2024-05-14T09:10:35.349 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-14T09:10:35.349 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} --archive /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670 --verbose --name vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670/orig.config.yaml 2024-05-14T09:10:35.404 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-14T09:10:35.496 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670 branch: wip-vshankar-testing-20240509.053109-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} email: vshankar@redhat.com first_in_suite: false job_id: '7705670' kernel: client: sha1: distro flavor: default kdb: true sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240509.053109-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: e2b755a64f0d72176b57299d02e880cf0620d780 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: e2b755a64f0d72176b57299d02e880cf0620d780 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240509.053109-debug sha1: e2b755a64f0d72176b57299d02e880cf0620d780 owner: scheduled_vshankar@teuthology priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6285 sha1: e2b755a64f0d72176b57299d02e880cf0620d780 sleep_before_teardown: 0 subset: 548/1024 suite: fs suite_branch: wip-vshankar-testing-20240509.053109-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: e2b755a64f0d72176b57299d02e880cf0620d780 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi002.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNKU1NbO3urIqJGt41AqPMnvxC3UOVVOMzAaBWER7gOR1mrCXB1fR6aBOlXoumsmo/kJS/Z9omknmy5yQTbbI8w= smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCn5pQSoMMrBGz6P2ksGr1UGMUGVYy2pyh2vnuWCt3viIRL2EcKGTtqzFV/evaUwD8UJ6Y7bXAuPwsljxLg9PpE= smithi029.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKoROA0d8fvC+XMKMpSo9QWZxKU/Q/yFOwkGDQgdv7uhoR9B/jXsXxbZ+QcJo7kGpT8dw1aLLR3Go0yYzVT7b5M= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-14_07:04:04 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-14T09:10:35.496 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa; will attempt to use it 2024-05-14T09:10:35.497 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks 2024-05-14T09:10:35.497 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-14T09:10:35.498 INFO:teuthology.task.internal:Checking packages... 2024-05-14T09:10:35.522 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'e2b755a64f0d72176b57299d02e880cf0620d780' 2024-05-14T09:10:35.522 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-14T09:10:35.523 INFO:teuthology.packaging:ref: None 2024-05-14T09:10:35.523 INFO:teuthology.packaging:tag: None 2024-05-14T09:10:35.523 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240509.053109-debug 2024-05-14T09:10:35.523 INFO:teuthology.packaging:sha1: e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T09:10:35.523 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240509.053109-debug 2024-05-14T09:10:35.749 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3588.ge2b755a6 2024-05-14T09:10:35.751 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-14T09:10:35.758 INFO:teuthology.task.internal:no buildpackages task found 2024-05-14T09:10:35.758 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-14T09:10:35.764 INFO:teuthology.task.internal:Saving configuration 2024-05-14T09:10:35.783 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-14T09:10:35.789 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T09:10:35.814 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 09:05:16.603156', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNKU1NbO3urIqJGt41AqPMnvxC3UOVVOMzAaBWER7gOR1mrCXB1fR6aBOlXoumsmo/kJS/Z9omknmy5yQTbbI8w='} 2024-05-14T09:10:35.837 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 09:05:16.605221', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCn5pQSoMMrBGz6P2ksGr1UGMUGVYy2pyh2vnuWCt3viIRL2EcKGTtqzFV/evaUwD8UJ6Y7bXAuPwsljxLg9PpE='} 2024-05-14T09:10:35.858 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi029.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 09:05:16.606985', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKoROA0d8fvC+XMKMpSo9QWZxKU/Q/yFOwkGDQgdv7uhoR9B/jXsXxbZ+QcJo7kGpT8dw1aLLR3Go0yYzVT7b5M='} 2024-05-14T09:10:35.858 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-14T09:10:35.864 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-14T09:10:35.864 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-14T09:10:35.865 INFO:teuthology.task.internal:roles: ubuntu@smithi029.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-14T09:10:35.865 INFO:teuthology.run_tasks:Running task console_log... 2024-05-14T09:10:35.974 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fda26ff40d0>, signals=[15]) 2024-05-14T09:10:35.974 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-14T09:10:35.981 INFO:teuthology.task.internal:Opening connections... 2024-05-14T09:10:35.981 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T09:10:35.983 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T09:10:36.058 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2024-05-14T09:10:36.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T09:10:36.136 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi029.front.sepia.ceph.com 2024-05-14T09:10:36.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T09:10:36.209 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-14T09:10:36.216 DEBUG:teuthology.orchestra.run.smithi002:> uname -m 2024-05-14T09:10:36.234 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2024-05-14T09:10:36.234 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2024-05-14T09:10:36.292 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2024-05-14T09:10:36.292 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="9" 2024-05-14T09:10:36.292 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2024-05-14T09:10:36.292 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2024-05-14T09:10:36.293 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="9" 2024-05-14T09:10:36.293 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el9" 2024-05-14T09:10:36.293 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-14T09:10:36.293 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2024-05-14T09:10:36.293 INFO:teuthology.orchestra.run.smithi002.stdout:LOGO="fedora-logo-icon" 2024-05-14T09:10:36.293 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-14T09:10:36.293 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2024-05-14T09:10:36.293 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-14T09:10:36.293 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-14T09:10:36.293 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-14T09:10:36.294 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2024-05-14T09:10:36.314 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2024-05-14T09:10:36.332 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2024-05-14T09:10:36.332 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2024-05-14T09:10:36.390 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Stream" 2024-05-14T09:10:36.390 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="9" 2024-05-14T09:10:36.390 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2024-05-14T09:10:36.390 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2024-05-14T09:10:36.390 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="9" 2024-05-14T09:10:36.390 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el9" 2024-05-14T09:10:36.390 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-14T09:10:36.390 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2024-05-14T09:10:36.390 INFO:teuthology.orchestra.run.smithi026.stdout:LOGO="fedora-logo-icon" 2024-05-14T09:10:36.391 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-14T09:10:36.391 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://centos.org/" 2024-05-14T09:10:36.391 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-14T09:10:36.391 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-14T09:10:36.391 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-14T09:10:36.391 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2024-05-14T09:10:36.414 DEBUG:teuthology.orchestra.run.smithi029:> uname -m 2024-05-14T09:10:36.432 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64 2024-05-14T09:10:36.432 DEBUG:teuthology.orchestra.run.smithi029:> cat /etc/os-release 2024-05-14T09:10:36.490 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="CentOS Stream" 2024-05-14T09:10:36.490 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="9" 2024-05-14T09:10:36.490 INFO:teuthology.orchestra.run.smithi029.stdout:ID="centos" 2024-05-14T09:10:36.490 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="rhel fedora" 2024-05-14T09:10:36.490 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="9" 2024-05-14T09:10:36.490 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el9" 2024-05-14T09:10:36.490 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-14T09:10:36.490 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31" 2024-05-14T09:10:36.491 INFO:teuthology.orchestra.run.smithi029.stdout:LOGO="fedora-logo-icon" 2024-05-14T09:10:36.491 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-14T09:10:36.491 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://centos.org/" 2024-05-14T09:10:36.491 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-14T09:10:36.491 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-14T09:10:36.491 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-14T09:10:36.491 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2024-05-14T09:10:36.513 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-14T09:10:36.524 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-14T09:10:36.574 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-14T09:10:36.574 DEBUG:teuthology.orchestra.run.smithi002:> test '!' -e /home/ubuntu/cephtest 2024-05-14T09:10:36.577 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2024-05-14T09:10:36.579 DEBUG:teuthology.orchestra.run.smithi029:> test '!' -e /home/ubuntu/cephtest 2024-05-14T09:10:36.596 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-14T09:10:36.602 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-14T09:10:36.602 DEBUG:teuthology.orchestra.run.smithi002:> test -z $(ls -A /var/lib/ceph) 2024-05-14T09:10:36.634 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2024-05-14T09:10:36.636 DEBUG:teuthology.orchestra.run.smithi029:> test -z $(ls -A /var/lib/ceph) 2024-05-14T09:10:36.660 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-14T09:10:36.734 INFO:teuthology.run_tasks:Running task kernel... 2024-05-14T09:10:36.751 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'} 2024-05-14T09:10:36.751 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-14T09:10:36.751 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-14T09:10:36.751 DEBUG:teuthology.orchestra.run.smithi002:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T09:10:36.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T09:10:36.768 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-05-14T09:10:36.825 INFO:teuthology.orchestra.run.smithi002.stdout:5.14.0-437.el9.x86_64 2024-05-14T09:10:36.826 INFO:teuthology.task.kernel:Running kernel on smithi002: 5.14.0-437.el9.x86_64 2024-05-14T09:10:36.826 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum install -y kernel 2024-05-14T09:10:43.013 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.1 MB 00:04 2024-05-14T09:10:44.781 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-05-14T09:10:50.273 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-05-14T09:10:51.865 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 16 kB 00:00 2024-05-14T09:10:54.492 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 21 MB 00:02 2024-05-14T09:10:59.866 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-05-14T09:11:00.078 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:01 ago on Tue 14 May 2024 09:10:59 AM UTC. 2024-05-14T09:11:01.923 INFO:teuthology.orchestra.run.smithi002.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-14T09:11:01.923 INFO:teuthology.orchestra.run.smithi002.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-14T09:11:02.022 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-05-14T09:11:02.028 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-05-14T09:11:02.028 INFO:teuthology.orchestra.run.smithi002.stdout: Package Architecture Version Repository Size 2024-05-14T09:11:02.028 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-05-14T09:11:02.028 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-05-14T09:11:02.028 INFO:teuthology.orchestra.run.smithi002.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-14T09:11:02.028 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-05-14T09:11:02.028 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-14T09:11:02.029 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-14T09:11:02.029 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-14T09:11:02.029 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:11:02.029 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-05-14T09:11:02.029 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-05-14T09:11:02.029 INFO:teuthology.orchestra.run.smithi002.stdout:Install 4 Packages 2024-05-14T09:11:02.029 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:11:02.029 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 78 M 2024-05-14T09:11:02.030 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 123 M 2024-05-14T09:11:02.030 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-05-14T09:11:02.351 INFO:teuthology.orchestra.run.smithi002.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 3.4 MB/s | 550 kB 00:00 2024-05-14T09:11:02.652 INFO:teuthology.orchestra.run.smithi002.stdout:(2/4): kernel-core-5.14.0-446.el9.x86_64.rpm 34 MB/s | 15 MB 00:00 2024-05-14T09:11:03.052 INFO:teuthology.orchestra.run.smithi002.stdout:(3/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 40 MB/s | 34 MB 00:00 2024-05-14T09:11:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:(4/4): kernel-modules-core-5.14.0-446.el9.x86_6 28 MB/s | 28 MB 00:01 2024-05-14T09:11:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-05-14T09:11:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:Total 59 MB/s | 78 MB 00:01 2024-05-14T09:11:04.066 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-05-14T09:11:04.451 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-05-14T09:11:04.451 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-05-14T09:11:05.409 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-05-14T09:11:05.410 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-05-14T09:11:07.931 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-05-14T09:11:08.599 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-14T09:11:08.685 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-14T09:11:11.509 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-14T09:11:11.637 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-14T09:11:16.715 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-14T09:11:16.757 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-14T09:11:21.855 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-14T09:11:52.741 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-14T09:11:52.780 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-14T09:11:53.138 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-14T09:11:53.139 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-14T09:11:53.139 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-14T09:11:53.139 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-14T09:11:53.904 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-14T09:11:53.904 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:11:53.904 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-05-14T09:11:53.904 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-14T09:11:53.904 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-14T09:11:53.904 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-14T09:11:53.904 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-14T09:11:53.904 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:11:53.904 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-05-14T09:11:54.235 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q kernel | sort -rV | head -n 1 2024-05-14T09:11:54.314 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-14T09:11:54.314 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-14T09:11:54.314 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-14T09:11:54.314 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-14T09:11:54.314 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-14T09:11:54.314 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-14T09:11:54.315 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi002.front.sepia.ceph.com, path=None, version=distro) 2024-05-14T09:11:54.315 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q kernel | sort -rV | head -n 1 2024-05-14T09:11:54.366 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-14T09:11:54.367 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-14T09:11:54.367 DEBUG:teuthology.orchestra.run.smithi002:> sudo rpm -qi grub2-tools 2024-05-14T09:11:54.408 INFO:teuthology.orchestra.run.smithi002.stdout:Name : grub2-tools 2024-05-14T09:11:54.408 INFO:teuthology.orchestra.run.smithi002.stdout:Epoch : 1 2024-05-14T09:11:54.408 INFO:teuthology.orchestra.run.smithi002.stdout:Version : 2.06 2024-05-14T09:11:54.408 INFO:teuthology.orchestra.run.smithi002.stdout:Release : 68.el9 2024-05-14T09:11:54.408 INFO:teuthology.orchestra.run.smithi002.stdout:Architecture: x86_64 2024-05-14T09:11:54.409 INFO:teuthology.orchestra.run.smithi002.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-14T09:11:54.409 INFO:teuthology.orchestra.run.smithi002.stdout:Group : Unspecified 2024-05-14T09:11:54.409 INFO:teuthology.orchestra.run.smithi002.stdout:Size : 8264254 2024-05-14T09:11:54.409 INFO:teuthology.orchestra.run.smithi002.stdout:License : GPLv3+ 2024-05-14T09:11:54.409 INFO:teuthology.orchestra.run.smithi002.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-14T09:11:54.409 INFO:teuthology.orchestra.run.smithi002.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-14T09:11:54.409 INFO:teuthology.orchestra.run.smithi002.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-14T09:11:54.409 INFO:teuthology.orchestra.run.smithi002.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-14T09:11:54.409 INFO:teuthology.orchestra.run.smithi002.stdout:Packager : builder@centos.org 2024-05-14T09:11:54.409 INFO:teuthology.orchestra.run.smithi002.stdout:Vendor : CentOS 2024-05-14T09:11:54.409 INFO:teuthology.orchestra.run.smithi002.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-14T09:11:54.409 INFO:teuthology.orchestra.run.smithi002.stdout:Summary : Support tools for GRUB. 2024-05-14T09:11:54.409 INFO:teuthology.orchestra.run.smithi002.stdout:Description : 2024-05-14T09:11:54.409 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:11:54.410 INFO:teuthology.orchestra.run.smithi002.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-14T09:11:54.410 INFO:teuthology.orchestra.run.smithi002.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-14T09:11:54.410 INFO:teuthology.orchestra.run.smithi002.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-14T09:11:54.410 INFO:teuthology.orchestra.run.smithi002.stdout:hardware devices. 2024-05-14T09:11:54.410 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:11:54.410 INFO:teuthology.orchestra.run.smithi002.stdout:This subpackage provides tools for support of all platforms. 2024-05-14T09:11:54.411 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-14T09:11:54.411 INFO:teuthology.task.kernel:Updating grub on smithi002 to boot 5.14.0-446.el9.x86_64 2024-05-14T09:11:54.411 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-14T09:11:55.001 INFO:teuthology.orchestra.run.smithi002.stderr:Generating grub configuration file ... 2024-05-14T09:11:56.454 INFO:teuthology.orchestra.run.smithi002.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-14T09:11:56.482 INFO:teuthology.orchestra.run.smithi002.stderr:done 2024-05-14T09:11:56.484 DEBUG:teuthology.orchestra.run.smithi002:> mktemp 2024-05-14T09:11:56.499 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.cWjVjffU3U 2024-05-14T09:11:56.500 DEBUG:teuthology.orchestra.run.smithi002:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.cWjVjffU3U 2024-05-14T09:11:56.566 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.cWjVjffU3U 2024-05-14T09:11:56.703 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.cWjVjffU3U is 6KB 2024-05-14T09:11:56.753 DEBUG:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.cWjVjffU3U 2024-05-14T09:11:56.768 DEBUG:teuthology.orchestra.run.smithi002:> sudo /bin/ls /boot/loader/entries || true 2024-05-14T09:11:56.830 INFO:teuthology.orchestra.run.smithi002.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-14T09:11:56.830 INFO:teuthology.orchestra.run.smithi002.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-14T09:11:56.831 INFO:teuthology.orchestra.run.smithi002.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-14T09:11:56.831 INFO:teuthology.orchestra.run.smithi002.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-14T09:11:56.832 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-14T09:11:56.976 DEBUG:teuthology.orchestra.run.smithi002:> sudo shutdown -r now 2024-05-14T09:12:26.979 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-14T09:12:26.980 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-05-14T09:12:26.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T09:12:45.347 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-05-14T09:12:54.347 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-05-14T09:12:54.348 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T09:12:57.410 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-05-14T09:13:09.411 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-05-14T09:13:09.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T09:13:41.455 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-05-14T09:13:41.998 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-05-14T09:13:41.999 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-446.el9.x86_64"... 2024-05-14T09:13:41.999 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-05-14T09:13:42.025 INFO:teuthology.orchestra.run.smithi002.stdout:5.14.0-446.el9.x86_64 2024-05-14T09:13:42.025 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-14T09:13:42.026 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-14T09:13:42.026 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-14T09:13:43.026 DEBUG:teuthology.parallel:result is None 2024-05-14T09:13:43.027 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-14T09:13:43.036 INFO:teuthology.task.internal:Creating test directory... 2024-05-14T09:13:43.037 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T09:13:43.039 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T09:13:43.042 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T09:13:43.061 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-14T09:13:43.068 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-14T09:13:43.077 INFO:teuthology.task.internal:Creating archive directory... 2024-05-14T09:13:43.078 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T09:13:43.101 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T09:13:43.103 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T09:13:43.137 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-14T09:13:43.144 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-14T09:13:43.144 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T09:13:43.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T09:13:43.183 DEBUG:teuthology.orchestra.run.smithi029:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T09:13:43.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T09:13:43.200 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T09:13:43.203 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T09:13:43.225 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T09:13:43.263 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T09:13:43.263 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T09:13:43.279 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T09:13:43.281 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T09:13:43.301 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T09:13:43.314 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T09:13:43.316 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-14T09:13:43.323 INFO:teuthology.task.internal:Configuring sudo... 2024-05-14T09:13:43.323 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T09:13:43.326 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T09:13:43.328 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T09:13:43.383 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-14T09:13:43.395 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-14T09:13:43.395 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T09:13:43.397 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T09:13:43.399 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T09:13:43.441 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T09:13:43.499 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T09:13:43.558 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T09:13:43.643 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T09:13:43.688 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:13:43.688 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T09:13:43.757 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T09:13:43.802 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T09:13:43.858 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T09:13:43.943 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T09:13:43.988 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-14T09:13:43.989 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T09:13:44.058 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T09:13:44.103 DEBUG:teuthology.orchestra.run.smithi029:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T09:13:44.157 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T09:13:44.241 DEBUG:teuthology.orchestra.run.smithi029:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T09:13:44.284 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-05-14T09:13:44.284 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T09:13:44.351 DEBUG:teuthology.orchestra.run.smithi002:> sudo service rsyslog restart 2024-05-14T09:13:44.354 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2024-05-14T09:13:44.357 DEBUG:teuthology.orchestra.run.smithi029:> sudo service rsyslog restart 2024-05-14T09:13:44.395 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T09:13:44.397 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T09:13:44.436 INFO:teuthology.orchestra.run.smithi029.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T09:13:44.960 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-14T09:13:44.969 INFO:teuthology.task.internal:Starting timer... 2024-05-14T09:13:44.969 INFO:teuthology.run_tasks:Running task pcp... 2024-05-14T09:13:44.995 INFO:teuthology.run_tasks:Running task selinux... 2024-05-14T09:13:45.003 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-14T09:13:45.003 DEBUG:teuthology.orchestra.run.smithi002:> sudo service auditd rotate 2024-05-14T09:13:45.065 INFO:teuthology.orchestra.run.smithi002.stdout:Rotating logs: 2024-05-14T09:13:45.067 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2024-05-14T09:13:45.148 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2024-05-14T09:13:45.150 DEBUG:teuthology.orchestra.run.smithi029:> sudo service auditd rotate 2024-05-14T09:13:45.229 INFO:teuthology.orchestra.run.smithi029.stdout:Rotating logs: 2024-05-14T09:13:45.231 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-14T09:13:45.232 DEBUG:teuthology.orchestra.run.smithi002:> /usr/sbin/getenforce 2024-05-14T09:13:45.261 INFO:teuthology.orchestra.run.smithi002.stdout:Permissive 2024-05-14T09:13:45.262 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2024-05-14T09:13:45.285 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2024-05-14T09:13:45.286 DEBUG:teuthology.orchestra.run.smithi029:> /usr/sbin/getenforce 2024-05-14T09:13:45.310 INFO:teuthology.orchestra.run.smithi029.stdout:Permissive 2024-05-14T09:13:45.310 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi002.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi026.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi029.front.sepia.ceph.com': 'permissive'} 2024-05-14T09:13:45.311 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T09:13:45.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T09:13:45.340 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T09:13:45.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T09:13:45.367 DEBUG:teuthology.orchestra.run.smithi029:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T09:13:45.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T09:13:45.395 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-14T09:13:45.395 DEBUG:teuthology.orchestra.run.smithi002:> sudo /usr/sbin/setenforce permissive 2024-05-14T09:13:45.430 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2024-05-14T09:13:45.456 DEBUG:teuthology.orchestra.run.smithi029:> sudo /usr/sbin/setenforce permissive 2024-05-14T09:13:45.497 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-14T09:13:45.508 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-14T09:13:45.513 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-14T09:13:45.643 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-14T09:13:45.659 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-14T09:13:45.660 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi002.front.sepia.ceph.com,smithi026.front.sepia.ceph.com,smithi029.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-14T09:19:06.123 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi002.front.sepia.ceph.com'), Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi029.front.sepia.ceph.com')] 2024-05-14T09:19:06.125 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-05-14T09:19:06.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T09:19:06.215 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-05-14T09:19:06.298 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-05-14T09:19:06.298 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-05-14T09:19:06.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T09:19:06.417 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-05-14T09:19:06.499 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-05-14T09:19:06.499 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-05-14T09:19:06.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T09:19:06.616 DEBUG:teuthology.orchestra.run.smithi029:> true 2024-05-14T09:19:06.699 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-05-14T09:19:06.699 INFO:teuthology.run_tasks:Running task clock... 2024-05-14T09:19:06.710 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-14T09:19:06.711 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T09:19:06.711 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T09:19:06.714 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T09:19:06.714 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T09:19:06.721 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T09:19:06.721 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T09:19:06.744 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-14T09:19:06.749 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-14T09:19:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-14T09:19:06.764 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-14T09:19:06.771 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-14T09:19:06.784 INFO:teuthology.orchestra.run.smithi002.stderr:sudo: ntpd: command not found 2024-05-14T09:19:06.786 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-14T09:19:06.789 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2024-05-14T09:19:06.796 INFO:teuthology.orchestra.run.smithi002.stdout:506 Cannot talk to daemon 2024-05-14T09:19:06.801 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2024-05-14T09:19:06.810 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-14T09:19:06.811 INFO:teuthology.orchestra.run.smithi029.stderr:sudo: ntpd: command not found 2024-05-14T09:19:06.815 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-14T09:19:06.822 INFO:teuthology.orchestra.run.smithi029.stdout:506 Cannot talk to daemon 2024-05-14T09:19:06.825 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-14T09:19:06.829 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-14T09:19:06.836 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-14T09:19:06.850 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-14T09:19:06.886 INFO:teuthology.orchestra.run.smithi002.stderr:bash: line 1: ntpq: command not found 2024-05-14T09:19:06.887 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-05-14T09:19:06.889 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T09:19:06.889 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2024-05-14T09:19:06.889 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T09:19:06.889 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T09:19:06.889 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T09:19:06.889 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T09:19:06.891 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T09:19:06.891 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-05-14T09:19:06.891 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T09:19:06.891 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T09:19:06.891 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T09:19:06.891 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T09:19:06.906 INFO:teuthology.orchestra.run.smithi029.stderr:bash: line 1: ntpq: command not found 2024-05-14T09:19:06.909 INFO:teuthology.orchestra.run.smithi029.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T09:19:06.909 INFO:teuthology.orchestra.run.smithi029.stdout:=============================================================================== 2024-05-14T09:19:06.909 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T09:19:06.909 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T09:19:06.910 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T09:19:06.910 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T09:19:06.910 INFO:teuthology.run_tasks:Running task install... 2024-05-14T09:19:06.919 DEBUG:teuthology.task.install:project ceph 2024-05-14T09:19:06.920 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780'}} 2024-05-14T09:19:06.920 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780'} 2024-05-14T09:19:06.920 INFO:teuthology.task.install:Using flavor: default 2024-05-14T09:19:06.928 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-14T09:19:06.928 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-14T09:19:06.929 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780', 'tag': None, 'wait_for_package': False} 2024-05-14T09:19:06.929 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T09:19:06.931 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780', 'tag': None, 'wait_for_package': False} 2024-05-14T09:19:06.932 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T09:19:06.932 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780', 'tag': None, 'wait_for_package': False} 2024-05-14T09:19:06.933 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T09:19:07.100 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/ 2024-05-14T09:19:07.100 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3588.ge2b755a6 2024-05-14T09:19:07.159 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/ 2024-05-14T09:19:07.159 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3588.ge2b755a6 2024-05-14T09:19:07.168 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/ 2024-05-14T09:19:07.168 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3588.ge2b755a6 2024-05-14T09:19:07.217 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-14T09:19:07.217 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-05-14T09:19:07.217 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-14T09:19:07.243 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-14T09:19:07.243 DEBUG:teuthology.orchestra.run.smithi029:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e2b755a64f0d72176b57299d02e880cf0620d780/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-14T09:19:07.271 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-14T09:19:07.272 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:19:07.272 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-14T09:19:07.300 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-14T09:19:07.300 DEBUG:teuthology.orchestra.run.smithi002:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e2b755a64f0d72176b57299d02e880cf0620d780/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-14T09:19:07.310 DEBUG:teuthology.orchestra.run.smithi029:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-14T09:19:07.345 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-14T09:19:07.345 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-14T09:19:07.345 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-14T09:19:07.369 DEBUG:teuthology.orchestra.run.smithi002:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-14T09:19:07.372 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-14T09:19:07.372 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e2b755a64f0d72176b57299d02e880cf0620d780/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-14T09:19:07.385 DEBUG:teuthology.orchestra.run.smithi029:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-14T09:19:07.439 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-14T09:19:07.447 DEBUG:teuthology.orchestra.run.smithi002:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-14T09:19:07.450 INFO:teuthology.orchestra.run.smithi029.stdout:check_obsoletes = 1 2024-05-14T09:19:07.451 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum clean all 2024-05-14T09:19:07.512 INFO:teuthology.orchestra.run.smithi002.stdout:check_obsoletes = 1 2024-05-14T09:19:07.514 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum clean all 2024-05-14T09:19:07.515 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-14T09:19:07.580 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2024-05-14T09:19:07.581 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2024-05-14T09:19:07.691 INFO:teuthology.orchestra.run.smithi029.stdout:57 files removed 2024-05-14T09:19:07.719 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-14T09:19:07.753 INFO:teuthology.orchestra.run.smithi002.stdout:57 files removed 2024-05-14T09:19:07.780 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-14T09:19:07.820 INFO:teuthology.orchestra.run.smithi026.stdout:57 files removed 2024-05-14T09:19:07.848 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-14T09:19:08.316 INFO:teuthology.orchestra.run.smithi029.stdout:ceph packages for x86_64 260 kB/s | 84 kB 00:00 2024-05-14T09:19:08.402 INFO:teuthology.orchestra.run.smithi002.stdout:ceph packages for x86_64 250 kB/s | 84 kB 00:00 2024-05-14T09:19:08.683 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 150 kB/s | 84 kB 00:00 2024-05-14T09:19:08.852 INFO:teuthology.orchestra.run.smithi002.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-05-14T09:19:08.875 INFO:teuthology.orchestra.run.smithi029.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-05-14T09:19:09.234 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-05-14T09:19:09.353 INFO:teuthology.orchestra.run.smithi002.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-05-14T09:19:09.385 INFO:teuthology.orchestra.run.smithi029.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-05-14T09:19:09.693 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-05-14T09:19:09.905 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.1 MB 00:00 2024-05-14T09:19:11.849 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-05-14T09:19:13.551 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.1 MB 00:03 2024-05-14T09:19:13.911 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.1 MB 00:04 2024-05-14T09:19:15.604 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - AppStream 34 MB/s | 19 MB 00:00 2024-05-14T09:19:18.434 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.2 MB 00:02 2024-05-14T09:19:19.869 INFO:teuthology.orchestra.run.smithi029.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 16 kB 00:00 2024-05-14T09:19:19.905 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-05-14T09:19:21.168 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 3.0 MB/s | 19 MB 00:06 2024-05-14T09:19:22.280 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-05-14T09:19:23.315 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-05-14T09:19:23.485 INFO:teuthology.orchestra.run.smithi029.stdout:Extra Packages for Enterprise Linux 6.1 MB/s | 21 MB 00:03 2024-05-14T09:19:27.219 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.2 MB 00:02 2024-05-14T09:19:28.619 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 16 kB 00:00 2024-05-14T09:19:28.865 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-14T09:19:28.925 INFO:teuthology.orchestra.run.smithi029.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-14T09:19:30.715 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-14T09:19:31.243 INFO:teuthology.orchestra.run.smithi002.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T09:19:31.243 INFO:teuthology.orchestra.run.smithi002.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T09:19:31.255 INFO:teuthology.orchestra.run.smithi002.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-14T09:19:31.256 INFO:teuthology.orchestra.run.smithi002.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-14T09:19:31.257 INFO:teuthology.orchestra.run.smithi002.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-14T09:19:31.257 INFO:teuthology.orchestra.run.smithi002.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-14T09:19:31.259 INFO:teuthology.orchestra.run.smithi002.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-14T09:19:31.261 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-14T09:19:31.262 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-14T09:19:31.329 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-05-14T09:19:31.335 INFO:teuthology.orchestra.run.smithi029.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T09:19:31.336 INFO:teuthology.orchestra.run.smithi029.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T09:19:31.344 INFO:teuthology.orchestra.run.smithi002.stdout:======================================================================================= 2024-05-14T09:19:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-05-14T09:19:31.345 INFO:teuthology.orchestra.run.smithi002.stdout:======================================================================================= 2024-05-14T09:19:31.345 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-05-14T09:19:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-14T09:19:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: ceph x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 6.6 k 2024-05-14T09:19:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 5.4 M 2024-05-14T09:19:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 863 k 2024-05-14T09:19:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 147 k 2024-05-14T09:19:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 1.5 M 2024-05-14T09:19:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 142 k 2024-05-14T09:19:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 3.5 M 2024-05-14T09:19:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 7.4 M 2024-05-14T09:19:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 49 k 2024-05-14T09:19:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 10 M 2024-05-14T09:19:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 43 M 2024-05-14T09:19:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 273 k 2024-05-14T09:19:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 763 k 2024-05-14T09:19:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 224 k 2024-05-14T09:19:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-shell x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 27 k 2024-05-14T09:19:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 18 k 2024-05-14T09:19:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-14T09:19:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-14T09:19:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-14T09:19:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 32 k 2024-05-14T09:19:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 756 k 2024-05-14T09:19:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 127 k 2024-05-14T09:19:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-14T09:19:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-14T09:19:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-14T09:19:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-14T09:19:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-14T09:19:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-14T09:19:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 162 k 2024-05-14T09:19:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 321 k 2024-05-14T09:19:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 301 k 2024-05-14T09:19:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 100 k 2024-05-14T09:19:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 86 k 2024-05-14T09:19:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.0 M 2024-05-14T09:19:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 170 k 2024-05-14T09:19:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-14T09:19:31.348 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrading: 2024-05-14T09:19:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-14T09:19:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-14T09:19:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-14T09:19:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-14T09:19:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: librados2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.3 M 2024-05-14T09:19:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.1 M 2024-05-14T09:19:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-14T09:19:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-14T09:19:31.348 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-05-14T09:19:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-14T09:19:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-14T09:19:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-14T09:19:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 20 M 2024-05-14T09:19:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 34 k 2024-05-14T09:19:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 2.3 M 2024-05-14T09:19:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 274 k 2024-05-14T09:19:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 4.9 M 2024-05-14T09:19:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 18 M 2024-05-14T09:19:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 16 k 2024-05-14T09:19:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 25 k 2024-05-14T09:19:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-14T09:19:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-14T09:19:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-14T09:19:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-14T09:19:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-14T09:19:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-14T09:19:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-14T09:19:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-14T09:19:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 157 k 2024-05-14T09:19:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-14T09:19:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-14T09:19:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-14T09:19:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-14T09:19:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-14T09:19:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 497 k 2024-05-14T09:19:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-14T09:19:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 5.2 M 2024-05-14T09:19:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-14T09:19:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-14T09:19:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-14T09:19:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-14T09:19:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-14T09:19:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-14T09:19:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-14T09:19:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-14T09:19:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-14T09:19:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-14T09:19:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-14T09:19:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-14T09:19:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-14T09:19:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-14T09:19:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-14T09:19:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-14T09:19:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-14T09:19:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 45 k 2024-05-14T09:19:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 135 k 2024-05-14T09:19:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-14T09:19:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-14T09:19:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-14T09:19:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-14T09:19:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-14T09:19:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-14T09:19:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-14T09:19:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-14T09:19:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-14T09:19:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-14T09:19:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-14T09:19:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-14T09:19:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-14T09:19:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-14T09:19:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-14T09:19:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-14T09:19:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-14T09:19:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-14T09:19:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-14T09:19:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-14T09:19:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-14T09:19:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-14T09:19:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-14T09:19:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-14T09:19:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-14T09:19:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-14T09:19:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-14T09:19:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-14T09:19:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-14T09:19:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-14T09:19:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-14T09:19:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-14T09:19:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-14T09:19:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-14T09:19:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-14T09:19:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-14T09:19:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-14T09:19:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-14T09:19:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-14T09:19:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-14T09:19:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-14T09:19:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-14T09:19:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-14T09:19:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-14T09:19:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-14T09:19:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-14T09:19:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-14T09:19:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-14T09:19:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-14T09:19:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-14T09:19:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-14T09:19:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-14T09:19:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-14T09:19:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-14T09:19:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-14T09:19:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-14T09:19:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-14T09:19:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-14T09:19:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-14T09:19:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-14T09:19:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-14T09:19:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-14T09:19:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-14T09:19:31.357 INFO:teuthology.orchestra.run.smithi002.stdout:Installing weak dependencies: 2024-05-14T09:19:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-14T09:19:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:19:31.357 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-05-14T09:19:31.357 INFO:teuthology.orchestra.run.smithi002.stdout:======================================================================================= 2024-05-14T09:19:31.357 INFO:teuthology.orchestra.run.smithi002.stdout:Install 148 Packages 2024-05-14T09:19:31.357 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrade 8 Packages 2024-05-14T09:19:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:19:31.358 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 219 M 2024-05-14T09:19:31.358 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-05-14T09:19:31.359 INFO:teuthology.orchestra.run.smithi029.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-14T09:19:31.359 INFO:teuthology.orchestra.run.smithi029.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-14T09:19:31.359 INFO:teuthology.orchestra.run.smithi029.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-14T09:19:31.359 INFO:teuthology.orchestra.run.smithi029.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-14T09:19:31.359 INFO:teuthology.orchestra.run.smithi029.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-14T09:19:31.359 INFO:teuthology.orchestra.run.smithi029.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-14T09:19:31.359 INFO:teuthology.orchestra.run.smithi029.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-14T09:19:31.420 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2024-05-14T09:19:31.447 INFO:teuthology.orchestra.run.smithi029.stdout:======================================================================================= 2024-05-14T09:19:31.447 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2024-05-14T09:19:31.447 INFO:teuthology.orchestra.run.smithi029.stdout:======================================================================================= 2024-05-14T09:19:31.447 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2024-05-14T09:19:31.447 INFO:teuthology.orchestra.run.smithi029.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-14T09:19:31.447 INFO:teuthology.orchestra.run.smithi029.stdout: ceph x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 6.6 k 2024-05-14T09:19:31.447 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-base x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 5.4 M 2024-05-14T09:19:31.447 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-fuse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 863 k 2024-05-14T09:19:31.447 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 147 k 2024-05-14T09:19:31.447 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 1.5 M 2024-05-14T09:19:31.447 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 142 k 2024-05-14T09:19:31.448 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 3.5 M 2024-05-14T09:19:31.448 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 7.4 M 2024-05-14T09:19:31.448 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-rook noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 49 k 2024-05-14T09:19:31.448 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-radosgw x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 10 M 2024-05-14T09:19:31.448 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-test x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 43 M 2024-05-14T09:19:31.448 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-volume noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 273 k 2024-05-14T09:19:31.448 INFO:teuthology.orchestra.run.smithi029.stdout: cephadm noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 763 k 2024-05-14T09:19:31.448 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-mirror x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 224 k 2024-05-14T09:19:31.448 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-shell x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 27 k 2024-05-14T09:19:31.448 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-top noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 18 k 2024-05-14T09:19:31.448 INFO:teuthology.orchestra.run.smithi029.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-14T09:19:31.448 INFO:teuthology.orchestra.run.smithi029.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-14T09:19:31.448 INFO:teuthology.orchestra.run.smithi029.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-14T09:19:31.448 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs-devel x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 32 k 2024-05-14T09:19:31.449 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 756 k 2024-05-14T09:19:31.449 INFO:teuthology.orchestra.run.smithi029.stdout: librados-devel x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 127 k 2024-05-14T09:19:31.449 INFO:teuthology.orchestra.run.smithi029.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-14T09:19:31.449 INFO:teuthology.orchestra.run.smithi029.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-14T09:19:31.449 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-14T09:19:31.449 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-14T09:19:31.449 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-14T09:19:31.449 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-14T09:19:31.449 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cephfs x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 162 k 2024-05-14T09:19:31.449 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rados x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 321 k 2024-05-14T09:19:31.449 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rbd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 301 k 2024-05-14T09:19:31.449 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rgw x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 100 k 2024-05-14T09:19:31.449 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-fuse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 86 k 2024-05-14T09:19:31.449 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-mirror x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.0 M 2024-05-14T09:19:31.450 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-nbd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 170 k 2024-05-14T09:19:31.450 INFO:teuthology.orchestra.run.smithi029.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-14T09:19:31.450 INFO:teuthology.orchestra.run.smithi029.stdout:Upgrading: 2024-05-14T09:19:31.450 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-14T09:19:31.450 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-14T09:19:31.450 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-14T09:19:31.450 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-14T09:19:31.450 INFO:teuthology.orchestra.run.smithi029.stdout: librados2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.3 M 2024-05-14T09:19:31.450 INFO:teuthology.orchestra.run.smithi029.stdout: librbd1 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.1 M 2024-05-14T09:19:31.450 INFO:teuthology.orchestra.run.smithi029.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-14T09:19:31.450 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-14T09:19:31.450 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2024-05-14T09:19:31.450 INFO:teuthology.orchestra.run.smithi029.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-14T09:19:31.450 INFO:teuthology.orchestra.run.smithi029.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-14T09:19:31.451 INFO:teuthology.orchestra.run.smithi029.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-14T09:19:31.451 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-common x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 20 M 2024-05-14T09:19:31.451 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 34 k 2024-05-14T09:19:31.451 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mds x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 2.3 M 2024-05-14T09:19:31.451 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 274 k 2024-05-14T09:19:31.451 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mon x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 4.9 M 2024-05-14T09:19:31.451 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-osd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 18 M 2024-05-14T09:19:31.451 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 16 k 2024-05-14T09:19:31.451 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-selinux x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 25 k 2024-05-14T09:19:31.451 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-14T09:19:31.451 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-14T09:19:31.451 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-14T09:19:31.451 INFO:teuthology.orchestra.run.smithi029.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-14T09:19:31.451 INFO:teuthology.orchestra.run.smithi029.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-14T09:19:31.451 INFO:teuthology.orchestra.run.smithi029.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-14T09:19:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-14T09:19:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-14T09:19:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: libcephsqlite x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 157 k 2024-05-14T09:19:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-14T09:19:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-14T09:19:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-14T09:19:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-14T09:19:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-14T09:19:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: libradosstriper1 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 497 k 2024-05-14T09:19:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-14T09:19:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: librgw2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 5.2 M 2024-05-14T09:19:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-14T09:19:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-14T09:19:31.452 INFO:teuthology.orchestra.run.smithi029.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-14T09:19:31.453 INFO:teuthology.orchestra.run.smithi029.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-14T09:19:31.453 INFO:teuthology.orchestra.run.smithi029.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-14T09:19:31.453 INFO:teuthology.orchestra.run.smithi029.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-14T09:19:31.453 INFO:teuthology.orchestra.run.smithi029.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-14T09:19:31.453 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-14T09:19:31.453 INFO:teuthology.orchestra.run.smithi029.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-14T09:19:31.453 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-14T09:19:31.453 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-14T09:19:31.453 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-14T09:19:31.453 INFO:teuthology.orchestra.run.smithi029.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-14T09:19:31.453 INFO:teuthology.orchestra.run.smithi029.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-14T09:19:31.453 INFO:teuthology.orchestra.run.smithi029.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-14T09:19:31.453 INFO:teuthology.orchestra.run.smithi029.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-14T09:19:31.453 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-14T09:19:31.454 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-argparse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 45 k 2024-05-14T09:19:31.454 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-common x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 135 k 2024-05-14T09:19:31.454 INFO:teuthology.orchestra.run.smithi029.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-14T09:19:31.454 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-14T09:19:31.454 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-14T09:19:31.454 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-14T09:19:31.454 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-14T09:19:31.454 INFO:teuthology.orchestra.run.smithi029.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-14T09:19:31.454 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-14T09:19:31.454 INFO:teuthology.orchestra.run.smithi029.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-14T09:19:31.454 INFO:teuthology.orchestra.run.smithi029.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-14T09:19:31.454 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-14T09:19:31.454 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-14T09:19:31.455 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-14T09:19:31.455 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-14T09:19:31.455 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-14T09:19:31.455 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-14T09:19:31.455 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-14T09:19:31.455 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-14T09:19:31.455 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-14T09:19:31.455 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-14T09:19:31.455 INFO:teuthology.orchestra.run.smithi029.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-14T09:19:31.455 INFO:teuthology.orchestra.run.smithi029.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-14T09:19:31.455 INFO:teuthology.orchestra.run.smithi029.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-14T09:19:31.455 INFO:teuthology.orchestra.run.smithi029.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-14T09:19:31.455 INFO:teuthology.orchestra.run.smithi029.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-14T09:19:31.455 INFO:teuthology.orchestra.run.smithi029.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-14T09:19:31.455 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-14T09:19:31.456 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-14T09:19:31.456 INFO:teuthology.orchestra.run.smithi029.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-14T09:19:31.456 INFO:teuthology.orchestra.run.smithi029.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-14T09:19:31.456 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-14T09:19:31.456 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-14T09:19:31.456 INFO:teuthology.orchestra.run.smithi029.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-14T09:19:31.456 INFO:teuthology.orchestra.run.smithi029.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-14T09:19:31.456 INFO:teuthology.orchestra.run.smithi029.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-14T09:19:31.456 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-14T09:19:31.456 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-14T09:19:31.456 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-14T09:19:31.456 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-14T09:19:31.456 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-14T09:19:31.456 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-14T09:19:31.456 INFO:teuthology.orchestra.run.smithi029.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-14T09:19:31.457 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-14T09:19:31.457 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-14T09:19:31.457 INFO:teuthology.orchestra.run.smithi029.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-14T09:19:31.457 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-14T09:19:31.457 INFO:teuthology.orchestra.run.smithi029.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-14T09:19:31.457 INFO:teuthology.orchestra.run.smithi029.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-14T09:19:31.457 INFO:teuthology.orchestra.run.smithi029.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-14T09:19:31.457 INFO:teuthology.orchestra.run.smithi029.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-14T09:19:31.457 INFO:teuthology.orchestra.run.smithi029.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-14T09:19:31.457 INFO:teuthology.orchestra.run.smithi029.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-14T09:19:31.457 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-14T09:19:31.457 INFO:teuthology.orchestra.run.smithi029.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-14T09:19:31.457 INFO:teuthology.orchestra.run.smithi029.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-14T09:19:31.457 INFO:teuthology.orchestra.run.smithi029.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-14T09:19:31.458 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-14T09:19:31.458 INFO:teuthology.orchestra.run.smithi029.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-14T09:19:31.458 INFO:teuthology.orchestra.run.smithi029.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-14T09:19:31.458 INFO:teuthology.orchestra.run.smithi029.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-14T09:19:31.458 INFO:teuthology.orchestra.run.smithi029.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-14T09:19:31.458 INFO:teuthology.orchestra.run.smithi029.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-14T09:19:31.458 INFO:teuthology.orchestra.run.smithi029.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-14T09:19:31.458 INFO:teuthology.orchestra.run.smithi029.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-14T09:19:31.458 INFO:teuthology.orchestra.run.smithi029.stdout:Installing weak dependencies: 2024-05-14T09:19:31.458 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-14T09:19:31.458 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:19:31.458 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2024-05-14T09:19:31.458 INFO:teuthology.orchestra.run.smithi029.stdout:======================================================================================= 2024-05-14T09:19:31.458 INFO:teuthology.orchestra.run.smithi029.stdout:Install 148 Packages 2024-05-14T09:19:31.459 INFO:teuthology.orchestra.run.smithi029.stdout:Upgrade 8 Packages 2024-05-14T09:19:31.459 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:19:31.459 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 219 M 2024-05-14T09:19:31.459 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2024-05-14T09:19:32.228 INFO:teuthology.orchestra.run.smithi029.stdout:(1/156): ceph-19.0.0-3588.ge2b755a6.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-05-14T09:19:32.341 INFO:teuthology.orchestra.run.smithi002.stdout:(1/156): ceph-19.0.0-3588.ge2b755a6.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-05-14T09:19:32.617 INFO:teuthology.orchestra.run.smithi002.stdout:(2/156): ceph-fuse-19.0.0-3588.ge2b755a6.el9.x8 3.1 MB/s | 863 kB 00:00 2024-05-14T09:19:32.636 INFO:teuthology.orchestra.run.smithi029.stdout:(2/156): ceph-fuse-19.0.0-3588.ge2b755a6.el9.x8 2.1 MB/s | 863 kB 00:00 2024-05-14T09:19:32.700 INFO:teuthology.orchestra.run.smithi002.stdout:(3/156): ceph-immutable-object-cache-19.0.0-358 1.7 MB/s | 147 kB 00:00 2024-05-14T09:19:32.764 INFO:teuthology.orchestra.run.smithi029.stdout:(3/156): ceph-immutable-object-cache-19.0.0-358 1.1 MB/s | 147 kB 00:00 2024-05-14T09:19:32.997 INFO:teuthology.orchestra.run.smithi029.stdout:(4/156): ceph-base-19.0.0-3588.ge2b755a6.el9.x8 5.3 MB/s | 5.4 MB 00:01 2024-05-14T09:19:33.234 INFO:teuthology.orchestra.run.smithi002.stdout:(4/156): ceph-mds-19.0.0-3588.ge2b755a6.el9.x86 4.2 MB/s | 2.3 MB 00:00 2024-05-14T09:19:33.256 INFO:teuthology.orchestra.run.smithi029.stdout:(5/156): ceph-mgr-19.0.0-3588.ge2b755a6.el9.x86 5.8 MB/s | 1.5 MB 00:00 2024-05-14T09:19:33.317 INFO:teuthology.orchestra.run.smithi002.stdout:(5/156): ceph-base-19.0.0-3588.ge2b755a6.el9.x8 4.7 MB/s | 5.4 MB 00:01 2024-05-14T09:19:33.607 INFO:teuthology.orchestra.run.smithi029.stdout:(6/156): ceph-mds-19.0.0-3588.ge2b755a6.el9.x86 2.7 MB/s | 2.3 MB 00:00 2024-05-14T09:19:33.626 INFO:teuthology.orchestra.run.smithi002.stdout:(6/156): ceph-mgr-19.0.0-3588.ge2b755a6.el9.x86 3.8 MB/s | 1.5 MB 00:00 2024-05-14T09:19:33.958 INFO:teuthology.orchestra.run.smithi029.stdout:(7/156): ceph-mon-19.0.0-3588.ge2b755a6.el9.x86 7.0 MB/s | 4.9 MB 00:00 2024-05-14T09:19:34.260 INFO:teuthology.orchestra.run.smithi002.stdout:(7/156): ceph-mon-19.0.0-3588.ge2b755a6.el9.x86 5.2 MB/s | 4.9 MB 00:00 2024-05-14T09:19:35.043 INFO:teuthology.orchestra.run.smithi002.stdout:(8/156): ceph-common-19.0.0-3588.ge2b755a6.el9. 6.8 MB/s | 20 MB 00:02 2024-05-14T09:19:35.102 INFO:teuthology.orchestra.run.smithi002.stdout:(9/156): ceph-selinux-19.0.0-3588.ge2b755a6.el9 432 kB/s | 25 kB 00:00 2024-05-14T09:19:35.988 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-14T09:19:36.169 INFO:teuthology.orchestra.run.smithi002.stdout:(10/156): ceph-radosgw-19.0.0-3588.ge2b755a6.el 5.3 MB/s | 10 MB 00:01 2024-05-14T09:19:36.269 INFO:teuthology.orchestra.run.smithi002.stdout:(11/156): cephfs-mirror-19.0.0-3588.ge2b755a6.e 2.2 MB/s | 224 kB 00:00 2024-05-14T09:19:36.352 INFO:teuthology.orchestra.run.smithi002.stdout:(12/156): cephfs-shell-19.0.0-3588.ge2b755a6.el 325 kB/s | 27 kB 00:00 2024-05-14T09:19:36.439 INFO:teuthology.orchestra.run.smithi002.stdout:(13/156): libcephfs-devel-19.0.0-3588.ge2b755a6 368 kB/s | 32 kB 00:00 2024-05-14T09:19:36.491 INFO:teuthology.orchestra.run.smithi029.stdout:(8/156): ceph-radosgw-19.0.0-3588.ge2b755a6.el9 4.0 MB/s | 10 MB 00:02 2024-05-14T09:19:36.592 INFO:teuthology.orchestra.run.smithi029.stdout:(9/156): ceph-selinux-19.0.0-3588.ge2b755a6.el9 251 kB/s | 25 kB 00:00 2024-05-14T09:19:36.603 INFO:teuthology.orchestra.run.smithi002.stdout:(14/156): libcephfs2-19.0.0-3588.ge2b755a6.el9. 4.5 MB/s | 756 kB 00:00 2024-05-14T09:19:36.695 INFO:teuthology.orchestra.run.smithi002.stdout:(15/156): libcephsqlite-19.0.0-3588.ge2b755a6.e 1.7 MB/s | 157 kB 00:00 2024-05-14T09:19:36.787 INFO:teuthology.orchestra.run.smithi002.stdout:(16/156): librados-devel-19.0.0-3588.ge2b755a6. 1.3 MB/s | 127 kB 00:00 2024-05-14T09:19:36.962 INFO:teuthology.orchestra.run.smithi002.stdout:(17/156): libradosstriper1-19.0.0-3588.ge2b755a 2.8 MB/s | 497 kB 00:00 2024-05-14T09:19:37.820 INFO:teuthology.orchestra.run.smithi002.stdout:(18/156): librgw2-19.0.0-3588.ge2b755a6.el9.x86 6.0 MB/s | 5.2 MB 00:00 2024-05-14T09:19:37.888 INFO:teuthology.orchestra.run.smithi002.stdout:(19/156): python3-ceph-argparse-19.0.0-3588.ge2 677 kB/s | 45 kB 00:00 2024-05-14T09:19:37.971 INFO:teuthology.orchestra.run.smithi002.stdout:(20/156): python3-ceph-common-19.0.0-3588.ge2b7 1.6 MB/s | 135 kB 00:00 2024-05-14T09:19:38.046 INFO:teuthology.orchestra.run.smithi002.stdout:(21/156): python3-cephfs-19.0.0-3588.ge2b755a6. 2.1 MB/s | 162 kB 00:00 2024-05-14T09:19:38.113 INFO:teuthology.orchestra.run.smithi002.stdout:(22/156): python3-rados-19.0.0-3588.ge2b755a6.e 4.7 MB/s | 321 kB 00:00 2024-05-14T09:19:38.180 INFO:teuthology.orchestra.run.smithi002.stdout:(23/156): python3-rbd-19.0.0-3588.ge2b755a6.el9 4.4 MB/s | 301 kB 00:00 2024-05-14T09:19:38.239 INFO:teuthology.orchestra.run.smithi002.stdout:(24/156): python3-rgw-19.0.0-3588.ge2b755a6.el9 1.7 MB/s | 100 kB 00:00 2024-05-14T09:19:38.276 INFO:teuthology.orchestra.run.smithi029.stdout:(10/156): ceph-osd-19.0.0-3588.ge2b755a6.el9.x8 3.8 MB/s | 18 MB 00:04 2024-05-14T09:19:38.315 INFO:teuthology.orchestra.run.smithi002.stdout:(25/156): rbd-fuse-19.0.0-3588.ge2b755a6.el9.x8 1.1 MB/s | 86 kB 00:00 2024-05-14T09:19:38.334 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T09:19:38.334 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T09:19:38.345 INFO:teuthology.orchestra.run.smithi026.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-14T09:19:38.346 INFO:teuthology.orchestra.run.smithi026.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-14T09:19:38.347 INFO:teuthology.orchestra.run.smithi026.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-14T09:19:38.348 INFO:teuthology.orchestra.run.smithi026.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-14T09:19:38.349 INFO:teuthology.orchestra.run.smithi026.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-14T09:19:38.352 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-14T09:19:38.353 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-14T09:19:38.416 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-05-14T09:19:38.431 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-05-14T09:19:38.432 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-05-14T09:19:38.432 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-05-14T09:19:38.432 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-05-14T09:19:38.432 INFO:teuthology.orchestra.run.smithi026.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-14T09:19:38.432 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 6.6 k 2024-05-14T09:19:38.432 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 5.4 M 2024-05-14T09:19:38.432 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 863 k 2024-05-14T09:19:38.432 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 147 k 2024-05-14T09:19:38.432 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 1.5 M 2024-05-14T09:19:38.432 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 142 k 2024-05-14T09:19:38.432 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 3.5 M 2024-05-14T09:19:38.432 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 7.4 M 2024-05-14T09:19:38.433 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 49 k 2024-05-14T09:19:38.433 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 10 M 2024-05-14T09:19:38.433 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 43 M 2024-05-14T09:19:38.433 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 273 k 2024-05-14T09:19:38.433 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 763 k 2024-05-14T09:19:38.433 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 224 k 2024-05-14T09:19:38.433 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 27 k 2024-05-14T09:19:38.433 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 18 k 2024-05-14T09:19:38.433 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-14T09:19:38.433 INFO:teuthology.orchestra.run.smithi026.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-14T09:19:38.433 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-14T09:19:38.433 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 32 k 2024-05-14T09:19:38.433 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 756 k 2024-05-14T09:19:38.433 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 127 k 2024-05-14T09:19:38.433 INFO:teuthology.orchestra.run.smithi026.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-14T09:19:38.434 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-14T09:19:38.434 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-14T09:19:38.434 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-14T09:19:38.434 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-14T09:19:38.434 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-14T09:19:38.434 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 162 k 2024-05-14T09:19:38.434 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 321 k 2024-05-14T09:19:38.434 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 301 k 2024-05-14T09:19:38.434 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 100 k 2024-05-14T09:19:38.434 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 86 k 2024-05-14T09:19:38.434 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.0 M 2024-05-14T09:19:38.434 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 170 k 2024-05-14T09:19:38.434 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-14T09:19:38.435 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2024-05-14T09:19:38.435 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-14T09:19:38.435 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-14T09:19:38.435 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-14T09:19:38.435 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-14T09:19:38.435 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.3 M 2024-05-14T09:19:38.435 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.1 M 2024-05-14T09:19:38.435 INFO:teuthology.orchestra.run.smithi026.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-14T09:19:38.435 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-14T09:19:38.435 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-05-14T09:19:38.435 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-14T09:19:38.435 INFO:teuthology.orchestra.run.smithi026.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-14T09:19:38.435 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-14T09:19:38.435 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 20 M 2024-05-14T09:19:38.436 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 34 k 2024-05-14T09:19:38.436 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 2.3 M 2024-05-14T09:19:38.436 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 274 k 2024-05-14T09:19:38.436 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 4.9 M 2024-05-14T09:19:38.436 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 18 M 2024-05-14T09:19:38.436 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 16 k 2024-05-14T09:19:38.436 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 25 k 2024-05-14T09:19:38.436 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-14T09:19:38.436 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-14T09:19:38.436 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-14T09:19:38.436 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-14T09:19:38.436 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-14T09:19:38.436 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-14T09:19:38.436 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-14T09:19:38.437 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-14T09:19:38.437 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 157 k 2024-05-14T09:19:38.437 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-14T09:19:38.437 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-14T09:19:38.437 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-14T09:19:38.437 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-14T09:19:38.437 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-14T09:19:38.437 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 497 k 2024-05-14T09:19:38.437 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-14T09:19:38.437 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 5.2 M 2024-05-14T09:19:38.437 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-14T09:19:38.437 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-14T09:19:38.437 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-14T09:19:38.437 INFO:teuthology.orchestra.run.smithi026.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-14T09:19:38.438 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-14T09:19:38.438 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-14T09:19:38.438 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-14T09:19:38.438 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-14T09:19:38.438 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-14T09:19:38.438 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-14T09:19:38.438 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-14T09:19:38.438 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-14T09:19:38.438 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-14T09:19:38.438 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-14T09:19:38.438 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-14T09:19:38.438 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-14T09:19:38.438 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-14T09:19:38.439 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 45 k 2024-05-14T09:19:38.439 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 135 k 2024-05-14T09:19:38.439 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-14T09:19:38.439 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-14T09:19:38.439 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-14T09:19:38.439 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-14T09:19:38.439 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-14T09:19:38.439 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-14T09:19:38.439 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-14T09:19:38.439 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-14T09:19:38.439 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-14T09:19:38.439 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-14T09:19:38.439 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-14T09:19:38.439 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-14T09:19:38.439 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-14T09:19:38.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-14T09:19:38.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-14T09:19:38.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-14T09:19:38.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-14T09:19:38.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-14T09:19:38.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-14T09:19:38.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-14T09:19:38.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-14T09:19:38.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-14T09:19:38.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-14T09:19:38.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-14T09:19:38.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-14T09:19:38.441 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-14T09:19:38.441 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-14T09:19:38.441 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-14T09:19:38.441 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-14T09:19:38.441 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-14T09:19:38.441 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-14T09:19:38.441 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-14T09:19:38.441 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-14T09:19:38.441 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-14T09:19:38.441 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-14T09:19:38.441 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-14T09:19:38.442 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-14T09:19:38.442 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-14T09:19:38.442 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-14T09:19:38.442 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-14T09:19:38.442 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-14T09:19:38.442 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-14T09:19:38.442 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-14T09:19:38.442 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-14T09:19:38.442 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-14T09:19:38.442 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-14T09:19:38.442 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-14T09:19:38.442 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-14T09:19:38.442 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-14T09:19:38.442 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-14T09:19:38.443 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-14T09:19:38.443 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-14T09:19:38.443 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-14T09:19:38.443 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-14T09:19:38.443 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-14T09:19:38.443 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-14T09:19:38.443 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-14T09:19:38.443 INFO:teuthology.orchestra.run.smithi026.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-14T09:19:38.443 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-14T09:19:38.443 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-14T09:19:38.443 INFO:teuthology.orchestra.run.smithi026.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-14T09:19:38.443 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-14T09:19:38.443 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-14T09:19:38.444 INFO:teuthology.orchestra.run.smithi026.stdout:Installing weak dependencies: 2024-05-14T09:19:38.444 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-14T09:19:38.444 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-14T09:19:38.444 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-05-14T09:19:38.444 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-05-14T09:19:38.444 INFO:teuthology.orchestra.run.smithi026.stdout:Install 148 Packages 2024-05-14T09:19:38.444 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 8 Packages 2024-05-14T09:19:38.444 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-14T09:19:38.444 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 219 M 2024-05-14T09:19:38.445 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-05-14T09:19:38.468 INFO:teuthology.orchestra.run.smithi029.stdout:(11/156): cephfs-mirror-19.0.0-3588.ge2b755a6.e 1.1 MB/s | 224 kB 00:00 2024-05-14T09:19:38.577 INFO:teuthology.orchestra.run.smithi029.stdout:(12/156): cephfs-shell-19.0.0-3588.ge2b755a6.el 249 kB/s | 27 kB 00:00 2024-05-14T09:19:38.686 INFO:teuthology.orchestra.run.smithi029.stdout:(13/156): libcephfs-devel-19.0.0-3588.ge2b755a6 292 kB/s | 32 kB 00:00 2024-05-14T09:19:38.798 INFO:teuthology.orchestra.run.smithi002.stdout:(26/156): rbd-mirror-19.0.0-3588.ge2b755a6.el9. 6.3 MB/s | 3.0 MB 00:00 2024-05-14T09:19:38.928 INFO:teuthology.orchestra.run.smithi029.stdout:(14/156): libcephfs2-19.0.0-3588.ge2b755a6.el9. 3.1 MB/s | 756 kB 00:00 2024-05-14T09:19:39.029 INFO:teuthology.orchestra.run.smithi029.stdout:(15/156): libcephsqlite-19.0.0-3588.ge2b755a6.e 1.5 MB/s | 157 kB 00:00 2024-05-14T09:19:39.164 INFO:teuthology.orchestra.run.smithi026.stdout:(1/156): ceph-19.0.0-3588.ge2b755a6.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-05-14T09:19:39.182 INFO:teuthology.orchestra.run.smithi002.stdout:(27/156): ceph-test-19.0.0-3588.ge2b755a6.el9.x 11 MB/s | 43 MB 00:04 2024-05-14T09:19:39.207 INFO:teuthology.orchestra.run.smithi002.stdout:(28/156): rbd-nbd-19.0.0-3588.ge2b755a6.el9.x86 415 kB/s | 170 kB 00:00 2024-05-14T09:19:39.232 INFO:teuthology.orchestra.run.smithi002.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3588.g 686 kB/s | 34 kB 00:00 2024-05-14T09:19:39.258 INFO:teuthology.orchestra.run.smithi002.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3588.ge2b755a 2.8 MB/s | 142 kB 00:00 2024-05-14T09:19:39.348 INFO:teuthology.orchestra.run.smithi029.stdout:(16/156): ceph-common-19.0.0-3588.ge2b755a6.el9 2.7 MB/s | 20 MB 00:07 2024-05-14T09:19:39.373 INFO:teuthology.orchestra.run.smithi029.stdout:(17/156): librados-devel-19.0.0-3588.ge2b755a6. 369 kB/s | 127 kB 00:00 2024-05-14T09:19:39.441 INFO:teuthology.orchestra.run.smithi002.stdout:(31/156): ceph-osd-19.0.0-3588.ge2b755a6.el9.x8 3.1 MB/s | 18 MB 00:05 2024-05-14T09:19:39.572 INFO:teuthology.orchestra.run.smithi026.stdout:(2/156): ceph-fuse-19.0.0-3588.ge2b755a6.el9.x8 2.1 MB/s | 863 kB 00:00 2024-05-14T09:19:39.608 INFO:teuthology.orchestra.run.smithi002.stdout:(32/156): ceph-mgr-dashboard-19.0.0-3588.ge2b75 9.4 MB/s | 3.5 MB 00:00 2024-05-14T09:19:39.675 INFO:teuthology.orchestra.run.smithi002.stdout:(33/156): ceph-mgr-modules-core-19.0.0-3588.ge2 1.1 MB/s | 274 kB 00:00 2024-05-14T09:19:39.681 INFO:teuthology.orchestra.run.smithi026.stdout:(3/156): ceph-immutable-object-cache-19.0.0-358 1.3 MB/s | 147 kB 00:00 2024-05-14T09:19:39.690 INFO:teuthology.orchestra.run.smithi029.stdout:(18/156): libradosstriper1-19.0.0-3588.ge2b755a 1.4 MB/s | 497 kB 00:00 2024-05-14T09:19:39.700 INFO:teuthology.orchestra.run.smithi002.stdout:(34/156): ceph-mgr-rook-19.0.0-3588.ge2b755a6.e 538 kB/s | 49 kB 00:00 2024-05-14T09:19:39.726 INFO:teuthology.orchestra.run.smithi002.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3588.ge 327 kB/s | 16 kB 00:00 2024-05-14T09:19:39.776 INFO:teuthology.orchestra.run.smithi002.stdout:(36/156): ceph-volume-19.0.0-3588.ge2b755a6.el9 3.5 MB/s | 273 kB 00:00 2024-05-14T09:19:39.782 INFO:teuthology.orchestra.run.smithi029.stdout:(19/156): python3-ceph-argparse-19.0.0-3588.ge2 491 kB/s | 45 kB 00:00 2024-05-14T09:19:39.843 INFO:teuthology.orchestra.run.smithi002.stdout:(37/156): cephfs-top-19.0.0-3588.ge2b755a6.el9. 273 kB/s | 18 kB 00:00 2024-05-14T09:19:39.893 INFO:teuthology.orchestra.run.smithi029.stdout:(20/156): python3-ceph-common-19.0.0-3588.ge2b7 1.2 MB/s | 135 kB 00:00 2024-05-14T09:19:39.993 INFO:teuthology.orchestra.run.smithi002.stdout:(38/156): cephadm-19.0.0-3588.ge2b755a6.el9.noa 2.8 MB/s | 763 kB 00:00 2024-05-14T09:19:40.000 INFO:teuthology.orchestra.run.smithi029.stdout:(21/156): python3-cephfs-19.0.0-3588.ge2b755a6. 1.5 MB/s | 162 kB 00:00 2024-05-14T09:19:40.184 INFO:teuthology.orchestra.run.smithi029.stdout:(22/156): python3-rados-19.0.0-3588.ge2b755a6.e 1.7 MB/s | 321 kB 00:00 2024-05-14T09:19:40.210 INFO:teuthology.orchestra.run.smithi002.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 7.8 MB/s | 7.4 MB 00:00 2024-05-14T09:19:40.231 INFO:teuthology.orchestra.run.smithi026.stdout:(4/156): ceph-mds-19.0.0-3588.ge2b755a6.el9.x86 4.1 MB/s | 2.3 MB 00:00 2024-05-14T09:19:40.327 INFO:teuthology.orchestra.run.smithi029.stdout:(23/156): python3-rbd-19.0.0-3588.ge2b755a6.el9 2.1 MB/s | 301 kB 00:00 2024-05-14T09:19:40.343 INFO:teuthology.orchestra.run.smithi002.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 160 kB/s | 80 kB 00:00 2024-05-14T09:19:40.419 INFO:teuthology.orchestra.run.smithi029.stdout:(24/156): python3-rgw-19.0.0-3588.ge2b755a6.el9 1.1 MB/s | 100 kB 00:00 2024-05-14T09:19:40.431 INFO:teuthology.orchestra.run.smithi026.stdout:(5/156): ceph-mgr-19.0.0-3588.ge2b755a6.el9.x86 7.5 MB/s | 1.5 MB 00:00 2024-05-14T09:19:40.494 INFO:teuthology.orchestra.run.smithi029.stdout:(25/156): rbd-fuse-19.0.0-3588.ge2b755a6.el9.x8 1.1 MB/s | 86 kB 00:00 2024-05-14T09:19:40.710 INFO:teuthology.orchestra.run.smithi002.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 691 kB/s | 253 kB 00:00 2024-05-14T09:19:40.753 INFO:teuthology.orchestra.run.smithi002.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 360 kB/s | 195 kB 00:00 2024-05-14T09:19:40.828 INFO:teuthology.orchestra.run.smithi002.stdout:(43/156): libgfortran-11.4.1-3.el9.x86_64.rpm 962 kB/s | 803 kB 00:00 2024-05-14T09:19:40.861 INFO:teuthology.orchestra.run.smithi002.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 982 kB/s | 106 kB 00:00 2024-05-14T09:19:40.936 INFO:teuthology.orchestra.run.smithi002.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-05-14T09:19:40.970 INFO:teuthology.orchestra.run.smithi002.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-14T09:19:40.984 INFO:teuthology.orchestra.run.smithi026.stdout:(6/156): ceph-base-19.0.0-3588.ge2b755a6.el9.x8 2.7 MB/s | 5.4 MB 00:01 2024-05-14T09:19:40.995 INFO:teuthology.orchestra.run.smithi029.stdout:(26/156): librgw2-19.0.0-3588.ge2b755a6.el9.x86 3.2 MB/s | 5.2 MB 00:01 2024-05-14T09:19:41.062 INFO:teuthology.orchestra.run.smithi002.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-05-14T09:19:41.070 INFO:teuthology.orchestra.run.smithi029.stdout:(27/156): rbd-nbd-19.0.0-3588.ge2b755a6.el9.x86 2.2 MB/s | 170 kB 00:00 2024-05-14T09:19:41.074 INFO:teuthology.orchestra.run.smithi026.stdout:(7/156): ceph-mon-19.0.0-3588.ge2b755a6.el9.x86 7.6 MB/s | 4.9 MB 00:00 2024-05-14T09:19:41.138 INFO:teuthology.orchestra.run.smithi029.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3588.g 510 kB/s | 34 kB 00:00 2024-05-14T09:19:41.162 INFO:teuthology.orchestra.run.smithi002.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-05-14T09:19:41.213 INFO:teuthology.orchestra.run.smithi002.stdout:(49/156): python3-cryptography-36.0.1-4.el9.x86 2.4 MB/s | 1.2 MB 00:00 2024-05-14T09:19:41.238 INFO:teuthology.orchestra.run.smithi002.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-05-14T09:19:41.238 INFO:teuthology.orchestra.run.smithi029.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3588.ge2b755a 1.4 MB/s | 142 kB 00:00 2024-05-14T09:19:41.288 INFO:teuthology.orchestra.run.smithi002.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-05-14T09:19:41.314 INFO:teuthology.orchestra.run.smithi002.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-05-14T09:19:41.331 INFO:teuthology.orchestra.run.smithi002.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 781 kB/s | 33 kB 00:00 2024-05-14T09:19:41.356 INFO:teuthology.orchestra.run.smithi002.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-05-14T09:19:41.373 INFO:teuthology.orchestra.run.smithi002.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 435 kB/s | 18 kB 00:00 2024-05-14T09:19:41.390 INFO:teuthology.orchestra.run.smithi002.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 333 kB/s | 11 kB 00:00 2024-05-14T09:19:41.415 INFO:teuthology.orchestra.run.smithi002.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-05-14T09:19:41.432 INFO:teuthology.orchestra.run.smithi002.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-05-14T09:19:41.474 INFO:teuthology.orchestra.run.smithi002.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-14T09:19:41.508 INFO:teuthology.orchestra.run.smithi002.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-05-14T09:19:41.515 INFO:teuthology.orchestra.run.smithi029.stdout:(30/156): rbd-mirror-19.0.0-3588.ge2b755a6.el9. 3.0 MB/s | 3.0 MB 00:01 2024-05-14T09:19:41.524 INFO:teuthology.orchestra.run.smithi002.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 508 kB/s | 25 kB 00:00 2024-05-14T09:19:41.541 INFO:teuthology.orchestra.run.smithi002.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-05-14T09:19:41.567 INFO:teuthology.orchestra.run.smithi002.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 292 kB 00:00 2024-05-14T09:19:41.592 INFO:teuthology.orchestra.run.smithi002.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-14T09:19:41.617 INFO:teuthology.orchestra.run.smithi002.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 699 kB/s | 35 kB 00:00 2024-05-14T09:19:41.668 INFO:teuthology.orchestra.run.smithi002.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.4 MB/s | 3.0 MB 00:00 2024-05-14T09:19:41.751 INFO:teuthology.orchestra.run.smithi002.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-05-14T09:19:41.793 INFO:teuthology.orchestra.run.smithi002.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-14T09:19:41.860 INFO:teuthology.orchestra.run.smithi002.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-05-14T09:19:41.885 INFO:teuthology.orchestra.run.smithi002.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-14T09:19:41.928 INFO:teuthology.orchestra.run.smithi002.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 5.1 MB/s | 911 kB 00:00 2024-05-14T09:19:41.961 INFO:teuthology.orchestra.run.smithi002.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-05-14T09:19:41.974 INFO:teuthology.orchestra.run.smithi026.stdout:(8/156): ceph-radosgw-19.0.0-3588.ge2b755a6.el9 11 MB/s | 10 MB 00:00 2024-05-14T09:19:41.986 INFO:teuthology.orchestra.run.smithi002.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-14T09:19:42.024 INFO:teuthology.orchestra.run.smithi026.stdout:(9/156): ceph-selinux-19.0.0-3588.ge2b755a6.el9 505 kB/s | 25 kB 00:00 2024-05-14T09:19:42.028 INFO:teuthology.orchestra.run.smithi002.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-05-14T09:19:42.112 INFO:teuthology.orchestra.run.smithi002.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-05-14T09:19:42.137 INFO:teuthology.orchestra.run.smithi002.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-05-14T09:19:42.154 INFO:teuthology.orchestra.run.smithi002.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 618 kB/s | 26 kB 00:00 2024-05-14T09:19:42.171 INFO:teuthology.orchestra.run.smithi002.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 573 kB/s | 19 kB 00:00 2024-05-14T09:19:42.196 INFO:teuthology.orchestra.run.smithi002.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-05-14T09:19:42.213 INFO:teuthology.orchestra.run.smithi002.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 833 kB/s | 35 kB 00:00 2024-05-14T09:19:42.297 INFO:teuthology.orchestra.run.smithi002.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-05-14T09:19:42.322 INFO:teuthology.orchestra.run.smithi002.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-05-14T09:19:42.348 INFO:teuthology.orchestra.run.smithi002.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-14T09:19:42.366 INFO:teuthology.orchestra.run.smithi002.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-05-14T09:19:42.381 INFO:teuthology.orchestra.run.smithi002.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 42 kB 00:00 2024-05-14T09:19:42.407 INFO:teuthology.orchestra.run.smithi002.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-05-14T09:19:42.432 INFO:teuthology.orchestra.run.smithi002.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-14T09:19:42.457 INFO:teuthology.orchestra.run.smithi002.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-14T09:19:42.474 INFO:teuthology.orchestra.run.smithi002.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-05-14T09:19:42.491 INFO:teuthology.orchestra.run.smithi002.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-05-14T09:19:42.575 INFO:teuthology.orchestra.run.smithi002.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-05-14T09:19:42.591 INFO:teuthology.orchestra.run.smithi002.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-05-14T09:19:42.608 INFO:teuthology.orchestra.run.smithi002.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-05-14T09:19:42.634 INFO:teuthology.orchestra.run.smithi002.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.1 MB/s | 221 kB 00:00 2024-05-14T09:19:42.651 INFO:teuthology.orchestra.run.smithi002.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 857 kB/s | 36 kB 00:00 2024-05-14T09:19:42.668 INFO:teuthology.orchestra.run.smithi002.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-05-14T09:19:42.693 INFO:teuthology.orchestra.run.smithi002.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 305 kB 00:00 2024-05-14T09:19:42.707 INFO:teuthology.orchestra.run.smithi029.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3588.ge2b75 2.4 MB/s | 3.5 MB 00:01 2024-05-14T09:19:42.710 INFO:teuthology.orchestra.run.smithi002.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-05-14T09:19:42.735 INFO:teuthology.orchestra.run.smithi002.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-05-14T09:19:42.752 INFO:teuthology.orchestra.run.smithi002.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-05-14T09:19:42.769 INFO:teuthology.orchestra.run.smithi002.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-05-14T09:19:42.908 INFO:teuthology.orchestra.run.smithi029.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3588.ge2 1.3 MB/s | 274 kB 00:00 2024-05-14T09:19:42.986 INFO:teuthology.orchestra.run.smithi002.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-05-14T09:19:43.003 INFO:teuthology.orchestra.run.smithi002.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 89 kB/s | 22 kB 00:00 2024-05-14T09:19:43.017 INFO:teuthology.orchestra.run.smithi029.stdout:(33/156): ceph-mgr-rook-19.0.0-3588.ge2b755a6.e 454 kB/s | 49 kB 00:00 2024-05-14T09:19:43.126 INFO:teuthology.orchestra.run.smithi029.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3588.ge 151 kB/s | 16 kB 00:00 2024-05-14T09:19:43.352 INFO:teuthology.orchestra.run.smithi029.stdout:(35/156): ceph-volume-19.0.0-3588.ge2b755a6.el9 1.2 MB/s | 273 kB 00:00 2024-05-14T09:19:43.570 INFO:teuthology.orchestra.run.smithi002.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-05-14T09:19:43.704 INFO:teuthology.orchestra.run.smithi002.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 441 kB/s | 309 kB 00:00 2024-05-14T09:19:43.771 INFO:teuthology.orchestra.run.smithi002.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 699 kB/s | 548 kB 00:00 2024-05-14T09:19:43.819 INFO:teuthology.orchestra.run.smithi029.stdout:(36/156): cephadm-19.0.0-3588.ge2b755a6.el9.noa 1.6 MB/s | 763 kB 00:00 2024-05-14T09:19:43.888 INFO:teuthology.orchestra.run.smithi002.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 222 kB/s | 26 kB 00:00 2024-05-14T09:19:43.913 INFO:teuthology.orchestra.run.smithi002.stdout:(108/156): grpc-data-1.46.7-10.el9.noarch.rpm 57 kB/s | 19 kB 00:00 2024-05-14T09:19:43.938 INFO:teuthology.orchestra.run.smithi029.stdout:(37/156): cephfs-top-19.0.0-3588.ge2b755a6.el9. 156 kB/s | 18 kB 00:00 2024-05-14T09:19:44.005 INFO:teuthology.orchestra.run.smithi002.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-05-14T09:19:44.054 INFO:teuthology.orchestra.run.smithi029.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 687 kB/s | 80 kB 00:00 2024-05-14T09:19:44.147 INFO:teuthology.orchestra.run.smithi002.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-05-14T09:19:44.171 INFO:teuthology.orchestra.run.smithi029.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-05-14T09:19:44.172 INFO:teuthology.orchestra.run.smithi002.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 260 kB/s | 67 kB 00:00 2024-05-14T09:19:44.213 INFO:teuthology.orchestra.run.smithi029.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-05-14T09:19:44.239 INFO:teuthology.orchestra.run.smithi002.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 4.4 MB 00:00 2024-05-14T09:19:44.241 INFO:teuthology.orchestra.run.smithi026.stdout:(10/156): ceph-common-19.0.0-3588.ge2b755a6.el9 3.7 MB/s | 20 MB 00:05 2024-05-14T09:19:44.264 INFO:teuthology.orchestra.run.smithi029.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-05-14T09:19:44.333 INFO:teuthology.orchestra.run.smithi026.stdout:(11/156): cephfs-mirror-19.0.0-3588.ge2b755a6.e 2.4 MB/s | 224 kB 00:00 2024-05-14T09:19:44.348 INFO:teuthology.orchestra.run.smithi029.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-05-14T09:19:44.348 INFO:teuthology.orchestra.run.smithi002.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 401 kB/s | 43 kB 00:00 2024-05-14T09:19:44.373 INFO:teuthology.orchestra.run.smithi002.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 839 kB 00:00 2024-05-14T09:19:44.391 INFO:teuthology.orchestra.run.smithi029.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.4 MB/s | 106 kB 00:00 2024-05-14T09:19:44.410 INFO:teuthology.orchestra.run.smithi026.stdout:(12/156): cephfs-shell-19.0.0-3588.ge2b755a6.el 362 kB/s | 27 kB 00:00 2024-05-14T09:19:44.442 INFO:teuthology.orchestra.run.smithi029.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-14T09:19:44.448 INFO:teuthology.orchestra.run.smithi002.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-05-14T09:19:44.465 INFO:teuthology.orchestra.run.smithi002.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-05-14T09:19:44.484 INFO:teuthology.orchestra.run.smithi029.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-05-14T09:19:44.492 INFO:teuthology.orchestra.run.smithi026.stdout:(13/156): libcephfs-devel-19.0.0-3588.ge2b755a6 380 kB/s | 32 kB 00:00 2024-05-14T09:19:44.535 INFO:teuthology.orchestra.run.smithi029.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-14T09:19:44.549 INFO:teuthology.orchestra.run.smithi002.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 1.4 MB/s | 548 kB 00:00 2024-05-14T09:19:44.574 INFO:teuthology.orchestra.run.smithi002.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-05-14T09:19:44.599 INFO:teuthology.orchestra.run.smithi002.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-05-14T09:19:44.602 INFO:teuthology.orchestra.run.smithi029.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 10 MB/s | 689 kB 00:00 2024-05-14T09:19:44.661 INFO:teuthology.orchestra.run.smithi029.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-05-14T09:19:44.666 INFO:teuthology.orchestra.run.smithi002.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-05-14T09:19:44.700 INFO:teuthology.orchestra.run.smithi002.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-05-14T09:19:44.703 INFO:teuthology.orchestra.run.smithi029.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-05-14T09:19:44.716 INFO:teuthology.orchestra.run.smithi026.stdout:(14/156): libcephfs2-19.0.0-3588.ge2b755a6.el9. 3.3 MB/s | 756 kB 00:00 2024-05-14T09:19:44.745 INFO:teuthology.orchestra.run.smithi029.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 514 kB/s | 21 kB 00:00 2024-05-14T09:19:44.759 INFO:teuthology.orchestra.run.smithi002.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 1.3 MB/s | 206 kB 00:00 2024-05-14T09:19:44.796 INFO:teuthology.orchestra.run.smithi029.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-14T09:19:44.826 INFO:teuthology.orchestra.run.smithi026.stdout:(15/156): libcephsqlite-19.0.0-3588.ge2b755a6.e 1.4 MB/s | 157 kB 00:00 2024-05-14T09:19:44.838 INFO:teuthology.orchestra.run.smithi029.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 778 kB/s | 33 kB 00:00 2024-05-14T09:19:44.859 INFO:teuthology.orchestra.run.smithi002.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-05-14T09:19:44.909 INFO:teuthology.orchestra.run.smithi002.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-05-14T09:19:44.934 INFO:teuthology.orchestra.run.smithi002.stdout:(125/156): python3-grpcio-tools-1.46.7-10.el9.x 617 kB/s | 144 kB 00:00 2024-05-14T09:19:44.956 INFO:teuthology.orchestra.run.smithi029.stdout:(53/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-05-14T09:19:44.960 INFO:teuthology.orchestra.run.smithi002.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 175 kB/s | 18 kB 00:00 2024-05-14T09:19:44.998 INFO:teuthology.orchestra.run.smithi029.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 435 kB/s | 18 kB 00:00 2024-05-14T09:19:45.027 INFO:teuthology.orchestra.run.smithi002.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 197 kB/s | 23 kB 00:00 2024-05-14T09:19:45.040 INFO:teuthology.orchestra.run.smithi029.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 265 kB/s | 11 kB 00:00 2024-05-14T09:19:45.043 INFO:teuthology.orchestra.run.smithi002.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 178 kB/s | 19 kB 00:00 2024-05-14T09:19:45.060 INFO:teuthology.orchestra.run.smithi002.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 195 kB/s | 20 kB 00:00 2024-05-14T09:19:45.136 INFO:teuthology.orchestra.run.smithi002.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 503 kB/s | 46 kB 00:00 2024-05-14T09:19:45.161 INFO:teuthology.orchestra.run.smithi002.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 786 kB/s | 79 kB 00:00 2024-05-14T09:19:45.191 INFO:teuthology.orchestra.run.smithi029.stdout:(56/156): ceph-mgr-diskprediction-local-19.0.0- 2.0 MB/s | 7.4 MB 00:03 2024-05-14T09:19:45.195 INFO:teuthology.orchestra.run.smithi002.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-05-14T09:19:45.218 INFO:teuthology.orchestra.run.smithi029.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 903 kB/s | 160 kB 00:00 2024-05-14T09:19:45.245 INFO:teuthology.orchestra.run.smithi002.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 531 kB/s | 58 kB 00:00 2024-05-14T09:19:45.260 INFO:teuthology.orchestra.run.smithi026.stdout:(16/156): ceph-test-19.0.0-3588.ge2b755a6.el9.x 13 MB/s | 43 MB 00:03 2024-05-14T09:19:45.268 INFO:teuthology.orchestra.run.smithi029.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-05-14T09:19:45.285 INFO:teuthology.orchestra.run.smithi026.stdout:(17/156): librados-devel-19.0.0-3588.ge2b755a6. 277 kB/s | 127 kB 00:00 2024-05-14T09:19:45.287 INFO:teuthology.orchestra.run.smithi002.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-05-14T09:19:45.294 INFO:teuthology.orchestra.run.smithi029.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 440 kB/s | 45 kB 00:00 2024-05-14T09:19:45.312 INFO:teuthology.orchestra.run.smithi002.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-05-14T09:19:45.318 INFO:teuthology.orchestra.run.smithi026.stdout:(18/156): libradosstriper1-19.0.0-3588.ge2b755a 8.4 MB/s | 497 kB 00:00 2024-05-14T09:19:45.328 INFO:teuthology.orchestra.run.smithi029.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 9.6 MB/s | 582 kB 00:00 2024-05-14T09:19:45.346 INFO:teuthology.orchestra.run.smithi002.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 894 kB/s | 90 kB 00:00 2024-05-14T09:19:45.353 INFO:teuthology.orchestra.run.smithi029.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 432 kB/s | 25 kB 00:00 2024-05-14T09:19:45.360 INFO:teuthology.orchestra.run.smithi026.stdout:(19/156): python3-ceph-argparse-19.0.0-3588.ge2 1.1 MB/s | 45 kB 00:00 2024-05-14T09:19:45.387 INFO:teuthology.orchestra.run.smithi029.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 803 kB/s | 47 kB 00:00 2024-05-14T09:19:45.396 INFO:teuthology.orchestra.run.smithi002.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-05-14T09:19:45.413 INFO:teuthology.orchestra.run.smithi002.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-05-14T09:19:45.418 INFO:teuthology.orchestra.run.smithi026.stdout:(20/156): python3-ceph-common-19.0.0-3588.ge2b7 2.3 MB/s | 135 kB 00:00 2024-05-14T09:19:45.471 INFO:teuthology.orchestra.run.smithi002.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-05-14T09:19:45.486 INFO:teuthology.orchestra.run.smithi026.stdout:(21/156): python3-cephfs-19.0.0-3588.ge2b755a6. 2.4 MB/s | 162 kB 00:00 2024-05-14T09:19:45.514 INFO:teuthology.orchestra.run.smithi002.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 504 kB/s | 59 kB 00:00 2024-05-14T09:19:45.531 INFO:teuthology.orchestra.run.smithi002.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 304 kB/s | 36 kB 00:00 2024-05-14T09:19:45.553 INFO:teuthology.orchestra.run.smithi026.stdout:(22/156): python3-rados-19.0.0-3588.ge2b755a6.e 4.7 MB/s | 321 kB 00:00 2024-05-14T09:19:45.589 INFO:teuthology.orchestra.run.smithi002.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 434 kB/s | 51 kB 00:00 2024-05-14T09:19:45.628 INFO:teuthology.orchestra.run.smithi026.stdout:(23/156): python3-rbd-19.0.0-3588.ge2b755a6.el9 3.9 MB/s | 301 kB 00:00 2024-05-14T09:19:45.687 INFO:teuthology.orchestra.run.smithi026.stdout:(24/156): python3-rgw-19.0.0-3588.ge2b755a6.el9 1.7 MB/s | 100 kB 00:00 2024-05-14T09:19:45.745 INFO:teuthology.orchestra.run.smithi026.stdout:(25/156): rbd-fuse-19.0.0-3588.ge2b755a6.el9.x8 1.4 MB/s | 86 kB 00:00 2024-05-14T09:19:45.748 INFO:teuthology.orchestra.run.smithi002.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 981 kB/s | 230 kB 00:00 2024-05-14T09:19:45.773 INFO:teuthology.orchestra.run.smithi002.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 370 kB/s | 90 kB 00:00 2024-05-14T09:19:45.799 INFO:teuthology.orchestra.run.smithi002.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.0 MB/s | 427 kB 00:00 2024-05-14T09:19:45.858 INFO:teuthology.orchestra.run.smithi002.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-05-14T09:19:45.883 INFO:teuthology.orchestra.run.smithi002.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-14T09:19:45.912 INFO:teuthology.orchestra.run.smithi026.stdout:(26/156): librgw2-19.0.0-3588.ge2b755a6.el9.x86 8.2 MB/s | 5.2 MB 00:00 2024-05-14T09:19:45.979 INFO:teuthology.orchestra.run.smithi026.stdout:(27/156): rbd-mirror-19.0.0-3588.ge2b755a6.el9. 13 MB/s | 3.0 MB 00:00 2024-05-14T09:19:46.005 INFO:teuthology.orchestra.run.smithi026.stdout:(28/156): rbd-nbd-19.0.0-3588.ge2b755a6.el9.x86 1.8 MB/s | 170 kB 00:00 2024-05-14T09:19:46.022 INFO:teuthology.orchestra.run.smithi026.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3588.g 815 kB/s | 34 kB 00:00 2024-05-14T09:19:46.058 INFO:teuthology.orchestra.run.smithi002.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-05-14T09:19:46.072 INFO:teuthology.orchestra.run.smithi026.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3588.ge2b755a 2.1 MB/s | 142 kB 00:00 2024-05-14T09:19:46.138 INFO:teuthology.orchestra.run.smithi029.stdout:(63/156): ceph-test-19.0.0-3588.ge2b755a6.el9.x 4.5 MB/s | 43 MB 00:09 2024-05-14T09:19:46.172 INFO:teuthology.orchestra.run.smithi029.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 240 kB/s | 188 kB 00:00 2024-05-14T09:19:46.206 INFO:teuthology.orchestra.run.smithi029.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 343 kB/s | 292 kB 00:00 2024-05-14T09:19:46.231 INFO:teuthology.orchestra.run.smithi029.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 380 kB/s | 35 kB 00:00 2024-05-14T09:19:46.347 INFO:teuthology.orchestra.run.smithi026.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3588.ge2b75 11 MB/s | 3.5 MB 00:00 2024-05-14T09:19:46.357 INFO:teuthology.orchestra.run.smithi029.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-05-14T09:19:46.374 INFO:teuthology.orchestra.run.smithi002.stdout:(149/156): librbd1-19.0.0-3588.ge2b755a6.el9.x8 6.3 MB/s | 3.1 MB 00:00 2024-05-14T09:19:46.398 INFO:teuthology.orchestra.run.smithi026.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3588.ge2 5.4 MB/s | 274 kB 00:00 2024-05-14T09:19:46.424 INFO:teuthology.orchestra.run.smithi029.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 13 MB/s | 911 kB 00:00 2024-05-14T09:19:46.431 INFO:teuthology.orchestra.run.smithi026.stdout:(33/156): ceph-mgr-rook-19.0.0-3588.ge2b755a6.e 1.5 MB/s | 49 kB 00:00 2024-05-14T09:19:46.465 INFO:teuthology.orchestra.run.smithi026.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3588.ge 493 kB/s | 16 kB 00:00 2024-05-14T09:19:46.541 INFO:teuthology.orchestra.run.smithi029.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-05-14T09:19:46.566 INFO:teuthology.orchestra.run.smithi002.stdout:(150/156): librados2-19.0.0-3588.ge2b755a6.el9. 4.7 MB/s | 3.3 MB 00:00 2024-05-14T09:19:46.575 INFO:teuthology.orchestra.run.smithi029.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 884 kB/s | 133 kB 00:00 2024-05-14T09:19:46.642 INFO:teuthology.orchestra.run.smithi029.stdout:(71/156): postgresql-13.14-1.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-05-14T09:19:46.648 INFO:teuthology.orchestra.run.smithi026.stdout:(35/156): ceph-osd-19.0.0-3588.ge2b755a6.el9.x8 3.1 MB/s | 18 MB 00:05 2024-05-14T09:19:46.674 INFO:teuthology.orchestra.run.smithi026.stdout:(36/156): ceph-volume-19.0.0-3588.ge2b755a6.el9 1.3 MB/s | 273 kB 00:00 2024-05-14T09:19:46.695 INFO:teuthology.orchestra.run.smithi029.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.5 MB/s | 1.0 MB 00:00 2024-05-14T09:19:46.715 INFO:teuthology.orchestra.run.smithi026.stdout:(37/156): cephfs-top-19.0.0-3588.ge2b755a6.el9. 441 kB/s | 18 kB 00:00 2024-05-14T09:19:46.728 INFO:teuthology.orchestra.run.smithi029.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-05-14T09:19:46.779 INFO:teuthology.orchestra.run.smithi029.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-14T09:19:46.807 INFO:teuthology.orchestra.run.smithi026.stdout:(38/156): cephadm-19.0.0-3588.ge2b755a6.el9.noa 4.7 MB/s | 763 kB 00:00 2024-05-14T09:19:46.896 INFO:teuthology.orchestra.run.smithi029.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-05-14T09:19:46.922 INFO:teuthology.orchestra.run.smithi029.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-05-14T09:19:46.947 INFO:teuthology.orchestra.run.smithi029.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 505 kB/s | 26 kB 00:00 2024-05-14T09:19:46.973 INFO:teuthology.orchestra.run.smithi029.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 377 kB/s | 19 kB 00:00 2024-05-14T09:19:47.000 INFO:teuthology.orchestra.run.smithi002.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 96 kB/s | 41 kB 00:00 2024-05-14T09:19:47.007 INFO:teuthology.orchestra.run.smithi029.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 172 kB 00:00 2024-05-14T09:19:47.032 INFO:teuthology.orchestra.run.smithi029.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 588 kB/s | 35 kB 00:00 2024-05-14T09:19:47.158 INFO:teuthology.orchestra.run.smithi029.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-05-14T09:19:47.192 INFO:teuthology.orchestra.run.smithi029.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-05-14T09:19:47.216 INFO:teuthology.orchestra.run.smithi026.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 160 kB/s | 80 kB 00:00 2024-05-14T09:19:47.218 INFO:teuthology.orchestra.run.smithi029.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-05-14T09:19:47.243 INFO:teuthology.orchestra.run.smithi029.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-14T09:19:47.269 INFO:teuthology.orchestra.run.smithi029.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 835 kB/s | 42 kB 00:00 2024-05-14T09:19:47.292 INFO:teuthology.orchestra.run.smithi002.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 850 kB/s | 248 kB 00:00 2024-05-14T09:19:47.302 INFO:teuthology.orchestra.run.smithi029.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-05-14T09:19:47.338 INFO:teuthology.orchestra.run.smithi029.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 159 kB 00:00 2024-05-14T09:19:47.372 INFO:teuthology.orchestra.run.smithi029.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.9 MB/s | 279 kB 00:00 2024-05-14T09:19:47.406 INFO:teuthology.orchestra.run.smithi029.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 757 kB/s | 51 kB 00:00 2024-05-14T09:19:47.440 INFO:teuthology.orchestra.run.smithi029.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 797 kB/s | 54 kB 00:00 2024-05-14T09:19:47.442 INFO:teuthology.orchestra.run.smithi002.stdout:(153/156): NetworkManager-libnm-1.47.90-1.el9.x 1.7 MB/s | 1.8 MB 00:01 2024-05-14T09:19:47.542 INFO:teuthology.orchestra.run.smithi026.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 601 kB/s | 195 kB 00:00 2024-05-14T09:19:47.548 INFO:teuthology.orchestra.run.smithi029.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.4 MB/s | 5.1 MB 00:00 2024-05-14T09:19:47.574 INFO:teuthology.orchestra.run.smithi029.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 311 kB/s | 42 kB 00:00 2024-05-14T09:19:47.600 INFO:teuthology.orchestra.run.smithi029.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 872 kB/s | 44 kB 00:00 2024-05-14T09:19:47.625 INFO:teuthology.orchestra.run.smithi026.stdout:(41/156): ceph-mgr-diskprediction-local-19.0.0- 4.8 MB/s | 7.4 MB 00:01 2024-05-14T09:19:47.625 INFO:teuthology.orchestra.run.smithi029.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-14T09:19:47.651 INFO:teuthology.orchestra.run.smithi029.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 709 kB/s | 36 kB 00:00 2024-05-14T09:19:47.667 INFO:teuthology.orchestra.run.smithi026.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 935 kB/s | 803 kB 00:00 2024-05-14T09:19:47.676 INFO:teuthology.orchestra.run.smithi029.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 936 kB/s | 47 kB 00:00 2024-05-14T09:19:47.701 INFO:teuthology.orchestra.run.smithi002.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.3 MB/s | 2.4 MB 00:00 2024-05-14T09:19:47.718 INFO:teuthology.orchestra.run.smithi029.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 305 kB 00:00 2024-05-14T09:19:47.744 INFO:teuthology.orchestra.run.smithi029.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 857 kB/s | 58 kB 00:00 2024-05-14T09:19:47.759 INFO:teuthology.orchestra.run.smithi026.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-05-14T09:19:47.769 INFO:teuthology.orchestra.run.smithi029.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-14T09:19:47.784 INFO:teuthology.orchestra.run.smithi026.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 911 kB/s | 106 kB 00:00 2024-05-14T09:19:47.796 INFO:teuthology.orchestra.run.smithi029.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-14T09:19:47.801 INFO:teuthology.orchestra.run.smithi002.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 2.7 MB/s | 1.3 MB 00:00 2024-05-14T09:19:47.823 INFO:teuthology.orchestra.run.smithi029.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 854 kB/s | 45 kB 00:00 2024-05-14T09:19:47.868 INFO:teuthology.orchestra.run.smithi026.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-05-14T09:19:47.893 INFO:teuthology.orchestra.run.smithi026.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-14T09:19:47.976 INFO:teuthology.orchestra.run.smithi026.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-14T09:19:48.085 INFO:teuthology.orchestra.run.smithi026.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-05-14T09:19:48.148 INFO:teuthology.orchestra.run.smithi029.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-05-14T09:19:48.194 INFO:teuthology.orchestra.run.smithi026.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 988 kB/s | 107 kB 00:00 2024-05-14T09:19:48.244 INFO:teuthology.orchestra.run.smithi026.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-05-14T09:19:48.274 INFO:teuthology.orchestra.run.smithi029.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 4.3 MB/s | 548 kB 00:00 2024-05-14T09:19:48.294 INFO:teuthology.orchestra.run.smithi026.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-05-14T09:19:48.316 INFO:teuthology.orchestra.run.smithi029.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 7.2 MB/s | 309 kB 00:00 2024-05-14T09:19:48.341 INFO:teuthology.orchestra.run.smithi029.stdout:(105/156): lua-devel-5.4.4-4.el9.x86_64.rpm 41 kB/s | 22 kB 00:00 2024-05-14T09:19:48.353 INFO:teuthology.orchestra.run.smithi026.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-14T09:19:48.367 INFO:teuthology.orchestra.run.smithi029.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 384 kB/s | 19 kB 00:00 2024-05-14T09:19:48.395 INFO:teuthology.orchestra.run.smithi026.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-05-14T09:19:48.400 INFO:teuthology.orchestra.run.smithi029.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 780 kB/s | 26 kB 00:00 2024-05-14T09:19:48.434 INFO:teuthology.orchestra.run.smithi029.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-05-14T09:19:48.477 INFO:teuthology.orchestra.run.smithi029.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-05-14T09:19:48.512 INFO:teuthology.orchestra.run.smithi026.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-05-14T09:19:48.618 INFO:teuthology.orchestra.run.smithi026.stdout:(55/156): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:00 2024-05-14T09:19:48.646 INFO:teuthology.orchestra.run.smithi026.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 83 kB/s | 11 kB 00:00 2024-05-14T09:19:48.719 INFO:teuthology.orchestra.run.smithi029.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 624 kB/s | 151 kB 00:00 2024-05-14T09:19:48.738 INFO:teuthology.orchestra.run.smithi026.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-05-14T09:19:48.763 INFO:teuthology.orchestra.run.smithi026.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-05-14T09:19:48.828 INFO:teuthology.orchestra.run.smithi029.stdout:(111/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-05-14T09:19:48.855 INFO:teuthology.orchestra.run.smithi026.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-05-14T09:19:48.870 INFO:teuthology.orchestra.run.smithi029.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.4 MB/s | 839 kB 00:00 2024-05-14T09:19:48.889 INFO:teuthology.orchestra.run.smithi026.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 582 kB 00:00 2024-05-14T09:19:48.904 INFO:teuthology.orchestra.run.smithi029.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-05-14T09:19:48.930 INFO:teuthology.orchestra.run.smithi029.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 736 kB/s | 43 kB 00:00 2024-05-14T09:19:48.949 INFO:teuthology.orchestra.run.smithi029.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 729 kB/s | 32 kB 00:00 2024-05-14T09:19:48.956 INFO:teuthology.orchestra.run.smithi026.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 253 kB/s | 25 kB 00:00 2024-05-14T09:19:48.975 INFO:teuthology.orchestra.run.smithi029.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 339 kB/s | 15 kB 00:00 2024-05-14T09:19:48.990 INFO:teuthology.orchestra.run.smithi026.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-05-14T09:19:49.016 INFO:teuthology.orchestra.run.smithi029.stdout:(117/156): protobuf-compiler-3.14.0-13.el9.x86_ 723 kB/s | 863 kB 00:01 2024-05-14T09:19:49.050 INFO:teuthology.orchestra.run.smithi029.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-05-14T09:19:49.065 INFO:teuthology.orchestra.run.smithi026.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-14T09:19:49.084 INFO:teuthology.orchestra.run.smithi029.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-05-14T09:19:49.110 INFO:teuthology.orchestra.run.smithi029.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 597 kB/s | 35 kB 00:00 2024-05-14T09:19:49.115 INFO:teuthology.orchestra.run.smithi026.stdout:(64/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.9 MB/s | 3.0 MB 00:00 2024-05-14T09:19:49.135 INFO:teuthology.orchestra.run.smithi029.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-05-14T09:19:49.140 INFO:teuthology.orchestra.run.smithi026.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-05-14T09:19:49.174 INFO:teuthology.orchestra.run.smithi026.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 323 kB/s | 35 kB 00:00 2024-05-14T09:19:49.194 INFO:teuthology.orchestra.run.smithi029.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-05-14T09:19:49.220 INFO:teuthology.orchestra.run.smithi029.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-05-14T09:19:49.245 INFO:teuthology.orchestra.run.smithi029.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 210 kB/s | 11 kB 00:00 2024-05-14T09:19:49.271 INFO:teuthology.orchestra.run.smithi029.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 348 kB/s | 18 kB 00:00 2024-05-14T09:19:49.296 INFO:teuthology.orchestra.run.smithi029.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 456 kB/s | 23 kB 00:00 2024-05-14T09:19:49.322 INFO:teuthology.orchestra.run.smithi029.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 383 kB/s | 19 kB 00:00 2024-05-14T09:19:49.348 INFO:teuthology.orchestra.run.smithi029.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 388 kB/s | 20 kB 00:00 2024-05-14T09:19:49.406 INFO:teuthology.orchestra.run.smithi029.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.9 MB/s | 2.0 MB 00:00 2024-05-14T09:19:49.434 INFO:teuthology.orchestra.run.smithi029.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 540 kB/s | 46 kB 00:00 2024-05-14T09:19:49.459 INFO:teuthology.orchestra.run.smithi029.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-14T09:19:49.466 INFO:teuthology.orchestra.run.smithi026.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-05-14T09:19:49.485 INFO:teuthology.orchestra.run.smithi029.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-14T09:19:49.510 INFO:teuthology.orchestra.run.smithi029.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.2 MB/s | 272 kB 00:00 2024-05-14T09:19:49.544 INFO:teuthology.orchestra.run.smithi029.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 278 kB/s | 16 kB 00:00 2024-05-14T09:19:49.578 INFO:teuthology.orchestra.run.smithi029.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-14T09:19:49.591 INFO:teuthology.orchestra.run.smithi026.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-05-14T09:19:49.612 INFO:teuthology.orchestra.run.smithi029.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 419 kB/s | 28 kB 00:00 2024-05-14T09:19:49.637 INFO:teuthology.orchestra.run.smithi029.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 520 kB/s | 31 kB 00:00 2024-05-14T09:19:49.680 INFO:teuthology.orchestra.run.smithi029.stdout:(138/156): python3-kubernetes-26.1.0-2.el9.noar 2.9 MB/s | 1.0 MB 00:00 2024-05-14T09:19:49.705 INFO:teuthology.orchestra.run.smithi029.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 875 kB/s | 59 kB 00:00 2024-05-14T09:19:49.716 INFO:teuthology.orchestra.run.smithi026.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-05-14T09:19:49.731 INFO:teuthology.orchestra.run.smithi029.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 704 kB/s | 36 kB 00:00 2024-05-14T09:19:49.757 INFO:teuthology.orchestra.run.smithi029.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-14T09:19:49.782 INFO:teuthology.orchestra.run.smithi029.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-05-14T09:19:49.808 INFO:teuthology.orchestra.run.smithi029.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-14T09:19:49.841 INFO:teuthology.orchestra.run.smithi029.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.0 MB/s | 230 kB 00:00 2024-05-14T09:19:49.869 INFO:teuthology.orchestra.run.smithi029.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 331 kB/s | 20 kB 00:00 2024-05-14T09:19:49.894 INFO:teuthology.orchestra.run.smithi029.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 3.6 MB/s | 192 kB 00:00 2024-05-14T09:19:49.928 INFO:teuthology.orchestra.run.smithi029.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-05-14T09:19:49.979 INFO:teuthology.orchestra.run.smithi029.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-14T09:19:50.171 INFO:teuthology.orchestra.run.smithi029.stdout:(149/156): NetworkManager-1.47.90-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-05-14T09:19:50.246 INFO:teuthology.orchestra.run.smithi029.stdout:(150/156): NetworkManager-libnm-1.47.90-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-05-14T09:19:50.289 INFO:teuthology.orchestra.run.smithi029.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 993 kB/s | 41 kB 00:00 2024-05-14T09:19:50.339 INFO:teuthology.orchestra.run.smithi029.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 4.8 MB/s | 248 kB 00:00 2024-05-14T09:19:50.350 INFO:teuthology.orchestra.run.smithi026.stdout:(70/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-05-14T09:19:50.406 INFO:teuthology.orchestra.run.smithi029.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-05-14T09:19:50.490 INFO:teuthology.orchestra.run.smithi029.stdout:(154/156): librbd1-19.0.0-3588.ge2b755a6.el9.x8 5.5 MB/s | 3.1 MB 00:00 2024-05-14T09:19:50.634 INFO:teuthology.orchestra.run.smithi029.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-14T09:19:50.768 INFO:teuthology.orchestra.run.smithi002.stdout:(156/156): NetworkManager-1.47.90-1.el9.x86_64. 502 kB/s | 2.3 MB 00:04 2024-05-14T09:19:50.771 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-05-14T09:19:50.771 INFO:teuthology.orchestra.run.smithi002.stdout:Total 11 MB/s | 219 MB 00:19 2024-05-14T09:19:50.999 INFO:teuthology.orchestra.run.smithi029.stdout:(156/156): librados2-19.0.0-3588.ge2b755a6.el9. 3.0 MB/s | 3.3 MB 00:01 2024-05-14T09:19:51.002 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2024-05-14T09:19:51.003 INFO:teuthology.orchestra.run.smithi029.stdout:Total 11 MB/s | 219 MB 00:19 2024-05-14T09:19:51.758 INFO:teuthology.orchestra.run.smithi026.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 734 kB/s | 1.0 MB 00:01 2024-05-14T09:19:51.974 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-05-14T09:19:51.978 INFO:teuthology.orchestra.run.smithi026.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 404 kB/s | 87 kB 00:00 2024-05-14T09:19:52.140 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-05-14T09:19:52.140 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-05-14T09:19:52.219 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2024-05-14T09:19:52.388 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2024-05-14T09:19:52.389 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2024-05-14T09:19:54.257 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2024-05-14T09:19:54.258 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2024-05-14T09:19:54.410 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-05-14T09:19:54.410 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-05-14T09:19:55.550 INFO:teuthology.orchestra.run.smithi026.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 1.0 MB/s | 5.8 MB 00:05 2024-05-14T09:19:55.934 INFO:teuthology.orchestra.run.smithi026.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 637 kB/s | 244 kB 00:00 2024-05-14T09:19:56.292 INFO:teuthology.orchestra.run.smithi026.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 692 kB/s | 248 kB 00:00 2024-05-14T09:19:56.351 INFO:teuthology.orchestra.run.smithi026.stdout:(76/156): openblas-openmp-0.3.21-2.el9.x86_64.r 671 kB/s | 4.7 MB 00:07 2024-05-14T09:19:56.393 INFO:teuthology.orchestra.run.smithi026.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 255 kB/s | 26 kB 00:00 2024-05-14T09:19:56.443 INFO:teuthology.orchestra.run.smithi026.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 208 kB/s | 19 kB 00:00 2024-05-14T09:19:56.502 INFO:teuthology.orchestra.run.smithi026.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-05-14T09:19:56.535 INFO:teuthology.orchestra.run.smithi026.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 378 kB/s | 35 kB 00:00 2024-05-14T09:19:56.886 INFO:teuthology.orchestra.run.smithi026.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 716 kB/s | 250 kB 00:00 2024-05-14T09:19:56.978 INFO:teuthology.orchestra.run.smithi026.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-05-14T09:19:57.078 INFO:teuthology.orchestra.run.smithi026.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 772 kB/s | 77 kB 00:00 2024-05-14T09:19:57.178 INFO:teuthology.orchestra.run.smithi026.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 423 kB/s | 42 kB 00:00 2024-05-14T09:19:57.245 INFO:teuthology.orchestra.run.smithi026.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 618 kB/s | 222 kB 00:00 2024-05-14T09:19:57.287 INFO:teuthology.orchestra.run.smithi026.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-05-14T09:19:57.396 INFO:teuthology.orchestra.run.smithi026.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-05-14T09:19:57.504 INFO:teuthology.orchestra.run.smithi026.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-05-14T09:19:57.529 INFO:teuthology.orchestra.run.smithi026.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 559 kB/s | 159 kB 00:00 2024-05-14T09:19:57.605 INFO:teuthology.orchestra.run.smithi026.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 535 kB/s | 54 kB 00:00 2024-05-14T09:19:57.706 INFO:teuthology.orchestra.run.smithi026.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-05-14T09:19:57.814 INFO:teuthology.orchestra.run.smithi026.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 408 kB/s | 44 kB 00:00 2024-05-14T09:19:57.922 INFO:teuthology.orchestra.run.smithi026.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-05-14T09:19:58.031 INFO:teuthology.orchestra.run.smithi026.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 331 kB/s | 36 kB 00:00 2024-05-14T09:19:58.148 INFO:teuthology.orchestra.run.smithi026.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 407 kB/s | 47 kB 00:00 2024-05-14T09:19:58.265 INFO:teuthology.orchestra.run.smithi026.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.5 MB/s | 305 kB 00:00 2024-05-14T09:19:58.373 INFO:teuthology.orchestra.run.smithi026.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 534 kB/s | 58 kB 00:00 2024-05-14T09:19:58.482 INFO:teuthology.orchestra.run.smithi026.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 865 kB/s | 94 kB 00:00 2024-05-14T09:19:58.599 INFO:teuthology.orchestra.run.smithi026.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 545 kB/s | 64 kB 00:00 2024-05-14T09:19:58.708 INFO:teuthology.orchestra.run.smithi026.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 414 kB/s | 45 kB 00:00 2024-05-14T09:19:58.842 INFO:teuthology.orchestra.run.smithi026.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-05-14T09:19:58.975 INFO:teuthology.orchestra.run.smithi026.stdout:(102/156): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-05-14T09:19:59.184 INFO:teuthology.orchestra.run.smithi026.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.6 MB/s | 548 kB 00:00 2024-05-14T09:19:59.242 INFO:teuthology.orchestra.run.smithi026.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-05-14T09:19:59.284 INFO:teuthology.orchestra.run.smithi026.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 467 kB/s | 19 kB 00:00 2024-05-14T09:19:59.460 INFO:teuthology.orchestra.run.smithi026.stdout:(106/156): libarrow-9.0.0-11.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-05-14T09:19:59.501 INFO:teuthology.orchestra.run.smithi026.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 628 kB/s | 26 kB 00:00 2024-05-14T09:19:59.546 INFO:teuthology.orchestra.run.smithi026.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-05-14T09:19:59.594 INFO:teuthology.orchestra.run.smithi026.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-05-14T09:19:59.644 INFO:teuthology.orchestra.run.smithi026.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-05-14T09:19:59.703 INFO:teuthology.orchestra.run.smithi026.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-05-14T09:19:59.753 INFO:teuthology.orchestra.run.smithi026.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-05-14T09:19:59.795 INFO:teuthology.orchestra.run.smithi026.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-05-14T09:19:59.837 INFO:teuthology.orchestra.run.smithi026.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 774 kB/s | 32 kB 00:00 2024-05-14T09:19:59.879 INFO:teuthology.orchestra.run.smithi026.stdout:(115/156): python3-certifi-2021.10.8-2.el9.noar 360 kB/s | 15 kB 00:00 2024-05-14T09:19:59.930 INFO:teuthology.orchestra.run.smithi026.stdout:(116/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-05-14T09:19:59.979 INFO:teuthology.orchestra.run.smithi026.stdout:(117/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-05-14T09:20:00.030 INFO:teuthology.orchestra.run.smithi026.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-05-14T09:20:00.088 INFO:teuthology.orchestra.run.smithi026.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 606 kB/s | 35 kB 00:00 2024-05-14T09:20:00.139 INFO:teuthology.orchestra.run.smithi026.stdout:(120/156): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-05-14T09:20:00.214 INFO:teuthology.orchestra.run.smithi026.stdout:(121/156): python3-grpcio-1.46.7-10.el9.x86_64. 27 MB/s | 2.0 MB 00:00 2024-05-14T09:20:00.264 INFO:teuthology.orchestra.run.smithi026.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-05-14T09:20:00.315 INFO:teuthology.orchestra.run.smithi026.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-05-14T09:20:00.365 INFO:teuthology.orchestra.run.smithi026.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-05-14T09:20:00.381 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2024-05-14T09:20:00.415 INFO:teuthology.orchestra.run.smithi026.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 464 kB/s | 23 kB 00:00 2024-05-14T09:20:00.465 INFO:teuthology.orchestra.run.smithi026.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-05-14T09:20:00.509 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-14T09:20:00.516 INFO:teuthology.orchestra.run.smithi026.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-05-14T09:20:00.548 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-14T09:20:00.574 INFO:teuthology.orchestra.run.smithi026.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-05-14T09:20:00.611 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-14T09:20:00.624 INFO:teuthology.orchestra.run.smithi026.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 929 kB/s | 46 kB 00:00 2024-05-14T09:20:00.654 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-14T09:20:00.675 INFO:teuthology.orchestra.run.smithi026.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-14T09:20:00.681 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-14T09:20:00.725 INFO:teuthology.orchestra.run.smithi026.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-14T09:20:00.733 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-14T09:20:00.810 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-05-14T09:20:00.825 INFO:teuthology.orchestra.run.smithi026.stdout:(132/156): python3-babel-2.9.1-2.el9.noarch.rpm 690 kB/s | 6.0 MB 00:08 2024-05-14T09:20:00.851 INFO:teuthology.orchestra.run.smithi026.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-05-14T09:20:00.876 INFO:teuthology.orchestra.run.smithi026.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-05-14T09:20:00.897 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-14T09:20:00.926 INFO:teuthology.orchestra.run.smithi026.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 568 kB/s | 28 kB 00:00 2024-05-14T09:20:00.936 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-14T09:20:00.977 INFO:teuthology.orchestra.run.smithi026.stdout:(136/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-05-14T09:20:01.001 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-14T09:20:01.002 INFO:teuthology.orchestra.run.smithi026.stdout:(137/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 597 kB/s | 90 kB 00:00 2024-05-14T09:20:01.028 INFO:teuthology.orchestra.run.smithi026.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-05-14T09:20:01.040 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-14T09:20:01.061 INFO:teuthology.orchestra.run.smithi026.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-14T09:20:01.065 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-14T09:20:01.086 INFO:teuthology.orchestra.run.smithi026.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-05-14T09:20:01.091 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-14T09:20:01.119 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-14T09:20:01.120 INFO:teuthology.orchestra.run.smithi026.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 869 kB/s | 51 kB 00:00 2024-05-14T09:20:01.145 INFO:teuthology.orchestra.run.smithi026.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-05-14T09:20:01.182 INFO:teuthology.orchestra.run.smithi026.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-05-14T09:20:01.204 INFO:teuthology.orchestra.run.smithi026.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-05-14T09:20:01.221 INFO:teuthology.orchestra.run.smithi026.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-05-14T09:20:01.255 INFO:teuthology.orchestra.run.smithi026.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-05-14T09:20:01.318 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T09:20:01.372 INFO:teuthology.orchestra.run.smithi026.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-14T09:20:01.451 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T09:20:01.470 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-14T09:20:01.483 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T09:20:01.572 INFO:teuthology.orchestra.run.smithi026.stdout:(148/156): librados2-19.0.0-3588.ge2b755a6.el9. 10 MB/s | 3.3 MB 00:00 2024-05-14T09:20:01.577 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T09:20:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T09:20:01.607 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 10/164 2024-05-14T09:20:01.719 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T09:20:01.750 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T09:20:01.764 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-14T09:20:01.843 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T09:20:01.843 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-14T09:20:01.871 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T09:20:01.878 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 10/164 2024-05-14T09:20:01.964 INFO:teuthology.orchestra.run.smithi026.stdout:(149/156): librbd1-19.0.0-3588.ge2b755a6.el9.x8 5.2 MB/s | 3.1 MB 00:00 2024-05-14T09:20:01.978 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T09:20:02.006 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 14/164 2024-05-14T09:20:02.054 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 14/164 2024-05-14T09:20:02.060 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-14T09:20:02.084 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T09:20:02.110 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T09:20:02.120 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-14T09:20:02.146 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T09:20:02.262 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T09:20:02.270 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T09:20:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 14/164 2024-05-14T09:20:02.352 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 14/164 2024-05-14T09:20:02.373 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T09:20:02.399 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T09:20:02.425 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T09:20:02.489 INFO:teuthology.orchestra.run.smithi026.stdout:(150/156): NetworkManager-libnm-1.47.90-1.el9.x 3.4 MB/s | 1.8 MB 00:00 2024-05-14T09:20:02.569 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T09:20:02.613 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T09:20:02.631 INFO:teuthology.orchestra.run.smithi026.stdout:(151/156): NetworkManager-1.47.90-1.el9.x86_64. 2.2 MB/s | 2.3 MB 00:01 2024-05-14T09:20:02.656 INFO:teuthology.orchestra.run.smithi026.stdout:(152/156): NetworkManager-team-1.47.90-1.el9.x8 249 kB/s | 41 kB 00:00 2024-05-14T09:20:02.674 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T09:20:02.740 INFO:teuthology.orchestra.run.smithi026.stdout:(153/156): NetworkManager-tui-1.47.90-1.el9.x86 2.2 MB/s | 248 kB 00:00 2024-05-14T09:20:02.743 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T09:20:02.773 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T09:20:02.790 INFO:teuthology.orchestra.run.smithi026.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 10 MB/s | 1.3 MB 00:00 2024-05-14T09:20:02.862 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T09:20:02.912 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T09:20:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T09:20:03.027 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T09:20:03.040 INFO:teuthology.orchestra.run.smithi026.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.0 MB/s | 2.4 MB 00:00 2024-05-14T09:20:03.090 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T09:20:03.129 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T09:20:03.238 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-14T09:20:03.276 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-14T09:20:03.308 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T09:20:03.320 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-14T09:20:03.397 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-14T09:20:03.416 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-14T09:20:03.448 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-14T09:20:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-14T09:20:03.496 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-14T09:20:03.501 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-14T09:20:03.543 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-14T09:20:03.572 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-14T09:20:03.579 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-14T09:20:03.612 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-14T09:20:03.656 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-14T09:20:03.660 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-14T09:20:03.689 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-14T09:20:03.716 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-14T09:20:03.721 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-14T09:20:03.758 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-14T09:20:03.787 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-14T09:20:03.804 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-14T09:20:03.847 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-14T09:20:03.877 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-14T09:20:03.883 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-14T09:20:03.913 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-14T09:20:03.916 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9. 32/164 2024-05-14T09:20:03.968 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el 33/164 2024-05-14T09:20:03.991 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-14T09:20:04.005 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_6 34/164 2024-05-14T09:20:04.023 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9. 32/164 2024-05-14T09:20:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el 33/164 2024-05-14T09:20:04.086 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-14T09:20:04.103 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_6 34/164 2024-05-14T09:20:04.138 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-14T09:20:04.177 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-14T09:20:04.227 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-14T09:20:04.227 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-14T09:20:04.303 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-14T09:20:04.375 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-14T09:20:04.429 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-14T09:20:04.448 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-14T09:20:04.505 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-14T09:20:05.000 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-14T09:20:05.059 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-14T09:20:05.097 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-14T09:20:05.133 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-14T09:20:05.143 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-14T09:20:05.178 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-14T09:20:05.184 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-14T09:20:05.222 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-14T09:20:05.223 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_6 44/164 2024-05-14T09:20:05.259 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_6 44/164 2024-05-14T09:20:05.273 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-14T09:20:05.306 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-14T09:20:05.450 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-14T09:20:05.487 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-14T09:20:05.522 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-14T09:20:05.545 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-14T09:20:05.626 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-14T09:20:05.640 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-14T09:20:05.691 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-14T09:20:05.703 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-14T09:20:05.772 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-14T09:20:05.777 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-14T09:20:05.811 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-14T09:20:05.813 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-14T09:20:05.848 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-14T09:20:05.850 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-14T09:20:05.889 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-14T09:20:05.898 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-14T09:20:05.924 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-14T09:20:05.949 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-14T09:20:05.967 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-14T09:20:06.001 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-14T09:20:06.010 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-14T09:20:06.052 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-14T09:20:06.151 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-14T09:20:06.199 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-14T09:20:06.253 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-14T09:20:06.257 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-14T09:20:06.287 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-14T09:20:06.291 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-14T09:20:06.329 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-14T09:20:06.342 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-14T09:20:06.427 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-14T09:20:06.441 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-14T09:20:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-14T09:20:06.511 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-14T09:20:06.544 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-14T09:20:06.568 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-14T09:20:06.627 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-14T09:20:06.656 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-14T09:20:06.669 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-14T09:20:06.711 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-14T09:20:06.745 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-14T09:20:06.783 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-14T09:20:06.817 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-14T09:20:06.825 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-14T09:20:06.846 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-14T09:20:06.880 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-14T09:20:06.922 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-14T09:20:06.959 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-14T09:20:06.966 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-14T09:20:06.988 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-14T09:20:07.009 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-14T09:20:07.023 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-14T09:20:07.045 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-14T09:20:07.056 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-14T09:20:07.088 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-14T09:20:07.098 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-14T09:20:07.132 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-14T09:20:07.137 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-14T09:20:07.179 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-14T09:20:07.217 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-14T09:20:07.222 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-14T09:20:07.252 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-14T09:20:07.264 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-14T09:20:07.286 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-14T09:20:07.294 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-14T09:20:07.330 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-14T09:20:07.342 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-14T09:20:07.363 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-14T09:20:07.447 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-14T09:20:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-14T09:20:07.498 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-14T09:20:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-14T09:20:07.538 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-14T09:20:07.594 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-14T09:20:07.705 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-14T09:20:07.767 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-14T09:20:08.271 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-14T09:20:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-14T09:20:08.351 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-14T09:20:08.453 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-14T09:20:08.510 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-14T09:20:08.554 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-14T09:20:08.599 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-14T09:20:08.609 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-14T09:20:08.636 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-14T09:20:08.652 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-14T09:20:08.766 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-14T09:20:08.821 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-14T09:20:08.823 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-14T09:20:08.842 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-14T09:20:08.914 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-14T09:20:08.915 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-14T09:20:08.959 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-14T09:20:09.143 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-14T09:20:09.174 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-14T09:20:09.264 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-14T09:20:09.611 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-14T09:20:09.727 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-14T09:20:09.967 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-14T09:20:10.103 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-14T09:20:11.047 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-14T09:20:11.123 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-14T09:20:11.151 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-14T09:20:11.184 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-14T09:20:11.418 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-14T09:20:11.520 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-14T09:20:11.521 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-14T09:20:11.557 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-14T09:20:11.563 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 101/164 2024-05-14T09:20:11.590 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-14T09:20:11.655 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 101/164 2024-05-14T09:20:11.685 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 102/164 2024-05-14T09:20:11.722 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-14T09:20:11.938 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-14T09:20:11.996 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 101/164 2024-05-14T09:20:12.099 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 101/164 2024-05-14T09:20:12.138 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 102/164 2024-05-14T09:20:12.183 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-14T09:20:12.238 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-14T09:20:12.266 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 105/164 2024-05-14T09:20:12.338 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 105/164 2024-05-14T09:20:12.368 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 106/164 2024-05-14T09:20:12.698 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-14T09:20:12.737 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 105/164 2024-05-14T09:20:12.818 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 105/164 2024-05-14T09:20:12.848 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 106/164 2024-05-14T09:20:14.224 INFO:teuthology.orchestra.run.smithi026.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 997 kB/s | 16 MB 00:16 2024-05-14T09:20:14.227 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-05-14T09:20:14.227 INFO:teuthology.orchestra.run.smithi026.stdout:Total 6.1 MB/s | 219 MB 00:35 2024-05-14T09:20:14.681 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T09:20:14.714 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T09:20:15.195 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T09:20:15.244 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T09:20:15.304 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T09:20:15.343 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 108/164 2024-05-14T09:20:15.386 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-05-14T09:20:15.426 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 108/164 2024-05-14T09:20:15.426 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-14T09:20:15.426 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-14T09:20:15.426 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:20:15.460 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 109/164 2024-05-14T09:20:15.554 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-05-14T09:20:15.554 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-05-14T09:20:15.832 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T09:20:15.884 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 108/164 2024-05-14T09:20:15.983 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 108/164 2024-05-14T09:20:15.983 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-14T09:20:15.983 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-14T09:20:15.983 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:20:16.013 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 109/164 2024-05-14T09:20:17.421 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-05-14T09:20:17.421 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-05-14T09:20:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-05-14T09:20:23.777 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-14T09:20:23.816 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-14T09:20:23.870 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-14T09:20:23.903 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-14T09:20:23.929 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-14T09:20:23.972 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-14T09:20:24.388 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-14T09:20:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T09:20:24.582 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T09:20:24.613 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T09:20:24.706 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T09:20:24.735 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 10/164 2024-05-14T09:20:24.890 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-14T09:20:24.950 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-14T09:20:24.977 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T09:20:25.066 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T09:20:25.094 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 14/164 2024-05-14T09:20:25.157 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 14/164 2024-05-14T09:20:25.179 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T09:20:25.199 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T09:20:25.396 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T09:20:25.519 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T09:20:25.685 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T09:20:25.745 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T09:20:25.843 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T09:20:25.912 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T09:20:26.114 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T09:20:26.220 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-14T09:20:26.250 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-14T09:20:26.293 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-14T09:20:26.368 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-14T09:20:26.418 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-14T09:20:26.462 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-14T09:20:26.494 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-14T09:20:26.522 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-14T09:20:26.593 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-14T09:20:26.653 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-14T09:20:26.683 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-14T09:20:26.718 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-14T09:20:26.797 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-14T09:20:26.829 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9. 32/164 2024-05-14T09:20:26.873 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el 33/164 2024-05-14T09:20:26.901 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_6 34/164 2024-05-14T09:20:26.977 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-14T09:20:27.050 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-14T09:20:27.126 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-14T09:20:27.263 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-14T09:20:27.319 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-14T09:20:27.361 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 109/164 2024-05-14T09:20:27.361 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /sys 2024-05-14T09:20:27.362 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /proc 2024-05-14T09:20:27.362 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /mnt 2024-05-14T09:20:27.362 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /var/tmp 2024-05-14T09:20:27.362 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /home 2024-05-14T09:20:27.362 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /root 2024-05-14T09:20:27.362 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /tmp 2024-05-14T09:20:27.362 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:20:27.541 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 110/164 2024-05-14T09:20:27.558 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 110/164 2024-05-14T09:20:27.558 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:27.558 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-14T09:20:27.558 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:20:27.872 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-14T09:20:27.957 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-14T09:20:27.993 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-14T09:20:28.036 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-14T09:20:28.074 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_6 44/164 2024-05-14T09:20:28.116 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-14T09:20:28.268 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-14T09:20:28.310 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-14T09:20:28.404 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-14T09:20:28.467 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-14T09:20:28.540 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-14T09:20:28.578 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-14T09:20:28.615 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-14T09:20:28.662 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-14T09:20:28.697 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-14T09:20:28.741 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-14T09:20:28.774 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-14T09:20:28.916 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-14T09:20:28.963 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-14T09:20:28.994 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-14T09:20:29.035 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-14T09:20:29.132 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-14T09:20:29.168 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-14T09:20:29.208 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-14T09:20:29.210 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 111/164 2024-05-14T09:20:29.238 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 111/164 2024-05-14T09:20:29.238 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:29.238 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-14T09:20:29.238 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T09:20:29.238 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T09:20:29.238 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:20:29.241 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-14T09:20:29.266 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-14T09:20:29.344 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-14T09:20:29.477 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 112/164 2024-05-14T09:20:29.480 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-14T09:20:29.502 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 112/164 2024-05-14T09:20:29.502 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:29.502 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-14T09:20:29.502 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T09:20:29.502 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T09:20:29.502 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:20:29.532 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-14T09:20:29.560 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-14T09:20:29.594 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-14T09:20:29.634 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-14T09:20:29.674 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-14T09:20:29.703 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-14T09:20:29.737 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-14T09:20:29.771 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-14T09:20:29.813 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-14T09:20:29.847 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-14T09:20:29.937 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-14T09:20:29.988 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-14T09:20:30.003 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 113/164 2024-05-14T09:20:30.027 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-14T09:20:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 113/164 2024-05-14T09:20:30.028 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:30.028 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-14T09:20:30.028 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T09:20:30.028 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T09:20:30.028 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:20:30.075 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-14T09:20:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-14T09:20:30.183 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-14T09:20:30.184 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-14T09:20:30.238 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-14T09:20:30.246 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-14T09:20:30.442 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-14T09:20:30.496 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-14T09:20:30.577 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-14T09:20:30.675 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-14T09:20:30.816 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-14T09:20:30.862 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-14T09:20:30.894 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-14T09:20:30.921 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-14T09:20:30.994 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-14T09:20:31.048 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-14T09:20:31.084 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-14T09:20:31.186 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-14T09:20:31.235 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-14T09:20:31.324 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-14T09:20:31.343 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 125/164 2024-05-14T09:20:31.361 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-14T09:20:31.370 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 125/164 2024-05-14T09:20:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.e 126/164 2024-05-14T09:20:31.466 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6. 127/164 2024-05-14T09:20:31.493 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 128/164 2024-05-14T09:20:31.527 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 109/164 2024-05-14T09:20:31.527 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /sys 2024-05-14T09:20:31.527 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /proc 2024-05-14T09:20:31.527 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /mnt 2024-05-14T09:20:31.527 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /var/tmp 2024-05-14T09:20:31.527 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /home 2024-05-14T09:20:31.527 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /root 2024-05-14T09:20:31.527 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /tmp 2024-05-14T09:20:31.528 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:20:31.544 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-14T09:20:31.567 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-14T09:20:31.638 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-14T09:20:31.721 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 110/164 2024-05-14T09:20:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 110/164 2024-05-14T09:20:31.738 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:31.738 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-14T09:20:31.738 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:20:32.313 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-14T09:20:32.435 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-14T09:20:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 128/164 2024-05-14T09:20:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 129/164 2024-05-14T09:20:33.379 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 111/164 2024-05-14T09:20:33.406 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 111/164 2024-05-14T09:20:33.407 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:33.407 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-14T09:20:33.407 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T09:20:33.407 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T09:20:33.407 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:20:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 129/164 2024-05-14T09:20:33.646 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 112/164 2024-05-14T09:20:33.661 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 130/164 2024-05-14T09:20:33.671 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 112/164 2024-05-14T09:20:33.671 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:33.671 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-14T09:20:33.671 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T09:20:33.671 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T09:20:33.671 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:20:33.718 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-14T09:20:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 130/164 2024-05-14T09:20:33.796 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-14T09:20:33.824 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-14T09:20:33.857 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-14T09:20:33.952 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el 131/164 2024-05-14T09:20:33.988 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 132/164 2024-05-14T09:20:34.013 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 132/164 2024-05-14T09:20:34.013 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:34.014 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-14T09:20:34.014 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T09:20:34.014 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T09:20:34.014 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:20:34.065 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 133/164 2024-05-14T09:20:34.077 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 133/164 2024-05-14T09:20:34.164 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 113/164 2024-05-14T09:20:34.178 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 134/164 2024-05-14T09:20:34.189 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 113/164 2024-05-14T09:20:34.189 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:34.189 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-14T09:20:34.189 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T09:20:34.189 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T09:20:34.189 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:20:34.194 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-14T09:20:34.228 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 101/164 2024-05-14T09:20:34.240 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 135/164 2024-05-14T09:20:34.298 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-14T09:20:34.319 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 101/164 2024-05-14T09:20:34.357 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-14T09:20:34.358 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 102/164 2024-05-14T09:20:34.395 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-14T09:20:34.421 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-14T09:20:34.624 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-14T09:20:34.691 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-14T09:20:34.774 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-14T09:20:34.881 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-14T09:20:34.909 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-14T09:20:34.939 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 105/164 2024-05-14T09:20:35.010 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 105/164 2024-05-14T09:20:35.030 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-14T09:20:35.041 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 106/164 2024-05-14T09:20:35.079 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-14T09:20:35.128 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-14T09:20:35.165 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-14T09:20:35.225 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-14T09:20:35.259 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 137/164 2024-05-14T09:20:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 137/164 2024-05-14T09:20:35.283 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:35.283 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-14T09:20:35.283 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T09:20:35.283 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T09:20:35.283 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:20:35.326 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 138/164 2024-05-14T09:20:35.349 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 138/164 2024-05-14T09:20:35.350 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:35.350 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-14T09:20:35.350 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:20:35.401 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 139/164 2024-05-14T09:20:35.425 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 139/164 2024-05-14T09:20:35.426 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:35.426 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-14T09:20:35.426 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T09:20:35.426 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T09:20:35.426 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:20:35.648 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 125/164 2024-05-14T09:20:35.677 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 125/164 2024-05-14T09:20:35.710 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.e 126/164 2024-05-14T09:20:35.760 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 140/164 2024-05-14T09:20:35.774 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6. 127/164 2024-05-14T09:20:35.784 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 140/164 2024-05-14T09:20:35.784 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:35.785 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-14T09:20:35.785 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T09:20:35.785 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T09:20:35.785 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:20:35.809 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 128/164 2024-05-14T09:20:36.906 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 128/164 2024-05-14T09:20:36.956 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 129/164 2024-05-14T09:20:37.390 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T09:20:37.428 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T09:20:37.971 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 129/164 2024-05-14T09:20:37.999 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 130/164 2024-05-14T09:20:38.013 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T09:20:38.049 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 108/164 2024-05-14T09:20:38.116 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 130/164 2024-05-14T09:20:38.141 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 108/164 2024-05-14T09:20:38.141 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-14T09:20:38.141 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-14T09:20:38.141 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-14T09:20:38.175 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 109/164 2024-05-14T09:20:38.290 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el 131/164 2024-05-14T09:20:38.319 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 132/164 2024-05-14T09:20:38.344 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 132/164 2024-05-14T09:20:38.344 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:38.344 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-14T09:20:38.344 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T09:20:38.344 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T09:20:38.345 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:20:38.396 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 133/164 2024-05-14T09:20:38.407 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 133/164 2024-05-14T09:20:38.510 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 134/164 2024-05-14T09:20:38.573 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 135/164 2024-05-14T09:20:39.559 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-14T09:20:39.602 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 137/164 2024-05-14T09:20:39.626 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 137/164 2024-05-14T09:20:39.627 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:39.627 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-14T09:20:39.627 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T09:20:39.627 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T09:20:39.627 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:20:39.679 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 138/164 2024-05-14T09:20:39.702 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 138/164 2024-05-14T09:20:39.702 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:39.702 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-14T09:20:39.702 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:20:39.754 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 139/164 2024-05-14T09:20:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 139/164 2024-05-14T09:20:39.779 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:39.779 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-14T09:20:39.779 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T09:20:39.779 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T09:20:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:20:40.106 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 140/164 2024-05-14T09:20:40.130 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 140/164 2024-05-14T09:20:40.130 INFO:teuthology.orchestra.run.smithi029.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:40.130 INFO:teuthology.orchestra.run.smithi029.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-14T09:20:40.130 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T09:20:40.130 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T09:20:40.131 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:20:40.356 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 141/164 2024-05-14T09:20:40.398 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 142/164 2024-05-14T09:20:40.438 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 143/164 2024-05-14T09:20:40.469 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-14T09:20:40.497 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-14T09:20:40.692 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 146/164 2024-05-14T09:20:40.743 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-14T09:20:40.773 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_ 148/164 2024-05-14T09:20:40.802 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-14T09:20:40.849 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-14T09:20:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T09:20:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T09:20:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T09:20:42.470 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 152/164 2024-05-14T09:20:42.570 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-14T09:20:42.618 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-14T09:20:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-14T09:20:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-14T09:20:42.791 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-14T09:20:42.792 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T09:20:42.875 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T09:20:42.917 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-14T09:20:42.917 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T09:20:43.010 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T09:20:43.034 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-14T09:20:43.041 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T09:20:43.041 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T09:20:43.228 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T09:20:43.229 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T09:20:43.313 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T09:20:43.380 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T09:20:44.660 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 141/164 2024-05-14T09:20:44.702 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 142/164 2024-05-14T09:20:44.752 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 143/164 2024-05-14T09:20:44.784 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-14T09:20:44.819 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-14T09:20:45.015 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 146/164 2024-05-14T09:20:45.072 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-14T09:20:45.102 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_ 148/164 2024-05-14T09:20:45.141 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-14T09:20:45.205 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-14T09:20:45.871 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T09:20:45.928 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T09:20:45.949 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T09:20:46.882 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 152/164 2024-05-14T09:20:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-14T09:20:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-14T09:20:47.134 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-14T09:20:47.181 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-14T09:20:47.207 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-14T09:20:47.207 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T09:20:47.309 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T09:20:47.342 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-14T09:20:47.343 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T09:20:47.444 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T09:20:47.478 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-14T09:20:47.484 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T09:20:47.484 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T09:20:47.674 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T09:20:47.674 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T09:20:47.776 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T09:20:47.851 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T09:20:48.996 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T09:20:48.996 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 1/164 2024-05-14T09:20:48.996 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 2/164 2024-05-14T09:20:48.996 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 3/164 2024-05-14T09:20:48.996 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 4/164 2024-05-14T09:20:48.996 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 5/164 2024-05-14T09:20:48.997 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 6/164 2024-05-14T09:20:48.997 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 7/164 2024-05-14T09:20:48.997 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T09:20:48.997 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T09:20:48.997 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 10/164 2024-05-14T09:20:48.997 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 11/164 2024-05-14T09:20:48.997 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 12/164 2024-05-14T09:20:48.997 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T09:20:48.997 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 14/164 2024-05-14T09:20:48.997 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_ 15/164 2024-05-14T09:20:48.997 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 16/164 2024-05-14T09:20:48.997 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 17/164 2024-05-14T09:20:48.997 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_6 18/164 2024-05-14T09:20:48.998 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 19/164 2024-05-14T09:20:48.998 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 20/164 2024-05-14T09:20:48.998 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el 21/164 2024-05-14T09:20:48.998 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9. 22/164 2024-05-14T09:20:48.998 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_6 23/164 2024-05-14T09:20:48.998 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 24/164 2024-05-14T09:20:48.998 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 25/164 2024-05-14T09:20:48.998 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 26/164 2024-05-14T09:20:48.998 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 27/164 2024-05-14T09:20:49.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 28/164 2024-05-14T09:20:49.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 29/164 2024-05-14T09:20:49.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6. 30/164 2024-05-14T09:20:49.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 31/164 2024-05-14T09:20:49.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 32/164 2024-05-14T09:20:49.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 33/164 2024-05-14T09:20:49.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el 34/164 2024-05-14T09:20:49.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 35/164 2024-05-14T09:20:49.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.e 36/164 2024-05-14T09:20:49.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 37/164 2024-05-14T09:20:49.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 38/164 2024-05-14T09:20:49.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 39/164 2024-05-14T09:20:49.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-14T09:20:49.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-14T09:20:49.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-14T09:20:49.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-14T09:20:49.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-14T09:20:49.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-14T09:20:49.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-14T09:20:49.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-14T09:20:49.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-14T09:20:49.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-14T09:20:49.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-14T09:20:49.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-14T09:20:49.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-14T09:20:49.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-14T09:20:49.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-14T09:20:49.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-14T09:20:49.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-14T09:20:49.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-14T09:20:49.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-14T09:20:49.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-14T09:20:49.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-14T09:20:49.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-14T09:20:49.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-14T09:20:49.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-14T09:20:49.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-14T09:20:49.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-14T09:20:49.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-14T09:20:49.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-14T09:20:49.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-14T09:20:49.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-14T09:20:49.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-14T09:20:49.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-14T09:20:49.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-14T09:20:49.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-14T09:20:49.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-14T09:20:49.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-14T09:20:49.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-14T09:20:49.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-14T09:20:49.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-14T09:20:49.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-14T09:20:49.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-14T09:20:49.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-14T09:20:49.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-14T09:20:49.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-14T09:20:49.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-14T09:20:49.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-14T09:20:49.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-14T09:20:49.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-14T09:20:49.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-14T09:20:49.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-14T09:20:49.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-14T09:20:49.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-14T09:20:49.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-14T09:20:49.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-14T09:20:49.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-14T09:20:49.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-14T09:20:49.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 149/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 151/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-14T09:20:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-14T09:20:49.020 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-14T09:20:49.020 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-14T09:20:49.020 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-14T09:20:49.020 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-14T09:20:49.020 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-14T09:20:49.020 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-14T09:20:50.247 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 109/164 2024-05-14T09:20:50.247 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2024-05-14T09:20:50.247 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2024-05-14T09:20:50.247 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2024-05-14T09:20:50.247 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2024-05-14T09:20:50.247 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2024-05-14T09:20:50.247 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /root 2024-05-14T09:20:50.247 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2024-05-14T09:20:50.247 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-14T09:20:50.422 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 110/164 2024-05-14T09:20:50.438 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 110/164 2024-05-14T09:20:50.438 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:50.438 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-14T09:20:50.438 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-14T09:20:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T09:20:51.930 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:20:51.930 INFO:teuthology.orchestra.run.smithi002.stdout:Upgraded: 2024-05-14T09:20:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-14T09:20:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-14T09:20:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-14T09:20:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-14T09:20:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-14T09:20:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-14T09:20:51.931 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-05-14T09:20:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-14T09:20:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: automake-1.16.2-8.el9.noarch 2024-05-14T09:20:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: bison-3.7.4-5.el9.x86_64 2024-05-14T09:20:51.931 INFO:teuthology.orchestra.run.smithi002.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-14T09:20:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:20:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:20:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:20:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:20:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:20:51.932 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:20:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:20:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:20:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:20:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:20:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-14T09:20:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: flex-2.6.4-9.el9.x86_64 2024-05-14T09:20:51.933 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-14T09:20:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-14T09:20:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-14T09:20:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-14T09:20:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-14T09:20:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-14T09:20:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-14T09:20:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-14T09:20:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-14T09:20:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-14T09:20:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-14T09:20:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-14T09:20:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-14T09:20:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-14T09:20:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-14T09:20:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-14T09:20:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-14T09:20:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-14T09:20:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-14T09:20:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-14T09:20:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: lua-5.4.4-4.el9.x86_64 2024-05-14T09:20:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-14T09:20:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-14T09:20:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-14T09:20:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-14T09:20:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-14T09:20:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-14T09:20:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-13.14-1.el9.x86_64 2024-05-14T09:20:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-14T09:20:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-14T09:20:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-14T09:20:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-14T09:20:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-14T09:20:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-14T09:20:51.936 INFO:teuthology.orchestra.run.smithi002.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-14T09:20:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-14T09:20:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-14T09:20:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-14T09:20:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-14T09:20:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-14T09:20:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-14T09:20:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-14T09:20:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-14T09:20:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-14T09:20:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-14T09:20:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-14T09:20:51.937 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-14T09:20:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-14T09:20:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-14T09:20:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-14T09:20:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-14T09:20:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-14T09:20:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-14T09:20:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-14T09:20:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-14T09:20:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-14T09:20:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-14T09:20:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-14T09:20:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-14T09:20:51.938 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-14T09:20:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-14T09:20:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-14T09:20:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-14T09:20:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-14T09:20:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-14T09:20:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-14T09:20:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-14T09:20:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-14T09:20:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ply-3.11-14.el9.noarch 2024-05-14T09:20:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-14T09:20:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-14T09:20:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-14T09:20:51.939 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-14T09:20:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-14T09:20:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-14T09:20:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-14T09:20:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-14T09:20:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-14T09:20:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-14T09:20:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-14T09:20:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-14T09:20:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-14T09:20:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-14T09:20:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-14T09:20:51.940 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-14T09:20:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-14T09:20:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-14T09:20:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-14T09:20:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-14T09:20:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-14T09:20:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-14T09:20:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-14T09:20:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-14T09:20:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-14T09:20:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-14T09:20:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-14T09:20:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.941 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:20:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: re2-1:20211101-3.el9.x86_64 2024-05-14T09:20:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-14T09:20:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-14T09:20:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-14T09:20:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-14T09:20:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-14T09:20:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-14T09:20:51.942 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:20:51.942 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-05-14T09:20:52.074 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 111/164 2024-05-14T09:20:52.101 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 111/164 2024-05-14T09:20:52.101 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:52.101 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-14T09:20:52.101 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T09:20:52.101 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T09:20:52.102 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-14T09:20:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 112/164 2024-05-14T09:20:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 112/164 2024-05-14T09:20:52.356 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:52.356 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-14T09:20:52.357 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T09:20:52.357 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T09:20:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-14T09:20:52.383 DEBUG:teuthology.parallel:result is None 2024-05-14T09:20:52.858 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 113/164 2024-05-14T09:20:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 113/164 2024-05-14T09:20:52.881 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:52.881 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-14T09:20:52.881 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T09:20:52.881 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T09:20:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-14T09:20:52.988 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-14T09:20:53.046 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-14T09:20:53.110 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-14T09:20:53.303 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-14T09:20:53.359 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-14T09:20:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-14T09:20:53.529 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-14T09:20:53.674 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-14T09:20:53.709 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-14T09:20:53.750 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-14T09:20:53.785 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-14T09:20:54.189 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 125/164 2024-05-14T09:20:54.217 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 125/164 2024-05-14T09:20:54.250 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.e 126/164 2024-05-14T09:20:54.313 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6. 127/164 2024-05-14T09:20:54.340 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 128/164 2024-05-14T09:20:55.431 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 128/164 2024-05-14T09:20:55.467 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 129/164 2024-05-14T09:20:56.482 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 129/164 2024-05-14T09:20:56.509 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 130/164 2024-05-14T09:20:56.626 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 130/164 2024-05-14T09:20:56.808 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el 131/164 2024-05-14T09:20:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 132/164 2024-05-14T09:20:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 132/164 2024-05-14T09:20:56.869 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:56.869 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-14T09:20:56.869 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T09:20:56.869 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T09:20:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-14T09:20:56.920 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 133/164 2024-05-14T09:20:56.932 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 133/164 2024-05-14T09:20:57.034 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 134/164 2024-05-14T09:20:57.087 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 135/164 2024-05-14T09:20:58.073 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-14T09:20:58.106 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 137/164 2024-05-14T09:20:58.129 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 137/164 2024-05-14T09:20:58.130 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:58.130 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-14T09:20:58.130 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T09:20:58.130 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T09:20:58.130 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-14T09:20:58.173 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 138/164 2024-05-14T09:20:58.196 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 138/164 2024-05-14T09:20:58.196 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:58.196 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-14T09:20:58.197 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-14T09:20:58.249 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 139/164 2024-05-14T09:20:58.272 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 139/164 2024-05-14T09:20:58.272 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:58.272 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-14T09:20:58.272 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T09:20:58.272 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T09:20:58.273 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-14T09:20:58.441 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T09:20:58.441 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 1/164 2024-05-14T09:20:58.441 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 2/164 2024-05-14T09:20:58.441 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 3/164 2024-05-14T09:20:58.441 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 4/164 2024-05-14T09:20:58.441 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 5/164 2024-05-14T09:20:58.441 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 6/164 2024-05-14T09:20:58.441 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 7/164 2024-05-14T09:20:58.442 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T09:20:58.442 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T09:20:58.442 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 10/164 2024-05-14T09:20:58.442 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 11/164 2024-05-14T09:20:58.442 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 12/164 2024-05-14T09:20:58.442 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T09:20:58.442 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 14/164 2024-05-14T09:20:58.442 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_ 15/164 2024-05-14T09:20:58.442 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 16/164 2024-05-14T09:20:58.442 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 17/164 2024-05-14T09:20:58.442 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_6 18/164 2024-05-14T09:20:58.442 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 19/164 2024-05-14T09:20:58.442 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 20/164 2024-05-14T09:20:58.442 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el 21/164 2024-05-14T09:20:58.443 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9. 22/164 2024-05-14T09:20:58.443 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_6 23/164 2024-05-14T09:20:58.443 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 24/164 2024-05-14T09:20:58.443 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 25/164 2024-05-14T09:20:58.443 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 26/164 2024-05-14T09:20:58.443 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 27/164 2024-05-14T09:20:58.456 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 28/164 2024-05-14T09:20:58.457 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 29/164 2024-05-14T09:20:58.457 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6. 30/164 2024-05-14T09:20:58.457 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 31/164 2024-05-14T09:20:58.457 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 32/164 2024-05-14T09:20:58.457 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 33/164 2024-05-14T09:20:58.457 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el 34/164 2024-05-14T09:20:58.457 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 35/164 2024-05-14T09:20:58.457 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.e 36/164 2024-05-14T09:20:58.457 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 37/164 2024-05-14T09:20:58.457 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 38/164 2024-05-14T09:20:58.457 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 39/164 2024-05-14T09:20:58.457 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-14T09:20:58.458 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-14T09:20:58.458 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-14T09:20:58.458 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-14T09:20:58.458 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-14T09:20:58.458 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-14T09:20:58.458 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-14T09:20:58.459 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-14T09:20:58.459 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-14T09:20:58.459 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-14T09:20:58.459 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-14T09:20:58.459 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-14T09:20:58.459 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-14T09:20:58.459 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-14T09:20:58.459 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-14T09:20:58.459 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-14T09:20:58.459 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-14T09:20:58.459 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-14T09:20:58.460 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-14T09:20:58.460 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-14T09:20:58.460 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-14T09:20:58.460 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-14T09:20:58.460 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-14T09:20:58.460 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-14T09:20:58.460 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-14T09:20:58.460 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-14T09:20:58.460 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-14T09:20:58.460 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-14T09:20:58.460 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-14T09:20:58.460 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-14T09:20:58.461 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-14T09:20:58.461 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-14T09:20:58.461 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-14T09:20:58.461 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-14T09:20:58.461 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-14T09:20:58.461 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-14T09:20:58.461 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-14T09:20:58.461 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-14T09:20:58.461 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-14T09:20:58.461 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-14T09:20:58.461 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-14T09:20:58.461 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-14T09:20:58.461 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-14T09:20:58.462 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-14T09:20:58.462 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-14T09:20:58.462 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-14T09:20:58.462 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-14T09:20:58.462 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-14T09:20:58.462 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-14T09:20:58.462 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-14T09:20:58.462 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-14T09:20:58.462 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-14T09:20:58.462 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-14T09:20:58.462 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-14T09:20:58.462 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-14T09:20:58.462 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-14T09:20:58.463 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-14T09:20:58.463 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-14T09:20:58.463 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-14T09:20:58.463 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-14T09:20:58.463 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-14T09:20:58.463 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-14T09:20:58.463 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-14T09:20:58.463 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-14T09:20:58.463 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-14T09:20:58.463 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-14T09:20:58.463 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-14T09:20:58.463 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-14T09:20:58.464 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-14T09:20:58.464 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-14T09:20:58.464 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-14T09:20:58.464 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-14T09:20:58.464 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-14T09:20:58.464 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-14T09:20:58.464 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-14T09:20:58.464 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-14T09:20:58.464 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-14T09:20:58.464 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-14T09:20:58.464 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-14T09:20:58.465 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-14T09:20:58.465 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-14T09:20:58.465 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-14T09:20:58.465 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-14T09:20:58.465 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-14T09:20:58.465 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-14T09:20:58.465 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-14T09:20:58.465 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-14T09:20:58.465 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-14T09:20:58.465 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-14T09:20:58.465 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-14T09:20:58.465 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-14T09:20:58.465 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-14T09:20:58.466 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-14T09:20:58.466 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-14T09:20:58.466 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-14T09:20:58.466 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-14T09:20:58.466 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-14T09:20:58.466 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-14T09:20:58.466 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-14T09:20:58.466 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-14T09:20:58.466 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-14T09:20:58.466 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-14T09:20:58.466 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-14T09:20:58.466 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-14T09:20:58.466 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-14T09:20:58.466 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-14T09:20:58.467 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-14T09:20:58.467 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-14T09:20:58.467 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-14T09:20:58.467 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 149/164 2024-05-14T09:20:58.467 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-14T09:20:58.467 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 151/164 2024-05-14T09:20:58.467 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-14T09:20:58.467 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-14T09:20:58.467 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-14T09:20:58.467 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-14T09:20:58.467 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-14T09:20:58.467 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-14T09:20:58.467 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-14T09:20:58.467 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-14T09:20:58.467 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-14T09:20:58.468 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-14T09:20:58.468 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-14T09:20:58.468 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-14T09:20:58.600 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 140/164 2024-05-14T09:20:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 140/164 2024-05-14T09:20:58.624 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T09:20:58.624 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-14T09:20:58.624 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T09:20:58.624 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T09:20:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-14T09:21:01.104 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T09:21:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:21:01.105 INFO:teuthology.orchestra.run.smithi029.stdout:Upgraded: 2024-05-14T09:21:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-14T09:21:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-14T09:21:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-14T09:21:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-14T09:21:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-14T09:21:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-14T09:21:01.105 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2024-05-14T09:21:01.105 INFO:teuthology.orchestra.run.smithi029.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-14T09:21:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: automake-1.16.2-8.el9.noarch 2024-05-14T09:21:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: bison-3.7.4-5.el9.x86_64 2024-05-14T09:21:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-14T09:21:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-immutable-object-cache-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.106 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:01.107 INFO:teuthology.orchestra.run.smithi029.stdout: cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-14T09:21:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: flex-2.6.4-9.el9.x86_64 2024-05-14T09:21:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-14T09:21:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-14T09:21:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-14T09:21:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-14T09:21:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-14T09:21:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-14T09:21:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-14T09:21:01.108 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-14T09:21:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-14T09:21:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-14T09:21:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-14T09:21:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-14T09:21:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-14T09:21:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-14T09:21:01.109 INFO:teuthology.orchestra.run.smithi029.stdout: librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-14T09:21:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-14T09:21:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-14T09:21:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-14T09:21:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-14T09:21:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-14T09:21:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: lua-5.4.4-4.el9.x86_64 2024-05-14T09:21:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-14T09:21:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-14T09:21:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-14T09:21:01.110 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-14T09:21:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-14T09:21:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-14T09:21:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-13.14-1.el9.x86_64 2024-05-14T09:21:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-14T09:21:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-14T09:21:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-14T09:21:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-14T09:21:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-14T09:21:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-14T09:21:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-14T09:21:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-14T09:21:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-14T09:21:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-14T09:21:01.111 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-14T09:21:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-14T09:21:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-14T09:21:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-14T09:21:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-14T09:21:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-14T09:21:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-14T09:21:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-14T09:21:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-14T09:21:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-14T09:21:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-14T09:21:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-14T09:21:01.112 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-14T09:21:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-14T09:21:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-14T09:21:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-14T09:21:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-14T09:21:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-14T09:21:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-14T09:21:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-14T09:21:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-14T09:21:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-14T09:21:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-14T09:21:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-14T09:21:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-14T09:21:01.113 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-14T09:21:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-14T09:21:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-14T09:21:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-14T09:21:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-14T09:21:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ply-3.11-14.el9.noarch 2024-05-14T09:21:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-14T09:21:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-14T09:21:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-14T09:21:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-14T09:21:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-14T09:21:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-14T09:21:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-14T09:21:01.114 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-14T09:21:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-14T09:21:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-14T09:21:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-14T09:21:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-14T09:21:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-14T09:21:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-14T09:21:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-14T09:21:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-14T09:21:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-14T09:21:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-14T09:21:01.115 INFO:teuthology.orchestra.run.smithi029.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-14T09:21:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-14T09:21:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-14T09:21:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-14T09:21:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-14T09:21:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-14T09:21:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-14T09:21:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-14T09:21:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-14T09:21:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: re2-1:20211101-3.el9.x86_64 2024-05-14T09:21:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-14T09:21:01.116 INFO:teuthology.orchestra.run.smithi029.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-14T09:21:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-14T09:21:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-14T09:21:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-14T09:21:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-14T09:21:01.117 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:21:01.117 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2024-05-14T09:21:01.639 DEBUG:teuthology.parallel:result is None 2024-05-14T09:21:03.146 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 141/164 2024-05-14T09:21:03.188 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 142/164 2024-05-14T09:21:03.229 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 143/164 2024-05-14T09:21:03.259 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-14T09:21:03.287 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-14T09:21:03.482 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 146/164 2024-05-14T09:21:03.525 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-14T09:21:03.554 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_ 148/164 2024-05-14T09:21:03.584 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-14T09:21:03.639 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-14T09:21:04.248 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T09:21:04.287 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T09:21:04.307 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T09:21:05.195 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 152/164 2024-05-14T09:21:05.291 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-14T09:21:05.333 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-14T09:21:05.418 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-14T09:21:05.452 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-14T09:21:05.478 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-14T09:21:05.479 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T09:21:05.546 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T09:21:05.579 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-14T09:21:05.580 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T09:21:05.664 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T09:21:05.688 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-14T09:21:05.693 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T09:21:05.694 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T09:21:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T09:21:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T09:21:05.966 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T09:21:06.032 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T09:21:11.597 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T09:21:11.597 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 1/164 2024-05-14T09:21:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 2/164 2024-05-14T09:21:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 3/164 2024-05-14T09:21:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 4/164 2024-05-14T09:21:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 5/164 2024-05-14T09:21:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 6/164 2024-05-14T09:21:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 7/164 2024-05-14T09:21:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T09:21:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T09:21:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 10/164 2024-05-14T09:21:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 11/164 2024-05-14T09:21:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 12/164 2024-05-14T09:21:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T09:21:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 14/164 2024-05-14T09:21:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_ 15/164 2024-05-14T09:21:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 16/164 2024-05-14T09:21:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 17/164 2024-05-14T09:21:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_6 18/164 2024-05-14T09:21:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 19/164 2024-05-14T09:21:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 20/164 2024-05-14T09:21:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el 21/164 2024-05-14T09:21:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9. 22/164 2024-05-14T09:21:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_6 23/164 2024-05-14T09:21:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 24/164 2024-05-14T09:21:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 25/164 2024-05-14T09:21:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 26/164 2024-05-14T09:21:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 27/164 2024-05-14T09:21:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 28/164 2024-05-14T09:21:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 29/164 2024-05-14T09:21:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6. 30/164 2024-05-14T09:21:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 31/164 2024-05-14T09:21:11.611 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 32/164 2024-05-14T09:21:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 33/164 2024-05-14T09:21:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el 34/164 2024-05-14T09:21:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 35/164 2024-05-14T09:21:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.e 36/164 2024-05-14T09:21:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 37/164 2024-05-14T09:21:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 38/164 2024-05-14T09:21:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 39/164 2024-05-14T09:21:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-14T09:21:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-14T09:21:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-14T09:21:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-14T09:21:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-14T09:21:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-14T09:21:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-14T09:21:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-14T09:21:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-14T09:21:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-14T09:21:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-14T09:21:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-14T09:21:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-14T09:21:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-14T09:21:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-14T09:21:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-14T09:21:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-14T09:21:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-14T09:21:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-14T09:21:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-14T09:21:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-14T09:21:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-14T09:21:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-14T09:21:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-14T09:21:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-14T09:21:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-14T09:21:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-14T09:21:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-14T09:21:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-14T09:21:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-14T09:21:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-14T09:21:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-14T09:21:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-14T09:21:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-14T09:21:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-14T09:21:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-14T09:21:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-14T09:21:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-14T09:21:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-14T09:21:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-14T09:21:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-14T09:21:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-14T09:21:11.616 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-14T09:21:11.617 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-14T09:21:11.617 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-14T09:21:11.617 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-14T09:21:11.617 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-14T09:21:11.617 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-14T09:21:11.617 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-14T09:21:11.617 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-14T09:21:11.617 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-14T09:21:11.617 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-14T09:21:11.617 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-14T09:21:11.617 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-14T09:21:11.617 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-14T09:21:11.617 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-14T09:21:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-14T09:21:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-14T09:21:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-14T09:21:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-14T09:21:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-14T09:21:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-14T09:21:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-14T09:21:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-14T09:21:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-14T09:21:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-14T09:21:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-14T09:21:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-14T09:21:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-14T09:21:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-14T09:21:11.619 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-14T09:21:11.619 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-14T09:21:11.619 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-14T09:21:11.619 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-14T09:21:11.619 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-14T09:21:11.619 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-14T09:21:11.619 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-14T09:21:11.619 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-14T09:21:11.619 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-14T09:21:11.619 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-14T09:21:11.619 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-14T09:21:11.619 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-14T09:21:11.619 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-14T09:21:11.620 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-14T09:21:11.620 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-14T09:21:11.620 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-14T09:21:11.620 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-14T09:21:11.620 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-14T09:21:11.620 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-14T09:21:11.620 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-14T09:21:11.620 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-14T09:21:11.620 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-14T09:21:11.620 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-14T09:21:11.620 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-14T09:21:11.620 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-14T09:21:11.620 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-14T09:21:11.620 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-14T09:21:11.620 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-14T09:21:11.621 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-14T09:21:11.621 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-14T09:21:11.621 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-14T09:21:11.621 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-14T09:21:11.621 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-14T09:21:11.621 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-14T09:21:11.621 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-14T09:21:11.621 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-14T09:21:11.621 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-14T09:21:11.621 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-14T09:21:11.621 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-14T09:21:11.621 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 149/164 2024-05-14T09:21:11.621 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-14T09:21:11.621 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 151/164 2024-05-14T09:21:11.621 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-14T09:21:11.622 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-14T09:21:11.622 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-14T09:21:11.622 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-14T09:21:11.622 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-14T09:21:11.622 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-14T09:21:11.622 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-14T09:21:11.622 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-14T09:21:11.622 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-14T09:21:11.622 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-14T09:21:11.622 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-14T09:21:11.622 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-14T09:21:14.300 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T09:21:14.300 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-14T09:21:14.300 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2024-05-14T09:21:14.300 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-14T09:21:14.300 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-14T09:21:14.300 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-14T09:21:14.300 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-14T09:21:14.300 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.300 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.301 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-14T09:21:14.301 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-14T09:21:14.301 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-05-14T09:21:14.301 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-14T09:21:14.301 INFO:teuthology.orchestra.run.smithi026.stdout: automake-1.16.2-8.el9.noarch 2024-05-14T09:21:14.301 INFO:teuthology.orchestra.run.smithi026.stdout: bison-3.7.4-5.el9.x86_64 2024-05-14T09:21:14.301 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-14T09:21:14.301 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.301 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.301 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.301 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.301 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:14.301 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.301 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.302 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.302 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:14.302 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:14.302 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:14.302 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:14.302 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:14.302 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.302 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.302 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:14.302 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.302 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.302 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.303 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:14.303 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:14.303 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.303 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.303 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T09:21:14.303 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-14T09:21:14.303 INFO:teuthology.orchestra.run.smithi026.stdout: flex-2.6.4-9.el9.x86_64 2024-05-14T09:21:14.303 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-14T09:21:14.303 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-14T09:21:14.303 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-14T09:21:14.303 INFO:teuthology.orchestra.run.smithi026.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-14T09:21:14.303 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-14T09:21:14.303 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-14T09:21:14.303 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-14T09:21:14.304 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-14T09:21:14.304 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-14T09:21:14.304 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.304 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.304 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.304 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-14T09:21:14.304 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-14T09:21:14.304 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-14T09:21:14.304 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-14T09:21:14.304 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-14T09:21:14.304 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.304 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.304 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-14T09:21:14.304 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.304 INFO:teuthology.orchestra.run.smithi026.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: lua-5.4.4-4.el9.x86_64 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-13.14-1.el9.x86_64 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-14T09:21:14.305 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-14T09:21:14.306 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-14T09:21:14.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-14T09:21:14.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-14T09:21:14.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-14T09:21:14.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-14T09:21:14.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-14T09:21:14.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-14T09:21:14.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-14T09:21:14.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-14T09:21:14.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-14T09:21:14.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-14T09:21:14.307 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-14T09:21:14.307 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-14T09:21:14.307 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-14T09:21:14.307 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-14T09:21:14.307 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-14T09:21:14.307 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-14T09:21:14.307 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-14T09:21:14.307 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-14T09:21:14.307 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-14T09:21:14.307 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-14T09:21:14.307 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-14T09:21:14.307 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-14T09:21:14.307 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-14T09:21:14.308 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-14T09:21:14.308 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-14T09:21:14.308 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-14T09:21:14.308 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-14T09:21:14.308 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-14T09:21:14.308 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-14T09:21:14.308 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-14T09:21:14.308 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-14T09:21:14.308 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-14T09:21:14.308 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-14T09:21:14.308 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-14T09:21:14.308 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-14T09:21:14.308 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply-3.11-14.el9.noarch 2024-05-14T09:21:14.308 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-14T09:21:14.308 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-14T09:21:14.309 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-14T09:21:14.309 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-14T09:21:14.309 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-14T09:21:14.309 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-14T09:21:14.309 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-14T09:21:14.309 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-14T09:21:14.309 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-14T09:21:14.309 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.309 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.309 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-14T09:21:14.309 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-14T09:21:14.309 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-14T09:21:14.309 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.309 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-14T09:21:14.310 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-14T09:21:14.310 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-14T09:21:14.310 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-14T09:21:14.310 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-14T09:21:14.310 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-14T09:21:14.310 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-14T09:21:14.310 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-14T09:21:14.310 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-14T09:21:14.310 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-14T09:21:14.310 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-14T09:21:14.310 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-14T09:21:14.310 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-14T09:21:14.310 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-14T09:21:14.311 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-14T09:21:14.311 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.311 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.311 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T09:21:14.311 INFO:teuthology.orchestra.run.smithi026.stdout: re2-1:20211101-3.el9.x86_64 2024-05-14T09:21:14.311 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-14T09:21:14.311 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-14T09:21:14.311 INFO:teuthology.orchestra.run.smithi026.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-14T09:21:14.311 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-14T09:21:14.311 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-14T09:21:14.311 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-14T09:21:14.311 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-05-14T09:21:14.311 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-05-14T09:21:14.714 DEBUG:teuthology.parallel:result is None 2024-05-14T09:21:14.715 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T09:21:14.864 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-14T09:21:14.892 INFO:teuthology.orchestra.run.smithi002.stdout:19.0.0-3588.ge2b755a6.el9 2024-05-14T09:21:14.892 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3588.ge2b755a6.el9 2024-05-14T09:21:14.892 INFO:teuthology.task.install:The correct ceph version 19.0.0-3588.ge2b755a6 is installed. 2024-05-14T09:21:14.894 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T09:21:15.114 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-14T09:21:15.136 INFO:teuthology.orchestra.run.smithi026.stdout:19.0.0-3588.ge2b755a6.el9 2024-05-14T09:21:15.137 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3588.ge2b755a6.el9 2024-05-14T09:21:15.137 INFO:teuthology.task.install:The correct ceph version 19.0.0-3588.ge2b755a6 is installed. 2024-05-14T09:21:15.139 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T09:21:15.278 DEBUG:teuthology.orchestra.run.smithi029:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-14T09:21:15.305 INFO:teuthology.orchestra.run.smithi029.stdout:19.0.0-3588.ge2b755a6.el9 2024-05-14T09:21:15.305 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3588.ge2b755a6.el9 2024-05-14T09:21:15.305 INFO:teuthology.task.install:The correct ceph version 19.0.0-3588.ge2b755a6 is installed. 2024-05-14T09:21:15.307 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-14T09:21:15.307 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:21:15.307 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T09:21:15.340 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-14T09:21:15.341 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T09:21:15.368 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-05-14T09:21:15.368 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T09:21:15.400 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-14T09:21:15.400 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:21:15.400 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T09:21:15.429 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T09:21:15.499 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-14T09:21:15.499 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T09:21:15.526 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T09:21:15.591 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-05-14T09:21:15.591 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T09:21:15.621 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T09:21:15.690 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-14T09:21:15.690 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:21:15.690 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T09:21:15.720 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T09:21:15.788 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-14T09:21:15.788 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T09:21:15.815 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T09:21:15.881 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-05-14T09:21:15.881 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T09:21:15.910 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T09:21:15.978 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-14T09:21:15.979 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:21:15.979 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T09:21:16.004 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T09:21:16.070 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-14T09:21:16.071 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T09:21:16.098 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T09:21:16.165 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-05-14T09:21:16.165 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T09:21:16.194 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T09:21:16.263 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-14T09:21:16.371 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780', 'subvols': {'create': 2}} 2024-05-14T09:21:16.372 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T09:21:16.372 INFO:tasks.cephadm:Cluster fsid is 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:21:16.372 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-14T09:21:16.373 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.2', 'mon.b': '172.21.15.26', 'mon.c': '172.21.15.29'} 2024-05-14T09:21:16.373 INFO:tasks.cephadm:First mon is mon.a on smithi002 2024-05-14T09:21:16.373 INFO:tasks.cephadm:First mgr is x 2024-05-14T09:21:16.373 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-14T09:21:16.373 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname $(hostname -s) 2024-05-14T09:21:16.401 DEBUG:teuthology.orchestra.run.smithi026:> sudo hostname $(hostname -s) 2024-05-14T09:21:16.429 DEBUG:teuthology.orchestra.run.smithi029:> sudo hostname $(hostname -s) 2024-05-14T09:21:16.459 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-14T09:21:16.459 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T09:21:16.605 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79237/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3588-ge2b755a6', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3588.ge2b755a6'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-09 07:03:10.506903', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240509.053109-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79236/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3588-ge2b755a6', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3588.ge2b755a6'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-09 06:43:33.316648', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240509.053109-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-14T09:21:16.825 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-vshankar-testing-20240509.053109-debug, sha1 e2b755a64f0d72176b57299d02e880cf0620d780 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T09:21:16.827 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/x86_64/flavors/default/cephadm 2024-05-14T09:21:16.827 DEBUG:teuthology.orchestra.run.smithi002:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-14T09:21:17.966 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-r--r--. 1 ubuntu ubuntu 782846 May 14 09:21 /home/ubuntu/cephtest/cephadm 2024-05-14T09:21:17.966 DEBUG:teuthology.orchestra.run.smithi026:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-14T09:21:18.756 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r--. 1 ubuntu ubuntu 782846 May 14 09:21 /home/ubuntu/cephtest/cephadm 2024-05-14T09:21:18.757 DEBUG:teuthology.orchestra.run.smithi029:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-14T09:21:19.376 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-r--r--. 1 ubuntu ubuntu 782846 May 14 09:21 /home/ubuntu/cephtest/cephadm 2024-05-14T09:21:19.376 DEBUG:teuthology.orchestra.run.smithi002:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-14T09:21:19.397 DEBUG:teuthology.orchestra.run.smithi026:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-14T09:21:19.417 DEBUG:teuthology.orchestra.run.smithi029:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-14T09:21:19.453 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 on all hosts... 2024-05-14T09:21:19.453 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 pull 2024-05-14T09:21:19.456 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 pull 2024-05-14T09:21:19.460 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 pull 2024-05-14T09:21:20.252 INFO:teuthology.orchestra.run.smithi002.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780... 2024-05-14T09:21:20.434 INFO:teuthology.orchestra.run.smithi029.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780... 2024-05-14T09:21:20.443 INFO:teuthology.orchestra.run.smithi026.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780... 2024-05-14T09:21:55.245 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:21:55.245 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T09:21:55.246 INFO:teuthology.orchestra.run.smithi002.stdout: "image_id": "bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345", 2024-05-14T09:21:55.246 INFO:teuthology.orchestra.run.smithi002.stdout: "repo_digests": [ 2024-05-14T09:21:55.246 INFO:teuthology.orchestra.run.smithi002.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430" 2024-05-14T09:21:55.246 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T09:21:55.246 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:22:22.159 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-14T09:22:22.160 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T09:22:22.160 INFO:teuthology.orchestra.run.smithi026.stdout: "image_id": "bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345", 2024-05-14T09:22:22.160 INFO:teuthology.orchestra.run.smithi026.stdout: "repo_digests": [ 2024-05-14T09:22:22.160 INFO:teuthology.orchestra.run.smithi026.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430" 2024-05-14T09:22:22.160 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-05-14T09:22:22.160 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-14T09:22:24.021 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-05-14T09:22:24.021 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T09:22:24.021 INFO:teuthology.orchestra.run.smithi029.stdout: "image_id": "bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345", 2024-05-14T09:22:24.022 INFO:teuthology.orchestra.run.smithi029.stdout: "repo_digests": [ 2024-05-14T09:22:24.022 INFO:teuthology.orchestra.run.smithi029.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430" 2024-05-14T09:22:24.022 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-05-14T09:22:24.022 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-05-14T09:22:24.048 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /etc/ceph 2024-05-14T09:22:24.083 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph 2024-05-14T09:22:24.116 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /etc/ceph 2024-05-14T09:22:24.144 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 777 /etc/ceph 2024-05-14T09:22:24.173 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /etc/ceph 2024-05-14T09:22:24.201 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 777 /etc/ceph 2024-05-14T09:22:24.227 INFO:tasks.cephadm:Writing seed config... 2024-05-14T09:22:24.228 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-14T09:22:24.228 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-14T09:22:24.228 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-14T09:22:24.228 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-14T09:22:24.229 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-14T09:22:24.229 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-14T09:22:24.229 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-14T09:22:24.229 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-14T09:22:24.229 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-14T09:22:24.229 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-14T09:22:24.229 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-14T09:22:24.229 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-14T09:22:24.229 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-14T09:22:24.229 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-14T09:22:24.229 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-14T09:22:24.229 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-14T09:22:24.230 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-14T09:22:24.230 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-14T09:22:24.230 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-14T09:22:24.230 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-14T09:22:24.230 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-14T09:22:24.230 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-14T09:22:24.230 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-14T09:22:24.230 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-14T09:22:24.230 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-14T09:22:24.230 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-14T09:22:24.230 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-14T09:22:24.230 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-14T09:22:24.231 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-14T09:22:24.231 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-14T09:22:24.231 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-14T09:22:24.231 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-14T09:22:24.231 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-14T09:22:24.231 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-14T09:22:24.231 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-14T09:22:24.231 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-14T09:22:24.231 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-14T09:22:24.231 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-14T09:22:24.231 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-14T09:22:24.231 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-14T09:22:24.231 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-14T09:22:24.232 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-14T09:22:24.232 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-14T09:22:24.232 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-14T09:22:24.232 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-14T09:22:24.233 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:22:24.233 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-14T09:22:24.251 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 50d5aebe-11d3-11ef-bc98-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-14T09:22:24.252 DEBUG:teuthology.orchestra.run.smithi002:mon.a> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.a.service 2024-05-14T09:22:24.295 DEBUG:teuthology.orchestra.run.smithi002:mgr.x> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mgr.x.service 2024-05-14T09:22:24.338 INFO:tasks.cephadm:Bootstrapping... 2024-05-14T09:22:24.339 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 -v bootstrap --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.2 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-14T09:22:24.544 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-05-14T09:22:24.544 INFO:teuthology.orchestra.run.smithi002.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780', '-v', 'bootstrap', '--fsid', '50d5aebe-11d3-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.2', '--skip-admin-label'] 2024-05-14T09:22:24.544 INFO:teuthology.orchestra.run.smithi002.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-14T09:22:24.544 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying podman|docker is present... 2024-05-14T09:22:24.571 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 5.0.2 2024-05-14T09:22:24.571 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying lvm2 is present... 2024-05-14T09:22:24.571 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying time synchronization is in place... 2024-05-14T09:22:24.579 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-14T09:22:24.579 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-14T09:22:24.585 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-14T09:22:24.585 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-05-14T09:22:24.592 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout enabled 2024-05-14T09:22:24.599 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout active 2024-05-14T09:22:24.599 INFO:teuthology.orchestra.run.smithi002.stdout:Unit chronyd.service is enabled and running 2024-05-14T09:22:24.599 INFO:teuthology.orchestra.run.smithi002.stdout:Repeating the final host check... 2024-05-14T09:22:24.623 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 5.0.2 2024-05-14T09:22:24.624 INFO:teuthology.orchestra.run.smithi002.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-14T09:22:24.624 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl is present 2024-05-14T09:22:24.624 INFO:teuthology.orchestra.run.smithi002.stdout:lvcreate is present 2024-05-14T09:22:24.631 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-14T09:22:24.631 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-14T09:22:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-14T09:22:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-05-14T09:22:24.644 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout enabled 2024-05-14T09:22:24.650 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout active 2024-05-14T09:22:24.651 INFO:teuthology.orchestra.run.smithi002.stdout:Unit chronyd.service is enabled and running 2024-05-14T09:22:24.651 INFO:teuthology.orchestra.run.smithi002.stdout:Host looks OK 2024-05-14T09:22:24.651 INFO:teuthology.orchestra.run.smithi002.stdout:Cluster fsid: 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:22:24.651 INFO:teuthology.orchestra.run.smithi002.stdout:Acquiring lock 140040022286288 on /run/cephadm/50d5aebe-11d3-11ef-bc98-c7b262605968.lock 2024-05-14T09:22:24.651 INFO:teuthology.orchestra.run.smithi002.stdout:Lock 140040022286288 acquired on /run/cephadm/50d5aebe-11d3-11ef-bc98-c7b262605968.lock 2024-05-14T09:22:24.651 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying IP 172.21.15.2 port 3300 ... 2024-05-14T09:22:24.652 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying IP 172.21.15.2 port 6789 ... 2024-05-14T09:22:24.652 INFO:teuthology.orchestra.run.smithi002.stdout:Base mon IP(s) is [172.21.15.2:3300, 172.21.15.2:6789], mon addrv is [v2:172.21.15.2:3300,v1:172.21.15.2:6789] 2024-05-14T09:22:24.655 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.2 metric 100 2024-05-14T09:22:24.655 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.2 metric 100 2024-05-14T09:22:24.658 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-14T09:22:24.658 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-14T09:22:24.658 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-14T09:22:24.660 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-14T09:22:24.660 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-14T09:22:24.661 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-14T09:22:24.661 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-14T09:22:24.661 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1920/64 scope link noprefixroute 2024-05-14T09:22:24.661 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-14T09:22:24.661 INFO:teuthology.orchestra.run.smithi002.stdout:Mon IP `172.21.15.2` is in CIDR network `172.21.0.0/20` 2024-05-14T09:22:24.661 INFO:teuthology.orchestra.run.smithi002.stdout:Mon IP `172.21.15.2` is in CIDR network `172.21.0.0/20` 2024-05-14T09:22:24.662 INFO:teuthology.orchestra.run.smithi002.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-14T09:22:24.662 INFO:teuthology.orchestra.run.smithi002.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-14T09:22:24.663 INFO:teuthology.orchestra.run.smithi002.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780... 2024-05-14T09:22:26.837 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 2024-05-14T09:22:26.838 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780... 2024-05-14T09:22:26.838 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Getting image source signatures 2024-05-14T09:22:26.838 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying blob sha256:d2511a189d36a438cd7259d98515a840197dc63bdcef8f78efcfd8f43785c919 2024-05-14T09:22:26.838 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-14T09:22:26.838 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying config sha256:bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 2024-05-14T09:22:26.838 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-14T09:22:27.791 INFO:teuthology.orchestra.run.smithi002.stdout:ceph: stdout ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev) 2024-05-14T09:22:27.791 INFO:teuthology.orchestra.run.smithi002.stdout:Ceph version: ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev) 2024-05-14T09:22:27.791 INFO:teuthology.orchestra.run.smithi002.stdout:Extracting ceph user uid/gid from container image... 2024-05-14T09:22:28.789 INFO:teuthology.orchestra.run.smithi002.stdout:stat: stdout 167 167 2024-05-14T09:22:28.789 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial keys... 2024-05-14T09:22:29.677 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQBVLUNmJLhaBhAAZLFDq5+cWRcFwwulO7AMWw== 2024-05-14T09:22:30.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQBWLUNmH78+ARAAs4wkkPlXJpOJ8gcKPJM/vw== 2024-05-14T09:22:31.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQBWLUNmco3IMhAA5cNX0CGblBjvWqHgYryZeA== 2024-05-14T09:22:31.398 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial monmap... 2024-05-14T09:22:32.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-14T09:22:32.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-14T09:22:32.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:22:32.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-14T09:22:32.521 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool for a [v2:172.21.15.2:3300,v1:172.21.15.2:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-14T09:22:32.521 INFO:teuthology.orchestra.run.smithi002.stdout:setting min_mon_release = quincy 2024-05-14T09:22:32.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: set fsid to 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:22:32.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-14T09:22:32.521 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:22:32.521 INFO:teuthology.orchestra.run.smithi002.stdout:Creating mon... 2024-05-14T09:22:33.418 INFO:teuthology.orchestra.run.smithi002.stdout:create mon.a on 2024-05-14T09:22:33.632 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-14T09:22:33.797 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-14T09:22:34.017 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-50d5aebe-11d3-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-50d5aebe-11d3-11ef-bc98-c7b262605968.target. 2024-05-14T09:22:34.017 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-50d5aebe-11d3-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-50d5aebe-11d3-11ef-bc98-c7b262605968.target. 2024-05-14T09:22:34.249 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.a 2024-05-14T09:22:34.249 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to reset failed state of unit ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-14T09:22:34.426 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-50d5aebe-11d3-11ef-bc98-c7b262605968.target.wants/ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@.service. 2024-05-14T09:22:34.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:34 smithi002 podman[31166]: 2024-05-14 09:22:34.700998469 +0000 UTC m=+0.144492363 container create 25946a5344925456c7ff9f079196b30d82eb6f590343eeb23ea45f224c569448 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-14T09:22:34.967 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T09:22:34.967 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-05-14T09:22:34.975 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T09:22:34.975 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-05-14T09:22:34.975 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-05-14T09:22:34.975 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to enable service . firewalld.service is not available 2024-05-14T09:22:34.975 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mon to start... 2024-05-14T09:22:34.975 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mon... 2024-05-14T09:22:35.069 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:34 smithi002 podman[31166]: 2024-05-14 09:22:34.878913971 +0000 UTC m=+0.322407898 container init 25946a5344925456c7ff9f079196b30d82eb6f590343eeb23ea45f224c569448 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-14T09:22:35.069 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:34 smithi002 podman[31166]: 2024-05-14 09:22:34.882694988 +0000 UTC m=+0.326188884 container start 25946a5344925456c7ff9f079196b30d82eb6f590343eeb23ea45f224c569448 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-14T09:22:35.069 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:34 smithi002 ceph-mon[31184]: mkfs 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:22:35.069 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:34 smithi002 ceph-mon[31184]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-14T09:22:35.069 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:34 smithi002 bash[31166]: 25946a5344925456c7ff9f079196b30d82eb6f590343eeb23ea45f224c569448 2024-05-14T09:22:35.069 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:34 smithi002 systemd[1]: Started Ceph mon.a for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:22:36.126 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:35 smithi002 ceph-mon[31184]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-14T09:22:36.126 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:35 smithi002 ceph-mon[31184]: monmap epoch 1 2024-05-14T09:22:36.126 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:35 smithi002 ceph-mon[31184]: fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:22:36.126 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:35 smithi002 ceph-mon[31184]: last_changed 2024-05-14T09:22:31.786962+0000 2024-05-14T09:22:36.126 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:35 smithi002 ceph-mon[31184]: created 2024-05-14T09:22:31.786962+0000 2024-05-14T09:22:36.127 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:35 smithi002 ceph-mon[31184]: min_mon_release 19 (squid) 2024-05-14T09:22:36.127 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:35 smithi002 ceph-mon[31184]: election_strategy: 1 2024-05-14T09:22:36.127 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:35 smithi002 ceph-mon[31184]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T09:22:36.127 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:35 smithi002 ceph-mon[31184]: fsmap 2024-05-14T09:22:36.127 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:35 smithi002 ceph-mon[31184]: osdmap e1: 0 total, 0 up, 0 in 2024-05-14T09:22:36.127 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:35 smithi002 ceph-mon[31184]: mgrmap e1: no daemons active 2024-05-14T09:22:36.127 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:35 smithi002 ceph-mon[31184]: from='client.? 172.21.15.2:0/1485514297' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-14T09:22:36.136 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout cluster: 2024-05-14T09:22:36.136 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout id: 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:22:36.136 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-14T09:22:36.136 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T09:22:36.136 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout services: 2024-05-14T09:22:36.136 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.58238s) 2024-05-14T09:22:36.136 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-14T09:22:36.136 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-14T09:22:36.136 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T09:22:36.136 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout data: 2024-05-14T09:22:36.136 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-14T09:22:36.136 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-14T09:22:36.136 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-14T09:22:36.137 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout pgs: 2024-05-14T09:22:36.137 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T09:22:36.137 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.436+0000 7f10f72db700 1 Processor -- start 2024-05-14T09:22:36.137 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.436+0000 7f10f72db700 1 -- start start 2024-05-14T09:22:36.137 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.436+0000 7f10f72db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f010a560 0x7f10f010c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:36.137 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.436+0000 7f10f72db700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f10f007d8c0 con 0x7f10f010a560 2024-05-14T09:22:36.137 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.437+0000 7f10f5077700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f010a560 0x7f10f010c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:36.137 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.437+0000 7f10f5077700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f010a560 0x7f10f010c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53890/0 (socket says 172.21.15.2:53890) 2024-05-14T09:22:36.137 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.437+0000 7f10f5077700 1 -- 172.21.15.2:0/3519335807 learned_addr learned my addr 172.21.15.2:0/3519335807 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:36.137 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.437+0000 7f10f5077700 1 -- 172.21.15.2:0/3519335807 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f010cec0 con 0x7f10f010a560 2024-05-14T09:22:36.137 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.437+0000 7f10f5077700 1 --2- 172.21.15.2:0/3519335807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f010a560 0x7f10f010c980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0098a0 tx=0x7f10ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38845bfc1155481d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:36.138 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.437+0000 7f10e7fff700 1 -- 172.21.15.2:0/3519335807 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10ec004160 con 0x7f10f010a560 2024-05-14T09:22:36.138 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.437+0000 7f10e7fff700 1 -- 172.21.15.2:0/3519335807 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10ec004300 con 0x7f10f010a560 2024-05-14T09:22:36.138 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.437+0000 7f10e7fff700 1 -- 172.21.15.2:0/3519335807 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10ec0036a0 con 0x7f10f010a560 2024-05-14T09:22:36.138 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.438+0000 7f10f72db700 1 -- 172.21.15.2:0/3519335807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f010a560 msgr2=0x7f10f010c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:36.138 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.438+0000 7f10f72db700 1 --2- 172.21.15.2:0/3519335807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f010a560 0x7f10f010c980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0098a0 tx=0x7f10ec009bb0 comp rx=0 tx=0).stop 2024-05-14T09:22:36.138 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.438+0000 7f10f72db700 1 -- 172.21.15.2:0/3519335807 shutdown_connections 2024-05-14T09:22:36.138 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.438+0000 7f10f72db700 1 --2- 172.21.15.2:0/3519335807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f010a560 0x7f10f010c980 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:36.138 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.438+0000 7f10f72db700 1 -- 172.21.15.2:0/3519335807 >> 172.21.15.2:0/3519335807 conn(0x7f10f0104040 msgr2=0x7f10f0106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:36.138 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.438+0000 7f10f72db700 1 -- 172.21.15.2:0/3519335807 shutdown_connections 2024-05-14T09:22:36.138 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.438+0000 7f10f72db700 1 -- 172.21.15.2:0/3519335807 wait complete. 2024-05-14T09:22:36.138 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.439+0000 7f10f72db700 1 Processor -- start 2024-05-14T09:22:36.138 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.439+0000 7f10f72db700 1 -- start start 2024-05-14T09:22:36.139 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.439+0000 7f10f72db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f010a560 0x7f10f019d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:36.139 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.439+0000 7f10f72db700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f10f010d330 con 0x7f10f010a560 2024-05-14T09:22:36.139 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.439+0000 7f10f5077700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f010a560 0x7f10f019d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:36.139 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.439+0000 7f10f5077700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f010a560 0x7f10f019d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53896/0 (socket says 172.21.15.2:53896) 2024-05-14T09:22:36.139 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.439+0000 7f10f5077700 1 -- 172.21.15.2:0/1485514297 learned_addr learned my addr 172.21.15.2:0/1485514297 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:36.139 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.439+0000 7f10f5077700 1 -- 172.21.15.2:0/1485514297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f019de30 con 0x7f10f010a560 2024-05-14T09:22:36.139 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.440+0000 7f10f5077700 1 --2- 172.21.15.2:0/1485514297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f010a560 0x7f10f019d8f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f10ec000c00 tx=0x7f10ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:36.139 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.440+0000 7f10e67fc700 1 -- 172.21.15.2:0/1485514297 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10ec003db0 con 0x7f10f010a560 2024-05-14T09:22:36.139 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.440+0000 7f10e67fc700 1 -- 172.21.15.2:0/1485514297 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10ec011630 con 0x7f10f010a560 2024-05-14T09:22:36.139 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.440+0000 7f10e67fc700 1 -- 172.21.15.2:0/1485514297 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10ec011950 con 0x7f10f010a560 2024-05-14T09:22:36.139 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.440+0000 7f10f72db700 1 -- 172.21.15.2:0/1485514297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f019e0c0 con 0x7f10f010a560 2024-05-14T09:22:36.139 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.440+0000 7f10f72db700 1 -- 172.21.15.2:0/1485514297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f01a0cd0 con 0x7f10f010a560 2024-05-14T09:22:36.139 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.440+0000 7f10e67fc700 1 -- 172.21.15.2:0/1485514297 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f10ec01a460 con 0x7f10f010a560 2024-05-14T09:22:36.139 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.440+0000 7f10e67fc700 1 -- 172.21.15.2:0/1485514297 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f10ec01b780 con 0x7f10f010a560 2024-05-14T09:22:36.140 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.441+0000 7f10f72db700 1 -- 172.21.15.2:0/1485514297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f0064e20 con 0x7f10f010a560 2024-05-14T09:22:36.140 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.444+0000 7f10e67fc700 1 -- 172.21.15.2:0/1485514297 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f10ec031020 con 0x7f10f010a560 2024-05-14T09:22:36.140 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.517+0000 7f10f72db700 1 -- 172.21.15.2:0/1485514297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f10f002e5f0 con 0x7f10f010a560 2024-05-14T09:22:36.140 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.517+0000 7f10e67fc700 1 -- 172.21.15.2:0/1485514297 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7f10ec01b9d0 con 0x7f10f010a560 2024-05-14T09:22:36.140 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.518+0000 7f10f72db700 1 -- 172.21.15.2:0/1485514297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f010a560 msgr2=0x7f10f019d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:36.140 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.518+0000 7f10f72db700 1 --2- 172.21.15.2:0/1485514297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f010a560 0x7f10f019d8f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f10ec000c00 tx=0x7f10ec004750 comp rx=0 tx=0).stop 2024-05-14T09:22:36.140 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.518+0000 7f10f72db700 1 -- 172.21.15.2:0/1485514297 shutdown_connections 2024-05-14T09:22:36.140 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.518+0000 7f10f72db700 1 --2- 172.21.15.2:0/1485514297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f010a560 0x7f10f019d8f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:36.140 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.518+0000 7f10f72db700 1 -- 172.21.15.2:0/1485514297 >> 172.21.15.2:0/1485514297 conn(0x7f10f0104040 msgr2=0x7f10f01064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:36.140 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.518+0000 7f10f72db700 1 -- 172.21.15.2:0/1485514297 shutdown_connections 2024-05-14T09:22:36.140 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:35.518+0000 7f10f72db700 1 -- 172.21.15.2:0/1485514297 wait complete. 2024-05-14T09:22:36.140 INFO:teuthology.orchestra.run.smithi002.stdout:mon is available 2024-05-14T09:22:36.140 INFO:teuthology.orchestra.run.smithi002.stdout:Assimilating anything we can from ceph.conf... 2024-05-14T09:22:37.224 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:36 smithi002 ceph-mon[31184]: from='client.? 172.21.15.2:0/3397912150' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-14T09:22:37.224 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:36 smithi002 ceph-mon[31184]: from='client.? 172.21.15.2:0/3397912150' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-14T09:22:37.226 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T09:22:37.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [global] 2024-05-14T09:22:37.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout fsid = 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:22:37.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-14T09:22:37.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.2:3300,v1:172.21.15.2:6789] 2024-05-14T09:22:37.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-14T09:22:37.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-14T09:22:37.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-14T09:22:37.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-14T09:22:37.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T09:22:37.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-14T09:22:37.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-14T09:22:37.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T09:22:37.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [osd] 2024-05-14T09:22:37.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-14T09:22:37.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-14T09:22:37.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-14T09:22:37.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-14T09:22:37.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-14T09:22:37.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.573+0000 7fed3085b700 1 Processor -- start 2024-05-14T09:22:37.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.573+0000 7fed3085b700 1 -- start start 2024-05-14T09:22:37.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.573+0000 7fed3085b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed2c1083b0 0x7fed2c1087c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:37.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.573+0000 7fed3085b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c07d860 con 0x7fed2c1083b0 2024-05-14T09:22:37.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.574+0000 7fed2a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed2c1083b0 0x7fed2c1087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:37.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.574+0000 7fed2a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed2c1083b0 0x7fed2c1087c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53910/0 (socket says 172.21.15.2:53910) 2024-05-14T09:22:37.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.574+0000 7fed2a59c700 1 -- 172.21.15.2:0/1882955883 learned_addr learned my addr 172.21.15.2:0/1882955883 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:37.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.574+0000 7fed2a59c700 1 -- 172.21.15.2:0/1882955883 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c108d00 con 0x7fed2c1083b0 2024-05-14T09:22:37.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.574+0000 7fed2a59c700 1 --2- 172.21.15.2:0/1882955883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed2c1083b0 0x7fed2c1087c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0098a0 tx=0x7fed1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348202c034fe064e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:37.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.574+0000 7fed2959a700 1 -- 172.21.15.2:0/1882955883 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed1c004160 con 0x7fed2c1083b0 2024-05-14T09:22:37.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.574+0000 7fed2959a700 1 -- 172.21.15.2:0/1882955883 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1c004300 con 0x7fed2c1083b0 2024-05-14T09:22:37.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.574+0000 7fed2959a700 1 -- 172.21.15.2:0/1882955883 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed1c0036a0 con 0x7fed2c1083b0 2024-05-14T09:22:37.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.575+0000 7fed3085b700 1 -- 172.21.15.2:0/1882955883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed2c1083b0 msgr2=0x7fed2c1087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:37.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.575+0000 7fed3085b700 1 --2- 172.21.15.2:0/1882955883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed2c1083b0 0x7fed2c1087c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0098a0 tx=0x7fed1c009bb0 comp rx=0 tx=0).stop 2024-05-14T09:22:37.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.575+0000 7fed3085b700 1 -- 172.21.15.2:0/1882955883 shutdown_connections 2024-05-14T09:22:37.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.575+0000 7fed3085b700 1 --2- 172.21.15.2:0/1882955883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed2c1083b0 0x7fed2c1087c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:37.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.575+0000 7fed3085b700 1 -- 172.21.15.2:0/1882955883 >> 172.21.15.2:0/1882955883 conn(0x7fed2c104040 msgr2=0x7fed2c106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:37.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.575+0000 7fed3085b700 1 -- 172.21.15.2:0/1882955883 shutdown_connections 2024-05-14T09:22:37.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.575+0000 7fed3085b700 1 -- 172.21.15.2:0/1882955883 wait complete. 2024-05-14T09:22:37.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.576+0000 7fed3085b700 1 Processor -- start 2024-05-14T09:22:37.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.576+0000 7fed3085b700 1 -- start start 2024-05-14T09:22:37.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.576+0000 7fed3085b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed2c1083b0 0x7fed2c19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:37.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.576+0000 7fed3085b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c109230 con 0x7fed2c1083b0 2024-05-14T09:22:37.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.576+0000 7fed2a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed2c1083b0 0x7fed2c19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:37.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.576+0000 7fed2a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed2c1083b0 0x7fed2c19faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53914/0 (socket says 172.21.15.2:53914) 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.576+0000 7fed2a59c700 1 -- 172.21.15.2:0/3397912150 learned_addr learned my addr 172.21.15.2:0/3397912150 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.577+0000 7fed2a59c700 1 -- 172.21.15.2:0/3397912150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c1a0030 con 0x7fed2c1083b0 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.577+0000 7fed2a59c700 1 --2- 172.21.15.2:0/3397912150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed2c1083b0 0x7fed2c19faf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fed1c000c00 tx=0x7fed1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.577+0000 7fed237fe700 1 -- 172.21.15.2:0/3397912150 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed1c003db0 con 0x7fed2c1083b0 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.577+0000 7fed237fe700 1 -- 172.21.15.2:0/3397912150 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1c011630 con 0x7fed2c1083b0 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.577+0000 7fed237fe700 1 -- 172.21.15.2:0/3397912150 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed1c011950 con 0x7fed2c1083b0 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.577+0000 7fed3085b700 1 -- 172.21.15.2:0/3397912150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c1a02c0 con 0x7fed2c1083b0 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.577+0000 7fed3085b700 1 -- 172.21.15.2:0/3397912150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c1a2ed0 con 0x7fed2c1083b0 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.577+0000 7fed237fe700 1 -- 172.21.15.2:0/3397912150 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fed1c01a460 con 0x7fed2c1083b0 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.577+0000 7fed237fe700 1 -- 172.21.15.2:0/3397912150 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fed1c01b780 con 0x7fed2c1083b0 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.578+0000 7fed3085b700 1 -- 172.21.15.2:0/3397912150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2c064e20 con 0x7fed2c1083b0 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.581+0000 7fed237fe700 1 -- 172.21.15.2:0/3397912150 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fed1c031020 con 0x7fed2c1083b0 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.651+0000 7fed3085b700 1 -- 172.21.15.2:0/3397912150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fed2c077510 con 0x7fed2c1083b0 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.655+0000 7fed237fe700 1 -- 172.21.15.2:0/3397912150 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+463 (secure 0 0 0) 0x7fed1c01b9d0 con 0x7fed2c1083b0 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.656+0000 7fed3085b700 1 -- 172.21.15.2:0/3397912150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed2c1083b0 msgr2=0x7fed2c19faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:37.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.656+0000 7fed3085b700 1 --2- 172.21.15.2:0/3397912150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed2c1083b0 0x7fed2c19faf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fed1c000c00 tx=0x7fed1c004750 comp rx=0 tx=0).stop 2024-05-14T09:22:37.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.656+0000 7fed3085b700 1 -- 172.21.15.2:0/3397912150 shutdown_connections 2024-05-14T09:22:37.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.656+0000 7fed3085b700 1 --2- 172.21.15.2:0/3397912150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed2c1083b0 0x7fed2c19faf0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:37.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.656+0000 7fed3085b700 1 -- 172.21.15.2:0/3397912150 >> 172.21.15.2:0/3397912150 conn(0x7fed2c104040 msgr2=0x7fed2c195dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:37.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.656+0000 7fed3085b700 1 -- 172.21.15.2:0/3397912150 shutdown_connections 2024-05-14T09:22:37.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:36.656+0000 7fed3085b700 1 -- 172.21.15.2:0/3397912150 wait complete. 2024-05-14T09:22:37.237 INFO:teuthology.orchestra.run.smithi002.stdout:Generating new minimal ceph.conf... 2024-05-14T09:22:38.288 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:37 smithi002 ceph-mon[31184]: from='client.? 172.21.15.2:0/1755013384' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:22:38.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.624+0000 7f9cd3c36700 1 Processor -- start 2024-05-14T09:22:38.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.624+0000 7f9cd3c36700 1 -- start start 2024-05-14T09:22:38.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.624+0000 7f9cd3c36700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ccc07f6d0 0x7f9ccc07db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:38.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.624+0000 7f9cd3c36700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc07e0d0 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.624+0000 7f9cd19d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ccc07f6d0 0x7f9ccc07db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:38.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.624+0000 7f9cd19d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ccc07f6d0 0x7f9ccc07db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53930/0 (socket says 172.21.15.2:53930) 2024-05-14T09:22:38.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.624+0000 7f9cd19d2700 1 -- 172.21.15.2:0/1653529731 learned_addr learned my addr 172.21.15.2:0/1653529731 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:38.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.625+0000 7f9cd19d2700 1 -- 172.21.15.2:0/1653529731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc07e250 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.625+0000 7f9cd19d2700 1 --2- 172.21.15.2:0/1653529731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ccc07f6d0 0x7f9ccc07db00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9cc80098a0 tx=0x7f9cc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428f0df7f180fae9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:38.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.625+0000 7f9cd09d0700 1 -- 172.21.15.2:0/1653529731 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cc8004160 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.625+0000 7f9cd09d0700 1 -- 172.21.15.2:0/1653529731 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9cc8004300 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.625+0000 7f9cd09d0700 1 -- 172.21.15.2:0/1653529731 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cc8011af0 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.626+0000 7f9cd3c36700 1 -- 172.21.15.2:0/1653529731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ccc07f6d0 msgr2=0x7f9ccc07db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:38.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.626+0000 7f9cd3c36700 1 --2- 172.21.15.2:0/1653529731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ccc07f6d0 0x7f9ccc07db00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9cc80098a0 tx=0x7f9cc8009bb0 comp rx=0 tx=0).stop 2024-05-14T09:22:38.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.626+0000 7f9cd3c36700 1 -- 172.21.15.2:0/1653529731 shutdown_connections 2024-05-14T09:22:38.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.626+0000 7f9cd3c36700 1 --2- 172.21.15.2:0/1653529731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ccc07f6d0 0x7f9ccc07db00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:38.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.626+0000 7f9cd3c36700 1 -- 172.21.15.2:0/1653529731 >> 172.21.15.2:0/1653529731 conn(0x7f9ccc1050b0 msgr2=0x7f9ccc107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:38.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.626+0000 7f9cd3c36700 1 -- 172.21.15.2:0/1653529731 shutdown_connections 2024-05-14T09:22:38.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.626+0000 7f9cd3c36700 1 -- 172.21.15.2:0/1653529731 wait complete. 2024-05-14T09:22:38.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.627+0000 7f9cd3c36700 1 Processor -- start 2024-05-14T09:22:38.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.627+0000 7f9cd3c36700 1 -- start start 2024-05-14T09:22:38.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.627+0000 7f9cd3c36700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ccc07f6d0 0x7f9ccc1a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:38.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.627+0000 7f9cd3c36700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc110020 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.627+0000 7f9cd19d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ccc07f6d0 0x7f9ccc1a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:38.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.627+0000 7f9cd19d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ccc07f6d0 0x7f9ccc1a0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53938/0 (socket says 172.21.15.2:53938) 2024-05-14T09:22:38.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.627+0000 7f9cd19d2700 1 -- 172.21.15.2:0/1755013384 learned_addr learned my addr 172.21.15.2:0/1755013384 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:38.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.627+0000 7f9cd19d2700 1 -- 172.21.15.2:0/1755013384 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc1a05a0 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.627+0000 7f9cd19d2700 1 --2- 172.21.15.2:0/1755013384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ccc07f6d0 0x7f9ccc1a0060 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009fb0 tx=0x7f9cc8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:38.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.628+0000 7f9cc2ffd700 1 -- 172.21.15.2:0/1755013384 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cc80036a0 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.628+0000 7f9cc2ffd700 1 -- 172.21.15.2:0/1755013384 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9cc801a430 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.628+0000 7f9cd3c36700 1 -- 172.21.15.2:0/1755013384 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ccc1a0830 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.628+0000 7f9cc2ffd700 1 -- 172.21.15.2:0/1755013384 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cc8019550 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.628+0000 7f9cd3c36700 1 -- 172.21.15.2:0/1755013384 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ccc1a3440 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.628+0000 7f9cc2ffd700 1 -- 172.21.15.2:0/1755013384 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9cc8019b90 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.628+0000 7f9cc2ffd700 1 -- 172.21.15.2:0/1755013384 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9cc8024730 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.629+0000 7f9cd3c36700 1 -- 172.21.15.2:0/1755013384 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ccc064e20 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.631+0000 7f9cc2ffd700 1 -- 172.21.15.2:0/1755013384 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f9cc8043ec0 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.701+0000 7f9cd3c36700 1 -- 172.21.15.2:0/1755013384 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9ccc1a3610 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.702+0000 7f9cc2ffd700 1 -- 172.21.15.2:0/1755013384 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7f9cc8032020 con 0x7f9ccc07f6d0 2024-05-14T09:22:38.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.702+0000 7f9cd3c36700 1 -- 172.21.15.2:0/1755013384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ccc07f6d0 msgr2=0x7f9ccc1a0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:38.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.702+0000 7f9cd3c36700 1 --2- 172.21.15.2:0/1755013384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ccc07f6d0 0x7f9ccc1a0060 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009fb0 tx=0x7f9cc8011940 comp rx=0 tx=0).stop 2024-05-14T09:22:38.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.702+0000 7f9cd3c36700 1 -- 172.21.15.2:0/1755013384 shutdown_connections 2024-05-14T09:22:38.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.702+0000 7f9cd3c36700 1 --2- 172.21.15.2:0/1755013384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ccc07f6d0 0x7f9ccc1a0060 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:38.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.702+0000 7f9cd3c36700 1 -- 172.21.15.2:0/1755013384 >> 172.21.15.2:0/1755013384 conn(0x7f9ccc1050b0 msgr2=0x7f9ccc1059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:38.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.703+0000 7f9cd3c36700 1 -- 172.21.15.2:0/1755013384 shutdown_connections 2024-05-14T09:22:38.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:37.703+0000 7f9cd3c36700 1 -- 172.21.15.2:0/1755013384 wait complete. 2024-05-14T09:22:38.298 INFO:teuthology.orchestra.run.smithi002.stdout:Restarting the monitor... 2024-05-14T09:22:38.560 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:38 smithi002 systemd[1]: Stopping Ceph mon.a for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T09:22:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:38 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a[31180]: 2024-05-14T09:22:38.558+0000 7f67fd6d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T09:22:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:38 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a[31180]: 2024-05-14T09:22:38.558+0000 7f67fd6d2700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-14T09:22:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:38 smithi002 podman[31432]: 2024-05-14 09:22:38.713552561 +0000 UTC m=+0.247874923 container died 25946a5344925456c7ff9f079196b30d82eb6f590343eeb23ea45f224c569448 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-14T09:22:39.242 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 podman[31432]: 2024-05-14 09:22:39.058614838 +0000 UTC m=+0.592937197 container cleanup 25946a5344925456c7ff9f079196b30d82eb6f590343eeb23ea45f224c569448 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T09:22:39.242 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 bash[31432]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a 2024-05-14T09:22:39.558 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 podman[31446]: 2024-05-14 09:22:39.242947617 +0000 UTC m=+0.523856311 container remove 25946a5344925456c7ff9f079196b30d82eb6f590343eeb23ea45f224c569448 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-14T09:22:39.558 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 systemd[1]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-14T09:22:39.558 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 systemd[1]: Stopped Ceph mon.a for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:22:39.558 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 systemd[1]: Starting Ceph mon.a for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T09:22:39.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 podman[31538]: 2024-05-14 09:22:39.693700714 +0000 UTC m=+0.101599217 container create 8434b6d71ae2c7dbb71fc1bfc2e5e96eb681516c5a81898881b8009d71a7e18d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2) 2024-05-14T09:22:39.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 podman[31538]: 2024-05-14 09:22:39.609913193 +0000 UTC m=+0.017811698 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T09:22:40.008 INFO:teuthology.orchestra.run.smithi002.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-14T09:22:40.130 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 podman[31538]: 2024-05-14 09:22:39.921242111 +0000 UTC m=+0.329140607 container init 8434b6d71ae2c7dbb71fc1bfc2e5e96eb681516c5a81898881b8009d71a7e18d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-14T09:22:40.130 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 podman[31538]: 2024-05-14 09:22:39.924444031 +0000 UTC m=+0.332342536 container start 8434b6d71ae2c7dbb71fc1bfc2e5e96eb681516c5a81898881b8009d71a7e18d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-14T09:22:40.130 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: set uid:gid to 167:167 (ceph:ceph) 2024-05-14T09:22:40.130 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev), process ceph-mon, pid 2 2024-05-14T09:22:40.130 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: pidfile_write: ignore empty --pid-file 2024-05-14T09:22:40.130 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: load: jerasure load: lrc 2024-05-14T09:22:40.130 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: RocksDB version: 7.9.2 2024-05-14T09:22:40.130 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Git sha 0 2024-05-14T09:22:40.131 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Compile date 2024-05-09 05:50:40 2024-05-14T09:22:40.131 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: DB SUMMARY 2024-05-14T09:22:40.131 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: DB Session ID: 943OH3DFSTJC2K2HEXUB 2024-05-14T09:22:40.131 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: CURRENT file: CURRENT 2024-05-14T09:22:40.131 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: IDENTITY file: IDENTITY 2024-05-14T09:22:40.131 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-14T09:22:40.131 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-14T09:22:40.131 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104060 ; 2024-05-14T09:22:40.131 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.error_if_exists: 0 2024-05-14T09:22:40.131 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.create_if_missing: 0 2024-05-14T09:22:40.131 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.paranoid_checks: 1 2024-05-14T09:22:40.131 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-14T09:22:40.131 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-14T09:22:40.132 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-14T09:22:40.132 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.env: 0x556a3f63f500 2024-05-14T09:22:40.132 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.fs: PosixFileSystem 2024-05-14T09:22:40.132 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.info_log: 0x556a40a84020 2024-05-14T09:22:40.132 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_file_opening_threads: 16 2024-05-14T09:22:40.133 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.statistics: (nil) 2024-05-14T09:22:40.133 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.use_fsync: 0 2024-05-14T09:22:40.133 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_log_file_size: 0 2024-05-14T09:22:40.133 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-14T09:22:40.133 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-14T09:22:40.133 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.keep_log_file_num: 1000 2024-05-14T09:22:40.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.recycle_log_file_num: 0 2024-05-14T09:22:40.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.allow_fallocate: 1 2024-05-14T09:22:40.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.allow_mmap_reads: 0 2024-05-14T09:22:40.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.allow_mmap_writes: 0 2024-05-14T09:22:40.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.use_direct_reads: 0 2024-05-14T09:22:40.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-14T09:22:40.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.create_missing_column_families: 0 2024-05-14T09:22:40.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.db_log_dir: 2024-05-14T09:22:40.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.wal_dir: 2024-05-14T09:22:40.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-14T09:22:40.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-14T09:22:40.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-14T09:22:40.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-14T09:22:40.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-14T09:22:40.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-14T09:22:40.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.advise_random_on_open: 1 2024-05-14T09:22:40.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.db_write_buffer_size: 0 2024-05-14T09:22:40.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.write_buffer_manager: 0x556a4094e8c0 2024-05-14T09:22:40.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-14T09:22:40.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-14T09:22:40.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-14T09:22:40.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.rate_limiter: (nil) 2024-05-14T09:22:40.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-14T09:22:40.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.wal_recovery_mode: 2 2024-05-14T09:22:40.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.enable_thread_tracking: 0 2024-05-14T09:22:40.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.enable_pipelined_write: 0 2024-05-14T09:22:40.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.unordered_write: 0 2024-05-14T09:22:40.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-14T09:22:40.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-14T09:22:40.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-14T09:22:40.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-14T09:22:40.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.row_cache: None 2024-05-14T09:22:40.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.wal_filter: None 2024-05-14T09:22:40.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-14T09:22:40.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.allow_ingest_behind: 0 2024-05-14T09:22:40.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.two_write_queues: 0 2024-05-14T09:22:40.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.manual_wal_flush: 0 2024-05-14T09:22:40.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.wal_compression: 0 2024-05-14T09:22:40.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.atomic_flush: 0 2024-05-14T09:22:40.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-14T09:22:40.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-14T09:22:40.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-14T09:22:40.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.log_readahead_size: 0 2024-05-14T09:22:40.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.best_efforts_recovery: 0 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.allow_data_in_errors: 0 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.db_host_id: __hostname__ 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_background_jobs: 2 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_background_compactions: -1 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_subcompactions: 1 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_total_wal_size: 0 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-14T09:22:40.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_open_files: -1 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.bytes_per_sync: 0 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compaction_readahead_size: 0 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_background_flushes: -1 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Compression algorithms supported: 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: kZSTD supported: 0 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: kXpressCompression supported: 0 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: kLZ4HCCompression supported: 1 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: kZlibCompression supported: 1 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: kSnappyCompression supported: 1 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: kLZ4Compression supported: 1 2024-05-14T09:22:40.138 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: kBZip2Compression supported: 0 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.merge_operator: 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compaction_filter: None 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compaction_filter_factory: None 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.sst_partitioner_factory: None 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556a40a841e0) 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout: cache_index_and_filter_blocks: 1 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-14T09:22:40.139 INFO:journalctl@ceph.mon.a.smithi002.stdout: pin_top_level_index_and_filter: 1 2024-05-14T09:22:40.140 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_type: 0 2024-05-14T09:22:40.140 INFO:journalctl@ceph.mon.a.smithi002.stdout: data_block_index_type: 0 2024-05-14T09:22:40.140 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_shortening: 1 2024-05-14T09:22:40.140 INFO:journalctl@ceph.mon.a.smithi002.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-14T09:22:40.140 INFO:journalctl@ceph.mon.a.smithi002.stdout: checksum: 4 2024-05-14T09:22:40.140 INFO:journalctl@ceph.mon.a.smithi002.stdout: no_block_cache: 0 2024-05-14T09:22:40.140 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache: 0x556a409811f0 2024-05-14T09:22:40.140 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_name: BinnedLRUCache 2024-05-14T09:22:40.140 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_options: 2024-05-14T09:22:40.140 INFO:journalctl@ceph.mon.a.smithi002.stdout: capacity : 536870912 2024-05-14T09:22:40.140 INFO:journalctl@ceph.mon.a.smithi002.stdout: num_shard_bits : 4 2024-05-14T09:22:40.140 INFO:journalctl@ceph.mon.a.smithi002.stdout: strict_capacity_limit : 0 2024-05-14T09:22:40.140 INFO:journalctl@ceph.mon.a.smithi002.stdout: high_pri_pool_ratio: 0.000 2024-05-14T09:22:40.140 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_compressed: (nil) 2024-05-14T09:22:40.140 INFO:journalctl@ceph.mon.a.smithi002.stdout: persistent_cache: (nil) 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_size: 4096 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_size_deviation: 10 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_restart_interval: 16 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_block_restart_interval: 1 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: metadata_block_size: 4096 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: partition_filters: 0 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: use_delta_encoding: 1 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: filter_policy: bloomfilter 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: whole_key_filtering: 1 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: verify_compression: 0 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: read_amp_bytes_per_bit: 0 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: format_version: 5 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: enable_index_compression: 1 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_align: 0 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: max_auto_readahead_size: 262144 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: prepopulate_block_cache: 0 2024-05-14T09:22:40.141 INFO:journalctl@ceph.mon.a.smithi002.stdout: initial_auto_readahead_size: 8192 2024-05-14T09:22:40.142 INFO:journalctl@ceph.mon.a.smithi002.stdout: num_file_reads_for_auto_readahead: 2 2024-05-14T09:22:40.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.write_buffer_size: 33554432 2024-05-14T09:22:40.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_write_buffer_number: 2 2024-05-14T09:22:40.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compression: NoCompression 2024-05-14T09:22:40.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.bottommost_compression: Disabled 2024-05-14T09:22:40.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.prefix_extractor: nullptr 2024-05-14T09:22:40.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-14T09:22:40.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.num_levels: 7 2024-05-14T09:22:40.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-14T09:22:40.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-14T09:22:40.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-14T09:22:40.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-14T09:22:40.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-14T09:22:40.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-14T09:22:40.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compression_opts.level: 32767 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compression_opts.strategy: 0 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compression_opts.enabled: false 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-14T09:22:40.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.target_file_size_base: 67108864 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-14T09:22:40.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-14T09:22:40.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.arena_block_size: 1048576 2024-05-14T09:22:40.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-14T09:22:40.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-14T09:22:40.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.disable_auto_compactions: 0 2024-05-14T09:22:40.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-14T09:22:40.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-14T09:22:40.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-14T09:22:40.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-14T09:22:40.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-14T09:22:40.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-14T09:22:40.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-14T09:22:40.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-14T09:22:40.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-14T09:22:40.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-14T09:22:40.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-14T09:22:40.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.inplace_update_support: 0 2024-05-14T09:22:40.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-14T09:22:40.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-14T09:22:40.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-14T09:22:40.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-14T09:22:40.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.bloom_locality: 0 2024-05-14T09:22:40.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.max_successive_merges: 0 2024-05-14T09:22:40.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-14T09:22:40.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.paranoid_file_checks: 0 2024-05-14T09:22:40.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.force_consistency_checks: 1 2024-05-14T09:22:40.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.report_bg_io_stats: 0 2024-05-14T09:22:40.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.ttl: 2592000 2024-05-14T09:22:40.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-14T09:22:40.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-14T09:22:40.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-14T09:22:40.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.enable_blob_files: false 2024-05-14T09:22:40.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.min_blob_size: 0 2024-05-14T09:22:40.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.blob_file_size: 268435456 2024-05-14T09:22:40.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-14T09:22:40.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-14T09:22:40.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-14T09:22:40.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-14T09:22:40.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-14T09:22:40.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.blob_file_starting_level: 0 2024-05-14T09:22:40.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-14T09:22:40.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-14T09:22:40.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-14T09:22:40.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 449e9f84-d656-4d86-a81c-fbf6588f12e6 2024-05-14T09:22:40.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715678559967481, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-14T09:22:40.148 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-14T09:22:40.148 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715678559968602, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99288, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97234, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13917, "raw_average_key_size": 49, "raw_value_size": 89628, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715678559, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "449e9f84-d656-4d86-a81c-fbf6588f12e6", "db_session_id": "943OH3DFSTJC2K2HEXUB", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-14T09:22:40.148 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715678559968687, "job": 1, "event": "recovery_finished"} 2024-05-14T09:22:40.148 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-14T09:22:40.148 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-14T09:22:40.148 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556a40965c00 2024-05-14T09:22:40.148 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: DB pointer 0x556a45758000 2024-05-14T09:22:40.148 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-14T09:22:40.148 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-14T09:22:40.148 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** DB Stats ** 2024-05-14T09:22:40.148 INFO:journalctl@ceph.mon.a.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T09:22:40.148 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-14T09:22:40.148 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T09:22:40.148 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** Compaction Stats [default] ** 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: L0 2/0 98.82 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: Sum 2/0 98.82 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** Compaction Stats [default] ** 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 153.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T09:22:40.149 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-05-14T09:22:40.150 INFO:journalctl@ceph.mon.a.smithi002.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-14T09:22:40.150 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-05-14T09:22:40.150 INFO:journalctl@ceph.mon.a.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T09:22:40.150 INFO:journalctl@ceph.mon.a.smithi002.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-14T09:22:40.150 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-14T09:22:40.150 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-14T09:22:40.150 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-14T09:22:40.150 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-14T09:22:40.150 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative compaction: 0.00 GB write, 28.09 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T09:22:40.150 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval compaction: 0.00 GB write, 28.09 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T09:22:40.150 INFO:journalctl@ceph.mon.a.smithi002.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-14T09:22:40.150 INFO:journalctl@ceph.mon.a.smithi002.stdout: Block cache BinnedLRUCache@0x556a409811f0#2 capacity: 512.00 MB usage: 1.41 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-14T09:22:40.150 INFO:journalctl@ceph.mon.a.smithi002.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.91 KB,0.000172853%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-14T09:22:40.150 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-05-14T09:22:40.150 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: starting mon.a rank 0 at public addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] at bind addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: mon.a@-1(???) e1 preinit fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: mon.a@-1(???).mds e1 new map 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: mon.a@-1(???).mds e1 print_map 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout: e1 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout: legacy client fscid: -1 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout: No filesystems configured 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:39 smithi002 ceph-mon[31557]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-14T09:22:40.151 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:40 smithi002 bash[31538]: 8434b6d71ae2c7dbb71fc1bfc2e5e96eb681516c5a81898881b8009d71a7e18d 2024-05-14T09:22:40.152 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:40 smithi002 systemd[1]: Started Ceph mon.a for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:22:40.152 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:40 smithi002 ceph-mon[31557]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-14T09:22:40.152 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:40 smithi002 ceph-mon[31557]: monmap epoch 1 2024-05-14T09:22:40.152 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:40 smithi002 ceph-mon[31557]: fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:22:40.152 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:40 smithi002 ceph-mon[31557]: last_changed 2024-05-14T09:22:31.786962+0000 2024-05-14T09:22:40.152 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:40 smithi002 ceph-mon[31557]: created 2024-05-14T09:22:31.786962+0000 2024-05-14T09:22:40.152 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:40 smithi002 ceph-mon[31557]: min_mon_release 19 (squid) 2024-05-14T09:22:40.152 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:40 smithi002 ceph-mon[31557]: election_strategy: 1 2024-05-14T09:22:40.152 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:40 smithi002 ceph-mon[31557]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T09:22:40.152 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:40 smithi002 ceph-mon[31557]: fsmap 2024-05-14T09:22:40.152 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:40 smithi002 ceph-mon[31557]: osdmap e1: 0 total, 0 up, 0 in 2024-05-14T09:22:40.152 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:40 smithi002 ceph-mon[31557]: mgrmap e1: no daemons active 2024-05-14T09:22:41.143 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.524+0000 7fa480063700 1 Processor -- start 2024-05-14T09:22:41.143 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.524+0000 7fa480063700 1 -- start start 2024-05-14T09:22:41.143 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.525+0000 7fa480063700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa478109a10 0x7fa478109e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:41.143 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.525+0000 7fa480063700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa47810a410 con 0x7fa478109a10 2024-05-14T09:22:41.143 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.525+0000 7fa47ddff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa478109a10 0x7fa478109e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:41.143 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.525+0000 7fa47ddff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa478109a10 0x7fa478109e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56610/0 (socket says 172.21.15.2:56610) 2024-05-14T09:22:41.143 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.525+0000 7fa47ddff700 1 -- 172.21.15.2:0/2014544780 learned_addr learned my addr 172.21.15.2:0/2014544780 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:41.143 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.526+0000 7fa47ddff700 1 -- 172.21.15.2:0/2014544780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47810abe0 con 0x7fa478109a10 2024-05-14T09:22:41.144 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.526+0000 7fa47ddff700 1 --2- 172.21.15.2:0/2014544780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa478109a10 0x7fa478109e20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa4740098a0 tx=0x7fa474009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204484bee3bc7ef6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:41.144 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.526+0000 7fa47cdfd700 1 -- 172.21.15.2:0/2014544780 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa474004160 con 0x7fa478109a10 2024-05-14T09:22:41.144 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.526+0000 7fa47cdfd700 1 -- 172.21.15.2:0/2014544780 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa474004300 con 0x7fa478109a10 2024-05-14T09:22:41.144 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.526+0000 7fa47cdfd700 1 -- 172.21.15.2:0/2014544780 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa474011af0 con 0x7fa478109a10 2024-05-14T09:22:41.144 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.527+0000 7fa480063700 1 -- 172.21.15.2:0/2014544780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa478109a10 msgr2=0x7fa478109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:41.144 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.527+0000 7fa480063700 1 --2- 172.21.15.2:0/2014544780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa478109a10 0x7fa478109e20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa4740098a0 tx=0x7fa474009bb0 comp rx=0 tx=0).stop 2024-05-14T09:22:41.144 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.527+0000 7fa480063700 1 -- 172.21.15.2:0/2014544780 shutdown_connections 2024-05-14T09:22:41.144 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.527+0000 7fa480063700 1 --2- 172.21.15.2:0/2014544780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa478109a10 0x7fa478109e20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:41.144 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.527+0000 7fa480063700 1 -- 172.21.15.2:0/2014544780 >> 172.21.15.2:0/2014544780 conn(0x7fa4781050b0 msgr2=0x7fa478107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:41.144 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.527+0000 7fa480063700 1 -- 172.21.15.2:0/2014544780 shutdown_connections 2024-05-14T09:22:41.144 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.527+0000 7fa480063700 1 -- 172.21.15.2:0/2014544780 wait complete. 2024-05-14T09:22:41.144 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.528+0000 7fa480063700 1 Processor -- start 2024-05-14T09:22:41.144 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.528+0000 7fa480063700 1 -- start start 2024-05-14T09:22:41.144 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.528+0000 7fa480063700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa478109a10 0x7fa47819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:41.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.528+0000 7fa480063700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa47810df90 con 0x7fa478109a10 2024-05-14T09:22:41.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.528+0000 7fa47ddff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa478109a10 0x7fa47819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:41.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.528+0000 7fa47ddff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa478109a10 0x7fa47819de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56624/0 (socket says 172.21.15.2:56624) 2024-05-14T09:22:41.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.528+0000 7fa47ddff700 1 -- 172.21.15.2:0/2221765964 learned_addr learned my addr 172.21.15.2:0/2221765964 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:41.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.528+0000 7fa47ddff700 1 -- 172.21.15.2:0/2221765964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47819e3d0 con 0x7fa478109a10 2024-05-14T09:22:41.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.529+0000 7fa47ddff700 1 --2- 172.21.15.2:0/2221765964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa478109a10 0x7fa47819de90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa474009fb0 tx=0x7fa474011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:41.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.529+0000 7fa46effd700 1 -- 172.21.15.2:0/2221765964 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4740036a0 con 0x7fa478109a10 2024-05-14T09:22:41.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.529+0000 7fa480063700 1 -- 172.21.15.2:0/2221765964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47819e660 con 0x7fa478109a10 2024-05-14T09:22:41.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.529+0000 7fa46effd700 1 -- 172.21.15.2:0/2221765964 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa474003cd0 con 0x7fa478109a10 2024-05-14T09:22:41.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.529+0000 7fa46effd700 1 -- 172.21.15.2:0/2221765964 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa47401a430 con 0x7fa478109a10 2024-05-14T09:22:41.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.529+0000 7fa480063700 1 -- 172.21.15.2:0/2221765964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4781a1270 con 0x7fa478109a10 2024-05-14T09:22:41.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.529+0000 7fa46effd700 1 -- 172.21.15.2:0/2221765964 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa47401aa80 con 0x7fa478109a10 2024-05-14T09:22:41.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.529+0000 7fa46effd700 1 -- 172.21.15.2:0/2221765964 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa4740193c0 con 0x7fa478109a10 2024-05-14T09:22:41.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.530+0000 7fa480063700 1 -- 172.21.15.2:0/2221765964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa478064e20 con 0x7fa478109a10 2024-05-14T09:22:41.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.533+0000 7fa46effd700 1 -- 172.21.15.2:0/2221765964 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa474024440 con 0x7fa478109a10 2024-05-14T09:22:41.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.603+0000 7fa480063700 1 -- 172.21.15.2:0/2221765964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fa4781a1580 con 0x7fa478109a10 2024-05-14T09:22:41.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.605+0000 7fa46effd700 1 -- 172.21.15.2:0/2221765964 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fa474024760 con 0x7fa478109a10 2024-05-14T09:22:41.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.606+0000 7fa480063700 1 -- 172.21.15.2:0/2221765964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa478109a10 msgr2=0x7fa47819de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:41.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.606+0000 7fa480063700 1 --2- 172.21.15.2:0/2221765964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa478109a10 0x7fa47819de90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa474009fb0 tx=0x7fa474011890 comp rx=0 tx=0).stop 2024-05-14T09:22:41.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.606+0000 7fa480063700 1 -- 172.21.15.2:0/2221765964 shutdown_connections 2024-05-14T09:22:41.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.606+0000 7fa480063700 1 --2- 172.21.15.2:0/2221765964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa478109a10 0x7fa47819de90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:41.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.606+0000 7fa480063700 1 -- 172.21.15.2:0/2221765964 >> 172.21.15.2:0/2221765964 conn(0x7fa4781050b0 msgr2=0x7fa478105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:41.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.606+0000 7fa480063700 1 -- 172.21.15.2:0/2221765964 shutdown_connections 2024-05-14T09:22:41.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:40.606+0000 7fa480063700 1 -- 172.21.15.2:0/2221765964 wait complete. 2024-05-14T09:22:41.146 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-14T09:22:41.160 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-14T09:22:41.160 INFO:teuthology.orchestra.run.smithi002.stdout:Creating mgr... 2024-05-14T09:22:41.161 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-14T09:22:41.161 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-14T09:22:41.411 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mgr.x 2024-05-14T09:22:41.411 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to reset failed state of unit ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-14T09:22:41.562 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-50d5aebe-11d3-11ef-bc98-c7b262605968.target.wants/ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@.service. 2024-05-14T09:22:41.909 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:41 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2221765964' entity='client.admin' 2024-05-14T09:22:42.082 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T09:22:42.082 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-05-14T09:22:42.090 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T09:22:42.090 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-05-14T09:22:42.090 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-05-14T09:22:42.090 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to enable service . firewalld.service is not available 2024-05-14T09:22:42.100 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T09:22:42.100 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-05-14T09:22:42.108 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T09:22:42.109 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-05-14T09:22:42.109 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-05-14T09:22:42.109 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-14T09:22:42.109 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr to start... 2024-05-14T09:22:42.109 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr... 2024-05-14T09:22:42.171 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:42 smithi002 systemd[1]: Started Ceph mgr.x for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:22:42.460 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:42 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:42.180+0000 7f22a67da200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T09:22:42.460 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:42 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:42.378+0000 7f22a67da200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T09:22:42.713 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:42 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:42.486+0000 7f22a67da200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T09:22:43.022 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:42 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/376230051' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T09:22:43.022 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:42 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:42.833+0000 7f22a67da200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T09:22:43.022 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:42 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:42.923+0000 7f22a67da200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T09:22:43.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T09:22:43.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-05-14T09:22:43.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "50d5aebe-11d3-11ef-bc98-c7b262605968", 2024-05-14T09:22:43.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T09:22:43.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T09:22:43.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T09:22:43.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T09:22:43.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:43.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T09:22:43.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T09:22:43.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-05-14T09:22:43.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T09:22:43.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T09:22:43.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T09:22:43.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T09:22:43.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-14T09:22:43.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T09:22:43.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:43.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T09:22:43.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T09:22:43.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:43.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T09:22:43.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:43.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T09:22:43.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T09:22:43.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T09:22:43.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T09:22:43.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T09:22:43.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T09:22:43.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:43.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T09:22:43.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-14T09:22:43.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T09:22:43.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T09:22:43.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T09:22:43.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T09:22:43.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T09:22:43.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T09:22:43.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T09:22:43.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:43.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T09:22:43.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:43.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T09:22:34.937680+0000", 2024-05-14T09:22:43.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T09:22:43.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:43.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T09:22:43.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-05-14T09:22:43.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.574+0000 7f6addc15700 1 Processor -- start 2024-05-14T09:22:43.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.574+0000 7f6addc15700 1 -- start start 2024-05-14T09:22:43.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.574+0000 7f6addc15700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ad810a5c0 0x7f6ad810c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:43.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.574+0000 7f6addc15700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad807ce10 con 0x7f6ad810a5c0 2024-05-14T09:22:43.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.575+0000 7f6ad77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ad810a5c0 0x7f6ad810c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:43.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.575+0000 7f6ad77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ad810a5c0 0x7f6ad810c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56644/0 (socket says 172.21.15.2:56644) 2024-05-14T09:22:43.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.575+0000 7f6ad77fe700 1 -- 172.21.15.2:0/1252798143 learned_addr learned my addr 172.21.15.2:0/1252798143 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:43.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.575+0000 7f6ad77fe700 1 -- 172.21.15.2:0/1252798143 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad810cf20 con 0x7f6ad810a5c0 2024-05-14T09:22:43.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.575+0000 7f6ad77fe700 1 --2- 172.21.15.2:0/1252798143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ad810a5c0 0x7f6ad810c9e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6ac00098a0 tx=0x7f6ac0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b810eebddf1306eb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:43.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.575+0000 7f6ad67fc700 1 -- 172.21.15.2:0/1252798143 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac0004160 con 0x7f6ad810a5c0 2024-05-14T09:22:43.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.576+0000 7f6ad67fc700 1 -- 172.21.15.2:0/1252798143 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6ac0004300 con 0x7f6ad810a5c0 2024-05-14T09:22:43.298 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.576+0000 7f6ad67fc700 1 -- 172.21.15.2:0/1252798143 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac0011af0 con 0x7f6ad810a5c0 2024-05-14T09:22:43.299 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.576+0000 7f6addc15700 1 -- 172.21.15.2:0/1252798143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ad810a5c0 msgr2=0x7f6ad810c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:43.299 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.576+0000 7f6addc15700 1 --2- 172.21.15.2:0/1252798143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ad810a5c0 0x7f6ad810c9e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6ac00098a0 tx=0x7f6ac0009bb0 comp rx=0 tx=0).stop 2024-05-14T09:22:43.299 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.576+0000 7f6addc15700 1 -- 172.21.15.2:0/1252798143 shutdown_connections 2024-05-14T09:22:43.299 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.576+0000 7f6addc15700 1 --2- 172.21.15.2:0/1252798143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ad810a5c0 0x7f6ad810c9e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:43.299 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.576+0000 7f6addc15700 1 -- 172.21.15.2:0/1252798143 >> 172.21.15.2:0/1252798143 conn(0x7f6ad8103fe0 msgr2=0x7f6ad8106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:43.299 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.577+0000 7f6addc15700 1 -- 172.21.15.2:0/1252798143 shutdown_connections 2024-05-14T09:22:43.299 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.577+0000 7f6addc15700 1 -- 172.21.15.2:0/1252798143 wait complete. 2024-05-14T09:22:43.299 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.577+0000 7f6addc15700 1 Processor -- start 2024-05-14T09:22:43.299 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.577+0000 7f6addc15700 1 -- start start 2024-05-14T09:22:43.299 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.578+0000 7f6addc15700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ad810a5c0 0x7f6ad8109d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:43.299 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.578+0000 7f6addc15700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad810de80 con 0x7f6ad810a5c0 2024-05-14T09:22:43.299 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.578+0000 7f6ad77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ad810a5c0 0x7f6ad8109d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:43.299 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.578+0000 7f6ad77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ad810a5c0 0x7f6ad8109d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56648/0 (socket says 172.21.15.2:56648) 2024-05-14T09:22:43.299 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.578+0000 7f6ad77fe700 1 -- 172.21.15.2:0/376230051 learned_addr learned my addr 172.21.15.2:0/376230051 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:43.300 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.578+0000 7f6ad77fe700 1 -- 172.21.15.2:0/376230051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad810a280 con 0x7f6ad810a5c0 2024-05-14T09:22:43.300 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.578+0000 7f6ad77fe700 1 --2- 172.21.15.2:0/376230051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ad810a5c0 0x7f6ad8109d40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0009fb0 tx=0x7f6ac0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:43.300 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.579+0000 7f6ad4ff9700 1 -- 172.21.15.2:0/376230051 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac0011bc0 con 0x7f6ad810a5c0 2024-05-14T09:22:43.300 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.579+0000 7f6addc15700 1 -- 172.21.15.2:0/376230051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad81084d0 con 0x7f6ad810a5c0 2024-05-14T09:22:43.300 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.579+0000 7f6ad4ff9700 1 -- 172.21.15.2:0/376230051 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6ac0024440 con 0x7f6ad810a5c0 2024-05-14T09:22:43.300 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.579+0000 7f6ad4ff9700 1 -- 172.21.15.2:0/376230051 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac001a5c0 con 0x7f6ad810a5c0 2024-05-14T09:22:43.300 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.579+0000 7f6addc15700 1 -- 172.21.15.2:0/376230051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad81089b0 con 0x7f6ad810a5c0 2024-05-14T09:22:43.300 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.579+0000 7f6ad4ff9700 1 -- 172.21.15.2:0/376230051 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6ac001ac10 con 0x7f6ad810a5c0 2024-05-14T09:22:43.300 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.579+0000 7f6ad4ff9700 1 -- 172.21.15.2:0/376230051 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6ac0019700 con 0x7f6ad810a5c0 2024-05-14T09:22:43.300 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.580+0000 7f6addc15700 1 -- 172.21.15.2:0/376230051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad8064e20 con 0x7f6ad810a5c0 2024-05-14T09:22:43.300 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.584+0000 7f6ad4ff9700 1 -- 172.21.15.2:0/376230051 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6ac002ddf0 con 0x7f6ad810a5c0 2024-05-14T09:22:43.300 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.660+0000 7f6addc15700 1 -- 172.21.15.2:0/376230051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6ad8108b80 con 0x7f6ad810a5c0 2024-05-14T09:22:43.300 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.661+0000 7f6ad4ff9700 1 -- 172.21.15.2:0/376230051 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6ac00245f0 con 0x7f6ad810a5c0 2024-05-14T09:22:43.300 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.662+0000 7f6addc15700 1 -- 172.21.15.2:0/376230051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ad810a5c0 msgr2=0x7f6ad8109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:43.301 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.662+0000 7f6addc15700 1 --2- 172.21.15.2:0/376230051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ad810a5c0 0x7f6ad8109d40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0009fb0 tx=0x7f6ac0011890 comp rx=0 tx=0).stop 2024-05-14T09:22:43.301 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.662+0000 7f6addc15700 1 -- 172.21.15.2:0/376230051 shutdown_connections 2024-05-14T09:22:43.301 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.662+0000 7f6addc15700 1 --2- 172.21.15.2:0/376230051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ad810a5c0 0x7f6ad8109d40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:43.301 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.662+0000 7f6addc15700 1 -- 172.21.15.2:0/376230051 >> 172.21.15.2:0/376230051 conn(0x7f6ad8103fe0 msgr2=0x7f6ad8104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:43.301 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.663+0000 7f6addc15700 1 -- 172.21.15.2:0/376230051 shutdown_connections 2024-05-14T09:22:43.301 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:42.663+0000 7f6addc15700 1 -- 172.21.15.2:0/376230051 wait complete. 2024-05-14T09:22:43.301 INFO:teuthology.orchestra.run.smithi002.stdout:mgr not available, waiting (1/15)... 2024-05-14T09:22:44.521 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:44 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:44.196+0000 7f22a67da200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T09:22:44.521 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:44 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:44.374+0000 7f22a67da200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T09:22:44.855 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:44 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:44.554+0000 7f22a67da200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T09:22:44.856 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:44 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:44.854+0000 7f22a67da200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T09:22:45.135 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:44 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:44.952+0000 7f22a67da200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T09:22:45.413 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:45 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:45.168+0000 7f22a67da200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T09:22:45.413 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:45 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:45.373+0000 7f22a67da200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T09:22:46.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:45 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1590670641' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T09:22:46.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:46 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:46.026+0000 7f22a67da200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T09:22:46.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:46 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:46.265+0000 7f22a67da200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T09:22:46.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T09:22:46.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-05-14T09:22:46.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "50d5aebe-11d3-11ef-bc98-c7b262605968", 2024-05-14T09:22:46.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T09:22:46.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T09:22:46.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T09:22:46.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T09:22:46.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:46.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T09:22:46.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T09:22:46.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-05-14T09:22:46.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T09:22:46.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T09:22:46.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T09:22:46.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T09:22:46.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-14T09:22:46.435 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T09:22:46.435 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:46.435 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T09:22:46.435 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T09:22:46.435 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:46.435 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T09:22:46.435 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:46.435 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T09:22:46.435 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T09:22:46.435 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T09:22:46.442 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T09:22:46.442 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T09:22:46.442 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T09:22:46.442 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:46.442 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T09:22:46.442 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T09:22:46.442 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T09:22:46.442 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T09:22:46.443 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T09:22:34.937680+0000", 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.756+0000 7f564887d700 1 Processor -- start 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.756+0000 7f564887d700 1 -- start start 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.756+0000 7f564887d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f564410a540 0x7f564410c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:46.444 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.756+0000 7f564887d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f564407ce90 con 0x7f564410a540 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.756+0000 7f564259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f564410a540 0x7f564410c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.756+0000 7f564259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f564410a540 0x7f564410c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56662/0 (socket says 172.21.15.2:56662) 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.756+0000 7f564259c700 1 -- 172.21.15.2:0/1472667179 learned_addr learned my addr 172.21.15.2:0/1472667179 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.757+0000 7f564259c700 1 -- 172.21.15.2:0/1472667179 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564410cea0 con 0x7f564410a540 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.757+0000 7f564259c700 1 --2- 172.21.15.2:0/1472667179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f564410a540 0x7f564410c960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56340098a0 tx=0x7f5634009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3974cd3b64beb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.757+0000 7f564159a700 1 -- 172.21.15.2:0/1472667179 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5634004160 con 0x7f564410a540 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.757+0000 7f564159a700 1 -- 172.21.15.2:0/1472667179 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5634004300 con 0x7f564410a540 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.757+0000 7f564159a700 1 -- 172.21.15.2:0/1472667179 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5634011af0 con 0x7f564410a540 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.758+0000 7f564887d700 1 -- 172.21.15.2:0/1472667179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f564410a540 msgr2=0x7f564410c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.758+0000 7f564887d700 1 --2- 172.21.15.2:0/1472667179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f564410a540 0x7f564410c960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56340098a0 tx=0x7f5634009bb0 comp rx=0 tx=0).stop 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.758+0000 7f564887d700 1 -- 172.21.15.2:0/1472667179 shutdown_connections 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.758+0000 7f564887d700 1 --2- 172.21.15.2:0/1472667179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f564410a540 0x7f564410c960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.758+0000 7f564887d700 1 -- 172.21.15.2:0/1472667179 >> 172.21.15.2:0/1472667179 conn(0x7f5644103fe0 msgr2=0x7f5644106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.758+0000 7f564887d700 1 -- 172.21.15.2:0/1472667179 shutdown_connections 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.758+0000 7f564887d700 1 -- 172.21.15.2:0/1472667179 wait complete. 2024-05-14T09:22:46.445 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.759+0000 7f564887d700 1 Processor -- start 2024-05-14T09:22:46.446 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.760+0000 7f564887d700 1 -- start start 2024-05-14T09:22:46.446 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.760+0000 7f564887d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f564410a540 0x7f564419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:46.446 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.760+0000 7f564887d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f564410de60 con 0x7f564410a540 2024-05-14T09:22:46.446 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.760+0000 7f564259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f564410a540 0x7f564419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:46.446 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.760+0000 7f564259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f564410a540 0x7f564419deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56666/0 (socket says 172.21.15.2:56666) 2024-05-14T09:22:46.446 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.760+0000 7f564259c700 1 -- 172.21.15.2:0/1590670641 learned_addr learned my addr 172.21.15.2:0/1590670641 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:46.446 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.760+0000 7f564259c700 1 -- 172.21.15.2:0/1590670641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564419e3f0 con 0x7f564410a540 2024-05-14T09:22:46.446 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.761+0000 7f564259c700 1 --2- 172.21.15.2:0/1590670641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f564410a540 0x7f564419deb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5634009fb0 tx=0x7f5634011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:46.446 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.761+0000 7f563b7fe700 1 -- 172.21.15.2:0/1590670641 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5634011bc0 con 0x7f564410a540 2024-05-14T09:22:46.446 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.761+0000 7f563b7fe700 1 -- 172.21.15.2:0/1590670641 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5634024440 con 0x7f564410a540 2024-05-14T09:22:46.446 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.761+0000 7f564887d700 1 -- 172.21.15.2:0/1590670641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f564419e680 con 0x7f564410a540 2024-05-14T09:22:46.446 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.761+0000 7f563b7fe700 1 -- 172.21.15.2:0/1590670641 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f563401a5c0 con 0x7f564410a540 2024-05-14T09:22:46.446 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.761+0000 7f564887d700 1 -- 172.21.15.2:0/1590670641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56441a1290 con 0x7f564410a540 2024-05-14T09:22:46.446 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.761+0000 7f563b7fe700 1 -- 172.21.15.2:0/1590670641 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f563401abc0 con 0x7f564410a540 2024-05-14T09:22:46.447 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.761+0000 7f563b7fe700 1 -- 172.21.15.2:0/1590670641 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5634019700 con 0x7f564410a540 2024-05-14T09:22:46.447 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.762+0000 7f564887d700 1 -- 172.21.15.2:0/1590670641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5644064e20 con 0x7f564410a540 2024-05-14T09:22:46.447 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.764+0000 7f563b7fe700 1 -- 172.21.15.2:0/1590670641 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f563402dd90 con 0x7f564410a540 2024-05-14T09:22:46.447 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.840+0000 7f564887d700 1 -- 172.21.15.2:0/1590670641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f56441a1460 con 0x7f564410a540 2024-05-14T09:22:46.447 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.840+0000 7f563b7fe700 1 -- 172.21.15.2:0/1590670641 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f56340245f0 con 0x7f564410a540 2024-05-14T09:22:46.447 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.841+0000 7f564887d700 1 -- 172.21.15.2:0/1590670641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f564410a540 msgr2=0x7f564419deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:46.447 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.841+0000 7f564887d700 1 --2- 172.21.15.2:0/1590670641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f564410a540 0x7f564419deb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5634009fb0 tx=0x7f5634011890 comp rx=0 tx=0).stop 2024-05-14T09:22:46.447 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.841+0000 7f564887d700 1 -- 172.21.15.2:0/1590670641 shutdown_connections 2024-05-14T09:22:46.447 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.841+0000 7f564887d700 1 --2- 172.21.15.2:0/1590670641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f564410a540 0x7f564419deb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:46.447 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.842+0000 7f564887d700 1 -- 172.21.15.2:0/1590670641 >> 172.21.15.2:0/1590670641 conn(0x7f5644103fe0 msgr2=0x7f5644104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:46.447 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.842+0000 7f564887d700 1 -- 172.21.15.2:0/1590670641 shutdown_connections 2024-05-14T09:22:46.447 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:45.842+0000 7f564887d700 1 -- 172.21.15.2:0/1590670641 wait complete. 2024-05-14T09:22:46.447 INFO:teuthology.orchestra.run.smithi002.stdout:mgr not available, waiting (2/15)... 2024-05-14T09:22:47.171 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:46 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:46.712+0000 7f22a67da200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T09:22:47.171 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:46 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:46.813+0000 7f22a67da200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T09:22:47.578 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:47 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:47.200+0000 7f22a67da200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T09:22:47.578 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:47 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:47.576+0000 7f22a67da200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T09:22:47.921 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:47 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:47.673+0000 7f22a67da200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T09:22:48.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:47 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:47.929+0000 7f22a67da200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T09:22:48.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:48 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:48.071+0000 7f22a67da200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T09:22:48.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:48 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:48.171+0000 7f22a67da200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T09:22:48.824 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:48 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:48.519+0000 7f22a67da200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T09:22:48.824 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:48 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:48.617+0000 7f22a67da200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T09:22:49.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:48 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/747208829' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T09:22:49.521 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:49 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:49.270+0000 7f22a67da200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T09:22:49.521 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:49 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:49.499+0000 7f22a67da200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T09:22:49.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T09:22:49.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-05-14T09:22:49.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "50d5aebe-11d3-11ef-bc98-c7b262605968", 2024-05-14T09:22:49.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T09:22:49.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T09:22:49.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T09:22:49.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T09:22:49.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:49.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T09:22:49.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T09:22:49.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-05-14T09:22:49.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T09:22:49.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T09:22:49.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T09:22:49.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:49.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T09:22:49.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T09:22:49.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T09:22:49.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T09:22:49.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T09:22:49.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T09:22:49.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T09:22:49.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T09:22:49.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T09:22:49.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:49.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T09:22:49.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:49.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T09:22:49.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T09:22:49.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:49.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T09:22:49.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-14T09:22:49.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T09:22:49.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T09:22:49.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T09:22:49.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T09:22:49.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T09:22:49.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T09:22:49.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T09:22:49.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:49.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T09:22:49.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:49.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T09:22:34.937680+0000", 2024-05-14T09:22:49.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T09:22:49.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:49.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T09:22:49.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-05-14T09:22:49.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.861+0000 7fc1bb981700 1 Processor -- start 2024-05-14T09:22:49.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.861+0000 7fc1bb981700 1 -- start start 2024-05-14T09:22:49.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.862+0000 7fc1bb981700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b4107de0 0x7fc1b41081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:49.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.862+0000 7fc1bb981700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4108800 con 0x7fc1b4107de0 2024-05-14T09:22:49.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.862+0000 7fc1b971d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b4107de0 0x7fc1b41081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:49.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.862+0000 7fc1b971d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b4107de0 0x7fc1b41081f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37708/0 (socket says 172.21.15.2:37708) 2024-05-14T09:22:49.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.862+0000 7fc1b971d700 1 -- 172.21.15.2:0/2622705470 learned_addr learned my addr 172.21.15.2:0/2622705470 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:49.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.862+0000 7fc1b971d700 1 -- 172.21.15.2:0/2622705470 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b4108980 con 0x7fc1b4107de0 2024-05-14T09:22:49.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.862+0000 7fc1b971d700 1 --2- 172.21.15.2:0/2622705470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b4107de0 0x7fc1b41081f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009e80 tx=0x7fc1a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6629b7c5611a9efb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:49.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.862+0000 7fc1b3fff700 1 -- 172.21.15.2:0/2622705470 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1a4017070 con 0x7fc1b4107de0 2024-05-14T09:22:49.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.863+0000 7fc1b3fff700 1 -- 172.21.15.2:0/2622705470 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc1a400b8c0 con 0x7fc1b4107de0 2024-05-14T09:22:49.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.863+0000 7fc1b3fff700 1 -- 172.21.15.2:0/2622705470 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1a4004490 con 0x7fc1b4107de0 2024-05-14T09:22:49.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.864+0000 7fc1bb981700 1 -- 172.21.15.2:0/2622705470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b4107de0 msgr2=0x7fc1b41081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:49.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.864+0000 7fc1bb981700 1 --2- 172.21.15.2:0/2622705470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b4107de0 0x7fc1b41081f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009e80 tx=0x7fc1a400b330 comp rx=0 tx=0).stop 2024-05-14T09:22:49.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.864+0000 7fc1bb981700 1 -- 172.21.15.2:0/2622705470 shutdown_connections 2024-05-14T09:22:49.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.864+0000 7fc1bb981700 1 --2- 172.21.15.2:0/2622705470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b4107de0 0x7fc1b41081f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:49.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.864+0000 7fc1bb981700 1 -- 172.21.15.2:0/2622705470 >> 172.21.15.2:0/2622705470 conn(0x7fc1b4103760 msgr2=0x7fc1b4105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:49.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.864+0000 7fc1bb981700 1 -- 172.21.15.2:0/2622705470 shutdown_connections 2024-05-14T09:22:49.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.864+0000 7fc1bb981700 1 -- 172.21.15.2:0/2622705470 wait complete. 2024-05-14T09:22:49.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.865+0000 7fc1bb981700 1 Processor -- start 2024-05-14T09:22:49.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.865+0000 7fc1bb981700 1 -- start start 2024-05-14T09:22:49.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.865+0000 7fc1bb981700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b4107de0 0x7fc1b419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:49.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.865+0000 7fc1bb981700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b410de60 con 0x7fc1b4107de0 2024-05-14T09:22:49.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.865+0000 7fc1b971d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b4107de0 0x7fc1b419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:49.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.865+0000 7fc1b971d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b4107de0 0x7fc1b419de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37724/0 (socket says 172.21.15.2:37724) 2024-05-14T09:22:49.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.865+0000 7fc1b971d700 1 -- 172.21.15.2:0/747208829 learned_addr learned my addr 172.21.15.2:0/747208829 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:49.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.866+0000 7fc1b971d700 1 -- 172.21.15.2:0/747208829 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b419e3c0 con 0x7fc1b4107de0 2024-05-14T09:22:49.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.866+0000 7fc1b971d700 1 --2- 172.21.15.2:0/747208829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b4107de0 0x7fc1b419de80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400b7b0 tx=0x7fc1a400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:49.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.866+0000 7fc1b27fc700 1 -- 172.21.15.2:0/747208829 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1a4020020 con 0x7fc1b4107de0 2024-05-14T09:22:49.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.866+0000 7fc1b27fc700 1 -- 172.21.15.2:0/747208829 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc1a4012430 con 0x7fc1b4107de0 2024-05-14T09:22:49.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.866+0000 7fc1bb981700 1 -- 172.21.15.2:0/747208829 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b419e650 con 0x7fc1b4107de0 2024-05-14T09:22:49.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.866+0000 7fc1b27fc700 1 -- 172.21.15.2:0/747208829 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1a4017040 con 0x7fc1b4107de0 2024-05-14T09:22:49.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.866+0000 7fc1bb981700 1 -- 172.21.15.2:0/747208829 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b41a1260 con 0x7fc1b4107de0 2024-05-14T09:22:49.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.866+0000 7fc1b27fc700 1 -- 172.21.15.2:0/747208829 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc1a4023020 con 0x7fc1b4107de0 2024-05-14T09:22:49.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.866+0000 7fc1b27fc700 1 -- 172.21.15.2:0/747208829 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc1a400f050 con 0x7fc1b4107de0 2024-05-14T09:22:49.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.867+0000 7fc1bb981700 1 -- 172.21.15.2:0/747208829 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b4064e20 con 0x7fc1b4107de0 2024-05-14T09:22:49.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.869+0000 7fc1b27fc700 1 -- 172.21.15.2:0/747208829 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc1a402a020 con 0x7fc1b4107de0 2024-05-14T09:22:49.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.945+0000 7fc1bb981700 1 -- 172.21.15.2:0/747208829 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc1b41a1430 con 0x7fc1b4107de0 2024-05-14T09:22:49.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.946+0000 7fc1b27fc700 1 -- 172.21.15.2:0/747208829 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc1b4108980 con 0x7fc1b4107de0 2024-05-14T09:22:49.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.947+0000 7fc1bb981700 1 -- 172.21.15.2:0/747208829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b4107de0 msgr2=0x7fc1b419de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:49.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.947+0000 7fc1bb981700 1 --2- 172.21.15.2:0/747208829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b4107de0 0x7fc1b419de80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400b7b0 tx=0x7fc1a400bd50 comp rx=0 tx=0).stop 2024-05-14T09:22:49.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.947+0000 7fc1bb981700 1 -- 172.21.15.2:0/747208829 shutdown_connections 2024-05-14T09:22:49.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.947+0000 7fc1bb981700 1 --2- 172.21.15.2:0/747208829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b4107de0 0x7fc1b419de80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:49.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.947+0000 7fc1bb981700 1 -- 172.21.15.2:0/747208829 >> 172.21.15.2:0/747208829 conn(0x7fc1b4103760 msgr2=0x7fc1b4104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:49.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.947+0000 7fc1bb981700 1 -- 172.21.15.2:0/747208829 shutdown_connections 2024-05-14T09:22:49.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:48.947+0000 7fc1bb981700 1 -- 172.21.15.2:0/747208829 wait complete. 2024-05-14T09:22:49.538 INFO:teuthology.orchestra.run.smithi002.stdout:mgr not available, waiting (3/15)... 2024-05-14T09:22:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:49 smithi002 ceph-mon[31557]: Activating manager daemon x 2024-05-14T09:22:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:49 smithi002 ceph-mon[31557]: mgrmap e2: x(active, starting, since 0.00403154s) 2024-05-14T09:22:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:49 smithi002 ceph-mon[31557]: from='mgr.14100 172.21.15.2:0/4178664488' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T09:22:50.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:49 smithi002 ceph-mon[31557]: from='mgr.14100 172.21.15.2:0/4178664488' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T09:22:50.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:49 smithi002 ceph-mon[31557]: from='mgr.14100 172.21.15.2:0/4178664488' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T09:22:50.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:49 smithi002 ceph-mon[31557]: from='mgr.14100 172.21.15.2:0/4178664488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:22:50.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:49 smithi002 ceph-mon[31557]: from='mgr.14100 172.21.15.2:0/4178664488' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T09:22:50.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:49 smithi002 ceph-mon[31557]: Manager daemon x is now available 2024-05-14T09:22:50.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:49 smithi002 ceph-mon[31557]: from='mgr.14100 172.21.15.2:0/4178664488' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T09:22:50.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:49 smithi002 ceph-mon[31557]: from='mgr.14100 172.21.15.2:0/4178664488' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T09:22:50.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:49 smithi002 ceph-mon[31557]: from='mgr.14100 172.21.15.2:0/4178664488' entity='mgr.x' 2024-05-14T09:22:50.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:49 smithi002 ceph-mon[31557]: from='mgr.14100 172.21.15.2:0/4178664488' entity='mgr.x' 2024-05-14T09:22:50.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:49 smithi002 ceph-mon[31557]: from='mgr.14100 172.21.15.2:0/4178664488' entity='mgr.x' 2024-05-14T09:22:51.767 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:51 smithi002 ceph-mon[31557]: mgrmap e3: x(active, since 1.00827s) 2024-05-14T09:22:52.582 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:52 smithi002 ceph-mon[31557]: mgrmap e4: x(active, since 2s) 2024-05-14T09:22:52.582 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:52 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3512643656' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T09:22:52.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T09:22:52.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-05-14T09:22:52.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "50d5aebe-11d3-11ef-bc98-c7b262605968", 2024-05-14T09:22:52.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T09:22:52.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T09:22:52.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T09:22:52.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T09:22:52.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:52.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T09:22:52.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T09:22:52.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-05-14T09:22:52.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T09:22:52.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T09:22:52.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T09:22:52.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T09:22:52.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-14T09:22:52.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T09:22:52.847 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:52.848 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T09:22:52.848 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T09:22:52.848 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:52.848 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T09:22:52.848 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:52.848 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T09:22:52.848 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T09:22:52.848 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T09:22:52.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T09:22:52.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T09:22:52.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T09:22:52.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:52.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T09:22:52.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T09:22:52.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T09:22:52.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T09:22:52.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T09:22:52.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T09:22:52.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T09:22:52.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T09:22:52.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T09:22:52.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:52.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T09:22:52.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:52.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T09:22:52.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T09:22:52.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:52.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T09:22:52.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-14T09:22:52.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T09:22:52.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T09:22:52.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T09:22:52.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T09:22:52.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T09:22:52.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T09:22:52.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T09:22:52.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:52.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T09:22:52.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T09:22:52.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T09:22:34.937680+0000", 2024-05-14T09:22:52.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T09:22:52.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T09:22:52.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T09:22:52.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-05-14T09:22:52.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.979+0000 7fa40a1b6700 1 Processor -- start 2024-05-14T09:22:52.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.979+0000 7fa40a1b6700 1 -- start start 2024-05-14T09:22:52.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.979+0000 7fa40a1b6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa404108920 0x7fa404108d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:52.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.979+0000 7fa40a1b6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa404109320 con 0x7fa404108920 2024-05-14T09:22:52.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.980+0000 7fa4037fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa404108920 0x7fa404108d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:52.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.980+0000 7fa4037fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa404108920 0x7fa404108d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37820/0 (socket says 172.21.15.2:37820) 2024-05-14T09:22:52.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.980+0000 7fa4037fe700 1 -- 172.21.15.2:0/1669801595 learned_addr learned my addr 172.21.15.2:0/1669801595 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:52.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.980+0000 7fa4037fe700 1 -- 172.21.15.2:0/1669801595 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa404109af0 con 0x7fa404108920 2024-05-14T09:22:52.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.980+0000 7fa4037fe700 1 --2- 172.21.15.2:0/1669801595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa404108920 0x7fa404108d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40098a0 tx=0x7fa3f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c709c57308e048e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:52.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.980+0000 7fa402ffd700 1 -- 172.21.15.2:0/1669801595 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3f4004160 con 0x7fa404108920 2024-05-14T09:22:52.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.980+0000 7fa402ffd700 1 -- 172.21.15.2:0/1669801595 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa3f4004300 con 0x7fa404108920 2024-05-14T09:22:52.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.980+0000 7fa402ffd700 1 -- 172.21.15.2:0/1669801595 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3f4011af0 con 0x7fa404108920 2024-05-14T09:22:52.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.981+0000 7fa40a1b6700 1 -- 172.21.15.2:0/1669801595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa404108920 msgr2=0x7fa404108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:52.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.981+0000 7fa40a1b6700 1 --2- 172.21.15.2:0/1669801595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa404108920 0x7fa404108d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40098a0 tx=0x7fa3f4009bb0 comp rx=0 tx=0).stop 2024-05-14T09:22:52.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.981+0000 7fa40a1b6700 1 -- 172.21.15.2:0/1669801595 shutdown_connections 2024-05-14T09:22:52.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.981+0000 7fa40a1b6700 1 --2- 172.21.15.2:0/1669801595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa404108920 0x7fa404108d30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:52.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.981+0000 7fa40a1b6700 1 -- 172.21.15.2:0/1669801595 >> 172.21.15.2:0/1669801595 conn(0x7fa404103fc0 msgr2=0x7fa404106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:52.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.981+0000 7fa40a1b6700 1 -- 172.21.15.2:0/1669801595 shutdown_connections 2024-05-14T09:22:52.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.981+0000 7fa40a1b6700 1 -- 172.21.15.2:0/1669801595 wait complete. 2024-05-14T09:22:52.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.982+0000 7fa40a1b6700 1 Processor -- start 2024-05-14T09:22:52.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.982+0000 7fa40a1b6700 1 -- start start 2024-05-14T09:22:52.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.982+0000 7fa40a1b6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa404108920 0x7fa404106070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:52.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.983+0000 7fa40a1b6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa40410cea0 con 0x7fa404108920 2024-05-14T09:22:52.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.983+0000 7fa4037fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa404108920 0x7fa404106070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:52.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.983+0000 7fa4037fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa404108920 0x7fa404106070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37836/0 (socket says 172.21.15.2:37836) 2024-05-14T09:22:52.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.983+0000 7fa4037fe700 1 -- 172.21.15.2:0/3512643656 learned_addr learned my addr 172.21.15.2:0/3512643656 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:52.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.983+0000 7fa4037fe700 1 -- 172.21.15.2:0/3512643656 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4041065b0 con 0x7fa404108920 2024-05-14T09:22:52.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.983+0000 7fa4037fe700 1 --2- 172.21.15.2:0/3512643656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa404108920 0x7fa404106070 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4011890 tx=0x7fa3f4011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:52.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.983+0000 7fa4017fa700 1 -- 172.21.15.2:0/3512643656 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3f40039e0 con 0x7fa404108920 2024-05-14T09:22:52.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.983+0000 7fa4017fa700 1 -- 172.21.15.2:0/3512643656 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa3f4003b80 con 0x7fa404108920 2024-05-14T09:22:52.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.983+0000 7fa40a1b6700 1 -- 172.21.15.2:0/3512643656 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa404104800 con 0x7fa404108920 2024-05-14T09:22:52.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.983+0000 7fa4017fa700 1 -- 172.21.15.2:0/3512643656 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3f40246e0 con 0x7fa404108920 2024-05-14T09:22:52.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.984+0000 7fa40a1b6700 1 -- 172.21.15.2:0/3512643656 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa404104ce0 con 0x7fa404108920 2024-05-14T09:22:52.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.984+0000 7fa4017fa700 1 -- 172.21.15.2:0/3512643656 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fa3f4022020 con 0x7fa404108920 2024-05-14T09:22:52.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.984+0000 7fa40a1b6700 1 -- 172.21.15.2:0/3512643656 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa404064e20 con 0x7fa404108920 2024-05-14T09:22:52.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.985+0000 7fa4017fa700 1 --2- 172.21.15.2:0/3512643656 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fa3f003e470 0x7fa3f0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:52.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.985+0000 7fa4017fa700 1 -- 172.21.15.2:0/3512643656 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa3f40533c0 con 0x7fa404108920 2024-05-14T09:22:52.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.985+0000 7fa3faffd700 1 --2- 172.21.15.2:0/3512643656 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fa3f003e470 0x7fa3f0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:52.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.985+0000 7fa3faffd700 1 --2- 172.21.15.2:0/3512643656 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fa3f003e470 0x7fa3f0040980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec006fd0 tx=0x7fa3ec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:52.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:51.991+0000 7fa4017fa700 1 -- 172.21.15.2:0/3512643656 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3f40172b0 con 0x7fa404108920 2024-05-14T09:22:52.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:52.301+0000 7fa40a1b6700 1 -- 172.21.15.2:0/3512643656 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa4041970e0 con 0x7fa404108920 2024-05-14T09:22:52.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:52.302+0000 7fa4017fa700 1 -- 172.21.15.2:0/3512643656 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa3f4053120 con 0x7fa404108920 2024-05-14T09:22:52.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:52.304+0000 7fa40a1b6700 1 -- 172.21.15.2:0/3512643656 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fa3f003e470 msgr2=0x7fa3f0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:52.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:52.304+0000 7fa40a1b6700 1 --2- 172.21.15.2:0/3512643656 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fa3f003e470 0x7fa3f0040980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec006fd0 tx=0x7fa3ec006e40 comp rx=0 tx=0).stop 2024-05-14T09:22:52.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:52.304+0000 7fa40a1b6700 1 -- 172.21.15.2:0/3512643656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa404108920 msgr2=0x7fa404106070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:52.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:52.304+0000 7fa40a1b6700 1 --2- 172.21.15.2:0/3512643656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa404108920 0x7fa404106070 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4011890 tx=0x7fa3f4011970 comp rx=0 tx=0).stop 2024-05-14T09:22:52.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:52.304+0000 7fa40a1b6700 1 -- 172.21.15.2:0/3512643656 shutdown_connections 2024-05-14T09:22:52.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:52.304+0000 7fa40a1b6700 1 --2- 172.21.15.2:0/3512643656 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fa3f003e470 0x7fa3f0040980 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:52.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:52.304+0000 7fa40a1b6700 1 --2- 172.21.15.2:0/3512643656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa404108920 0x7fa404106070 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:52.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:52.304+0000 7fa40a1b6700 1 -- 172.21.15.2:0/3512643656 >> 172.21.15.2:0/3512643656 conn(0x7fa404103fc0 msgr2=0x7fa404194450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:52.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:52.304+0000 7fa40a1b6700 1 -- 172.21.15.2:0/3512643656 shutdown_connections 2024-05-14T09:22:52.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:52.304+0000 7fa40a1b6700 1 -- 172.21.15.2:0/3512643656 wait complete. 2024-05-14T09:22:52.863 INFO:teuthology.orchestra.run.smithi002.stdout:mgr is available 2024-05-14T09:22:53.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:53 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/570419047' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-14T09:22:54.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T09:22:54.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [global] 2024-05-14T09:22:54.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout fsid = 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:22:54.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-14T09:22:54.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-14T09:22:54.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-14T09:22:54.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-14T09:22:54.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-14T09:22:54.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T09:22:54.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-14T09:22:54.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-14T09:22:54.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T09:22:54.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [osd] 2024-05-14T09:22:54.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-14T09:22:54.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-14T09:22:54.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-14T09:22:54.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-14T09:22:54.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-14T09:22:54.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.438+0000 7eff662f1700 1 Processor -- start 2024-05-14T09:22:54.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.438+0000 7eff662f1700 1 -- start start 2024-05-14T09:22:54.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.438+0000 7eff662f1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff60109a90 0x7eff60109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:54.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.438+0000 7eff662f1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010a490 con 0x7eff60109a90 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.438+0000 7eff5ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff60109a90 0x7eff60109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.438+0000 7eff5ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff60109a90 0x7eff60109ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37840/0 (socket says 172.21.15.2:37840) 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.438+0000 7eff5ffff700 1 -- 172.21.15.2:0/2404923536 learned_addr learned my addr 172.21.15.2:0/2404923536 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.439+0000 7eff5ffff700 1 -- 172.21.15.2:0/2404923536 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6010ac60 con 0x7eff60109a90 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.439+0000 7eff5ffff700 1 --2- 172.21.15.2:0/2404923536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff60109a90 0x7eff60109ea0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7eff500098a0 tx=0x7eff50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ea3dfabf70d5d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.439+0000 7eff5effd700 1 -- 172.21.15.2:0/2404923536 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff50004160 con 0x7eff60109a90 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.439+0000 7eff5effd700 1 -- 172.21.15.2:0/2404923536 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7eff50004300 con 0x7eff60109a90 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.439+0000 7eff5effd700 1 -- 172.21.15.2:0/2404923536 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff50011af0 con 0x7eff60109a90 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.440+0000 7eff662f1700 1 -- 172.21.15.2:0/2404923536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff60109a90 msgr2=0x7eff60109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.440+0000 7eff662f1700 1 --2- 172.21.15.2:0/2404923536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff60109a90 0x7eff60109ea0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7eff500098a0 tx=0x7eff50009bb0 comp rx=0 tx=0).stop 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.440+0000 7eff662f1700 1 -- 172.21.15.2:0/2404923536 shutdown_connections 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.440+0000 7eff662f1700 1 --2- 172.21.15.2:0/2404923536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff60109a90 0x7eff60109ea0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.440+0000 7eff662f1700 1 -- 172.21.15.2:0/2404923536 >> 172.21.15.2:0/2404923536 conn(0x7eff601050f0 msgr2=0x7eff60107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.440+0000 7eff662f1700 1 -- 172.21.15.2:0/2404923536 shutdown_connections 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.440+0000 7eff662f1700 1 -- 172.21.15.2:0/2404923536 wait complete. 2024-05-14T09:22:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.441+0000 7eff662f1700 1 Processor -- start 2024-05-14T09:22:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.441+0000 7eff662f1700 1 -- start start 2024-05-14T09:22:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.441+0000 7eff662f1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff60109a90 0x7eff6007f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.441+0000 7eff662f1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010de90 con 0x7eff60109a90 2024-05-14T09:22:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.441+0000 7eff5ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff60109a90 0x7eff6007f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.441+0000 7eff5ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff60109a90 0x7eff6007f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37846/0 (socket says 172.21.15.2:37846) 2024-05-14T09:22:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.441+0000 7eff5ffff700 1 -- 172.21.15.2:0/570419047 learned_addr learned my addr 172.21.15.2:0/570419047 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.441+0000 7eff5ffff700 1 -- 172.21.15.2:0/570419047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6007f9f0 con 0x7eff60109a90 2024-05-14T09:22:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.442+0000 7eff5ffff700 1 --2- 172.21.15.2:0/570419047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff60109a90 0x7eff6007f4b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7eff50011890 tx=0x7eff50011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.442+0000 7eff5d7fa700 1 -- 172.21.15.2:0/570419047 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff500039e0 con 0x7eff60109a90 2024-05-14T09:22:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.442+0000 7eff5d7fa700 1 -- 172.21.15.2:0/570419047 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7eff50003b80 con 0x7eff60109a90 2024-05-14T09:22:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.442+0000 7eff662f1700 1 -- 172.21.15.2:0/570419047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6007dc40 con 0x7eff60109a90 2024-05-14T09:22:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.442+0000 7eff5d7fa700 1 -- 172.21.15.2:0/570419047 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff500246e0 con 0x7eff60109a90 2024-05-14T09:22:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.442+0000 7eff662f1700 1 -- 172.21.15.2:0/570419047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6007e120 con 0x7eff60109a90 2024-05-14T09:22:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.443+0000 7eff5d7fa700 1 -- 172.21.15.2:0/570419047 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7eff50022020 con 0x7eff60109a90 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.443+0000 7eff662f1700 1 -- 172.21.15.2:0/570419047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff60064e20 con 0x7eff60109a90 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.443+0000 7eff5d7fa700 1 --2- 172.21.15.2:0/570419047 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7eff4803e470 0x7eff48040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.443+0000 7eff5d7fa700 1 -- 172.21.15.2:0/570419047 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7eff500533f0 con 0x7eff60109a90 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.443+0000 7eff5f7fe700 1 --2- 172.21.15.2:0/570419047 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7eff4803e470 0x7eff48040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.444+0000 7eff5f7fe700 1 --2- 172.21.15.2:0/570419047 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7eff4803e470 0x7eff48040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7eff54006fd0 tx=0x7eff54006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.448+0000 7eff5d7fa700 1 -- 172.21.15.2:0/570419047 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff500172b0 con 0x7eff60109a90 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.661+0000 7eff662f1700 1 -- 172.21.15.2:0/570419047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7eff6007e860 con 0x7eff60109a90 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.663+0000 7eff5d7fa700 1 -- 172.21.15.2:0/570419047 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7eff5000fdd0 con 0x7eff60109a90 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.665+0000 7eff662f1700 1 -- 172.21.15.2:0/570419047 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7eff4803e470 msgr2=0x7eff48040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.665+0000 7eff662f1700 1 --2- 172.21.15.2:0/570419047 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7eff4803e470 0x7eff48040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7eff54006fd0 tx=0x7eff54006e40 comp rx=0 tx=0).stop 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.665+0000 7eff662f1700 1 -- 172.21.15.2:0/570419047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff60109a90 msgr2=0x7eff6007f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.665+0000 7eff662f1700 1 --2- 172.21.15.2:0/570419047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff60109a90 0x7eff6007f4b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7eff50011890 tx=0x7eff50011970 comp rx=0 tx=0).stop 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.665+0000 7eff662f1700 1 -- 172.21.15.2:0/570419047 shutdown_connections 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.665+0000 7eff662f1700 1 --2- 172.21.15.2:0/570419047 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7eff4803e470 0x7eff48040980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.665+0000 7eff662f1700 1 --2- 172.21.15.2:0/570419047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff60109a90 0x7eff6007f4b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.665+0000 7eff662f1700 1 -- 172.21.15.2:0/570419047 >> 172.21.15.2:0/570419047 conn(0x7eff601050f0 msgr2=0x7eff60105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:54.187 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.666+0000 7eff662f1700 1 -- 172.21.15.2:0/570419047 shutdown_connections 2024-05-14T09:22:54.187 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:53.666+0000 7eff662f1700 1 -- 172.21.15.2:0/570419047 wait complete. 2024-05-14T09:22:54.187 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling cephadm module... 2024-05-14T09:22:55.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:54 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2514590300' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-14T09:22:55.171 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:55 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: ignoring --setuser ceph since I am not root 2024-05-14T09:22:55.171 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:55 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: ignoring --setgroup ceph since I am not root 2024-05-14T09:22:55.479 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:55 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:55.173+0000 7f442fdf1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T09:22:55.480 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:55 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:55.370+0000 7f442fdf1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T09:22:55.480 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:55 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:55.479+0000 7f442fdf1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T09:22:55.563 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.626+0000 7f707eaa2700 1 Processor -- start 2024-05-14T09:22:55.563 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.626+0000 7f707eaa2700 1 -- start start 2024-05-14T09:22:55.563 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.626+0000 7f707eaa2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f707810a5c0 0x7f707810c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:55.563 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.626+0000 7f707eaa2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f707807d860 con 0x7f707810a5c0 2024-05-14T09:22:55.563 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.626+0000 7f707c83e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f707810a5c0 0x7f707810c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:55.563 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.626+0000 7f707c83e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f707810a5c0 0x7f707810c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37858/0 (socket says 172.21.15.2:37858) 2024-05-14T09:22:55.563 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.626+0000 7f707c83e700 1 -- 172.21.15.2:0/1143596832 learned_addr learned my addr 172.21.15.2:0/1143596832 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:55.563 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.627+0000 7f707c83e700 1 -- 172.21.15.2:0/1143596832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707810cf20 con 0x7f707810a5c0 2024-05-14T09:22:55.564 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.627+0000 7f707c83e700 1 --2- 172.21.15.2:0/1143596832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f707810a5c0 0x7f707810c9e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f706c0098a0 tx=0x7f706c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a639470e5bed6bbd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:55.564 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.627+0000 7f70777fe700 1 -- 172.21.15.2:0/1143596832 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f706c004160 con 0x7f707810a5c0 2024-05-14T09:22:55.564 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.627+0000 7f70777fe700 1 -- 172.21.15.2:0/1143596832 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f706c004300 con 0x7f707810a5c0 2024-05-14T09:22:55.564 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.627+0000 7f70777fe700 1 -- 172.21.15.2:0/1143596832 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f706c011af0 con 0x7f707810a5c0 2024-05-14T09:22:55.564 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.628+0000 7f707eaa2700 1 -- 172.21.15.2:0/1143596832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f707810a5c0 msgr2=0x7f707810c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:55.564 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.628+0000 7f707eaa2700 1 --2- 172.21.15.2:0/1143596832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f707810a5c0 0x7f707810c9e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f706c0098a0 tx=0x7f706c009bb0 comp rx=0 tx=0).stop 2024-05-14T09:22:55.564 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.628+0000 7f707eaa2700 1 -- 172.21.15.2:0/1143596832 shutdown_connections 2024-05-14T09:22:55.564 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.628+0000 7f707eaa2700 1 --2- 172.21.15.2:0/1143596832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f707810a5c0 0x7f707810c9e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:55.564 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.628+0000 7f707eaa2700 1 -- 172.21.15.2:0/1143596832 >> 172.21.15.2:0/1143596832 conn(0x7f7078104040 msgr2=0x7f7078106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:55.564 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.628+0000 7f707eaa2700 1 -- 172.21.15.2:0/1143596832 shutdown_connections 2024-05-14T09:22:55.564 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.628+0000 7f707eaa2700 1 -- 172.21.15.2:0/1143596832 wait complete. 2024-05-14T09:22:55.564 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.629+0000 7f707eaa2700 1 Processor -- start 2024-05-14T09:22:55.564 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.629+0000 7f707eaa2700 1 -- start start 2024-05-14T09:22:55.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.629+0000 7f707eaa2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f707810a5c0 0x7f707819e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:55.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.629+0000 7f707eaa2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f707810de10 con 0x7f707810a5c0 2024-05-14T09:22:55.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.629+0000 7f707c83e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f707810a5c0 0x7f707819e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:55.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.629+0000 7f707c83e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f707810a5c0 0x7f707819e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37860/0 (socket says 172.21.15.2:37860) 2024-05-14T09:22:55.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.629+0000 7f707c83e700 1 -- 172.21.15.2:0/2514590300 learned_addr learned my addr 172.21.15.2:0/2514590300 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:55.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.629+0000 7f707c83e700 1 -- 172.21.15.2:0/2514590300 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707819e6c0 con 0x7f707810a5c0 2024-05-14T09:22:55.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.629+0000 7f707c83e700 1 --2- 172.21.15.2:0/2514590300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f707810a5c0 0x7f707819e180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f706c009fb0 tx=0x7f706c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:55.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.630+0000 7f7075ffb700 1 -- 172.21.15.2:0/2514590300 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f706c011bc0 con 0x7f707810a5c0 2024-05-14T09:22:55.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.630+0000 7f7075ffb700 1 -- 172.21.15.2:0/2514590300 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f706c024440 con 0x7f707810a5c0 2024-05-14T09:22:55.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.630+0000 7f7075ffb700 1 -- 172.21.15.2:0/2514590300 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f706c01a5c0 con 0x7f707810a5c0 2024-05-14T09:22:55.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.630+0000 7f707eaa2700 1 -- 172.21.15.2:0/2514590300 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f707819e950 con 0x7f707810a5c0 2024-05-14T09:22:55.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.630+0000 7f707eaa2700 1 -- 172.21.15.2:0/2514590300 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70781a1560 con 0x7f707810a5c0 2024-05-14T09:22:55.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.631+0000 7f7075ffb700 1 -- 172.21.15.2:0/2514590300 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f706c0245f0 con 0x7f707810a5c0 2024-05-14T09:22:55.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.631+0000 7f707eaa2700 1 -- 172.21.15.2:0/2514590300 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7078064e20 con 0x7f707810a5c0 2024-05-14T09:22:55.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.631+0000 7f7075ffb700 1 --2- 172.21.15.2:0/2514590300 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7f706803e3d0 0x7f70680408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:55.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.631+0000 7f7075ffb700 1 -- 172.21.15.2:0/2514590300 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f706c0537d0 con 0x7f707810a5c0 2024-05-14T09:22:55.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.631+0000 7f7077fff700 1 --2- 172.21.15.2:0/2514590300 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7f706803e3d0 0x7f70680408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:55.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.632+0000 7f7077fff700 1 --2- 172.21.15.2:0/2514590300 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7f706803e3d0 0x7f70680408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7064006fd0 tx=0x7f7064006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:55.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.637+0000 7f7075ffb700 1 -- 172.21.15.2:0/2514590300 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f706c017dd0 con 0x7f707810a5c0 2024-05-14T09:22:55.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.910+0000 7f707eaa2700 1 -- 172.21.15.2:0/2514590300 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f707802e620 con 0x7f707810a5c0 2024-05-14T09:22:55.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.962+0000 7f7075ffb700 1 -- 172.21.15.2:0/2514590300 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f706c0531f0 con 0x7f707810a5c0 2024-05-14T09:22:55.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.964+0000 7f7075ffb700 1 -- 172.21.15.2:0/2514590300 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f706c0038b0 con 0x7f707810a5c0 2024-05-14T09:22:55.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.965+0000 7f707eaa2700 1 -- 172.21.15.2:0/2514590300 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7f706803e3d0 msgr2=0x7f70680408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:55.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.965+0000 7f707eaa2700 1 --2- 172.21.15.2:0/2514590300 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7f706803e3d0 0x7f70680408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7064006fd0 tx=0x7f7064006e40 comp rx=0 tx=0).stop 2024-05-14T09:22:55.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.965+0000 7f707eaa2700 1 -- 172.21.15.2:0/2514590300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f707810a5c0 msgr2=0x7f707819e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:55.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.965+0000 7f707eaa2700 1 --2- 172.21.15.2:0/2514590300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f707810a5c0 0x7f707819e180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f706c009fb0 tx=0x7f706c011890 comp rx=0 tx=0).stop 2024-05-14T09:22:55.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.965+0000 7f707eaa2700 1 -- 172.21.15.2:0/2514590300 shutdown_connections 2024-05-14T09:22:55.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.965+0000 7f707eaa2700 1 --2- 172.21.15.2:0/2514590300 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7f706803e3d0 0x7f70680408e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:55.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.965+0000 7f707eaa2700 1 --2- 172.21.15.2:0/2514590300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f707810a5c0 0x7f707819e180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:55.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.965+0000 7f707eaa2700 1 -- 172.21.15.2:0/2514590300 >> 172.21.15.2:0/2514590300 conn(0x7f7078104040 msgr2=0x7f7078105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:55.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.965+0000 7f707eaa2700 1 -- 172.21.15.2:0/2514590300 shutdown_connections 2024-05-14T09:22:55.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:54.965+0000 7f707eaa2700 1 -- 172.21.15.2:0/2514590300 wait complete. 2024-05-14T09:22:56.082 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:55 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2514590300' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-14T09:22:56.082 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:55 smithi002 ceph-mon[31557]: mgrmap e5: x(active, since 5s) 2024-05-14T09:22:56.082 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:55 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:55.813+0000 7f442fdf1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T09:22:56.082 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:55 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:55.907+0000 7f442fdf1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T09:22:56.878 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-05-14T09:22:56.878 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-14T09:22:56.878 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-14T09:22:56.878 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-14T09:22:56.878 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-14T09:22:56.878 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.021+0000 7fe1a49d4700 1 Processor -- start 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.021+0000 7fe1a49d4700 1 -- start start 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.022+0000 7fe1a49d4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a0108920 0x7fe1a0108d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.022+0000 7fe1a49d4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a0109320 con 0x7fe1a0108920 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.022+0000 7fe19e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a0108920 0x7fe1a0108d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.022+0000 7fe19e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a0108920 0x7fe1a0108d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37882/0 (socket says 172.21.15.2:37882) 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.022+0000 7fe19e59c700 1 -- 172.21.15.2:0/1992427714 learned_addr learned my addr 172.21.15.2:0/1992427714 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.023+0000 7fe19e59c700 1 -- 172.21.15.2:0/1992427714 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a0109af0 con 0x7fe1a0108920 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.023+0000 7fe19e59c700 1 --2- 172.21.15.2:0/1992427714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a0108920 0x7fe1a0108d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe1900098a0 tx=0x7fe190009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c342f776360042e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.023+0000 7fe19d59a700 1 -- 172.21.15.2:0/1992427714 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe190004160 con 0x7fe1a0108920 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.023+0000 7fe19d59a700 1 -- 172.21.15.2:0/1992427714 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe190004300 con 0x7fe1a0108920 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.023+0000 7fe19d59a700 1 -- 172.21.15.2:0/1992427714 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe190011af0 con 0x7fe1a0108920 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.024+0000 7fe1a49d4700 1 -- 172.21.15.2:0/1992427714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a0108920 msgr2=0x7fe1a0108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.024+0000 7fe1a49d4700 1 --2- 172.21.15.2:0/1992427714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a0108920 0x7fe1a0108d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe1900098a0 tx=0x7fe190009bb0 comp rx=0 tx=0).stop 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.024+0000 7fe1a49d4700 1 -- 172.21.15.2:0/1992427714 shutdown_connections 2024-05-14T09:22:56.879 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.024+0000 7fe1a49d4700 1 --2- 172.21.15.2:0/1992427714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a0108920 0x7fe1a0108d30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:56.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.024+0000 7fe1a49d4700 1 -- 172.21.15.2:0/1992427714 >> 172.21.15.2:0/1992427714 conn(0x7fe1a0103fc0 msgr2=0x7fe1a0106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:56.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.024+0000 7fe1a49d4700 1 -- 172.21.15.2:0/1992427714 shutdown_connections 2024-05-14T09:22:56.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.024+0000 7fe1a49d4700 1 -- 172.21.15.2:0/1992427714 wait complete. 2024-05-14T09:22:56.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.025+0000 7fe1a49d4700 1 Processor -- start 2024-05-14T09:22:56.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.025+0000 7fe1a49d4700 1 -- start start 2024-05-14T09:22:56.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.026+0000 7fe1a49d4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a0108920 0x7fe1a019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:56.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.026+0000 7fe1a49d4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a010cea0 con 0x7fe1a0108920 2024-05-14T09:22:56.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.026+0000 7fe19e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a0108920 0x7fe1a019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:22:56.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.026+0000 7fe19e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a0108920 0x7fe1a019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37888/0 (socket says 172.21.15.2:37888) 2024-05-14T09:22:56.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.026+0000 7fe19e59c700 1 -- 172.21.15.2:0/3168880509 learned_addr learned my addr 172.21.15.2:0/3168880509 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:22:56.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.026+0000 7fe19e59c700 1 -- 172.21.15.2:0/3168880509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a019d2e0 con 0x7fe1a0108920 2024-05-14T09:22:56.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.026+0000 7fe19e59c700 1 --2- 172.21.15.2:0/3168880509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a0108920 0x7fe1a019cda0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe190009fb0 tx=0x7fe190011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:22:56.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.026+0000 7fe18f7fe700 1 -- 172.21.15.2:0/3168880509 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1900036a0 con 0x7fe1a0108920 2024-05-14T09:22:56.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.026+0000 7fe18f7fe700 1 -- 172.21.15.2:0/3168880509 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe190003cd0 con 0x7fe1a0108920 2024-05-14T09:22:56.880 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.027+0000 7fe1a49d4700 1 -- 172.21.15.2:0/3168880509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a019d570 con 0x7fe1a0108920 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.027+0000 7fe18f7fe700 1 -- 172.21.15.2:0/3168880509 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe19001a430 con 0x7fe1a0108920 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.027+0000 7fe1a49d4700 1 -- 172.21.15.2:0/3168880509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a01a0180 con 0x7fe1a0108920 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.028+0000 7fe18f7fe700 1 -- 172.21.15.2:0/3168880509 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fe19001a6f0 con 0x7fe1a0108920 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.028+0000 7fe1a49d4700 1 -- 172.21.15.2:0/3168880509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a0064e20 con 0x7fe1a0108920 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.028+0000 7fe18f7fe700 1 --2- 172.21.15.2:0/3168880509 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe18803e420 0x7fe188040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.028+0000 7fe18f7fe700 1 -- 172.21.15.2:0/3168880509 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe190054680 con 0x7fe1a0108920 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.028+0000 7fe19dd9b700 1 -- 172.21.15.2:0/3168880509 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe18803e420 msgr2=0x7fe188040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3593583257 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.028+0000 7fe19dd9b700 1 --2- 172.21.15.2:0/3168880509 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe18803e420 0x7fe188040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.034+0000 7fe18f7fe700 1 -- 172.21.15.2:0/3168880509 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe190017e90 con 0x7fe1a0108920 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.229+0000 7fe19dd9b700 1 -- 172.21.15.2:0/3168880509 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe18803e420 msgr2=0x7fe188040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3593583257 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.229+0000 7fe19dd9b700 1 --2- 172.21.15.2:0/3168880509 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe18803e420 0x7fe188040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.315+0000 7fe1a49d4700 1 -- 172.21.15.2:0/3168880509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe1a007cea0 con 0x7fe1a0108920 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.317+0000 7fe18f7fe700 1 -- 172.21.15.2:0/3168880509 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fe19002c350 con 0x7fe1a0108920 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.322+0000 7fe1a49d4700 1 -- 172.21.15.2:0/3168880509 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe18803e420 msgr2=0x7fe188040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.322+0000 7fe1a49d4700 1 --2- 172.21.15.2:0/3168880509 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe18803e420 0x7fe188040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:56.881 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.322+0000 7fe1a49d4700 1 -- 172.21.15.2:0/3168880509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a0108920 msgr2=0x7fe1a019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:22:56.882 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.322+0000 7fe1a49d4700 1 --2- 172.21.15.2:0/3168880509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a0108920 0x7fe1a019cda0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe190009fb0 tx=0x7fe190011890 comp rx=0 tx=0).stop 2024-05-14T09:22:56.882 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.323+0000 7fe1a49d4700 1 -- 172.21.15.2:0/3168880509 shutdown_connections 2024-05-14T09:22:56.882 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.323+0000 7fe1a49d4700 1 --2- 172.21.15.2:0/3168880509 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe18803e420 0x7fe188040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:56.882 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.323+0000 7fe1a49d4700 1 --2- 172.21.15.2:0/3168880509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a0108920 0x7fe1a019cda0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:22:56.882 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.323+0000 7fe1a49d4700 1 -- 172.21.15.2:0/3168880509 >> 172.21.15.2:0/3168880509 conn(0x7fe1a0103fc0 msgr2=0x7fe1a0104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:22:56.882 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.324+0000 7fe1a49d4700 1 -- 172.21.15.2:0/3168880509 shutdown_connections 2024-05-14T09:22:56.882 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:56.324+0000 7fe1a49d4700 1 -- 172.21.15.2:0/3168880509 wait complete. 2024-05-14T09:22:56.882 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for the mgr to restart... 2024-05-14T09:22:56.882 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr epoch 5... 2024-05-14T09:22:57.222 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:22:56 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3168880509' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-14T09:22:57.222 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:57.214+0000 7f442fdf1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T09:22:57.546 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:57.402+0000 7f442fdf1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T09:22:57.888 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:57.582+0000 7f442fdf1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T09:22:57.888 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:57.888+0000 7f442fdf1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T09:22:58.171 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:57.986+0000 7f442fdf1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T09:22:58.671 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:58 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:58.199+0000 7f442fdf1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T09:22:58.671 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:58 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:58.402+0000 7f442fdf1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T09:22:59.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:59 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:59.037+0000 7f442fdf1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T09:22:59.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:59 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:59.271+0000 7f442fdf1200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T09:23:00.171 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:59 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:59.719+0000 7f442fdf1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T09:23:00.171 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:22:59 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:22:59.822+0000 7f442fdf1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T09:23:00.589 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:00.212+0000 7f442fdf1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T09:23:00.589 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:00.589+0000 7f442fdf1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T09:23:00.921 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:00.688+0000 7f442fdf1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T09:23:01.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:00.947+0000 7f442fdf1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T09:23:01.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:01 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:01.088+0000 7f442fdf1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T09:23:01.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:01 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:01.190+0000 7f442fdf1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T09:23:01.921 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:01 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:01.532+0000 7f442fdf1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T09:23:01.921 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:01 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:01.630+0000 7f442fdf1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T09:23:02.549 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:02 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:02.268+0000 7f442fdf1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T09:23:02.549 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:02 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:02.495+0000 7f442fdf1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T09:23:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:02 smithi002 ceph-mon[31557]: Active manager daemon x restarted 2024-05-14T09:23:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:02 smithi002 ceph-mon[31557]: Activating manager daemon x 2024-05-14T09:23:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:02 smithi002 ceph-mon[31557]: osdmap e2: 0 total, 0 up, 0 in 2024-05-14T09:23:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:02 smithi002 ceph-mon[31557]: mgrmap e6: x(active, starting, since 0.00376941s) 2024-05-14T09:23:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:02 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:23:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:02 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T09:23:02.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:02 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T09:23:02.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:02 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T09:23:02.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:02 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T09:23:02.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:02 smithi002 ceph-mon[31557]: Manager daemon x is now available 2024-05-14T09:23:02.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:02 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' 2024-05-14T09:23:02.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:02 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' 2024-05-14T09:23:02.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:02 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:23:02.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:02 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:23:03.842 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:03 smithi002 ceph-mon[31557]: Found migration_current of "None". Setting to last migration. 2024-05-14T09:23:03.842 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:03 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T09:23:03.842 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:03 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T09:23:03.842 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:03 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' 2024-05-14T09:23:03.842 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:03 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' 2024-05-14T09:23:03.843 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:03 smithi002 ceph-mon[31557]: mgrmap e7: x(active, since 1.00692s) 2024-05-14T09:23:04.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-05-14T09:23:04.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-14T09:23:04.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-14T09:23:04.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-05-14T09:23:04.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.338+0000 7fe163520700 1 Processor -- start 2024-05-14T09:23:04.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.339+0000 7fe163520700 1 -- start start 2024-05-14T09:23:04.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.339+0000 7fe163520700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe15c108960 0x7fe15c108d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:04.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.339+0000 7fe163520700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c109360 con 0x7fe15c108960 2024-05-14T09:23:04.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.339+0000 7fe1612bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe15c108960 0x7fe15c108d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:04.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.339+0000 7fe1612bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe15c108960 0x7fe15c108d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37904/0 (socket says 172.21.15.2:37904) 2024-05-14T09:23:04.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.339+0000 7fe1612bc700 1 -- 172.21.15.2:0/2604386696 learned_addr learned my addr 172.21.15.2:0/2604386696 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:04.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.339+0000 7fe1612bc700 1 -- 172.21.15.2:0/2604386696 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15c109b30 con 0x7fe15c108960 2024-05-14T09:23:04.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.340+0000 7fe1612bc700 1 --2- 172.21.15.2:0/2604386696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe15c108960 0x7fe15c108d70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe1500098a0 tx=0x7fe150009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f4c63a4632fb99 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:04.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.340+0000 7fe14ffff700 1 -- 172.21.15.2:0/2604386696 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe150004160 con 0x7fe15c108960 2024-05-14T09:23:04.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.340+0000 7fe14ffff700 1 -- 172.21.15.2:0/2604386696 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe150004300 con 0x7fe15c108960 2024-05-14T09:23:04.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.340+0000 7fe14ffff700 1 -- 172.21.15.2:0/2604386696 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe150011af0 con 0x7fe15c108960 2024-05-14T09:23:04.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.341+0000 7fe163520700 1 -- 172.21.15.2:0/2604386696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe15c108960 msgr2=0x7fe15c108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:04.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.341+0000 7fe163520700 1 --2- 172.21.15.2:0/2604386696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe15c108960 0x7fe15c108d70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe1500098a0 tx=0x7fe150009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:04.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.341+0000 7fe163520700 1 -- 172.21.15.2:0/2604386696 shutdown_connections 2024-05-14T09:23:04.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.341+0000 7fe163520700 1 --2- 172.21.15.2:0/2604386696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe15c108960 0x7fe15c108d70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:04.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.341+0000 7fe163520700 1 -- 172.21.15.2:0/2604386696 >> 172.21.15.2:0/2604386696 conn(0x7fe15c103fe0 msgr2=0x7fe15c106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:04.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.341+0000 7fe163520700 1 -- 172.21.15.2:0/2604386696 shutdown_connections 2024-05-14T09:23:04.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.341+0000 7fe163520700 1 -- 172.21.15.2:0/2604386696 wait complete. 2024-05-14T09:23:04.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.342+0000 7fe163520700 1 Processor -- start 2024-05-14T09:23:04.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.342+0000 7fe163520700 1 -- start start 2024-05-14T09:23:04.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.342+0000 7fe163520700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe15c108960 0x7fe15c19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:04.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.342+0000 7fe163520700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c10cee0 con 0x7fe15c108960 2024-05-14T09:23:04.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.342+0000 7fe1612bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe15c108960 0x7fe15c19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:04.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.342+0000 7fe1612bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe15c108960 0x7fe15c19df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37914/0 (socket says 172.21.15.2:37914) 2024-05-14T09:23:04.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.342+0000 7fe1612bc700 1 -- 172.21.15.2:0/2234086110 learned_addr learned my addr 172.21.15.2:0/2234086110 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:04.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.342+0000 7fe1612bc700 1 -- 172.21.15.2:0/2234086110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15c19e440 con 0x7fe15c108960 2024-05-14T09:23:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.343+0000 7fe1612bc700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe15c108960 0x7fe15c19df00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe1500090e0 tx=0x7fe150011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.343+0000 7fe14e7fc700 1 -- 172.21.15.2:0/2234086110 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1500039e0 con 0x7fe15c108960 2024-05-14T09:23:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.343+0000 7fe14e7fc700 1 -- 172.21.15.2:0/2234086110 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe150003b80 con 0x7fe15c108960 2024-05-14T09:23:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.343+0000 7fe163520700 1 -- 172.21.15.2:0/2234086110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15c19e6d0 con 0x7fe15c108960 2024-05-14T09:23:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.343+0000 7fe14e7fc700 1 -- 172.21.15.2:0/2234086110 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe150024690 con 0x7fe15c108960 2024-05-14T09:23:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.343+0000 7fe163520700 1 -- 172.21.15.2:0/2234086110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe15c1a12e0 con 0x7fe15c108960 2024-05-14T09:23:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.344+0000 7fe14e7fc700 1 -- 172.21.15.2:0/2234086110 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fe150022020 con 0x7fe15c108960 2024-05-14T09:23:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.345+0000 7fe14e7fc700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe14803e4c0 0x7fe1480409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.345+0000 7fe14e7fc700 1 -- 172.21.15.2:0/2234086110 --> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe1480410e0 con 0x7fe14803e4c0 2024-05-14T09:23:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.345+0000 7fe160abb700 1 -- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe14803e4c0 msgr2=0x7fe1480409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3593583257 2024-05-14T09:23:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.345+0000 7fe14e7fc700 1 -- 172.21.15.2:0/2234086110 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe150053350 con 0x7fe15c108960 2024-05-14T09:23:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.345+0000 7fe160abb700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe14803e4c0 0x7fe1480409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T09:23:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.546+0000 7fe160abb700 1 -- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe14803e4c0 msgr2=0x7fe1480409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3593583257 2024-05-14T09:23:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.546+0000 7fe160abb700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe14803e4c0 0x7fe1480409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T09:23:04.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.947+0000 7fe160abb700 1 -- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe14803e4c0 msgr2=0x7fe1480409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3593583257 2024-05-14T09:23:04.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:57.947+0000 7fe160abb700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe14803e4c0 0x7fe1480409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-14T09:23:04.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:58.748+0000 7fe160abb700 1 -- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe14803e4c0 msgr2=0x7fe1480409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3593583257 2024-05-14T09:23:04.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:22:58.748+0000 7fe160abb700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe14803e4c0 0x7fe1480409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-14T09:23:04.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:00.349+0000 7fe160abb700 1 -- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe14803e4c0 msgr2=0x7fe1480409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3593583257 2024-05-14T09:23:04.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:00.349+0000 7fe160abb700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe14803e4c0 0x7fe1480409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-14T09:23:04.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:02.499+0000 7fe14e7fc700 1 -- 172.21.15.2:0/2234086110 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7fe15002ccd0 con 0x7fe15c108960 2024-05-14T09:23:04.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:02.500+0000 7fe14e7fc700 1 -- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe14803e4c0 msgr2=0x7fe1480409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T09:23:04.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:02.500+0000 7fe14e7fc700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe14803e4c0 0x7fe1480409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:04.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.501+0000 7fe14e7fc700 1 -- 172.21.15.2:0/2234086110 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fe150055420 con 0x7fe15c108960 2024-05-14T09:23:04.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.501+0000 7fe14e7fc700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fe148041e00 0x7fe148044220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:04.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.501+0000 7fe14e7fc700 1 -- 172.21.15.2:0/2234086110 --> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe1480410e0 con 0x7fe148041e00 2024-05-14T09:23:04.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.502+0000 7fe160abb700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fe148041e00 0x7fe148044220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:04.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.503+0000 7fe160abb700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fe148041e00 0x7fe148044220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe158003a10 tx=0x7fe1580092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:04.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.503+0000 7fe14e7fc700 1 -- 172.21.15.2:0/2234086110 <== mgr.14120 v2:172.21.15.2:6800/1423742578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fe1480410e0 con 0x7fe148041e00 2024-05-14T09:23:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.512+0000 7fe163520700 1 -- 172.21.15.2:0/2234086110 --> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fe15c064e20 con 0x7fe148041e00 2024-05-14T09:23:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.512+0000 7fe14e7fc700 1 -- 172.21.15.2:0/2234086110 <== mgr.14120 v2:172.21.15.2:6800/1423742578 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fe15c064e20 con 0x7fe148041e00 2024-05-14T09:23:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.513+0000 7fe163520700 1 -- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fe148041e00 msgr2=0x7fe148044220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.513+0000 7fe163520700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fe148041e00 0x7fe148044220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe158003a10 tx=0x7fe1580092b0 comp rx=0 tx=0).stop 2024-05-14T09:23:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.513+0000 7fe163520700 1 -- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe15c108960 msgr2=0x7fe15c19df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.513+0000 7fe163520700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe15c108960 0x7fe15c19df00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe1500090e0 tx=0x7fe150011940 comp rx=0 tx=0).stop 2024-05-14T09:23:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.513+0000 7fe163520700 1 -- 172.21.15.2:0/2234086110 shutdown_connections 2024-05-14T09:23:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.513+0000 7fe163520700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fe148041e00 0x7fe148044220 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.513+0000 7fe163520700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe15c108960 0x7fe15c19df00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.513+0000 7fe163520700 1 --2- 172.21.15.2:0/2234086110 >> [v2:172.21.15.2:6800/3593583257,v1:172.21.15.2:6801/3593583257] conn(0x7fe14803e4c0 0x7fe1480409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.513+0000 7fe163520700 1 -- 172.21.15.2:0/2234086110 >> 172.21.15.2:0/2234086110 conn(0x7fe15c103fe0 msgr2=0x7fe15c1944b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.513+0000 7fe163520700 1 -- 172.21.15.2:0/2234086110 shutdown_connections 2024-05-14T09:23:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:03.513+0000 7fe163520700 1 -- 172.21.15.2:0/2234086110 wait complete. 2024-05-14T09:23:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:mgr epoch 5 is available 2024-05-14T09:23:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:Setting orchestrator backend to cephadm... 2024-05-14T09:23:04.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:04 smithi002 ceph-mon[31557]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-14T09:23:04.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:04 smithi002 ceph-mon[31557]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-14T09:23:04.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:04 smithi002 ceph-mon[31557]: [14/May/2024:09:23:03] ENGINE Bus STARTING 2024-05-14T09:23:04.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:04 smithi002 ceph-mon[31557]: [14/May/2024:09:23:03] ENGINE Serving on http://172.21.15.2:8765 2024-05-14T09:23:04.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:04 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:23:05.448 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.621+0000 7f202ecee700 1 Processor -- start 2024-05-14T09:23:05.448 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.621+0000 7f202ecee700 1 -- start start 2024-05-14T09:23:05.448 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.621+0000 7f202ecee700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f202810a500 0x7f202810c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:05.448 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.621+0000 7f202ecee700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f202807d720 con 0x7f202810a500 2024-05-14T09:23:05.448 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.621+0000 7f202ca8a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f202810a500 0x7f202810c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:05.448 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.621+0000 7f202ca8a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f202810a500 0x7f202810c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50624/0 (socket says 172.21.15.2:50624) 2024-05-14T09:23:05.448 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.622+0000 7f202ca8a700 1 -- 172.21.15.2:0/3521207609 learned_addr learned my addr 172.21.15.2:0/3521207609 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.622+0000 7f202ca8a700 1 -- 172.21.15.2:0/3521207609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202810ce60 con 0x7f202810a500 2024-05-14T09:23:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.622+0000 7f202ca8a700 1 --2- 172.21.15.2:0/3521207609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f202810a500 0x7f202810c920 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f201c0098a0 tx=0x7f201c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6c3034a7f275c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.622+0000 7f20277fe700 1 -- 172.21.15.2:0/3521207609 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f201c004160 con 0x7f202810a500 2024-05-14T09:23:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.622+0000 7f20277fe700 1 -- 172.21.15.2:0/3521207609 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f201c004300 con 0x7f202810a500 2024-05-14T09:23:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.622+0000 7f20277fe700 1 -- 172.21.15.2:0/3521207609 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f201c011af0 con 0x7f202810a500 2024-05-14T09:23:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.623+0000 7f202ecee700 1 -- 172.21.15.2:0/3521207609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f202810a500 msgr2=0x7f202810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.623+0000 7f202ecee700 1 --2- 172.21.15.2:0/3521207609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f202810a500 0x7f202810c920 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f201c0098a0 tx=0x7f201c009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.623+0000 7f202ecee700 1 -- 172.21.15.2:0/3521207609 shutdown_connections 2024-05-14T09:23:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.623+0000 7f202ecee700 1 --2- 172.21.15.2:0/3521207609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f202810a500 0x7f202810c920 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.623+0000 7f202ecee700 1 -- 172.21.15.2:0/3521207609 >> 172.21.15.2:0/3521207609 conn(0x7f2028103fc0 msgr2=0x7f2028106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.623+0000 7f202ecee700 1 -- 172.21.15.2:0/3521207609 shutdown_connections 2024-05-14T09:23:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.623+0000 7f202ecee700 1 -- 172.21.15.2:0/3521207609 wait complete. 2024-05-14T09:23:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.624+0000 7f202ecee700 1 Processor -- start 2024-05-14T09:23:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.624+0000 7f202ecee700 1 -- start start 2024-05-14T09:23:05.449 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.624+0000 7f202ecee700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f202810a500 0x7f202819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.624+0000 7f202ecee700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f202810de20 con 0x7f202810a500 2024-05-14T09:23:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.624+0000 7f202ca8a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f202810a500 0x7f202819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.624+0000 7f202ca8a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f202810a500 0x7f202819de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50628/0 (socket says 172.21.15.2:50628) 2024-05-14T09:23:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.624+0000 7f202ca8a700 1 -- 172.21.15.2:0/3537402971 learned_addr learned my addr 172.21.15.2:0/3537402971 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.624+0000 7f202ca8a700 1 -- 172.21.15.2:0/3537402971 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202819e390 con 0x7f202810a500 2024-05-14T09:23:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.625+0000 7f202ca8a700 1 --2- 172.21.15.2:0/3537402971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f202810a500 0x7f202819de50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f201c0117b0 tx=0x7f201c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.625+0000 7f2025ffb700 1 -- 172.21.15.2:0/3537402971 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f201c003af0 con 0x7f202810a500 2024-05-14T09:23:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.625+0000 7f202ecee700 1 -- 172.21.15.2:0/3537402971 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202819e620 con 0x7f202810a500 2024-05-14T09:23:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.625+0000 7f2025ffb700 1 -- 172.21.15.2:0/3537402971 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f201c020070 con 0x7f202810a500 2024-05-14T09:23:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.625+0000 7f2025ffb700 1 -- 172.21.15.2:0/3537402971 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f201c019630 con 0x7f202810a500 2024-05-14T09:23:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.625+0000 7f202ecee700 1 -- 172.21.15.2:0/3537402971 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20281a1230 con 0x7f202810a500 2024-05-14T09:23:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.626+0000 7f2025ffb700 1 -- 172.21.15.2:0/3537402971 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f201c0198f0 con 0x7f202810a500 2024-05-14T09:23:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.626+0000 7f2025ffb700 1 --2- 172.21.15.2:0/3537402971 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f201803e470 0x7f2018040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:05.450 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.626+0000 7f2025ffb700 1 -- 172.21.15.2:0/3537402971 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f201c054a10 con 0x7f202810a500 2024-05-14T09:23:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.626+0000 7f202ecee700 1 -- 172.21.15.2:0/3537402971 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2028064e20 con 0x7f202810a500 2024-05-14T09:23:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.626+0000 7f2027fff700 1 --2- 172.21.15.2:0/3537402971 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f201803e470 0x7f2018040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.627+0000 7f2027fff700 1 --2- 172.21.15.2:0/3537402971 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f201803e470 0x7f2018040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2014006fd0 tx=0x7f2014006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.632+0000 7f2025ffb700 1 -- 172.21.15.2:0/3537402971 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f201c016260 con 0x7f202810a500 2024-05-14T09:23:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.869+0000 7f202ecee700 1 -- 172.21.15.2:0/3537402971 --> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f2028105070 con 0x7f201803e470 2024-05-14T09:23:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.874+0000 7f2025ffb700 1 -- 172.21.15.2:0/3537402971 <== mgr.14120 v2:172.21.15.2:6800/1423742578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2028105070 con 0x7f201803e470 2024-05-14T09:23:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.877+0000 7f202ecee700 1 -- 172.21.15.2:0/3537402971 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f201803e470 msgr2=0x7f2018040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.877+0000 7f202ecee700 1 --2- 172.21.15.2:0/3537402971 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f201803e470 0x7f2018040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2014006fd0 tx=0x7f2014006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.877+0000 7f202ecee700 1 -- 172.21.15.2:0/3537402971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f202810a500 msgr2=0x7f202819de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.877+0000 7f202ecee700 1 --2- 172.21.15.2:0/3537402971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f202810a500 0x7f202819de50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f201c0117b0 tx=0x7f201c011b70 comp rx=0 tx=0).stop 2024-05-14T09:23:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.877+0000 7f202ecee700 1 -- 172.21.15.2:0/3537402971 shutdown_connections 2024-05-14T09:23:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.877+0000 7f202ecee700 1 --2- 172.21.15.2:0/3537402971 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f201803e470 0x7f2018040980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.877+0000 7f202ecee700 1 --2- 172.21.15.2:0/3537402971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f202810a500 0x7f202819de50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.877+0000 7f202ecee700 1 -- 172.21.15.2:0/3537402971 >> 172.21.15.2:0/3537402971 conn(0x7f2028103fc0 msgr2=0x7f20281048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:05.451 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.877+0000 7f202ecee700 1 -- 172.21.15.2:0/3537402971 shutdown_connections 2024-05-14T09:23:05.452 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:04.877+0000 7f202ecee700 1 -- 172.21.15.2:0/3537402971 wait complete. 2024-05-14T09:23:05.737 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:05 smithi002 ceph-mon[31557]: [14/May/2024:09:23:04] ENGINE Serving on https://172.21.15.2:7150 2024-05-14T09:23:05.737 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:05 smithi002 ceph-mon[31557]: [14/May/2024:09:23:04] ENGINE Bus STARTED 2024-05-14T09:23:05.737 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:05 smithi002 ceph-mon[31557]: mgrmap e8: x(active, since 2s) 2024-05-14T09:23:05.737 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:05 smithi002 ceph-mon[31557]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:23:05.737 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:05 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' 2024-05-14T09:23:05.737 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:05 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:23:06.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-14T09:23:06.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.850+0000 7f5e17724700 1 Processor -- start 2024-05-14T09:23:06.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.851+0000 7f5e17724700 1 -- start start 2024-05-14T09:23:06.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.851+0000 7f5e17724700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1010a560 0x7f5e1010c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:06.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.851+0000 7f5e17724700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1007d6c0 con 0x7f5e1010a560 2024-05-14T09:23:06.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.851+0000 7f5e154c0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1010a560 0x7f5e1010c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:06.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.851+0000 7f5e154c0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1010a560 0x7f5e1010c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50640/0 (socket says 172.21.15.2:50640) 2024-05-14T09:23:06.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.851+0000 7f5e154c0700 1 -- 172.21.15.2:0/662616861 learned_addr learned my addr 172.21.15.2:0/662616861 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:06.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.851+0000 7f5e154c0700 1 -- 172.21.15.2:0/662616861 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1010cec0 con 0x7f5e1010a560 2024-05-14T09:23:06.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.852+0000 7f5e154c0700 1 --2- 172.21.15.2:0/662616861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1010a560 0x7f5e1010c980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0098a0 tx=0x7f5e0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4e57db039bce40 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:06.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.852+0000 7f5e07fff700 1 -- 172.21.15.2:0/662616861 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e0c004160 con 0x7f5e1010a560 2024-05-14T09:23:06.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.852+0000 7f5e07fff700 1 -- 172.21.15.2:0/662616861 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5e0c004300 con 0x7f5e1010a560 2024-05-14T09:23:06.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.852+0000 7f5e07fff700 1 -- 172.21.15.2:0/662616861 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e0c011af0 con 0x7f5e1010a560 2024-05-14T09:23:06.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.852+0000 7f5e17724700 1 -- 172.21.15.2:0/662616861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1010a560 msgr2=0x7f5e1010c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:06.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.852+0000 7f5e17724700 1 --2- 172.21.15.2:0/662616861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1010a560 0x7f5e1010c980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0098a0 tx=0x7f5e0c009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:06.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.853+0000 7f5e17724700 1 -- 172.21.15.2:0/662616861 shutdown_connections 2024-05-14T09:23:06.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.853+0000 7f5e17724700 1 --2- 172.21.15.2:0/662616861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1010a560 0x7f5e1010c980 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:06.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.853+0000 7f5e17724700 1 -- 172.21.15.2:0/662616861 >> 172.21.15.2:0/662616861 conn(0x7f5e10103fc0 msgr2=0x7f5e10106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:06.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.853+0000 7f5e17724700 1 -- 172.21.15.2:0/662616861 shutdown_connections 2024-05-14T09:23:06.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.853+0000 7f5e17724700 1 -- 172.21.15.2:0/662616861 wait complete. 2024-05-14T09:23:06.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.853+0000 7f5e17724700 1 Processor -- start 2024-05-14T09:23:06.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.853+0000 7f5e17724700 1 -- start start 2024-05-14T09:23:06.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.854+0000 7f5e17724700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1010a560 0x7f5e1019de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:06.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.854+0000 7f5e17724700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1010de20 con 0x7f5e1010a560 2024-05-14T09:23:06.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.854+0000 7f5e154c0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1010a560 0x7f5e1019de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:06.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.854+0000 7f5e154c0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1010a560 0x7f5e1019de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50650/0 (socket says 172.21.15.2:50650) 2024-05-14T09:23:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.854+0000 7f5e154c0700 1 -- 172.21.15.2:0/2559184879 learned_addr learned my addr 172.21.15.2:0/2559184879 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.854+0000 7f5e154c0700 1 -- 172.21.15.2:0/2559184879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1019e370 con 0x7f5e1010a560 2024-05-14T09:23:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.854+0000 7f5e154c0700 1 --2- 172.21.15.2:0/2559184879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1010a560 0x7f5e1019de30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0117b0 tx=0x7f5e0c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.854+0000 7f5e067fc700 1 -- 172.21.15.2:0/2559184879 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e0c003af0 con 0x7f5e1010a560 2024-05-14T09:23:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.854+0000 7f5e067fc700 1 -- 172.21.15.2:0/2559184879 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5e0c020070 con 0x7f5e1010a560 2024-05-14T09:23:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.854+0000 7f5e17724700 1 -- 172.21.15.2:0/2559184879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1019e600 con 0x7f5e1010a560 2024-05-14T09:23:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.854+0000 7f5e067fc700 1 -- 172.21.15.2:0/2559184879 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e0c019630 con 0x7f5e1010a560 2024-05-14T09:23:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.854+0000 7f5e17724700 1 -- 172.21.15.2:0/2559184879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e101a1210 con 0x7f5e1010a560 2024-05-14T09:23:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.855+0000 7f5e17724700 1 -- 172.21.15.2:0/2559184879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e10064e20 con 0x7f5e1010a560 2024-05-14T09:23:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.855+0000 7f5e067fc700 1 -- 172.21.15.2:0/2559184879 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5e0c0198f0 con 0x7f5e1010a560 2024-05-14T09:23:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.856+0000 7f5e067fc700 1 --2- 172.21.15.2:0/2559184879 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f5dfc03e470 0x7f5dfc040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.856+0000 7f5e067fc700 1 -- 172.21.15.2:0/2559184879 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5e0c053b40 con 0x7f5e1010a560 2024-05-14T09:23:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.856+0000 7f5e14cbf700 1 --2- 172.21.15.2:0/2559184879 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f5dfc03e470 0x7f5dfc040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.856+0000 7f5e14cbf700 1 --2- 172.21.15.2:0/2559184879 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f5dfc03e470 0x7f5dfc040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5e00006fd0 tx=0x7f5e00006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:06.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:05.861+0000 7f5e067fc700 1 -- 172.21.15.2:0/2559184879 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5e0c016260 con 0x7f5e1010a560 2024-05-14T09:23:06.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:06.079+0000 7f5e17724700 1 -- 172.21.15.2:0/2559184879 --> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f5e10105000 con 0x7f5dfc03e470 2024-05-14T09:23:06.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:06.080+0000 7f5e067fc700 1 -- 172.21.15.2:0/2559184879 <== mgr.14120 v2:172.21.15.2:6800/1423742578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f5e10105000 con 0x7f5dfc03e470 2024-05-14T09:23:06.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:06.082+0000 7f5e17724700 1 -- 172.21.15.2:0/2559184879 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f5dfc03e470 msgr2=0x7f5dfc040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:06.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:06.082+0000 7f5e17724700 1 --2- 172.21.15.2:0/2559184879 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f5dfc03e470 0x7f5dfc040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5e00006fd0 tx=0x7f5e00006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:06.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:06.082+0000 7f5e17724700 1 -- 172.21.15.2:0/2559184879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1010a560 msgr2=0x7f5e1019de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:06.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:06.082+0000 7f5e17724700 1 --2- 172.21.15.2:0/2559184879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1010a560 0x7f5e1019de30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0117b0 tx=0x7f5e0c011b70 comp rx=0 tx=0).stop 2024-05-14T09:23:06.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:06.082+0000 7f5e17724700 1 -- 172.21.15.2:0/2559184879 shutdown_connections 2024-05-14T09:23:06.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:06.082+0000 7f5e17724700 1 --2- 172.21.15.2:0/2559184879 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f5dfc03e470 0x7f5dfc040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:06.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:06.082+0000 7f5e17724700 1 --2- 172.21.15.2:0/2559184879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1010a560 0x7f5e1019de30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:06.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:06.082+0000 7f5e17724700 1 -- 172.21.15.2:0/2559184879 >> 172.21.15.2:0/2559184879 conn(0x7f5e10103fc0 msgr2=0x7f5e101048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:06.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:06.082+0000 7f5e17724700 1 -- 172.21.15.2:0/2559184879 shutdown_connections 2024-05-14T09:23:06.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:06.082+0000 7f5e17724700 1 -- 172.21.15.2:0/2559184879 wait complete. 2024-05-14T09:23:06.643 INFO:teuthology.orchestra.run.smithi002.stdout:Generating ssh key... 2024-05-14T09:23:07.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:07 smithi002 ceph-mon[31557]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:23:07.574 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: Generating public/private rsa key pair. 2024-05-14T09:23:07.574 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: Your identification has been saved in /tmp/tmppvrvbhve/key. 2024-05-14T09:23:07.574 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: Your public key has been saved in /tmp/tmppvrvbhve/key.pub. 2024-05-14T09:23:07.574 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: The key fingerprint is: 2024-05-14T09:23:07.574 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: SHA256:8IHuRCG750T9w+TaQBx6B3XPLMPBhE1MO2A5z2ylgCE ceph-50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:23:07.574 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: The key's randomart image is: 2024-05-14T09:23:07.574 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: +---[RSA 3072]----+ 2024-05-14T09:23:07.574 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: | . E +++X* | 2024-05-14T09:23:07.575 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: | o B.+=++*. | 2024-05-14T09:23:07.575 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: | . * * o**o+ | 2024-05-14T09:23:07.575 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: | = = B *+ | 2024-05-14T09:23:07.575 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: | . = S =. | 2024-05-14T09:23:07.575 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: | * + . | 2024-05-14T09:23:07.575 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: | o . . | 2024-05-14T09:23:07.575 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: | | 2024-05-14T09:23:07.575 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: | | 2024-05-14T09:23:07.575 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: +----[SHA256]-----+ 2024-05-14T09:23:08.129 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.088+0000 7fecd0197700 1 Processor -- start 2024-05-14T09:23:08.130 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.088+0000 7fecd0197700 1 -- start start 2024-05-14T09:23:08.130 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.089+0000 7fecd0197700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc8107d50 0x7fecc8108160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:08.130 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.089+0000 7fecd0197700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fecc8108730 con 0x7fecc8107d50 2024-05-14T09:23:08.130 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.089+0000 7feccdf33700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc8107d50 0x7fecc8108160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:08.130 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.089+0000 7feccdf33700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc8107d50 0x7fecc8108160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50662/0 (socket says 172.21.15.2:50662) 2024-05-14T09:23:08.130 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.089+0000 7feccdf33700 1 -- 172.21.15.2:0/1924862979 learned_addr learned my addr 172.21.15.2:0/1924862979 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:08.130 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.089+0000 7feccdf33700 1 -- 172.21.15.2:0/1924862979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc810ad50 con 0x7fecc8107d50 2024-05-14T09:23:08.130 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.089+0000 7feccdf33700 1 --2- 172.21.15.2:0/1924862979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc8107d50 0x7fecc8108160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fecc40098a0 tx=0x7fecc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f7fc415dc52d2b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:08.130 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.089+0000 7fecccf31700 1 -- 172.21.15.2:0/1924862979 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecc4004160 con 0x7fecc8107d50 2024-05-14T09:23:08.130 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.090+0000 7fecccf31700 1 -- 172.21.15.2:0/1924862979 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fecc4004300 con 0x7fecc8107d50 2024-05-14T09:23:08.130 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.090+0000 7fecccf31700 1 -- 172.21.15.2:0/1924862979 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecc4011af0 con 0x7fecc8107d50 2024-05-14T09:23:08.130 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.090+0000 7fecd0197700 1 -- 172.21.15.2:0/1924862979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc8107d50 msgr2=0x7fecc8108160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:08.130 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.090+0000 7fecd0197700 1 --2- 172.21.15.2:0/1924862979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc8107d50 0x7fecc8108160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fecc40098a0 tx=0x7fecc4009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:08.130 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.090+0000 7fecd0197700 1 -- 172.21.15.2:0/1924862979 shutdown_connections 2024-05-14T09:23:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.090+0000 7fecd0197700 1 --2- 172.21.15.2:0/1924862979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc8107d50 0x7fecc8108160 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.090+0000 7fecd0197700 1 -- 172.21.15.2:0/1924862979 >> 172.21.15.2:0/1924862979 conn(0x7fecc8103760 msgr2=0x7fecc8105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.090+0000 7fecd0197700 1 -- 172.21.15.2:0/1924862979 shutdown_connections 2024-05-14T09:23:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.091+0000 7fecd0197700 1 -- 172.21.15.2:0/1924862979 wait complete. 2024-05-14T09:23:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.091+0000 7fecd0197700 1 Processor -- start 2024-05-14T09:23:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.091+0000 7fecd0197700 1 -- start start 2024-05-14T09:23:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.091+0000 7fecd0197700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc8107d50 0x7fecc819e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.091+0000 7fecd0197700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fecc810df80 con 0x7fecc8107d50 2024-05-14T09:23:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.091+0000 7feccdf33700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc8107d50 0x7fecc819e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.091+0000 7feccdf33700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc8107d50 0x7fecc819e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50676/0 (socket says 172.21.15.2:50676) 2024-05-14T09:23:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.091+0000 7feccdf33700 1 -- 172.21.15.2:0/2446733827 learned_addr learned my addr 172.21.15.2:0/2446733827 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.092+0000 7feccdf33700 1 -- 172.21.15.2:0/2446733827 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc819e670 con 0x7fecc8107d50 2024-05-14T09:23:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.092+0000 7feccdf33700 1 --2- 172.21.15.2:0/2446733827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc8107d50 0x7fecc819e130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fecc4011670 tx=0x7fecc4004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.092+0000 7fecbaffd700 1 -- 172.21.15.2:0/2446733827 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecc40036a0 con 0x7fecc8107d50 2024-05-14T09:23:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.092+0000 7fecbaffd700 1 -- 172.21.15.2:0/2446733827 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fecc401a430 con 0x7fecc8107d50 2024-05-14T09:23:08.132 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.092+0000 7fecd0197700 1 -- 172.21.15.2:0/2446733827 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc819e900 con 0x7fecc8107d50 2024-05-14T09:23:08.132 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.092+0000 7fecbaffd700 1 -- 172.21.15.2:0/2446733827 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecc4019550 con 0x7fecc8107d50 2024-05-14T09:23:08.132 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.092+0000 7fecd0197700 1 -- 172.21.15.2:0/2446733827 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc81a1510 con 0x7fecc8107d50 2024-05-14T09:23:08.132 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.093+0000 7fecd0197700 1 -- 172.21.15.2:0/2446733827 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc8064e20 con 0x7fecc8107d50 2024-05-14T09:23:08.132 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.093+0000 7fecbaffd700 1 -- 172.21.15.2:0/2446733827 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fecc40197c0 con 0x7fecc8107d50 2024-05-14T09:23:08.132 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.094+0000 7fecbaffd700 1 --2- 172.21.15.2:0/2446733827 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fecb403e470 0x7fecb4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:08.132 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.094+0000 7fecbaffd700 1 -- 172.21.15.2:0/2446733827 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fecc40537a0 con 0x7fecc8107d50 2024-05-14T09:23:08.132 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.094+0000 7feccd732700 1 --2- 172.21.15.2:0/2446733827 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fecb403e470 0x7fecb4040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:08.132 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.094+0000 7feccd732700 1 --2- 172.21.15.2:0/2446733827 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fecb403e470 0x7fecb4040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fecbc006fd0 tx=0x7fecbc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:08.132 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.099+0000 7fecbaffd700 1 -- 172.21.15.2:0/2446733827 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fecc4023d40 con 0x7fecc8107d50 2024-05-14T09:23:08.132 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.323+0000 7fecd0197700 1 -- 172.21.15.2:0/2446733827 --> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fecc8104880 con 0x7fecb403e470 2024-05-14T09:23:08.132 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.559+0000 7fecbaffd700 1 -- 172.21.15.2:0/2446733827 <== mgr.14120 v2:172.21.15.2:6800/1423742578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fecc8104880 con 0x7fecb403e470 2024-05-14T09:23:08.132 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.561+0000 7fecd0197700 1 -- 172.21.15.2:0/2446733827 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fecb403e470 msgr2=0x7fecb4040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:08.132 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.561+0000 7fecd0197700 1 --2- 172.21.15.2:0/2446733827 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fecb403e470 0x7fecb4040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fecbc006fd0 tx=0x7fecbc006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:08.133 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.561+0000 7fecd0197700 1 -- 172.21.15.2:0/2446733827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc8107d50 msgr2=0x7fecc819e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:08.133 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.561+0000 7fecd0197700 1 --2- 172.21.15.2:0/2446733827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc8107d50 0x7fecc819e130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fecc4011670 tx=0x7fecc4004280 comp rx=0 tx=0).stop 2024-05-14T09:23:08.133 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.561+0000 7fecd0197700 1 -- 172.21.15.2:0/2446733827 shutdown_connections 2024-05-14T09:23:08.133 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.561+0000 7fecd0197700 1 --2- 172.21.15.2:0/2446733827 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fecb403e470 0x7fecb4040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:08.133 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.561+0000 7fecd0197700 1 --2- 172.21.15.2:0/2446733827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc8107d50 0x7fecc819e130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:08.133 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.561+0000 7fecd0197700 1 -- 172.21.15.2:0/2446733827 >> 172.21.15.2:0/2446733827 conn(0x7fecc8103760 msgr2=0x7fecc81040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:08.133 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.561+0000 7fecd0197700 1 -- 172.21.15.2:0/2446733827 shutdown_connections 2024-05-14T09:23:08.133 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:07.561+0000 7fecd0197700 1 -- 172.21.15.2:0/2446733827 wait complete. 2024-05-14T09:23:08.851 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:08 smithi002 ceph-mon[31557]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:23:08.851 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:08 smithi002 ceph-mon[31557]: Generating ssh key... 2024-05-14T09:23:08.851 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:08 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' 2024-05-14T09:23:08.851 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:08 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' 2024-05-14T09:23:09.428 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHo3ka/VaO+EpZB3Hau8HJ1zdu6l4P3i0W54/WnuUjqLq/m8Zyh4jysm7SuzoJAPmbo2+tLgOZpNzDhpG61rDOMsy5LYPgdx5HzL5KD3hqLLgHCbQe3JTTo83zEmdnHAbZOAqsRhX+0plI3gvXtoh7uas3MnMFvzupIsCwwnfTYLPvuJSULpQnvohOKLNoc+ouJUjQLpwbsV+0h587L6oi+7TxIVpDhk7J7nqwl6HBGQg9CARTkb+bC825EnA4F5CEHxLPf/x9OjXyqMc1LNav7cRkZDPChzSJ2Q+8HeUwdv93SIMLvvfvR+WxCHrA/uy6Ck7Q21FOxT19C9YVykh0jjokA+x3N8k+WbiU/C3yXJMzTOfd4iPoc24ipV4FltecPXS6FJcEbVJFt+cbMlQiUO9pukrfnIDyM5bsrKYcxT2P2t7VGxmSAI5SB/olUy9bAxYeuPlH9AJvI9qqWYHslrY2beC3uj45S7vjSZh6ZzOIansSeG6WdN+rQULYqZ8= ceph-50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:23:09.428 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.619+0000 7fac414dc700 1 Processor -- start 2024-05-14T09:23:09.428 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.619+0000 7fac414dc700 1 -- start start 2024-05-14T09:23:09.428 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.619+0000 7fac414dc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c07d710 0x7fac3c10c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:09.428 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.619+0000 7fac414dc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c10cc60 con 0x7fac3c07d710 2024-05-14T09:23:09.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.619+0000 7fac3affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c07d710 0x7fac3c10c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:09.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.619+0000 7fac3affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c07d710 0x7fac3c10c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41466/0 (socket says 172.21.15.2:41466) 2024-05-14T09:23:09.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.619+0000 7fac3affd700 1 -- 172.21.15.2:0/1900758242 learned_addr learned my addr 172.21.15.2:0/1900758242 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:09.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.620+0000 7fac3affd700 1 -- 172.21.15.2:0/1900758242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c10cde0 con 0x7fac3c07d710 2024-05-14T09:23:09.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.620+0000 7fac3affd700 1 --2- 172.21.15.2:0/1900758242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c07d710 0x7fac3c10c650 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0098a0 tx=0x7fac2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c778c41278f48a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:09.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.620+0000 7fac39ffb700 1 -- 172.21.15.2:0/1900758242 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac2c004160 con 0x7fac3c07d710 2024-05-14T09:23:09.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.620+0000 7fac39ffb700 1 -- 172.21.15.2:0/1900758242 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac2c004300 con 0x7fac3c07d710 2024-05-14T09:23:09.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.620+0000 7fac39ffb700 1 -- 172.21.15.2:0/1900758242 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac2c011af0 con 0x7fac3c07d710 2024-05-14T09:23:09.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.621+0000 7fac414dc700 1 -- 172.21.15.2:0/1900758242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c07d710 msgr2=0x7fac3c10c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:09.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.621+0000 7fac414dc700 1 --2- 172.21.15.2:0/1900758242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c07d710 0x7fac3c10c650 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0098a0 tx=0x7fac2c009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:09.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.621+0000 7fac414dc700 1 -- 172.21.15.2:0/1900758242 shutdown_connections 2024-05-14T09:23:09.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.621+0000 7fac414dc700 1 --2- 172.21.15.2:0/1900758242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c07d710 0x7fac3c10c650 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:09.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.621+0000 7fac414dc700 1 -- 172.21.15.2:0/1900758242 >> 172.21.15.2:0/1900758242 conn(0x7fac3c104040 msgr2=0x7fac3c106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:09.429 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.621+0000 7fac414dc700 1 -- 172.21.15.2:0/1900758242 shutdown_connections 2024-05-14T09:23:09.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.621+0000 7fac414dc700 1 -- 172.21.15.2:0/1900758242 wait complete. 2024-05-14T09:23:09.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.622+0000 7fac414dc700 1 Processor -- start 2024-05-14T09:23:09.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.622+0000 7fac414dc700 1 -- start start 2024-05-14T09:23:09.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.622+0000 7fac414dc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c07d710 0x7fac3c19e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:09.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.622+0000 7fac414dc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c10ddf0 con 0x7fac3c07d710 2024-05-14T09:23:09.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.622+0000 7fac3affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c07d710 0x7fac3c19e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:09.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.622+0000 7fac3affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c07d710 0x7fac3c19e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41470/0 (socket says 172.21.15.2:41470) 2024-05-14T09:23:09.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.622+0000 7fac3affd700 1 -- 172.21.15.2:0/1261094393 learned_addr learned my addr 172.21.15.2:0/1261094393 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:09.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.623+0000 7fac3affd700 1 -- 172.21.15.2:0/1261094393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c19e6c0 con 0x7fac3c07d710 2024-05-14T09:23:09.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.623+0000 7fac3affd700 1 --2- 172.21.15.2:0/1261094393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c07d710 0x7fac3c19e180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0117b0 tx=0x7fac2c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:09.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.623+0000 7fac23fff700 1 -- 172.21.15.2:0/1261094393 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac2c0039c0 con 0x7fac3c07d710 2024-05-14T09:23:09.430 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.623+0000 7fac23fff700 1 -- 172.21.15.2:0/1261094393 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac2c003b60 con 0x7fac3c07d710 2024-05-14T09:23:09.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.623+0000 7fac414dc700 1 -- 172.21.15.2:0/1261094393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac3c19e950 con 0x7fac3c07d710 2024-05-14T09:23:09.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.623+0000 7fac23fff700 1 -- 172.21.15.2:0/1261094393 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac2c0196a0 con 0x7fac3c07d710 2024-05-14T09:23:09.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.623+0000 7fac414dc700 1 -- 172.21.15.2:0/1261094393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac3c1a1560 con 0x7fac3c07d710 2024-05-14T09:23:09.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.624+0000 7fac414dc700 1 -- 172.21.15.2:0/1261094393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac3c064e20 con 0x7fac3c07d710 2024-05-14T09:23:09.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.624+0000 7fac23fff700 1 -- 172.21.15.2:0/1261094393 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fac2c019960 con 0x7fac3c07d710 2024-05-14T09:23:09.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.624+0000 7fac23fff700 1 --2- 172.21.15.2:0/1261094393 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fac2403e470 0x7fac24040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:09.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.625+0000 7fac23fff700 1 -- 172.21.15.2:0/1261094393 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fac2c053b50 con 0x7fac3c07d710 2024-05-14T09:23:09.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.625+0000 7fac3a7fc700 1 --2- 172.21.15.2:0/1261094393 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fac2403e470 0x7fac24040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:09.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.625+0000 7fac3a7fc700 1 --2- 172.21.15.2:0/1261094393 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fac2403e470 0x7fac24040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fac30006fd0 tx=0x7fac30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:09.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.630+0000 7fac23fff700 1 -- 172.21.15.2:0/1261094393 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac2c017e90 con 0x7fac3c07d710 2024-05-14T09:23:09.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.851+0000 7fac414dc700 1 -- 172.21.15.2:0/1261094393 --> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fac3c02e660 con 0x7fac2403e470 2024-05-14T09:23:09.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.851+0000 7fac23fff700 1 -- 172.21.15.2:0/1261094393 <== mgr.14120 v2:172.21.15.2:6800/1423742578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fac3c02e660 con 0x7fac2403e470 2024-05-14T09:23:09.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.854+0000 7fac414dc700 1 -- 172.21.15.2:0/1261094393 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fac2403e470 msgr2=0x7fac24040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:09.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.854+0000 7fac414dc700 1 --2- 172.21.15.2:0/1261094393 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fac2403e470 0x7fac24040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fac30006fd0 tx=0x7fac30006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:09.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.854+0000 7fac414dc700 1 -- 172.21.15.2:0/1261094393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c07d710 msgr2=0x7fac3c19e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:09.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.854+0000 7fac414dc700 1 --2- 172.21.15.2:0/1261094393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c07d710 0x7fac3c19e180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0117b0 tx=0x7fac2c011b70 comp rx=0 tx=0).stop 2024-05-14T09:23:09.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.854+0000 7fac414dc700 1 -- 172.21.15.2:0/1261094393 shutdown_connections 2024-05-14T09:23:09.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.854+0000 7fac414dc700 1 --2- 172.21.15.2:0/1261094393 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fac2403e470 0x7fac24040980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:09.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.854+0000 7fac414dc700 1 --2- 172.21.15.2:0/1261094393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c07d710 0x7fac3c19e180 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:09.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.854+0000 7fac414dc700 1 -- 172.21.15.2:0/1261094393 >> 172.21.15.2:0/1261094393 conn(0x7fac3c104040 msgr2=0x7fac3c105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:09.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.854+0000 7fac414dc700 1 -- 172.21.15.2:0/1261094393 shutdown_connections 2024-05-14T09:23:09.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:08.854+0000 7fac414dc700 1 -- 172.21.15.2:0/1261094393 wait complete. 2024-05-14T09:23:09.432 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-14T09:23:09.432 INFO:teuthology.orchestra.run.smithi002.stdout:Adding key to root@localhost authorized_keys... 2024-05-14T09:23:09.432 INFO:teuthology.orchestra.run.smithi002.stdout:Adding host smithi002... 2024-05-14T09:23:09.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:09 smithi002 ceph-mon[31557]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:23:10.660 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:10 smithi002 ceph-mon[31557]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi002", "addr": "172.21.15.2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:23:11.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:11 smithi002 ceph-mon[31557]: Deploying cephadm binary to smithi002 2024-05-14T09:23:14.927 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Added host 'smithi002' with addr '172.21.15.2' 2024-05-14T09:23:14.927 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.813+0000 7faaf883d700 1 Processor -- start 2024-05-14T09:23:14.927 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.813+0000 7faaf883d700 1 -- start start 2024-05-14T09:23:14.927 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.813+0000 7faaf883d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf410a540 0x7faaf410c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:14.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.813+0000 7faaf883d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faaf407d6e0 con 0x7faaf410a540 2024-05-14T09:23:14.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.813+0000 7faaf259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf410a540 0x7faaf410c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:14.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.813+0000 7faaf259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf410a540 0x7faaf410c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41482/0 (socket says 172.21.15.2:41482) 2024-05-14T09:23:14.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.813+0000 7faaf259c700 1 -- 172.21.15.2:0/3404289358 learned_addr learned my addr 172.21.15.2:0/3404289358 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:14.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.813+0000 7faaf259c700 1 -- 172.21.15.2:0/3404289358 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf410cea0 con 0x7faaf410a540 2024-05-14T09:23:14.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.814+0000 7faaf259c700 1 --2- 172.21.15.2:0/3404289358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf410a540 0x7faaf410c960 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faae40098a0 tx=0x7faae4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead1d016be256d01 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:14.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.814+0000 7faaf1d9b700 1 -- 172.21.15.2:0/3404289358 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faae4004160 con 0x7faaf410a540 2024-05-14T09:23:14.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.814+0000 7faaf1d9b700 1 -- 172.21.15.2:0/3404289358 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faae4004300 con 0x7faaf410a540 2024-05-14T09:23:14.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.814+0000 7faaf1d9b700 1 -- 172.21.15.2:0/3404289358 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faae4011af0 con 0x7faaf410a540 2024-05-14T09:23:14.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.815+0000 7faaf883d700 1 -- 172.21.15.2:0/3404289358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf410a540 msgr2=0x7faaf410c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:14.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.815+0000 7faaf883d700 1 --2- 172.21.15.2:0/3404289358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf410a540 0x7faaf410c960 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faae40098a0 tx=0x7faae4009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:14.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.815+0000 7faaf883d700 1 -- 172.21.15.2:0/3404289358 shutdown_connections 2024-05-14T09:23:14.928 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.815+0000 7faaf883d700 1 --2- 172.21.15.2:0/3404289358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf410a540 0x7faaf410c960 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:14.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.815+0000 7faaf883d700 1 -- 172.21.15.2:0/3404289358 >> 172.21.15.2:0/3404289358 conn(0x7faaf4103fc0 msgr2=0x7faaf4106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:14.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.815+0000 7faaf883d700 1 -- 172.21.15.2:0/3404289358 shutdown_connections 2024-05-14T09:23:14.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.815+0000 7faaf883d700 1 -- 172.21.15.2:0/3404289358 wait complete. 2024-05-14T09:23:14.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.815+0000 7faaf883d700 1 Processor -- start 2024-05-14T09:23:14.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.816+0000 7faaf883d700 1 -- start start 2024-05-14T09:23:14.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.816+0000 7faaf883d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf410a540 0x7faaf419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:14.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.816+0000 7faaf883d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faaf410de00 con 0x7faaf410a540 2024-05-14T09:23:14.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.816+0000 7faaf259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf410a540 0x7faaf419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:14.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.816+0000 7faaf259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf410a540 0x7faaf419de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41494/0 (socket says 172.21.15.2:41494) 2024-05-14T09:23:14.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.816+0000 7faaf259c700 1 -- 172.21.15.2:0/4096272577 learned_addr learned my addr 172.21.15.2:0/4096272577 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:14.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.816+0000 7faaf259c700 1 -- 172.21.15.2:0/4096272577 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf419e370 con 0x7faaf410a540 2024-05-14T09:23:14.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.816+0000 7faaf259c700 1 --2- 172.21.15.2:0/4096272577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf410a540 0x7faaf419de30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faae40117b0 tx=0x7faae4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:14.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.817+0000 7faaeb7fe700 1 -- 172.21.15.2:0/4096272577 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faae4003af0 con 0x7faaf410a540 2024-05-14T09:23:14.929 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.817+0000 7faaf883d700 1 -- 172.21.15.2:0/4096272577 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf419e600 con 0x7faaf410a540 2024-05-14T09:23:14.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.817+0000 7faaeb7fe700 1 -- 172.21.15.2:0/4096272577 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faae4020070 con 0x7faaf410a540 2024-05-14T09:23:14.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.817+0000 7faaeb7fe700 1 -- 172.21.15.2:0/4096272577 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faae4019630 con 0x7faaf410a540 2024-05-14T09:23:14.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.817+0000 7faaf883d700 1 -- 172.21.15.2:0/4096272577 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf41a1210 con 0x7faaf410a540 2024-05-14T09:23:14.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.818+0000 7faaf883d700 1 -- 172.21.15.2:0/4096272577 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf4064e20 con 0x7faaf410a540 2024-05-14T09:23:14.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.818+0000 7faaeb7fe700 1 -- 172.21.15.2:0/4096272577 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7faae40198f0 con 0x7faaf410a540 2024-05-14T09:23:14.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.818+0000 7faaeb7fe700 1 --2- 172.21.15.2:0/4096272577 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7faae003e470 0x7faae0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:14.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.818+0000 7faaeb7fe700 1 -- 172.21.15.2:0/4096272577 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faae4053d60 con 0x7faaf410a540 2024-05-14T09:23:14.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.818+0000 7faaebfff700 1 --2- 172.21.15.2:0/4096272577 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7faae003e470 0x7faae0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:14.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.819+0000 7faaebfff700 1 --2- 172.21.15.2:0/4096272577 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7faae003e470 0x7faae0040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faadc006fd0 tx=0x7faadc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:14.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:09.823+0000 7faaeb7fe700 1 -- 172.21.15.2:0/4096272577 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faae4016260 con 0x7faaf410a540 2024-05-14T09:23:14.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:10.041+0000 7faaf883d700 1 -- 172.21.15.2:0/4096272577 --> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi002", "addr": "172.21.15.2", "target": ["mon-mgr", ""]}) -- 0x7faaf4105000 con 0x7faae003e470 2024-05-14T09:23:14.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:14.316+0000 7faaeb7fe700 1 -- 172.21.15.2:0/4096272577 <== mgr.14120 v2:172.21.15.2:6800/1423742578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7faaf4105000 con 0x7faae003e470 2024-05-14T09:23:14.930 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:14.319+0000 7faaf883d700 1 -- 172.21.15.2:0/4096272577 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7faae003e470 msgr2=0x7faae0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:14.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:14.319+0000 7faaf883d700 1 --2- 172.21.15.2:0/4096272577 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7faae003e470 0x7faae0040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faadc006fd0 tx=0x7faadc006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:14.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:14.319+0000 7faaf883d700 1 -- 172.21.15.2:0/4096272577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf410a540 msgr2=0x7faaf419de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:14.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:14.319+0000 7faaf883d700 1 --2- 172.21.15.2:0/4096272577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf410a540 0x7faaf419de30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faae40117b0 tx=0x7faae4011b70 comp rx=0 tx=0).stop 2024-05-14T09:23:14.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:14.319+0000 7faaf883d700 1 -- 172.21.15.2:0/4096272577 shutdown_connections 2024-05-14T09:23:14.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:14.319+0000 7faaf883d700 1 --2- 172.21.15.2:0/4096272577 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7faae003e470 0x7faae0040980 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:14.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:14.319+0000 7faaf883d700 1 --2- 172.21.15.2:0/4096272577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf410a540 0x7faaf419de30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:14.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:14.319+0000 7faaf883d700 1 -- 172.21.15.2:0/4096272577 >> 172.21.15.2:0/4096272577 conn(0x7faaf4103fc0 msgr2=0x7faaf41048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:14.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:14.319+0000 7faaf883d700 1 -- 172.21.15.2:0/4096272577 shutdown_connections 2024-05-14T09:23:14.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:14.319+0000 7faaf883d700 1 -- 172.21.15.2:0/4096272577 wait complete. 2024-05-14T09:23:14.931 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying unmanaged mon service... 2024-05-14T09:23:15.623 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:15 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' 2024-05-14T09:23:15.623 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:15 smithi002 ceph-mon[31557]: Added host smithi002 2024-05-14T09:23:15.623 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:15 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:23:16.353 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-14T09:23:16.353 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.382+0000 7f4089ca5700 1 Processor -- start 2024-05-14T09:23:16.353 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.382+0000 7f4089ca5700 1 -- start start 2024-05-14T09:23:16.353 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.382+0000 7f4089ca5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084109f00 0x7f408410a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:16.353 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.382+0000 7f4089ca5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f408410a900 con 0x7f4084109f00 2024-05-14T09:23:16.353 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.382+0000 7f40837fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084109f00 0x7f408410a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:16.353 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.382+0000 7f40837fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084109f00 0x7f408410a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41502/0 (socket says 172.21.15.2:41502) 2024-05-14T09:23:16.353 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.382+0000 7f40837fe700 1 -- 172.21.15.2:0/1850866434 learned_addr learned my addr 172.21.15.2:0/1850866434 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:16.353 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.382+0000 7f40837fe700 1 -- 172.21.15.2:0/1850866434 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4084108c20 con 0x7f4084109f00 2024-05-14T09:23:16.353 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.383+0000 7f40837fe700 1 --2- 172.21.15.2:0/1850866434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084109f00 0x7f408410a310 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f40740098a0 tx=0x7f4074009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec3921d184ecd3e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:16.354 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.383+0000 7f40827fc700 1 -- 172.21.15.2:0/1850866434 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4074004160 con 0x7f4084109f00 2024-05-14T09:23:16.354 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.383+0000 7f40827fc700 1 -- 172.21.15.2:0/1850866434 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4074004300 con 0x7f4084109f00 2024-05-14T09:23:16.354 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.383+0000 7f40827fc700 1 -- 172.21.15.2:0/1850866434 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4074011af0 con 0x7f4084109f00 2024-05-14T09:23:16.354 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.384+0000 7f4089ca5700 1 -- 172.21.15.2:0/1850866434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084109f00 msgr2=0x7f408410a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:16.354 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.384+0000 7f4089ca5700 1 --2- 172.21.15.2:0/1850866434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084109f00 0x7f408410a310 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f40740098a0 tx=0x7f4074009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:16.354 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.384+0000 7f4089ca5700 1 -- 172.21.15.2:0/1850866434 shutdown_connections 2024-05-14T09:23:16.354 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.384+0000 7f4089ca5700 1 --2- 172.21.15.2:0/1850866434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084109f00 0x7f408410a310 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:16.354 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.384+0000 7f4089ca5700 1 -- 172.21.15.2:0/1850866434 >> 172.21.15.2:0/1850866434 conn(0x7f4084103760 msgr2=0x7f4084105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:16.354 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.384+0000 7f4089ca5700 1 -- 172.21.15.2:0/1850866434 shutdown_connections 2024-05-14T09:23:16.354 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.384+0000 7f4089ca5700 1 -- 172.21.15.2:0/1850866434 wait complete. 2024-05-14T09:23:16.354 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.385+0000 7f4089ca5700 1 Processor -- start 2024-05-14T09:23:16.354 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.385+0000 7f4089ca5700 1 -- start start 2024-05-14T09:23:16.354 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.385+0000 7f4089ca5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084109f00 0x7f408419df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:16.354 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.385+0000 7f4089ca5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f408410ce60 con 0x7f4084109f00 2024-05-14T09:23:16.355 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.385+0000 7f40837fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084109f00 0x7f408419df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:16.355 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.385+0000 7f40837fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084109f00 0x7f408419df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41512/0 (socket says 172.21.15.2:41512) 2024-05-14T09:23:16.355 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.385+0000 7f40837fe700 1 -- 172.21.15.2:0/1388324515 learned_addr learned my addr 172.21.15.2:0/1388324515 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:16.355 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.385+0000 7f40837fe700 1 -- 172.21.15.2:0/1388324515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408419e4a0 con 0x7f4084109f00 2024-05-14T09:23:16.355 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.385+0000 7f40837fe700 1 --2- 172.21.15.2:0/1388324515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084109f00 0x7f408419df60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4074009fb0 tx=0x7f4074011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:16.355 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.386+0000 7f4080ff9700 1 -- 172.21.15.2:0/1388324515 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40740036a0 con 0x7f4084109f00 2024-05-14T09:23:16.355 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.386+0000 7f4080ff9700 1 -- 172.21.15.2:0/1388324515 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4074024440 con 0x7f4084109f00 2024-05-14T09:23:16.355 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.386+0000 7f4089ca5700 1 -- 172.21.15.2:0/1388324515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408419e730 con 0x7f4084109f00 2024-05-14T09:23:16.355 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.386+0000 7f4080ff9700 1 -- 172.21.15.2:0/1388324515 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f407401a5c0 con 0x7f4084109f00 2024-05-14T09:23:16.355 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.386+0000 7f4089ca5700 1 -- 172.21.15.2:0/1388324515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40841a1340 con 0x7f4084109f00 2024-05-14T09:23:16.355 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.387+0000 7f4089ca5700 1 -- 172.21.15.2:0/1388324515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4084064e20 con 0x7f4084109f00 2024-05-14T09:23:16.355 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.387+0000 7f4080ff9700 1 -- 172.21.15.2:0/1388324515 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f407401a830 con 0x7f4084109f00 2024-05-14T09:23:16.355 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.387+0000 7f4080ff9700 1 --2- 172.21.15.2:0/1388324515 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f406c03e470 0x7f406c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:16.355 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.387+0000 7f4080ff9700 1 -- 172.21.15.2:0/1388324515 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4074053730 con 0x7f4084109f00 2024-05-14T09:23:16.356 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.387+0000 7f4082ffd700 1 --2- 172.21.15.2:0/1388324515 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f406c03e470 0x7f406c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:16.356 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.388+0000 7f4082ffd700 1 --2- 172.21.15.2:0/1388324515 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f406c03e470 0x7f406c040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4078006fd0 tx=0x7f4078006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:16.356 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.393+0000 7f4080ff9700 1 -- 172.21.15.2:0/1388324515 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4074017e90 con 0x7f4084109f00 2024-05-14T09:23:16.356 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.623+0000 7f4089ca5700 1 -- 172.21.15.2:0/1388324515 --> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4084063c00 con 0x7f406c03e470 2024-05-14T09:23:16.356 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.628+0000 7f4080ff9700 1 -- 172.21.15.2:0/1388324515 <== mgr.14120 v2:172.21.15.2:6800/1423742578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4084063c00 con 0x7f406c03e470 2024-05-14T09:23:16.356 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.630+0000 7f4089ca5700 1 -- 172.21.15.2:0/1388324515 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f406c03e470 msgr2=0x7f406c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:16.356 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.630+0000 7f4089ca5700 1 --2- 172.21.15.2:0/1388324515 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f406c03e470 0x7f406c040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4078006fd0 tx=0x7f4078006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:16.356 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.630+0000 7f4089ca5700 1 -- 172.21.15.2:0/1388324515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084109f00 msgr2=0x7f408419df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:16.356 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.630+0000 7f4089ca5700 1 --2- 172.21.15.2:0/1388324515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084109f00 0x7f408419df60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4074009fb0 tx=0x7f4074011890 comp rx=0 tx=0).stop 2024-05-14T09:23:16.356 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.630+0000 7f4089ca5700 1 -- 172.21.15.2:0/1388324515 shutdown_connections 2024-05-14T09:23:16.356 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.630+0000 7f4089ca5700 1 --2- 172.21.15.2:0/1388324515 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f406c03e470 0x7f406c040980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:16.356 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.630+0000 7f4089ca5700 1 --2- 172.21.15.2:0/1388324515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084109f00 0x7f408419df60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:16.356 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.630+0000 7f4089ca5700 1 -- 172.21.15.2:0/1388324515 >> 172.21.15.2:0/1388324515 conn(0x7f4084103760 msgr2=0x7f40841944e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:16.356 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.631+0000 7f4089ca5700 1 -- 172.21.15.2:0/1388324515 shutdown_connections 2024-05-14T09:23:16.357 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:15.631+0000 7f4089ca5700 1 -- 172.21.15.2:0/1388324515 wait complete. 2024-05-14T09:23:16.357 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying unmanaged mgr service... 2024-05-14T09:23:16.865 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:16 smithi002 ceph-mon[31557]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:23:16.865 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:16 smithi002 ceph-mon[31557]: Saving service mon spec with placement count:5 2024-05-14T09:23:16.865 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:16 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' 2024-05-14T09:23:17.529 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-14T09:23:17.529 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.732+0000 7f2c866b1700 1 Processor -- start 2024-05-14T09:23:17.529 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.732+0000 7f2c866b1700 1 -- start start 2024-05-14T09:23:17.529 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.732+0000 7f2c866b1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c8010a580 0x7f2c8010c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:17.529 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.732+0000 7f2c866b1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8007d6a0 con 0x7f2c8010a580 2024-05-14T09:23:17.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.732+0000 7f2c7ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c8010a580 0x7f2c8010c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:17.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.732+0000 7f2c7ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c8010a580 0x7f2c8010c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41528/0 (socket says 172.21.15.2:41528) 2024-05-14T09:23:17.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.732+0000 7f2c7ffff700 1 -- 172.21.15.2:0/701970242 learned_addr learned my addr 172.21.15.2:0/701970242 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:17.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.733+0000 7f2c7ffff700 1 -- 172.21.15.2:0/701970242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8010cee0 con 0x7f2c8010a580 2024-05-14T09:23:17.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.733+0000 7f2c7ffff700 1 --2- 172.21.15.2:0/701970242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c8010a580 0x7f2c8010c9a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2c700098a0 tx=0x7f2c70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb90c6da6d9eabf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:17.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.733+0000 7f2c7effd700 1 -- 172.21.15.2:0/701970242 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c70004160 con 0x7f2c8010a580 2024-05-14T09:23:17.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.733+0000 7f2c7effd700 1 -- 172.21.15.2:0/701970242 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2c70004300 con 0x7f2c8010a580 2024-05-14T09:23:17.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.733+0000 7f2c7effd700 1 -- 172.21.15.2:0/701970242 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c70011af0 con 0x7f2c8010a580 2024-05-14T09:23:17.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.734+0000 7f2c866b1700 1 -- 172.21.15.2:0/701970242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c8010a580 msgr2=0x7f2c8010c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:17.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.734+0000 7f2c866b1700 1 --2- 172.21.15.2:0/701970242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c8010a580 0x7f2c8010c9a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2c700098a0 tx=0x7f2c70009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:17.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.734+0000 7f2c866b1700 1 -- 172.21.15.2:0/701970242 shutdown_connections 2024-05-14T09:23:17.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.734+0000 7f2c866b1700 1 --2- 172.21.15.2:0/701970242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c8010a580 0x7f2c8010c9a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:17.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.734+0000 7f2c866b1700 1 -- 172.21.15.2:0/701970242 >> 172.21.15.2:0/701970242 conn(0x7f2c80103fc0 msgr2=0x7f2c80106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:17.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.734+0000 7f2c866b1700 1 -- 172.21.15.2:0/701970242 shutdown_connections 2024-05-14T09:23:17.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.734+0000 7f2c866b1700 1 -- 172.21.15.2:0/701970242 wait complete. 2024-05-14T09:23:17.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.735+0000 7f2c866b1700 1 Processor -- start 2024-05-14T09:23:17.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.735+0000 7f2c866b1700 1 -- start start 2024-05-14T09:23:17.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.735+0000 7f2c866b1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c8010a580 0x7f2c80109d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:17.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.735+0000 7f2c866b1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8010de40 con 0x7f2c8010a580 2024-05-14T09:23:17.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.735+0000 7f2c7ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c8010a580 0x7f2c80109d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:17.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.735+0000 7f2c7ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c8010a580 0x7f2c80109d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41536/0 (socket says 172.21.15.2:41536) 2024-05-14T09:23:17.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.735+0000 7f2c7ffff700 1 -- 172.21.15.2:0/2361606746 learned_addr learned my addr 172.21.15.2:0/2361606746 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:17.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.736+0000 7f2c7ffff700 1 -- 172.21.15.2:0/2361606746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8010a240 con 0x7f2c8010a580 2024-05-14T09:23:17.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.736+0000 7f2c7ffff700 1 --2- 172.21.15.2:0/2361606746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c8010a580 0x7f2c80109d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2c700117b0 tx=0x7f2c70011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:17.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.736+0000 7f2c7d7fa700 1 -- 172.21.15.2:0/2361606746 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c70003af0 con 0x7f2c8010a580 2024-05-14T09:23:17.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.736+0000 7f2c7d7fa700 1 -- 172.21.15.2:0/2361606746 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2c70020070 con 0x7f2c8010a580 2024-05-14T09:23:17.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.736+0000 7f2c866b1700 1 -- 172.21.15.2:0/2361606746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c80108490 con 0x7f2c8010a580 2024-05-14T09:23:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.736+0000 7f2c866b1700 1 -- 172.21.15.2:0/2361606746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c80108970 con 0x7f2c8010a580 2024-05-14T09:23:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.736+0000 7f2c7d7fa700 1 -- 172.21.15.2:0/2361606746 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c70019630 con 0x7f2c8010a580 2024-05-14T09:23:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.737+0000 7f2c7d7fa700 1 -- 172.21.15.2:0/2361606746 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2c700198a0 con 0x7f2c8010a580 2024-05-14T09:23:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.737+0000 7f2c866b1700 1 -- 172.21.15.2:0/2361606746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c80064e20 con 0x7f2c8010a580 2024-05-14T09:23:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.737+0000 7f2c7d7fa700 1 --2- 172.21.15.2:0/2361606746 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f2c6803e4c0 0x7f2c680409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.737+0000 7f2c7d7fa700 1 -- 172.21.15.2:0/2361606746 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2c70053ab0 con 0x7f2c8010a580 2024-05-14T09:23:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.737+0000 7f2c7f7fe700 1 --2- 172.21.15.2:0/2361606746 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f2c6803e4c0 0x7f2c680409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.738+0000 7f2c7f7fe700 1 --2- 172.21.15.2:0/2361606746 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f2c6803e4c0 0x7f2c680409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2c74006fd0 tx=0x7f2c74006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.745+0000 7f2c7d7fa700 1 -- 172.21.15.2:0/2361606746 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c70016260 con 0x7f2c8010a580 2024-05-14T09:23:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.978+0000 7f2c866b1700 1 -- 172.21.15.2:0/2361606746 --> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2c80109120 con 0x7f2c6803e4c0 2024-05-14T09:23:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.983+0000 7f2c7d7fa700 1 -- 172.21.15.2:0/2361606746 <== mgr.14120 v2:172.21.15.2:6800/1423742578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2c80109120 con 0x7f2c6803e4c0 2024-05-14T09:23:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.985+0000 7f2c866b1700 1 -- 172.21.15.2:0/2361606746 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f2c6803e4c0 msgr2=0x7f2c680409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.985+0000 7f2c866b1700 1 --2- 172.21.15.2:0/2361606746 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f2c6803e4c0 0x7f2c680409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2c74006fd0 tx=0x7f2c74006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.985+0000 7f2c866b1700 1 -- 172.21.15.2:0/2361606746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c8010a580 msgr2=0x7f2c80109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.985+0000 7f2c866b1700 1 --2- 172.21.15.2:0/2361606746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c8010a580 0x7f2c80109d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2c700117b0 tx=0x7f2c70011b70 comp rx=0 tx=0).stop 2024-05-14T09:23:17.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.986+0000 7f2c866b1700 1 -- 172.21.15.2:0/2361606746 shutdown_connections 2024-05-14T09:23:17.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.986+0000 7f2c866b1700 1 --2- 172.21.15.2:0/2361606746 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f2c6803e4c0 0x7f2c680409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:17.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.986+0000 7f2c866b1700 1 --2- 172.21.15.2:0/2361606746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c8010a580 0x7f2c80109d00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:17.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.986+0000 7f2c866b1700 1 -- 172.21.15.2:0/2361606746 >> 172.21.15.2:0/2361606746 conn(0x7f2c80103fc0 msgr2=0x7f2c801048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:17.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.986+0000 7f2c866b1700 1 -- 172.21.15.2:0/2361606746 shutdown_connections 2024-05-14T09:23:17.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:16.986+0000 7f2c866b1700 1 -- 172.21.15.2:0/2361606746 wait complete. 2024-05-14T09:23:18.099 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:17 smithi002 ceph-mon[31557]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:23:18.099 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:17 smithi002 ceph-mon[31557]: Saving service mgr spec with placement count:2 2024-05-14T09:23:18.099 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:17 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' 2024-05-14T09:23:19.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.135+0000 7fb380266700 1 Processor -- start 2024-05-14T09:23:19.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.135+0000 7fb380266700 1 -- start start 2024-05-14T09:23:19.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.135+0000 7fb380266700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb37810a500 0x7fb37810c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:19.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.135+0000 7fb380266700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb37807d720 con 0x7fb37810a500 2024-05-14T09:23:19.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.135+0000 7fb37e002700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb37810a500 0x7fb37810c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:19.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.136+0000 7fb37e002700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb37810a500 0x7fb37810c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52170/0 (socket says 172.21.15.2:52170) 2024-05-14T09:23:19.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.136+0000 7fb37e002700 1 -- 172.21.15.2:0/1715188925 learned_addr learned my addr 172.21.15.2:0/1715188925 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:19.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.136+0000 7fb37e002700 1 -- 172.21.15.2:0/1715188925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37810ce60 con 0x7fb37810a500 2024-05-14T09:23:19.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.136+0000 7fb37e002700 1 --2- 172.21.15.2:0/1715188925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb37810a500 0x7fb37810c920 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb3740098a0 tx=0x7fb374009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a636650fe3291e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:19.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.136+0000 7fb37d000700 1 -- 172.21.15.2:0/1715188925 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb374004160 con 0x7fb37810a500 2024-05-14T09:23:19.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.136+0000 7fb37d000700 1 -- 172.21.15.2:0/1715188925 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb374004300 con 0x7fb37810a500 2024-05-14T09:23:19.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.136+0000 7fb37d000700 1 -- 172.21.15.2:0/1715188925 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb374011af0 con 0x7fb37810a500 2024-05-14T09:23:19.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.137+0000 7fb380266700 1 -- 172.21.15.2:0/1715188925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb37810a500 msgr2=0x7fb37810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:19.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.137+0000 7fb380266700 1 --2- 172.21.15.2:0/1715188925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb37810a500 0x7fb37810c920 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb3740098a0 tx=0x7fb374009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:19.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.137+0000 7fb380266700 1 -- 172.21.15.2:0/1715188925 shutdown_connections 2024-05-14T09:23:19.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.137+0000 7fb380266700 1 --2- 172.21.15.2:0/1715188925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb37810a500 0x7fb37810c920 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:19.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.137+0000 7fb380266700 1 -- 172.21.15.2:0/1715188925 >> 172.21.15.2:0/1715188925 conn(0x7fb378103fc0 msgr2=0x7fb378106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:19.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.137+0000 7fb380266700 1 -- 172.21.15.2:0/1715188925 shutdown_connections 2024-05-14T09:23:19.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.137+0000 7fb380266700 1 -- 172.21.15.2:0/1715188925 wait complete. 2024-05-14T09:23:19.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.138+0000 7fb380266700 1 Processor -- start 2024-05-14T09:23:19.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.138+0000 7fb380266700 1 -- start start 2024-05-14T09:23:19.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.138+0000 7fb380266700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb37810a500 0x7fb3781a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:19.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.138+0000 7fb380266700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb37810de20 con 0x7fb37810a500 2024-05-14T09:23:19.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.138+0000 7fb37e002700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb37810a500 0x7fb3781a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:19.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.138+0000 7fb37e002700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb37810a500 0x7fb3781a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52186/0 (socket says 172.21.15.2:52186) 2024-05-14T09:23:19.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.138+0000 7fb37e002700 1 -- 172.21.15.2:0/1072732046 learned_addr learned my addr 172.21.15.2:0/1072732046 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:19.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.139+0000 7fb37e002700 1 -- 172.21.15.2:0/1072732046 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3781a05b0 con 0x7fb37810a500 2024-05-14T09:23:19.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.139+0000 7fb37e002700 1 --2- 172.21.15.2:0/1072732046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb37810a500 0x7fb3781a0070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb374009fb0 tx=0x7fb374011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:19.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.139+0000 7fb36effd700 1 -- 172.21.15.2:0/1072732046 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3740036a0 con 0x7fb37810a500 2024-05-14T09:23:19.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.139+0000 7fb36effd700 1 -- 172.21.15.2:0/1072732046 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb37401a430 con 0x7fb37810a500 2024-05-14T09:23:19.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.139+0000 7fb380266700 1 -- 172.21.15.2:0/1072732046 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3781a0840 con 0x7fb37810a500 2024-05-14T09:23:19.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.139+0000 7fb36effd700 1 -- 172.21.15.2:0/1072732046 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb374019550 con 0x7fb37810a500 2024-05-14T09:23:19.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.139+0000 7fb380266700 1 -- 172.21.15.2:0/1072732046 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3781a3450 con 0x7fb37810a500 2024-05-14T09:23:19.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.140+0000 7fb36effd700 1 -- 172.21.15.2:0/1072732046 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fb374019800 con 0x7fb37810a500 2024-05-14T09:23:19.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.140+0000 7fb380266700 1 -- 172.21.15.2:0/1072732046 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb378064e20 con 0x7fb37810a500 2024-05-14T09:23:19.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.141+0000 7fb36effd700 1 --2- 172.21.15.2:0/1072732046 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fb36403e470 0x7fb364040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:19.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.141+0000 7fb36effd700 1 -- 172.21.15.2:0/1072732046 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb374053850 con 0x7fb37810a500 2024-05-14T09:23:19.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.141+0000 7fb37d801700 1 --2- 172.21.15.2:0/1072732046 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fb36403e470 0x7fb364040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:19.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.141+0000 7fb37d801700 1 --2- 172.21.15.2:0/1072732046 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fb36403e470 0x7fb364040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb368006fd0 tx=0x7fb368006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:19.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.146+0000 7fb36effd700 1 -- 172.21.15.2:0/1072732046 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb374017e90 con 0x7fb37810a500 2024-05-14T09:23:19.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.359+0000 7fb380266700 1 -- 172.21.15.2:0/1072732046 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fb37802e8f0 con 0x7fb37810a500 2024-05-14T09:23:19.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.362+0000 7fb36effd700 1 -- 172.21.15.2:0/1072732046 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7fb374023390 con 0x7fb37810a500 2024-05-14T09:23:19.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.365+0000 7fb380266700 1 -- 172.21.15.2:0/1072732046 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fb36403e470 msgr2=0x7fb364040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:19.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.365+0000 7fb380266700 1 --2- 172.21.15.2:0/1072732046 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fb36403e470 0x7fb364040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb368006fd0 tx=0x7fb368006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:19.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.365+0000 7fb380266700 1 -- 172.21.15.2:0/1072732046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb37810a500 msgr2=0x7fb3781a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:19.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.365+0000 7fb380266700 1 --2- 172.21.15.2:0/1072732046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb37810a500 0x7fb3781a0070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb374009fb0 tx=0x7fb374011940 comp rx=0 tx=0).stop 2024-05-14T09:23:19.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.365+0000 7fb380266700 1 -- 172.21.15.2:0/1072732046 shutdown_connections 2024-05-14T09:23:19.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.365+0000 7fb380266700 1 --2- 172.21.15.2:0/1072732046 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7fb36403e470 0x7fb364040980 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:19.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.365+0000 7fb380266700 1 --2- 172.21.15.2:0/1072732046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb37810a500 0x7fb3781a0070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:19.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.365+0000 7fb380266700 1 -- 172.21.15.2:0/1072732046 >> 172.21.15.2:0/1072732046 conn(0x7fb378103fc0 msgr2=0x7fb378108430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:19.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.365+0000 7fb380266700 1 -- 172.21.15.2:0/1072732046 shutdown_connections 2024-05-14T09:23:19.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:18.365+0000 7fb380266700 1 -- 172.21.15.2:0/1072732046 wait complete. 2024-05-14T09:23:19.515 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:19 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1072732046' entity='client.admin' 2024-05-14T09:23:19.515 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:19 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' 2024-05-14T09:23:20.529 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:20 smithi002 ceph-mon[31557]: from='mgr.14120 172.21.15.2:0/839411151' entity='mgr.x' 2024-05-14T09:23:20.529 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:20 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3797300091' entity='client.admin' 2024-05-14T09:23:20.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.718+0000 7f062a711700 1 Processor -- start 2024-05-14T09:23:20.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.718+0000 7f062a711700 1 -- start start 2024-05-14T09:23:20.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.719+0000 7f062a711700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f062410a540 0x7f062410c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:20.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.719+0000 7f062a711700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f062407ce90 con 0x7f062410a540 2024-05-14T09:23:20.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.719+0000 7f0623fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f062410a540 0x7f062410c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:20.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.719+0000 7f0623fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f062410a540 0x7f062410c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52198/0 (socket says 172.21.15.2:52198) 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.719+0000 7f0623fff700 1 -- 172.21.15.2:0/557482588 learned_addr learned my addr 172.21.15.2:0/557482588 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.719+0000 7f0623fff700 1 -- 172.21.15.2:0/557482588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062410cea0 con 0x7f062410a540 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.719+0000 7f0623fff700 1 --2- 172.21.15.2:0/557482588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f062410a540 0x7f062410c960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06140098a0 tx=0x7f0614009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc035ca01162bc89 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.719+0000 7f0622ffd700 1 -- 172.21.15.2:0/557482588 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0614004160 con 0x7f062410a540 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.720+0000 7f0622ffd700 1 -- 172.21.15.2:0/557482588 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0614004300 con 0x7f062410a540 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.720+0000 7f0622ffd700 1 -- 172.21.15.2:0/557482588 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0614011a20 con 0x7f062410a540 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.720+0000 7f062a711700 1 -- 172.21.15.2:0/557482588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f062410a540 msgr2=0x7f062410c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.720+0000 7f062a711700 1 --2- 172.21.15.2:0/557482588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f062410a540 0x7f062410c960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06140098a0 tx=0x7f0614009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.720+0000 7f062a711700 1 -- 172.21.15.2:0/557482588 shutdown_connections 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.720+0000 7f062a711700 1 --2- 172.21.15.2:0/557482588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f062410a540 0x7f062410c960 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.720+0000 7f062a711700 1 -- 172.21.15.2:0/557482588 >> 172.21.15.2:0/557482588 conn(0x7f0624103fe0 msgr2=0x7f0624106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.721+0000 7f062a711700 1 -- 172.21.15.2:0/557482588 shutdown_connections 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.721+0000 7f062a711700 1 -- 172.21.15.2:0/557482588 wait complete. 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.721+0000 7f062a711700 1 Processor -- start 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.721+0000 7f062a711700 1 -- start start 2024-05-14T09:23:20.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.722+0000 7f062a711700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f062410a540 0x7f0624109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:20.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.722+0000 7f062a711700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f062410dea0 con 0x7f062410a540 2024-05-14T09:23:20.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.722+0000 7f0623fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f062410a540 0x7f0624109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:20.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.722+0000 7f0623fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f062410a540 0x7f0624109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52210/0 (socket says 172.21.15.2:52210) 2024-05-14T09:23:20.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.722+0000 7f0623fff700 1 -- 172.21.15.2:0/3797300091 learned_addr learned my addr 172.21.15.2:0/3797300091 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:20.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.722+0000 7f0623fff700 1 -- 172.21.15.2:0/3797300091 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062410a200 con 0x7f062410a540 2024-05-14T09:23:20.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.722+0000 7f0623fff700 1 --2- 172.21.15.2:0/3797300091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f062410a540 0x7f0624109cc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0614004660 tx=0x7f0614011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:20.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.722+0000 7f06217fa700 1 -- 172.21.15.2:0/3797300091 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0614003a60 con 0x7f062410a540 2024-05-14T09:23:20.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.723+0000 7f06217fa700 1 -- 172.21.15.2:0/3797300091 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061401b8f0 con 0x7f062410a540 2024-05-14T09:23:20.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.723+0000 7f062a711700 1 -- 172.21.15.2:0/3797300091 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0624108450 con 0x7f062410a540 2024-05-14T09:23:20.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.723+0000 7f06217fa700 1 -- 172.21.15.2:0/3797300091 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0614023890 con 0x7f062410a540 2024-05-14T09:23:20.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.723+0000 7f062a711700 1 -- 172.21.15.2:0/3797300091 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0624108930 con 0x7f062410a540 2024-05-14T09:23:20.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.724+0000 7f062a711700 1 -- 172.21.15.2:0/3797300091 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0624064e20 con 0x7f062410a540 2024-05-14T09:23:20.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.724+0000 7f06217fa700 1 -- 172.21.15.2:0/3797300091 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f061402c3f0 con 0x7f062410a540 2024-05-14T09:23:20.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.724+0000 7f06217fa700 1 --2- 172.21.15.2:0/3797300091 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f061003e420 0x7f0610040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:20.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.724+0000 7f06217fa700 1 -- 172.21.15.2:0/3797300091 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0614053f60 con 0x7f062410a540 2024-05-14T09:23:20.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.724+0000 7f06237fe700 1 --2- 172.21.15.2:0/3797300091 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f061003e420 0x7f0610040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:20.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.725+0000 7f06237fe700 1 --2- 172.21.15.2:0/3797300091 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f061003e420 0x7f0610040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f060c006fd0 tx=0x7f060c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:20.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.729+0000 7f06217fa700 1 -- 172.21.15.2:0/3797300091 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f061401f030 con 0x7f062410a540 2024-05-14T09:23:20.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.951+0000 7f062a711700 1 -- 172.21.15.2:0/3797300091 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f0624109090 con 0x7f062410a540 2024-05-14T09:23:20.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.955+0000 7f06217fa700 1 -- 172.21.15.2:0/3797300091 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f0614017e90 con 0x7f062410a540 2024-05-14T09:23:20.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.957+0000 7f062a711700 1 -- 172.21.15.2:0/3797300091 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f061003e420 msgr2=0x7f0610040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:20.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.957+0000 7f062a711700 1 --2- 172.21.15.2:0/3797300091 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f061003e420 0x7f0610040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f060c006fd0 tx=0x7f060c006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:20.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.957+0000 7f062a711700 1 -- 172.21.15.2:0/3797300091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f062410a540 msgr2=0x7f0624109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:20.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.957+0000 7f062a711700 1 --2- 172.21.15.2:0/3797300091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f062410a540 0x7f0624109cc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0614004660 tx=0x7f0614011aa0 comp rx=0 tx=0).stop 2024-05-14T09:23:20.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.958+0000 7f062a711700 1 -- 172.21.15.2:0/3797300091 shutdown_connections 2024-05-14T09:23:20.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.958+0000 7f062a711700 1 --2- 172.21.15.2:0/3797300091 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f061003e420 0x7f0610040930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:20.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.958+0000 7f062a711700 1 --2- 172.21.15.2:0/3797300091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f062410a540 0x7f0624109cc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:20.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.958+0000 7f062a711700 1 -- 172.21.15.2:0/3797300091 >> 172.21.15.2:0/3797300091 conn(0x7f0624103fe0 msgr2=0x7f0624104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:20.540 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.958+0000 7f062a711700 1 -- 172.21.15.2:0/3797300091 shutdown_connections 2024-05-14T09:23:20.540 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:19.958+0000 7f062a711700 1 -- 172.21.15.2:0/3797300091 wait complete. 2024-05-14T09:23:20.540 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling the dashboard module... 2024-05-14T09:23:21.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:21 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/546728498' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-14T09:23:21.671 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:21 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: ignoring --setuser ceph since I am not root 2024-05-14T09:23:21.671 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:21 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: ignoring --setgroup ceph since I am not root 2024-05-14T09:23:21.671 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:21 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:21.584+0000 7f1a42647200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T09:23:22.047 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:21 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:21.783+0000 7f1a42647200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T09:23:22.047 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:21 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:21.893+0000 7f1a42647200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T09:23:22.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.961+0000 7f7cb0b0d700 1 Processor -- start 2024-05-14T09:23:22.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.961+0000 7f7cb0b0d700 1 -- start start 2024-05-14T09:23:22.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.962+0000 7f7cb0b0d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac108980 0x7f7cac108d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:22.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.962+0000 7f7cb0b0d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac109380 con 0x7f7cac108980 2024-05-14T09:23:22.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.962+0000 7f7caa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac108980 0x7f7cac108d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:22.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.962+0000 7f7caa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac108980 0x7f7cac108d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52214/0 (socket says 172.21.15.2:52214) 2024-05-14T09:23:22.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.962+0000 7f7caa59c700 1 -- 172.21.15.2:0/1732908778 learned_addr learned my addr 172.21.15.2:0/1732908778 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:22.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.962+0000 7f7caa59c700 1 -- 172.21.15.2:0/1732908778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cac109b50 con 0x7f7cac108980 2024-05-14T09:23:22.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.962+0000 7f7caa59c700 1 --2- 172.21.15.2:0/1732908778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac108980 0x7f7cac108d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00bb40 tx=0x7f7c9c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abff52fd5f4bc8e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:22.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.962+0000 7f7ca959a700 1 -- 172.21.15.2:0/1732908778 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c9c00dbe0 con 0x7f7cac108980 2024-05-14T09:23:22.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.962+0000 7f7ca959a700 1 -- 172.21.15.2:0/1732908778 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c9c0044e0 con 0x7f7cac108980 2024-05-14T09:23:22.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.962+0000 7f7ca959a700 1 -- 172.21.15.2:0/1732908778 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c9c010470 con 0x7f7cac108980 2024-05-14T09:23:22.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.963+0000 7f7cb0b0d700 1 -- 172.21.15.2:0/1732908778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac108980 msgr2=0x7f7cac108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:22.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.963+0000 7f7cb0b0d700 1 --2- 172.21.15.2:0/1732908778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac108980 0x7f7cac108d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00bb40 tx=0x7f7c9c00be50 comp rx=0 tx=0).stop 2024-05-14T09:23:22.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.963+0000 7f7cb0b0d700 1 -- 172.21.15.2:0/1732908778 shutdown_connections 2024-05-14T09:23:22.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.963+0000 7f7cb0b0d700 1 --2- 172.21.15.2:0/1732908778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac108980 0x7f7cac108d90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:22.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.963+0000 7f7cb0b0d700 1 -- 172.21.15.2:0/1732908778 >> 172.21.15.2:0/1732908778 conn(0x7f7cac104040 msgr2=0x7f7cac106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:22.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.963+0000 7f7cb0b0d700 1 -- 172.21.15.2:0/1732908778 shutdown_connections 2024-05-14T09:23:22.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.963+0000 7f7cb0b0d700 1 -- 172.21.15.2:0/1732908778 wait complete. 2024-05-14T09:23:22.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.964+0000 7f7cb0b0d700 1 Processor -- start 2024-05-14T09:23:22.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.964+0000 7f7cb0b0d700 1 -- start start 2024-05-14T09:23:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.964+0000 7f7cb0b0d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac108980 0x7f7cac1a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.964+0000 7f7cb0b0d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac07cb90 con 0x7f7cac108980 2024-05-14T09:23:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.965+0000 7f7caa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac108980 0x7f7cac1a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.965+0000 7f7caa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac108980 0x7f7cac1a0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52226/0 (socket says 172.21.15.2:52226) 2024-05-14T09:23:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.965+0000 7f7caa59c700 1 -- 172.21.15.2:0/546728498 learned_addr learned my addr 172.21.15.2:0/546728498 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.965+0000 7f7caa59c700 1 -- 172.21.15.2:0/546728498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cac1a08b0 con 0x7f7cac108980 2024-05-14T09:23:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.965+0000 7f7caa59c700 1 --2- 172.21.15.2:0/546728498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac108980 0x7f7cac1a0370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c003c30 tx=0x7f7c9c003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.965+0000 7f7c9b7fe700 1 -- 172.21.15.2:0/546728498 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c9c010720 con 0x7f7cac108980 2024-05-14T09:23:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.965+0000 7f7c9b7fe700 1 -- 172.21.15.2:0/546728498 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c9c01d070 con 0x7f7cac108980 2024-05-14T09:23:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.965+0000 7f7c9b7fe700 1 -- 172.21.15.2:0/546728498 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c9c019a80 con 0x7f7cac108980 2024-05-14T09:23:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.965+0000 7f7cb0b0d700 1 -- 172.21.15.2:0/546728498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cac1a0b40 con 0x7f7cac108980 2024-05-14T09:23:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.965+0000 7f7cb0b0d700 1 -- 172.21.15.2:0/546728498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cac1a3750 con 0x7f7cac108980 2024-05-14T09:23:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.966+0000 7f7c9b7fe700 1 -- 172.21.15.2:0/546728498 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7c9c028060 con 0x7f7cac108980 2024-05-14T09:23:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.967+0000 7f7cb0b0d700 1 -- 172.21.15.2:0/546728498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cac064e20 con 0x7f7cac108980 2024-05-14T09:23:22.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.967+0000 7f7c9b7fe700 1 --2- 172.21.15.2:0/546728498 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f7c9403e4c0 0x7f7c940409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:22.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.967+0000 7f7c9b7fe700 1 -- 172.21.15.2:0/546728498 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7c9c053840 con 0x7f7cac108980 2024-05-14T09:23:22.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.967+0000 7f7ca9d9b700 1 --2- 172.21.15.2:0/546728498 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f7c9403e4c0 0x7f7c940409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:22.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.967+0000 7f7ca9d9b700 1 --2- 172.21.15.2:0/546728498 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f7c9403e4c0 0x7f7c940409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0006fd0 tx=0x7f7ca0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:22.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:20.972+0000 7f7c9b7fe700 1 -- 172.21.15.2:0/546728498 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c9c015070 con 0x7f7cac108980 2024-05-14T09:23:22.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:21.254+0000 7f7cb0b0d700 1 -- 172.21.15.2:0/546728498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f7cac1951c0 con 0x7f7cac108980 2024-05-14T09:23:22.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:21.368+0000 7f7c9b7fe700 1 -- 172.21.15.2:0/546728498 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f7c9c01fde0 con 0x7f7cac108980 2024-05-14T09:23:22.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:21.370+0000 7f7c9b7fe700 1 -- 172.21.15.2:0/546728498 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f7c9c01f030 con 0x7f7cac108980 2024-05-14T09:23:22.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:21.371+0000 7f7cb0b0d700 1 -- 172.21.15.2:0/546728498 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f7c9403e4c0 msgr2=0x7f7c940409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:22.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:21.371+0000 7f7cb0b0d700 1 --2- 172.21.15.2:0/546728498 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f7c9403e4c0 0x7f7c940409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0006fd0 tx=0x7f7ca0006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:22.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:21.371+0000 7f7cb0b0d700 1 -- 172.21.15.2:0/546728498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac108980 msgr2=0x7f7cac1a0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:22.059 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:21.371+0000 7f7cb0b0d700 1 --2- 172.21.15.2:0/546728498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac108980 0x7f7cac1a0370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c003c30 tx=0x7f7c9c003d10 comp rx=0 tx=0).stop 2024-05-14T09:23:22.059 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:21.371+0000 7f7cb0b0d700 1 -- 172.21.15.2:0/546728498 shutdown_connections 2024-05-14T09:23:22.059 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:21.371+0000 7f7cb0b0d700 1 --2- 172.21.15.2:0/546728498 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f7c9403e4c0 0x7f7c940409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:22.059 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:21.371+0000 7f7cb0b0d700 1 --2- 172.21.15.2:0/546728498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac108980 0x7f7cac1a0370 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:22.059 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:21.371+0000 7f7cb0b0d700 1 -- 172.21.15.2:0/546728498 >> 172.21.15.2:0/546728498 conn(0x7f7cac104040 msgr2=0x7f7cac192280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:22.059 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:21.371+0000 7f7cb0b0d700 1 -- 172.21.15.2:0/546728498 shutdown_connections 2024-05-14T09:23:22.059 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:21.371+0000 7f7cb0b0d700 1 -- 172.21.15.2:0/546728498 wait complete. 2024-05-14T09:23:22.319 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:22 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:22.227+0000 7f1a42647200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T09:23:22.671 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:22 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:22.319+0000 7f1a42647200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T09:23:22.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:22 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/546728498' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-14T09:23:22.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:22 smithi002 ceph-mon[31557]: mgrmap e9: x(active, since 18s) 2024-05-14T09:23:23.419 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-05-14T09:23:23.420 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-14T09:23:23.420 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-14T09:23:23.420 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-14T09:23:23.420 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-14T09:23:23.420 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-05-14T09:23:23.420 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.541+0000 7f73641be700 1 Processor -- start 2024-05-14T09:23:23.420 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.541+0000 7f73641be700 1 -- start start 2024-05-14T09:23:23.420 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.541+0000 7f73641be700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f735c108940 0x7f735c108d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:23.420 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.541+0000 7f73641be700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f735c109340 con 0x7f735c108940 2024-05-14T09:23:23.420 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.542+0000 7f7361f5a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f735c108940 0x7f735c108d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:23.420 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.542+0000 7f7361f5a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f735c108940 0x7f735c108d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52242/0 (socket says 172.21.15.2:52242) 2024-05-14T09:23:23.420 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.542+0000 7f7361f5a700 1 -- 172.21.15.2:0/287386107 learned_addr learned my addr 172.21.15.2:0/287386107 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:23.420 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.542+0000 7f7361f5a700 1 -- 172.21.15.2:0/287386107 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735c109b10 con 0x7f735c108940 2024-05-14T09:23:23.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.542+0000 7f7361f5a700 1 --2- 172.21.15.2:0/287386107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f735c108940 0x7f735c108d50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f73580098a0 tx=0x7f7358009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f13bbc954cd28bf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:23.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.542+0000 7f7360f58700 1 -- 172.21.15.2:0/287386107 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7358004160 con 0x7f735c108940 2024-05-14T09:23:23.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.542+0000 7f7360f58700 1 -- 172.21.15.2:0/287386107 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7358004300 con 0x7f735c108940 2024-05-14T09:23:23.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.542+0000 7f7360f58700 1 -- 172.21.15.2:0/287386107 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7358011a20 con 0x7f735c108940 2024-05-14T09:23:23.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.543+0000 7f73641be700 1 -- 172.21.15.2:0/287386107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f735c108940 msgr2=0x7f735c108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:23.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.543+0000 7f73641be700 1 --2- 172.21.15.2:0/287386107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f735c108940 0x7f735c108d50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f73580098a0 tx=0x7f7358009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:23.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.543+0000 7f73641be700 1 -- 172.21.15.2:0/287386107 shutdown_connections 2024-05-14T09:23:23.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.543+0000 7f73641be700 1 --2- 172.21.15.2:0/287386107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f735c108940 0x7f735c108d50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:23.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.543+0000 7f73641be700 1 -- 172.21.15.2:0/287386107 >> 172.21.15.2:0/287386107 conn(0x7f735c103fc0 msgr2=0x7f735c106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:23.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.543+0000 7f73641be700 1 -- 172.21.15.2:0/287386107 shutdown_connections 2024-05-14T09:23:23.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.543+0000 7f73641be700 1 -- 172.21.15.2:0/287386107 wait complete. 2024-05-14T09:23:23.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.544+0000 7f73641be700 1 Processor -- start 2024-05-14T09:23:23.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.544+0000 7f73641be700 1 -- start start 2024-05-14T09:23:23.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.544+0000 7f73641be700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f735c108940 0x7f735c19d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:23.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.544+0000 7f73641be700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f735c10cec0 con 0x7f735c108940 2024-05-14T09:23:23.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.544+0000 7f7361f5a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f735c108940 0x7f735c19d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:23.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.544+0000 7f7361f5a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f735c108940 0x7f735c19d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52254/0 (socket says 172.21.15.2:52254) 2024-05-14T09:23:23.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.544+0000 7f7361f5a700 1 -- 172.21.15.2:0/994071578 learned_addr learned my addr 172.21.15.2:0/994071578 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:23.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.545+0000 7f7361f5a700 1 -- 172.21.15.2:0/994071578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735c19d5c0 con 0x7f735c108940 2024-05-14T09:23:23.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.545+0000 7f7361f5a700 1 --2- 172.21.15.2:0/994071578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f735c108940 0x7f735c19d080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7358011a20 tx=0x7f7358011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:23.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.545+0000 7f7352ffd700 1 -- 172.21.15.2:0/994071578 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73580039c0 con 0x7f735c108940 2024-05-14T09:23:23.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.545+0000 7f7352ffd700 1 -- 172.21.15.2:0/994071578 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7358003b60 con 0x7f735c108940 2024-05-14T09:23:23.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.545+0000 7f73641be700 1 -- 172.21.15.2:0/994071578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f735c19d850 con 0x7f735c108940 2024-05-14T09:23:23.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.545+0000 7f7352ffd700 1 -- 172.21.15.2:0/994071578 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7358019700 con 0x7f735c108940 2024-05-14T09:23:23.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.545+0000 7f73641be700 1 -- 172.21.15.2:0/994071578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f735c1a0460 con 0x7f735c108940 2024-05-14T09:23:23.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.546+0000 7f73641be700 1 -- 172.21.15.2:0/994071578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f735c064e20 con 0x7f735c108940 2024-05-14T09:23:23.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.546+0000 7f7352ffd700 1 -- 172.21.15.2:0/994071578 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f735801ab00 con 0x7f735c108940 2024-05-14T09:23:23.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.547+0000 7f7352ffd700 1 --2- 172.21.15.2:0/994071578 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f734803e470 0x7f7348040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:23.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.547+0000 7f7352ffd700 1 -- 172.21.15.2:0/994071578 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7358053a30 con 0x7f735c108940 2024-05-14T09:23:23.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.547+0000 7f7361759700 1 -- 172.21.15.2:0/994071578 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f734803e470 msgr2=0x7f7348040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1423742578 2024-05-14T09:23:23.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.547+0000 7f7361759700 1 --2- 172.21.15.2:0/994071578 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f734803e470 0x7f7348040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T09:23:23.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.552+0000 7f7352ffd700 1 -- 172.21.15.2:0/994071578 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7358053790 con 0x7f735c108940 2024-05-14T09:23:23.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.748+0000 7f7361759700 1 -- 172.21.15.2:0/994071578 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f734803e470 msgr2=0x7f7348040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1423742578 2024-05-14T09:23:23.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.748+0000 7f7361759700 1 --2- 172.21.15.2:0/994071578 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f734803e470 0x7f7348040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T09:23:23.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.872+0000 7f73641be700 1 -- 172.21.15.2:0/994071578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f735c02e5f0 con 0x7f735c108940 2024-05-14T09:23:23.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.873+0000 7f7352ffd700 1 -- 172.21.15.2:0/994071578 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f735801a860 con 0x7f735c108940 2024-05-14T09:23:23.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.875+0000 7f73641be700 1 -- 172.21.15.2:0/994071578 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f734803e470 msgr2=0x7f7348040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T09:23:23.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.875+0000 7f73641be700 1 --2- 172.21.15.2:0/994071578 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f734803e470 0x7f7348040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:23.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.875+0000 7f73641be700 1 -- 172.21.15.2:0/994071578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f735c108940 msgr2=0x7f735c19d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:23.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.875+0000 7f73641be700 1 --2- 172.21.15.2:0/994071578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f735c108940 0x7f735c19d080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7358011a20 tx=0x7f7358011b00 comp rx=0 tx=0).stop 2024-05-14T09:23:23.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.875+0000 7f73641be700 1 -- 172.21.15.2:0/994071578 shutdown_connections 2024-05-14T09:23:23.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.875+0000 7f73641be700 1 --2- 172.21.15.2:0/994071578 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f734803e470 0x7f7348040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:23.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.875+0000 7f73641be700 1 --2- 172.21.15.2:0/994071578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f735c108940 0x7f735c19d080 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:23.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.875+0000 7f73641be700 1 -- 172.21.15.2:0/994071578 >> 172.21.15.2:0/994071578 conn(0x7f735c103fc0 msgr2=0x7f735c104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:23.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.875+0000 7f73641be700 1 -- 172.21.15.2:0/994071578 shutdown_connections 2024-05-14T09:23:23.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:22.875+0000 7f73641be700 1 -- 172.21.15.2:0/994071578 wait complete. 2024-05-14T09:23:23.424 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for the mgr to restart... 2024-05-14T09:23:23.424 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr epoch 9... 2024-05-14T09:23:23.529 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:23 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/994071578' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-14T09:23:23.799 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:23 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:23.612+0000 7f1a42647200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T09:23:24.171 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:23 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:23.798+0000 7f1a42647200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T09:23:24.171 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:23 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:23.980+0000 7f1a42647200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T09:23:24.448 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:24 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:24.287+0000 7f1a42647200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T09:23:24.448 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:24 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:24.385+0000 7f1a42647200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T09:23:24.804 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:24 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:24.598+0000 7f1a42647200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T09:23:25.171 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:24 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:24.803+0000 7f1a42647200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T09:23:25.921 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:25 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:25.441+0000 7f1a42647200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T09:23:25.921 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:25 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:25.684+0000 7f1a42647200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T09:23:26.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:26 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:26.148+0000 7f1a42647200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T09:23:26.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:26 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:26.254+0000 7f1a42647200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T09:23:26.850 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:26 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:26.644+0000 7f1a42647200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T09:23:27.122 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:27 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:27.022+0000 7f1a42647200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T09:23:27.123 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:27 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:27.121+0000 7f1a42647200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T09:23:27.671 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:27 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:27.382+0000 7f1a42647200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T09:23:27.671 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:27 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:27.523+0000 7f1a42647200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T09:23:27.671 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:27 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:27.626+0000 7f1a42647200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T09:23:28.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:27 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:27.967+0000 7f1a42647200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T09:23:28.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:28 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:28.066+0000 7f1a42647200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T09:23:28.991 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:28 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:28.705+0000 7f1a42647200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T09:23:28.991 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:23:28 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:23:28.937+0000 7f1a42647200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T09:23:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:28 smithi002 ceph-mon[31557]: Active manager daemon x restarted 2024-05-14T09:23:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:28 smithi002 ceph-mon[31557]: Activating manager daemon x 2024-05-14T09:23:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:28 smithi002 ceph-mon[31557]: osdmap e3: 0 total, 0 up, 0 in 2024-05-14T09:23:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:28 smithi002 ceph-mon[31557]: mgrmap e10: x(active, starting, since 0.00418536s) 2024-05-14T09:23:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:23:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T09:23:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T09:23:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T09:23:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T09:23:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:28 smithi002 ceph-mon[31557]: Manager daemon x is now available 2024-05-14T09:23:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:23:30.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T09:23:30.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T09:23:30.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:29 smithi002 ceph-mon[31557]: [14/May/2024:09:23:29] ENGINE Bus STARTING 2024-05-14T09:23:30.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:29 smithi002 ceph-mon[31557]: [14/May/2024:09:23:29] ENGINE Serving on http://172.21.15.2:8765 2024-05-14T09:23:30.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:29 smithi002 ceph-mon[31557]: mgrmap e11: x(active, since 1.00773s) 2024-05-14T09:23:30.738 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-05-14T09:23:30.739 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-14T09:23:30.739 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-14T09:23:30.739 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-05-14T09:23:30.739 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.838+0000 7f6a7f846700 1 Processor -- start 2024-05-14T09:23:30.739 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.838+0000 7f6a7f846700 1 -- start start 2024-05-14T09:23:30.739 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.839+0000 7f6a7f846700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a7810a560 0x7f6a7810c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:30.739 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.839+0000 7f6a7f846700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7807d880 con 0x7f6a7810a560 2024-05-14T09:23:30.739 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.839+0000 7f6a7d5e2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a7810a560 0x7f6a7810c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:30.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.839+0000 7f6a7d5e2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a7810a560 0x7f6a7810c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52258/0 (socket says 172.21.15.2:52258) 2024-05-14T09:23:30.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.839+0000 7f6a7d5e2700 1 -- 172.21.15.2:0/2467883986 learned_addr learned my addr 172.21.15.2:0/2467883986 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:30.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.839+0000 7f6a7d5e2700 1 -- 172.21.15.2:0/2467883986 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7810cec0 con 0x7f6a7810a560 2024-05-14T09:23:30.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.840+0000 7f6a7d5e2700 1 --2- 172.21.15.2:0/2467883986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a7810a560 0x7f6a7810c980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a740098a0 tx=0x7f6a74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0de939fe5e50366 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:30.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.840+0000 7f6a6bfff700 1 -- 172.21.15.2:0/2467883986 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a74004160 con 0x7f6a7810a560 2024-05-14T09:23:30.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.840+0000 7f6a6bfff700 1 -- 172.21.15.2:0/2467883986 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a74004300 con 0x7f6a7810a560 2024-05-14T09:23:30.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.840+0000 7f6a6bfff700 1 -- 172.21.15.2:0/2467883986 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a74011a20 con 0x7f6a7810a560 2024-05-14T09:23:30.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.841+0000 7f6a7f846700 1 -- 172.21.15.2:0/2467883986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a7810a560 msgr2=0x7f6a7810c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:30.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.841+0000 7f6a7f846700 1 --2- 172.21.15.2:0/2467883986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a7810a560 0x7f6a7810c980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a740098a0 tx=0x7f6a74009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:30.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.841+0000 7f6a7f846700 1 -- 172.21.15.2:0/2467883986 shutdown_connections 2024-05-14T09:23:30.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.841+0000 7f6a7f846700 1 --2- 172.21.15.2:0/2467883986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a7810a560 0x7f6a7810c980 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:30.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.841+0000 7f6a7f846700 1 -- 172.21.15.2:0/2467883986 >> 172.21.15.2:0/2467883986 conn(0x7f6a78103fc0 msgr2=0x7f6a78106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:30.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.841+0000 7f6a7f846700 1 -- 172.21.15.2:0/2467883986 shutdown_connections 2024-05-14T09:23:30.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.841+0000 7f6a7f846700 1 -- 172.21.15.2:0/2467883986 wait complete. 2024-05-14T09:23:30.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.842+0000 7f6a7f846700 1 Processor -- start 2024-05-14T09:23:30.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.842+0000 7f6a7f846700 1 -- start start 2024-05-14T09:23:30.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.842+0000 7f6a7f846700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a7810a560 0x7f6a7819e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:30.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.842+0000 7f6a7f846700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7810de60 con 0x7f6a7810a560 2024-05-14T09:23:30.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.842+0000 7f6a7d5e2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a7810a560 0x7f6a7819e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:30.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.842+0000 7f6a7d5e2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a7810a560 0x7f6a7819e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52262/0 (socket says 172.21.15.2:52262) 2024-05-14T09:23:30.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.843+0000 7f6a7d5e2700 1 -- 172.21.15.2:0/246266789 learned_addr learned my addr 172.21.15.2:0/246266789 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:30.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.843+0000 7f6a7d5e2700 1 -- 172.21.15.2:0/246266789 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7819e660 con 0x7f6a7810a560 2024-05-14T09:23:30.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.843+0000 7f6a7d5e2700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a7810a560 0x7f6a7819e120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a74004660 tx=0x7f6a74011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:30.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.843+0000 7f6a6a7fc700 1 -- 172.21.15.2:0/246266789 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a74003a60 con 0x7f6a7810a560 2024-05-14T09:23:30.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.843+0000 7f6a6a7fc700 1 -- 172.21.15.2:0/246266789 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a7401b8f0 con 0x7f6a7810a560 2024-05-14T09:23:30.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.843+0000 7f6a6a7fc700 1 -- 172.21.15.2:0/246266789 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a74023890 con 0x7f6a7810a560 2024-05-14T09:23:30.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.843+0000 7f6a7f846700 1 -- 172.21.15.2:0/246266789 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a7819e8f0 con 0x7f6a7810a560 2024-05-14T09:23:30.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.843+0000 7f6a7f846700 1 -- 172.21.15.2:0/246266789 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a781a1500 con 0x7f6a7810a560 2024-05-14T09:23:30.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.844+0000 7f6a6a7fc700 1 -- 172.21.15.2:0/246266789 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f6a7402c3f0 con 0x7f6a7810a560 2024-05-14T09:23:30.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.845+0000 7f6a6a7fc700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f6a6403e470 0x7f6a64040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:30.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.845+0000 7f6a6a7fc700 1 -- 172.21.15.2:0/246266789 --> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6a64041090 con 0x7f6a6403e470 2024-05-14T09:23:30.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.845+0000 7f6a6a7fc700 1 -- 172.21.15.2:0/246266789 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6a74054f30 con 0x7f6a7810a560 2024-05-14T09:23:30.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.845+0000 7f6a7cde1700 1 -- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f6a6403e470 msgr2=0x7f6a64040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1423742578 2024-05-14T09:23:30.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:23.845+0000 7f6a7cde1700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f6a6403e470 0x7f6a64040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T09:23:30.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:24.046+0000 7f6a7cde1700 1 -- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f6a6403e470 msgr2=0x7f6a64040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1423742578 2024-05-14T09:23:30.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:24.046+0000 7f6a7cde1700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f6a6403e470 0x7f6a64040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T09:23:30.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:24.447+0000 7f6a7cde1700 1 -- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f6a6403e470 msgr2=0x7f6a64040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1423742578 2024-05-14T09:23:30.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:24.447+0000 7f6a7cde1700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f6a6403e470 0x7f6a64040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-14T09:23:30.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:25.248+0000 7f6a7cde1700 1 -- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f6a6403e470 msgr2=0x7f6a64040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1423742578 2024-05-14T09:23:30.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:25.248+0000 7f6a7cde1700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f6a6403e470 0x7f6a64040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-14T09:23:30.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:26.849+0000 7f6a7cde1700 1 -- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f6a6403e470 msgr2=0x7f6a64040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1423742578 2024-05-14T09:23:30.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:26.849+0000 7f6a7cde1700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f6a6403e470 0x7f6a64040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-14T09:23:30.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:28.941+0000 7f6a6a7fc700 1 -- 172.21.15.2:0/246266789 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f6a7402c780 con 0x7f6a7810a560 2024-05-14T09:23:30.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:28.941+0000 7f6a6a7fc700 1 -- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f6a6403e470 msgr2=0x7f6a64040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T09:23:30.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:28.941+0000 7f6a6a7fc700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f6a6403e470 0x7f6a64040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.943+0000 7f6a6a7fc700 1 -- 172.21.15.2:0/246266789 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f6a7402cca0 con 0x7f6a7810a560 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.943+0000 7f6a6a7fc700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6a74054ac0 0x7f6a64043f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.943+0000 7f6a6a7fc700 1 -- 172.21.15.2:0/246266789 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6a74012ec0 con 0x7f6a74054ac0 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.943+0000 7f6a7cde1700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6a74054ac0 0x7f6a64043f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.944+0000 7f6a7cde1700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6a74054ac0 0x7f6a64043f70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c003a10 tx=0x7f6a6c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.945+0000 7f6a6a7fc700 1 -- 172.21.15.2:0/246266789 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f6a74012ec0 con 0x7f6a74054ac0 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.954+0000 7f6a7f846700 1 -- 172.21.15.2:0/246266789 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6a78064e20 con 0x7f6a74054ac0 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.954+0000 7f6a6a7fc700 1 -- 172.21.15.2:0/246266789 <== mgr.14152 v2:172.21.15.2:6800/3050449557 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f6a78064e20 con 0x7f6a74054ac0 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.954+0000 7f6a7f846700 1 -- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6a74054ac0 msgr2=0x7f6a64043f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.954+0000 7f6a7f846700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6a74054ac0 0x7f6a64043f70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c003a10 tx=0x7f6a6c0092b0 comp rx=0 tx=0).stop 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.954+0000 7f6a7f846700 1 -- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a7810a560 msgr2=0x7f6a7819e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.954+0000 7f6a7f846700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a7810a560 0x7f6a7819e120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a74004660 tx=0x7f6a74011aa0 comp rx=0 tx=0).stop 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.955+0000 7f6a7f846700 1 -- 172.21.15.2:0/246266789 shutdown_connections 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.955+0000 7f6a7f846700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6a74054ac0 0x7f6a64043f70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.955+0000 7f6a7f846700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a7810a560 0x7f6a7819e120 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:30.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.955+0000 7f6a7f846700 1 --2- 172.21.15.2:0/246266789 >> [v2:172.21.15.2:6800/1423742578,v1:172.21.15.2:6801/1423742578] conn(0x7f6a6403e470 0x7f6a64040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:30.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.955+0000 7f6a7f846700 1 -- 172.21.15.2:0/246266789 >> 172.21.15.2:0/246266789 conn(0x7f6a78103fc0 msgr2=0x7f6a78104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:30.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.955+0000 7f6a7f846700 1 -- 172.21.15.2:0/246266789 shutdown_connections 2024-05-14T09:23:30.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:29.955+0000 7f6a7f846700 1 -- 172.21.15.2:0/246266789 wait complete. 2024-05-14T09:23:30.745 INFO:teuthology.orchestra.run.smithi002.stdout:mgr epoch 9 is available 2024-05-14T09:23:30.745 INFO:teuthology.orchestra.run.smithi002.stdout:Generating a dashboard self-signed certificate... 2024-05-14T09:23:31.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:30 smithi002 ceph-mon[31557]: [14/May/2024:09:23:29] ENGINE Serving on https://172.21.15.2:7150 2024-05-14T09:23:31.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:30 smithi002 ceph-mon[31557]: [14/May/2024:09:23:29] ENGINE Bus STARTED 2024-05-14T09:23:31.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:30 smithi002 ceph-mon[31557]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-14T09:23:31.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:30 smithi002 ceph-mon[31557]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-14T09:23:32.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:31 smithi002 ceph-mon[31557]: mgrmap e12: x(active, since 2s) 2024-05-14T09:23:32.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:31 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:32.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:31 smithi002 ceph-mon[31557]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:23:32.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:31 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:32.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:31 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:32.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-14T09:23:32.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.218+0000 7f87e2b11700 1 Processor -- start 2024-05-14T09:23:32.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.218+0000 7f87e2b11700 1 -- start start 2024-05-14T09:23:32.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.218+0000 7f87e2b11700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87dc10a560 0x7f87dc10c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:32.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.218+0000 7f87e2b11700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc07d880 con 0x7f87dc10a560 2024-05-14T09:23:32.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.218+0000 7f87e08ad700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87dc10a560 0x7f87dc10c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:32.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.218+0000 7f87e08ad700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87dc10a560 0x7f87dc10c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38266/0 (socket says 172.21.15.2:38266) 2024-05-14T09:23:32.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.218+0000 7f87e08ad700 1 -- 172.21.15.2:0/1695306190 learned_addr learned my addr 172.21.15.2:0/1695306190 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:32.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.219+0000 7f87e08ad700 1 -- 172.21.15.2:0/1695306190 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc10cec0 con 0x7f87dc10a560 2024-05-14T09:23:32.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.219+0000 7f87e08ad700 1 --2- 172.21.15.2:0/1695306190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87dc10a560 0x7f87dc10c980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f87d00098a0 tx=0x7f87d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e57bd976b62178a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:32.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.219+0000 7f87db7fe700 1 -- 172.21.15.2:0/1695306190 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87d0004160 con 0x7f87dc10a560 2024-05-14T09:23:32.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.219+0000 7f87db7fe700 1 -- 172.21.15.2:0/1695306190 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87d0004300 con 0x7f87dc10a560 2024-05-14T09:23:32.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.219+0000 7f87db7fe700 1 -- 172.21.15.2:0/1695306190 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87d0011a20 con 0x7f87dc10a560 2024-05-14T09:23:32.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.220+0000 7f87e2b11700 1 -- 172.21.15.2:0/1695306190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87dc10a560 msgr2=0x7f87dc10c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:32.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.220+0000 7f87e2b11700 1 --2- 172.21.15.2:0/1695306190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87dc10a560 0x7f87dc10c980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f87d00098a0 tx=0x7f87d0009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:32.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.220+0000 7f87e2b11700 1 -- 172.21.15.2:0/1695306190 shutdown_connections 2024-05-14T09:23:32.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.220+0000 7f87e2b11700 1 --2- 172.21.15.2:0/1695306190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87dc10a560 0x7f87dc10c980 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:32.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.220+0000 7f87e2b11700 1 -- 172.21.15.2:0/1695306190 >> 172.21.15.2:0/1695306190 conn(0x7f87dc103fc0 msgr2=0x7f87dc106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:32.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.220+0000 7f87e2b11700 1 -- 172.21.15.2:0/1695306190 shutdown_connections 2024-05-14T09:23:32.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.220+0000 7f87e2b11700 1 -- 172.21.15.2:0/1695306190 wait complete. 2024-05-14T09:23:32.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.221+0000 7f87e2b11700 1 Processor -- start 2024-05-14T09:23:32.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.221+0000 7f87e2b11700 1 -- start start 2024-05-14T09:23:32.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.222+0000 7f87e2b11700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87dc10a560 0x7f87dc19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:32.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.222+0000 7f87e2b11700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc10de60 con 0x7f87dc10a560 2024-05-14T09:23:32.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.222+0000 7f87e08ad700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87dc10a560 0x7f87dc19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:32.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.222+0000 7f87e08ad700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87dc10a560 0x7f87dc19e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38274/0 (socket says 172.21.15.2:38274) 2024-05-14T09:23:32.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.222+0000 7f87e08ad700 1 -- 172.21.15.2:0/817229817 learned_addr learned my addr 172.21.15.2:0/817229817 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:32.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.222+0000 7f87e08ad700 1 -- 172.21.15.2:0/817229817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc19e640 con 0x7f87dc10a560 2024-05-14T09:23:32.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.222+0000 7f87e08ad700 1 --2- 172.21.15.2:0/817229817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87dc10a560 0x7f87dc19e100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f87d0004660 tx=0x7f87d0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:32.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.222+0000 7f87d9ffb700 1 -- 172.21.15.2:0/817229817 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87d0003a60 con 0x7f87dc10a560 2024-05-14T09:23:32.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.222+0000 7f87e2b11700 1 -- 172.21.15.2:0/817229817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87dc19e8d0 con 0x7f87dc10a560 2024-05-14T09:23:32.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.223+0000 7f87d9ffb700 1 -- 172.21.15.2:0/817229817 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87d001b8f0 con 0x7f87dc10a560 2024-05-14T09:23:32.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.223+0000 7f87d9ffb700 1 -- 172.21.15.2:0/817229817 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87d0023890 con 0x7f87dc10a560 2024-05-14T09:23:32.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.223+0000 7f87e2b11700 1 -- 172.21.15.2:0/817229817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87dc1a14e0 con 0x7f87dc10a560 2024-05-14T09:23:32.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.224+0000 7f87d9ffb700 1 -- 172.21.15.2:0/817229817 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f87d002c3f0 con 0x7f87dc10a560 2024-05-14T09:23:32.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.224+0000 7f87d9ffb700 1 --2- 172.21.15.2:0/817229817 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f87cc03e470 0x7f87cc040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:32.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.224+0000 7f87d9ffb700 1 -- 172.21.15.2:0/817229817 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f87d001b5f0 con 0x7f87dc10a560 2024-05-14T09:23:32.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.224+0000 7f87dbfff700 1 --2- 172.21.15.2:0/817229817 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f87cc03e470 0x7f87cc040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:32.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.224+0000 7f87e2b11700 1 -- 172.21.15.2:0/817229817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87dc064e20 con 0x7f87dc10a560 2024-05-14T09:23:32.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.225+0000 7f87dbfff700 1 --2- 172.21.15.2:0/817229817 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f87cc03e470 0x7f87cc040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f87c8006fd0 tx=0x7f87c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:32.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.230+0000 7f87d9ffb700 1 -- 172.21.15.2:0/817229817 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87d001f030 con 0x7f87dc10a560 2024-05-14T09:23:32.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.452+0000 7f87e2b11700 1 -- 172.21.15.2:0/817229817 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f87dc063c00 con 0x7f87cc03e470 2024-05-14T09:23:32.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.602+0000 7f87d9ffb700 1 -- 172.21.15.2:0/817229817 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f87dc063c00 con 0x7f87cc03e470 2024-05-14T09:23:32.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.604+0000 7f87e2b11700 1 -- 172.21.15.2:0/817229817 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f87cc03e470 msgr2=0x7f87cc040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:32.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.604+0000 7f87e2b11700 1 --2- 172.21.15.2:0/817229817 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f87cc03e470 0x7f87cc040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f87c8006fd0 tx=0x7f87c8006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:32.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.604+0000 7f87e2b11700 1 -- 172.21.15.2:0/817229817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87dc10a560 msgr2=0x7f87dc19e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:32.212 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.604+0000 7f87e2b11700 1 --2- 172.21.15.2:0/817229817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87dc10a560 0x7f87dc19e100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f87d0004660 tx=0x7f87d0011aa0 comp rx=0 tx=0).stop 2024-05-14T09:23:32.213 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.604+0000 7f87e2b11700 1 -- 172.21.15.2:0/817229817 shutdown_connections 2024-05-14T09:23:32.213 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.604+0000 7f87e2b11700 1 --2- 172.21.15.2:0/817229817 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f87cc03e470 0x7f87cc040980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:32.213 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.604+0000 7f87e2b11700 1 --2- 172.21.15.2:0/817229817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87dc10a560 0x7f87dc19e100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:32.213 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.604+0000 7f87e2b11700 1 -- 172.21.15.2:0/817229817 >> 172.21.15.2:0/817229817 conn(0x7f87dc103fc0 msgr2=0x7f87dc104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:32.213 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.604+0000 7f87e2b11700 1 -- 172.21.15.2:0/817229817 shutdown_connections 2024-05-14T09:23:32.213 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:31.604+0000 7f87e2b11700 1 -- 172.21.15.2:0/817229817 wait complete. 2024-05-14T09:23:32.213 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial admin user... 2024-05-14T09:23:33.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$ryqbyqmojFFAP7s7XTedpucqJdLGC64rZe9TsDHB83lCO/MfLLEu6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715678613, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-14T09:23:33.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.665+0000 7fb004b2d700 1 Processor -- start 2024-05-14T09:23:33.740 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.665+0000 7fb004b2d700 1 -- start start 2024-05-14T09:23:33.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.665+0000 7fb004b2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb00010a560 0x7fb00010c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:33.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.665+0000 7fb004b2d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb00007d880 con 0x7fb00010a560 2024-05-14T09:23:33.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.666+0000 7faffe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb00010a560 0x7fb00010c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:33.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.666+0000 7faffe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb00010a560 0x7fb00010c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38282/0 (socket says 172.21.15.2:38282) 2024-05-14T09:23:33.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.666+0000 7faffe59c700 1 -- 172.21.15.2:0/1279345489 learned_addr learned my addr 172.21.15.2:0/1279345489 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:33.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.666+0000 7faffe59c700 1 -- 172.21.15.2:0/1279345489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00010cec0 con 0x7fb00010a560 2024-05-14T09:23:33.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.666+0000 7faffe59c700 1 --2- 172.21.15.2:0/1279345489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb00010a560 0x7fb00010c980 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faff00098a0 tx=0x7faff0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d73917a86feb82 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:33.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.666+0000 7faffd59a700 1 -- 172.21.15.2:0/1279345489 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faff0004160 con 0x7fb00010a560 2024-05-14T09:23:33.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.666+0000 7faffd59a700 1 -- 172.21.15.2:0/1279345489 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faff0004300 con 0x7fb00010a560 2024-05-14T09:23:33.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.666+0000 7faffd59a700 1 -- 172.21.15.2:0/1279345489 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faff0011a20 con 0x7fb00010a560 2024-05-14T09:23:33.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.667+0000 7fb004b2d700 1 -- 172.21.15.2:0/1279345489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb00010a560 msgr2=0x7fb00010c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:33.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.667+0000 7fb004b2d700 1 --2- 172.21.15.2:0/1279345489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb00010a560 0x7fb00010c980 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faff00098a0 tx=0x7faff0009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:33.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.667+0000 7fb004b2d700 1 -- 172.21.15.2:0/1279345489 shutdown_connections 2024-05-14T09:23:33.741 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.667+0000 7fb004b2d700 1 --2- 172.21.15.2:0/1279345489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb00010a560 0x7fb00010c980 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.667+0000 7fb004b2d700 1 -- 172.21.15.2:0/1279345489 >> 172.21.15.2:0/1279345489 conn(0x7fb000103fc0 msgr2=0x7fb000106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.668+0000 7fb004b2d700 1 -- 172.21.15.2:0/1279345489 shutdown_connections 2024-05-14T09:23:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.668+0000 7fb004b2d700 1 -- 172.21.15.2:0/1279345489 wait complete. 2024-05-14T09:23:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.669+0000 7fb004b2d700 1 Processor -- start 2024-05-14T09:23:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.669+0000 7fb004b2d700 1 -- start start 2024-05-14T09:23:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.669+0000 7fb004b2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb00010a560 0x7fb00019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.669+0000 7fb004b2d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb00010de60 con 0x7fb00010a560 2024-05-14T09:23:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.669+0000 7faffe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb00010a560 0x7fb00019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.669+0000 7faffe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb00010a560 0x7fb00019e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38296/0 (socket says 172.21.15.2:38296) 2024-05-14T09:23:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.669+0000 7faffe59c700 1 -- 172.21.15.2:0/3875598648 learned_addr learned my addr 172.21.15.2:0/3875598648 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.670+0000 7faffe59c700 1 -- 172.21.15.2:0/3875598648 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00019e640 con 0x7fb00010a560 2024-05-14T09:23:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.670+0000 7faffe59c700 1 --2- 172.21.15.2:0/3875598648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb00010a560 0x7fb00019e100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faff0009520 tx=0x7faff0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.670+0000 7fafef7fe700 1 -- 172.21.15.2:0/3875598648 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faff00039e0 con 0x7fb00010a560 2024-05-14T09:23:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.670+0000 7fafef7fe700 1 -- 172.21.15.2:0/3875598648 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faff001b8f0 con 0x7fb00010a560 2024-05-14T09:23:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.670+0000 7fb004b2d700 1 -- 172.21.15.2:0/3875598648 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00019e8d0 con 0x7fb00010a560 2024-05-14T09:23:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.670+0000 7fafef7fe700 1 -- 172.21.15.2:0/3875598648 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faff00197d0 con 0x7fb00010a560 2024-05-14T09:23:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.670+0000 7fb004b2d700 1 -- 172.21.15.2:0/3875598648 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0001a14e0 con 0x7fb00010a560 2024-05-14T09:23:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.671+0000 7fb004b2d700 1 -- 172.21.15.2:0/3875598648 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb000064e20 con 0x7fb00010a560 2024-05-14T09:23:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.671+0000 7fafef7fe700 1 -- 172.21.15.2:0/3875598648 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7faff0019a80 con 0x7fb00010a560 2024-05-14T09:23:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.672+0000 7fafef7fe700 1 --2- 172.21.15.2:0/3875598648 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fafe803e520 0x7fafe8040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.672+0000 7fafef7fe700 1 -- 172.21.15.2:0/3875598648 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faff0054080 con 0x7fb00010a560 2024-05-14T09:23:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.672+0000 7faffdd9b700 1 --2- 172.21.15.2:0/3875598648 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fafe803e520 0x7fafe8040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.672+0000 7faffdd9b700 1 --2- 172.21.15.2:0/3875598648 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fafe803e520 0x7fafe8040a30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faff4006fd0 tx=0x7faff4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.677+0000 7fafef7fe700 1 -- 172.21.15.2:0/3875598648 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faff001f030 con 0x7fb00010a560 2024-05-14T09:23:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:32.912+0000 7fb004b2d700 1 -- 172.21.15.2:0/3875598648 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fb000196190 con 0x7fafe803e520 2024-05-14T09:23:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:33.162+0000 7fafef7fe700 1 -- 172.21.15.2:0/3875598648 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fb000196190 con 0x7fafe803e520 2024-05-14T09:23:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:33.165+0000 7fb004b2d700 1 -- 172.21.15.2:0/3875598648 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fafe803e520 msgr2=0x7fafe8040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:33.165+0000 7fb004b2d700 1 --2- 172.21.15.2:0/3875598648 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fafe803e520 0x7fafe8040a30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faff4006fd0 tx=0x7faff4006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:33.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:33.165+0000 7fb004b2d700 1 -- 172.21.15.2:0/3875598648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb00010a560 msgr2=0x7fb00019e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:33.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:33.165+0000 7fb004b2d700 1 --2- 172.21.15.2:0/3875598648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb00010a560 0x7fb00019e100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faff0009520 tx=0x7faff0011aa0 comp rx=0 tx=0).stop 2024-05-14T09:23:33.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:33.165+0000 7fb004b2d700 1 -- 172.21.15.2:0/3875598648 shutdown_connections 2024-05-14T09:23:33.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:33.165+0000 7fb004b2d700 1 --2- 172.21.15.2:0/3875598648 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fafe803e520 0x7fafe8040a30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:33.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:33.165+0000 7fb004b2d700 1 --2- 172.21.15.2:0/3875598648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb00010a560 0x7fb00019e100 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:33.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:33.165+0000 7fb004b2d700 1 -- 172.21.15.2:0/3875598648 >> 172.21.15.2:0/3875598648 conn(0x7fb000103fc0 msgr2=0x7fb000104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:33.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:33.165+0000 7fb004b2d700 1 -- 172.21.15.2:0/3875598648 shutdown_connections 2024-05-14T09:23:33.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:33.165+0000 7fb004b2d700 1 -- 172.21.15.2:0/3875598648 wait complete. 2024-05-14T09:23:33.744 INFO:teuthology.orchestra.run.smithi002.stdout:Fetching dashboard port number... 2024-05-14T09:23:34.044 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:33 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:34.044 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:33 smithi002 ceph-mon[31557]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:23:34.044 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:33 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:35.075 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:34 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3543243184' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-14T09:23:35.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 8443 2024-05-14T09:23:35.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.172+0000 7f1df94d1700 1 Processor -- start 2024-05-14T09:23:35.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.172+0000 7f1df94d1700 1 -- start start 2024-05-14T09:23:35.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.173+0000 7f1df94d1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1df4107d50 0x7f1df4108160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:35.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.173+0000 7f1df94d1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1df4108730 con 0x7f1df4107d50 2024-05-14T09:23:35.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.173+0000 7f1df2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1df4107d50 0x7f1df4108160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:35.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.173+0000 7f1df2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1df4107d50 0x7f1df4108160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38312/0 (socket says 172.21.15.2:38312) 2024-05-14T09:23:35.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.173+0000 7f1df2ffd700 1 -- 172.21.15.2:0/3040495642 learned_addr learned my addr 172.21.15.2:0/3040495642 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:35.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.173+0000 7f1df2ffd700 1 -- 172.21.15.2:0/3040495642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df410ad50 con 0x7f1df4107d50 2024-05-14T09:23:35.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.173+0000 7f1df2ffd700 1 --2- 172.21.15.2:0/3040495642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1df4107d50 0x7f1df4108160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1de40098a0 tx=0x7f1de4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00b0d1e0aebed08 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:35.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.174+0000 7f1df27fc700 1 -- 172.21.15.2:0/3040495642 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1de4004160 con 0x7f1df4107d50 2024-05-14T09:23:35.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.174+0000 7f1df27fc700 1 -- 172.21.15.2:0/3040495642 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1de4004300 con 0x7f1df4107d50 2024-05-14T09:23:35.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.174+0000 7f1df27fc700 1 -- 172.21.15.2:0/3040495642 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1de4011a20 con 0x7f1df4107d50 2024-05-14T09:23:35.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.174+0000 7f1df94d1700 1 -- 172.21.15.2:0/3040495642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1df4107d50 msgr2=0x7f1df4108160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:35.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.174+0000 7f1df94d1700 1 --2- 172.21.15.2:0/3040495642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1df4107d50 0x7f1df4108160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1de40098a0 tx=0x7f1de4009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:35.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.175+0000 7f1df94d1700 1 -- 172.21.15.2:0/3040495642 shutdown_connections 2024-05-14T09:23:35.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.175+0000 7f1df94d1700 1 --2- 172.21.15.2:0/3040495642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1df4107d50 0x7f1df4108160 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:35.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.175+0000 7f1df94d1700 1 -- 172.21.15.2:0/3040495642 >> 172.21.15.2:0/3040495642 conn(0x7f1df4103760 msgr2=0x7f1df4105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:35.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.175+0000 7f1df94d1700 1 -- 172.21.15.2:0/3040495642 shutdown_connections 2024-05-14T09:23:35.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.175+0000 7f1df94d1700 1 -- 172.21.15.2:0/3040495642 wait complete. 2024-05-14T09:23:35.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.175+0000 7f1df94d1700 1 Processor -- start 2024-05-14T09:23:35.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.175+0000 7f1df94d1700 1 -- start start 2024-05-14T09:23:35.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.176+0000 7f1df94d1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1df4107d50 0x7f1df419e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:35.640 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.176+0000 7f1df94d1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1df410df80 con 0x7f1df4107d50 2024-05-14T09:23:35.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.176+0000 7f1df2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1df4107d50 0x7f1df419e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:35.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.176+0000 7f1df2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1df4107d50 0x7f1df419e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38322/0 (socket says 172.21.15.2:38322) 2024-05-14T09:23:35.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.176+0000 7f1df2ffd700 1 -- 172.21.15.2:0/3543243184 learned_addr learned my addr 172.21.15.2:0/3543243184 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:35.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.176+0000 7f1df2ffd700 1 -- 172.21.15.2:0/3543243184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df419e550 con 0x7f1df4107d50 2024-05-14T09:23:35.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.177+0000 7f1df2ffd700 1 --2- 172.21.15.2:0/3543243184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1df4107d50 0x7f1df419e010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1de4011a20 tx=0x7f1de4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:35.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.177+0000 7f1df0ff9700 1 -- 172.21.15.2:0/3543243184 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1de40039e0 con 0x7f1df4107d50 2024-05-14T09:23:35.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.177+0000 7f1df94d1700 1 -- 172.21.15.2:0/3543243184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df419e7e0 con 0x7f1df4107d50 2024-05-14T09:23:35.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.177+0000 7f1df0ff9700 1 -- 172.21.15.2:0/3543243184 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1de401b8f0 con 0x7f1df4107d50 2024-05-14T09:23:35.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.177+0000 7f1df0ff9700 1 -- 172.21.15.2:0/3543243184 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1de4023770 con 0x7f1df4107d50 2024-05-14T09:23:35.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.177+0000 7f1df94d1700 1 -- 172.21.15.2:0/3543243184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df419ec00 con 0x7f1df4107d50 2024-05-14T09:23:35.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.178+0000 7f1df0ff9700 1 -- 172.21.15.2:0/3543243184 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f1de402c3f0 con 0x7f1df4107d50 2024-05-14T09:23:35.641 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.178+0000 7f1df0ff9700 1 --2- 172.21.15.2:0/3543243184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1de003e520 0x7f1de0040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:35.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.178+0000 7f1df94d1700 1 -- 172.21.15.2:0/3543243184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df410e100 con 0x7f1df4107d50 2024-05-14T09:23:35.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.179+0000 7f1df0ff9700 1 -- 172.21.15.2:0/3543243184 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1de4054f80 con 0x7f1df4107d50 2024-05-14T09:23:35.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.179+0000 7f1debfff700 1 --2- 172.21.15.2:0/3543243184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1de003e520 0x7f1de0040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:35.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.179+0000 7f1debfff700 1 --2- 172.21.15.2:0/3543243184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1de003e520 0x7f1de0040a30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc006fd0 tx=0x7f1ddc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:35.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.184+0000 7f1df0ff9700 1 -- 172.21.15.2:0/3543243184 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1de401f030 con 0x7f1df4107d50 2024-05-14T09:23:35.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.403+0000 7f1df94d1700 1 -- 172.21.15.2:0/3543243184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f1df41a1570 con 0x7f1df4107d50 2024-05-14T09:23:35.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.403+0000 7f1df0ff9700 1 -- 172.21.15.2:0/3543243184 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f1de4053300 con 0x7f1df4107d50 2024-05-14T09:23:35.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.406+0000 7f1df94d1700 1 -- 172.21.15.2:0/3543243184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1de003e520 msgr2=0x7f1de0040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:35.642 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.406+0000 7f1df94d1700 1 --2- 172.21.15.2:0/3543243184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1de003e520 0x7f1de0040a30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc006fd0 tx=0x7f1ddc006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:35.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.406+0000 7f1df94d1700 1 -- 172.21.15.2:0/3543243184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1df4107d50 msgr2=0x7f1df419e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:35.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.406+0000 7f1df94d1700 1 --2- 172.21.15.2:0/3543243184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1df4107d50 0x7f1df419e010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1de4011a20 tx=0x7f1de4011b00 comp rx=0 tx=0).stop 2024-05-14T09:23:35.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.406+0000 7f1df94d1700 1 -- 172.21.15.2:0/3543243184 shutdown_connections 2024-05-14T09:23:35.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.406+0000 7f1df94d1700 1 --2- 172.21.15.2:0/3543243184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1de003e520 0x7f1de0040a30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:35.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.406+0000 7f1df94d1700 1 --2- 172.21.15.2:0/3543243184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1df4107d50 0x7f1df419e010 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:35.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.406+0000 7f1df94d1700 1 -- 172.21.15.2:0/3543243184 >> 172.21.15.2:0/3543243184 conn(0x7f1df4103760 msgr2=0x7f1df4104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:35.643 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.406+0000 7f1df94d1700 1 -- 172.21.15.2:0/3543243184 shutdown_connections 2024-05-14T09:23:35.644 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:34.406+0000 7f1df94d1700 1 -- 172.21.15.2:0/3543243184 wait complete. 2024-05-14T09:23:35.647 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T09:23:35.647 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-05-14T09:23:35.656 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T09:23:35.656 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-05-14T09:23:35.656 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-05-14T09:23:35.656 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-14T09:23:35.658 INFO:teuthology.orchestra.run.smithi002.stdout:Ceph Dashboard is now available at: 2024-05-14T09:23:35.658 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:23:35.658 INFO:teuthology.orchestra.run.smithi002.stdout: URL: https://smithi002.front.sepia.ceph.com:8443/ 2024-05-14T09:23:35.658 INFO:teuthology.orchestra.run.smithi002.stdout: User: admin 2024-05-14T09:23:35.659 INFO:teuthology.orchestra.run.smithi002.stdout: Password: fek2b5atyy 2024-05-14T09:23:35.659 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:23:35.659 INFO:teuthology.orchestra.run.smithi002.stdout:Saving cluster configuration to /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config directory 2024-05-14T09:23:35.660 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling autotune for osd_memory_target 2024-05-14T09:23:36.860 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:36 smithi002 ceph-mon[31557]: mgrmap e13: x(active, since 6s) 2024-05-14T09:23:36.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.052+0000 7fc61ad81700 1 Processor -- start 2024-05-14T09:23:36.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.052+0000 7fc61ad81700 1 -- start start 2024-05-14T09:23:36.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.052+0000 7fc61ad81700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc61407f6f0 0x7fc61407db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:36.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.052+0000 7fc61ad81700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc61407e0f0 con 0x7fc61407f6f0 2024-05-14T09:23:36.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.052+0000 7fc618b1d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc61407f6f0 0x7fc61407db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:36.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.052+0000 7fc618b1d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc61407f6f0 0x7fc61407db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38326/0 (socket says 172.21.15.2:38326) 2024-05-14T09:23:36.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.052+0000 7fc618b1d700 1 -- 172.21.15.2:0/2282305529 learned_addr learned my addr 172.21.15.2:0/2282305529 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:36.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.052+0000 7fc618b1d700 1 -- 172.21.15.2:0/2282305529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61407e270 con 0x7fc61407f6f0 2024-05-14T09:23:36.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.053+0000 7fc618b1d700 1 --2- 172.21.15.2:0/2282305529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc61407f6f0 0x7fc61407db20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc6080098a0 tx=0x7fc608009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9661ed7814d5727c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:36.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.053+0000 7fc6137fe700 1 -- 172.21.15.2:0/2282305529 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc608004160 con 0x7fc61407f6f0 2024-05-14T09:23:36.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.053+0000 7fc6137fe700 1 -- 172.21.15.2:0/2282305529 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc608004300 con 0x7fc61407f6f0 2024-05-14T09:23:36.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.053+0000 7fc6137fe700 1 -- 172.21.15.2:0/2282305529 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc608011a20 con 0x7fc61407f6f0 2024-05-14T09:23:36.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.054+0000 7fc61ad81700 1 -- 172.21.15.2:0/2282305529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc61407f6f0 msgr2=0x7fc61407db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:36.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.054+0000 7fc61ad81700 1 --2- 172.21.15.2:0/2282305529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc61407f6f0 0x7fc61407db20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc6080098a0 tx=0x7fc608009bb0 comp rx=0 tx=0).stop 2024-05-14T09:23:36.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.054+0000 7fc61ad81700 1 -- 172.21.15.2:0/2282305529 shutdown_connections 2024-05-14T09:23:36.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.054+0000 7fc61ad81700 1 --2- 172.21.15.2:0/2282305529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc61407f6f0 0x7fc61407db20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:36.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.054+0000 7fc61ad81700 1 -- 172.21.15.2:0/2282305529 >> 172.21.15.2:0/2282305529 conn(0x7fc6141050b0 msgr2=0x7fc614107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.054+0000 7fc61ad81700 1 -- 172.21.15.2:0/2282305529 shutdown_connections 2024-05-14T09:23:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.054+0000 7fc61ad81700 1 -- 172.21.15.2:0/2282305529 wait complete. 2024-05-14T09:23:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.054+0000 7fc61ad81700 1 Processor -- start 2024-05-14T09:23:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.054+0000 7fc61ad81700 1 -- start start 2024-05-14T09:23:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.055+0000 7fc61ad81700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc61407f6f0 0x7fc61419ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.055+0000 7fc61ad81700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc614110060 con 0x7fc61407f6f0 2024-05-14T09:23:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.055+0000 7fc618b1d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc61407f6f0 0x7fc61419ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.055+0000 7fc618b1d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc61407f6f0 0x7fc61419ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38340/0 (socket says 172.21.15.2:38340) 2024-05-14T09:23:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.055+0000 7fc618b1d700 1 -- 172.21.15.2:0/67640430 learned_addr learned my addr 172.21.15.2:0/67640430 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.055+0000 7fc618b1d700 1 -- 172.21.15.2:0/67640430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6141a0510 con 0x7fc61407f6f0 2024-05-14T09:23:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.055+0000 7fc618b1d700 1 --2- 172.21.15.2:0/67640430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc61407f6f0 0x7fc61419ffd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc608000c00 tx=0x7fc608011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.055+0000 7fc611ffb700 1 -- 172.21.15.2:0/67640430 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc608003aa0 con 0x7fc61407f6f0 2024-05-14T09:23:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.055+0000 7fc611ffb700 1 -- 172.21.15.2:0/67640430 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc60801b8f0 con 0x7fc61407f6f0 2024-05-14T09:23:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.055+0000 7fc61ad81700 1 -- 172.21.15.2:0/67640430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6141a07a0 con 0x7fc61407f6f0 2024-05-14T09:23:36.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.056+0000 7fc611ffb700 1 -- 172.21.15.2:0/67640430 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc608023890 con 0x7fc61407f6f0 2024-05-14T09:23:36.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.056+0000 7fc61ad81700 1 -- 172.21.15.2:0/67640430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6141a33b0 con 0x7fc61407f6f0 2024-05-14T09:23:36.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.057+0000 7fc61ad81700 1 -- 172.21.15.2:0/67640430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc614064e20 con 0x7fc61407f6f0 2024-05-14T09:23:36.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.057+0000 7fc611ffb700 1 -- 172.21.15.2:0/67640430 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fc60802c3f0 con 0x7fc61407f6f0 2024-05-14T09:23:36.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.057+0000 7fc611ffb700 1 --2- 172.21.15.2:0/67640430 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc60403e560 0x7fc604040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:36.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.057+0000 7fc611ffb700 1 -- 172.21.15.2:0/67640430 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc608053fe0 con 0x7fc61407f6f0 2024-05-14T09:23:36.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.057+0000 7fc613fff700 1 --2- 172.21.15.2:0/67640430 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc60403e560 0x7fc604040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:36.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.057+0000 7fc613fff700 1 --2- 172.21.15.2:0/67640430 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc60403e560 0x7fc604040a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc600006fd0 tx=0x7fc600006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:36.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.062+0000 7fc611ffb700 1 -- 172.21.15.2:0/67640430 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc60801f030 con 0x7fc61407f6f0 2024-05-14T09:23:36.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.275+0000 7fc61ad81700 1 -- 172.21.15.2:0/67640430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fc6141a3580 con 0x7fc61407f6f0 2024-05-14T09:23:36.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.276+0000 7fc611ffb700 1 -- 172.21.15.2:0/67640430 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fc608010380 con 0x7fc61407f6f0 2024-05-14T09:23:36.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.278+0000 7fc61ad81700 1 -- 172.21.15.2:0/67640430 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc60403e560 msgr2=0x7fc604040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:36.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.278+0000 7fc61ad81700 1 --2- 172.21.15.2:0/67640430 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc60403e560 0x7fc604040a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc600006fd0 tx=0x7fc600006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:36.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.278+0000 7fc61ad81700 1 -- 172.21.15.2:0/67640430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc61407f6f0 msgr2=0x7fc61419ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:36.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.278+0000 7fc61ad81700 1 --2- 172.21.15.2:0/67640430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc61407f6f0 0x7fc61419ffd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc608000c00 tx=0x7fc608011aa0 comp rx=0 tx=0).stop 2024-05-14T09:23:36.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.278+0000 7fc61ad81700 1 -- 172.21.15.2:0/67640430 shutdown_connections 2024-05-14T09:23:36.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.278+0000 7fc61ad81700 1 --2- 172.21.15.2:0/67640430 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc60403e560 0x7fc604040a70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:36.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.278+0000 7fc61ad81700 1 --2- 172.21.15.2:0/67640430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc61407f6f0 0x7fc61419ffd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:36.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.278+0000 7fc61ad81700 1 -- 172.21.15.2:0/67640430 >> 172.21.15.2:0/67640430 conn(0x7fc6141050b0 msgr2=0x7fc614105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:36.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.279+0000 7fc61ad81700 1 -- 172.21.15.2:0/67640430 shutdown_connections 2024-05-14T09:23:36.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:36.279+0000 7fc61ad81700 1 -- 172.21.15.2:0/67640430 wait complete. 2024-05-14T09:23:38.169 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.322+0000 7f1ca82e0700 1 Processor -- start 2024-05-14T09:23:38.175 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.322+0000 7f1ca82e0700 1 -- start start 2024-05-14T09:23:38.175 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.322+0000 7f1ca82e0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ca010a5e0 0x7f1ca010ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:38.175 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.323+0000 7f1ca82e0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca007cdf0 con 0x7f1ca010a5e0 2024-05-14T09:23:38.175 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.323+0000 7f1ca607c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ca010a5e0 0x7f1ca010ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:38.175 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.323+0000 7f1ca607c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ca010a5e0 0x7f1ca010ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38350/0 (socket says 172.21.15.2:38350) 2024-05-14T09:23:38.175 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.323+0000 7f1ca607c700 1 -- 172.21.15.2:0/2500165613 learned_addr learned my addr 172.21.15.2:0/2500165613 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:38.175 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.323+0000 7f1ca607c700 1 -- 172.21.15.2:0/2500165613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca010cf40 con 0x7f1ca010a5e0 2024-05-14T09:23:38.176 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.323+0000 7f1ca607c700 1 --2- 172.21.15.2:0/2500165613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ca010a5e0 0x7f1ca010ca00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009e80 tx=0x7f1c9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d22216602f58b14 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:38.176 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.324+0000 7f1ca507a700 1 -- 172.21.15.2:0/2500165613 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c90017070 con 0x7f1ca010a5e0 2024-05-14T09:23:38.176 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.324+0000 7f1ca507a700 1 -- 172.21.15.2:0/2500165613 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c9000b8c0 con 0x7f1ca010a5e0 2024-05-14T09:23:38.176 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.324+0000 7f1ca507a700 1 -- 172.21.15.2:0/2500165613 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c90004490 con 0x7f1ca010a5e0 2024-05-14T09:23:38.176 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.324+0000 7f1ca82e0700 1 -- 172.21.15.2:0/2500165613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ca010a5e0 msgr2=0x7f1ca010ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:38.176 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.324+0000 7f1ca82e0700 1 --2- 172.21.15.2:0/2500165613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ca010a5e0 0x7f1ca010ca00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009e80 tx=0x7f1c9000b330 comp rx=0 tx=0).stop 2024-05-14T09:23:38.176 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.324+0000 7f1ca82e0700 1 -- 172.21.15.2:0/2500165613 shutdown_connections 2024-05-14T09:23:38.176 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.324+0000 7f1ca82e0700 1 --2- 172.21.15.2:0/2500165613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ca010a5e0 0x7f1ca010ca00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:38.176 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.324+0000 7f1ca82e0700 1 -- 172.21.15.2:0/2500165613 >> 172.21.15.2:0/2500165613 conn(0x7f1ca0103fe0 msgr2=0x7f1ca0106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:38.176 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.325+0000 7f1ca82e0700 1 -- 172.21.15.2:0/2500165613 shutdown_connections 2024-05-14T09:23:38.176 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.325+0000 7f1ca82e0700 1 -- 172.21.15.2:0/2500165613 wait complete. 2024-05-14T09:23:38.176 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.325+0000 7f1ca82e0700 1 Processor -- start 2024-05-14T09:23:38.176 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.325+0000 7f1ca82e0700 1 -- start start 2024-05-14T09:23:38.177 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.326+0000 7f1ca82e0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ca010a5e0 0x7f1ca019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:38.177 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.326+0000 7f1ca82e0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca010de80 con 0x7f1ca010a5e0 2024-05-14T09:23:38.177 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.326+0000 7f1ca607c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ca010a5e0 0x7f1ca019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:38.177 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.326+0000 7f1ca607c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ca010a5e0 0x7f1ca019de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38352/0 (socket says 172.21.15.2:38352) 2024-05-14T09:23:38.177 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.326+0000 7f1ca607c700 1 -- 172.21.15.2:0/2689017336 learned_addr learned my addr 172.21.15.2:0/2689017336 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:38.177 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.326+0000 7f1ca607c700 1 -- 172.21.15.2:0/2689017336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca019e3a0 con 0x7f1ca010a5e0 2024-05-14T09:23:38.177 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.326+0000 7f1ca607c700 1 --2- 172.21.15.2:0/2689017336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ca010a5e0 0x7f1ca019de60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1c900072a0 tx=0x7f1c9000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:38.177 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.326+0000 7f1c9f7fe700 1 -- 172.21.15.2:0/2689017336 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c90020020 con 0x7f1ca010a5e0 2024-05-14T09:23:38.177 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.326+0000 7f1c9f7fe700 1 -- 172.21.15.2:0/2689017336 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c90012430 con 0x7f1ca010a5e0 2024-05-14T09:23:38.177 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.326+0000 7f1ca82e0700 1 -- 172.21.15.2:0/2689017336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca019e630 con 0x7f1ca010a5e0 2024-05-14T09:23:38.177 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.326+0000 7f1c9f7fe700 1 -- 172.21.15.2:0/2689017336 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c90017040 con 0x7f1ca010a5e0 2024-05-14T09:23:38.177 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.326+0000 7f1ca82e0700 1 -- 172.21.15.2:0/2689017336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca01a1240 con 0x7f1ca010a5e0 2024-05-14T09:23:38.177 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.327+0000 7f1ca82e0700 1 -- 172.21.15.2:0/2689017336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca0064e20 con 0x7f1ca010a5e0 2024-05-14T09:23:38.178 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.327+0000 7f1c9f7fe700 1 -- 172.21.15.2:0/2689017336 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f1c90023020 con 0x7f1ca010a5e0 2024-05-14T09:23:38.178 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.328+0000 7f1c9f7fe700 1 --2- 172.21.15.2:0/2689017336 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c8803e570 0x7f1c88040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:38.178 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.328+0000 7f1c9f7fe700 1 -- 172.21.15.2:0/2689017336 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1c9000f050 con 0x7f1ca010a5e0 2024-05-14T09:23:38.178 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.328+0000 7f1ca587b700 1 --2- 172.21.15.2:0/2689017336 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c8803e570 0x7f1c88040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:38.178 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.328+0000 7f1ca587b700 1 --2- 172.21.15.2:0/2689017336 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c8803e570 0x7f1c88040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1c94006fd0 tx=0x7f1c94006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:38.178 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.333+0000 7f1c9f7fe700 1 -- 172.21.15.2:0/2689017336 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1c9002a020 con 0x7f1ca010a5e0 2024-05-14T09:23:38.178 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.669+0000 7f1ca82e0700 1 -- 172.21.15.2:0/2689017336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f1ca01a1410 con 0x7f1ca010a5e0 2024-05-14T09:23:38.178 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.671+0000 7f1c9f7fe700 1 -- 172.21.15.2:0/2689017336 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7f1c90012650 con 0x7f1ca010a5e0 2024-05-14T09:23:38.178 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.674+0000 7f1ca82e0700 1 -- 172.21.15.2:0/2689017336 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c8803e570 msgr2=0x7f1c88040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:38.178 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.674+0000 7f1ca82e0700 1 --2- 172.21.15.2:0/2689017336 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c8803e570 0x7f1c88040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1c94006fd0 tx=0x7f1c94006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:38.178 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.674+0000 7f1ca82e0700 1 -- 172.21.15.2:0/2689017336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ca010a5e0 msgr2=0x7f1ca019de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:38.179 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.674+0000 7f1ca82e0700 1 --2- 172.21.15.2:0/2689017336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ca010a5e0 0x7f1ca019de60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1c900072a0 tx=0x7f1c9000bd50 comp rx=0 tx=0).stop 2024-05-14T09:23:38.179 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.674+0000 7f1ca82e0700 1 -- 172.21.15.2:0/2689017336 shutdown_connections 2024-05-14T09:23:38.179 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.674+0000 7f1ca82e0700 1 --2- 172.21.15.2:0/2689017336 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c8803e570 0x7f1c88040a80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:38.179 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.674+0000 7f1ca82e0700 1 --2- 172.21.15.2:0/2689017336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ca010a5e0 0x7f1ca019de60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:38.179 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.674+0000 7f1ca82e0700 1 -- 172.21.15.2:0/2689017336 >> 172.21.15.2:0/2689017336 conn(0x7f1ca0103fe0 msgr2=0x7f1ca01048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:38.179 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.674+0000 7f1ca82e0700 1 -- 172.21.15.2:0/2689017336 shutdown_connections 2024-05-14T09:23:38.179 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T09:23:37.674+0000 7f1ca82e0700 1 -- 172.21.15.2:0/2689017336 wait complete. 2024-05-14T09:23:38.179 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-14T09:23:38.179 INFO:teuthology.orchestra.run.smithi002.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-14T09:23:38.179 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:23:38.179 INFO:teuthology.orchestra.run.smithi002.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-14T09:23:38.179 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:23:38.179 INFO:teuthology.orchestra.run.smithi002.stdout:Or, if you are only running a single cluster on this host: 2024-05-14T09:23:38.180 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:23:38.180 INFO:teuthology.orchestra.run.smithi002.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-14T09:23:38.180 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:23:38.180 INFO:teuthology.orchestra.run.smithi002.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-14T09:23:38.180 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:23:38.180 INFO:teuthology.orchestra.run.smithi002.stdout: ceph telemetry on 2024-05-14T09:23:38.180 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:23:38.180 INFO:teuthology.orchestra.run.smithi002.stdout:For more information see: 2024-05-14T09:23:38.180 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:23:38.180 INFO:teuthology.orchestra.run.smithi002.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-14T09:23:38.180 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:23:38.180 INFO:teuthology.orchestra.run.smithi002.stdout:Bootstrap complete. 2024-05-14T09:23:38.214 INFO:tasks.cephadm:Fetching config... 2024-05-14T09:23:38.214 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:23:38.214 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-14T09:23:38.231 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-14T09:23:38.231 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:23:38.231 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-14T09:23:38.287 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-14T09:23:38.287 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:23:38.287 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-14T09:23:38.352 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-14T09:23:38.353 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:23:38.353 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-14T09:23:38.408 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-14T09:23:38.409 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHo3ka/VaO+EpZB3Hau8HJ1zdu6l4P3i0W54/WnuUjqLq/m8Zyh4jysm7SuzoJAPmbo2+tLgOZpNzDhpG61rDOMsy5LYPgdx5HzL5KD3hqLLgHCbQe3JTTo83zEmdnHAbZOAqsRhX+0plI3gvXtoh7uas3MnMFvzupIsCwwnfTYLPvuJSULpQnvohOKLNoc+ouJUjQLpwbsV+0h587L6oi+7TxIVpDhk7J7nqwl6HBGQg9CARTkb+bC825EnA4F5CEHxLPf/x9OjXyqMc1LNav7cRkZDPChzSJ2Q+8HeUwdv93SIMLvvfvR+WxCHrA/uy6Ck7Q21FOxT19C9YVykh0jjokA+x3N8k+WbiU/C3yXJMzTOfd4iPoc24ipV4FltecPXS6FJcEbVJFt+cbMlQiUO9pukrfnIDyM5bsrKYcxT2P2t7VGxmSAI5SB/olUy9bAxYeuPlH9AJvI9qqWYHslrY2beC3uj45S7vjSZh6ZzOIansSeG6WdN+rQULYqZ8= ceph-50d5aebe-11d3-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-14T09:23:38.490 INFO:teuthology.orchestra.run.smithi002.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHo3ka/VaO+EpZB3Hau8HJ1zdu6l4P3i0W54/WnuUjqLq/m8Zyh4jysm7SuzoJAPmbo2+tLgOZpNzDhpG61rDOMsy5LYPgdx5HzL5KD3hqLLgHCbQe3JTTo83zEmdnHAbZOAqsRhX+0plI3gvXtoh7uas3MnMFvzupIsCwwnfTYLPvuJSULpQnvohOKLNoc+ouJUjQLpwbsV+0h587L6oi+7TxIVpDhk7J7nqwl6HBGQg9CARTkb+bC825EnA4F5CEHxLPf/x9OjXyqMc1LNav7cRkZDPChzSJ2Q+8HeUwdv93SIMLvvfvR+WxCHrA/uy6Ck7Q21FOxT19C9YVykh0jjokA+x3N8k+WbiU/C3yXJMzTOfd4iPoc24ipV4FltecPXS6FJcEbVJFt+cbMlQiUO9pukrfnIDyM5bsrKYcxT2P2t7VGxmSAI5SB/olUy9bAxYeuPlH9AJvI9qqWYHslrY2beC3uj45S7vjSZh6ZzOIansSeG6WdN+rQULYqZ8= ceph-50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:23:38.503 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHo3ka/VaO+EpZB3Hau8HJ1zdu6l4P3i0W54/WnuUjqLq/m8Zyh4jysm7SuzoJAPmbo2+tLgOZpNzDhpG61rDOMsy5LYPgdx5HzL5KD3hqLLgHCbQe3JTTo83zEmdnHAbZOAqsRhX+0plI3gvXtoh7uas3MnMFvzupIsCwwnfTYLPvuJSULpQnvohOKLNoc+ouJUjQLpwbsV+0h587L6oi+7TxIVpDhk7J7nqwl6HBGQg9CARTkb+bC825EnA4F5CEHxLPf/x9OjXyqMc1LNav7cRkZDPChzSJ2Q+8HeUwdv93SIMLvvfvR+WxCHrA/uy6Ck7Q21FOxT19C9YVykh0jjokA+x3N8k+WbiU/C3yXJMzTOfd4iPoc24ipV4FltecPXS6FJcEbVJFt+cbMlQiUO9pukrfnIDyM5bsrKYcxT2P2t7VGxmSAI5SB/olUy9bAxYeuPlH9AJvI9qqWYHslrY2beC3uj45S7vjSZh6ZzOIansSeG6WdN+rQULYqZ8= ceph-50d5aebe-11d3-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-14T09:23:38.544 INFO:teuthology.orchestra.run.smithi026.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHo3ka/VaO+EpZB3Hau8HJ1zdu6l4P3i0W54/WnuUjqLq/m8Zyh4jysm7SuzoJAPmbo2+tLgOZpNzDhpG61rDOMsy5LYPgdx5HzL5KD3hqLLgHCbQe3JTTo83zEmdnHAbZOAqsRhX+0plI3gvXtoh7uas3MnMFvzupIsCwwnfTYLPvuJSULpQnvohOKLNoc+ouJUjQLpwbsV+0h587L6oi+7TxIVpDhk7J7nqwl6HBGQg9CARTkb+bC825EnA4F5CEHxLPf/x9OjXyqMc1LNav7cRkZDPChzSJ2Q+8HeUwdv93SIMLvvfvR+WxCHrA/uy6Ck7Q21FOxT19C9YVykh0jjokA+x3N8k+WbiU/C3yXJMzTOfd4iPoc24ipV4FltecPXS6FJcEbVJFt+cbMlQiUO9pukrfnIDyM5bsrKYcxT2P2t7VGxmSAI5SB/olUy9bAxYeuPlH9AJvI9qqWYHslrY2beC3uj45S7vjSZh6ZzOIansSeG6WdN+rQULYqZ8= ceph-50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:23:38.556 DEBUG:teuthology.orchestra.run.smithi029:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHo3ka/VaO+EpZB3Hau8HJ1zdu6l4P3i0W54/WnuUjqLq/m8Zyh4jysm7SuzoJAPmbo2+tLgOZpNzDhpG61rDOMsy5LYPgdx5HzL5KD3hqLLgHCbQe3JTTo83zEmdnHAbZOAqsRhX+0plI3gvXtoh7uas3MnMFvzupIsCwwnfTYLPvuJSULpQnvohOKLNoc+ouJUjQLpwbsV+0h587L6oi+7TxIVpDhk7J7nqwl6HBGQg9CARTkb+bC825EnA4F5CEHxLPf/x9OjXyqMc1LNav7cRkZDPChzSJ2Q+8HeUwdv93SIMLvvfvR+WxCHrA/uy6Ck7Q21FOxT19C9YVykh0jjokA+x3N8k+WbiU/C3yXJMzTOfd4iPoc24ipV4FltecPXS6FJcEbVJFt+cbMlQiUO9pukrfnIDyM5bsrKYcxT2P2t7VGxmSAI5SB/olUy9bAxYeuPlH9AJvI9qqWYHslrY2beC3uj45S7vjSZh6ZzOIansSeG6WdN+rQULYqZ8= ceph-50d5aebe-11d3-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-14T09:23:38.598 INFO:teuthology.orchestra.run.smithi029.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHo3ka/VaO+EpZB3Hau8HJ1zdu6l4P3i0W54/WnuUjqLq/m8Zyh4jysm7SuzoJAPmbo2+tLgOZpNzDhpG61rDOMsy5LYPgdx5HzL5KD3hqLLgHCbQe3JTTo83zEmdnHAbZOAqsRhX+0plI3gvXtoh7uas3MnMFvzupIsCwwnfTYLPvuJSULpQnvohOKLNoc+ouJUjQLpwbsV+0h587L6oi+7TxIVpDhk7J7nqwl6HBGQg9CARTkb+bC825EnA4F5CEHxLPf/x9OjXyqMc1LNav7cRkZDPChzSJ2Q+8HeUwdv93SIMLvvfvR+WxCHrA/uy6Ck7Q21FOxT19C9YVykh0jjokA+x3N8k+WbiU/C3yXJMzTOfd4iPoc24ipV4FltecPXS6FJcEbVJFt+cbMlQiUO9pukrfnIDyM5bsrKYcxT2P2t7VGxmSAI5SB/olUy9bAxYeuPlH9AJvI9qqWYHslrY2beC3uj45S7vjSZh6ZzOIansSeG6WdN+rQULYqZ8= ceph-50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:23:38.611 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-14T09:23:38.858 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:23:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:38 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2689017336' entity='client.admin' 2024-05-14T09:23:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.193+0000 7f2285f53700 1 -- 172.21.15.2:0/3733820310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2280102ae0 msgr2=0x7f2280102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.193+0000 7f2285f53700 1 --2- 172.21.15.2:0/3733820310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2280102ae0 0x7f2280102ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2270009910 tx=0x7f2270009c20 comp rx=0 tx=0).stop 2024-05-14T09:23:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.193+0000 7f2285f53700 1 -- 172.21.15.2:0/3733820310 shutdown_connections 2024-05-14T09:23:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.193+0000 7f2285f53700 1 --2- 172.21.15.2:0/3733820310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2280102ae0 0x7f2280102ef0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.193+0000 7f2285f53700 1 -- 172.21.15.2:0/3733820310 >> 172.21.15.2:0/3733820310 conn(0x7f22800fe140 msgr2=0x7f22801005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.194+0000 7f2285f53700 1 -- 172.21.15.2:0/3733820310 shutdown_connections 2024-05-14T09:23:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.194+0000 7f2285f53700 1 -- 172.21.15.2:0/3733820310 wait complete. 2024-05-14T09:23:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.194+0000 7f2285f53700 1 Processor -- start 2024-05-14T09:23:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.194+0000 7f2285f53700 1 -- start start 2024-05-14T09:23:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.195+0000 7f2285f53700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2280102ae0 0x7f2280198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.195+0000 7f2285f53700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2280108aa0 con 0x7f2280102ae0 2024-05-14T09:23:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.195+0000 7f227f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2280102ae0 0x7f2280198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.195+0000 7f227f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2280102ae0 0x7f2280198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34372/0 (socket says 172.21.15.2:34372) 2024-05-14T09:23:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.195+0000 7f227f7fe700 1 -- 172.21.15.2:0/1203966770 learned_addr learned my addr 172.21.15.2:0/1203966770 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.195+0000 7f227f7fe700 1 -- 172.21.15.2:0/1203966770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2280198b40 con 0x7f2280102ae0 2024-05-14T09:23:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.195+0000 7f227f7fe700 1 --2- 172.21.15.2:0/1203966770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2280102ae0 0x7f2280198600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f22700057d0 tx=0x7f2270005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:41.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.195+0000 7f227cff9700 1 -- 172.21.15.2:0/1203966770 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2270027070 con 0x7f2280102ae0 2024-05-14T09:23:41.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.195+0000 7f2285f53700 1 -- 172.21.15.2:0/1203966770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2280198dd0 con 0x7f2280102ae0 2024-05-14T09:23:41.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.195+0000 7f227cff9700 1 -- 172.21.15.2:0/1203966770 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f227000be10 con 0x7f2280102ae0 2024-05-14T09:23:41.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.195+0000 7f227cff9700 1 -- 172.21.15.2:0/1203966770 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2270022dd0 con 0x7f2280102ae0 2024-05-14T09:23:41.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.195+0000 7f2285f53700 1 -- 172.21.15.2:0/1203966770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f228019b9e0 con 0x7f2280102ae0 2024-05-14T09:23:41.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.196+0000 7f227cff9700 1 -- 172.21.15.2:0/1203966770 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f227002a070 con 0x7f2280102ae0 2024-05-14T09:23:41.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.196+0000 7f2285f53700 1 -- 172.21.15.2:0/1203966770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2280109220 con 0x7f2280102ae0 2024-05-14T09:23:41.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.197+0000 7f227cff9700 1 --2- 172.21.15.2:0/1203966770 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f226803e4c0 0x7f22680409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:41.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.197+0000 7f227cff9700 1 -- 172.21.15.2:0/1203966770 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f227001e030 con 0x7f2280102ae0 2024-05-14T09:23:41.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.197+0000 7f227effd700 1 --2- 172.21.15.2:0/1203966770 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f226803e4c0 0x7f22680409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:41.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.197+0000 7f227effd700 1 --2- 172.21.15.2:0/1203966770 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f226803e4c0 0x7f22680409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2274006fd0 tx=0x7f2274006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:41.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.202+0000 7f227cff9700 1 -- 172.21.15.2:0/1203966770 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2270013070 con 0x7f2280102ae0 2024-05-14T09:23:41.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.420+0000 7f2285f53700 1 -- 172.21.15.2:0/1203966770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f228002e700 con 0x7f2280102ae0 2024-05-14T09:23:41.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.422+0000 7f227cff9700 1 -- 172.21.15.2:0/1203966770 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f22700310c0 con 0x7f2280102ae0 2024-05-14T09:23:41.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.425+0000 7f2285f53700 1 -- 172.21.15.2:0/1203966770 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f226803e4c0 msgr2=0x7f22680409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:41.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.425+0000 7f2285f53700 1 --2- 172.21.15.2:0/1203966770 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f226803e4c0 0x7f22680409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2274006fd0 tx=0x7f2274006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:41.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.425+0000 7f2285f53700 1 -- 172.21.15.2:0/1203966770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2280102ae0 msgr2=0x7f2280198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:41.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.425+0000 7f2285f53700 1 --2- 172.21.15.2:0/1203966770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2280102ae0 0x7f2280198600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f22700057d0 tx=0x7f2270005800 comp rx=0 tx=0).stop 2024-05-14T09:23:41.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.425+0000 7f2285f53700 1 -- 172.21.15.2:0/1203966770 shutdown_connections 2024-05-14T09:23:41.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.425+0000 7f2285f53700 1 --2- 172.21.15.2:0/1203966770 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f226803e4c0 0x7f22680409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:41.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.425+0000 7f2285f53700 1 --2- 172.21.15.2:0/1203966770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2280102ae0 0x7f2280198600 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:41.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.425+0000 7f2285f53700 1 -- 172.21.15.2:0/1203966770 >> 172.21.15.2:0/1203966770 conn(0x7f22800fe140 msgr2=0x7f22800fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:41.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.425+0000 7f2285f53700 1 -- 172.21.15.2:0/1203966770 shutdown_connections 2024-05-14T09:23:41.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:41.425+0000 7f2285f53700 1 -- 172.21.15.2:0/1203966770 wait complete. 2024-05-14T09:23:41.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:41.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:41.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi002", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:23:41.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:41.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:23:41.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:41.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:41 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1203966770' entity='client.admin' 2024-05-14T09:23:41.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:23:41.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:42.024 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-14T09:23:42.025 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-14T09:23:42.269 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:23:43.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.735+0000 7f9b17e89700 1 -- 172.21.15.2:0/2209910629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b10102af0 msgr2=0x7f9b10102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:43.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.735+0000 7f9b17e89700 1 --2- 172.21.15.2:0/2209910629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b10102af0 0x7f9b10102f00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00b0a0 tx=0x7f9b0c00b3b0 comp rx=0 tx=0).stop 2024-05-14T09:23:43.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.735+0000 7f9b17e89700 1 -- 172.21.15.2:0/2209910629 shutdown_connections 2024-05-14T09:23:43.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.735+0000 7f9b17e89700 1 --2- 172.21.15.2:0/2209910629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b10102af0 0x7f9b10102f00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:43.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.735+0000 7f9b17e89700 1 -- 172.21.15.2:0/2209910629 >> 172.21.15.2:0/2209910629 conn(0x7f9b100fe130 msgr2=0x7f9b101005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:43.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.735+0000 7f9b17e89700 1 -- 172.21.15.2:0/2209910629 shutdown_connections 2024-05-14T09:23:43.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.735+0000 7f9b17e89700 1 -- 172.21.15.2:0/2209910629 wait complete. 2024-05-14T09:23:43.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.736+0000 7f9b17e89700 1 Processor -- start 2024-05-14T09:23:43.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.736+0000 7f9b17e89700 1 -- start start 2024-05-14T09:23:43.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.736+0000 7f9b17e89700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b10102af0 0x7f9b1006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:43.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.736+0000 7f9b17e89700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10108ab0 con 0x7f9b10102af0 2024-05-14T09:23:43.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.736+0000 7f9b15c25700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b10102af0 0x7f9b1006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:43.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.736+0000 7f9b15c25700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b10102af0 0x7f9b1006c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34400/0 (socket says 172.21.15.2:34400) 2024-05-14T09:23:43.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.736+0000 7f9b15c25700 1 -- 172.21.15.2:0/2825077890 learned_addr learned my addr 172.21.15.2:0/2825077890 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:43.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.736+0000 7f9b15c25700 1 -- 172.21.15.2:0/2825077890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1006ca20 con 0x7f9b10102af0 2024-05-14T09:23:43.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.737+0000 7f9b15c25700 1 --2- 172.21.15.2:0/2825077890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b10102af0 0x7f9b1006c4e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c015040 tx=0x7f9b0c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:43.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.737+0000 7f9b06ffd700 1 -- 172.21.15.2:0/2825077890 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b0c021020 con 0x7f9b10102af0 2024-05-14T09:23:43.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.737+0000 7f9b06ffd700 1 -- 172.21.15.2:0/2825077890 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b0c00bdd0 con 0x7f9b10102af0 2024-05-14T09:23:43.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.737+0000 7f9b17e89700 1 -- 172.21.15.2:0/2825077890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1019c430 con 0x7f9b10102af0 2024-05-14T09:23:43.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.737+0000 7f9b06ffd700 1 -- 172.21.15.2:0/2825077890 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b0c017040 con 0x7f9b10102af0 2024-05-14T09:23:43.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.737+0000 7f9b17e89700 1 -- 172.21.15.2:0/2825077890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b1019c890 con 0x7f9b10102af0 2024-05-14T09:23:43.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.738+0000 7f9b06ffd700 1 -- 172.21.15.2:0/2825077890 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f9b0c024020 con 0x7f9b10102af0 2024-05-14T09:23:43.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.738+0000 7f9b17e89700 1 -- 172.21.15.2:0/2825077890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b10109230 con 0x7f9b10102af0 2024-05-14T09:23:43.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.738+0000 7f9b06ffd700 1 --2- 172.21.15.2:0/2825077890 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9afc03e570 0x7f9afc040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:43.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.738+0000 7f9b06ffd700 1 -- 172.21.15.2:0/2825077890 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9b0c00f050 con 0x7f9b10102af0 2024-05-14T09:23:43.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.738+0000 7f9b15424700 1 --2- 172.21.15.2:0/2825077890 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9afc03e570 0x7f9afc040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:43.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.739+0000 7f9b15424700 1 --2- 172.21.15.2:0/2825077890 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9afc03e570 0x7f9afc040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9b00006fd0 tx=0x7f9b00006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:43.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.744+0000 7f9b06ffd700 1 -- 172.21.15.2:0/2825077890 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b0c02b020 con 0x7f9b10102af0 2024-05-14T09:23:43.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.961+0000 7f9b17e89700 1 -- 172.21.15.2:0/2825077890 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f9b1002e820 con 0x7f9afc03e570 2024-05-14T09:23:43.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.965+0000 7f9b06ffd700 1 -- 172.21.15.2:0/2825077890 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9b1002e820 con 0x7f9afc03e570 2024-05-14T09:23:43.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.968+0000 7f9b17e89700 1 -- 172.21.15.2:0/2825077890 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9afc03e570 msgr2=0x7f9afc040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:43.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.968+0000 7f9b17e89700 1 --2- 172.21.15.2:0/2825077890 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9afc03e570 0x7f9afc040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9b00006fd0 tx=0x7f9b00006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:43.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.968+0000 7f9b17e89700 1 -- 172.21.15.2:0/2825077890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b10102af0 msgr2=0x7f9b1006c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:43.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.968+0000 7f9b17e89700 1 --2- 172.21.15.2:0/2825077890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b10102af0 0x7f9b1006c4e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c015040 tx=0x7f9b0c003ce0 comp rx=0 tx=0).stop 2024-05-14T09:23:43.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.968+0000 7f9b17e89700 1 -- 172.21.15.2:0/2825077890 shutdown_connections 2024-05-14T09:23:43.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.968+0000 7f9b17e89700 1 --2- 172.21.15.2:0/2825077890 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9afc03e570 0x7f9afc040a80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:43.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.968+0000 7f9b17e89700 1 --2- 172.21.15.2:0/2825077890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b10102af0 0x7f9b1006c4e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:43.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.968+0000 7f9b17e89700 1 -- 172.21.15.2:0/2825077890 >> 172.21.15.2:0/2825077890 conn(0x7f9b100fe130 msgr2=0x7f9b100fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:43.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.968+0000 7f9b17e89700 1 -- 172.21.15.2:0/2825077890 shutdown_connections 2024-05-14T09:23:43.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:43.968+0000 7f9b17e89700 1 -- 172.21.15.2:0/2825077890 wait complete. 2024-05-14T09:23:44.705 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi026 2024-05-14T09:23:44.706 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-14T09:23:44.706 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/etc/ceph/ceph.conf 2024-05-14T09:23:44.726 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-14T09:23:44.726 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T09:23:44.784 INFO:tasks.cephadm:Adding host smithi026 to orchestrator... 2024-05-14T09:23:44.784 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch host add smithi026 2024-05-14T09:23:45.037 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:23:45.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:44 smithi002 ceph-mon[31557]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:23:45.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:45.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:23:45.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:23:45.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:23:45.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:44 smithi002 ceph-mon[31557]: Updating smithi002:/etc/ceph/ceph.conf 2024-05-14T09:23:45.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:44 smithi002 ceph-mon[31557]: Updating smithi002:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:23:46.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:46 smithi002 ceph-mon[31557]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-05-14T09:23:46.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:46 smithi002 ceph-mon[31557]: Updating smithi002:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T09:23:46.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:46.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:46.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:46.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.851+0000 7f89943f3700 1 -- 172.21.15.2:0/503720707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f898c107aa0 msgr2=0x7f898c107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:46.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.851+0000 7f89943f3700 1 --2- 172.21.15.2:0/503720707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f898c107aa0 0x7f898c107eb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8988009910 tx=0x7f8988009c20 comp rx=0 tx=0).stop 2024-05-14T09:23:46.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.851+0000 7f89943f3700 1 -- 172.21.15.2:0/503720707 shutdown_connections 2024-05-14T09:23:46.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.851+0000 7f89943f3700 1 --2- 172.21.15.2:0/503720707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f898c107aa0 0x7f898c107eb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:46.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.851+0000 7f89943f3700 1 -- 172.21.15.2:0/503720707 >> 172.21.15.2:0/503720707 conn(0x7f898c0fde30 msgr2=0x7f898c100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:46.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.851+0000 7f89943f3700 1 -- 172.21.15.2:0/503720707 shutdown_connections 2024-05-14T09:23:46.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.851+0000 7f89943f3700 1 -- 172.21.15.2:0/503720707 wait complete. 2024-05-14T09:23:46.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.852+0000 7f89943f3700 1 Processor -- start 2024-05-14T09:23:46.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.852+0000 7f89943f3700 1 -- start start 2024-05-14T09:23:46.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.852+0000 7f89943f3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f898c107aa0 0x7f898c19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:46.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.852+0000 7f89943f3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f898c10d030 con 0x7f898c107aa0 2024-05-14T09:23:46.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.852+0000 7f899218f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f898c107aa0 0x7f898c19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:46.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.852+0000 7f899218f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f898c107aa0 0x7f898c19e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34412/0 (socket says 172.21.15.2:34412) 2024-05-14T09:23:46.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.852+0000 7f899218f700 1 -- 172.21.15.2:0/4116088505 learned_addr learned my addr 172.21.15.2:0/4116088505 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:46.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.853+0000 7f899218f700 1 -- 172.21.15.2:0/4116088505 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898c19ea70 con 0x7f898c107aa0 2024-05-14T09:23:46.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.853+0000 7f899218f700 1 --2- 172.21.15.2:0/4116088505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f898c107aa0 0x7f898c19e530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f898800b5c0 tx=0x7f8988005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:46.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.853+0000 7f89837fe700 1 -- 172.21.15.2:0/4116088505 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8988026070 con 0x7f898c107aa0 2024-05-14T09:23:46.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.853+0000 7f89837fe700 1 -- 172.21.15.2:0/4116088505 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f898800b750 con 0x7f898c107aa0 2024-05-14T09:23:46.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.853+0000 7f89943f3700 1 -- 172.21.15.2:0/4116088505 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898c19ed00 con 0x7f898c107aa0 2024-05-14T09:23:46.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.853+0000 7f89837fe700 1 -- 172.21.15.2:0/4116088505 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89880037d0 con 0x7f898c107aa0 2024-05-14T09:23:46.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.853+0000 7f89943f3700 1 -- 172.21.15.2:0/4116088505 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898c1986c0 con 0x7f898c107aa0 2024-05-14T09:23:46.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.854+0000 7f89943f3700 1 -- 172.21.15.2:0/4116088505 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f898c104380 con 0x7f898c107aa0 2024-05-14T09:23:46.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.854+0000 7f89837fe700 1 -- 172.21.15.2:0/4116088505 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f8988029070 con 0x7f898c107aa0 2024-05-14T09:23:46.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.855+0000 7f89837fe700 1 --2- 172.21.15.2:0/4116088505 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f897803e510 0x7f8978040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:46.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.855+0000 7f89837fe700 1 -- 172.21.15.2:0/4116088505 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f898801e070 con 0x7f898c107aa0 2024-05-14T09:23:46.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.855+0000 7f899198e700 1 --2- 172.21.15.2:0/4116088505 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f897803e510 0x7f8978040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:46.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.855+0000 7f899198e700 1 --2- 172.21.15.2:0/4116088505 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f897803e510 0x7f8978040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f897c006fd0 tx=0x7f897c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:46.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:46.860+0000 7f89837fe700 1 -- 172.21.15.2:0/4116088505 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8988013070 con 0x7f898c107aa0 2024-05-14T09:23:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:47.079+0000 7f89943f3700 1 -- 172.21.15.2:0/4116088505 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi026", "target": ["mon-mgr", ""]}) -- 0x7f898c199bd0 con 0x7f897803e510 2024-05-14T09:23:47.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:47 smithi002 ceph-mon[31557]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi026", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:23:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:48 smithi002 ceph-mon[31557]: Deploying cephadm binary to smithi026 2024-05-14T09:23:50.174 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:49 smithi002 ceph-mon[31557]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:23:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:51.453+0000 7f89837fe700 1 -- 172.21.15.2:0/4116088505 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f898c199bd0 con 0x7f897803e510 2024-05-14T09:23:51.454 INFO:teuthology.orchestra.run.smithi002.stdout:Added host 'smithi026' with addr '172.21.15.26' 2024-05-14T09:23:51.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:51.456+0000 7f89943f3700 1 -- 172.21.15.2:0/4116088505 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f897803e510 msgr2=0x7f8978040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:51.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:51.456+0000 7f89943f3700 1 --2- 172.21.15.2:0/4116088505 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f897803e510 0x7f8978040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f897c006fd0 tx=0x7f897c006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:51.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:51.456+0000 7f89943f3700 1 -- 172.21.15.2:0/4116088505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f898c107aa0 msgr2=0x7f898c19e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:51.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:51.456+0000 7f89943f3700 1 --2- 172.21.15.2:0/4116088505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f898c107aa0 0x7f898c19e530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f898800b5c0 tx=0x7f8988005780 comp rx=0 tx=0).stop 2024-05-14T09:23:51.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:51.456+0000 7f89943f3700 1 -- 172.21.15.2:0/4116088505 shutdown_connections 2024-05-14T09:23:51.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:51.456+0000 7f89943f3700 1 --2- 172.21.15.2:0/4116088505 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f897803e510 0x7f8978040a20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:51.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:51.456+0000 7f89943f3700 1 --2- 172.21.15.2:0/4116088505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f898c107aa0 0x7f898c19e530 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:51.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:51.456+0000 7f89943f3700 1 -- 172.21.15.2:0/4116088505 >> 172.21.15.2:0/4116088505 conn(0x7f898c0fde30 msgr2=0x7f898c1046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:51.456+0000 7f89943f3700 1 -- 172.21.15.2:0/4116088505 shutdown_connections 2024-05-14T09:23:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:51.456+0000 7f89943f3700 1 -- 172.21.15.2:0/4116088505 wait complete. 2024-05-14T09:23:52.033 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-14T09:23:52.283 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:23:52.650 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:52 smithi002 ceph-mon[31557]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:23:52.650 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:52.650 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:52 smithi002 ceph-mon[31557]: Added host smithi026 2024-05-14T09:23:52.650 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:23:52.650 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:53.906 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:53.907 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:53 smithi002 ceph-mon[31557]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:23:54.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.024+0000 7fcc6815e700 1 -- 172.21.15.2:0/3830538042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc60103dd0 msgr2=0x7fcc601041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:54.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.024+0000 7fcc6815e700 1 --2- 172.21.15.2:0/3830538042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc60103dd0 0x7fcc601041e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009910 tx=0x7fcc5c009c20 comp rx=0 tx=0).stop 2024-05-14T09:23:54.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.024+0000 7fcc6815e700 1 -- 172.21.15.2:0/3830538042 shutdown_connections 2024-05-14T09:23:54.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.024+0000 7fcc6815e700 1 --2- 172.21.15.2:0/3830538042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc60103dd0 0x7fcc601041e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:54.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.024+0000 7fcc6815e700 1 -- 172.21.15.2:0/3830538042 >> 172.21.15.2:0/3830538042 conn(0x7fcc600ff680 msgr2=0x7fcc60101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:54.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.024+0000 7fcc6815e700 1 -- 172.21.15.2:0/3830538042 shutdown_connections 2024-05-14T09:23:54.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.024+0000 7fcc6815e700 1 -- 172.21.15.2:0/3830538042 wait complete. 2024-05-14T09:23:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.024+0000 7fcc6815e700 1 Processor -- start 2024-05-14T09:23:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.024+0000 7fcc6815e700 1 -- start start 2024-05-14T09:23:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.025+0000 7fcc6815e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc60103dd0 0x7fcc6019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.025+0000 7fcc6815e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6010e0c0 con 0x7fcc60103dd0 2024-05-14T09:23:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.025+0000 7fcc65efa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc60103dd0 0x7fcc6019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.025+0000 7fcc65efa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc60103dd0 0x7fcc6019f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57374/0 (socket says 172.21.15.2:57374) 2024-05-14T09:23:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.025+0000 7fcc65efa700 1 -- 172.21.15.2:0/2349525743 learned_addr learned my addr 172.21.15.2:0/2349525743 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.025+0000 7fcc65efa700 1 -- 172.21.15.2:0/2349525743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6019fb20 con 0x7fcc60103dd0 2024-05-14T09:23:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.025+0000 7fcc65efa700 1 --2- 172.21.15.2:0/2349525743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc60103dd0 0x7fcc6019f5e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c006010 tx=0x7fcc5c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.025+0000 7fcc56ffd700 1 -- 172.21.15.2:0/2349525743 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc5c026070 con 0x7fcc60103dd0 2024-05-14T09:23:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.026+0000 7fcc56ffd700 1 -- 172.21.15.2:0/2349525743 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc5c00bba0 con 0x7fcc60103dd0 2024-05-14T09:23:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.026+0000 7fcc6815e700 1 -- 172.21.15.2:0/2349525743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6019fe10 con 0x7fcc60103dd0 2024-05-14T09:23:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.026+0000 7fcc56ffd700 1 -- 172.21.15.2:0/2349525743 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc5c003910 con 0x7fcc60103dd0 2024-05-14T09:23:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.026+0000 7fcc6815e700 1 -- 172.21.15.2:0/2349525743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc60199990 con 0x7fcc60103dd0 2024-05-14T09:23:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.026+0000 7fcc56ffd700 1 -- 172.21.15.2:0/2349525743 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fcc5c029070 con 0x7fcc60103dd0 2024-05-14T09:23:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.027+0000 7fcc6815e700 1 -- 172.21.15.2:0/2349525743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc60105280 con 0x7fcc60103dd0 2024-05-14T09:23:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.027+0000 7fcc56ffd700 1 --2- 172.21.15.2:0/2349525743 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcc4c03e510 0x7fcc4c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.027+0000 7fcc56ffd700 1 -- 172.21.15.2:0/2349525743 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcc5c01e070 con 0x7fcc60103dd0 2024-05-14T09:23:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.027+0000 7fcc656f9700 1 --2- 172.21.15.2:0/2349525743 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcc4c03e510 0x7fcc4c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.027+0000 7fcc656f9700 1 --2- 172.21.15.2:0/2349525743 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcc4c03e510 0x7fcc4c040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcc50006fd0 tx=0x7fcc50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.032+0000 7fcc56ffd700 1 -- 172.21.15.2:0/2349525743 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc5c013070 con 0x7fcc60103dd0 2024-05-14T09:23:54.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.256+0000 7fcc6815e700 1 -- 172.21.15.2:0/2349525743 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc6019acf0 con 0x7fcc4c03e510 2024-05-14T09:23:54.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.257+0000 7fcc56ffd700 1 -- 172.21.15.2:0/2349525743 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+158 (secure 0 0 0) 0x7fcc6019acf0 con 0x7fcc4c03e510 2024-05-14T09:23:54.257 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:23:54.257 INFO:teuthology.orchestra.run.smithi002.stdout:[{"addr": "172.21.15.2", "hostname": "smithi002", "labels": [], "status": ""}, {"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}] 2024-05-14T09:23:54.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.259+0000 7fcc6815e700 1 -- 172.21.15.2:0/2349525743 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcc4c03e510 msgr2=0x7fcc4c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:54.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.259+0000 7fcc6815e700 1 --2- 172.21.15.2:0/2349525743 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcc4c03e510 0x7fcc4c040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcc50006fd0 tx=0x7fcc50006e40 comp rx=0 tx=0).stop 2024-05-14T09:23:54.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.259+0000 7fcc6815e700 1 -- 172.21.15.2:0/2349525743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc60103dd0 msgr2=0x7fcc6019f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:54.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.259+0000 7fcc6815e700 1 --2- 172.21.15.2:0/2349525743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc60103dd0 0x7fcc6019f5e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c006010 tx=0x7fcc5c005780 comp rx=0 tx=0).stop 2024-05-14T09:23:54.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.260+0000 7fcc6815e700 1 -- 172.21.15.2:0/2349525743 shutdown_connections 2024-05-14T09:23:54.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.260+0000 7fcc6815e700 1 --2- 172.21.15.2:0/2349525743 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcc4c03e510 0x7fcc4c040a20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:54.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.260+0000 7fcc6815e700 1 --2- 172.21.15.2:0/2349525743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc60103dd0 0x7fcc6019f5e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:54.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.260+0000 7fcc6815e700 1 -- 172.21.15.2:0/2349525743 >> 172.21.15.2:0/2349525743 conn(0x7fcc600ff680 msgr2=0x7fcc60101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:54.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.260+0000 7fcc6815e700 1 -- 172.21.15.2:0/2349525743 shutdown_connections 2024-05-14T09:23:54.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:54.260+0000 7fcc6815e700 1 -- 172.21.15.2:0/2349525743 wait complete. 2024-05-14T09:23:54.821 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi029 2024-05-14T09:23:54.821 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-05-14T09:23:54.821 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/etc/ceph/ceph.conf 2024-05-14T09:23:54.847 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-05-14T09:23:54.847 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T09:23:54.915 INFO:tasks.cephadm:Adding host smithi029 to orchestrator... 2024-05-14T09:23:54.915 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch host add smithi029 2024-05-14T09:23:55.167 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:23:55.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:55 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:55.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:55 smithi002 ceph-mon[31557]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:23:56.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:56 smithi002 ceph-mon[31557]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:23:56.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.694+0000 7fbb96154700 1 -- 172.21.15.2:0/3749310380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb90108ba0 msgr2=0x7fbb90108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:23:56.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.694+0000 7fbb96154700 1 --2- 172.21.15.2:0/3749310380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb90108ba0 0x7fbb90108fb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009910 tx=0x7fbb80009c20 comp rx=0 tx=0).stop 2024-05-14T09:23:56.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.694+0000 7fbb96154700 1 -- 172.21.15.2:0/3749310380 shutdown_connections 2024-05-14T09:23:56.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.694+0000 7fbb96154700 1 --2- 172.21.15.2:0/3749310380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb90108ba0 0x7fbb90108fb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:23:56.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.694+0000 7fbb96154700 1 -- 172.21.15.2:0/3749310380 >> 172.21.15.2:0/3749310380 conn(0x7fbb900fde30 msgr2=0x7fbb90100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:23:56.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.694+0000 7fbb96154700 1 -- 172.21.15.2:0/3749310380 shutdown_connections 2024-05-14T09:23:56.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.694+0000 7fbb96154700 1 -- 172.21.15.2:0/3749310380 wait complete. 2024-05-14T09:23:56.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.695+0000 7fbb96154700 1 Processor -- start 2024-05-14T09:23:56.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.695+0000 7fbb96154700 1 -- start start 2024-05-14T09:23:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.696+0000 7fbb96154700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb90108ba0 0x7fbb9019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.696+0000 7fbb96154700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9010e220 con 0x7fbb90108ba0 2024-05-14T09:23:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.696+0000 7fbb8f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb90108ba0 0x7fbb9019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.696+0000 7fbb8f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb90108ba0 0x7fbb9019f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57396/0 (socket says 172.21.15.2:57396) 2024-05-14T09:23:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.696+0000 7fbb8f7fe700 1 -- 172.21.15.2:0/2957995353 learned_addr learned my addr 172.21.15.2:0/2957995353 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:23:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.696+0000 7fbb8f7fe700 1 -- 172.21.15.2:0/2957995353 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9019fb30 con 0x7fbb90108ba0 2024-05-14T09:23:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.696+0000 7fbb8f7fe700 1 --2- 172.21.15.2:0/2957995353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb90108ba0 0x7fbb9019f5f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000b5c0 tx=0x7fbb80005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.696+0000 7fbb8cff9700 1 -- 172.21.15.2:0/2957995353 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb80026070 con 0x7fbb90108ba0 2024-05-14T09:23:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.697+0000 7fbb8cff9700 1 -- 172.21.15.2:0/2957995353 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb8000b750 con 0x7fbb90108ba0 2024-05-14T09:23:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.697+0000 7fbb8cff9700 1 -- 172.21.15.2:0/2957995353 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb800037d0 con 0x7fbb90108ba0 2024-05-14T09:23:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.697+0000 7fbb96154700 1 -- 172.21.15.2:0/2957995353 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9019fdc0 con 0x7fbb90108ba0 2024-05-14T09:23:56.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.697+0000 7fbb96154700 1 -- 172.21.15.2:0/2957995353 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb90199830 con 0x7fbb90108ba0 2024-05-14T09:23:56.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.698+0000 7fbb8cff9700 1 -- 172.21.15.2:0/2957995353 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fbb80029070 con 0x7fbb90108ba0 2024-05-14T09:23:56.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.698+0000 7fbb96154700 1 -- 172.21.15.2:0/2957995353 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb901052b0 con 0x7fbb90108ba0 2024-05-14T09:23:56.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.698+0000 7fbb8cff9700 1 --2- 172.21.15.2:0/2957995353 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbb7c03e510 0x7fbb7c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:23:56.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.698+0000 7fbb8cff9700 1 -- 172.21.15.2:0/2957995353 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbb8001e070 con 0x7fbb90108ba0 2024-05-14T09:23:56.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.698+0000 7fbb8effd700 1 --2- 172.21.15.2:0/2957995353 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbb7c03e510 0x7fbb7c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:23:56.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.698+0000 7fbb8effd700 1 --2- 172.21.15.2:0/2957995353 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbb7c03e510 0x7fbb7c040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbb78006fd0 tx=0x7fbb78006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:23:56.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.703+0000 7fbb8cff9700 1 -- 172.21.15.2:0/2957995353 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb80013070 con 0x7fbb90108ba0 2024-05-14T09:23:56.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:23:56.924+0000 7fbb96154700 1 -- 172.21.15.2:0/2957995353 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi029", "target": ["mon-mgr", ""]}) -- 0x7fbb9019ad40 con 0x7fbb7c03e510 2024-05-14T09:23:58.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:58 smithi002 ceph-mon[31557]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi029", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:23:58.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:58 smithi002 ceph-mon[31557]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:23:58.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:58 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:58.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:58 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:58.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:58 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:58.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:58 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:23:58.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:58 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi026", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:23:58.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:58 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:23:58.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:58 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:23:58.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:58 smithi002 ceph-mon[31557]: Updating smithi026:/etc/ceph/ceph.conf 2024-05-14T09:23:58.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:58 smithi002 ceph-mon[31557]: Updating smithi026:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:23:59.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:59 smithi002 ceph-mon[31557]: Deploying cephadm binary to smithi029 2024-05-14T09:23:59.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:59 smithi002 ceph-mon[31557]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-05-14T09:23:59.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:23:59 smithi002 ceph-mon[31557]: Updating smithi026:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T09:24:00.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:00 smithi002 ceph-mon[31557]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:01.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:01.259+0000 7fbb8cff9700 1 -- 172.21.15.2:0/2957995353 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fbb9019ad40 con 0x7fbb7c03e510 2024-05-14T09:24:01.259 INFO:teuthology.orchestra.run.smithi002.stdout:Added host 'smithi029' with addr '172.21.15.29' 2024-05-14T09:24:01.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:01.262+0000 7fbb96154700 1 -- 172.21.15.2:0/2957995353 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbb7c03e510 msgr2=0x7fbb7c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:01.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:01.262+0000 7fbb96154700 1 --2- 172.21.15.2:0/2957995353 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbb7c03e510 0x7fbb7c040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbb78006fd0 tx=0x7fbb78006e40 comp rx=0 tx=0).stop 2024-05-14T09:24:01.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:01.262+0000 7fbb96154700 1 -- 172.21.15.2:0/2957995353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb90108ba0 msgr2=0x7fbb9019f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:01.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:01.262+0000 7fbb96154700 1 --2- 172.21.15.2:0/2957995353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb90108ba0 0x7fbb9019f5f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000b5c0 tx=0x7fbb80005780 comp rx=0 tx=0).stop 2024-05-14T09:24:01.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:01.262+0000 7fbb96154700 1 -- 172.21.15.2:0/2957995353 shutdown_connections 2024-05-14T09:24:01.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:01.262+0000 7fbb96154700 1 --2- 172.21.15.2:0/2957995353 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbb7c03e510 0x7fbb7c040a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:01.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:01.262+0000 7fbb96154700 1 --2- 172.21.15.2:0/2957995353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb90108ba0 0x7fbb9019f5f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:01.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:01.262+0000 7fbb96154700 1 -- 172.21.15.2:0/2957995353 >> 172.21.15.2:0/2957995353 conn(0x7fbb900fde30 msgr2=0x7fbb901057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:01.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:01.262+0000 7fbb96154700 1 -- 172.21.15.2:0/2957995353 shutdown_connections 2024-05-14T09:24:01.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:01.262+0000 7fbb96154700 1 -- 172.21.15.2:0/2957995353 wait complete. 2024-05-14T09:24:01.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:01.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:01.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:01.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:01 smithi002 ceph-mon[31557]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:01.824 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:01.824 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:01 smithi002 ceph-mon[31557]: Added host smithi029 2024-05-14T09:24:01.824 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:01.855 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-14T09:24:02.107 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:24:03.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:02 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:03.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:02 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:03.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.647+0000 7ff8b44c5700 1 -- 172.21.15.2:0/1918475089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8ac103dd0 msgr2=0x7ff8ac1041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:03.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.647+0000 7ff8b44c5700 1 --2- 172.21.15.2:0/1918475089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8ac103dd0 0x7ff8ac1041e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009910 tx=0x7ff8a8009c20 comp rx=0 tx=0).stop 2024-05-14T09:24:03.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.647+0000 7ff8b44c5700 1 -- 172.21.15.2:0/1918475089 shutdown_connections 2024-05-14T09:24:03.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.647+0000 7ff8b44c5700 1 --2- 172.21.15.2:0/1918475089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8ac103dd0 0x7ff8ac1041e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:03.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.647+0000 7ff8b44c5700 1 -- 172.21.15.2:0/1918475089 >> 172.21.15.2:0/1918475089 conn(0x7ff8ac0ff660 msgr2=0x7ff8ac101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:03.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.647+0000 7ff8b44c5700 1 -- 172.21.15.2:0/1918475089 shutdown_connections 2024-05-14T09:24:03.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.647+0000 7ff8b44c5700 1 -- 172.21.15.2:0/1918475089 wait complete. 2024-05-14T09:24:03.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.648+0000 7ff8b44c5700 1 Processor -- start 2024-05-14T09:24:03.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.648+0000 7ff8b44c5700 1 -- start start 2024-05-14T09:24:03.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.648+0000 7ff8b44c5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8ac103dd0 0x7ff8ac19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:03.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.648+0000 7ff8b44c5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac10e0c0 con 0x7ff8ac103dd0 2024-05-14T09:24:03.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.648+0000 7ff8b2261700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8ac103dd0 0x7ff8ac19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:03.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.648+0000 7ff8b2261700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8ac103dd0 0x7ff8ac19f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47602/0 (socket says 172.21.15.2:47602) 2024-05-14T09:24:03.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.648+0000 7ff8b2261700 1 -- 172.21.15.2:0/3462374493 learned_addr learned my addr 172.21.15.2:0/3462374493 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:24:03.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.649+0000 7ff8b2261700 1 -- 172.21.15.2:0/3462374493 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ac19fb30 con 0x7ff8ac103dd0 2024-05-14T09:24:03.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.649+0000 7ff8b2261700 1 --2- 172.21.15.2:0/3462374493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8ac103dd0 0x7ff8ac19f5f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80056d0 tx=0x7ff8a80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:03.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.649+0000 7ff8a37fe700 1 -- 172.21.15.2:0/3462374493 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8a8026070 con 0x7ff8ac103dd0 2024-05-14T09:24:03.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.649+0000 7ff8b44c5700 1 -- 172.21.15.2:0/3462374493 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ac19fe20 con 0x7ff8ac103dd0 2024-05-14T09:24:03.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.649+0000 7ff8a37fe700 1 -- 172.21.15.2:0/3462374493 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a800bba0 con 0x7ff8ac103dd0 2024-05-14T09:24:03.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.649+0000 7ff8a37fe700 1 -- 172.21.15.2:0/3462374493 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8a8003920 con 0x7ff8ac103dd0 2024-05-14T09:24:03.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.649+0000 7ff8b44c5700 1 -- 172.21.15.2:0/3462374493 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ac199950 con 0x7ff8ac103dd0 2024-05-14T09:24:03.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.650+0000 7ff8b44c5700 1 -- 172.21.15.2:0/3462374493 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ac105280 con 0x7ff8ac103dd0 2024-05-14T09:24:03.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.650+0000 7ff8a37fe700 1 -- 172.21.15.2:0/3462374493 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7ff8a8029070 con 0x7ff8ac103dd0 2024-05-14T09:24:03.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.650+0000 7ff8a37fe700 1 --2- 172.21.15.2:0/3462374493 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff89803e510 0x7ff898040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:03.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.651+0000 7ff8a37fe700 1 -- 172.21.15.2:0/3462374493 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff8a801e070 con 0x7ff8ac103dd0 2024-05-14T09:24:03.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.651+0000 7ff8b1a60700 1 --2- 172.21.15.2:0/3462374493 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff89803e510 0x7ff898040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:03.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.651+0000 7ff8b1a60700 1 --2- 172.21.15.2:0/3462374493 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff89803e510 0x7ff898040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff89c006fd0 tx=0x7ff89c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:03.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.656+0000 7ff8a37fe700 1 -- 172.21.15.2:0/3462374493 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff8a8013070 con 0x7ff8ac103dd0 2024-05-14T09:24:03.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.874+0000 7ff8b44c5700 1 -- 172.21.15.2:0/3462374493 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8ac19ac90 con 0x7ff89803e510 2024-05-14T09:24:03.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.875+0000 7ff8a37fe700 1 -- 172.21.15.2:0/3462374493 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+237 (secure 0 0 0) 0x7ff8ac19ac90 con 0x7ff89803e510 2024-05-14T09:24:03.875 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:24:03.875 INFO:teuthology.orchestra.run.smithi002.stdout:[{"addr": "172.21.15.2", "hostname": "smithi002", "labels": [], "status": ""}, {"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}, {"addr": "172.21.15.29", "hostname": "smithi029", "labels": [], "status": ""}] 2024-05-14T09:24:03.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.877+0000 7ff8b44c5700 1 -- 172.21.15.2:0/3462374493 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff89803e510 msgr2=0x7ff898040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:03.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.877+0000 7ff8b44c5700 1 --2- 172.21.15.2:0/3462374493 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff89803e510 0x7ff898040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff89c006fd0 tx=0x7ff89c006e40 comp rx=0 tx=0).stop 2024-05-14T09:24:03.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.877+0000 7ff8b44c5700 1 -- 172.21.15.2:0/3462374493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8ac103dd0 msgr2=0x7ff8ac19f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:03.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.877+0000 7ff8b44c5700 1 --2- 172.21.15.2:0/3462374493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8ac103dd0 0x7ff8ac19f5f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80056d0 tx=0x7ff8a80057b0 comp rx=0 tx=0).stop 2024-05-14T09:24:03.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.877+0000 7ff8b44c5700 1 -- 172.21.15.2:0/3462374493 shutdown_connections 2024-05-14T09:24:03.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.877+0000 7ff8b44c5700 1 --2- 172.21.15.2:0/3462374493 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff89803e510 0x7ff898040a20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:03.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.877+0000 7ff8b44c5700 1 --2- 172.21.15.2:0/3462374493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8ac103dd0 0x7ff8ac19f5f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:03.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.877+0000 7ff8b44c5700 1 -- 172.21.15.2:0/3462374493 >> 172.21.15.2:0/3462374493 conn(0x7ff8ac0ff660 msgr2=0x7ff8ac109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:03.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.878+0000 7ff8b44c5700 1 -- 172.21.15.2:0/3462374493 shutdown_connections 2024-05-14T09:24:03.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:03.878+0000 7ff8b44c5700 1 -- 172.21.15.2:0/3462374493 wait complete. 2024-05-14T09:24:04.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:03 smithi002 ceph-mon[31557]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:04.447 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-14T09:24:04.447 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-14T09:24:04.694 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:24:05.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:04 smithi002 ceph-mon[31557]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:24:05.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:04 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:06.066 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:05 smithi002 ceph-mon[31557]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:06.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.185+0000 7f8bc964f700 1 -- 172.21.15.2:0/2197759493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bc4103dd0 msgr2=0x7f8bc41041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:06.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.185+0000 7f8bc964f700 1 --2- 172.21.15.2:0/2197759493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bc4103dd0 0x7f8bc41041e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4009910 tx=0x7f8bb4009c20 comp rx=0 tx=0).stop 2024-05-14T09:24:06.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.185+0000 7f8bc964f700 1 -- 172.21.15.2:0/2197759493 shutdown_connections 2024-05-14T09:24:06.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.185+0000 7f8bc964f700 1 --2- 172.21.15.2:0/2197759493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bc4103dd0 0x7f8bc41041e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:06.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.185+0000 7f8bc964f700 1 -- 172.21.15.2:0/2197759493 >> 172.21.15.2:0/2197759493 conn(0x7f8bc40ff660 msgr2=0x7f8bc4101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:06.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.185+0000 7f8bc964f700 1 -- 172.21.15.2:0/2197759493 shutdown_connections 2024-05-14T09:24:06.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.185+0000 7f8bc964f700 1 -- 172.21.15.2:0/2197759493 wait complete. 2024-05-14T09:24:06.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.185+0000 7f8bc964f700 1 Processor -- start 2024-05-14T09:24:06.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.186+0000 7f8bc964f700 1 -- start start 2024-05-14T09:24:06.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.186+0000 7f8bc964f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bc4103dd0 0x7f8bc419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:06.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.186+0000 7f8bc964f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc410e0c0 con 0x7f8bc4103dd0 2024-05-14T09:24:06.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.186+0000 7f8bc2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bc4103dd0 0x7f8bc419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:06.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.186+0000 7f8bc2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bc4103dd0 0x7f8bc419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47618/0 (socket says 172.21.15.2:47618) 2024-05-14T09:24:06.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.186+0000 7f8bc2ffd700 1 -- 172.21.15.2:0/557067254 learned_addr learned my addr 172.21.15.2:0/557067254 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:24:06.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.186+0000 7f8bc2ffd700 1 -- 172.21.15.2:0/557067254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc419fb40 con 0x7f8bc4103dd0 2024-05-14T09:24:06.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.186+0000 7f8bc2ffd700 1 --2- 172.21.15.2:0/557067254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bc4103dd0 0x7f8bc419f600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40056d0 tx=0x7f8bb40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:06.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.186+0000 7f8bc0ff9700 1 -- 172.21.15.2:0/557067254 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bb4026070 con 0x7f8bc4103dd0 2024-05-14T09:24:06.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.187+0000 7f8bc0ff9700 1 -- 172.21.15.2:0/557067254 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb400bba0 con 0x7f8bc4103dd0 2024-05-14T09:24:06.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.187+0000 7f8bc964f700 1 -- 172.21.15.2:0/557067254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc419fe30 con 0x7f8bc4103dd0 2024-05-14T09:24:06.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.187+0000 7f8bc0ff9700 1 -- 172.21.15.2:0/557067254 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bb4003920 con 0x7f8bc4103dd0 2024-05-14T09:24:06.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.187+0000 7f8bc964f700 1 -- 172.21.15.2:0/557067254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc4199960 con 0x7f8bc4103dd0 2024-05-14T09:24:06.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.188+0000 7f8bc964f700 1 -- 172.21.15.2:0/557067254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc4105280 con 0x7f8bc4103dd0 2024-05-14T09:24:06.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.188+0000 7f8bc0ff9700 1 -- 172.21.15.2:0/557067254 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f8bb4029070 con 0x7f8bc4103dd0 2024-05-14T09:24:06.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.188+0000 7f8bc0ff9700 1 --2- 172.21.15.2:0/557067254 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8bb003e570 0x7f8bb0040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:06.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.188+0000 7f8bc0ff9700 1 -- 172.21.15.2:0/557067254 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8bb401e070 con 0x7f8bc4103dd0 2024-05-14T09:24:06.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.188+0000 7f8bba7fc700 1 --2- 172.21.15.2:0/557067254 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8bb003e570 0x7f8bb0040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:06.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.189+0000 7f8bba7fc700 1 --2- 172.21.15.2:0/557067254 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8bb003e570 0x7f8bb0040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8bac006fd0 tx=0x7f8bac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:06.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.193+0000 7f8bc0ff9700 1 -- 172.21.15.2:0/557067254 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8bb4013070 con 0x7f8bc4103dd0 2024-05-14T09:24:06.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.414+0000 7f8bc964f700 1 -- 172.21.15.2:0/557067254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f8bc41a0060 con 0x7f8bc4103dd0 2024-05-14T09:24:06.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.961+0000 7f8bc0ff9700 1 -- 172.21.15.2:0/557067254 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f8bb4015350 con 0x7f8bc4103dd0 2024-05-14T09:24:06.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.965+0000 7f8bc964f700 1 -- 172.21.15.2:0/557067254 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8bb003e570 msgr2=0x7f8bb0040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:06.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.965+0000 7f8bc964f700 1 --2- 172.21.15.2:0/557067254 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8bb003e570 0x7f8bb0040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8bac006fd0 tx=0x7f8bac006e40 comp rx=0 tx=0).stop 2024-05-14T09:24:06.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.965+0000 7f8bc964f700 1 -- 172.21.15.2:0/557067254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bc4103dd0 msgr2=0x7f8bc419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:06.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.965+0000 7f8bc964f700 1 --2- 172.21.15.2:0/557067254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bc4103dd0 0x7f8bc419f600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40056d0 tx=0x7f8bb40057b0 comp rx=0 tx=0).stop 2024-05-14T09:24:06.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.965+0000 7f8bc964f700 1 -- 172.21.15.2:0/557067254 shutdown_connections 2024-05-14T09:24:06.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.965+0000 7f8bc964f700 1 --2- 172.21.15.2:0/557067254 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8bb003e570 0x7f8bb0040a80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:06.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.965+0000 7f8bc964f700 1 --2- 172.21.15.2:0/557067254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bc4103dd0 0x7f8bc419f600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:06.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.965+0000 7f8bc964f700 1 -- 172.21.15.2:0/557067254 >> 172.21.15.2:0/557067254 conn(0x7f8bc40ff660 msgr2=0x7f8bc4109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:06.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.965+0000 7f8bc964f700 1 -- 172.21.15.2:0/557067254 shutdown_connections 2024-05-14T09:24:06.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:06.965+0000 7f8bc964f700 1 -- 172.21.15.2:0/557067254 wait complete. 2024-05-14T09:24:06.966 INFO:teuthology.orchestra.run.smithi002.stderr:adjusted tunables profile to default 2024-05-14T09:24:07.278 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:06 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/557067254' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-14T09:24:07.570 INFO:tasks.cephadm:Adding mon.a on smithi002 2024-05-14T09:24:07.571 INFO:tasks.cephadm:Adding mon.b on smithi026 2024-05-14T09:24:07.571 INFO:tasks.cephadm:Adding mon.c on smithi029 2024-05-14T09:24:07.571 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi002:172.21.15.2=a;smithi026:172.21.15.26=b;smithi029:172.21.15.29=c' 2024-05-14T09:24:07.806 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:08.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:07 smithi002 ceph-mon[31557]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:08.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:07 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/557067254' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-14T09:24:08.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:07 smithi002 ceph-mon[31557]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T09:24:08.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:07 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:08.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:07 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:08.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:07 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:08.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:07 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:08.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:07 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi029", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:24:08.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:07 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:08.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:07 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:08.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:07 smithi002 ceph-mon[31557]: Updating smithi029:/etc/ceph/ceph.conf 2024-05-14T09:24:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.287+0000 7f122c904700 1 -- 172.21.15.29:0/302128362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1228108ac0 msgr2=0x7f1228108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.287+0000 7f122c904700 1 --2- 172.21.15.29:0/302128362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1228108ac0 0x7f1228108ed0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f121c009910 tx=0x7f121c009c20 comp rx=0 tx=0).stop 2024-05-14T09:24:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.287+0000 7f122c904700 1 -- 172.21.15.29:0/302128362 shutdown_connections 2024-05-14T09:24:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.287+0000 7f122c904700 1 --2- 172.21.15.29:0/302128362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1228108ac0 0x7f1228108ed0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.287+0000 7f122c904700 1 -- 172.21.15.29:0/302128362 >> 172.21.15.29:0/302128362 conn(0x7f12280fe5b0 msgr2=0x7f1228100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:09.287 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.287+0000 7f122c904700 1 -- 172.21.15.29:0/302128362 shutdown_connections 2024-05-14T09:24:09.288 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.287+0000 7f122c904700 1 -- 172.21.15.29:0/302128362 wait complete. 2024-05-14T09:24:09.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.288+0000 7f122c904700 1 Processor -- start 2024-05-14T09:24:09.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.288+0000 7f122c904700 1 -- start start 2024-05-14T09:24:09.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.288+0000 7f122c904700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1228108ac0 0x7f122819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:09.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.288+0000 7f122c904700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f122810d010 con 0x7f1228108ac0 2024-05-14T09:24:09.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.289+0000 7f122659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1228108ac0 0x7f122819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:09.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.289+0000 7f122659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1228108ac0 0x7f122819f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.29:48386/0 (socket says 172.21.15.29:48386) 2024-05-14T09:24:09.289 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.289+0000 7f122659c700 1 -- 172.21.15.29:0/3455502 learned_addr learned my addr 172.21.15.29:0/3455502 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-05-14T09:24:09.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.289+0000 7f122659c700 1 -- 172.21.15.29:0/3455502 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122819fb30 con 0x7f1228108ac0 2024-05-14T09:24:09.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.289+0000 7f122659c700 1 --2- 172.21.15.29:0/3455502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1228108ac0 0x7f122819f5f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f121c0056d0 tx=0x7f121c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:09.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.290+0000 7f12177fe700 1 -- 172.21.15.29:0/3455502 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f121c027070 con 0x7f1228108ac0 2024-05-14T09:24:09.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.290+0000 7f122c904700 1 -- 172.21.15.29:0/3455502 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122819fe20 con 0x7f1228108ac0 2024-05-14T09:24:09.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.290+0000 7f12177fe700 1 -- 172.21.15.29:0/3455502 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f121c00be10 con 0x7f1228108ac0 2024-05-14T09:24:09.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.290+0000 7f12177fe700 1 -- 172.21.15.29:0/3455502 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f121c022d20 con 0x7f1228108ac0 2024-05-14T09:24:09.290 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.290+0000 7f122c904700 1 -- 172.21.15.29:0/3455502 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1228199830 con 0x7f1228108ac0 2024-05-14T09:24:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.291+0000 7f122c904700 1 -- 172.21.15.29:0/3455502 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12281041d0 con 0x7f1228108ac0 2024-05-14T09:24:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.291+0000 7f12177fe700 1 -- 172.21.15.29:0/3455502 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f121c02a070 con 0x7f1228108ac0 2024-05-14T09:24:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.291+0000 7f12177fe700 1 --2- 172.21.15.29:0/3455502 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f121003e510 0x7f1210040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.291+0000 7f12177fe700 1 -- 172.21.15.29:0/3455502 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f121c01e030 con 0x7f1228108ac0 2024-05-14T09:24:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.291+0000 7f1225d9b700 1 --2- 172.21.15.29:0/3455502 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f121003e510 0x7f1210040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:09.291 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.292+0000 7f1225d9b700 1 --2- 172.21.15.29:0/3455502 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f121003e510 0x7f1210040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1218006fd0 tx=0x7f1218006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:09.296 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.297+0000 7f12177fe700 1 -- 172.21.15.29:0/3455502 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f121c013070 con 0x7f1228108ac0 2024-05-14T09:24:09.527 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.528+0000 7f122c904700 1 -- 172.21.15.29:0/3455502 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi026:172.21.15.26=b;smithi029:172.21.15.29=c", "target": ["mon-mgr", ""]}) -- 0x7f122819adf0 con 0x7f121003e510 2024-05-14T09:24:09.533 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.534+0000 7f12177fe700 1 -- 172.21.15.29:0/3455502 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f122819adf0 con 0x7f121003e510 2024-05-14T09:24:09.534 INFO:teuthology.orchestra.run.smithi029.stdout:Scheduled mon update... 2024-05-14T09:24:09.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.536+0000 7f122c904700 1 -- 172.21.15.29:0/3455502 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f121003e510 msgr2=0x7f1210040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:09.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.536+0000 7f122c904700 1 --2- 172.21.15.29:0/3455502 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f121003e510 0x7f1210040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1218006fd0 tx=0x7f1218006e40 comp rx=0 tx=0).stop 2024-05-14T09:24:09.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.536+0000 7f122c904700 1 -- 172.21.15.29:0/3455502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1228108ac0 msgr2=0x7f122819f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:09.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.536+0000 7f122c904700 1 --2- 172.21.15.29:0/3455502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1228108ac0 0x7f122819f5f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f121c0056d0 tx=0x7f121c0057b0 comp rx=0 tx=0).stop 2024-05-14T09:24:09.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.536+0000 7f122c904700 1 -- 172.21.15.29:0/3455502 shutdown_connections 2024-05-14T09:24:09.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.536+0000 7f122c904700 1 --2- 172.21.15.29:0/3455502 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f121003e510 0x7f1210040a20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:09.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.536+0000 7f122c904700 1 --2- 172.21.15.29:0/3455502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1228108ac0 0x7f122819f5f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:09.536 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.536+0000 7f122c904700 1 -- 172.21.15.29:0/3455502 >> 172.21.15.29:0/3455502 conn(0x7f12280fe5b0 msgr2=0x7f12281065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:09.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.537+0000 7f122c904700 1 -- 172.21.15.29:0/3455502 shutdown_connections 2024-05-14T09:24:09.537 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:09.537+0000 7f122c904700 1 -- 172.21.15.29:0/3455502 wait complete. 2024-05-14T09:24:09.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:09 smithi002 ceph-mon[31557]: Updating smithi029:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:09.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:09 smithi002 ceph-mon[31557]: Updating smithi029:/etc/ceph/ceph.client.admin.keyring 2024-05-14T09:24:09.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:09 smithi002 ceph-mon[31557]: Updating smithi029:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T09:24:09.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:09.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:09.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:10.117 DEBUG:teuthology.orchestra.run.smithi026:mon.b> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.b.service 2024-05-14T09:24:10.120 DEBUG:teuthology.orchestra.run.smithi029:mon.c> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.c.service 2024-05-14T09:24:10.122 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T09:24:10.122 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T09:24:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:10 smithi002 ceph-mon[31557]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:10 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:10 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:10 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:10.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:10 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:10.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:10 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:10.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:10 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T09:24:10.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:10 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:11.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:11 smithi002 ceph-mon[31557]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi026:172.21.15.26=b;smithi029:172.21.15.29=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:24:11.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:11 smithi002 ceph-mon[31557]: Saving service mon spec with placement smithi002:172.21.15.2=a;smithi026:172.21.15.26=b;smithi029:172.21.15.29=c;count:3 2024-05-14T09:24:11.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:11 smithi002 ceph-mon[31557]: Deploying daemon mon.c on smithi029 2024-05-14T09:24:11.811 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:12.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:12 smithi002 ceph-mon[31557]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:13.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.987+0000 7f68f4550700 1 -- 172.21.15.29:0/993583597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68ec103dd0 msgr2=0x7f68ec1041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:13.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.987+0000 7f68f4550700 1 --2- 172.21.15.29:0/993583597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68ec103dd0 0x7f68ec1041e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009910 tx=0x7f68e8009c20 comp rx=0 tx=0).stop 2024-05-14T09:24:13.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.987+0000 7f68f4550700 1 -- 172.21.15.29:0/993583597 shutdown_connections 2024-05-14T09:24:13.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.987+0000 7f68f4550700 1 --2- 172.21.15.29:0/993583597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68ec103dd0 0x7f68ec1041e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:13.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.987+0000 7f68f4550700 1 -- 172.21.15.29:0/993583597 >> 172.21.15.29:0/993583597 conn(0x7f68ec0ff660 msgr2=0x7f68ec101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:13.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.987+0000 7f68f4550700 1 -- 172.21.15.29:0/993583597 shutdown_connections 2024-05-14T09:24:13.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.987+0000 7f68f4550700 1 -- 172.21.15.29:0/993583597 wait complete. 2024-05-14T09:24:13.988 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.988+0000 7f68f4550700 1 Processor -- start 2024-05-14T09:24:13.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.988+0000 7f68f4550700 1 -- start start 2024-05-14T09:24:13.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.988+0000 7f68f4550700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68ec103dd0 0x7f68ec075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:13.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.988+0000 7f68f4550700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec10e0c0 con 0x7f68ec103dd0 2024-05-14T09:24:13.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.988+0000 7f68f22ec700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68ec103dd0 0x7f68ec075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:13.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.988+0000 7f68f22ec700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68ec103dd0 0x7f68ec075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.29:55442/0 (socket says 172.21.15.29:55442) 2024-05-14T09:24:13.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.988+0000 7f68f22ec700 1 -- 172.21.15.29:0/870274451 learned_addr learned my addr 172.21.15.29:0/870274451 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-05-14T09:24:13.989 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.989+0000 7f68f22ec700 1 -- 172.21.15.29:0/870274451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ec077910 con 0x7f68ec103dd0 2024-05-14T09:24:13.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.989+0000 7f68f22ec700 1 --2- 172.21.15.29:0/870274451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68ec103dd0 0x7f68ec075ac0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f68e80056d0 tx=0x7f68e80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:13.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.989+0000 7f68e37fe700 1 -- 172.21.15.29:0/870274451 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68e8026070 con 0x7f68ec103dd0 2024-05-14T09:24:13.990 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.989+0000 7f68f4550700 1 -- 172.21.15.29:0/870274451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ec076000 con 0x7f68ec103dd0 2024-05-14T09:24:13.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.989+0000 7f68e37fe700 1 -- 172.21.15.29:0/870274451 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68e800bba0 con 0x7f68ec103dd0 2024-05-14T09:24:13.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.989+0000 7f68e37fe700 1 -- 172.21.15.29:0/870274451 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68e8003920 con 0x7f68ec103dd0 2024-05-14T09:24:13.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.989+0000 7f68f4550700 1 -- 172.21.15.29:0/870274451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ec0764c0 con 0x7f68ec103dd0 2024-05-14T09:24:13.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.990+0000 7f68f4550700 1 -- 172.21.15.29:0/870274451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68ec105280 con 0x7f68ec103dd0 2024-05-14T09:24:13.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.990+0000 7f68e37fe700 1 -- 172.21.15.29:0/870274451 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f68e8029070 con 0x7f68ec103dd0 2024-05-14T09:24:13.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.991+0000 7f68e37fe700 1 --2- 172.21.15.29:0/870274451 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f68d803e510 0x7f68d8040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:13.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.991+0000 7f68e37fe700 1 -- 172.21.15.29:0/870274451 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f68e801e070 con 0x7f68ec103dd0 2024-05-14T09:24:13.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.991+0000 7f68f1aeb700 1 --2- 172.21.15.29:0/870274451 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f68d803e510 0x7f68d8040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:13.991 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.991+0000 7f68f1aeb700 1 --2- 172.21.15.29:0/870274451 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f68d803e510 0x7f68d8040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f68dc006fd0 tx=0x7f68dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:13.996 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:13.996+0000 7f68e37fe700 1 -- 172.21.15.29:0/870274451 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f68e8013070 con 0x7f68ec103dd0 2024-05-14T09:24:14.313 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:14.314+0000 7f68f4550700 1 -- 172.21.15.29:0/870274451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f68ec077020 con 0x7f68ec103dd0 2024-05-14T09:24:14.314 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:14.314+0000 7f68e37fe700 1 -- 172.21.15.29:0/870274451 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f68e800d340 con 0x7f68ec103dd0 2024-05-14T09:24:14.314 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:24:14.314 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":1,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","modified":"2024-05-14T09:22:31.786962Z","created":"2024-05-14T09:22:31.786962Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-14T09:24:14.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:14.316+0000 7f68f4550700 1 -- 172.21.15.29:0/870274451 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f68d803e510 msgr2=0x7f68d8040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:14.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:14.316+0000 7f68f4550700 1 --2- 172.21.15.29:0/870274451 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f68d803e510 0x7f68d8040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f68dc006fd0 tx=0x7f68dc006e40 comp rx=0 tx=0).stop 2024-05-14T09:24:14.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:14.316+0000 7f68f4550700 1 -- 172.21.15.29:0/870274451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68ec103dd0 msgr2=0x7f68ec075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:14.316 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:14.316+0000 7f68f4550700 1 --2- 172.21.15.29:0/870274451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68ec103dd0 0x7f68ec075ac0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f68e80056d0 tx=0x7f68e80057b0 comp rx=0 tx=0).stop 2024-05-14T09:24:14.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:14.316+0000 7f68f4550700 1 -- 172.21.15.29:0/870274451 shutdown_connections 2024-05-14T09:24:14.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:14.316+0000 7f68f4550700 1 --2- 172.21.15.29:0/870274451 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f68d803e510 0x7f68d8040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:14.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:14.316+0000 7f68f4550700 1 --2- 172.21.15.29:0/870274451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68ec103dd0 0x7f68ec075ac0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:14.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:14.316+0000 7f68f4550700 1 -- 172.21.15.29:0/870274451 >> 172.21.15.29:0/870274451 conn(0x7f68ec0ff660 msgr2=0x7f68ec109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:14.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:14.317+0000 7f68f4550700 1 -- 172.21.15.29:0/870274451 shutdown_connections 2024-05-14T09:24:14.317 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:14.317+0000 7f68f4550700 1 -- 172.21.15.29:0/870274451 wait complete. 2024-05-14T09:24:14.317 INFO:teuthology.orchestra.run.smithi029.stderr:dumped monmap epoch 1 2024-05-14T09:24:14.462 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 podman[36216]: 2024-05-14 09:24:14.426860675 +0000 UTC m=+0.330374194 container start d42860fea015e86df8dd2f51b1382b69ad8d6dea7bad53b44ace96d225e993c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-14T09:24:14.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:14 smithi002 ceph-mon[31557]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:14.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:14 smithi002 ceph-mon[31557]: from='client.? 172.21.15.29:0/870274451' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T09:24:14.794 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: set uid:gid to 167:167 (ceph:ceph) 2024-05-14T09:24:14.794 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev), process ceph-mon, pid 2 2024-05-14T09:24:14.795 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: pidfile_write: ignore empty --pid-file 2024-05-14T09:24:14.795 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: load: jerasure load: lrc 2024-05-14T09:24:14.795 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: RocksDB version: 7.9.2 2024-05-14T09:24:14.795 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Git sha 0 2024-05-14T09:24:14.795 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Compile date 2024-05-09 05:50:40 2024-05-14T09:24:14.795 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: DB SUMMARY 2024-05-14T09:24:14.795 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: DB Session ID: ZONZK8EZKE6AGV6S68IS 2024-05-14T09:24:14.795 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: CURRENT file: CURRENT 2024-05-14T09:24:14.795 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: IDENTITY file: IDENTITY 2024-05-14T09:24:14.795 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-14T09:24:14.795 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-14T09:24:14.795 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-14T09:24:14.796 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.error_if_exists: 0 2024-05-14T09:24:14.796 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.create_if_missing: 0 2024-05-14T09:24:14.796 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.paranoid_checks: 1 2024-05-14T09:24:14.796 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-14T09:24:14.796 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-14T09:24:14.796 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-14T09:24:14.796 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.env: 0x560e1d62a500 2024-05-14T09:24:14.796 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.fs: PosixFileSystem 2024-05-14T09:24:14.796 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.info_log: 0x560e1e3f2020 2024-05-14T09:24:14.796 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_file_opening_threads: 16 2024-05-14T09:24:14.796 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.statistics: (nil) 2024-05-14T09:24:14.796 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.use_fsync: 0 2024-05-14T09:24:14.796 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_log_file_size: 0 2024-05-14T09:24:14.797 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-14T09:24:14.797 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-14T09:24:14.797 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.keep_log_file_num: 1000 2024-05-14T09:24:14.797 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.recycle_log_file_num: 0 2024-05-14T09:24:14.797 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.allow_fallocate: 1 2024-05-14T09:24:14.797 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.allow_mmap_reads: 0 2024-05-14T09:24:14.797 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.allow_mmap_writes: 0 2024-05-14T09:24:14.797 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.use_direct_reads: 0 2024-05-14T09:24:14.797 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-14T09:24:14.797 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.create_missing_column_families: 0 2024-05-14T09:24:14.797 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.db_log_dir: 2024-05-14T09:24:14.797 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.wal_dir: 2024-05-14T09:24:14.797 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-14T09:24:14.798 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-14T09:24:14.798 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-14T09:24:14.798 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-14T09:24:14.798 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-14T09:24:14.798 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-14T09:24:14.798 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.advise_random_on_open: 1 2024-05-14T09:24:14.798 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.db_write_buffer_size: 0 2024-05-14T09:24:14.798 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.write_buffer_manager: 0x560e1e2bc8c0 2024-05-14T09:24:14.798 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-14T09:24:14.798 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-14T09:24:14.798 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-14T09:24:14.798 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.rate_limiter: (nil) 2024-05-14T09:24:14.798 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-14T09:24:14.798 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.wal_recovery_mode: 2 2024-05-14T09:24:14.799 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.enable_thread_tracking: 0 2024-05-14T09:24:14.799 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.enable_pipelined_write: 0 2024-05-14T09:24:14.799 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.unordered_write: 0 2024-05-14T09:24:14.799 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-14T09:24:14.799 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-14T09:24:14.799 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-14T09:24:14.799 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-14T09:24:14.799 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.row_cache: None 2024-05-14T09:24:14.800 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.wal_filter: None 2024-05-14T09:24:14.800 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-14T09:24:14.800 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.allow_ingest_behind: 0 2024-05-14T09:24:14.800 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.two_write_queues: 0 2024-05-14T09:24:14.800 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.manual_wal_flush: 0 2024-05-14T09:24:14.800 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.wal_compression: 0 2024-05-14T09:24:14.800 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.atomic_flush: 0 2024-05-14T09:24:14.800 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-14T09:24:14.800 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-14T09:24:14.800 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-14T09:24:14.800 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.log_readahead_size: 0 2024-05-14T09:24:14.800 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-14T09:24:14.800 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.best_efforts_recovery: 0 2024-05-14T09:24:14.800 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-14T09:24:14.801 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-14T09:24:14.801 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.allow_data_in_errors: 0 2024-05-14T09:24:14.801 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.db_host_id: __hostname__ 2024-05-14T09:24:14.801 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-14T09:24:14.802 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_background_jobs: 2 2024-05-14T09:24:14.802 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_background_compactions: -1 2024-05-14T09:24:14.802 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_subcompactions: 1 2024-05-14T09:24:14.803 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-14T09:24:14.803 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-14T09:24:14.803 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-14T09:24:14.803 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_total_wal_size: 0 2024-05-14T09:24:14.803 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-14T09:24:14.803 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-14T09:24:14.803 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-14T09:24:14.803 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-14T09:24:14.803 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_open_files: -1 2024-05-14T09:24:14.803 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.bytes_per_sync: 0 2024-05-14T09:24:14.803 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-14T09:24:14.803 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-14T09:24:14.803 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compaction_readahead_size: 0 2024-05-14T09:24:14.803 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_background_flushes: -1 2024-05-14T09:24:14.803 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Compression algorithms supported: 2024-05-14T09:24:14.804 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-14T09:24:14.804 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: kZSTD supported: 0 2024-05-14T09:24:14.804 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: kXpressCompression supported: 0 2024-05-14T09:24:14.804 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: kLZ4HCCompression supported: 1 2024-05-14T09:24:14.804 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: kZlibCompression supported: 1 2024-05-14T09:24:14.804 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: kSnappyCompression supported: 1 2024-05-14T09:24:14.804 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: kLZ4Compression supported: 1 2024-05-14T09:24:14.804 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: kBZip2Compression supported: 0 2024-05-14T09:24:14.804 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-14T09:24:14.804 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-14T09:24:14.804 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-14T09:24:14.804 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-14T09:24:14.804 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-14T09:24:14.804 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.merge_operator: 2024-05-14T09:24:14.804 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compaction_filter: None 2024-05-14T09:24:14.805 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compaction_filter_factory: None 2024-05-14T09:24:14.805 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.sst_partitioner_factory: None 2024-05-14T09:24:14.805 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-14T09:24:14.805 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-14T09:24:14.805 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560e1e3f21e0) 2024-05-14T09:24:14.805 INFO:journalctl@ceph.mon.c.smithi029.stdout: cache_index_and_filter_blocks: 1 2024-05-14T09:24:14.805 INFO:journalctl@ceph.mon.c.smithi029.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-14T09:24:14.805 INFO:journalctl@ceph.mon.c.smithi029.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-14T09:24:14.805 INFO:journalctl@ceph.mon.c.smithi029.stdout: pin_top_level_index_and_filter: 1 2024-05-14T09:24:14.805 INFO:journalctl@ceph.mon.c.smithi029.stdout: index_type: 0 2024-05-14T09:24:14.805 INFO:journalctl@ceph.mon.c.smithi029.stdout: data_block_index_type: 0 2024-05-14T09:24:14.805 INFO:journalctl@ceph.mon.c.smithi029.stdout: index_shortening: 1 2024-05-14T09:24:14.805 INFO:journalctl@ceph.mon.c.smithi029.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-14T09:24:14.805 INFO:journalctl@ceph.mon.c.smithi029.stdout: checksum: 4 2024-05-14T09:24:14.805 INFO:journalctl@ceph.mon.c.smithi029.stdout: no_block_cache: 0 2024-05-14T09:24:14.806 INFO:journalctl@ceph.mon.c.smithi029.stdout: block_cache: 0x560e1e2ef1f0 2024-05-14T09:24:14.806 INFO:journalctl@ceph.mon.c.smithi029.stdout: block_cache_name: BinnedLRUCache 2024-05-14T09:24:14.806 INFO:journalctl@ceph.mon.c.smithi029.stdout: block_cache_options: 2024-05-14T09:24:14.806 INFO:journalctl@ceph.mon.c.smithi029.stdout: capacity : 536870912 2024-05-14T09:24:14.806 INFO:journalctl@ceph.mon.c.smithi029.stdout: num_shard_bits : 4 2024-05-14T09:24:14.806 INFO:journalctl@ceph.mon.c.smithi029.stdout: strict_capacity_limit : 0 2024-05-14T09:24:14.806 INFO:journalctl@ceph.mon.c.smithi029.stdout: high_pri_pool_ratio: 0.000 2024-05-14T09:24:14.806 INFO:journalctl@ceph.mon.c.smithi029.stdout: block_cache_compressed: (nil) 2024-05-14T09:24:14.806 INFO:journalctl@ceph.mon.c.smithi029.stdout: persistent_cache: (nil) 2024-05-14T09:24:14.806 INFO:journalctl@ceph.mon.c.smithi029.stdout: block_size: 4096 2024-05-14T09:24:14.806 INFO:journalctl@ceph.mon.c.smithi029.stdout: block_size_deviation: 10 2024-05-14T09:24:14.806 INFO:journalctl@ceph.mon.c.smithi029.stdout: block_restart_interval: 16 2024-05-14T09:24:14.806 INFO:journalctl@ceph.mon.c.smithi029.stdout: index_block_restart_interval: 1 2024-05-14T09:24:14.806 INFO:journalctl@ceph.mon.c.smithi029.stdout: metadata_block_size: 4096 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout: partition_filters: 0 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout: use_delta_encoding: 1 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout: filter_policy: bloomfilter 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout: whole_key_filtering: 1 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout: verify_compression: 0 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout: read_amp_bytes_per_bit: 0 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout: format_version: 5 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout: enable_index_compression: 1 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout: block_align: 0 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout: max_auto_readahead_size: 262144 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout: prepopulate_block_cache: 0 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout: initial_auto_readahead_size: 8192 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout: num_file_reads_for_auto_readahead: 2 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.write_buffer_size: 33554432 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_write_buffer_number: 2 2024-05-14T09:24:14.807 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compression: NoCompression 2024-05-14T09:24:14.808 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.bottommost_compression: Disabled 2024-05-14T09:24:14.808 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.prefix_extractor: nullptr 2024-05-14T09:24:14.808 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-14T09:24:14.808 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.num_levels: 7 2024-05-14T09:24:14.808 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-14T09:24:14.808 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-14T09:24:14.808 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-14T09:24:14.808 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-14T09:24:14.808 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-14T09:24:14.808 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-14T09:24:14.808 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-14T09:24:14.808 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-14T09:24:14.808 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-14T09:24:14.808 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-14T09:24:14.809 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-14T09:24:14.809 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-14T09:24:14.809 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-14T09:24:14.809 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compression_opts.level: 32767 2024-05-14T09:24:14.809 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compression_opts.strategy: 0 2024-05-14T09:24:14.809 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-14T09:24:14.809 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-14T09:24:14.809 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-14T09:24:14.809 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-14T09:24:14.809 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compression_opts.enabled: false 2024-05-14T09:24:14.809 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-14T09:24:14.809 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-14T09:24:14.809 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-14T09:24:14.809 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-14T09:24:14.809 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.target_file_size_base: 67108864 2024-05-14T09:24:14.810 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-14T09:24:14.810 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-14T09:24:14.810 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-14T09:24:14.810 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-14T09:24:14.810 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-14T09:24:14.810 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-14T09:24:14.810 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-14T09:24:14.810 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-14T09:24:14.810 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-14T09:24:14.810 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-14T09:24:14.810 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-14T09:24:14.810 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-14T09:24:14.810 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-14T09:24:14.810 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-14T09:24:14.811 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.arena_block_size: 1048576 2024-05-14T09:24:14.811 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-14T09:24:14.811 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-14T09:24:14.811 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.disable_auto_compactions: 0 2024-05-14T09:24:14.811 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-14T09:24:14.811 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-14T09:24:14.811 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-14T09:24:14.811 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-14T09:24:14.811 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-14T09:24:14.811 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-14T09:24:14.811 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-14T09:24:14.811 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-14T09:24:14.811 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-14T09:24:14.811 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-14T09:24:14.812 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-14T09:24:14.812 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.inplace_update_support: 0 2024-05-14T09:24:14.812 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-14T09:24:14.812 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-14T09:24:14.812 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-14T09:24:14.812 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-14T09:24:14.812 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.bloom_locality: 0 2024-05-14T09:24:14.812 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.max_successive_merges: 0 2024-05-14T09:24:14.812 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-14T09:24:14.812 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.paranoid_file_checks: 0 2024-05-14T09:24:14.812 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.force_consistency_checks: 1 2024-05-14T09:24:14.812 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.report_bg_io_stats: 0 2024-05-14T09:24:14.812 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.ttl: 2592000 2024-05-14T09:24:14.812 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-14T09:24:14.812 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.enable_blob_files: false 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.min_blob_size: 0 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.blob_file_size: 268435456 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.blob_file_starting_level: 0 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6f012845-aacc-440b-b3cc-2ba6c445a79f 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715678654481534, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-14T09:24:14.813 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-14T09:24:14.814 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715678654481998, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715678654, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6f012845-aacc-440b-b3cc-2ba6c445a79f", "db_session_id": "ZONZK8EZKE6AGV6S68IS", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-14T09:24:14.814 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715678654482087, "job": 1, "event": "recovery_finished"} 2024-05-14T09:24:14.814 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-14T09:24:14.814 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-14T09:24:14.814 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560e1e2d3c00 2024-05-14T09:24:14.814 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: DB pointer 0x560e230c6000 2024-05-14T09:24:14.814 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-14T09:24:14.814 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-14T09:24:14.814 INFO:journalctl@ceph.mon.c.smithi029.stdout: ** DB Stats ** 2024-05-14T09:24:14.814 INFO:journalctl@ceph.mon.c.smithi029.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T09:24:14.814 INFO:journalctl@ceph.mon.c.smithi029.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-14T09:24:14.814 INFO:journalctl@ceph.mon.c.smithi029.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T09:24:14.814 INFO:journalctl@ceph.mon.c.smithi029.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T09:24:14.814 INFO:journalctl@ceph.mon.c.smithi029.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-14T09:24:14.815 INFO:journalctl@ceph.mon.c.smithi029.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T09:24:14.815 INFO:journalctl@ceph.mon.c.smithi029.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T09:24:14.815 INFO:journalctl@ceph.mon.c.smithi029.stdout: 2024-05-14T09:24:14.815 INFO:journalctl@ceph.mon.c.smithi029.stdout: ** Compaction Stats [default] ** 2024-05-14T09:24:14.815 INFO:journalctl@ceph.mon.c.smithi029.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T09:24:14.815 INFO:journalctl@ceph.mon.c.smithi029.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-14T09:24:14.815 INFO:journalctl@ceph.mon.c.smithi029.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-14T09:24:14.815 INFO:journalctl@ceph.mon.c.smithi029.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-14T09:24:14.815 INFO:journalctl@ceph.mon.c.smithi029.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-14T09:24:14.815 INFO:journalctl@ceph.mon.c.smithi029.stdout: 2024-05-14T09:24:14.815 INFO:journalctl@ceph.mon.c.smithi029.stdout: ** Compaction Stats [default] ** 2024-05-14T09:24:14.815 INFO:journalctl@ceph.mon.c.smithi029.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T09:24:14.815 INFO:journalctl@ceph.mon.c.smithi029.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-14T09:24:14.815 INFO:journalctl@ceph.mon.c.smithi029.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-14T09:24:14.815 INFO:journalctl@ceph.mon.c.smithi029.stdout: 2024-05-14T09:24:14.816 INFO:journalctl@ceph.mon.c.smithi029.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-14T09:24:14.816 INFO:journalctl@ceph.mon.c.smithi029.stdout: 2024-05-14T09:24:14.816 INFO:journalctl@ceph.mon.c.smithi029.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T09:24:14.816 INFO:journalctl@ceph.mon.c.smithi029.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-14T09:24:14.816 INFO:journalctl@ceph.mon.c.smithi029.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-14T09:24:14.816 INFO:journalctl@ceph.mon.c.smithi029.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-14T09:24:14.816 INFO:journalctl@ceph.mon.c.smithi029.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-14T09:24:14.816 INFO:journalctl@ceph.mon.c.smithi029.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-14T09:24:14.816 INFO:journalctl@ceph.mon.c.smithi029.stdout: Cumulative compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T09:24:14.816 INFO:journalctl@ceph.mon.c.smithi029.stdout: Interval compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T09:24:14.816 INFO:journalctl@ceph.mon.c.smithi029.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-14T09:24:14.816 INFO:journalctl@ceph.mon.c.smithi029.stdout: Block cache BinnedLRUCache@0x560e1e2ef1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-14T09:24:14.816 INFO:journalctl@ceph.mon.c.smithi029.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-14T09:24:14.816 INFO:journalctl@ceph.mon.c.smithi029.stdout: 2024-05-14T09:24:14.817 INFO:journalctl@ceph.mon.c.smithi029.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-14T09:24:14.817 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-14T09:24:14.817 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: using public_addr v2:172.21.15.29:0/0 -> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] 2024-05-14T09:24:14.817 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: starting mon.c rank -1 at public addrs [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] at bind addrs [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:24:14.817 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: mon.c@-1(???) e0 preinit fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:24:14.817 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: mon.c@-1(synchronizing).mds e1 new map 2024-05-14T09:24:14.817 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-14T09:24:14.817 INFO:journalctl@ceph.mon.c.smithi029.stdout: e1 2024-05-14T09:24:14.817 INFO:journalctl@ceph.mon.c.smithi029.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T09:24:14.817 INFO:journalctl@ceph.mon.c.smithi029.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T09:24:14.817 INFO:journalctl@ceph.mon.c.smithi029.stdout: legacy client fscid: -1 2024-05-14T09:24:14.817 INFO:journalctl@ceph.mon.c.smithi029.stdout: 2024-05-14T09:24:14.817 INFO:journalctl@ceph.mon.c.smithi029.stdout: No filesystems configured 2024-05-14T09:24:14.817 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-14T09:24:14.818 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-14T09:24:14.818 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-14T09:24:14.818 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-14T09:24:14.818 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-14T09:24:14.818 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-14T09:24:14.818 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-14T09:24:14.818 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T09:24:14.818 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T09:24:14.818 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T09:24:14.818 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:14.818 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi029", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:24:14.818 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:14.818 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.818 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi026", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:24:14.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:14.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:14.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: Updating smithi026:/etc/ceph/ceph.conf 2024-05-14T09:24:14.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: Updating smithi026:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:14.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: Deploying cephadm binary to smithi029 2024-05-14T09:24:14.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-05-14T09:24:14.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: Updating smithi026:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T09:24:14.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:14.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.820 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:14.820 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.820 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: Added host smithi029 2024-05-14T09:24:14.820 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:14.820 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.820 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.820 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:14.820 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:24:14.820 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.820 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:14.820 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/557067254' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-14T09:24:14.820 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:14.820 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/557067254' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-14T09:24:14.821 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T09:24:14.821 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.821 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.821 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.821 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.821 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi029", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:24:14.821 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:14.821 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:14.821 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: Updating smithi029:/etc/ceph/ceph.conf 2024-05-14T09:24:14.821 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: Updating smithi029:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:14.821 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: Updating smithi029:/etc/ceph/ceph.client.admin.keyring 2024-05-14T09:24:14.821 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: Updating smithi029:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T09:24:14.821 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.821 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.821 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.822 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:14.822 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.822 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:14.822 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:14.822 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:14.822 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:14.822 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T09:24:14.822 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:14.822 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi026:172.21.15.26=b;smithi029:172.21.15.29=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:24:14.822 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: Saving service mon spec with placement smithi002:172.21.15.2=a;smithi026:172.21.15.26=b;smithi029:172.21.15.29=c;count:3 2024-05-14T09:24:14.822 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: Deploying daemon mon.c on smithi029 2024-05-14T09:24:14.822 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:14.822 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:14.823 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: from='client.? 172.21.15.29:0/870274451' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T09:24:14.823 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 ceph-mon[36249]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-14T09:24:14.823 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 bash[36216]: d42860fea015e86df8dd2f51b1382b69ad8d6dea7bad53b44ace96d225e993c5 2024-05-14T09:24:14.823 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:14 smithi029 systemd[1]: Started Ceph mon.c for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:24:16.070 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T09:24:16.070 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T09:24:16.325 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T09:24:19.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:24:19.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:19.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: mon.a calling monitor election 2024-05-14T09:24:19.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:19.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:19.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:19.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: mon.c calling monitor election 2024-05-14T09:24:19.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:19.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:19.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:19.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:19.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:19.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-14T09:24:19.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: monmap epoch 2 2024-05-14T09:24:19.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:24:19.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: last_changed 2024-05-14T09:24:14.507521+0000 2024-05-14T09:24:19.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: created 2024-05-14T09:22:31.786962+0000 2024-05-14T09:24:19.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: min_mon_release 19 (squid) 2024-05-14T09:24:19.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: election_strategy: 1 2024-05-14T09:24:19.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T09:24:19.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: 1: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.c 2024-05-14T09:24:19.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: fsmap 2024-05-14T09:24:19.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T09:24:19.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: mgrmap e13: x(active, since 50s) 2024-05-14T09:24:19.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: overall HEALTH_OK 2024-05-14T09:24:19.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:19.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:19 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:19.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:24:19.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:19.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: mon.a calling monitor election 2024-05-14T09:24:19.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:19.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:19.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:19.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: mon.c calling monitor election 2024-05-14T09:24:19.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:19.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:19.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:19.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:19.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:19.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-14T09:24:19.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: monmap epoch 2 2024-05-14T09:24:19.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:24:19.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: last_changed 2024-05-14T09:24:14.507521+0000 2024-05-14T09:24:19.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: created 2024-05-14T09:22:31.786962+0000 2024-05-14T09:24:19.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: min_mon_release 19 (squid) 2024-05-14T09:24:19.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: election_strategy: 1 2024-05-14T09:24:19.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T09:24:19.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: 1: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.c 2024-05-14T09:24:19.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: fsmap 2024-05-14T09:24:19.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T09:24:19.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: mgrmap e13: x(active, since 50s) 2024-05-14T09:24:19.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: overall HEALTH_OK 2024-05-14T09:24:19.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:19.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:19 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:20.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.781+0000 7fd49341a700 1 -- 172.21.15.29:0/519905489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4780035c0 msgr2=0x7fd478005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:20.782 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.781+0000 7fd49341a700 1 --2- 172.21.15.29:0/519905489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4780035c0 0x7fd478005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd48c06bd50 tx=0x7fd48800feb0 comp rx=0 tx=0).stop 2024-05-14T09:24:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.781+0000 7fd49341a700 1 -- 172.21.15.29:0/519905489 shutdown_connections 2024-05-14T09:24:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.781+0000 7fd49341a700 1 --2- 172.21.15.29:0/519905489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4780035c0 0x7fd478005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.781+0000 7fd49341a700 1 -- 172.21.15.29:0/519905489 >> 172.21.15.29:0/519905489 conn(0x7fd48c0fe6c0 msgr2=0x7fd48c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.781+0000 7fd49341a700 1 -- 172.21.15.29:0/519905489 shutdown_connections 2024-05-14T09:24:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.781+0000 7fd49341a700 1 -- 172.21.15.29:0/519905489 wait complete. 2024-05-14T09:24:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.782+0000 7fd49341a700 1 Processor -- start 2024-05-14T09:24:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.782+0000 7fd49341a700 1 -- start start 2024-05-14T09:24:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.782+0000 7fd49341a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4780035c0 0x7fd48c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.782+0000 7fd49341a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd48c19fc90 0x7fd48c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.782+0000 7fd49341a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c10bc40 con 0x7fd48c19fc90 2024-05-14T09:24:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.782+0000 7fd49341a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c10bac0 con 0x7fd4780035c0 2024-05-14T09:24:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.783+0000 7fd4911b6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4780035c0 0x7fd48c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.783+0000 7fd4911b6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4780035c0 0x7fd48c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54564/0 (socket says 172.21.15.29:54564) 2024-05-14T09:24:20.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.783+0000 7fd4911b6700 1 -- 172.21.15.29:0/1559853545 learned_addr learned my addr 172.21.15.29:0/1559853545 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-05-14T09:24:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.783+0000 7fd4911b6700 1 -- 172.21.15.29:0/1559853545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4780035c0 msgr2=0x7fd48c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-14T09:24:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.783+0000 7fd4911b6700 1 -- 172.21.15.29:0/1559853545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4780035c0 msgr2=0x7fd48c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-14T09:24:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.783+0000 7fd4911b6700 1 --2- 172.21.15.29:0/1559853545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4780035c0 0x7fd48c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T09:24:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.783+0000 7fd4911b6700 1 --2- 172.21.15.29:0/1559853545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4780035c0 0x7fd48c19f750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T09:24:20.784 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.783+0000 7fd4909b5700 1 --2- 172.21.15.29:0/1559853545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd48c19fc90 0x7fd48c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.783+0000 7fd4909b5700 1 -- 172.21.15.29:0/1559853545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4780035c0 msgr2=0x7fd48c19f750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T09:24:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.783+0000 7fd4909b5700 1 --2- 172.21.15.29:0/1559853545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4780035c0 0x7fd48c19f750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:20.785 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.783+0000 7fd4909b5700 1 -- 172.21.15.29:0/1559853545 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c1a0260 con 0x7fd48c19fc90 2024-05-14T09:24:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.784+0000 7fd4909b5700 1 --2- 172.21.15.29:0/1559853545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd48c19fc90 0x7fd48c199970 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00b4d0 tx=0x7fd47c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.784+0000 7fd4827fc700 1 -- 172.21.15.29:0/1559853545 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd47c011630 con 0x7fd48c19fc90 2024-05-14T09:24:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.784+0000 7fd4827fc700 1 -- 172.21.15.29:0/1559853545 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47c011c80 con 0x7fd48c19fc90 2024-05-14T09:24:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.784+0000 7fd49341a700 1 -- 172.21.15.29:0/1559853545 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c199f10 con 0x7fd48c19fc90 2024-05-14T09:24:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.784+0000 7fd4827fc700 1 -- 172.21.15.29:0/1559853545 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd47c010da0 con 0x7fd48c19fc90 2024-05-14T09:24:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.784+0000 7fd49341a700 1 -- 172.21.15.29:0/1559853545 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c19a4a0 con 0x7fd48c19fc90 2024-05-14T09:24:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.785+0000 7fd4827fc700 1 -- 172.21.15.29:0/1559853545 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fd47c018020 con 0x7fd48c19fc90 2024-05-14T09:24:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.785+0000 7fd4827fc700 1 --2- 172.21.15.29:0/1559853545 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd47003e300 0x7fd470040810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.785+0000 7fd4827fc700 1 -- 172.21.15.29:0/1559853545 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd47c0530d0 con 0x7fd48c19fc90 2024-05-14T09:24:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.785+0000 7fd49341a700 1 -- 172.21.15.29:0/1559853545 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c19a7b0 con 0x7fd48c19fc90 2024-05-14T09:24:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.786+0000 7fd4911b6700 1 --2- 172.21.15.29:0/1559853545 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd47003e300 0x7fd470040810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:20.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.786+0000 7fd4911b6700 1 --2- 172.21.15.29:0/1559853545 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd47003e300 0x7fd470040810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd48800f520 tx=0x7fd488005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:20.791 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:20.792+0000 7fd4827fc700 1 -- 172.21.15.29:0/1559853545 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd47c01f020 con 0x7fd48c19fc90 2024-05-14T09:24:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:20 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:20 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T09:24:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:20 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:20 smithi002 ceph-mon[31557]: Deploying daemon mon.b on smithi026 2024-05-14T09:24:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:20 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:20 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:20 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T09:24:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:20 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:20 smithi029 ceph-mon[36249]: Deploying daemon mon.b on smithi026 2024-05-14T09:24:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:20 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:21.101 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:21.102+0000 7fd49341a700 1 -- 172.21.15.29:0/1559853545 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd48c19aea0 con 0x7fd48c19fc90 2024-05-14T09:24:21.102 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:21.102+0000 7fd4827fc700 1 -- 172.21.15.29:0/1559853545 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1007 (secure 0 0 0) 0x7fd47c0187b0 con 0x7fd48c19fc90 2024-05-14T09:24:21.102 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:24:21.102 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":2,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","modified":"2024-05-14T09:24:14.507521Z","created":"2024-05-14T09:22:31.786962Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:3300","nonce":0},{"type":"v1","addr":"172.21.15.29:6789","nonce":0}]},"addr":"172.21.15.29:6789/0","public_addr":"172.21.15.29:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-14T09:24:21.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:21.105+0000 7fd49341a700 1 -- 172.21.15.29:0/1559853545 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd47003e300 msgr2=0x7fd470040810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:21.104 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:21.105+0000 7fd49341a700 1 --2- 172.21.15.29:0/1559853545 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd47003e300 0x7fd470040810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd48800f520 tx=0x7fd488005960 comp rx=0 tx=0).stop 2024-05-14T09:24:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:21.105+0000 7fd49341a700 1 -- 172.21.15.29:0/1559853545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd48c19fc90 msgr2=0x7fd48c199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:21.105+0000 7fd49341a700 1 --2- 172.21.15.29:0/1559853545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd48c19fc90 0x7fd48c199970 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00b4d0 tx=0x7fd47c00b890 comp rx=0 tx=0).stop 2024-05-14T09:24:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:21.105+0000 7fd49341a700 1 -- 172.21.15.29:0/1559853545 shutdown_connections 2024-05-14T09:24:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:21.105+0000 7fd49341a700 1 --2- 172.21.15.29:0/1559853545 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd47003e300 0x7fd470040810 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:21.105+0000 7fd49341a700 1 --2- 172.21.15.29:0/1559853545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4780035c0 0x7fd48c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:21.105+0000 7fd49341a700 1 --2- 172.21.15.29:0/1559853545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd48c19fc90 0x7fd48c199970 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:21.105+0000 7fd49341a700 1 -- 172.21.15.29:0/1559853545 >> 172.21.15.29:0/1559853545 conn(0x7fd48c0fe6c0 msgr2=0x7fd48c101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:21.105+0000 7fd49341a700 1 -- 172.21.15.29:0/1559853545 shutdown_connections 2024-05-14T09:24:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:21.105+0000 7fd49341a700 1 -- 172.21.15.29:0/1559853545 wait complete. 2024-05-14T09:24:21.105 INFO:teuthology.orchestra.run.smithi029.stderr:dumped monmap epoch 2 2024-05-14T09:24:21.666 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:21 smithi029 ceph-mon[36249]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:21.666 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:21 smithi029 ceph-mon[36249]: from='client.? 172.21.15.29:0/1559853545' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T09:24:21.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:21 smithi002 ceph-mon[31557]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:21.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:21 smithi002 ceph-mon[31557]: from='client.? 172.21.15.29:0/1559853545' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T09:24:22.697 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T09:24:22.697 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T09:24:22.948 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T09:24:23.066 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 systemd[1]: Starting Ceph mon.b for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T09:24:23.353 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 podman[36037]: 2024-05-14 09:24:23.255115612 +0000 UTC m=+0.110243442 container create a44b30303c5ae2dca3a01ff65ee13b96cff9ae70e6c885e19a827f6434bbcf6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-b, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2) 2024-05-14T09:24:23.353 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 podman[36037]: 2024-05-14 09:24:23.163049881 +0000 UTC m=+0.018177729 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T09:24:23.633 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 podman[36037]: 2024-05-14 09:24:23.440638362 +0000 UTC m=+0.295766199 container init a44b30303c5ae2dca3a01ff65ee13b96cff9ae70e6c885e19a827f6434bbcf6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD) 2024-05-14T09:24:23.633 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 podman[36037]: 2024-05-14 09:24:23.443670892 +0000 UTC m=+0.298798738 container start a44b30303c5ae2dca3a01ff65ee13b96cff9ae70e6c885e19a827f6434bbcf6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-b, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-14T09:24:23.633 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: set uid:gid to 167:167 (ceph:ceph) 2024-05-14T09:24:23.634 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev), process ceph-mon, pid 2 2024-05-14T09:24:23.634 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: pidfile_write: ignore empty --pid-file 2024-05-14T09:24:23.634 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: load: jerasure load: lrc 2024-05-14T09:24:23.634 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: RocksDB version: 7.9.2 2024-05-14T09:24:23.634 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Git sha 0 2024-05-14T09:24:23.634 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Compile date 2024-05-09 05:50:40 2024-05-14T09:24:23.634 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: DB SUMMARY 2024-05-14T09:24:23.634 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: DB Session ID: 0I3QG1YA0O4R7M5FHR3N 2024-05-14T09:24:23.634 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: CURRENT file: CURRENT 2024-05-14T09:24:23.634 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: IDENTITY file: IDENTITY 2024-05-14T09:24:23.634 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-14T09:24:23.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-14T09:24:23.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-14T09:24:23.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.error_if_exists: 0 2024-05-14T09:24:23.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.create_if_missing: 0 2024-05-14T09:24:23.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.paranoid_checks: 1 2024-05-14T09:24:23.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-14T09:24:23.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-14T09:24:23.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-14T09:24:23.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.env: 0x56383a8f3500 2024-05-14T09:24:23.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.fs: PosixFileSystem 2024-05-14T09:24:23.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.info_log: 0x56383c8c2020 2024-05-14T09:24:23.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_file_opening_threads: 16 2024-05-14T09:24:23.637 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.statistics: (nil) 2024-05-14T09:24:23.637 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.use_fsync: 0 2024-05-14T09:24:23.637 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_log_file_size: 0 2024-05-14T09:24:23.637 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-14T09:24:23.638 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-14T09:24:23.638 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.keep_log_file_num: 1000 2024-05-14T09:24:23.638 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.recycle_log_file_num: 0 2024-05-14T09:24:23.638 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.allow_fallocate: 1 2024-05-14T09:24:23.638 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.allow_mmap_reads: 0 2024-05-14T09:24:23.638 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.allow_mmap_writes: 0 2024-05-14T09:24:23.638 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.use_direct_reads: 0 2024-05-14T09:24:23.638 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-14T09:24:23.638 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.create_missing_column_families: 0 2024-05-14T09:24:23.638 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.db_log_dir: 2024-05-14T09:24:23.638 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.wal_dir: 2024-05-14T09:24:23.638 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-14T09:24:23.639 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-14T09:24:23.639 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-14T09:24:23.639 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-14T09:24:23.639 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-14T09:24:23.639 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-14T09:24:23.639 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.advise_random_on_open: 1 2024-05-14T09:24:23.639 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.db_write_buffer_size: 0 2024-05-14T09:24:23.639 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.write_buffer_manager: 0x56383c78c8c0 2024-05-14T09:24:23.639 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-14T09:24:23.639 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-14T09:24:23.639 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-14T09:24:23.639 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.rate_limiter: (nil) 2024-05-14T09:24:23.640 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-14T09:24:23.640 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.wal_recovery_mode: 2 2024-05-14T09:24:23.640 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.enable_thread_tracking: 0 2024-05-14T09:24:23.640 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.enable_pipelined_write: 0 2024-05-14T09:24:23.640 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.unordered_write: 0 2024-05-14T09:24:23.640 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-14T09:24:23.640 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-14T09:24:23.640 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-14T09:24:23.640 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-14T09:24:23.640 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.row_cache: None 2024-05-14T09:24:23.640 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.wal_filter: None 2024-05-14T09:24:23.640 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-14T09:24:23.640 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.allow_ingest_behind: 0 2024-05-14T09:24:23.641 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.two_write_queues: 0 2024-05-14T09:24:23.641 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.manual_wal_flush: 0 2024-05-14T09:24:23.641 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.wal_compression: 0 2024-05-14T09:24:23.641 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.atomic_flush: 0 2024-05-14T09:24:23.641 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-14T09:24:23.641 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-14T09:24:23.641 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-14T09:24:23.641 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.log_readahead_size: 0 2024-05-14T09:24:23.641 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-14T09:24:23.641 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.best_efforts_recovery: 0 2024-05-14T09:24:23.641 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-14T09:24:23.641 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-14T09:24:23.642 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.allow_data_in_errors: 0 2024-05-14T09:24:23.642 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.db_host_id: __hostname__ 2024-05-14T09:24:23.642 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-14T09:24:23.642 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_background_jobs: 2 2024-05-14T09:24:23.642 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_background_compactions: -1 2024-05-14T09:24:23.642 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_subcompactions: 1 2024-05-14T09:24:23.642 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-14T09:24:23.642 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-14T09:24:23.642 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-14T09:24:23.642 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_total_wal_size: 0 2024-05-14T09:24:23.642 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-14T09:24:23.642 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-14T09:24:23.643 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-14T09:24:23.643 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-14T09:24:23.643 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_open_files: -1 2024-05-14T09:24:23.643 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.bytes_per_sync: 0 2024-05-14T09:24:23.643 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-14T09:24:23.643 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-14T09:24:23.643 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compaction_readahead_size: 0 2024-05-14T09:24:23.643 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_background_flushes: -1 2024-05-14T09:24:23.643 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Compression algorithms supported: 2024-05-14T09:24:23.643 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-14T09:24:23.643 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: kZSTD supported: 0 2024-05-14T09:24:23.643 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: kXpressCompression supported: 0 2024-05-14T09:24:23.644 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: kLZ4HCCompression supported: 1 2024-05-14T09:24:23.644 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: kZlibCompression supported: 1 2024-05-14T09:24:23.644 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: kSnappyCompression supported: 1 2024-05-14T09:24:23.644 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: kLZ4Compression supported: 1 2024-05-14T09:24:23.644 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: kBZip2Compression supported: 0 2024-05-14T09:24:23.644 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-14T09:24:23.644 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-14T09:24:23.644 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-14T09:24:23.644 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-14T09:24:23.644 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-14T09:24:23.644 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.merge_operator: 2024-05-14T09:24:23.644 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compaction_filter: None 2024-05-14T09:24:23.644 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compaction_filter_factory: None 2024-05-14T09:24:23.645 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.sst_partitioner_factory: None 2024-05-14T09:24:23.645 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-14T09:24:23.645 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-14T09:24:23.645 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56383c8c21e0) 2024-05-14T09:24:23.645 INFO:journalctl@ceph.mon.b.smithi026.stdout: cache_index_and_filter_blocks: 1 2024-05-14T09:24:23.645 INFO:journalctl@ceph.mon.b.smithi026.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-14T09:24:23.645 INFO:journalctl@ceph.mon.b.smithi026.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-14T09:24:23.645 INFO:journalctl@ceph.mon.b.smithi026.stdout: pin_top_level_index_and_filter: 1 2024-05-14T09:24:23.645 INFO:journalctl@ceph.mon.b.smithi026.stdout: index_type: 0 2024-05-14T09:24:23.645 INFO:journalctl@ceph.mon.b.smithi026.stdout: data_block_index_type: 0 2024-05-14T09:24:23.645 INFO:journalctl@ceph.mon.b.smithi026.stdout: index_shortening: 1 2024-05-14T09:24:23.645 INFO:journalctl@ceph.mon.b.smithi026.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-14T09:24:23.645 INFO:journalctl@ceph.mon.b.smithi026.stdout: checksum: 4 2024-05-14T09:24:23.646 INFO:journalctl@ceph.mon.b.smithi026.stdout: no_block_cache: 0 2024-05-14T09:24:23.646 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_cache: 0x56383c7bf1f0 2024-05-14T09:24:23.646 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_cache_name: BinnedLRUCache 2024-05-14T09:24:23.646 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_cache_options: 2024-05-14T09:24:23.646 INFO:journalctl@ceph.mon.b.smithi026.stdout: capacity : 536870912 2024-05-14T09:24:23.646 INFO:journalctl@ceph.mon.b.smithi026.stdout: num_shard_bits : 4 2024-05-14T09:24:23.646 INFO:journalctl@ceph.mon.b.smithi026.stdout: strict_capacity_limit : 0 2024-05-14T09:24:23.646 INFO:journalctl@ceph.mon.b.smithi026.stdout: high_pri_pool_ratio: 0.000 2024-05-14T09:24:23.646 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_cache_compressed: (nil) 2024-05-14T09:24:23.646 INFO:journalctl@ceph.mon.b.smithi026.stdout: persistent_cache: (nil) 2024-05-14T09:24:23.646 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_size: 4096 2024-05-14T09:24:23.646 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_size_deviation: 10 2024-05-14T09:24:23.646 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_restart_interval: 16 2024-05-14T09:24:23.647 INFO:journalctl@ceph.mon.b.smithi026.stdout: index_block_restart_interval: 1 2024-05-14T09:24:23.647 INFO:journalctl@ceph.mon.b.smithi026.stdout: metadata_block_size: 4096 2024-05-14T09:24:23.647 INFO:journalctl@ceph.mon.b.smithi026.stdout: partition_filters: 0 2024-05-14T09:24:23.647 INFO:journalctl@ceph.mon.b.smithi026.stdout: use_delta_encoding: 1 2024-05-14T09:24:23.647 INFO:journalctl@ceph.mon.b.smithi026.stdout: filter_policy: bloomfilter 2024-05-14T09:24:23.647 INFO:journalctl@ceph.mon.b.smithi026.stdout: whole_key_filtering: 1 2024-05-14T09:24:23.647 INFO:journalctl@ceph.mon.b.smithi026.stdout: verify_compression: 0 2024-05-14T09:24:23.647 INFO:journalctl@ceph.mon.b.smithi026.stdout: read_amp_bytes_per_bit: 0 2024-05-14T09:24:23.647 INFO:journalctl@ceph.mon.b.smithi026.stdout: format_version: 5 2024-05-14T09:24:23.647 INFO:journalctl@ceph.mon.b.smithi026.stdout: enable_index_compression: 1 2024-05-14T09:24:23.647 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_align: 0 2024-05-14T09:24:23.647 INFO:journalctl@ceph.mon.b.smithi026.stdout: max_auto_readahead_size: 262144 2024-05-14T09:24:23.647 INFO:journalctl@ceph.mon.b.smithi026.stdout: prepopulate_block_cache: 0 2024-05-14T09:24:23.648 INFO:journalctl@ceph.mon.b.smithi026.stdout: initial_auto_readahead_size: 8192 2024-05-14T09:24:23.648 INFO:journalctl@ceph.mon.b.smithi026.stdout: num_file_reads_for_auto_readahead: 2 2024-05-14T09:24:23.648 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.write_buffer_size: 33554432 2024-05-14T09:24:23.648 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_write_buffer_number: 2 2024-05-14T09:24:23.648 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compression: NoCompression 2024-05-14T09:24:23.648 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.bottommost_compression: Disabled 2024-05-14T09:24:23.648 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.prefix_extractor: nullptr 2024-05-14T09:24:23.648 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-14T09:24:23.648 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.num_levels: 7 2024-05-14T09:24:23.648 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-14T09:24:23.648 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-14T09:24:23.648 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-14T09:24:23.648 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-14T09:24:23.648 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-14T09:24:23.649 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-14T09:24:23.649 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-14T09:24:23.649 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-14T09:24:23.649 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-14T09:24:23.649 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-14T09:24:23.649 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-14T09:24:23.649 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-14T09:24:23.649 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-14T09:24:23.649 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compression_opts.level: 32767 2024-05-14T09:24:23.649 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compression_opts.strategy: 0 2024-05-14T09:24:23.649 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-14T09:24:23.649 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-14T09:24:23.649 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-14T09:24:23.650 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-14T09:24:23.650 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compression_opts.enabled: false 2024-05-14T09:24:23.650 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-14T09:24:23.650 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-14T09:24:23.650 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-14T09:24:23.650 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-14T09:24:23.650 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.target_file_size_base: 67108864 2024-05-14T09:24:23.650 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-14T09:24:23.650 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-14T09:24:23.650 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-14T09:24:23.650 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-14T09:24:23.650 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-14T09:24:23.651 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-14T09:24:23.651 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-14T09:24:23.651 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-14T09:24:23.651 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-14T09:24:23.651 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-14T09:24:23.651 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-14T09:24:23.651 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-14T09:24:23.651 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-14T09:24:23.651 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-14T09:24:23.651 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.arena_block_size: 1048576 2024-05-14T09:24:23.651 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-14T09:24:23.652 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-14T09:24:23.652 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.disable_auto_compactions: 0 2024-05-14T09:24:23.652 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-14T09:24:23.652 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-14T09:24:23.652 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-14T09:24:23.652 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-14T09:24:23.652 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-14T09:24:23.652 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-14T09:24:23.652 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-14T09:24:23.652 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-14T09:24:23.652 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-14T09:24:23.652 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-14T09:24:23.653 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-14T09:24:23.653 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.inplace_update_support: 0 2024-05-14T09:24:23.653 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-14T09:24:23.653 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-14T09:24:23.653 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-14T09:24:23.653 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-14T09:24:23.653 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.bloom_locality: 0 2024-05-14T09:24:23.653 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.max_successive_merges: 0 2024-05-14T09:24:23.653 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-14T09:24:23.653 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.paranoid_file_checks: 0 2024-05-14T09:24:23.653 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.force_consistency_checks: 1 2024-05-14T09:24:23.653 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.report_bg_io_stats: 0 2024-05-14T09:24:23.653 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.ttl: 2592000 2024-05-14T09:24:23.654 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-14T09:24:23.654 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-14T09:24:23.654 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-14T09:24:23.654 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.enable_blob_files: false 2024-05-14T09:24:23.654 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.min_blob_size: 0 2024-05-14T09:24:23.654 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.blob_file_size: 268435456 2024-05-14T09:24:23.654 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-14T09:24:23.654 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-14T09:24:23.654 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-14T09:24:23.654 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-14T09:24:23.654 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-14T09:24:23.655 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.blob_file_starting_level: 0 2024-05-14T09:24:23.655 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-14T09:24:23.655 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-14T09:24:23.655 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-14T09:24:23.655 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 720bc8e6-e3a6-4ea1-ae79-7b644acaf15c 2024-05-14T09:24:23.655 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715678663485124, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-14T09:24:23.655 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-14T09:24:23.655 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715678663485733, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715678663, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "720bc8e6-e3a6-4ea1-ae79-7b644acaf15c", "db_session_id": "0I3QG1YA0O4R7M5FHR3N", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-14T09:24:23.655 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715678663485853, "job": 1, "event": "recovery_finished"} 2024-05-14T09:24:23.655 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-14T09:24:23.655 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-14T09:24:23.655 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56383c7a3c00 2024-05-14T09:24:23.656 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: DB pointer 0x563841596000 2024-05-14T09:24:23.656 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-14T09:24:23.656 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-14T09:24:23.656 INFO:journalctl@ceph.mon.b.smithi026.stdout: ** DB Stats ** 2024-05-14T09:24:23.656 INFO:journalctl@ceph.mon.b.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T09:24:23.656 INFO:journalctl@ceph.mon.b.smithi026.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-14T09:24:23.656 INFO:journalctl@ceph.mon.b.smithi026.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T09:24:23.656 INFO:journalctl@ceph.mon.b.smithi026.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T09:24:23.656 INFO:journalctl@ceph.mon.b.smithi026.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-14T09:24:23.656 INFO:journalctl@ceph.mon.b.smithi026.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T09:24:23.656 INFO:journalctl@ceph.mon.b.smithi026.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T09:24:23.656 INFO:journalctl@ceph.mon.b.smithi026.stdout: 2024-05-14T09:24:23.657 INFO:journalctl@ceph.mon.b.smithi026.stdout: ** Compaction Stats [default] ** 2024-05-14T09:24:23.657 INFO:journalctl@ceph.mon.b.smithi026.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T09:24:23.657 INFO:journalctl@ceph.mon.b.smithi026.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-14T09:24:23.657 INFO:journalctl@ceph.mon.b.smithi026.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T09:24:23.657 INFO:journalctl@ceph.mon.b.smithi026.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T09:24:23.657 INFO:journalctl@ceph.mon.b.smithi026.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T09:24:23.657 INFO:journalctl@ceph.mon.b.smithi026.stdout: 2024-05-14T09:24:23.657 INFO:journalctl@ceph.mon.b.smithi026.stdout: ** Compaction Stats [default] ** 2024-05-14T09:24:23.657 INFO:journalctl@ceph.mon.b.smithi026.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T09:24:23.657 INFO:journalctl@ceph.mon.b.smithi026.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-14T09:24:23.657 INFO:journalctl@ceph.mon.b.smithi026.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T09:24:23.657 INFO:journalctl@ceph.mon.b.smithi026.stdout: 2024-05-14T09:24:23.658 INFO:journalctl@ceph.mon.b.smithi026.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-14T09:24:23.658 INFO:journalctl@ceph.mon.b.smithi026.stdout: 2024-05-14T09:24:23.658 INFO:journalctl@ceph.mon.b.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T09:24:23.658 INFO:journalctl@ceph.mon.b.smithi026.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-14T09:24:23.658 INFO:journalctl@ceph.mon.b.smithi026.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-14T09:24:23.658 INFO:journalctl@ceph.mon.b.smithi026.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-14T09:24:23.658 INFO:journalctl@ceph.mon.b.smithi026.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-14T09:24:23.658 INFO:journalctl@ceph.mon.b.smithi026.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-14T09:24:23.658 INFO:journalctl@ceph.mon.b.smithi026.stdout: Cumulative compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T09:24:23.658 INFO:journalctl@ceph.mon.b.smithi026.stdout: Interval compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T09:24:23.659 INFO:journalctl@ceph.mon.b.smithi026.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-14T09:24:23.659 INFO:journalctl@ceph.mon.b.smithi026.stdout: Block cache BinnedLRUCache@0x56383c7bf1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-14T09:24:23.659 INFO:journalctl@ceph.mon.b.smithi026.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-14T09:24:23.659 INFO:journalctl@ceph.mon.b.smithi026.stdout: 2024-05-14T09:24:23.659 INFO:journalctl@ceph.mon.b.smithi026.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-14T09:24:23.659 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-14T09:24:23.659 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: using public_addr v2:172.21.15.26:0/0 -> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] 2024-05-14T09:24:23.659 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: starting mon.b rank -1 at public addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] at bind addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:24:23.659 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.b@-1(???) e0 preinit fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:24:23.659 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.b@-1(synchronizing).mds e1 new map 2024-05-14T09:24:23.659 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-14T09:24:23.659 INFO:journalctl@ceph.mon.b.smithi026.stdout: e1 2024-05-14T09:24:23.660 INFO:journalctl@ceph.mon.b.smithi026.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T09:24:23.660 INFO:journalctl@ceph.mon.b.smithi026.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T09:24:23.660 INFO:journalctl@ceph.mon.b.smithi026.stdout: legacy client fscid: -1 2024-05-14T09:24:23.660 INFO:journalctl@ceph.mon.b.smithi026.stdout: 2024-05-14T09:24:23.660 INFO:journalctl@ceph.mon.b.smithi026.stdout: No filesystems configured 2024-05-14T09:24:23.660 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-14T09:24:23.660 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-14T09:24:23.661 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-14T09:24:23.661 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-14T09:24:23.661 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-14T09:24:23.661 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-14T09:24:23.661 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-14T09:24:23.661 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T09:24:23.661 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T09:24:23.661 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T09:24:23.661 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:23.661 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi029", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:24:23.661 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:23.662 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.662 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.662 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.662 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.662 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi026", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:24:23.662 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:23.662 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:23.662 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: Updating smithi026:/etc/ceph/ceph.conf 2024-05-14T09:24:23.662 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: Updating smithi026:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:23.662 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: Deploying cephadm binary to smithi029 2024-05-14T09:24:23.662 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-05-14T09:24:23.663 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: Updating smithi026:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T09:24:23.663 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:23.663 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.663 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.663 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.663 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:23.663 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.663 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: Added host smithi029 2024-05-14T09:24:23.663 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:23.663 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.663 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.663 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:23.664 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:24:23.664 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.664 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:23.664 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/557067254' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-14T09:24:23.664 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:23.664 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/557067254' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-14T09:24:23.664 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T09:24:23.664 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.664 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.664 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.664 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.665 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi029", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:24:23.665 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:23.665 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:23.665 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: Updating smithi029:/etc/ceph/ceph.conf 2024-05-14T09:24:23.665 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: Updating smithi029:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:23.665 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: Updating smithi029:/etc/ceph/ceph.client.admin.keyring 2024-05-14T09:24:23.665 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: Updating smithi029:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T09:24:23.666 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.666 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.666 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.666 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:23.666 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.666 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:23.666 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:23.666 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:23.666 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.666 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T09:24:23.666 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:23.666 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi026:172.21.15.26=b;smithi029:172.21.15.29=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:24:23.666 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: Saving service mon spec with placement smithi002:172.21.15.2=a;smithi026:172.21.15.26=b;smithi029:172.21.15.29=c;count:3 2024-05-14T09:24:23.667 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: Deploying daemon mon.c on smithi029 2024-05-14T09:24:23.667 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:23.667 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:23.667 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='client.? 172.21.15.29:0/870274451' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T09:24:23.667 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:24:23.667 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:23.667 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.a calling monitor election 2024-05-14T09:24:23.667 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:23.667 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:23.667 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:23.667 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.c calling monitor election 2024-05-14T09:24:23.667 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:23.667 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:23.668 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:23.668 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:23.668 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:23.668 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-14T09:24:23.668 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: monmap epoch 2 2024-05-14T09:24:23.668 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:24:23.668 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: last_changed 2024-05-14T09:24:14.507521+0000 2024-05-14T09:24:23.668 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: created 2024-05-14T09:22:31.786962+0000 2024-05-14T09:24:23.668 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: min_mon_release 19 (squid) 2024-05-14T09:24:23.668 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: election_strategy: 1 2024-05-14T09:24:23.668 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T09:24:23.668 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: 1: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.c 2024-05-14T09:24:23.668 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: fsmap 2024-05-14T09:24:23.669 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T09:24:23.669 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mgrmap e13: x(active, since 50s) 2024-05-14T09:24:23.669 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: overall HEALTH_OK 2024-05-14T09:24:23.669 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.669 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.669 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:23.669 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T09:24:23.669 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:23.669 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: Deploying daemon mon.b on smithi026 2024-05-14T09:24:23.669 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:23.669 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:23.669 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: from='client.? 172.21.15.29:0/1559853545' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T09:24:23.669 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 ceph-mon[36056]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-14T09:24:23.670 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 bash[36037]: a44b30303c5ae2dca3a01ff65ee13b96cff9ae70e6c885e19a827f6434bbcf6f 2024-05-14T09:24:23.670 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:23 smithi026 systemd[1]: Started Ceph mon.b for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:24:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.746+0000 7f8fc8a38700 1 -- 172.21.15.29:0/1208295441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fb00035c0 msgr2=0x7f8fb0005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.746+0000 7f8fc8a38700 1 --2- 172.21.15.29:0/1208295441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fb00035c0 0x7f8fb0005aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4103b70 tx=0x7f8fb400fe60 comp rx=0 tx=0).stop 2024-05-14T09:24:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.746+0000 7f8fc8a38700 1 -- 172.21.15.29:0/1208295441 shutdown_connections 2024-05-14T09:24:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.746+0000 7f8fc8a38700 1 --2- 172.21.15.29:0/1208295441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fb00035c0 0x7f8fb0005aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:28.747 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.746+0000 7f8fc8a38700 1 -- 172.21.15.29:0/1208295441 >> 172.21.15.29:0/1208295441 conn(0x7f8fc40ff760 msgr2=0x7f8fc4101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:28.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.746+0000 7f8fc8a38700 1 -- 172.21.15.29:0/1208295441 shutdown_connections 2024-05-14T09:24:28.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.746+0000 7f8fc8a38700 1 -- 172.21.15.29:0/1208295441 wait complete. 2024-05-14T09:24:28.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.748+0000 7f8fc8a38700 1 Processor -- start 2024-05-14T09:24:28.748 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.748+0000 7f8fc8a38700 1 -- start start 2024-05-14T09:24:28.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.748+0000 7f8fc8a38700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb00035c0 0x7f8fc419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:28.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.748+0000 7f8fc8a38700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc419fd90 0x7f8fc4199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:28.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.748+0000 7f8fc8a38700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fc4199fb0 0x7f8fc419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:28.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.748+0000 7f8fc8a38700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc410bba0 con 0x7f8fc4199fb0 2024-05-14T09:24:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.748+0000 7f8fc8a38700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc410ba20 con 0x7f8fc419fd90 2024-05-14T09:24:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.748+0000 7f8fc8a38700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc410bd20 con 0x7f8fb00035c0 2024-05-14T09:24:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb00035c0 0x7f8fc419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fc4199fb0 0x7f8fc419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb00035c0 0x7f8fc419f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:54572/0 (socket says 172.21.15.29:54572) 2024-05-14T09:24:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fc4199fb0 0x7f8fc419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.29:53936/0 (socket says 172.21.15.29:53936) 2024-05-14T09:24:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc259c700 1 -- 172.21.15.29:0/329240651 learned_addr learned my addr 172.21.15.29:0/329240651 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-05-14T09:24:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc259c700 1 -- 172.21.15.29:0/329240651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb00035c0 msgr2=0x7f8fc419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-14T09:24:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc259c700 1 -- 172.21.15.29:0/329240651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb00035c0 msgr2=0x7f8fc419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-14T09:24:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc259c700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb00035c0 0x7f8fc419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T09:24:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc259c700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb00035c0 0x7f8fc419f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T09:24:28.751 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc1d9b700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc419fd90 0x7f8fc4199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:28.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc2d9d700 1 -- 172.21.15.29:0/329240651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb00035c0 msgr2=0x7f8fc419f850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T09:24:28.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc2d9d700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb00035c0 0x7f8fc419f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:28.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc2d9d700 1 -- 172.21.15.29:0/329240651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc419fd90 msgr2=0x7f8fc4199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:28.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc1d9b700 1 -- 172.21.15.29:0/329240651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc419fd90 msgr2=0x7f8fc4199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-14T09:24:28.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc1d9b700 1 -- 172.21.15.29:0/329240651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc419fd90 msgr2=0x7f8fc4199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-14T09:24:28.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc1d9b700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc419fd90 0x7f8fc4199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T09:24:28.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc1d9b700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc419fd90 0x7f8fc4199a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T09:24:28.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc2d9d700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc419fd90 0x7f8fc4199a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:28.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc2d9d700 1 -- 172.21.15.29:0/329240651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc41a66e0 con 0x7f8fc4199fb0 2024-05-14T09:24:28.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fc2d9d700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fc4199fb0 0x7f8fc419a3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00bd50 tx=0x7f8fbc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:28.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.749+0000 7f8fbb7fe700 1 -- 172.21.15.29:0/329240651 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc019070 con 0x7f8fc4199fb0 2024-05-14T09:24:28.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.750+0000 7f8fbb7fe700 1 -- 172.21.15.29:0/329240651 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fbc00cd10 con 0x7f8fc4199fb0 2024-05-14T09:24:28.752 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.750+0000 7f8fbb7fe700 1 -- 172.21.15.29:0/329240651 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc012700 con 0x7f8fc4199fb0 2024-05-14T09:24:28.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.750+0000 7f8fc8a38700 1 -- 172.21.15.29:0/329240651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc41a6970 con 0x7f8fc4199fb0 2024-05-14T09:24:28.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.750+0000 7f8fc8a38700 1 -- 172.21.15.29:0/329240651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc41a6e30 con 0x7f8fc4199fb0 2024-05-14T09:24:28.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.751+0000 7f8fbb7fe700 1 -- 172.21.15.29:0/329240651 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f8fbc01c070 con 0x7f8fc4199fb0 2024-05-14T09:24:28.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.751+0000 7f8fbb7fe700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8fa403e9a0 0x7f8fa4040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:28.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.751+0000 7f8fbb7fe700 1 -- 172.21.15.29:0/329240651 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8fbc052f70 con 0x7f8fc4199fb0 2024-05-14T09:24:28.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.751+0000 7f8fc8a38700 1 -- 172.21.15.29:0/329240651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc4064e20 con 0x7f8fc4199fb0 2024-05-14T09:24:28.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.751+0000 7f8fc259c700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8fa403e9a0 0x7f8fa4040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:28.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.752+0000 7f8fc259c700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8fa403e9a0 0x7f8fa4040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400fa10 tx=0x7f8fb4005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:28.757 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:28.757+0000 7f8fbb7fe700 1 -- 172.21.15.29:0/329240651 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8fbc022170 con 0x7f8fc4199fb0 2024-05-14T09:24:28.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:24:28.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: mon.c calling monitor election 2024-05-14T09:24:28.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: mon.a calling monitor election 2024-05-14T09:24:28.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:28.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:28.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:28.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:28.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:28.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:28.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:28.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:28.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:28.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-14T09:24:28.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: monmap epoch 3 2024-05-14T09:24:28.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:24:28.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: last_changed 2024-05-14T09:24:23.513689+0000 2024-05-14T09:24:28.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: created 2024-05-14T09:22:31.786962+0000 2024-05-14T09:24:28.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: min_mon_release 19 (squid) 2024-05-14T09:24:28.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: election_strategy: 1 2024-05-14T09:24:28.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T09:24:28.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: 1: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.c 2024-05-14T09:24:28.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: 2: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.b 2024-05-14T09:24:28.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: fsmap 2024-05-14T09:24:28.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T09:24:28.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: mgrmap e13: x(active, since 59s) 2024-05-14T09:24:28.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: overall HEALTH_OK 2024-05-14T09:24:28.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:28.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:24:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: mon.c calling monitor election 2024-05-14T09:24:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: mon.a calling monitor election 2024-05-14T09:24:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:28.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:28.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:28.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:28.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:28.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:28.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:28.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:28.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-14T09:24:28.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: monmap epoch 3 2024-05-14T09:24:28.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:24:28.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: last_changed 2024-05-14T09:24:23.513689+0000 2024-05-14T09:24:28.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: created 2024-05-14T09:22:31.786962+0000 2024-05-14T09:24:28.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: min_mon_release 19 (squid) 2024-05-14T09:24:28.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: election_strategy: 1 2024-05-14T09:24:28.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T09:24:28.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: 1: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.c 2024-05-14T09:24:28.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: 2: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.b 2024-05-14T09:24:28.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: fsmap 2024-05-14T09:24:28.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T09:24:28.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: mgrmap e13: x(active, since 59s) 2024-05-14T09:24:28.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: overall HEALTH_OK 2024-05-14T09:24:28.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:28.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:29.068 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:29.068+0000 7f8fc8a38700 1 -- 172.21.15.29:0/329240651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8fc402e5f0 con 0x7f8fc4199fb0 2024-05-14T09:24:29.069 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:29.068+0000 7f8fbb7fe700 1 -- 172.21.15.29:0/329240651 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1271 (secure 0 0 0) 0x7f8fbc014a10 con 0x7f8fc4199fb0 2024-05-14T09:24:29.069 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-05-14T09:24:29.069 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":3,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","modified":"2024-05-14T09:24:23.513689Z","created":"2024-05-14T09:22:31.786962Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:3300","nonce":0},{"type":"v1","addr":"172.21.15.29:6789","nonce":0}]},"addr":"172.21.15.29:6789/0","public_addr":"172.21.15.29:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-14T09:24:29.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:29.071+0000 7f8fc8a38700 1 -- 172.21.15.29:0/329240651 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8fa403e9a0 msgr2=0x7f8fa4040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:29.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:29.071+0000 7f8fc8a38700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8fa403e9a0 0x7f8fa4040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400fa10 tx=0x7f8fb4005a90 comp rx=0 tx=0).stop 2024-05-14T09:24:29.071 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:29.071+0000 7f8fc8a38700 1 -- 172.21.15.29:0/329240651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fc4199fb0 msgr2=0x7f8fc419a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:29.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:29.071+0000 7f8fc8a38700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fc4199fb0 0x7f8fc419a3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00bd50 tx=0x7f8fbc00be30 comp rx=0 tx=0).stop 2024-05-14T09:24:29.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:29.071+0000 7f8fc8a38700 1 -- 172.21.15.29:0/329240651 shutdown_connections 2024-05-14T09:24:29.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:29.071+0000 7f8fc8a38700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8fa403e9a0 0x7f8fa4040eb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:29.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:29.071+0000 7f8fc8a38700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fc4199fb0 0x7f8fc419a3c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:29.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:29.071+0000 7f8fc8a38700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8fb00035c0 0x7f8fc419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:29.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:29.071+0000 7f8fc8a38700 1 --2- 172.21.15.29:0/329240651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc419fd90 0x7f8fc4199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:29.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:29.071+0000 7f8fc8a38700 1 -- 172.21.15.29:0/329240651 >> 172.21.15.29:0/329240651 conn(0x7f8fc40ff760 msgr2=0x7f8fc4107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:29.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:29.071+0000 7f8fc8a38700 1 -- 172.21.15.29:0/329240651 shutdown_connections 2024-05-14T09:24:29.072 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:29.071+0000 7f8fc8a38700 1 -- 172.21.15.29:0/329240651 wait complete. 2024-05-14T09:24:29.072 INFO:teuthology.orchestra.run.smithi029.stderr:dumped monmap epoch 3 2024-05-14T09:24:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:29 smithi029 ceph-mon[36249]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:29 smithi029 ceph-mon[36249]: from='client.? 172.21.15.29:0/329240651' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T09:24:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:29.764 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-14T09:24:29.764 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-14T09:24:29.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:29.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:29.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:29.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:29 smithi002 ceph-mon[31557]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:29.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:29.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:29 smithi002 ceph-mon[31557]: from='client.? 172.21.15.29:0/329240651' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T09:24:29.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:30.029 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:24:30.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:24:30.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: mon.c calling monitor election 2024-05-14T09:24:30.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: mon.a calling monitor election 2024-05-14T09:24:30.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:30.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:24:30.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:30.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:30.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:30.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:30.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:30.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:30.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:30.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-14T09:24:30.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: monmap epoch 3 2024-05-14T09:24:30.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:24:30.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: last_changed 2024-05-14T09:24:23.513689+0000 2024-05-14T09:24:30.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: created 2024-05-14T09:22:31.786962+0000 2024-05-14T09:24:30.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: min_mon_release 19 (squid) 2024-05-14T09:24:30.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: election_strategy: 1 2024-05-14T09:24:30.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T09:24:30.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: 1: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.c 2024-05-14T09:24:30.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: 2: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.b 2024-05-14T09:24:30.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: fsmap 2024-05-14T09:24:30.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T09:24:30.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: mgrmap e13: x(active, since 59s) 2024-05-14T09:24:30.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: overall HEALTH_OK 2024-05-14T09:24:30.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:30.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:30.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:30.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:30.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:30.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:30.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:30.955 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='client.? 172.21.15.29:0/329240651' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T09:24:30.955 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:31.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: mon.b calling monitor election 2024-05-14T09:24:31.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: mon.b calling monitor election 2024-05-14T09:24:31.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: mon.c calling monitor election 2024-05-14T09:24:31.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: mon.a calling monitor election 2024-05-14T09:24:31.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-14T09:24:31.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: monmap epoch 3 2024-05-14T09:24:31.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:24:31.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: last_changed 2024-05-14T09:24:23.513689+0000 2024-05-14T09:24:31.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: created 2024-05-14T09:22:31.786962+0000 2024-05-14T09:24:31.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: min_mon_release 19 (squid) 2024-05-14T09:24:31.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: election_strategy: 1 2024-05-14T09:24:31.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T09:24:31.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: 1: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.c 2024-05-14T09:24:31.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: 2: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.b 2024-05-14T09:24:31.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: fsmap 2024-05-14T09:24:31.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T09:24:31.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: mgrmap e13: x(active, since 61s) 2024-05-14T09:24:31.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: overall HEALTH_OK 2024-05-14T09:24:31.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:31.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:31.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:31.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:31 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:31.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: mon.b calling monitor election 2024-05-14T09:24:31.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: mon.b calling monitor election 2024-05-14T09:24:31.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: mon.c calling monitor election 2024-05-14T09:24:31.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: mon.a calling monitor election 2024-05-14T09:24:31.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-14T09:24:31.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: monmap epoch 3 2024-05-14T09:24:31.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:24:31.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: last_changed 2024-05-14T09:24:23.513689+0000 2024-05-14T09:24:31.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: created 2024-05-14T09:22:31.786962+0000 2024-05-14T09:24:31.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: min_mon_release 19 (squid) 2024-05-14T09:24:31.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: election_strategy: 1 2024-05-14T09:24:31.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T09:24:31.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: 1: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.c 2024-05-14T09:24:31.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: 2: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.b 2024-05-14T09:24:31.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: fsmap 2024-05-14T09:24:31.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T09:24:31.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: mgrmap e13: x(active, since 61s) 2024-05-14T09:24:31.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: overall HEALTH_OK 2024-05-14T09:24:31.439 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:31.439 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:31.439 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:31.439 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:31 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:31.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: mon.b calling monitor election 2024-05-14T09:24:31.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: mon.b calling monitor election 2024-05-14T09:24:31.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: mon.c calling monitor election 2024-05-14T09:24:31.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: mon.a calling monitor election 2024-05-14T09:24:31.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-14T09:24:31.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: monmap epoch 3 2024-05-14T09:24:31.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:24:31.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: last_changed 2024-05-14T09:24:23.513689+0000 2024-05-14T09:24:31.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: created 2024-05-14T09:22:31.786962+0000 2024-05-14T09:24:31.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: min_mon_release 19 (squid) 2024-05-14T09:24:31.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: election_strategy: 1 2024-05-14T09:24:31.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T09:24:31.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: 1: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.c 2024-05-14T09:24:31.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: 2: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.b 2024-05-14T09:24:31.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: fsmap 2024-05-14T09:24:31.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T09:24:31.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: mgrmap e13: x(active, since 61s) 2024-05-14T09:24:31.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: overall HEALTH_OK 2024-05-14T09:24:31.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:31.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:31.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:31.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:31 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:31.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.817+0000 7f20cb45e700 1 -- 172.21.15.2:0/1661265393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4103df0 msgr2=0x7f20c4104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:31.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.817+0000 7f20cb45e700 1 --2- 172.21.15.2:0/1661265393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4103df0 0x7f20c4104200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009910 tx=0x7f20c0009c20 comp rx=0 tx=0).stop 2024-05-14T09:24:31.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.817+0000 7f20cb45e700 1 -- 172.21.15.2:0/1661265393 shutdown_connections 2024-05-14T09:24:31.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.817+0000 7f20cb45e700 1 --2- 172.21.15.2:0/1661265393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4103df0 0x7f20c4104200 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:31.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.817+0000 7f20cb45e700 1 -- 172.21.15.2:0/1661265393 >> 172.21.15.2:0/1661265393 conn(0x7f20c40ff6a0 msgr2=0x7f20c4101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:31.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.817+0000 7f20cb45e700 1 -- 172.21.15.2:0/1661265393 shutdown_connections 2024-05-14T09:24:31.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.817+0000 7f20cb45e700 1 -- 172.21.15.2:0/1661265393 wait complete. 2024-05-14T09:24:31.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.818+0000 7f20cb45e700 1 Processor -- start 2024-05-14T09:24:31.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.818+0000 7f20cb45e700 1 -- start start 2024-05-14T09:24:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20cb45e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4103df0 0x7f20c419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20cb45e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20c419fd50 0x7f20c4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20cb45e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20c4199f70 0x7f20c419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20cb45e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f20c410e0e0 con 0x7f20c4103df0 2024-05-14T09:24:31.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20cb45e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f20c410df60 con 0x7f20c4199f70 2024-05-14T09:24:31.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20cb45e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f20c410e260 con 0x7f20c419fd50 2024-05-14T09:24:31.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20c91fa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4103df0 0x7f20c419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:31.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20c91fa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4103df0 0x7f20c419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39666/0 (socket says 172.21.15.2:39666) 2024-05-14T09:24:31.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20c91fa700 1 -- 172.21.15.2:0/467217184 learned_addr learned my addr 172.21.15.2:0/467217184 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:24:31.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20c89f9700 1 --2- 172.21.15.2:0/467217184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20c419fd50 0x7f20c4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:31.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20c99fb700 1 --2- 172.21.15.2:0/467217184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20c4199f70 0x7f20c419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:31.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20c89f9700 1 -- 172.21.15.2:0/467217184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20c4199f70 msgr2=0x7f20c419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20c89f9700 1 --2- 172.21.15.2:0/467217184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20c4199f70 0x7f20c419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20c89f9700 1 -- 172.21.15.2:0/467217184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4103df0 msgr2=0x7f20c419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20c89f9700 1 --2- 172.21.15.2:0/467217184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4103df0 0x7f20c419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20c89f9700 1 -- 172.21.15.2:0/467217184 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c40759f0 con 0x7f20c419fd50 2024-05-14T09:24:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.819+0000 7f20c91fa700 1 --2- 172.21.15.2:0/467217184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4103df0 0x7f20c419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:24:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.820+0000 7f20c89f9700 1 --2- 172.21.15.2:0/467217184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20c419fd50 0x7f20c4199a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f20b400b550 tx=0x7f20b400b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.820+0000 7f20ba7fc700 1 -- 172.21.15.2:0/467217184 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4004230 con 0x7f20c419fd50 2024-05-14T09:24:31.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.820+0000 7f20ba7fc700 1 -- 172.21.15.2:0/467217184 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b40043d0 con 0x7f20c419fd50 2024-05-14T09:24:31.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.820+0000 7f20cb45e700 1 -- 172.21.15.2:0/467217184 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c4075ce0 con 0x7f20c419fd50 2024-05-14T09:24:31.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.820+0000 7f20ba7fc700 1 -- 172.21.15.2:0/467217184 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4011a50 con 0x7f20c419fd50 2024-05-14T09:24:31.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.820+0000 7f20cb45e700 1 -- 172.21.15.2:0/467217184 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c4076270 con 0x7f20c419fd50 2024-05-14T09:24:31.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.821+0000 7f20ba7fc700 1 -- 172.21.15.2:0/467217184 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f20b4011cf0 con 0x7f20c419fd50 2024-05-14T09:24:31.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.821+0000 7f20cb45e700 1 -- 172.21.15.2:0/467217184 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c4064e20 con 0x7f20c419fd50 2024-05-14T09:24:31.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.821+0000 7f20ba7fc700 1 --2- 172.21.15.2:0/467217184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f20b003e9a0 0x7f20b0040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:31.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.821+0000 7f20ba7fc700 1 -- 172.21.15.2:0/467217184 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f20b40521b0 con 0x7f20c419fd50 2024-05-14T09:24:31.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.822+0000 7f20c91fa700 1 --2- 172.21.15.2:0/467217184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f20b003e9a0 0x7f20b0040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:31.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.822+0000 7f20c91fa700 1 --2- 172.21.15.2:0/467217184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f20b003e9a0 0x7f20b0040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f20c0006010 tx=0x7f20c001e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:31.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:31.827+0000 7f20ba7fc700 1 -- 172.21.15.2:0/467217184 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f20b4020740 con 0x7f20c419fd50 2024-05-14T09:24:32.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:32.047+0000 7f20cb45e700 1 -- 172.21.15.2:0/467217184 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f20c4076580 con 0x7f20c419fd50 2024-05-14T09:24:32.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:32.048+0000 7f20ba7fc700 1 -- 172.21.15.2:0/467217184 <== mon.1 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+269 (secure 0 0 0) 0x7f20b401fcc0 con 0x7f20c419fd50 2024-05-14T09:24:32.048 INFO:teuthology.orchestra.run.smithi002.stdout:# minimal ceph.conf for 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:24:32.048 INFO:teuthology.orchestra.run.smithi002.stdout:[global] 2024-05-14T09:24:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: fsid = 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:24:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: mon_host = [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] 2024-05-14T09:24:32.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:32.050+0000 7f20cb45e700 1 -- 172.21.15.2:0/467217184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f20b003e9a0 msgr2=0x7f20b0040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:32.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:32.050+0000 7f20cb45e700 1 --2- 172.21.15.2:0/467217184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f20b003e9a0 0x7f20b0040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f20c0006010 tx=0x7f20c001e040 comp rx=0 tx=0).stop 2024-05-14T09:24:32.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:32.050+0000 7f20cb45e700 1 -- 172.21.15.2:0/467217184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20c419fd50 msgr2=0x7f20c4199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:32.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:32.050+0000 7f20cb45e700 1 --2- 172.21.15.2:0/467217184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20c419fd50 0x7f20c4199a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f20b400b550 tx=0x7f20b400b860 comp rx=0 tx=0).stop 2024-05-14T09:24:32.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:32.050+0000 7f20cb45e700 1 -- 172.21.15.2:0/467217184 shutdown_connections 2024-05-14T09:24:32.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:32.050+0000 7f20cb45e700 1 --2- 172.21.15.2:0/467217184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f20b003e9a0 0x7f20b0040eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:32.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:32.050+0000 7f20cb45e700 1 --2- 172.21.15.2:0/467217184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20c4199f70 0x7f20c419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:32.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:32.050+0000 7f20cb45e700 1 --2- 172.21.15.2:0/467217184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4103df0 0x7f20c419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:32.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:32.050+0000 7f20cb45e700 1 --2- 172.21.15.2:0/467217184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f20c419fd50 0x7f20c4199a30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:32.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:32.050+0000 7f20cb45e700 1 -- 172.21.15.2:0/467217184 >> 172.21.15.2:0/467217184 conn(0x7f20c40ff6a0 msgr2=0x7f20c4109920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:32.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:32.050+0000 7f20cb45e700 1 -- 172.21.15.2:0/467217184 shutdown_connections 2024-05-14T09:24:32.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:32.050+0000 7f20cb45e700 1 -- 172.21.15.2:0/467217184 wait complete. 2024-05-14T09:24:32.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: Updating smithi002:/etc/ceph/ceph.conf 2024-05-14T09:24:32.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: Updating smithi026:/etc/ceph/ceph.conf 2024-05-14T09:24:32.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: Updating smithi029:/etc/ceph/ceph.conf 2024-05-14T09:24:32.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: Updating smithi029:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:32.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: Updating smithi026:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:32.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: Updating smithi002:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:32.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:32.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: Reconfiguring mon.a (unknown last config time)... 2024-05-14T09:24:32.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T09:24:32.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T09:24:32.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:32.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: Reconfiguring daemon mon.a on smithi002 2024-05-14T09:24:32.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:32.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:32 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/467217184' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: Updating smithi002:/etc/ceph/ceph.conf 2024-05-14T09:24:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: Updating smithi026:/etc/ceph/ceph.conf 2024-05-14T09:24:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: Updating smithi029:/etc/ceph/ceph.conf 2024-05-14T09:24:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: Updating smithi029:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: Updating smithi026:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: Updating smithi002:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: Reconfiguring mon.a (unknown last config time)... 2024-05-14T09:24:32.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T09:24:32.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T09:24:32.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:32.439 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: Reconfiguring daemon mon.a on smithi002 2024-05-14T09:24:32.439 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:32.439 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:32 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/467217184' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:32.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: Updating smithi002:/etc/ceph/ceph.conf 2024-05-14T09:24:32.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: Updating smithi026:/etc/ceph/ceph.conf 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: Updating smithi029:/etc/ceph/ceph.conf 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: Updating smithi029:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: Updating smithi026:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: Updating smithi002:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:32.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: Reconfiguring mon.a (unknown last config time)... 2024-05-14T09:24:32.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T09:24:32.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T09:24:32.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:32.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: Reconfiguring daemon mon.a on smithi002 2024-05-14T09:24:32.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:24:32.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:32 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/467217184' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:33.107 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-14T09:24:33.107 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:24:33.107 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.conf 2024-05-14T09:24:33.139 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:24:33.139 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T09:24:33.208 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-14T09:24:33.208 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.conf 2024-05-14T09:24:33.238 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-14T09:24:33.238 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T09:24:33.305 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-05-14T09:24:33.305 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/ceph/ceph.conf 2024-05-14T09:24:33.337 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-05-14T09:24:33.337 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T09:24:33.407 INFO:tasks.cephadm:Adding mgr.x on smithi002 2024-05-14T09:24:33.407 INFO:tasks.cephadm:Adding mgr.y on smithi026 2024-05-14T09:24:33.407 INFO:tasks.cephadm:Adding mgr.z on smithi029 2024-05-14T09:24:33.408 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi002=x;smithi026=y;smithi029=z' 2024-05-14T09:24:33.691 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T09:24:34.420 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:34 smithi026 ceph-mon[36056]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:34.421 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:34 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:34.421 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:34 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:34.421 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:34 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T09:24:34.421 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:34 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T09:24:34.421 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:34 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:34.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:34 smithi002 ceph-mon[31557]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:34.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:34 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:34.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:34 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:34.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:34 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T09:24:34.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:34 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T09:24:34.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:34 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:34.434 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:34 smithi029 ceph-mon[36249]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:34.434 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:34 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:34.435 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:34 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:34.435 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:34 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T09:24:34.435 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:34 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T09:24:34.435 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:34 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:35.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:35 smithi002 ceph-mon[31557]: Reconfiguring mon.b (monmap changed)... 2024-05-14T09:24:35.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:35 smithi002 ceph-mon[31557]: Reconfiguring daemon mon.b on smithi026 2024-05-14T09:24:35.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:35 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:35.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:35 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:35.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:35 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T09:24:35.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:35 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T09:24:35.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:35 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:35.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:35 smithi029 ceph-mon[36249]: Reconfiguring mon.b (monmap changed)... 2024-05-14T09:24:35.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:35 smithi029 ceph-mon[36249]: Reconfiguring daemon mon.b on smithi026 2024-05-14T09:24:35.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:35 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:35.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:35 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:35.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:35 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T09:24:35.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:35 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T09:24:35.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:35 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:35.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:35 smithi026 ceph-mon[36056]: Reconfiguring mon.b (monmap changed)... 2024-05-14T09:24:35.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:35 smithi026 ceph-mon[36056]: Reconfiguring daemon mon.b on smithi026 2024-05-14T09:24:35.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:35 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:35.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:35 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:35.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:35 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T09:24:35.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:35 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T09:24:35.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:35 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:36 smithi002 ceph-mon[31557]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:36 smithi002 ceph-mon[31557]: Reconfiguring mon.c (monmap changed)... 2024-05-14T09:24:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:36 smithi002 ceph-mon[31557]: Reconfiguring daemon mon.c on smithi029 2024-05-14T09:24:36.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:36 smithi029 ceph-mon[36249]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:36.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:36 smithi029 ceph-mon[36249]: Reconfiguring mon.c (monmap changed)... 2024-05-14T09:24:36.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:36 smithi029 ceph-mon[36249]: Reconfiguring daemon mon.c on smithi029 2024-05-14T09:24:36.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:36 smithi026 ceph-mon[36056]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:36.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:36 smithi026 ceph-mon[36056]: Reconfiguring mon.c (monmap changed)... 2024-05-14T09:24:36.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:36 smithi026 ceph-mon[36056]: Reconfiguring daemon mon.c on smithi029 2024-05-14T09:24:38.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:38 smithi002 ceph-mon[31557]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:38.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:38 smithi029 ceph-mon[36249]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:38.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:38 smithi026 ceph-mon[36056]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:39.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.808+0000 7fc90e65c700 1 -- 172.21.15.29:0/511031583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc908103d30 msgr2=0x7fc908104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:39.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.808+0000 7fc90e65c700 1 --2- 172.21.15.29:0/511031583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc908103d30 0x7fc908104140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009910 tx=0x7fc8f8009c20 comp rx=0 tx=0).stop 2024-05-14T09:24:39.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.808+0000 7fc90e65c700 1 -- 172.21.15.29:0/511031583 shutdown_connections 2024-05-14T09:24:39.811 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.808+0000 7fc90e65c700 1 --2- 172.21.15.29:0/511031583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc908103d30 0x7fc908104140 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.808+0000 7fc90e65c700 1 -- 172.21.15.29:0/511031583 >> 172.21.15.29:0/511031583 conn(0x7fc9080ff370 msgr2=0x7fc908101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.808+0000 7fc90e65c700 1 -- 172.21.15.29:0/511031583 shutdown_connections 2024-05-14T09:24:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.808+0000 7fc90e65c700 1 -- 172.21.15.29:0/511031583 wait complete. 2024-05-14T09:24:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.809+0000 7fc90e65c700 1 Processor -- start 2024-05-14T09:24:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.809+0000 7fc90e65c700 1 -- start start 2024-05-14T09:24:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.809+0000 7fc90e65c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc908103d30 0x7fc908077460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.809+0000 7fc90e65c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc908075b10 0x7fc908075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.809+0000 7fc90e65c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc908076460 0x7fc9081a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.809+0000 7fc90e65c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc908109cf0 con 0x7fc908075b10 2024-05-14T09:24:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.809+0000 7fc90e65c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc908109b70 con 0x7fc908076460 2024-05-14T09:24:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.809+0000 7fc90e65c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc908109e70 con 0x7fc908103d30 2024-05-14T09:24:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.810+0000 7fc9077fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc908075b10 0x7fc908075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:39.812 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.810+0000 7fc907fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc908103d30 0x7fc908077460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.810+0000 7fc9077fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc908075b10 0x7fc908075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.29:55522/0 (socket says 172.21.15.29:55522) 2024-05-14T09:24:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.810+0000 7fc9077fe700 1 -- 172.21.15.29:0/2958384832 learned_addr learned my addr 172.21.15.29:0/2958384832 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-05-14T09:24:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.810+0000 7fc90cbf9700 1 --2- 172.21.15.29:0/2958384832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc908076460 0x7fc9081a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.810+0000 7fc9077fe700 1 -- 172.21.15.29:0/2958384832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc908103d30 msgr2=0x7fc908077460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.810+0000 7fc9077fe700 1 --2- 172.21.15.29:0/2958384832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc908103d30 0x7fc908077460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.810+0000 7fc9077fe700 1 -- 172.21.15.29:0/2958384832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc908076460 msgr2=0x7fc9081a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.810+0000 7fc9077fe700 1 --2- 172.21.15.29:0/2958384832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc908076460 0x7fc9081a3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.810+0000 7fc9077fe700 1 -- 172.21.15.29:0/2958384832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9081a3f50 con 0x7fc908075b10 2024-05-14T09:24:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.810+0000 7fc9077fe700 1 --2- 172.21.15.29:0/2958384832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc908075b10 0x7fc908075f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00b780 tx=0x7fc8fc00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.810+0000 7fc9057fa700 1 -- 172.21.15.29:0/2958384832 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc016070 con 0x7fc908075b10 2024-05-14T09:24:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.811+0000 7fc9057fa700 1 -- 172.21.15.29:0/2958384832 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8fc0036a0 con 0x7fc908075b10 2024-05-14T09:24:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.811+0000 7fc90e65c700 1 -- 172.21.15.29:0/2958384832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9081a41e0 con 0x7fc908075b10 2024-05-14T09:24:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.811+0000 7fc9057fa700 1 -- 172.21.15.29:0/2958384832 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc0119c0 con 0x7fc908075b10 2024-05-14T09:24:39.813 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.811+0000 7fc90e65c700 1 -- 172.21.15.29:0/2958384832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9081a47c0 con 0x7fc908075b10 2024-05-14T09:24:39.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.812+0000 7fc9057fa700 1 -- 172.21.15.29:0/2958384832 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fc8fc019070 con 0x7fc908075b10 2024-05-14T09:24:39.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.812+0000 7fc90e65c700 1 -- 172.21.15.29:0/2958384832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc908064e20 con 0x7fc908075b10 2024-05-14T09:24:39.814 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.812+0000 7fc9057fa700 1 --2- 172.21.15.29:0/2958384832 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc8f003e9a0 0x7fc8f0040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:39.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.812+0000 7fc9057fa700 1 -- 172.21.15.29:0/2958384832 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc8fc051270 con 0x7fc908075b10 2024-05-14T09:24:39.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.812+0000 7fc907fff700 1 --2- 172.21.15.29:0/2958384832 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc8f003e9a0 0x7fc8f0040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:39.815 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.813+0000 7fc907fff700 1 --2- 172.21.15.29:0/2958384832 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc8f003e9a0 0x7fc8f0040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8006010 tx=0x7fc8f801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:39.818 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:39.818+0000 7fc9057fa700 1 -- 172.21.15.29:0/2958384832 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc8fc019360 con 0x7fc908075b10 2024-05-14T09:24:40.047 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:40.047+0000 7fc90e65c700 1 -- 172.21.15.29:0/2958384832 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi026=y;smithi029=z", "target": ["mon-mgr", ""]}) -- 0x7fc908101140 con 0x7fc8f003e9a0 2024-05-14T09:24:40.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:40.054+0000 7fc9057fa700 1 -- 172.21.15.29:0/2958384832 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc908101140 con 0x7fc8f003e9a0 2024-05-14T09:24:40.054 INFO:teuthology.orchestra.run.smithi029.stdout:Scheduled mgr update... 2024-05-14T09:24:40.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:40.056+0000 7fc90e65c700 1 -- 172.21.15.29:0/2958384832 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc8f003e9a0 msgr2=0x7fc8f0040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:40.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:40.056+0000 7fc90e65c700 1 --2- 172.21.15.29:0/2958384832 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc8f003e9a0 0x7fc8f0040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8006010 tx=0x7fc8f801a040 comp rx=0 tx=0).stop 2024-05-14T09:24:40.057 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:40.056+0000 7fc90e65c700 1 -- 172.21.15.29:0/2958384832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc908075b10 msgr2=0x7fc908075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:40.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:40.056+0000 7fc90e65c700 1 --2- 172.21.15.29:0/2958384832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc908075b10 0x7fc908075f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00b780 tx=0x7fc8fc00ba90 comp rx=0 tx=0).stop 2024-05-14T09:24:40.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:40.056+0000 7fc90e65c700 1 -- 172.21.15.29:0/2958384832 shutdown_connections 2024-05-14T09:24:40.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:40.056+0000 7fc90e65c700 1 --2- 172.21.15.29:0/2958384832 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc8f003e9a0 0x7fc8f0040eb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:40.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:40.056+0000 7fc90e65c700 1 --2- 172.21.15.29:0/2958384832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc908076460 0x7fc9081a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:40.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:40.056+0000 7fc90e65c700 1 --2- 172.21.15.29:0/2958384832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc908103d30 0x7fc908077460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:40.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:40.056+0000 7fc90e65c700 1 --2- 172.21.15.29:0/2958384832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc908075b10 0x7fc908075f20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:40.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:40.056+0000 7fc90e65c700 1 -- 172.21.15.29:0/2958384832 >> 172.21.15.29:0/2958384832 conn(0x7fc9080ff370 msgr2=0x7fc908100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:40.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:40.056+0000 7fc90e65c700 1 -- 172.21.15.29:0/2958384832 shutdown_connections 2024-05-14T09:24:40.058 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:24:40.056+0000 7fc90e65c700 1 -- 172.21.15.29:0/2958384832 wait complete. 2024-05-14T09:24:40.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:40 smithi002 ceph-mon[31557]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:40.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:40.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:40 smithi029 ceph-mon[36249]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:40.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:40.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:40 smithi026 ceph-mon[36056]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:40.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:40.707 DEBUG:teuthology.orchestra.run.smithi026:mgr.y> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mgr.y.service 2024-05-14T09:24:40.709 DEBUG:teuthology.orchestra.run.smithi029:mgr.z> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mgr.z.service 2024-05-14T09:24:40.712 INFO:tasks.cephadm:Deploying OSDs... 2024-05-14T09:24:40.712 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:24:40.712 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T09:24:40.729 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T09:24:40.729 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_1 2024-05-14T09:24:40.788 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T09:24:40.788 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T09:24:40.788 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-14T09:24:40.788 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T09:24:40.788 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T09:24:40.788 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-05-14 09:23:39.145649481 +0000 2024-05-14T09:24:40.788 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-05-14 09:17:31.964867312 +0000 2024-05-14T09:24:40.788 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-05-14 09:17:31.964867312 +0000 2024-05-14T09:24:40.788 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-05-14 09:17:31.964867312 +0000 2024-05-14T09:24:40.788 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T09:24:40.853 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-05-14T09:24:40.853 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-05-14T09:24:40.853 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000109148 s, 4.7 MB/s 2024-05-14T09:24:40.855 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T09:24:40.912 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_2 2024-05-14T09:24:40.968 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T09:24:40.969 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T09:24:40.969 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-14T09:24:40.969 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T09:24:40.969 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T09:24:40.969 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-05-14 09:23:39.173648862 +0000 2024-05-14T09:24:40.969 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-05-14 09:17:32.328859279 +0000 2024-05-14T09:24:40.969 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-05-14 09:17:32.328859279 +0000 2024-05-14T09:24:40.969 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-05-14 09:17:32.328859279 +0000 2024-05-14T09:24:40.969 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T09:24:41.033 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-05-14T09:24:41.033 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-05-14T09:24:41.034 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000109997 s, 4.7 MB/s 2024-05-14T09:24:41.035 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T09:24:41.092 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_3 2024-05-14T09:24:41.149 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T09:24:41.150 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T09:24:41.150 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-14T09:24:41.150 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T09:24:41.150 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T09:24:41.150 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-05-14 09:23:39.199648288 +0000 2024-05-14T09:24:41.150 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-05-14 09:17:32.667851798 +0000 2024-05-14T09:24:41.150 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-05-14 09:17:32.667851798 +0000 2024-05-14T09:24:41.150 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-05-14 09:17:32.667851798 +0000 2024-05-14T09:24:41.150 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T09:24:41.213 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-05-14T09:24:41.213 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-05-14T09:24:41.213 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 9.5436e-05 s, 5.4 MB/s 2024-05-14T09:24:41.215 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T09:24:41.271 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_4 2024-05-14T09:24:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T09:24:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T09:24:41.328 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-14T09:24:41.328 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T09:24:41.329 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T09:24:41.329 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-05-14 09:23:39.226647692 +0000 2024-05-14T09:24:41.329 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-05-14 09:17:33.014844139 +0000 2024-05-14T09:24:41.329 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-05-14 09:17:33.014844139 +0000 2024-05-14T09:24:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-05-14 09:17:33.014844139 +0000 2024-05-14T09:24:41.330 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T09:24:41.393 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-05-14T09:24:41.393 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-05-14T09:24:41.393 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000107608 s, 4.8 MB/s 2024-05-14T09:24:41.394 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T09:24:41.452 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-05-14T09:24:41.452 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T09:24:41.472 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T09:24:41.472 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2024-05-14T09:24:41.530 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T09:24:41.530 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T09:24:41.531 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-14T09:24:41.531 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T09:24:41.531 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T09:24:41.531 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-05-14 09:24:22.943058115 +0000 2024-05-14T09:24:41.531 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-05-14 09:18:06.991165781 +0000 2024-05-14T09:24:41.531 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-05-14 09:18:06.991165781 +0000 2024-05-14T09:24:41.531 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-05-14 09:18:06.991165781 +0000 2024-05-14T09:24:41.531 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T09:24:41.532 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:41 smithi029 ceph-mon[36249]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi026=y;smithi029=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:24:41.532 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:41 smithi029 ceph-mon[36249]: Saving service mgr spec with placement smithi002=x;smithi026=y;smithi029=z;count:3 2024-05-14T09:24:41.532 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:41.532 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:41.532 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:41.533 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:41.533 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:41.533 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:41.533 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T09:24:41.533 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T09:24:41.533 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T09:24:41.533 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:41.595 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:41 smithi026 ceph-mon[36056]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi026=y;smithi029=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:24:41.595 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:41 smithi026 ceph-mon[36056]: Saving service mgr spec with placement smithi002=x;smithi026=y;smithi029=z;count:3 2024-05-14T09:24:41.595 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:41.595 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:41.596 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:41.596 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:41.596 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:41.596 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:41.596 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T09:24:41.596 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T09:24:41.596 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T09:24:41.596 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:41.598 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-05-14T09:24:41.598 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-05-14T09:24:41.598 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.00011268 s, 4.5 MB/s 2024-05-14T09:24:41.599 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T09:24:41.658 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2024-05-14T09:24:41.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:41 smithi002 ceph-mon[31557]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi026=y;smithi029=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:24:41.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:41 smithi002 ceph-mon[31557]: Saving service mgr spec with placement smithi002=x;smithi026=y;smithi029=z;count:3 2024-05-14T09:24:41.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:41.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:41.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:41.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:41.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:41.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:41.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T09:24:41.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T09:24:41.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T09:24:41.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:41.714 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T09:24:41.714 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T09:24:41.714 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-14T09:24:41.714 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T09:24:41.714 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T09:24:41.714 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-05-14 09:24:22.943058115 +0000 2024-05-14T09:24:41.714 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-05-14 09:18:07.328161028 +0000 2024-05-14T09:24:41.714 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-05-14 09:18:07.328161028 +0000 2024-05-14T09:24:41.714 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-05-14 09:18:07.328161028 +0000 2024-05-14T09:24:41.714 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T09:24:41.779 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-05-14T09:24:41.779 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-05-14T09:24:41.779 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000102044 s, 5.0 MB/s 2024-05-14T09:24:41.781 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T09:24:41.837 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2024-05-14T09:24:41.894 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T09:24:41.894 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T09:24:41.894 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-14T09:24:41.894 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T09:24:41.894 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T09:24:41.894 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-05-14 09:24:22.944058102 +0000 2024-05-14T09:24:41.894 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-05-14 09:18:07.656156403 +0000 2024-05-14T09:24:41.894 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-05-14 09:18:07.656156403 +0000 2024-05-14T09:24:41.894 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-05-14 09:18:07.656156403 +0000 2024-05-14T09:24:41.894 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T09:24:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-05-14T09:24:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-05-14T09:24:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000116768 s, 4.4 MB/s 2024-05-14T09:24:41.960 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T09:24:42.017 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2024-05-14T09:24:42.073 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T09:24:42.073 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T09:24:42.073 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-14T09:24:42.073 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T09:24:42.073 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T09:24:42.073 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-05-14 09:24:22.944058102 +0000 2024-05-14T09:24:42.073 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-05-14 09:18:07.994151637 +0000 2024-05-14T09:24:42.073 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-05-14 09:18:07.994151637 +0000 2024-05-14T09:24:42.073 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-05-14 09:18:07.994151637 +0000 2024-05-14T09:24:42.073 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T09:24:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-05-14T09:24:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-05-14T09:24:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000111979 s, 4.6 MB/s 2024-05-14T09:24:42.139 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T09:24:42.196 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-05-14T09:24:42.196 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T09:24:42.214 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T09:24:42.214 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_1 2024-05-14T09:24:42.271 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T09:24:42.271 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T09:24:42.271 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-14T09:24:42.271 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T09:24:42.271 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T09:24:42.271 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-05-14 09:24:13.910131541 +0000 2024-05-14T09:24:42.271 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-05-14 09:18:12.445782059 +0000 2024-05-14T09:24:42.271 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-05-14 09:18:12.445782059 +0000 2024-05-14T09:24:42.271 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-05-14 09:18:12.445782059 +0000 2024-05-14T09:24:42.271 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T09:24:42.336 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-05-14T09:24:42.336 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-05-14T09:24:42.336 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000110887 s, 4.6 MB/s 2024-05-14T09:24:42.338 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T09:24:42.395 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_2 2024-05-14T09:24:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:42 smithi002 ceph-mon[31557]: Deploying daemon mgr.z on smithi029 2024-05-14T09:24:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:42 smithi002 ceph-mon[31557]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:42.451 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T09:24:42.451 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T09:24:42.451 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-14T09:24:42.451 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T09:24:42.451 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T09:24:42.451 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-05-14 09:24:13.910131541 +0000 2024-05-14T09:24:42.451 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-05-14 09:18:12.777776353 +0000 2024-05-14T09:24:42.451 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-05-14 09:18:12.777776353 +0000 2024-05-14T09:24:42.451 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-05-14 09:18:12.777776353 +0000 2024-05-14T09:24:42.451 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T09:24:42.512 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:42 smithi029 ceph-mon[36249]: Deploying daemon mgr.z on smithi029 2024-05-14T09:24:42.512 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:42 smithi029 ceph-mon[36249]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:42.515 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-05-14T09:24:42.515 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-05-14T09:24:42.515 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000109755 s, 4.7 MB/s 2024-05-14T09:24:42.516 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T09:24:42.576 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_3 2024-05-14T09:24:42.631 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T09:24:42.631 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T09:24:42.631 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-14T09:24:42.631 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T09:24:42.632 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T09:24:42.632 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-05-14 09:24:13.911131524 +0000 2024-05-14T09:24:42.632 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-05-14 09:18:13.106770698 +0000 2024-05-14T09:24:42.632 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-05-14 09:18:13.106770698 +0000 2024-05-14T09:24:42.632 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-05-14 09:18:13.106770698 +0000 2024-05-14T09:24:42.632 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T09:24:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:42 smithi026 ceph-mon[36056]: Deploying daemon mgr.z on smithi029 2024-05-14T09:24:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:42 smithi026 ceph-mon[36056]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:42.702 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-05-14T09:24:42.702 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-05-14T09:24:42.702 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.00010891 s, 4.7 MB/s 2024-05-14T09:24:42.703 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T09:24:42.766 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_4 2024-05-14T09:24:42.822 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T09:24:42.822 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T09:24:42.822 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-14T09:24:42.822 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T09:24:42.822 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T09:24:42.822 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-05-14 09:24:42.637657817 +0000 2024-05-14T09:24:42.822 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-05-14 09:18:13.450764785 +0000 2024-05-14T09:24:42.822 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-05-14 09:18:13.450764785 +0000 2024-05-14T09:24:42.822 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: 2024-05-14 09:18:13.450764785 +0000 2024-05-14T09:24:42.823 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T09:24:42.900 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-05-14T09:24:42.900 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-05-14T09:24:42.900 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000107512 s, 4.8 MB/s 2024-05-14T09:24:42.902 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T09:24:42.959 INFO:tasks.cephadm:Deploying osd.0 on smithi002 with /dev/vg_nvme/lv_4... 2024-05-14T09:24:42.959 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-14T09:24:43.187 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:43 smithi029 podman[38073]: 2024-05-14 09:24:43.020750264 +0000 UTC m=+0.017930793 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T09:24:43.214 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:24:43.507 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:43 smithi029 podman[38073]: 2024-05-14 09:24:43.329469171 +0000 UTC m=+0.326649700 container init aeefc0356c8a7ded73ee6430df419b01caec3ad5381f757639dafdddcfde8428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-14T09:24:43.507 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:43 smithi029 podman[38073]: 2024-05-14 09:24:43.332331779 +0000 UTC m=+0.329512310 container start aeefc0356c8a7ded73ee6430df419b01caec3ad5381f757639dafdddcfde8428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-05-14T09:24:43.508 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:43 smithi029 ceph-mgr[38092]: -- 172.21.15.29:0/1474244029 <== mon.1 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5610c7eee5a0 con 0x5610ccc78480 2024-05-14T09:24:43.508 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:43 smithi029 bash[38073]: aeefc0356c8a7ded73ee6430df419b01caec3ad5381f757639dafdddcfde8428 2024-05-14T09:24:43.508 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:43 smithi029 systemd[1]: Started Ceph mgr.z for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:24:43.806 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:43 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:43.506+0000 7fd6831ce200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T09:24:43.806 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:43 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:43.699+0000 7fd6831ce200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T09:24:43.806 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:43 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:43.806+0000 7fd6831ce200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T09:24:44.437 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:44 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:44.129+0000 7fd6831ce200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T09:24:44.437 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:44 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:44.218+0000 7fd6831ce200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T09:24:44.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:44 smithi029 ceph-mon[36249]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:44.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:44.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:44.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:44.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T09:24:44.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T09:24:44.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T09:24:44.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:44.455 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:44 smithi002 ceph-mon[31557]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:44.455 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:44.455 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:44.455 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:44.455 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T09:24:44.455 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T09:24:44.455 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T09:24:44.455 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:44.551 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:44 smithi026 ceph-mon[36056]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:44.551 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:44.551 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:44.551 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:44.551 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T09:24:44.551 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T09:24:44.551 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T09:24:44.551 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:45.036 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-14T09:24:45.037 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-14T09:24:45.037 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2024-05-14T09:24:45.037 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2024-05-14T09:24:45.037 INFO:teuthology.orchestra.run.smithi002.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00851379 s, 1.2 GB/s 2024-05-14T09:24:45.037 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2024-05-14T09:24:45.358 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:45 smithi026 ceph-mon[36056]: Deploying daemon mgr.y on smithi026 2024-05-14T09:24:45.394 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:45 smithi002 ceph-mon[31557]: Deploying daemon mgr.y on smithi026 2024-05-14T09:24:45.490 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:45 smithi029 ceph-mon[36249]: Deploying daemon mgr.y on smithi026 2024-05-14T09:24:45.685 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi002:vg_nvme/lv_4 2024-05-14T09:24:45.701 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:45 smithi026 systemd[1]: Starting Ceph mgr.y for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T09:24:45.842 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:45 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:45.489+0000 7fd6831ce200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T09:24:45.842 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:45 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:45.666+0000 7fd6831ce200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T09:24:45.842 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:45 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:45.841+0000 7fd6831ce200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T09:24:45.933 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:24:46.027 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:45 smithi026 podman[37423]: 2024-05-14 09:24:45.659266883 +0000 UTC m=+0.017779148 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T09:24:46.027 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:45 smithi026 podman[37423]: 2024-05-14 09:24:45.75689108 +0000 UTC m=+0.115403329 container create 3a307e8b7787fbb512be24aade2bd243edefb6965d0847a2a4ae09903bf0791f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T09:24:46.027 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:45 smithi026 podman[37423]: 2024-05-14 09:24:45.934063468 +0000 UTC m=+0.292575721 container init 3a307e8b7787fbb512be24aade2bd243edefb6965d0847a2a4ae09903bf0791f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-14T09:24:46.028 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:45 smithi026 podman[37423]: 2024-05-14 09:24:45.937112844 +0000 UTC m=+0.295625093 container start 3a307e8b7787fbb512be24aade2bd243edefb6965d0847a2a4ae09903bf0791f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T09:24:46.028 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:45 smithi026 ceph-mgr[37442]: -- 172.21.15.26:0/3715711044 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5594f2f665a0 con 0x5594f7cf0480 2024-05-14T09:24:46.028 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:46 smithi026 bash[37423]: 3a307e8b7787fbb512be24aade2bd243edefb6965d0847a2a4ae09903bf0791f 2024-05-14T09:24:46.296 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:46 smithi026 ceph-mon[36056]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:46.297 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:46.297 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:46.297 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:46.297 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:46.297 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:46 smithi026 systemd[1]: Started Ceph mgr.y for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:24:46.297 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:46 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:46.103+0000 7fbc746f3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T09:24:46.297 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:46 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:46.295+0000 7fbc746f3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T09:24:46.331 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:46 smithi002 ceph-mon[31557]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:46.331 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:46.331 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:46.331 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:46.331 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:46.437 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:46 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:46.140+0000 7fd6831ce200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T09:24:46.437 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:46 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:46.236+0000 7fd6831ce200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T09:24:46.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:46 smithi029 ceph-mon[36249]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:46.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:46.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:46.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:46.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:46.701 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:46 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:46.404+0000 7fbc746f3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T09:24:46.901 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:46 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:46.448+0000 7fd6831ce200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T09:24:46.902 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:46 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:46.654+0000 7fd6831ce200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T09:24:47.157 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:46 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:46.737+0000 7fbc746f3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T09:24:47.157 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:46 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:46.832+0000 7fbc746f3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T09:24:47.350 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:47.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:47.437 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:47 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:47.338+0000 7fd6831ce200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T09:24:47.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:47.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.579+0000 7fe8110c8700 1 -- 172.21.15.2:0/682424370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c104380 msgr2=0x7fe80c104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:47.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.579+0000 7fe8110c8700 1 --2- 172.21.15.2:0/682424370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c104380 0x7fe80c104790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe800009990 tx=0x7fe800009ca0 comp rx=0 tx=0).stop 2024-05-14T09:24:47.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.579+0000 7fe8110c8700 1 -- 172.21.15.2:0/682424370 shutdown_connections 2024-05-14T09:24:47.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.579+0000 7fe8110c8700 1 --2- 172.21.15.2:0/682424370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe80c103250 0x7fe80c103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:47.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.579+0000 7fe8110c8700 1 --2- 172.21.15.2:0/682424370 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe80c104d20 0x7fe80c10b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:47.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.579+0000 7fe8110c8700 1 --2- 172.21.15.2:0/682424370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c104380 0x7fe80c104790 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:47.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.579+0000 7fe8110c8700 1 -- 172.21.15.2:0/682424370 >> 172.21.15.2:0/682424370 conn(0x7fe80c0fe890 msgr2=0x7fe80c100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:24:47.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.579+0000 7fe8110c8700 1 -- 172.21.15.2:0/682424370 shutdown_connections 2024-05-14T09:24:47.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.579+0000 7fe8110c8700 1 -- 172.21.15.2:0/682424370 wait complete. 2024-05-14T09:24:47.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.580+0000 7fe8110c8700 1 Processor -- start 2024-05-14T09:24:47.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.580+0000 7fe8110c8700 1 -- start start 2024-05-14T09:24:47.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.580+0000 7fe8110c8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe80c103250 0x7fe80c19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:47.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.580+0000 7fe8110c8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe80c104380 0x7fe80c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:47.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.580+0000 7fe8110c8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c104d20 0x7fe80c1a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:47.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.580+0000 7fe8110c8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c10ecb0 con 0x7fe80c104d20 2024-05-14T09:24:47.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.580+0000 7fe8110c8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c10eb30 con 0x7fe80c103250 2024-05-14T09:24:47.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.580+0000 7fe8110c8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c10ee30 con 0x7fe80c104380 2024-05-14T09:24:47.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.580+0000 7fe80b59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c104d20 0x7fe80c1a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:47.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.580+0000 7fe80a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe80c104380 0x7fe80c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:47.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.580+0000 7fe80ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe80c103250 0x7fe80c19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:47.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.580+0000 7fe80b59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c104d20 0x7fe80c1a1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47470/0 (socket says 172.21.15.2:47470) 2024-05-14T09:24:47.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.580+0000 7fe80ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe80c103250 0x7fe80c19a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:50260/0 (socket says 172.21.15.2:50260) 2024-05-14T09:24:47.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.580+0000 7fe80a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe80c104380 0x7fe80c19ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:53932/0 (socket says 172.21.15.2:53932) 2024-05-14T09:24:47.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.580+0000 7fe80b59e700 1 -- 172.21.15.2:0/2753300680 learned_addr learned my addr 172.21.15.2:0/2753300680 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:24:47.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.581+0000 7fe80b59e700 1 -- 172.21.15.2:0/2753300680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe80c104380 msgr2=0x7fe80c19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:47.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.581+0000 7fe80b59e700 1 --2- 172.21.15.2:0/2753300680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe80c104380 0x7fe80c19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:47.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.581+0000 7fe80b59e700 1 -- 172.21.15.2:0/2753300680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe80c103250 msgr2=0x7fe80c19a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:24:47.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.581+0000 7fe80b59e700 1 --2- 172.21.15.2:0/2753300680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe80c103250 0x7fe80c19a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:24:47.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.581+0000 7fe80b59e700 1 -- 172.21.15.2:0/2753300680 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c1a2420 con 0x7fe80c104d20 2024-05-14T09:24:47.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.581+0000 7fe80a59c700 1 --2- 172.21.15.2:0/2753300680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe80c104380 0x7fe80c19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:24:47.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.581+0000 7fe80b59e700 1 --2- 172.21.15.2:0/2753300680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c104d20 0x7fe80c1a1da0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe80400ef30 tx=0x7fe80400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:47.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.581+0000 7fe7f3fff700 1 -- 172.21.15.2:0/2753300680 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8040092e0 con 0x7fe80c104d20 2024-05-14T09:24:47.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.581+0000 7fe7f3fff700 1 -- 172.21.15.2:0/2753300680 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe804004bb0 con 0x7fe80c104d20 2024-05-14T09:24:47.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.581+0000 7fe8110c8700 1 -- 172.21.15.2:0/2753300680 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80c1a26b0 con 0x7fe80c104d20 2024-05-14T09:24:47.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.581+0000 7fe7f3fff700 1 -- 172.21.15.2:0/2753300680 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804004020 con 0x7fe80c104d20 2024-05-14T09:24:47.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.581+0000 7fe8110c8700 1 -- 172.21.15.2:0/2753300680 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80c1a2ba0 con 0x7fe80c104d20 2024-05-14T09:24:47.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.582+0000 7fe7f3fff700 1 -- 172.21.15.2:0/2753300680 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fe80401b030 con 0x7fe80c104d20 2024-05-14T09:24:47.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.582+0000 7fe8110c8700 1 -- 172.21.15.2:0/2753300680 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80c0512c0 con 0x7fe80c104d20 2024-05-14T09:24:47.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.582+0000 7fe7f3fff700 1 --2- 172.21.15.2:0/2753300680 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe7f403e750 0x7fe7f4040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:24:47.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.582+0000 7fe7f3fff700 1 -- 172.21.15.2:0/2753300680 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe804052420 con 0x7fe80c104d20 2024-05-14T09:24:47.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.583+0000 7fe80ad9d700 1 --2- 172.21.15.2:0/2753300680 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe7f403e750 0x7fe7f4040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:24:47.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.583+0000 7fe80ad9d700 1 --2- 172.21.15.2:0/2753300680 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe7f403e750 0x7fe7f4040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe80c104280 tx=0x7fe7fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:24:47.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.588+0000 7fe7f3fff700 1 -- 172.21.15.2:0/2753300680 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe804022020 con 0x7fe80c104d20 2024-05-14T09:24:47.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:47.818+0000 7fe8110c8700 1 -- 172.21.15.2:0/2753300680 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe80c1008a0 con 0x7fe7f403e750 2024-05-14T09:24:47.937 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:47 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:47.570+0000 7fd6831ce200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T09:24:48.437 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:48 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:48.023+0000 7fd6831ce200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T09:24:48.437 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:48 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:48.123+0000 7fd6831ce200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T09:24:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:48 smithi029 ceph-mon[36249]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:48.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:48.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:48.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:48.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:48.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:24:48.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:24:48.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:48.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T09:24:48.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T09:24:48.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:48.451 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:48 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:48.139+0000 7fbc746f3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T09:24:48.451 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:48 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:48.315+0000 7fbc746f3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T09:24:48.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:48 smithi026 ceph-mon[36056]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:48.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:48.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:48.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:48.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:48.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:48.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:48.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:48.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:24:48.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:24:48.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:48.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T09:24:48.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T09:24:48.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:48.576 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:48 smithi002 ceph-mon[31557]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:48.576 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:48.576 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:48.576 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:48.577 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:48.577 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:48.577 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:48.577 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:48.577 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:24:48.577 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:24:48.577 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:48.577 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T09:24:48.577 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T09:24:48.577 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:48.789 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:48 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:48.492+0000 7fbc746f3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T09:24:48.789 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:48 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:48.789+0000 7fbc746f3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T09:24:48.877 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:48 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:48.505+0000 7fd6831ce200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T09:24:48.877 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:48 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:48.876+0000 7fd6831ce200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T09:24:49.098 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:48 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:48.885+0000 7fbc746f3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T09:24:49.187 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:48 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:48.972+0000 7fd6831ce200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T09:24:49.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:49 smithi002 ceph-mon[31557]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:24:49.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:49 smithi002 ceph-mon[31557]: Reconfiguring mgr.x (unknown last config time)... 2024-05-14T09:24:49.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:49 smithi002 ceph-mon[31557]: Reconfiguring daemon mgr.x on smithi002 2024-05-14T09:24:49.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:49 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:49.451 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:49 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:49.098+0000 7fbc746f3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T09:24:49.451 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:49 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:49.297+0000 7fbc746f3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T09:24:49.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:49 smithi026 ceph-mon[36056]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:24:49.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:49 smithi026 ceph-mon[36056]: Reconfiguring mgr.x (unknown last config time)... 2024-05-14T09:24:49.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:49 smithi026 ceph-mon[36056]: Reconfiguring daemon mgr.x on smithi002 2024-05-14T09:24:49.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:49 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:49.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:49 smithi029 ceph-mon[36249]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:24:49.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:49 smithi029 ceph-mon[36249]: Reconfiguring mgr.x (unknown last config time)... 2024-05-14T09:24:49.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:49 smithi029 ceph-mon[36249]: Reconfiguring daemon mgr.x on smithi002 2024-05-14T09:24:49.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:49 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:49.687 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:49 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:49.228+0000 7fd6831ce200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T09:24:49.687 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:49 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:49.368+0000 7fd6831ce200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T09:24:49.687 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:49 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:49.466+0000 7fd6831ce200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T09:24:50.187 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:49 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:49.806+0000 7fd6831ce200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T09:24:50.187 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:49 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:49.901+0000 7fd6831ce200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T09:24:50.201 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:49 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:49.950+0000 7fbc746f3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T09:24:50.201 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:50 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:50.181+0000 7fbc746f3200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T09:24:50.540 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:50 smithi029 ceph-mon[36249]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:50.609 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:50 smithi002 ceph-mon[31557]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:50.632 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:50 smithi026 ceph-mon[36056]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:50.937 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:50 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:50.540+0000 7fd6831ce200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T09:24:50.937 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 09:24:50 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z[38088]: 2024-05-14T09:24:50.766+0000 7fd6831ce200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T09:24:50.951 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:50 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:50.632+0000 7fbc746f3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T09:24:50.951 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:50 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:50.734+0000 7fbc746f3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T09:24:51.447 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:51 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:51.135+0000 7fbc746f3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T09:24:51.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:51.508+0000 7fe7f3fff700 1 -- 172.21.15.2:0/2753300680 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 14) ==== 99936+0+0 (secure 0 0 0) 0x7fe804012050 con 0x7fe80c104d20 2024-05-14T09:24:51.701 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:51 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:51.510+0000 7fbc746f3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T09:24:51.702 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:51 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:51.606+0000 7fbc746f3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T09:24:51.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:51.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:51.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:51.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:51.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:51.703 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:51.703 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:51 smithi026 ceph-mon[36056]: Standby manager daemon z started 2024-05-14T09:24:51.703 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:51 smithi026 ceph-mon[36056]: from='mgr.? 172.21.15.29:0/33153433' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T09:24:51.703 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:51 smithi026 ceph-mon[36056]: from='mgr.? 172.21.15.29:0/33153433' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T09:24:51.703 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:51 smithi026 ceph-mon[36056]: from='mgr.? 172.21.15.29:0/33153433' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T09:24:51.703 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:51 smithi026 ceph-mon[36056]: from='mgr.? 172.21.15.29:0/33153433' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T09:24:51.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:51.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:51.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:51.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:51.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:51.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:51.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:51 smithi002 ceph-mon[31557]: Standby manager daemon z started 2024-05-14T09:24:51.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:51 smithi002 ceph-mon[31557]: from='mgr.? 172.21.15.29:0/33153433' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T09:24:51.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:51 smithi002 ceph-mon[31557]: from='mgr.? 172.21.15.29:0/33153433' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T09:24:51.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:51 smithi002 ceph-mon[31557]: from='mgr.? 172.21.15.29:0/33153433' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T09:24:51.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:51 smithi002 ceph-mon[31557]: from='mgr.? 172.21.15.29:0/33153433' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T09:24:51.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:51.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:51.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:24:51.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:51.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:24:51.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:24:51.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:51 smithi029 ceph-mon[36249]: Standby manager daemon z started 2024-05-14T09:24:51.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:51 smithi029 ceph-mon[36249]: from='mgr.? 172.21.15.29:0/33153433' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T09:24:51.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:51 smithi029 ceph-mon[36249]: from='mgr.? 172.21.15.29:0/33153433' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T09:24:51.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:51 smithi029 ceph-mon[36249]: from='mgr.? 172.21.15.29:0/33153433' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T09:24:51.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:51 smithi029 ceph-mon[36249]: from='mgr.? 172.21.15.29:0/33153433' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T09:24:52.201 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:51 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:51.863+0000 7fbc746f3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T09:24:52.201 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:52 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:52.005+0000 7fbc746f3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T09:24:52.202 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:52 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:52.106+0000 7fbc746f3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T09:24:52.701 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:52 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:52.449+0000 7fbc746f3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T09:24:52.701 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:52 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:52.547+0000 7fbc746f3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T09:24:52.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:52 smithi026 ceph-mon[36056]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:52.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:52 smithi026 ceph-mon[36056]: mgrmap e14: x(active, since 82s), standbys: z 2024-05-14T09:24:52.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:52 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T09:24:52.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:52 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3595035773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10d34289-414e-4b1b-bc7c-ba1dd24c7297"}]: dispatch 2024-05-14T09:24:52.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:52 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3595035773' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10d34289-414e-4b1b-bc7c-ba1dd24c7297"}]': finished 2024-05-14T09:24:52.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:52 smithi026 ceph-mon[36056]: osdmap e5: 1 total, 0 up, 1 in 2024-05-14T09:24:52.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:52 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T09:24:52.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:52 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3100697498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:24:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:52 smithi002 ceph-mon[31557]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:52 smithi002 ceph-mon[31557]: mgrmap e14: x(active, since 82s), standbys: z 2024-05-14T09:24:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T09:24:52.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:52 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3595035773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10d34289-414e-4b1b-bc7c-ba1dd24c7297"}]: dispatch 2024-05-14T09:24:52.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:52 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3595035773' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10d34289-414e-4b1b-bc7c-ba1dd24c7297"}]': finished 2024-05-14T09:24:52.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:52 smithi002 ceph-mon[31557]: osdmap e5: 1 total, 0 up, 1 in 2024-05-14T09:24:52.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T09:24:52.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:52 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3100697498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:24:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:52 smithi029 ceph-mon[36249]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:52 smithi029 ceph-mon[36249]: mgrmap e14: x(active, since 82s), standbys: z 2024-05-14T09:24:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:52 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T09:24:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:52 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3595035773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10d34289-414e-4b1b-bc7c-ba1dd24c7297"}]: dispatch 2024-05-14T09:24:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:52 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3595035773' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10d34289-414e-4b1b-bc7c-ba1dd24c7297"}]': finished 2024-05-14T09:24:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:52 smithi029 ceph-mon[36249]: osdmap e5: 1 total, 0 up, 1 in 2024-05-14T09:24:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:52 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T09:24:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:52 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3100697498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:24:53.451 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:53 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:53.181+0000 7fbc746f3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T09:24:53.451 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 09:24:53 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y[37438]: 2024-05-14T09:24:53.409+0000 7fbc746f3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T09:24:53.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:24:53.459+0000 7fe7f3fff700 1 -- 172.21.15.2:0/2753300680 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7fe804007880 con 0x7fe80c104d20 2024-05-14T09:24:53.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:53 smithi002 ceph-mon[31557]: Standby manager daemon y started 2024-05-14T09:24:53.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:53 smithi002 ceph-mon[31557]: from='mgr.? 172.21.15.26:0/1112395294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T09:24:53.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:53 smithi002 ceph-mon[31557]: from='mgr.? 172.21.15.26:0/1112395294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T09:24:53.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:53 smithi002 ceph-mon[31557]: from='mgr.? 172.21.15.26:0/1112395294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T09:24:53.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:53 smithi002 ceph-mon[31557]: from='mgr.? 172.21.15.26:0/1112395294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T09:24:53.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:53 smithi029 ceph-mon[36249]: Standby manager daemon y started 2024-05-14T09:24:53.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:53 smithi029 ceph-mon[36249]: from='mgr.? 172.21.15.26:0/1112395294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T09:24:53.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:53 smithi029 ceph-mon[36249]: from='mgr.? 172.21.15.26:0/1112395294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T09:24:53.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:53 smithi029 ceph-mon[36249]: from='mgr.? 172.21.15.26:0/1112395294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T09:24:53.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:53 smithi029 ceph-mon[36249]: from='mgr.? 172.21.15.26:0/1112395294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T09:24:53.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:53 smithi026 ceph-mon[36056]: Standby manager daemon y started 2024-05-14T09:24:53.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:53 smithi026 ceph-mon[36056]: from='mgr.? 172.21.15.26:0/1112395294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T09:24:53.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:53 smithi026 ceph-mon[36056]: from='mgr.? 172.21.15.26:0/1112395294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T09:24:53.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:53 smithi026 ceph-mon[36056]: from='mgr.? 172.21.15.26:0/1112395294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T09:24:53.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:53 smithi026 ceph-mon[36056]: from='mgr.? 172.21.15.26:0/1112395294' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T09:24:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:54 smithi002 ceph-mon[31557]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:54.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:54 smithi002 ceph-mon[31557]: mgrmap e15: x(active, since 84s), standbys: z, y 2024-05-14T09:24:54.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:54 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T09:24:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:54 smithi029 ceph-mon[36249]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:54 smithi029 ceph-mon[36249]: mgrmap e15: x(active, since 84s), standbys: z, y 2024-05-14T09:24:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:54 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T09:24:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:54 smithi026 ceph-mon[36056]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:54 smithi026 ceph-mon[36056]: mgrmap e15: x(active, since 84s), standbys: z, y 2024-05-14T09:24:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:54 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T09:24:56.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:56 smithi002 ceph-mon[31557]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:56 smithi029 ceph-mon[36249]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:56 smithi026 ceph-mon[36056]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:58.669 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:58 smithi002 ceph-mon[31557]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:58 smithi029 ceph-mon[36249]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:58 smithi026 ceph-mon[36056]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:24:59.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-14T09:24:59.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:24:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:59.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-14T09:24:59.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:24:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:24:59.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-14T09:24:59.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:24:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:00.565 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:00 smithi002 ceph-mon[31557]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:00.565 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:00 smithi002 ceph-mon[31557]: Deploying daemon osd.0 on smithi002 2024-05-14T09:25:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:00 smithi029 ceph-mon[36249]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:00 smithi029 ceph-mon[36249]: Deploying daemon osd.0 on smithi002 2024-05-14T09:25:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:00 smithi026 ceph-mon[36056]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:00 smithi026 ceph-mon[36056]: Deploying daemon osd.0 on smithi002 2024-05-14T09:25:02.655 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:02 smithi002 ceph-mon[31557]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:02 smithi029 ceph-mon[36249]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:02 smithi026 ceph-mon[36056]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:04.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:04 smithi002 ceph-mon[31557]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:04.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:04 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:04.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:04 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:04.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:04 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:04 smithi029 ceph-mon[36249]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:04 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:04 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:04 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:04 smithi026 ceph-mon[36056]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:04 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:04 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:04 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:05.813 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:05 smithi002 ceph-mon[31557]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:05.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:05 smithi029 ceph-mon[36249]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:05.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:05 smithi026 ceph-mon[36056]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:08.299 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:08 smithi002 ceph-mon[31557]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:08.299 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:08 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:08.299 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:08 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:08.299 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:08 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:08.299 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:08 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:25:08.299 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:08 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:08.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:25:08.378+0000 7fe7f3fff700 1 -- 172.21.15.2:0/2753300680 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe80c1008a0 con 0x7fe7f403e750 2024-05-14T09:25:08.378 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 0 on host 'smithi002' 2024-05-14T09:25:08.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:25:08.381+0000 7fe8110c8700 1 -- 172.21.15.2:0/2753300680 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe7f403e750 msgr2=0x7fe7f4040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:25:08.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:25:08.381+0000 7fe8110c8700 1 --2- 172.21.15.2:0/2753300680 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe7f403e750 0x7fe7f4040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe80c104280 tx=0x7fe7fc009260 comp rx=0 tx=0).stop 2024-05-14T09:25:08.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:25:08.381+0000 7fe8110c8700 1 -- 172.21.15.2:0/2753300680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c104d20 msgr2=0x7fe80c1a1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:25:08.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:25:08.381+0000 7fe8110c8700 1 --2- 172.21.15.2:0/2753300680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c104d20 0x7fe80c1a1da0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe80400ef30 tx=0x7fe80400c3a0 comp rx=0 tx=0).stop 2024-05-14T09:25:08.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:25:08.381+0000 7fe8110c8700 1 -- 172.21.15.2:0/2753300680 shutdown_connections 2024-05-14T09:25:08.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:25:08.381+0000 7fe8110c8700 1 --2- 172.21.15.2:0/2753300680 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe7f403e750 0x7fe7f4040c60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:08.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:25:08.381+0000 7fe8110c8700 1 --2- 172.21.15.2:0/2753300680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe80c103250 0x7fe80c19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:08.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:25:08.381+0000 7fe8110c8700 1 --2- 172.21.15.2:0/2753300680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c104d20 0x7fe80c1a1da0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:08.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:25:08.381+0000 7fe8110c8700 1 --2- 172.21.15.2:0/2753300680 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe80c104380 0x7fe80c19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:08.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:25:08.381+0000 7fe8110c8700 1 -- 172.21.15.2:0/2753300680 >> 172.21.15.2:0/2753300680 conn(0x7fe80c0fe890 msgr2=0x7fe80c100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:25:08.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:25:08.381+0000 7fe8110c8700 1 -- 172.21.15.2:0/2753300680 shutdown_connections 2024-05-14T09:25:08.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:25:08.381+0000 7fe8110c8700 1 -- 172.21.15.2:0/2753300680 wait complete. 2024-05-14T09:25:08.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:08 smithi029 ceph-mon[36249]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:08.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:08 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:08.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:08 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:08.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:08 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:08.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:08 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:25:08.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:08 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:08.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:08 smithi026 ceph-mon[36056]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:08.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:08 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:08.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:08 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:08.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:08 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:08.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:08 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:25:08.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:08 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:09.000 DEBUG:teuthology.orchestra.run.smithi002:osd.0> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.0.service 2024-05-14T09:25:09.003 INFO:tasks.cephadm:Deploying osd.1 on smithi026 with /dev/vg_nvme/lv_4... 2024-05-14T09:25:09.003 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-14T09:25:09.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:09 smithi002 ceph-mon[31557]: from='osd.0 [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-14T09:25:09.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:09.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:09.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:09.267 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T09:25:09.310 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:09 smithi026 ceph-mon[36056]: from='osd.0 [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-14T09:25:09.311 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:09 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:09.311 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:09 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:09.311 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:09 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:09.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:09 smithi029 ceph-mon[36249]: from='osd.0 [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-14T09:25:09.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:09 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:09.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:09 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:09.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:09 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:10.283 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:10 smithi026 ceph-mon[36056]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:10.283 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:10 smithi026 ceph-mon[36056]: from='osd.0 [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-14T09:25:10.283 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:10 smithi026 ceph-mon[36056]: osdmap e6: 1 total, 0 up, 1 in 2024-05-14T09:25:10.283 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:10 smithi026 ceph-mon[36056]: from='osd.0 [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T09:25:10.283 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:10 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T09:25:10.293 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:10 smithi002 ceph-mon[31557]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:10.293 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:10 smithi002 ceph-mon[31557]: from='osd.0 [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-14T09:25:10.293 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:10 smithi002 ceph-mon[31557]: osdmap e6: 1 total, 0 up, 1 in 2024-05-14T09:25:10.293 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:10 smithi002 ceph-mon[31557]: from='osd.0 [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T09:25:10.293 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:10 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T09:25:10.293 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 09:25:10 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-0[43457]: 2024-05-14T09:25:10.036+0000 7f79bc1c3700 -1 osd.0 0 waiting for initial osdmap 2024-05-14T09:25:10.293 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 09:25:10 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-0[43457]: 2024-05-14T09:25:10.039+0000 7f79b7f78700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T09:25:10.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:10 smithi029 ceph-mon[36249]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:10.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:10 smithi029 ceph-mon[36249]: from='osd.0 [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-14T09:25:10.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:10 smithi029 ceph-mon[36249]: osdmap e6: 1 total, 0 up, 1 in 2024-05-14T09:25:10.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:10 smithi029 ceph-mon[36249]: from='osd.0 [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T09:25:10.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:10 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T09:25:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-14T09:25:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-14T09:25:11.100 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-05-14T09:25:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-05-14T09:25:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00841288 s, 1.2 GB/s 2024-05-14T09:25:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-05-14T09:25:11.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:11 smithi002 ceph-mon[31557]: from='osd.0 [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T09:25:11.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:11 smithi002 ceph-mon[31557]: osdmap e7: 1 total, 0 up, 1 in 2024-05-14T09:25:11.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:11 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T09:25:11.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:11 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T09:25:11.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:11 smithi029 ceph-mon[36249]: from='osd.0 [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T09:25:11.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:11 smithi029 ceph-mon[36249]: osdmap e7: 1 total, 0 up, 1 in 2024-05-14T09:25:11.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:11 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T09:25:11.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:11 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T09:25:11.445 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:11 smithi026 ceph-mon[36056]: from='osd.0 [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T09:25:11.445 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:11 smithi026 ceph-mon[36056]: osdmap e7: 1 total, 0 up, 1 in 2024-05-14T09:25:11.445 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:11 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T09:25:11.445 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:11 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T09:25:11.735 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_4 2024-05-14T09:25:11.977 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T09:25:12.359 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:12 smithi026 ceph-mon[36056]: purged_snaps scrub starts 2024-05-14T09:25:12.359 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:12 smithi026 ceph-mon[36056]: purged_snaps scrub ok 2024-05-14T09:25:12.359 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:12 smithi026 ceph-mon[36056]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:12.359 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:12 smithi026 ceph-mon[36056]: osd.0 [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634] boot 2024-05-14T09:25:12.359 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:12 smithi026 ceph-mon[36056]: osdmap e8: 1 total, 1 up, 1 in 2024-05-14T09:25:12.359 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:12 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T09:25:12.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:12 smithi002 ceph-mon[31557]: purged_snaps scrub starts 2024-05-14T09:25:12.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:12 smithi002 ceph-mon[31557]: purged_snaps scrub ok 2024-05-14T09:25:12.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:12 smithi002 ceph-mon[31557]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:12.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:12 smithi002 ceph-mon[31557]: osd.0 [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634] boot 2024-05-14T09:25:12.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:12 smithi002 ceph-mon[31557]: osdmap e8: 1 total, 1 up, 1 in 2024-05-14T09:25:12.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:12 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T09:25:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:12 smithi029 ceph-mon[36249]: purged_snaps scrub starts 2024-05-14T09:25:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:12 smithi029 ceph-mon[36249]: purged_snaps scrub ok 2024-05-14T09:25:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:12 smithi029 ceph-mon[36249]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T09:25:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:12 smithi029 ceph-mon[36249]: osd.0 [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634] boot 2024-05-14T09:25:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:12 smithi029 ceph-mon[36249]: osdmap e8: 1 total, 1 up, 1 in 2024-05-14T09:25:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:12 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T09:25:13.140 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:13 smithi026 ceph-mon[36056]: osdmap e9: 1 total, 1 up, 1 in 2024-05-14T09:25:13.140 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:13 smithi026 ceph-mon[36056]: Detected new or changed devices on smithi002 2024-05-14T09:25:13.140 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:13 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:13.140 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:13 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:13.140 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:13 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:25:13.140 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:13 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi002 to 17093M 2024-05-14T09:25:13.140 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:13 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:13.140 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:13 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:13.140 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:13 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:25:13.140 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:13 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.365+0000 7f903b219700 1 -- 172.21.15.26:0/845854912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90341059b0 msgr2=0x7f9034105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:25:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.365+0000 7f903b219700 1 --2- 172.21.15.26:0/845854912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90341059b0 0x7f9034105dc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9028009990 tx=0x7f9028009ca0 comp rx=0 tx=0).stop 2024-05-14T09:25:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.365+0000 7f903b219700 1 -- 172.21.15.26:0/845854912 shutdown_connections 2024-05-14T09:25:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.365+0000 7f903b219700 1 --2- 172.21.15.26:0/845854912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90341059b0 0x7f9034105dc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.365+0000 7f903b219700 1 --2- 172.21.15.26:0/845854912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90341049e0 0x7f903410c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.365+0000 7f903b219700 1 --2- 172.21.15.26:0/845854912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9034103f60 0x7f9034104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.365+0000 7f903b219700 1 -- 172.21.15.26:0/845854912 >> 172.21.15.26:0/845854912 conn(0x7f90340ff9d0 msgr2=0x7f9034101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:25:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.365+0000 7f903b219700 1 -- 172.21.15.26:0/845854912 shutdown_connections 2024-05-14T09:25:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.365+0000 7f903b219700 1 -- 172.21.15.26:0/845854912 wait complete. 2024-05-14T09:25:13.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.366+0000 7f903b219700 1 Processor -- start 2024-05-14T09:25:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.366+0000 7f903b219700 1 -- start start 2024-05-14T09:25:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.366+0000 7f903b219700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9034103f60 0x7f903419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:25:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.366+0000 7f903b219700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90341049e0 0x7f903419af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:25:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.366+0000 7f903b219700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90341059b0 0x7f90341a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:25:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.366+0000 7f903b219700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f903410fed0 con 0x7f90341049e0 2024-05-14T09:25:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.366+0000 7f903b219700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f903410fd50 con 0x7f9034103f60 2024-05-14T09:25:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.366+0000 7f903b219700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9034110050 con 0x7f90341059b0 2024-05-14T09:25:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.366+0000 7f9038fb5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9034103f60 0x7f903419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:25:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.366+0000 7f9038fb5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9034103f60 0x7f903419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45452/0 (socket says 172.21.15.26:45452) 2024-05-14T09:25:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.366+0000 7f9038fb5700 1 -- 172.21.15.26:0/115786561 learned_addr learned my addr 172.21.15.26:0/115786561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-14T09:25:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.366+0000 7f90397b6700 1 --2- 172.21.15.26:0/115786561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90341059b0 0x7f90341a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:25:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.366+0000 7f9033fff700 1 --2- 172.21.15.26:0/115786561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90341049e0 0x7f903419af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:25:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.366+0000 7f9038fb5700 1 -- 172.21.15.26:0/115786561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90341059b0 msgr2=0x7f90341a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:25:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.366+0000 7f9038fb5700 1 --2- 172.21.15.26:0/115786561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90341059b0 0x7f90341a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.367+0000 7f9038fb5700 1 -- 172.21.15.26:0/115786561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90341049e0 msgr2=0x7f903419af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:25:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.367+0000 7f9038fb5700 1 --2- 172.21.15.26:0/115786561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90341049e0 0x7f903419af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.367+0000 7f9038fb5700 1 -- 172.21.15.26:0/115786561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90341a24e0 con 0x7f9034103f60 2024-05-14T09:25:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.367+0000 7f9038fb5700 1 --2- 172.21.15.26:0/115786561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9034103f60 0x7f903419aa10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f90280057c0 tx=0x7f90280057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:25:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.367+0000 7f9031ffb700 1 -- 172.21.15.26:0/115786561 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028027070 con 0x7f9034103f60 2024-05-14T09:25:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.367+0000 7f9031ffb700 1 -- 172.21.15.26:0/115786561 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9028004000 con 0x7f9034103f60 2024-05-14T09:25:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.367+0000 7f903b219700 1 -- 172.21.15.26:0/115786561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90341a2770 con 0x7f9034103f60 2024-05-14T09:25:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.367+0000 7f9031ffb700 1 -- 172.21.15.26:0/115786561 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028022be0 con 0x7f9034103f60 2024-05-14T09:25:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.367+0000 7f903b219700 1 -- 172.21.15.26:0/115786561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90341a2c20 con 0x7f9034103f60 2024-05-14T09:25:13.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.368+0000 7f903b219700 1 -- 172.21.15.26:0/115786561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90340512c0 con 0x7f9034103f60 2024-05-14T09:25:13.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.370+0000 7f9031ffb700 1 -- 172.21.15.26:0/115786561 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f902802a070 con 0x7f9034103f60 2024-05-14T09:25:13.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.370+0000 7f9031ffb700 1 --2- 172.21.15.26:0/115786561 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f90240b2870 0x7f90240b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:25:13.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.370+0000 7f9031ffb700 1 -- 172.21.15.26:0/115786561 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f90280e1680 con 0x7f9034103f60 2024-05-14T09:25:13.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.370+0000 7f9033fff700 1 --2- 172.21.15.26:0/115786561 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f90240b2870 0x7f90240b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:25:13.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.371+0000 7f9033fff700 1 --2- 172.21.15.26:0/115786561 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f90240b2870 0x7f90240b4d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f903419bac0 tx=0x7f902000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:25:13.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.375+0000 7f9031ffb700 1 -- 172.21.15.26:0/115786561 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9028013070 con 0x7f9034103f60 2024-05-14T09:25:13.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:13 smithi002 ceph-mon[31557]: osdmap e9: 1 total, 1 up, 1 in 2024-05-14T09:25:13.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:13 smithi002 ceph-mon[31557]: Detected new or changed devices on smithi002 2024-05-14T09:25:13.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:13 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:13.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:13 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:13.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:13 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:25:13.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:13 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi002 to 17093M 2024-05-14T09:25:13.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:13 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:13.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:13 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:13.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:13 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:25:13.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:13 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:13.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:13 smithi029 ceph-mon[36249]: osdmap e9: 1 total, 1 up, 1 in 2024-05-14T09:25:13.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:13 smithi029 ceph-mon[36249]: Detected new or changed devices on smithi002 2024-05-14T09:25:13.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:13 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:13.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:13 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:13.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:13 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:25:13.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:13 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi002 to 17093M 2024-05-14T09:25:13.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:13 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:13.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:13 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:13.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:13 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:25:13.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:13 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:13.601+0000 7f903b219700 1 -- 172.21.15.26:0/115786561 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f903402e9b0 con 0x7f90240b2870 2024-05-14T09:25:14.392 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:14 smithi026 ceph-mon[36056]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:14.392 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:14 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:25:14.393 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:14 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:25:14.393 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:14 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:14.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:14 smithi002 ceph-mon[31557]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:14.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:14 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:25:14.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:14 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:25:14.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:14 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:14.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:14 smithi029 ceph-mon[36249]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:14.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:25:14.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:25:14.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:15.098 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:15 smithi026 ceph-mon[36056]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:25:15.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:15 smithi002 ceph-mon[31557]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:25:15.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:15 smithi029 ceph-mon[36249]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:25:16.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:16 smithi002 ceph-mon[31557]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:16.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:16 smithi029 ceph-mon[36249]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:16.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:16 smithi026 ceph-mon[36056]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:17 smithi002 ceph-mon[31557]: from='client.? 172.21.15.26:0/3718214916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc4aefc2-706e-4faf-9042-64b87ad097b5"}]: dispatch 2024-05-14T09:25:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:17 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc4aefc2-706e-4faf-9042-64b87ad097b5"}]: dispatch 2024-05-14T09:25:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:17 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc4aefc2-706e-4faf-9042-64b87ad097b5"}]': finished 2024-05-14T09:25:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:17 smithi002 ceph-mon[31557]: osdmap e10: 2 total, 1 up, 2 in 2024-05-14T09:25:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:17 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:17.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:17 smithi002 ceph-mon[31557]: from='client.? 172.21.15.26:0/1527743558' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:25:17.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:17 smithi029 ceph-mon[36249]: from='client.? 172.21.15.26:0/3718214916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc4aefc2-706e-4faf-9042-64b87ad097b5"}]: dispatch 2024-05-14T09:25:17.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:17 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc4aefc2-706e-4faf-9042-64b87ad097b5"}]: dispatch 2024-05-14T09:25:17.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:17 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc4aefc2-706e-4faf-9042-64b87ad097b5"}]': finished 2024-05-14T09:25:17.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:17 smithi029 ceph-mon[36249]: osdmap e10: 2 total, 1 up, 2 in 2024-05-14T09:25:17.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:17 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:17.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:17 smithi029 ceph-mon[36249]: from='client.? 172.21.15.26:0/1527743558' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:25:17.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:17 smithi026 ceph-mon[36056]: from='client.? 172.21.15.26:0/3718214916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc4aefc2-706e-4faf-9042-64b87ad097b5"}]: dispatch 2024-05-14T09:25:17.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:17 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc4aefc2-706e-4faf-9042-64b87ad097b5"}]: dispatch 2024-05-14T09:25:17.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:17 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc4aefc2-706e-4faf-9042-64b87ad097b5"}]': finished 2024-05-14T09:25:17.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:17 smithi026 ceph-mon[36056]: osdmap e10: 2 total, 1 up, 2 in 2024-05-14T09:25:17.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:17 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:17.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:17 smithi026 ceph-mon[36056]: from='client.? 172.21.15.26:0/1527743558' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:25:18.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:18 smithi002 ceph-mon[31557]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:18.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:18 smithi029 ceph-mon[36249]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:18.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:18 smithi026 ceph-mon[36056]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:20 smithi002 ceph-mon[31557]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:20 smithi029 ceph-mon[36249]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:20 smithi026 ceph-mon[36056]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:22.412 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:22 smithi026 ceph-mon[36056]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:22.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:22 smithi002 ceph-mon[31557]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:22.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:22 smithi029 ceph-mon[36249]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:24.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:24 smithi026 ceph-mon[36056]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:24.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:24 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-14T09:25:24.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:24 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:24.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:24 smithi002 ceph-mon[31557]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:24.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:24 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-14T09:25:24.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:24 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:24.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:24 smithi029 ceph-mon[36249]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:24.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:24 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-14T09:25:24.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:24 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:25.618 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:25 smithi026 ceph-mon[36056]: Deploying daemon osd.1 on smithi026 2024-05-14T09:25:25.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:25 smithi002 ceph-mon[31557]: Deploying daemon osd.1 on smithi026 2024-05-14T09:25:25.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:25 smithi029 ceph-mon[36249]: Deploying daemon osd.1 on smithi026 2024-05-14T09:25:26.557 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:26 smithi026 ceph-mon[36056]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:26.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:26 smithi002 ceph-mon[31557]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:26 smithi029 ceph-mon[36249]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:28.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:28 smithi002 ceph-mon[31557]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:28.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:28 smithi029 ceph-mon[36249]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:28.691 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:28 smithi026 ceph-mon[36056]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:30.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:30 smithi029 ceph-mon[36249]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:30.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:30.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:30.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:30.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:30 smithi026 ceph-mon[36056]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:30.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:30.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:30.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:30.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:30 smithi002 ceph-mon[31557]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:30.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:30.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:30.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:32 smithi029 ceph-mon[36249]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:25:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:32.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:32 smithi026 ceph-mon[36056]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:25:32.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:32.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:32 smithi002 ceph-mon[31557]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:32.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:32.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:32.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:32.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:25:32.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:32.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:32.972+0000 7f9031ffb700 1 -- 172.21.15.26:0/115786561 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f903402e9b0 con 0x7f90240b2870 2024-05-14T09:25:32.973 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 1 on host 'smithi026' 2024-05-14T09:25:32.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:32.975+0000 7f903b219700 1 -- 172.21.15.26:0/115786561 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f90240b2870 msgr2=0x7f90240b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:25:32.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:32.975+0000 7f903b219700 1 --2- 172.21.15.26:0/115786561 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f90240b2870 0x7f90240b4d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f903419bac0 tx=0x7f902000a380 comp rx=0 tx=0).stop 2024-05-14T09:25:32.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:32.975+0000 7f903b219700 1 -- 172.21.15.26:0/115786561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9034103f60 msgr2=0x7f903419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:25:32.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:32.975+0000 7f903b219700 1 --2- 172.21.15.26:0/115786561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9034103f60 0x7f903419aa10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f90280057c0 tx=0x7f90280057f0 comp rx=0 tx=0).stop 2024-05-14T09:25:32.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:32.975+0000 7f903b219700 1 -- 172.21.15.26:0/115786561 shutdown_connections 2024-05-14T09:25:32.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:32.975+0000 7f903b219700 1 --2- 172.21.15.26:0/115786561 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f90240b2870 0x7f90240b4d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:32.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:32.975+0000 7f903b219700 1 --2- 172.21.15.26:0/115786561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9034103f60 0x7f903419aa10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:32.975+0000 7f903b219700 1 --2- 172.21.15.26:0/115786561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f90341059b0 0x7f90341a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:32.975+0000 7f903b219700 1 --2- 172.21.15.26:0/115786561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90341049e0 0x7f903419af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:32.975+0000 7f903b219700 1 -- 172.21.15.26:0/115786561 >> 172.21.15.26:0/115786561 conn(0x7f90340ff9d0 msgr2=0x7f90341086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:25:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:32.975+0000 7f903b219700 1 -- 172.21.15.26:0/115786561 shutdown_connections 2024-05-14T09:25:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:25:32.975+0000 7f903b219700 1 -- 172.21.15.26:0/115786561 wait complete. 2024-05-14T09:25:33.641 DEBUG:teuthology.orchestra.run.smithi026:osd.1> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.1.service 2024-05-14T09:25:33.644 INFO:tasks.cephadm:Deploying osd.2 on smithi029 with /dev/vg_nvme/lv_4... 2024-05-14T09:25:33.644 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-14T09:25:33.897 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T09:25:34.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:33 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:34.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:33 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:34.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:33 smithi026 ceph-mon[36056]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:34.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:33 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:34.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:33 smithi026 ceph-mon[36056]: from='osd.1 [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T09:25:34.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:33 smithi026 ceph-mon[36056]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T09:25:34.228 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:33 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:34.228 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:33 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:34.228 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:33 smithi029 ceph-mon[36249]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:34.228 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:33 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:34.228 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:33 smithi029 ceph-mon[36249]: from='osd.1 [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T09:25:34.228 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:33 smithi029 ceph-mon[36249]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T09:25:34.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:33 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:34.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:33 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:34.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:33 smithi002 ceph-mon[31557]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:34.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:33 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:34.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:33 smithi002 ceph-mon[31557]: from='osd.1 [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T09:25:34.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:33 smithi002 ceph-mon[31557]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T09:25:35.111 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:34 smithi026 ceph-mon[36056]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-14T09:25:35.111 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:34 smithi026 ceph-mon[36056]: from='osd.1 [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:25:35.111 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:34 smithi026 ceph-mon[36056]: osdmap e11: 2 total, 1 up, 2 in 2024-05-14T09:25:35.112 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:34 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:35.112 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:34 smithi026 ceph-mon[36056]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:25:35.133 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:34 smithi029 ceph-mon[36249]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-14T09:25:35.133 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:34 smithi029 ceph-mon[36249]: from='osd.1 [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:25:35.133 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:34 smithi029 ceph-mon[36249]: osdmap e11: 2 total, 1 up, 2 in 2024-05-14T09:25:35.133 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:34 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:35.133 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:34 smithi029 ceph-mon[36249]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:25:35.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:34 smithi002 ceph-mon[31557]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-14T09:25:35.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:34 smithi002 ceph-mon[31557]: from='osd.1 [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:25:35.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:34 smithi002 ceph-mon[31557]: osdmap e11: 2 total, 1 up, 2 in 2024-05-14T09:25:35.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:34 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:35.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:34 smithi002 ceph-mon[31557]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:25:35.701 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 09:25:35 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-1[40621]: 2024-05-14T09:25:35.434+0000 7fdacef8f700 -1 osd.1 0 waiting for initial osdmap 2024-05-14T09:25:35.702 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 09:25:35 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-1[40621]: 2024-05-14T09:25:35.438+0000 7fdac653b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T09:25:35.737 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-14T09:25:35.737 INFO:teuthology.orchestra.run.smithi029.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-14T09:25:35.737 INFO:teuthology.orchestra.run.smithi029.stderr: stderr: 10+0 records in 2024-05-14T09:25:35.737 INFO:teuthology.orchestra.run.smithi029.stderr:10+0 records out 2024-05-14T09:25:35.737 INFO:teuthology.orchestra.run.smithi029.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00844072 s, 1.2 GB/s 2024-05-14T09:25:35.737 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping successful for: 2024-05-14T09:25:36.064 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:35 smithi029 ceph-mon[36249]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-14T09:25:36.064 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:35 smithi029 ceph-mon[36249]: osdmap e12: 2 total, 1 up, 2 in 2024-05-14T09:25:36.064 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:35 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:36.064 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:35 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:36.064 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:35 smithi029 ceph-mon[36249]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:36.064 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:35 smithi029 ceph-mon[36249]: from='osd.1 ' entity='osd.1' 2024-05-14T09:25:36.273 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:35 smithi026 ceph-mon[36056]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-14T09:25:36.274 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:35 smithi026 ceph-mon[36056]: osdmap e12: 2 total, 1 up, 2 in 2024-05-14T09:25:36.274 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:35 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:36.274 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:35 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:36.274 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:35 smithi026 ceph-mon[36056]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:36.274 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:35 smithi026 ceph-mon[36056]: from='osd.1 ' entity='osd.1' 2024-05-14T09:25:36.389 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi029:vg_nvme/lv_4 2024-05-14T09:25:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:35 smithi002 ceph-mon[31557]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-14T09:25:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:35 smithi002 ceph-mon[31557]: osdmap e12: 2 total, 1 up, 2 in 2024-05-14T09:25:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:35 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:35 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:35 smithi002 ceph-mon[31557]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-14T09:25:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:35 smithi002 ceph-mon[31557]: from='osd.1 ' entity='osd.1' 2024-05-14T09:25:36.645 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T09:25:37.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:36 smithi002 ceph-mon[31557]: purged_snaps scrub starts 2024-05-14T09:25:37.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:36 smithi002 ceph-mon[31557]: purged_snaps scrub ok 2024-05-14T09:25:37.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:37.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:36 smithi002 ceph-mon[31557]: osd.1 [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226] boot 2024-05-14T09:25:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:36 smithi002 ceph-mon[31557]: osdmap e13: 2 total, 2 up, 2 in 2024-05-14T09:25:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:25:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:25:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:37.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:36 smithi029 ceph-mon[36249]: purged_snaps scrub starts 2024-05-14T09:25:37.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:36 smithi029 ceph-mon[36249]: purged_snaps scrub ok 2024-05-14T09:25:37.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:37.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:36 smithi029 ceph-mon[36249]: osd.1 [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226] boot 2024-05-14T09:25:37.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:36 smithi029 ceph-mon[36249]: osdmap e13: 2 total, 2 up, 2 in 2024-05-14T09:25:37.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:37.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:37.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:37.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:25:37.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:37.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:37.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:25:37.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:37.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:36 smithi026 ceph-mon[36056]: purged_snaps scrub starts 2024-05-14T09:25:37.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:36 smithi026 ceph-mon[36056]: purged_snaps scrub ok 2024-05-14T09:25:37.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:37.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:36 smithi026 ceph-mon[36056]: osd.1 [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226] boot 2024-05-14T09:25:37.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:36 smithi026 ceph-mon[36056]: osdmap e13: 2 total, 2 up, 2 in 2024-05-14T09:25:37.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T09:25:37.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:37.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:37.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:25:37.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:37.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:37.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:25:37.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:37.985 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:37 smithi029 ceph-mon[36249]: Detected new or changed devices on smithi026 2024-05-14T09:25:37.985 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:37 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi026 to 17093M 2024-05-14T09:25:37.985 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:37 smithi029 ceph-mon[36249]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:38.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.195+0000 7fb747894700 1 -- 172.21.15.29:0/3478055325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb740103240 msgr2=0x7fb740103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:25:38.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.195+0000 7fb747894700 1 --2- 172.21.15.29:0/3478055325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb740103240 0x7fb740103650 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009990 tx=0x7fb73c009ca0 comp rx=0 tx=0).stop 2024-05-14T09:25:38.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.196+0000 7fb747894700 1 -- 172.21.15.29:0/3478055325 shutdown_connections 2024-05-14T09:25:38.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.196+0000 7fb747894700 1 --2- 172.21.15.29:0/3478055325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb740104d10 0x7fb74010b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:38.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.196+0000 7fb747894700 1 --2- 172.21.15.29:0/3478055325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb740103240 0x7fb740103650 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:38.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.196+0000 7fb747894700 1 --2- 172.21.15.29:0/3478055325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb740104370 0x7fb740104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:38.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.196+0000 7fb747894700 1 -- 172.21.15.29:0/3478055325 >> 172.21.15.29:0/3478055325 conn(0x7fb7400fe880 msgr2=0x7fb740100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:25:38.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.196+0000 7fb747894700 1 -- 172.21.15.29:0/3478055325 shutdown_connections 2024-05-14T09:25:38.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.196+0000 7fb747894700 1 -- 172.21.15.29:0/3478055325 wait complete. 2024-05-14T09:25:38.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.196+0000 7fb747894700 1 Processor -- start 2024-05-14T09:25:38.197 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.196+0000 7fb747894700 1 -- start start 2024-05-14T09:25:38.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb747894700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb740103240 0x7fb740199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:25:38.199 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb747894700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb740104370 0x7fb740199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:25:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb747894700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb740104d10 0x7fb7401a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:25:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb747894700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb74010ec80 con 0x7fb740103240 2024-05-14T09:25:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb747894700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb74010eb00 con 0x7fb740104370 2024-05-14T09:25:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb747894700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb74010ee00 con 0x7fb740104d10 2024-05-14T09:25:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb745e31700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb740104d10 0x7fb7401a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:25:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb745e31700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb740104d10 0x7fb7401a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:56252/0 (socket says 172.21.15.29:56252) 2024-05-14T09:25:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb745e31700 1 -- 172.21.15.29:0/3153864647 learned_addr learned my addr 172.21.15.29:0/3153864647 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-05-14T09:25:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb744e2f700 1 --2- 172.21.15.29:0/3153864647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb740104370 0x7fb740199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:25:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb745630700 1 --2- 172.21.15.29:0/3153864647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb740103240 0x7fb740199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:25:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb745e31700 1 -- 172.21.15.29:0/3153864647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb740104370 msgr2=0x7fb740199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:25:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb745e31700 1 --2- 172.21.15.29:0/3153864647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb740104370 0x7fb740199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb745e31700 1 -- 172.21.15.29:0/3153864647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb740103240 msgr2=0x7fb740199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:25:38.200 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb745e31700 1 --2- 172.21.15.29:0/3153864647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb740103240 0x7fb740199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:38.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb745e31700 1 -- 172.21.15.29:0/3153864647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7401a1390 con 0x7fb740104d10 2024-05-14T09:25:38.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb745e31700 1 --2- 172.21.15.29:0/3153864647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb740104d10 0x7fb7401a0d10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb738007f10 tx=0x7fb73800f5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:25:38.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.197+0000 7fb7367fc700 1 -- 172.21.15.29:0/3153864647 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738018070 con 0x7fb740104d10 2024-05-14T09:25:38.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.198+0000 7fb7367fc700 1 -- 172.21.15.29:0/3153864647 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb738004020 con 0x7fb740104d10 2024-05-14T09:25:38.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.198+0000 7fb747894700 1 -- 172.21.15.29:0/3153864647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74006b670 con 0x7fb740104d10 2024-05-14T09:25:38.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.198+0000 7fb7367fc700 1 -- 172.21.15.29:0/3153864647 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7380126c0 con 0x7fb740104d10 2024-05-14T09:25:38.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.198+0000 7fb747894700 1 -- 172.21.15.29:0/3153864647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb74006bbb0 con 0x7fb740104d10 2024-05-14T09:25:38.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.199+0000 7fb747894700 1 -- 172.21.15.29:0/3153864647 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7400512c0 con 0x7fb740104d10 2024-05-14T09:25:38.201 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.201+0000 7fb7367fc700 1 -- 172.21.15.29:0/3153864647 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7fb7380044f0 con 0x7fb740104d10 2024-05-14T09:25:38.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.202+0000 7fb7367fc700 1 --2- 172.21.15.29:0/3153864647 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb72c0b2870 0x7fb72c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:25:38.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.202+0000 7fb7367fc700 1 -- 172.21.15.29:0/3153864647 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fb7380e2fa0 con 0x7fb740104d10 2024-05-14T09:25:38.202 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.202+0000 7fb745630700 1 --2- 172.21.15.29:0/3153864647 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb72c0b2870 0x7fb72c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:25:38.203 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.203+0000 7fb745630700 1 --2- 172.21.15.29:0/3153864647 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb72c0b2870 0x7fb72c0b4d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0038a0 tx=0x7fb73c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:25:38.206 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.206+0000 7fb7367fc700 1 -- 172.21.15.29:0/3153864647 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7380ac470 con 0x7fb740104d10 2024-05-14T09:25:38.354 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:37 smithi026 ceph-mon[36056]: Detected new or changed devices on smithi026 2024-05-14T09:25:38.354 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:37 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi026 to 17093M 2024-05-14T09:25:38.354 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:37 smithi026 ceph-mon[36056]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:38.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:37 smithi002 ceph-mon[31557]: Detected new or changed devices on smithi026 2024-05-14T09:25:38.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:37 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi026 to 17093M 2024-05-14T09:25:38.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:37 smithi002 ceph-mon[31557]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:38.429 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:38.429+0000 7fb747894700 1 -- 172.21.15.29:0/3153864647 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb74006be10 con 0x7fb72c0b2870 2024-05-14T09:25:39.244 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:38 smithi029 ceph-mon[36249]: osdmap e14: 2 total, 2 up, 2 in 2024-05-14T09:25:39.244 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:38 smithi029 ceph-mon[36249]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:25:39.244 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:25:39.244 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:25:39.244 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:39.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:38 smithi002 ceph-mon[31557]: osdmap e14: 2 total, 2 up, 2 in 2024-05-14T09:25:39.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:38 smithi002 ceph-mon[31557]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:25:39.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:25:39.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:25:39.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:39.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:38 smithi026 ceph-mon[36056]: osdmap e14: 2 total, 2 up, 2 in 2024-05-14T09:25:39.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:38 smithi026 ceph-mon[36056]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:25:39.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:25:39.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:25:39.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:40.180 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:39 smithi029 ceph-mon[36249]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:40.232 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:39 smithi002 ceph-mon[31557]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:40.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:39 smithi026 ceph-mon[36056]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:41 smithi002 ceph-mon[31557]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:41 smithi002 ceph-mon[31557]: from='client.? 172.21.15.29:0/3667728695' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c5bb589-56e2-469f-9a6f-6035b3ddf4bd"}]: dispatch 2024-05-14T09:25:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:41 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c5bb589-56e2-469f-9a6f-6035b3ddf4bd"}]: dispatch 2024-05-14T09:25:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:41 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c5bb589-56e2-469f-9a6f-6035b3ddf4bd"}]': finished 2024-05-14T09:25:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:41 smithi002 ceph-mon[31557]: osdmap e15: 3 total, 2 up, 3 in 2024-05-14T09:25:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:25:42.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:41 smithi002 ceph-mon[31557]: from='client.? 172.21.15.29:0/1261545158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:25:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:41 smithi029 ceph-mon[36249]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:41 smithi029 ceph-mon[36249]: from='client.? 172.21.15.29:0/3667728695' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c5bb589-56e2-469f-9a6f-6035b3ddf4bd"}]: dispatch 2024-05-14T09:25:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:41 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c5bb589-56e2-469f-9a6f-6035b3ddf4bd"}]: dispatch 2024-05-14T09:25:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:41 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c5bb589-56e2-469f-9a6f-6035b3ddf4bd"}]': finished 2024-05-14T09:25:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:41 smithi029 ceph-mon[36249]: osdmap e15: 3 total, 2 up, 3 in 2024-05-14T09:25:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:25:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:41 smithi029 ceph-mon[36249]: from='client.? 172.21.15.29:0/1261545158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:25:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:41 smithi026 ceph-mon[36056]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:41 smithi026 ceph-mon[36056]: from='client.? 172.21.15.29:0/3667728695' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c5bb589-56e2-469f-9a6f-6035b3ddf4bd"}]: dispatch 2024-05-14T09:25:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:41 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c5bb589-56e2-469f-9a6f-6035b3ddf4bd"}]: dispatch 2024-05-14T09:25:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:41 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c5bb589-56e2-469f-9a6f-6035b3ddf4bd"}]': finished 2024-05-14T09:25:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:41 smithi026 ceph-mon[36056]: osdmap e15: 3 total, 2 up, 3 in 2024-05-14T09:25:42.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:25:42.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:41 smithi026 ceph-mon[36056]: from='client.? 172.21.15.29:0/1261545158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:25:44.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:43 smithi002 ceph-mon[31557]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:44.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:43 smithi029 ceph-mon[36249]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:44.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:43 smithi026 ceph-mon[36056]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:46.258 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:45 smithi029 ceph-mon[36249]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:46.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:45 smithi002 ceph-mon[31557]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:46.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:45 smithi026 ceph-mon[36056]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:48.159 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:47 smithi029 ceph-mon[36249]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:48.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:47 smithi002 ceph-mon[31557]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:48.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:47 smithi026 ceph-mon[36056]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:49.312 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-14T09:25:49.313 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:49.313 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:48 smithi029 ceph-mon[36249]: Deploying daemon osd.2 on smithi029 2024-05-14T09:25:49.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-14T09:25:49.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:49.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:48 smithi002 ceph-mon[31557]: Deploying daemon osd.2 on smithi029 2024-05-14T09:25:49.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-14T09:25:49.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:49.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:48 smithi026 ceph-mon[36056]: Deploying daemon osd.2 on smithi029 2024-05-14T09:25:50.247 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:49 smithi029 ceph-mon[36249]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:49 smithi002 ceph-mon[31557]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:50.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:49 smithi026 ceph-mon[36056]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:52.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:51 smithi029 ceph-mon[36249]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:52.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:51 smithi002 ceph-mon[31557]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:52.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:51 smithi026 ceph-mon[36056]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:54.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:53 smithi029 ceph-mon[36249]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:54.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:53 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:54.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:53 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:54.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:53 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:53 smithi002 ceph-mon[31557]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:54.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:53 smithi026 ceph-mon[36056]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:54.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:53 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:54.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:53 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:54.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:53 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:56.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:56 smithi029 ceph-mon[36249]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:56.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:56 smithi002 ceph-mon[31557]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:56 smithi026 ceph-mon[36056]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:57.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:57.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:57.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:57.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:25:57.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:57.750 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:57.750+0000 7fb7367fc700 1 -- 172.21.15.29:0/3153864647 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb74006be10 con 0x7fb72c0b2870 2024-05-14T09:25:57.751 INFO:teuthology.orchestra.run.smithi029.stdout:Created osd(s) 2 on host 'smithi029' 2024-05-14T09:25:57.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:57.752+0000 7fb747894700 1 -- 172.21.15.29:0/3153864647 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb72c0b2870 msgr2=0x7fb72c0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:25:57.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:57.752+0000 7fb747894700 1 --2- 172.21.15.29:0/3153864647 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb72c0b2870 0x7fb72c0b4d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0038a0 tx=0x7fb73c01a040 comp rx=0 tx=0).stop 2024-05-14T09:25:57.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:57.753+0000 7fb747894700 1 -- 172.21.15.29:0/3153864647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb740104d10 msgr2=0x7fb7401a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:25:57.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:57.753+0000 7fb747894700 1 --2- 172.21.15.29:0/3153864647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb740104d10 0x7fb7401a0d10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb738007f10 tx=0x7fb73800f5c0 comp rx=0 tx=0).stop 2024-05-14T09:25:57.753 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:57.753+0000 7fb747894700 1 -- 172.21.15.29:0/3153864647 shutdown_connections 2024-05-14T09:25:57.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:57.753+0000 7fb747894700 1 --2- 172.21.15.29:0/3153864647 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb72c0b2870 0x7fb72c0b4d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:57.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:57.753+0000 7fb747894700 1 --2- 172.21.15.29:0/3153864647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb740104d10 0x7fb7401a0d10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:57.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:57.753+0000 7fb747894700 1 --2- 172.21.15.29:0/3153864647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb740103240 0x7fb740199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:57.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:57.753+0000 7fb747894700 1 --2- 172.21.15.29:0/3153864647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb740104370 0x7fb740199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:25:57.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:57.753+0000 7fb747894700 1 -- 172.21.15.29:0/3153864647 >> 172.21.15.29:0/3153864647 conn(0x7fb7400fe880 msgr2=0x7fb7401075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:25:57.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:57.753+0000 7fb747894700 1 -- 172.21.15.29:0/3153864647 shutdown_connections 2024-05-14T09:25:57.754 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:25:57.753+0000 7fb747894700 1 -- 172.21.15.29:0/3153864647 wait complete. 2024-05-14T09:25:57.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:57.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:57.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:57.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:25:57.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:57.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:57.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:57.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:25:57.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:25:57.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:58.381 DEBUG:teuthology.orchestra.run.smithi029:osd.2> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.2.service 2024-05-14T09:25:58.384 INFO:tasks.cephadm:Deploying osd.3 on smithi002 with /dev/vg_nvme/lv_3... 2024-05-14T09:25:58.384 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-14T09:25:58.643 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:25:58.664 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:58 smithi029 ceph-mon[36249]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:58.664 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:58 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:58.664 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:58 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:58.664 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:58 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:58.665 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:58 smithi029 ceph-mon[36249]: from='osd.2 [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T09:25:58.665 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:58 smithi029 ceph-mon[36249]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T09:25:58.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:58 smithi002 ceph-mon[31557]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:58.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:58 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:58.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:58 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:58.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:58 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:58.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:58 smithi002 ceph-mon[31557]: from='osd.2 [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T09:25:58.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:58 smithi002 ceph-mon[31557]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T09:25:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:58 smithi026 ceph-mon[36056]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:25:58.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:58 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:58.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:58 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:25:58.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:58 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:25:58.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:58 smithi026 ceph-mon[36056]: from='osd.2 [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T09:25:58.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:58 smithi026 ceph-mon[36056]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T09:25:59.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:59 smithi029 ceph-mon[36249]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-14T09:25:59.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:59 smithi029 ceph-mon[36249]: osdmap e16: 3 total, 2 up, 3 in 2024-05-14T09:25:59.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:59 smithi029 ceph-mon[36249]: from='osd.2 [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:25:59.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:25:59.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:59 smithi029 ceph-mon[36249]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:25:59.819 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:25:59 smithi029 ceph-mon[36249]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:26:00.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:59 smithi002 ceph-mon[31557]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-14T09:26:00.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:59 smithi002 ceph-mon[31557]: osdmap e16: 3 total, 2 up, 3 in 2024-05-14T09:26:00.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:59 smithi002 ceph-mon[31557]: from='osd.2 [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:26:00.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:26:00.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:59 smithi002 ceph-mon[31557]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:26:00.007 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:25:59 smithi002 ceph-mon[31557]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:26:00.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:59 smithi026 ceph-mon[36056]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-14T09:26:00.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:59 smithi026 ceph-mon[36056]: osdmap e16: 3 total, 2 up, 3 in 2024-05-14T09:26:00.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:59 smithi026 ceph-mon[36056]: from='osd.2 [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:26:00.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:26:00.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:59 smithi026 ceph-mon[36056]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:26:00.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:25:59 smithi026 ceph-mon[36056]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:26:00.555 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-14T09:26:00.555 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-14T09:26:00.555 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2024-05-14T09:26:00.555 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2024-05-14T09:26:00.555 INFO:teuthology.orchestra.run.smithi002.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00853592 s, 1.2 GB/s 2024-05-14T09:26:00.555 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2024-05-14T09:26:00.687 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 09:26:00 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-2[41339]: 2024-05-14T09:26:00.254+0000 7f65dab48700 -1 osd.2 0 waiting for initial osdmap 2024-05-14T09:26:00.687 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 09:26:00 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-2[41339]: 2024-05-14T09:26:00.258+0000 7f65d28f5700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T09:26:01.017 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:00 smithi029 ceph-mon[36249]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-05-14T09:26:01.017 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:00 smithi029 ceph-mon[36249]: osdmap e17: 3 total, 2 up, 3 in 2024-05-14T09:26:01.017 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:00 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:26:01.017 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:00 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:26:01.017 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:00 smithi029 ceph-mon[36249]: from='osd.2 ' entity='osd.2' 2024-05-14T09:26:01.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:00 smithi002 ceph-mon[31557]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-05-14T09:26:01.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:00 smithi002 ceph-mon[31557]: osdmap e17: 3 total, 2 up, 3 in 2024-05-14T09:26:01.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:00 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:26:01.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:00 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:26:01.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:00 smithi002 ceph-mon[31557]: from='osd.2 ' entity='osd.2' 2024-05-14T09:26:01.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:00 smithi026 ceph-mon[36056]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-05-14T09:26:01.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:00 smithi026 ceph-mon[36056]: osdmap e17: 3 total, 2 up, 3 in 2024-05-14T09:26:01.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:00 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:26:01.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:00 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:26:01.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:00 smithi026 ceph-mon[36056]: from='osd.2 ' entity='osd.2' 2024-05-14T09:26:01.225 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi002:vg_nvme/lv_3 2024-05-14T09:26:01.485 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:26:01.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: purged_snaps scrub starts 2024-05-14T09:26:01.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: purged_snaps scrub ok 2024-05-14T09:26:01.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:26:01.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:26:01.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: osd.2 [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086] boot 2024-05-14T09:26:01.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: osdmap e18: 3 total, 3 up, 3 in 2024-05-14T09:26:01.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:26:01.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: Detected new or changed devices on smithi029 2024-05-14T09:26:01.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:01.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:01.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:26:01.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi029 to 17093M 2024-05-14T09:26:01.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:01.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:01.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:26:01.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:02.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: purged_snaps scrub starts 2024-05-14T09:26:02.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: purged_snaps scrub ok 2024-05-14T09:26:02.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:26:02.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:26:02.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: osd.2 [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086] boot 2024-05-14T09:26:02.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: osdmap e18: 3 total, 3 up, 3 in 2024-05-14T09:26:02.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:26:02.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: Detected new or changed devices on smithi029 2024-05-14T09:26:02.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:02.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:02.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:26:02.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi029 to 17093M 2024-05-14T09:26:02.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:02.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:02.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:26:02.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:01 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:02.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: purged_snaps scrub starts 2024-05-14T09:26:02.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: purged_snaps scrub ok 2024-05-14T09:26:02.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:26:02.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T09:26:02.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: osd.2 [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086] boot 2024-05-14T09:26:02.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: osdmap e18: 3 total, 3 up, 3 in 2024-05-14T09:26:02.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T09:26:02.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: Detected new or changed devices on smithi029 2024-05-14T09:26:02.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:02.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:02.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:26:02.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi029 to 17093M 2024-05-14T09:26:02.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:02.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:02.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:26:02.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:01 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:02.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.991+0000 7fc4b8e26700 1 -- 172.21.15.2:0/1162033484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4b4104380 msgr2=0x7fc4b4104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:26:02.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.991+0000 7fc4b8e26700 1 --2- 172.21.15.2:0/1162033484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4b4104380 0x7fc4b4104790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009990 tx=0x7fc4a8009ca0 comp rx=0 tx=0).stop 2024-05-14T09:26:02.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.991+0000 7fc4b8e26700 1 -- 172.21.15.2:0/1162033484 shutdown_connections 2024-05-14T09:26:02.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.991+0000 7fc4b8e26700 1 --2- 172.21.15.2:0/1162033484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b4104d20 0x7fc4b410b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:02.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.991+0000 7fc4b8e26700 1 --2- 172.21.15.2:0/1162033484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4b4103250 0x7fc4b4103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:02.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.991+0000 7fc4b8e26700 1 --2- 172.21.15.2:0/1162033484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4b4104380 0x7fc4b4104790 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:02.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.991+0000 7fc4b8e26700 1 -- 172.21.15.2:0/1162033484 >> 172.21.15.2:0/1162033484 conn(0x7fc4b40fe890 msgr2=0x7fc4b4100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:26:02.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.991+0000 7fc4b8e26700 1 -- 172.21.15.2:0/1162033484 shutdown_connections 2024-05-14T09:26:02.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.991+0000 7fc4b8e26700 1 -- 172.21.15.2:0/1162033484 wait complete. 2024-05-14T09:26:02.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.992+0000 7fc4b8e26700 1 Processor -- start 2024-05-14T09:26:02.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.992+0000 7fc4b8e26700 1 -- start start 2024-05-14T09:26:02.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.992+0000 7fc4b8e26700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4b4103250 0x7fc4b4076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:26:02.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.992+0000 7fc4b8e26700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b4104380 0x7fc4b41a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:26:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.992+0000 7fc4b8e26700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4b4104d20 0x7fc4b41a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:26:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.992+0000 7fc4b8e26700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b410ecb0 con 0x7fc4b4103250 2024-05-14T09:26:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.992+0000 7fc4b8e26700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b410eb30 con 0x7fc4b4104380 2024-05-14T09:26:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.992+0000 7fc4b8e26700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b410ee30 con 0x7fc4b4104d20 2024-05-14T09:26:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.992+0000 7fc4b259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4b4103250 0x7fc4b4076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:26:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.992+0000 7fc4b259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4b4103250 0x7fc4b4076450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46926/0 (socket says 172.21.15.2:46926) 2024-05-14T09:26:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.992+0000 7fc4b259c700 1 -- 172.21.15.2:0/3892778072 learned_addr learned my addr 172.21.15.2:0/3892778072 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:26:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.992+0000 7fc4b1d9b700 1 --2- 172.21.15.2:0/3892778072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b4104380 0x7fc4b41a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:26:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.993+0000 7fc4b2d9d700 1 --2- 172.21.15.2:0/3892778072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4b4104d20 0x7fc4b41a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:26:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.993+0000 7fc4b259c700 1 -- 172.21.15.2:0/3892778072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4b4104d20 msgr2=0x7fc4b41a4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:26:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.993+0000 7fc4b259c700 1 --2- 172.21.15.2:0/3892778072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4b4104d20 0x7fc4b41a4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.993+0000 7fc4b259c700 1 -- 172.21.15.2:0/3892778072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b4104380 msgr2=0x7fc4b41a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:26:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.993+0000 7fc4b259c700 1 --2- 172.21.15.2:0/3892778072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b4104380 0x7fc4b41a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.993+0000 7fc4b259c700 1 -- 172.21.15.2:0/3892778072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b41a4700 con 0x7fc4b4103250 2024-05-14T09:26:02.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.993+0000 7fc4b2d9d700 1 --2- 172.21.15.2:0/3892778072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4b4104d20 0x7fc4b41a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:26:02.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.993+0000 7fc4b259c700 1 --2- 172.21.15.2:0/3892778072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4b4103250 0x7fc4b4076450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400d950 tx=0x7fc4a400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:26:02.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.993+0000 7fc4a37fe700 1 -- 172.21.15.2:0/3892778072 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4016070 con 0x7fc4b4103250 2024-05-14T09:26:02.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.993+0000 7fc4a37fe700 1 -- 172.21.15.2:0/3892778072 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4a400be00 con 0x7fc4b4103250 2024-05-14T09:26:02.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.993+0000 7fc4b8e26700 1 -- 172.21.15.2:0/3892778072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b41a49f0 con 0x7fc4b4103250 2024-05-14T09:26:02.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.993+0000 7fc4a37fe700 1 -- 172.21.15.2:0/3892778072 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a400f3c0 con 0x7fc4b4103250 2024-05-14T09:26:02.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.993+0000 7fc4b8e26700 1 -- 172.21.15.2:0/3892778072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b41a4f30 con 0x7fc4b4103250 2024-05-14T09:26:02.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.994+0000 7fc4b8e26700 1 -- 172.21.15.2:0/3892778072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b40512c0 con 0x7fc4b4103250 2024-05-14T09:26:02.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.997+0000 7fc4a37fe700 1 -- 172.21.15.2:0/3892778072 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7fc4a4019070 con 0x7fc4b4103250 2024-05-14T09:26:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.998+0000 7fc4a37fe700 1 --2- 172.21.15.2:0/3892778072 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc49c0b27f0 0x7fc49c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:26:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.998+0000 7fc4b1d9b700 1 --2- 172.21.15.2:0/3892778072 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc49c0b27f0 0x7fc49c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:26:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.998+0000 7fc4a37fe700 1 -- 172.21.15.2:0/3892778072 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7fc4a40e0f80 con 0x7fc4b4103250 2024-05-14T09:26:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:02.998+0000 7fc4b1d9b700 1 --2- 172.21.15.2:0/3892778072 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc49c0b27f0 0x7fc49c0b4d00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800b5c0 tx=0x7fc4a801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:26:03.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:03.001+0000 7fc4a37fe700 1 -- 172.21.15.2:0/3892778072 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc4a4020020 con 0x7fc4b4103250 2024-05-14T09:26:03.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:03.226+0000 7fc4b8e26700 1 -- 172.21.15.2:0/3892778072 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc4b4063c00 con 0x7fc49c0b27f0 2024-05-14T09:26:04.105 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:03 smithi002 ceph-mon[31557]: osdmap e19: 3 total, 3 up, 3 in 2024-05-14T09:26:04.105 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:03 smithi002 ceph-mon[31557]: pgmap v86: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:04.106 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:03 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-14T09:26:04.106 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:03 smithi002 ceph-mon[31557]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:26:04.106 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:03 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:26:04.106 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:03 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:26:04.106 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:03 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:04.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:03 smithi029 ceph-mon[36249]: osdmap e19: 3 total, 3 up, 3 in 2024-05-14T09:26:04.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:03 smithi029 ceph-mon[36249]: pgmap v86: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:04.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:03 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-14T09:26:04.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:03 smithi029 ceph-mon[36249]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:26:04.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:03 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:26:04.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:03 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:26:04.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:03 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:04.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:03 smithi026 ceph-mon[36056]: osdmap e19: 3 total, 3 up, 3 in 2024-05-14T09:26:04.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:03 smithi026 ceph-mon[36056]: pgmap v86: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:04.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:03 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-14T09:26:04.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:03 smithi026 ceph-mon[36056]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:26:04.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:03 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:26:04.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:03 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:26:04.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:03 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:04.821 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:04 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-14T09:26:04.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:04 smithi002 ceph-mon[31557]: osdmap e20: 3 total, 3 up, 3 in 2024-05-14T09:26:04.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:04 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T09:26:05.066 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:04 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-14T09:26:05.066 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:04 smithi029 ceph-mon[36249]: osdmap e20: 3 total, 3 up, 3 in 2024-05-14T09:26:05.066 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:04 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T09:26:05.066 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 09:26:04 smithi029 sudo[44940]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T09:26:05.066 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 09:26:04 smithi029 sudo[44940]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T09:26:05.066 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 09:26:04 smithi029 sudo[44940]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T09:26:05.066 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 09:26:04 smithi029 sudo[44940]: pam_unix(sudo:session): session closed for user root 2024-05-14T09:26:05.066 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 09:26:04 smithi029 sudo[44943]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T09:26:05.066 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 09:26:04 smithi029 sudo[44943]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T09:26:05.067 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 09:26:04 smithi029 sudo[44943]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T09:26:05.067 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 09:26:04 smithi029 sudo[44943]: pam_unix(sudo:session): session closed for user root 2024-05-14T09:26:05.104 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:04 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-14T09:26:05.104 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:04 smithi026 ceph-mon[36056]: osdmap e20: 3 total, 3 up, 3 in 2024-05-14T09:26:05.104 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:04 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T09:26:05.104 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 09:26:04 smithi026 sudo[44728]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T09:26:05.104 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 09:26:04 smithi026 sudo[44728]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T09:26:05.104 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 09:26:04 smithi026 sudo[44728]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T09:26:05.104 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 09:26:04 smithi026 sudo[44728]: pam_unix(sudo:session): session closed for user root 2024-05-14T09:26:05.104 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 09:26:04 smithi026 sudo[44731]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T09:26:05.104 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 09:26:04 smithi026 sudo[44731]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T09:26:05.104 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 09:26:04 smithi026 sudo[44731]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T09:26:05.105 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 09:26:04 smithi026 sudo[44731]: pam_unix(sudo:session): session closed for user root 2024-05-14T09:26:05.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 sudo[48287]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T09:26:05.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 sudo[48287]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T09:26:05.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 sudo[48287]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T09:26:05.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 sudo[48287]: pam_unix(sudo:session): session closed for user root 2024-05-14T09:26:05.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 sudo[48290]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T09:26:05.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 sudo[48290]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T09:26:05.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 sudo[48290]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T09:26:05.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 sudo[48290]: pam_unix(sudo:session): session closed for user root 2024-05-14T09:26:05.169 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 09:26:04 smithi002 sudo[48260]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T09:26:05.169 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 09:26:04 smithi002 sudo[48260]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T09:26:05.169 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 09:26:04 smithi002 sudo[48260]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T09:26:05.169 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 09:26:04 smithi002 sudo[48260]: pam_unix(sudo:session): session closed for user root 2024-05-14T09:26:05.169 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 09:26:04 smithi002 sudo[48263]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T09:26:05.169 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 09:26:04 smithi002 sudo[48263]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T09:26:05.169 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 09:26:04 smithi002 sudo[48263]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T09:26:05.169 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 09:26:04 smithi002 sudo[48263]: pam_unix(sudo:session): session closed for user root 2024-05-14T09:26:05.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 sudo[44946]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T09:26:05.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 sudo[44946]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T09:26:05.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 sudo[44946]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T09:26:05.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 sudo[44946]: pam_unix(sudo:session): session closed for user root 2024-05-14T09:26:05.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 sudo[44949]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T09:26:05.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 sudo[44949]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T09:26:05.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 sudo[44949]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T09:26:05.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 sudo[44949]: pam_unix(sudo:session): session closed for user root 2024-05-14T09:26:05.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 sudo[44734]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T09:26:05.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 sudo[44734]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T09:26:05.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 sudo[44734]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T09:26:05.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 sudo[44734]: pam_unix(sudo:session): session closed for user root 2024-05-14T09:26:05.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 sudo[44737]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T09:26:05.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 sudo[44737]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T09:26:05.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 sudo[44737]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T09:26:05.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 sudo[44737]: pam_unix(sudo:session): session closed for user root 2024-05-14T09:26:05.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:05.961+0000 7fc4a37fe700 1 -- 172.21.15.2:0/3892778072 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fc4a40a5e60 con 0x7fc4b4103250 2024-05-14T09:26:06.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-14T09:26:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: osdmap e21: 3 total, 3 up, 3 in 2024-05-14T09:26:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T09:26:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T09:26:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:26:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:26:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:26:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T09:26:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:26:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:26:06.173 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:26:06.173 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T09:26:06.173 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T09:26:06.173 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:26:06.173 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:26:06.173 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:26:06.173 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:05 smithi002 ceph-mon[31557]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T09:26:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-14T09:26:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: osdmap e21: 3 total, 3 up, 3 in 2024-05-14T09:26:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T09:26:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T09:26:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:26:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:26:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:26:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T09:26:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:26:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:26:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:26:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T09:26:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T09:26:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:26:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:26:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:26:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:05 smithi029 ceph-mon[36249]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T09:26:06.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-14T09:26:06.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: osdmap e21: 3 total, 3 up, 3 in 2024-05-14T09:26:06.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:06.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:06.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:06.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:06.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T09:26:06.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T09:26:06.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:26:06.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:26:06.203 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:26:06.203 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T09:26:06.203 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:26:06.203 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:26:06.203 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:26:06.203 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T09:26:06.203 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T09:26:06.203 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T09:26:06.203 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T09:26:06.203 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T09:26:06.203 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:05 smithi026 ceph-mon[36056]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T09:26:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:06 smithi002 ceph-mon[31557]: osdmap e22: 3 total, 3 up, 3 in 2024-05-14T09:26:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:06 smithi002 ceph-mon[31557]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-14T09:26:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:06 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/4090790072' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fbfc078-85e7-4dc2-acda-5e7b0ab927e1"}]: dispatch 2024-05-14T09:26:07.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:06 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/4090790072' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fbfc078-85e7-4dc2-acda-5e7b0ab927e1"}]': finished 2024-05-14T09:26:07.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:06 smithi002 ceph-mon[31557]: osdmap e23: 4 total, 3 up, 4 in 2024-05-14T09:26:07.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:06 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T09:26:07.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:06 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/907303419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:26:07.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:06 smithi029 ceph-mon[36249]: osdmap e22: 3 total, 3 up, 3 in 2024-05-14T09:26:07.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:06 smithi029 ceph-mon[36249]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-14T09:26:07.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:06 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/4090790072' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fbfc078-85e7-4dc2-acda-5e7b0ab927e1"}]: dispatch 2024-05-14T09:26:07.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:06 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/4090790072' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fbfc078-85e7-4dc2-acda-5e7b0ab927e1"}]': finished 2024-05-14T09:26:07.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:06 smithi029 ceph-mon[36249]: osdmap e23: 4 total, 3 up, 4 in 2024-05-14T09:26:07.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:06 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T09:26:07.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:06 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/907303419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:26:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:06 smithi026 ceph-mon[36056]: osdmap e22: 3 total, 3 up, 3 in 2024-05-14T09:26:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:06 smithi026 ceph-mon[36056]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-14T09:26:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:06 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/4090790072' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fbfc078-85e7-4dc2-acda-5e7b0ab927e1"}]: dispatch 2024-05-14T09:26:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:06 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/4090790072' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fbfc078-85e7-4dc2-acda-5e7b0ab927e1"}]': finished 2024-05-14T09:26:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:06 smithi026 ceph-mon[36056]: osdmap e23: 4 total, 3 up, 4 in 2024-05-14T09:26:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:06 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T09:26:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:06 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/907303419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:26:08.088 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:07 smithi029 ceph-mon[36249]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:08.102 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:07 smithi026 ceph-mon[36056]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:08.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:07 smithi002 ceph-mon[31557]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:10.126 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:10 smithi002 ceph-mon[31557]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:10.439 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:10 smithi029 ceph-mon[36249]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:10.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:10 smithi026 ceph-mon[36056]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:12.250 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:12 smithi002 ceph-mon[31557]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:12 smithi029 ceph-mon[36249]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:12.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:12 smithi026 ceph-mon[36056]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:14.312 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:14 smithi002 ceph-mon[31557]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:14.312 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:14 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-14T09:26:14.313 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:14 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:14.313 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:14 smithi002 ceph-mon[31557]: Deploying daemon osd.3 on smithi002 2024-05-14T09:26:14.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:14 smithi029 ceph-mon[36249]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:14.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-14T09:26:14.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:14.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:14 smithi029 ceph-mon[36249]: Deploying daemon osd.3 on smithi002 2024-05-14T09:26:14.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:14 smithi026 ceph-mon[36056]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:14.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:14 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-14T09:26:14.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:14 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:14.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:14 smithi026 ceph-mon[36056]: Deploying daemon osd.3 on smithi002 2024-05-14T09:26:16.391 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:16 smithi002 ceph-mon[31557]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:16.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:16 smithi029 ceph-mon[36249]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:16.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:16 smithi026 ceph-mon[36056]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:18.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:18 smithi002 ceph-mon[31557]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:18.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:18 smithi029 ceph-mon[36249]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:18.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:18 smithi026 ceph-mon[36056]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:20.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:20 smithi002 ceph-mon[31557]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:20.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:20 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:20.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:20 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:20.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:20 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:26:20.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:20 smithi029 ceph-mon[36249]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:20.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:20 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:20.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:20 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:20.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:20 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:26:20.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:20 smithi026 ceph-mon[36056]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:20.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:20 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:20.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:20 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:20.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:20 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:26:22.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:22 smithi002 ceph-mon[31557]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:22.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:22 smithi029 ceph-mon[36249]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:22.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:22 smithi026 ceph-mon[36056]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:22.482+0000 7fc4a37fe700 1 -- 172.21.15.2:0/3892778072 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc4b4063c00 con 0x7fc49c0b27f0 2024-05-14T09:26:22.483 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 3 on host 'smithi002' 2024-05-14T09:26:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:22.485+0000 7fc4b8e26700 1 -- 172.21.15.2:0/3892778072 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc49c0b27f0 msgr2=0x7fc49c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:26:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:22.485+0000 7fc4b8e26700 1 --2- 172.21.15.2:0/3892778072 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc49c0b27f0 0x7fc49c0b4d00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800b5c0 tx=0x7fc4a801a040 comp rx=0 tx=0).stop 2024-05-14T09:26:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:22.485+0000 7fc4b8e26700 1 -- 172.21.15.2:0/3892778072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4b4103250 msgr2=0x7fc4b4076450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:26:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:22.485+0000 7fc4b8e26700 1 --2- 172.21.15.2:0/3892778072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4b4103250 0x7fc4b4076450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400d950 tx=0x7fc4a400dc60 comp rx=0 tx=0).stop 2024-05-14T09:26:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:22.485+0000 7fc4b8e26700 1 -- 172.21.15.2:0/3892778072 shutdown_connections 2024-05-14T09:26:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:22.485+0000 7fc4b8e26700 1 --2- 172.21.15.2:0/3892778072 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc49c0b27f0 0x7fc49c0b4d00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:22.485+0000 7fc4b8e26700 1 --2- 172.21.15.2:0/3892778072 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc4b4104d20 0x7fc4b41a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:22.485+0000 7fc4b8e26700 1 --2- 172.21.15.2:0/3892778072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4b4103250 0x7fc4b4076450 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:22.485+0000 7fc4b8e26700 1 --2- 172.21.15.2:0/3892778072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b4104380 0x7fc4b41a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:22.485+0000 7fc4b8e26700 1 -- 172.21.15.2:0/3892778072 >> 172.21.15.2:0/3892778072 conn(0x7fc4b40fe890 msgr2=0x7fc4b41075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:26:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:22.485+0000 7fc4b8e26700 1 -- 172.21.15.2:0/3892778072 shutdown_connections 2024-05-14T09:26:22.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:26:22.485+0000 7fc4b8e26700 1 -- 172.21.15.2:0/3892778072 wait complete. 2024-05-14T09:26:23.115 DEBUG:teuthology.orchestra.run.smithi002:osd.3> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.3.service 2024-05-14T09:26:23.117 INFO:tasks.cephadm:Deploying osd.4 on smithi026 with /dev/vg_nvme/lv_3... 2024-05-14T09:26:23.118 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-14T09:26:23.374 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T09:26:23.727 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:23.727 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:23.727 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:23 smithi026 ceph-mon[36056]: from='osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-14T09:26:23.727 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:23.727 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:23.905 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:23 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:23.905 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:23 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:23.905 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:23 smithi002 ceph-mon[31557]: from='osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-14T09:26:23.905 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:23 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:23.905 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:23 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:23.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:23 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:23.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:23 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:23.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:23 smithi029 ceph-mon[36249]: from='osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-14T09:26:23.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:23 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:23.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:23 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:24.749 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:24 smithi026 ceph-mon[36056]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:24.749 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:24 smithi026 ceph-mon[36056]: from='osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-14T09:26:24.749 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:24 smithi026 ceph-mon[36056]: osdmap e24: 4 total, 3 up, 4 in 2024-05-14T09:26:24.749 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:24 smithi026 ceph-mon[36056]: from='osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T09:26:24.749 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:24 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T09:26:24.777 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:24 smithi002 ceph-mon[31557]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:24.778 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:24 smithi002 ceph-mon[31557]: from='osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-14T09:26:24.778 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:24 smithi002 ceph-mon[31557]: osdmap e24: 4 total, 3 up, 4 in 2024-05-14T09:26:24.778 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:24 smithi002 ceph-mon[31557]: from='osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T09:26:24.778 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:24 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T09:26:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:24 smithi029 ceph-mon[36249]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:24 smithi029 ceph-mon[36249]: from='osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-14T09:26:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:24 smithi029 ceph-mon[36249]: osdmap e24: 4 total, 3 up, 4 in 2024-05-14T09:26:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:24 smithi029 ceph-mon[36249]: from='osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T09:26:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:24 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T09:26:25.119 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 09:26:24 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-3[50388]: 2024-05-14T09:26:24.981+0000 7f49916fb700 -1 osd.3 0 waiting for initial osdmap 2024-05-14T09:26:25.120 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 09:26:24 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-3[50388]: 2024-05-14T09:26:24.986+0000 7f49894a8700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T09:26:25.318 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-14T09:26:25.318 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-14T09:26:25.318 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-05-14T09:26:25.318 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-05-14T09:26:25.318 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0083375 s, 1.3 GB/s 2024-05-14T09:26:25.319 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-05-14T09:26:25.859 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:25 smithi026 ceph-mon[36056]: from='osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T09:26:25.859 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:25 smithi026 ceph-mon[36056]: osdmap e25: 4 total, 3 up, 4 in 2024-05-14T09:26:25.859 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:25 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T09:26:25.859 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:25 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T09:26:25.859 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:25 smithi026 ceph-mon[36056]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:25.860 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:25 smithi026 ceph-mon[36056]: from='osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451]' entity='osd.3' 2024-05-14T09:26:25.860 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:25 smithi026 ceph-mon[36056]: osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451] boot 2024-05-14T09:26:25.860 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:25 smithi026 ceph-mon[36056]: osdmap e26: 4 total, 4 up, 4 in 2024-05-14T09:26:25.860 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:25 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T09:26:25.887 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_3 2024-05-14T09:26:25.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:25 smithi002 ceph-mon[31557]: from='osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T09:26:25.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:25 smithi002 ceph-mon[31557]: osdmap e25: 4 total, 3 up, 4 in 2024-05-14T09:26:25.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:25 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T09:26:25.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:25 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T09:26:25.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:25 smithi002 ceph-mon[31557]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:25.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:25 smithi002 ceph-mon[31557]: from='osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451]' entity='osd.3' 2024-05-14T09:26:25.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:25 smithi002 ceph-mon[31557]: osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451] boot 2024-05-14T09:26:25.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:25 smithi002 ceph-mon[31557]: osdmap e26: 4 total, 4 up, 4 in 2024-05-14T09:26:25.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:25 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T09:26:25.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:25 smithi029 ceph-mon[36249]: from='osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T09:26:25.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:25 smithi029 ceph-mon[36249]: osdmap e25: 4 total, 3 up, 4 in 2024-05-14T09:26:25.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:25 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T09:26:25.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:25 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T09:26:25.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:25 smithi029 ceph-mon[36249]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T09:26:25.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:25 smithi029 ceph-mon[36249]: from='osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451]' entity='osd.3' 2024-05-14T09:26:25.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:25 smithi029 ceph-mon[36249]: osd.3 [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451] boot 2024-05-14T09:26:25.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:25 smithi029 ceph-mon[36249]: osdmap e26: 4 total, 4 up, 4 in 2024-05-14T09:26:25.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:25 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T09:26:26.136 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T09:26:26.529 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:26 smithi026 ceph-mon[36056]: purged_snaps scrub starts 2024-05-14T09:26:26.529 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:26 smithi026 ceph-mon[36056]: purged_snaps scrub ok 2024-05-14T09:26:26.529 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:26 smithi026 ceph-mon[36056]: osdmap e27: 4 total, 4 up, 4 in 2024-05-14T09:26:26.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:26 smithi002 ceph-mon[31557]: purged_snaps scrub starts 2024-05-14T09:26:26.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:26 smithi002 ceph-mon[31557]: purged_snaps scrub ok 2024-05-14T09:26:26.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:26 smithi002 ceph-mon[31557]: osdmap e27: 4 total, 4 up, 4 in 2024-05-14T09:26:26.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:26 smithi029 ceph-mon[36249]: purged_snaps scrub starts 2024-05-14T09:26:26.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:26 smithi029 ceph-mon[36249]: purged_snaps scrub ok 2024-05-14T09:26:26.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:26 smithi029 ceph-mon[36249]: osdmap e27: 4 total, 4 up, 4 in 2024-05-14T09:26:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.913+0000 7f358ba48700 1 -- 172.21.15.26:0/3471963088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3584104d20 msgr2=0x7f358410b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:26:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.913+0000 7f358ba48700 1 --2- 172.21.15.26:0/3471963088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3584104d20 0x7f358410b5d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f357c00b0a0 tx=0x7f357c00b3b0 comp rx=0 tx=0).stop 2024-05-14T09:26:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.913+0000 7f358ba48700 1 -- 172.21.15.26:0/3471963088 shutdown_connections 2024-05-14T09:26:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.913+0000 7f358ba48700 1 --2- 172.21.15.26:0/3471963088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3584104d20 0x7f358410b5d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.913+0000 7f358ba48700 1 --2- 172.21.15.26:0/3471963088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3584103250 0x7f3584103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.913+0000 7f358ba48700 1 --2- 172.21.15.26:0/3471963088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3584104380 0x7f3584104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.913+0000 7f358ba48700 1 -- 172.21.15.26:0/3471963088 >> 172.21.15.26:0/3471963088 conn(0x7f35840fe890 msgr2=0x7f3584100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:26:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.913+0000 7f358ba48700 1 -- 172.21.15.26:0/3471963088 shutdown_connections 2024-05-14T09:26:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.913+0000 7f358ba48700 1 -- 172.21.15.26:0/3471963088 wait complete. 2024-05-14T09:26:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f358ba48700 1 Processor -- start 2024-05-14T09:26:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f358ba48700 1 -- start start 2024-05-14T09:26:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f358ba48700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3584103250 0x7f3584076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:26:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f358ba48700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3584104380 0x7f358419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:26:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f358ba48700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3584104d20 0x7f35841a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:26:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f358ba48700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f358410ecb0 con 0x7f3584104d20 2024-05-14T09:26:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f358ba48700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f358410eb30 con 0x7f3584104380 2024-05-14T09:26:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f358ba48700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f358410ee30 con 0x7f3584103250 2024-05-14T09:26:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f3588fe3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3584104380 0x7f358419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:26:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f3588fe3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3584104380 0x7f358419f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40004/0 (socket says 172.21.15.26:40004) 2024-05-14T09:26:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f3588fe3700 1 -- 172.21.15.26:0/1482276192 learned_addr learned my addr 172.21.15.26:0/1482276192 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-14T09:26:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f3589fe5700 1 --2- 172.21.15.26:0/1482276192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3584104d20 0x7f35841a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:26:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f35897e4700 1 --2- 172.21.15.26:0/1482276192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3584103250 0x7f3584076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:26:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f3588fe3700 1 -- 172.21.15.26:0/1482276192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3584103250 msgr2=0x7f3584076450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:26:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f3588fe3700 1 --2- 172.21.15.26:0/1482276192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3584103250 0x7f3584076450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f3588fe3700 1 -- 172.21.15.26:0/1482276192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3584104d20 msgr2=0x7f35841a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:26:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f3588fe3700 1 --2- 172.21.15.26:0/1482276192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3584104d20 0x7f35841a1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f3588fe3700 1 -- 172.21.15.26:0/1482276192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3584199400 con 0x7f3584104380 2024-05-14T09:26:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.914+0000 7f3589fe5700 1 --2- 172.21.15.26:0/1482276192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3584104d20 0x7f35841a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:26:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.915+0000 7f3588fe3700 1 --2- 172.21.15.26:0/1482276192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3584104380 0x7f358419f630 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f357400e9a0 tx=0x7f357400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:26:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.915+0000 7f357a7fc700 1 -- 172.21.15.26:0/1482276192 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357400cbd0 con 0x7f3584104380 2024-05-14T09:26:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.915+0000 7f357a7fc700 1 -- 172.21.15.26:0/1482276192 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f357400cd70 con 0x7f3584104380 2024-05-14T09:26:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.915+0000 7f358ba48700 1 -- 172.21.15.26:0/1482276192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35841996f0 con 0x7f3584104380 2024-05-14T09:26:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.915+0000 7f357a7fc700 1 -- 172.21.15.26:0/1482276192 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574018680 con 0x7f3584104380 2024-05-14T09:26:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.915+0000 7f358ba48700 1 -- 172.21.15.26:0/1482276192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3584199c30 con 0x7f3584104380 2024-05-14T09:26:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.916+0000 7f358ba48700 1 -- 172.21.15.26:0/1482276192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35840512c0 con 0x7f3584104380 2024-05-14T09:26:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.917+0000 7f357a7fc700 1 -- 172.21.15.26:0/1482276192 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f3574010970 con 0x7f3584104380 2024-05-14T09:26:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.918+0000 7f357a7fc700 1 --2- 172.21.15.26:0/1482276192 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f35700b2880 0x7f35700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:26:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.918+0000 7f357a7fc700 1 -- 172.21.15.26:0/1482276192 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f35740e1860 con 0x7f3584104380 2024-05-14T09:26:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.918+0000 7f35897e4700 1 --2- 172.21.15.26:0/1482276192 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f35700b2880 0x7f35700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:26:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.919+0000 7f35897e4700 1 --2- 172.21.15.26:0/1482276192 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f35700b2880 0x7f35700b4d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3580005fd0 tx=0x7f3580005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:26:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:27.922+0000 7f357a7fc700 1 -- 172.21.15.26:0/1482276192 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f35740aa700 con 0x7f3584104380 2024-05-14T09:26:28.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:28.144+0000 7f358ba48700 1 -- 172.21.15.26:0/1482276192 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3584109fc0 con 0x7f35700b2880 2024-05-14T09:26:28.144 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 163 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:28.144 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: Detected new or changed devices on smithi002 2024-05-14T09:26:28.144 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:28.144 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:28.144 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:26:28.144 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:26:28.144 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi002 to 8546M 2024-05-14T09:26:28.145 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:28.145 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: osdmap e28: 4 total, 4 up, 4 in 2024-05-14T09:26:28.145 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:28.145 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:26:28.145 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:28.145 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:26:28.145 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:28.145 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:26:28.145 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 163 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: Detected new or changed devices on smithi002 2024-05-14T09:26:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:26:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:26:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi002 to 8546M 2024-05-14T09:26:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:28.324 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: osdmap e28: 4 total, 4 up, 4 in 2024-05-14T09:26:28.324 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:28.324 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:26:28.324 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:28.324 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:26:28.324 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:28.324 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:26:28.324 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:28.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 163 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:28.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: Detected new or changed devices on smithi002 2024-05-14T09:26:28.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:28.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:28.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:26:28.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:26:28.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi002 to 8546M 2024-05-14T09:26:28.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:28.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: osdmap e28: 4 total, 4 up, 4 in 2024-05-14T09:26:28.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:28.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:26:28.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:28.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:26:28.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:28.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:26:28.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:29.106 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:29 smithi026 ceph-mon[36056]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:26:29.106 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:26:29.106 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:26:29.106 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:29 smithi002 ceph-mon[31557]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:26:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:26:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:26:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:29.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:29 smithi029 ceph-mon[36249]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:26:29.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:26:29.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:26:29.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:30.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:30 smithi002 ceph-mon[31557]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 163 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:30.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:30 smithi029 ceph-mon[36249]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 163 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:30.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:30 smithi026 ceph-mon[36056]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 163 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:31.286 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:31 smithi026 ceph-mon[36056]: from='client.? 172.21.15.26:0/2022647011' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2515448-e9c1-48a2-b270-1e5e781a2fad"}]: dispatch 2024-05-14T09:26:31.286 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:31 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2515448-e9c1-48a2-b270-1e5e781a2fad"}]: dispatch 2024-05-14T09:26:31.286 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:31 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2515448-e9c1-48a2-b270-1e5e781a2fad"}]': finished 2024-05-14T09:26:31.286 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:31 smithi026 ceph-mon[36056]: osdmap e29: 5 total, 4 up, 5 in 2024-05-14T09:26:31.286 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:31 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:31.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:31 smithi002 ceph-mon[31557]: from='client.? 172.21.15.26:0/2022647011' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2515448-e9c1-48a2-b270-1e5e781a2fad"}]: dispatch 2024-05-14T09:26:31.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:31 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2515448-e9c1-48a2-b270-1e5e781a2fad"}]: dispatch 2024-05-14T09:26:31.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:31 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2515448-e9c1-48a2-b270-1e5e781a2fad"}]': finished 2024-05-14T09:26:31.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:31 smithi002 ceph-mon[31557]: osdmap e29: 5 total, 4 up, 5 in 2024-05-14T09:26:31.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:31 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:31.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:31 smithi029 ceph-mon[36249]: from='client.? 172.21.15.26:0/2022647011' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2515448-e9c1-48a2-b270-1e5e781a2fad"}]: dispatch 2024-05-14T09:26:31.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:31 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2515448-e9c1-48a2-b270-1e5e781a2fad"}]: dispatch 2024-05-14T09:26:31.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:31 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2515448-e9c1-48a2-b270-1e5e781a2fad"}]': finished 2024-05-14T09:26:31.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:31 smithi029 ceph-mon[36249]: osdmap e29: 5 total, 4 up, 5 in 2024-05-14T09:26:31.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:31 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:32.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:32 smithi002 ceph-mon[31557]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:32.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:32 smithi002 ceph-mon[31557]: from='client.? 172.21.15.26:0/3479477755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:26:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:32 smithi029 ceph-mon[36249]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:32 smithi029 ceph-mon[36249]: from='client.? 172.21.15.26:0/3479477755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:26:32.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:32 smithi026 ceph-mon[36056]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:32.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:32 smithi026 ceph-mon[36056]: from='client.? 172.21.15.26:0/3479477755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:26:34.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:34 smithi002 ceph-mon[31557]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-14T09:26:34.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:34 smithi029 ceph-mon[36249]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-14T09:26:34.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:34 smithi026 ceph-mon[36056]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-14T09:26:36.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:36 smithi026 ceph-mon[36056]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T09:26:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:36 smithi002 ceph-mon[31557]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T09:26:36.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:36 smithi029 ceph-mon[36249]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T09:26:38.373 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:38 smithi026 ceph-mon[36056]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:26:38.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:38 smithi002 ceph-mon[31557]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:26:38.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:38 smithi029 ceph-mon[36249]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:26:39.350 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:39 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-14T09:26:39.350 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:39 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:39.350 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:39 smithi026 ceph-mon[36056]: Deploying daemon osd.4 on smithi026 2024-05-14T09:26:39.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:39 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-14T09:26:39.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:39 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:39.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:39 smithi002 ceph-mon[31557]: Deploying daemon osd.4 on smithi026 2024-05-14T09:26:39.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:39 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-14T09:26:39.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:39 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:39.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:39 smithi029 ceph-mon[36249]: Deploying daemon osd.4 on smithi026 2024-05-14T09:26:40.307 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:40 smithi026 ceph-mon[36056]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:26:40.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:40 smithi002 ceph-mon[31557]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:26:40.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:40 smithi029 ceph-mon[36249]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:26:42.282 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:42 smithi026 ceph-mon[36056]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:26:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:42 smithi002 ceph-mon[31557]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:26:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:42 smithi029 ceph-mon[36249]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:26:44.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:44 smithi002 ceph-mon[31557]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T09:26:44.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:44 smithi029 ceph-mon[36249]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T09:26:44.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:44 smithi026 ceph-mon[36056]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T09:26:45.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:45 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:45.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:45 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:45.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:45 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:26:45.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:45 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:45.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:45 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:45.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:45 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:26:45.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:45 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:45.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:45 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:45.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:45 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:26:46.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:46 smithi002 ceph-mon[31557]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:46.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:46 smithi029 ceph-mon[36249]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:46.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:46 smithi026 ceph-mon[36056]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:48.014+0000 7f357a7fc700 1 -- 172.21.15.26:0/1482276192 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3584109fc0 con 0x7f35700b2880 2024-05-14T09:26:48.015 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 4 on host 'smithi026' 2024-05-14T09:26:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:48.017+0000 7f358ba48700 1 -- 172.21.15.26:0/1482276192 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f35700b2880 msgr2=0x7f35700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:26:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:48.017+0000 7f358ba48700 1 --2- 172.21.15.26:0/1482276192 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f35700b2880 0x7f35700b4d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3580005fd0 tx=0x7f3580005f60 comp rx=0 tx=0).stop 2024-05-14T09:26:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:48.017+0000 7f358ba48700 1 -- 172.21.15.26:0/1482276192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3584104380 msgr2=0x7f358419f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:26:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:48.017+0000 7f358ba48700 1 --2- 172.21.15.26:0/1482276192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3584104380 0x7f358419f630 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f357400e9a0 tx=0x7f357400ed60 comp rx=0 tx=0).stop 2024-05-14T09:26:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:48.017+0000 7f358ba48700 1 -- 172.21.15.26:0/1482276192 shutdown_connections 2024-05-14T09:26:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:48.017+0000 7f358ba48700 1 --2- 172.21.15.26:0/1482276192 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f35700b2880 0x7f35700b4d90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:48.017+0000 7f358ba48700 1 --2- 172.21.15.26:0/1482276192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3584104d20 0x7f35841a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:48.017+0000 7f358ba48700 1 --2- 172.21.15.26:0/1482276192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3584103250 0x7f3584076450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:48.017+0000 7f358ba48700 1 --2- 172.21.15.26:0/1482276192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3584104380 0x7f358419f630 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:48.017+0000 7f358ba48700 1 -- 172.21.15.26:0/1482276192 >> 172.21.15.26:0/1482276192 conn(0x7f35840fe890 msgr2=0x7f35841075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:26:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:48.018+0000 7f358ba48700 1 -- 172.21.15.26:0/1482276192 shutdown_connections 2024-05-14T09:26:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:26:48.018+0000 7f358ba48700 1 -- 172.21.15.26:0/1482276192 wait complete. 2024-05-14T09:26:48.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:48.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:48.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:48.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:26:48.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:48.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:47 smithi026 ceph-mon[36056]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:48.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:48.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:48.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:48.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:26:48.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:48.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:47 smithi002 ceph-mon[31557]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:26:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:47 smithi029 ceph-mon[36249]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:48.669 DEBUG:teuthology.orchestra.run.smithi026:osd.4> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.4.service 2024-05-14T09:26:48.672 INFO:tasks.cephadm:Deploying osd.5 on smithi029 with /dev/vg_nvme/lv_3... 2024-05-14T09:26:48.672 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-14T09:26:48.927 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T09:26:49.290 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:49.290 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:49.290 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:26:49.290 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:48 smithi029 ceph-mon[36249]: from='osd.4 [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T09:26:49.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:49.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:49.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:26:49.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:48 smithi002 ceph-mon[31557]: from='osd.4 [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T09:26:49.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:49.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:49.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:26:49.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:48 smithi026 ceph-mon[36056]: from='osd.4 [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T09:26:50.158 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:49 smithi026 ceph-mon[36056]: from='osd.4 [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-14T09:26:50.158 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:49 smithi026 ceph-mon[36056]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:50.158 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:49 smithi026 ceph-mon[36056]: osdmap e30: 5 total, 4 up, 5 in 2024-05-14T09:26:50.158 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:49 smithi026 ceph-mon[36056]: from='osd.4 [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:26:50.159 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:49 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:50.159 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 09:26:49 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4[47536]: 2024-05-14T09:26:49.989+0000 7f0a9a873700 -1 osd.4 0 waiting for initial osdmap 2024-05-14T09:26:50.159 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 09:26:49 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4[47536]: 2024-05-14T09:26:49.995+0000 7f0a96628700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T09:26:50.242 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:49 smithi029 ceph-mon[36249]: from='osd.4 [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-14T09:26:50.243 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:49 smithi029 ceph-mon[36249]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:50.243 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:49 smithi029 ceph-mon[36249]: osdmap e30: 5 total, 4 up, 5 in 2024-05-14T09:26:50.243 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:49 smithi029 ceph-mon[36249]: from='osd.4 [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:26:50.243 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:49 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:49 smithi002 ceph-mon[31557]: from='osd.4 [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-14T09:26:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:49 smithi002 ceph-mon[31557]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:49 smithi002 ceph-mon[31557]: osdmap e30: 5 total, 4 up, 5 in 2024-05-14T09:26:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:49 smithi002 ceph-mon[31557]: from='osd.4 [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:26:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:49 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-14T09:26:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-14T09:26:50.976 INFO:teuthology.orchestra.run.smithi029.stderr: stderr: 10+0 records in 2024-05-14T09:26:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:10+0 records out 2024-05-14T09:26:50.976 INFO:teuthology.orchestra.run.smithi029.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00842546 s, 1.2 GB/s 2024-05-14T09:26:50.977 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping successful for: 2024-05-14T09:26:51.296 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:50 smithi029 ceph-mon[36249]: from='osd.4 [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-14T09:26:51.296 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:50 smithi029 ceph-mon[36249]: osdmap e31: 5 total, 4 up, 5 in 2024-05-14T09:26:51.296 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:50 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:51.296 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:50 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:51.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:50 smithi002 ceph-mon[31557]: from='osd.4 [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-14T09:26:51.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:50 smithi002 ceph-mon[31557]: osdmap e31: 5 total, 4 up, 5 in 2024-05-14T09:26:51.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:50 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:51.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:50 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:51.440 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:50 smithi026 ceph-mon[36056]: from='osd.4 [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-14T09:26:51.440 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:50 smithi026 ceph-mon[36056]: osdmap e31: 5 total, 4 up, 5 in 2024-05-14T09:26:51.440 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:50 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:51.440 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:50 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:51.649 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi029:vg_nvme/lv_3 2024-05-14T09:26:51.900 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T09:26:52.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:51 smithi026 ceph-mon[36056]: purged_snaps scrub starts 2024-05-14T09:26:52.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:51 smithi026 ceph-mon[36056]: purged_snaps scrub ok 2024-05-14T09:26:52.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:51 smithi026 ceph-mon[36056]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:52.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:52.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:51 smithi026 ceph-mon[36056]: osd.4 [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752] boot 2024-05-14T09:26:52.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:51 smithi026 ceph-mon[36056]: osdmap e32: 5 total, 5 up, 5 in 2024-05-14T09:26:52.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:52.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:52.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:52.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:26:52.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:26:52.206 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:52.206 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:52.206 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:26:52.207 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:52.272 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:51 smithi029 ceph-mon[36249]: purged_snaps scrub starts 2024-05-14T09:26:52.272 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:51 smithi029 ceph-mon[36249]: purged_snaps scrub ok 2024-05-14T09:26:52.272 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:51 smithi029 ceph-mon[36249]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:52.272 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:52.272 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:51 smithi029 ceph-mon[36249]: osd.4 [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752] boot 2024-05-14T09:26:52.272 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:51 smithi029 ceph-mon[36249]: osdmap e32: 5 total, 5 up, 5 in 2024-05-14T09:26:52.272 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:52.273 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:52.273 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:52.273 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:26:52.273 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:26:52.273 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:52.273 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:52.273 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:26:52.273 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:52.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:51 smithi002 ceph-mon[31557]: purged_snaps scrub starts 2024-05-14T09:26:52.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:51 smithi002 ceph-mon[31557]: purged_snaps scrub ok 2024-05-14T09:26:52.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:51 smithi002 ceph-mon[31557]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T09:26:52.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:52.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:51 smithi002 ceph-mon[31557]: osd.4 [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752] boot 2024-05-14T09:26:52.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:51 smithi002 ceph-mon[31557]: osdmap e32: 5 total, 5 up, 5 in 2024-05-14T09:26:52.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T09:26:52.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:52.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:52.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:26:52.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:26:52.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:52.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:52.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:26:52.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:26:53.326 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:53 smithi029 ceph-mon[36249]: Detected new or changed devices on smithi026 2024-05-14T09:26:53.327 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:53 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi026 to 8546M 2024-05-14T09:26:53.327 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:53 smithi029 ceph-mon[36249]: osdmap e33: 5 total, 5 up, 5 in 2024-05-14T09:26:53.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:53 smithi002 ceph-mon[31557]: Detected new or changed devices on smithi026 2024-05-14T09:26:53.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:53 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi026 to 8546M 2024-05-14T09:26:53.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:53 smithi002 ceph-mon[31557]: osdmap e33: 5 total, 5 up, 5 in 2024-05-14T09:26:53.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:53 smithi026 ceph-mon[36056]: Detected new or changed devices on smithi026 2024-05-14T09:26:53.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:53 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi026 to 8546M 2024-05-14T09:26:53.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:53 smithi026 ceph-mon[36056]: osdmap e33: 5 total, 5 up, 5 in 2024-05-14T09:26:53.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.586+0000 7fdc23862700 1 -- 172.21.15.29:0/3568257977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc1c105e40 msgr2=0x7fdc1c10c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:26:53.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.586+0000 7fdc23862700 1 --2- 172.21.15.29:0/3568257977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc1c105e40 0x7fdc1c10c6f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009990 tx=0x7fdc0c009ca0 comp rx=0 tx=0).stop 2024-05-14T09:26:53.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.586+0000 7fdc23862700 1 -- 172.21.15.29:0/3568257977 shutdown_connections 2024-05-14T09:26:53.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.586+0000 7fdc23862700 1 --2- 172.21.15.29:0/3568257977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc1c104370 0x7fdc1c104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:53.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.586+0000 7fdc23862700 1 --2- 172.21.15.29:0/3568257977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc1c105e40 0x7fdc1c10c6f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:53.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.586+0000 7fdc23862700 1 --2- 172.21.15.29:0/3568257977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc1c1054a0 0x7fdc1c1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:53.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.586+0000 7fdc23862700 1 -- 172.21.15.29:0/3568257977 >> 172.21.15.29:0/3568257977 conn(0x7fdc1c0ff9d0 msgr2=0x7fdc1c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:26:53.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.586+0000 7fdc23862700 1 -- 172.21.15.29:0/3568257977 shutdown_connections 2024-05-14T09:26:53.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.586+0000 7fdc23862700 1 -- 172.21.15.29:0/3568257977 wait complete. 2024-05-14T09:26:53.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.586+0000 7fdc23862700 1 Processor -- start 2024-05-14T09:26:53.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc23862700 1 -- start start 2024-05-14T09:26:53.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc23862700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc1c104370 0x7fdc1c078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:26:53.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc23862700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc1c1054a0 0x7fdc1c076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:26:53.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc23862700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc1c105e40 0x7fdc1c077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:26:53.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc23862700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c10fdd0 con 0x7fdc1c1054a0 2024-05-14T09:26:53.590 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc23862700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c10fc50 con 0x7fdc1c104370 2024-05-14T09:26:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc23862700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c10ff50 con 0x7fdc1c105e40 2024-05-14T09:26:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc21dff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc1c105e40 0x7fdc1c077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:26:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc215fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc1c104370 0x7fdc1c078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:26:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc21dff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc1c105e40 0x7fdc1c077160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:43184/0 (socket says 172.21.15.29:43184) 2024-05-14T09:26:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc21dff700 1 -- 172.21.15.29:0/2355457153 learned_addr learned my addr 172.21.15.29:0/2355457153 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-05-14T09:26:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc20dfd700 1 --2- 172.21.15.29:0/2355457153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc1c1054a0 0x7fdc1c076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:26:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc21dff700 1 -- 172.21.15.29:0/2355457153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc1c104370 msgr2=0x7fdc1c078570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:26:53.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc21dff700 1 --2- 172.21.15.29:0/2355457153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc1c104370 0x7fdc1c078570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:53.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc21dff700 1 -- 172.21.15.29:0/2355457153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc1c1054a0 msgr2=0x7fdc1c076c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:26:53.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc21dff700 1 --2- 172.21.15.29:0/2355457153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc1c1054a0 0x7fdc1c076c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:26:53.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc21dff700 1 -- 172.21.15.29:0/2355457153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c0778c0 con 0x7fdc1c105e40 2024-05-14T09:26:53.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.587+0000 7fdc21dff700 1 --2- 172.21.15.29:0/2355457153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc1c105e40 0x7fdc1c077160 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0059d0 tx=0x7fdc0c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:26:53.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.588+0000 7fdc1a7fc700 1 -- 172.21.15.29:0/2355457153 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c027070 con 0x7fdc1c105e40 2024-05-14T09:26:53.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.588+0000 7fdc23862700 1 -- 172.21.15.29:0/2355457153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1c1a4930 con 0x7fdc1c105e40 2024-05-14T09:26:53.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.588+0000 7fdc1a7fc700 1 -- 172.21.15.29:0/2355457153 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0c022470 con 0x7fdc1c105e40 2024-05-14T09:26:53.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.588+0000 7fdc23862700 1 -- 172.21.15.29:0/2355457153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc1c1a4d20 con 0x7fdc1c105e40 2024-05-14T09:26:53.593 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.588+0000 7fdc1a7fc700 1 -- 172.21.15.29:0/2355457153 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c017650 con 0x7fdc1c105e40 2024-05-14T09:26:53.594 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.589+0000 7fdc23862700 1 -- 172.21.15.29:0/2355457153 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc1c0512c0 con 0x7fdc1c105e40 2024-05-14T09:26:53.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.590+0000 7fdc1a7fc700 1 -- 172.21.15.29:0/2355457153 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fdc0c02a070 con 0x7fdc1c105e40 2024-05-14T09:26:53.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.591+0000 7fdc1a7fc700 1 --2- 172.21.15.29:0/2355457153 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdc040b2900 0x7fdc040b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:26:53.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.591+0000 7fdc1a7fc700 1 -- 172.21.15.29:0/2355457153 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7fdc0c0e2390 con 0x7fdc1c105e40 2024-05-14T09:26:53.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.591+0000 7fdc215fe700 1 --2- 172.21.15.29:0/2355457153 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdc040b2900 0x7fdc040b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:26:53.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.592+0000 7fdc215fe700 1 --2- 172.21.15.29:0/2355457153 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdc040b2900 0x7fdc040b4e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009550 tx=0x7fdc140091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:26:53.595 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.595+0000 7fdc1a7fc700 1 -- 172.21.15.29:0/2355457153 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdc0c013070 con 0x7fdc1c105e40 2024-05-14T09:26:53.820 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:26:53.819+0000 7fdc23862700 1 -- 172.21.15.29:0/2355457153 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdc1c02ea00 con 0x7fdc040b2900 2024-05-14T09:26:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:54 smithi002 ceph-mon[31557]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:26:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:54 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:26:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:54 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:26:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:54 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:54.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:54 smithi029 ceph-mon[36249]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:26:54.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:54 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:26:54.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:54 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:26:54.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:54 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:54.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:54 smithi026 ceph-mon[36056]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:26:54.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:54 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:26:54.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:54 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:26:54.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:54 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:26:55.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:55 smithi002 ceph-mon[31557]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:26:55.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:55 smithi029 ceph-mon[36249]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:26:55.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:55 smithi026 ceph-mon[36056]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:26:56.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:56 smithi002 ceph-mon[31557]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:26:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:56 smithi029 ceph-mon[36249]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:26:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:56 smithi026 ceph-mon[36056]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:26:57.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:57 smithi002 ceph-mon[31557]: from='client.? 172.21.15.29:0/448506541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20030857-a1e6-4b0a-b186-1cfa44f2e4ef"}]: dispatch 2024-05-14T09:26:57.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:57 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20030857-a1e6-4b0a-b186-1cfa44f2e4ef"}]: dispatch 2024-05-14T09:26:57.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:57 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20030857-a1e6-4b0a-b186-1cfa44f2e4ef"}]': finished 2024-05-14T09:26:57.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:57 smithi002 ceph-mon[31557]: osdmap e34: 6 total, 5 up, 6 in 2024-05-14T09:26:57.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T09:26:57.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:57 smithi029 ceph-mon[36249]: from='client.? 172.21.15.29:0/448506541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20030857-a1e6-4b0a-b186-1cfa44f2e4ef"}]: dispatch 2024-05-14T09:26:57.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:57 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20030857-a1e6-4b0a-b186-1cfa44f2e4ef"}]: dispatch 2024-05-14T09:26:57.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:57 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20030857-a1e6-4b0a-b186-1cfa44f2e4ef"}]': finished 2024-05-14T09:26:57.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:57 smithi029 ceph-mon[36249]: osdmap e34: 6 total, 5 up, 6 in 2024-05-14T09:26:57.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T09:26:57.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:57 smithi026 ceph-mon[36056]: from='client.? 172.21.15.29:0/448506541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20030857-a1e6-4b0a-b186-1cfa44f2e4ef"}]: dispatch 2024-05-14T09:26:57.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:57 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20030857-a1e6-4b0a-b186-1cfa44f2e4ef"}]: dispatch 2024-05-14T09:26:57.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:57 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20030857-a1e6-4b0a-b186-1cfa44f2e4ef"}]': finished 2024-05-14T09:26:57.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:57 smithi026 ceph-mon[36056]: osdmap e34: 6 total, 5 up, 6 in 2024-05-14T09:26:57.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T09:26:58.322 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:58 smithi002 ceph-mon[31557]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:26:58.322 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:26:58 smithi002 ceph-mon[31557]: from='client.? 172.21.15.29:0/982168028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:26:58.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:58 smithi029 ceph-mon[36249]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:26:58.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:26:58 smithi029 ceph-mon[36249]: from='client.? 172.21.15.29:0/982168028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:26:58.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:58 smithi026 ceph-mon[36056]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:26:58.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:26:58 smithi026 ceph-mon[36056]: from='client.? 172.21.15.29:0/982168028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:27:00.411 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:00 smithi029 ceph-mon[36249]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:00.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:00 smithi002 ceph-mon[31557]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:00.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:00 smithi026 ceph-mon[36056]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:02.364 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:02 smithi029 ceph-mon[36249]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:02.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:02 smithi002 ceph-mon[31557]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:02.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:02 smithi026 ceph-mon[36056]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:04.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:04 smithi029 ceph-mon[36249]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:04.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:04 smithi002 ceph-mon[31557]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:04.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:04 smithi026 ceph-mon[36056]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:05.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-14T09:27:05.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:05.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:05 smithi029 ceph-mon[36249]: Deploying daemon osd.5 on smithi029 2024-05-14T09:27:05.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-14T09:27:05.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:05.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:05 smithi002 ceph-mon[31557]: Deploying daemon osd.5 on smithi029 2024-05-14T09:27:05.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-14T09:27:05.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:05.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:05 smithi026 ceph-mon[36056]: Deploying daemon osd.5 on smithi029 2024-05-14T09:27:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:06 smithi029 ceph-mon[36249]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:06.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:06 smithi002 ceph-mon[31557]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:06.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:06 smithi026 ceph-mon[36056]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:08.273 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:08 smithi029 ceph-mon[36249]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:08.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:08 smithi002 ceph-mon[31557]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:08.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:08 smithi026 ceph-mon[36056]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:10.177 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:10 smithi029 ceph-mon[36249]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:10.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:10 smithi002 ceph-mon[31557]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:10.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:10 smithi026 ceph-mon[36056]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:11.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:11 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:11.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:11 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:27:11.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:11 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:11.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:11 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:11.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:11 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:27:11.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:11 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:11.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:11 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:11.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:11 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:27:11.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:11 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:12.381 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:12 smithi029 ceph-mon[36249]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:12.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:12 smithi002 ceph-mon[31557]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:12.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:12 smithi026 ceph-mon[36056]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:13.589 INFO:teuthology.orchestra.run.smithi029.stdout:Created osd(s) 5 on host 'smithi029' 2024-05-14T09:27:13.589 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:27:13.589+0000 7fdc1a7fc700 1 -- 172.21.15.29:0/2355457153 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdc1c02ea00 con 0x7fdc040b2900 2024-05-14T09:27:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:27:13.592+0000 7fdc23862700 1 -- 172.21.15.29:0/2355457153 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdc040b2900 msgr2=0x7fdc040b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:27:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:27:13.592+0000 7fdc23862700 1 --2- 172.21.15.29:0/2355457153 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdc040b2900 0x7fdc040b4e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009550 tx=0x7fdc140091b0 comp rx=0 tx=0).stop 2024-05-14T09:27:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:27:13.592+0000 7fdc23862700 1 -- 172.21.15.29:0/2355457153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc1c105e40 msgr2=0x7fdc1c077160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:27:13.591 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:27:13.592+0000 7fdc23862700 1 --2- 172.21.15.29:0/2355457153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc1c105e40 0x7fdc1c077160 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0059d0 tx=0x7fdc0c0047b0 comp rx=0 tx=0).stop 2024-05-14T09:27:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:27:13.592+0000 7fdc23862700 1 -- 172.21.15.29:0/2355457153 shutdown_connections 2024-05-14T09:27:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:27:13.592+0000 7fdc23862700 1 --2- 172.21.15.29:0/2355457153 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdc040b2900 0x7fdc040b4e10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:27:13.592+0000 7fdc23862700 1 --2- 172.21.15.29:0/2355457153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc1c104370 0x7fdc1c078570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:27:13.592+0000 7fdc23862700 1 --2- 172.21.15.29:0/2355457153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc1c105e40 0x7fdc1c077160 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:27:13.592+0000 7fdc23862700 1 --2- 172.21.15.29:0/2355457153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc1c1054a0 0x7fdc1c076c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:27:13.592+0000 7fdc23862700 1 -- 172.21.15.29:0/2355457153 >> 172.21.15.29:0/2355457153 conn(0x7fdc1c0ff9d0 msgr2=0x7fdc1c1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:27:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:27:13.592+0000 7fdc23862700 1 -- 172.21.15.29:0/2355457153 shutdown_connections 2024-05-14T09:27:13.592 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:27:13.592+0000 7fdc23862700 1 -- 172.21.15.29:0/2355457153 wait complete. 2024-05-14T09:27:14.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:14 smithi029 ceph-mon[36249]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:14.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:14.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:14.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:14 smithi029 ceph-mon[36249]: from='osd.5 [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T09:27:14.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:14 smithi029 ceph-mon[36249]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T09:27:14.265 DEBUG:teuthology.orchestra.run.smithi029:osd.5> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.5.service 2024-05-14T09:27:14.268 INFO:tasks.cephadm:Deploying osd.6 on smithi002 with /dev/vg_nvme/lv_2... 2024-05-14T09:27:14.268 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-14T09:27:14.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:14 smithi002 ceph-mon[31557]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:14.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:14 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:14.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:14 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:14.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:14 smithi002 ceph-mon[31557]: from='osd.5 [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T09:27:14.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:14 smithi002 ceph-mon[31557]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T09:27:14.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:14 smithi026 ceph-mon[36056]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:14.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:14 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:14.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:14 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:14.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:14 smithi026 ceph-mon[36056]: from='osd.5 [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T09:27:14.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:14 smithi026 ceph-mon[36056]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T09:27:14.544 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:27:15.657 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:15 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:15.657 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:15 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:15.657 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:15 smithi002 ceph-mon[31557]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-14T09:27:15.657 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:15 smithi002 ceph-mon[31557]: osdmap e35: 6 total, 5 up, 6 in 2024-05-14T09:27:15.657 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:15 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T09:27:15.657 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:15 smithi002 ceph-mon[31557]: from='osd.5 [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:27:15.657 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:15 smithi002 ceph-mon[31557]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:27:15.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:15 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:15.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:15 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:15.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:15 smithi029 ceph-mon[36249]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-14T09:27:15.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:15 smithi029 ceph-mon[36249]: osdmap e35: 6 total, 5 up, 6 in 2024-05-14T09:27:15.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:15 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T09:27:15.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:15 smithi029 ceph-mon[36249]: from='osd.5 [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:27:15.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:15 smithi029 ceph-mon[36249]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:27:15.688 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 09:27:15 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T09:27:15.553+0000 7f2c9b077700 -1 osd.5 0 waiting for initial osdmap 2024-05-14T09:27:15.688 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 09:27:15 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T09:27:15.561+0000 7f2c96e2c700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T09:27:15.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:15 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:15.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:15 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:15.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:15 smithi026 ceph-mon[36056]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-14T09:27:15.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:15 smithi026 ceph-mon[36056]: osdmap e35: 6 total, 5 up, 6 in 2024-05-14T09:27:15.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:15 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T09:27:15.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:15 smithi026 ceph-mon[36056]: from='osd.5 [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:27:15.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:15 smithi026 ceph-mon[36056]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:27:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-14T09:27:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-14T09:27:16.547 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2024-05-14T09:27:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2024-05-14T09:27:16.548 INFO:teuthology.orchestra.run.smithi002.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00832293 s, 1.3 GB/s 2024-05-14T09:27:16.548 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2024-05-14T09:27:16.670 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:16 smithi002 ceph-mon[31557]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:16.670 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:16 smithi002 ceph-mon[31557]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-05-14T09:27:16.670 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:16 smithi002 ceph-mon[31557]: osdmap e36: 6 total, 5 up, 6 in 2024-05-14T09:27:16.670 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:16 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T09:27:16.670 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:16 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T09:27:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:16 smithi029 ceph-mon[36249]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:16 smithi029 ceph-mon[36249]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-05-14T09:27:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:16 smithi029 ceph-mon[36249]: osdmap e36: 6 total, 5 up, 6 in 2024-05-14T09:27:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:16 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T09:27:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:16 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T09:27:16.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:16 smithi026 ceph-mon[36056]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:16.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:16 smithi026 ceph-mon[36056]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-05-14T09:27:16.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:16 smithi026 ceph-mon[36056]: osdmap e36: 6 total, 5 up, 6 in 2024-05-14T09:27:16.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:16 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T09:27:16.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:16 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T09:27:17.269 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi002:vg_nvme/lv_2 2024-05-14T09:27:17.517 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:27:17.785 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:17 smithi029 ceph-mon[36249]: purged_snaps scrub starts 2024-05-14T09:27:17.786 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:17 smithi029 ceph-mon[36249]: purged_snaps scrub ok 2024-05-14T09:27:17.786 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:17 smithi029 ceph-mon[36249]: osd.5 [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093] boot 2024-05-14T09:27:17.786 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:17 smithi029 ceph-mon[36249]: osdmap e37: 6 total, 6 up, 6 in 2024-05-14T09:27:17.786 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:17 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T09:27:17.786 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:17 smithi029 ceph-mon[36249]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:17.867 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:17 smithi002 ceph-mon[31557]: purged_snaps scrub starts 2024-05-14T09:27:17.868 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:17 smithi002 ceph-mon[31557]: purged_snaps scrub ok 2024-05-14T09:27:17.868 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:17 smithi002 ceph-mon[31557]: osd.5 [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093] boot 2024-05-14T09:27:17.868 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:17 smithi002 ceph-mon[31557]: osdmap e37: 6 total, 6 up, 6 in 2024-05-14T09:27:17.868 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:17 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T09:27:17.868 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:17 smithi002 ceph-mon[31557]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:17.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:17 smithi026 ceph-mon[36056]: purged_snaps scrub starts 2024-05-14T09:27:17.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:17 smithi026 ceph-mon[36056]: purged_snaps scrub ok 2024-05-14T09:27:17.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:17 smithi026 ceph-mon[36056]: osd.5 [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093] boot 2024-05-14T09:27:17.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:17 smithi026 ceph-mon[36056]: osdmap e37: 6 total, 6 up, 6 in 2024-05-14T09:27:17.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:17 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T09:27:17.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:17 smithi026 ceph-mon[36056]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-05-14T09:27:18.697 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:18 smithi002 ceph-mon[31557]: osdmap e38: 6 total, 6 up, 6 in 2024-05-14T09:27:18.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:18 smithi002 ceph-mon[31557]: Detected new or changed devices on smithi029 2024-05-14T09:27:18.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:18 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:18.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:18 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:18.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:18 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:27:18.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:18 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:27:18.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:18 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi029 to 8546M 2024-05-14T09:27:18.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:18 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:18.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:18 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:18.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:18 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:27:18.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:18 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:18.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:18 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:27:18.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:18 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:18.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:18 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:27:18.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:18 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:18.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:18 smithi029 ceph-mon[36249]: osdmap e38: 6 total, 6 up, 6 in 2024-05-14T09:27:18.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:18 smithi029 ceph-mon[36249]: Detected new or changed devices on smithi029 2024-05-14T09:27:18.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:18 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:18.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:18 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:18.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:18 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:27:18.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:18 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:27:18.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:18 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi029 to 8546M 2024-05-14T09:27:18.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:18 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:18.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:18 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:18.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:18 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:27:18.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:18 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:18.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:18 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:27:18.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:18 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:18.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:18 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:27:18.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:18 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:18 smithi026 ceph-mon[36056]: osdmap e38: 6 total, 6 up, 6 in 2024-05-14T09:27:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:18 smithi026 ceph-mon[36056]: Detected new or changed devices on smithi029 2024-05-14T09:27:18.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:18 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:18.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:18 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:18.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:18 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:27:18.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:18 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:27:18.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:18 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi029 to 8546M 2024-05-14T09:27:18.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:18 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:18.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:18 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:18.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:18 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:27:18.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:18 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:18.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:18 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:27:18.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:18 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:18.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:18 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:27:18.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:18 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:19.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.185+0000 7f969c3d9700 1 -- 172.21.15.2:0/3263968930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f969410a110 msgr2=0x7f969410c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:27:19.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.185+0000 7f969c3d9700 1 --2- 172.21.15.2:0/3263968930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f969410a110 0x7f969410c530 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f968c00b0a0 tx=0x7f968c00b3b0 comp rx=0 tx=0).stop 2024-05-14T09:27:19.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.185+0000 7f969c3d9700 1 -- 172.21.15.2:0/3263968930 shutdown_connections 2024-05-14T09:27:19.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.185+0000 7f969c3d9700 1 --2- 172.21.15.2:0/3263968930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9694104e50 0x7f9694107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:19.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.185+0000 7f969c3d9700 1 --2- 172.21.15.2:0/3263968930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f969410a110 0x7f969410c530 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:19.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.185+0000 7f969c3d9700 1 --2- 172.21.15.2:0/3263968930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f96941077b0 0x7f9694109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:19.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.185+0000 7f969c3d9700 1 -- 172.21.15.2:0/3263968930 >> 172.21.15.2:0/3263968930 conn(0x7f96940fe890 msgr2=0x7f9694100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:27:19.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.185+0000 7f969c3d9700 1 -- 172.21.15.2:0/3263968930 shutdown_connections 2024-05-14T09:27:19.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.185+0000 7f969c3d9700 1 -- 172.21.15.2:0/3263968930 wait complete. 2024-05-14T09:27:19.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f969c3d9700 1 Processor -- start 2024-05-14T09:27:19.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f969c3d9700 1 -- start start 2024-05-14T09:27:19.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f969c3d9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9694104e50 0x7f96941046b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:27:19.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f969c3d9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96941077b0 0x7f9694102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:27:19.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f969c3d9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f969410a110 0x7f96941032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:27:19.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f969c3d9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f969410fdd0 con 0x7f96941077b0 2024-05-14T09:27:19.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f969c3d9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f969410fc50 con 0x7f969410a110 2024-05-14T09:27:19.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f969c3d9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f969410ff50 con 0x7f9694104e50 2024-05-14T09:27:19.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f9699974700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96941077b0 0x7f9694102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:27:19.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f9699974700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96941077b0 0x7f9694102d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51672/0 (socket says 172.21.15.2:51672) 2024-05-14T09:27:19.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f9699974700 1 -- 172.21.15.2:0/1547072749 learned_addr learned my addr 172.21.15.2:0/1547072749 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:27:19.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f969a976700 1 --2- 172.21.15.2:0/1547072749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f969410a110 0x7f96941032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:27:19.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f969a175700 1 --2- 172.21.15.2:0/1547072749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9694104e50 0x7f96941046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:27:19.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f9699974700 1 -- 172.21.15.2:0/1547072749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9694104e50 msgr2=0x7f96941046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:27:19.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f9699974700 1 --2- 172.21.15.2:0/1547072749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9694104e50 0x7f96941046b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:19.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f9699974700 1 -- 172.21.15.2:0/1547072749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f969410a110 msgr2=0x7f96941032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:27:19.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f9699974700 1 --2- 172.21.15.2:0/1547072749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f969410a110 0x7f96941032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:19.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.186+0000 7f9699974700 1 -- 172.21.15.2:0/1547072749 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9694103a00 con 0x7f96941077b0 2024-05-14T09:27:19.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.187+0000 7f9699974700 1 --2- 172.21.15.2:0/1547072749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96941077b0 0x7f9694102d60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f969000d950 tx=0x7f969000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:27:19.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.187+0000 7f96877fe700 1 -- 172.21.15.2:0/1547072749 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690016070 con 0x7f96941077b0 2024-05-14T09:27:19.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.187+0000 7f96877fe700 1 -- 172.21.15.2:0/1547072749 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f969000fd40 con 0x7f96941077b0 2024-05-14T09:27:19.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.187+0000 7f969c3d9700 1 -- 172.21.15.2:0/1547072749 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96941a4960 con 0x7f96941077b0 2024-05-14T09:27:19.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.187+0000 7f96877fe700 1 -- 172.21.15.2:0/1547072749 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969000e3b0 con 0x7f96941077b0 2024-05-14T09:27:19.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.187+0000 7f969c3d9700 1 -- 172.21.15.2:0/1547072749 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96941a4e90 con 0x7f96941077b0 2024-05-14T09:27:19.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.188+0000 7f969c3d9700 1 -- 172.21.15.2:0/1547072749 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f969402e5f0 con 0x7f96941077b0 2024-05-14T09:27:19.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.190+0000 7f96877fe700 1 -- 172.21.15.2:0/1547072749 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f9690019070 con 0x7f96941077b0 2024-05-14T09:27:19.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.190+0000 7f96877fe700 1 --2- 172.21.15.2:0/1547072749 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f96800b2900 0x7f96800b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:27:19.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.190+0000 7f969a175700 1 --2- 172.21.15.2:0/1547072749 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f96800b2900 0x7f96800b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:27:19.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.190+0000 7f96877fe700 1 -- 172.21.15.2:0/1547072749 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4533+0+0 (secure 0 0 0) 0x7f96900e0df0 con 0x7f96941077b0 2024-05-14T09:27:19.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.191+0000 7f969a175700 1 --2- 172.21.15.2:0/1547072749 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f96800b2900 0x7f96800b4e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9694076410 tx=0x7f968800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:27:19.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.194+0000 7f96877fe700 1 -- 172.21.15.2:0/1547072749 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f96900a9930 con 0x7f96941077b0 2024-05-14T09:27:19.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:19.418+0000 7f969c3d9700 1 -- 172.21.15.2:0/1547072749 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9694063c00 con 0x7f96800b2900 2024-05-14T09:27:19.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:19 smithi002 ceph-mon[31557]: osdmap e39: 6 total, 6 up, 6 in 2024-05-14T09:27:19.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:19 smithi002 ceph-mon[31557]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:19.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:19 smithi002 ceph-mon[31557]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:27:19.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:19 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:27:19.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:19 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:27:19.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:19 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:19.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:19 smithi029 ceph-mon[36249]: osdmap e39: 6 total, 6 up, 6 in 2024-05-14T09:27:19.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:19 smithi029 ceph-mon[36249]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:19.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:19 smithi029 ceph-mon[36249]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:27:19.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:19 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:27:19.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:19 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:27:19.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:19 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:19.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:19 smithi026 ceph-mon[36056]: osdmap e39: 6 total, 6 up, 6 in 2024-05-14T09:27:19.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:19 smithi026 ceph-mon[36056]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:19.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:19 smithi026 ceph-mon[36056]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:27:19.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:19 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:27:19.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:19 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:27:19.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:19 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:22.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:22 smithi002 ceph-mon[31557]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:22.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:22 smithi029 ceph-mon[36249]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:22.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:22 smithi026 ceph-mon[36056]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:23.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:23 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1205723615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d757c72a-29f0-4f14-9f13-a2e61eb5e6a2"}]: dispatch 2024-05-14T09:27:23.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:23 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1205723615' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d757c72a-29f0-4f14-9f13-a2e61eb5e6a2"}]': finished 2024-05-14T09:27:23.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:23 smithi002 ceph-mon[31557]: osdmap e40: 7 total, 6 up, 7 in 2024-05-14T09:27:23.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:23 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T09:27:23.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:23 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3457390541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:27:23.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:23 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1205723615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d757c72a-29f0-4f14-9f13-a2e61eb5e6a2"}]: dispatch 2024-05-14T09:27:23.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:23 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1205723615' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d757c72a-29f0-4f14-9f13-a2e61eb5e6a2"}]': finished 2024-05-14T09:27:23.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:23 smithi029 ceph-mon[36249]: osdmap e40: 7 total, 6 up, 7 in 2024-05-14T09:27:23.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:23 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T09:27:23.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:23 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3457390541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:27:23.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:23 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1205723615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d757c72a-29f0-4f14-9f13-a2e61eb5e6a2"}]: dispatch 2024-05-14T09:27:23.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:23 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1205723615' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d757c72a-29f0-4f14-9f13-a2e61eb5e6a2"}]': finished 2024-05-14T09:27:23.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:23 smithi026 ceph-mon[36056]: osdmap e40: 7 total, 6 up, 7 in 2024-05-14T09:27:23.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T09:27:23.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:23 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3457390541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:27:24.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:24 smithi002 ceph-mon[31557]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T09:27:24.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:24 smithi029 ceph-mon[36249]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T09:27:24.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:24 smithi026 ceph-mon[36056]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T09:27:26.356 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:26 smithi002 ceph-mon[31557]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-14T09:27:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:26 smithi029 ceph-mon[36249]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-14T09:27:26.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:26 smithi026 ceph-mon[36056]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-14T09:27:28.636 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:28 smithi002 ceph-mon[31557]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-14T09:27:28.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:28 smithi029 ceph-mon[36249]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-14T09:27:28.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:28 smithi026 ceph-mon[36056]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-14T09:27:30.624 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:30 smithi002 ceph-mon[31557]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:27:30.624 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-14T09:27:30.624 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:30 smithi029 ceph-mon[36249]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:27:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-14T09:27:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:30 smithi026 ceph-mon[36056]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:27:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-14T09:27:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:31.627 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:31 smithi002 ceph-mon[31557]: Deploying daemon osd.6 on smithi002 2024-05-14T09:27:31.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:31 smithi029 ceph-mon[36249]: Deploying daemon osd.6 on smithi002 2024-05-14T09:27:31.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:31 smithi026 ceph-mon[36056]: Deploying daemon osd.6 on smithi002 2024-05-14T09:27:32.454 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:32 smithi002 ceph-mon[31557]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:27:32.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:32 smithi029 ceph-mon[36249]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:27:32.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:32 smithi026 ceph-mon[36056]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:27:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:34 smithi002 ceph-mon[31557]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:34 smithi029 ceph-mon[36249]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:34 smithi026 ceph-mon[36056]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:36 smithi002 ceph-mon[31557]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:27:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:36 smithi029 ceph-mon[36249]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:27:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:36 smithi026 ceph-mon[36056]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:27:38.543 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:38 smithi002 ceph-mon[31557]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:38.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:38 smithi029 ceph-mon[36249]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:38.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:38 smithi026 ceph-mon[36056]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:39.783 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 6 on host 'smithi002' 2024-05-14T09:27:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:39.783+0000 7f96877fe700 1 -- 172.21.15.2:0/1547072749 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9694063c00 con 0x7f96800b2900 2024-05-14T09:27:39.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:39.786+0000 7f969c3d9700 1 -- 172.21.15.2:0/1547072749 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f96800b2900 msgr2=0x7f96800b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:27:39.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:39.786+0000 7f969c3d9700 1 --2- 172.21.15.2:0/1547072749 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f96800b2900 0x7f96800b4e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9694076410 tx=0x7f968800a300 comp rx=0 tx=0).stop 2024-05-14T09:27:39.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:39.786+0000 7f969c3d9700 1 -- 172.21.15.2:0/1547072749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96941077b0 msgr2=0x7f9694102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:27:39.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:39.786+0000 7f969c3d9700 1 --2- 172.21.15.2:0/1547072749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96941077b0 0x7f9694102d60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f969000d950 tx=0x7f969000dc60 comp rx=0 tx=0).stop 2024-05-14T09:27:39.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:39.786+0000 7f969c3d9700 1 -- 172.21.15.2:0/1547072749 shutdown_connections 2024-05-14T09:27:39.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:39.786+0000 7f969c3d9700 1 --2- 172.21.15.2:0/1547072749 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f96800b2900 0x7f96800b4e10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:39.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:39.786+0000 7f969c3d9700 1 --2- 172.21.15.2:0/1547072749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f969410a110 0x7f96941032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:39.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:39.786+0000 7f969c3d9700 1 --2- 172.21.15.2:0/1547072749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9694104e50 0x7f96941046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:39.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:39.786+0000 7f969c3d9700 1 --2- 172.21.15.2:0/1547072749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96941077b0 0x7f9694102d60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:39.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:39.786+0000 7f969c3d9700 1 -- 172.21.15.2:0/1547072749 >> 172.21.15.2:0/1547072749 conn(0x7f96940fe890 msgr2=0x7f9694105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:27:39.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:39.786+0000 7f969c3d9700 1 -- 172.21.15.2:0/1547072749 shutdown_connections 2024-05-14T09:27:39.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:27:39.786+0000 7f969c3d9700 1 -- 172.21.15.2:0/1547072749 wait complete. 2024-05-14T09:27:40.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:40 smithi002 ceph-mon[31557]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:40.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:40.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:40.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:40 smithi002 ceph-mon[31557]: from='osd.6 [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-14T09:27:40.501 DEBUG:teuthology.orchestra.run.smithi002:osd.6> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.6.service 2024-05-14T09:27:40.504 INFO:tasks.cephadm:Deploying osd.7 on smithi026 with /dev/vg_nvme/lv_2... 2024-05-14T09:27:40.504 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-14T09:27:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:40 smithi029 ceph-mon[36249]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:40 smithi029 ceph-mon[36249]: from='osd.6 [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-14T09:27:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:40 smithi026 ceph-mon[36056]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:40 smithi026 ceph-mon[36056]: from='osd.6 [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-14T09:27:40.761 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T09:27:41.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:41.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:41.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:41 smithi002 ceph-mon[31557]: from='osd.6 [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-14T09:27:41.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:41 smithi002 ceph-mon[31557]: osdmap e41: 7 total, 6 up, 7 in 2024-05-14T09:27:41.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:41 smithi002 ceph-mon[31557]: from='osd.6 [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T09:27:41.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T09:27:41.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:41 smithi002 ceph-mon[31557]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:41.922 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 09:27:41 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T09:27:41.791+0000 7fb3bd3c6700 -1 osd.6 0 waiting for initial osdmap 2024-05-14T09:27:41.922 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 09:27:41 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T09:27:41.801+0000 7fb3b917b700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T09:27:41.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:41.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:41.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:41 smithi029 ceph-mon[36249]: from='osd.6 [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-14T09:27:41.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:41 smithi029 ceph-mon[36249]: osdmap e41: 7 total, 6 up, 7 in 2024-05-14T09:27:41.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:41 smithi029 ceph-mon[36249]: from='osd.6 [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T09:27:41.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T09:27:41.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:41 smithi029 ceph-mon[36249]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:41.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:41.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:41.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:41 smithi026 ceph-mon[36056]: from='osd.6 [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-14T09:27:41.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:41 smithi026 ceph-mon[36056]: osdmap e41: 7 total, 6 up, 7 in 2024-05-14T09:27:41.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:41 smithi026 ceph-mon[36056]: from='osd.6 [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T09:27:41.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T09:27:41.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:41 smithi026 ceph-mon[36056]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-14T09:27:42.837 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-14T09:27:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-14T09:27:42.838 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-05-14T09:27:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-05-14T09:27:42.838 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00827846 s, 1.3 GB/s 2024-05-14T09:27:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-05-14T09:27:43.033 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:42 smithi002 ceph-mon[31557]: from='osd.6 [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T09:27:43.034 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:42 smithi002 ceph-mon[31557]: osdmap e42: 7 total, 6 up, 7 in 2024-05-14T09:27:43.034 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:42 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T09:27:43.034 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:42 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T09:27:43.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:42 smithi029 ceph-mon[36249]: from='osd.6 [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T09:27:43.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:42 smithi029 ceph-mon[36249]: osdmap e42: 7 total, 6 up, 7 in 2024-05-14T09:27:43.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:42 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T09:27:43.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:42 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T09:27:43.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:42 smithi026 ceph-mon[36056]: from='osd.6 [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T09:27:43.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:42 smithi026 ceph-mon[36056]: osdmap e42: 7 total, 6 up, 7 in 2024-05-14T09:27:43.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:42 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T09:27:43.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:42 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T09:27:43.579 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_2 2024-05-14T09:27:43.796 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:43 smithi026 ceph-mon[36056]: purged_snaps scrub starts 2024-05-14T09:27:43.796 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:43 smithi026 ceph-mon[36056]: purged_snaps scrub ok 2024-05-14T09:27:43.796 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:43 smithi026 ceph-mon[36056]: osd.6 [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595] boot 2024-05-14T09:27:43.796 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:43 smithi026 ceph-mon[36056]: osdmap e43: 7 total, 7 up, 7 in 2024-05-14T09:27:43.796 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:43 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T09:27:43.844 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T09:27:43.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:43 smithi002 ceph-mon[31557]: purged_snaps scrub starts 2024-05-14T09:27:43.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:43 smithi002 ceph-mon[31557]: purged_snaps scrub ok 2024-05-14T09:27:43.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:43 smithi002 ceph-mon[31557]: osd.6 [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595] boot 2024-05-14T09:27:43.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:43 smithi002 ceph-mon[31557]: osdmap e43: 7 total, 7 up, 7 in 2024-05-14T09:27:43.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:43 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T09:27:43.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:43 smithi002 ceph-mon[31557]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:44.061 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:43 smithi026 ceph-mon[36056]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:44.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:43 smithi029 ceph-mon[36249]: purged_snaps scrub starts 2024-05-14T09:27:44.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:43 smithi029 ceph-mon[36249]: purged_snaps scrub ok 2024-05-14T09:27:44.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:43 smithi029 ceph-mon[36249]: osd.6 [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595] boot 2024-05-14T09:27:44.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:43 smithi029 ceph-mon[36249]: osdmap e43: 7 total, 7 up, 7 in 2024-05-14T09:27:44.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:43 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T09:27:44.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:43 smithi029 ceph-mon[36249]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: osdmap e44: 7 total, 7 up, 7 in 2024-05-14T09:27:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: Detected new or changed devices on smithi002 2024-05-14T09:27:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:27:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:27:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:27:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi002 to 5697M 2024-05-14T09:27:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:27:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:27:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:27:44.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:45.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: osdmap e44: 7 total, 7 up, 7 in 2024-05-14T09:27:45.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: Detected new or changed devices on smithi002 2024-05-14T09:27:45.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:45.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:45.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:27:45.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:27:45.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:27:45.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi002 to 5697M 2024-05-14T09:27:45.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:45.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:45.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:27:45.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:45.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:27:45.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:45.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:27:45.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:45.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: osdmap e44: 7 total, 7 up, 7 in 2024-05-14T09:27:45.213 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: Detected new or changed devices on smithi002 2024-05-14T09:27:45.213 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:45.214 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:45.214 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:27:45.214 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:27:45.214 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:27:45.214 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi002 to 5697M 2024-05-14T09:27:45.214 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:45.214 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:45.214 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:27:45.214 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:45.214 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:27:45.214 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:45.214 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:27:45.214 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:27:45.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.675+0000 7ff2ea4c3700 1 -- 172.21.15.26:0/3658692037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2e4104380 msgr2=0x7ff2e4104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:27:45.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.675+0000 7ff2ea4c3700 1 --2- 172.21.15.26:0/3658692037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2e4104380 0x7ff2e4104790 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009990 tx=0x7ff2d8009ca0 comp rx=0 tx=0).stop 2024-05-14T09:27:45.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.675+0000 7ff2ea4c3700 1 -- 172.21.15.26:0/3658692037 shutdown_connections 2024-05-14T09:27:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.675+0000 7ff2ea4c3700 1 --2- 172.21.15.26:0/3658692037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2e4104d20 0x7ff2e410b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.675+0000 7ff2ea4c3700 1 --2- 172.21.15.26:0/3658692037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2e4103250 0x7ff2e4103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.675+0000 7ff2ea4c3700 1 --2- 172.21.15.26:0/3658692037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2e4104380 0x7ff2e4104790 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.675+0000 7ff2ea4c3700 1 -- 172.21.15.26:0/3658692037 >> 172.21.15.26:0/3658692037 conn(0x7ff2e40fe890 msgr2=0x7ff2e4100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:27:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.675+0000 7ff2ea4c3700 1 -- 172.21.15.26:0/3658692037 shutdown_connections 2024-05-14T09:27:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.675+0000 7ff2ea4c3700 1 -- 172.21.15.26:0/3658692037 wait complete. 2024-05-14T09:27:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.676+0000 7ff2ea4c3700 1 Processor -- start 2024-05-14T09:27:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.676+0000 7ff2ea4c3700 1 -- start start 2024-05-14T09:27:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.676+0000 7ff2ea4c3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2e4103250 0x7ff2e419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:27:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.676+0000 7ff2ea4c3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2e4104380 0x7ff2e419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:27:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.676+0000 7ff2ea4c3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2e4104d20 0x7ff2e41a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:27:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.676+0000 7ff2ea4c3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e410ecb0 con 0x7ff2e4104d20 2024-05-14T09:27:45.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.676+0000 7ff2ea4c3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e410eb30 con 0x7ff2e4103250 2024-05-14T09:27:45.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.676+0000 7ff2ea4c3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e410ee30 con 0x7ff2e4104380 2024-05-14T09:27:45.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.676+0000 7ff2e3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2e4103250 0x7ff2e419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:27:45.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.676+0000 7ff2e3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2e4103250 0x7ff2e419a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54856/0 (socket says 172.21.15.26:54856) 2024-05-14T09:27:45.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.676+0000 7ff2e3fff700 1 -- 172.21.15.26:0/2925997086 learned_addr learned my addr 172.21.15.26:0/2925997086 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-14T09:27:45.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.677+0000 7ff2e37fe700 1 --2- 172.21.15.26:0/2925997086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2e4104380 0x7ff2e419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:27:45.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.677+0000 7ff2e8a60700 1 --2- 172.21.15.26:0/2925997086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2e4104d20 0x7ff2e41a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:27:45.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.677+0000 7ff2e3fff700 1 -- 172.21.15.26:0/2925997086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2e4104380 msgr2=0x7ff2e419ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:27:45.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.677+0000 7ff2e3fff700 1 --2- 172.21.15.26:0/2925997086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2e4104380 0x7ff2e419ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:45.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.677+0000 7ff2e3fff700 1 -- 172.21.15.26:0/2925997086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2e4104d20 msgr2=0x7ff2e41a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:27:45.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.677+0000 7ff2e3fff700 1 --2- 172.21.15.26:0/2925997086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2e4104d20 0x7ff2e41a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:27:45.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.677+0000 7ff2e3fff700 1 -- 172.21.15.26:0/2925997086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e41a24a0 con 0x7ff2e4103250 2024-05-14T09:27:45.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.677+0000 7ff2e8a60700 1 --2- 172.21.15.26:0/2925997086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2e4104d20 0x7ff2e41a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:27:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.677+0000 7ff2e3fff700 1 --2- 172.21.15.26:0/2925997086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2e4103250 0x7ff2e419a940 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400e940 tx=0x7ff2d400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:27:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.677+0000 7ff2e17fa700 1 -- 172.21.15.26:0/2925997086 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400cbd0 con 0x7ff2e4103250 2024-05-14T09:27:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.677+0000 7ff2ea4c3700 1 -- 172.21.15.26:0/2925997086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e41a2790 con 0x7ff2e4103250 2024-05-14T09:27:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.677+0000 7ff2e17fa700 1 -- 172.21.15.26:0/2925997086 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d400ee90 con 0x7ff2e4103250 2024-05-14T09:27:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.677+0000 7ff2e17fa700 1 -- 172.21.15.26:0/2925997086 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4003dc0 con 0x7ff2e4103250 2024-05-14T09:27:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.677+0000 7ff2ea4c3700 1 -- 172.21.15.26:0/2925997086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e41a2cd0 con 0x7ff2e4103250 2024-05-14T09:27:45.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.678+0000 7ff2ea4c3700 1 -- 172.21.15.26:0/2925997086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e40512c0 con 0x7ff2e4103250 2024-05-14T09:27:45.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.680+0000 7ff2e17fa700 1 -- 172.21.15.26:0/2925997086 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7ff2d4003f60 con 0x7ff2e4103250 2024-05-14T09:27:45.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.681+0000 7ff2e17fa700 1 --2- 172.21.15.26:0/2925997086 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff2cc0b2880 0x7ff2cc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:27:45.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.681+0000 7ff2e17fa700 1 -- 172.21.15.26:0/2925997086 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7ff2d40e1810 con 0x7ff2e4103250 2024-05-14T09:27:45.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.681+0000 7ff2e37fe700 1 --2- 172.21.15.26:0/2925997086 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff2cc0b2880 0x7ff2cc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:27:45.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.681+0000 7ff2e37fe700 1 --2- 172.21.15.26:0/2925997086 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff2cc0b2880 0x7ff2cc0b4d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff2e419b9f0 tx=0x7ff2d800b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:27:45.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.684+0000 7ff2e17fa700 1 -- 172.21.15.26:0/2925997086 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2d40aa2d0 con 0x7ff2e4103250 2024-05-14T09:27:45.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:27:45.906+0000 7ff2ea4c3700 1 -- 172.21.15.26:0/2925997086 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff2e41a2f90 con 0x7ff2cc0b2880 2024-05-14T09:27:45.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:45 smithi026 ceph-mon[36056]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:46.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:45 smithi002 ceph-mon[31557]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:46.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:45 smithi029 ceph-mon[36249]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:47.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:46 smithi002 ceph-mon[31557]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:27:47.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:27:47.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:27:47.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:47.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:46 smithi029 ceph-mon[36249]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:27:47.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:27:47.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:27:47.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:47.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:46 smithi026 ceph-mon[36056]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:27:47.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:27:47.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:27:47.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:48.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:47 smithi002 ceph-mon[31557]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:48.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:47 smithi029 ceph-mon[36249]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:48.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:47 smithi026 ceph-mon[36056]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:49.081 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:48 smithi026 ceph-mon[36056]: from='client.? 172.21.15.26:0/640891646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22ddef80-3633-44c5-bc0e-edc75b546ea9"}]: dispatch 2024-05-14T09:27:49.081 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:48 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22ddef80-3633-44c5-bc0e-edc75b546ea9"}]: dispatch 2024-05-14T09:27:49.081 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:48 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22ddef80-3633-44c5-bc0e-edc75b546ea9"}]': finished 2024-05-14T09:27:49.082 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:48 smithi026 ceph-mon[36056]: osdmap e45: 8 total, 7 up, 8 in 2024-05-14T09:27:49.082 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T09:27:49.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:48 smithi002 ceph-mon[31557]: from='client.? 172.21.15.26:0/640891646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22ddef80-3633-44c5-bc0e-edc75b546ea9"}]: dispatch 2024-05-14T09:27:49.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:48 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22ddef80-3633-44c5-bc0e-edc75b546ea9"}]: dispatch 2024-05-14T09:27:49.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:48 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22ddef80-3633-44c5-bc0e-edc75b546ea9"}]': finished 2024-05-14T09:27:49.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:48 smithi002 ceph-mon[31557]: osdmap e45: 8 total, 7 up, 8 in 2024-05-14T09:27:49.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T09:27:49.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:48 smithi029 ceph-mon[36249]: from='client.? 172.21.15.26:0/640891646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22ddef80-3633-44c5-bc0e-edc75b546ea9"}]: dispatch 2024-05-14T09:27:49.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:48 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22ddef80-3633-44c5-bc0e-edc75b546ea9"}]: dispatch 2024-05-14T09:27:49.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:48 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22ddef80-3633-44c5-bc0e-edc75b546ea9"}]': finished 2024-05-14T09:27:49.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:48 smithi029 ceph-mon[36249]: osdmap e45: 8 total, 7 up, 8 in 2024-05-14T09:27:49.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T09:27:50.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:49 smithi002 ceph-mon[31557]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:50.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:49 smithi002 ceph-mon[31557]: from='client.? 172.21.15.26:0/4224443541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:27:50.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:49 smithi029 ceph-mon[36249]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:50.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:49 smithi029 ceph-mon[36249]: from='client.? 172.21.15.26:0/4224443541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:27:50.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:49 smithi026 ceph-mon[36056]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:50.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:49 smithi026 ceph-mon[36056]: from='client.? 172.21.15.26:0/4224443541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:27:52.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:52 smithi002 ceph-mon[31557]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:52.436 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:52 smithi026 ceph-mon[36056]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:52.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:52 smithi029 ceph-mon[36249]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:54.346 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:54 smithi026 ceph-mon[36056]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:54 smithi002 ceph-mon[31557]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:54.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:54 smithi029 ceph-mon[36249]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:56.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:56 smithi002 ceph-mon[31557]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:56 smithi029 ceph-mon[36249]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:56 smithi026 ceph-mon[36056]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:57.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-14T09:27:57.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:57.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:57 smithi002 ceph-mon[31557]: Deploying daemon osd.7 on smithi026 2024-05-14T09:27:57.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-14T09:27:57.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:57.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:57 smithi029 ceph-mon[36249]: Deploying daemon osd.7 on smithi026 2024-05-14T09:27:57.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-14T09:27:57.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:27:57.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:57 smithi026 ceph-mon[36056]: Deploying daemon osd.7 on smithi026 2024-05-14T09:27:58.322 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:27:58 smithi002 ceph-mon[31557]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:58.392 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:27:58 smithi026 ceph-mon[36056]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:27:58.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:27:58 smithi029 ceph-mon[36249]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:00.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:00 smithi026 ceph-mon[36056]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:00.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:00 smithi002 ceph-mon[31557]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:00.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:00 smithi029 ceph-mon[36249]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:02.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:02 smithi002 ceph-mon[31557]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:02.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:02 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:02.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:02 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:02.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:02 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:28:02.431 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:02 smithi026 ceph-mon[36056]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:02.431 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:02 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:02.431 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:02 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:02.431 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:02 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:28:02.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:02 smithi029 ceph-mon[36249]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:02.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:02 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:02.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:02 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:02.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:02 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:28:04.368 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:04 smithi026 ceph-mon[36056]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:04.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:04 smithi002 ceph-mon[31557]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:04.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:04 smithi029 ceph-mon[36249]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:05.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:28:05.775+0000 7ff2e17fa700 1 -- 172.21.15.26:0/2925997086 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff2e41a2f90 con 0x7ff2cc0b2880 2024-05-14T09:28:05.776 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 7 on host 'smithi026' 2024-05-14T09:28:05.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:28:05.778+0000 7ff2ea4c3700 1 -- 172.21.15.26:0/2925997086 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff2cc0b2880 msgr2=0x7ff2cc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:28:05.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:28:05.778+0000 7ff2ea4c3700 1 --2- 172.21.15.26:0/2925997086 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff2cc0b2880 0x7ff2cc0b4d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff2e419b9f0 tx=0x7ff2d800b720 comp rx=0 tx=0).stop 2024-05-14T09:28:05.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:28:05.778+0000 7ff2ea4c3700 1 -- 172.21.15.26:0/2925997086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2e4103250 msgr2=0x7ff2e419a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:28:05.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:28:05.778+0000 7ff2ea4c3700 1 --2- 172.21.15.26:0/2925997086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2e4103250 0x7ff2e419a940 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400e940 tx=0x7ff2d400ec50 comp rx=0 tx=0).stop 2024-05-14T09:28:05.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:28:05.778+0000 7ff2ea4c3700 1 -- 172.21.15.26:0/2925997086 shutdown_connections 2024-05-14T09:28:05.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:28:05.778+0000 7ff2ea4c3700 1 --2- 172.21.15.26:0/2925997086 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff2cc0b2880 0x7ff2cc0b4d90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:05.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:28:05.778+0000 7ff2ea4c3700 1 --2- 172.21.15.26:0/2925997086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2e4103250 0x7ff2e419a940 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:05.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:28:05.778+0000 7ff2ea4c3700 1 --2- 172.21.15.26:0/2925997086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2e4104d20 0x7ff2e41a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:05.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:28:05.778+0000 7ff2ea4c3700 1 --2- 172.21.15.26:0/2925997086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2e4104380 0x7ff2e419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:05.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:28:05.778+0000 7ff2ea4c3700 1 -- 172.21.15.26:0/2925997086 >> 172.21.15.26:0/2925997086 conn(0x7ff2e40fe890 msgr2=0x7ff2e41001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:28:05.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:28:05.778+0000 7ff2ea4c3700 1 -- 172.21.15.26:0/2925997086 shutdown_connections 2024-05-14T09:28:05.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:28:05.778+0000 7ff2ea4c3700 1 -- 172.21.15.26:0/2925997086 wait complete. 2024-05-14T09:28:06.080 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:06.081 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:06.081 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:06.081 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:28:06.081 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:06.081 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:05 smithi026 ceph-mon[36056]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:06.081 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:05 smithi026 ceph-mon[36056]: from='osd.7 [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T09:28:06.081 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:05 smithi026 ceph-mon[36056]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T09:28:06.081 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:28:06.081 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:06.081 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:06.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:06.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:28:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:05 smithi002 ceph-mon[31557]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:05 smithi002 ceph-mon[31557]: from='osd.7 [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T09:28:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:05 smithi002 ceph-mon[31557]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T09:28:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:28:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:06.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:28:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:05 smithi029 ceph-mon[36249]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:05 smithi029 ceph-mon[36249]: from='osd.7 [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T09:28:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:05 smithi029 ceph-mon[36249]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T09:28:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:28:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:06.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:06.391 DEBUG:teuthology.orchestra.run.smithi026:osd.7> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.7.service 2024-05-14T09:28:06.393 INFO:tasks.cephadm:Deploying osd.8 on smithi029 with /dev/vg_nvme/lv_2... 2024-05-14T09:28:06.393 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-14T09:28:06.650 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T09:28:07.029 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:06 smithi029 ceph-mon[36249]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-14T09:28:07.030 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:06 smithi029 ceph-mon[36249]: osdmap e46: 8 total, 7 up, 8 in 2024-05-14T09:28:07.030 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:06 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T09:28:07.030 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:06 smithi029 ceph-mon[36249]: from='osd.7 [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:28:07.030 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:06 smithi029 ceph-mon[36249]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:28:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:06 smithi002 ceph-mon[31557]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-14T09:28:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:06 smithi002 ceph-mon[31557]: osdmap e46: 8 total, 7 up, 8 in 2024-05-14T09:28:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:06 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T09:28:07.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:06 smithi002 ceph-mon[31557]: from='osd.7 [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:28:07.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:06 smithi002 ceph-mon[31557]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:28:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:06 smithi026 ceph-mon[36056]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-14T09:28:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:06 smithi026 ceph-mon[36056]: osdmap e46: 8 total, 7 up, 8 in 2024-05-14T09:28:07.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:06 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T09:28:07.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:06 smithi026 ceph-mon[36056]: from='osd.7 [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:28:07.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:06 smithi026 ceph-mon[36056]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:28:07.488 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 09:28:07 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T09:28:07.255+0000 7f12c0b97700 -1 osd.7 0 waiting for initial osdmap 2024-05-14T09:28:07.488 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 09:28:07 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T09:28:07.268+0000 7f12b8143700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T09:28:08.079 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:07 smithi029 ceph-mon[36249]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-14T09:28:08.079 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:07 smithi029 ceph-mon[36249]: osdmap e47: 8 total, 7 up, 8 in 2024-05-14T09:28:08.079 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:07 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T09:28:08.079 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:07 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T09:28:08.079 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:07 smithi029 ceph-mon[36249]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:08.079 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:07 smithi029 ceph-mon[36249]: from='osd.7 ' entity='osd.7' 2024-05-14T09:28:08.079 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:07 smithi029 ceph-mon[36249]: osd.7 [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611] boot 2024-05-14T09:28:08.151 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:07 smithi026 ceph-mon[36056]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-14T09:28:08.151 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:07 smithi026 ceph-mon[36056]: osdmap e47: 8 total, 7 up, 8 in 2024-05-14T09:28:08.151 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:07 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T09:28:08.151 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:07 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T09:28:08.151 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:07 smithi026 ceph-mon[36056]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:08.151 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:07 smithi026 ceph-mon[36056]: from='osd.7 ' entity='osd.7' 2024-05-14T09:28:08.151 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:07 smithi026 ceph-mon[36056]: osd.7 [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611] boot 2024-05-14T09:28:08.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:07 smithi002 ceph-mon[31557]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-14T09:28:08.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:07 smithi002 ceph-mon[31557]: osdmap e47: 8 total, 7 up, 8 in 2024-05-14T09:28:08.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:07 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T09:28:08.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:07 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T09:28:08.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:07 smithi002 ceph-mon[31557]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-14T09:28:08.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:07 smithi002 ceph-mon[31557]: from='osd.7 ' entity='osd.7' 2024-05-14T09:28:08.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:07 smithi002 ceph-mon[31557]: osd.7 [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611] boot 2024-05-14T09:28:08.795 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-14T09:28:08.795 INFO:teuthology.orchestra.run.smithi029.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-14T09:28:08.795 INFO:teuthology.orchestra.run.smithi029.stderr: stderr: 10+0 records in 2024-05-14T09:28:08.795 INFO:teuthology.orchestra.run.smithi029.stderr:10+0 records out 2024-05-14T09:28:08.796 INFO:teuthology.orchestra.run.smithi029.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00853031 s, 1.2 GB/s 2024-05-14T09:28:08.796 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping successful for: 2024-05-14T09:28:09.077 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:08 smithi026 ceph-mon[36056]: purged_snaps scrub starts 2024-05-14T09:28:09.077 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:08 smithi026 ceph-mon[36056]: purged_snaps scrub ok 2024-05-14T09:28:09.077 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:08 smithi026 ceph-mon[36056]: osdmap e48: 8 total, 8 up, 8 in 2024-05-14T09:28:09.077 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:08 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T09:28:09.088 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:08 smithi002 ceph-mon[31557]: purged_snaps scrub starts 2024-05-14T09:28:09.089 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:08 smithi002 ceph-mon[31557]: purged_snaps scrub ok 2024-05-14T09:28:09.089 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:08 smithi002 ceph-mon[31557]: osdmap e48: 8 total, 8 up, 8 in 2024-05-14T09:28:09.089 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:08 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T09:28:09.136 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:08 smithi029 ceph-mon[36249]: purged_snaps scrub starts 2024-05-14T09:28:09.137 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:08 smithi029 ceph-mon[36249]: purged_snaps scrub ok 2024-05-14T09:28:09.137 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:08 smithi029 ceph-mon[36249]: osdmap e48: 8 total, 8 up, 8 in 2024-05-14T09:28:09.137 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:08 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T09:28:09.464 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi029:vg_nvme/lv_2 2024-05-14T09:28:09.721 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T09:28:09.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:09 smithi026 ceph-mon[36056]: osdmap e49: 8 total, 8 up, 8 in 2024-05-14T09:28:09.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:09 smithi026 ceph-mon[36056]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:09.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:09 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:09.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:09 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:09.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:09 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:09.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:09 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:09.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:09 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:09.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:09 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:09.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:09 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:09.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:09 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:28:09.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:09 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:10.096 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:09 smithi029 ceph-mon[36249]: osdmap e49: 8 total, 8 up, 8 in 2024-05-14T09:28:10.097 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:09 smithi029 ceph-mon[36249]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:10.097 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:09 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:10.097 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:09 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:10.097 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:09 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:10.097 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:09 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:10.097 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:09 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:10.097 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:09 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:10.097 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:09 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:10.097 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:09 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:28:10.097 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:09 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:10.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:09 smithi002 ceph-mon[31557]: osdmap e49: 8 total, 8 up, 8 in 2024-05-14T09:28:10.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:09 smithi002 ceph-mon[31557]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:10.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:10.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:10.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:10.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:10.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:10.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:10.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:10.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:28:10.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:11.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:10 smithi002 ceph-mon[31557]: Detected new or changed devices on smithi026 2024-05-14T09:28:11.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:10 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi026 to 5697M 2024-05-14T09:28:11.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:10 smithi002 ceph-mon[31557]: osdmap e50: 8 total, 8 up, 8 in 2024-05-14T09:28:11.178 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:10 smithi029 ceph-mon[36249]: Detected new or changed devices on smithi026 2024-05-14T09:28:11.178 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:10 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi026 to 5697M 2024-05-14T09:28:11.179 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:10 smithi029 ceph-mon[36249]: osdmap e50: 8 total, 8 up, 8 in 2024-05-14T09:28:11.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:10 smithi026 ceph-mon[36056]: Detected new or changed devices on smithi026 2024-05-14T09:28:11.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:10 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi026 to 5697M 2024-05-14T09:28:11.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:10 smithi026 ceph-mon[36056]: osdmap e50: 8 total, 8 up, 8 in 2024-05-14T09:28:11.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.432+0000 7fe6c51c3700 1 -- 172.21.15.29:0/1101003757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0103240 msgr2=0x7fe6c0103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:28:11.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.432+0000 7fe6c51c3700 1 --2- 172.21.15.29:0/1101003757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0103240 0x7fe6c0103650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009990 tx=0x7fe6b0009ca0 comp rx=0 tx=0).stop 2024-05-14T09:28:11.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.432+0000 7fe6c51c3700 1 -- 172.21.15.29:0/1101003757 shutdown_connections 2024-05-14T09:28:11.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.432+0000 7fe6c51c3700 1 --2- 172.21.15.29:0/1101003757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c0104d10 0x7fe6c010b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:11.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.432+0000 7fe6c51c3700 1 --2- 172.21.15.29:0/1101003757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0103240 0x7fe6c0103650 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:11.432 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.432+0000 7fe6c51c3700 1 --2- 172.21.15.29:0/1101003757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6c0104370 0x7fe6c0104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:11.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.432+0000 7fe6c51c3700 1 -- 172.21.15.29:0/1101003757 >> 172.21.15.29:0/1101003757 conn(0x7fe6c00fe880 msgr2=0x7fe6c0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:28:11.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.432+0000 7fe6c51c3700 1 -- 172.21.15.29:0/1101003757 shutdown_connections 2024-05-14T09:28:11.433 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.432+0000 7fe6c51c3700 1 -- 172.21.15.29:0/1101003757 wait complete. 2024-05-14T09:28:11.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.433+0000 7fe6c51c3700 1 Processor -- start 2024-05-14T09:28:11.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6c51c3700 1 -- start start 2024-05-14T09:28:11.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6c51c3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0103240 0x7fe6c006c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:28:11.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6c51c3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6c0104370 0x7fe6c019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:28:11.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6c51c3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c0104d10 0x7fe6c01a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:28:11.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6c51c3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c010ec80 con 0x7fe6c0104d10 2024-05-14T09:28:11.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6c51c3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c010eb00 con 0x7fe6c0104370 2024-05-14T09:28:11.435 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6c51c3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c010ee00 con 0x7fe6c0103240 2024-05-14T09:28:11.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6bed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0103240 0x7fe6c006c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:28:11.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6bed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0103240 0x7fe6c006c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:49702/0 (socket says 172.21.15.29:49702) 2024-05-14T09:28:11.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6bed9d700 1 -- 172.21.15.29:0/4225072205 learned_addr learned my addr 172.21.15.29:0/4225072205 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-05-14T09:28:11.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6be59c700 1 --2- 172.21.15.29:0/4225072205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6c0104370 0x7fe6c019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:28:11.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6bf59e700 1 --2- 172.21.15.29:0/4225072205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c0104d10 0x7fe6c01a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:28:11.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6bed9d700 1 -- 172.21.15.29:0/4225072205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6c0104370 msgr2=0x7fe6c019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:28:11.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6bed9d700 1 --2- 172.21.15.29:0/4225072205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6c0104370 0x7fe6c019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:11.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6bed9d700 1 -- 172.21.15.29:0/4225072205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c0104d10 msgr2=0x7fe6c01a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:28:11.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6bed9d700 1 --2- 172.21.15.29:0/4225072205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c0104d10 0x7fe6c01a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:11.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6bed9d700 1 -- 172.21.15.29:0/4225072205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c01a2460 con 0x7fe6c0103240 2024-05-14T09:28:11.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.434+0000 7fe6bed9d700 1 --2- 172.21.15.29:0/4225072205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0103240 0x7fe6c006c070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00057c0 tx=0x7fe6b00057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:28:11.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.435+0000 7fe6a7fff700 1 -- 172.21.15.29:0/4225072205 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0027070 con 0x7fe6c0103240 2024-05-14T09:28:11.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.435+0000 7fe6c51c3700 1 -- 172.21.15.29:0/4225072205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c01a26f0 con 0x7fe6c0103240 2024-05-14T09:28:11.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.435+0000 7fe6a7fff700 1 -- 172.21.15.29:0/4225072205 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6b0004000 con 0x7fe6c0103240 2024-05-14T09:28:11.436 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.435+0000 7fe6a7fff700 1 -- 172.21.15.29:0/4225072205 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0022c30 con 0x7fe6c0103240 2024-05-14T09:28:11.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.435+0000 7fe6c51c3700 1 -- 172.21.15.29:0/4225072205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c01a2ba0 con 0x7fe6c0103240 2024-05-14T09:28:11.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.436+0000 7fe6c51c3700 1 -- 172.21.15.29:0/4225072205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c00512c0 con 0x7fe6c0103240 2024-05-14T09:28:11.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.437+0000 7fe6a7fff700 1 -- 172.21.15.29:0/4225072205 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fe6b002a070 con 0x7fe6c0103240 2024-05-14T09:28:11.437 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.438+0000 7fe6a7fff700 1 --2- 172.21.15.29:0/4225072205 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe6a80b2880 0x7fe6a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:28:11.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.438+0000 7fe6a7fff700 1 -- 172.21.15.29:0/4225072205 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7fe6b00e2e60 con 0x7fe6c0103240 2024-05-14T09:28:11.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.438+0000 7fe6be59c700 1 --2- 172.21.15.29:0/4225072205 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe6a80b2880 0x7fe6a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:28:11.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.439+0000 7fe6be59c700 1 --2- 172.21.15.29:0/4225072205 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe6a80b2880 0x7fe6a80b4d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe6c006ba20 tx=0x7fe6b40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:28:11.441 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.442+0000 7fe6a7fff700 1 -- 172.21.15.29:0/4225072205 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe6b0031080 con 0x7fe6c0103240 2024-05-14T09:28:11.663 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:11.664+0000 7fe6c51c3700 1 -- 172.21.15.29:0/4225072205 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe6c0063c00 con 0x7fe6a80b2880 2024-05-14T09:28:12.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:11 smithi002 ceph-mon[31557]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:12.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:11 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:28:12.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:11 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:28:12.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:11 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:12.186 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:11 smithi029 ceph-mon[36249]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:12.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:11 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:28:12.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:11 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:28:12.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:11 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:12.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:11 smithi026 ceph-mon[36056]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:12.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:11 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:28:12.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:11 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:28:12.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:11 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:12.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:12 smithi029 ceph-mon[36249]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:28:13.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:12 smithi002 ceph-mon[31557]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:28:13.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:12 smithi026 ceph-mon[36056]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:28:13.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:13 smithi029 ceph-mon[36249]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T09:28:14.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:13 smithi002 ceph-mon[31557]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T09:28:14.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:13 smithi026 ceph-mon[36056]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T09:28:15.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:14 smithi002 ceph-mon[31557]: from='client.? 172.21.15.29:0/2160919550' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "466e7ac4-3ff7-4254-a3fd-8096420b1397"}]: dispatch 2024-05-14T09:28:15.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:14 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "466e7ac4-3ff7-4254-a3fd-8096420b1397"}]: dispatch 2024-05-14T09:28:15.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:14 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "466e7ac4-3ff7-4254-a3fd-8096420b1397"}]': finished 2024-05-14T09:28:15.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:14 smithi002 ceph-mon[31557]: osdmap e51: 9 total, 8 up, 9 in 2024-05-14T09:28:15.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:14 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T09:28:15.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:14 smithi029 ceph-mon[36249]: from='client.? 172.21.15.29:0/2160919550' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "466e7ac4-3ff7-4254-a3fd-8096420b1397"}]: dispatch 2024-05-14T09:28:15.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:14 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "466e7ac4-3ff7-4254-a3fd-8096420b1397"}]: dispatch 2024-05-14T09:28:15.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:14 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "466e7ac4-3ff7-4254-a3fd-8096420b1397"}]': finished 2024-05-14T09:28:15.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:14 smithi029 ceph-mon[36249]: osdmap e51: 9 total, 8 up, 9 in 2024-05-14T09:28:15.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T09:28:15.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:14 smithi026 ceph-mon[36056]: from='client.? 172.21.15.29:0/2160919550' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "466e7ac4-3ff7-4254-a3fd-8096420b1397"}]: dispatch 2024-05-14T09:28:15.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:14 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "466e7ac4-3ff7-4254-a3fd-8096420b1397"}]: dispatch 2024-05-14T09:28:15.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:14 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "466e7ac4-3ff7-4254-a3fd-8096420b1397"}]': finished 2024-05-14T09:28:15.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:14 smithi026 ceph-mon[36056]: osdmap e51: 9 total, 8 up, 9 in 2024-05-14T09:28:15.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:14 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T09:28:16.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:15 smithi002 ceph-mon[31557]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 186 KiB/s, 0 objects/s recovering 2024-05-14T09:28:16.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:15 smithi002 ceph-mon[31557]: from='client.? 172.21.15.29:0/3799148134' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:28:16.186 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:15 smithi029 ceph-mon[36249]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 186 KiB/s, 0 objects/s recovering 2024-05-14T09:28:16.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:15 smithi029 ceph-mon[36249]: from='client.? 172.21.15.29:0/3799148134' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:28:16.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:15 smithi026 ceph-mon[36056]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 186 KiB/s, 0 objects/s recovering 2024-05-14T09:28:16.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:15 smithi026 ceph-mon[36056]: from='client.? 172.21.15.29:0/3799148134' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:28:18.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:18 smithi002 ceph-mon[31557]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-14T09:28:18.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:18 smithi029 ceph-mon[36249]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-14T09:28:18.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:18 smithi026 ceph-mon[36056]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-14T09:28:20.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:20 smithi002 ceph-mon[31557]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-05-14T09:28:20.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:20 smithi029 ceph-mon[36249]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-05-14T09:28:20.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:20 smithi026 ceph-mon[36056]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-05-14T09:28:22.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:22 smithi029 ceph-mon[36249]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-14T09:28:22.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:22 smithi002 ceph-mon[31557]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-14T09:28:22.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:22 smithi026 ceph-mon[36056]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-14T09:28:23.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:23 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-14T09:28:23.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:23 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:23.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:23 smithi029 ceph-mon[36249]: Deploying daemon osd.8 on smithi029 2024-05-14T09:28:23.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:23 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-14T09:28:23.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:23 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:23.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:23 smithi002 ceph-mon[31557]: Deploying daemon osd.8 on smithi029 2024-05-14T09:28:23.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-14T09:28:23.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:23.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:23 smithi026 ceph-mon[36056]: Deploying daemon osd.8 on smithi029 2024-05-14T09:28:24.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:24 smithi029 ceph-mon[36249]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:24.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:24 smithi002 ceph-mon[31557]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:24.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:24 smithi026 ceph-mon[36056]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:26.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:26 smithi029 ceph-mon[36249]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:26.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:26 smithi002 ceph-mon[31557]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:26.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:26 smithi026 ceph-mon[36056]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:28.173 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:28 smithi029 ceph-mon[36249]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:28.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:28 smithi002 ceph-mon[31557]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:28.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:28 smithi026 ceph-mon[36056]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:28:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:29.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:29.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:28:29.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:29.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:29.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:28:29.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:30 smithi002 ceph-mon[31557]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:30 smithi029 ceph-mon[36249]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:30 smithi026 ceph-mon[36056]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:32.551 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:32 smithi029 ceph-mon[36249]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:32.551 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:32.551 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:32.551 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:32.551 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:28:32.551 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:32.551 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:32 smithi029 ceph-mon[36249]: from='osd.8 [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T09:28:32.551 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:32 smithi029 ceph-mon[36249]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T09:28:32.637 INFO:teuthology.orchestra.run.smithi029.stdout:Created osd(s) 8 on host 'smithi029' 2024-05-14T09:28:32.637 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:32.637+0000 7fe6a7fff700 1 -- 172.21.15.29:0/4225072205 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe6c0063c00 con 0x7fe6a80b2880 2024-05-14T09:28:32.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:32.639+0000 7fe6c51c3700 1 -- 172.21.15.29:0/4225072205 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe6a80b2880 msgr2=0x7fe6a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:28:32.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:32.639+0000 7fe6c51c3700 1 --2- 172.21.15.29:0/4225072205 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe6a80b2880 0x7fe6a80b4d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe6c006ba20 tx=0x7fe6b40073d0 comp rx=0 tx=0).stop 2024-05-14T09:28:32.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:32.639+0000 7fe6c51c3700 1 -- 172.21.15.29:0/4225072205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0103240 msgr2=0x7fe6c006c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:28:32.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:32.639+0000 7fe6c51c3700 1 --2- 172.21.15.29:0/4225072205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0103240 0x7fe6c006c070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00057c0 tx=0x7fe6b00057f0 comp rx=0 tx=0).stop 2024-05-14T09:28:32.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:32.639+0000 7fe6c51c3700 1 -- 172.21.15.29:0/4225072205 shutdown_connections 2024-05-14T09:28:32.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:32.639+0000 7fe6c51c3700 1 --2- 172.21.15.29:0/4225072205 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe6a80b2880 0x7fe6a80b4d90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:32.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:32.639+0000 7fe6c51c3700 1 --2- 172.21.15.29:0/4225072205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c0104d10 0x7fe6c01a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:32.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:32.639+0000 7fe6c51c3700 1 --2- 172.21.15.29:0/4225072205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6c0103240 0x7fe6c006c070 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:32.640 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:32.639+0000 7fe6c51c3700 1 --2- 172.21.15.29:0/4225072205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6c0104370 0x7fe6c019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:32.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:32.639+0000 7fe6c51c3700 1 -- 172.21.15.29:0/4225072205 >> 172.21.15.29:0/4225072205 conn(0x7fe6c00fe880 msgr2=0x7fe6c01075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:28:32.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:32.639+0000 7fe6c51c3700 1 -- 172.21.15.29:0/4225072205 shutdown_connections 2024-05-14T09:28:32.641 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:28:32.639+0000 7fe6c51c3700 1 -- 172.21.15.29:0/4225072205 wait complete. 2024-05-14T09:28:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:32 smithi002 ceph-mon[31557]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:32.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:32.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:28:32.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:32.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:32 smithi002 ceph-mon[31557]: from='osd.8 [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T09:28:32.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:32 smithi002 ceph-mon[31557]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T09:28:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:32 smithi026 ceph-mon[36056]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:32.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:32.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:32.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:28:32.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:32.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:32 smithi026 ceph-mon[36056]: from='osd.8 [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T09:28:32.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:32 smithi026 ceph-mon[36056]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T09:28:33.265 DEBUG:teuthology.orchestra.run.smithi029:osd.8> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.8.service 2024-05-14T09:28:33.267 INFO:tasks.cephadm:Deploying osd.9 on smithi002 with /dev/vg_nvme/lv_1... 2024-05-14T09:28:33.267 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-14T09:28:33.528 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:28:33.871 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:33 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:33.872 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:33 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:33.872 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:33 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:28:33.872 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:33 smithi002 ceph-mon[31557]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-14T09:28:33.872 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:33 smithi002 ceph-mon[31557]: from='osd.8 [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:28:33.872 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:33 smithi002 ceph-mon[31557]: osdmap e52: 9 total, 8 up, 9 in 2024-05-14T09:28:33.872 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:33 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T09:28:33.872 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:33 smithi002 ceph-mon[31557]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:28:33.872 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:33 smithi002 ceph-mon[31557]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:33.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:33 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:33.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:33 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:33.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:33 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:28:33.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:33 smithi029 ceph-mon[36249]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-14T09:28:33.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:33 smithi029 ceph-mon[36249]: from='osd.8 [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:28:33.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:33 smithi029 ceph-mon[36249]: osdmap e52: 9 total, 8 up, 9 in 2024-05-14T09:28:33.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:33 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T09:28:33.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:33 smithi029 ceph-mon[36249]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:28:33.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:33 smithi029 ceph-mon[36249]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:33.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:33 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:33.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:33 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:33.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:33 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:28:33.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:33 smithi026 ceph-mon[36056]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-14T09:28:33.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:33 smithi026 ceph-mon[36056]: from='osd.8 [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:28:33.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:33 smithi026 ceph-mon[36056]: osdmap e52: 9 total, 8 up, 9 in 2024-05-14T09:28:33.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:33 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T09:28:33.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:33 smithi026 ceph-mon[36056]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:28:33.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:33 smithi026 ceph-mon[36056]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:34.529 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 09:28:34 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T09:28:34.190+0000 7f6822f00700 -1 osd.8 0 waiting for initial osdmap 2024-05-14T09:28:34.529 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 09:28:34 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T09:28:34.200+0000 7f681acad700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T09:28:34.837 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:34 smithi029 ceph-mon[36249]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-05-14T09:28:34.838 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:34 smithi029 ceph-mon[36249]: osdmap e53: 9 total, 8 up, 9 in 2024-05-14T09:28:34.838 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:34 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T09:28:34.838 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:34 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T09:28:34.838 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:34 smithi029 ceph-mon[36249]: from='osd.8 ' entity='osd.8' 2024-05-14T09:28:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:34 smithi002 ceph-mon[31557]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-05-14T09:28:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:34 smithi002 ceph-mon[31557]: osdmap e53: 9 total, 8 up, 9 in 2024-05-14T09:28:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:34 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T09:28:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:34 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T09:28:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:34 smithi002 ceph-mon[31557]: from='osd.8 ' entity='osd.8' 2024-05-14T09:28:35.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:34 smithi026 ceph-mon[36056]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-05-14T09:28:35.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:34 smithi026 ceph-mon[36056]: osdmap e53: 9 total, 8 up, 9 in 2024-05-14T09:28:35.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:34 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T09:28:35.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:34 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T09:28:35.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:34 smithi026 ceph-mon[36056]: from='osd.8 ' entity='osd.8' 2024-05-14T09:28:35.773 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-14T09:28:35.773 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-14T09:28:35.773 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2024-05-14T09:28:35.773 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2024-05-14T09:28:35.773 INFO:teuthology.orchestra.run.smithi002.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00968277 s, 1.1 GB/s 2024-05-14T09:28:35.773 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2024-05-14T09:28:35.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:35 smithi002 ceph-mon[31557]: purged_snaps scrub starts 2024-05-14T09:28:35.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:35 smithi002 ceph-mon[31557]: purged_snaps scrub ok 2024-05-14T09:28:35.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:35 smithi002 ceph-mon[31557]: osd.8 [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446] boot 2024-05-14T09:28:35.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:35 smithi002 ceph-mon[31557]: osdmap e54: 9 total, 9 up, 9 in 2024-05-14T09:28:35.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:35 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T09:28:35.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:35 smithi002 ceph-mon[31557]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:35.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:35 smithi002 ceph-mon[31557]: osdmap e55: 9 total, 9 up, 9 in 2024-05-14T09:28:35.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:35 smithi029 ceph-mon[36249]: purged_snaps scrub starts 2024-05-14T09:28:35.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:35 smithi029 ceph-mon[36249]: purged_snaps scrub ok 2024-05-14T09:28:35.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:35 smithi029 ceph-mon[36249]: osd.8 [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446] boot 2024-05-14T09:28:35.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:35 smithi029 ceph-mon[36249]: osdmap e54: 9 total, 9 up, 9 in 2024-05-14T09:28:35.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:35 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T09:28:35.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:35 smithi029 ceph-mon[36249]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:35.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:35 smithi029 ceph-mon[36249]: osdmap e55: 9 total, 9 up, 9 in 2024-05-14T09:28:36.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:35 smithi026 ceph-mon[36056]: purged_snaps scrub starts 2024-05-14T09:28:36.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:35 smithi026 ceph-mon[36056]: purged_snaps scrub ok 2024-05-14T09:28:36.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:35 smithi026 ceph-mon[36056]: osd.8 [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446] boot 2024-05-14T09:28:36.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:35 smithi026 ceph-mon[36056]: osdmap e54: 9 total, 9 up, 9 in 2024-05-14T09:28:36.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:35 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T09:28:36.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:35 smithi026 ceph-mon[36056]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-14T09:28:36.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:35 smithi026 ceph-mon[36056]: osdmap e55: 9 total, 9 up, 9 in 2024-05-14T09:28:36.546 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi002:vg_nvme/lv_1 2024-05-14T09:28:36.794 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:28:37.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:37 smithi002 ceph-mon[31557]: osdmap e56: 9 total, 9 up, 9 in 2024-05-14T09:28:37.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:37 smithi029 ceph-mon[36249]: osdmap e56: 9 total, 9 up, 9 in 2024-05-14T09:28:37.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:37 smithi026 ceph-mon[36056]: osdmap e56: 9 total, 9 up, 9 in 2024-05-14T09:28:38.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:38 smithi002 ceph-mon[31557]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:38.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:38 smithi002 ceph-mon[31557]: Detected new or changed devices on smithi029 2024-05-14T09:28:38.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:38.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:38.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:38.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:38.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:38.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:38 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi029 to 5697M 2024-05-14T09:28:38.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:38.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:38.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:28:38.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:38.339 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:38 smithi029 ceph-mon[36249]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:38.339 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:38 smithi029 ceph-mon[36249]: Detected new or changed devices on smithi029 2024-05-14T09:28:38.339 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:38.340 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:38.340 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:38.340 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:38.340 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:38.340 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:38 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi029 to 5697M 2024-05-14T09:28:38.340 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:38.340 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:38.340 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:28:38.340 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:38.353 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:38 smithi026 ceph-mon[36056]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:38.353 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:38 smithi026 ceph-mon[36056]: Detected new or changed devices on smithi029 2024-05-14T09:28:38.353 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:38.353 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:38.353 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:38.353 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:38.353 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:28:38.353 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:38 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi029 to 5697M 2024-05-14T09:28:38.353 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:38.353 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:38.353 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:28:38.354 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.484+0000 7f9b72a5f700 1 -- 172.21.15.2:0/1702172096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b6c1054a0 msgr2=0x7f9b6c1058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:28:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.484+0000 7f9b72a5f700 1 --2- 172.21.15.2:0/1702172096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b6c1054a0 0x7f9b6c1058b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009990 tx=0x7f9b54009ca0 comp rx=0 tx=0).stop 2024-05-14T09:28:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.484+0000 7f9b72a5f700 1 -- 172.21.15.2:0/1702172096 shutdown_connections 2024-05-14T09:28:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.484+0000 7f9b72a5f700 1 --2- 172.21.15.2:0/1702172096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b6c105e40 0x7f9b6c10c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.484+0000 7f9b72a5f700 1 --2- 172.21.15.2:0/1702172096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b6c104370 0x7f9b6c104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.484+0000 7f9b72a5f700 1 --2- 172.21.15.2:0/1702172096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b6c1054a0 0x7f9b6c1058b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.484+0000 7f9b72a5f700 1 -- 172.21.15.2:0/1702172096 >> 172.21.15.2:0/1702172096 conn(0x7f9b6c0ff9d0 msgr2=0x7f9b6c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:28:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.485+0000 7f9b72a5f700 1 -- 172.21.15.2:0/1702172096 shutdown_connections 2024-05-14T09:28:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.485+0000 7f9b72a5f700 1 -- 172.21.15.2:0/1702172096 wait complete. 2024-05-14T09:28:38.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.485+0000 7f9b72a5f700 1 Processor -- start 2024-05-14T09:28:38.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.485+0000 7f9b72a5f700 1 -- start start 2024-05-14T09:28:38.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.485+0000 7f9b72a5f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b6c104370 0x7f9b6c078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:28:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.485+0000 7f9b72a5f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b6c1054a0 0x7f9b6c076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:28:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.485+0000 7f9b72a5f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b6c105e40 0x7f9b6c077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:28:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b72a5f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c10fdd0 con 0x7f9b6c104370 2024-05-14T09:28:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b72a5f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c10fc50 con 0x7f9b6c105e40 2024-05-14T09:28:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b72a5f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c10ff50 con 0x7f9b6c1054a0 2024-05-14T09:28:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b6bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b6c104370 0x7f9b6c078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:28:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b6bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b6c104370 0x7f9b6c078570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39272/0 (socket says 172.21.15.2:39272) 2024-05-14T09:28:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b6bfff700 1 -- 172.21.15.2:0/3785281519 learned_addr learned my addr 172.21.15.2:0/3785281519 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:28:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b70ffc700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b6c105e40 0x7f9b6c077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:28:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b63fff700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b6c1054a0 0x7f9b6c076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:28:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b6bfff700 1 -- 172.21.15.2:0/3785281519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b6c1054a0 msgr2=0x7f9b6c076c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:28:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b6bfff700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b6c1054a0 0x7f9b6c076c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b6bfff700 1 -- 172.21.15.2:0/3785281519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b6c105e40 msgr2=0x7f9b6c077160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:28:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b6bfff700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b6c105e40 0x7f9b6c077160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b6bfff700 1 -- 172.21.15.2:0/3785281519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6c077830 con 0x7f9b6c104370 2024-05-14T09:28:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b63fff700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b6c1054a0 0x7f9b6c076c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:28:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b70ffc700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b6c105e40 0x7f9b6c077160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:28:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b6bfff700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b6c104370 0x7f9b6c078570 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00c970 tx=0x7f9b5c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:28:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b69ffb700 1 -- 172.21.15.2:0/3785281519 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c007b00 con 0x7f9b6c104370 2024-05-14T09:28:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b72a5f700 1 -- 172.21.15.2:0/3785281519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6c1a4980 con 0x7f9b6c104370 2024-05-14T09:28:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b69ffb700 1 -- 172.21.15.2:0/3785281519 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b5c007ca0 con 0x7f9b6c104370 2024-05-14T09:28:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b69ffb700 1 -- 172.21.15.2:0/3785281519 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c018660 con 0x7f9b6c104370 2024-05-14T09:28:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.486+0000 7f9b72a5f700 1 -- 172.21.15.2:0/3785281519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b6c1a4dd0 con 0x7f9b6c104370 2024-05-14T09:28:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.487+0000 7f9b72a5f700 1 -- 172.21.15.2:0/3785281519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b6c0512c0 con 0x7f9b6c104370 2024-05-14T09:28:38.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.489+0000 7f9b69ffb700 1 -- 172.21.15.2:0/3785281519 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f9b5c00f450 con 0x7f9b6c104370 2024-05-14T09:28:38.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.490+0000 7f9b69ffb700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9b580b2880 0x7f9b580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:28:38.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.490+0000 7f9b63fff700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9b580b2880 0x7f9b580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:28:38.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.490+0000 7f9b69ffb700 1 -- 172.21.15.2:0/3785281519 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7f9b5c0e1890 con 0x7f9b6c104370 2024-05-14T09:28:38.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.490+0000 7f9b63fff700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9b580b2880 0x7f9b580b4d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400b6c0 tx=0x7f9b54005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:28:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.494+0000 7f9b69ffb700 1 -- 172.21.15.2:0/3785281519 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b5c014030 con 0x7f9b6c104370 2024-05-14T09:28:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:38.718+0000 7f9b72a5f700 1 -- 172.21.15.2:0/3785281519 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9b6c063c00 con 0x7f9b580b2880 2024-05-14T09:28:39.355 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:39 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:28:39.356 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:39 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:28:39.356 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:39 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:39.358 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:39 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:28:39.358 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:39 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:28:39.358 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:39 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:39.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:39 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:28:39.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:39 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:28:39.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:39 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:40.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:40 smithi002 ceph-mon[31557]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:28:40.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:40 smithi002 ceph-mon[31557]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-14T09:28:40.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:40 smithi029 ceph-mon[36249]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:28:40.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:40 smithi029 ceph-mon[36249]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-14T09:28:40.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:40 smithi026 ceph-mon[36056]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:28:40.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:40 smithi026 ceph-mon[36056]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-14T09:28:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:42 smithi002 ceph-mon[31557]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-14T09:28:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:42 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2817102279' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "119eeac9-29f7-4086-a1ad-04a0bc154b0e"}]: dispatch 2024-05-14T09:28:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:42 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2817102279' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "119eeac9-29f7-4086-a1ad-04a0bc154b0e"}]': finished 2024-05-14T09:28:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:42 smithi002 ceph-mon[31557]: osdmap e57: 10 total, 9 up, 10 in 2024-05-14T09:28:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:42 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:28:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:42 smithi029 ceph-mon[36249]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-14T09:28:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:42 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2817102279' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "119eeac9-29f7-4086-a1ad-04a0bc154b0e"}]: dispatch 2024-05-14T09:28:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:42 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2817102279' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "119eeac9-29f7-4086-a1ad-04a0bc154b0e"}]': finished 2024-05-14T09:28:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:42 smithi029 ceph-mon[36249]: osdmap e57: 10 total, 9 up, 10 in 2024-05-14T09:28:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:42 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:28:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:42 smithi026 ceph-mon[36056]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-14T09:28:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:42 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2817102279' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "119eeac9-29f7-4086-a1ad-04a0bc154b0e"}]: dispatch 2024-05-14T09:28:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:42 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2817102279' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "119eeac9-29f7-4086-a1ad-04a0bc154b0e"}]': finished 2024-05-14T09:28:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:42 smithi026 ceph-mon[36056]: osdmap e57: 10 total, 9 up, 10 in 2024-05-14T09:28:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:42 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:28:43.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:43 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/20139567' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:28:43.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:43 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/20139567' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:28:43.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:43 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/20139567' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:28:44.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:44 smithi002 ceph-mon[31557]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T09:28:44.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:44 smithi029 ceph-mon[36249]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T09:28:44.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:44 smithi026 ceph-mon[36056]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T09:28:46.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:46 smithi002 ceph-mon[31557]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-14T09:28:46.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:46 smithi029 ceph-mon[36249]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-14T09:28:46.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:46 smithi026 ceph-mon[36056]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-14T09:28:48.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:48 smithi002 ceph-mon[31557]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:28:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:48 smithi029 ceph-mon[36249]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:28:48.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:48 smithi026 ceph-mon[36056]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:28:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:50 smithi002 ceph-mon[31557]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:50.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:50 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-14T09:28:50.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:50 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:50.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:50 smithi002 ceph-mon[31557]: Deploying daemon osd.9 on smithi002 2024-05-14T09:28:50.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:50 smithi029 ceph-mon[36249]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:50.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:50 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-14T09:28:50.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:50 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:50.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:50 smithi029 ceph-mon[36249]: Deploying daemon osd.9 on smithi002 2024-05-14T09:28:50.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:50 smithi026 ceph-mon[36056]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:50.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:50 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-14T09:28:50.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:50 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:50.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:50 smithi026 ceph-mon[36056]: Deploying daemon osd.9 on smithi002 2024-05-14T09:28:52.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:52 smithi002 ceph-mon[31557]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:52.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:52 smithi029 ceph-mon[36249]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:52.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:52 smithi026 ceph-mon[36056]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:54 smithi002 ceph-mon[31557]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:54.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:54 smithi029 ceph-mon[36249]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:54.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:54 smithi026 ceph-mon[36056]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:56.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:56 smithi002 ceph-mon[31557]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:56.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:56 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:56.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:56 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:56.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:56 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:28:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:56 smithi029 ceph-mon[36249]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:56 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:56 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:56 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:28:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:56 smithi026 ceph-mon[36056]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:56 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:56 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:56 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:28:58.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:58 smithi002 ceph-mon[31557]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:58.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:58 smithi029 ceph-mon[36249]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:58.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:58 smithi026 ceph-mon[36056]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:28:59.230 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 9 on host 'smithi002' 2024-05-14T09:28:59.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:59.230+0000 7f9b69ffb700 1 -- 172.21.15.2:0/3785281519 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9b6c063c00 con 0x7f9b580b2880 2024-05-14T09:28:59.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:59.233+0000 7f9b72a5f700 1 -- 172.21.15.2:0/3785281519 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9b580b2880 msgr2=0x7f9b580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:28:59.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:59.233+0000 7f9b72a5f700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9b580b2880 0x7f9b580b4d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400b6c0 tx=0x7f9b54005fb0 comp rx=0 tx=0).stop 2024-05-14T09:28:59.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:59.233+0000 7f9b72a5f700 1 -- 172.21.15.2:0/3785281519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b6c104370 msgr2=0x7f9b6c078570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:28:59.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:59.233+0000 7f9b72a5f700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b6c104370 0x7f9b6c078570 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00c970 tx=0x7f9b5c00cc80 comp rx=0 tx=0).stop 2024-05-14T09:28:59.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:59.233+0000 7f9b72a5f700 1 -- 172.21.15.2:0/3785281519 shutdown_connections 2024-05-14T09:28:59.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:59.233+0000 7f9b72a5f700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9b580b2880 0x7f9b580b4d90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:59.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:59.233+0000 7f9b72a5f700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b6c105e40 0x7f9b6c077160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:59.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:59.233+0000 7f9b72a5f700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b6c104370 0x7f9b6c078570 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:59.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:59.233+0000 7f9b72a5f700 1 --2- 172.21.15.2:0/3785281519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b6c1054a0 0x7f9b6c076c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:28:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:59.233+0000 7f9b72a5f700 1 -- 172.21.15.2:0/3785281519 >> 172.21.15.2:0/3785281519 conn(0x7f9b6c0ff9d0 msgr2=0x7f9b6c1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:28:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:59.234+0000 7f9b72a5f700 1 -- 172.21.15.2:0/3785281519 shutdown_connections 2024-05-14T09:28:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:28:59.234+0000 7f9b72a5f700 1 -- 172.21.15.2:0/3785281519 wait complete. 2024-05-14T09:28:59.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:59.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:59.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:59.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:28:59.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:59.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:59 smithi002 ceph-mon[31557]: from='osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-14T09:28:59.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:59.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:28:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:59.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:59.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:59.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:59.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:28:59.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:59.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:59 smithi029 ceph-mon[36249]: from='osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-14T09:28:59.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:59.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:28:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:59.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:59.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:59.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:28:59.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:28:59.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:59.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:59 smithi026 ceph-mon[36056]: from='osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-14T09:28:59.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:59.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:28:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:28:59.915 DEBUG:teuthology.orchestra.run.smithi002:osd.9> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.9.service 2024-05-14T09:28:59.917 INFO:tasks.cephadm:Deploying osd.10 on smithi026 with /dev/vg_nvme/lv_1... 2024-05-14T09:28:59.918 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-14T09:29:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T09:29:00.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:00 smithi002 ceph-mon[31557]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:29:00.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:00 smithi002 ceph-mon[31557]: from='osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-14T09:29:00.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:00 smithi002 ceph-mon[31557]: osdmap e58: 10 total, 9 up, 10 in 2024-05-14T09:29:00.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:00 smithi002 ceph-mon[31557]: from='osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T09:29:00.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:00 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:29:00.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:00 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:29:00.539 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:00 smithi026 ceph-mon[36056]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:29:00.539 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:00 smithi026 ceph-mon[36056]: from='osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-14T09:29:00.539 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:00 smithi026 ceph-mon[36056]: osdmap e58: 10 total, 9 up, 10 in 2024-05-14T09:29:00.539 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:00 smithi026 ceph-mon[36056]: from='osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T09:29:00.540 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:00 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:29:00.540 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:00 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:29:00.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:00 smithi029 ceph-mon[36249]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:29:00.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:00 smithi029 ceph-mon[36249]: from='osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-14T09:29:00.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:00 smithi029 ceph-mon[36249]: osdmap e58: 10 total, 9 up, 10 in 2024-05-14T09:29:00.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:00 smithi029 ceph-mon[36249]: from='osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T09:29:00.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:00 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:29:00.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:00 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:29:01.138 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 09:29:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T09:29:00.725+0000 7f9cfd73e700 -1 osd.9 0 waiting for initial osdmap 2024-05-14T09:29:01.138 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 09:29:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T09:29:00.737+0000 7f9cf4cea700 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T09:29:01.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:01 smithi002 ceph-mon[31557]: from='osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T09:29:01.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:01 smithi002 ceph-mon[31557]: osdmap e59: 10 total, 9 up, 10 in 2024-05-14T09:29:01.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:29:01.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:01 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:29:01.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:01 smithi002 ceph-mon[31557]: from='osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226]' entity='osd.9' 2024-05-14T09:29:01.505 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:01 smithi026 ceph-mon[36056]: from='osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T09:29:01.505 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:01 smithi026 ceph-mon[36056]: osdmap e59: 10 total, 9 up, 10 in 2024-05-14T09:29:01.505 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:01 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:29:01.505 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:01 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:29:01.505 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:01 smithi026 ceph-mon[36056]: from='osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226]' entity='osd.9' 2024-05-14T09:29:01.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:01 smithi029 ceph-mon[36249]: from='osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T09:29:01.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:01 smithi029 ceph-mon[36249]: osdmap e59: 10 total, 9 up, 10 in 2024-05-14T09:29:01.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:01 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:29:01.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:01 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:29:01.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:01 smithi029 ceph-mon[36249]: from='osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226]' entity='osd.9' 2024-05-14T09:29:02.418 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-14T09:29:02.418 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-14T09:29:02.418 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-05-14T09:29:02.418 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-05-14T09:29:02.418 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00842533 s, 1.2 GB/s 2024-05-14T09:29:02.418 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-05-14T09:29:02.484 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:02 smithi026 ceph-mon[36056]: purged_snaps scrub starts 2024-05-14T09:29:02.485 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:02 smithi026 ceph-mon[36056]: purged_snaps scrub ok 2024-05-14T09:29:02.485 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:02 smithi026 ceph-mon[36056]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:29:02.485 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:02 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:29:02.485 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:02 smithi026 ceph-mon[36056]: osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226] boot 2024-05-14T09:29:02.485 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:02 smithi026 ceph-mon[36056]: osdmap e60: 10 total, 10 up, 10 in 2024-05-14T09:29:02.485 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:02 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:29:02.586 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:02 smithi002 ceph-mon[31557]: purged_snaps scrub starts 2024-05-14T09:29:02.587 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:02 smithi002 ceph-mon[31557]: purged_snaps scrub ok 2024-05-14T09:29:02.587 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:02 smithi002 ceph-mon[31557]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:29:02.587 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:02 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:29:02.587 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:02 smithi002 ceph-mon[31557]: osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226] boot 2024-05-14T09:29:02.587 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:02 smithi002 ceph-mon[31557]: osdmap e60: 10 total, 10 up, 10 in 2024-05-14T09:29:02.587 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:02 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:29:02.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:02 smithi029 ceph-mon[36249]: purged_snaps scrub starts 2024-05-14T09:29:02.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:02 smithi029 ceph-mon[36249]: purged_snaps scrub ok 2024-05-14T09:29:02.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:02 smithi029 ceph-mon[36249]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-05-14T09:29:02.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:02 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:29:02.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:02 smithi029 ceph-mon[36249]: osd.9 [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226] boot 2024-05-14T09:29:02.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:02 smithi029 ceph-mon[36249]: osdmap e60: 10 total, 10 up, 10 in 2024-05-14T09:29:02.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:02 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T09:29:03.122 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_1 2024-05-14T09:29:03.374 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T09:29:04.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:03 smithi002 ceph-mon[31557]: osdmap e61: 10 total, 10 up, 10 in 2024-05-14T09:29:04.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:03 smithi002 ceph-mon[31557]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:04.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:03 smithi002 ceph-mon[31557]: Detected new or changed devices on smithi002 2024-05-14T09:29:04.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:03 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:04.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:03 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:04.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:03 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:04.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:03 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:04.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:03 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:04.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:03 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:04.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:03 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi002 to 4273M 2024-05-14T09:29:04.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:03 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:04.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:03 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:04.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:03 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:29:04.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:03 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:04.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:03 smithi029 ceph-mon[36249]: osdmap e61: 10 total, 10 up, 10 in 2024-05-14T09:29:04.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:03 smithi029 ceph-mon[36249]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:04.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:03 smithi029 ceph-mon[36249]: Detected new or changed devices on smithi002 2024-05-14T09:29:04.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:03 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:04.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:03 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:04.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:03 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:04.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:03 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:04.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:03 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:04.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:03 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:04.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:03 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi002 to 4273M 2024-05-14T09:29:04.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:03 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:04.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:03 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:04.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:03 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:29:04.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:03 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:04.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:03 smithi026 ceph-mon[36056]: osdmap e61: 10 total, 10 up, 10 in 2024-05-14T09:29:04.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:03 smithi026 ceph-mon[36056]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:04.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:03 smithi026 ceph-mon[36056]: Detected new or changed devices on smithi002 2024-05-14T09:29:04.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:03 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:04.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:03 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:04.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:03 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:04.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:03 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:04.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:03 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:04.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:03 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:04.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:03 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi002 to 4273M 2024-05-14T09:29:04.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:03 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:04.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:03 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:04.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:03 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:29:04.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:03 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.009+0000 7f5671014700 1 -- 172.21.15.26:0/372166144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f566c103260 msgr2=0x7f566c103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.009+0000 7f5671014700 1 --2- 172.21.15.26:0/372166144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f566c103260 0x7f566c103670 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f565c009990 tx=0x7f565c009ca0 comp rx=0 tx=0).stop 2024-05-14T09:29:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.009+0000 7f5671014700 1 -- 172.21.15.26:0/372166144 shutdown_connections 2024-05-14T09:29:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.009+0000 7f5671014700 1 --2- 172.21.15.26:0/372166144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f566c103260 0x7f566c103670 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.009+0000 7f5671014700 1 --2- 172.21.15.26:0/372166144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f566c104d30 0x7f566c10b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.009+0000 7f5671014700 1 --2- 172.21.15.26:0/372166144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f566c104390 0x7f566c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.009+0000 7f5671014700 1 -- 172.21.15.26:0/372166144 >> 172.21.15.26:0/372166144 conn(0x7f566c0fe8c0 msgr2=0x7f566c100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:29:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.009+0000 7f5671014700 1 -- 172.21.15.26:0/372166144 shutdown_connections 2024-05-14T09:29:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.009+0000 7f5671014700 1 -- 172.21.15.26:0/372166144 wait complete. 2024-05-14T09:29:05.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.010+0000 7f5671014700 1 Processor -- start 2024-05-14T09:29:05.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.010+0000 7f5671014700 1 -- start start 2024-05-14T09:29:05.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.010+0000 7f5671014700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f566c103260 0x7f566c076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:05.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.010+0000 7f5671014700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f566c104390 0x7f566c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:05.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.010+0000 7f5671014700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f566c104d30 0x7f566c1a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:05.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.010+0000 7f5671014700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f566c10eca0 con 0x7f566c103260 2024-05-14T09:29:05.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.010+0000 7f5671014700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f566c10eb20 con 0x7f566c104d30 2024-05-14T09:29:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.010+0000 7f5671014700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f566c10ee20 con 0x7f566c104390 2024-05-14T09:29:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.010+0000 7f566b59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f566c104d30 0x7f566c1a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.010+0000 7f566b59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f566c104d30 0x7f566c1a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35578/0 (socket says 172.21.15.26:35578) 2024-05-14T09:29:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.010+0000 7f566b59e700 1 -- 172.21.15.26:0/2843985494 learned_addr learned my addr 172.21.15.26:0/2843985494 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-05-14T09:29:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.010+0000 7f566ad9d700 1 --2- 172.21.15.26:0/2843985494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f566c103260 0x7f566c076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.010+0000 7f566a59c700 1 --2- 172.21.15.26:0/2843985494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f566c104390 0x7f566c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.010+0000 7f566b59e700 1 -- 172.21.15.26:0/2843985494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f566c104390 msgr2=0x7f566c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.011+0000 7f566b59e700 1 --2- 172.21.15.26:0/2843985494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f566c104390 0x7f566c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.011+0000 7f566b59e700 1 -- 172.21.15.26:0/2843985494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f566c103260 msgr2=0x7f566c076450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.011+0000 7f566b59e700 1 --2- 172.21.15.26:0/2843985494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f566c103260 0x7f566c076450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.011+0000 7f566b59e700 1 -- 172.21.15.26:0/2843985494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c1a24a0 con 0x7f566c104d30 2024-05-14T09:29:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.011+0000 7f566b59e700 1 --2- 172.21.15.26:0/2843985494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f566c104d30 0x7f566c1a1c70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f566400ada0 tx=0x7f566400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:29:05.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.011+0000 7f5653fff700 1 -- 172.21.15.26:0/2843985494 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664017020 con 0x7f566c104d30 2024-05-14T09:29:05.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.011+0000 7f5671014700 1 -- 172.21.15.26:0/2843985494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566c1a2790 con 0x7f566c104d30 2024-05-14T09:29:05.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.011+0000 7f5653fff700 1 -- 172.21.15.26:0/2843985494 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566400cd80 con 0x7f566c104d30 2024-05-14T09:29:05.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.011+0000 7f5653fff700 1 -- 172.21.15.26:0/2843985494 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664005260 con 0x7f566c104d30 2024-05-14T09:29:05.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.011+0000 7f5671014700 1 -- 172.21.15.26:0/2843985494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566c1a2c80 con 0x7f566c104d30 2024-05-14T09:29:05.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.012+0000 7f5671014700 1 -- 172.21.15.26:0/2843985494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f566c0512c0 con 0x7f566c104d30 2024-05-14T09:29:05.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.014+0000 7f5653fff700 1 -- 172.21.15.26:0/2843985494 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f566400a090 con 0x7f566c104d30 2024-05-14T09:29:05.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.015+0000 7f5653fff700 1 --2- 172.21.15.26:0/2843985494 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f56540b2880 0x7f56540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:05.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.015+0000 7f5653fff700 1 -- 172.21.15.26:0/2843985494 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7f56640e1750 con 0x7f566c104d30 2024-05-14T09:29:05.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.015+0000 7f566ad9d700 1 --2- 172.21.15.26:0/2843985494 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f56540b2880 0x7f56540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:05.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.015+0000 7f566ad9d700 1 --2- 172.21.15.26:0/2843985494 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f56540b2880 0x7f56540b4d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f565c0098e0 tx=0x7f565c00b680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:29:05.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.019+0000 7f5653fff700 1 -- 172.21.15.26:0/2843985494 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5664021020 con 0x7f566c104d30 2024-05-14T09:29:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:05.240+0000 7f5671014700 1 -- 172.21.15.26:0/2843985494 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f566c063c00 con 0x7f56540b2880 2024-05-14T09:29:06.141 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:06 smithi026 ceph-mon[36056]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:06.141 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:06 smithi026 ceph-mon[36056]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:29:06.141 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:06 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:29:06.141 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:06 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:29:06.142 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:06 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:06.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:06 smithi002 ceph-mon[31557]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:06.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:06 smithi002 ceph-mon[31557]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:29:06.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:06 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:29:06.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:06 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:29:06.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:06 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:06.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:06 smithi029 ceph-mon[36249]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:06.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:06 smithi029 ceph-mon[36249]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:29:06.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:06 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:29:06.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:06 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:29:06.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:06 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:08.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:08 smithi002 ceph-mon[31557]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:08.436 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:08 smithi029 ceph-mon[36249]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:08.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:08 smithi026 ceph-mon[36056]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:09.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:09 smithi002 ceph-mon[31557]: from='client.? 172.21.15.26:0/1406371414' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce"}]: dispatch 2024-05-14T09:29:09.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:09 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce"}]: dispatch 2024-05-14T09:29:09.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:09 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce"}]': finished 2024-05-14T09:29:09.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:09 smithi002 ceph-mon[31557]: osdmap e62: 11 total, 10 up, 11 in 2024-05-14T09:29:09.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:09 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T09:29:09.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:09 smithi002 ceph-mon[31557]: from='client.? 172.21.15.26:0/177774332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:29:09.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:09 smithi029 ceph-mon[36249]: from='client.? 172.21.15.26:0/1406371414' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce"}]: dispatch 2024-05-14T09:29:09.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:09 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce"}]: dispatch 2024-05-14T09:29:09.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:09 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce"}]': finished 2024-05-14T09:29:09.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:09 smithi029 ceph-mon[36249]: osdmap e62: 11 total, 10 up, 11 in 2024-05-14T09:29:09.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:09 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T09:29:09.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:09 smithi029 ceph-mon[36249]: from='client.? 172.21.15.26:0/177774332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:29:09.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:09 smithi026 ceph-mon[36056]: from='client.? 172.21.15.26:0/1406371414' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce"}]: dispatch 2024-05-14T09:29:09.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:09 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce"}]: dispatch 2024-05-14T09:29:09.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:09 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce"}]': finished 2024-05-14T09:29:09.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:09 smithi026 ceph-mon[36056]: osdmap e62: 11 total, 10 up, 11 in 2024-05-14T09:29:09.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:09 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T09:29:09.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:09 smithi026 ceph-mon[36056]: from='client.? 172.21.15.26:0/177774332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:29:10.499 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:10 smithi029 ceph-mon[36249]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:10 smithi002 ceph-mon[31557]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:10.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:10 smithi026 ceph-mon[36056]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:12.648 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:12 smithi026 ceph-mon[36056]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:12.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:12 smithi002 ceph-mon[31557]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:12.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:12 smithi029 ceph-mon[36249]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:14.513 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:14 smithi026 ceph-mon[36056]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:14.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:14 smithi002 ceph-mon[31557]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:14.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:14 smithi029 ceph-mon[36249]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:16.597 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:16 smithi026 ceph-mon[36056]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:16.597 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:16 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-14T09:29:16.597 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:16 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:16.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:16 smithi002 ceph-mon[31557]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:16.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:16 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-14T09:29:16.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:16 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:16 smithi029 ceph-mon[36249]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:16 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-14T09:29:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:16 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:17.529 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:17 smithi026 ceph-mon[36056]: Deploying daemon osd.10 on smithi026 2024-05-14T09:29:17.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:17 smithi002 ceph-mon[31557]: Deploying daemon osd.10 on smithi026 2024-05-14T09:29:17.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:17 smithi029 ceph-mon[36249]: Deploying daemon osd.10 on smithi026 2024-05-14T09:29:18.525 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:18 smithi026 ceph-mon[36056]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:18.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:18 smithi002 ceph-mon[31557]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:18.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:18 smithi029 ceph-mon[36249]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:20.519 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:20 smithi026 ceph-mon[36056]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:20.572 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:20 smithi029 ceph-mon[36249]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:20 smithi002 ceph-mon[31557]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:22.576 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:22 smithi026 ceph-mon[36056]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:22.577 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:22 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:22.577 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:22 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:22.577 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:22 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:29:22.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:22 smithi002 ceph-mon[31557]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:22.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:22 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:22.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:22 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:22.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:22 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:29:22.690 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:22 smithi029 ceph-mon[36249]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:22.690 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:22 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:22.690 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:22 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:22.690 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:22 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:29:24.601 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:24 smithi026 ceph-mon[36056]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:24.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:24 smithi002 ceph-mon[31557]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:24.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:24 smithi029 ceph-mon[36249]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:25.812 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 10 on host 'smithi026' 2024-05-14T09:29:25.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:25.811+0000 7f5653fff700 1 -- 172.21.15.26:0/2843985494 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f566c063c00 con 0x7f56540b2880 2024-05-14T09:29:25.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:25.814+0000 7f5671014700 1 -- 172.21.15.26:0/2843985494 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f56540b2880 msgr2=0x7f56540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:25.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:25.814+0000 7f5671014700 1 --2- 172.21.15.26:0/2843985494 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f56540b2880 0x7f56540b4d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f565c0098e0 tx=0x7f565c00b680 comp rx=0 tx=0).stop 2024-05-14T09:29:25.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:25.814+0000 7f5671014700 1 -- 172.21.15.26:0/2843985494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f566c104d30 msgr2=0x7f566c1a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:25.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:25.814+0000 7f5671014700 1 --2- 172.21.15.26:0/2843985494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f566c104d30 0x7f566c1a1c70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f566400ada0 tx=0x7f566400c330 comp rx=0 tx=0).stop 2024-05-14T09:29:25.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:25.814+0000 7f5671014700 1 -- 172.21.15.26:0/2843985494 shutdown_connections 2024-05-14T09:29:25.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:25.814+0000 7f5671014700 1 --2- 172.21.15.26:0/2843985494 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f56540b2880 0x7f56540b4d90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:25.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:25.814+0000 7f5671014700 1 --2- 172.21.15.26:0/2843985494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f566c104d30 0x7f566c1a1c70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:25.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:25.814+0000 7f5671014700 1 --2- 172.21.15.26:0/2843985494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f566c103260 0x7f566c076450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:25.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:25.814+0000 7f5671014700 1 --2- 172.21.15.26:0/2843985494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f566c104390 0x7f566c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:25.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:25.814+0000 7f5671014700 1 -- 172.21.15.26:0/2843985494 >> 172.21.15.26:0/2843985494 conn(0x7f566c0fe8c0 msgr2=0x7f566c1075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:29:25.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:25.814+0000 7f5671014700 1 -- 172.21.15.26:0/2843985494 shutdown_connections 2024-05-14T09:29:25.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-05-14T09:29:25.814+0000 7f5671014700 1 -- 172.21.15.26:0/2843985494 wait complete. 2024-05-14T09:29:26.143 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:25 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:26.143 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:25 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:26.143 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:25 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:26.143 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:25 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:29:26.143 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:25 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:26.143 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:25 smithi026 ceph-mon[36056]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:26.143 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:25 smithi026 ceph-mon[36056]: from='osd.10 [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T09:29:26.143 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:25 smithi026 ceph-mon[36056]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T09:29:26.144 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:25 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:29:26.144 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:25 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:26.144 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:25 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:26.174 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:25 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:26.174 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:25 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:26.174 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:25 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:26.174 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:25 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:29:26.174 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:25 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:26.174 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:25 smithi002 ceph-mon[31557]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:26.175 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:25 smithi002 ceph-mon[31557]: from='osd.10 [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T09:29:26.175 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:25 smithi002 ceph-mon[31557]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T09:29:26.175 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:25 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:29:26.175 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:25 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:26.175 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:25 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:26.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:25 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:26.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:25 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:26.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:25 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:26.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:25 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:29:26.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:25 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:26.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:25 smithi029 ceph-mon[36249]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:26.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:25 smithi029 ceph-mon[36249]: from='osd.10 [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T09:29:26.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:25 smithi029 ceph-mon[36249]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T09:29:26.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:25 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:29:26.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:25 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:26.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:25 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:26.489 DEBUG:teuthology.orchestra.run.smithi026:osd.10> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.10.service 2024-05-14T09:29:26.492 INFO:tasks.cephadm:Deploying osd.11 on smithi029 with /dev/vg_nvme/lv_1... 2024-05-14T09:29:26.492 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-14T09:29:26.750 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T09:29:27.123 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:26 smithi029 ceph-mon[36249]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-14T09:29:27.123 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:26 smithi029 ceph-mon[36249]: osdmap e63: 11 total, 10 up, 11 in 2024-05-14T09:29:27.123 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:26 smithi029 ceph-mon[36249]: from='osd.10 [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:29:27.123 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:26 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T09:29:27.123 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:26 smithi029 ceph-mon[36249]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:29:27.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:26 smithi026 ceph-mon[36056]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-14T09:29:27.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:26 smithi026 ceph-mon[36056]: osdmap e63: 11 total, 10 up, 11 in 2024-05-14T09:29:27.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:26 smithi026 ceph-mon[36056]: from='osd.10 [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:29:27.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:26 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T09:29:27.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:26 smithi026 ceph-mon[36056]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:29:27.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:26 smithi002 ceph-mon[31557]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-14T09:29:27.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:26 smithi002 ceph-mon[31557]: osdmap e63: 11 total, 10 up, 11 in 2024-05-14T09:29:27.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:26 smithi002 ceph-mon[31557]: from='osd.10 [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:29:27.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:26 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T09:29:27.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:26 smithi002 ceph-mon[31557]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-05-14T09:29:27.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 09:29:27 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T09:29:27.431+0000 7fb3ece37700 -1 osd.10 0 waiting for initial osdmap 2024-05-14T09:29:27.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 09:29:27 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T09:29:27.444+0000 7fb3e43e3700 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T09:29:28.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:27 smithi026 ceph-mon[36056]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-14T09:29:28.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:27 smithi026 ceph-mon[36056]: osdmap e64: 11 total, 10 up, 11 in 2024-05-14T09:29:28.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:27 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T09:29:28.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:27 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T09:29:28.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:27 smithi026 ceph-mon[36056]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:28.017 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:27 smithi026 ceph-mon[36056]: from='osd.10 ' entity='osd.10' 2024-05-14T09:29:28.017 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:27 smithi026 ceph-mon[36056]: osd.10 [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515] boot 2024-05-14T09:29:28.017 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:27 smithi026 ceph-mon[36056]: osdmap e65: 11 total, 11 up, 11 in 2024-05-14T09:29:28.017 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:27 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T09:29:28.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:27 smithi029 ceph-mon[36249]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-14T09:29:28.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:27 smithi029 ceph-mon[36249]: osdmap e64: 11 total, 10 up, 11 in 2024-05-14T09:29:28.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:27 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T09:29:28.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:27 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T09:29:28.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:27 smithi029 ceph-mon[36249]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:28.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:27 smithi029 ceph-mon[36249]: from='osd.10 ' entity='osd.10' 2024-05-14T09:29:28.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:27 smithi029 ceph-mon[36249]: osd.10 [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515] boot 2024-05-14T09:29:28.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:27 smithi029 ceph-mon[36249]: osdmap e65: 11 total, 11 up, 11 in 2024-05-14T09:29:28.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:27 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T09:29:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:27 smithi002 ceph-mon[31557]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-05-14T09:29:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:27 smithi002 ceph-mon[31557]: osdmap e64: 11 total, 10 up, 11 in 2024-05-14T09:29:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:27 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T09:29:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:27 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T09:29:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:27 smithi002 ceph-mon[31557]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 894 GiB / 894 GiB avail 2024-05-14T09:29:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:27 smithi002 ceph-mon[31557]: from='osd.10 ' entity='osd.10' 2024-05-14T09:29:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:27 smithi002 ceph-mon[31557]: osd.10 [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515] boot 2024-05-14T09:29:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:27 smithi002 ceph-mon[31557]: osdmap e65: 11 total, 11 up, 11 in 2024-05-14T09:29:28.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:27 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T09:29:28.949 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-14T09:29:28.950 INFO:teuthology.orchestra.run.smithi029.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-14T09:29:28.950 INFO:teuthology.orchestra.run.smithi029.stderr: stderr: 10+0 records in 2024-05-14T09:29:28.950 INFO:teuthology.orchestra.run.smithi029.stderr:10+0 records out 2024-05-14T09:29:28.950 INFO:teuthology.orchestra.run.smithi029.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00942118 s, 1.1 GB/s 2024-05-14T09:29:28.950 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping successful for: 2024-05-14T09:29:29.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:28 smithi026 ceph-mon[36056]: purged_snaps scrub starts 2024-05-14T09:29:29.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:28 smithi026 ceph-mon[36056]: purged_snaps scrub ok 2024-05-14T09:29:29.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:28 smithi026 ceph-mon[36056]: osdmap e66: 11 total, 11 up, 11 in 2024-05-14T09:29:29.337 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:28 smithi029 ceph-mon[36249]: purged_snaps scrub starts 2024-05-14T09:29:29.337 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:28 smithi029 ceph-mon[36249]: purged_snaps scrub ok 2024-05-14T09:29:29.337 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:28 smithi029 ceph-mon[36249]: osdmap e66: 11 total, 11 up, 11 in 2024-05-14T09:29:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:28 smithi002 ceph-mon[31557]: purged_snaps scrub starts 2024-05-14T09:29:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:28 smithi002 ceph-mon[31557]: purged_snaps scrub ok 2024-05-14T09:29:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:28 smithi002 ceph-mon[31557]: osdmap e66: 11 total, 11 up, 11 in 2024-05-14T09:29:29.802 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi029:vg_nvme/lv_1 2024-05-14T09:29:30.070 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T09:29:30.105 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:29 smithi029 ceph-mon[36249]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 354 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:30.105 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:30.105 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:30.105 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:30.106 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:30.106 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:30.106 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:30.106 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:30.106 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:30.106 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:29:30.106 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:30.106 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:29 smithi029 ceph-mon[36249]: osdmap e67: 11 total, 11 up, 11 in 2024-05-14T09:29:30.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:29 smithi026 ceph-mon[36056]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 354 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:30.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:30.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:30.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:30.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:30.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:30.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:30.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:30.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:30.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:29:30.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:30.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:29 smithi026 ceph-mon[36056]: osdmap e67: 11 total, 11 up, 11 in 2024-05-14T09:29:30.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:29 smithi002 ceph-mon[31557]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 354 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:30.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:30.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:30.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:30.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:30.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:30.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:30.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:30.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:30.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:29:30.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:30.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:29 smithi002 ceph-mon[31557]: osdmap e67: 11 total, 11 up, 11 in 2024-05-14T09:29:31.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:30 smithi029 ceph-mon[36249]: Detected new or changed devices on smithi026 2024-05-14T09:29:31.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:30 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi026 to 4273M 2024-05-14T09:29:31.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:30 smithi002 ceph-mon[31557]: Detected new or changed devices on smithi026 2024-05-14T09:29:31.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:30 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi026 to 4273M 2024-05-14T09:29:31.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:30 smithi026 ceph-mon[36056]: Detected new or changed devices on smithi026 2024-05-14T09:29:31.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:30 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi026 to 4273M 2024-05-14T09:29:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.788+0000 7fe060237700 1 -- 172.21.15.29:0/1375649118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe058103250 msgr2=0x7fe058103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.788+0000 7fe060237700 1 --2- 172.21.15.29:0/1375649118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe058103250 0x7fe058103660 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe054009990 tx=0x7fe054009ca0 comp rx=0 tx=0).stop 2024-05-14T09:29:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.788+0000 7fe060237700 1 -- 172.21.15.29:0/1375649118 shutdown_connections 2024-05-14T09:29:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.788+0000 7fe060237700 1 --2- 172.21.15.29:0/1375649118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe058104d20 0x7fe05810b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:31.789 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.788+0000 7fe060237700 1 --2- 172.21.15.29:0/1375649118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe058103250 0x7fe058103660 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:31.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.788+0000 7fe060237700 1 --2- 172.21.15.29:0/1375649118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe058104380 0x7fe058104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:31.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.788+0000 7fe060237700 1 -- 172.21.15.29:0/1375649118 >> 172.21.15.29:0/1375649118 conn(0x7fe0580fe890 msgr2=0x7fe058100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:29:31.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.788+0000 7fe060237700 1 -- 172.21.15.29:0/1375649118 shutdown_connections 2024-05-14T09:29:31.790 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.788+0000 7fe060237700 1 -- 172.21.15.29:0/1375649118 wait complete. 2024-05-14T09:29:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.789+0000 7fe060237700 1 Processor -- start 2024-05-14T09:29:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.789+0000 7fe060237700 1 -- start start 2024-05-14T09:29:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.789+0000 7fe060237700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe058103250 0x7fe058076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:31.792 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.789+0000 7fe060237700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe058104380 0x7fe05819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.789+0000 7fe060237700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe058104d20 0x7fe0581a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.789+0000 7fe060237700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe05810ecf0 con 0x7fe058103250 2024-05-14T09:29:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.789+0000 7fe060237700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe05810eb70 con 0x7fe058104d20 2024-05-14T09:29:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.789+0000 7fe060237700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe05810ee70 con 0x7fe058104380 2024-05-14T09:29:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.789+0000 7fe05d7d2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe058104380 0x7fe05819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.789+0000 7fe05d7d2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe058104380 0x7fe05819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:59248/0 (socket says 172.21.15.29:59248) 2024-05-14T09:29:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.789+0000 7fe05d7d2700 1 -- 172.21.15.29:0/3649913581 learned_addr learned my addr 172.21.15.29:0/3649913581 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-05-14T09:29:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.789+0000 7fe05dfd3700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe058103250 0x7fe058076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.790+0000 7fe05e7d4700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe058104d20 0x7fe0581a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.790+0000 7fe05d7d2700 1 -- 172.21.15.29:0/3649913581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe058104d20 msgr2=0x7fe0581a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:31.793 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.790+0000 7fe05d7d2700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe058104d20 0x7fe0581a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.790+0000 7fe05d7d2700 1 -- 172.21.15.29:0/3649913581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe058103250 msgr2=0x7fe058076450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.790+0000 7fe05d7d2700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe058103250 0x7fe058076450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.790+0000 7fe05d7d2700 1 -- 172.21.15.29:0/3649913581 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0581a24f0 con 0x7fe058104380 2024-05-14T09:29:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.790+0000 7fe05e7d4700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe058104d20 0x7fe0581a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:29:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.790+0000 7fe05dfd3700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe058103250 0x7fe058076450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:29:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.790+0000 7fe05d7d2700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe058104380 0x7fe05819f750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe04800d980 tx=0x7fe04800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:29:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.790+0000 7fe04effd700 1 -- 172.21.15.29:0/3649913581 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe048016070 con 0x7fe058104380 2024-05-14T09:29:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.790+0000 7fe060237700 1 -- 172.21.15.29:0/3649913581 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0581a27e0 con 0x7fe058104380 2024-05-14T09:29:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.790+0000 7fe04effd700 1 -- 172.21.15.29:0/3649913581 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe048004830 con 0x7fe058104380 2024-05-14T09:29:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.790+0000 7fe04effd700 1 -- 172.21.15.29:0/3649913581 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0480054c0 con 0x7fe058104380 2024-05-14T09:29:31.794 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.790+0000 7fe060237700 1 -- 172.21.15.29:0/3649913581 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0581a2d70 con 0x7fe058104380 2024-05-14T09:29:31.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.791+0000 7fe060237700 1 -- 172.21.15.29:0/3649913581 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0580512c0 con 0x7fe058104380 2024-05-14T09:29:31.795 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.794+0000 7fe04effd700 1 -- 172.21.15.29:0/3649913581 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fe048004220 con 0x7fe058104380 2024-05-14T09:29:31.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.794+0000 7fe04effd700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0440b2880 0x7fe0440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:31.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.795+0000 7fe04effd700 1 -- 172.21.15.29:0/3649913581 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6694+0+0 (secure 0 0 0) 0x7fe0480e1ff0 con 0x7fe058104380 2024-05-14T09:29:31.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.795+0000 7fe05dfd3700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0440b2880 0x7fe0440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:31.796 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.795+0000 7fe05dfd3700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0440b2880 0x7fe0440b4d90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe05400b6c0 tx=0x7fe05400b2a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:29:31.798 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:31.798+0000 7fe04effd700 1 -- 172.21.15.29:0/3649913581 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe04801f020 con 0x7fe058104380 2024-05-14T09:29:32.021 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:32.021+0000 7fe060237700 1 -- 172.21.15.29:0/3649913581 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe058063c00 con 0x7fe0440b2880 2024-05-14T09:29:32.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:31 smithi029 ceph-mon[36249]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 354 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:32.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:31 smithi002 ceph-mon[31557]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 354 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:32.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:31 smithi026 ceph-mon[36056]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 354 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:33.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:32 smithi029 ceph-mon[36249]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:29:33.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:29:33.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:29:33.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:33.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:32 smithi002 ceph-mon[31557]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:29:33.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:29:33.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:29:33.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:33.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:32 smithi026 ceph-mon[36056]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:29:33.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T09:29:33.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T09:29:33.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:34.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:33 smithi002 ceph-mon[31557]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T09:29:34.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:33 smithi029 ceph-mon[36249]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T09:29:34.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:33 smithi026 ceph-mon[36056]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T09:29:35.280 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:34 smithi029 ceph-mon[36249]: from='client.? 172.21.15.29:0/3546682164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5246763-c4f6-4ae3-8a93-5929c541ac5d"}]: dispatch 2024-05-14T09:29:35.280 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:34 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5246763-c4f6-4ae3-8a93-5929c541ac5d"}]: dispatch 2024-05-14T09:29:35.280 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:34 smithi029 ceph-mon[36249]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5246763-c4f6-4ae3-8a93-5929c541ac5d"}]': finished 2024-05-14T09:29:35.280 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:34 smithi029 ceph-mon[36249]: osdmap e68: 12 total, 11 up, 12 in 2024-05-14T09:29:35.280 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:34 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T09:29:35.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:34 smithi002 ceph-mon[31557]: from='client.? 172.21.15.29:0/3546682164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5246763-c4f6-4ae3-8a93-5929c541ac5d"}]: dispatch 2024-05-14T09:29:35.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:34 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5246763-c4f6-4ae3-8a93-5929c541ac5d"}]: dispatch 2024-05-14T09:29:35.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:34 smithi002 ceph-mon[31557]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5246763-c4f6-4ae3-8a93-5929c541ac5d"}]': finished 2024-05-14T09:29:35.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:34 smithi002 ceph-mon[31557]: osdmap e68: 12 total, 11 up, 12 in 2024-05-14T09:29:35.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:34 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T09:29:35.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:34 smithi026 ceph-mon[36056]: from='client.? 172.21.15.29:0/3546682164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5246763-c4f6-4ae3-8a93-5929c541ac5d"}]: dispatch 2024-05-14T09:29:35.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:34 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5246763-c4f6-4ae3-8a93-5929c541ac5d"}]: dispatch 2024-05-14T09:29:35.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:34 smithi026 ceph-mon[36056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5246763-c4f6-4ae3-8a93-5929c541ac5d"}]': finished 2024-05-14T09:29:35.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:34 smithi026 ceph-mon[36056]: osdmap e68: 12 total, 11 up, 12 in 2024-05-14T09:29:35.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:34 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T09:29:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:35 smithi002 ceph-mon[31557]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-05-14T09:29:36.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:35 smithi002 ceph-mon[31557]: from='client.? 172.21.15.29:0/3207851519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:29:36.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:35 smithi029 ceph-mon[36249]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-05-14T09:29:36.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:35 smithi029 ceph-mon[36249]: from='client.? 172.21.15.29:0/3207851519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:29:36.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:35 smithi026 ceph-mon[36056]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-05-14T09:29:36.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:35 smithi026 ceph-mon[36056]: from='client.? 172.21.15.29:0/3207851519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T09:29:38.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:38 smithi002 ceph-mon[31557]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T09:29:38.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:38 smithi029 ceph-mon[36249]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T09:29:38.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:38 smithi026 ceph-mon[36056]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T09:29:40.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:40 smithi002 ceph-mon[31557]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-14T09:29:40.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:40 smithi029 ceph-mon[36249]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-14T09:29:40.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:40 smithi026 ceph-mon[36056]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-14T09:29:42.401 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:42 smithi029 ceph-mon[36249]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:29:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:42 smithi002 ceph-mon[31557]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:29:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:42 smithi026 ceph-mon[36056]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T09:29:43.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:43 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-14T09:29:43.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:43 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:43.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:43 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-14T09:29:43.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:43 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:43.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:43 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-14T09:29:43.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:43 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:44.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:44 smithi002 ceph-mon[31557]: Deploying daemon osd.11 on smithi029 2024-05-14T09:29:44.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:44 smithi002 ceph-mon[31557]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:44.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:44 smithi029 ceph-mon[36249]: Deploying daemon osd.11 on smithi029 2024-05-14T09:29:44.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:44 smithi029 ceph-mon[36249]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:44.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:44 smithi026 ceph-mon[36056]: Deploying daemon osd.11 on smithi029 2024-05-14T09:29:44.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:44 smithi026 ceph-mon[36056]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:46.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:46 smithi002 ceph-mon[31557]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:46.428 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:46 smithi029 ceph-mon[36249]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:46.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:46 smithi026 ceph-mon[36056]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:48.375 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:48 smithi029 ceph-mon[36249]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:48.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:48 smithi002 ceph-mon[31557]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:48.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:48 smithi026 ceph-mon[36056]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:50 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:50 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:29:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:50 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:50 smithi002 ceph-mon[31557]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:50.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:50 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:50.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:50 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:29:50.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:50 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:50.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:50 smithi029 ceph-mon[36249]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:50.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:50 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:50.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:50 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:29:50.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:50 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:50.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:50 smithi026 ceph-mon[36056]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:52.420 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:52 smithi029 ceph-mon[36249]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:52.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:52 smithi002 ceph-mon[31557]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:52.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:52 smithi026 ceph-mon[36056]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:53.147 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:53.147+0000 7fe04effd700 1 -- 172.21.15.29:0/3649913581 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe058063c00 con 0x7fe0440b2880 2024-05-14T09:29:53.147 INFO:teuthology.orchestra.run.smithi029.stdout:Created osd(s) 11 on host 'smithi029' 2024-05-14T09:29:53.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:53.150+0000 7fe060237700 1 -- 172.21.15.29:0/3649913581 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0440b2880 msgr2=0x7fe0440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:53.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:53.150+0000 7fe060237700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0440b2880 0x7fe0440b4d90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe05400b6c0 tx=0x7fe05400b2a0 comp rx=0 tx=0).stop 2024-05-14T09:29:53.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:53.150+0000 7fe060237700 1 -- 172.21.15.29:0/3649913581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe058104380 msgr2=0x7fe05819f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:53.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:53.150+0000 7fe060237700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe058104380 0x7fe05819f750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe04800d980 tx=0x7fe04800dc90 comp rx=0 tx=0).stop 2024-05-14T09:29:53.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:53.150+0000 7fe060237700 1 -- 172.21.15.29:0/3649913581 shutdown_connections 2024-05-14T09:29:53.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:53.150+0000 7fe060237700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0440b2880 0x7fe0440b4d90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:53.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:53.150+0000 7fe060237700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe058104d20 0x7fe0581a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:53.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:53.150+0000 7fe060237700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe058103250 0x7fe058076450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:53.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:53.150+0000 7fe060237700 1 --2- 172.21.15.29:0/3649913581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe058104380 0x7fe05819f750 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:53.151 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:53.150+0000 7fe060237700 1 -- 172.21.15.29:0/3649913581 >> 172.21.15.29:0/3649913581 conn(0x7fe0580fe890 msgr2=0x7fe0581075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:29:53.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:53.150+0000 7fe060237700 1 -- 172.21.15.29:0/3649913581 shutdown_connections 2024-05-14T09:29:53.152 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:29:53.150+0000 7fe060237700 1 -- 172.21.15.29:0/3649913581 wait complete. 2024-05-14T09:29:53.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:53.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:53.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:53.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:29:53.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:53.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:53 smithi002 ceph-mon[31557]: from='osd.11 [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T09:29:53.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:53 smithi002 ceph-mon[31557]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T09:29:53.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:53.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:53.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:29:53.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:53 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:53.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:53 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:53.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:53 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:53.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:53 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:29:53.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:53 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:53.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:53 smithi029 ceph-mon[36249]: from='osd.11 [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T09:29:53.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:53 smithi029 ceph-mon[36249]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T09:29:53.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:53 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:53.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:53 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:53.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:53 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:29:53.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:53 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:53.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:53 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:53.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:53 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:53.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:53 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:29:53.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:53 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:53.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:53 smithi026 ceph-mon[36056]: from='osd.11 [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T09:29:53.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:53 smithi026 ceph-mon[36056]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T09:29:53.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:53 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:53.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:53 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:53.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:53 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:29:53.841 DEBUG:teuthology.orchestra.run.smithi029:osd.11> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.11.service 2024-05-14T09:29:53.843 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-14T09:29:53.844 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-14T09:29:54.105 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:29:54.334 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:54 smithi029 ceph-mon[36249]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:54.334 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:54 smithi029 ceph-mon[36249]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-14T09:29:54.334 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:54 smithi029 ceph-mon[36249]: osdmap e69: 12 total, 11 up, 12 in 2024-05-14T09:29:54.334 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:54 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T09:29:54.334 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:54 smithi029 ceph-mon[36249]: from='osd.11 [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:29:54.334 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:54 smithi029 ceph-mon[36249]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:29:54.335 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 09:29:54 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T09:29:54.160+0000 7fb80a69b700 -1 osd.11 0 waiting for initial osdmap 2024-05-14T09:29:54.335 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 09:29:54 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T09:29:54.174+0000 7fb806450700 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T09:29:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:54 smithi002 ceph-mon[31557]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:54 smithi002 ceph-mon[31557]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-14T09:29:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:54 smithi002 ceph-mon[31557]: osdmap e69: 12 total, 11 up, 12 in 2024-05-14T09:29:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:54 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T09:29:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:54 smithi002 ceph-mon[31557]: from='osd.11 [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:29:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:54 smithi002 ceph-mon[31557]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:29:54.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:54 smithi026 ceph-mon[36056]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:54.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:54 smithi026 ceph-mon[36056]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-14T09:29:54.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:54 smithi026 ceph-mon[36056]: osdmap e69: 12 total, 11 up, 12 in 2024-05-14T09:29:54.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:54 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T09:29:54.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:54 smithi026 ceph-mon[36056]: from='osd.11 [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:29:54.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:54 smithi026 ceph-mon[36056]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2024-05-14T09:29:55.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:55 smithi002 ceph-mon[31557]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-05-14T09:29:55.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:55 smithi002 ceph-mon[31557]: osdmap e70: 12 total, 11 up, 12 in 2024-05-14T09:29:55.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:55 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T09:29:55.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:55 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T09:29:55.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:55 smithi029 ceph-mon[36249]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-05-14T09:29:55.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:55 smithi029 ceph-mon[36249]: osdmap e70: 12 total, 11 up, 12 in 2024-05-14T09:29:55.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:55 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T09:29:55.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:55 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T09:29:55.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:55 smithi026 ceph-mon[36056]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2024-05-14T09:29:55.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:55 smithi026 ceph-mon[36056]: osdmap e70: 12 total, 11 up, 12 in 2024-05-14T09:29:55.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:55 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T09:29:55.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:55 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T09:29:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.831+0000 7f1465ecc700 1 -- 172.21.15.2:0/3647802761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1460104380 msgr2=0x7f1460104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.831+0000 7f1465ecc700 1 --2- 172.21.15.2:0/3647802761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1460104380 0x7f1460104790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1454009990 tx=0x7f1454009ca0 comp rx=0 tx=0).stop 2024-05-14T09:29:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.831+0000 7f1465ecc700 1 -- 172.21.15.2:0/3647802761 shutdown_connections 2024-05-14T09:29:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.831+0000 7f1465ecc700 1 --2- 172.21.15.2:0/3647802761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1460104d20 0x7f146010b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.831+0000 7f1465ecc700 1 --2- 172.21.15.2:0/3647802761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1460103250 0x7f1460103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.831+0000 7f1465ecc700 1 --2- 172.21.15.2:0/3647802761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1460104380 0x7f1460104790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.831+0000 7f1465ecc700 1 -- 172.21.15.2:0/3647802761 >> 172.21.15.2:0/3647802761 conn(0x7f14600fe890 msgr2=0x7f1460100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:29:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.831+0000 7f1465ecc700 1 -- 172.21.15.2:0/3647802761 shutdown_connections 2024-05-14T09:29:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.831+0000 7f1465ecc700 1 -- 172.21.15.2:0/3647802761 wait complete. 2024-05-14T09:29:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.831+0000 7f1465ecc700 1 Processor -- start 2024-05-14T09:29:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f1465ecc700 1 -- start start 2024-05-14T09:29:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f1465ecc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1460103250 0x7f1460076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f1465ecc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1460104380 0x7f146019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f1465ecc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1460104d20 0x7f14601a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f1465ecc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f146010ecb0 con 0x7f1460103250 2024-05-14T09:29:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f1465ecc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f146010eb30 con 0x7f1460104380 2024-05-14T09:29:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f1465ecc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f146010ee30 con 0x7f1460104d20 2024-05-14T09:29:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f145f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1460103250 0x7f1460076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f145f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1460103250 0x7f1460076450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44318/0 (socket says 172.21.15.2:44318) 2024-05-14T09:29:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f145f7fe700 1 -- 172.21.15.2:0/992294046 learned_addr learned my addr 172.21.15.2:0/992294046 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:29:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f145ffff700 1 --2- 172.21.15.2:0/992294046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1460104d20 0x7f14601a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f145effd700 1 --2- 172.21.15.2:0/992294046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1460104380 0x7f146019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f145f7fe700 1 -- 172.21.15.2:0/992294046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1460104d20 msgr2=0x7f14601a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f145f7fe700 1 --2- 172.21.15.2:0/992294046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1460104d20 0x7f14601a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f145f7fe700 1 -- 172.21.15.2:0/992294046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1460104380 msgr2=0x7f146019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.832+0000 7f145f7fe700 1 --2- 172.21.15.2:0/992294046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1460104380 0x7f146019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.833+0000 7f145f7fe700 1 -- 172.21.15.2:0/992294046 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14601a2370 con 0x7f1460103250 2024-05-14T09:29:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.833+0000 7f145f7fe700 1 --2- 172.21.15.2:0/992294046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1460103250 0x7f1460076450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f145000c970 tx=0x7f145000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:29:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.833+0000 7f145cff9700 1 -- 172.21.15.2:0/992294046 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1450007910 con 0x7f1460103250 2024-05-14T09:29:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.833+0000 7f145cff9700 1 -- 172.21.15.2:0/992294046 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f145000f450 con 0x7f1460103250 2024-05-14T09:29:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.833+0000 7f1465ecc700 1 -- 172.21.15.2:0/992294046 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14601a2660 con 0x7f1460103250 2024-05-14T09:29:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.833+0000 7f145cff9700 1 -- 172.21.15.2:0/992294046 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1450018660 con 0x7f1460103250 2024-05-14T09:29:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.833+0000 7f1465ecc700 1 -- 172.21.15.2:0/992294046 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14601a2ba0 con 0x7f1460103250 2024-05-14T09:29:55.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.834+0000 7f1465ecc700 1 -- 172.21.15.2:0/992294046 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14600512c0 con 0x7f1460103250 2024-05-14T09:29:55.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.836+0000 7f145cff9700 1 -- 172.21.15.2:0/992294046 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f145000f920 con 0x7f1460103250 2024-05-14T09:29:55.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.836+0000 7f145cff9700 1 --2- 172.21.15.2:0/992294046 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14480b2880 0x7f14480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:55.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.837+0000 7f145effd700 1 --2- 172.21.15.2:0/992294046 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14480b2880 0x7f14480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:55.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.837+0000 7f145cff9700 1 -- 172.21.15.2:0/992294046 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f14500e1e00 con 0x7f1460103250 2024-05-14T09:29:55.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.837+0000 7f145effd700 1 --2- 172.21.15.2:0/992294046 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14480b2880 0x7f14480b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f145400b5c0 tx=0x7f145401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:29:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:55.840+0000 7f145cff9700 1 -- 172.21.15.2:0/992294046 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1450014030 con 0x7f1460103250 2024-05-14T09:29:56.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:56.056+0000 7f1465ecc700 1 -- 172.21.15.2:0/992294046 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f146002e5f0 con 0x7f1460103250 2024-05-14T09:29:56.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:56.056+0000 7f145cff9700 1 -- 172.21.15.2:0/992294046 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f14500a9fc0 con 0x7f1460103250 2024-05-14T09:29:56.056 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:29:56.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:56.059+0000 7f1465ecc700 1 -- 172.21.15.2:0/992294046 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14480b2880 msgr2=0x7f14480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:56.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:56.059+0000 7f1465ecc700 1 --2- 172.21.15.2:0/992294046 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14480b2880 0x7f14480b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f145400b5c0 tx=0x7f145401a040 comp rx=0 tx=0).stop 2024-05-14T09:29:56.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:56.059+0000 7f1465ecc700 1 -- 172.21.15.2:0/992294046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1460103250 msgr2=0x7f1460076450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:56.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:56.059+0000 7f1465ecc700 1 --2- 172.21.15.2:0/992294046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1460103250 0x7f1460076450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f145000c970 tx=0x7f145000cc80 comp rx=0 tx=0).stop 2024-05-14T09:29:56.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:56.059+0000 7f1465ecc700 1 -- 172.21.15.2:0/992294046 shutdown_connections 2024-05-14T09:29:56.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:56.059+0000 7f1465ecc700 1 --2- 172.21.15.2:0/992294046 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14480b2880 0x7f14480b4d90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:56.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:56.059+0000 7f1465ecc700 1 --2- 172.21.15.2:0/992294046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1460104d20 0x7f14601a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:56.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:56.059+0000 7f1465ecc700 1 --2- 172.21.15.2:0/992294046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1460103250 0x7f1460076450 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:56.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:56.059+0000 7f1465ecc700 1 --2- 172.21.15.2:0/992294046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1460104380 0x7f146019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:56.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:56.059+0000 7f1465ecc700 1 -- 172.21.15.2:0/992294046 >> 172.21.15.2:0/992294046 conn(0x7f14600fe890 msgr2=0x7f14601075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:29:56.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:56.059+0000 7f1465ecc700 1 -- 172.21.15.2:0/992294046 shutdown_connections 2024-05-14T09:29:56.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:56.059+0000 7f1465ecc700 1 -- 172.21.15.2:0/992294046 wait complete. 2024-05-14T09:29:56.371 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:56 smithi002 ceph-mon[31557]: purged_snaps scrub starts 2024-05-14T09:29:56.372 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:56 smithi002 ceph-mon[31557]: purged_snaps scrub ok 2024-05-14T09:29:56.372 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:56 smithi002 ceph-mon[31557]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:56.372 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:56 smithi002 ceph-mon[31557]: osd.11 [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417] boot 2024-05-14T09:29:56.372 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:56 smithi002 ceph-mon[31557]: osdmap e71: 12 total, 12 up, 12 in 2024-05-14T09:29:56.372 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:56 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T09:29:56.372 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:56 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/992294046' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T09:29:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:56 smithi029 ceph-mon[36249]: purged_snaps scrub starts 2024-05-14T09:29:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:56 smithi029 ceph-mon[36249]: purged_snaps scrub ok 2024-05-14T09:29:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:56 smithi029 ceph-mon[36249]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:56 smithi029 ceph-mon[36249]: osd.11 [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417] boot 2024-05-14T09:29:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:56 smithi029 ceph-mon[36249]: osdmap e71: 12 total, 12 up, 12 in 2024-05-14T09:29:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:56 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T09:29:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:56 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/992294046' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T09:29:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:56 smithi026 ceph-mon[36056]: purged_snaps scrub starts 2024-05-14T09:29:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:56 smithi026 ceph-mon[36056]: purged_snaps scrub ok 2024-05-14T09:29:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:56 smithi026 ceph-mon[36056]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 983 GiB / 983 GiB avail 2024-05-14T09:29:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:56 smithi026 ceph-mon[36056]: osd.11 [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417] boot 2024-05-14T09:29:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:56 smithi026 ceph-mon[36056]: osdmap e71: 12 total, 12 up, 12 in 2024-05-14T09:29:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:56 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T09:29:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:56 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/992294046' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T09:29:56.719 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":71,"num_osds":12,"num_up_osds":12,"osd_up_since":1715678995,"num_in_osds":12,"osd_in_since":1715678974,"num_remapped_pgs":0} 2024-05-14T09:29:56.720 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T09:29:56.970 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:29:57.343 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:57 smithi002 ceph-mon[31557]: osdmap e72: 12 total, 12 up, 12 in 2024-05-14T09:29:57.344 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:57.344 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:57.344 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:57.344 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:57.344 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:57.344 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:57.344 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:57.344 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:57.344 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:29:57.345 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:57.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:57 smithi029 ceph-mon[36249]: osdmap e72: 12 total, 12 up, 12 in 2024-05-14T09:29:57.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:57.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:57.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:57.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:57.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:57.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:57.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:57.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:57.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:29:57.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:57.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:57 smithi026 ceph-mon[36056]: osdmap e72: 12 total, 12 up, 12 in 2024-05-14T09:29:57.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:57.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:57.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:57.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:57.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:57.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:29:57.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:57.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:29:57.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:29:57.453 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:29:58.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:58 smithi002 ceph-mon[31557]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:29:58.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:58 smithi002 ceph-mon[31557]: Detected new or changed devices on smithi029 2024-05-14T09:29:58.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:58 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi029 to 4273M 2024-05-14T09:29:58.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:58 smithi029 ceph-mon[36249]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:29:58.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:58 smithi029 ceph-mon[36249]: Detected new or changed devices on smithi029 2024-05-14T09:29:58.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:58 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi029 to 4273M 2024-05-14T09:29:58.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:58 smithi026 ceph-mon[36056]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:29:58.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:58 smithi026 ceph-mon[36056]: Detected new or changed devices on smithi029 2024-05-14T09:29:58.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:58 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi029 to 4273M 2024-05-14T09:29:58.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.655+0000 7f321ad23700 1 -- 172.21.15.2:0/2671920927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3214110c30 msgr2=0x7f3214113050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:58.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.655+0000 7f321ad23700 1 --2- 172.21.15.2:0/2671920927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3214110c30 0x7f3214113050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f320c00b0a0 tx=0x7f320c00b3b0 comp rx=0 tx=0).stop 2024-05-14T09:29:58.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.655+0000 7f321ad23700 1 -- 172.21.15.2:0/2671920927 shutdown_connections 2024-05-14T09:29:58.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.655+0000 7f321ad23700 1 --2- 172.21.15.2:0/2671920927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3214110c30 0x7f3214113050 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:58.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.655+0000 7f321ad23700 1 --2- 172.21.15.2:0/2671920927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3214103080 0x7f3214103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:58.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.655+0000 7f321ad23700 1 --2- 172.21.15.2:0/2671920927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32141039d0 0x7f32141106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:58.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.655+0000 7f321ad23700 1 -- 172.21.15.2:0/2671920927 >> 172.21.15.2:0/2671920927 conn(0x7f32140fedf0 msgr2=0x7f3214101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:29:58.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.655+0000 7f321ad23700 1 -- 172.21.15.2:0/2671920927 shutdown_connections 2024-05-14T09:29:58.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.655+0000 7f321ad23700 1 -- 172.21.15.2:0/2671920927 wait complete. 2024-05-14T09:29:58.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.656+0000 7f321ad23700 1 Processor -- start 2024-05-14T09:29:58.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.656+0000 7f321ad23700 1 -- start start 2024-05-14T09:29:58.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.656+0000 7f321ad23700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3214103080 0x7f321419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:58.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.656+0000 7f321ad23700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32141039d0 0x7f321419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:58.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.656+0000 7f321ad23700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3214110c30 0x7f32141a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:58.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.656+0000 7f321ad23700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f32141166b0 con 0x7f3214110c30 2024-05-14T09:29:58.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.656+0000 7f321ad23700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3214116530 con 0x7f3214103080 2024-05-14T09:29:58.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.656+0000 7f321ad23700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3214116830 con 0x7f32141039d0 2024-05-14T09:29:58.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.656+0000 7f32192c0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3214110c30 0x7f32141a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:58.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.656+0000 7f32192c0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3214110c30 0x7f32141a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35054/0 (socket says 172.21.15.2:35054) 2024-05-14T09:29:58.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.656+0000 7f32192c0700 1 -- 172.21.15.2:0/2988985691 learned_addr learned my addr 172.21.15.2:0/2988985691 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:29:58.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.657+0000 7f3213fff700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32141039d0 0x7f321419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:58.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.657+0000 7f3218abf700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3214103080 0x7f321419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:58.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.657+0000 7f32192c0700 1 -- 172.21.15.2:0/2988985691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32141039d0 msgr2=0x7f321419d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:58.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.657+0000 7f32192c0700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32141039d0 0x7f321419d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:58.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.657+0000 7f32192c0700 1 -- 172.21.15.2:0/2988985691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3214103080 msgr2=0x7f321419cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:58.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.657+0000 7f32192c0700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3214103080 0x7f321419cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:58.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.657+0000 7f32192c0700 1 -- 172.21.15.2:0/2988985691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32141a1eb0 con 0x7f3214110c30 2024-05-14T09:29:58.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.657+0000 7f3213fff700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32141039d0 0x7f321419d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:29:58.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.657+0000 7f3218abf700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3214103080 0x7f321419cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:29:58.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.657+0000 7f32192c0700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3214110c30 0x7f32141a1810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f320c0062a0 tx=0x7f320c004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:29:58.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.657+0000 7f3211ffb700 1 -- 172.21.15.2:0/2988985691 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c021070 con 0x7f3214110c30 2024-05-14T09:29:58.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.657+0000 7f321ad23700 1 -- 172.21.15.2:0/2988985691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32141a2140 con 0x7f3214110c30 2024-05-14T09:29:58.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.657+0000 7f3211ffb700 1 -- 172.21.15.2:0/2988985691 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f320c007d00 con 0x7f3214110c30 2024-05-14T09:29:58.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.657+0000 7f3211ffb700 1 -- 172.21.15.2:0/2988985691 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c01d450 con 0x7f3214110c30 2024-05-14T09:29:58.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.657+0000 7f321ad23700 1 -- 172.21.15.2:0/2988985691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32141a9ac0 con 0x7f3214110c30 2024-05-14T09:29:58.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.659+0000 7f321ad23700 1 -- 172.21.15.2:0/2988985691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32140762d0 con 0x7f3214110c30 2024-05-14T09:29:58.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.660+0000 7f3211ffb700 1 -- 172.21.15.2:0/2988985691 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f320c004020 con 0x7f3214110c30 2024-05-14T09:29:58.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.661+0000 7f3211ffb700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f32040b2880 0x7f32040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:29:58.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.661+0000 7f3218abf700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f32040b2880 0x7f32040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:29:58.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.661+0000 7f3211ffb700 1 -- 172.21.15.2:0/2988985691 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f320c00f020 con 0x7f3214110c30 2024-05-14T09:29:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.661+0000 7f3218abf700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f32040b2880 0x7f32040b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3208005950 tx=0x7f3208005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:29:58.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.666+0000 7f3211ffb700 1 -- 172.21.15.2:0/2988985691 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f320c02a020 con 0x7f3214110c30 2024-05-14T09:29:58.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.885+0000 7f321ad23700 1 -- 172.21.15.2:0/2988985691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f321419e340 con 0x7f3214110c30 2024-05-14T09:29:58.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.886+0000 7f3211ffb700 1 -- 172.21.15.2:0/2988985691 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18279 (secure 0 0 0) 0x7f320c0ab6d0 con 0x7f3214110c30 2024-05-14T09:29:58.887 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:29:58.887 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":72,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","created":"2024-05-14T09:22:34.937047+0000","modified":"2024-05-14T09:29:56.165114+0000","last_up_change":"2024-05-14T09:29:55.157569+0000","last_in_change":"2024-05-14T09:29:34.922562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T09:26:03.046454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"10d34289-414e-4b1b-bc7c-ba1dd24c7297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6803","nonce":2622184634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6805","nonce":2622184634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6809","nonce":2622184634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6807","nonce":2622184634}]},"public_addr":"172.21.15.2:6803/2622184634","cluster_addr":"172.21.15.2:6805/2622184634","heartbeat_back_addr":"172.21.15.2:6809/2622184634","heartbeat_front_addr":"172.21.15.2:6807/2622184634","state":["exists","up"]},{"osd":1,"uuid":"bc4aefc2-706e-4faf-9042-64b87ad097b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6801","nonce":3084747226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6803","nonce":3084747226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6807","nonce":3084747226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6805","nonce":3084747226}]},"public_addr":"172.21.15.26:6801/3084747226","cluster_addr":"172.21.15.26:6803/3084747226","heartbeat_back_addr":"172.21.15.26:6807/3084747226","heartbeat_front_addr":"172.21.15.26:6805/3084747226","state":["exists","up"]},{"osd":2,"uuid":"8c5bb589-56e2-469f-9a6f-6035b3ddf4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6801","nonce":814048086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6803","nonce":814048086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6807","nonce":814048086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6805","nonce":814048086}]},"public_addr":"172.21.15.29:6801/814048086","cluster_addr":"172.21.15.29:6803/814048086","heartbeat_back_addr":"172.21.15.29:6807/814048086","heartbeat_front_addr":"172.21.15.29:6805/814048086","state":["exists","up"]},{"osd":3,"uuid":"8fbfc078-85e7-4dc2-acda-5e7b0ab927e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6811","nonce":879639451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6813","nonce":879639451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6817","nonce":879639451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6815","nonce":879639451}]},"public_addr":"172.21.15.2:6811/879639451","cluster_addr":"172.21.15.2:6813/879639451","heartbeat_back_addr":"172.21.15.2:6817/879639451","heartbeat_front_addr":"172.21.15.2:6815/879639451","state":["exists","up"]},{"osd":4,"uuid":"f2515448-e9c1-48a2-b270-1e5e781a2fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6809","nonce":457100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6811","nonce":457100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6815","nonce":457100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6813","nonce":457100752}]},"public_addr":"172.21.15.26:6809/457100752","cluster_addr":"172.21.15.26:6811/457100752","heartbeat_back_addr":"172.21.15.26:6815/457100752","heartbeat_front_addr":"172.21.15.26:6813/457100752","state":["exists","up"]},{"osd":5,"uuid":"20030857-a1e6-4b0a-b186-1cfa44f2e4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6809","nonce":1554045093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6811","nonce":1554045093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6815","nonce":1554045093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6813","nonce":1554045093}]},"public_addr":"172.21.15.29:6809/1554045093","cluster_addr":"172.21.15.29:6811/1554045093","heartbeat_back_addr":"172.21.15.29:6815/1554045093","heartbeat_front_addr":"172.21.15.29:6813/1554045093","state":["exists","up"]},{"osd":6,"uuid":"d757c72a-29f0-4f14-9f13-a2e61eb5e6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6819","nonce":2749897595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6821","nonce":2749897595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6825","nonce":2749897595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6823","nonce":2749897595}]},"public_addr":"172.21.15.2:6819/2749897595","cluster_addr":"172.21.15.2:6821/2749897595","heartbeat_back_addr":"172.21.15.2:6825/2749897595","heartbeat_front_addr":"172.21.15.2:6823/2749897595","state":["exists","up"]},{"osd":7,"uuid":"22ddef80-3633-44c5-bc0e-edc75b546ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6817","nonce":1180401611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6819","nonce":1180401611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6823","nonce":1180401611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6821","nonce":1180401611}]},"public_addr":"172.21.15.26:6817/1180401611","cluster_addr":"172.21.15.26:6819/1180401611","heartbeat_back_addr":"172.21.15.26:6823/1180401611","heartbeat_front_addr":"172.21.15.26:6821/1180401611","state":["exists","up"]},{"osd":8,"uuid":"466e7ac4-3ff7-4254-a3fd-8096420b1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6817","nonce":2858408446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6819","nonce":2858408446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6823","nonce":2858408446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6821","nonce":2858408446}]},"public_addr":"172.21.15.29:6817/2858408446","cluster_addr":"172.21.15.29:6819/2858408446","heartbeat_back_addr":"172.21.15.29:6823/2858408446","heartbeat_front_addr":"172.21.15.29:6821/2858408446","state":["exists","up"]},{"osd":9,"uuid":"119eeac9-29f7-4086-a1ad-04a0bc154b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6827","nonce":176898226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6829","nonce":176898226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6833","nonce":176898226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6831","nonce":176898226}]},"public_addr":"172.21.15.2:6827/176898226","cluster_addr":"172.21.15.2:6829/176898226","heartbeat_back_addr":"172.21.15.2:6833/176898226","heartbeat_front_addr":"172.21.15.2:6831/176898226","state":["exists","up"]},{"osd":10,"uuid":"55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6825","nonce":4065159515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6827","nonce":4065159515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6831","nonce":4065159515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6829","nonce":4065159515}]},"public_addr":"172.21.15.26:6825/4065159515","cluster_addr":"172.21.15.26:6827/4065159515","heartbeat_back_addr":"172.21.15.26:6831/4065159515","heartbeat_front_addr":"172.21.15.26:6829/4065159515","state":["exists","up"]},{"osd":11,"uuid":"c5246763-c4f6-4ae3-8a93-5929c541ac5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6825","nonce":378800417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6827","nonce":378800417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6831","nonce":378800417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6829","nonce":378800417}]},"public_addr":"172.21.15.29:6825/378800417","cluster_addr":"172.21.15.29:6827/378800417","heartbeat_back_addr":"172.21.15.29:6831/378800417","heartbeat_front_addr":"172.21.15.29:6829/378800417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:09.342387+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:34.156359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:58.860653+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:24.009323+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:49.057146+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:15.007172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:41.085748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:06.674923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:33.409128+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:00.106186+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:26.803487+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:53.668283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4051922409":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/2353790181":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/4268171474":"2024-05-15T09:23:28.940670+0000","172.21.15.2:6801/1423742578":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/969850441":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3252991268":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6801/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3789067476":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/1423742578":"2024-05-15T09:23:28.940670+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T09:29:58.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.889+0000 7f321ad23700 1 -- 172.21.15.2:0/2988985691 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f32040b2880 msgr2=0x7f32040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:58.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.889+0000 7f321ad23700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f32040b2880 0x7f32040b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3208005950 tx=0x7f3208005f50 comp rx=0 tx=0).stop 2024-05-14T09:29:58.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.889+0000 7f321ad23700 1 -- 172.21.15.2:0/2988985691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3214110c30 msgr2=0x7f32141a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:29:58.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.889+0000 7f321ad23700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3214110c30 0x7f32141a1810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f320c0062a0 tx=0x7f320c004ed0 comp rx=0 tx=0).stop 2024-05-14T09:29:58.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.889+0000 7f321ad23700 1 -- 172.21.15.2:0/2988985691 shutdown_connections 2024-05-14T09:29:58.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.889+0000 7f321ad23700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f32040b2880 0x7f32040b4d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:58.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.889+0000 7f321ad23700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3214103080 0x7f321419cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:58.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.889+0000 7f321ad23700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3214110c30 0x7f32141a1810 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:58.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.889+0000 7f321ad23700 1 --2- 172.21.15.2:0/2988985691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32141039d0 0x7f321419d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:29:58.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.889+0000 7f321ad23700 1 -- 172.21.15.2:0/2988985691 >> 172.21.15.2:0/2988985691 conn(0x7f32140fedf0 msgr2=0x7f32141126b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:29:58.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.890+0000 7f321ad23700 1 -- 172.21.15.2:0/2988985691 shutdown_connections 2024-05-14T09:29:58.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:29:58.890+0000 7f321ad23700 1 -- 172.21.15.2:0/2988985691 wait complete. 2024-05-14T09:29:59.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:29:59 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2988985691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:29:59.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:29:59 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2988985691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:29:59.657 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:29:59 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2988985691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:29:59.687 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T09:26:03.046454+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-14T09:29:59.688 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-14T09:29:59.937 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:30:00.360 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:00 smithi002 ceph-mon[31557]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:00.360 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:00 smithi002 ceph-mon[31557]: overall HEALTH_OK 2024-05-14T09:30:00.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:00 smithi029 ceph-mon[36249]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:00.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:00 smithi029 ceph-mon[36249]: overall HEALTH_OK 2024-05-14T09:30:00.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:00 smithi026 ceph-mon[36056]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:00.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:00 smithi026 ceph-mon[36056]: overall HEALTH_OK 2024-05-14T09:30:01.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.511+0000 7f23a5b98700 1 -- 172.21.15.2:0/1504622628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23a0108740 msgr2=0x7f23a01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:01.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.511+0000 7f23a5b98700 1 --2- 172.21.15.2:0/1504622628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23a0108740 0x7f23a01100a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f23980099c0 tx=0x7f2398009cd0 comp rx=0 tx=0).stop 2024-05-14T09:30:01.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.511+0000 7f23a5b98700 1 -- 172.21.15.2:0/1504622628 shutdown_connections 2024-05-14T09:30:01.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.511+0000 7f23a5b98700 1 --2- 172.21.15.2:0/1504622628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23a0108740 0x7f23a01100a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:01.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.511+0000 7f23a5b98700 1 --2- 172.21.15.2:0/1504622628 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23a01034c0 0x7f23a01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:01.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.511+0000 7f23a5b98700 1 --2- 172.21.15.2:0/1504622628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a0107df0 0x7f23a0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:01.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.511+0000 7f23a5b98700 1 -- 172.21.15.2:0/1504622628 >> 172.21.15.2:0/1504622628 conn(0x7f23a00fedf0 msgr2=0x7f23a0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:01.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.511+0000 7f23a5b98700 1 -- 172.21.15.2:0/1504622628 shutdown_connections 2024-05-14T09:30:01.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.511+0000 7f23a5b98700 1 -- 172.21.15.2:0/1504622628 wait complete. 2024-05-14T09:30:01.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.512+0000 7f23a5b98700 1 Processor -- start 2024-05-14T09:30:01.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f23a5b98700 1 -- start start 2024-05-14T09:30:01.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f23a5b98700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a01034c0 0x7f23a019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:01.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f23a5b98700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23a0107df0 0x7f23a019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:01.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f23a5b98700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23a0108740 0x7f23a019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:01.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f23a5b98700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f23a0113370 con 0x7f23a0107df0 2024-05-14T09:30:01.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f23a5b98700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f23a01131f0 con 0x7f23a01034c0 2024-05-14T09:30:01.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f23a5b98700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f23a01134f0 con 0x7f23a0108740 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f239effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23a0107df0 0x7f23a019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f239effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23a0107df0 0x7f23a019b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35070/0 (socket says 172.21.15.2:35070) 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f239effd700 1 -- 172.21.15.2:0/119881989 learned_addr learned my addr 172.21.15.2:0/119881989 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f239ffff700 1 --2- 172.21.15.2:0/119881989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23a0108740 0x7f23a019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f239f7fe700 1 --2- 172.21.15.2:0/119881989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a01034c0 0x7f23a019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f239effd700 1 -- 172.21.15.2:0/119881989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23a0108740 msgr2=0x7f23a019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f239effd700 1 --2- 172.21.15.2:0/119881989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23a0108740 0x7f23a019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f239effd700 1 -- 172.21.15.2:0/119881989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a01034c0 msgr2=0x7f23a019ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f239effd700 1 --2- 172.21.15.2:0/119881989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a01034c0 0x7f23a019ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.513+0000 7f239effd700 1 -- 172.21.15.2:0/119881989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a019fc60 con 0x7f23a0107df0 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.514+0000 7f239effd700 1 --2- 172.21.15.2:0/119881989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23a0107df0 0x7f23a019b330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f238800d980 tx=0x7f238800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.514+0000 7f239cff9700 1 -- 172.21.15.2:0/119881989 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388017070 con 0x7f23a0107df0 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.514+0000 7f23a5b98700 1 -- 172.21.15.2:0/119881989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a019ff50 con 0x7f23a0107df0 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.514+0000 7f239cff9700 1 -- 172.21.15.2:0/119881989 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f238800fd40 con 0x7f23a0107df0 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.514+0000 7f239cff9700 1 -- 172.21.15.2:0/119881989 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238800e3b0 con 0x7f23a0107df0 2024-05-14T09:30:01.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.514+0000 7f23a5b98700 1 -- 172.21.15.2:0/119881989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a01a7870 con 0x7f23a0107df0 2024-05-14T09:30:01.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.515+0000 7f23a5b98700 1 -- 172.21.15.2:0/119881989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a00512c0 con 0x7f23a0107df0 2024-05-14T09:30:01.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.517+0000 7f239cff9700 1 -- 172.21.15.2:0/119881989 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f2388010610 con 0x7f23a0107df0 2024-05-14T09:30:01.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.518+0000 7f239cff9700 1 --2- 172.21.15.2:0/119881989 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f238c0b2880 0x7f238c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:01.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.518+0000 7f239f7fe700 1 --2- 172.21.15.2:0/119881989 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f238c0b2880 0x7f238c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:01.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.518+0000 7f239cff9700 1 -- 172.21.15.2:0/119881989 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f23880e17d0 con 0x7f23a0107df0 2024-05-14T09:30:01.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.519+0000 7f239f7fe700 1 --2- 172.21.15.2:0/119881989 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f238c0b2880 0x7f238c0b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2390009550 tx=0x7f23900091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:01.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.522+0000 7f239cff9700 1 -- 172.21.15.2:0/119881989 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f23880a98f0 con 0x7f23a0107df0 2024-05-14T09:30:01.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.735+0000 7f23a5b98700 1 -- 172.21.15.2:0/119881989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f23a019c0f0 con 0x7f23a0107df0 2024-05-14T09:30:01.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.736+0000 7f239cff9700 1 -- 172.21.15.2:0/119881989 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f23880a91f0 con 0x7f23a0107df0 2024-05-14T09:30:01.736 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-05-14T09:30:01.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.738+0000 7f23a5b98700 1 -- 172.21.15.2:0/119881989 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f238c0b2880 msgr2=0x7f238c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:01.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.738+0000 7f23a5b98700 1 --2- 172.21.15.2:0/119881989 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f238c0b2880 0x7f238c0b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2390009550 tx=0x7f23900091b0 comp rx=0 tx=0).stop 2024-05-14T09:30:01.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.738+0000 7f23a5b98700 1 -- 172.21.15.2:0/119881989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23a0107df0 msgr2=0x7f23a019b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:01.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.738+0000 7f23a5b98700 1 --2- 172.21.15.2:0/119881989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23a0107df0 0x7f23a019b330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f238800d980 tx=0x7f238800dc90 comp rx=0 tx=0).stop 2024-05-14T09:30:01.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.739+0000 7f23a5b98700 1 -- 172.21.15.2:0/119881989 shutdown_connections 2024-05-14T09:30:01.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.739+0000 7f23a5b98700 1 --2- 172.21.15.2:0/119881989 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f238c0b2880 0x7f238c0b4d90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:01.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.739+0000 7f23a5b98700 1 --2- 172.21.15.2:0/119881989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a01034c0 0x7f23a019ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:01.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.739+0000 7f23a5b98700 1 --2- 172.21.15.2:0/119881989 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23a0108740 0x7f23a019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:01.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.739+0000 7f23a5b98700 1 --2- 172.21.15.2:0/119881989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23a0107df0 0x7f23a019b330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:01.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.739+0000 7f23a5b98700 1 -- 172.21.15.2:0/119881989 >> 172.21.15.2:0/119881989 conn(0x7f23a00fedf0 msgr2=0x7f23a0075c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:01.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.739+0000 7f23a5b98700 1 -- 172.21.15.2:0/119881989 shutdown_connections 2024-05-14T09:30:01.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:01.739+0000 7f23a5b98700 1 -- 172.21.15.2:0/119881989 wait complete. 2024-05-14T09:30:02.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:02 smithi002 ceph-mon[31557]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:02.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:02 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/119881989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T09:30:02.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:02 smithi029 ceph-mon[36249]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:02.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:02 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/119881989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T09:30:02.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:02 smithi026 ceph-mon[36056]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:02.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:02 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/119881989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T09:30:02.548 INFO:tasks.cephadm:Adding mds.a on smithi002 2024-05-14T09:30:02.548 INFO:tasks.cephadm:Adding mds.d on smithi002 2024-05-14T09:30:02.548 INFO:tasks.cephadm:Adding mds.g on smithi002 2024-05-14T09:30:02.548 INFO:tasks.cephadm:Adding mds.j on smithi002 2024-05-14T09:30:02.548 INFO:tasks.cephadm:Adding mds.b on smithi026 2024-05-14T09:30:02.549 INFO:tasks.cephadm:Adding mds.e on smithi026 2024-05-14T09:30:02.549 INFO:tasks.cephadm:Adding mds.h on smithi026 2024-05-14T09:30:02.549 INFO:tasks.cephadm:Adding mds.k on smithi026 2024-05-14T09:30:02.549 INFO:tasks.cephadm:Adding mds.c on smithi029 2024-05-14T09:30:02.549 INFO:tasks.cephadm:Adding mds.f on smithi029 2024-05-14T09:30:02.549 INFO:tasks.cephadm:Adding mds.i on smithi029 2024-05-14T09:30:02.549 INFO:tasks.cephadm:Adding mds.l on smithi029 2024-05-14T09:30:02.549 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi029=c;smithi029=f;smithi029=i;smithi029=l' 2024-05-14T09:30:02.811 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T09:30:04.278 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:04 smithi029 ceph-mon[36249]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:04.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:04 smithi026 ceph-mon[36056]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.540+0000 7fc0c3cea700 1 -- 172.21.15.29:0/853395795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0bc1077c0 msgr2=0x7fc0bc109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.540+0000 7fc0c3cea700 1 --2- 172.21.15.29:0/853395795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0bc1077c0 0x7fc0bc109be0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009990 tx=0x7fc0ac009ca0 comp rx=0 tx=0).stop 2024-05-14T09:30:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.540+0000 7fc0c3cea700 1 -- 172.21.15.29:0/853395795 shutdown_connections 2024-05-14T09:30:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.540+0000 7fc0c3cea700 1 --2- 172.21.15.29:0/853395795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0bc104e60 0x7fc0bc107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:04.540 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.540+0000 7fc0c3cea700 1 --2- 172.21.15.29:0/853395795 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0bc10a120 0x7fc0bc10c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.540+0000 7fc0c3cea700 1 --2- 172.21.15.29:0/853395795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0bc1077c0 0x7fc0bc109be0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.540+0000 7fc0c3cea700 1 -- 172.21.15.29:0/853395795 >> 172.21.15.29:0/853395795 conn(0x7fc0bc0fe8c0 msgr2=0x7fc0bc100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.540+0000 7fc0c3cea700 1 -- 172.21.15.29:0/853395795 shutdown_connections 2024-05-14T09:30:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.540+0000 7fc0c3cea700 1 -- 172.21.15.29:0/853395795 wait complete. 2024-05-14T09:30:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.541+0000 7fc0c3cea700 1 Processor -- start 2024-05-14T09:30:04.541 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.541+0000 7fc0c3cea700 1 -- start start 2024-05-14T09:30:04.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.541+0000 7fc0c3cea700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0bc104e60 0x7fc0bc19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:04.542 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.541+0000 7fc0c3cea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0bc1077c0 0x7fc0bc19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:04.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.541+0000 7fc0c3cea700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0bc10a120 0x7fc0bc1a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:04.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.541+0000 7fc0c3cea700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc10fe20 con 0x7fc0bc10a120 2024-05-14T09:30:04.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.541+0000 7fc0c3cea700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc10fca0 con 0x7fc0bc1077c0 2024-05-14T09:30:04.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.541+0000 7fc0c3cea700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc10ffa0 con 0x7fc0bc104e60 2024-05-14T09:30:04.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.541+0000 7fc0c1a86700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0bc104e60 0x7fc0bc19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:04.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.541+0000 7fc0c1a86700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0bc104e60 0x7fc0bc19a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35152/0 (socket says 172.21.15.29:35152) 2024-05-14T09:30:04.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.541+0000 7fc0c1a86700 1 -- 172.21.15.29:0/1901073231 learned_addr learned my addr 172.21.15.29:0/1901073231 (peer_addr_for_me v2:172.21.15.29:0/0) 2024-05-14T09:30:04.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.541+0000 7fc0c1285700 1 --2- 172.21.15.29:0/1901073231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0bc1077c0 0x7fc0bc19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:04.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.542+0000 7fc0c2287700 1 --2- 172.21.15.29:0/1901073231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0bc10a120 0x7fc0bc1a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:04.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.542+0000 7fc0c1a86700 1 -- 172.21.15.29:0/1901073231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0bc1077c0 msgr2=0x7fc0bc19aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:04.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.542+0000 7fc0c1a86700 1 --2- 172.21.15.29:0/1901073231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0bc1077c0 0x7fc0bc19aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:04.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.542+0000 7fc0c1a86700 1 -- 172.21.15.29:0/1901073231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0bc10a120 msgr2=0x7fc0bc1a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:04.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.542+0000 7fc0c1a86700 1 --2- 172.21.15.29:0/1901073231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0bc10a120 0x7fc0bc1a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:04.543 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.542+0000 7fc0c1a86700 1 -- 172.21.15.29:0/1901073231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0bc1a24c0 con 0x7fc0bc104e60 2024-05-14T09:30:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.542+0000 7fc0c1a86700 1 --2- 172.21.15.29:0/1901073231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0bc104e60 0x7fc0bc19a960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc0b800d980 tx=0x7fc0b800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.542+0000 7fc0b2ffd700 1 -- 172.21.15.29:0/1901073231 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8017070 con 0x7fc0bc104e60 2024-05-14T09:30:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.542+0000 7fc0b2ffd700 1 -- 172.21.15.29:0/1901073231 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0b8009de0 con 0x7fc0bc104e60 2024-05-14T09:30:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.542+0000 7fc0c3cea700 1 -- 172.21.15.29:0/1901073231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0bc1a27b0 con 0x7fc0bc104e60 2024-05-14T09:30:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.543+0000 7fc0b2ffd700 1 -- 172.21.15.29:0/1901073231 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b800f3c0 con 0x7fc0bc104e60 2024-05-14T09:30:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.543+0000 7fc0c3cea700 1 -- 172.21.15.29:0/1901073231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0bc1a2d40 con 0x7fc0bc104e60 2024-05-14T09:30:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.544+0000 7fc0c3cea700 1 -- 172.21.15.29:0/1901073231 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0bc0512c0 con 0x7fc0bc104e60 2024-05-14T09:30:04.544 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.545+0000 7fc0b2ffd700 1 -- 172.21.15.29:0/1901073231 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fc0b8009930 con 0x7fc0bc104e60 2024-05-14T09:30:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.546+0000 7fc0b2ffd700 1 --2- 172.21.15.29:0/1901073231 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0a80b2900 0x7fc0a80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.546+0000 7fc0b2ffd700 1 -- 172.21.15.29:0/1901073231 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fc0b80e1b60 con 0x7fc0bc104e60 2024-05-14T09:30:04.545 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.546+0000 7fc0c1285700 1 --2- 172.21.15.29:0/1901073231 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0a80b2900 0x7fc0a80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:04.546 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.547+0000 7fc0c1285700 1 --2- 172.21.15.29:0/1901073231 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0a80b2900 0x7fc0a80b4e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc19ba10 tx=0x7fc0ac00b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:04.549 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.550+0000 7fc0b2ffd700 1 -- 172.21.15.29:0/1901073231 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc0b80a9da0 con 0x7fc0bc104e60 2024-05-14T09:30:04.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:04 smithi002 ceph-mon[31557]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:04.775 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.776+0000 7fc0c3cea700 1 -- 172.21.15.29:0/1901073231 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi029=c;smithi029=f;smithi029=i;smithi029=l", "target": ["mon-mgr", ""]}) -- 0x7fc0bc063d00 con 0x7fc0a80b2900 2024-05-14T09:30:04.783 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.784+0000 7fc0b2ffd700 1 -- 172.21.15.29:0/1901073231 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fc0bc063d00 con 0x7fc0a80b2900 2024-05-14T09:30:04.783 INFO:teuthology.orchestra.run.smithi029.stdout:Scheduled mds.all update... 2024-05-14T09:30:04.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.786+0000 7fc0c3cea700 1 -- 172.21.15.29:0/1901073231 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0a80b2900 msgr2=0x7fc0a80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:04.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.786+0000 7fc0c3cea700 1 --2- 172.21.15.29:0/1901073231 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0a80b2900 0x7fc0a80b4e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc19ba10 tx=0x7fc0ac00b720 comp rx=0 tx=0).stop 2024-05-14T09:30:04.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.786+0000 7fc0c3cea700 1 -- 172.21.15.29:0/1901073231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0bc104e60 msgr2=0x7fc0bc19a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:04.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.786+0000 7fc0c3cea700 1 --2- 172.21.15.29:0/1901073231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0bc104e60 0x7fc0bc19a960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc0b800d980 tx=0x7fc0b800dc90 comp rx=0 tx=0).stop 2024-05-14T09:30:04.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.787+0000 7fc0c3cea700 1 -- 172.21.15.29:0/1901073231 shutdown_connections 2024-05-14T09:30:04.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.787+0000 7fc0c3cea700 1 --2- 172.21.15.29:0/1901073231 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0a80b2900 0x7fc0a80b4e10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:04.786 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.787+0000 7fc0c3cea700 1 --2- 172.21.15.29:0/1901073231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0bc10a120 0x7fc0bc1a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:04.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.787+0000 7fc0c3cea700 1 --2- 172.21.15.29:0/1901073231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0bc104e60 0x7fc0bc19a960 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:04.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.787+0000 7fc0c3cea700 1 --2- 172.21.15.29:0/1901073231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0bc1077c0 0x7fc0bc19aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:04.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.787+0000 7fc0c3cea700 1 -- 172.21.15.29:0/1901073231 >> 172.21.15.29:0/1901073231 conn(0x7fc0bc0fe8c0 msgr2=0x7fc0bc105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:04.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.787+0000 7fc0c3cea700 1 -- 172.21.15.29:0/1901073231 shutdown_connections 2024-05-14T09:30:04.787 INFO:teuthology.orchestra.run.smithi029.stderr:2024-05-14T09:30:04.787+0000 7fc0c3cea700 1 -- 172.21.15.29:0/1901073231 wait complete. 2024-05-14T09:30:05.511 DEBUG:teuthology.orchestra.run.smithi002:mds.a> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.a.service 2024-05-14T09:30:05.514 DEBUG:teuthology.orchestra.run.smithi002:mds.d> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.d.service 2024-05-14T09:30:05.557 DEBUG:teuthology.orchestra.run.smithi002:mds.g> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.g.service 2024-05-14T09:30:05.601 DEBUG:teuthology.orchestra.run.smithi002:mds.j> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.j.service 2024-05-14T09:30:05.643 DEBUG:teuthology.orchestra.run.smithi026:mds.b> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.b.service 2024-05-14T09:30:05.646 DEBUG:teuthology.orchestra.run.smithi026:mds.e> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.e.service 2024-05-14T09:30:05.690 DEBUG:teuthology.orchestra.run.smithi026:mds.h> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.h.service 2024-05-14T09:30:05.732 DEBUG:teuthology.orchestra.run.smithi026:mds.k> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.k.service 2024-05-14T09:30:05.775 DEBUG:teuthology.orchestra.run.smithi029:mds.c> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.c.service 2024-05-14T09:30:05.778 DEBUG:teuthology.orchestra.run.smithi029:mds.f> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.f.service 2024-05-14T09:30:05.821 DEBUG:teuthology.orchestra.run.smithi029:mds.i> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.i.service 2024-05-14T09:30:05.864 DEBUG:teuthology.orchestra.run.smithi029:mds.l> sudo journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.l.service 2024-05-14T09:30:05.907 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-14T09:30:05.907 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:30:05.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:05 smithi002 ceph-mon[31557]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi029=c;smithi029=f;smithi029=i;smithi029=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:30:05.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:05 smithi002 ceph-mon[31557]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi029=c;smithi029=f;smithi029=i;smithi029=l;count:12 2024-05-14T09:30:05.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:05.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:30:05.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:05 smithi002 ceph-mon[31557]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:05.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:05.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:30:05.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:05.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:05.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:05.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:05.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:05 smithi026 ceph-mon[36056]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi029=c;smithi029=f;smithi029=i;smithi029=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:30:05.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:05 smithi026 ceph-mon[36056]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi029=c;smithi029=f;smithi029=i;smithi029=l;count:12 2024-05-14T09:30:05.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:05.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:30:05.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:05 smithi026 ceph-mon[36056]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:05.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:05.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:30:05.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:05.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:05.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:05.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:06.154 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:30:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:05 smithi029 ceph-mon[36249]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi029=c;smithi029=f;smithi029=i;smithi029=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:30:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:05 smithi029 ceph-mon[36249]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi029=c;smithi029=f;smithi029=i;smithi029=l;count:12 2024-05-14T09:30:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:30:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:05 smithi029 ceph-mon[36249]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:30:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:06.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:06 smithi002 ceph-mon[31557]: Deploying daemon mds.l on smithi029 2024-05-14T09:30:07.077 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:06 smithi029 ceph-mon[36249]: Deploying daemon mds.l on smithi029 2024-05-14T09:30:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:06 smithi026 ceph-mon[36056]: Deploying daemon mds.l on smithi029 2024-05-14T09:30:07.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:07 smithi002 ceph-mon[31557]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:07.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.791+0000 7f83ee127700 1 -- 172.21.15.2:0/624118265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e8108f20 msgr2=0x7f83e8109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:07.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.791+0000 7f83ee127700 1 --2- 172.21.15.2:0/624118265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e8108f20 0x7f83e8109330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f83d00099c0 tx=0x7f83d0009cd0 comp rx=0 tx=0).stop 2024-05-14T09:30:07.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.791+0000 7f83ee127700 1 -- 172.21.15.2:0/624118265 shutdown_connections 2024-05-14T09:30:07.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.791+0000 7f83ee127700 1 --2- 172.21.15.2:0/624118265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e8109870 0x7f83e81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:07.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.791+0000 7f83ee127700 1 --2- 172.21.15.2:0/624118265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83e81045f0 0x7f83e8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:07.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.791+0000 7f83ee127700 1 --2- 172.21.15.2:0/624118265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e8108f20 0x7f83e8109330 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:07.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.791+0000 7f83ee127700 1 -- 172.21.15.2:0/624118265 >> 172.21.15.2:0/624118265 conn(0x7f83e8100040 msgr2=0x7f83e8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:07.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.791+0000 7f83ee127700 1 -- 172.21.15.2:0/624118265 shutdown_connections 2024-05-14T09:30:07.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.792+0000 7f83ee127700 1 -- 172.21.15.2:0/624118265 wait complete. 2024-05-14T09:30:07.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.792+0000 7f83ee127700 1 Processor -- start 2024-05-14T09:30:07.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.793+0000 7f83ee127700 1 -- start start 2024-05-14T09:30:07.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.793+0000 7f83ee127700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e81045f0 0x7f83e819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:07.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.793+0000 7f83ee127700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e8108f20 0x7f83e819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:07.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.793+0000 7f83ee127700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83e8109870 0x7f83e819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:07.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.793+0000 7f83ee127700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8114480 con 0x7f83e8108f20 2024-05-14T09:30:07.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.793+0000 7f83ee127700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8114300 con 0x7f83e81045f0 2024-05-14T09:30:07.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.793+0000 7f83ee127700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8114600 con 0x7f83e8109870 2024-05-14T09:30:07.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.793+0000 7f83deffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e8108f20 0x7f83e819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:07.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.793+0000 7f83deffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e8108f20 0x7f83e819b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57564/0 (socket says 172.21.15.2:57564) 2024-05-14T09:30:07.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.793+0000 7f83deffd700 1 -- 172.21.15.2:0/958452369 learned_addr learned my addr 172.21.15.2:0/958452369 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:30:07.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.793+0000 7f83e7fff700 1 --2- 172.21.15.2:0/958452369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83e8109870 0x7f83e819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:07.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.793+0000 7f83e77fe700 1 --2- 172.21.15.2:0/958452369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e81045f0 0x7f83e819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:07.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.794+0000 7f83e7fff700 1 -- 172.21.15.2:0/958452369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e81045f0 msgr2=0x7f83e819ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:07.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.794+0000 7f83e7fff700 1 --2- 172.21.15.2:0/958452369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e81045f0 0x7f83e819ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:07.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.794+0000 7f83e7fff700 1 -- 172.21.15.2:0/958452369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e8108f20 msgr2=0x7f83e819b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:07.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.794+0000 7f83e7fff700 1 --2- 172.21.15.2:0/958452369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e8108f20 0x7f83e819b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:07.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.794+0000 7f83e7fff700 1 -- 172.21.15.2:0/958452369 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e819fd60 con 0x7f83e8109870 2024-05-14T09:30:07.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.794+0000 7f83deffd700 1 --2- 172.21.15.2:0/958452369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e8108f20 0x7f83e819b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:30:07.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.794+0000 7f83e7fff700 1 --2- 172.21.15.2:0/958452369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83e8109870 0x7f83e819f6c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f83e000e970 tx=0x7f83e000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:07.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.794+0000 7f83e57fa700 1 -- 172.21.15.2:0/958452369 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0004220 con 0x7f83e8109870 2024-05-14T09:30:07.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.794+0000 7f83ee127700 1 -- 172.21.15.2:0/958452369 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e81a0050 con 0x7f83e8109870 2024-05-14T09:30:07.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.794+0000 7f83e57fa700 1 -- 172.21.15.2:0/958452369 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83e00043c0 con 0x7f83e8109870 2024-05-14T09:30:07.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.794+0000 7f83e57fa700 1 -- 172.21.15.2:0/958452369 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0010700 con 0x7f83e8109870 2024-05-14T09:30:07.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.794+0000 7f83ee127700 1 -- 172.21.15.2:0/958452369 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e81a7970 con 0x7f83e8109870 2024-05-14T09:30:07.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.796+0000 7f83ee127700 1 -- 172.21.15.2:0/958452369 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83e80512c0 con 0x7f83e8109870 2024-05-14T09:30:07.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.796+0000 7f83e57fa700 1 -- 172.21.15.2:0/958452369 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f83e00108a0 con 0x7f83e8109870 2024-05-14T09:30:07.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.797+0000 7f83e57fa700 1 --2- 172.21.15.2:0/958452369 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83d40b2880 0x7f83d40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:07.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.797+0000 7f83e77fe700 1 --2- 172.21.15.2:0/958452369 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83d40b2880 0x7f83d40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:07.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.797+0000 7f83e57fa700 1 -- 172.21.15.2:0/958452369 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f83e00e1b30 con 0x7f83e8109870 2024-05-14T09:30:07.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.798+0000 7f83e77fe700 1 --2- 172.21.15.2:0/958452369 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83d40b2880 0x7f83d40b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f83d800f540 tx=0x7f83d800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:07.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:07.802+0000 7f83e57fa700 1 -- 172.21.15.2:0/958452369 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f83e00a9cf0 con 0x7f83e8109870 2024-05-14T09:30:08.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:08.071+0000 7f83ee127700 1 -- 172.21.15.2:0/958452369 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83e819c1f0 con 0x7f83e8109870 2024-05-14T09:30:08.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:08.072+0000 7f83e57fa700 1 -- 172.21.15.2:0/958452369 <== mon.1 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f83e00a95f0 con 0x7f83e8109870 2024-05-14T09:30:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:30:08.072 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-14T09:30:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:08.074+0000 7f83ee127700 1 -- 172.21.15.2:0/958452369 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83d40b2880 msgr2=0x7f83d40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:08.074+0000 7f83ee127700 1 --2- 172.21.15.2:0/958452369 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83d40b2880 0x7f83d40b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f83d800f540 tx=0x7f83d800b040 comp rx=0 tx=0).stop 2024-05-14T09:30:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:08.074+0000 7f83ee127700 1 -- 172.21.15.2:0/958452369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83e8109870 msgr2=0x7f83e819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:08.074+0000 7f83ee127700 1 --2- 172.21.15.2:0/958452369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83e8109870 0x7f83e819f6c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f83e000e970 tx=0x7f83e000ec80 comp rx=0 tx=0).stop 2024-05-14T09:30:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:08.074+0000 7f83ee127700 1 -- 172.21.15.2:0/958452369 shutdown_connections 2024-05-14T09:30:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:08.074+0000 7f83ee127700 1 --2- 172.21.15.2:0/958452369 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83d40b2880 0x7f83d40b4d90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:08.074+0000 7f83ee127700 1 --2- 172.21.15.2:0/958452369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e81045f0 0x7f83e819ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:08.074+0000 7f83ee127700 1 --2- 172.21.15.2:0/958452369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83e8109870 0x7f83e819f6c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:08.074+0000 7f83ee127700 1 --2- 172.21.15.2:0/958452369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83e8108f20 0x7f83e819b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:08.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:08.074+0000 7f83ee127700 1 -- 172.21.15.2:0/958452369 >> 172.21.15.2:0/958452369 conn(0x7f83e8100040 msgr2=0x7f83e810a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:08.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:08.074+0000 7f83ee127700 1 -- 172.21.15.2:0/958452369 shutdown_connections 2024-05-14T09:30:08.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:08.074+0000 7f83ee127700 1 -- 172.21.15.2:0/958452369 wait complete. 2024-05-14T09:30:08.075 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 1 2024-05-14T09:30:08.100 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:07 smithi029 ceph-mon[36249]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:08.101 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 09:30:07 smithi029 systemd[1]: Starting Ceph mds.l for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T09:30:08.102 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:07 smithi026 ceph-mon[36056]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:08.366 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 09:30:08 smithi029 podman[65754]: 2024-05-14 09:30:08.004181977 +0000 UTC m=+0.017642067 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T09:30:08.366 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 09:30:08 smithi029 podman[65754]: 2024-05-14 09:30:08.119411096 +0000 UTC m=+0.132871219 container create e34a19116bb8a0da72d9dc9e95adf2686acb24884e7c67409e4f0ed32a2c7ed7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-l, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-14T09:30:08.366 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 09:30:08 smithi029 podman[65754]: 2024-05-14 09:30:08.367096408 +0000 UTC m=+0.380556477 container init e34a19116bb8a0da72d9dc9e95adf2686acb24884e7c67409e4f0ed32a2c7ed7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-l, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-14T09:30:08.687 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 09:30:08 smithi029 podman[65754]: 2024-05-14 09:30:08.369965064 +0000 UTC m=+0.383425150 container start e34a19116bb8a0da72d9dc9e95adf2686acb24884e7c67409e4f0ed32a2c7ed7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-l, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T09:30:08.687 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 09:30:08 smithi029 ceph-mds[65774]: -- 172.21.15.29:0/3528298040 <== mon.1 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564f13cafa40 con 0x564f13cf7600 2024-05-14T09:30:08.687 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 09:30:08 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-l[65770]: starting mds.l at 2024-05-14T09:30:08.688 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 09:30:08 smithi029 bash[65754]: e34a19116bb8a0da72d9dc9e95adf2686acb24884e7c67409e4f0ed32a2c7ed7 2024-05-14T09:30:08.688 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 09:30:08 smithi029 systemd[1]: Started Ceph mds.l for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:30:09.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:08 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/958452369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:09.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:08 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.29:6832/2153165231,v1:172.21.15.29:6833/2153165231] up:boot 2024-05-14T09:30:09.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:08 smithi002 ceph-mon[31557]: fsmap 1 up:standby 2024-05-14T09:30:09.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:08 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T09:30:09.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:08 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:09.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:08 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:09.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:08 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:09.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:08 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:09.173 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:08 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:09.173 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:08 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:09.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:08 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/958452369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:09.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:08 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.29:6832/2153165231,v1:172.21.15.29:6833/2153165231] up:boot 2024-05-14T09:30:09.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:08 smithi029 ceph-mon[36249]: fsmap 1 up:standby 2024-05-14T09:30:09.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:08 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T09:30:09.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:08 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:09.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:08 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:09.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:08 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:09.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:08 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:09.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:08 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:09.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:08 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:09.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:08 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/958452369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:09.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:08 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.29:6832/2153165231,v1:172.21.15.29:6833/2153165231] up:boot 2024-05-14T09:30:09.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:08 smithi026 ceph-mon[36056]: fsmap 1 up:standby 2024-05-14T09:30:09.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:08 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T09:30:09.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:08 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:09.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:08 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:09.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:08 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:09.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:08 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:09.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:08 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:09.203 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:08 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:10.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:09 smithi002 ceph-mon[31557]: Deploying daemon mds.k on smithi026 2024-05-14T09:30:10.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:09 smithi002 ceph-mon[31557]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:10.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:09 smithi029 ceph-mon[36249]: Deploying daemon mds.k on smithi026 2024-05-14T09:30:10.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:09 smithi029 ceph-mon[36249]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:10.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:09 smithi026 ceph-mon[36056]: Deploying daemon mds.k on smithi026 2024-05-14T09:30:10.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:09 smithi026 ceph-mon[36056]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:10.716 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:30:10.977 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:30:11.113 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 09:30:10 smithi026 systemd[1]: Starting Ceph mds.k for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T09:30:11.397 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 09:30:11 smithi026 podman[65804]: 2024-05-14 09:30:11.016456612 +0000 UTC m=+0.017706357 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T09:30:11.398 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 09:30:11 smithi026 podman[65804]: 2024-05-14 09:30:11.128732011 +0000 UTC m=+0.129981764 container create d4e5220632df18aaa51bb2d978f139ae91584f21d8d4b1f1b737c60880e555a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T09:30:11.398 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 09:30:11 smithi026 podman[65804]: 2024-05-14 09:30:11.398535172 +0000 UTC m=+0.399784914 container init d4e5220632df18aaa51bb2d978f139ae91584f21d8d4b1f1b737c60880e555a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-05-14T09:30:11.701 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 09:30:11 smithi026 podman[65804]: 2024-05-14 09:30:11.401536156 +0000 UTC m=+0.402785905 container start d4e5220632df18aaa51bb2d978f139ae91584f21d8d4b1f1b737c60880e555a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-14T09:30:11.701 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 09:30:11 smithi026 ceph-mds[65823]: -- 172.21.15.26:0/3496461365 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564fcbbfba40 con 0x564fcbc43600 2024-05-14T09:30:11.702 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 09:30:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-k[65819]: starting mds.k at 2024-05-14T09:30:11.702 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 09:30:11 smithi026 bash[65804]: d4e5220632df18aaa51bb2d978f139ae91584f21d8d4b1f1b737c60880e555a1 2024-05-14T09:30:11.702 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 09:30:11 smithi026 systemd[1]: Started Ceph mds.k for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:30:12.354 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:12 smithi002 ceph-mon[31557]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:12.354 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:12 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:12.354 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:12 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:12.354 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:12 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:12.354 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:12 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:12.354 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:12 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:12.354 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:12 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:12 smithi029 ceph-mon[36249]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:12 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:12 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:12 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:12 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:12 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:12 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:12.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:12 smithi026 ceph-mon[36056]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:12.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:12 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:12.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:12 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:12.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:12 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:12.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:12 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:12.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:12 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:12.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:12 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:12.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.713+0000 7f7222fbe700 1 -- 172.21.15.2:0/435167024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f721c107e10 msgr2=0x7f721c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:12.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.713+0000 7f7222fbe700 1 --2- 172.21.15.2:0/435167024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f721c107e10 0x7f721c108220 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f72100099c0 tx=0x7f7210009cd0 comp rx=0 tx=0).stop 2024-05-14T09:30:12.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.713+0000 7f7222fbe700 1 -- 172.21.15.2:0/435167024 shutdown_connections 2024-05-14T09:30:12.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.713+0000 7f7222fbe700 1 --2- 172.21.15.2:0/435167024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f721c1034e0 0x7f721c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:12.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.713+0000 7f7222fbe700 1 --2- 172.21.15.2:0/435167024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f721c108760 0x7f721c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:12.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.713+0000 7f7222fbe700 1 --2- 172.21.15.2:0/435167024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f721c107e10 0x7f721c108220 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:12.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.713+0000 7f7222fbe700 1 -- 172.21.15.2:0/435167024 >> 172.21.15.2:0/435167024 conn(0x7f721c0fef30 msgr2=0x7f721c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:12.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.713+0000 7f7222fbe700 1 -- 172.21.15.2:0/435167024 shutdown_connections 2024-05-14T09:30:12.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.713+0000 7f7222fbe700 1 -- 172.21.15.2:0/435167024 wait complete. 2024-05-14T09:30:12.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.714+0000 7f7222fbe700 1 Processor -- start 2024-05-14T09:30:12.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.714+0000 7f7222fbe700 1 -- start start 2024-05-14T09:30:12.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.714+0000 7f7222fbe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f721c1034e0 0x7f721c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:12.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.714+0000 7f7222fbe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f721c107e10 0x7f721c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:12.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.714+0000 7f7222fbe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f721c108760 0x7f721c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:12.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.714+0000 7f7222fbe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f721c113370 con 0x7f721c1034e0 2024-05-14T09:30:12.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.714+0000 7f7222fbe700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1131f0 con 0x7f721c108760 2024-05-14T09:30:12.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.714+0000 7f7222fbe700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1134f0 con 0x7f721c107e10 2024-05-14T09:30:12.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.715+0000 7f7220d5a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f721c1034e0 0x7f721c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:12.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.715+0000 7f7220d5a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f721c1034e0 0x7f721c19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57576/0 (socket says 172.21.15.2:57576) 2024-05-14T09:30:12.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.715+0000 7f7220d5a700 1 -- 172.21.15.2:0/3436289844 learned_addr learned my addr 172.21.15.2:0/3436289844 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:30:12.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.715+0000 7f721bfff700 1 --2- 172.21.15.2:0/3436289844 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f721c107e10 0x7f721c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:12.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.715+0000 7f722155b700 1 --2- 172.21.15.2:0/3436289844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f721c108760 0x7f721c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:12.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.715+0000 7f7220d5a700 1 -- 172.21.15.2:0/3436289844 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f721c107e10 msgr2=0x7f721c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:12.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.715+0000 7f7220d5a700 1 --2- 172.21.15.2:0/3436289844 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f721c107e10 0x7f721c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:12.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.715+0000 7f7220d5a700 1 -- 172.21.15.2:0/3436289844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f721c108760 msgr2=0x7f721c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:12.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.715+0000 7f7220d5a700 1 --2- 172.21.15.2:0/3436289844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f721c108760 0x7f721c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:12.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.715+0000 7f7220d5a700 1 -- 172.21.15.2:0/3436289844 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721c19fce0 con 0x7f721c1034e0 2024-05-14T09:30:12.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.715+0000 7f7220d5a700 1 --2- 172.21.15.2:0/3436289844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f721c1034e0 0x7f721c19ae10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f720c00e9a0 tx=0x7f720c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:12.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.715+0000 7f7219ffb700 1 -- 172.21.15.2:0/3436289844 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c00c990 con 0x7f721c1034e0 2024-05-14T09:30:12.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.715+0000 7f7219ffb700 1 -- 172.21.15.2:0/3436289844 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f720c010430 con 0x7f721c1034e0 2024-05-14T09:30:12.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.715+0000 7f7219ffb700 1 -- 172.21.15.2:0/3436289844 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c018680 con 0x7f721c1034e0 2024-05-14T09:30:12.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.715+0000 7f7222fbe700 1 -- 172.21.15.2:0/3436289844 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721c19ffd0 con 0x7f721c1034e0 2024-05-14T09:30:12.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.716+0000 7f7222fbe700 1 -- 172.21.15.2:0/3436289844 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721c0ff340 con 0x7f721c1034e0 2024-05-14T09:30:12.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.716+0000 7f7222fbe700 1 -- 172.21.15.2:0/3436289844 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f721c104990 con 0x7f721c1034e0 2024-05-14T09:30:12.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.718+0000 7f7219ffb700 1 -- 172.21.15.2:0/3436289844 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f720c00cb30 con 0x7f721c1034e0 2024-05-14T09:30:12.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.719+0000 7f7219ffb700 1 --2- 172.21.15.2:0/3436289844 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f72040b2900 0x7f72040b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:12.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.719+0000 7f7219ffb700 1 -- 172.21.15.2:0/3436289844 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f720c0e2040 con 0x7f721c1034e0 2024-05-14T09:30:12.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.719+0000 7f721bfff700 1 --2- 172.21.15.2:0/3436289844 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f72040b2900 0x7f72040b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:12.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.720+0000 7f721bfff700 1 --2- 172.21.15.2:0/3436289844 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f72040b2900 0x7f72040b4e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7210000c00 tx=0x7f721001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:12.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.723+0000 7f7219ffb700 1 -- 172.21.15.2:0/3436289844 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f720c00ce20 con 0x7f721c1034e0 2024-05-14T09:30:13.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:12.999+0000 7f7222fbe700 1 -- 172.21.15.2:0/3436289844 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f721c19c170 con 0x7f721c1034e0 2024-05-14T09:30:13.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:13.000+0000 7f7219ffb700 1 -- 172.21.15.2:0/3436289844 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2212 (secure 0 0 0) 0x7f720c0aa200 con 0x7f721c1034e0 2024-05-14T09:30:13.001 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:30:13.001 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-14T09:30:13.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:13.002+0000 7f7222fbe700 1 -- 172.21.15.2:0/3436289844 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f72040b2900 msgr2=0x7f72040b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:13.002+0000 7f7222fbe700 1 --2- 172.21.15.2:0/3436289844 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f72040b2900 0x7f72040b4e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7210000c00 tx=0x7f721001a040 comp rx=0 tx=0).stop 2024-05-14T09:30:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:13.002+0000 7f7222fbe700 1 -- 172.21.15.2:0/3436289844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f721c1034e0 msgr2=0x7f721c19ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:13.002+0000 7f7222fbe700 1 --2- 172.21.15.2:0/3436289844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f721c1034e0 0x7f721c19ae10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f720c00e9a0 tx=0x7f720c00ecb0 comp rx=0 tx=0).stop 2024-05-14T09:30:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:13.002+0000 7f7222fbe700 1 -- 172.21.15.2:0/3436289844 shutdown_connections 2024-05-14T09:30:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:13.003+0000 7f7222fbe700 1 --2- 172.21.15.2:0/3436289844 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f72040b2900 0x7f72040b4e10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:13.003+0000 7f7222fbe700 1 --2- 172.21.15.2:0/3436289844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f721c108760 0x7f721c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:13.003+0000 7f7222fbe700 1 --2- 172.21.15.2:0/3436289844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f721c1034e0 0x7f721c19ae10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:13.003+0000 7f7222fbe700 1 --2- 172.21.15.2:0/3436289844 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f721c107e10 0x7f721c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:13.003+0000 7f7222fbe700 1 -- 172.21.15.2:0/3436289844 >> 172.21.15.2:0/3436289844 conn(0x7f721c0fef30 msgr2=0x7f721c06c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:13.003+0000 7f7222fbe700 1 -- 172.21.15.2:0/3436289844 shutdown_connections 2024-05-14T09:30:13.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:13.003+0000 7f7222fbe700 1 -- 172.21.15.2:0/3436289844 wait complete. 2024-05-14T09:30:13.005 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 3 2024-05-14T09:30:13.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:13 smithi002 ceph-mon[31557]: Deploying daemon mds.j on smithi002 2024-05-14T09:30:13.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:13 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.26:6832/39434912,v1:172.21.15.26:6833/39434912] up:boot 2024-05-14T09:30:13.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:13 smithi002 ceph-mon[31557]: fsmap 2 up:standby 2024-05-14T09:30:13.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:13 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T09:30:13.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:13 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3436289844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:13.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:13 smithi029 ceph-mon[36249]: Deploying daemon mds.j on smithi002 2024-05-14T09:30:13.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:13 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.26:6832/39434912,v1:172.21.15.26:6833/39434912] up:boot 2024-05-14T09:30:13.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:13 smithi029 ceph-mon[36249]: fsmap 2 up:standby 2024-05-14T09:30:13.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:13 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T09:30:13.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:13 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3436289844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:13.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:13 smithi026 ceph-mon[36056]: Deploying daemon mds.j on smithi002 2024-05-14T09:30:13.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:13 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.26:6832/39434912,v1:172.21.15.26:6833/39434912] up:boot 2024-05-14T09:30:13.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:13 smithi026 ceph-mon[36056]: fsmap 2 up:standby 2024-05-14T09:30:13.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:13 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T09:30:13.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:13 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3436289844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:14.294 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:14 smithi002 ceph-mon[31557]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:14.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:14 smithi029 ceph-mon[36249]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:14.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:14 smithi026 ceph-mon[36056]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:15.171 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 09:30:15 smithi002 systemd[1]: Started Ceph mds.j for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:30:16.169 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:16 smithi002 ceph-mon[31557]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:16.170 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:16 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.2:6834/3590687009,v1:172.21.15.2:6835/3590687009] up:boot 2024-05-14T09:30:16.170 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:16 smithi002 ceph-mon[31557]: fsmap 3 up:standby 2024-05-14T09:30:16.170 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:16 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T09:30:16.170 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:16 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:16.170 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:16 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:16.170 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:16 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:16.170 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:16 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:16.170 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:16 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:16.170 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:16 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:16.170 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:16 smithi002 ceph-mon[31557]: Deploying daemon mds.d on smithi002 2024-05-14T09:30:16.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:16 smithi029 ceph-mon[36249]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:16.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:16 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.2:6834/3590687009,v1:172.21.15.2:6835/3590687009] up:boot 2024-05-14T09:30:16.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:16 smithi029 ceph-mon[36249]: fsmap 3 up:standby 2024-05-14T09:30:16.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:16 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T09:30:16.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:16 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:16.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:16 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:16.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:16 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:16.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:16 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:16.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:16 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:16.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:16 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:16.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:16 smithi029 ceph-mon[36249]: Deploying daemon mds.d on smithi002 2024-05-14T09:30:16.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:16 smithi026 ceph-mon[36056]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:16.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:16 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.2:6834/3590687009,v1:172.21.15.2:6835/3590687009] up:boot 2024-05-14T09:30:16.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:16 smithi026 ceph-mon[36056]: fsmap 3 up:standby 2024-05-14T09:30:16.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:16 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T09:30:16.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:16 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:16.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:16 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:16.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:16 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:16.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:16 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:16.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:16 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:16.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:16 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:16.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:16 smithi026 ceph-mon[36056]: Deploying daemon mds.d on smithi002 2024-05-14T09:30:16.561 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:30:16.996 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:30:18.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:18 smithi002 ceph-mon[31557]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:18.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:18 smithi029 ceph-mon[36249]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:18.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:18 smithi026 ceph-mon[36056]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.814+0000 7f866fa48700 1 -- 172.21.15.2:0/2530302866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8668108f40 msgr2=0x7f8668109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.814+0000 7f866fa48700 1 --2- 172.21.15.2:0/2530302866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8668108f40 0x7f8668109350 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86580099c0 tx=0x7f8658009cd0 comp rx=0 tx=0).stop 2024-05-14T09:30:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.814+0000 7f866fa48700 1 -- 172.21.15.2:0/2530302866 shutdown_connections 2024-05-14T09:30:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.814+0000 7f866fa48700 1 --2- 172.21.15.2:0/2530302866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8668109890 0x7f86681111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.814+0000 7f866fa48700 1 --2- 172.21.15.2:0/2530302866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8668104610 0x7f8668104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.814+0000 7f866fa48700 1 --2- 172.21.15.2:0/2530302866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8668108f40 0x7f8668109350 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.814+0000 7f866fa48700 1 -- 172.21.15.2:0/2530302866 >> 172.21.15.2:0/2530302866 conn(0x7f866807af70 msgr2=0x7f866807b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.814+0000 7f866fa48700 1 -- 172.21.15.2:0/2530302866 shutdown_connections 2024-05-14T09:30:18.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.814+0000 7f866fa48700 1 -- 172.21.15.2:0/2530302866 wait complete. 2024-05-14T09:30:18.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.816+0000 7f866fa48700 1 Processor -- start 2024-05-14T09:30:18.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.816+0000 7f866fa48700 1 -- start start 2024-05-14T09:30:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866fa48700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8668104610 0x7f866819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866fa48700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8668108f40 0x7f866819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866fa48700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8668109890 0x7f866819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866fa48700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f86681144a0 con 0x7f8668104610 2024-05-14T09:30:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866fa48700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8668114320 con 0x7f8668108f40 2024-05-14T09:30:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866fa48700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8668114620 con 0x7f8668109890 2024-05-14T09:30:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866d7e4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8668104610 0x7f866819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866d7e4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8668104610 0x7f866819ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50080/0 (socket says 172.21.15.2:50080) 2024-05-14T09:30:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866d7e4700 1 -- 172.21.15.2:0/483743952 learned_addr learned my addr 172.21.15.2:0/483743952 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:30:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866cfe3700 1 --2- 172.21.15.2:0/483743952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8668108f40 0x7f866819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866dfe5700 1 --2- 172.21.15.2:0/483743952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8668109890 0x7f866819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866d7e4700 1 -- 172.21.15.2:0/483743952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8668109890 msgr2=0x7f866819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866d7e4700 1 --2- 172.21.15.2:0/483743952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8668109890 0x7f866819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866d7e4700 1 -- 172.21.15.2:0/483743952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8668108f40 msgr2=0x7f866819b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866d7e4700 1 --2- 172.21.15.2:0/483743952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8668108f40 0x7f866819b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.817+0000 7f866d7e4700 1 -- 172.21.15.2:0/483743952 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866819fd40 con 0x7f8668104610 2024-05-14T09:30:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.818+0000 7f866d7e4700 1 --2- 172.21.15.2:0/483743952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8668104610 0x7f866819ae70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f866400e9a0 tx=0x7f866400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.818+0000 7f865e7fc700 1 -- 172.21.15.2:0/483743952 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866400ca10 con 0x7f8668104610 2024-05-14T09:30:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.818+0000 7f865e7fc700 1 -- 172.21.15.2:0/483743952 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f866400cbb0 con 0x7f8668104610 2024-05-14T09:30:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.818+0000 7f866fa48700 1 -- 172.21.15.2:0/483743952 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86681a0030 con 0x7f8668104610 2024-05-14T09:30:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.818+0000 7f865e7fc700 1 -- 172.21.15.2:0/483743952 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664018680 con 0x7f8668104610 2024-05-14T09:30:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.818+0000 7f866fa48700 1 -- 172.21.15.2:0/483743952 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86680759f0 con 0x7f8668104610 2024-05-14T09:30:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.819+0000 7f866fa48700 1 -- 172.21.15.2:0/483743952 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8668105ac0 con 0x7f8668104610 2024-05-14T09:30:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.821+0000 7f865e7fc700 1 -- 172.21.15.2:0/483743952 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f8664010430 con 0x7f8668104610 2024-05-14T09:30:18.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.822+0000 7f865e7fc700 1 --2- 172.21.15.2:0/483743952 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f86540b2880 0x7f86540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:18.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.822+0000 7f866cfe3700 1 --2- 172.21.15.2:0/483743952 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f86540b2880 0x7f86540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:18.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.822+0000 7f865e7fc700 1 -- 172.21.15.2:0/483743952 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f86640e22a0 con 0x7f8668104610 2024-05-14T09:30:18.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.822+0000 7f866cfe3700 1 --2- 172.21.15.2:0/483743952 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f86540b2880 0x7f86540b4d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8658000c00 tx=0x7f865801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:18.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:18.825+0000 7f865e7fc700 1 -- 172.21.15.2:0/483743952 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f86640aa3c0 con 0x7f8668104610 2024-05-14T09:30:19.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:19.096+0000 7f866fa48700 1 -- 172.21.15.2:0/483743952 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f866802e5f0 con 0x7f8668104610 2024-05-14T09:30:19.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:19.097+0000 7f865e7fc700 1 -- 172.21.15.2:0/483743952 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3034 (secure 0 0 0) 0x7f86640a9cc0 con 0x7f8668104610 2024-05-14T09:30:19.097 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:30:19.097 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-14T09:30:19.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:19.099+0000 7f866fa48700 1 -- 172.21.15.2:0/483743952 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f86540b2880 msgr2=0x7f86540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:19.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:19.099+0000 7f866fa48700 1 --2- 172.21.15.2:0/483743952 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f86540b2880 0x7f86540b4d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8658000c00 tx=0x7f865801a040 comp rx=0 tx=0).stop 2024-05-14T09:30:19.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:19.099+0000 7f866fa48700 1 -- 172.21.15.2:0/483743952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8668104610 msgr2=0x7f866819ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:19.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:19.099+0000 7f866fa48700 1 --2- 172.21.15.2:0/483743952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8668104610 0x7f866819ae70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f866400e9a0 tx=0x7f866400ecb0 comp rx=0 tx=0).stop 2024-05-14T09:30:19.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:19.099+0000 7f866fa48700 1 -- 172.21.15.2:0/483743952 shutdown_connections 2024-05-14T09:30:19.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:19.099+0000 7f866fa48700 1 --2- 172.21.15.2:0/483743952 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f86540b2880 0x7f86540b4d90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:19.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:19.099+0000 7f866fa48700 1 --2- 172.21.15.2:0/483743952 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8668109890 0x7f866819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:19.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:19.099+0000 7f866fa48700 1 --2- 172.21.15.2:0/483743952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8668104610 0x7f866819ae70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:19.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:19.099+0000 7f866fa48700 1 --2- 172.21.15.2:0/483743952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8668108f40 0x7f866819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:19.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:19.099+0000 7f866fa48700 1 -- 172.21.15.2:0/483743952 >> 172.21.15.2:0/483743952 conn(0x7f866807af70 msgr2=0x7f8668101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:19.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:19.099+0000 7f866fa48700 1 -- 172.21.15.2:0/483743952 shutdown_connections 2024-05-14T09:30:19.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:19.099+0000 7f866fa48700 1 -- 172.21.15.2:0/483743952 wait complete. 2024-05-14T09:30:19.100 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 4 2024-05-14T09:30:19.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:19 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/483743952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:19.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:19 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/483743952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:19.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:19 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/483743952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:20 smithi002 ceph-mon[31557]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:20 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:20 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:20 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:20 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:20.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:20 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:20.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:20 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:20.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:20 smithi002 ceph-mon[31557]: Deploying daemon mds.e on smithi026 2024-05-14T09:30:20.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:20 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.2:6836/2302731082,v1:172.21.15.2:6837/2302731082] up:boot 2024-05-14T09:30:20.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:20 smithi002 ceph-mon[31557]: fsmap 4 up:standby 2024-05-14T09:30:20.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:20 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T09:30:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:20 smithi029 ceph-mon[36249]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:20 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:20 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:20 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:20 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:20 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:20 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:20 smithi029 ceph-mon[36249]: Deploying daemon mds.e on smithi026 2024-05-14T09:30:20.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:20 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.2:6836/2302731082,v1:172.21.15.2:6837/2302731082] up:boot 2024-05-14T09:30:20.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:20 smithi029 ceph-mon[36249]: fsmap 4 up:standby 2024-05-14T09:30:20.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:20 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T09:30:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:20 smithi026 ceph-mon[36056]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:20 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:20.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:20 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:20.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:20 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:20.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:20 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:20.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:20 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:20.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:20 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:20.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:20 smithi026 ceph-mon[36056]: Deploying daemon mds.e on smithi026 2024-05-14T09:30:20.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:20 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.2:6836/2302731082,v1:172.21.15.2:6837/2302731082] up:boot 2024-05-14T09:30:20.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:20 smithi026 ceph-mon[36056]: fsmap 4 up:standby 2024-05-14T09:30:20.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:20 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T09:30:21.798 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:30:21.952 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:30:21 smithi026 systemd[1]: Starting Ceph mds.e for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T09:30:22.063 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:30:22.283 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:30:21 smithi026 podman[66103]: 2024-05-14 09:30:21.859417125 +0000 UTC m=+0.017644671 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T09:30:22.283 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:30:22 smithi026 podman[66103]: 2024-05-14 09:30:22.0420792 +0000 UTC m=+0.200306732 container create eb79eacbc98a660363ccf3b839297da2af590244dbc44b39237ab2d8ef71654c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-14T09:30:22.497 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:22 smithi002 ceph-mon[31557]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:22.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:22 smithi029 ceph-mon[36249]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:22.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:22 smithi026 ceph-mon[36056]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:22.701 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:30:22 smithi026 podman[66103]: 2024-05-14 09:30:22.406871601 +0000 UTC m=+0.565099124 container init eb79eacbc98a660363ccf3b839297da2af590244dbc44b39237ab2d8ef71654c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-14T09:30:22.702 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:30:22 smithi026 podman[66103]: 2024-05-14 09:30:22.409818922 +0000 UTC m=+0.568046454 container start eb79eacbc98a660363ccf3b839297da2af590244dbc44b39237ab2d8ef71654c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-14T09:30:22.702 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:30:22 smithi026 ceph-mds[66122]: -- 172.21.15.26:0/4113265515 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b01b549a40 con 0x55b01b591a80 2024-05-14T09:30:22.702 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:30:22 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e[66118]: starting mds.e at 2024-05-14T09:30:22.702 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:30:22 smithi026 bash[66103]: eb79eacbc98a660363ccf3b839297da2af590244dbc44b39237ab2d8ef71654c 2024-05-14T09:30:22.702 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:30:22 smithi026 systemd[1]: Started Ceph mds.e for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:30:23.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.809+0000 7fe822e4a700 1 -- 172.21.15.2:0/512555776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe81c1034c0 msgr2=0x7fe81c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:23.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.809+0000 7fe822e4a700 1 --2- 172.21.15.2:0/512555776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe81c1034c0 0x7fe81c1038d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe8040099c0 tx=0x7fe804009cd0 comp rx=0 tx=0).stop 2024-05-14T09:30:23.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.810+0000 7fe822e4a700 1 -- 172.21.15.2:0/512555776 shutdown_connections 2024-05-14T09:30:23.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.810+0000 7fe822e4a700 1 --2- 172.21.15.2:0/512555776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe81c108740 0x7fe81c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:23.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.810+0000 7fe822e4a700 1 --2- 172.21.15.2:0/512555776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe81c1034c0 0x7fe81c1038d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:23.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.810+0000 7fe822e4a700 1 --2- 172.21.15.2:0/512555776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe81c107df0 0x7fe81c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:23.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.810+0000 7fe822e4a700 1 -- 172.21.15.2:0/512555776 >> 172.21.15.2:0/512555776 conn(0x7fe81c0fedc0 msgr2=0x7fe81c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:23.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.810+0000 7fe822e4a700 1 -- 172.21.15.2:0/512555776 shutdown_connections 2024-05-14T09:30:23.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.810+0000 7fe822e4a700 1 -- 172.21.15.2:0/512555776 wait complete. 2024-05-14T09:30:23.811 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:23 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:23.811 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:23 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:23.811 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:23 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:23.811 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:23 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:23.811 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:23 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:23.811 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:23 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:23.811 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:23 smithi002 ceph-mon[31557]: Deploying daemon mds.f on smithi029 2024-05-14T09:30:23.812 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:23 smithi002 ceph-mon[31557]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:23.812 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:23 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.26:6834/2529273890,v1:172.21.15.26:6835/2529273890] up:boot 2024-05-14T09:30:23.812 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:23 smithi002 ceph-mon[31557]: fsmap 5 up:standby 2024-05-14T09:30:23.812 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:23 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T09:30:23.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.810+0000 7fe822e4a700 1 Processor -- start 2024-05-14T09:30:23.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.810+0000 7fe822e4a700 1 -- start start 2024-05-14T09:30:23.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe822e4a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe81c1034c0 0x7fe81c19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:23.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe822e4a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe81c107df0 0x7fe81c19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:23.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe822e4a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe81c108740 0x7fe81c19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:23.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe822e4a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c113370 con 0x7fe81c108740 2024-05-14T09:30:23.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe822e4a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c1131f0 con 0x7fe81c107df0 2024-05-14T09:30:23.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe822e4a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c1134f0 con 0x7fe81c1034c0 2024-05-14T09:30:23.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe8213e7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe81c108740 0x7fe81c19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:23.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe8213e7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe81c108740 0x7fe81c19c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50112/0 (socket says 172.21.15.2:50112) 2024-05-14T09:30:23.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe8213e7700 1 -- 172.21.15.2:0/3771399266 learned_addr learned my addr 172.21.15.2:0/3771399266 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:30:23.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe820be6700 1 --2- 172.21.15.2:0/3771399266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe81c1034c0 0x7fe81c19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:23.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe81bfff700 1 --2- 172.21.15.2:0/3771399266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe81c107df0 0x7fe81c19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:23.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe8213e7700 1 -- 172.21.15.2:0/3771399266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe81c1034c0 msgr2=0x7fe81c19bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:23.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe8213e7700 1 --2- 172.21.15.2:0/3771399266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe81c1034c0 0x7fe81c19bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:23.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe8213e7700 1 -- 172.21.15.2:0/3771399266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe81c107df0 msgr2=0x7fe81c19c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:23.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe8213e7700 1 --2- 172.21.15.2:0/3771399266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe81c107df0 0x7fe81c19c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:23.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe8213e7700 1 -- 172.21.15.2:0/3771399266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81c1a6f30 con 0x7fe81c108740 2024-05-14T09:30:23.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.811+0000 7fe820be6700 1 --2- 172.21.15.2:0/3771399266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe81c1034c0 0x7fe81c19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:30:23.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.812+0000 7fe8213e7700 1 --2- 172.21.15.2:0/3771399266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe81c108740 0x7fe81c19c710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe81400ef30 tx=0x7fe81400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:23.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.812+0000 7fe819ffb700 1 -- 172.21.15.2:0/3771399266 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8140092e0 con 0x7fe81c108740 2024-05-14T09:30:23.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.812+0000 7fe819ffb700 1 -- 172.21.15.2:0/3771399266 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe814004bb0 con 0x7fe81c108740 2024-05-14T09:30:23.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.812+0000 7fe819ffb700 1 -- 172.21.15.2:0/3771399266 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814004280 con 0x7fe81c108740 2024-05-14T09:30:23.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.812+0000 7fe822e4a700 1 -- 172.21.15.2:0/3771399266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81c1a71c0 con 0x7fe81c108740 2024-05-14T09:30:23.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.812+0000 7fe822e4a700 1 -- 172.21.15.2:0/3771399266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe81c1a76b0 con 0x7fe81c108740 2024-05-14T09:30:23.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.813+0000 7fe822e4a700 1 -- 172.21.15.2:0/3771399266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe81c0512c0 con 0x7fe81c108740 2024-05-14T09:30:23.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.815+0000 7fe819ffb700 1 -- 172.21.15.2:0/3771399266 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fe81401b030 con 0x7fe81c108740 2024-05-14T09:30:23.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.815+0000 7fe819ffb700 1 --2- 172.21.15.2:0/3771399266 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8080b2900 0x7fe8080b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:23.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.815+0000 7fe820be6700 1 --2- 172.21.15.2:0/3771399266 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8080b2900 0x7fe8080b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:23.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.815+0000 7fe819ffb700 1 -- 172.21.15.2:0/3771399266 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fe8140e1940 con 0x7fe81c108740 2024-05-14T09:30:23.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.816+0000 7fe820be6700 1 --2- 172.21.15.2:0/3771399266 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8080b2900 0x7fe8080b4e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe804000c00 tx=0x7fe80401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:23.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:23.820+0000 7fe819ffb700 1 -- 172.21.15.2:0/3771399266 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe814022020 con 0x7fe81c108740 2024-05-14T09:30:23.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:23 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:23.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:23 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:23.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:23 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:23.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:23 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:23.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:23 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:23.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:23 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:23.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:23 smithi029 ceph-mon[36249]: Deploying daemon mds.f on smithi029 2024-05-14T09:30:23.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:23 smithi029 ceph-mon[36249]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:23.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:23 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.26:6834/2529273890,v1:172.21.15.26:6835/2529273890] up:boot 2024-05-14T09:30:23.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:23 smithi029 ceph-mon[36249]: fsmap 5 up:standby 2024-05-14T09:30:23.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:23 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T09:30:23.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:23.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:23.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:23.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:23.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:23.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:23.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:23 smithi026 ceph-mon[36056]: Deploying daemon mds.f on smithi029 2024-05-14T09:30:23.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:23 smithi026 ceph-mon[36056]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:23.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:23 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.26:6834/2529273890,v1:172.21.15.26:6835/2529273890] up:boot 2024-05-14T09:30:23.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:23 smithi026 ceph-mon[36056]: fsmap 5 up:standby 2024-05-14T09:30:23.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:23 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T09:30:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:24.080+0000 7fe822e4a700 1 -- 172.21.15.2:0/3771399266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe81c02e5f0 con 0x7fe81c108740 2024-05-14T09:30:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:24.081+0000 7fe819ffb700 1 -- 172.21.15.2:0/3771399266 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4681 (secure 0 0 0) 0x7fe814010040 con 0x7fe81c108740 2024-05-14T09:30:24.081 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:30:24.081 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-14T09:30:24.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:24.083+0000 7fe822e4a700 1 -- 172.21.15.2:0/3771399266 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8080b2900 msgr2=0x7fe8080b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:24.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:24.083+0000 7fe822e4a700 1 --2- 172.21.15.2:0/3771399266 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8080b2900 0x7fe8080b4e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe804000c00 tx=0x7fe80401a040 comp rx=0 tx=0).stop 2024-05-14T09:30:24.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:24.083+0000 7fe822e4a700 1 -- 172.21.15.2:0/3771399266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe81c108740 msgr2=0x7fe81c19c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:24.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:24.083+0000 7fe822e4a700 1 --2- 172.21.15.2:0/3771399266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe81c108740 0x7fe81c19c710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe81400ef30 tx=0x7fe81400c450 comp rx=0 tx=0).stop 2024-05-14T09:30:24.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:24.083+0000 7fe822e4a700 1 -- 172.21.15.2:0/3771399266 shutdown_connections 2024-05-14T09:30:24.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:24.083+0000 7fe822e4a700 1 --2- 172.21.15.2:0/3771399266 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8080b2900 0x7fe8080b4e10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:24.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:24.083+0000 7fe822e4a700 1 --2- 172.21.15.2:0/3771399266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe81c108740 0x7fe81c19c710 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:24.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:24.083+0000 7fe822e4a700 1 --2- 172.21.15.2:0/3771399266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe81c1034c0 0x7fe81c19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:24.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:24.083+0000 7fe822e4a700 1 --2- 172.21.15.2:0/3771399266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe81c107df0 0x7fe81c19c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:24.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:24.083+0000 7fe822e4a700 1 -- 172.21.15.2:0/3771399266 >> 172.21.15.2:0/3771399266 conn(0x7fe81c0fedc0 msgr2=0x7fe81c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:24.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:24.083+0000 7fe822e4a700 1 -- 172.21.15.2:0/3771399266 shutdown_connections 2024-05-14T09:30:24.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:24.083+0000 7fe822e4a700 1 -- 172.21.15.2:0/3771399266 wait complete. 2024-05-14T09:30:24.084 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 6 2024-05-14T09:30:24.636 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:24 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3771399266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:24 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3771399266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:24.937 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 09:30:24 smithi029 systemd[1]: Starting Ceph mds.f for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T09:30:24.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:24 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3771399266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:25.314 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 09:30:25 smithi029 podman[66061]: 2024-05-14 09:30:24.941220896 +0000 UTC m=+0.017582712 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T09:30:25.314 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 09:30:25 smithi029 podman[66061]: 2024-05-14 09:30:25.05893134 +0000 UTC m=+0.135293144 container create 56f5739413b1245e958cd2673b854786dba1237267b538167bdb24fabb507b60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-14T09:30:25.314 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 09:30:25 smithi029 podman[66061]: 2024-05-14 09:30:25.315084927 +0000 UTC m=+0.391446731 container init 56f5739413b1245e958cd2673b854786dba1237267b538167bdb24fabb507b60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-14T09:30:25.687 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 09:30:25 smithi029 podman[66061]: 2024-05-14 09:30:25.317868597 +0000 UTC m=+0.394230408 container start 56f5739413b1245e958cd2673b854786dba1237267b538167bdb24fabb507b60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-f, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD) 2024-05-14T09:30:25.687 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 09:30:25 smithi029 ceph-mds[66080]: -- 172.21.15.29:0/857475101 <== mon.1 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d7c5f17a40 con 0x55d7c5f5fa80 2024-05-14T09:30:25.687 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 09:30:25 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-f[66076]: starting mds.f at 2024-05-14T09:30:25.687 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 09:30:25 smithi029 bash[66061]: 56f5739413b1245e958cd2673b854786dba1237267b538167bdb24fabb507b60 2024-05-14T09:30:25.687 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 09:30:25 smithi029 systemd[1]: Started Ceph mds.f for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:30:26.669 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:30:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:26 smithi029 ceph-mon[36249]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:26 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:26 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:26 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:26 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:26.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:26 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:26.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:26 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:26.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:26 smithi029 ceph-mon[36249]: Deploying daemon mds.c on smithi029 2024-05-14T09:30:26.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:26 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.29:6834/3647517494,v1:172.21.15.29:6835/3647517494] up:boot 2024-05-14T09:30:26.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:26 smithi029 ceph-mon[36249]: fsmap 6 up:standby 2024-05-14T09:30:26.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:26 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T09:30:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:26 smithi002 ceph-mon[31557]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:26 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:26 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:26 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:26 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:26 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:26 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:26 smithi002 ceph-mon[31557]: Deploying daemon mds.c on smithi029 2024-05-14T09:30:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:26 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.29:6834/3647517494,v1:172.21.15.29:6835/3647517494] up:boot 2024-05-14T09:30:26.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:26 smithi002 ceph-mon[31557]: fsmap 6 up:standby 2024-05-14T09:30:26.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:26 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T09:30:26.930 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:30:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:26 smithi026 ceph-mon[36056]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:26 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:26 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:26 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:26 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:26 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:26.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:26 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:26.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:26 smithi026 ceph-mon[36056]: Deploying daemon mds.c on smithi029 2024-05-14T09:30:26.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:26 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.29:6834/3647517494,v1:172.21.15.29:6835/3647517494] up:boot 2024-05-14T09:30:26.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:26 smithi026 ceph-mon[36056]: fsmap 6 up:standby 2024-05-14T09:30:26.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:26 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T09:30:27.882 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 09:30:27 smithi029 systemd[1]: Starting Ceph mds.c for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T09:30:28.152 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 09:30:27 smithi029 podman[66366]: 2024-05-14 09:30:27.784525693 +0000 UTC m=+0.017908332 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T09:30:28.152 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 09:30:27 smithi029 podman[66366]: 2024-05-14 09:30:27.911955716 +0000 UTC m=+0.145338364 container create 1128ca72187843c129e5b30f52047885e6d63590c3c0c28c49fbba62773ae7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-14T09:30:28.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.598+0000 7f0eede61700 1 -- 172.21.15.2:0/1730346004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ee81087a0 msgr2=0x7f0ee8110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:28.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.598+0000 7f0eede61700 1 --2- 172.21.15.2:0/1730346004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ee81087a0 0x7f0ee8110140 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b0d0 tx=0x7f0ee000b3e0 comp rx=0 tx=0).stop 2024-05-14T09:30:28.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.598+0000 7f0eede61700 1 -- 172.21.15.2:0/1730346004 shutdown_connections 2024-05-14T09:30:28.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.598+0000 7f0eede61700 1 --2- 172.21.15.2:0/1730346004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ee8103c50 0x7f0ee8104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:28.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.598+0000 7f0eede61700 1 --2- 172.21.15.2:0/1730346004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ee81087a0 0x7f0ee8110140 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:28.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.598+0000 7f0eede61700 1 --2- 172.21.15.2:0/1730346004 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ee8107e50 0x7f0ee8108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:28.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.598+0000 7f0eede61700 1 -- 172.21.15.2:0/1730346004 >> 172.21.15.2:0/1730346004 conn(0x7f0ee80fe4e0 msgr2=0x7f0ee8100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:28.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.598+0000 7f0eede61700 1 -- 172.21.15.2:0/1730346004 shutdown_connections 2024-05-14T09:30:28.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.598+0000 7f0eede61700 1 -- 172.21.15.2:0/1730346004 wait complete. 2024-05-14T09:30:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.599+0000 7f0eede61700 1 Processor -- start 2024-05-14T09:30:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.599+0000 7f0eede61700 1 -- start start 2024-05-14T09:30:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.599+0000 7f0eede61700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ee8103c50 0x7f0ee819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.599+0000 7f0eede61700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ee8107e50 0x7f0ee819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.599+0000 7f0eede61700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ee81087a0 0x7f0ee819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.599+0000 7f0eede61700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee8114560 con 0x7f0ee8107e50 2024-05-14T09:30:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.599+0000 7f0eede61700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee81143e0 con 0x7f0ee8103c50 2024-05-14T09:30:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.599+0000 7f0eede61700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee81146e0 con 0x7f0ee81087a0 2024-05-14T09:30:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0ee6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ee8107e50 0x7f0ee819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ee8103c50 0x7f0ee819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0ee6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ee8107e50 0x7f0ee819b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39640/0 (socket says 172.21.15.2:39640) 2024-05-14T09:30:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ee8103c50 0x7f0ee819ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:33834/0 (socket says 172.21.15.2:33834) 2024-05-14T09:30:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0ee6ffd700 1 -- 172.21.15.2:0/2803013846 learned_addr learned my addr 172.21.15.2:0/2803013846 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:30:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0ee7fff700 1 --2- 172.21.15.2:0/2803013846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ee81087a0 0x7f0ee819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0ee77fe700 1 -- 172.21.15.2:0/2803013846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ee81087a0 msgr2=0x7f0ee819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0ee77fe700 1 --2- 172.21.15.2:0/2803013846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ee81087a0 0x7f0ee819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0ee77fe700 1 -- 172.21.15.2:0/2803013846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ee8107e50 msgr2=0x7f0ee819b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0ee77fe700 1 --2- 172.21.15.2:0/2803013846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ee8107e50 0x7f0ee819b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0ee77fe700 1 -- 172.21.15.2:0/2803013846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee819fd20 con 0x7f0ee8103c50 2024-05-14T09:30:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0ee77fe700 1 --2- 172.21.15.2:0/2803013846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ee8103c50 0x7f0ee819ae50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800e9f0 tx=0x7f0ed800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0ee4ff9700 1 -- 172.21.15.2:0/2803013846 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed800c9c0 con 0x7f0ee8103c50 2024-05-14T09:30:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0eede61700 1 -- 172.21.15.2:0/2803013846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee81a0010 con 0x7f0ee8103c50 2024-05-14T09:30:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0ee4ff9700 1 -- 172.21.15.2:0/2803013846 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed800cb60 con 0x7f0ee8103c50 2024-05-14T09:30:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0ee4ff9700 1 -- 172.21.15.2:0/2803013846 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8018680 con 0x7f0ee8103c50 2024-05-14T09:30:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.600+0000 7f0eede61700 1 -- 172.21.15.2:0/2803013846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee8110850 con 0x7f0ee8103c50 2024-05-14T09:30:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.602+0000 7f0eede61700 1 -- 172.21.15.2:0/2803013846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee80512c0 con 0x7f0ee8103c50 2024-05-14T09:30:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.603+0000 7f0ee4ff9700 1 -- 172.21.15.2:0/2803013846 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f0ed804aab0 con 0x7f0ee8103c50 2024-05-14T09:30:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.604+0000 7f0ee4ff9700 1 --2- 172.21.15.2:0/2803013846 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0ed00b2880 0x7f0ed00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.604+0000 7f0ee6ffd700 1 --2- 172.21.15.2:0/2803013846 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0ed00b2880 0x7f0ed00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.604+0000 7f0ee4ff9700 1 -- 172.21.15.2:0/2803013846 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f0ed80e20e0 con 0x7f0ee8103c50 2024-05-14T09:30:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.604+0000 7f0ee6ffd700 1 --2- 172.21.15.2:0/2803013846 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0ed00b2880 0x7f0ed00b4d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8106bf0 tx=0x7f0edc00b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:28.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.608+0000 7f0ee4ff9700 1 -- 172.21.15.2:0/2803013846 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0ed80aa200 con 0x7f0ee8103c50 2024-05-14T09:30:28.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:28 smithi002 ceph-mon[31557]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:28.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:28.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:28.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:28.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:28.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:28.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:28 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:28.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:28 smithi029 ceph-mon[36249]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:28.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:28.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:28.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:28.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:28.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:28.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:28 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:28.687 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 09:30:28 smithi029 podman[66366]: 2024-05-14 09:30:28.285704453 +0000 UTC m=+0.519087079 container init 1128ca72187843c129e5b30f52047885e6d63590c3c0c28c49fbba62773ae7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507) 2024-05-14T09:30:28.687 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 09:30:28 smithi029 podman[66366]: 2024-05-14 09:30:28.28858764 +0000 UTC m=+0.521970268 container start 1128ca72187843c129e5b30f52047885e6d63590c3c0c28c49fbba62773ae7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-c, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-14T09:30:28.688 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 09:30:28 smithi029 ceph-mds[66385]: -- 172.21.15.29:0/4026176657 <== mon.1 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a8123f5a40 con 0x55a81243da80 2024-05-14T09:30:28.688 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 09:30:28 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-c[66381]: starting mds.c at 2024-05-14T09:30:28.688 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 09:30:28 smithi029 bash[66366]: 1128ca72187843c129e5b30f52047885e6d63590c3c0c28c49fbba62773ae7a0 2024-05-14T09:30:28.688 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 09:30:28 smithi029 systemd[1]: Started Ceph mds.c for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:30:28.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.870+0000 7f0eede61700 1 -- 172.21.15.2:0/2803013846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ee802e5f0 con 0x7f0ee8103c50 2024-05-14T09:30:28.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.871+0000 7f0ee4ff9700 1 -- 172.21.15.2:0/2803013846 <== mon.2 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6331 (secure 0 0 0) 0x7f0ed80b2280 con 0x7f0ee8103c50 2024-05-14T09:30:28.871 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:30:28.871 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-14T09:30:28.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.873+0000 7f0eede61700 1 -- 172.21.15.2:0/2803013846 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0ed00b2880 msgr2=0x7f0ed00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:28.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.873+0000 7f0eede61700 1 --2- 172.21.15.2:0/2803013846 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0ed00b2880 0x7f0ed00b4d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8106bf0 tx=0x7f0edc00b480 comp rx=0 tx=0).stop 2024-05-14T09:30:28.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.873+0000 7f0eede61700 1 -- 172.21.15.2:0/2803013846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ee8103c50 msgr2=0x7f0ee819ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:28.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.873+0000 7f0eede61700 1 --2- 172.21.15.2:0/2803013846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ee8103c50 0x7f0ee819ae50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800e9f0 tx=0x7f0ed800ed00 comp rx=0 tx=0).stop 2024-05-14T09:30:28.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.873+0000 7f0eede61700 1 -- 172.21.15.2:0/2803013846 shutdown_connections 2024-05-14T09:30:28.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.873+0000 7f0eede61700 1 --2- 172.21.15.2:0/2803013846 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0ed00b2880 0x7f0ed00b4d90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:28.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.873+0000 7f0eede61700 1 --2- 172.21.15.2:0/2803013846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ee8103c50 0x7f0ee819ae50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:28.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.873+0000 7f0eede61700 1 --2- 172.21.15.2:0/2803013846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ee81087a0 0x7f0ee819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:28.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.873+0000 7f0eede61700 1 --2- 172.21.15.2:0/2803013846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ee8107e50 0x7f0ee819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:28.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.873+0000 7f0eede61700 1 -- 172.21.15.2:0/2803013846 >> 172.21.15.2:0/2803013846 conn(0x7f0ee80fe4e0 msgr2=0x7f0ee8107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:28.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.873+0000 7f0eede61700 1 -- 172.21.15.2:0/2803013846 shutdown_connections 2024-05-14T09:30:28.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:28.873+0000 7f0eede61700 1 -- 172.21.15.2:0/2803013846 wait complete. 2024-05-14T09:30:28.874 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2024-05-14T09:30:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:28 smithi026 ceph-mon[36056]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:28 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:29.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:29 smithi026 ceph-mon[36056]: Deploying daemon mds.h on smithi026 2024-05-14T09:30:29.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:29 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.29:6836/1333834114,v1:172.21.15.29:6837/1333834114] up:boot 2024-05-14T09:30:29.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:29 smithi026 ceph-mon[36056]: fsmap 7 up:standby 2024-05-14T09:30:29.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T09:30:29.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:29 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2803013846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:29.785 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:29 smithi002 ceph-mon[31557]: Deploying daemon mds.h on smithi026 2024-05-14T09:30:29.785 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:29 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.29:6836/1333834114,v1:172.21.15.29:6837/1333834114] up:boot 2024-05-14T09:30:29.785 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:29 smithi002 ceph-mon[31557]: fsmap 7 up:standby 2024-05-14T09:30:29.785 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T09:30:29.785 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:29 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2803013846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:29.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:29 smithi029 ceph-mon[36249]: Deploying daemon mds.h on smithi026 2024-05-14T09:30:29.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:29 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.29:6836/1333834114,v1:172.21.15.29:6837/1333834114] up:boot 2024-05-14T09:30:29.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:29 smithi029 ceph-mon[36249]: fsmap 7 up:standby 2024-05-14T09:30:29.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T09:30:29.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:29 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2803013846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:30.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:30 smithi002 ceph-mon[31557]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:30.911 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:30 smithi026 ceph-mon[36056]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:30.911 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:30:30 smithi026 systemd[1]: Starting Ceph mds.h for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T09:30:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:30 smithi029 ceph-mon[36249]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:31.201 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:30:30 smithi026 podman[66407]: 2024-05-14 09:30:30.814131621 +0000 UTC m=+0.017819095 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T09:30:31.201 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:30:30 smithi026 podman[66407]: 2024-05-14 09:30:30.922904785 +0000 UTC m=+0.126592266 container create 58bf2a4d34111b0e7f4ebb901fe9bd9e86866099b7c0ed64e0f00eb6b09e44e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-14T09:30:31.509 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:30:31.701 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:30:31 smithi026 podman[66407]: 2024-05-14 09:30:31.208878143 +0000 UTC m=+0.412565610 container init 58bf2a4d34111b0e7f4ebb901fe9bd9e86866099b7c0ed64e0f00eb6b09e44e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T09:30:31.702 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:30:31 smithi026 podman[66407]: 2024-05-14 09:30:31.211486662 +0000 UTC m=+0.415174113 container start 58bf2a4d34111b0e7f4ebb901fe9bd9e86866099b7c0ed64e0f00eb6b09e44e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-14T09:30:31.702 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:30:31 smithi026 ceph-mds[66426]: -- 172.21.15.26:0/1616327368 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ece56e1a40 con 0x55ece5729180 2024-05-14T09:30:31.702 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:30:31 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: starting mds.h at 2024-05-14T09:30:31.702 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:30:31 smithi026 bash[66407]: 58bf2a4d34111b0e7f4ebb901fe9bd9e86866099b7c0ed64e0f00eb6b09e44e5 2024-05-14T09:30:31.702 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:30:31 smithi026 systemd[1]: Started Ceph mds.h for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:30:31.768 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:30:32.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:32 smithi002 ceph-mon[31557]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:32.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:32.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:32.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:32.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:32.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:32.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:32.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:32 smithi002 ceph-mon[31557]: Deploying daemon mds.i on smithi029 2024-05-14T09:30:32.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:32 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] up:boot 2024-05-14T09:30:32.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:32 smithi002 ceph-mon[31557]: fsmap 8 up:standby 2024-05-14T09:30:32.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T09:30:32.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:32 smithi029 ceph-mon[36249]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:32.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:32.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:32.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:32.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:32.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:32.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:32.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:32 smithi029 ceph-mon[36249]: Deploying daemon mds.i on smithi029 2024-05-14T09:30:32.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:32 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] up:boot 2024-05-14T09:30:32.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:32 smithi029 ceph-mon[36249]: fsmap 8 up:standby 2024-05-14T09:30:32.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T09:30:32.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:32 smithi026 ceph-mon[36056]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:32.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:32.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:32.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:32.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:32.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:32.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:32.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:32 smithi026 ceph-mon[36056]: Deploying daemon mds.i on smithi029 2024-05-14T09:30:32.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:32 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] up:boot 2024-05-14T09:30:32.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:32 smithi026 ceph-mon[36056]: fsmap 8 up:standby 2024-05-14T09:30:32.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T09:30:33.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.462+0000 7f6b4d17f700 1 -- 172.21.15.2:0/2847122224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b48109860 msgr2=0x7f6b481111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:33.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.462+0000 7f6b4d17f700 1 --2- 172.21.15.2:0/2847122224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b48109860 0x7f6b481111c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000b0d0 tx=0x7f6b4000b3e0 comp rx=0 tx=0).stop 2024-05-14T09:30:33.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.462+0000 7f6b4d17f700 1 -- 172.21.15.2:0/2847122224 shutdown_connections 2024-05-14T09:30:33.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.462+0000 7f6b4d17f700 1 --2- 172.21.15.2:0/2847122224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b48109860 0x7f6b481111c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:33.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.462+0000 7f6b4d17f700 1 --2- 172.21.15.2:0/2847122224 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b481045e0 0x7f6b481049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:33.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.462+0000 7f6b4d17f700 1 --2- 172.21.15.2:0/2847122224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b48108f10 0x7f6b48109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:33.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.462+0000 7f6b4d17f700 1 -- 172.21.15.2:0/2847122224 >> 172.21.15.2:0/2847122224 conn(0x7f6b480ffed0 msgr2=0x7f6b48102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:33.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.462+0000 7f6b4d17f700 1 -- 172.21.15.2:0/2847122224 shutdown_connections 2024-05-14T09:30:33.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.462+0000 7f6b4d17f700 1 -- 172.21.15.2:0/2847122224 wait complete. 2024-05-14T09:30:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.463+0000 7f6b4d17f700 1 Processor -- start 2024-05-14T09:30:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.463+0000 7f6b4d17f700 1 -- start start 2024-05-14T09:30:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.463+0000 7f6b4d17f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b481045e0 0x7f6b4819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.463+0000 7f6b4d17f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b48108f10 0x7f6b4819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.463+0000 7f6b4d17f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b48109860 0x7f6b4819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.464+0000 7f6b4d17f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6b48114490 con 0x7f6b48109860 2024-05-14T09:30:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.464+0000 7f6b4d17f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6b48114310 con 0x7f6b481045e0 2024-05-14T09:30:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.464+0000 7f6b4d17f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6b48114610 con 0x7f6b48108f10 2024-05-14T09:30:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.464+0000 7f6b46d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b481045e0 0x7f6b4819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.464+0000 7f6b4659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b48108f10 0x7f6b4819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.464+0000 7f6b46d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b481045e0 0x7f6b4819adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:33856/0 (socket says 172.21.15.2:33856) 2024-05-14T09:30:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.464+0000 7f6b4759e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b48109860 0x7f6b4819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.464+0000 7f6b46d9d700 1 -- 172.21.15.2:0/710807298 learned_addr learned my addr 172.21.15.2:0/710807298 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:30:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.464+0000 7f6b4659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b48108f10 0x7f6b4819b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:51804/0 (socket says 172.21.15.2:51804) 2024-05-14T09:30:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.464+0000 7f6b46d9d700 1 -- 172.21.15.2:0/710807298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b48108f10 msgr2=0x7f6b4819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.464+0000 7f6b46d9d700 1 --2- 172.21.15.2:0/710807298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b48108f10 0x7f6b4819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.464+0000 7f6b46d9d700 1 -- 172.21.15.2:0/710807298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b48109860 msgr2=0x7f6b4819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.464+0000 7f6b46d9d700 1 --2- 172.21.15.2:0/710807298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b48109860 0x7f6b4819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.464+0000 7f6b46d9d700 1 -- 172.21.15.2:0/710807298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4819fc90 con 0x7f6b481045e0 2024-05-14T09:30:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.464+0000 7f6b46d9d700 1 --2- 172.21.15.2:0/710807298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b481045e0 0x7f6b4819adc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800e9a0 tx=0x7f6b3800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.465+0000 7f6b2ffff700 1 -- 172.21.15.2:0/710807298 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3800ca10 con 0x7f6b481045e0 2024-05-14T09:30:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.465+0000 7f6b2ffff700 1 -- 172.21.15.2:0/710807298 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b3800cbb0 con 0x7f6b481045e0 2024-05-14T09:30:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.465+0000 7f6b4d17f700 1 -- 172.21.15.2:0/710807298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b4819ff80 con 0x7f6b481045e0 2024-05-14T09:30:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.465+0000 7f6b4d17f700 1 -- 172.21.15.2:0/710807298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b48076b00 con 0x7f6b481045e0 2024-05-14T09:30:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.465+0000 7f6b2ffff700 1 -- 172.21.15.2:0/710807298 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b38018680 con 0x7f6b481045e0 2024-05-14T09:30:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.466+0000 7f6b4d17f700 1 -- 172.21.15.2:0/710807298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b48105a90 con 0x7f6b481045e0 2024-05-14T09:30:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.467+0000 7f6b2ffff700 1 -- 172.21.15.2:0/710807298 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f6b38010430 con 0x7f6b481045e0 2024-05-14T09:30:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.468+0000 7f6b2ffff700 1 --2- 172.21.15.2:0/710807298 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6b300b2880 0x7f6b300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.468+0000 7f6b4659c700 1 --2- 172.21.15.2:0/710807298 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6b300b2880 0x7f6b300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.468+0000 7f6b2ffff700 1 -- 172.21.15.2:0/710807298 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f6b380e1fb0 con 0x7f6b481045e0 2024-05-14T09:30:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.469+0000 7f6b4659c700 1 --2- 172.21.15.2:0/710807298 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6b300b2880 0x7f6b300b4d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6b481038b0 tx=0x7f6b3c00f820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:33.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.473+0000 7f6b2ffff700 1 -- 172.21.15.2:0/710807298 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b380aa0d0 con 0x7f6b481045e0 2024-05-14T09:30:33.687 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 09:30:33 smithi029 systemd[1]: Starting Ceph mds.i for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T09:30:33.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.741+0000 7f6b4d17f700 1 -- 172.21.15.2:0/710807298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b4802e5f0 con 0x7f6b481045e0 2024-05-14T09:30:33.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.742+0000 7f6b2ffff700 1 -- 172.21.15.2:0/710807298 <== mon.2 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7156 (secure 0 0 0) 0x7f6b380b2150 con 0x7f6b481045e0 2024-05-14T09:30:33.741 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:30:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-14T09:30:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.744+0000 7f6b4d17f700 1 -- 172.21.15.2:0/710807298 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6b300b2880 msgr2=0x7f6b300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.744+0000 7f6b4d17f700 1 --2- 172.21.15.2:0/710807298 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6b300b2880 0x7f6b300b4d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6b481038b0 tx=0x7f6b3c00f820 comp rx=0 tx=0).stop 2024-05-14T09:30:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.744+0000 7f6b4d17f700 1 -- 172.21.15.2:0/710807298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b481045e0 msgr2=0x7f6b4819adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.744+0000 7f6b4d17f700 1 --2- 172.21.15.2:0/710807298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b481045e0 0x7f6b4819adc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800e9a0 tx=0x7f6b3800ecb0 comp rx=0 tx=0).stop 2024-05-14T09:30:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.744+0000 7f6b4d17f700 1 -- 172.21.15.2:0/710807298 shutdown_connections 2024-05-14T09:30:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.744+0000 7f6b4d17f700 1 --2- 172.21.15.2:0/710807298 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6b300b2880 0x7f6b300b4d90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.744+0000 7f6b4d17f700 1 --2- 172.21.15.2:0/710807298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b481045e0 0x7f6b4819adc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.744+0000 7f6b4d17f700 1 --2- 172.21.15.2:0/710807298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b48109860 0x7f6b4819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.744+0000 7f6b4d17f700 1 --2- 172.21.15.2:0/710807298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b48108f10 0x7f6b4819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.744+0000 7f6b4d17f700 1 -- 172.21.15.2:0/710807298 >> 172.21.15.2:0/710807298 conn(0x7f6b480ffed0 msgr2=0x7f6b48108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.744+0000 7f6b4d17f700 1 -- 172.21.15.2:0/710807298 shutdown_connections 2024-05-14T09:30:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:33.744+0000 7f6b4d17f700 1 -- 172.21.15.2:0/710807298 wait complete. 2024-05-14T09:30:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 9 2024-05-14T09:30:34.110 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 09:30:33 smithi029 podman[66670]: 2024-05-14 09:30:33.713559031 +0000 UTC m=+0.017414390 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T09:30:34.110 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 09:30:33 smithi029 podman[66670]: 2024-05-14 09:30:33.856319237 +0000 UTC m=+0.160174589 container create 9e4f54c74b3e06c278ce449e189e9bc81565ec0e609c714995171ba796d697aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-14T09:30:34.370 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 09:30:34 smithi029 podman[66670]: 2024-05-14 09:30:34.110966693 +0000 UTC m=+0.414822052 container init 9e4f54c74b3e06c278ce449e189e9bc81565ec0e609c714995171ba796d697aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-14T09:30:34.370 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 09:30:34 smithi029 podman[66670]: 2024-05-14 09:30:34.113767825 +0000 UTC m=+0.417623186 container start 9e4f54c74b3e06c278ce449e189e9bc81565ec0e609c714995171ba796d697aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-i, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-14T09:30:34.370 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 09:30:34 smithi029 ceph-mds[66689]: -- 172.21.15.29:0/499681450 <== mon.1 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570363dba40 con 0x557036423600 2024-05-14T09:30:34.370 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 09:30:34 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-i[66685]: starting mds.i at 2024-05-14T09:30:34.370 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 09:30:34 smithi029 bash[66670]: 9e4f54c74b3e06c278ce449e189e9bc81565ec0e609c714995171ba796d697aa 2024-05-14T09:30:34.370 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 09:30:34 smithi029 systemd[1]: Started Ceph mds.i for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:30:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:34 smithi002 ceph-mon[31557]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:34 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/710807298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:34 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:34 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:34 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:34 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:34.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:34 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:34.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:34 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:34 smithi029 ceph-mon[36249]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:34 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/710807298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:34 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:34 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:34 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:34 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:34 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:34 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:34 smithi026 ceph-mon[36056]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:34 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/710807298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:34 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:34 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:34 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:34.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:34 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:34.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:34 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:34.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:34 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:35.540 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:35 smithi002 ceph-mon[31557]: Deploying daemon mds.g on smithi002 2024-05-14T09:30:35.540 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:35 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] up:boot 2024-05-14T09:30:35.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:35 smithi002 ceph-mon[31557]: fsmap 9 up:standby 2024-05-14T09:30:35.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:35 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T09:30:35.547 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:35 smithi029 ceph-mon[36249]: Deploying daemon mds.g on smithi002 2024-05-14T09:30:35.547 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:35 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] up:boot 2024-05-14T09:30:35.547 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:35 smithi029 ceph-mon[36249]: fsmap 9 up:standby 2024-05-14T09:30:35.547 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:35 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T09:30:35.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:35 smithi026 ceph-mon[36056]: Deploying daemon mds.g on smithi002 2024-05-14T09:30:35.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:35 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] up:boot 2024-05-14T09:30:35.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:35 smithi026 ceph-mon[36056]: fsmap 9 up:standby 2024-05-14T09:30:35.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:35 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T09:30:36.380 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 09:30:36 smithi002 systemd[1]: Starting Ceph mds.g for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T09:30:36.430 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:30:36.651 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:36 smithi002 ceph-mon[31557]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:36.651 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 09:30:36 smithi002 podman[70394]: 2024-05-14 09:30:36.412852687 +0000 UTC m=+0.017887835 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T09:30:36.651 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 09:30:36 smithi002 podman[70394]: 2024-05-14 09:30:36.53013751 +0000 UTC m=+0.135172645 container create f3fcf57575ab9c44eaa2e1d7a9f92f752b63a96b79a46ac199eaee71b976fecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-g, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-14T09:30:36.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:36 smithi029 ceph-mon[36249]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:36 smithi026 ceph-mon[36056]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:36.913 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:30:36.921 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 09:30:36 smithi002 podman[70394]: 2024-05-14 09:30:36.774646091 +0000 UTC m=+0.379681221 container init f3fcf57575ab9c44eaa2e1d7a9f92f752b63a96b79a46ac199eaee71b976fecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-14T09:30:36.921 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 09:30:36 smithi002 podman[70394]: 2024-05-14 09:30:36.777758975 +0000 UTC m=+0.382794113 container start f3fcf57575ab9c44eaa2e1d7a9f92f752b63a96b79a46ac199eaee71b976fecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-14T09:30:36.921 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 09:30:36 smithi002 ceph-mds[70459]: -- 172.21.15.2:0/1142930974 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55577e721a40 con 0x55577e769a80 2024-05-14T09:30:36.921 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 09:30:36 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-g[70443]: starting mds.g at 2024-05-14T09:30:36.921 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 09:30:36 smithi002 bash[70394]: f3fcf57575ab9c44eaa2e1d7a9f92f752b63a96b79a46ac199eaee71b976fecf 2024-05-14T09:30:36.921 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 09:30:36 smithi002 systemd[1]: Started Ceph mds.g for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:30:38.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:37 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:38.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:37 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:38.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:37 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:38.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:37 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:38.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:37 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:38.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:37 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:38.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:37 smithi002 ceph-mon[31557]: Deploying daemon mds.a on smithi002 2024-05-14T09:30:38.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:37 smithi002 ceph-mon[31557]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:38.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:37 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.2:6838/2887812543,v1:172.21.15.2:6839/2887812543] up:boot 2024-05-14T09:30:38.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:37 smithi002 ceph-mon[31557]: fsmap 10 up:standby 2024-05-14T09:30:38.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:37 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T09:30:38.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:37 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:38.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:37 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:38.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:37 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:38.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:37 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:38.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:37 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:38.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:37 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:38.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:37 smithi029 ceph-mon[36249]: Deploying daemon mds.a on smithi002 2024-05-14T09:30:38.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:37 smithi029 ceph-mon[36249]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:38.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:37 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.2:6838/2887812543,v1:172.21.15.2:6839/2887812543] up:boot 2024-05-14T09:30:38.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:37 smithi029 ceph-mon[36249]: fsmap 10 up:standby 2024-05-14T09:30:38.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:37 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T09:30:38.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:37 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:38.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:37 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:38.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:37 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:38.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:37 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:38.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:37 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:38.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:37 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:38.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:37 smithi026 ceph-mon[36056]: Deploying daemon mds.a on smithi002 2024-05-14T09:30:38.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:37 smithi026 ceph-mon[36056]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:38.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:37 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.2:6838/2887812543,v1:172.21.15.2:6839/2887812543] up:boot 2024-05-14T09:30:38.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:37 smithi026 ceph-mon[36056]: fsmap 10 up:standby 2024-05-14T09:30:38.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:37 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T09:30:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.878+0000 7faded921700 1 -- 172.21.15.2:0/180329569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fade8102880 msgr2=0x7fade8102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.878+0000 7faded921700 1 --2- 172.21.15.2:0/180329569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fade8102880 0x7fade8102c90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faddc009990 tx=0x7faddc009ca0 comp rx=0 tx=0).stop 2024-05-14T09:30:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.878+0000 7faded921700 1 -- 172.21.15.2:0/180329569 shutdown_connections 2024-05-14T09:30:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.878+0000 7faded921700 1 --2- 172.21.15.2:0/180329569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade8109300 0x7fade8109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.878+0000 7faded921700 1 --2- 172.21.15.2:0/180329569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fade81031d0 0x7fade8111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.878+0000 7faded921700 1 --2- 172.21.15.2:0/180329569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fade8102880 0x7fade8102c90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.878+0000 7faded921700 1 -- 172.21.15.2:0/180329569 >> 172.21.15.2:0/180329569 conn(0x7fade80fe6f0 msgr2=0x7fade8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.878+0000 7faded921700 1 -- 172.21.15.2:0/180329569 shutdown_connections 2024-05-14T09:30:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.878+0000 7faded921700 1 -- 172.21.15.2:0/180329569 wait complete. 2024-05-14T09:30:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.879+0000 7faded921700 1 Processor -- start 2024-05-14T09:30:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.879+0000 7faded921700 1 -- start start 2024-05-14T09:30:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.879+0000 7faded921700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fade8102880 0x7fade819af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.879+0000 7faded921700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fade81031d0 0x7fade819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7faded921700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade8109300 0x7fade819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7faded921700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fade81145a0 con 0x7fade8102880 2024-05-14T09:30:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7faded921700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fade8114420 con 0x7fade8109300 2024-05-14T09:30:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7faded921700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fade8114720 con 0x7fade81031d0 2024-05-14T09:30:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7fade6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fade8102880 0x7fade819af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7fade6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fade8102880 0x7fade819af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55960/0 (socket says 172.21.15.2:55960) 2024-05-14T09:30:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7fade6ffd700 1 -- 172.21.15.2:0/2807856068 learned_addr learned my addr 172.21.15.2:0/2807856068 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:30:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7fade77fe700 1 --2- 172.21.15.2:0/2807856068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade8109300 0x7fade819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7fade67fc700 1 --2- 172.21.15.2:0/2807856068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fade81031d0 0x7fade819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7fade6ffd700 1 -- 172.21.15.2:0/2807856068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fade81031d0 msgr2=0x7fade819b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7fade6ffd700 1 --2- 172.21.15.2:0/2807856068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fade81031d0 0x7fade819b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7fade6ffd700 1 -- 172.21.15.2:0/2807856068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade8109300 msgr2=0x7fade819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7fade6ffd700 1 --2- 172.21.15.2:0/2807856068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade8109300 0x7fade819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7fade6ffd700 1 -- 172.21.15.2:0/2807856068 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade819fd90 con 0x7fade8102880 2024-05-14T09:30:39.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7fade6ffd700 1 --2- 172.21.15.2:0/2807856068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fade8102880 0x7fade819af70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fadd800c970 tx=0x7fadd800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:39.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.880+0000 7fadec91f700 1 -- 172.21.15.2:0/2807856068 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8007910 con 0x7fade8102880 2024-05-14T09:30:39.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.881+0000 7faded921700 1 -- 172.21.15.2:0/2807856068 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade81a0080 con 0x7fade8102880 2024-05-14T09:30:39.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.881+0000 7faded921700 1 -- 172.21.15.2:0/2807856068 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade81a05e0 con 0x7fade8102880 2024-05-14T09:30:39.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.881+0000 7fadec91f700 1 -- 172.21.15.2:0/2807856068 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd800f450 con 0x7fade8102880 2024-05-14T09:30:39.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.881+0000 7fadec91f700 1 -- 172.21.15.2:0/2807856068 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8018660 con 0x7fade8102880 2024-05-14T09:30:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.882+0000 7faded921700 1 -- 172.21.15.2:0/2807856068 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade80512c0 con 0x7fade8102880 2024-05-14T09:30:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.884+0000 7fadec91f700 1 -- 172.21.15.2:0/2807856068 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fadd8007ab0 con 0x7fade8102880 2024-05-14T09:30:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.884+0000 7fadec91f700 1 --2- 172.21.15.2:0/2807856068 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fadd00b2980 0x7fadd00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:39.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.885+0000 7fade67fc700 1 --2- 172.21.15.2:0/2807856068 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fadd00b2980 0x7fadd00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:39.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.885+0000 7fadec91f700 1 -- 172.21.15.2:0/2807856068 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fadd80e1cb0 con 0x7fade8102880 2024-05-14T09:30:39.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.885+0000 7fade67fc700 1 --2- 172.21.15.2:0/2807856068 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fadd00b2980 0x7fadd00b4e90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faddc006010 tx=0x7faddc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:39.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:39.888+0000 7fadec91f700 1 -- 172.21.15.2:0/2807856068 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fadd8053df0 con 0x7fade8102880 2024-05-14T09:30:40.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:40.163+0000 7faded921700 1 -- 172.21.15.2:0/2807856068 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fade819c2d0 con 0x7fade8102880 2024-05-14T09:30:40.164 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:40 smithi002 ceph-mon[31557]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:40.164 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 09:30:40 smithi002 podman[70838]: 2024-05-14 09:30:40.119688454 +0000 UTC m=+0.130266957 container create 73d41c4e0cf74a51b9ecabc66d601c96caf70f1a4f1b083ae463f58e4eb513a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-14T09:30:40.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:40.165+0000 7fadec91f700 1 -- 172.21.15.2:0/2807856068 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8806 (secure 0 0 0) 0x7fadd8014030 con 0x7fade8102880 2024-05-14T09:30:40.165 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:30:40.165 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-14T09:30:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:40.167+0000 7faded921700 1 -- 172.21.15.2:0/2807856068 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fadd00b2980 msgr2=0x7fadd00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:40.167+0000 7faded921700 1 --2- 172.21.15.2:0/2807856068 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fadd00b2980 0x7fadd00b4e90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faddc006010 tx=0x7faddc005960 comp rx=0 tx=0).stop 2024-05-14T09:30:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:40.167+0000 7faded921700 1 -- 172.21.15.2:0/2807856068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fade8102880 msgr2=0x7fade819af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:40.167+0000 7faded921700 1 --2- 172.21.15.2:0/2807856068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fade8102880 0x7fade819af70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fadd800c970 tx=0x7fadd800cc80 comp rx=0 tx=0).stop 2024-05-14T09:30:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:40.167+0000 7faded921700 1 -- 172.21.15.2:0/2807856068 shutdown_connections 2024-05-14T09:30:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:40.167+0000 7faded921700 1 --2- 172.21.15.2:0/2807856068 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fadd00b2980 0x7fadd00b4e90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:40.167+0000 7faded921700 1 --2- 172.21.15.2:0/2807856068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade8109300 0x7fade819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:40.167+0000 7faded921700 1 --2- 172.21.15.2:0/2807856068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fade8102880 0x7fade819af70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:40.167+0000 7faded921700 1 --2- 172.21.15.2:0/2807856068 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fade81031d0 0x7fade819b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:40.167+0000 7faded921700 1 -- 172.21.15.2:0/2807856068 >> 172.21.15.2:0/2807856068 conn(0x7fade80fe6f0 msgr2=0x7fade810a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:40.167+0000 7faded921700 1 -- 172.21.15.2:0/2807856068 shutdown_connections 2024-05-14T09:30:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:40.167+0000 7faded921700 1 -- 172.21.15.2:0/2807856068 wait complete. 2024-05-14T09:30:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 11 2024-05-14T09:30:40.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:40 smithi029 ceph-mon[36249]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:40.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:40 smithi026 ceph-mon[36056]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:40.826 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 09:30:40 smithi002 podman[70838]: 2024-05-14 09:30:40.557598457 +0000 UTC m=+0.568176958 container init 73d41c4e0cf74a51b9ecabc66d601c96caf70f1a4f1b083ae463f58e4eb513a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T09:30:40.826 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 09:30:40 smithi002 podman[70838]: 2024-05-14 09:30:40.560875948 +0000 UTC m=+0.571454459 container start 73d41c4e0cf74a51b9ecabc66d601c96caf70f1a4f1b083ae463f58e4eb513a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-14T09:30:40.826 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 09:30:40 smithi002 ceph-mds[70871]: -- 172.21.15.2:0/3530379796 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c0b55dfa40 con 0x55c0b5627180 2024-05-14T09:30:40.826 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 09:30:40 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-a[70867]: starting mds.a at 2024-05-14T09:30:40.827 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 09:30:40 smithi002 bash[70838]: 73d41c4e0cf74a51b9ecabc66d601c96caf70f1a4f1b083ae463f58e4eb513a4 2024-05-14T09:30:40.827 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 09:30:40 smithi002 systemd[1]: Started Ceph mds.a for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:30:41.417 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:41 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2807856068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:41.417 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:41.417 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:41.417 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:41.417 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:41.417 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:41.417 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:41.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:41 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2807856068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:41.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:41.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:41.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:41.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:41.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:41.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:41.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:41 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2807856068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:41.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:41.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:41.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:41.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T09:30:41.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T09:30:41.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:42 smithi002 ceph-mon[31557]: Deploying daemon mds.b on smithi026 2024-05-14T09:30:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:42 smithi002 ceph-mon[31557]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:42 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.2:6840/4114057120,v1:172.21.15.2:6841/4114057120] up:boot 2024-05-14T09:30:42.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:42 smithi002 ceph-mon[31557]: fsmap 11 up:standby 2024-05-14T09:30:42.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:42 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T09:30:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:42 smithi029 ceph-mon[36249]: Deploying daemon mds.b on smithi026 2024-05-14T09:30:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:42 smithi029 ceph-mon[36249]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:42 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.2:6840/4114057120,v1:172.21.15.2:6841/4114057120] up:boot 2024-05-14T09:30:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:42 smithi029 ceph-mon[36249]: fsmap 11 up:standby 2024-05-14T09:30:42.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:42 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T09:30:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:42 smithi026 ceph-mon[36056]: Deploying daemon mds.b on smithi026 2024-05-14T09:30:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:42 smithi026 ceph-mon[36056]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:42 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.2:6840/4114057120,v1:172.21.15.2:6841/4114057120] up:boot 2024-05-14T09:30:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:42 smithi026 ceph-mon[36056]: fsmap 11 up:standby 2024-05-14T09:30:42.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:42 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T09:30:43.189 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:30:43.452 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 09:30:43 smithi026 podman[66726]: 2024-05-14 09:30:43.394825353 +0000 UTC m=+0.128367112 container create 968bdd4c0b6373c1a07bb45e26ca37dc9545f3500f7ff4c71ab3e6d8685fddec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-05-14T09:30:43.455 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:30:43.951 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 09:30:43 smithi026 podman[66726]: 2024-05-14 09:30:43.655956806 +0000 UTC m=+0.389498560 container init 968bdd4c0b6373c1a07bb45e26ca37dc9545f3500f7ff4c71ab3e6d8685fddec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T09:30:43.951 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 09:30:43 smithi026 podman[66726]: 2024-05-14 09:30:43.65856475 +0000 UTC m=+0.392106504 container start 968bdd4c0b6373c1a07bb45e26ca37dc9545f3500f7ff4c71ab3e6d8685fddec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True) 2024-05-14T09:30:43.952 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 09:30:43 smithi026 ceph-mds[66745]: -- 172.21.15.26:0/2039930986 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56084324ba40 con 0x560843293a80 2024-05-14T09:30:43.952 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 09:30:43 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b[66741]: starting mds.b at 2024-05-14T09:30:43.952 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 09:30:43 smithi026 bash[66726]: 968bdd4c0b6373c1a07bb45e26ca37dc9545f3500f7ff4c71ab3e6d8685fddec 2024-05-14T09:30:43.952 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 09:30:43 smithi026 systemd[1]: Started Ceph mds.b for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T09:30:44.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:44 smithi002 ceph-mon[31557]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:44.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:30:44.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:44 smithi029 ceph-mon[36249]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:44.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:30:44.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:44 smithi026 ceph-mon[36056]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:44.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:44.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:30:45.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.018+0000 7f4057a58700 1 -- 172.21.15.2:0/201185453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4050108f20 msgr2=0x7f4050109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:45.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.018+0000 7f4057a58700 1 --2- 172.21.15.2:0/201185453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4050108f20 0x7f4050109330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f40400099c0 tx=0x7f4040009cd0 comp rx=0 tx=0).stop 2024-05-14T09:30:45.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.019+0000 7f4057a58700 1 -- 172.21.15.2:0/201185453 shutdown_connections 2024-05-14T09:30:45.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.019+0000 7f4057a58700 1 --2- 172.21.15.2:0/201185453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40501045f0 0x7f4050104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:45.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.019+0000 7f4057a58700 1 --2- 172.21.15.2:0/201185453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4050109870 0x7f40501111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:45.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.019+0000 7f4057a58700 1 --2- 172.21.15.2:0/201185453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4050108f20 0x7f4050109330 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:45.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.019+0000 7f4057a58700 1 -- 172.21.15.2:0/201185453 >> 172.21.15.2:0/201185453 conn(0x7f4050100040 msgr2=0x7f4050102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:45.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.019+0000 7f4057a58700 1 -- 172.21.15.2:0/201185453 shutdown_connections 2024-05-14T09:30:45.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.019+0000 7f4057a58700 1 -- 172.21.15.2:0/201185453 wait complete. 2024-05-14T09:30:45.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.020+0000 7f4057a58700 1 Processor -- start 2024-05-14T09:30:45.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.020+0000 7f4057a58700 1 -- start start 2024-05-14T09:30:45.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.020+0000 7f4057a58700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40501045f0 0x7f405019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:45.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.020+0000 7f4057a58700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4050108f20 0x7f405019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:45.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.020+0000 7f4057a58700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4050109870 0x7f405019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:45.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.020+0000 7f4057a58700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4050114480 con 0x7f4050109870 2024-05-14T09:30:45.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.020+0000 7f4057a58700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4050114300 con 0x7f40501045f0 2024-05-14T09:30:45.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.020+0000 7f4057a58700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4050114600 con 0x7f4050108f20 2024-05-14T09:30:45.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.020+0000 7f4055ff5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4050109870 0x7f405019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:45.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.020+0000 7f4054ff3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4050108f20 0x7f405019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:45.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.020+0000 7f4054ff3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4050108f20 0x7f405019b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:38666/0 (socket says 172.21.15.2:38666) 2024-05-14T09:30:45.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.020+0000 7f4055ff5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4050109870 0x7f405019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55990/0 (socket says 172.21.15.2:55990) 2024-05-14T09:30:45.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.020+0000 7f4054ff3700 1 -- 172.21.15.2:0/503736997 learned_addr learned my addr 172.21.15.2:0/503736997 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:30:45.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.020+0000 7f40557f4700 1 --2- 172.21.15.2:0/503736997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40501045f0 0x7f405019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:45.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.021+0000 7f4054ff3700 1 -- 172.21.15.2:0/503736997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40501045f0 msgr2=0x7f405019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:45.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.021+0000 7f4054ff3700 1 --2- 172.21.15.2:0/503736997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40501045f0 0x7f405019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:45.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.021+0000 7f4054ff3700 1 -- 172.21.15.2:0/503736997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4050109870 msgr2=0x7f405019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:45.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.021+0000 7f4054ff3700 1 --2- 172.21.15.2:0/503736997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4050109870 0x7f405019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:45.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.021+0000 7f4054ff3700 1 -- 172.21.15.2:0/503736997 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405019fd00 con 0x7f4050108f20 2024-05-14T09:30:45.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.021+0000 7f4054ff3700 1 --2- 172.21.15.2:0/503736997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4050108f20 0x7f405019b3d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4040005f50 tx=0x7f4040004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:45.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.021+0000 7f40467fc700 1 -- 172.21.15.2:0/503736997 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040026070 con 0x7f4050108f20 2024-05-14T09:30:45.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.021+0000 7f40467fc700 1 -- 172.21.15.2:0/503736997 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4040003c70 con 0x7f4050108f20 2024-05-14T09:30:45.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.021+0000 7f40467fc700 1 -- 172.21.15.2:0/503736997 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404000fa80 con 0x7f4050108f20 2024-05-14T09:30:45.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.021+0000 7f4057a58700 1 -- 172.21.15.2:0/503736997 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405019ff90 con 0x7f4050108f20 2024-05-14T09:30:45.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.021+0000 7f4057a58700 1 -- 172.21.15.2:0/503736997 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40501a0420 con 0x7f4050108f20 2024-05-14T09:30:45.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.022+0000 7f4057a58700 1 -- 172.21.15.2:0/503736997 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40500512c0 con 0x7f4050108f20 2024-05-14T09:30:45.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.024+0000 7f40467fc700 1 -- 172.21.15.2:0/503736997 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f4040029070 con 0x7f4050108f20 2024-05-14T09:30:45.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.025+0000 7f40467fc700 1 --2- 172.21.15.2:0/503736997 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f403c0b2900 0x7f403c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:45.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.025+0000 7f40557f4700 1 --2- 172.21.15.2:0/503736997 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f403c0b2900 0x7f403c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:45.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.025+0000 7f40467fc700 1 -- 172.21.15.2:0/503736997 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f40400e24e0 con 0x7f4050108f20 2024-05-14T09:30:45.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.025+0000 7f40557f4700 1 --2- 172.21.15.2:0/503736997 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f403c0b2900 0x7f403c0b4e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f404c009bf0 tx=0x7f404c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:45.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.029+0000 7f40467fc700 1 -- 172.21.15.2:0/503736997 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4040013070 con 0x7f4050108f20 2024-05-14T09:30:45.116 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:45 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] up:boot 2024-05-14T09:30:45.116 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:45 smithi026 ceph-mon[36056]: fsmap 12 up:standby 2024-05-14T09:30:45.116 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:45 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T09:30:45.137 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:45 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] up:boot 2024-05-14T09:30:45.137 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:45 smithi029 ceph-mon[36249]: fsmap 12 up:standby 2024-05-14T09:30:45.137 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:45 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T09:30:45.327 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:45 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] up:boot 2024-05-14T09:30:45.327 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:45 smithi002 ceph-mon[31557]: fsmap 12 up:standby 2024-05-14T09:30:45.327 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:45 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T09:30:45.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.339+0000 7f4057a58700 1 -- 172.21.15.2:0/503736997 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f405002e5f0 con 0x7f4050108f20 2024-05-14T09:30:45.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.340+0000 7f40467fc700 1 -- 172.21.15.2:0/503736997 <== mon.1 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10452 (secure 0 0 0) 0x7f40400aa690 con 0x7f4050108f20 2024-05-14T09:30:45.340 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:30:45.340 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-14T09:30:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.343+0000 7f4057a58700 1 -- 172.21.15.2:0/503736997 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f403c0b2900 msgr2=0x7f403c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.343+0000 7f4057a58700 1 --2- 172.21.15.2:0/503736997 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f403c0b2900 0x7f403c0b4e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f404c009bf0 tx=0x7f404c009260 comp rx=0 tx=0).stop 2024-05-14T09:30:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.343+0000 7f4057a58700 1 -- 172.21.15.2:0/503736997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4050108f20 msgr2=0x7f405019b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.343+0000 7f4057a58700 1 --2- 172.21.15.2:0/503736997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4050108f20 0x7f405019b3d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4040005f50 tx=0x7f4040004270 comp rx=0 tx=0).stop 2024-05-14T09:30:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.343+0000 7f4057a58700 1 -- 172.21.15.2:0/503736997 shutdown_connections 2024-05-14T09:30:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.343+0000 7f4057a58700 1 --2- 172.21.15.2:0/503736997 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f403c0b2900 0x7f403c0b4e10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.343+0000 7f4057a58700 1 --2- 172.21.15.2:0/503736997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40501045f0 0x7f405019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.343+0000 7f4057a58700 1 --2- 172.21.15.2:0/503736997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4050109870 0x7f405019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.343+0000 7f4057a58700 1 --2- 172.21.15.2:0/503736997 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4050108f20 0x7f405019b3d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:45.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.343+0000 7f4057a58700 1 -- 172.21.15.2:0/503736997 >> 172.21.15.2:0/503736997 conn(0x7f4050100040 msgr2=0x7f405010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:45.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.343+0000 7f4057a58700 1 -- 172.21.15.2:0/503736997 shutdown_connections 2024-05-14T09:30:45.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:45.343+0000 7f4057a58700 1 -- 172.21.15.2:0/503736997 wait complete. 2024-05-14T09:30:45.344 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 13 2024-05-14T09:30:46.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:46 smithi002 ceph-mon[31557]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:46.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:46 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/503736997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:46.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:46.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:46.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:46.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:46.385 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-14T09:30:46.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:46 smithi029 ceph-mon[36249]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:46.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:46 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/503736997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:46.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:46.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:46.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:46.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:46.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:46 smithi026 ceph-mon[36056]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:46.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:46 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/503736997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:30:46.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:46.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:46.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:46.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:30:48.049 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:48.049 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:48.049 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:48.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:30:48.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:48.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:47 smithi002 ceph-mon[31557]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:48.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:30:48.283 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:48.284 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:48.284 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:48.284 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:30:48.284 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:48.284 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:47 smithi026 ceph-mon[36056]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:48.284 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:30:48.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.326+0000 7f87b8c29700 1 -- 172.21.15.2:0/1186728192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b4108f20 msgr2=0x7f87b4109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:48.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.326+0000 7f87b8c29700 1 --2- 172.21.15.2:0/1186728192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b4108f20 0x7f87b4109330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f87a80099c0 tx=0x7f87a8009cd0 comp rx=0 tx=0).stop 2024-05-14T09:30:48.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.326+0000 7f87b8c29700 1 -- 172.21.15.2:0/1186728192 shutdown_connections 2024-05-14T09:30:48.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.326+0000 7f87b8c29700 1 --2- 172.21.15.2:0/1186728192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b41045f0 0x7f87b4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:48.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.326+0000 7f87b8c29700 1 --2- 172.21.15.2:0/1186728192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87b4109870 0x7f87b41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:48.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.326+0000 7f87b8c29700 1 --2- 172.21.15.2:0/1186728192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b4108f20 0x7f87b4109330 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:48.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.326+0000 7f87b8c29700 1 -- 172.21.15.2:0/1186728192 >> 172.21.15.2:0/1186728192 conn(0x7f87b4100040 msgr2=0x7f87b4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:48.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.326+0000 7f87b8c29700 1 -- 172.21.15.2:0/1186728192 shutdown_connections 2024-05-14T09:30:48.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.326+0000 7f87b8c29700 1 -- 172.21.15.2:0/1186728192 wait complete. 2024-05-14T09:30:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.327+0000 7f87b8c29700 1 Processor -- start 2024-05-14T09:30:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.327+0000 7f87b8c29700 1 -- start start 2024-05-14T09:30:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.327+0000 7f87b8c29700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87b41045f0 0x7f87b419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.327+0000 7f87b8c29700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b4108f20 0x7f87b419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b8c29700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b4109870 0x7f87b419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b8c29700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4114480 con 0x7f87b4108f20 2024-05-14T09:30:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b8c29700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4114300 con 0x7f87b4109870 2024-05-14T09:30:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b8c29700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4114600 con 0x7f87b41045f0 2024-05-14T09:30:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b1d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b4108f20 0x7f87b419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b1d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b4108f20 0x7f87b419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53828/0 (socket says 172.21.15.2:53828) 2024-05-14T09:30:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b1d9b700 1 -- 172.21.15.2:0/1470868197 learned_addr learned my addr 172.21.15.2:0/1470868197 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:30:48.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b259c700 1 --2- 172.21.15.2:0/1470868197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87b41045f0 0x7f87b419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:48.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b2d9d700 1 --2- 172.21.15.2:0/1470868197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b4109870 0x7f87b419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:48.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b1d9b700 1 -- 172.21.15.2:0/1470868197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87b41045f0 msgr2=0x7f87b419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:48.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b1d9b700 1 --2- 172.21.15.2:0/1470868197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87b41045f0 0x7f87b419ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:48.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b1d9b700 1 -- 172.21.15.2:0/1470868197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b4109870 msgr2=0x7f87b419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:48.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b1d9b700 1 --2- 172.21.15.2:0/1470868197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b4109870 0x7f87b419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:48.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b1d9b700 1 -- 172.21.15.2:0/1470868197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b419fd60 con 0x7f87b4108f20 2024-05-14T09:30:48.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b1d9b700 1 --2- 172.21.15.2:0/1470868197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b4108f20 0x7f87b419b430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f87a80098e0 tx=0x7f87a80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:48.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87a37fe700 1 -- 172.21.15.2:0/1470868197 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8027070 con 0x7f87b4108f20 2024-05-14T09:30:48.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87a37fe700 1 -- 172.21.15.2:0/1470868197 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a8003f20 con 0x7f87b4108f20 2024-05-14T09:30:48.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.328+0000 7f87b8c29700 1 -- 172.21.15.2:0/1470868197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b419fff0 con 0x7f87b4108f20 2024-05-14T09:30:48.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.329+0000 7f87a37fe700 1 -- 172.21.15.2:0/1470868197 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8022c70 con 0x7f87b4108f20 2024-05-14T09:30:48.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.329+0000 7f87b8c29700 1 -- 172.21.15.2:0/1470868197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b41a04d0 con 0x7f87b4108f20 2024-05-14T09:30:48.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.329+0000 7f87b8c29700 1 -- 172.21.15.2:0/1470868197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b40512c0 con 0x7f87b4108f20 2024-05-14T09:30:48.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.331+0000 7f87a37fe700 1 -- 172.21.15.2:0/1470868197 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f87a802a070 con 0x7f87b4108f20 2024-05-14T09:30:48.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.332+0000 7f87a37fe700 1 --2- 172.21.15.2:0/1470868197 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f879c0b2880 0x7f879c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:48.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.332+0000 7f87b259c700 1 --2- 172.21.15.2:0/1470868197 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f879c0b2880 0x7f879c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:48.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.332+0000 7f87a37fe700 1 -- 172.21.15.2:0/1470868197 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f87a80e33d0 con 0x7f87b4108f20 2024-05-14T09:30:48.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.333+0000 7f87b259c700 1 --2- 172.21.15.2:0/1470868197 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f879c0b2880 0x7f879c0b4d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f87a4005950 tx=0x7f87a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:48.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.335+0000 7f87a37fe700 1 -- 172.21.15.2:0/1470868197 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87a8031080 con 0x7f87b4108f20 2024-05-14T09:30:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:30:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:47 smithi029 ceph-mon[36249]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:30:48.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.552+0000 7f87b8c29700 1 -- 172.21.15.2:0/1470868197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f87b402e5f0 con 0x7f87b4108f20 2024-05-14T09:30:48.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.552+0000 7f87a37fe700 1 -- 172.21.15.2:0/1470868197 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f87a8013070 con 0x7f87b4108f20 2024-05-14T09:30:48.552 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:30:48.552 INFO:teuthology.orchestra.run.smithi002.stdout:[ 2024-05-14T09:30:48.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:30:48.552 INFO:teuthology.orchestra.run.smithi002.stdout: "poolnum": 1, 2024-05-14T09:30:48.552 INFO:teuthology.orchestra.run.smithi002.stdout: "poolname": ".mgr" 2024-05-14T09:30:48.553 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T09:30:48.553 INFO:teuthology.orchestra.run.smithi002.stdout:] 2024-05-14T09:30:48.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.555+0000 7f87b8c29700 1 -- 172.21.15.2:0/1470868197 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f879c0b2880 msgr2=0x7f879c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:48.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.555+0000 7f87b8c29700 1 --2- 172.21.15.2:0/1470868197 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f879c0b2880 0x7f879c0b4d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f87a4005950 tx=0x7f87a400b410 comp rx=0 tx=0).stop 2024-05-14T09:30:48.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.555+0000 7f87b8c29700 1 -- 172.21.15.2:0/1470868197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b4108f20 msgr2=0x7f87b419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:48.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.555+0000 7f87b8c29700 1 --2- 172.21.15.2:0/1470868197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b4108f20 0x7f87b419b430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f87a80098e0 tx=0x7f87a80057c0 comp rx=0 tx=0).stop 2024-05-14T09:30:48.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.555+0000 7f87b8c29700 1 -- 172.21.15.2:0/1470868197 shutdown_connections 2024-05-14T09:30:48.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.555+0000 7f87b8c29700 1 --2- 172.21.15.2:0/1470868197 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f879c0b2880 0x7f879c0b4d90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:48.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.555+0000 7f87b8c29700 1 --2- 172.21.15.2:0/1470868197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b4109870 0x7f87b419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:48.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.555+0000 7f87b8c29700 1 --2- 172.21.15.2:0/1470868197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f87b41045f0 0x7f87b419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:48.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.555+0000 7f87b8c29700 1 --2- 172.21.15.2:0/1470868197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b4108f20 0x7f87b419b430 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:48.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.555+0000 7f87b8c29700 1 -- 172.21.15.2:0/1470868197 >> 172.21.15.2:0/1470868197 conn(0x7f87b4100040 msgr2=0x7f87b410a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:48.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.555+0000 7f87b8c29700 1 -- 172.21.15.2:0/1470868197 shutdown_connections 2024-05-14T09:30:48.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:48.555+0000 7f87b8c29700 1 -- 172.21.15.2:0/1470868197 wait complete. 2024-05-14T09:30:49.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:48 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1470868197' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-14T09:30:49.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:49.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:49.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:49.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:49.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:48 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1470868197' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-14T09:30:49.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:49.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:49.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:49.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:49.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:48 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1470868197' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-14T09:30:49.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:49.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:49.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:49.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:49.629 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-14T09:30:49.629 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-14T09:30:49.900 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:30:50.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:49 smithi002 ceph-mon[31557]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:50.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:49 smithi029 ceph-mon[36249]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:50.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:49 smithi026 ceph-mon[36056]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:51.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.575+0000 7f377ed29700 1 -- 172.21.15.2:0/4162078035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3778105dd0 msgr2=0x7f377810e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:51.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.575+0000 7f377ed29700 1 --2- 172.21.15.2:0/4162078035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3778105dd0 0x7f377810e320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3764009990 tx=0x7f3764009ca0 comp rx=0 tx=0).stop 2024-05-14T09:30:51.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.575+0000 7f377ed29700 1 -- 172.21.15.2:0/4162078035 shutdown_connections 2024-05-14T09:30:51.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.575+0000 7f377ed29700 1 --2- 172.21.15.2:0/4162078035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377810e8f0 0x7f3778110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:51.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.575+0000 7f377ed29700 1 --2- 172.21.15.2:0/4162078035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3778105480 0x7f3778105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:51.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.575+0000 7f377ed29700 1 --2- 172.21.15.2:0/4162078035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3778105dd0 0x7f377810e320 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:51.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.575+0000 7f377ed29700 1 -- 172.21.15.2:0/4162078035 >> 172.21.15.2:0/4162078035 conn(0x7f37780fef30 msgr2=0x7f3778101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:51.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.575+0000 7f377ed29700 1 -- 172.21.15.2:0/4162078035 shutdown_connections 2024-05-14T09:30:51.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.575+0000 7f377ed29700 1 -- 172.21.15.2:0/4162078035 wait complete. 2024-05-14T09:30:51.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.576+0000 7f377ed29700 1 Processor -- start 2024-05-14T09:30:51.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.576+0000 7f377ed29700 1 -- start start 2024-05-14T09:30:51.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.576+0000 7f377ed29700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3778105480 0x7f377819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:51.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.576+0000 7f377ed29700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3778105dd0 0x7f377819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:51.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.576+0000 7f377ed29700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f377810e8f0 0x7f377819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:51.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.577+0000 7f377ed29700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3778114450 con 0x7f377810e8f0 2024-05-14T09:30:51.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.577+0000 7f377ed29700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f37781142d0 con 0x7f3778105480 2024-05-14T09:30:51.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.577+0000 7f377ed29700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f37781145d0 con 0x7f3778105dd0 2024-05-14T09:30:51.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.577+0000 7f377d2c6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f377810e8f0 0x7f377819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:51.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.577+0000 7f377cac5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3778105480 0x7f377819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:51.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.577+0000 7f377d2c6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f377810e8f0 0x7f377819f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53836/0 (socket says 172.21.15.2:53836) 2024-05-14T09:30:51.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.577+0000 7f377d2c6700 1 -- 172.21.15.2:0/1175448003 learned_addr learned my addr 172.21.15.2:0/1175448003 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:30:51.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.577+0000 7f3777fff700 1 --2- 172.21.15.2:0/1175448003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3778105dd0 0x7f377819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:51.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.577+0000 7f377d2c6700 1 -- 172.21.15.2:0/1175448003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3778105dd0 msgr2=0x7f377819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:51.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.577+0000 7f377d2c6700 1 --2- 172.21.15.2:0/1175448003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3778105dd0 0x7f377819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:51.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.577+0000 7f377d2c6700 1 -- 172.21.15.2:0/1175448003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3778105480 msgr2=0x7f377819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:51.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.577+0000 7f377d2c6700 1 --2- 172.21.15.2:0/1175448003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3778105480 0x7f377819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:51.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.577+0000 7f377d2c6700 1 -- 172.21.15.2:0/1175448003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377819fca0 con 0x7f377810e8f0 2024-05-14T09:30:51.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.577+0000 7f377d2c6700 1 --2- 172.21.15.2:0/1175448003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f377810e8f0 0x7f377819f600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3770007ee0 tx=0x7f377000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:51.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.577+0000 7f3775ffb700 1 -- 172.21.15.2:0/1175448003 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770004bb0 con 0x7f377810e8f0 2024-05-14T09:30:51.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.578+0000 7f377ed29700 1 -- 172.21.15.2:0/1175448003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377819ff90 con 0x7f377810e8f0 2024-05-14T09:30:51.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.578+0000 7f3775ffb700 1 -- 172.21.15.2:0/1175448003 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3770004d50 con 0x7f377810e8f0 2024-05-14T09:30:51.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.578+0000 7f3775ffb700 1 -- 172.21.15.2:0/1175448003 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770005920 con 0x7f377810e8f0 2024-05-14T09:30:51.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.578+0000 7f377ed29700 1 -- 172.21.15.2:0/1175448003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37781a78b0 con 0x7f377810e8f0 2024-05-14T09:30:51.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.579+0000 7f377ed29700 1 -- 172.21.15.2:0/1175448003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f377806c920 con 0x7f377810e8f0 2024-05-14T09:30:51.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.580+0000 7f3775ffb700 1 -- 172.21.15.2:0/1175448003 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f377001a020 con 0x7f377810e8f0 2024-05-14T09:30:51.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.582+0000 7f3775ffb700 1 --2- 172.21.15.2:0/1175448003 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f37680b2880 0x7f37680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:51.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.582+0000 7f377cac5700 1 --2- 172.21.15.2:0/1175448003 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f37680b2880 0x7f37680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:51.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.582+0000 7f3775ffb700 1 -- 172.21.15.2:0/1175448003 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f37700e19e0 con 0x7f377810e8f0 2024-05-14T09:30:51.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.582+0000 7f377cac5700 1 --2- 172.21.15.2:0/1175448003 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f37680b2880 0x7f37680b4d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f376c009c60 tx=0x7f376c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.586+0000 7f3775ffb700 1 -- 172.21.15.2:0/1175448003 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3770010070 con 0x7f377810e8f0 2024-05-14T09:30:51.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:51.809+0000 7f377ed29700 1 -- 172.21.15.2:0/1175448003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f377802e5f0 con 0x7f377810e8f0 2024-05-14T09:30:52.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:52 smithi002 ceph-mon[31557]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:52.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:52.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:52.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:52.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:30:52.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:52.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:52 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1175448003' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-14T09:30:52.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:52 smithi029 ceph-mon[36249]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:52.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:52 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:52.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:52 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:52.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:52 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:52.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:52 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:30:52.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:52 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:52.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:52 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1175448003' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-14T09:30:52.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:52 smithi026 ceph-mon[36056]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:52.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:52 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:52.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:52 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:52.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:52 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:30:52.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:52 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:30:52.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:52 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:30:52.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:52 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1175448003' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-14T09:30:52.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:52.748+0000 7f3775ffb700 1 -- 172.21.15.2:0/1175448003 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f37700a9ba0 con 0x7f377810e8f0 2024-05-14T09:30:52.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:52.752+0000 7f377ed29700 1 -- 172.21.15.2:0/1175448003 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f37680b2880 msgr2=0x7f37680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:52.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:52.752+0000 7f377ed29700 1 --2- 172.21.15.2:0/1175448003 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f37680b2880 0x7f37680b4d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f376c009c60 tx=0x7f376c009260 comp rx=0 tx=0).stop 2024-05-14T09:30:52.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:52.752+0000 7f377ed29700 1 -- 172.21.15.2:0/1175448003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f377810e8f0 msgr2=0x7f377819f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:52.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:52.752+0000 7f377ed29700 1 --2- 172.21.15.2:0/1175448003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f377810e8f0 0x7f377819f600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3770007ee0 tx=0x7f377000d3b0 comp rx=0 tx=0).stop 2024-05-14T09:30:52.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:52.752+0000 7f377ed29700 1 -- 172.21.15.2:0/1175448003 shutdown_connections 2024-05-14T09:30:52.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:52.752+0000 7f377ed29700 1 --2- 172.21.15.2:0/1175448003 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f37680b2880 0x7f37680b4d90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:52.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:52.752+0000 7f377ed29700 1 --2- 172.21.15.2:0/1175448003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3778105480 0x7f377819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:52.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:52.752+0000 7f377ed29700 1 --2- 172.21.15.2:0/1175448003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f377810e8f0 0x7f377819f600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:52.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:52.752+0000 7f377ed29700 1 --2- 172.21.15.2:0/1175448003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3778105dd0 0x7f377819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:52.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:52.752+0000 7f377ed29700 1 -- 172.21.15.2:0/1175448003 >> 172.21.15.2:0/1175448003 conn(0x7f37780fef30 msgr2=0x7f37780ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:52.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:52.752+0000 7f377ed29700 1 -- 172.21.15.2:0/1175448003 shutdown_connections 2024-05-14T09:30:52.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:52.753+0000 7f377ed29700 1 -- 172.21.15.2:0/1175448003 wait complete. 2024-05-14T09:30:52.755 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' created 2024-05-14T09:30:53.408 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-14T09:30:53.667 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:30:54.069 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:53 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1175448003' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-14T09:30:54.069 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:53 smithi002 ceph-mon[31557]: osdmap e73: 12 total, 12 up, 12 in 2024-05-14T09:30:54.069 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:53 smithi002 ceph-mon[31557]: pgmap v285: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:54.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:53 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1175448003' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-14T09:30:54.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:53 smithi029 ceph-mon[36249]: osdmap e73: 12 total, 12 up, 12 in 2024-05-14T09:30:54.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:53 smithi029 ceph-mon[36249]: pgmap v285: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:54.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:53 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1175448003' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-14T09:30:54.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:53 smithi026 ceph-mon[36056]: osdmap e73: 12 total, 12 up, 12 in 2024-05-14T09:30:54.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:53 smithi026 ceph-mon[36056]: pgmap v285: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:54 smithi002 ceph-mon[31557]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T09:30:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:54 smithi002 ceph-mon[31557]: osdmap e74: 12 total, 12 up, 12 in 2024-05-14T09:30:55.159 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:54 smithi026 ceph-mon[36056]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T09:30:55.159 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:54 smithi026 ceph-mon[36056]: osdmap e74: 12 total, 12 up, 12 in 2024-05-14T09:30:55.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:54 smithi029 ceph-mon[36249]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T09:30:55.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:54 smithi029 ceph-mon[36249]: osdmap e74: 12 total, 12 up, 12 in 2024-05-14T09:30:55.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.269+0000 7f7a894b1700 1 -- 172.21.15.2:0/3697483008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a8410e8f0 msgr2=0x7f7a84110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:55.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.269+0000 7f7a894b1700 1 --2- 172.21.15.2:0/3697483008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a8410e8f0 0x7f7a84110e00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00b0a0 tx=0x7f7a7c00b3b0 comp rx=0 tx=0).stop 2024-05-14T09:30:55.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.269+0000 7f7a894b1700 1 -- 172.21.15.2:0/3697483008 shutdown_connections 2024-05-14T09:30:55.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.269+0000 7f7a894b1700 1 --2- 172.21.15.2:0/3697483008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a84105480 0x7f7a84105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:55.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.269+0000 7f7a894b1700 1 --2- 172.21.15.2:0/3697483008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a8410e8f0 0x7f7a84110e00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:55.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.269+0000 7f7a894b1700 1 --2- 172.21.15.2:0/3697483008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a84105dd0 0x7f7a8410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:55.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.269+0000 7f7a894b1700 1 -- 172.21.15.2:0/3697483008 >> 172.21.15.2:0/3697483008 conn(0x7f7a840fef50 msgr2=0x7f7a84101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:55.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.269+0000 7f7a894b1700 1 -- 172.21.15.2:0/3697483008 shutdown_connections 2024-05-14T09:30:55.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.269+0000 7f7a894b1700 1 -- 172.21.15.2:0/3697483008 wait complete. 2024-05-14T09:30:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.270+0000 7f7a894b1700 1 Processor -- start 2024-05-14T09:30:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.270+0000 7f7a894b1700 1 -- start start 2024-05-14T09:30:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a894b1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a84105480 0x7f7a84103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a894b1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a84105dd0 0x7f7a841036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a894b1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a8410e8f0 0x7f7a84103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a894b1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7a841144a0 con 0x7f7a84105dd0 2024-05-14T09:30:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a894b1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7a84114320 con 0x7f7a84105480 2024-05-14T09:30:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a894b1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7a84114620 con 0x7f7a8410e8f0 2024-05-14T09:30:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a827fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a84105dd0 0x7f7a841036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a827fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a84105dd0 0x7f7a841036b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53866/0 (socket says 172.21.15.2:53866) 2024-05-14T09:30:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a827fc700 1 -- 172.21.15.2:0/3180027002 learned_addr learned my addr 172.21.15.2:0/3180027002 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:30:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a82ffd700 1 --2- 172.21.15.2:0/3180027002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a84105480 0x7f7a84103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a837fe700 1 --2- 172.21.15.2:0/3180027002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a8410e8f0 0x7f7a84103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a827fc700 1 -- 172.21.15.2:0/3180027002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a8410e8f0 msgr2=0x7f7a84103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a827fc700 1 --2- 172.21.15.2:0/3180027002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a8410e8f0 0x7f7a84103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a827fc700 1 -- 172.21.15.2:0/3180027002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a84105480 msgr2=0x7f7a84103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a827fc700 1 --2- 172.21.15.2:0/3180027002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a84105480 0x7f7a84103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a827fc700 1 -- 172.21.15.2:0/3180027002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a841a9130 con 0x7f7a84105dd0 2024-05-14T09:30:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a82ffd700 1 --2- 172.21.15.2:0/3180027002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a84105480 0x7f7a84103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:30:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.271+0000 7f7a827fc700 1 --2- 172.21.15.2:0/3180027002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a84105dd0 0x7f7a841036b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800d950 tx=0x7f7a7800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.272+0000 7f7a6bfff700 1 -- 172.21.15.2:0/3180027002 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78016070 con 0x7f7a84105dd0 2024-05-14T09:30:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.272+0000 7f7a894b1700 1 -- 172.21.15.2:0/3180027002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a841a9420 con 0x7f7a84105dd0 2024-05-14T09:30:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.272+0000 7f7a6bfff700 1 -- 172.21.15.2:0/3180027002 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a78004830 con 0x7f7a84105dd0 2024-05-14T09:30:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.272+0000 7f7a6bfff700 1 -- 172.21.15.2:0/3180027002 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78005520 con 0x7f7a84105dd0 2024-05-14T09:30:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.272+0000 7f7a894b1700 1 -- 172.21.15.2:0/3180027002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a841a9960 con 0x7f7a84105dd0 2024-05-14T09:30:55.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.273+0000 7f7a894b1700 1 -- 172.21.15.2:0/3180027002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a840512c0 con 0x7f7a84105dd0 2024-05-14T09:30:55.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.275+0000 7f7a6bfff700 1 -- 172.21.15.2:0/3180027002 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f7a78004220 con 0x7f7a84105dd0 2024-05-14T09:30:55.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.276+0000 7f7a6bfff700 1 --2- 172.21.15.2:0/3180027002 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7a6c0b2880 0x7f7a6c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:55.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.276+0000 7f7a82ffd700 1 --2- 172.21.15.2:0/3180027002 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7a6c0b2880 0x7f7a6c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:55.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.276+0000 7f7a6bfff700 1 -- 172.21.15.2:0/3180027002 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7497+0+0 (secure 0 0 0) 0x7f7a780e2120 con 0x7f7a84105dd0 2024-05-14T09:30:55.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.276+0000 7f7a82ffd700 1 --2- 172.21.15.2:0/3180027002 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7a6c0b2880 0x7f7a6c0b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7a8406d150 tx=0x7f7a74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:55.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.279+0000 7f7a6bfff700 1 -- 172.21.15.2:0/3180027002 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a7801f020 con 0x7f7a84105dd0 2024-05-14T09:30:55.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.494+0000 7f7a894b1700 1 -- 172.21.15.2:0/3180027002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f7a8402e5f0 con 0x7f7a84105dd0 2024-05-14T09:30:55.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.767+0000 7f7a6bfff700 1 -- 172.21.15.2:0/3180027002 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v76) ==== 171+0+0 (secure 0 0 0) 0x7f7a780aa0d0 con 0x7f7a84105dd0 2024-05-14T09:30:55.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.770+0000 7f7a894b1700 1 -- 172.21.15.2:0/3180027002 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7a6c0b2880 msgr2=0x7f7a6c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:55.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.770+0000 7f7a894b1700 1 --2- 172.21.15.2:0/3180027002 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7a6c0b2880 0x7f7a6c0b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7a8406d150 tx=0x7f7a74009260 comp rx=0 tx=0).stop 2024-05-14T09:30:55.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.770+0000 7f7a894b1700 1 -- 172.21.15.2:0/3180027002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a84105dd0 msgr2=0x7f7a841036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:55.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.770+0000 7f7a894b1700 1 --2- 172.21.15.2:0/3180027002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a84105dd0 0x7f7a841036b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800d950 tx=0x7f7a7800dc60 comp rx=0 tx=0).stop 2024-05-14T09:30:55.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.770+0000 7f7a894b1700 1 -- 172.21.15.2:0/3180027002 shutdown_connections 2024-05-14T09:30:55.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.770+0000 7f7a894b1700 1 --2- 172.21.15.2:0/3180027002 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7a6c0b2880 0x7f7a6c0b4d90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:55.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.770+0000 7f7a894b1700 1 --2- 172.21.15.2:0/3180027002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a84105480 0x7f7a84103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.770+0000 7f7a894b1700 1 --2- 172.21.15.2:0/3180027002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a8410e8f0 0x7f7a84103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.770+0000 7f7a894b1700 1 --2- 172.21.15.2:0/3180027002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a84105dd0 0x7f7a841036b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.770+0000 7f7a894b1700 1 -- 172.21.15.2:0/3180027002 >> 172.21.15.2:0/3180027002 conn(0x7f7a840fef50 msgr2=0x7f7a840ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.770+0000 7f7a894b1700 1 -- 172.21.15.2:0/3180027002 shutdown_connections 2024-05-14T09:30:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:55.770+0000 7f7a894b1700 1 -- 172.21.15.2:0/3180027002 wait complete. 2024-05-14T09:30:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' created 2024-05-14T09:30:56.096 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:55 smithi002 ceph-mon[31557]: osdmap e75: 12 total, 12 up, 12 in 2024-05-14T09:30:56.096 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:55 smithi002 ceph-mon[31557]: pgmap v288: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:56.096 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:55 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3180027002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-14T09:30:56.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:55 smithi029 ceph-mon[36249]: osdmap e75: 12 total, 12 up, 12 in 2024-05-14T09:30:56.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:55 smithi029 ceph-mon[36249]: pgmap v288: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:56.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:55 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3180027002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-14T09:30:56.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:55 smithi026 ceph-mon[36056]: osdmap e75: 12 total, 12 up, 12 in 2024-05-14T09:30:56.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:55 smithi026 ceph-mon[36056]: pgmap v288: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:56.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:55 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3180027002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-14T09:30:56.439 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-14T09:30:56.694 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:30:57.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:56 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3180027002' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-14T09:30:57.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:56 smithi002 ceph-mon[31557]: osdmap e76: 12 total, 12 up, 12 in 2024-05-14T09:30:57.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:56 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3180027002' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-14T09:30:57.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:56 smithi029 ceph-mon[36249]: osdmap e76: 12 total, 12 up, 12 in 2024-05-14T09:30:57.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:56 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3180027002' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-14T09:30:57.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:56 smithi026 ceph-mon[36056]: osdmap e76: 12 total, 12 up, 12 in 2024-05-14T09:30:58.106 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:57 smithi002 ceph-mon[31557]: osdmap e77: 12 total, 12 up, 12 in 2024-05-14T09:30:58.107 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:57 smithi002 ceph-mon[31557]: pgmap v291: 97 pgs: 20 creating+peering, 53 unknown, 24 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:58.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:57 smithi029 ceph-mon[36249]: osdmap e77: 12 total, 12 up, 12 in 2024-05-14T09:30:58.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:57 smithi029 ceph-mon[36249]: pgmap v291: 97 pgs: 20 creating+peering, 53 unknown, 24 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:58.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:57 smithi026 ceph-mon[36056]: osdmap e77: 12 total, 12 up, 12 in 2024-05-14T09:30:58.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:57 smithi026 ceph-mon[36056]: pgmap v291: 97 pgs: 20 creating+peering, 53 unknown, 24 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:30:58.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.372+0000 7fb0571c1700 1 -- 172.21.15.2:0/3907778157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0501034c0 msgr2=0x7fb0501038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:58.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.372+0000 7fb0571c1700 1 --2- 172.21.15.2:0/3907778157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0501034c0 0x7fb0501038d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb0440099c0 tx=0x7fb044009cd0 comp rx=0 tx=0).stop 2024-05-14T09:30:58.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.372+0000 7fb0571c1700 1 -- 172.21.15.2:0/3907778157 shutdown_connections 2024-05-14T09:30:58.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.372+0000 7fb0571c1700 1 --2- 172.21.15.2:0/3907778157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb050108740 0x7fb0501100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:58.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.372+0000 7fb0571c1700 1 --2- 172.21.15.2:0/3907778157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0501034c0 0x7fb0501038d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:58.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.372+0000 7fb0571c1700 1 --2- 172.21.15.2:0/3907778157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb050107df0 0x7fb050108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:58.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.372+0000 7fb0571c1700 1 -- 172.21.15.2:0/3907778157 >> 172.21.15.2:0/3907778157 conn(0x7fb0500fedf0 msgr2=0x7fb050101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:58.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.372+0000 7fb0571c1700 1 -- 172.21.15.2:0/3907778157 shutdown_connections 2024-05-14T09:30:58.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.372+0000 7fb0571c1700 1 -- 172.21.15.2:0/3907778157 wait complete. 2024-05-14T09:30:58.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.373+0000 7fb0571c1700 1 Processor -- start 2024-05-14T09:30:58.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.373+0000 7fb0571c1700 1 -- start start 2024-05-14T09:30:58.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.373+0000 7fb0571c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0501034c0 0x7fb05019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:58.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.373+0000 7fb0571c1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb050107df0 0x7fb05019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:58.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.373+0000 7fb0571c1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb050108740 0x7fb05019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:58.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.373+0000 7fb0571c1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb050113370 con 0x7fb050107df0 2024-05-14T09:30:58.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.373+0000 7fb0571c1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb0501131f0 con 0x7fb0501034c0 2024-05-14T09:30:58.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.373+0000 7fb0571c1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb0501134f0 con 0x7fb050108740 2024-05-14T09:30:58.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.373+0000 7fb04ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb050107df0 0x7fb05019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:58.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.374+0000 7fb04ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb050107df0 0x7fb05019b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36740/0 (socket says 172.21.15.2:36740) 2024-05-14T09:30:58.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.374+0000 7fb04ffff700 1 -- 172.21.15.2:0/2881612878 learned_addr learned my addr 172.21.15.2:0/2881612878 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:30:58.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.374+0000 7fb054f5d700 1 --2- 172.21.15.2:0/2881612878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0501034c0 0x7fb05019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:58.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.374+0000 7fb05575e700 1 --2- 172.21.15.2:0/2881612878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb050108740 0x7fb05019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:58.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.374+0000 7fb04ffff700 1 -- 172.21.15.2:0/2881612878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb050108740 msgr2=0x7fb05019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:58.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.374+0000 7fb04ffff700 1 --2- 172.21.15.2:0/2881612878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb050108740 0x7fb05019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:58.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.374+0000 7fb04ffff700 1 -- 172.21.15.2:0/2881612878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0501034c0 msgr2=0x7fb05019aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:58.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.374+0000 7fb04ffff700 1 --2- 172.21.15.2:0/2881612878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0501034c0 0x7fb05019aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:58.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.374+0000 7fb04ffff700 1 -- 172.21.15.2:0/2881612878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05019fd80 con 0x7fb050107df0 2024-05-14T09:30:58.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.374+0000 7fb054f5d700 1 --2- 172.21.15.2:0/2881612878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0501034c0 0x7fb05019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:30:58.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.374+0000 7fb04ffff700 1 --2- 172.21.15.2:0/2881612878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb050107df0 0x7fb05019b450 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00d980 tx=0x7fb03c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:58.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.374+0000 7fb04dffb700 1 -- 172.21.15.2:0/2881612878 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c017070 con 0x7fb050107df0 2024-05-14T09:30:58.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.374+0000 7fb0571c1700 1 -- 172.21.15.2:0/2881612878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0501a0070 con 0x7fb050107df0 2024-05-14T09:30:58.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.374+0000 7fb04dffb700 1 -- 172.21.15.2:0/2881612878 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03c00fd40 con 0x7fb050107df0 2024-05-14T09:30:58.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.375+0000 7fb04dffb700 1 -- 172.21.15.2:0/2881612878 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00e3b0 con 0x7fb050107df0 2024-05-14T09:30:58.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.375+0000 7fb0571c1700 1 -- 172.21.15.2:0/2881612878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0501a7990 con 0x7fb050107df0 2024-05-14T09:30:58.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.376+0000 7fb0571c1700 1 -- 172.21.15.2:0/2881612878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0500512c0 con 0x7fb050107df0 2024-05-14T09:30:58.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.377+0000 7fb04dffb700 1 -- 172.21.15.2:0/2881612878 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fb03c010610 con 0x7fb050107df0 2024-05-14T09:30:58.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.378+0000 7fb04dffb700 1 --2- 172.21.15.2:0/2881612878 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb0400b2900 0x7fb0400b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:30:58.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.378+0000 7fb054f5d700 1 --2- 172.21.15.2:0/2881612878 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb0400b2900 0x7fb0400b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:30:58.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.378+0000 7fb04dffb700 1 -- 172.21.15.2:0/2881612878 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7fb03c0e1db0 con 0x7fb050107df0 2024-05-14T09:30:58.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.379+0000 7fb054f5d700 1 --2- 172.21.15.2:0/2881612878 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb0400b2900 0x7fb0400b4e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb044009990 tx=0x7fb04401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:30:58.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.382+0000 7fb04dffb700 1 -- 172.21.15.2:0/2881612878 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb03c0a9d00 con 0x7fb050107df0 2024-05-14T09:30:58.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.642+0000 7fb0571c1700 1 -- 172.21.15.2:0/2881612878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb05002e5f0 con 0x7fb050107df0 2024-05-14T09:30:58.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.782+0000 7fb04dffb700 1 -- 172.21.15.2:0/2881612878 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fb03c0a9600 con 0x7fb050107df0 2024-05-14T09:30:58.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.785+0000 7fb0571c1700 1 -- 172.21.15.2:0/2881612878 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb0400b2900 msgr2=0x7fb0400b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:58.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.785+0000 7fb0571c1700 1 --2- 172.21.15.2:0/2881612878 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb0400b2900 0x7fb0400b4e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb044009990 tx=0x7fb04401a040 comp rx=0 tx=0).stop 2024-05-14T09:30:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.785+0000 7fb0571c1700 1 -- 172.21.15.2:0/2881612878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb050107df0 msgr2=0x7fb05019b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:30:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.785+0000 7fb0571c1700 1 --2- 172.21.15.2:0/2881612878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb050107df0 0x7fb05019b450 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00d980 tx=0x7fb03c00dc90 comp rx=0 tx=0).stop 2024-05-14T09:30:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.786+0000 7fb0571c1700 1 -- 172.21.15.2:0/2881612878 shutdown_connections 2024-05-14T09:30:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.786+0000 7fb0571c1700 1 --2- 172.21.15.2:0/2881612878 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb0400b2900 0x7fb0400b4e10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.786+0000 7fb0571c1700 1 --2- 172.21.15.2:0/2881612878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0501034c0 0x7fb05019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.786+0000 7fb0571c1700 1 --2- 172.21.15.2:0/2881612878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb050108740 0x7fb05019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.786+0000 7fb0571c1700 1 --2- 172.21.15.2:0/2881612878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb050107df0 0x7fb05019b450 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:30:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.786+0000 7fb0571c1700 1 -- 172.21.15.2:0/2881612878 >> 172.21.15.2:0/2881612878 conn(0x7fb0500fedf0 msgr2=0x7fb050075dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:30:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.786+0000 7fb0571c1700 1 -- 172.21.15.2:0/2881612878 shutdown_connections 2024-05-14T09:30:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:30:58.786+0000 7fb0571c1700 1 -- 172.21.15.2:0/2881612878 wait complete. 2024-05-14T09:30:58.788 INFO:teuthology.orchestra.run.smithi002.stderr:new fs with metadata pool 2 and data pool 3 2024-05-14T09:30:58.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:58 smithi002 ceph-mon[31557]: osdmap e78: 12 total, 12 up, 12 in 2024-05-14T09:30:58.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:58 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2881612878' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-14T09:30:58.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:58 smithi002 ceph-mon[31557]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-14T09:30:59.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:58 smithi029 ceph-mon[36249]: osdmap e78: 12 total, 12 up, 12 in 2024-05-14T09:30:59.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:58 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2881612878' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-14T09:30:59.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:58 smithi029 ceph-mon[36249]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-14T09:30:59.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:58 smithi026 ceph-mon[36056]: osdmap e78: 12 total, 12 up, 12 in 2024-05-14T09:30:59.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:58 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2881612878' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-14T09:30:59.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:58 smithi026 ceph-mon[36056]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-14T09:30:59.590 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-14T09:30:59.591 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-14T09:30:59.855 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:30:59.892 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:59 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2881612878' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-14T09:30:59.892 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:59 smithi002 ceph-mon[31557]: osdmap e79: 12 total, 12 up, 12 in 2024-05-14T09:30:59.893 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:59 smithi002 ceph-mon[31557]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-14T09:30:59.893 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:59 smithi002 ceph-mon[31557]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-14T09:30:59.893 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:30:59 smithi002 ceph-mon[31557]: pgmap v294: 97 pgs: 11 creating+peering, 12 unknown, 74 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:00.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:59 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2881612878' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-14T09:31:00.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:59 smithi029 ceph-mon[36249]: osdmap e79: 12 total, 12 up, 12 in 2024-05-14T09:31:00.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:59 smithi029 ceph-mon[36249]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-14T09:31:00.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:59 smithi029 ceph-mon[36249]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-14T09:31:00.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:30:59 smithi029 ceph-mon[36249]: pgmap v294: 97 pgs: 11 creating+peering, 12 unknown, 74 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:00.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:59 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2881612878' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-14T09:31:00.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:59 smithi026 ceph-mon[36056]: osdmap e79: 12 total, 12 up, 12 in 2024-05-14T09:31:00.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:59 smithi026 ceph-mon[36056]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-14T09:31:00.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:59 smithi026 ceph-mon[36056]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-14T09:31:00.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:30:59 smithi026 ceph-mon[36056]: pgmap v294: 97 pgs: 11 creating+peering, 12 unknown, 74 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:01.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:00 smithi002 ceph-mon[31557]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-14T09:31:01.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:00 smithi002 ceph-mon[31557]: Cluster is now healthy 2024-05-14T09:31:01.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:00 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] up:active 2024-05-14T09:31:01.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:00 smithi002 ceph-mon[31557]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T09:31:01.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:00 smithi002 ceph-mon[31557]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T09:31:01.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:00 smithi029 ceph-mon[36249]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-14T09:31:01.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:00 smithi029 ceph-mon[36249]: Cluster is now healthy 2024-05-14T09:31:01.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:00 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] up:active 2024-05-14T09:31:01.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:00 smithi029 ceph-mon[36249]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T09:31:01.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:00 smithi029 ceph-mon[36249]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T09:31:01.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:00 smithi026 ceph-mon[36056]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-14T09:31:01.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:00 smithi026 ceph-mon[36056]: Cluster is now healthy 2024-05-14T09:31:01.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:00 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] up:active 2024-05-14T09:31:01.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:00 smithi026 ceph-mon[36056]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T09:31:01.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:00 smithi026 ceph-mon[36056]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T09:31:01.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.571+0000 7f89e6ac5700 1 -- 172.21.15.2:0/3902758369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89e0109850 msgr2=0x7f89e01111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:01.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.571+0000 7f89e6ac5700 1 --2- 172.21.15.2:0/3902758369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89e0109850 0x7f89e01111b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f89d800b0d0 tx=0x7f89d800b3e0 comp rx=0 tx=0).stop 2024-05-14T09:31:01.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.571+0000 7f89e6ac5700 1 -- 172.21.15.2:0/3902758369 shutdown_connections 2024-05-14T09:31:01.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.571+0000 7f89e6ac5700 1 --2- 172.21.15.2:0/3902758369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89e0109850 0x7f89e01111b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:01.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.571+0000 7f89e6ac5700 1 --2- 172.21.15.2:0/3902758369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89e01045d0 0x7f89e01049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:01.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.571+0000 7f89e6ac5700 1 --2- 172.21.15.2:0/3902758369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89e0108f00 0x7f89e0109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:01.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.571+0000 7f89e6ac5700 1 -- 172.21.15.2:0/3902758369 >> 172.21.15.2:0/3902758369 conn(0x7f89e00ffea0 msgr2=0x7f89e01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:01.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.571+0000 7f89e6ac5700 1 -- 172.21.15.2:0/3902758369 shutdown_connections 2024-05-14T09:31:01.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.571+0000 7f89e6ac5700 1 -- 172.21.15.2:0/3902758369 wait complete. 2024-05-14T09:31:01.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.572+0000 7f89e6ac5700 1 Processor -- start 2024-05-14T09:31:01.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.572+0000 7f89e6ac5700 1 -- start start 2024-05-14T09:31:01.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.572+0000 7f89e6ac5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89e01045d0 0x7f89e0076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:01.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.572+0000 7f89e6ac5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89e0108f00 0x7f89e0077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:01.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.572+0000 7f89e6ac5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89e0109850 0x7f89e0077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:01.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.572+0000 7f89e6ac5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f89e0114480 con 0x7f89e01045d0 2024-05-14T09:31:01.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.572+0000 7f89e6ac5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f89e0114300 con 0x7f89e0109850 2024-05-14T09:31:01.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.572+0000 7f89e4861700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89e01045d0 0x7f89e0076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:01.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.572+0000 7f89e6ac5700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f89e0114600 con 0x7f89e0108f00 2024-05-14T09:31:01.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.572+0000 7f89e4861700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89e01045d0 0x7f89e0076bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36754/0 (socket says 172.21.15.2:36754) 2024-05-14T09:31:01.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.572+0000 7f89e4861700 1 -- 172.21.15.2:0/3941032123 learned_addr learned my addr 172.21.15.2:0/3941032123 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:01.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.573+0000 7f89dffff700 1 --2- 172.21.15.2:0/3941032123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89e0108f00 0x7f89e0077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:01.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.573+0000 7f89e5062700 1 --2- 172.21.15.2:0/3941032123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89e0109850 0x7f89e0077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:01.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.573+0000 7f89e4861700 1 -- 172.21.15.2:0/3941032123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89e0108f00 msgr2=0x7f89e0077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:01.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.573+0000 7f89e4861700 1 --2- 172.21.15.2:0/3941032123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89e0108f00 0x7f89e0077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:01.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.573+0000 7f89e4861700 1 -- 172.21.15.2:0/3941032123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89e0109850 msgr2=0x7f89e0077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:01.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.573+0000 7f89e4861700 1 --2- 172.21.15.2:0/3941032123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89e0109850 0x7f89e0077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:01.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.573+0000 7f89e4861700 1 -- 172.21.15.2:0/3941032123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e01a90a0 con 0x7f89e01045d0 2024-05-14T09:31:01.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.573+0000 7f89e4861700 1 --2- 172.21.15.2:0/3941032123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89e01045d0 0x7f89e0076bd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f89d400e9a0 tx=0x7f89d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:01.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.573+0000 7f89ddffb700 1 -- 172.21.15.2:0/3941032123 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d400cbd0 con 0x7f89e01045d0 2024-05-14T09:31:01.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.573+0000 7f89ddffb700 1 -- 172.21.15.2:0/3941032123 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89d400cd70 con 0x7f89e01045d0 2024-05-14T09:31:01.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.573+0000 7f89e6ac5700 1 -- 172.21.15.2:0/3941032123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e01a9390 con 0x7f89e01045d0 2024-05-14T09:31:01.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.573+0000 7f89ddffb700 1 -- 172.21.15.2:0/3941032123 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d4018680 con 0x7f89e01045d0 2024-05-14T09:31:01.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.574+0000 7f89e6ac5700 1 -- 172.21.15.2:0/3941032123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e01a98d0 con 0x7f89e01045d0 2024-05-14T09:31:01.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.575+0000 7f89e6ac5700 1 -- 172.21.15.2:0/3941032123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89e00512c0 con 0x7f89e01045d0 2024-05-14T09:31:01.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.576+0000 7f89ddffb700 1 -- 172.21.15.2:0/3941032123 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f89d4010430 con 0x7f89e01045d0 2024-05-14T09:31:01.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.577+0000 7f89ddffb700 1 --2- 172.21.15.2:0/3941032123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f89d00b2880 0x7f89d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:01.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.577+0000 7f89dffff700 1 --2- 172.21.15.2:0/3941032123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f89d00b2880 0x7f89d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:01.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.577+0000 7f89ddffb700 1 -- 172.21.15.2:0/3941032123 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f89d40e23e0 con 0x7f89e01045d0 2024-05-14T09:31:01.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.578+0000 7f89dffff700 1 --2- 172.21.15.2:0/3941032123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f89d00b2880 0x7f89d00b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f89e0103440 tx=0x7f89cc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:01.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.582+0000 7f89ddffb700 1 -- 172.21.15.2:0/3941032123 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f89d40aa1b0 con 0x7f89e01045d0 2024-05-14T09:31:01.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:01.808+0000 7f89e6ac5700 1 -- 172.21.15.2:0/3941032123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f89e002e840 con 0x7f89e01045d0 2024-05-14T09:31:01.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:01 smithi002 ceph-mon[31557]: pgmap v295: 97 pgs: 11 creating+peering, 86 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 194 B/s wr, 3 op/s 2024-05-14T09:31:02.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:01 smithi029 ceph-mon[36249]: pgmap v295: 97 pgs: 11 creating+peering, 86 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 194 B/s wr, 3 op/s 2024-05-14T09:31:02.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:01 smithi026 ceph-mon[36056]: pgmap v295: 97 pgs: 11 creating+peering, 86 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 194 B/s wr, 3 op/s 2024-05-14T09:31:02.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:02.808+0000 7f89ddffb700 1 -- 172.21.15.2:0/3941032123 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) ==== 157+0+0 (secure 0 0 0) 0x7f89d40a9ab0 con 0x7f89e01045d0 2024-05-14T09:31:02.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:02.812+0000 7f89e6ac5700 1 -- 172.21.15.2:0/3941032123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f89d00b2880 msgr2=0x7f89d00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:02.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:02.812+0000 7f89e6ac5700 1 --2- 172.21.15.2:0/3941032123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f89d00b2880 0x7f89d00b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f89e0103440 tx=0x7f89cc00a380 comp rx=0 tx=0).stop 2024-05-14T09:31:02.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:02.812+0000 7f89e6ac5700 1 -- 172.21.15.2:0/3941032123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89e01045d0 msgr2=0x7f89e0076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:02.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:02.812+0000 7f89e6ac5700 1 --2- 172.21.15.2:0/3941032123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89e01045d0 0x7f89e0076bd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f89d400e9a0 tx=0x7f89d400ecb0 comp rx=0 tx=0).stop 2024-05-14T09:31:02.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:02.812+0000 7f89e6ac5700 1 -- 172.21.15.2:0/3941032123 shutdown_connections 2024-05-14T09:31:02.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:02.812+0000 7f89e6ac5700 1 --2- 172.21.15.2:0/3941032123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f89d00b2880 0x7f89d00b4d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:02.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:02.812+0000 7f89e6ac5700 1 --2- 172.21.15.2:0/3941032123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89e0109850 0x7f89e0077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:02.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:02.812+0000 7f89e6ac5700 1 --2- 172.21.15.2:0/3941032123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89e01045d0 0x7f89e0076bd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:02.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:02.812+0000 7f89e6ac5700 1 --2- 172.21.15.2:0/3941032123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f89e0108f00 0x7f89e0077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:02.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:02.812+0000 7f89e6ac5700 1 -- 172.21.15.2:0/3941032123 >> 172.21.15.2:0/3941032123 conn(0x7f89e00ffea0 msgr2=0x7f89e0108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:02.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:02.812+0000 7f89e6ac5700 1 -- 172.21.15.2:0/3941032123 shutdown_connections 2024-05-14T09:31:02.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:02.812+0000 7f89e6ac5700 1 -- 172.21.15.2:0/3941032123 wait complete. 2024-05-14T09:31:03.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:02 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3941032123' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-14T09:31:03.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:02 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3941032123' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-14T09:31:03.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:02 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3941032123' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-14T09:31:03.616 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-14T09:31:03.875 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:03.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:03 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3941032123' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-14T09:31:03.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:03 smithi002 ceph-mon[31557]: osdmap e80: 12 total, 12 up, 12 in 2024-05-14T09:31:03.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:03 smithi002 ceph-mon[31557]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-05-14T09:31:04.107 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:03 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3941032123' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-14T09:31:04.107 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:03 smithi026 ceph-mon[36056]: osdmap e80: 12 total, 12 up, 12 in 2024-05-14T09:31:04.107 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:03 smithi026 ceph-mon[36056]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-05-14T09:31:04.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:03 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3941032123' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-14T09:31:04.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:03 smithi029 ceph-mon[36249]: osdmap e80: 12 total, 12 up, 12 in 2024-05-14T09:31:04.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:03 smithi029 ceph-mon[36249]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-05-14T09:31:04.451 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 09:31:04 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b[66741]: 2024-05-14T09:31:04.106+0000 7f081c936700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T09:31:05.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.686+0000 7fe0d6187700 1 -- 172.21.15.2:0/665450662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0d01045d0 msgr2=0x7fe0d01049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:05.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.686+0000 7fe0d6187700 1 --2- 172.21.15.2:0/665450662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0d01045d0 0x7fe0d01049e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000b0d0 tx=0x7fe0c000b3e0 comp rx=0 tx=0).stop 2024-05-14T09:31:05.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.686+0000 7fe0d6187700 1 -- 172.21.15.2:0/665450662 shutdown_connections 2024-05-14T09:31:05.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.686+0000 7fe0d6187700 1 --2- 172.21.15.2:0/665450662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0d0109850 0x7fe0d01111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:05.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.686+0000 7fe0d6187700 1 --2- 172.21.15.2:0/665450662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0d01045d0 0x7fe0d01049e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:05.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.686+0000 7fe0d6187700 1 --2- 172.21.15.2:0/665450662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0d0108f00 0x7fe0d0109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:05.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.686+0000 7fe0d6187700 1 -- 172.21.15.2:0/665450662 >> 172.21.15.2:0/665450662 conn(0x7fe0d00ffea0 msgr2=0x7fe0d01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:05.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.686+0000 7fe0d6187700 1 -- 172.21.15.2:0/665450662 shutdown_connections 2024-05-14T09:31:05.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.686+0000 7fe0d6187700 1 -- 172.21.15.2:0/665450662 wait complete. 2024-05-14T09:31:05.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.686+0000 7fe0d6187700 1 Processor -- start 2024-05-14T09:31:05.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.687+0000 7fe0d6187700 1 -- start start 2024-05-14T09:31:05.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.687+0000 7fe0d6187700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0d01045d0 0x7fe0d019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:05.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.687+0000 7fe0d6187700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0d0108f00 0x7fe0d019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:05.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.687+0000 7fe0d6187700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0d0109850 0x7fe0d019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:05.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.687+0000 7fe0d6187700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d0114480 con 0x7fe0d0109850 2024-05-14T09:31:05.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.687+0000 7fe0d6187700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d0114300 con 0x7fe0d01045d0 2024-05-14T09:31:05.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.687+0000 7fe0d6187700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d0114600 con 0x7fe0d0108f00 2024-05-14T09:31:05.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.687+0000 7fe0cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0d0109850 0x7fe0d019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:05.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.687+0000 7fe0cf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0d01045d0 0x7fe0d019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:05.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.687+0000 7fe0cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0d0109850 0x7fe0d019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36774/0 (socket says 172.21.15.2:36774) 2024-05-14T09:31:05.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.687+0000 7fe0ceffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0d0108f00 0x7fe0d019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:05.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.687+0000 7fe0cffff700 1 -- 172.21.15.2:0/2763668969 learned_addr learned my addr 172.21.15.2:0/2763668969 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:05.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.687+0000 7fe0cf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0d01045d0 0x7fe0d019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:40746/0 (socket says 172.21.15.2:40746) 2024-05-14T09:31:05.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.688+0000 7fe0cffff700 1 -- 172.21.15.2:0/2763668969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0d0108f00 msgr2=0x7fe0d019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:05.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.688+0000 7fe0cffff700 1 --2- 172.21.15.2:0/2763668969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0d0108f00 0x7fe0d019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:05.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.688+0000 7fe0cffff700 1 -- 172.21.15.2:0/2763668969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0d01045d0 msgr2=0x7fe0d019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:05.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.688+0000 7fe0cffff700 1 --2- 172.21.15.2:0/2763668969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0d01045d0 0x7fe0d019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:05.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.688+0000 7fe0cffff700 1 -- 172.21.15.2:0/2763668969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d019fc50 con 0x7fe0d0109850 2024-05-14T09:31:05.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.688+0000 7fe0cf7fe700 1 --2- 172.21.15.2:0/2763668969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0d01045d0 0x7fe0d019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:31:05.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.688+0000 7fe0cffff700 1 --2- 172.21.15.2:0/2763668969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0d0109850 0x7fe0d019f660 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800c900 tx=0x7fe0c800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:05.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.688+0000 7fe0ccff9700 1 -- 172.21.15.2:0/2763668969 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80078d0 con 0x7fe0d0109850 2024-05-14T09:31:05.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.688+0000 7fe0d6187700 1 -- 172.21.15.2:0/2763668969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d019ff40 con 0x7fe0d0109850 2024-05-14T09:31:05.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.688+0000 7fe0ccff9700 1 -- 172.21.15.2:0/2763668969 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0c800f450 con 0x7fe0d0109850 2024-05-14T09:31:05.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.688+0000 7fe0ccff9700 1 -- 172.21.15.2:0/2763668969 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8018620 con 0x7fe0d0109850 2024-05-14T09:31:05.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.688+0000 7fe0d6187700 1 -- 172.21.15.2:0/2763668969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d01a0480 con 0x7fe0d0109850 2024-05-14T09:31:05.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.689+0000 7fe0d6187700 1 -- 172.21.15.2:0/2763668969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d00512c0 con 0x7fe0d0109850 2024-05-14T09:31:05.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.691+0000 7fe0ccff9700 1 -- 172.21.15.2:0/2763668969 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fe0c8007a70 con 0x7fe0d0109850 2024-05-14T09:31:05.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.692+0000 7fe0ccff9700 1 --2- 172.21.15.2:0/2763668969 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0b80b2880 0x7fe0b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:05.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.692+0000 7fe0cf7fe700 1 --2- 172.21.15.2:0/2763668969 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0b80b2880 0x7fe0b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:05.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.692+0000 7fe0ccff9700 1 -- 172.21.15.2:0/2763668969 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8277+0+0 (secure 0 0 0) 0x7fe0c80e2100 con 0x7fe0d0109850 2024-05-14T09:31:05.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.692+0000 7fe0cf7fe700 1 --2- 172.21.15.2:0/2763668969 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0b80b2880 0x7fe0b80b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009900 tx=0x7fe0c000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:05.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.695+0000 7fe0ccff9700 1 -- 172.21.15.2:0/2763668969 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe0c8014030 con 0x7fe0d0109850 2024-05-14T09:31:05.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:05.910+0000 7fe0d6187700 1 -- 172.21.15.2:0/2763668969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe0d002e5f0 con 0x7fe0d0109850 2024-05-14T09:31:06.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:06 smithi002 ceph-mon[31557]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-14T09:31:06.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:06 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2763668969' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T09:31:06.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:06 smithi029 ceph-mon[36249]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-14T09:31:06.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:06 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2763668969' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T09:31:06.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:06 smithi026 ceph-mon[36056]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-14T09:31:06.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:06 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2763668969' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T09:31:07.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:07.093+0000 7fe0ccff9700 1 -- 172.21.15.2:0/2763668969 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) ==== 228+0+0 (secure 0 0 0) 0x7fe0c80b0e20 con 0x7fe0d0109850 2024-05-14T09:31:07.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:07.097+0000 7fe0d6187700 1 -- 172.21.15.2:0/2763668969 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0b80b2880 msgr2=0x7fe0b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:07.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:07.097+0000 7fe0d6187700 1 --2- 172.21.15.2:0/2763668969 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0b80b2880 0x7fe0b80b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009900 tx=0x7fe0c000b040 comp rx=0 tx=0).stop 2024-05-14T09:31:07.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:07.097+0000 7fe0d6187700 1 -- 172.21.15.2:0/2763668969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0d0109850 msgr2=0x7fe0d019f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:07.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:07.097+0000 7fe0d6187700 1 --2- 172.21.15.2:0/2763668969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0d0109850 0x7fe0d019f660 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800c900 tx=0x7fe0c800cc10 comp rx=0 tx=0).stop 2024-05-14T09:31:07.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:07.097+0000 7fe0d6187700 1 -- 172.21.15.2:0/2763668969 shutdown_connections 2024-05-14T09:31:07.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:07.097+0000 7fe0d6187700 1 --2- 172.21.15.2:0/2763668969 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0b80b2880 0x7fe0b80b4d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:07.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:07.097+0000 7fe0d6187700 1 --2- 172.21.15.2:0/2763668969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0d01045d0 0x7fe0d019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:07.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:07.097+0000 7fe0d6187700 1 --2- 172.21.15.2:0/2763668969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0d0109850 0x7fe0d019f660 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:07.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:07.097+0000 7fe0d6187700 1 --2- 172.21.15.2:0/2763668969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0d0108f00 0x7fe0d019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:07.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:07.097+0000 7fe0d6187700 1 -- 172.21.15.2:0/2763668969 >> 172.21.15.2:0/2763668969 conn(0x7fe0d00ffea0 msgr2=0x7fe0d0102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:07.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:07.097+0000 7fe0d6187700 1 -- 172.21.15.2:0/2763668969 shutdown_connections 2024-05-14T09:31:07.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:07.097+0000 7fe0d6187700 1 -- 172.21.15.2:0/2763668969 wait complete. 2024-05-14T09:31:07.101 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data_ec' created 2024-05-14T09:31:07.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:07 smithi002 ceph-mon[31557]: osdmap e81: 12 total, 12 up, 12 in 2024-05-14T09:31:07.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:07 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2763668969' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-14T09:31:07.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:07 smithi029 ceph-mon[36249]: osdmap e81: 12 total, 12 up, 12 in 2024-05-14T09:31:07.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:07 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2763668969' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-14T09:31:07.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:07 smithi026 ceph-mon[36056]: osdmap e81: 12 total, 12 up, 12 in 2024-05-14T09:31:07.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:07 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2763668969' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-14T09:31:07.774 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-14T09:31:08.032 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:08.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:08 smithi002 ceph-mon[31557]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-14T09:31:08.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:08 smithi002 ceph-mon[31557]: osdmap e82: 12 total, 12 up, 12 in 2024-05-14T09:31:08.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:08 smithi002 ceph-mon[31557]: osdmap e83: 12 total, 12 up, 12 in 2024-05-14T09:31:08.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:08 smithi029 ceph-mon[36249]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-14T09:31:08.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:08 smithi029 ceph-mon[36249]: osdmap e82: 12 total, 12 up, 12 in 2024-05-14T09:31:08.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:08 smithi029 ceph-mon[36249]: osdmap e83: 12 total, 12 up, 12 in 2024-05-14T09:31:08.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:08 smithi026 ceph-mon[36056]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-14T09:31:08.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:08 smithi026 ceph-mon[36056]: osdmap e82: 12 total, 12 up, 12 in 2024-05-14T09:31:08.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:08 smithi026 ceph-mon[36056]: osdmap e83: 12 total, 12 up, 12 in 2024-05-14T09:31:09.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.529+0000 7f27668cb700 1 -- 172.21.15.2:0/2515559805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27601034e0 msgr2=0x7f27601038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:09.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.529+0000 7f27668cb700 1 --2- 172.21.15.2:0/2515559805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27601034e0 0x7f27601038f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f27500099c0 tx=0x7f2750009cd0 comp rx=0 tx=0).stop 2024-05-14T09:31:09.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.529+0000 7f27668cb700 1 -- 172.21.15.2:0/2515559805 shutdown_connections 2024-05-14T09:31:09.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.529+0000 7f27668cb700 1 --2- 172.21.15.2:0/2515559805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2760108760 0x7f27601100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:09.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.529+0000 7f27668cb700 1 --2- 172.21.15.2:0/2515559805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27601034e0 0x7f27601038f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:09.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.529+0000 7f27668cb700 1 --2- 172.21.15.2:0/2515559805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2760107e10 0x7f2760108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:09.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.529+0000 7f27668cb700 1 -- 172.21.15.2:0/2515559805 >> 172.21.15.2:0/2515559805 conn(0x7f27600fef30 msgr2=0x7f2760101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:09.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.529+0000 7f27668cb700 1 -- 172.21.15.2:0/2515559805 shutdown_connections 2024-05-14T09:31:09.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.529+0000 7f27668cb700 1 -- 172.21.15.2:0/2515559805 wait complete. 2024-05-14T09:31:09.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.530+0000 7f27668cb700 1 Processor -- start 2024-05-14T09:31:09.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.530+0000 7f27668cb700 1 -- start start 2024-05-14T09:31:09.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.530+0000 7f27668cb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27601034e0 0x7f2760199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:09.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.530+0000 7f27668cb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2760107e10 0x7f276019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:09.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.530+0000 7f27668cb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2760108760 0x7f276019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:09.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.530+0000 7f27668cb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2760113400 con 0x7f2760107e10 2024-05-14T09:31:09.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.530+0000 7f27668cb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2760113280 con 0x7f27601034e0 2024-05-14T09:31:09.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.530+0000 7f27668cb700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2760113580 con 0x7f2760108760 2024-05-14T09:31:09.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.530+0000 7f2757fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2760107e10 0x7f276019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:09.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.530+0000 7f2757fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2760107e10 0x7f276019a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41062/0 (socket says 172.21.15.2:41062) 2024-05-14T09:31:09.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.530+0000 7f2757fff700 1 -- 172.21.15.2:0/3058592301 learned_addr learned my addr 172.21.15.2:0/3058592301 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:09.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.530+0000 7f2764e68700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2760108760 0x7f276019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:09.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.531+0000 7f275ffff700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27601034e0 0x7f2760199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:09.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.531+0000 7f2764e68700 1 -- 172.21.15.2:0/3058592301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27601034e0 msgr2=0x7f2760199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:09.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.531+0000 7f2764e68700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27601034e0 0x7f2760199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:09.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.531+0000 7f2764e68700 1 -- 172.21.15.2:0/3058592301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2760107e10 msgr2=0x7f276019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:09.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.531+0000 7f2764e68700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2760107e10 0x7f276019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:09.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.531+0000 7f2764e68700 1 -- 172.21.15.2:0/3058592301 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27601a9210 con 0x7f2760108760 2024-05-14T09:31:09.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.531+0000 7f2757fff700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2760107e10 0x7f276019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:31:09.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.531+0000 7f275ffff700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27601034e0 0x7f2760199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:31:09.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.531+0000 7f2764e68700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2760108760 0x7f276019a590 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f275800efc0 tx=0x7f275800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:09.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.531+0000 7f275dffb700 1 -- 172.21.15.2:0/3058592301 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27580092e0 con 0x7f2760108760 2024-05-14T09:31:09.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.531+0000 7f275dffb700 1 -- 172.21.15.2:0/3058592301 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2758004bb0 con 0x7f2760108760 2024-05-14T09:31:09.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.531+0000 7f27668cb700 1 -- 172.21.15.2:0/3058592301 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27601a94a0 con 0x7f2760108760 2024-05-14T09:31:09.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.531+0000 7f275dffb700 1 -- 172.21.15.2:0/3058592301 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2758004080 con 0x7f2760108760 2024-05-14T09:31:09.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.531+0000 7f27668cb700 1 -- 172.21.15.2:0/3058592301 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27601a9990 con 0x7f2760108760 2024-05-14T09:31:09.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.532+0000 7f27668cb700 1 -- 172.21.15.2:0/3058592301 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2760104990 con 0x7f2760108760 2024-05-14T09:31:09.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.534+0000 7f275dffb700 1 -- 172.21.15.2:0/3058592301 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f275801b030 con 0x7f2760108760 2024-05-14T09:31:09.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.535+0000 7f275dffb700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f274c0b2880 0x7f274c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:09.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.535+0000 7f275ffff700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f274c0b2880 0x7f274c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:09.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.535+0000 7f275dffb700 1 -- 172.21.15.2:0/3058592301 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f27580e29c0 con 0x7f2760108760 2024-05-14T09:31:09.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.535+0000 7f275ffff700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f274c0b2880 0x7f274c0b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2750003760 tx=0x7f275001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:09.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.538+0000 7f275dffb700 1 -- 172.21.15.2:0/3058592301 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2758022020 con 0x7f2760108760 2024-05-14T09:31:09.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:09.754+0000 7f27668cb700 1 -- 172.21.15.2:0/3058592301 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f276002e5f0 con 0x7f2760108760 2024-05-14T09:31:10.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:10.117+0000 7f275dffb700 1 -- 172.21.15.2:0/3058592301 <== mon.1 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7f2758010040 con 0x7f2760108760 2024-05-14T09:31:10.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:10.119+0000 7f27668cb700 1 -- 172.21.15.2:0/3058592301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f274c0b2880 msgr2=0x7f274c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:10.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:10.119+0000 7f27668cb700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f274c0b2880 0x7f274c0b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2750003760 tx=0x7f275001a040 comp rx=0 tx=0).stop 2024-05-14T09:31:10.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:10.119+0000 7f27668cb700 1 -- 172.21.15.2:0/3058592301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2760108760 msgr2=0x7f276019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:10.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:10.119+0000 7f27668cb700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2760108760 0x7f276019a590 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f275800efc0 tx=0x7f275800c3a0 comp rx=0 tx=0).stop 2024-05-14T09:31:10.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:10.119+0000 7f27668cb700 1 -- 172.21.15.2:0/3058592301 shutdown_connections 2024-05-14T09:31:10.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:10.119+0000 7f27668cb700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f274c0b2880 0x7f274c0b4d90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:10.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:10.119+0000 7f27668cb700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27601034e0 0x7f2760199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:10.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:10.119+0000 7f27668cb700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2760108760 0x7f276019a590 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:10.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:10.119+0000 7f27668cb700 1 --2- 172.21.15.2:0/3058592301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2760107e10 0x7f276019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:10.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:10.119+0000 7f27668cb700 1 -- 172.21.15.2:0/3058592301 >> 172.21.15.2:0/3058592301 conn(0x7f27600fef30 msgr2=0x7f27601096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:10.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:10.119+0000 7f27668cb700 1 -- 172.21.15.2:0/3058592301 shutdown_connections 2024-05-14T09:31:10.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:10.119+0000 7f27668cb700 1 -- 172.21.15.2:0/3058592301 wait complete. 2024-05-14T09:31:10.120 INFO:teuthology.orchestra.run.smithi002.stderr:set pool 4 allow_ec_overwrites to true 2024-05-14T09:31:10.356 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:10 smithi002 ceph-mon[31557]: pgmap v303: 129 pgs: 12 creating+peering, 20 unknown, 97 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:10.356 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:10 smithi002 ceph-mon[31557]: osdmap e84: 12 total, 12 up, 12 in 2024-05-14T09:31:10.356 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:10 smithi002 ceph-mon[31557]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T09:31:10.356 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:10 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3058592301' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-14T09:31:10.356 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:10 smithi002 ceph-mon[31557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-14T09:31:10.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:10 smithi029 ceph-mon[36249]: pgmap v303: 129 pgs: 12 creating+peering, 20 unknown, 97 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:10.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:10 smithi029 ceph-mon[36249]: osdmap e84: 12 total, 12 up, 12 in 2024-05-14T09:31:10.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:10 smithi029 ceph-mon[36249]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T09:31:10.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:10 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3058592301' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-14T09:31:10.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:10 smithi029 ceph-mon[36249]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-14T09:31:10.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:10 smithi026 ceph-mon[36056]: pgmap v303: 129 pgs: 12 creating+peering, 20 unknown, 97 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:10.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:10 smithi026 ceph-mon[36056]: osdmap e84: 12 total, 12 up, 12 in 2024-05-14T09:31:10.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:10 smithi026 ceph-mon[36056]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T09:31:10.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:10 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3058592301' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-14T09:31:10.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:10 smithi026 ceph-mon[36056]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-14T09:31:10.891 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-14T09:31:11.155 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:11.386 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:11 smithi002 ceph-mon[31557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-14T09:31:11.387 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:11 smithi002 ceph-mon[31557]: osdmap e85: 12 total, 12 up, 12 in 2024-05-14T09:31:11.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:11 smithi029 ceph-mon[36249]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-14T09:31:11.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:11 smithi029 ceph-mon[36249]: osdmap e85: 12 total, 12 up, 12 in 2024-05-14T09:31:11.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:11 smithi026 ceph-mon[36056]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-14T09:31:11.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:11 smithi026 ceph-mon[36056]: osdmap e85: 12 total, 12 up, 12 in 2024-05-14T09:31:12.339 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:12 smithi002 ceph-mon[31557]: pgmap v306: 129 pgs: 12 creating+peering, 17 unknown, 100 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:12 smithi029 ceph-mon[36249]: pgmap v306: 129 pgs: 12 creating+peering, 17 unknown, 100 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:12.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:12 smithi026 ceph-mon[36056]: pgmap v306: 129 pgs: 12 creating+peering, 17 unknown, 100 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.892+0000 7f8c1c33a700 1 -- 172.21.15.2:0/2339385722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c141034e0 msgr2=0x7f8c141038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.892+0000 7f8c1c33a700 1 --2- 172.21.15.2:0/2339385722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c141034e0 0x7f8c141038f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8c100099c0 tx=0x7f8c10009cd0 comp rx=0 tx=0).stop 2024-05-14T09:31:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.892+0000 7f8c1c33a700 1 -- 172.21.15.2:0/2339385722 shutdown_connections 2024-05-14T09:31:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.892+0000 7f8c1c33a700 1 --2- 172.21.15.2:0/2339385722 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c14108760 0x7f8c141100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.892+0000 7f8c1c33a700 1 --2- 172.21.15.2:0/2339385722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c141034e0 0x7f8c141038f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.892+0000 7f8c1c33a700 1 --2- 172.21.15.2:0/2339385722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c14107e10 0x7f8c14108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.892+0000 7f8c1c33a700 1 -- 172.21.15.2:0/2339385722 >> 172.21.15.2:0/2339385722 conn(0x7f8c140fef30 msgr2=0x7f8c14101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.892+0000 7f8c1c33a700 1 -- 172.21.15.2:0/2339385722 shutdown_connections 2024-05-14T09:31:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.892+0000 7f8c1c33a700 1 -- 172.21.15.2:0/2339385722 wait complete. 2024-05-14T09:31:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.893+0000 7f8c1c33a700 1 Processor -- start 2024-05-14T09:31:12.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.893+0000 7f8c1c33a700 1 -- start start 2024-05-14T09:31:12.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.893+0000 7f8c1c33a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c141034e0 0x7f8c1419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:12.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.893+0000 7f8c1c33a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c14107e10 0x7f8c1419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:12.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.893+0000 7f8c1c33a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c14108760 0x7f8c1419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:12.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.893+0000 7f8c1c33a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8c141133e0 con 0x7f8c14107e10 2024-05-14T09:31:12.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.893+0000 7f8c1c33a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8c14113260 con 0x7f8c141034e0 2024-05-14T09:31:12.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.893+0000 7f8c1c33a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8c14113560 con 0x7f8c14108760 2024-05-14T09:31:12.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c1a0d6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c141034e0 0x7f8c1419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:12.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c198d5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c14107e10 0x7f8c1419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:12.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c198d5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c14107e10 0x7f8c1419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41088/0 (socket says 172.21.15.2:41088) 2024-05-14T09:31:12.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c198d5700 1 -- 172.21.15.2:0/3648703110 learned_addr learned my addr 172.21.15.2:0/3648703110 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:12.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c1a0d6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c141034e0 0x7f8c1419ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:34230/0 (socket says 172.21.15.2:34230) 2024-05-14T09:31:12.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c1a8d7700 1 --2- 172.21.15.2:0/3648703110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c14108760 0x7f8c1419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:12.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c198d5700 1 -- 172.21.15.2:0/3648703110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c14108760 msgr2=0x7f8c1419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:12.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c198d5700 1 --2- 172.21.15.2:0/3648703110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c14108760 0x7f8c1419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:12.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c198d5700 1 -- 172.21.15.2:0/3648703110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c141034e0 msgr2=0x7f8c1419ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:12.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c198d5700 1 --2- 172.21.15.2:0/3648703110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c141034e0 0x7f8c1419ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:12.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c198d5700 1 -- 172.21.15.2:0/3648703110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1419fd10 con 0x7f8c14107e10 2024-05-14T09:31:12.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c1a8d7700 1 --2- 172.21.15.2:0/3648703110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c14108760 0x7f8c1419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:31:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c198d5700 1 --2- 172.21.15.2:0/3648703110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c14107e10 0x7f8c1419b3e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400d980 tx=0x7f8c0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c0b7fe700 1 -- 172.21.15.2:0/3648703110 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04017070 con 0x7f8c14107e10 2024-05-14T09:31:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c1c33a700 1 -- 172.21.15.2:0/3648703110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c141a0000 con 0x7f8c14107e10 2024-05-14T09:31:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.894+0000 7f8c0b7fe700 1 -- 172.21.15.2:0/3648703110 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c04009de0 con 0x7f8c14107e10 2024-05-14T09:31:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.895+0000 7f8c0b7fe700 1 -- 172.21.15.2:0/3648703110 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0400f3c0 con 0x7f8c14107e10 2024-05-14T09:31:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.895+0000 7f8c1c33a700 1 -- 172.21.15.2:0/3648703110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c141a7920 con 0x7f8c14107e10 2024-05-14T09:31:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.896+0000 7f8c1c33a700 1 -- 172.21.15.2:0/3648703110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c140512c0 con 0x7f8c14107e10 2024-05-14T09:31:12.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.899+0000 7f8c0b7fe700 1 -- 172.21.15.2:0/3648703110 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f8c04009930 con 0x7f8c14107e10 2024-05-14T09:31:12.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.899+0000 7f8c0b7fe700 1 --2- 172.21.15.2:0/3648703110 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8c000b2880 0x7f8c000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:12.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.899+0000 7f8c1a0d6700 1 --2- 172.21.15.2:0/3648703110 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8c000b2880 0x7f8c000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:12.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.899+0000 7f8c0b7fe700 1 -- 172.21.15.2:0/3648703110 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7f8c040e2210 con 0x7f8c14107e10 2024-05-14T09:31:12.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.900+0000 7f8c1a0d6700 1 --2- 172.21.15.2:0/3648703110 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8c000b2880 0x7f8c000b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000b580 tx=0x7f8c100036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:12.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:12.903+0000 7f8c0b7fe700 1 -- 172.21.15.2:0/3648703110 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8c040a9db0 con 0x7f8c14107e10 2024-05-14T09:31:13.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:13.171+0000 7f8c1c33a700 1 -- 172.21.15.2:0/3648703110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f8c1402e5f0 con 0x7f8c14107e10 2024-05-14T09:31:14.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:14.131+0000 7f8c0b7fe700 1 -- 172.21.15.2:0/3648703110 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f8c040a96b0 con 0x7f8c14107e10 2024-05-14T09:31:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:14.134+0000 7f8c1c33a700 1 -- 172.21.15.2:0/3648703110 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8c000b2880 msgr2=0x7f8c000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:14.134+0000 7f8c1c33a700 1 --2- 172.21.15.2:0/3648703110 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8c000b2880 0x7f8c000b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000b580 tx=0x7f8c100036e0 comp rx=0 tx=0).stop 2024-05-14T09:31:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:14.134+0000 7f8c1c33a700 1 -- 172.21.15.2:0/3648703110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c14107e10 msgr2=0x7f8c1419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:14.134+0000 7f8c1c33a700 1 --2- 172.21.15.2:0/3648703110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c14107e10 0x7f8c1419b3e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400d980 tx=0x7f8c0400dc90 comp rx=0 tx=0).stop 2024-05-14T09:31:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:14.134+0000 7f8c1c33a700 1 -- 172.21.15.2:0/3648703110 shutdown_connections 2024-05-14T09:31:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:14.134+0000 7f8c1c33a700 1 --2- 172.21.15.2:0/3648703110 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8c000b2880 0x7f8c000b4d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:14.134+0000 7f8c1c33a700 1 --2- 172.21.15.2:0/3648703110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c141034e0 0x7f8c1419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:14.134+0000 7f8c1c33a700 1 --2- 172.21.15.2:0/3648703110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c14108760 0x7f8c1419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:14.134+0000 7f8c1c33a700 1 --2- 172.21.15.2:0/3648703110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c14107e10 0x7f8c1419b3e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:14.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:14.134+0000 7f8c1c33a700 1 -- 172.21.15.2:0/3648703110 >> 172.21.15.2:0/3648703110 conn(0x7f8c140fef30 msgr2=0x7f8c1406c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:14.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:14.134+0000 7f8c1c33a700 1 -- 172.21.15.2:0/3648703110 shutdown_connections 2024-05-14T09:31:14.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:14.134+0000 7f8c1c33a700 1 -- 172.21.15.2:0/3648703110 wait complete. 2024-05-14T09:31:14.135 INFO:teuthology.orchestra.run.smithi002.stderr:added data pool 4 to fsmap 2024-05-14T09:31:14.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:14 smithi002 ceph-mon[31557]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:14.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:14 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3648703110' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-14T09:31:14.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:14 smithi029 ceph-mon[36249]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:14.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:14 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3648703110' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-14T09:31:14.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:14 smithi026 ceph-mon[36056]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:14.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:14 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3648703110' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-14T09:31:15.024 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:31:15.286 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:15.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:15 smithi002 ceph-mon[31557]: osdmap e86: 12 total, 12 up, 12 in 2024-05-14T09:31:15.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:15 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3648703110' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-14T09:31:15.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:15 smithi002 ceph-mon[31557]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T09:31:15.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:15 smithi029 ceph-mon[36249]: osdmap e86: 12 total, 12 up, 12 in 2024-05-14T09:31:15.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:15 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3648703110' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-14T09:31:15.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:15 smithi029 ceph-mon[36249]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T09:31:15.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:15 smithi026 ceph-mon[36056]: osdmap e86: 12 total, 12 up, 12 in 2024-05-14T09:31:15.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:15 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3648703110' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-14T09:31:15.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:15 smithi026 ceph-mon[36056]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T09:31:16.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:16 smithi002 ceph-mon[31557]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:16.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:16 smithi002 ceph-mon[31557]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-14T09:31:16.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:16 smithi002 ceph-mon[31557]: Cluster is now healthy 2024-05-14T09:31:16.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:16 smithi029 ceph-mon[36249]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:16.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:16 smithi029 ceph-mon[36249]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-14T09:31:16.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:16 smithi029 ceph-mon[36249]: Cluster is now healthy 2024-05-14T09:31:16.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:16 smithi026 ceph-mon[36056]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:16.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:16 smithi026 ceph-mon[36056]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-14T09:31:16.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:16 smithi026 ceph-mon[36056]: Cluster is now healthy 2024-05-14T09:31:17.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.040+0000 7fa430041700 1 -- 172.21.15.2:0/482705584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa428108f60 msgr2=0x7fa428109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:17.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.040+0000 7fa430041700 1 --2- 172.21.15.2:0/482705584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa428108f60 0x7fa428109370 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009990 tx=0x7fa41c009ca0 comp rx=0 tx=0).stop 2024-05-14T09:31:17.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.040+0000 7fa430041700 1 -- 172.21.15.2:0/482705584 shutdown_connections 2024-05-14T09:31:17.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.040+0000 7fa430041700 1 --2- 172.21.15.2:0/482705584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa428102b40 0x7fa428102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:17.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.040+0000 7fa430041700 1 --2- 172.21.15.2:0/482705584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa4281098b0 0x7fa4281111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:17.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.040+0000 7fa430041700 1 --2- 172.21.15.2:0/482705584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa428108f60 0x7fa428109370 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:17.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.040+0000 7fa430041700 1 -- 172.21.15.2:0/482705584 >> 172.21.15.2:0/482705584 conn(0x7fa4280fe540 msgr2=0x7fa428100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:17.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.040+0000 7fa430041700 1 -- 172.21.15.2:0/482705584 shutdown_connections 2024-05-14T09:31:17.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.040+0000 7fa430041700 1 -- 172.21.15.2:0/482705584 wait complete. 2024-05-14T09:31:17.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.041+0000 7fa430041700 1 Processor -- start 2024-05-14T09:31:17.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.041+0000 7fa430041700 1 -- start start 2024-05-14T09:31:17.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.041+0000 7fa430041700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa428102b40 0x7fa42819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:17.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.041+0000 7fa430041700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa428108f60 0x7fa42819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:17.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.041+0000 7fa430041700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4281098b0 0x7fa42819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:17.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.041+0000 7fa430041700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa428114470 con 0x7fa428102b40 2024-05-14T09:31:17.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.041+0000 7fa430041700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281142f0 con 0x7fa4281098b0 2024-05-14T09:31:17.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.041+0000 7fa430041700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281145f0 con 0x7fa428108f60 2024-05-14T09:31:17.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.041+0000 7fa42dddd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa428102b40 0x7fa42819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.041+0000 7fa42dddd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa428102b40 0x7fa42819ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41108/0 (socket says 172.21.15.2:41108) 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.041+0000 7fa42dddd700 1 -- 172.21.15.2:0/1600277262 learned_addr learned my addr 172.21.15.2:0/1600277262 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.042+0000 7fa42d5dc700 1 --2- 172.21.15.2:0/1600277262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa428108f60 0x7fa42819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.042+0000 7fa42e5de700 1 --2- 172.21.15.2:0/1600277262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4281098b0 0x7fa42819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.042+0000 7fa42dddd700 1 -- 172.21.15.2:0/1600277262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa428108f60 msgr2=0x7fa42819b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.042+0000 7fa42dddd700 1 --2- 172.21.15.2:0/1600277262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa428108f60 0x7fa42819b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.042+0000 7fa42dddd700 1 -- 172.21.15.2:0/1600277262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4281098b0 msgr2=0x7fa42819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.042+0000 7fa42dddd700 1 --2- 172.21.15.2:0/1600277262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4281098b0 0x7fa42819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.042+0000 7fa42dddd700 1 -- 172.21.15.2:0/1600277262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42819fc60 con 0x7fa428102b40 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.042+0000 7fa42d5dc700 1 --2- 172.21.15.2:0/1600277262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa428108f60 0x7fa42819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.042+0000 7fa42dddd700 1 --2- 172.21.15.2:0/1600277262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa428102b40 0x7fa42819ae40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa42400c8f0 tx=0x7fa42400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.042+0000 7fa41affd700 1 -- 172.21.15.2:0/1600277262 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424007b00 con 0x7fa428102b40 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.042+0000 7fa430041700 1 -- 172.21.15.2:0/1600277262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42819ff50 con 0x7fa428102b40 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.042+0000 7fa41affd700 1 -- 172.21.15.2:0/1600277262 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa424007ca0 con 0x7fa428102b40 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.042+0000 7fa41affd700 1 -- 172.21.15.2:0/1600277262 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424018660 con 0x7fa428102b40 2024-05-14T09:31:17.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.042+0000 7fa430041700 1 -- 172.21.15.2:0/1600277262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4281a0490 con 0x7fa428102b40 2024-05-14T09:31:17.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.043+0000 7fa430041700 1 -- 172.21.15.2:0/1600277262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4280512c0 con 0x7fa428102b40 2024-05-14T09:31:17.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.045+0000 7fa41affd700 1 -- 172.21.15.2:0/1600277262 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fa42400f450 con 0x7fa428102b40 2024-05-14T09:31:17.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.046+0000 7fa41affd700 1 --2- 172.21.15.2:0/1600277262 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa4140b2880 0x7fa4140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:17.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.046+0000 7fa42d5dc700 1 --2- 172.21.15.2:0/1600277262 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa4140b2880 0x7fa4140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:17.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.046+0000 7fa41affd700 1 -- 172.21.15.2:0/1600277262 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fa4240e2530 con 0x7fa428102b40 2024-05-14T09:31:17.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.046+0000 7fa42d5dc700 1 --2- 172.21.15.2:0/1600277262 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa4140b2880 0x7fa4140b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa428108ef0 tx=0x7fa41c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:17.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.050+0000 7fa41affd700 1 -- 172.21.15.2:0/1600277262 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa424014030 con 0x7fa428102b40 2024-05-14T09:31:17.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.310+0000 7fa430041700 1 -- 172.21.15.2:0/1600277262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa42819c1d0 con 0x7fa428102b40 2024-05-14T09:31:17.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.311+0000 7fa41affd700 1 -- 172.21.15.2:0/1600277262 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11770 (secure 0 0 0) 0x7fa42819c1d0 con 0x7fa428102b40 2024-05-14T09:31:17.311 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:31:17.311 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T09:31:13.172624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24524,"qdb_cluster":[24524]},"id":1}]} 2024-05-14T09:31:17.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.313+0000 7fa430041700 1 -- 172.21.15.2:0/1600277262 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa4140b2880 msgr2=0x7fa4140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:17.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.313+0000 7fa430041700 1 --2- 172.21.15.2:0/1600277262 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa4140b2880 0x7fa4140b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa428108ef0 tx=0x7fa41c01a040 comp rx=0 tx=0).stop 2024-05-14T09:31:17.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.313+0000 7fa430041700 1 -- 172.21.15.2:0/1600277262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa428102b40 msgr2=0x7fa42819ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:17.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.313+0000 7fa430041700 1 --2- 172.21.15.2:0/1600277262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa428102b40 0x7fa42819ae40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa42400c8f0 tx=0x7fa42400ccb0 comp rx=0 tx=0).stop 2024-05-14T09:31:17.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.313+0000 7fa430041700 1 -- 172.21.15.2:0/1600277262 shutdown_connections 2024-05-14T09:31:17.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.313+0000 7fa430041700 1 --2- 172.21.15.2:0/1600277262 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa4140b2880 0x7fa4140b4d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:17.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.313+0000 7fa430041700 1 --2- 172.21.15.2:0/1600277262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4281098b0 0x7fa42819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:17.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.313+0000 7fa430041700 1 --2- 172.21.15.2:0/1600277262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa428102b40 0x7fa42819ae40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:17.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.313+0000 7fa430041700 1 --2- 172.21.15.2:0/1600277262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa428108f60 0x7fa42819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:17.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.313+0000 7fa430041700 1 -- 172.21.15.2:0/1600277262 >> 172.21.15.2:0/1600277262 conn(0x7fa4280fe540 msgr2=0x7fa428100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:17.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.313+0000 7fa430041700 1 -- 172.21.15.2:0/1600277262 shutdown_connections 2024-05-14T09:31:17.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:17.313+0000 7fa430041700 1 -- 172.21.15.2:0/1600277262 wait complete. 2024-05-14T09:31:17.314 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-05-14T09:31:18.039 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T09:31:18.294 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:18.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:18 smithi002 ceph-mon[31557]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:18.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:18 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1600277262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:31:18.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:18 smithi029 ceph-mon[36249]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:18.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:18 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1600277262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:31:18.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:18 smithi026 ceph-mon[36056]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:18.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:18 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1600277262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:31:20.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.103+0000 7f1822113700 1 -- 172.21.15.2:0/1762058054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f181c108740 msgr2=0x7f181c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:20.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.103+0000 7f1822113700 1 --2- 172.21.15.2:0/1762058054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f181c108740 0x7f181c1100a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f181400b0d0 tx=0x7f181400b3e0 comp rx=0 tx=0).stop 2024-05-14T09:31:20.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.103+0000 7f1822113700 1 -- 172.21.15.2:0/1762058054 shutdown_connections 2024-05-14T09:31:20.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.103+0000 7f1822113700 1 --2- 172.21.15.2:0/1762058054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f181c1034c0 0x7f181c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:20.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.103+0000 7f1822113700 1 --2- 172.21.15.2:0/1762058054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f181c108740 0x7f181c1100a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:20.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.103+0000 7f1822113700 1 --2- 172.21.15.2:0/1762058054 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f181c107df0 0x7f181c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:20.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.103+0000 7f1822113700 1 -- 172.21.15.2:0/1762058054 >> 172.21.15.2:0/1762058054 conn(0x7f181c0fedf0 msgr2=0x7f181c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:20.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.103+0000 7f1822113700 1 -- 172.21.15.2:0/1762058054 shutdown_connections 2024-05-14T09:31:20.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.103+0000 7f1822113700 1 -- 172.21.15.2:0/1762058054 wait complete. 2024-05-14T09:31:20.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.104+0000 7f1822113700 1 Processor -- start 2024-05-14T09:31:20.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.104+0000 7f1822113700 1 -- start start 2024-05-14T09:31:20.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.104+0000 7f1822113700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f181c1034c0 0x7f181c0ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:20.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.104+0000 7f1822113700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f181c107df0 0x7f181c0ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:20.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.104+0000 7f1822113700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f181c108740 0x7f181c0ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:20.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.104+0000 7f1822113700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f181c113370 con 0x7f181c108740 2024-05-14T09:31:20.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.104+0000 7f1822113700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1131f0 con 0x7f181c1034c0 2024-05-14T09:31:20.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.104+0000 7f1822113700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1134f0 con 0x7f181c107df0 2024-05-14T09:31:20.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f181bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f181c108740 0x7f181c0ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:20.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f181bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f181c108740 0x7f181c0ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33000/0 (socket says 172.21.15.2:33000) 2024-05-14T09:31:20.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f181bfff700 1 -- 172.21.15.2:0/135227123 learned_addr learned my addr 172.21.15.2:0/135227123 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:20.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f181b7fe700 1 --2- 172.21.15.2:0/135227123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f181c1034c0 0x7f181c0ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:20.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f181affd700 1 --2- 172.21.15.2:0/135227123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f181c107df0 0x7f181c0ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:20.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f181bfff700 1 -- 172.21.15.2:0/135227123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f181c107df0 msgr2=0x7f181c0ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:20.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f181bfff700 1 --2- 172.21.15.2:0/135227123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f181c107df0 0x7f181c0ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:20.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f181bfff700 1 -- 172.21.15.2:0/135227123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f181c1034c0 msgr2=0x7f181c0ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:20.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f181bfff700 1 --2- 172.21.15.2:0/135227123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f181c1034c0 0x7f181c0ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:20.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f181bfff700 1 -- 172.21.15.2:0/135227123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181c1a9110 con 0x7f181c108740 2024-05-14T09:31:20.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f181b7fe700 1 --2- 172.21.15.2:0/135227123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f181c1034c0 0x7f181c0ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:31:20.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f181bfff700 1 --2- 172.21.15.2:0/135227123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f181c108740 0x7f181c0ffd50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f18140095c0 tx=0x7f1814007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:20.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f1818ff9700 1 -- 172.21.15.2:0/135227123 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181401f070 con 0x7f181c108740 2024-05-14T09:31:20.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f1818ff9700 1 -- 172.21.15.2:0/135227123 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1814003c00 con 0x7f181c108740 2024-05-14T09:31:20.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f1818ff9700 1 -- 172.21.15.2:0/135227123 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181401b6a0 con 0x7f181c108740 2024-05-14T09:31:20.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f1822113700 1 -- 172.21.15.2:0/135227123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181c1a93a0 con 0x7f181c108740 2024-05-14T09:31:20.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.105+0000 7f1822113700 1 -- 172.21.15.2:0/135227123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181c1a97a0 con 0x7f181c108740 2024-05-14T09:31:20.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.106+0000 7f1822113700 1 -- 172.21.15.2:0/135227123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f181c0512c0 con 0x7f181c108740 2024-05-14T09:31:20.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.109+0000 7f1818ff9700 1 -- 172.21.15.2:0/135227123 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f1814022070 con 0x7f181c108740 2024-05-14T09:31:20.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.110+0000 7f1818ff9700 1 --2- 172.21.15.2:0/135227123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f18040b2900 0x7f18040b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:20.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.110+0000 7f181b7fe700 1 --2- 172.21.15.2:0/135227123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f18040b2900 0x7f18040b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:20.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.110+0000 7f1818ff9700 1 -- 172.21.15.2:0/135227123 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f181400f020 con 0x7f181c108740 2024-05-14T09:31:20.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.110+0000 7f181b7fe700 1 --2- 172.21.15.2:0/135227123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f18040b2900 0x7f18040b4e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f180c005950 tx=0x7f180c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:20.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.113+0000 7f1818ff9700 1 -- 172.21.15.2:0/135227123 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1814029020 con 0x7f181c108740 2024-05-14T09:31:20.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.327+0000 7f1822113700 1 -- 172.21.15.2:0/135227123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f181c02e850 con 0x7f181c108740 2024-05-14T09:31:20.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.329+0000 7f1818ff9700 1 -- 172.21.15.2:0/135227123 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+23951 (secure 0 0 0) 0x7f18140ab700 con 0x7f181c108740 2024-05-14T09:31:20.329 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:31:20.330 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":86,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","created":"2024-05-14T09:22:34.937047+0000","modified":"2024-05-14T09:31:13.172434+0000","last_up_change":"2024-05-14T09:29:55.157569+0000","last_in_change":"2024-05-14T09:29:34.922562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T09:26:03.046454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T09:30:51.814350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T09:30:55.500000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T09:31:06.099087+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10d34289-414e-4b1b-bc7c-ba1dd24c7297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6803","nonce":2622184634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6805","nonce":2622184634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6809","nonce":2622184634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6807","nonce":2622184634}]},"public_addr":"172.21.15.2:6803/2622184634","cluster_addr":"172.21.15.2:6805/2622184634","heartbeat_back_addr":"172.21.15.2:6809/2622184634","heartbeat_front_addr":"172.21.15.2:6807/2622184634","state":["exists","up"]},{"osd":1,"uuid":"bc4aefc2-706e-4faf-9042-64b87ad097b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6801","nonce":3084747226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6803","nonce":3084747226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6807","nonce":3084747226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6805","nonce":3084747226}]},"public_addr":"172.21.15.26:6801/3084747226","cluster_addr":"172.21.15.26:6803/3084747226","heartbeat_back_addr":"172.21.15.26:6807/3084747226","heartbeat_front_addr":"172.21.15.26:6805/3084747226","state":["exists","up"]},{"osd":2,"uuid":"8c5bb589-56e2-469f-9a6f-6035b3ddf4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6801","nonce":814048086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6803","nonce":814048086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6807","nonce":814048086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6805","nonce":814048086}]},"public_addr":"172.21.15.29:6801/814048086","cluster_addr":"172.21.15.29:6803/814048086","heartbeat_back_addr":"172.21.15.29:6807/814048086","heartbeat_front_addr":"172.21.15.29:6805/814048086","state":["exists","up"]},{"osd":3,"uuid":"8fbfc078-85e7-4dc2-acda-5e7b0ab927e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6811","nonce":879639451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6813","nonce":879639451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6817","nonce":879639451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6815","nonce":879639451}]},"public_addr":"172.21.15.2:6811/879639451","cluster_addr":"172.21.15.2:6813/879639451","heartbeat_back_addr":"172.21.15.2:6817/879639451","heartbeat_front_addr":"172.21.15.2:6815/879639451","state":["exists","up"]},{"osd":4,"uuid":"f2515448-e9c1-48a2-b270-1e5e781a2fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6809","nonce":457100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6811","nonce":457100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6815","nonce":457100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6813","nonce":457100752}]},"public_addr":"172.21.15.26:6809/457100752","cluster_addr":"172.21.15.26:6811/457100752","heartbeat_back_addr":"172.21.15.26:6815/457100752","heartbeat_front_addr":"172.21.15.26:6813/457100752","state":["exists","up"]},{"osd":5,"uuid":"20030857-a1e6-4b0a-b186-1cfa44f2e4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6809","nonce":1554045093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6811","nonce":1554045093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6815","nonce":1554045093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6813","nonce":1554045093}]},"public_addr":"172.21.15.29:6809/1554045093","cluster_addr":"172.21.15.29:6811/1554045093","heartbeat_back_addr":"172.21.15.29:6815/1554045093","heartbeat_front_addr":"172.21.15.29:6813/1554045093","state":["exists","up"]},{"osd":6,"uuid":"d757c72a-29f0-4f14-9f13-a2e61eb5e6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6819","nonce":2749897595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6821","nonce":2749897595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6825","nonce":2749897595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6823","nonce":2749897595}]},"public_addr":"172.21.15.2:6819/2749897595","cluster_addr":"172.21.15.2:6821/2749897595","heartbeat_back_addr":"172.21.15.2:6825/2749897595","heartbeat_front_addr":"172.21.15.2:6823/2749897595","state":["exists","up"]},{"osd":7,"uuid":"22ddef80-3633-44c5-bc0e-edc75b546ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6817","nonce":1180401611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6819","nonce":1180401611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6823","nonce":1180401611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6821","nonce":1180401611}]},"public_addr":"172.21.15.26:6817/1180401611","cluster_addr":"172.21.15.26:6819/1180401611","heartbeat_back_addr":"172.21.15.26:6823/1180401611","heartbeat_front_addr":"172.21.15.26:6821/1180401611","state":["exists","up"]},{"osd":8,"uuid":"466e7ac4-3ff7-4254-a3fd-8096420b1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6817","nonce":2858408446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6819","nonce":2858408446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6823","nonce":2858408446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6821","nonce":2858408446}]},"public_addr":"172.21.15.29:6817/2858408446","cluster_addr":"172.21.15.29:6819/2858408446","heartbeat_back_addr":"172.21.15.29:6823/2858408446","heartbeat_front_addr":"172.21.15.29:6821/2858408446","state":["exists","up"]},{"osd":9,"uuid":"119eeac9-29f7-4086-a1ad-04a0bc154b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6827","nonce":176898226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6829","nonce":176898226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6833","nonce":176898226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6831","nonce":176898226}]},"public_addr":"172.21.15.2:6827/176898226","cluster_addr":"172.21.15.2:6829/176898226","heartbeat_back_addr":"172.21.15.2:6833/176898226","heartbeat_front_addr":"172.21.15.2:6831/176898226","state":["exists","up"]},{"osd":10,"uuid":"55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6825","nonce":4065159515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6827","nonce":4065159515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6831","nonce":4065159515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6829","nonce":4065159515}]},"public_addr":"172.21.15.26:6825/4065159515","cluster_addr":"172.21.15.26:6827/4065159515","heartbeat_back_addr":"172.21.15.26:6831/4065159515","heartbeat_front_addr":"172.21.15.26:6829/4065159515","state":["exists","up"]},{"osd":11,"uuid":"c5246763-c4f6-4ae3-8a93-5929c541ac5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6825","nonce":378800417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6827","nonce":378800417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6831","nonce":378800417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6829","nonce":378800417}]},"public_addr":"172.21.15.29:6825/378800417","cluster_addr":"172.21.15.29:6827/378800417","heartbeat_back_addr":"172.21.15.29:6831/378800417","heartbeat_front_addr":"172.21.15.29:6829/378800417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:09.342387+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:34.156359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:58.860653+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:24.009323+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:49.057146+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:15.007172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:41.085748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:06.674923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:33.409128+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:00.106186+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:26.803487+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:53.668283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4051922409":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/2353790181":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/4268171474":"2024-05-15T09:23:28.940670+0000","172.21.15.2:6801/1423742578":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/969850441":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3252991268":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6801/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3789067476":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/1423742578":"2024-05-15T09:23:28.940670+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T09:31:20.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.332+0000 7f1822113700 1 -- 172.21.15.2:0/135227123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f18040b2900 msgr2=0x7f18040b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:20.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.332+0000 7f1822113700 1 --2- 172.21.15.2:0/135227123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f18040b2900 0x7f18040b4e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f180c005950 tx=0x7f180c0058e0 comp rx=0 tx=0).stop 2024-05-14T09:31:20.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.332+0000 7f1822113700 1 -- 172.21.15.2:0/135227123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f181c108740 msgr2=0x7f181c0ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:20.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.332+0000 7f1822113700 1 --2- 172.21.15.2:0/135227123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f181c108740 0x7f181c0ffd50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f18140095c0 tx=0x7f1814007ad0 comp rx=0 tx=0).stop 2024-05-14T09:31:20.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.332+0000 7f1822113700 1 -- 172.21.15.2:0/135227123 shutdown_connections 2024-05-14T09:31:20.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.332+0000 7f1822113700 1 --2- 172.21.15.2:0/135227123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f18040b2900 0x7f18040b4e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:20.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.332+0000 7f1822113700 1 --2- 172.21.15.2:0/135227123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f181c1034c0 0x7f181c0ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:20.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.332+0000 7f1822113700 1 --2- 172.21.15.2:0/135227123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f181c108740 0x7f181c0ffd50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:20.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.332+0000 7f1822113700 1 --2- 172.21.15.2:0/135227123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f181c107df0 0x7f181c0ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:20.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.332+0000 7f1822113700 1 -- 172.21.15.2:0/135227123 >> 172.21.15.2:0/135227123 conn(0x7f181c0fedf0 msgr2=0x7f181c075c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:20.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.332+0000 7f1822113700 1 -- 172.21.15.2:0/135227123 shutdown_connections 2024-05-14T09:31:20.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:20.332+0000 7f1822113700 1 -- 172.21.15.2:0/135227123 wait complete. 2024-05-14T09:31:20.343 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:20 smithi002 ceph-mon[31557]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:20.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:20 smithi029 ceph-mon[36249]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:20.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:20 smithi026 ceph-mon[36056]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:21.111 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T09:31:21.363 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:21.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:21 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/135227123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:31:21.436 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:21 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/135227123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:31:21.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:21 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/135227123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:31:22.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:22 smithi002 ceph-mon[31557]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:22.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:22 smithi029 ceph-mon[36249]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:22.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:22 smithi026 ceph-mon[36056]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:23.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.185+0000 7fba82952700 1 -- 172.21.15.2:0/3453301294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba7c1034c0 msgr2=0x7fba7c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:23.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.185+0000 7fba82952700 1 --2- 172.21.15.2:0/3453301294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba7c1034c0 0x7fba7c1038d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0099c0 tx=0x7fba6c009cd0 comp rx=0 tx=0).stop 2024-05-14T09:31:23.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.185+0000 7fba82952700 1 -- 172.21.15.2:0/3453301294 shutdown_connections 2024-05-14T09:31:23.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.185+0000 7fba82952700 1 --2- 172.21.15.2:0/3453301294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba7c108740 0x7fba7c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:23.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.185+0000 7fba82952700 1 --2- 172.21.15.2:0/3453301294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba7c1034c0 0x7fba7c1038d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:23.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.185+0000 7fba82952700 1 --2- 172.21.15.2:0/3453301294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba7c107df0 0x7fba7c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:23.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.185+0000 7fba82952700 1 -- 172.21.15.2:0/3453301294 >> 172.21.15.2:0/3453301294 conn(0x7fba7c0fedf0 msgr2=0x7fba7c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:23.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.185+0000 7fba82952700 1 -- 172.21.15.2:0/3453301294 shutdown_connections 2024-05-14T09:31:23.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.185+0000 7fba82952700 1 -- 172.21.15.2:0/3453301294 wait complete. 2024-05-14T09:31:23.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.186+0000 7fba82952700 1 Processor -- start 2024-05-14T09:31:23.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.186+0000 7fba82952700 1 -- start start 2024-05-14T09:31:23.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.186+0000 7fba82952700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba7c1034c0 0x7fba7c19bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:23.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.186+0000 7fba82952700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba7c107df0 0x7fba7c19c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:23.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.186+0000 7fba82952700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba7c108740 0x7fba7c19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:23.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.186+0000 7fba82952700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1133c0 con 0x7fba7c108740 2024-05-14T09:31:23.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.186+0000 7fba82952700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c113240 con 0x7fba7c1034c0 2024-05-14T09:31:23.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.186+0000 7fba82952700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c113540 con 0x7fba7c107df0 2024-05-14T09:31:23.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.187+0000 7fba80eef700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba7c108740 0x7fba7c19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:23.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.187+0000 7fba80eef700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba7c108740 0x7fba7c19c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33014/0 (socket says 172.21.15.2:33014) 2024-05-14T09:31:23.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.187+0000 7fba80eef700 1 -- 172.21.15.2:0/2631054526 learned_addr learned my addr 172.21.15.2:0/2631054526 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:23.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.187+0000 7fba73fff700 1 --2- 172.21.15.2:0/2631054526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba7c107df0 0x7fba7c19c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:23.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.187+0000 7fba7bfff700 1 --2- 172.21.15.2:0/2631054526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba7c1034c0 0x7fba7c19bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:23.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.187+0000 7fba80eef700 1 -- 172.21.15.2:0/2631054526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba7c107df0 msgr2=0x7fba7c19c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:23.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.187+0000 7fba80eef700 1 --2- 172.21.15.2:0/2631054526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba7c107df0 0x7fba7c19c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:23.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.187+0000 7fba80eef700 1 -- 172.21.15.2:0/2631054526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba7c1034c0 msgr2=0x7fba7c19bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:23.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.187+0000 7fba80eef700 1 --2- 172.21.15.2:0/2631054526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba7c1034c0 0x7fba7c19bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:23.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.187+0000 7fba80eef700 1 -- 172.21.15.2:0/2631054526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c1a7010 con 0x7fba7c108740 2024-05-14T09:31:23.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.187+0000 7fba73fff700 1 --2- 172.21.15.2:0/2631054526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba7c107df0 0x7fba7c19c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:31:23.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.187+0000 7fba80eef700 1 --2- 172.21.15.2:0/2631054526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba7c108740 0x7fba7c19c760 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fba74007ee0 tx=0x7fba7400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:23.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.187+0000 7fba79ffb700 1 -- 172.21.15.2:0/2631054526 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74004bb0 con 0x7fba7c108740 2024-05-14T09:31:23.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.188+0000 7fba82952700 1 -- 172.21.15.2:0/2631054526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7c1a7300 con 0x7fba7c108740 2024-05-14T09:31:23.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.188+0000 7fba79ffb700 1 -- 172.21.15.2:0/2631054526 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba74004d50 con 0x7fba7c108740 2024-05-14T09:31:23.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.188+0000 7fba79ffb700 1 -- 172.21.15.2:0/2631054526 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74005920 con 0x7fba7c108740 2024-05-14T09:31:23.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.188+0000 7fba82952700 1 -- 172.21.15.2:0/2631054526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7c1a7840 con 0x7fba7c108740 2024-05-14T09:31:23.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.189+0000 7fba82952700 1 -- 172.21.15.2:0/2631054526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7c0512c0 con 0x7fba7c108740 2024-05-14T09:31:23.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.190+0000 7fba79ffb700 1 -- 172.21.15.2:0/2631054526 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fba7401a020 con 0x7fba7c108740 2024-05-14T09:31:23.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.191+0000 7fba79ffb700 1 --2- 172.21.15.2:0/2631054526 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fba680b2900 0x7fba680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:23.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.191+0000 7fba7bfff700 1 --2- 172.21.15.2:0/2631054526 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fba680b2900 0x7fba680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:23.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.191+0000 7fba79ffb700 1 -- 172.21.15.2:0/2631054526 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fba740e1a30 con 0x7fba7c108740 2024-05-14T09:31:23.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.192+0000 7fba7bfff700 1 --2- 172.21.15.2:0/2631054526 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fba680b2900 0x7fba680b4e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00b580 tx=0x7fba6c0036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:23.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.196+0000 7fba79ffb700 1 -- 172.21.15.2:0/2631054526 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba74010070 con 0x7fba7c108740 2024-05-14T09:31:23.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.418+0000 7fba82952700 1 -- 172.21.15.2:0/2631054526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba7c02e5f0 con 0x7fba7c108740 2024-05-14T09:31:23.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.420+0000 7fba79ffb700 1 -- 172.21.15.2:0/2631054526 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+23951 (secure 0 0 0) 0x7fba740a94b0 con 0x7fba7c108740 2024-05-14T09:31:23.420 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:31:23.420 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":86,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","created":"2024-05-14T09:22:34.937047+0000","modified":"2024-05-14T09:31:13.172434+0000","last_up_change":"2024-05-14T09:29:55.157569+0000","last_in_change":"2024-05-14T09:29:34.922562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T09:26:03.046454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T09:30:51.814350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T09:30:55.500000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T09:31:06.099087+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10d34289-414e-4b1b-bc7c-ba1dd24c7297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6803","nonce":2622184634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6805","nonce":2622184634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6809","nonce":2622184634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6807","nonce":2622184634}]},"public_addr":"172.21.15.2:6803/2622184634","cluster_addr":"172.21.15.2:6805/2622184634","heartbeat_back_addr":"172.21.15.2:6809/2622184634","heartbeat_front_addr":"172.21.15.2:6807/2622184634","state":["exists","up"]},{"osd":1,"uuid":"bc4aefc2-706e-4faf-9042-64b87ad097b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6801","nonce":3084747226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6803","nonce":3084747226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6807","nonce":3084747226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6805","nonce":3084747226}]},"public_addr":"172.21.15.26:6801/3084747226","cluster_addr":"172.21.15.26:6803/3084747226","heartbeat_back_addr":"172.21.15.26:6807/3084747226","heartbeat_front_addr":"172.21.15.26:6805/3084747226","state":["exists","up"]},{"osd":2,"uuid":"8c5bb589-56e2-469f-9a6f-6035b3ddf4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6801","nonce":814048086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6803","nonce":814048086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6807","nonce":814048086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6805","nonce":814048086}]},"public_addr":"172.21.15.29:6801/814048086","cluster_addr":"172.21.15.29:6803/814048086","heartbeat_back_addr":"172.21.15.29:6807/814048086","heartbeat_front_addr":"172.21.15.29:6805/814048086","state":["exists","up"]},{"osd":3,"uuid":"8fbfc078-85e7-4dc2-acda-5e7b0ab927e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6811","nonce":879639451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6813","nonce":879639451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6817","nonce":879639451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6815","nonce":879639451}]},"public_addr":"172.21.15.2:6811/879639451","cluster_addr":"172.21.15.2:6813/879639451","heartbeat_back_addr":"172.21.15.2:6817/879639451","heartbeat_front_addr":"172.21.15.2:6815/879639451","state":["exists","up"]},{"osd":4,"uuid":"f2515448-e9c1-48a2-b270-1e5e781a2fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6809","nonce":457100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6811","nonce":457100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6815","nonce":457100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6813","nonce":457100752}]},"public_addr":"172.21.15.26:6809/457100752","cluster_addr":"172.21.15.26:6811/457100752","heartbeat_back_addr":"172.21.15.26:6815/457100752","heartbeat_front_addr":"172.21.15.26:6813/457100752","state":["exists","up"]},{"osd":5,"uuid":"20030857-a1e6-4b0a-b186-1cfa44f2e4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6809","nonce":1554045093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6811","nonce":1554045093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6815","nonce":1554045093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6813","nonce":1554045093}]},"public_addr":"172.21.15.29:6809/1554045093","cluster_addr":"172.21.15.29:6811/1554045093","heartbeat_back_addr":"172.21.15.29:6815/1554045093","heartbeat_front_addr":"172.21.15.29:6813/1554045093","state":["exists","up"]},{"osd":6,"uuid":"d757c72a-29f0-4f14-9f13-a2e61eb5e6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6819","nonce":2749897595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6821","nonce":2749897595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6825","nonce":2749897595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6823","nonce":2749897595}]},"public_addr":"172.21.15.2:6819/2749897595","cluster_addr":"172.21.15.2:6821/2749897595","heartbeat_back_addr":"172.21.15.2:6825/2749897595","heartbeat_front_addr":"172.21.15.2:6823/2749897595","state":["exists","up"]},{"osd":7,"uuid":"22ddef80-3633-44c5-bc0e-edc75b546ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6817","nonce":1180401611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6819","nonce":1180401611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6823","nonce":1180401611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6821","nonce":1180401611}]},"public_addr":"172.21.15.26:6817/1180401611","cluster_addr":"172.21.15.26:6819/1180401611","heartbeat_back_addr":"172.21.15.26:6823/1180401611","heartbeat_front_addr":"172.21.15.26:6821/1180401611","state":["exists","up"]},{"osd":8,"uuid":"466e7ac4-3ff7-4254-a3fd-8096420b1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6817","nonce":2858408446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6819","nonce":2858408446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6823","nonce":2858408446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6821","nonce":2858408446}]},"public_addr":"172.21.15.29:6817/2858408446","cluster_addr":"172.21.15.29:6819/2858408446","heartbeat_back_addr":"172.21.15.29:6823/2858408446","heartbeat_front_addr":"172.21.15.29:6821/2858408446","state":["exists","up"]},{"osd":9,"uuid":"119eeac9-29f7-4086-a1ad-04a0bc154b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6827","nonce":176898226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6829","nonce":176898226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6833","nonce":176898226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6831","nonce":176898226}]},"public_addr":"172.21.15.2:6827/176898226","cluster_addr":"172.21.15.2:6829/176898226","heartbeat_back_addr":"172.21.15.2:6833/176898226","heartbeat_front_addr":"172.21.15.2:6831/176898226","state":["exists","up"]},{"osd":10,"uuid":"55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6825","nonce":4065159515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6827","nonce":4065159515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6831","nonce":4065159515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6829","nonce":4065159515}]},"public_addr":"172.21.15.26:6825/4065159515","cluster_addr":"172.21.15.26:6827/4065159515","heartbeat_back_addr":"172.21.15.26:6831/4065159515","heartbeat_front_addr":"172.21.15.26:6829/4065159515","state":["exists","up"]},{"osd":11,"uuid":"c5246763-c4f6-4ae3-8a93-5929c541ac5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6825","nonce":378800417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6827","nonce":378800417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6831","nonce":378800417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6829","nonce":378800417}]},"public_addr":"172.21.15.29:6825/378800417","cluster_addr":"172.21.15.29:6827/378800417","heartbeat_back_addr":"172.21.15.29:6831/378800417","heartbeat_front_addr":"172.21.15.29:6829/378800417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:09.342387+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:34.156359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:58.860653+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:24.009323+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:49.057146+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:15.007172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:41.085748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:06.674923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:33.409128+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:00.106186+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:26.803487+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:53.668283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4051922409":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/2353790181":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/4268171474":"2024-05-15T09:23:28.940670+0000","172.21.15.2:6801/1423742578":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/969850441":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3252991268":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6801/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3789067476":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/1423742578":"2024-05-15T09:23:28.940670+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T09:31:23.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.422+0000 7fba82952700 1 -- 172.21.15.2:0/2631054526 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fba680b2900 msgr2=0x7fba680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:23.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.422+0000 7fba82952700 1 --2- 172.21.15.2:0/2631054526 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fba680b2900 0x7fba680b4e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00b580 tx=0x7fba6c0036e0 comp rx=0 tx=0).stop 2024-05-14T09:31:23.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.422+0000 7fba82952700 1 -- 172.21.15.2:0/2631054526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba7c108740 msgr2=0x7fba7c19c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:23.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.422+0000 7fba82952700 1 --2- 172.21.15.2:0/2631054526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba7c108740 0x7fba7c19c760 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fba74007ee0 tx=0x7fba7400d3b0 comp rx=0 tx=0).stop 2024-05-14T09:31:23.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.422+0000 7fba82952700 1 -- 172.21.15.2:0/2631054526 shutdown_connections 2024-05-14T09:31:23.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.422+0000 7fba82952700 1 --2- 172.21.15.2:0/2631054526 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fba680b2900 0x7fba680b4e10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:23.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.422+0000 7fba82952700 1 --2- 172.21.15.2:0/2631054526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba7c1034c0 0x7fba7c19bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:23.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.422+0000 7fba82952700 1 --2- 172.21.15.2:0/2631054526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba7c108740 0x7fba7c19c760 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:23.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.422+0000 7fba82952700 1 --2- 172.21.15.2:0/2631054526 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba7c107df0 0x7fba7c19c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:23.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.422+0000 7fba82952700 1 -- 172.21.15.2:0/2631054526 >> 172.21.15.2:0/2631054526 conn(0x7fba7c0fedf0 msgr2=0x7fba7c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:23.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.423+0000 7fba82952700 1 -- 172.21.15.2:0/2631054526 shutdown_connections 2024-05-14T09:31:23.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:23.423+0000 7fba82952700 1 -- 172.21.15.2:0/2631054526 wait complete. 2024-05-14T09:31:24.074 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:31:24.328 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:24.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:24 smithi002 ceph-mon[31557]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:24.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:24 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2631054526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:31:24.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:24 smithi029 ceph-mon[36249]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:24.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:24 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2631054526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:31:24.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:24 smithi026 ceph-mon[36056]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:24.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:24 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2631054526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:31:26.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.110+0000 7f3d9ee6f700 1 -- 172.21.15.2:0/1592641749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d98103b30 msgr2=0x7f3d98103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:26.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.110+0000 7f3d9ee6f700 1 --2- 172.21.15.2:0/1592641749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d98103b30 0x7f3d98103f40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0099c0 tx=0x7f3d8c009cd0 comp rx=0 tx=0).stop 2024-05-14T09:31:26.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.110+0000 7f3d9ee6f700 1 -- 172.21.15.2:0/1592641749 shutdown_connections 2024-05-14T09:31:26.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.110+0000 7f3d9ee6f700 1 --2- 172.21.15.2:0/1592641749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d981087a0 0x7f3d98110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:26.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.110+0000 7f3d9ee6f700 1 --2- 172.21.15.2:0/1592641749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d98103b30 0x7f3d98103f40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:26.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.110+0000 7f3d9ee6f700 1 --2- 172.21.15.2:0/1592641749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d98107e50 0x7f3d98108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:26.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.110+0000 7f3d9ee6f700 1 -- 172.21.15.2:0/1592641749 >> 172.21.15.2:0/1592641749 conn(0x7f3d980fe4e0 msgr2=0x7f3d98100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:26.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.110+0000 7f3d9ee6f700 1 -- 172.21.15.2:0/1592641749 shutdown_connections 2024-05-14T09:31:26.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.110+0000 7f3d9ee6f700 1 -- 172.21.15.2:0/1592641749 wait complete. 2024-05-14T09:31:26.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.111+0000 7f3d9ee6f700 1 Processor -- start 2024-05-14T09:31:26.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.111+0000 7f3d9ee6f700 1 -- start start 2024-05-14T09:31:26.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.111+0000 7f3d9ee6f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d98103b30 0x7f3d9819bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:26.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.111+0000 7f3d9ee6f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d98107e50 0x7f3d9819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:26.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.111+0000 7f3d9ee6f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d981087a0 0x7f3d9819c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:26.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.111+0000 7f3d9ee6f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3d98113500 con 0x7f3d98103b30 2024-05-14T09:31:26.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.111+0000 7f3d9ee6f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3d98113380 con 0x7f3d98107e50 2024-05-14T09:31:26.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.111+0000 7f3d9ee6f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3d98113680 con 0x7f3d981087a0 2024-05-14T09:31:26.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.111+0000 7f3d9cc0b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d98103b30 0x7f3d9819bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:26.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.112+0000 7f3d9cc0b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d98103b30 0x7f3d9819bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33034/0 (socket says 172.21.15.2:33034) 2024-05-14T09:31:26.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.112+0000 7f3d9cc0b700 1 -- 172.21.15.2:0/3350775442 learned_addr learned my addr 172.21.15.2:0/3350775442 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:26.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.112+0000 7f3d97fff700 1 --2- 172.21.15.2:0/3350775442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d98107e50 0x7f3d9819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:26.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.112+0000 7f3d9d40c700 1 --2- 172.21.15.2:0/3350775442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d981087a0 0x7f3d9819c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:26.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.112+0000 7f3d9cc0b700 1 -- 172.21.15.2:0/3350775442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d981087a0 msgr2=0x7f3d9819c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:26.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.112+0000 7f3d9cc0b700 1 --2- 172.21.15.2:0/3350775442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d981087a0 0x7f3d9819c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:26.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.112+0000 7f3d9cc0b700 1 -- 172.21.15.2:0/3350775442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d98107e50 msgr2=0x7f3d9819c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:26.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.112+0000 7f3d9cc0b700 1 --2- 172.21.15.2:0/3350775442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d98107e50 0x7f3d9819c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:26.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.112+0000 7f3d9cc0b700 1 -- 172.21.15.2:0/3350775442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d981a71c0 con 0x7f3d98103b30 2024-05-14T09:31:26.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.112+0000 7f3d9cc0b700 1 --2- 172.21.15.2:0/3350775442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d98103b30 0x7f3d9819bea0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00b580 tx=0x7f3d8c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:26.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.112+0000 7f3d95ffb700 1 -- 172.21.15.2:0/3350775442 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c027070 con 0x7f3d98103b30 2024-05-14T09:31:26.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.112+0000 7f3d95ffb700 1 -- 172.21.15.2:0/3350775442 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d8c0040a0 con 0x7f3d98103b30 2024-05-14T09:31:26.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.112+0000 7f3d9ee6f700 1 -- 172.21.15.2:0/3350775442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d981a7450 con 0x7f3d98103b30 2024-05-14T09:31:26.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.112+0000 7f3d95ffb700 1 -- 172.21.15.2:0/3350775442 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c00f460 con 0x7f3d98103b30 2024-05-14T09:31:26.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.113+0000 7f3d9ee6f700 1 -- 172.21.15.2:0/3350775442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d981a78a0 con 0x7f3d98103b30 2024-05-14T09:31:26.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.113+0000 7f3d9ee6f700 1 -- 172.21.15.2:0/3350775442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9810b7c0 con 0x7f3d98103b30 2024-05-14T09:31:26.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.115+0000 7f3d95ffb700 1 -- 172.21.15.2:0/3350775442 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f3d8c02a070 con 0x7f3d98103b30 2024-05-14T09:31:26.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.116+0000 7f3d95ffb700 1 --2- 172.21.15.2:0/3350775442 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3d880b2880 0x7f3d880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:26.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.116+0000 7f3d97fff700 1 --2- 172.21.15.2:0/3350775442 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3d880b2880 0x7f3d880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:26.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.116+0000 7f3d95ffb700 1 -- 172.21.15.2:0/3350775442 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f3d8c0e3910 con 0x7f3d98103b30 2024-05-14T09:31:26.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.116+0000 7f3d97fff700 1 --2- 172.21.15.2:0/3350775442 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3d880b2880 0x7f3d880b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3d981079f0 tx=0x7f3d84009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:26.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.119+0000 7f3d95ffb700 1 -- 172.21.15.2:0/3350775442 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3d8c031080 con 0x7f3d98103b30 2024-05-14T09:31:26.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.382+0000 7f3d9ee6f700 1 -- 172.21.15.2:0/3350775442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d980512c0 con 0x7f3d98103b30 2024-05-14T09:31:26.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:26 smithi002 ceph-mon[31557]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:26.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.383+0000 7f3d95ffb700 1 -- 172.21.15.2:0/3350775442 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11770 (secure 0 0 0) 0x7f3d8c013070 con 0x7f3d98103b30 2024-05-14T09:31:26.383 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:31:26.384 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T09:31:13.172624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24524,"qdb_cluster":[24524]},"id":1}]} 2024-05-14T09:31:26.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.385+0000 7f3d9ee6f700 1 -- 172.21.15.2:0/3350775442 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3d880b2880 msgr2=0x7f3d880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:26.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.385+0000 7f3d9ee6f700 1 --2- 172.21.15.2:0/3350775442 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3d880b2880 0x7f3d880b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3d981079f0 tx=0x7f3d84009190 comp rx=0 tx=0).stop 2024-05-14T09:31:26.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.385+0000 7f3d9ee6f700 1 -- 172.21.15.2:0/3350775442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d98103b30 msgr2=0x7f3d9819bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:26.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.385+0000 7f3d9ee6f700 1 --2- 172.21.15.2:0/3350775442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d98103b30 0x7f3d9819bea0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00b580 tx=0x7f3d8c0047b0 comp rx=0 tx=0).stop 2024-05-14T09:31:26.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.386+0000 7f3d9ee6f700 1 -- 172.21.15.2:0/3350775442 shutdown_connections 2024-05-14T09:31:26.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.386+0000 7f3d9ee6f700 1 --2- 172.21.15.2:0/3350775442 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3d880b2880 0x7f3d880b4d90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:26.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.386+0000 7f3d9ee6f700 1 --2- 172.21.15.2:0/3350775442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d981087a0 0x7f3d9819c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:26.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.386+0000 7f3d9ee6f700 1 --2- 172.21.15.2:0/3350775442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d98103b30 0x7f3d9819bea0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:26.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.386+0000 7f3d9ee6f700 1 --2- 172.21.15.2:0/3350775442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d98107e50 0x7f3d9819c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:26.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.386+0000 7f3d9ee6f700 1 -- 172.21.15.2:0/3350775442 >> 172.21.15.2:0/3350775442 conn(0x7f3d980fe4e0 msgr2=0x7f3d98106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:26.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.386+0000 7f3d9ee6f700 1 -- 172.21.15.2:0/3350775442 shutdown_connections 2024-05-14T09:31:26.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:26.386+0000 7f3d9ee6f700 1 -- 172.21.15.2:0/3350775442 wait complete. 2024-05-14T09:31:26.387 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-05-14T09:31:26.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:26 smithi029 ceph-mon[36249]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:26.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:26 smithi026 ceph-mon[36056]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:26.992 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T09:31:27.241 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:27.354 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:27 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3350775442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:31:27.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:27 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3350775442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:31:27.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:27 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3350775442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:31:28.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:28 smithi002 ceph-mon[31557]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:28.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:28 smithi029 ceph-mon[36249]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:28.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:28 smithi026 ceph-mon[36056]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:28.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.868+0000 7f4c924a9700 1 -- 172.21.15.2:0/4126105080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c8c109860 msgr2=0x7f4c8c1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:28.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.868+0000 7f4c924a9700 1 --2- 172.21.15.2:0/4126105080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c8c109860 0x7f4c8c1111c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400b0d0 tx=0x7f4c8400b3e0 comp rx=0 tx=0).stop 2024-05-14T09:31:28.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.868+0000 7f4c924a9700 1 -- 172.21.15.2:0/4126105080 shutdown_connections 2024-05-14T09:31:28.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.868+0000 7f4c924a9700 1 --2- 172.21.15.2:0/4126105080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c8c1045e0 0x7f4c8c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:28.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.868+0000 7f4c924a9700 1 --2- 172.21.15.2:0/4126105080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c8c109860 0x7f4c8c1111c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:28.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.868+0000 7f4c924a9700 1 --2- 172.21.15.2:0/4126105080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c8c108f10 0x7f4c8c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:28.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.868+0000 7f4c924a9700 1 -- 172.21.15.2:0/4126105080 >> 172.21.15.2:0/4126105080 conn(0x7f4c8c0ffed0 msgr2=0x7f4c8c102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:28.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.868+0000 7f4c924a9700 1 -- 172.21.15.2:0/4126105080 shutdown_connections 2024-05-14T09:31:28.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.868+0000 7f4c924a9700 1 -- 172.21.15.2:0/4126105080 wait complete. 2024-05-14T09:31:28.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.869+0000 7f4c924a9700 1 Processor -- start 2024-05-14T09:31:28.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.869+0000 7f4c924a9700 1 -- start start 2024-05-14T09:31:28.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.869+0000 7f4c924a9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c8c1045e0 0x7f4c8c076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:28.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.869+0000 7f4c924a9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c8c108f10 0x7f4c8c077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:28.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.869+0000 7f4c924a9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c8c109860 0x7f4c8c077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:28.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.869+0000 7f4c924a9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c114510 con 0x7f4c8c108f10 2024-05-14T09:31:28.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.869+0000 7f4c924a9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c114390 con 0x7f4c8c1045e0 2024-05-14T09:31:28.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.869+0000 7f4c924a9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c114690 con 0x7f4c8c109860 2024-05-14T09:31:28.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c83fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c8c108f10 0x7f4c8c077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:28.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c83fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c8c108f10 0x7f4c8c077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42324/0 (socket says 172.21.15.2:42324) 2024-05-14T09:31:28.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c83fff700 1 -- 172.21.15.2:0/203422744 learned_addr learned my addr 172.21.15.2:0/203422744 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:28.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c8bfff700 1 --2- 172.21.15.2:0/203422744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c8c1045e0 0x7f4c8c076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:28.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c90a46700 1 --2- 172.21.15.2:0/203422744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c8c109860 0x7f4c8c077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:28.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c83fff700 1 -- 172.21.15.2:0/203422744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c8c109860 msgr2=0x7f4c8c077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:28.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c83fff700 1 --2- 172.21.15.2:0/203422744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c8c109860 0x7f4c8c077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:28.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c83fff700 1 -- 172.21.15.2:0/203422744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c8c1045e0 msgr2=0x7f4c8c076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:28.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c83fff700 1 --2- 172.21.15.2:0/203422744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c8c1045e0 0x7f4c8c076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:28.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c83fff700 1 -- 172.21.15.2:0/203422744 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c1a91f0 con 0x7f4c8c108f10 2024-05-14T09:31:28.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c83fff700 1 --2- 172.21.15.2:0/203422744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c8c108f10 0x7f4c8c077110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400d980 tx=0x7f4c7400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:28.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c89ffb700 1 -- 172.21.15.2:0/203422744 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74017070 con 0x7f4c8c108f10 2024-05-14T09:31:28.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c924a9700 1 -- 172.21.15.2:0/203422744 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c1a94e0 con 0x7f4c8c108f10 2024-05-14T09:31:28.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c89ffb700 1 -- 172.21.15.2:0/203422744 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c74009de0 con 0x7f4c8c108f10 2024-05-14T09:31:28.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c89ffb700 1 -- 172.21.15.2:0/203422744 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400f3c0 con 0x7f4c8c108f10 2024-05-14T09:31:28.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.870+0000 7f4c924a9700 1 -- 172.21.15.2:0/203422744 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c1a9a70 con 0x7f4c8c108f10 2024-05-14T09:31:28.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.871+0000 7f4c924a9700 1 -- 172.21.15.2:0/203422744 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8c0512c0 con 0x7f4c8c108f10 2024-05-14T09:31:28.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.873+0000 7f4c89ffb700 1 -- 172.21.15.2:0/203422744 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f4c74009930 con 0x7f4c8c108f10 2024-05-14T09:31:28.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.874+0000 7f4c89ffb700 1 --2- 172.21.15.2:0/203422744 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4c780b2880 0x7f4c780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:28.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.874+0000 7f4c89ffb700 1 -- 172.21.15.2:0/203422744 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f4c740e1f40 con 0x7f4c8c108f10 2024-05-14T09:31:28.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.874+0000 7f4c8bfff700 1 --2- 172.21.15.2:0/203422744 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4c780b2880 0x7f4c780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:28.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.874+0000 7f4c8bfff700 1 --2- 172.21.15.2:0/203422744 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4c780b2880 0x7f4c780b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009a10 tx=0x7f4c7c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:28.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:28.877+0000 7f4c89ffb700 1 -- 172.21.15.2:0/203422744 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4c740a9a20 con 0x7f4c8c108f10 2024-05-14T09:31:29.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:29.095+0000 7f4c924a9700 1 -- 172.21.15.2:0/203422744 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c8c078100 con 0x7f4c8c108f10 2024-05-14T09:31:29.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:29.096+0000 7f4c89ffb700 1 -- 172.21.15.2:0/203422744 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+23951 (secure 0 0 0) 0x7f4c740a9320 con 0x7f4c8c108f10 2024-05-14T09:31:29.097 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:31:29.097 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":86,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","created":"2024-05-14T09:22:34.937047+0000","modified":"2024-05-14T09:31:13.172434+0000","last_up_change":"2024-05-14T09:29:55.157569+0000","last_in_change":"2024-05-14T09:29:34.922562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T09:26:03.046454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T09:30:51.814350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T09:30:55.500000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T09:31:06.099087+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10d34289-414e-4b1b-bc7c-ba1dd24c7297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6803","nonce":2622184634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6805","nonce":2622184634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6809","nonce":2622184634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6807","nonce":2622184634}]},"public_addr":"172.21.15.2:6803/2622184634","cluster_addr":"172.21.15.2:6805/2622184634","heartbeat_back_addr":"172.21.15.2:6809/2622184634","heartbeat_front_addr":"172.21.15.2:6807/2622184634","state":["exists","up"]},{"osd":1,"uuid":"bc4aefc2-706e-4faf-9042-64b87ad097b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6801","nonce":3084747226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6803","nonce":3084747226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6807","nonce":3084747226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6805","nonce":3084747226}]},"public_addr":"172.21.15.26:6801/3084747226","cluster_addr":"172.21.15.26:6803/3084747226","heartbeat_back_addr":"172.21.15.26:6807/3084747226","heartbeat_front_addr":"172.21.15.26:6805/3084747226","state":["exists","up"]},{"osd":2,"uuid":"8c5bb589-56e2-469f-9a6f-6035b3ddf4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6801","nonce":814048086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6803","nonce":814048086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6807","nonce":814048086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6805","nonce":814048086}]},"public_addr":"172.21.15.29:6801/814048086","cluster_addr":"172.21.15.29:6803/814048086","heartbeat_back_addr":"172.21.15.29:6807/814048086","heartbeat_front_addr":"172.21.15.29:6805/814048086","state":["exists","up"]},{"osd":3,"uuid":"8fbfc078-85e7-4dc2-acda-5e7b0ab927e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6811","nonce":879639451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6813","nonce":879639451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6817","nonce":879639451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6815","nonce":879639451}]},"public_addr":"172.21.15.2:6811/879639451","cluster_addr":"172.21.15.2:6813/879639451","heartbeat_back_addr":"172.21.15.2:6817/879639451","heartbeat_front_addr":"172.21.15.2:6815/879639451","state":["exists","up"]},{"osd":4,"uuid":"f2515448-e9c1-48a2-b270-1e5e781a2fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6809","nonce":457100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6811","nonce":457100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6815","nonce":457100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6813","nonce":457100752}]},"public_addr":"172.21.15.26:6809/457100752","cluster_addr":"172.21.15.26:6811/457100752","heartbeat_back_addr":"172.21.15.26:6815/457100752","heartbeat_front_addr":"172.21.15.26:6813/457100752","state":["exists","up"]},{"osd":5,"uuid":"20030857-a1e6-4b0a-b186-1cfa44f2e4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6809","nonce":1554045093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6811","nonce":1554045093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6815","nonce":1554045093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6813","nonce":1554045093}]},"public_addr":"172.21.15.29:6809/1554045093","cluster_addr":"172.21.15.29:6811/1554045093","heartbeat_back_addr":"172.21.15.29:6815/1554045093","heartbeat_front_addr":"172.21.15.29:6813/1554045093","state":["exists","up"]},{"osd":6,"uuid":"d757c72a-29f0-4f14-9f13-a2e61eb5e6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6819","nonce":2749897595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6821","nonce":2749897595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6825","nonce":2749897595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6823","nonce":2749897595}]},"public_addr":"172.21.15.2:6819/2749897595","cluster_addr":"172.21.15.2:6821/2749897595","heartbeat_back_addr":"172.21.15.2:6825/2749897595","heartbeat_front_addr":"172.21.15.2:6823/2749897595","state":["exists","up"]},{"osd":7,"uuid":"22ddef80-3633-44c5-bc0e-edc75b546ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6817","nonce":1180401611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6819","nonce":1180401611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6823","nonce":1180401611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6821","nonce":1180401611}]},"public_addr":"172.21.15.26:6817/1180401611","cluster_addr":"172.21.15.26:6819/1180401611","heartbeat_back_addr":"172.21.15.26:6823/1180401611","heartbeat_front_addr":"172.21.15.26:6821/1180401611","state":["exists","up"]},{"osd":8,"uuid":"466e7ac4-3ff7-4254-a3fd-8096420b1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6817","nonce":2858408446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6819","nonce":2858408446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6823","nonce":2858408446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6821","nonce":2858408446}]},"public_addr":"172.21.15.29:6817/2858408446","cluster_addr":"172.21.15.29:6819/2858408446","heartbeat_back_addr":"172.21.15.29:6823/2858408446","heartbeat_front_addr":"172.21.15.29:6821/2858408446","state":["exists","up"]},{"osd":9,"uuid":"119eeac9-29f7-4086-a1ad-04a0bc154b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6827","nonce":176898226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6829","nonce":176898226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6833","nonce":176898226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6831","nonce":176898226}]},"public_addr":"172.21.15.2:6827/176898226","cluster_addr":"172.21.15.2:6829/176898226","heartbeat_back_addr":"172.21.15.2:6833/176898226","heartbeat_front_addr":"172.21.15.2:6831/176898226","state":["exists","up"]},{"osd":10,"uuid":"55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6825","nonce":4065159515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6827","nonce":4065159515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6831","nonce":4065159515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6829","nonce":4065159515}]},"public_addr":"172.21.15.26:6825/4065159515","cluster_addr":"172.21.15.26:6827/4065159515","heartbeat_back_addr":"172.21.15.26:6831/4065159515","heartbeat_front_addr":"172.21.15.26:6829/4065159515","state":["exists","up"]},{"osd":11,"uuid":"c5246763-c4f6-4ae3-8a93-5929c541ac5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6825","nonce":378800417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6827","nonce":378800417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6831","nonce":378800417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6829","nonce":378800417}]},"public_addr":"172.21.15.29:6825/378800417","cluster_addr":"172.21.15.29:6827/378800417","heartbeat_back_addr":"172.21.15.29:6831/378800417","heartbeat_front_addr":"172.21.15.29:6829/378800417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:09.342387+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:34.156359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:58.860653+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:24.009323+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:49.057146+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:15.007172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:41.085748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:06.674923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:33.409128+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:00.106186+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:26.803487+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:53.668283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4051922409":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/2353790181":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/4268171474":"2024-05-15T09:23:28.940670+0000","172.21.15.2:6801/1423742578":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/969850441":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3252991268":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6801/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3789067476":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/1423742578":"2024-05-15T09:23:28.940670+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T09:31:29.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:29.099+0000 7f4c924a9700 1 -- 172.21.15.2:0/203422744 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4c780b2880 msgr2=0x7f4c780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:29.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:29.099+0000 7f4c924a9700 1 --2- 172.21.15.2:0/203422744 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4c780b2880 0x7f4c780b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009a10 tx=0x7f4c7c009190 comp rx=0 tx=0).stop 2024-05-14T09:31:29.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:29.099+0000 7f4c924a9700 1 -- 172.21.15.2:0/203422744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c8c108f10 msgr2=0x7f4c8c077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:29.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:29.099+0000 7f4c924a9700 1 --2- 172.21.15.2:0/203422744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c8c108f10 0x7f4c8c077110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400d980 tx=0x7f4c7400dc90 comp rx=0 tx=0).stop 2024-05-14T09:31:29.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:29.099+0000 7f4c924a9700 1 -- 172.21.15.2:0/203422744 shutdown_connections 2024-05-14T09:31:29.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:29.099+0000 7f4c924a9700 1 --2- 172.21.15.2:0/203422744 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4c780b2880 0x7f4c780b4d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:29.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:29.099+0000 7f4c924a9700 1 --2- 172.21.15.2:0/203422744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c8c1045e0 0x7f4c8c076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:29.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:29.099+0000 7f4c924a9700 1 --2- 172.21.15.2:0/203422744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c8c109860 0x7f4c8c077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:29.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:29.099+0000 7f4c924a9700 1 --2- 172.21.15.2:0/203422744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c8c108f10 0x7f4c8c077110 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:29.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:29.099+0000 7f4c924a9700 1 -- 172.21.15.2:0/203422744 >> 172.21.15.2:0/203422744 conn(0x7f4c8c0ffed0 msgr2=0x7f4c8c108620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:29.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:29.100+0000 7f4c924a9700 1 -- 172.21.15.2:0/203422744 shutdown_connections 2024-05-14T09:31:29.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:29.100+0000 7f4c924a9700 1 -- 172.21.15.2:0/203422744 wait complete. 2024-05-14T09:31:29.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-14T09:31:29.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-14T09:31:29.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-05-14T09:31:29.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [5, 4]}]: dispatch 2024-05-14T09:31:29.407 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11, 8, 6]}]: dispatch 2024-05-14T09:31:29.407 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-05-14T09:31:29.407 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-05-14T09:31:29.407 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-05-14T09:31:29.407 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-14T09:31:29.407 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:29 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/203422744' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:31:29.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-14T09:31:29.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-14T09:31:29.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-05-14T09:31:29.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [5, 4]}]: dispatch 2024-05-14T09:31:29.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11, 8, 6]}]: dispatch 2024-05-14T09:31:29.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-05-14T09:31:29.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-05-14T09:31:29.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-05-14T09:31:29.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-14T09:31:29.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:29 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/203422744' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:31:29.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-14T09:31:29.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-14T09:31:29.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-05-14T09:31:29.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [5, 4]}]: dispatch 2024-05-14T09:31:29.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11, 8, 6]}]: dispatch 2024-05-14T09:31:29.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-05-14T09:31:29.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-05-14T09:31:29.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-05-14T09:31:29.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-14T09:31:29.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:29 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/203422744' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:31:29.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T09:30:58.644918+0000', 'modified': '2024-05-14T09:31:13.172624+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24524}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.26:6839/903985766', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 903985766}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 903985766}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24524, 'qdb_cluster': [24524]} 2024-05-14T09:31:29.724 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T09:31:29.724 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-14T09:31:29.978 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:30.373 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:30 smithi002 ceph-mon[31557]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:30.373 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-14T09:31:30.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-14T09:31:30.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-05-14T09:31:30.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [5, 4]}]': finished 2024-05-14T09:31:30.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11, 8, 6]}]': finished 2024-05-14T09:31:30.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-05-14T09:31:30.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-05-14T09:31:30.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-05-14T09:31:30.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-14T09:31:30.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:30 smithi002 ceph-mon[31557]: osdmap e87: 12 total, 12 up, 12 in 2024-05-14T09:31:30.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:30 smithi029 ceph-mon[36249]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:30.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-14T09:31:30.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-14T09:31:30.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-05-14T09:31:30.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [5, 4]}]': finished 2024-05-14T09:31:30.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11, 8, 6]}]': finished 2024-05-14T09:31:30.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-05-14T09:31:30.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-05-14T09:31:30.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-05-14T09:31:30.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-14T09:31:30.438 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:30 smithi029 ceph-mon[36249]: osdmap e87: 12 total, 12 up, 12 in 2024-05-14T09:31:30.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:30 smithi026 ceph-mon[36056]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T09:31:30.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-14T09:31:30.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-14T09:31:30.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-05-14T09:31:30.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [5, 4]}]': finished 2024-05-14T09:31:30.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11, 8, 6]}]': finished 2024-05-14T09:31:30.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-05-14T09:31:30.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-05-14T09:31:30.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-05-14T09:31:30.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-14T09:31:30.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:30 smithi026 ceph-mon[36056]: osdmap e87: 12 total, 12 up, 12 in 2024-05-14T09:31:31.371 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:31 smithi002 ceph-mon[31557]: osdmap e88: 12 total, 12 up, 12 in 2024-05-14T09:31:31.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:31 smithi029 ceph-mon[36249]: osdmap e88: 12 total, 12 up, 12 in 2024-05-14T09:31:31.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:31 smithi026 ceph-mon[36056]: osdmap e88: 12 total, 12 up, 12 in 2024-05-14T09:31:31.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.615+0000 7fae31a13700 1 -- 172.21.15.2:0/123524851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae2c107df0 msgr2=0x7fae2c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:31.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.615+0000 7fae31a13700 1 --2- 172.21.15.2:0/123524851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae2c107df0 0x7fae2c108200 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fae200099c0 tx=0x7fae20009cd0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.615+0000 7fae31a13700 1 -- 172.21.15.2:0/123524851 shutdown_connections 2024-05-14T09:31:31.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.615+0000 7fae31a13700 1 --2- 172.21.15.2:0/123524851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae2c108740 0x7fae2c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.615+0000 7fae31a13700 1 --2- 172.21.15.2:0/123524851 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae2c1034c0 0x7fae2c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.615+0000 7fae31a13700 1 --2- 172.21.15.2:0/123524851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae2c107df0 0x7fae2c108200 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.615+0000 7fae31a13700 1 -- 172.21.15.2:0/123524851 >> 172.21.15.2:0/123524851 conn(0x7fae2c0fedf0 msgr2=0x7fae2c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:31.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.615+0000 7fae31a13700 1 -- 172.21.15.2:0/123524851 shutdown_connections 2024-05-14T09:31:31.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.615+0000 7fae31a13700 1 -- 172.21.15.2:0/123524851 wait complete. 2024-05-14T09:31:31.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.616+0000 7fae31a13700 1 Processor -- start 2024-05-14T09:31:31.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.616+0000 7fae31a13700 1 -- start start 2024-05-14T09:31:31.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae31a13700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae2c1034c0 0x7fae2c19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae31a13700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae2c107df0 0x7fae2c19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae31a13700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae2c108740 0x7fae2c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae31a13700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c1144c0 con 0x7fae2c108740 2024-05-14T09:31:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae31a13700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c114340 con 0x7fae2c1034c0 2024-05-14T09:31:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae31a13700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c114640 con 0x7fae2c107df0 2024-05-14T09:31:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae2b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae2c108740 0x7fae2c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae2b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae2c108740 0x7fae2c19f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42336/0 (socket says 172.21.15.2:42336) 2024-05-14T09:31:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae2b7fe700 1 -- 172.21.15.2:0/2491679129 learned_addr learned my addr 172.21.15.2:0/2491679129 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae2a7fc700 1 --2- 172.21.15.2:0/2491679129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae2c107df0 0x7fae2c19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae2affd700 1 --2- 172.21.15.2:0/2491679129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae2c1034c0 0x7fae2c19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae2b7fe700 1 -- 172.21.15.2:0/2491679129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae2c107df0 msgr2=0x7fae2c19b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae2b7fe700 1 --2- 172.21.15.2:0/2491679129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae2c107df0 0x7fae2c19b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae2b7fe700 1 -- 172.21.15.2:0/2491679129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae2c1034c0 msgr2=0x7fae2c19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae2b7fe700 1 --2- 172.21.15.2:0/2491679129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae2c1034c0 0x7fae2c19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae2b7fe700 1 -- 172.21.15.2:0/2491679129 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2c19fd80 con 0x7fae2c108740 2024-05-14T09:31:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae2b7fe700 1 --2- 172.21.15.2:0/2491679129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae2c108740 0x7fae2c19f6e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fae24007ee0 tx=0x7fae2400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.617+0000 7fae30a11700 1 -- 172.21.15.2:0/2491679129 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2400dd20 con 0x7fae2c108740 2024-05-14T09:31:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.618+0000 7fae31a13700 1 -- 172.21.15.2:0/2491679129 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2c1a0070 con 0x7fae2c108740 2024-05-14T09:31:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.618+0000 7fae30a11700 1 -- 172.21.15.2:0/2491679129 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae24004020 con 0x7fae2c108740 2024-05-14T09:31:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.618+0000 7fae30a11700 1 -- 172.21.15.2:0/2491679129 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24015660 con 0x7fae2c108740 2024-05-14T09:31:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.618+0000 7fae31a13700 1 -- 172.21.15.2:0/2491679129 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae2c1a7990 con 0x7fae2c108740 2024-05-14T09:31:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.620+0000 7fae30a11700 1 -- 172.21.15.2:0/2491679129 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fae240041d0 con 0x7fae2c108740 2024-05-14T09:31:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.621+0000 7fae30a11700 1 --2- 172.21.15.2:0/2491679129 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fae140b2880 0x7fae140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.621+0000 7fae2affd700 1 --2- 172.21.15.2:0/2491679129 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fae140b2880 0x7fae140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.621+0000 7fae30a11700 1 -- 172.21.15.2:0/2491679129 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fae240e1fb0 con 0x7fae2c108740 2024-05-14T09:31:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.621+0000 7fae40427b80 1 Processor -- start 2024-05-14T09:31:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.621+0000 7fae40427b80 1 -- start start 2024-05-14T09:31:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae40427b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727d37df0 0x555727eb9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae40427b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727eba020 0x555727ebe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae2affd700 1 --2- 172.21.15.2:0/2491679129 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fae140b2880 0x7fae140b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009a10 tx=0x7fae1c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae40427b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727ebe830 0x555727ebec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae40427b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555727dd7810 con 0x555727eba020 2024-05-14T09:31:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae40427b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555727d76770 con 0x555727d37df0 2024-05-14T09:31:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae40427b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555727dd37d0 con 0x555727ebe830 2024-05-14T09:31:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae2a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727eba020 0x555727ebe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae2a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727eba020 0x555727ebe2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42342/0 (socket says 172.21.15.2:42342) 2024-05-14T09:31:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae2a7fc700 1 -- 172.21.15.2:0/3738281012 learned_addr learned my addr 172.21.15.2:0/3738281012 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae2affd700 1 --2- 172.21.15.2:0/3738281012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727d37df0 0x555727eb9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae2b7fe700 1 --2- 172.21.15.2:0/3738281012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727ebe830 0x555727ebec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae2a7fc700 1 -- 172.21.15.2:0/3738281012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727ebe830 msgr2=0x555727ebec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:31:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae2a7fc700 1 --2- 172.21.15.2:0/3738281012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727ebe830 0x555727ebec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae2a7fc700 1 -- 172.21.15.2:0/3738281012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727d37df0 msgr2=0x555727eb9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:31:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae2a7fc700 1 --2- 172.21.15.2:0/3738281012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727d37df0 0x555727eb9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae2a7fc700 1 -- 172.21.15.2:0/3738281012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555727d24880 con 0x555727eba020 2024-05-14T09:31:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.622+0000 7fae2b7fe700 1 --2- 172.21.15.2:0/3738281012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727ebe830 0x555727ebec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:31:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.623+0000 7fae2a7fc700 1 --2- 172.21.15.2:0/3738281012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727eba020 0x555727ebe2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fae200099c0 tx=0x7fae20009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dd18e6c9a82460 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.623+0000 7fae11ffb700 1 -- 172.21.15.2:0/3738281012 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20003b90 con 0x555727eba020 2024-05-14T09:31:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.623+0000 7fae11ffb700 1 -- 172.21.15.2:0/3738281012 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae20003d30 con 0x555727eba020 2024-05-14T09:31:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.623+0000 7fae11ffb700 1 -- 172.21.15.2:0/3738281012 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2000bd60 con 0x555727eba020 2024-05-14T09:31:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.624+0000 7fae40427b80 1 -- 172.21.15.2:0/3738281012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727eba020 msgr2=0x555727ebe2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.624+0000 7fae40427b80 1 --2- 172.21.15.2:0/3738281012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727eba020 0x555727ebe2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fae200099c0 tx=0x7fae20009710 comp rx=0 tx=0).stop 2024-05-14T09:31:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.624+0000 7fae40427b80 1 -- 172.21.15.2:0/3738281012 shutdown_connections 2024-05-14T09:31:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.624+0000 7fae40427b80 1 --2- 172.21.15.2:0/3738281012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727d37df0 0x555727eb9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.624+0000 7fae40427b80 1 --2- 172.21.15.2:0/3738281012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727ebe830 0x555727ebec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.624+0000 7fae40427b80 1 --2- 172.21.15.2:0/3738281012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727eba020 0x555727ebe2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.624+0000 7fae40427b80 1 -- 172.21.15.2:0/3738281012 >> 172.21.15.2:0/3738281012 conn(0x555727d7f5b0 msgr2=0x555727eb0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.624+0000 7fae40427b80 1 -- 172.21.15.2:0/3738281012 shutdown_connections 2024-05-14T09:31:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.624+0000 7fae40427b80 1 -- 172.21.15.2:0/3738281012 wait complete. 2024-05-14T09:31:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.624+0000 7fae40427b80 1 -- start start 2024-05-14T09:31:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae40427b80 1 Processor -- start 2024-05-14T09:31:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae40427b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727d37df0 0x555727ec3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae40427b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727eba020 0x555727ec39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae40427b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727ebe830 0x555727ec3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae40427b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555727ec66c0 con 0x555727d37df0 2024-05-14T09:31:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae40427b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555727d3dbe0 con 0x555727ebe830 2024-05-14T09:31:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae40427b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555727ec6840 con 0x555727eba020 2024-05-14T09:31:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae40427b80 10 client.0 _ll_register_callbacks cb 0x555727d6f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:31:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae2affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727d37df0 0x555727ec3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae2affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727d37df0 0x555727ec3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42350/0 (socket says 172.21.15.2:42350) 2024-05-14T09:31:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae2affd700 1 -- 172.21.15.2:0/512384497 learned_addr learned my addr 172.21.15.2:0/512384497 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae2a7fc700 1 --2- 172.21.15.2:0/512384497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727eba020 0x555727ec39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae2b7fe700 1 --2- 172.21.15.2:0/512384497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727ebe830 0x555727ec3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae2affd700 1 -- 172.21.15.2:0/512384497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727eba020 msgr2=0x555727ec39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:31:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae2affd700 1 --2- 172.21.15.2:0/512384497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727eba020 0x555727ec39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae2affd700 1 -- 172.21.15.2:0/512384497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727ebe830 msgr2=0x555727ec3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:31:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae2affd700 1 --2- 172.21.15.2:0/512384497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727ebe830 0x555727ec3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae2affd700 1 -- 172.21.15.2:0/512384497 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555727d73640 con 0x555727d37df0 2024-05-14T09:31:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae2a7fc700 1 --2- 172.21.15.2:0/512384497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727eba020 0x555727ec39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:31:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.625+0000 7fae2b7fe700 1 --2- 172.21.15.2:0/512384497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727ebe830 0x555727ec3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:31:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.626+0000 7fae2affd700 1 --2- 172.21.15.2:0/512384497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727d37df0 0x555727ec3480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0154d0 tx=0x7fae1c004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9092aed8da0abbb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.626+0000 7fae0affd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:31:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.626+0000 7fae0affd700 1 -- 172.21.15.2:0/512384497 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c025070 con 0x555727d37df0 2024-05-14T09:31:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.626+0000 7fae40427b80 1 -- 172.21.15.2:0/512384497 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae20003d30 con 0x555727d37df0 2024-05-14T09:31:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.626+0000 7fae0affd700 1 -- 172.21.15.2:0/512384497 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fae1c015d00 con 0x555727d37df0 2024-05-14T09:31:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.626+0000 7fae0affd700 1 -- 172.21.15.2:0/512384497 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fae1c01b040 con 0x555727d37df0 2024-05-14T09:31:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.626+0000 7fae40427b80 10 client.14736 fetch_fsmap learned FSMap version 17 2024-05-14T09:31:31.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.626+0000 7fae40427b80 1 -- 172.21.15.2:0/512384497 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x555727ed12e0 con 0x555727d37df0 2024-05-14T09:31:31.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.626+0000 7fae0affd700 1 -- 172.21.15.2:0/512384497 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae20003d30 con 0x555727d37df0 2024-05-14T09:31:31.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.627+0000 7fae0affd700 1 -- 172.21.15.2:0/512384497 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fae1c006730 con 0x555727d37df0 2024-05-14T09:31:31.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.627+0000 7fae40427b80 10 client.14736 fetch_fsmap finished waiting for FSMap version 17 2024-05-14T09:31:31.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.627+0000 7fae40427b80 10 client.14736 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T09:31:31.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.627+0000 7fae40427b80 20 client.14736 populate_metadata read hostname 'smithi002' 2024-05-14T09:31:31.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.627+0000 7fae40427b80 1 --2- 172.21.15.2:0/512384497 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555727ed39b0 0x555727ed4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.627+0000 7fae40427b80 4 client.14736 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:31:31.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.627+0000 7fae40427b80 1 -- 172.21.15.2:0/512384497 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555727de51f0 con 0x555727ed39b0 2024-05-14T09:31:31.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.627+0000 7fae2a7fc700 1 --2- 172.21.15.2:0/512384497 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555727ed39b0 0x555727ed4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.628+0000 7fae2a7fc700 1 --2- 172.21.15.2:0/512384497 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555727ed39b0 0x555727ed4e20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f33f8de9d606f1f server_cookie=9e6526f80c30db5a in_seq=0 out_seq=0 2024-05-14T09:31:31.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.628+0000 7fae0affd700 10 client.14736 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:31:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.628+0000 7fae0affd700 1 -- 172.21.15.2:0/512384497 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555727de51f0 con 0x555727ed39b0 2024-05-14T09:31:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.628+0000 7fae0affd700 10 client.14736 handle_command_reply: tid=0 2024-05-14T09:31:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.629+0000 7fae40427b80 1 client.14736 shutdown 2024-05-14T09:31:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.629+0000 7fae40427b80 1 -- 172.21.15.2:0/512384497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727d37df0 msgr2=0x555727ec3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.629+0000 7fae40427b80 1 --2- 172.21.15.2:0/512384497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727d37df0 0x555727ec3480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0154d0 tx=0x7fae1c004860 comp rx=0 tx=0).stop 2024-05-14T09:31:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.629+0000 7fae40427b80 1 -- 172.21.15.2:0/512384497 shutdown_connections 2024-05-14T09:31:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.629+0000 7fae40427b80 1 --2- 172.21.15.2:0/512384497 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555727ed39b0 0x555727ed4e20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.629+0000 7fae40427b80 1 --2- 172.21.15.2:0/512384497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727ebe830 0x555727ec3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.629+0000 7fae40427b80 1 --2- 172.21.15.2:0/512384497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727d37df0 0x555727ec3480 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.629+0000 7fae40427b80 1 --2- 172.21.15.2:0/512384497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727eba020 0x555727ec39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.629+0000 7fae40427b80 1 -- 172.21.15.2:0/512384497 >> 172.21.15.2:0/512384497 conn(0x555727d7f5b0 msgr2=0x555727eb8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.629+0000 7fae0affd700 0 client.14736 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:31:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.629+0000 7fae40427b80 1 -- 172.21.15.2:0/512384497 shutdown_connections 2024-05-14T09:31:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.629+0000 7fae40427b80 1 -- 172.21.15.2:0/512384497 wait complete. 2024-05-14T09:31:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.629+0000 7fae40427b80 20 client.14736 trim_cache size 0 max 16384 2024-05-14T09:31:31.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.650+0000 7fae40427b80 1 Processor -- start 2024-05-14T09:31:31.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.650+0000 7fae40427b80 1 -- start start 2024-05-14T09:31:31.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.650+0000 7fae40427b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727d37df0 0x555727ecfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.650+0000 7fae40427b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727eba020 0x555727eb0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.650+0000 7fae40427b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727ebe830 0x555727eba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.650+0000 7fae40427b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555727ed3160 con 0x555727ebe830 2024-05-14T09:31:31.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.650+0000 7fae40427b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555727ed3830 con 0x555727d37df0 2024-05-14T09:31:31.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.650+0000 7fae40427b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555727ed3f00 con 0x555727eba020 2024-05-14T09:31:31.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.651+0000 7fae2b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727ebe830 0x555727eba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.651+0000 7fae2b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727ebe830 0x555727eba430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42360/0 (socket says 172.21.15.2:42360) 2024-05-14T09:31:31.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.651+0000 7fae2b7fe700 1 -- 172.21.15.2:0/72026219 learned_addr learned my addr 172.21.15.2:0/72026219 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.651+0000 7fae2a7fc700 1 --2- 172.21.15.2:0/72026219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727eba020 0x555727eb0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.651+0000 7fae2affd700 1 --2- 172.21.15.2:0/72026219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727d37df0 0x555727ecfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.651+0000 7fae2b7fe700 1 -- 172.21.15.2:0/72026219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727eba020 msgr2=0x555727eb0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:31:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.651+0000 7fae2b7fe700 1 --2- 172.21.15.2:0/72026219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727eba020 0x555727eb0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.651+0000 7fae2b7fe700 1 -- 172.21.15.2:0/72026219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727d37df0 msgr2=0x555727ecfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:31:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.651+0000 7fae2b7fe700 1 --2- 172.21.15.2:0/72026219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727d37df0 0x555727ecfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.651+0000 7fae2b7fe700 1 -- 172.21.15.2:0/72026219 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555727ed43f0 con 0x555727ebe830 2024-05-14T09:31:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.651+0000 7fae2b7fe700 1 --2- 172.21.15.2:0/72026219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727ebe830 0x555727eba430 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fae24004b80 tx=0x7fae240a8a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc7a4c67b07940d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.651+0000 7fae117fa700 1 -- 172.21.15.2:0/72026219 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae240a9930 con 0x555727ebe830 2024-05-14T09:31:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.651+0000 7fae117fa700 1 -- 172.21.15.2:0/72026219 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae240a9230 con 0x555727ebe830 2024-05-14T09:31:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.651+0000 7fae117fa700 1 -- 172.21.15.2:0/72026219 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24004bd0 con 0x555727ebe830 2024-05-14T09:31:31.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.652+0000 7fae40427b80 1 -- 172.21.15.2:0/72026219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727ebe830 msgr2=0x555727eba430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:31.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.652+0000 7fae40427b80 1 --2- 172.21.15.2:0/72026219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727ebe830 0x555727eba430 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fae24004b80 tx=0x7fae240a8a70 comp rx=0 tx=0).stop 2024-05-14T09:31:31.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.652+0000 7fae40427b80 1 -- 172.21.15.2:0/72026219 shutdown_connections 2024-05-14T09:31:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.652+0000 7fae40427b80 1 --2- 172.21.15.2:0/72026219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727d37df0 0x555727ecfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.652+0000 7fae40427b80 1 --2- 172.21.15.2:0/72026219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727ebe830 0x555727eba430 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.652+0000 7fae40427b80 1 --2- 172.21.15.2:0/72026219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727eba020 0x555727eb0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.652+0000 7fae40427b80 1 -- 172.21.15.2:0/72026219 >> 172.21.15.2:0/72026219 conn(0x555727d7f5b0 msgr2=0x555727eaa130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.652+0000 7fae40427b80 1 -- 172.21.15.2:0/72026219 shutdown_connections 2024-05-14T09:31:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.652+0000 7fae40427b80 1 -- 172.21.15.2:0/72026219 wait complete. 2024-05-14T09:31:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.653+0000 7fae40427b80 1 -- start start 2024-05-14T09:31:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.653+0000 7fae40427b80 1 Processor -- start 2024-05-14T09:31:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.653+0000 7fae40427b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727d37df0 0x555727ede910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.653+0000 7fae40427b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727eba020 0x555727ed9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.653+0000 7fae40427b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727ebe830 0x555727eda2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.653+0000 7fae40427b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555727ed4ca0 con 0x555727d37df0 2024-05-14T09:31:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.653+0000 7fae40427b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555727ed45d0 con 0x555727eba020 2024-05-14T09:31:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.653+0000 7fae40427b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555727ed5370 con 0x555727ebe830 2024-05-14T09:31:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.653+0000 7fae40427b80 10 client.0 _ll_register_callbacks cb 0x555727d6f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:31:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.653+0000 7fae2affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727d37df0 0x555727ede910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.653+0000 7fae2affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727d37df0 0x555727ede910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42372/0 (socket says 172.21.15.2:42372) 2024-05-14T09:31:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.653+0000 7fae2affd700 1 -- 172.21.15.2:0/3829373332 learned_addr learned my addr 172.21.15.2:0/3829373332 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.653+0000 7fae2b7fe700 1 --2- 172.21.15.2:0/3829373332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727ebe830 0x555727eda2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae2a7fc700 1 --2- 172.21.15.2:0/3829373332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727eba020 0x555727ed9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae2affd700 1 -- 172.21.15.2:0/3829373332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727ebe830 msgr2=0x555727eda2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:31:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae2affd700 1 --2- 172.21.15.2:0/3829373332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727ebe830 0x555727eda2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae2affd700 1 -- 172.21.15.2:0/3829373332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727eba020 msgr2=0x555727ed9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:31:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae2affd700 1 --2- 172.21.15.2:0/3829373332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727eba020 0x555727ed9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae2affd700 1 -- 172.21.15.2:0/3829373332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555727edaa60 con 0x555727d37df0 2024-05-14T09:31:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae2a7fc700 1 --2- 172.21.15.2:0/3829373332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727eba020 0x555727ed9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:31:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae2affd700 1 --2- 172.21.15.2:0/3829373332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727d37df0 0x555727ede910 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0134d0 tx=0x7fae1c013d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3586260b1e4167a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae0affd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:31:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae0affd700 1 -- 172.21.15.2:0/3829373332 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c03a020 con 0x555727d37df0 2024-05-14T09:31:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae40427b80 1 -- 172.21.15.2:0/3829373332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae240a9230 con 0x555727d37df0 2024-05-14T09:31:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae0affd700 1 -- 172.21.15.2:0/3829373332 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fae1c035890 con 0x555727d37df0 2024-05-14T09:31:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae0affd700 1 -- 172.21.15.2:0/3829373332 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fae1c045050 con 0x555727d37df0 2024-05-14T09:31:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae40427b80 10 client.14742 fetch_fsmap learned FSMap version 17 2024-05-14T09:31:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae40427b80 1 -- 172.21.15.2:0/3829373332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x555727f3ee60 con 0x555727d37df0 2024-05-14T09:31:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.654+0000 7fae0affd700 1 -- 172.21.15.2:0/3829373332 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae240a9230 con 0x555727d37df0 2024-05-14T09:31:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.655+0000 7fae0affd700 1 -- 172.21.15.2:0/3829373332 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fae1c02aa00 con 0x555727d37df0 2024-05-14T09:31:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.655+0000 7fae40427b80 10 client.14742 fetch_fsmap finished waiting for FSMap version 17 2024-05-14T09:31:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.655+0000 7fae40427b80 10 client.14742 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T09:31:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.655+0000 7fae40427b80 20 client.14742 populate_metadata read hostname 'smithi002' 2024-05-14T09:31:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.655+0000 7fae40427b80 1 --2- 172.21.15.2:0/3829373332 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555727f41380 0x555727f427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.655+0000 7fae40427b80 4 client.14742 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T09:31:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.655+0000 7fae40427b80 1 -- 172.21.15.2:0/3829373332 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "status"}) -- 0x555727ddf290 con 0x555727f41380 2024-05-14T09:31:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.655+0000 7fae2a7fc700 1 --2- 172.21.15.2:0/3829373332 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555727f41380 0x555727f427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae2a7fc700 1 --2- 172.21.15.2:0/3829373332 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555727f41380 0x555727f427f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=709acbce430b50f1 server_cookie=f8db32759a3287dd in_seq=0 out_seq=0 2024-05-14T09:31:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae0affd700 10 client.14742 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:31:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae0affd700 1 -- 172.21.15.2:0/3829373332 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555727ddf290 con 0x555727f41380 2024-05-14T09:31:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae0affd700 10 client.14742 handle_command_reply: tid=0 2024-05-14T09:31:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae40427b80 1 client.14742 shutdown 2024-05-14T09:31:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae40427b80 1 -- 172.21.15.2:0/3829373332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727d37df0 msgr2=0x555727ede910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae40427b80 1 --2- 172.21.15.2:0/3829373332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727d37df0 0x555727ede910 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0134d0 tx=0x7fae1c013d90 comp rx=0 tx=0).stop 2024-05-14T09:31:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae40427b80 1 -- 172.21.15.2:0/3829373332 shutdown_connections 2024-05-14T09:31:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae40427b80 1 --2- 172.21.15.2:0/3829373332 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555727f41380 0x555727f427f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae0affd700 0 client.14742 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:31:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae40427b80 1 --2- 172.21.15.2:0/3829373332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555727ebe830 0x555727eda2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae40427b80 1 --2- 172.21.15.2:0/3829373332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727d37df0 0x555727ede910 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae40427b80 1 --2- 172.21.15.2:0/3829373332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555727eba020 0x555727ed9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae40427b80 1 -- 172.21.15.2:0/3829373332 >> 172.21.15.2:0/3829373332 conn(0x555727d7f5b0 msgr2=0x555727eaa130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae40427b80 1 -- 172.21.15.2:0/3829373332 shutdown_connections 2024-05-14T09:31:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae40427b80 1 -- 172.21.15.2:0/3829373332 wait complete. 2024-05-14T09:31:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.656+0000 7fae40427b80 20 client.14742 trim_cache size 0 max 16384 2024-05-14T09:31:31.661 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:31:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "50d5aebe-11d3-11ef-bc98-c7b262605968", 2024-05-14T09:31:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2024-05-14T09:31:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24524, 2024-05-14T09:31:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T09:31:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T09:31:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T09:31:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 32.870708606000001, 2024-05-14T09:31:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 17, 2024-05-14T09:31:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 88, 2024-05-14T09:31:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 86, 2024-05-14T09:31:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 47.961128610999999 2024-05-14T09:31:31.662 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:31:31.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.657+0000 7fae31a13700 1 -- 172.21.15.2:0/2491679129 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fae140b2880 msgr2=0x7fae140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:31.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.657+0000 7fae31a13700 1 --2- 172.21.15.2:0/2491679129 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fae140b2880 0x7fae140b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009a10 tx=0x7fae1c009190 comp rx=0 tx=0).stop 2024-05-14T09:31:31.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.657+0000 7fae31a13700 1 -- 172.21.15.2:0/2491679129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae2c108740 msgr2=0x7fae2c19f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:31.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.657+0000 7fae31a13700 1 --2- 172.21.15.2:0/2491679129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae2c108740 0x7fae2c19f6e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fae24007ee0 tx=0x7fae2400d3b0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.657+0000 7fae31a13700 1 -- 172.21.15.2:0/2491679129 shutdown_connections 2024-05-14T09:31:31.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.657+0000 7fae31a13700 1 --2- 172.21.15.2:0/2491679129 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fae140b2880 0x7fae140b4d90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.657+0000 7fae31a13700 1 --2- 172.21.15.2:0/2491679129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae2c1034c0 0x7fae2c19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.657+0000 7fae31a13700 1 --2- 172.21.15.2:0/2491679129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae2c108740 0x7fae2c19f6e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.657+0000 7fae31a13700 1 --2- 172.21.15.2:0/2491679129 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae2c107df0 0x7fae2c19b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:31.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.657+0000 7fae31a13700 1 -- 172.21.15.2:0/2491679129 >> 172.21.15.2:0/2491679129 conn(0x7fae2c0fedf0 msgr2=0x7fae2c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:31.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.657+0000 7fae31a13700 1 -- 172.21.15.2:0/2491679129 shutdown_connections 2024-05-14T09:31:31.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:31.657+0000 7fae31a13700 1 -- 172.21.15.2:0/2491679129 wait complete. 2024-05-14T09:31:32.293 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:32 smithi002 ceph-mon[31557]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-14T09:31:32.324 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-14T09:31:32.324 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-14T09:31:32.324 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-14T09:31:32.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:32 smithi029 ceph-mon[36249]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-14T09:31:32.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:32 smithi026 ceph-mon[36056]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-14T09:31:32.577 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:34.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:34 smithi002 ceph-mon[31557]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s, 0 objects/s recovering 2024-05-14T09:31:34.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.288+0000 7f3b4ddf6700 1 -- 172.21.15.2:0/639601630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b48108760 msgr2=0x7f3b481100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:34.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.288+0000 7f3b4ddf6700 1 --2- 172.21.15.2:0/639601630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b48108760 0x7f3b481100c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000b0d0 tx=0x7f3b4000b3e0 comp rx=0 tx=0).stop 2024-05-14T09:31:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.288+0000 7f3b4ddf6700 1 -- 172.21.15.2:0/639601630 shutdown_connections 2024-05-14T09:31:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.288+0000 7f3b4ddf6700 1 --2- 172.21.15.2:0/639601630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b481034e0 0x7f3b481038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.288+0000 7f3b4ddf6700 1 --2- 172.21.15.2:0/639601630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b48108760 0x7f3b481100c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.288+0000 7f3b4ddf6700 1 --2- 172.21.15.2:0/639601630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b48107e10 0x7f3b48108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.288+0000 7f3b4ddf6700 1 -- 172.21.15.2:0/639601630 >> 172.21.15.2:0/639601630 conn(0x7f3b480fef30 msgr2=0x7f3b48101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.288+0000 7f3b4ddf6700 1 -- 172.21.15.2:0/639601630 shutdown_connections 2024-05-14T09:31:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.288+0000 7f3b4ddf6700 1 -- 172.21.15.2:0/639601630 wait complete. 2024-05-14T09:31:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.289+0000 7f3b4ddf6700 1 Processor -- start 2024-05-14T09:31:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.289+0000 7f3b4ddf6700 1 -- start start 2024-05-14T09:31:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.289+0000 7f3b4ddf6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b481034e0 0x7f3b4819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.289+0000 7f3b4ddf6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b48107e10 0x7f3b4819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b4ddf6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b48108760 0x7f3b4819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b4ddf6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3b48113410 con 0x7f3b481034e0 2024-05-14T09:31:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b4ddf6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3b48113290 con 0x7f3b48108760 2024-05-14T09:31:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b4ddf6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3b48113590 con 0x7f3b48107e10 2024-05-14T09:31:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b481034e0 0x7f3b4819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b481034e0 0x7f3b4819bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42384/0 (socket says 172.21.15.2:42384) 2024-05-14T09:31:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b477fe700 1 -- 172.21.15.2:0/282926778 learned_addr learned my addr 172.21.15.2:0/282926778 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b47fff700 1 --2- 172.21.15.2:0/282926778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b48108760 0x7f3b4819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b46ffd700 1 --2- 172.21.15.2:0/282926778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b48107e10 0x7f3b4819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b477fe700 1 -- 172.21.15.2:0/282926778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b48107e10 msgr2=0x7f3b4819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b477fe700 1 --2- 172.21.15.2:0/282926778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b48107e10 0x7f3b4819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b477fe700 1 -- 172.21.15.2:0/282926778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b48108760 msgr2=0x7f3b4819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b477fe700 1 --2- 172.21.15.2:0/282926778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b48108760 0x7f3b4819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b477fe700 1 -- 172.21.15.2:0/282926778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b481a70a0 con 0x7f3b481034e0 2024-05-14T09:31:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b477fe700 1 --2- 172.21.15.2:0/282926778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b481034e0 0x7f3b4819bd50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000b530 tx=0x7f3b3000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b44ff9700 1 -- 172.21.15.2:0/282926778 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30015020 con 0x7f3b481034e0 2024-05-14T09:31:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b44ff9700 1 -- 172.21.15.2:0/282926778 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b30011ce0 con 0x7f3b481034e0 2024-05-14T09:31:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b4ddf6700 1 -- 172.21.15.2:0/282926778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b481a7390 con 0x7f3b481034e0 2024-05-14T09:31:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.290+0000 7f3b44ff9700 1 -- 172.21.15.2:0/282926778 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30010dd0 con 0x7f3b481034e0 2024-05-14T09:31:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.291+0000 7f3b4ddf6700 1 -- 172.21.15.2:0/282926778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b481a78d0 con 0x7f3b481034e0 2024-05-14T09:31:34.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.291+0000 7f3b4ddf6700 1 -- 172.21.15.2:0/282926778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b480512c0 con 0x7f3b481034e0 2024-05-14T09:31:34.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.294+0000 7f3b44ff9700 1 -- 172.21.15.2:0/282926778 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f3b30018020 con 0x7f3b481034e0 2024-05-14T09:31:34.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.295+0000 7f3b44ff9700 1 --2- 172.21.15.2:0/282926778 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3b340b2880 0x7f3b340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:34.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.295+0000 7f3b46ffd700 1 --2- 172.21.15.2:0/282926778 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3b340b2880 0x7f3b340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:34.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.295+0000 7f3b44ff9700 1 -- 172.21.15.2:0/282926778 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f3b300e2200 con 0x7f3b481034e0 2024-05-14T09:31:34.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.295+0000 7f3b46ffd700 1 --2- 172.21.15.2:0/282926778 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3b340b2880 0x7f3b340b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3b481066e0 tx=0x7f3b3800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:34.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.298+0000 7f3b44ff9700 1 -- 172.21.15.2:0/282926778 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3b3001f020 con 0x7f3b481034e0 2024-05-14T09:31:34.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:34 smithi029 ceph-mon[36249]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s, 0 objects/s recovering 2024-05-14T09:31:34.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:34 smithi026 ceph-mon[36056]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s, 0 objects/s recovering 2024-05-14T09:31:34.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.561+0000 7f3b4ddf6700 1 -- 172.21.15.2:0/282926778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3b4802e5f0 con 0x7f3b481034e0 2024-05-14T09:31:34.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.561+0000 7f3b44ff9700 1 -- 172.21.15.2:0/282926778 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f3b300a9b80 con 0x7f3b481034e0 2024-05-14T09:31:34.561 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-14T09:31:34.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.564+0000 7f3b4ddf6700 1 -- 172.21.15.2:0/282926778 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3b340b2880 msgr2=0x7f3b340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:34.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.564+0000 7f3b4ddf6700 1 --2- 172.21.15.2:0/282926778 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3b340b2880 0x7f3b340b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3b481066e0 tx=0x7f3b3800a300 comp rx=0 tx=0).stop 2024-05-14T09:31:34.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.564+0000 7f3b4ddf6700 1 -- 172.21.15.2:0/282926778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b481034e0 msgr2=0x7f3b4819bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:34.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.564+0000 7f3b4ddf6700 1 --2- 172.21.15.2:0/282926778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b481034e0 0x7f3b4819bd50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000b530 tx=0x7f3b3000b840 comp rx=0 tx=0).stop 2024-05-14T09:31:34.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.564+0000 7f3b4ddf6700 1 -- 172.21.15.2:0/282926778 shutdown_connections 2024-05-14T09:31:34.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.564+0000 7f3b4ddf6700 1 --2- 172.21.15.2:0/282926778 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3b340b2880 0x7f3b340b4d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:34.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.564+0000 7f3b4ddf6700 1 --2- 172.21.15.2:0/282926778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b48108760 0x7f3b4819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:34.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.564+0000 7f3b4ddf6700 1 --2- 172.21.15.2:0/282926778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b481034e0 0x7f3b4819bd50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:34.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.564+0000 7f3b4ddf6700 1 --2- 172.21.15.2:0/282926778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3b48107e10 0x7f3b4819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:34.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.564+0000 7f3b4ddf6700 1 -- 172.21.15.2:0/282926778 >> 172.21.15.2:0/282926778 conn(0x7f3b480fef30 msgr2=0x7f3b48107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:34.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.564+0000 7f3b4ddf6700 1 -- 172.21.15.2:0/282926778 shutdown_connections 2024-05-14T09:31:34.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:34.564+0000 7f3b4ddf6700 1 -- 172.21.15.2:0/282926778 wait complete. 2024-05-14T09:31:35.198 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-14T09:31:35.199 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T09:31:35.199 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi002.front.sepia.ceph.com 2024-05-14T09:31:35.199 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-14T09:31:35.199 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-14T09:31:35.199 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-14T09:31:35.199 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T09:31:35.199 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T09:31:35.199 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-14T09:31:35.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:31:35.199 DEBUG:teuthology.orchestra.run.smithi002:> ip addr 2024-05-14T09:31:35.217 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T09:31:35.217 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T09:31:35.217 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T09:31:35.217 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-05-14T09:31:35.217 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2024-05-14T09:31:35.217 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-05-14T09:31:35.217 INFO:teuthology.orchestra.run.smithi002.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T09:31:35.217 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:02 brd ff:ff:ff:ff:ff:ff 2024-05-14T09:31:35.217 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f0 2024-05-14T09:31:35.217 INFO:teuthology.orchestra.run.smithi002.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T09:31:35.218 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:03 brd ff:ff:ff:ff:ff:ff 2024-05-14T09:31:35.218 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f1 2024-05-14T09:31:35.218 INFO:teuthology.orchestra.run.smithi002.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T09:31:35.218 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:20 brd ff:ff:ff:ff:ff:ff 2024-05-14T09:31:35.218 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f0 2024-05-14T09:31:35.218 INFO:teuthology.orchestra.run.smithi002.stdout: inet 172.21.15.2/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-14T09:31:35.218 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft 42119sec preferred_lft 42119sec 2024-05-14T09:31:35.218 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::ec4:7aff:febd:1920/64 scope link noprefixroute 2024-05-14T09:31:35.218 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-05-14T09:31:35.218 INFO:teuthology.orchestra.run.smithi002.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T09:31:35.218 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:21 brd ff:ff:ff:ff:ff:ff 2024-05-14T09:31:35.218 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f1 2024-05-14T09:31:35.219 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-14T09:31:35.219 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T09:31:35.219 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-05-14T09:31:35.219 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add name ceph-brx type bridge 2024-05-14T09:31:35.219 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr flush dev ceph-brx 2024-05-14T09:31:35.219 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set ceph-brx up 2024-05-14T09:31:35.219 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-14T09:31:35.219 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-05-14T09:31:35.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T09:31:35.420 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:35 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/282926778' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T09:31:35.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:31:35.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:35 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/282926778' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T09:31:35.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:31:35.442 DEBUG:teuthology.orchestra.run.smithi002:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T09:31:35.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:35 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/282926778' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T09:31:35.468 INFO:teuthology.orchestra.run.smithi002.stdout:1 2024-05-14T09:31:35.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:31:35.470 DEBUG:teuthology.orchestra.run.smithi002:> route 2024-05-14T09:31:35.537 INFO:teuthology.orchestra.run.smithi002.stdout:Kernel IP routing table 2024-05-14T09:31:35.537 INFO:teuthology.orchestra.run.smithi002.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T09:31:35.537 INFO:teuthology.orchestra.run.smithi002.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-14T09:31:35.537 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-14T09:31:35.537 INFO:teuthology.orchestra.run.smithi002.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T09:31:35.537 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T09:31:35.537 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-05-14T09:31:35.537 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-14T09:31:35.538 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-14T09:31:35.538 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-14T09:31:35.538 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-05-14T09:31:35.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T09:31:35.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:31:35.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:31:35.852 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-05-14T09:31:35.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:31:35.869 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list-id 2024-05-14T09:31:35.924 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T09:31:35.924 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-05-14T09:31:35.925 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T09:31:35.925 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-14T09:31:35.925 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-05-14T09:31:36.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T09:31:36.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:31:36.046 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-14T09:31:36.047 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T09:31:36.047 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-05-14T09:31:36.047 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-14T09:31:36.047 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-14T09:31:36.047 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-14T09:31:36.047 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-14T09:31:36.047 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-14T09:31:36.047 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-05-14T09:31:36.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T09:31:36.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:31:36.177 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T09:31:36.178 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-05-14T09:31:36.178 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set brx.0 up 2024-05-14T09:31:36.178 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set dev brx.0 master ceph-brx 2024-05-14T09:31:36.178 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-05-14T09:31:36.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T09:31:36.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:31:36.258 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-14T09:31:36.258 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T09:31:36.258 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-14T09:31:36.274 INFO:teuthology.orchestra.run.smithi002.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-14T09:31:36.274 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T09:31:36.274 DEBUG:teuthology.orchestra.run.smithi002:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-14T09:31:36.330 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2024-05-14T09:31:36.393 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:36 smithi002 ceph-mon[31557]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s, 0 objects/s recovering 2024-05-14T09:31:36.397 DEBUG:teuthology.orchestra.run.smithi002:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T09:31:36.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:36 smithi029 ceph-mon[36249]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s, 0 objects/s recovering 2024-05-14T09:31:36.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:36 smithi026 ceph-mon[36056]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s, 0 objects/s recovering 2024-05-14T09:31:36.454 INFO:teuthology.orchestra.run.smithi002.stdout:/proc 2024-05-14T09:31:36.454 INFO:teuthology.orchestra.run.smithi002.stdout:/sys 2024-05-14T09:31:36.454 INFO:teuthology.orchestra.run.smithi002.stdout:/dev 2024-05-14T09:31:36.454 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/security 2024-05-14T09:31:36.454 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/shm 2024-05-14T09:31:36.454 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/pts 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/run 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/pstore 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/bpf 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/ 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/selinux 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/mqueue 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/hugepages 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/debug 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/tracing 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/fuse/connections 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/config 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-sysctl.service 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-14T09:31:36.455 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/nfs/rpc_pipefs 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/1000 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/ceph 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/d3845f32443d93d840c9e7c075cbc2e63890eeeeb826535b6b7a1b59f11a8af2/merged 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f3564f39eb0ccb4334754840b560fc084ad97e3f0c7182efb8832e04738ba2d2/merged 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/0 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/26ddc5c72de0dfdfe0f6afff12d2986882ee8cdbc6e9ce04d13ef047b9ba0af2/merged 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/8d9f499420fc82e8c9c434b1804b50a6f7caedc103ac2963de8ee990936dd096/merged 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/865d157079ded1f0608cfd392a71f6a394c9eb4c66fed870bf299ee0f6e7d305/merged 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/658bd59d389641a37b5b84b36e0c1fd8b0d1ac7548f97dd369ce0a41d05852d5/merged 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/16eb213df9a547367cc4be26f2718248d777e59fe9354ce45ac9a0db1ba8b26d/merged 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/0e5f1c3b30c22a7789aa02867831843eccf489fdf642c3ef838106c1f4f279f2/merged 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/494885cfbc4bd3a8d050c7e5ebeaea673b35440fb700ab8bc6e8cdd428f89ad2/merged 2024-05-14T09:31:36.456 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/3c39ff43924d545676eccd20bf8eade336b0bbcb5c8e578ce3a9ac99d4278084/merged 2024-05-14T09:31:36.457 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns 2024-05-14T09:31:36.457 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T09:31:36.457 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T09:31:36.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:31:36.457 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2024-05-14T09:31:36.510 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T09:31:36.510 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-14T09:31:36.553 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2024-05-14T09:31:36.578 DEBUG:teuthology.orchestra.run.smithi002:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T09:31:36.635 INFO:teuthology.orchestra.run.smithi002.stdout:/proc 2024-05-14T09:31:36.635 INFO:teuthology.orchestra.run.smithi002.stdout:/sys 2024-05-14T09:31:36.635 INFO:teuthology.orchestra.run.smithi002.stdout:/dev 2024-05-14T09:31:36.635 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/security 2024-05-14T09:31:36.635 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/shm 2024-05-14T09:31:36.635 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/pts 2024-05-14T09:31:36.635 INFO:teuthology.orchestra.run.smithi002.stdout:/run 2024-05-14T09:31:36.635 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup 2024-05-14T09:31:36.635 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/pstore 2024-05-14T09:31:36.636 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/bpf 2024-05-14T09:31:36.636 INFO:teuthology.orchestra.run.smithi002.stdout:/ 2024-05-14T09:31:36.636 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/selinux 2024-05-14T09:31:36.636 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T09:31:36.636 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/mqueue 2024-05-14T09:31:36.636 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/hugepages 2024-05-14T09:31:36.636 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/debug 2024-05-14T09:31:36.636 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/tracing 2024-05-14T09:31:36.636 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/fuse/connections 2024-05-14T09:31:36.636 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/config 2024-05-14T09:31:36.636 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-sysctl.service 2024-05-14T09:31:36.636 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-14T09:31:36.636 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-14T09:31:36.636 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/nfs/rpc_pipefs 2024-05-14T09:31:36.636 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/1000 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/ceph 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/d3845f32443d93d840c9e7c075cbc2e63890eeeeb826535b6b7a1b59f11a8af2/merged 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f3564f39eb0ccb4334754840b560fc084ad97e3f0c7182efb8832e04738ba2d2/merged 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/0 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/26ddc5c72de0dfdfe0f6afff12d2986882ee8cdbc6e9ce04d13ef047b9ba0af2/merged 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/8d9f499420fc82e8c9c434b1804b50a6f7caedc103ac2963de8ee990936dd096/merged 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/865d157079ded1f0608cfd392a71f6a394c9eb4c66fed870bf299ee0f6e7d305/merged 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/658bd59d389641a37b5b84b36e0c1fd8b0d1ac7548f97dd369ce0a41d05852d5/merged 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/16eb213df9a547367cc4be26f2718248d777e59fe9354ce45ac9a0db1ba8b26d/merged 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/0e5f1c3b30c22a7789aa02867831843eccf489fdf642c3ef838106c1f4f279f2/merged 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/494885cfbc4bd3a8d050c7e5ebeaea673b35440fb700ab8bc6e8cdd428f89ad2/merged 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/3c39ff43924d545676eccd20bf8eade336b0bbcb5c8e578ce3a9ac99d4278084/merged 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T09:31:36.637 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T09:31:36.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:31:36.638 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2024-05-14T09:31:36.684 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:2024-05-14T09:31:36.685+0000 7f68bac10480 -1 init, newargv = 0x7f689c004720 newargc=15 2024-05-14T09:31:36.685 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:ceph-fuse[74716]: starting ceph client 2024-05-14T09:31:36.692 INFO:teuthology.orchestra.run.smithi002.stdout:107 2024-05-14T09:31:36.692 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [107] 2024-05-14T09:31:36.693 DEBUG:teuthology.orchestra.run.smithi002:> sudo stdin-killer -- python3 -c ' 2024-05-14T09:31:36.693 DEBUG:teuthology.orchestra.run.smithi002:> import glob 2024-05-14T09:31:36.693 DEBUG:teuthology.orchestra.run.smithi002:> import re 2024-05-14T09:31:36.693 DEBUG:teuthology.orchestra.run.smithi002:> import os 2024-05-14T09:31:36.693 DEBUG:teuthology.orchestra.run.smithi002:> import subprocess 2024-05-14T09:31:36.693 DEBUG:teuthology.orchestra.run.smithi002:> 2024-05-14T09:31:36.693 DEBUG:teuthology.orchestra.run.smithi002:> def _find_admin_socket(client_name): 2024-05-14T09:31:36.693 DEBUG:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-14T09:31:36.693 DEBUG:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2024-05-14T09:31:36.693 DEBUG:teuthology.orchestra.run.smithi002:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-14T09:31:36.693 DEBUG:teuthology.orchestra.run.smithi002:> 2024-05-14T09:31:36.693 DEBUG:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2024-05-14T09:31:36.693 DEBUG:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2024-05-14T09:31:36.693 DEBUG:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2024-05-14T09:31:36.693 DEBUG:teuthology.orchestra.run.smithi002:> return files[0] 2024-05-14T09:31:36.694 DEBUG:teuthology.orchestra.run.smithi002:> 2024-05-14T09:31:36.694 DEBUG:teuthology.orchestra.run.smithi002:> for f in files: 2024-05-14T09:31:36.694 DEBUG:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T09:31:36.694 DEBUG:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T09:31:36.694 DEBUG:teuthology.orchestra.run.smithi002:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T09:31:36.694 DEBUG:teuthology.orchestra.run.smithi002:> contents = proc_f.read() 2024-05-14T09:31:36.694 DEBUG:teuthology.orchestra.run.smithi002:> if mountpoint in contents: 2024-05-14T09:31:36.694 DEBUG:teuthology.orchestra.run.smithi002:> return f 2024-05-14T09:31:36.694 DEBUG:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T09:31:36.694 DEBUG:teuthology.orchestra.run.smithi002:> 2024-05-14T09:31:36.694 DEBUG:teuthology.orchestra.run.smithi002:> print(_find_admin_socket("client.admin")) 2024-05-14T09:31:36.694 DEBUG:teuthology.orchestra.run.smithi002:> ' 2024-05-14T09:31:36.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:ceph-fuse[74716]: starting fuse 2024-05-14T09:31:36.816 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.admin.74716.asok 2024-05-14T09:31:36.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:31:36.828 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74716.asok 2024-05-14T09:31:36.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:31:36.828 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74716.asok status 2024-05-14T09:31:36.975 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:31:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2024-05-14T09:31:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T09:31:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T09:31:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "admin", 2024-05-14T09:31:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2024-05-14T09:31:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-14T09:31:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "74716", 2024-05-14T09:31:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/" 2024-05-14T09:31:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:31:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2024-05-14T09:31:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2024-05-14T09:31:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 14754, 2024-05-14T09:31:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2024-05-14T09:31:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2024-05-14T09:31:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2024-05-14T09:31:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 14754 2024-05-14T09:31:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:31:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2024-05-14T09:31:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-05-14T09:31:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "192.168.0.1:0", 2024-05-14T09:31:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1023300866 2024-05-14T09:31:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T09:31:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:31:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2024-05-14T09:31:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-05-14T09:31:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "192.168.0.1:0", 2024-05-14T09:31:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1023300866 2024-05-14T09:31:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:31:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.14754 192.168.0.1:0/1023300866", 2024-05-14T09:31:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "192.168.0.1:0/1023300866", 2024-05-14T09:31:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 1, 2024-05-14T09:31:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 17, 2024-05-14T09:31:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 88, 2024-05-14T09:31:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2024-05-14T09:31:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklisted": false, 2024-05-14T09:31:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs" 2024-05-14T09:31:36.978 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:31:36.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:31:36.984 DEBUG:teuthology.orchestra.run.smithi002:> stat --file-system '--printf=%T 2024-05-14T09:31:36.984 DEBUG:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-14T09:31:37.000 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2024-05-14T09:31:37.001 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-14T09:31:37.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:31:37.001 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-14T09:31:37.067 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-14T09:31:37.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T09:31:37.184 INFO:teuthology.orchestra.run.smithi002.stdout:# file: . 2024-05-14T09:31:37.184 INFO:teuthology.orchestra.run.smithi002.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-14T09:31:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:31:37.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:31:37.191 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:31:37.191 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T09:31:37.208 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-14T09:31:37.498 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:38.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:38 smithi026 ceph-mon[36056]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s, 0 objects/s recovering 2024-05-14T09:31:38.641 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:38 smithi002 ceph-mon[31557]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s, 0 objects/s recovering 2024-05-14T09:31:38.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:38 smithi029 ceph-mon[36249]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s, 0 objects/s recovering 2024-05-14T09:31:39.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.211+0000 7fadf4e0b700 1 -- 172.21.15.2:0/599756674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadf0107df0 msgr2=0x7fadf0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:39.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.211+0000 7fadf4e0b700 1 --2- 172.21.15.2:0/599756674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadf0107df0 0x7fadf0108200 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fadd80099c0 tx=0x7fadd8009cd0 comp rx=0 tx=0).stop 2024-05-14T09:31:39.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.212+0000 7fadf4e0b700 1 -- 172.21.15.2:0/599756674 shutdown_connections 2024-05-14T09:31:39.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.212+0000 7fadf4e0b700 1 --2- 172.21.15.2:0/599756674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf01034c0 0x7fadf01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:39.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.212+0000 7fadf4e0b700 1 --2- 172.21.15.2:0/599756674 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadf0108740 0x7fadf01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:39.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.212+0000 7fadf4e0b700 1 --2- 172.21.15.2:0/599756674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadf0107df0 0x7fadf0108200 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:39.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.212+0000 7fadf4e0b700 1 -- 172.21.15.2:0/599756674 >> 172.21.15.2:0/599756674 conn(0x7fadf00fedc0 msgr2=0x7fadf01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:39.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.212+0000 7fadf4e0b700 1 -- 172.21.15.2:0/599756674 shutdown_connections 2024-05-14T09:31:39.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.212+0000 7fadf4e0b700 1 -- 172.21.15.2:0/599756674 wait complete. 2024-05-14T09:31:39.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.213+0000 7fadf4e0b700 1 Processor -- start 2024-05-14T09:31:39.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.213+0000 7fadf4e0b700 1 -- start start 2024-05-14T09:31:39.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.213+0000 7fadf4e0b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadf01034c0 0x7fadf019bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:39.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.213+0000 7fadf4e0b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadf0107df0 0x7fadf019c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:39.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.213+0000 7fadf4e0b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf0108740 0x7fadf019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:39.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.213+0000 7fadf4e0b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fadf01133a0 con 0x7fadf01034c0 2024-05-14T09:31:39.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.213+0000 7fadf4e0b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0113220 con 0x7fadf0108740 2024-05-14T09:31:39.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.213+0000 7fadf4e0b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0113520 con 0x7fadf0107df0 2024-05-14T09:31:39.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.213+0000 7fadee59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadf01034c0 0x7fadf019bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:39.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.213+0000 7fadee59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadf01034c0 0x7fadf019bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59140/0 (socket says 172.21.15.2:59140) 2024-05-14T09:31:39.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.213+0000 7fadee59c700 1 -- 172.21.15.2:0/3289055553 learned_addr learned my addr 172.21.15.2:0/3289055553 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:39.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.213+0000 7fadedd9b700 1 --2- 172.21.15.2:0/3289055553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadf0107df0 0x7fadf019c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:39.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.214+0000 7fadeed9d700 1 --2- 172.21.15.2:0/3289055553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf0108740 0x7fadf019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:39.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.214+0000 7fadee59c700 1 -- 172.21.15.2:0/3289055553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadf0107df0 msgr2=0x7fadf019c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:39.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.214+0000 7fadee59c700 1 --2- 172.21.15.2:0/3289055553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadf0107df0 0x7fadf019c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:39.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.214+0000 7fadee59c700 1 -- 172.21.15.2:0/3289055553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf0108740 msgr2=0x7fadf019c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:39.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.214+0000 7fadee59c700 1 --2- 172.21.15.2:0/3289055553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf0108740 0x7fadf019c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:39.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.214+0000 7fadee59c700 1 -- 172.21.15.2:0/3289055553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf01a6ff0 con 0x7fadf01034c0 2024-05-14T09:31:39.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.214+0000 7fadeed9d700 1 --2- 172.21.15.2:0/3289055553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf0108740 0x7fadf019c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:31:39.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.214+0000 7fadee59c700 1 --2- 172.21.15.2:0/3289055553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadf01034c0 0x7fadf019bca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fade000ada0 tx=0x7fade000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:39.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.214+0000 7fade77fe700 1 -- 172.21.15.2:0/3289055553 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0017020 con 0x7fadf01034c0 2024-05-14T09:31:39.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.214+0000 7fadf4e0b700 1 -- 172.21.15.2:0/3289055553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf01a72e0 con 0x7fadf01034c0 2024-05-14T09:31:39.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.214+0000 7fade77fe700 1 -- 172.21.15.2:0/3289055553 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade000ce30 con 0x7fadf01034c0 2024-05-14T09:31:39.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.214+0000 7fade77fe700 1 -- 172.21.15.2:0/3289055553 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0005420 con 0x7fadf01034c0 2024-05-14T09:31:39.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.214+0000 7fadf4e0b700 1 -- 172.21.15.2:0/3289055553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf01a7840 con 0x7fadf01034c0 2024-05-14T09:31:39.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.215+0000 7fadf4e0b700 1 -- 172.21.15.2:0/3289055553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf00512c0 con 0x7fadf01034c0 2024-05-14T09:31:39.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.217+0000 7fade77fe700 1 -- 172.21.15.2:0/3289055553 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fade0004ad0 con 0x7fadf01034c0 2024-05-14T09:31:39.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.218+0000 7fade77fe700 1 --2- 172.21.15.2:0/3289055553 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7faddc0b2880 0x7faddc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:39.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.218+0000 7fadedd9b700 1 --2- 172.21.15.2:0/3289055553 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7faddc0b2880 0x7faddc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:39.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.218+0000 7fade77fe700 1 -- 172.21.15.2:0/3289055553 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fade00e2340 con 0x7fadf01034c0 2024-05-14T09:31:39.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.219+0000 7fadedd9b700 1 --2- 172.21.15.2:0/3289055553 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7faddc0b2880 0x7faddc0b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fadd8006010 tx=0x7fadd8009710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:39.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.221+0000 7fade77fe700 1 -- 172.21.15.2:0/3289055553 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fade0021020 con 0x7fadf01034c0 2024-05-14T09:31:39.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.455+0000 7fadf4e0b700 1 -- 172.21.15.2:0/3289055553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fadf002e5f0 con 0x7fadf01034c0 2024-05-14T09:31:39.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.455+0000 7fade77fe700 1 -- 172.21.15.2:0/3289055553 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+581 (secure 0 0 0) 0x7fade00a9d50 con 0x7fadf01034c0 2024-05-14T09:31:39.455 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/4051922409 2024-05-15T09:23:28.940670+0000 2024-05-14T09:31:39.455 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2353790181 2024-05-15T09:23:28.940670+0000 2024-05-14T09:31:39.455 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/4268171474 2024-05-15T09:23:28.940670+0000 2024-05-14T09:31:39.455 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/1423742578 2024-05-15T09:23:28.940670+0000 2024-05-14T09:31:39.456 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/969850441 2024-05-15T09:23:02.498446+0000 2024-05-14T09:31:39.456 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3252991268 2024-05-15T09:23:02.498446+0000 2024-05-14T09:31:39.456 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3593583257 2024-05-15T09:23:02.498446+0000 2024-05-14T09:31:39.456 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3593583257 2024-05-15T09:23:02.498446+0000 2024-05-14T09:31:39.456 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3789067476 2024-05-15T09:23:02.498446+0000 2024-05-14T09:31:39.456 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/1423742578 2024-05-15T09:23:28.940670+0000 2024-05-14T09:31:39.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.458+0000 7fadf4e0b700 1 -- 172.21.15.2:0/3289055553 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7faddc0b2880 msgr2=0x7faddc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:39.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.458+0000 7fadf4e0b700 1 --2- 172.21.15.2:0/3289055553 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7faddc0b2880 0x7faddc0b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fadd8006010 tx=0x7fadd8009710 comp rx=0 tx=0).stop 2024-05-14T09:31:39.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.458+0000 7fadf4e0b700 1 -- 172.21.15.2:0/3289055553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadf01034c0 msgr2=0x7fadf019bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:39.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.458+0000 7fadf4e0b700 1 --2- 172.21.15.2:0/3289055553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadf01034c0 0x7fadf019bca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fade000ada0 tx=0x7fade000c330 comp rx=0 tx=0).stop 2024-05-14T09:31:39.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.458+0000 7fadf4e0b700 1 -- 172.21.15.2:0/3289055553 shutdown_connections 2024-05-14T09:31:39.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.458+0000 7fadf4e0b700 1 --2- 172.21.15.2:0/3289055553 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7faddc0b2880 0x7faddc0b4d90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:39.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.458+0000 7fadf4e0b700 1 --2- 172.21.15.2:0/3289055553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadf0108740 0x7fadf019c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:39.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.458+0000 7fadf4e0b700 1 --2- 172.21.15.2:0/3289055553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadf01034c0 0x7fadf019bca0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:39.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.458+0000 7fadf4e0b700 1 --2- 172.21.15.2:0/3289055553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadf0107df0 0x7fadf019c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:39.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.458+0000 7fadf4e0b700 1 -- 172.21.15.2:0/3289055553 >> 172.21.15.2:0/3289055553 conn(0x7fadf00fedc0 msgr2=0x7fadf01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:39.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.458+0000 7fadf4e0b700 1 -- 172.21.15.2:0/3289055553 shutdown_connections 2024-05-14T09:31:39.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:39.458+0000 7fadf4e0b700 1 -- 172.21.15.2:0/3289055553 wait complete. 2024-05-14T09:31:39.460 INFO:teuthology.orchestra.run.smithi002.stderr:listed 10 entries 2024-05-14T09:31:40.126 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:31:40.127 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T09:31:40.145 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-14T09:31:40.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:40 smithi002 ceph-mon[31557]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 414 B/s wr, 0 op/s; 221 B/s, 0 objects/s recovering 2024-05-14T09:31:40.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:40 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3289055553' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T09:31:40.439 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:40.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:40 smithi026 ceph-mon[36056]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 414 B/s wr, 0 op/s; 221 B/s, 0 objects/s recovering 2024-05-14T09:31:40.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:40 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3289055553' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T09:31:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:40 smithi029 ceph-mon[36249]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 414 B/s wr, 0 op/s; 221 B/s, 0 objects/s recovering 2024-05-14T09:31:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:40 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3289055553' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T09:31:42.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.108+0000 7fedadb99700 1 -- 172.21.15.2:0/1831682664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8103550 msgr2=0x7feda8103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:42.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.108+0000 7fedadb99700 1 --2- 172.21.15.2:0/1831682664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8103550 0x7feda8103960 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fed980099c0 tx=0x7fed98009cd0 comp rx=0 tx=0).stop 2024-05-14T09:31:42.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.108+0000 7fedadb99700 1 -- 172.21.15.2:0/1831682664 shutdown_connections 2024-05-14T09:31:42.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.108+0000 7fedadb99700 1 --2- 172.21.15.2:0/1831682664 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda8108780 0x7feda81100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:42.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.108+0000 7fedadb99700 1 --2- 172.21.15.2:0/1831682664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8103550 0x7feda8103960 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:42.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.108+0000 7fedadb99700 1 --2- 172.21.15.2:0/1831682664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda8107e30 0x7feda8108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:42.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.108+0000 7fedadb99700 1 -- 172.21.15.2:0/1831682664 >> 172.21.15.2:0/1831682664 conn(0x7feda8078e00 msgr2=0x7feda8079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:42.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.108+0000 7fedadb99700 1 -- 172.21.15.2:0/1831682664 shutdown_connections 2024-05-14T09:31:42.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.108+0000 7fedadb99700 1 -- 172.21.15.2:0/1831682664 wait complete. 2024-05-14T09:31:42.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.109+0000 7fedadb99700 1 Processor -- start 2024-05-14T09:31:42.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.109+0000 7fedadb99700 1 -- start start 2024-05-14T09:31:42.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.109+0000 7fedadb99700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda8103550 0x7feda8199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:42.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.109+0000 7fedadb99700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8107e30 0x7feda8199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:42.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.109+0000 7fedadb99700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda8108780 0x7feda819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:42.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.109+0000 7fedadb99700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7feda81133b0 con 0x7feda8107e30 2024-05-14T09:31:42.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.109+0000 7fedadb99700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feda8113230 con 0x7feda8103550 2024-05-14T09:31:42.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.109+0000 7fedadb99700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7feda8113530 con 0x7feda8108780 2024-05-14T09:31:42.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.109+0000 7feda6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8107e30 0x7feda8199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:42.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.109+0000 7feda6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8107e30 0x7feda8199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59170/0 (socket says 172.21.15.2:59170) 2024-05-14T09:31:42.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.109+0000 7feda6ffd700 1 -- 172.21.15.2:0/4202936589 learned_addr learned my addr 172.21.15.2:0/4202936589 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:42.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.109+0000 7feda77fe700 1 --2- 172.21.15.2:0/4202936589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda8103550 0x7feda8199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:42.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.109+0000 7feda7fff700 1 --2- 172.21.15.2:0/4202936589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda8108780 0x7feda819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:42.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.110+0000 7feda6ffd700 1 -- 172.21.15.2:0/4202936589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda8108780 msgr2=0x7feda819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:42.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.110+0000 7feda6ffd700 1 --2- 172.21.15.2:0/4202936589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda8108780 0x7feda819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:42.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.110+0000 7feda6ffd700 1 -- 172.21.15.2:0/4202936589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda8103550 msgr2=0x7feda8199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:42.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.110+0000 7feda6ffd700 1 --2- 172.21.15.2:0/4202936589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda8103550 0x7feda8199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:42.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.110+0000 7feda6ffd700 1 -- 172.21.15.2:0/4202936589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda81a91a0 con 0x7feda8107e30 2024-05-14T09:31:42.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.110+0000 7feda7fff700 1 --2- 172.21.15.2:0/4202936589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda8108780 0x7feda819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:31:42.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.110+0000 7feda6ffd700 1 --2- 172.21.15.2:0/4202936589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8107e30 0x7feda8199fe0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00e970 tx=0x7fed9c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:42.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.110+0000 7feda4ff9700 1 -- 172.21.15.2:0/4202936589 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c00cbd0 con 0x7feda8107e30 2024-05-14T09:31:42.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.110+0000 7feda4ff9700 1 -- 172.21.15.2:0/4202936589 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed9c00cd70 con 0x7feda8107e30 2024-05-14T09:31:42.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.110+0000 7fedadb99700 1 -- 172.21.15.2:0/4202936589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda81a9490 con 0x7feda8107e30 2024-05-14T09:31:42.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.110+0000 7feda4ff9700 1 -- 172.21.15.2:0/4202936589 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c010430 con 0x7feda8107e30 2024-05-14T09:31:42.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.110+0000 7fedadb99700 1 -- 172.21.15.2:0/4202936589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda81a99d0 con 0x7feda8107e30 2024-05-14T09:31:42.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.111+0000 7fedadb99700 1 -- 172.21.15.2:0/4202936589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda81049b0 con 0x7feda8107e30 2024-05-14T09:31:42.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.113+0000 7feda4ff9700 1 -- 172.21.15.2:0/4202936589 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fed9c004d70 con 0x7feda8107e30 2024-05-14T09:31:42.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.113+0000 7feda4ff9700 1 --2- 172.21.15.2:0/4202936589 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fed900b2880 0x7fed900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:42.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.114+0000 7feda77fe700 1 --2- 172.21.15.2:0/4202936589 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fed900b2880 0x7fed900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:42.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.114+0000 7feda4ff9700 1 -- 172.21.15.2:0/4202936589 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fed9c0e2a80 con 0x7feda8107e30 2024-05-14T09:31:42.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.114+0000 7feda77fe700 1 --2- 172.21.15.2:0/4202936589 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fed900b2880 0x7fed900b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fed98003760 tx=0x7fed9801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:42.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.117+0000 7feda4ff9700 1 -- 172.21.15.2:0/4202936589 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fed9c054df0 con 0x7feda8107e30 2024-05-14T09:31:42.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.345+0000 7fedadb99700 1 -- 172.21.15.2:0/4202936589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feda8075fe0 con 0x7feda8107e30 2024-05-14T09:31:42.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.345+0000 7feda4ff9700 1 -- 172.21.15.2:0/4202936589 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+581 (secure 0 0 0) 0x7fed9c0aa460 con 0x7feda8107e30 2024-05-14T09:31:42.346 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/4051922409 2024-05-15T09:23:28.940670+0000 2024-05-14T09:31:42.346 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2353790181 2024-05-15T09:23:28.940670+0000 2024-05-14T09:31:42.346 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/4268171474 2024-05-15T09:23:28.940670+0000 2024-05-14T09:31:42.346 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/1423742578 2024-05-15T09:23:28.940670+0000 2024-05-14T09:31:42.346 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/969850441 2024-05-15T09:23:02.498446+0000 2024-05-14T09:31:42.347 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3252991268 2024-05-15T09:23:02.498446+0000 2024-05-14T09:31:42.347 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3593583257 2024-05-15T09:23:02.498446+0000 2024-05-14T09:31:42.347 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3593583257 2024-05-15T09:23:02.498446+0000 2024-05-14T09:31:42.347 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3789067476 2024-05-15T09:23:02.498446+0000 2024-05-14T09:31:42.347 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/1423742578 2024-05-15T09:23:28.940670+0000 2024-05-14T09:31:42.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.348+0000 7fedadb99700 1 -- 172.21.15.2:0/4202936589 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fed900b2880 msgr2=0x7fed900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:42.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.348+0000 7fedadb99700 1 --2- 172.21.15.2:0/4202936589 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fed900b2880 0x7fed900b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fed98003760 tx=0x7fed9801a040 comp rx=0 tx=0).stop 2024-05-14T09:31:42.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.348+0000 7fedadb99700 1 -- 172.21.15.2:0/4202936589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8107e30 msgr2=0x7feda8199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:42.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.348+0000 7fedadb99700 1 --2- 172.21.15.2:0/4202936589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8107e30 0x7feda8199fe0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00e970 tx=0x7fed9c00ed30 comp rx=0 tx=0).stop 2024-05-14T09:31:42.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.348+0000 7fedadb99700 1 -- 172.21.15.2:0/4202936589 shutdown_connections 2024-05-14T09:31:42.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.348+0000 7fedadb99700 1 --2- 172.21.15.2:0/4202936589 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fed900b2880 0x7fed900b4d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:42.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.348+0000 7fedadb99700 1 --2- 172.21.15.2:0/4202936589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda8103550 0x7feda8199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:42.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.348+0000 7fedadb99700 1 --2- 172.21.15.2:0/4202936589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feda8108780 0x7feda819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:42.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.348+0000 7fedadb99700 1 --2- 172.21.15.2:0/4202936589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8107e30 0x7feda8199fe0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:42.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.348+0000 7fedadb99700 1 -- 172.21.15.2:0/4202936589 >> 172.21.15.2:0/4202936589 conn(0x7feda8078e00 msgr2=0x7feda8100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:42.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.348+0000 7fedadb99700 1 -- 172.21.15.2:0/4202936589 shutdown_connections 2024-05-14T09:31:42.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:42.348+0000 7fedadb99700 1 -- 172.21.15.2:0/4202936589 wait complete. 2024-05-14T09:31:42.350 INFO:teuthology.orchestra.run.smithi002.stderr:listed 10 entries 2024-05-14T09:31:42.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:42 smithi002 ceph-mon[31557]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 376 B/s wr, 0 op/s; 201 B/s, 0 objects/s recovering 2024-05-14T09:31:42.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:42 smithi026 ceph-mon[36056]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 376 B/s wr, 0 op/s; 201 B/s, 0 objects/s recovering 2024-05-14T09:31:42.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:42 smithi029 ceph-mon[36249]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 376 B/s wr, 0 op/s; 201 B/s, 0 objects/s recovering 2024-05-14T09:31:42.981 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2024-05-14T09:31:42.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:31:42.982 DEBUG:teuthology.orchestra.run.smithi002:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-14T09:31:43.009 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:ceph-fuse[74716]: fuse finished with error 0 and tester_r 0 2024-05-14T09:31:43.010 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T09:31:43.058 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T09:31:43.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:31:43.058 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-14T09:31:43.083 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T09:31:43.375 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:43.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:43 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/4202936589' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T09:31:43.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:43 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/4202936589' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T09:31:43.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:43 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/4202936589' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T09:31:44.527 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:44 smithi002 ceph-mon[31557]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 182 B/s, 0 objects/s recovering 2024-05-14T09:31:44.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:44 smithi029 ceph-mon[36249]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 182 B/s, 0 objects/s recovering 2024-05-14T09:31:44.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:44 smithi026 ceph-mon[36056]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 182 B/s, 0 objects/s recovering 2024-05-14T09:31:45.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.108+0000 7fe203a7a700 1 -- 172.21.15.2:0/96323240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1fc108760 msgr2=0x7fe1fc1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:45.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.108+0000 7fe203a7a700 1 --2- 172.21.15.2:0/96323240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1fc108760 0x7fe1fc1100c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400b0d0 tx=0x7fe1f400b3e0 comp rx=0 tx=0).stop 2024-05-14T09:31:45.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.108+0000 7fe203a7a700 1 -- 172.21.15.2:0/96323240 shutdown_connections 2024-05-14T09:31:45.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.108+0000 7fe203a7a700 1 --2- 172.21.15.2:0/96323240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1fc108760 0x7fe1fc1100c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:45.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.108+0000 7fe203a7a700 1 --2- 172.21.15.2:0/96323240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc1034e0 0x7fe1fc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:45.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.108+0000 7fe203a7a700 1 --2- 172.21.15.2:0/96323240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1fc107e10 0x7fe1fc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.108+0000 7fe203a7a700 1 -- 172.21.15.2:0/96323240 >> 172.21.15.2:0/96323240 conn(0x7fe1fc0fef30 msgr2=0x7fe1fc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.109+0000 7fe203a7a700 1 -- 172.21.15.2:0/96323240 shutdown_connections 2024-05-14T09:31:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.109+0000 7fe203a7a700 1 -- 172.21.15.2:0/96323240 wait complete. 2024-05-14T09:31:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.109+0000 7fe203a7a700 1 Processor -- start 2024-05-14T09:31:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.109+0000 7fe203a7a700 1 -- start start 2024-05-14T09:31:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe203a7a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc1034e0 0x7fe1fc19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe203a7a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1fc107e10 0x7fe1fc19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe203a7a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1fc108760 0x7fe1fc19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe203a7a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe1fc113410 con 0x7fe1fc107e10 2024-05-14T09:31:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe203a7a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe1fc113290 con 0x7fe1fc108760 2024-05-14T09:31:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe203a7a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe1fc113590 con 0x7fe1fc1034e0 2024-05-14T09:31:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe201015700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1fc107e10 0x7fe1fc19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe201015700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1fc107e10 0x7fe1fc19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59180/0 (socket says 172.21.15.2:59180) 2024-05-14T09:31:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe201015700 1 -- 172.21.15.2:0/561432927 learned_addr learned my addr 172.21.15.2:0/561432927 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe202017700 1 --2- 172.21.15.2:0/561432927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1fc108760 0x7fe1fc19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:45.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe201816700 1 --2- 172.21.15.2:0/561432927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc1034e0 0x7fe1fc19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:45.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe201015700 1 -- 172.21.15.2:0/561432927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc1034e0 msgr2=0x7fe1fc19bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:45.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe201015700 1 --2- 172.21.15.2:0/561432927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc1034e0 0x7fe1fc19bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:45.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe201015700 1 -- 172.21.15.2:0/561432927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1fc108760 msgr2=0x7fe1fc19c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:45.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe201015700 1 --2- 172.21.15.2:0/561432927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1fc108760 0x7fe1fc19c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:45.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe201015700 1 -- 172.21.15.2:0/561432927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1fc1a70c0 con 0x7fe1fc107e10 2024-05-14T09:31:45.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe201015700 1 --2- 172.21.15.2:0/561432927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1fc107e10 0x7fe1fc19c2b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00c8f0 tx=0x7fe1ec00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:45.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.110+0000 7fe1f2ffd700 1 -- 172.21.15.2:0/561432927 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec007940 con 0x7fe1fc107e10 2024-05-14T09:31:45.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.111+0000 7fe1f2ffd700 1 -- 172.21.15.2:0/561432927 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1ec00f450 con 0x7fe1fc107e10 2024-05-14T09:31:45.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.111+0000 7fe203a7a700 1 -- 172.21.15.2:0/561432927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1fc1a73b0 con 0x7fe1fc107e10 2024-05-14T09:31:45.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.111+0000 7fe1f2ffd700 1 -- 172.21.15.2:0/561432927 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec018660 con 0x7fe1fc107e10 2024-05-14T09:31:45.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.111+0000 7fe203a7a700 1 -- 172.21.15.2:0/561432927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1fc1a7940 con 0x7fe1fc107e10 2024-05-14T09:31:45.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.112+0000 7fe203a7a700 1 -- 172.21.15.2:0/561432927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1fc0512c0 con 0x7fe1fc107e10 2024-05-14T09:31:45.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.113+0000 7fe1f2ffd700 1 -- 172.21.15.2:0/561432927 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fe1ec00f890 con 0x7fe1fc107e10 2024-05-14T09:31:45.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.114+0000 7fe1f2ffd700 1 --2- 172.21.15.2:0/561432927 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe1e80b2900 0x7fe1e80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:45.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.114+0000 7fe201816700 1 --2- 172.21.15.2:0/561432927 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe1e80b2900 0x7fe1e80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:45.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.115+0000 7fe1f2ffd700 1 -- 172.21.15.2:0/561432927 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fe1ec0e2480 con 0x7fe1fc107e10 2024-05-14T09:31:45.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.115+0000 7fe201816700 1 --2- 172.21.15.2:0/561432927 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe1e80b2900 0x7fe1e80b4e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8006fd0 tx=0x7fe1f8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:45.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.119+0000 7fe1f2ffd700 1 -- 172.21.15.2:0/561432927 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe1ec014030 con 0x7fe1fc107e10 2024-05-14T09:31:45.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.332+0000 7fe203a7a700 1 -- 172.21.15.2:0/561432927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1fc06c630 con 0x7fe1fc107e10 2024-05-14T09:31:45.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.334+0000 7fe1f2ffd700 1 -- 172.21.15.2:0/561432927 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24395 (secure 0 0 0) 0x7fe1ec0a9f70 con 0x7fe1fc107e10 2024-05-14T09:31:45.335 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:31:45.335 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":88,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","created":"2024-05-14T09:22:34.937047+0000","modified":"2024-05-14T09:31:30.160963+0000","last_up_change":"2024-05-14T09:29:55.157569+0000","last_in_change":"2024-05-14T09:29:34.922562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T09:26:03.046454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T09:30:51.814350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T09:30:55.500000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T09:31:06.099087+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10d34289-414e-4b1b-bc7c-ba1dd24c7297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6803","nonce":2622184634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6805","nonce":2622184634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6809","nonce":2622184634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6807","nonce":2622184634}]},"public_addr":"172.21.15.2:6803/2622184634","cluster_addr":"172.21.15.2:6805/2622184634","heartbeat_back_addr":"172.21.15.2:6809/2622184634","heartbeat_front_addr":"172.21.15.2:6807/2622184634","state":["exists","up"]},{"osd":1,"uuid":"bc4aefc2-706e-4faf-9042-64b87ad097b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6801","nonce":3084747226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6803","nonce":3084747226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6807","nonce":3084747226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6805","nonce":3084747226}]},"public_addr":"172.21.15.26:6801/3084747226","cluster_addr":"172.21.15.26:6803/3084747226","heartbeat_back_addr":"172.21.15.26:6807/3084747226","heartbeat_front_addr":"172.21.15.26:6805/3084747226","state":["exists","up"]},{"osd":2,"uuid":"8c5bb589-56e2-469f-9a6f-6035b3ddf4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6801","nonce":814048086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6803","nonce":814048086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6807","nonce":814048086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6805","nonce":814048086}]},"public_addr":"172.21.15.29:6801/814048086","cluster_addr":"172.21.15.29:6803/814048086","heartbeat_back_addr":"172.21.15.29:6807/814048086","heartbeat_front_addr":"172.21.15.29:6805/814048086","state":["exists","up"]},{"osd":3,"uuid":"8fbfc078-85e7-4dc2-acda-5e7b0ab927e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6811","nonce":879639451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6813","nonce":879639451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6817","nonce":879639451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6815","nonce":879639451}]},"public_addr":"172.21.15.2:6811/879639451","cluster_addr":"172.21.15.2:6813/879639451","heartbeat_back_addr":"172.21.15.2:6817/879639451","heartbeat_front_addr":"172.21.15.2:6815/879639451","state":["exists","up"]},{"osd":4,"uuid":"f2515448-e9c1-48a2-b270-1e5e781a2fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6809","nonce":457100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6811","nonce":457100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6815","nonce":457100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6813","nonce":457100752}]},"public_addr":"172.21.15.26:6809/457100752","cluster_addr":"172.21.15.26:6811/457100752","heartbeat_back_addr":"172.21.15.26:6815/457100752","heartbeat_front_addr":"172.21.15.26:6813/457100752","state":["exists","up"]},{"osd":5,"uuid":"20030857-a1e6-4b0a-b186-1cfa44f2e4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6809","nonce":1554045093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6811","nonce":1554045093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6815","nonce":1554045093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6813","nonce":1554045093}]},"public_addr":"172.21.15.29:6809/1554045093","cluster_addr":"172.21.15.29:6811/1554045093","heartbeat_back_addr":"172.21.15.29:6815/1554045093","heartbeat_front_addr":"172.21.15.29:6813/1554045093","state":["exists","up"]},{"osd":6,"uuid":"d757c72a-29f0-4f14-9f13-a2e61eb5e6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6819","nonce":2749897595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6821","nonce":2749897595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6825","nonce":2749897595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6823","nonce":2749897595}]},"public_addr":"172.21.15.2:6819/2749897595","cluster_addr":"172.21.15.2:6821/2749897595","heartbeat_back_addr":"172.21.15.2:6825/2749897595","heartbeat_front_addr":"172.21.15.2:6823/2749897595","state":["exists","up"]},{"osd":7,"uuid":"22ddef80-3633-44c5-bc0e-edc75b546ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6817","nonce":1180401611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6819","nonce":1180401611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6823","nonce":1180401611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6821","nonce":1180401611}]},"public_addr":"172.21.15.26:6817/1180401611","cluster_addr":"172.21.15.26:6819/1180401611","heartbeat_back_addr":"172.21.15.26:6823/1180401611","heartbeat_front_addr":"172.21.15.26:6821/1180401611","state":["exists","up"]},{"osd":8,"uuid":"466e7ac4-3ff7-4254-a3fd-8096420b1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6817","nonce":2858408446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6819","nonce":2858408446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6823","nonce":2858408446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6821","nonce":2858408446}]},"public_addr":"172.21.15.29:6817/2858408446","cluster_addr":"172.21.15.29:6819/2858408446","heartbeat_back_addr":"172.21.15.29:6823/2858408446","heartbeat_front_addr":"172.21.15.29:6821/2858408446","state":["exists","up"]},{"osd":9,"uuid":"119eeac9-29f7-4086-a1ad-04a0bc154b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6827","nonce":176898226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6829","nonce":176898226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6833","nonce":176898226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6831","nonce":176898226}]},"public_addr":"172.21.15.2:6827/176898226","cluster_addr":"172.21.15.2:6829/176898226","heartbeat_back_addr":"172.21.15.2:6833/176898226","heartbeat_front_addr":"172.21.15.2:6831/176898226","state":["exists","up"]},{"osd":10,"uuid":"55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6825","nonce":4065159515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6827","nonce":4065159515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6831","nonce":4065159515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6829","nonce":4065159515}]},"public_addr":"172.21.15.26:6825/4065159515","cluster_addr":"172.21.15.26:6827/4065159515","heartbeat_back_addr":"172.21.15.26:6831/4065159515","heartbeat_front_addr":"172.21.15.26:6829/4065159515","state":["exists","up"]},{"osd":11,"uuid":"c5246763-c4f6-4ae3-8a93-5929c541ac5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6825","nonce":378800417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6827","nonce":378800417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6831","nonce":378800417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6829","nonce":378800417}]},"public_addr":"172.21.15.29:6825/378800417","cluster_addr":"172.21.15.29:6827/378800417","heartbeat_back_addr":"172.21.15.29:6831/378800417","heartbeat_front_addr":"172.21.15.29:6829/378800417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:09.342387+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:34.156359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:58.860653+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:24.009323+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:49.057146+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:15.007172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:41.085748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:06.674923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:33.409128+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:00.106186+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:26.803487+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:53.668283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11},{"from":8,"to":6}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4051922409":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/2353790181":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/4268171474":"2024-05-15T09:23:28.940670+0000","172.21.15.2:6801/1423742578":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/969850441":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3252991268":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6801/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3789067476":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/1423742578":"2024-05-15T09:23:28.940670+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T09:31:45.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.336+0000 7fe203a7a700 1 -- 172.21.15.2:0/561432927 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe1e80b2900 msgr2=0x7fe1e80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:45.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.336+0000 7fe203a7a700 1 --2- 172.21.15.2:0/561432927 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe1e80b2900 0x7fe1e80b4e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8006fd0 tx=0x7fe1f8008040 comp rx=0 tx=0).stop 2024-05-14T09:31:45.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.336+0000 7fe203a7a700 1 -- 172.21.15.2:0/561432927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1fc107e10 msgr2=0x7fe1fc19c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:45.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.336+0000 7fe203a7a700 1 --2- 172.21.15.2:0/561432927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1fc107e10 0x7fe1fc19c2b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00c8f0 tx=0x7fe1ec00cc00 comp rx=0 tx=0).stop 2024-05-14T09:31:45.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.336+0000 7fe203a7a700 1 -- 172.21.15.2:0/561432927 shutdown_connections 2024-05-14T09:31:45.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.336+0000 7fe203a7a700 1 --2- 172.21.15.2:0/561432927 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe1e80b2900 0x7fe1e80b4e10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:45.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.336+0000 7fe203a7a700 1 --2- 172.21.15.2:0/561432927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1fc108760 0x7fe1fc19c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:45.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.336+0000 7fe203a7a700 1 --2- 172.21.15.2:0/561432927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1fc1034e0 0x7fe1fc19bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:45.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.336+0000 7fe203a7a700 1 --2- 172.21.15.2:0/561432927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1fc107e10 0x7fe1fc19c2b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:45.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.337+0000 7fe203a7a700 1 -- 172.21.15.2:0/561432927 >> 172.21.15.2:0/561432927 conn(0x7fe1fc0fef30 msgr2=0x7fe1fc107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:45.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.337+0000 7fe203a7a700 1 -- 172.21.15.2:0/561432927 shutdown_connections 2024-05-14T09:31:45.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:45.337+0000 7fe203a7a700 1 -- 172.21.15.2:0/561432927 wait complete. 2024-05-14T09:31:45.978 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T09:31:46.235 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:46.497 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:46 smithi002 ceph-mon[31557]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-14T09:31:46.497 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:46 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/561432927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:31:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:46 smithi029 ceph-mon[36249]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-14T09:31:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:46 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/561432927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:31:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:46 smithi026 ceph-mon[36056]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-14T09:31:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:46 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/561432927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:31:47.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.924+0000 7ff0d673e700 1 -- 172.21.15.2:0/2119161141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0d0109850 msgr2=0x7ff0d01111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:47.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.924+0000 7ff0d673e700 1 --2- 172.21.15.2:0/2119161141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0d0109850 0x7ff0d01111b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800b0d0 tx=0x7ff0c800b3e0 comp rx=0 tx=0).stop 2024-05-14T09:31:47.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.924+0000 7ff0d673e700 1 -- 172.21.15.2:0/2119161141 shutdown_connections 2024-05-14T09:31:47.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.924+0000 7ff0d673e700 1 --2- 172.21.15.2:0/2119161141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0d01045d0 0x7ff0d01049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:47.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.924+0000 7ff0d673e700 1 --2- 172.21.15.2:0/2119161141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0d0109850 0x7ff0d01111b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:47.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.924+0000 7ff0d673e700 1 --2- 172.21.15.2:0/2119161141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0d0108f00 0x7ff0d0109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:47.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.924+0000 7ff0d673e700 1 -- 172.21.15.2:0/2119161141 >> 172.21.15.2:0/2119161141 conn(0x7ff0d00ffea0 msgr2=0x7ff0d01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:47.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.924+0000 7ff0d673e700 1 -- 172.21.15.2:0/2119161141 shutdown_connections 2024-05-14T09:31:47.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.924+0000 7ff0d673e700 1 -- 172.21.15.2:0/2119161141 wait complete. 2024-05-14T09:31:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0d673e700 1 Processor -- start 2024-05-14T09:31:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0d673e700 1 -- start start 2024-05-14T09:31:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0d673e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0d01045d0 0x7ff0d0076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0d673e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0d0108f00 0x7ff0d0077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0d673e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0d0109850 0x7ff0d0077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0d673e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0114480 con 0x7ff0d0108f00 2024-05-14T09:31:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0d673e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0114300 con 0x7ff0d0109850 2024-05-14T09:31:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0d673e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0114600 con 0x7ff0d01045d0 2024-05-14T09:31:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0c7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0d0108f00 0x7ff0d0077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0c7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0d0108f00 0x7ff0d0077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59396/0 (socket says 172.21.15.2:59396) 2024-05-14T09:31:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0c7fff700 1 -- 172.21.15.2:0/895158587 learned_addr learned my addr 172.21.15.2:0/895158587 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0cffff700 1 --2- 172.21.15.2:0/895158587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0d01045d0 0x7ff0d0076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0d4cdb700 1 --2- 172.21.15.2:0/895158587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0d0109850 0x7ff0d0077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0c7fff700 1 -- 172.21.15.2:0/895158587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0d01045d0 msgr2=0x7ff0d0076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0c7fff700 1 --2- 172.21.15.2:0/895158587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0d01045d0 0x7ff0d0076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0c7fff700 1 -- 172.21.15.2:0/895158587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0d0109850 msgr2=0x7ff0d0077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0c7fff700 1 --2- 172.21.15.2:0/895158587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0d0109850 0x7ff0d0077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0c7fff700 1 -- 172.21.15.2:0/895158587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d01a91c0 con 0x7ff0d0108f00 2024-05-14T09:31:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.925+0000 7ff0d4cdb700 1 --2- 172.21.15.2:0/895158587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0d0109850 0x7ff0d0077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:31:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.926+0000 7ff0c7fff700 1 --2- 172.21.15.2:0/895158587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0d0108f00 0x7ff0d0077110 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800c8f0 tx=0x7ff0b800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.926+0000 7ff0cdffb700 1 -- 172.21.15.2:0/895158587 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8007940 con 0x7ff0d0108f00 2024-05-14T09:31:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.926+0000 7ff0cdffb700 1 -- 172.21.15.2:0/895158587 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0b800f450 con 0x7ff0d0108f00 2024-05-14T09:31:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.926+0000 7ff0d673e700 1 -- 172.21.15.2:0/895158587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d01a94b0 con 0x7ff0d0108f00 2024-05-14T09:31:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.926+0000 7ff0cdffb700 1 -- 172.21.15.2:0/895158587 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8018660 con 0x7ff0d0108f00 2024-05-14T09:31:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.926+0000 7ff0d673e700 1 -- 172.21.15.2:0/895158587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d01a9a40 con 0x7ff0d0108f00 2024-05-14T09:31:47.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.927+0000 7ff0d673e700 1 -- 172.21.15.2:0/895158587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d00512c0 con 0x7ff0d0108f00 2024-05-14T09:31:47.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.930+0000 7ff0cdffb700 1 -- 172.21.15.2:0/895158587 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7ff0b800f890 con 0x7ff0d0108f00 2024-05-14T09:31:47.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.930+0000 7ff0cdffb700 1 --2- 172.21.15.2:0/895158587 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff0bc0b2900 0x7ff0bc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:47.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.931+0000 7ff0cffff700 1 --2- 172.21.15.2:0/895158587 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff0bc0b2900 0x7ff0bc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:47.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.931+0000 7ff0cdffb700 1 -- 172.21.15.2:0/895158587 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7ff0b80e2320 con 0x7ff0d0108f00 2024-05-14T09:31:47.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.931+0000 7ff0cffff700 1 --2- 172.21.15.2:0/895158587 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff0bc0b2900 0x7ff0bc0b4e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0005950 tx=0x7ff0c000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:47.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:47.934+0000 7ff0cdffb700 1 -- 172.21.15.2:0/895158587 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff0b8014030 con 0x7ff0d0108f00 2024-05-14T09:31:48.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:48.157+0000 7ff0d673e700 1 -- 172.21.15.2:0/895158587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0d0078100 con 0x7ff0d0108f00 2024-05-14T09:31:48.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:48.159+0000 7ff0cdffb700 1 -- 172.21.15.2:0/895158587 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24395 (secure 0 0 0) 0x7ff0b80a9eb0 con 0x7ff0d0108f00 2024-05-14T09:31:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:31:48.160 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":88,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","created":"2024-05-14T09:22:34.937047+0000","modified":"2024-05-14T09:31:30.160963+0000","last_up_change":"2024-05-14T09:29:55.157569+0000","last_in_change":"2024-05-14T09:29:34.922562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T09:26:03.046454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T09:30:51.814350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T09:30:55.500000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T09:31:06.099087+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10d34289-414e-4b1b-bc7c-ba1dd24c7297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6803","nonce":2622184634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6805","nonce":2622184634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6809","nonce":2622184634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6807","nonce":2622184634}]},"public_addr":"172.21.15.2:6803/2622184634","cluster_addr":"172.21.15.2:6805/2622184634","heartbeat_back_addr":"172.21.15.2:6809/2622184634","heartbeat_front_addr":"172.21.15.2:6807/2622184634","state":["exists","up"]},{"osd":1,"uuid":"bc4aefc2-706e-4faf-9042-64b87ad097b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6801","nonce":3084747226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6803","nonce":3084747226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6807","nonce":3084747226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6805","nonce":3084747226}]},"public_addr":"172.21.15.26:6801/3084747226","cluster_addr":"172.21.15.26:6803/3084747226","heartbeat_back_addr":"172.21.15.26:6807/3084747226","heartbeat_front_addr":"172.21.15.26:6805/3084747226","state":["exists","up"]},{"osd":2,"uuid":"8c5bb589-56e2-469f-9a6f-6035b3ddf4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6801","nonce":814048086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6803","nonce":814048086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6807","nonce":814048086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6805","nonce":814048086}]},"public_addr":"172.21.15.29:6801/814048086","cluster_addr":"172.21.15.29:6803/814048086","heartbeat_back_addr":"172.21.15.29:6807/814048086","heartbeat_front_addr":"172.21.15.29:6805/814048086","state":["exists","up"]},{"osd":3,"uuid":"8fbfc078-85e7-4dc2-acda-5e7b0ab927e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6811","nonce":879639451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6813","nonce":879639451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6817","nonce":879639451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6815","nonce":879639451}]},"public_addr":"172.21.15.2:6811/879639451","cluster_addr":"172.21.15.2:6813/879639451","heartbeat_back_addr":"172.21.15.2:6817/879639451","heartbeat_front_addr":"172.21.15.2:6815/879639451","state":["exists","up"]},{"osd":4,"uuid":"f2515448-e9c1-48a2-b270-1e5e781a2fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6809","nonce":457100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6811","nonce":457100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6815","nonce":457100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6813","nonce":457100752}]},"public_addr":"172.21.15.26:6809/457100752","cluster_addr":"172.21.15.26:6811/457100752","heartbeat_back_addr":"172.21.15.26:6815/457100752","heartbeat_front_addr":"172.21.15.26:6813/457100752","state":["exists","up"]},{"osd":5,"uuid":"20030857-a1e6-4b0a-b186-1cfa44f2e4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6809","nonce":1554045093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6811","nonce":1554045093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6815","nonce":1554045093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6813","nonce":1554045093}]},"public_addr":"172.21.15.29:6809/1554045093","cluster_addr":"172.21.15.29:6811/1554045093","heartbeat_back_addr":"172.21.15.29:6815/1554045093","heartbeat_front_addr":"172.21.15.29:6813/1554045093","state":["exists","up"]},{"osd":6,"uuid":"d757c72a-29f0-4f14-9f13-a2e61eb5e6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6819","nonce":2749897595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6821","nonce":2749897595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6825","nonce":2749897595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6823","nonce":2749897595}]},"public_addr":"172.21.15.2:6819/2749897595","cluster_addr":"172.21.15.2:6821/2749897595","heartbeat_back_addr":"172.21.15.2:6825/2749897595","heartbeat_front_addr":"172.21.15.2:6823/2749897595","state":["exists","up"]},{"osd":7,"uuid":"22ddef80-3633-44c5-bc0e-edc75b546ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6817","nonce":1180401611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6819","nonce":1180401611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6823","nonce":1180401611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6821","nonce":1180401611}]},"public_addr":"172.21.15.26:6817/1180401611","cluster_addr":"172.21.15.26:6819/1180401611","heartbeat_back_addr":"172.21.15.26:6823/1180401611","heartbeat_front_addr":"172.21.15.26:6821/1180401611","state":["exists","up"]},{"osd":8,"uuid":"466e7ac4-3ff7-4254-a3fd-8096420b1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6817","nonce":2858408446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6819","nonce":2858408446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6823","nonce":2858408446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6821","nonce":2858408446}]},"public_addr":"172.21.15.29:6817/2858408446","cluster_addr":"172.21.15.29:6819/2858408446","heartbeat_back_addr":"172.21.15.29:6823/2858408446","heartbeat_front_addr":"172.21.15.29:6821/2858408446","state":["exists","up"]},{"osd":9,"uuid":"119eeac9-29f7-4086-a1ad-04a0bc154b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6827","nonce":176898226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6829","nonce":176898226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6833","nonce":176898226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6831","nonce":176898226}]},"public_addr":"172.21.15.2:6827/176898226","cluster_addr":"172.21.15.2:6829/176898226","heartbeat_back_addr":"172.21.15.2:6833/176898226","heartbeat_front_addr":"172.21.15.2:6831/176898226","state":["exists","up"]},{"osd":10,"uuid":"55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6825","nonce":4065159515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6827","nonce":4065159515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6831","nonce":4065159515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6829","nonce":4065159515}]},"public_addr":"172.21.15.26:6825/4065159515","cluster_addr":"172.21.15.26:6827/4065159515","heartbeat_back_addr":"172.21.15.26:6831/4065159515","heartbeat_front_addr":"172.21.15.26:6829/4065159515","state":["exists","up"]},{"osd":11,"uuid":"c5246763-c4f6-4ae3-8a93-5929c541ac5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6825","nonce":378800417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6827","nonce":378800417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6831","nonce":378800417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6829","nonce":378800417}]},"public_addr":"172.21.15.29:6825/378800417","cluster_addr":"172.21.15.29:6827/378800417","heartbeat_back_addr":"172.21.15.29:6831/378800417","heartbeat_front_addr":"172.21.15.29:6829/378800417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:09.342387+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:34.156359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:58.860653+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:24.009323+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:49.057146+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:15.007172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:41.085748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:06.674923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:33.409128+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:00.106186+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:26.803487+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:53.668283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11},{"from":8,"to":6}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4051922409":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/2353790181":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/4268171474":"2024-05-15T09:23:28.940670+0000","172.21.15.2:6801/1423742578":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/969850441":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3252991268":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6801/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3789067476":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/1423742578":"2024-05-15T09:23:28.940670+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T09:31:48.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:48.161+0000 7ff0d673e700 1 -- 172.21.15.2:0/895158587 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff0bc0b2900 msgr2=0x7ff0bc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:48.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:48.161+0000 7ff0d673e700 1 --2- 172.21.15.2:0/895158587 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff0bc0b2900 0x7ff0bc0b4e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0005950 tx=0x7ff0c000a300 comp rx=0 tx=0).stop 2024-05-14T09:31:48.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:48.161+0000 7ff0d673e700 1 -- 172.21.15.2:0/895158587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0d0108f00 msgr2=0x7ff0d0077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:48.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:48.161+0000 7ff0d673e700 1 --2- 172.21.15.2:0/895158587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0d0108f00 0x7ff0d0077110 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800c8f0 tx=0x7ff0b800cc00 comp rx=0 tx=0).stop 2024-05-14T09:31:48.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:48.162+0000 7ff0d673e700 1 -- 172.21.15.2:0/895158587 shutdown_connections 2024-05-14T09:31:48.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:48.162+0000 7ff0d673e700 1 --2- 172.21.15.2:0/895158587 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff0bc0b2900 0x7ff0bc0b4e10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:48.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:48.162+0000 7ff0d673e700 1 --2- 172.21.15.2:0/895158587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0d0109850 0x7ff0d0077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:48.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:48.162+0000 7ff0d673e700 1 --2- 172.21.15.2:0/895158587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff0d01045d0 0x7ff0d0076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:48.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:48.162+0000 7ff0d673e700 1 --2- 172.21.15.2:0/895158587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0d0108f00 0x7ff0d0077110 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:48.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:48.162+0000 7ff0d673e700 1 -- 172.21.15.2:0/895158587 >> 172.21.15.2:0/895158587 conn(0x7ff0d00ffea0 msgr2=0x7ff0d01086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:48.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:48.162+0000 7ff0d673e700 1 -- 172.21.15.2:0/895158587 shutdown_connections 2024-05-14T09:31:48.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:48.162+0000 7ff0d673e700 1 -- 172.21.15.2:0/895158587 wait complete. 2024-05-14T09:31:48.476 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:48 smithi002 ceph-mon[31557]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-14T09:31:48.476 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:48 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/895158587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:31:48.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:48 smithi029 ceph-mon[36249]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-14T09:31:48.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:48 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/895158587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:31:48.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:48 smithi026 ceph-mon[36056]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-14T09:31:48.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:48 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/895158587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:31:48.781 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-14T09:31:49.034 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:50.514 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:50 smithi002 ceph-mon[31557]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-14T09:31:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:50 smithi029 ceph-mon[36249]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-14T09:31:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:50 smithi026 ceph-mon[36056]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-14T09:31:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.785+0000 7f6303184700 1 -- 172.21.15.2:0/2326275488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62fc103550 msgr2=0x7f62fc103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.785+0000 7f6303184700 1 --2- 172.21.15.2:0/2326275488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62fc103550 0x7f62fc103960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f62f80099c0 tx=0x7f62f8009cd0 comp rx=0 tx=0).stop 2024-05-14T09:31:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.785+0000 7f6303184700 1 -- 172.21.15.2:0/2326275488 shutdown_connections 2024-05-14T09:31:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.785+0000 7f6303184700 1 --2- 172.21.15.2:0/2326275488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62fc108780 0x7f62fc1100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.785+0000 7f6303184700 1 --2- 172.21.15.2:0/2326275488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62fc103550 0x7f62fc103960 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.785+0000 7f6303184700 1 --2- 172.21.15.2:0/2326275488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62fc107e30 0x7f62fc108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.785+0000 7f6303184700 1 -- 172.21.15.2:0/2326275488 >> 172.21.15.2:0/2326275488 conn(0x7f62fc078e00 msgr2=0x7f62fc079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.786+0000 7f6303184700 1 -- 172.21.15.2:0/2326275488 shutdown_connections 2024-05-14T09:31:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.786+0000 7f6303184700 1 -- 172.21.15.2:0/2326275488 wait complete. 2024-05-14T09:31:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.786+0000 7f6303184700 1 Processor -- start 2024-05-14T09:31:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.786+0000 7f6303184700 1 -- start start 2024-05-14T09:31:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f6303184700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62fc103550 0x7f62fc19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f6303184700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62fc107e30 0x7f62fc19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f6303184700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62fc108780 0x7f62fc19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f6303184700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc114500 con 0x7f62fc107e30 2024-05-14T09:31:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f6303184700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc114380 con 0x7f62fc103550 2024-05-14T09:31:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f6303184700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc114680 con 0x7f62fc108780 2024-05-14T09:31:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62fc107e30 0x7f62fc19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f6300f20700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62fc103550 0x7f62fc19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62fc107e30 0x7f62fc19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59422/0 (socket says 172.21.15.2:59422) 2024-05-14T09:31:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f62f3fff700 1 -- 172.21.15.2:0/2771474363 learned_addr learned my addr 172.21.15.2:0/2771474363 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f6301721700 1 --2- 172.21.15.2:0/2771474363 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62fc108780 0x7f62fc19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f6300f20700 1 -- 172.21.15.2:0/2771474363 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62fc108780 msgr2=0x7f62fc19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f6300f20700 1 --2- 172.21.15.2:0/2771474363 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62fc108780 0x7f62fc19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f6300f20700 1 -- 172.21.15.2:0/2771474363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62fc107e30 msgr2=0x7f62fc19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:50.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f6300f20700 1 --2- 172.21.15.2:0/2771474363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62fc107e30 0x7f62fc19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:50.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f6300f20700 1 -- 172.21.15.2:0/2771474363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62fc19fce0 con 0x7f62fc103550 2024-05-14T09:31:50.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.787+0000 7f62f3fff700 1 --2- 172.21.15.2:0/2771474363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62fc107e30 0x7f62fc19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:31:50.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.788+0000 7f6300f20700 1 --2- 172.21.15.2:0/2771474363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62fc103550 0x7f62fc19ae10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f62f80098e0 tx=0x7f62f80045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:50.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.788+0000 7f62f1ffb700 1 -- 172.21.15.2:0/2771474363 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8026070 con 0x7f62fc103550 2024-05-14T09:31:50.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.788+0000 7f6303184700 1 -- 172.21.15.2:0/2771474363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62fc19ff70 con 0x7f62fc103550 2024-05-14T09:31:50.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.788+0000 7f62f1ffb700 1 -- 172.21.15.2:0/2771474363 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62f800f460 con 0x7f62fc103550 2024-05-14T09:31:50.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.788+0000 7f62f1ffb700 1 -- 172.21.15.2:0/2771474363 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8009e60 con 0x7f62fc103550 2024-05-14T09:31:50.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.788+0000 7f6303184700 1 -- 172.21.15.2:0/2771474363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62fc1a0450 con 0x7f62fc103550 2024-05-14T09:31:50.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.789+0000 7f6303184700 1 -- 172.21.15.2:0/2771474363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62fc0512c0 con 0x7f62fc103550 2024-05-14T09:31:50.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.791+0000 7f62f1ffb700 1 -- 172.21.15.2:0/2771474363 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f62f8029070 con 0x7f62fc103550 2024-05-14T09:31:50.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.792+0000 7f62f1ffb700 1 --2- 172.21.15.2:0/2771474363 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f62e40b2880 0x7f62e40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:50.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.792+0000 7f62f3fff700 1 --2- 172.21.15.2:0/2771474363 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f62e40b2880 0x7f62e40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:50.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.792+0000 7f62f1ffb700 1 -- 172.21.15.2:0/2771474363 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f62f80e2fb0 con 0x7f62fc103550 2024-05-14T09:31:50.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.792+0000 7f62f3fff700 1 --2- 172.21.15.2:0/2771474363 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f62e40b2880 0x7f62e40b4d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f62fc079250 tx=0x7f62ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:50.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:50.796+0000 7f62f1ffb700 1 -- 172.21.15.2:0/2771474363 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62f8013070 con 0x7f62fc103550 2024-05-14T09:31:51.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:51.059+0000 7f6303184700 1 -- 172.21.15.2:0/2771474363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f62fc02e5f0 con 0x7f62fc103550 2024-05-14T09:31:51.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:51.219+0000 7f62f1ffb700 1 -- 172.21.15.2:0/2771474363 <== mon.2 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f62f80aa9c0 con 0x7f62fc103550 2024-05-14T09:31:51.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:51.222+0000 7f6303184700 1 -- 172.21.15.2:0/2771474363 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f62e40b2880 msgr2=0x7f62e40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:51.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:51.222+0000 7f6303184700 1 --2- 172.21.15.2:0/2771474363 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f62e40b2880 0x7f62e40b4d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f62fc079250 tx=0x7f62ec009260 comp rx=0 tx=0).stop 2024-05-14T09:31:51.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:51.222+0000 7f6303184700 1 -- 172.21.15.2:0/2771474363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62fc103550 msgr2=0x7f62fc19ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:51.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:51.222+0000 7f6303184700 1 --2- 172.21.15.2:0/2771474363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62fc103550 0x7f62fc19ae10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f62f80098e0 tx=0x7f62f80045e0 comp rx=0 tx=0).stop 2024-05-14T09:31:51.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:51.222+0000 7f6303184700 1 -- 172.21.15.2:0/2771474363 shutdown_connections 2024-05-14T09:31:51.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:51.222+0000 7f6303184700 1 --2- 172.21.15.2:0/2771474363 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f62e40b2880 0x7f62e40b4d90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:51.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:51.222+0000 7f6303184700 1 --2- 172.21.15.2:0/2771474363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62fc103550 0x7f62fc19ae10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:51.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:51.222+0000 7f6303184700 1 --2- 172.21.15.2:0/2771474363 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62fc108780 0x7f62fc19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:51.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:51.222+0000 7f6303184700 1 --2- 172.21.15.2:0/2771474363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62fc107e30 0x7f62fc19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:51.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:51.222+0000 7f6303184700 1 -- 172.21.15.2:0/2771474363 >> 172.21.15.2:0/2771474363 conn(0x7f62fc078e00 msgr2=0x7f62fc100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:51.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:51.222+0000 7f6303184700 1 -- 172.21.15.2:0/2771474363 shutdown_connections 2024-05-14T09:31:51.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:51.222+0000 7f6303184700 1 -- 172.21.15.2:0/2771474363 wait complete. 2024-05-14T09:31:51.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:51 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2771474363' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T09:31:51.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:51 smithi002 ceph-mon[31557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T09:31:51.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:51 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2771474363' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T09:31:51.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:51 smithi029 ceph-mon[36249]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T09:31:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:51 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2771474363' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T09:31:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:51 smithi026 ceph-mon[36056]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T09:31:51.838 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-14T09:31:51.839 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-14T09:31:52.109 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:52.471 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:52 smithi002 ceph-mon[31557]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-14T09:31:52.471 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:52 smithi002 ceph-mon[31557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-14T09:31:52.471 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:52 smithi002 ceph-mon[31557]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T09:31:52.471 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:31:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:52 smithi029 ceph-mon[36249]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-14T09:31:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:52 smithi029 ceph-mon[36249]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-14T09:31:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:52 smithi029 ceph-mon[36249]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T09:31:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:52 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:31:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:52 smithi026 ceph-mon[36056]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-14T09:31:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:52 smithi026 ceph-mon[36056]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-14T09:31:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:52 smithi026 ceph-mon[36056]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T09:31:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:52 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:31:53.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.751+0000 7f711d356700 1 -- 172.21.15.2:0/3593085546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7118108f10 msgr2=0x7f7118109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:53.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.751+0000 7f711d356700 1 --2- 172.21.15.2:0/3593085546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7118108f10 0x7f7118109320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f710c0099c0 tx=0x7f710c009cd0 comp rx=0 tx=0).stop 2024-05-14T09:31:53.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.751+0000 7f711d356700 1 -- 172.21.15.2:0/3593085546 shutdown_connections 2024-05-14T09:31:53.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.751+0000 7f711d356700 1 --2- 172.21.15.2:0/3593085546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71181045e0 0x7f71181049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:53.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.751+0000 7f711d356700 1 --2- 172.21.15.2:0/3593085546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7118109860 0x7f71181111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:53.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.751+0000 7f711d356700 1 --2- 172.21.15.2:0/3593085546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7118108f10 0x7f7118109320 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:53.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.751+0000 7f711d356700 1 -- 172.21.15.2:0/3593085546 >> 172.21.15.2:0/3593085546 conn(0x7f71180ffed0 msgr2=0x7f7118102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:53.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.751+0000 7f711d356700 1 -- 172.21.15.2:0/3593085546 shutdown_connections 2024-05-14T09:31:53.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.751+0000 7f711d356700 1 -- 172.21.15.2:0/3593085546 wait complete. 2024-05-14T09:31:53.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.752+0000 7f711d356700 1 Processor -- start 2024-05-14T09:31:53.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.752+0000 7f711d356700 1 -- start start 2024-05-14T09:31:53.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.752+0000 7f711d356700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71181045e0 0x7f711819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:53.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.752+0000 7f711d356700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7118108f10 0x7f711819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:53.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.752+0000 7f711d356700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7118109860 0x7f711819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:53.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.752+0000 7f711d356700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7118114490 con 0x7f7118108f10 2024-05-14T09:31:53.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.752+0000 7f711d356700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7118114310 con 0x7f71181045e0 2024-05-14T09:31:53.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.752+0000 7f711d356700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7118114610 con 0x7f7118109860 2024-05-14T09:31:53.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.752+0000 7f71167fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7118108f10 0x7f711819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:53.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.752+0000 7f71167fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7118108f10 0x7f711819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59444/0 (socket says 172.21.15.2:59444) 2024-05-14T09:31:53.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.752+0000 7f71167fc700 1 -- 172.21.15.2:0/174316349 learned_addr learned my addr 172.21.15.2:0/174316349 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:53.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.752+0000 7f71177fe700 1 --2- 172.21.15.2:0/174316349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7118109860 0x7f711819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:53.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.752+0000 7f7116ffd700 1 --2- 172.21.15.2:0/174316349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71181045e0 0x7f711819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:53.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.753+0000 7f71167fc700 1 -- 172.21.15.2:0/174316349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7118109860 msgr2=0x7f711819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:53.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.753+0000 7f71167fc700 1 --2- 172.21.15.2:0/174316349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7118109860 0x7f711819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:53.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.753+0000 7f71167fc700 1 -- 172.21.15.2:0/174316349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71181045e0 msgr2=0x7f711819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:53.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.753+0000 7f71167fc700 1 --2- 172.21.15.2:0/174316349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71181045e0 0x7f711819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:53.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.753+0000 7f71167fc700 1 -- 172.21.15.2:0/174316349 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711819fcd0 con 0x7f7118108f10 2024-05-14T09:31:53.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.753+0000 7f71177fe700 1 --2- 172.21.15.2:0/174316349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7118109860 0x7f711819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:31:53.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.753+0000 7f71167fc700 1 --2- 172.21.15.2:0/174316349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7118108f10 0x7f711819b3a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f710c005f50 tx=0x7f710c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:53.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.753+0000 7f70fffff700 1 -- 172.21.15.2:0/174316349 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c026070 con 0x7f7118108f10 2024-05-14T09:31:53.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.753+0000 7f711d356700 1 -- 172.21.15.2:0/174316349 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711819ff60 con 0x7f7118108f10 2024-05-14T09:31:53.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.753+0000 7f70fffff700 1 -- 172.21.15.2:0/174316349 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f710c003c70 con 0x7f7118108f10 2024-05-14T09:31:53.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.753+0000 7f70fffff700 1 -- 172.21.15.2:0/174316349 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c00f9c0 con 0x7f7118108f10 2024-05-14T09:31:53.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.753+0000 7f711d356700 1 -- 172.21.15.2:0/174316349 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71181a0440 con 0x7f7118108f10 2024-05-14T09:31:53.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.754+0000 7f711d356700 1 -- 172.21.15.2:0/174316349 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71180512c0 con 0x7f7118108f10 2024-05-14T09:31:53.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.756+0000 7f70fffff700 1 -- 172.21.15.2:0/174316349 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f710c029070 con 0x7f7118108f10 2024-05-14T09:31:53.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.757+0000 7f70fffff700 1 --2- 172.21.15.2:0/174316349 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f71000b2880 0x7f71000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:53.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.757+0000 7f7116ffd700 1 --2- 172.21.15.2:0/174316349 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f71000b2880 0x7f71000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:53.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.757+0000 7f70fffff700 1 -- 172.21.15.2:0/174316349 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f710c0e31e0 con 0x7f7118108f10 2024-05-14T09:31:53.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.757+0000 7f7116ffd700 1 --2- 172.21.15.2:0/174316349 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f71000b2880 0x7f71000b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7108009c60 tx=0x7f7108009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:53.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:53.760+0000 7f70fffff700 1 -- 172.21.15.2:0/174316349 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f710c030080 con 0x7f7118108f10 2024-05-14T09:31:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:54.027+0000 7f711d356700 1 -- 172.21.15.2:0/174316349 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f711819c160 con 0x7f7118108f10 2024-05-14T09:31:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:54 smithi002 ceph-mon[31557]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-14T09:31:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:54 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:54 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:54.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:54 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:54.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:54 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:54.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:54 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/174316349' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-14T09:31:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:54 smithi029 ceph-mon[36249]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-14T09:31:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:54 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:54 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:54 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:54 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:54 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/174316349' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-14T09:31:54.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:54.691+0000 7f70fffff700 1 -- 172.21.15.2:0/174316349 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f710c013070 con 0x7f7118108f10 2024-05-14T09:31:54.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:54.694+0000 7f711d356700 1 -- 172.21.15.2:0/174316349 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f71000b2880 msgr2=0x7f71000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:54.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:54.694+0000 7f711d356700 1 --2- 172.21.15.2:0/174316349 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f71000b2880 0x7f71000b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7108009c60 tx=0x7f7108009260 comp rx=0 tx=0).stop 2024-05-14T09:31:54.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:54.694+0000 7f711d356700 1 -- 172.21.15.2:0/174316349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7118108f10 msgr2=0x7f711819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:54.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:54.694+0000 7f711d356700 1 --2- 172.21.15.2:0/174316349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7118108f10 0x7f711819b3a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f710c005f50 tx=0x7f710c004270 comp rx=0 tx=0).stop 2024-05-14T09:31:54.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:54.694+0000 7f711d356700 1 -- 172.21.15.2:0/174316349 shutdown_connections 2024-05-14T09:31:54.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:54.694+0000 7f711d356700 1 --2- 172.21.15.2:0/174316349 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f71000b2880 0x7f71000b4d90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:54.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:54.694+0000 7f711d356700 1 --2- 172.21.15.2:0/174316349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71181045e0 0x7f711819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:54.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:54.694+0000 7f711d356700 1 --2- 172.21.15.2:0/174316349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7118109860 0x7f711819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:54.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:54.694+0000 7f711d356700 1 --2- 172.21.15.2:0/174316349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7118108f10 0x7f711819b3a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:54.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:54.694+0000 7f711d356700 1 -- 172.21.15.2:0/174316349 >> 172.21.15.2:0/174316349 conn(0x7f71180ffed0 msgr2=0x7f71181086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:54.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:54.694+0000 7f711d356700 1 -- 172.21.15.2:0/174316349 shutdown_connections 2024-05-14T09:31:54.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:54.694+0000 7f711d356700 1 -- 172.21.15.2:0/174316349 wait complete. 2024-05-14T09:31:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:54 smithi026 ceph-mon[36056]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-14T09:31:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:54 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:54 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:54 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:54.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:54 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:54.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:54 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/174316349' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-14T09:31:55.354 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:55 smithi002 ceph-mon[31557]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-14T09:31:55.354 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:55 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/174316349' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-14T09:31:55.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:55 smithi002 ceph-mon[31557]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-14T09:31:55.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:55 smithi002 ceph-mon[31557]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T09:31:55.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:55 smithi002 ceph-mon[31557]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-14T09:31:55.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:55 smithi002 ceph-mon[31557]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-14T09:31:55.383 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-14T09:31:55.651 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:55.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:55 smithi029 ceph-mon[36249]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-14T09:31:55.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:55 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/174316349' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-14T09:31:55.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:55 smithi029 ceph-mon[36249]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-14T09:31:55.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:55 smithi029 ceph-mon[36249]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T09:31:55.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:55 smithi029 ceph-mon[36249]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-14T09:31:55.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:55 smithi029 ceph-mon[36249]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-14T09:31:55.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:55 smithi026 ceph-mon[36056]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-14T09:31:55.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:55 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/174316349' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-14T09:31:55.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:55 smithi026 ceph-mon[36056]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-14T09:31:55.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:55 smithi026 ceph-mon[36056]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T09:31:55.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:55 smithi026 ceph-mon[36056]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-14T09:31:55.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:55 smithi026 ceph-mon[36056]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-14T09:31:56.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:56 smithi002 ceph-mon[31557]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-14T09:31:56.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:56 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:56.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:56 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:56.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:56 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] up:active 2024-05-14T09:31:56.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:56 smithi002 ceph-mon[31557]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-14T09:31:56.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:56 smithi002 ceph-mon[31557]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-14T09:31:56.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:56 smithi002 ceph-mon[31557]: Cluster is now healthy 2024-05-14T09:31:56.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:56 smithi002 ceph-mon[31557]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-14T09:31:56.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:56 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-14T09:31:56.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:56 smithi002 ceph-mon[31557]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-14T09:31:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:56 smithi029 ceph-mon[36249]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-14T09:31:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:56 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:56 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:56 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] up:active 2024-05-14T09:31:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:56 smithi029 ceph-mon[36249]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-14T09:31:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:56 smithi029 ceph-mon[36249]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-14T09:31:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:56 smithi029 ceph-mon[36249]: Cluster is now healthy 2024-05-14T09:31:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:56 smithi029 ceph-mon[36249]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-14T09:31:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:56 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-14T09:31:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:56 smithi029 ceph-mon[36249]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-14T09:31:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:56 smithi026 ceph-mon[36056]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-14T09:31:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:56 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:56 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:56 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] up:active 2024-05-14T09:31:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:56 smithi026 ceph-mon[36056]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-14T09:31:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:56 smithi026 ceph-mon[36056]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-14T09:31:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:56 smithi026 ceph-mon[36056]: Cluster is now healthy 2024-05-14T09:31:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:56 smithi026 ceph-mon[36056]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-14T09:31:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:56 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-14T09:31:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:56 smithi026 ceph-mon[36056]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-14T09:31:57.361 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:31:57.361 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:31:57.361 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:57.361 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:57 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] up:active 2024-05-14T09:31:57.362 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:57 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-14T09:31:57.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.361+0000 7fa6a8a35700 1 -- 172.21.15.2:0/1999019228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6a4109850 msgr2=0x7fa6a41111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:57.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.361+0000 7fa6a8a35700 1 --2- 172.21.15.2:0/1999019228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6a4109850 0x7fa6a41111b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00b0d0 tx=0x7fa69c00b3e0 comp rx=0 tx=0).stop 2024-05-14T09:31:57.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.361+0000 7fa6a8a35700 1 -- 172.21.15.2:0/1999019228 shutdown_connections 2024-05-14T09:31:57.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.361+0000 7fa6a8a35700 1 --2- 172.21.15.2:0/1999019228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a41045d0 0x7fa6a41049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:57.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.361+0000 7fa6a8a35700 1 --2- 172.21.15.2:0/1999019228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6a4109850 0x7fa6a41111b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:57.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.361+0000 7fa6a8a35700 1 --2- 172.21.15.2:0/1999019228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6a4108f00 0x7fa6a4109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:57.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.361+0000 7fa6a8a35700 1 -- 172.21.15.2:0/1999019228 >> 172.21.15.2:0/1999019228 conn(0x7fa6a40ffea0 msgr2=0x7fa6a41022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:57.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.361+0000 7fa6a8a35700 1 -- 172.21.15.2:0/1999019228 shutdown_connections 2024-05-14T09:31:57.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.361+0000 7fa6a8a35700 1 -- 172.21.15.2:0/1999019228 wait complete. 2024-05-14T09:31:57.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.362+0000 7fa6a8a35700 1 Processor -- start 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.362+0000 7fa6a8a35700 1 -- start start 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a8a35700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6a41045d0 0x7fa6a419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a8a35700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a4108f00 0x7fa6a419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a8a35700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6a4109850 0x7fa6a419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a8a35700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a4114480 con 0x7fa6a4109850 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a8a35700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a4114300 con 0x7fa6a4108f00 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a8a35700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a4114600 con 0x7fa6a41045d0 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6a4109850 0x7fa6a419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6a4109850 0x7fa6a419f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59458/0 (socket says 172.21.15.2:59458) 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a2d9d700 1 -- 172.21.15.2:0/942238110 learned_addr learned my addr 172.21.15.2:0/942238110 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a1d9b700 1 --2- 172.21.15.2:0/942238110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a4108f00 0x7fa6a419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a259c700 1 --2- 172.21.15.2:0/942238110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6a41045d0 0x7fa6a419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a2d9d700 1 -- 172.21.15.2:0/942238110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6a41045d0 msgr2=0x7fa6a419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a2d9d700 1 --2- 172.21.15.2:0/942238110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6a41045d0 0x7fa6a419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a2d9d700 1 -- 172.21.15.2:0/942238110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a4108f00 msgr2=0x7fa6a419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:57.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a2d9d700 1 --2- 172.21.15.2:0/942238110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a4108f00 0x7fa6a419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:57.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a2d9d700 1 -- 172.21.15.2:0/942238110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a419fc30 con 0x7fa6a4109850 2024-05-14T09:31:57.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a1d9b700 1 --2- 172.21.15.2:0/942238110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a4108f00 0x7fa6a419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:31:57.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.363+0000 7fa6a2d9d700 1 --2- 172.21.15.2:0/942238110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6a4109850 0x7fa6a419f590 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa69c007a00 tx=0x7fa69c007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:57.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.364+0000 7fa6937fe700 1 -- 172.21.15.2:0/942238110 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c01f070 con 0x7fa6a4109850 2024-05-14T09:31:57.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.364+0000 7fa6a8a35700 1 -- 172.21.15.2:0/942238110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a419fec0 con 0x7fa6a4109850 2024-05-14T09:31:57.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.364+0000 7fa6937fe700 1 -- 172.21.15.2:0/942238110 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa69c004c50 con 0x7fa6a4109850 2024-05-14T09:31:57.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.364+0000 7fa6937fe700 1 -- 172.21.15.2:0/942238110 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c01b690 con 0x7fa6a4109850 2024-05-14T09:31:57.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.364+0000 7fa6a8a35700 1 -- 172.21.15.2:0/942238110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a41a03a0 con 0x7fa6a4109850 2024-05-14T09:31:57.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.365+0000 7fa6a8a35700 1 -- 172.21.15.2:0/942238110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a40512c0 con 0x7fa6a4109850 2024-05-14T09:31:57.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.366+0000 7fa6937fe700 1 -- 172.21.15.2:0/942238110 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fa69c022070 con 0x7fa6a4109850 2024-05-14T09:31:57.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.367+0000 7fa6937fe700 1 --2- 172.21.15.2:0/942238110 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa68c0b2880 0x7fa68c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:31:57.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.367+0000 7fa6a259c700 1 --2- 172.21.15.2:0/942238110 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa68c0b2880 0x7fa68c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:31:57.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.367+0000 7fa6937fe700 1 -- 172.21.15.2:0/942238110 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fa69c00f020 con 0x7fa6a4109850 2024-05-14T09:31:57.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.368+0000 7fa6a259c700 1 --2- 172.21.15.2:0/942238110 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa68c0b2880 0x7fa68c0b4d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa694009680 tx=0x7fa694009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:31:57.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.372+0000 7fa6937fe700 1 -- 172.21.15.2:0/942238110 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa69c029020 con 0x7fa6a4109850 2024-05-14T09:31:57.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:57.637+0000 7fa6a8a35700 1 -- 172.21.15.2:0/942238110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fa6a402e5f0 con 0x7fa6a4109850 2024-05-14T09:31:57.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:31:57.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:31:57.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:57.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:57 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] up:active 2024-05-14T09:31:57.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:57 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-14T09:31:57.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:31:57.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:31:57.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:31:57.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:57 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] up:active 2024-05-14T09:31:57.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:57 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-14T09:31:58.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:58.226+0000 7fa6937fe700 1 -- 172.21.15.2:0/942238110 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7fa69c0ab0f0 con 0x7fa6a4109850 2024-05-14T09:31:58.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:58.229+0000 7fa6a8a35700 1 -- 172.21.15.2:0/942238110 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa68c0b2880 msgr2=0x7fa68c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:58.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:58.229+0000 7fa6a8a35700 1 --2- 172.21.15.2:0/942238110 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa68c0b2880 0x7fa68c0b4d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa694009680 tx=0x7fa694009260 comp rx=0 tx=0).stop 2024-05-14T09:31:58.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:58.229+0000 7fa6a8a35700 1 -- 172.21.15.2:0/942238110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6a4109850 msgr2=0x7fa6a419f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:31:58.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:58.229+0000 7fa6a8a35700 1 --2- 172.21.15.2:0/942238110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6a4109850 0x7fa6a419f590 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa69c007a00 tx=0x7fa69c007a30 comp rx=0 tx=0).stop 2024-05-14T09:31:58.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:58.229+0000 7fa6a8a35700 1 -- 172.21.15.2:0/942238110 shutdown_connections 2024-05-14T09:31:58.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:58.229+0000 7fa6a8a35700 1 --2- 172.21.15.2:0/942238110 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa68c0b2880 0x7fa68c0b4d90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:58.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:58.229+0000 7fa6a8a35700 1 --2- 172.21.15.2:0/942238110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6a4109850 0x7fa6a419f590 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:58.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:58.229+0000 7fa6a8a35700 1 --2- 172.21.15.2:0/942238110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa6a41045d0 0x7fa6a419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:58.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:58.229+0000 7fa6a8a35700 1 --2- 172.21.15.2:0/942238110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a4108f00 0x7fa6a419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:31:58.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:58.229+0000 7fa6a8a35700 1 -- 172.21.15.2:0/942238110 >> 172.21.15.2:0/942238110 conn(0x7fa6a40ffea0 msgr2=0x7fa6a4108640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:31:58.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:58.229+0000 7fa6a8a35700 1 -- 172.21.15.2:0/942238110 shutdown_connections 2024-05-14T09:31:58.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:31:58.229+0000 7fa6a8a35700 1 -- 172.21.15.2:0/942238110 wait complete. 2024-05-14T09:31:58.603 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:58 smithi002 ceph-mon[31557]: pgmap v332: 129 pgs: 129 active+clean; 582 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 3 op/s 2024-05-14T09:31:58.603 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:58 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/942238110' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T09:31:58.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:58 smithi029 ceph-mon[36249]: pgmap v332: 129 pgs: 129 active+clean; 582 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 3 op/s 2024-05-14T09:31:58.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:58 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/942238110' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T09:31:58.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:58 smithi026 ceph-mon[36056]: pgmap v332: 129 pgs: 129 active+clean; 582 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 3 op/s 2024-05-14T09:31:58.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:58 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/942238110' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T09:31:58.941 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-14T09:31:59.198 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:31:59.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:59 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/942238110' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-14T09:31:59.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:59 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-14T09:31:59.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:31:59 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:31:59.452 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 09:31:59 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b[66741]: 2024-05-14T09:31:59.109+0000 7f081c936700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-14T09:31:59.452 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 09:31:59 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b[66741]: 2024-05-14T09:31:59.109+0000 7f081c936700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-14T09:31:59.549 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:59 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/942238110' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-14T09:31:59.549 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:59 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-14T09:31:59.549 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:31:59 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:31:59.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:59 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/942238110' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-14T09:31:59.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:59 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-14T09:31:59.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:31:59 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:32:00.314 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:00 smithi002 ceph-mon[31557]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-14T09:32:00.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:00 smithi029 ceph-mon[36249]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-14T09:32:00.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:00 smithi026 ceph-mon[36056]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-14T09:32:00.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.887+0000 7f8231ed5700 1 -- 172.21.15.2:0/2635071125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c103550 msgr2=0x7f822c103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:00.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.887+0000 7f8231ed5700 1 --2- 172.21.15.2:0/2635071125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c103550 0x7f822c103960 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f82140099c0 tx=0x7f8214009cd0 comp rx=0 tx=0).stop 2024-05-14T09:32:00.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.887+0000 7f8231ed5700 1 -- 172.21.15.2:0/2635071125 shutdown_connections 2024-05-14T09:32:00.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.887+0000 7f8231ed5700 1 --2- 172.21.15.2:0/2635071125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f822c108780 0x7f822c1100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:00.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.887+0000 7f8231ed5700 1 --2- 172.21.15.2:0/2635071125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c103550 0x7f822c103960 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:00.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.887+0000 7f8231ed5700 1 --2- 172.21.15.2:0/2635071125 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f822c107e30 0x7f822c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:00.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.887+0000 7f8231ed5700 1 -- 172.21.15.2:0/2635071125 >> 172.21.15.2:0/2635071125 conn(0x7f822c078e00 msgr2=0x7f822c079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:00.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.887+0000 7f8231ed5700 1 -- 172.21.15.2:0/2635071125 shutdown_connections 2024-05-14T09:32:00.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.887+0000 7f8231ed5700 1 -- 172.21.15.2:0/2635071125 wait complete. 2024-05-14T09:32:00.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.888+0000 7f8231ed5700 1 Processor -- start 2024-05-14T09:32:00.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.888+0000 7f8231ed5700 1 -- start start 2024-05-14T09:32:00.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.888+0000 7f8231ed5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f822c103550 0x7f822c19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:00.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.888+0000 7f8231ed5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c107e30 0x7f822c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.888+0000 7f8231ed5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f822c108780 0x7f822c19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.888+0000 7f8231ed5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f822c1133b0 con 0x7f822c107e30 2024-05-14T09:32:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.888+0000 7f8231ed5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f822c113230 con 0x7f822c103550 2024-05-14T09:32:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.888+0000 7f8231ed5700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f822c113530 con 0x7f822c108780 2024-05-14T09:32:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.888+0000 7f822affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c107e30 0x7f822c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.888+0000 7f822affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c107e30 0x7f822c19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52028/0 (socket says 172.21.15.2:52028) 2024-05-14T09:32:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.888+0000 7f822affd700 1 -- 172.21.15.2:0/2412088905 learned_addr learned my addr 172.21.15.2:0/2412088905 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.888+0000 7f822bfff700 1 --2- 172.21.15.2:0/2412088905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f822c108780 0x7f822c19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.889+0000 7f822b7fe700 1 --2- 172.21.15.2:0/2412088905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f822c103550 0x7f822c19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.889+0000 7f822affd700 1 -- 172.21.15.2:0/2412088905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f822c108780 msgr2=0x7f822c19c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.889+0000 7f822affd700 1 --2- 172.21.15.2:0/2412088905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f822c108780 0x7f822c19c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.889+0000 7f822affd700 1 -- 172.21.15.2:0/2412088905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f822c103550 msgr2=0x7f822c19bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.889+0000 7f822affd700 1 --2- 172.21.15.2:0/2412088905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f822c103550 0x7f822c19bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.889+0000 7f822affd700 1 -- 172.21.15.2:0/2412088905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822c1a70c0 con 0x7f822c107e30 2024-05-14T09:32:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.889+0000 7f822affd700 1 --2- 172.21.15.2:0/2412088905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c107e30 0x7f822c19c2b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e970 tx=0x7f821c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.889+0000 7f8228ff9700 1 -- 172.21.15.2:0/2412088905 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c004a30 con 0x7f822c107e30 2024-05-14T09:32:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.889+0000 7f8228ff9700 1 -- 172.21.15.2:0/2412088905 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f821c00cc90 con 0x7f822c107e30 2024-05-14T09:32:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.890+0000 7f8231ed5700 1 -- 172.21.15.2:0/2412088905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822c199bc0 con 0x7f822c107e30 2024-05-14T09:32:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.890+0000 7f8228ff9700 1 -- 172.21.15.2:0/2412088905 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c005610 con 0x7f822c107e30 2024-05-14T09:32:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.890+0000 7f8231ed5700 1 -- 172.21.15.2:0/2412088905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f822c19a0d0 con 0x7f822c107e30 2024-05-14T09:32:00.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.891+0000 7f8231ed5700 1 -- 172.21.15.2:0/2412088905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f822c1049b0 con 0x7f822c107e30 2024-05-14T09:32:00.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.893+0000 7f8228ff9700 1 -- 172.21.15.2:0/2412088905 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f821c004bd0 con 0x7f822c107e30 2024-05-14T09:32:00.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.894+0000 7f8228ff9700 1 --2- 172.21.15.2:0/2412088905 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f82180b27d0 0x7f82180b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:00.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.894+0000 7f822b7fe700 1 --2- 172.21.15.2:0/2412088905 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f82180b27d0 0x7f82180b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:00.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.894+0000 7f8228ff9700 1 -- 172.21.15.2:0/2412088905 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f821c0e2950 con 0x7f822c107e30 2024-05-14T09:32:00.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.895+0000 7f822b7fe700 1 --2- 172.21.15.2:0/2412088905 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f82180b27d0 0x7f82180b4ce0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8214000c00 tx=0x7f821401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:00.898+0000 7f8228ff9700 1 -- 172.21.15.2:0/2412088905 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f821c0aa3c0 con 0x7f822c107e30 2024-05-14T09:32:01.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:01.166+0000 7f8231ed5700 1 -- 172.21.15.2:0/2412088905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f822c0512c0 con 0x7f822c107e30 2024-05-14T09:32:01.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:01.244+0000 7f8228ff9700 1 -- 172.21.15.2:0/2412088905 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f821c0a9cc0 con 0x7f822c107e30 2024-05-14T09:32:01.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:01.247+0000 7f8231ed5700 1 -- 172.21.15.2:0/2412088905 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f82180b27d0 msgr2=0x7f82180b4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:01.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:01.247+0000 7f8231ed5700 1 --2- 172.21.15.2:0/2412088905 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f82180b27d0 0x7f82180b4ce0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8214000c00 tx=0x7f821401a040 comp rx=0 tx=0).stop 2024-05-14T09:32:01.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:01.247+0000 7f8231ed5700 1 -- 172.21.15.2:0/2412088905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c107e30 msgr2=0x7f822c19c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:01.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:01.247+0000 7f8231ed5700 1 --2- 172.21.15.2:0/2412088905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c107e30 0x7f822c19c2b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e970 tx=0x7f821c00ec80 comp rx=0 tx=0).stop 2024-05-14T09:32:01.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:01.247+0000 7f8231ed5700 1 -- 172.21.15.2:0/2412088905 shutdown_connections 2024-05-14T09:32:01.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:01.247+0000 7f8231ed5700 1 --2- 172.21.15.2:0/2412088905 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f82180b27d0 0x7f82180b4ce0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:01.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:01.247+0000 7f8231ed5700 1 --2- 172.21.15.2:0/2412088905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f822c103550 0x7f822c19bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:01.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:01.247+0000 7f8231ed5700 1 --2- 172.21.15.2:0/2412088905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f822c108780 0x7f822c19c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:01.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:01.247+0000 7f8231ed5700 1 --2- 172.21.15.2:0/2412088905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c107e30 0x7f822c19c2b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:01.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:01.247+0000 7f8231ed5700 1 -- 172.21.15.2:0/2412088905 >> 172.21.15.2:0/2412088905 conn(0x7f822c078e00 msgr2=0x7f822c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:01.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:01.247+0000 7f8231ed5700 1 -- 172.21.15.2:0/2412088905 shutdown_connections 2024-05-14T09:32:01.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:01.247+0000 7f8231ed5700 1 -- 172.21.15.2:0/2412088905 wait complete. 2024-05-14T09:32:01.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:01 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2412088905' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-14T09:32:01.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:01 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2412088905' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-14T09:32:01.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:01 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2412088905' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-14T09:32:01.958 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-14T09:32:01.958 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-14T09:32:02.208 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:02.567 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:02 smithi002 ceph-mon[31557]: pgmap v334: 129 pgs: 129 active+clean; 584 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-14T09:32:02.567 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:02 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2412088905' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-14T09:32:02.568 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:02 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:32:02.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:02 smithi029 ceph-mon[36249]: pgmap v334: 129 pgs: 129 active+clean; 584 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-14T09:32:02.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:02 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2412088905' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-14T09:32:02.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:02 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:32:02.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:02 smithi026 ceph-mon[36056]: pgmap v334: 129 pgs: 129 active+clean; 584 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-14T09:32:02.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:02 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2412088905' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-14T09:32:02.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:02 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:32:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.904+0000 7f0cb973b700 1 -- 172.21.15.2:0/1464079427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0cb4104650 msgr2=0x7f0cb4104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.904+0000 7f0cb973b700 1 --2- 172.21.15.2:0/1464079427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0cb4104650 0x7f0cb4104a60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40099c0 tx=0x7f0ca4009cd0 comp rx=0 tx=0).stop 2024-05-14T09:32:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.904+0000 7f0cb973b700 1 -- 172.21.15.2:0/1464079427 shutdown_connections 2024-05-14T09:32:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.904+0000 7f0cb973b700 1 --2- 172.21.15.2:0/1464079427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cb4109880 0x7f0cb41111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.904+0000 7f0cb973b700 1 --2- 172.21.15.2:0/1464079427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0cb4104650 0x7f0cb4104a60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.904+0000 7f0cb973b700 1 --2- 172.21.15.2:0/1464079427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb4108f30 0x7f0cb4109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.904+0000 7f0cb973b700 1 -- 172.21.15.2:0/1464079427 >> 172.21.15.2:0/1464079427 conn(0x7f0cb4079f00 msgr2=0x7f0cb407a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.904+0000 7f0cb973b700 1 -- 172.21.15.2:0/1464079427 shutdown_connections 2024-05-14T09:32:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.904+0000 7f0cb973b700 1 -- 172.21.15.2:0/1464079427 wait complete. 2024-05-14T09:32:03.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.905+0000 7f0cb973b700 1 Processor -- start 2024-05-14T09:32:03.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.905+0000 7f0cb973b700 1 -- start start 2024-05-14T09:32:03.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.905+0000 7f0cb973b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cb4104650 0x7f0cb4075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:03.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.905+0000 7f0cb973b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb4108f30 0x7f0cb4076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:03.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.905+0000 7f0cb973b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0cb4109880 0x7f0cb4076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:03.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.905+0000 7f0cb973b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41144b0 con 0x7f0cb4109880 2024-05-14T09:32:03.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.905+0000 7f0cb973b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb4114330 con 0x7f0cb4108f30 2024-05-14T09:32:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.905+0000 7f0cb973b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb4114630 con 0x7f0cb4104650 2024-05-14T09:32:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.905+0000 7f0cb37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0cb4109880 0x7f0cb4076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.905+0000 7f0cb2ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cb4104650 0x7f0cb4075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.905+0000 7f0cb37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0cb4109880 0x7f0cb4076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52048/0 (socket says 172.21.15.2:52048) 2024-05-14T09:32:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.905+0000 7f0cb37fe700 1 -- 172.21.15.2:0/1748786275 learned_addr learned my addr 172.21.15.2:0/1748786275 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.905+0000 7f0cb27fc700 1 --2- 172.21.15.2:0/1748786275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb4108f30 0x7f0cb4076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.906+0000 7f0cb37fe700 1 -- 172.21.15.2:0/1748786275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cb4104650 msgr2=0x7f0cb4075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.906+0000 7f0cb37fe700 1 --2- 172.21.15.2:0/1748786275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cb4104650 0x7f0cb4075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.906+0000 7f0cb37fe700 1 -- 172.21.15.2:0/1748786275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb4108f30 msgr2=0x7f0cb4076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.906+0000 7f0cb37fe700 1 --2- 172.21.15.2:0/1748786275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb4108f30 0x7f0cb4076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.906+0000 7f0cb37fe700 1 -- 172.21.15.2:0/1748786275 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb41a91c0 con 0x7f0cb4109880 2024-05-14T09:32:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.906+0000 7f0cb37fe700 1 --2- 172.21.15.2:0/1748786275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0cb4109880 0x7f0cb4076540 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0cac007ee0 tx=0x7f0cac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.906+0000 7f0c9bfff700 1 -- 172.21.15.2:0/1748786275 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac004bb0 con 0x7f0cb4109880 2024-05-14T09:32:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.906+0000 7f0c9bfff700 1 -- 172.21.15.2:0/1748786275 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cac004d50 con 0x7f0cb4109880 2024-05-14T09:32:03.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.906+0000 7f0cb973b700 1 -- 172.21.15.2:0/1748786275 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb41a94b0 con 0x7f0cb4109880 2024-05-14T09:32:03.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.906+0000 7f0c9bfff700 1 -- 172.21.15.2:0/1748786275 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac005920 con 0x7f0cb4109880 2024-05-14T09:32:03.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.906+0000 7f0cb973b700 1 -- 172.21.15.2:0/1748786275 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb41a99a0 con 0x7f0cb4109880 2024-05-14T09:32:03.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.907+0000 7f0cb973b700 1 -- 172.21.15.2:0/1748786275 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb40512c0 con 0x7f0cb4109880 2024-05-14T09:32:03.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.909+0000 7f0c9bfff700 1 -- 172.21.15.2:0/1748786275 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f0cac01a020 con 0x7f0cb4109880 2024-05-14T09:32:03.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.909+0000 7f0c9bfff700 1 --2- 172.21.15.2:0/1748786275 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c9c0b2880 0x7f0c9c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:03.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.910+0000 7f0cb2ffd700 1 --2- 172.21.15.2:0/1748786275 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c9c0b2880 0x7f0c9c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:03.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.910+0000 7f0c9bfff700 1 -- 172.21.15.2:0/1748786275 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f0cac0e21e0 con 0x7f0cb4109880 2024-05-14T09:32:03.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.910+0000 7f0cb2ffd700 1 --2- 172.21.15.2:0/1748786275 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c9c0b2880 0x7f0c9c0b4d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009990 tx=0x7f0ca40036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:03.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:03.913+0000 7f0c9bfff700 1 -- 172.21.15.2:0/1748786275 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0cac010070 con 0x7f0cb4109880 2024-05-14T09:32:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:04.159+0000 7f0cb973b700 1 -- 172.21.15.2:0/1748786275 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0cb4063c00 con 0x7f0c9c0b2880 2024-05-14T09:32:04.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:04.187+0000 7f0c9bfff700 1 -- 172.21.15.2:0/1748786275 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0cb4063c00 con 0x7f0c9c0b2880 2024-05-14T09:32:04.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:04.189+0000 7f0cb973b700 1 -- 172.21.15.2:0/1748786275 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c9c0b2880 msgr2=0x7f0c9c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:04.189+0000 7f0cb973b700 1 --2- 172.21.15.2:0/1748786275 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c9c0b2880 0x7f0c9c0b4d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009990 tx=0x7f0ca40036e0 comp rx=0 tx=0).stop 2024-05-14T09:32:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:04.189+0000 7f0cb973b700 1 -- 172.21.15.2:0/1748786275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0cb4109880 msgr2=0x7f0cb4076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:04.189+0000 7f0cb973b700 1 --2- 172.21.15.2:0/1748786275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0cb4109880 0x7f0cb4076540 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0cac007ee0 tx=0x7f0cac00d3b0 comp rx=0 tx=0).stop 2024-05-14T09:32:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:04.189+0000 7f0cb973b700 1 -- 172.21.15.2:0/1748786275 shutdown_connections 2024-05-14T09:32:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:04.189+0000 7f0cb973b700 1 --2- 172.21.15.2:0/1748786275 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c9c0b2880 0x7f0c9c0b4d90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:04.189+0000 7f0cb973b700 1 --2- 172.21.15.2:0/1748786275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0cb4109880 0x7f0cb4076540 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:04.189+0000 7f0cb973b700 1 --2- 172.21.15.2:0/1748786275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0cb4104650 0x7f0cb4075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:04.189+0000 7f0cb973b700 1 --2- 172.21.15.2:0/1748786275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb4108f30 0x7f0cb4076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:04.189+0000 7f0cb973b700 1 -- 172.21.15.2:0/1748786275 >> 172.21.15.2:0/1748786275 conn(0x7f0cb4079f00 msgr2=0x7f0cb41020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:04.190+0000 7f0cb973b700 1 -- 172.21.15.2:0/1748786275 shutdown_connections 2024-05-14T09:32:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:04.190+0000 7f0cb973b700 1 -- 172.21.15.2:0/1748786275 wait complete. 2024-05-14T09:32:04.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:04 smithi002 ceph-mon[31557]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T09:32:04.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:32:04 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:32:04.165+0000 7f19a3751700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T09:32:04.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:32:04 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:32:04.165+0000 7f19a3751700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T09:32:04.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:32:04 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:32:04.165+0000 7f19a3751700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T09:32:04.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:32:04 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:32:04.165+0000 7f19a3751700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T09:32:04.421 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 09:32:04 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T09:32:04.165+0000 7f19a3751700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T09:32:04.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:04 smithi029 ceph-mon[36249]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T09:32:04.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:04 smithi026 ceph-mon[36056]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T09:32:04.852 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-14T09:32:05.104 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:05.484 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:05 smithi002 ceph-mon[31557]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:32:05.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:05 smithi029 ceph-mon[36249]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:32:05.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:05 smithi026 ceph-mon[36056]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:32:06.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:06 smithi002 ceph-mon[31557]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T09:32:06.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:06 smithi002 ceph-mon[31557]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-14T09:32:06.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:06 smithi029 ceph-mon[36249]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T09:32:06.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:06 smithi029 ceph-mon[36249]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-14T09:32:06.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:06 smithi026 ceph-mon[36056]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T09:32:06.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:06 smithi026 ceph-mon[36056]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-14T09:32:06.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.827+0000 7f2e7ff9d700 1 -- 172.21.15.2:0/3805507612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e78107e10 msgr2=0x7f2e78108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:06.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.827+0000 7f2e7ff9d700 1 --2- 172.21.15.2:0/3805507612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e78107e10 0x7f2e78108220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2e680099c0 tx=0x7f2e68009cd0 comp rx=0 tx=0).stop 2024-05-14T09:32:06.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.827+0000 7f2e7ff9d700 1 -- 172.21.15.2:0/3805507612 shutdown_connections 2024-05-14T09:32:06.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.827+0000 7f2e7ff9d700 1 --2- 172.21.15.2:0/3805507612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e78108760 0x7f2e781100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:06.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.827+0000 7f2e7ff9d700 1 --2- 172.21.15.2:0/3805507612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e781034e0 0x7f2e781038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:06.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.827+0000 7f2e7ff9d700 1 --2- 172.21.15.2:0/3805507612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e78107e10 0x7f2e78108220 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:06.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.827+0000 7f2e7ff9d700 1 -- 172.21.15.2:0/3805507612 >> 172.21.15.2:0/3805507612 conn(0x7f2e780fef30 msgr2=0x7f2e78101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:06.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.827+0000 7f2e7ff9d700 1 -- 172.21.15.2:0/3805507612 shutdown_connections 2024-05-14T09:32:06.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.827+0000 7f2e7ff9d700 1 -- 172.21.15.2:0/3805507612 wait complete. 2024-05-14T09:32:06.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.828+0000 7f2e7ff9d700 1 Processor -- start 2024-05-14T09:32:06.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.828+0000 7f2e7ff9d700 1 -- start start 2024-05-14T09:32:06.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.828+0000 7f2e7ff9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e781034e0 0x7f2e7819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:06.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.828+0000 7f2e7ff9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e78107e10 0x7f2e7819c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:06.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.828+0000 7f2e7ff9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e78108760 0x7f2e781a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:06.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.828+0000 7f2e7ff9d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2e78113370 con 0x7f2e78108760 2024-05-14T09:32:06.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.828+0000 7f2e7ff9d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781131f0 con 0x7f2e78107e10 2024-05-14T09:32:06.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.828+0000 7f2e7ff9d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781134f0 con 0x7f2e781034e0 2024-05-14T09:32:06.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.828+0000 7f2e7e53a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e78108760 0x7f2e781a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:06.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.828+0000 7f2e7dd39700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e781034e0 0x7f2e7819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:06.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.828+0000 7f2e7e53a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e78108760 0x7f2e781a6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52078/0 (socket says 172.21.15.2:52078) 2024-05-14T09:32:06.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.828+0000 7f2e7e53a700 1 -- 172.21.15.2:0/2033305517 learned_addr learned my addr 172.21.15.2:0/2033305517 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:06.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.828+0000 7f2e7d538700 1 --2- 172.21.15.2:0/2033305517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e78107e10 0x7f2e7819c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:06.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.829+0000 7f2e7e53a700 1 -- 172.21.15.2:0/2033305517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e781034e0 msgr2=0x7f2e7819bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:06.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.829+0000 7f2e7e53a700 1 --2- 172.21.15.2:0/2033305517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e781034e0 0x7f2e7819bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:06.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.829+0000 7f2e7e53a700 1 -- 172.21.15.2:0/2033305517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e78107e10 msgr2=0x7f2e7819c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:06.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.829+0000 7f2e7e53a700 1 --2- 172.21.15.2:0/2033305517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e78107e10 0x7f2e7819c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:06.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.829+0000 7f2e7e53a700 1 -- 172.21.15.2:0/2033305517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7819c9d0 con 0x7f2e78108760 2024-05-14T09:32:06.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.829+0000 7f2e7e53a700 1 --2- 172.21.15.2:0/2033305517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e78108760 0x7f2e781a6cf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000ada0 tx=0x7f2e7000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:06.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.829+0000 7f2e6effd700 1 -- 172.21.15.2:0/2033305517 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70017020 con 0x7f2e78108760 2024-05-14T09:32:06.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.829+0000 7f2e7ff9d700 1 -- 172.21.15.2:0/2033305517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e781a7290 con 0x7f2e78108760 2024-05-14T09:32:06.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.829+0000 7f2e6effd700 1 -- 172.21.15.2:0/2033305517 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e7000cd80 con 0x7f2e78108760 2024-05-14T09:32:06.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.829+0000 7f2e6effd700 1 -- 172.21.15.2:0/2033305517 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70005260 con 0x7f2e78108760 2024-05-14T09:32:06.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.829+0000 7f2e7ff9d700 1 -- 172.21.15.2:0/2033305517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e781a77a0 con 0x7f2e78108760 2024-05-14T09:32:06.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.830+0000 7f2e7ff9d700 1 -- 172.21.15.2:0/2033305517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e780512c0 con 0x7f2e78108760 2024-05-14T09:32:06.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.833+0000 7f2e6effd700 1 -- 172.21.15.2:0/2033305517 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f2e7000a090 con 0x7f2e78108760 2024-05-14T09:32:06.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.833+0000 7f2e6effd700 1 --2- 172.21.15.2:0/2033305517 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2e640b2990 0x7f2e640b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:06.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.833+0000 7f2e7dd39700 1 --2- 172.21.15.2:0/2033305517 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2e640b2990 0x7f2e640b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:06.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.833+0000 7f2e6effd700 1 -- 172.21.15.2:0/2033305517 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f2e700e22e0 con 0x7f2e78108760 2024-05-14T09:32:06.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.834+0000 7f2e7dd39700 1 --2- 172.21.15.2:0/2033305517 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2e640b2990 0x7f2e640b4ea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400ac50 tx=0x7f2e7400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:06.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:06.836+0000 7f2e6effd700 1 -- 172.21.15.2:0/2033305517 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2e70021020 con 0x7f2e78108760 2024-05-14T09:32:07.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:07.085+0000 7f2e7ff9d700 1 -- 172.21.15.2:0/2033305517 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2e78063c00 con 0x7f2e640b2990 2024-05-14T09:32:07.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:07.113+0000 7f2e6effd700 1 -- 172.21.15.2:0/2033305517 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2e78063c00 con 0x7f2e640b2990 2024-05-14T09:32:07.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:07.115+0000 7f2e7ff9d700 1 -- 172.21.15.2:0/2033305517 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2e640b2990 msgr2=0x7f2e640b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:07.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:07.115+0000 7f2e7ff9d700 1 --2- 172.21.15.2:0/2033305517 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2e640b2990 0x7f2e640b4ea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400ac50 tx=0x7f2e7400a380 comp rx=0 tx=0).stop 2024-05-14T09:32:07.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:07.115+0000 7f2e7ff9d700 1 -- 172.21.15.2:0/2033305517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e78108760 msgr2=0x7f2e781a6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:07.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:07.115+0000 7f2e7ff9d700 1 --2- 172.21.15.2:0/2033305517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e78108760 0x7f2e781a6cf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000ada0 tx=0x7f2e7000c330 comp rx=0 tx=0).stop 2024-05-14T09:32:07.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:07.115+0000 7f2e7ff9d700 1 -- 172.21.15.2:0/2033305517 shutdown_connections 2024-05-14T09:32:07.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:07.115+0000 7f2e7ff9d700 1 --2- 172.21.15.2:0/2033305517 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2e640b2990 0x7f2e640b4ea0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:07.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:07.115+0000 7f2e7ff9d700 1 --2- 172.21.15.2:0/2033305517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e78108760 0x7f2e781a6cf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:07.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:07.115+0000 7f2e7ff9d700 1 --2- 172.21.15.2:0/2033305517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2e781034e0 0x7f2e7819bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:07.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:07.115+0000 7f2e7ff9d700 1 --2- 172.21.15.2:0/2033305517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e78107e10 0x7f2e7819c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:07.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:07.115+0000 7f2e7ff9d700 1 -- 172.21.15.2:0/2033305517 >> 172.21.15.2:0/2033305517 conn(0x7f2e780fef30 msgr2=0x7f2e781096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:07.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:07.115+0000 7f2e7ff9d700 1 -- 172.21.15.2:0/2033305517 shutdown_connections 2024-05-14T09:32:07.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:07.115+0000 7f2e7ff9d700 1 -- 172.21.15.2:0/2033305517 wait complete. 2024-05-14T09:32:07.816 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-14T09:32:08.075 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:08.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:08 smithi002 ceph-mon[31557]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-14T09:32:08.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:08 smithi002 ceph-mon[31557]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:32:08.517 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:08 smithi026 ceph-mon[36056]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-14T09:32:08.518 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:08 smithi026 ceph-mon[36056]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:32:08.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:08 smithi029 ceph-mon[36249]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-14T09:32:08.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:08 smithi029 ceph-mon[36249]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:32:09.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.723+0000 7f6452861700 1 -- 172.21.15.2:0/1859193299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f644c108f10 msgr2=0x7f644c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:09.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.723+0000 7f6452861700 1 --2- 172.21.15.2:0/1859193299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f644c108f10 0x7f644c109320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f64340099c0 tx=0x7f6434009cd0 comp rx=0 tx=0).stop 2024-05-14T09:32:09.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.723+0000 7f6452861700 1 -- 172.21.15.2:0/1859193299 shutdown_connections 2024-05-14T09:32:09.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.723+0000 7f6452861700 1 --2- 172.21.15.2:0/1859193299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c1045e0 0x7f644c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:09.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.723+0000 7f6452861700 1 --2- 172.21.15.2:0/1859193299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f644c109860 0x7f644c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:09.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.723+0000 7f6452861700 1 --2- 172.21.15.2:0/1859193299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f644c108f10 0x7f644c109320 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:09.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.723+0000 7f6452861700 1 -- 172.21.15.2:0/1859193299 >> 172.21.15.2:0/1859193299 conn(0x7f644c0ffe70 msgr2=0x7f644c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:09.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.723+0000 7f6452861700 1 -- 172.21.15.2:0/1859193299 shutdown_connections 2024-05-14T09:32:09.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.724+0000 7f6452861700 1 -- 172.21.15.2:0/1859193299 wait complete. 2024-05-14T09:32:09.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.724+0000 7f6452861700 1 Processor -- start 2024-05-14T09:32:09.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.724+0000 7f6452861700 1 -- start start 2024-05-14T09:32:09.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f6452861700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f644c1045e0 0x7f644c19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:09.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f6452861700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f644c108f10 0x7f644c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:09.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f6452861700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c109860 0x7f644c1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:09.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f6452861700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f644c114470 con 0x7f644c1045e0 2024-05-14T09:32:09.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f6452861700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f644c1142f0 con 0x7f644c109860 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f6452861700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f644c1145f0 con 0x7f644c108f10 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f644bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f644c1045e0 0x7f644c19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f644bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f644c1045e0 0x7f644c19d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34260/0 (socket says 172.21.15.2:34260) 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f644bfff700 1 -- 172.21.15.2:0/4205065962 learned_addr learned my addr 172.21.15.2:0/4205065962 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f6443fff700 1 --2- 172.21.15.2:0/4205065962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f644c108f10 0x7f644c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f6450dfe700 1 --2- 172.21.15.2:0/4205065962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c109860 0x7f644c1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f6443fff700 1 -- 172.21.15.2:0/4205065962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c109860 msgr2=0x7f644c1a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f6443fff700 1 --2- 172.21.15.2:0/4205065962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c109860 0x7f644c1a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f6443fff700 1 -- 172.21.15.2:0/4205065962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f644c1045e0 msgr2=0x7f644c19d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f6443fff700 1 --2- 172.21.15.2:0/4205065962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f644c1045e0 0x7f644c19d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f6443fff700 1 -- 172.21.15.2:0/4205065962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c1a1ef0 con 0x7f644c108f10 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.725+0000 7f644bfff700 1 --2- 172.21.15.2:0/4205065962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f644c1045e0 0x7f644c19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.726+0000 7f6443fff700 1 --2- 172.21.15.2:0/4205065962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f644c108f10 0x7f644c19d5c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f64340098e0 tx=0x7f6434004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.726+0000 7f6449ffb700 1 -- 172.21.15.2:0/4205065962 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434026070 con 0x7f644c108f10 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.726+0000 7f6449ffb700 1 -- 172.21.15.2:0/4205065962 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6434003c70 con 0x7f644c108f10 2024-05-14T09:32:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.726+0000 7f6452861700 1 -- 172.21.15.2:0/4205065962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644c1a2180 con 0x7f644c108f10 2024-05-14T09:32:09.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.726+0000 7f6449ffb700 1 -- 172.21.15.2:0/4205065962 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643400f9c0 con 0x7f644c108f10 2024-05-14T09:32:09.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.726+0000 7f6452861700 1 -- 172.21.15.2:0/4205065962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644c1a2660 con 0x7f644c108f10 2024-05-14T09:32:09.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.727+0000 7f6452861700 1 -- 172.21.15.2:0/4205065962 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f644c0512c0 con 0x7f644c108f10 2024-05-14T09:32:09.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.729+0000 7f6449ffb700 1 -- 172.21.15.2:0/4205065962 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f6434029070 con 0x7f644c108f10 2024-05-14T09:32:09.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.729+0000 7f6449ffb700 1 --2- 172.21.15.2:0/4205065962 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64380b2910 0x7f64380b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:09.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.729+0000 7f644bfff700 1 --2- 172.21.15.2:0/4205065962 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64380b2910 0x7f64380b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:09.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.729+0000 7f6449ffb700 1 -- 172.21.15.2:0/4205065962 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f64340e3110 con 0x7f644c108f10 2024-05-14T09:32:09.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.730+0000 7f644bfff700 1 --2- 172.21.15.2:0/4205065962 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64380b2910 0x7f64380b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f643c011550 tx=0x7f643c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:09.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.733+0000 7f6449ffb700 1 -- 172.21.15.2:0/4205065962 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6434013070 con 0x7f644c108f10 2024-05-14T09:32:09.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.985+0000 7f6452861700 1 -- 172.21.15.2:0/4205065962 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f644c0759f0 con 0x7f64380b2910 2024-05-14T09:32:09.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.991+0000 7f6449ffb700 1 -- 172.21.15.2:0/4205065962 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f644c0759f0 con 0x7f64380b2910 2024-05-14T09:32:09.991 INFO:teuthology.orchestra.run.smithi002.stdout:/volumes/qa/sv_0/21e0f0b4-de4e-4f07-8e71-c1923c49882c 2024-05-14T09:32:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.994+0000 7f6452861700 1 -- 172.21.15.2:0/4205065962 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64380b2910 msgr2=0x7f64380b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.994+0000 7f6452861700 1 --2- 172.21.15.2:0/4205065962 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64380b2910 0x7f64380b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f643c011550 tx=0x7f643c009260 comp rx=0 tx=0).stop 2024-05-14T09:32:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.994+0000 7f6452861700 1 -- 172.21.15.2:0/4205065962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f644c108f10 msgr2=0x7f644c19d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.994+0000 7f6452861700 1 --2- 172.21.15.2:0/4205065962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f644c108f10 0x7f644c19d5c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f64340098e0 tx=0x7f6434004270 comp rx=0 tx=0).stop 2024-05-14T09:32:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.994+0000 7f6452861700 1 -- 172.21.15.2:0/4205065962 shutdown_connections 2024-05-14T09:32:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.994+0000 7f6452861700 1 --2- 172.21.15.2:0/4205065962 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64380b2910 0x7f64380b4e20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.994+0000 7f6452861700 1 --2- 172.21.15.2:0/4205065962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f644c109860 0x7f644c1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.994+0000 7f6452861700 1 --2- 172.21.15.2:0/4205065962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f644c1045e0 0x7f644c19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.994+0000 7f6452861700 1 --2- 172.21.15.2:0/4205065962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f644c108f10 0x7f644c19d5c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.994+0000 7f6452861700 1 -- 172.21.15.2:0/4205065962 >> 172.21.15.2:0/4205065962 conn(0x7f644c0ffe70 msgr2=0x7f644c10fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.994+0000 7f6452861700 1 -- 172.21.15.2:0/4205065962 shutdown_connections 2024-05-14T09:32:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:09.994+0000 7f6452861700 1 -- 172.21.15.2:0/4205065962 wait complete. 2024-05-14T09:32:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:10 smithi002 ceph-mon[31557]: pgmap v338: 129 pgs: 129 active+clean; 605 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-14T09:32:10.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:10 smithi029 ceph-mon[36249]: pgmap v338: 129 pgs: 129 active+clean; 605 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-14T09:32:10.701 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-14T09:32:10.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:10 smithi026 ceph-mon[36056]: pgmap v338: 129 pgs: 129 active+clean; 605 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-14T09:32:10.953 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:11.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:11 smithi002 ceph-mon[31557]: from='client.24658 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:32:11.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:11 smithi029 ceph-mon[36249]: from='client.24658 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:32:11.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:11 smithi026 ceph-mon[36056]: from='client.24658 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:32:12.485 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:12 smithi002 ceph-mon[31557]: pgmap v339: 129 pgs: 129 active+clean; 605 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-14T09:32:12.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:12 smithi029 ceph-mon[36249]: pgmap v339: 129 pgs: 129 active+clean; 605 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-14T09:32:12.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:12 smithi026 ceph-mon[36056]: pgmap v339: 129 pgs: 129 active+clean; 605 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-14T09:32:12.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.737+0000 7f2560e5a700 1 -- 172.21.15.2:0/2804599281 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f255c108740 msgr2=0x7f255c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:12.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.737+0000 7f2560e5a700 1 --2- 172.21.15.2:0/2804599281 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f255c108740 0x7f255c1100a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f255400b0d0 tx=0x7f255400b3e0 comp rx=0 tx=0).stop 2024-05-14T09:32:12.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.737+0000 7f2560e5a700 1 -- 172.21.15.2:0/2804599281 shutdown_connections 2024-05-14T09:32:12.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.737+0000 7f2560e5a700 1 --2- 172.21.15.2:0/2804599281 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f255c108740 0x7f255c1100a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:12.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.737+0000 7f2560e5a700 1 --2- 172.21.15.2:0/2804599281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f255c1034c0 0x7f255c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:12.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.737+0000 7f2560e5a700 1 --2- 172.21.15.2:0/2804599281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f255c107df0 0x7f255c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:12.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.737+0000 7f2560e5a700 1 -- 172.21.15.2:0/2804599281 >> 172.21.15.2:0/2804599281 conn(0x7f255c0fedf0 msgr2=0x7f255c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:12.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.737+0000 7f2560e5a700 1 -- 172.21.15.2:0/2804599281 shutdown_connections 2024-05-14T09:32:12.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.737+0000 7f2560e5a700 1 -- 172.21.15.2:0/2804599281 wait complete. 2024-05-14T09:32:12.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.738+0000 7f2560e5a700 1 Processor -- start 2024-05-14T09:32:12.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.738+0000 7f2560e5a700 1 -- start start 2024-05-14T09:32:12.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f2560e5a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f255c1034c0 0x7f255c19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:12.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f2560e5a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f255c107df0 0x7f255c19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:12.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f2560e5a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f255c108740 0x7f255c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:12.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f2560e5a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f255c113370 con 0x7f255c1034c0 2024-05-14T09:32:12.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f2560e5a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f255c1131f0 con 0x7f255c108740 2024-05-14T09:32:12.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f2560e5a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f255c1134f0 con 0x7f255c107df0 2024-05-14T09:32:12.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f255a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f255c1034c0 0x7f255c19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:12.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f255a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f255c1034c0 0x7f255c19bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34280/0 (socket says 172.21.15.2:34280) 2024-05-14T09:32:12.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f255a59c700 1 -- 172.21.15.2:0/2750281596 learned_addr learned my addr 172.21.15.2:0/2750281596 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:12.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f2559d9b700 1 --2- 172.21.15.2:0/2750281596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f255c107df0 0x7f255c19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:12.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f255ad9d700 1 --2- 172.21.15.2:0/2750281596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f255c108740 0x7f255c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:12.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f255a59c700 1 -- 172.21.15.2:0/2750281596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f255c107df0 msgr2=0x7f255c19c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:12.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f255a59c700 1 --2- 172.21.15.2:0/2750281596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f255c107df0 0x7f255c19c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:12.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f255a59c700 1 -- 172.21.15.2:0/2750281596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f255c108740 msgr2=0x7f255c19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:12.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f255a59c700 1 --2- 172.21.15.2:0/2750281596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f255c108740 0x7f255c19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:12.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f255a59c700 1 -- 172.21.15.2:0/2750281596 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255c1a6f10 con 0x7f255c1034c0 2024-05-14T09:32:12.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.739+0000 7f2559d9b700 1 --2- 172.21.15.2:0/2750281596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f255c107df0 0x7f255c19c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:32:12.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.740+0000 7f255a59c700 1 --2- 172.21.15.2:0/2750281596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f255c1034c0 0x7f255c19bc50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f254c00d950 tx=0x7f254c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:12.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.740+0000 7f25537fe700 1 -- 172.21.15.2:0/2750281596 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c016070 con 0x7f255c1034c0 2024-05-14T09:32:12.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.740+0000 7f25537fe700 1 -- 172.21.15.2:0/2750281596 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f254c00fd40 con 0x7f255c1034c0 2024-05-14T09:32:12.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.740+0000 7f2560e5a700 1 -- 172.21.15.2:0/2750281596 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f255c1a7200 con 0x7f255c1034c0 2024-05-14T09:32:12.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.740+0000 7f25537fe700 1 -- 172.21.15.2:0/2750281596 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c00e3b0 con 0x7f255c1034c0 2024-05-14T09:32:12.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.740+0000 7f2560e5a700 1 -- 172.21.15.2:0/2750281596 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f255c1a7790 con 0x7f255c1034c0 2024-05-14T09:32:12.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.741+0000 7f2560e5a700 1 -- 172.21.15.2:0/2750281596 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f255c0512c0 con 0x7f255c1034c0 2024-05-14T09:32:12.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.742+0000 7f25537fe700 1 -- 172.21.15.2:0/2750281596 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f254c019070 con 0x7f255c1034c0 2024-05-14T09:32:12.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.743+0000 7f25537fe700 1 --2- 172.21.15.2:0/2750281596 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f25480b2990 0x7f25480b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:12.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.743+0000 7f2559d9b700 1 --2- 172.21.15.2:0/2750281596 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f25480b2990 0x7f25480b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:12.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.743+0000 7f25537fe700 1 -- 172.21.15.2:0/2750281596 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f254c0e20b0 con 0x7f255c1034c0 2024-05-14T09:32:12.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.744+0000 7f2559d9b700 1 --2- 172.21.15.2:0/2750281596 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f25480b2990 0x7f25480b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f255c106750 tx=0x7f254400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:12.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.747+0000 7f25537fe700 1 -- 172.21.15.2:0/2750281596 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f254c0a9930 con 0x7f255c1034c0 2024-05-14T09:32:12.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:12.999+0000 7f2560e5a700 1 -- 172.21.15.2:0/2750281596 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f255c063c00 con 0x7f25480b2990 2024-05-14T09:32:13.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:13.030+0000 7f25537fe700 1 -- 172.21.15.2:0/2750281596 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f255c063c00 con 0x7f25480b2990 2024-05-14T09:32:13.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:13.032+0000 7f2560e5a700 1 -- 172.21.15.2:0/2750281596 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f25480b2990 msgr2=0x7f25480b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:13.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:13.032+0000 7f2560e5a700 1 --2- 172.21.15.2:0/2750281596 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f25480b2990 0x7f25480b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f255c106750 tx=0x7f254400a380 comp rx=0 tx=0).stop 2024-05-14T09:32:13.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:13.032+0000 7f2560e5a700 1 -- 172.21.15.2:0/2750281596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f255c1034c0 msgr2=0x7f255c19bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:13.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:13.032+0000 7f2560e5a700 1 --2- 172.21.15.2:0/2750281596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f255c1034c0 0x7f255c19bc50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f254c00d950 tx=0x7f254c00dc60 comp rx=0 tx=0).stop 2024-05-14T09:32:13.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:13.032+0000 7f2560e5a700 1 -- 172.21.15.2:0/2750281596 shutdown_connections 2024-05-14T09:32:13.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:13.032+0000 7f2560e5a700 1 --2- 172.21.15.2:0/2750281596 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f25480b2990 0x7f25480b4ea0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:13.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:13.032+0000 7f2560e5a700 1 --2- 172.21.15.2:0/2750281596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f255c108740 0x7f255c19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:13.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:13.032+0000 7f2560e5a700 1 --2- 172.21.15.2:0/2750281596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f255c1034c0 0x7f255c19bc50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:13.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:13.032+0000 7f2560e5a700 1 --2- 172.21.15.2:0/2750281596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f255c107df0 0x7f255c19c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:13.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:13.032+0000 7f2560e5a700 1 -- 172.21.15.2:0/2750281596 >> 172.21.15.2:0/2750281596 conn(0x7f255c0fedf0 msgr2=0x7f255c107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:13.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:13.032+0000 7f2560e5a700 1 -- 172.21.15.2:0/2750281596 shutdown_connections 2024-05-14T09:32:13.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:13.032+0000 7f2560e5a700 1 -- 172.21.15.2:0/2750281596 wait complete. 2024-05-14T09:32:13.652 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-14T09:32:13.908 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:14.299 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:14 smithi002 ceph-mon[31557]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:32:14.299 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:14 smithi002 ceph-mon[31557]: pgmap v340: 129 pgs: 129 active+clean; 609 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-05-14T09:32:14.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:14 smithi029 ceph-mon[36249]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:32:14.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:14 smithi029 ceph-mon[36249]: pgmap v340: 129 pgs: 129 active+clean; 609 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-05-14T09:32:14.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:14 smithi026 ceph-mon[36056]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:32:14.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:14 smithi026 ceph-mon[36056]: pgmap v340: 129 pgs: 129 active+clean; 609 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-05-14T09:32:15.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.699+0000 7f436b29c700 1 -- 172.21.15.2:0/1482502877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4364110c30 msgr2=0x7f4364113050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:15.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.699+0000 7f436b29c700 1 --2- 172.21.15.2:0/1482502877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4364110c30 0x7f4364113050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f435c00b0a0 tx=0x7f435c00b3b0 comp rx=0 tx=0).stop 2024-05-14T09:32:15.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.699+0000 7f436b29c700 1 -- 172.21.15.2:0/1482502877 shutdown_connections 2024-05-14T09:32:15.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.699+0000 7f436b29c700 1 --2- 172.21.15.2:0/1482502877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4364110c30 0x7f4364113050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:15.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.699+0000 7f436b29c700 1 --2- 172.21.15.2:0/1482502877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4364103080 0x7f4364103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:15.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.699+0000 7f436b29c700 1 --2- 172.21.15.2:0/1482502877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43641039d0 0x7f43641106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:15.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.699+0000 7f436b29c700 1 -- 172.21.15.2:0/1482502877 >> 172.21.15.2:0/1482502877 conn(0x7f43640fedf0 msgr2=0x7f4364101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:15.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.699+0000 7f436b29c700 1 -- 172.21.15.2:0/1482502877 shutdown_connections 2024-05-14T09:32:15.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.699+0000 7f436b29c700 1 -- 172.21.15.2:0/1482502877 wait complete. 2024-05-14T09:32:15.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.700+0000 7f436b29c700 1 Processor -- start 2024-05-14T09:32:15.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.700+0000 7f436b29c700 1 -- start start 2024-05-14T09:32:15.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.700+0000 7f436b29c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4364103080 0x7f436419d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:15.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.700+0000 7f436b29c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43641039d0 0x7f436419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:15.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.700+0000 7f436b29c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4364110c30 0x7f43641a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:15.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.700+0000 7f436b29c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f43641166c0 con 0x7f4364110c30 2024-05-14T09:32:15.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.700+0000 7f436b29c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4364116540 con 0x7f43641039d0 2024-05-14T09:32:15.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.700+0000 7f436b29c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4364116840 con 0x7f4364103080 2024-05-14T09:32:15.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.701+0000 7f4369839700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4364110c30 0x7f43641a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:15.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.701+0000 7f4369839700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4364110c30 0x7f43641a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34308/0 (socket says 172.21.15.2:34308) 2024-05-14T09:32:15.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.701+0000 7f4369839700 1 -- 172.21.15.2:0/385832568 learned_addr learned my addr 172.21.15.2:0/385832568 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:15.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.701+0000 7f4369038700 1 --2- 172.21.15.2:0/385832568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4364103080 0x7f436419d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:15.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.701+0000 7f4368837700 1 --2- 172.21.15.2:0/385832568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43641039d0 0x7f436419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:15.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.701+0000 7f4369839700 1 -- 172.21.15.2:0/385832568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4364103080 msgr2=0x7f436419d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:15.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.701+0000 7f4369839700 1 --2- 172.21.15.2:0/385832568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4364103080 0x7f436419d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:15.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.701+0000 7f4369839700 1 -- 172.21.15.2:0/385832568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43641039d0 msgr2=0x7f436419d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:15.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.701+0000 7f4369839700 1 --2- 172.21.15.2:0/385832568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43641039d0 0x7f436419d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:15.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.701+0000 7f4369839700 1 -- 172.21.15.2:0/385832568 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43641a1f00 con 0x7f4364110c30 2024-05-14T09:32:15.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.701+0000 7f4369839700 1 --2- 172.21.15.2:0/385832568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4364110c30 0x7f43641a1860 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f435c0062a0 tx=0x7f435c004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:15.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.701+0000 7f435a7fc700 1 -- 172.21.15.2:0/385832568 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c021070 con 0x7f4364110c30 2024-05-14T09:32:15.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.701+0000 7f435a7fc700 1 -- 172.21.15.2:0/385832568 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f435c003c00 con 0x7f4364110c30 2024-05-14T09:32:15.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.701+0000 7f436b29c700 1 -- 172.21.15.2:0/385832568 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43641a2190 con 0x7f4364110c30 2024-05-14T09:32:15.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.701+0000 7f435a7fc700 1 -- 172.21.15.2:0/385832568 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c004830 con 0x7f4364110c30 2024-05-14T09:32:15.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.702+0000 7f436b29c700 1 -- 172.21.15.2:0/385832568 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43641a9b10 con 0x7f4364110c30 2024-05-14T09:32:15.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.703+0000 7f436b29c700 1 -- 172.21.15.2:0/385832568 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43640762d0 con 0x7f4364110c30 2024-05-14T09:32:15.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.704+0000 7f435a7fc700 1 -- 172.21.15.2:0/385832568 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f435c003db0 con 0x7f4364110c30 2024-05-14T09:32:15.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.705+0000 7f435a7fc700 1 --2- 172.21.15.2:0/385832568 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f43500b2910 0x7f43500b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:15.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.705+0000 7f4369038700 1 --2- 172.21.15.2:0/385832568 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f43500b2910 0x7f43500b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:15.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.705+0000 7f435a7fc700 1 -- 172.21.15.2:0/385832568 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f435c00f020 con 0x7f4364110c30 2024-05-14T09:32:15.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.706+0000 7f4369038700 1 --2- 172.21.15.2:0/385832568 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f43500b2910 0x7f43500b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4360005950 tx=0x7f436000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:15.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.709+0000 7f435a7fc700 1 -- 172.21.15.2:0/385832568 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f435c02a020 con 0x7f4364110c30 2024-05-14T09:32:15.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.959+0000 7f436b29c700 1 -- 172.21.15.2:0/385832568 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f436419e390 con 0x7f43500b2910 2024-05-14T09:32:15.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.964+0000 7f435a7fc700 1 -- 172.21.15.2:0/385832568 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f436419e390 con 0x7f43500b2910 2024-05-14T09:32:15.964 INFO:teuthology.orchestra.run.smithi002.stdout:/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 2024-05-14T09:32:15.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.967+0000 7f436b29c700 1 -- 172.21.15.2:0/385832568 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f43500b2910 msgr2=0x7f43500b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:15.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.967+0000 7f436b29c700 1 --2- 172.21.15.2:0/385832568 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f43500b2910 0x7f43500b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4360005950 tx=0x7f436000a300 comp rx=0 tx=0).stop 2024-05-14T09:32:15.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.967+0000 7f436b29c700 1 -- 172.21.15.2:0/385832568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4364110c30 msgr2=0x7f43641a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:15.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.967+0000 7f436b29c700 1 --2- 172.21.15.2:0/385832568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4364110c30 0x7f43641a1860 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f435c0062a0 tx=0x7f435c004e20 comp rx=0 tx=0).stop 2024-05-14T09:32:15.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.967+0000 7f436b29c700 1 -- 172.21.15.2:0/385832568 shutdown_connections 2024-05-14T09:32:15.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.967+0000 7f436b29c700 1 --2- 172.21.15.2:0/385832568 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f43500b2910 0x7f43500b4e20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:15.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.967+0000 7f436b29c700 1 --2- 172.21.15.2:0/385832568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4364110c30 0x7f43641a1860 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:15.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.967+0000 7f436b29c700 1 --2- 172.21.15.2:0/385832568 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4364103080 0x7f436419d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:15.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.967+0000 7f436b29c700 1 --2- 172.21.15.2:0/385832568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43641039d0 0x7f436419d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:15.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.967+0000 7f436b29c700 1 -- 172.21.15.2:0/385832568 >> 172.21.15.2:0/385832568 conn(0x7f43640fedf0 msgr2=0x7f43641125f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:15.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.967+0000 7f436b29c700 1 -- 172.21.15.2:0/385832568 shutdown_connections 2024-05-14T09:32:15.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:15.967+0000 7f436b29c700 1 -- 172.21.15.2:0/385832568 wait complete. 2024-05-14T09:32:16.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:16 smithi002 ceph-mon[31557]: pgmap v341: 129 pgs: 129 active+clean; 609 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-05-14T09:32:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:16 smithi029 ceph-mon[36249]: pgmap v341: 129 pgs: 129 active+clean; 609 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-05-14T09:32:16.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:16 smithi026 ceph-mon[36056]: pgmap v341: 129 pgs: 129 active+clean; 609 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-05-14T09:32:16.703 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:32:16.966 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:17.357 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:17 smithi002 ceph-mon[31557]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:32:17.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:17 smithi029 ceph-mon[36249]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:32:17.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:17 smithi026 ceph-mon[36056]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:32:18.396 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:18 smithi002 ceph-mon[31557]: pgmap v342: 129 pgs: 129 active+clean; 609 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-14T09:32:18.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:18 smithi029 ceph-mon[36249]: pgmap v342: 129 pgs: 129 active+clean; 609 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-14T09:32:18.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:18 smithi026 ceph-mon[36056]: pgmap v342: 129 pgs: 129 active+clean; 609 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-14T09:32:18.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.774+0000 7ff3aed7c700 1 -- 172.21.15.2:0/2513546682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3a8107df0 msgr2=0x7ff3a8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:18.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.774+0000 7ff3aed7c700 1 --2- 172.21.15.2:0/2513546682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3a8107df0 0x7ff3a8108200 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff3940099c0 tx=0x7ff394009cd0 comp rx=0 tx=0).stop 2024-05-14T09:32:18.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.774+0000 7ff3aed7c700 1 -- 172.21.15.2:0/2513546682 shutdown_connections 2024-05-14T09:32:18.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.774+0000 7ff3aed7c700 1 --2- 172.21.15.2:0/2513546682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a81034c0 0x7ff3a81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:18.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.774+0000 7ff3aed7c700 1 --2- 172.21.15.2:0/2513546682 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3a8108740 0x7ff3a81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:18.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.774+0000 7ff3aed7c700 1 --2- 172.21.15.2:0/2513546682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3a8107df0 0x7ff3a8108200 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:18.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.774+0000 7ff3aed7c700 1 -- 172.21.15.2:0/2513546682 >> 172.21.15.2:0/2513546682 conn(0x7ff3a80fedf0 msgr2=0x7ff3a8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:18.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.774+0000 7ff3aed7c700 1 -- 172.21.15.2:0/2513546682 shutdown_connections 2024-05-14T09:32:18.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.774+0000 7ff3aed7c700 1 -- 172.21.15.2:0/2513546682 wait complete. 2024-05-14T09:32:18.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.775+0000 7ff3aed7c700 1 Processor -- start 2024-05-14T09:32:18.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.775+0000 7ff3aed7c700 1 -- start start 2024-05-14T09:32:18.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.775+0000 7ff3aed7c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3a81034c0 0x7ff3a819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:18.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.775+0000 7ff3aed7c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3a8107df0 0x7ff3a819c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:18.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.775+0000 7ff3aed7c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a8108740 0x7ff3a81a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:18.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.775+0000 7ff3aed7c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a8113350 con 0x7ff3a81034c0 2024-05-14T09:32:18.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.775+0000 7ff3aed7c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81131d0 con 0x7ff3a8108740 2024-05-14T09:32:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.775+0000 7ff3aed7c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81134d0 con 0x7ff3a8107df0 2024-05-14T09:32:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.776+0000 7ff3acb18700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3a81034c0 0x7ff3a819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.776+0000 7ff3acb18700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3a81034c0 0x7ff3a819bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47662/0 (socket says 172.21.15.2:47662) 2024-05-14T09:32:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.776+0000 7ff3acb18700 1 -- 172.21.15.2:0/366762064 learned_addr learned my addr 172.21.15.2:0/366762064 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.776+0000 7ff3a7fff700 1 --2- 172.21.15.2:0/366762064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3a8107df0 0x7ff3a819c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.776+0000 7ff3ad319700 1 --2- 172.21.15.2:0/366762064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a8108740 0x7ff3a81a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.776+0000 7ff3acb18700 1 -- 172.21.15.2:0/366762064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3a8107df0 msgr2=0x7ff3a819c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.776+0000 7ff3acb18700 1 --2- 172.21.15.2:0/366762064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3a8107df0 0x7ff3a819c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.776+0000 7ff3acb18700 1 -- 172.21.15.2:0/366762064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a8108740 msgr2=0x7ff3a81a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.776+0000 7ff3acb18700 1 --2- 172.21.15.2:0/366762064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a8108740 0x7ff3a81a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.776+0000 7ff3acb18700 1 -- 172.21.15.2:0/366762064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a819ca60 con 0x7ff3a81034c0 2024-05-14T09:32:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.776+0000 7ff3acb18700 1 --2- 172.21.15.2:0/366762064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3a81034c0 0x7ff3a819bc70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00c920 tx=0x7ff39c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:18.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.776+0000 7ff3a5ffb700 1 -- 172.21.15.2:0/366762064 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c007b00 con 0x7ff3a81034c0 2024-05-14T09:32:18.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.776+0000 7ff3a5ffb700 1 -- 172.21.15.2:0/366762064 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff39c004ce0 con 0x7ff3a81034c0 2024-05-14T09:32:18.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.776+0000 7ff3aed7c700 1 -- 172.21.15.2:0/366762064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a81a7350 con 0x7ff3a81034c0 2024-05-14T09:32:18.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.776+0000 7ff3a5ffb700 1 -- 172.21.15.2:0/366762064 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c005650 con 0x7ff3a81034c0 2024-05-14T09:32:18.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.777+0000 7ff3aed7c700 1 -- 172.21.15.2:0/366762064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a81a7810 con 0x7ff3a81034c0 2024-05-14T09:32:18.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.778+0000 7ff3aed7c700 1 -- 172.21.15.2:0/366762064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a80512c0 con 0x7ff3a81034c0 2024-05-14T09:32:18.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.779+0000 7ff3a5ffb700 1 -- 172.21.15.2:0/366762064 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7ff39c004750 con 0x7ff3a81034c0 2024-05-14T09:32:18.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.780+0000 7ff3a5ffb700 1 --2- 172.21.15.2:0/366762064 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3980b2910 0x7ff3980b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:18.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.780+0000 7ff3a7fff700 1 --2- 172.21.15.2:0/366762064 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3980b2910 0x7ff3980b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:18.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.780+0000 7ff3a5ffb700 1 -- 172.21.15.2:0/366762064 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7ff39c0e2710 con 0x7ff3a81034c0 2024-05-14T09:32:18.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.781+0000 7ff3a7fff700 1 --2- 172.21.15.2:0/366762064 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3980b2910 0x7ff3980b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff394000c00 tx=0x7ff39401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:18.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:18.783+0000 7ff3a5ffb700 1 -- 172.21.15.2:0/366762064 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff39c014030 con 0x7ff3a81034c0 2024-05-14T09:32:19.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:19.047+0000 7ff3aed7c700 1 -- 172.21.15.2:0/366762064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3a802e5f0 con 0x7ff3a81034c0 2024-05-14T09:32:19.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:19.048+0000 7ff3a5ffb700 1 -- 172.21.15.2:0/366762064 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11843 (secure 0 0 0) 0x7ff39c0a9f30 con 0x7ff3a81034c0 2024-05-14T09:32:19.048 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:32:19.048 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T09:32:01.167752+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24497,24524,24487]},"id":1}]} 2024-05-14T09:32:19.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:19.050+0000 7ff3aed7c700 1 -- 172.21.15.2:0/366762064 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3980b2910 msgr2=0x7ff3980b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:19.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:19.050+0000 7ff3aed7c700 1 --2- 172.21.15.2:0/366762064 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3980b2910 0x7ff3980b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff394000c00 tx=0x7ff39401a040 comp rx=0 tx=0).stop 2024-05-14T09:32:19.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:19.050+0000 7ff3aed7c700 1 -- 172.21.15.2:0/366762064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3a81034c0 msgr2=0x7ff3a819bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:19.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:19.050+0000 7ff3aed7c700 1 --2- 172.21.15.2:0/366762064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3a81034c0 0x7ff3a819bc70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00c920 tx=0x7ff39c00cce0 comp rx=0 tx=0).stop 2024-05-14T09:32:19.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:19.051+0000 7ff3aed7c700 1 -- 172.21.15.2:0/366762064 shutdown_connections 2024-05-14T09:32:19.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:19.051+0000 7ff3aed7c700 1 --2- 172.21.15.2:0/366762064 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3980b2910 0x7ff3980b4e20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:19.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:19.051+0000 7ff3aed7c700 1 --2- 172.21.15.2:0/366762064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a8108740 0x7ff3a81a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:19.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:19.051+0000 7ff3aed7c700 1 --2- 172.21.15.2:0/366762064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3a81034c0 0x7ff3a819bc70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:19.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:19.051+0000 7ff3aed7c700 1 --2- 172.21.15.2:0/366762064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3a8107df0 0x7ff3a819c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:19.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:19.051+0000 7ff3aed7c700 1 -- 172.21.15.2:0/366762064 >> 172.21.15.2:0/366762064 conn(0x7ff3a80fedf0 msgr2=0x7ff3a81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:19.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:19.051+0000 7ff3aed7c700 1 -- 172.21.15.2:0/366762064 shutdown_connections 2024-05-14T09:32:19.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:19.051+0000 7ff3aed7c700 1 -- 172.21.15.2:0/366762064 wait complete. 2024-05-14T09:32:19.052 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-05-14T09:32:19.345 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:19 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/366762064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:32:19.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:19 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/366762064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:32:19.691 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T09:32:19.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:19 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/366762064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:32:19.941 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:20.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:20 smithi002 ceph-mon[31557]: pgmap v343: 129 pgs: 129 active+clean; 634 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-05-14T09:32:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:20 smithi029 ceph-mon[36249]: pgmap v343: 129 pgs: 129 active+clean; 634 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-05-14T09:32:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:20 smithi026 ceph-mon[36056]: pgmap v343: 129 pgs: 129 active+clean; 634 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-05-14T09:32:21.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.797+0000 7f8c1b1c8700 1 -- 172.21.15.2:0/1042707958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c141045e0 msgr2=0x7f8c141049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:21.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.797+0000 7f8c1b1c8700 1 --2- 172.21.15.2:0/1042707958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c141045e0 0x7f8c141049f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8c080099c0 tx=0x7f8c08009cd0 comp rx=0 tx=0).stop 2024-05-14T09:32:21.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.797+0000 7f8c1b1c8700 1 -- 172.21.15.2:0/1042707958 shutdown_connections 2024-05-14T09:32:21.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.797+0000 7f8c1b1c8700 1 --2- 172.21.15.2:0/1042707958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c14109860 0x7f8c141111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:21.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.797+0000 7f8c1b1c8700 1 --2- 172.21.15.2:0/1042707958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c141045e0 0x7f8c141049f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:21.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.797+0000 7f8c1b1c8700 1 --2- 172.21.15.2:0/1042707958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c14108f10 0x7f8c14109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:21.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.797+0000 7f8c1b1c8700 1 -- 172.21.15.2:0/1042707958 >> 172.21.15.2:0/1042707958 conn(0x7f8c140ffed0 msgr2=0x7f8c14102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:21.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.797+0000 7f8c1b1c8700 1 -- 172.21.15.2:0/1042707958 shutdown_connections 2024-05-14T09:32:21.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.797+0000 7f8c1b1c8700 1 -- 172.21.15.2:0/1042707958 wait complete. 2024-05-14T09:32:21.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.798+0000 7f8c1b1c8700 1 Processor -- start 2024-05-14T09:32:21.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.798+0000 7f8c1b1c8700 1 -- start start 2024-05-14T09:32:21.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.798+0000 7f8c1b1c8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c141045e0 0x7f8c1419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:21.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.798+0000 7f8c1b1c8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c14108f10 0x7f8c1419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:21.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.798+0000 7f8c1b1c8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c14109860 0x7f8c1419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:21.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.798+0000 7f8c1b1c8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8c14114490 con 0x7f8c141045e0 2024-05-14T09:32:21.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.798+0000 7f8c1b1c8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8c14114310 con 0x7f8c14109860 2024-05-14T09:32:21.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.798+0000 7f8c1b1c8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8c14114610 con 0x7f8c14108f10 2024-05-14T09:32:21.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.798+0000 7f8c18f64700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c141045e0 0x7f8c1419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:21.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.798+0000 7f8c18f64700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c141045e0 0x7f8c1419ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47668/0 (socket says 172.21.15.2:47668) 2024-05-14T09:32:21.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.799+0000 7f8c18f64700 1 -- 172.21.15.2:0/2728760065 learned_addr learned my addr 172.21.15.2:0/2728760065 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:21.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.799+0000 7f8c19765700 1 --2- 172.21.15.2:0/2728760065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c14109860 0x7f8c1419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:21.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.799+0000 7f8c13fff700 1 --2- 172.21.15.2:0/2728760065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c14108f10 0x7f8c1419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:21.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.799+0000 7f8c18f64700 1 -- 172.21.15.2:0/2728760065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c14108f10 msgr2=0x7f8c1419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:21.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.799+0000 7f8c18f64700 1 --2- 172.21.15.2:0/2728760065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c14108f10 0x7f8c1419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:21.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.799+0000 7f8c18f64700 1 -- 172.21.15.2:0/2728760065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c14109860 msgr2=0x7f8c1419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:21.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.799+0000 7f8c18f64700 1 --2- 172.21.15.2:0/2728760065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c14109860 0x7f8c1419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:21.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.799+0000 7f8c18f64700 1 -- 172.21.15.2:0/2728760065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1419fcd0 con 0x7f8c141045e0 2024-05-14T09:32:21.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.799+0000 7f8c13fff700 1 --2- 172.21.15.2:0/2728760065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c14108f10 0x7f8c1419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:32:21.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.799+0000 7f8c18f64700 1 --2- 172.21.15.2:0/2728760065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c141045e0 0x7f8c1419ae00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800b580 tx=0x7f8c080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:21.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.799+0000 7f8c11ffb700 1 -- 172.21.15.2:0/2728760065 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08027070 con 0x7f8c141045e0 2024-05-14T09:32:21.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.799+0000 7f8c11ffb700 1 -- 172.21.15.2:0/2728760065 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c08003f20 con 0x7f8c141045e0 2024-05-14T09:32:21.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.799+0000 7f8c1b1c8700 1 -- 172.21.15.2:0/2728760065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1419ff60 con 0x7f8c141045e0 2024-05-14T09:32:21.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.799+0000 7f8c11ffb700 1 -- 172.21.15.2:0/2728760065 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08022d70 con 0x7f8c141045e0 2024-05-14T09:32:21.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.800+0000 7f8c1b1c8700 1 -- 172.21.15.2:0/2728760065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c141a78e0 con 0x7f8c141045e0 2024-05-14T09:32:21.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.800+0000 7f8c1b1c8700 1 -- 172.21.15.2:0/2728760065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c14105a90 con 0x7f8c141045e0 2024-05-14T09:32:21.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.802+0000 7f8c11ffb700 1 -- 172.21.15.2:0/2728760065 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f8c08022500 con 0x7f8c141045e0 2024-05-14T09:32:21.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.803+0000 7f8c11ffb700 1 --2- 172.21.15.2:0/2728760065 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8c040b2990 0x7f8c040b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:21.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.803+0000 7f8c11ffb700 1 -- 172.21.15.2:0/2728760065 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f8c080e3780 con 0x7f8c141045e0 2024-05-14T09:32:21.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.803+0000 7f8c13fff700 1 --2- 172.21.15.2:0/2728760065 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8c040b2990 0x7f8c040b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:21.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.804+0000 7f8c13fff700 1 --2- 172.21.15.2:0/2728760065 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8c040b2990 0x7f8c040b4ea0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8c00005950 tx=0x7f8c000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:21.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:21.807+0000 7f8c11ffb700 1 -- 172.21.15.2:0/2728760065 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8c08013070 con 0x7f8c141045e0 2024-05-14T09:32:22.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:22.022+0000 7f8c1b1c8700 1 -- 172.21.15.2:0/2728760065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c1419c160 con 0x7f8c141045e0 2024-05-14T09:32:22.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:22.024+0000 7f8c11ffb700 1 -- 172.21.15.2:0/2728760065 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24395 (secure 0 0 0) 0x7f8c080ab090 con 0x7f8c141045e0 2024-05-14T09:32:22.024 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:32:22.024 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":88,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","created":"2024-05-14T09:22:34.937047+0000","modified":"2024-05-14T09:31:30.160963+0000","last_up_change":"2024-05-14T09:29:55.157569+0000","last_in_change":"2024-05-14T09:29:34.922562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T09:26:03.046454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T09:30:51.814350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T09:30:55.500000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T09:31:06.099087+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10d34289-414e-4b1b-bc7c-ba1dd24c7297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6803","nonce":2622184634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6805","nonce":2622184634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6809","nonce":2622184634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6807","nonce":2622184634}]},"public_addr":"172.21.15.2:6803/2622184634","cluster_addr":"172.21.15.2:6805/2622184634","heartbeat_back_addr":"172.21.15.2:6809/2622184634","heartbeat_front_addr":"172.21.15.2:6807/2622184634","state":["exists","up"]},{"osd":1,"uuid":"bc4aefc2-706e-4faf-9042-64b87ad097b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6801","nonce":3084747226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6803","nonce":3084747226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6807","nonce":3084747226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6805","nonce":3084747226}]},"public_addr":"172.21.15.26:6801/3084747226","cluster_addr":"172.21.15.26:6803/3084747226","heartbeat_back_addr":"172.21.15.26:6807/3084747226","heartbeat_front_addr":"172.21.15.26:6805/3084747226","state":["exists","up"]},{"osd":2,"uuid":"8c5bb589-56e2-469f-9a6f-6035b3ddf4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6801","nonce":814048086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6803","nonce":814048086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6807","nonce":814048086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6805","nonce":814048086}]},"public_addr":"172.21.15.29:6801/814048086","cluster_addr":"172.21.15.29:6803/814048086","heartbeat_back_addr":"172.21.15.29:6807/814048086","heartbeat_front_addr":"172.21.15.29:6805/814048086","state":["exists","up"]},{"osd":3,"uuid":"8fbfc078-85e7-4dc2-acda-5e7b0ab927e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6811","nonce":879639451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6813","nonce":879639451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6817","nonce":879639451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6815","nonce":879639451}]},"public_addr":"172.21.15.2:6811/879639451","cluster_addr":"172.21.15.2:6813/879639451","heartbeat_back_addr":"172.21.15.2:6817/879639451","heartbeat_front_addr":"172.21.15.2:6815/879639451","state":["exists","up"]},{"osd":4,"uuid":"f2515448-e9c1-48a2-b270-1e5e781a2fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6809","nonce":457100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6811","nonce":457100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6815","nonce":457100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6813","nonce":457100752}]},"public_addr":"172.21.15.26:6809/457100752","cluster_addr":"172.21.15.26:6811/457100752","heartbeat_back_addr":"172.21.15.26:6815/457100752","heartbeat_front_addr":"172.21.15.26:6813/457100752","state":["exists","up"]},{"osd":5,"uuid":"20030857-a1e6-4b0a-b186-1cfa44f2e4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6809","nonce":1554045093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6811","nonce":1554045093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6815","nonce":1554045093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6813","nonce":1554045093}]},"public_addr":"172.21.15.29:6809/1554045093","cluster_addr":"172.21.15.29:6811/1554045093","heartbeat_back_addr":"172.21.15.29:6815/1554045093","heartbeat_front_addr":"172.21.15.29:6813/1554045093","state":["exists","up"]},{"osd":6,"uuid":"d757c72a-29f0-4f14-9f13-a2e61eb5e6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6819","nonce":2749897595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6821","nonce":2749897595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6825","nonce":2749897595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6823","nonce":2749897595}]},"public_addr":"172.21.15.2:6819/2749897595","cluster_addr":"172.21.15.2:6821/2749897595","heartbeat_back_addr":"172.21.15.2:6825/2749897595","heartbeat_front_addr":"172.21.15.2:6823/2749897595","state":["exists","up"]},{"osd":7,"uuid":"22ddef80-3633-44c5-bc0e-edc75b546ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6817","nonce":1180401611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6819","nonce":1180401611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6823","nonce":1180401611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6821","nonce":1180401611}]},"public_addr":"172.21.15.26:6817/1180401611","cluster_addr":"172.21.15.26:6819/1180401611","heartbeat_back_addr":"172.21.15.26:6823/1180401611","heartbeat_front_addr":"172.21.15.26:6821/1180401611","state":["exists","up"]},{"osd":8,"uuid":"466e7ac4-3ff7-4254-a3fd-8096420b1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6817","nonce":2858408446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6819","nonce":2858408446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6823","nonce":2858408446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6821","nonce":2858408446}]},"public_addr":"172.21.15.29:6817/2858408446","cluster_addr":"172.21.15.29:6819/2858408446","heartbeat_back_addr":"172.21.15.29:6823/2858408446","heartbeat_front_addr":"172.21.15.29:6821/2858408446","state":["exists","up"]},{"osd":9,"uuid":"119eeac9-29f7-4086-a1ad-04a0bc154b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6827","nonce":176898226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6829","nonce":176898226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6833","nonce":176898226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6831","nonce":176898226}]},"public_addr":"172.21.15.2:6827/176898226","cluster_addr":"172.21.15.2:6829/176898226","heartbeat_back_addr":"172.21.15.2:6833/176898226","heartbeat_front_addr":"172.21.15.2:6831/176898226","state":["exists","up"]},{"osd":10,"uuid":"55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6825","nonce":4065159515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6827","nonce":4065159515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6831","nonce":4065159515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6829","nonce":4065159515}]},"public_addr":"172.21.15.26:6825/4065159515","cluster_addr":"172.21.15.26:6827/4065159515","heartbeat_back_addr":"172.21.15.26:6831/4065159515","heartbeat_front_addr":"172.21.15.26:6829/4065159515","state":["exists","up"]},{"osd":11,"uuid":"c5246763-c4f6-4ae3-8a93-5929c541ac5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6825","nonce":378800417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6827","nonce":378800417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6831","nonce":378800417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6829","nonce":378800417}]},"public_addr":"172.21.15.29:6825/378800417","cluster_addr":"172.21.15.29:6827/378800417","heartbeat_back_addr":"172.21.15.29:6831/378800417","heartbeat_front_addr":"172.21.15.29:6829/378800417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:09.342387+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:34.156359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:58.860653+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:24.009323+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:49.057146+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:15.007172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:41.085748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:06.674923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:33.409128+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:00.106186+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:26.803487+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:53.668283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11},{"from":8,"to":6}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4051922409":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/2353790181":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/4268171474":"2024-05-15T09:23:28.940670+0000","172.21.15.2:6801/1423742578":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/969850441":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3252991268":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6801/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3789067476":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/1423742578":"2024-05-15T09:23:28.940670+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T09:32:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:22.026+0000 7f8c1b1c8700 1 -- 172.21.15.2:0/2728760065 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8c040b2990 msgr2=0x7f8c040b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:22.026+0000 7f8c1b1c8700 1 --2- 172.21.15.2:0/2728760065 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8c040b2990 0x7f8c040b4ea0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8c00005950 tx=0x7f8c000058e0 comp rx=0 tx=0).stop 2024-05-14T09:32:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:22.026+0000 7f8c1b1c8700 1 -- 172.21.15.2:0/2728760065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c141045e0 msgr2=0x7f8c1419ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:22.026+0000 7f8c1b1c8700 1 --2- 172.21.15.2:0/2728760065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c141045e0 0x7f8c1419ae00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800b580 tx=0x7f8c080047b0 comp rx=0 tx=0).stop 2024-05-14T09:32:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:22.026+0000 7f8c1b1c8700 1 -- 172.21.15.2:0/2728760065 shutdown_connections 2024-05-14T09:32:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:22.026+0000 7f8c1b1c8700 1 --2- 172.21.15.2:0/2728760065 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8c040b2990 0x7f8c040b4ea0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:22.026+0000 7f8c1b1c8700 1 --2- 172.21.15.2:0/2728760065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c14109860 0x7f8c1419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:22.026+0000 7f8c1b1c8700 1 --2- 172.21.15.2:0/2728760065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c141045e0 0x7f8c1419ae00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:22.027+0000 7f8c1b1c8700 1 --2- 172.21.15.2:0/2728760065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c14108f10 0x7f8c1419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:22.027+0000 7f8c1b1c8700 1 -- 172.21.15.2:0/2728760065 >> 172.21.15.2:0/2728760065 conn(0x7f8c140ffed0 msgr2=0x7f8c1410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:22.027+0000 7f8c1b1c8700 1 -- 172.21.15.2:0/2728760065 shutdown_connections 2024-05-14T09:32:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:22.027+0000 7f8c1b1c8700 1 -- 172.21.15.2:0/2728760065 wait complete. 2024-05-14T09:32:22.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:22 smithi002 ceph-mon[31557]: pgmap v344: 129 pgs: 129 active+clean; 634 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-05-14T09:32:22.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:22 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2728760065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:32:22.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:22 smithi029 ceph-mon[36249]: pgmap v344: 129 pgs: 129 active+clean; 634 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-05-14T09:32:22.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:22 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2728760065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:32:22.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:22 smithi026 ceph-mon[36056]: pgmap v344: 129 pgs: 129 active+clean; 634 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-05-14T09:32:22.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:22 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2728760065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:32:22.729 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-14T09:32:22.729 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T09:32:22.986 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:24.407 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:24 smithi002 ceph-mon[31557]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-14T09:32:24.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.659+0000 7fcb31f96700 1 -- 172.21.15.2:0/4085401199 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb2c108f10 msgr2=0x7fcb2c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:24.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.659+0000 7fcb31f96700 1 --2- 172.21.15.2:0/4085401199 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb2c108f10 0x7fcb2c109320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcb200099c0 tx=0x7fcb20009cd0 comp rx=0 tx=0).stop 2024-05-14T09:32:24.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.659+0000 7fcb31f96700 1 -- 172.21.15.2:0/4085401199 shutdown_connections 2024-05-14T09:32:24.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.659+0000 7fcb31f96700 1 --2- 172.21.15.2:0/4085401199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb2c109860 0x7fcb2c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:24.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.659+0000 7fcb31f96700 1 --2- 172.21.15.2:0/4085401199 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb2c1045e0 0x7fcb2c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:24.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.659+0000 7fcb31f96700 1 --2- 172.21.15.2:0/4085401199 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb2c108f10 0x7fcb2c109320 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:24.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.659+0000 7fcb31f96700 1 -- 172.21.15.2:0/4085401199 >> 172.21.15.2:0/4085401199 conn(0x7fcb2c0ffed0 msgr2=0x7fcb2c102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:24.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.659+0000 7fcb31f96700 1 -- 172.21.15.2:0/4085401199 shutdown_connections 2024-05-14T09:32:24.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.659+0000 7fcb31f96700 1 -- 172.21.15.2:0/4085401199 wait complete. 2024-05-14T09:32:24.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.660+0000 7fcb31f96700 1 Processor -- start 2024-05-14T09:32:24.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.660+0000 7fcb31f96700 1 -- start start 2024-05-14T09:32:24.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.660+0000 7fcb31f96700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb2c1045e0 0x7fcb2c19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:24.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.660+0000 7fcb31f96700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb2c108f10 0x7fcb2c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:24.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.660+0000 7fcb31f96700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb2c109860 0x7fcb2c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.660+0000 7fcb31f96700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c114490 con 0x7fcb2c108f10 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.660+0000 7fcb31f96700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c114310 con 0x7fcb2c109860 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.660+0000 7fcb31f96700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c114610 con 0x7fcb2c1045e0 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.661+0000 7fcb2affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb2c108f10 0x7fcb2c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.661+0000 7fcb2affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb2c108f10 0x7fcb2c19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47684/0 (socket says 172.21.15.2:47684) 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.661+0000 7fcb2affd700 1 -- 172.21.15.2:0/4245860369 learned_addr learned my addr 172.21.15.2:0/4245860369 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.661+0000 7fcb2b7fe700 1 --2- 172.21.15.2:0/4245860369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb2c1045e0 0x7fcb2c19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.661+0000 7fcb2bfff700 1 --2- 172.21.15.2:0/4245860369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb2c109860 0x7fcb2c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.661+0000 7fcb2affd700 1 -- 172.21.15.2:0/4245860369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb2c1045e0 msgr2=0x7fcb2c19ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.661+0000 7fcb2affd700 1 --2- 172.21.15.2:0/4245860369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb2c1045e0 0x7fcb2c19ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.661+0000 7fcb2affd700 1 -- 172.21.15.2:0/4245860369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb2c109860 msgr2=0x7fcb2c19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.661+0000 7fcb2affd700 1 --2- 172.21.15.2:0/4245860369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb2c109860 0x7fcb2c19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.661+0000 7fcb2affd700 1 -- 172.21.15.2:0/4245860369 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2c19fd50 con 0x7fcb2c108f10 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.661+0000 7fcb2affd700 1 --2- 172.21.15.2:0/4245860369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb2c108f10 0x7fcb2c19b420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcb20006010 tx=0x7fcb200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.661+0000 7fcb28ff9700 1 -- 172.21.15.2:0/4245860369 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20027070 con 0x7fcb2c108f10 2024-05-14T09:32:24.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.661+0000 7fcb28ff9700 1 -- 172.21.15.2:0/4245860369 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb20003f20 con 0x7fcb2c108f10 2024-05-14T09:32:24.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.662+0000 7fcb28ff9700 1 -- 172.21.15.2:0/4245860369 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20022c60 con 0x7fcb2c108f10 2024-05-14T09:32:24.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.662+0000 7fcb31f96700 1 -- 172.21.15.2:0/4245860369 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2c19ffe0 con 0x7fcb2c108f10 2024-05-14T09:32:24.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.662+0000 7fcb31f96700 1 -- 172.21.15.2:0/4245860369 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2c1a03b0 con 0x7fcb2c108f10 2024-05-14T09:32:24.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.664+0000 7fcb31f96700 1 -- 172.21.15.2:0/4245860369 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb2c0512c0 con 0x7fcb2c108f10 2024-05-14T09:32:24.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.665+0000 7fcb28ff9700 1 -- 172.21.15.2:0/4245860369 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fcb2002a070 con 0x7fcb2c108f10 2024-05-14T09:32:24.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.665+0000 7fcb28ff9700 1 --2- 172.21.15.2:0/4245860369 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcb140b2990 0x7fcb140b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:24.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.666+0000 7fcb2b7fe700 1 --2- 172.21.15.2:0/4245860369 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcb140b2990 0x7fcb140b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:24.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.666+0000 7fcb28ff9700 1 -- 172.21.15.2:0/4245860369 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fcb20022e00 con 0x7fcb2c108f10 2024-05-14T09:32:24.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.666+0000 7fcb2b7fe700 1 --2- 172.21.15.2:0/4245860369 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcb140b2990 0x7fcb140b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c004630 tx=0x7fcb1c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:24.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.670+0000 7fcb28ff9700 1 -- 172.21.15.2:0/4245860369 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb20013070 con 0x7fcb2c108f10 2024-05-14T09:32:24.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:24 smithi029 ceph-mon[36249]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-14T09:32:24.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:24 smithi026 ceph-mon[36056]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-14T09:32:24.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.884+0000 7fcb31f96700 1 -- 172.21.15.2:0/4245860369 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcb2c19c1e0 con 0x7fcb140b2990 2024-05-14T09:32:24.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.894+0000 7fcb28ff9700 1 -- 172.21.15.2:0/4245860369 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340610 (secure 0 0 0) 0x7fcb2c19c1e0 con 0x7fcb140b2990 2024-05-14T09:32:24.900 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:32:24.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.898+0000 7fcb31f96700 1 -- 172.21.15.2:0/4245860369 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcb140b2990 msgr2=0x7fcb140b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:24.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.898+0000 7fcb31f96700 1 --2- 172.21.15.2:0/4245860369 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcb140b2990 0x7fcb140b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c004630 tx=0x7fcb1c00b3c0 comp rx=0 tx=0).stop 2024-05-14T09:32:24.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.898+0000 7fcb31f96700 1 -- 172.21.15.2:0/4245860369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb2c108f10 msgr2=0x7fcb2c19b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:24.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.898+0000 7fcb31f96700 1 --2- 172.21.15.2:0/4245860369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb2c108f10 0x7fcb2c19b420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcb20006010 tx=0x7fcb200047b0 comp rx=0 tx=0).stop 2024-05-14T09:32:24.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.898+0000 7fcb31f96700 1 -- 172.21.15.2:0/4245860369 shutdown_connections 2024-05-14T09:32:24.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.898+0000 7fcb31f96700 1 --2- 172.21.15.2:0/4245860369 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcb140b2990 0x7fcb140b4ea0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:24.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.898+0000 7fcb31f96700 1 --2- 172.21.15.2:0/4245860369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb2c109860 0x7fcb2c19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:24.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.898+0000 7fcb31f96700 1 --2- 172.21.15.2:0/4245860369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb2c1045e0 0x7fcb2c19ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:24.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.898+0000 7fcb31f96700 1 --2- 172.21.15.2:0/4245860369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb2c108f10 0x7fcb2c19b420 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:24.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.898+0000 7fcb31f96700 1 -- 172.21.15.2:0/4245860369 >> 172.21.15.2:0/4245860369 conn(0x7fcb2c0ffed0 msgr2=0x7fcb2c10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:24.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.898+0000 7fcb31f96700 1 -- 172.21.15.2:0/4245860369 shutdown_connections 2024-05-14T09:32:24.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:24.898+0000 7fcb31f96700 1 -- 172.21.15.2:0/4245860369 wait complete. 2024-05-14T09:32:24.904 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-05-14T09:32:25.578 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-05-14T09:32:23.060976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653260,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":252,"num_read_kb":219,"num_write":290,"num_write_kb":2262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2954034,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":394200,"kb_used_data":11768,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124547624,"statfs":{"total":1151940427776,"available":1151536766976,"internally_reserved":0,"allocated":12050432,"data_stored":7926350,"data_compressed":424731,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19140,"internal_metadata":391492924},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34198,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":94,"num_read_kb":80,"num_write":26,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003691"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166300+0000","last_change":"2024-05-14T09:30:56.797051+0000","last_active":"2024-05-14T09:31:30.166300+0000","last_peered":"2024-05-14T09:31:30.166300+0000","last_clean":"2024-05-14T09:31:30.166300+0000","last_became_active":"2024-05-14T09:30:56.796517+0000","last_became_peered":"2024-05-14T09:30:56.796517+0000","last_unstale":"2024-05-14T09:31:30.166300+0000","last_undegraded":"2024-05-14T09:31:30.166300+0000","last_fullsized":"2024-05-14T09:31:30.166300+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:58:22.170187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295855+0000","last_change":"2024-05-14T09:30:56.795325+0000","last_active":"2024-05-14T09:31:30.295855+0000","last_peered":"2024-05-14T09:31:30.295855+0000","last_clean":"2024-05-14T09:31:30.295855+0000","last_became_active":"2024-05-14T09:30:56.794962+0000","last_became_peered":"2024-05-14T09:30:56.794962+0000","last_unstale":"2024-05-14T09:31:30.295855+0000","last_undegraded":"2024-05-14T09:31:30.295855+0000","last_fullsized":"2024-05-14T09:31:30.295855+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:15:28.890565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295685+0000","last_change":"2024-05-14T09:30:56.793200+0000","last_active":"2024-05-14T09:31:30.295685+0000","last_peered":"2024-05-14T09:31:30.295685+0000","last_clean":"2024-05-14T09:31:30.295685+0000","last_became_active":"2024-05-14T09:30:56.792835+0000","last_became_peered":"2024-05-14T09:30:56.792835+0000","last_unstale":"2024-05-14T09:31:30.295685+0000","last_undegraded":"2024-05-14T09:31:30.295685+0000","last_fullsized":"2024-05-14T09:31:30.295685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:19.851276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168183+0000","last_change":"2024-05-14T09:30:56.790238+0000","last_active":"2024-05-14T09:31:30.168183+0000","last_peered":"2024-05-14T09:31:30.168183+0000","last_clean":"2024-05-14T09:31:30.168183+0000","last_became_active":"2024-05-14T09:30:56.789826+0000","last_became_peered":"2024-05-14T09:30:56.789826+0000","last_unstale":"2024-05-14T09:31:30.168183+0000","last_undegraded":"2024-05-14T09:31:30.168183+0000","last_fullsized":"2024-05-14T09:31:30.168183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:19:19.853931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166364+0000","last_change":"2024-05-14T09:30:56.788021+0000","last_active":"2024-05-14T09:31:30.166364+0000","last_peered":"2024-05-14T09:31:30.166364+0000","last_clean":"2024-05-14T09:31:30.166364+0000","last_became_active":"2024-05-14T09:30:56.787657+0000","last_became_peered":"2024-05-14T09:30:56.787657+0000","last_unstale":"2024-05-14T09:31:30.166364+0000","last_undegraded":"2024-05-14T09:31:30.166364+0000","last_fullsized":"2024-05-14T09:31:30.166364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:37:03.232137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165471+0000","last_change":"2024-05-14T09:30:56.795985+0000","last_active":"2024-05-14T09:31:30.165471+0000","last_peered":"2024-05-14T09:31:30.165471+0000","last_clean":"2024-05-14T09:31:30.165471+0000","last_became_active":"2024-05-14T09:30:56.795575+0000","last_became_peered":"2024-05-14T09:30:56.795575+0000","last_unstale":"2024-05-14T09:31:30.165471+0000","last_undegraded":"2024-05-14T09:31:30.165471+0000","last_fullsized":"2024-05-14T09:31:30.165471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:42.923901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.175394+0000","last_change":"2024-05-14T09:30:56.795647+0000","last_active":"2024-05-14T09:31:30.175394+0000","last_peered":"2024-05-14T09:31:30.175394+0000","last_clean":"2024-05-14T09:31:30.175394+0000","last_became_active":"2024-05-14T09:30:56.795186+0000","last_became_peered":"2024-05-14T09:30:56.795186+0000","last_unstale":"2024-05-14T09:31:30.175394+0000","last_undegraded":"2024-05-14T09:31:30.175394+0000","last_fullsized":"2024-05-14T09:31:30.175394+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:04:53.471305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167059+0000","last_change":"2024-05-14T09:30:56.796834+0000","last_active":"2024-05-14T09:31:30.167059+0000","last_peered":"2024-05-14T09:31:30.167059+0000","last_clean":"2024-05-14T09:31:30.167059+0000","last_became_active":"2024-05-14T09:30:56.796492+0000","last_became_peered":"2024-05-14T09:30:56.796492+0000","last_unstale":"2024-05-14T09:31:30.167059+0000","last_undegraded":"2024-05-14T09:31:30.167059+0000","last_fullsized":"2024-05-14T09:31:30.167059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:46.606129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.172227+0000","last_change":"2024-05-14T09:30:56.795020+0000","last_active":"2024-05-14T09:31:30.172227+0000","last_peered":"2024-05-14T09:31:30.172227+0000","last_clean":"2024-05-14T09:31:30.172227+0000","last_became_active":"2024-05-14T09:30:56.794625+0000","last_became_peered":"2024-05-14T09:30:56.794625+0000","last_unstale":"2024-05-14T09:31:30.172227+0000","last_undegraded":"2024-05-14T09:31:30.172227+0000","last_fullsized":"2024-05-14T09:31:30.172227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:09:38.597296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166641+0000","last_change":"2024-05-14T09:30:56.796439+0000","last_active":"2024-05-14T09:31:30.166641+0000","last_peered":"2024-05-14T09:31:30.166641+0000","last_clean":"2024-05-14T09:31:30.166641+0000","last_became_active":"2024-05-14T09:30:56.795168+0000","last_became_peered":"2024-05-14T09:30:56.795168+0000","last_unstale":"2024-05-14T09:31:30.166641+0000","last_undegraded":"2024-05-14T09:31:30.166641+0000","last_fullsized":"2024-05-14T09:31:30.166641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:47.660286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.175736+0000","last_change":"2024-05-14T09:30:56.795994+0000","last_active":"2024-05-14T09:31:30.175736+0000","last_peered":"2024-05-14T09:31:30.175736+0000","last_clean":"2024-05-14T09:31:30.175736+0000","last_became_active":"2024-05-14T09:30:56.795079+0000","last_became_peered":"2024-05-14T09:30:56.795079+0000","last_unstale":"2024-05-14T09:31:30.175736+0000","last_undegraded":"2024-05-14T09:31:30.175736+0000","last_fullsized":"2024-05-14T09:31:30.175736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:37:42.930195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168036+0000","last_change":"2024-05-14T09:30:56.792997+0000","last_active":"2024-05-14T09:31:30.168036+0000","last_peered":"2024-05-14T09:31:30.168036+0000","last_clean":"2024-05-14T09:31:30.168036+0000","last_became_active":"2024-05-14T09:30:56.792302+0000","last_became_peered":"2024-05-14T09:30:56.792302+0000","last_unstale":"2024-05-14T09:31:30.168036+0000","last_undegraded":"2024-05-14T09:31:30.168036+0000","last_fullsized":"2024-05-14T09:31:30.168036+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:52.986662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169028+0000","last_change":"2024-05-14T09:30:56.795903+0000","last_active":"2024-05-14T09:31:30.169028+0000","last_peered":"2024-05-14T09:31:30.169028+0000","last_clean":"2024-05-14T09:31:30.169028+0000","last_became_active":"2024-05-14T09:30:56.795466+0000","last_became_peered":"2024-05-14T09:30:56.795466+0000","last_unstale":"2024-05-14T09:31:30.169028+0000","last_undegraded":"2024-05-14T09:31:30.169028+0000","last_fullsized":"2024-05-14T09:31:30.169028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:02:10.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295513+0000","last_change":"2024-05-14T09:30:56.795911+0000","last_active":"2024-05-14T09:31:30.295513+0000","last_peered":"2024-05-14T09:31:30.295513+0000","last_clean":"2024-05-14T09:31:30.295513+0000","last_became_active":"2024-05-14T09:30:56.795547+0000","last_became_peered":"2024-05-14T09:30:56.795547+0000","last_unstale":"2024-05-14T09:31:30.295513+0000","last_undegraded":"2024-05-14T09:31:30.295513+0000","last_fullsized":"2024-05-14T09:31:30.295513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:56:29.621783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295455+0000","last_change":"2024-05-14T09:30:56.787745+0000","last_active":"2024-05-14T09:31:30.295455+0000","last_peered":"2024-05-14T09:31:30.295455+0000","last_clean":"2024-05-14T09:31:30.295455+0000","last_became_active":"2024-05-14T09:30:56.787385+0000","last_became_peered":"2024-05-14T09:30:56.787385+0000","last_unstale":"2024-05-14T09:31:30.295455+0000","last_undegraded":"2024-05-14T09:31:30.295455+0000","last_fullsized":"2024-05-14T09:31:30.295455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:11.256775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171640+0000","last_change":"2024-05-14T09:30:56.795785+0000","last_active":"2024-05-14T09:31:30.171640+0000","last_peered":"2024-05-14T09:31:30.171640+0000","last_clean":"2024-05-14T09:31:30.171640+0000","last_became_active":"2024-05-14T09:30:56.795436+0000","last_became_peered":"2024-05-14T09:30:56.795436+0000","last_unstale":"2024-05-14T09:31:30.171640+0000","last_undegraded":"2024-05-14T09:31:30.171640+0000","last_fullsized":"2024-05-14T09:31:30.171640+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:12:09.221929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166125+0000","last_change":"2024-05-14T09:30:56.800688+0000","last_active":"2024-05-14T09:31:30.166125+0000","last_peered":"2024-05-14T09:31:30.166125+0000","last_clean":"2024-05-14T09:31:30.166125+0000","last_became_active":"2024-05-14T09:30:56.800204+0000","last_became_peered":"2024-05-14T09:30:56.800204+0000","last_unstale":"2024-05-14T09:31:30.166125+0000","last_undegraded":"2024-05-14T09:31:30.166125+0000","last_fullsized":"2024-05-14T09:31:30.166125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:32:27.057688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.164256+0000","last_change":"2024-05-14T09:30:56.795604+0000","last_active":"2024-05-14T09:31:30.164256+0000","last_peered":"2024-05-14T09:31:30.164256+0000","last_clean":"2024-05-14T09:31:30.164256+0000","last_became_active":"2024-05-14T09:30:56.795245+0000","last_became_peered":"2024-05-14T09:30:56.795245+0000","last_unstale":"2024-05-14T09:31:30.164256+0000","last_undegraded":"2024-05-14T09:31:30.164256+0000","last_fullsized":"2024-05-14T09:31:30.164256+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:14.072584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171084+0000","last_change":"2024-05-14T09:30:56.794815+0000","last_active":"2024-05-14T09:31:30.171084+0000","last_peered":"2024-05-14T09:31:30.171084+0000","last_clean":"2024-05-14T09:31:30.171084+0000","last_became_active":"2024-05-14T09:30:56.794429+0000","last_became_peered":"2024-05-14T09:30:56.794429+0000","last_unstale":"2024-05-14T09:31:30.171084+0000","last_undegraded":"2024-05-14T09:31:30.171084+0000","last_fullsized":"2024-05-14T09:31:30.171084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:32:08.367046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.170256+0000","last_change":"2024-05-14T09:31:08.107895+0000","last_active":"2024-05-14T09:31:30.170256+0000","last_peered":"2024-05-14T09:31:30.170256+0000","last_clean":"2024-05-14T09:31:30.170256+0000","last_became_active":"2024-05-14T09:31:08.107458+0000","last_became_peered":"2024-05-14T09:31:08.107458+0000","last_unstale":"2024-05-14T09:31:30.170256+0000","last_undegraded":"2024-05-14T09:31:30.170256+0000","last_fullsized":"2024-05-14T09:31:30.170256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:13:11.717753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":64,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.720901+0000","last_change":"2024-05-14T09:31:30.178197+0000","last_active":"2024-05-14T09:31:55.720901+0000","last_peered":"2024-05-14T09:31:55.720901+0000","last_clean":"2024-05-14T09:31:55.720901+0000","last_became_active":"2024-05-14T09:31:30.171154+0000","last_became_peered":"2024-05-14T09:31:30.171154+0000","last_unstale":"2024-05-14T09:31:55.720901+0000","last_undegraded":"2024-05-14T09:31:55.720901+0000","last_fullsized":"2024-05-14T09:31:55.720901+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:43:46.208439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167609+0000","last_change":"2024-05-14T09:30:56.800526+0000","last_active":"2024-05-14T09:31:30.167609+0000","last_peered":"2024-05-14T09:31:30.167609+0000","last_clean":"2024-05-14T09:31:30.167609+0000","last_became_active":"2024-05-14T09:30:56.797692+0000","last_became_peered":"2024-05-14T09:30:56.797692+0000","last_unstale":"2024-05-14T09:31:30.167609+0000","last_undegraded":"2024-05-14T09:31:30.167609+0000","last_fullsized":"2024-05-14T09:31:30.167609+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:09:14.471173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173810+0000","last_change":"2024-05-14T09:31:30.173810+0000","last_active":"2024-05-14T09:31:30.173810+0000","last_peered":"2024-05-14T09:31:30.173810+0000","last_clean":"2024-05-14T09:31:30.173810+0000","last_became_active":"2024-05-14T09:31:30.173242+0000","last_became_peered":"2024-05-14T09:31:30.173242+0000","last_unstale":"2024-05-14T09:31:30.173810+0000","last_undegraded":"2024-05-14T09:31:30.173810+0000","last_fullsized":"2024-05-14T09:31:30.173810+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:37:15.991805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168122+0000","last_change":"2024-05-14T09:30:54.015636+0000","last_active":"2024-05-14T09:31:30.168122+0000","last_peered":"2024-05-14T09:31:30.168122+0000","last_clean":"2024-05-14T09:31:30.168122+0000","last_became_active":"2024-05-14T09:30:54.015290+0000","last_became_peered":"2024-05-14T09:30:54.015290+0000","last_unstale":"2024-05-14T09:31:30.168122+0000","last_undegraded":"2024-05-14T09:31:30.168122+0000","last_fullsized":"2024-05-14T09:31:30.168122+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:29:17.548817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295746+0000","last_change":"2024-05-14T09:30:56.799506+0000","last_active":"2024-05-14T09:31:30.295746+0000","last_peered":"2024-05-14T09:31:30.295746+0000","last_clean":"2024-05-14T09:31:30.295746+0000","last_became_active":"2024-05-14T09:30:56.799153+0000","last_became_peered":"2024-05-14T09:30:56.799153+0000","last_unstale":"2024-05-14T09:31:30.295746+0000","last_undegraded":"2024-05-14T09:31:30.295746+0000","last_fullsized":"2024-05-14T09:31:30.295746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:04:08.217337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168111+0000","last_change":"2024-05-14T09:31:08.115533+0000","last_active":"2024-05-14T09:31:30.168111+0000","last_peered":"2024-05-14T09:31:30.168111+0000","last_clean":"2024-05-14T09:31:30.168111+0000","last_became_active":"2024-05-14T09:31:08.115033+0000","last_became_peered":"2024-05-14T09:31:08.115033+0000","last_unstale":"2024-05-14T09:31:30.168111+0000","last_undegraded":"2024-05-14T09:31:30.168111+0000","last_fullsized":"2024-05-14T09:31:30.168111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:15.668357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.719855+0000","last_change":"2024-05-14T09:30:53.770547+0000","last_active":"2024-05-14T09:31:55.719855+0000","last_peered":"2024-05-14T09:31:55.719855+0000","last_clean":"2024-05-14T09:31:55.719855+0000","last_became_active":"2024-05-14T09:30:53.770110+0000","last_became_peered":"2024-05-14T09:30:53.770110+0000","last_unstale":"2024-05-14T09:31:55.719855+0000","last_undegraded":"2024-05-14T09:31:55.719855+0000","last_fullsized":"2024-05-14T09:31:55.719855+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:18.371318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171215+0000","last_change":"2024-05-14T09:30:56.799427+0000","last_active":"2024-05-14T09:31:30.171215+0000","last_peered":"2024-05-14T09:31:30.171215+0000","last_clean":"2024-05-14T09:31:30.171215+0000","last_became_active":"2024-05-14T09:30:56.799018+0000","last_became_peered":"2024-05-14T09:30:56.799018+0000","last_unstale":"2024-05-14T09:31:30.171215+0000","last_undegraded":"2024-05-14T09:31:30.171215+0000","last_fullsized":"2024-05-14T09:31:30.171215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:13:31.837985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167589+0000","last_change":"2024-05-14T09:31:08.116258+0000","last_active":"2024-05-14T09:31:30.167589+0000","last_peered":"2024-05-14T09:31:30.167589+0000","last_clean":"2024-05-14T09:31:30.167589+0000","last_became_active":"2024-05-14T09:31:08.115040+0000","last_became_peered":"2024-05-14T09:31:08.115040+0000","last_unstale":"2024-05-14T09:31:30.167589+0000","last_undegraded":"2024-05-14T09:31:30.167589+0000","last_fullsized":"2024-05-14T09:31:30.167589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:57:54.839203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.188984+0000","last_change":"2024-05-14T09:31:30.188984+0000","last_active":"2024-05-14T09:31:30.188984+0000","last_peered":"2024-05-14T09:31:30.188984+0000","last_clean":"2024-05-14T09:31:30.188984+0000","last_became_active":"2024-05-14T09:31:30.174410+0000","last_became_peered":"2024-05-14T09:31:30.174410+0000","last_unstale":"2024-05-14T09:31:30.188984+0000","last_undegraded":"2024-05-14T09:31:30.188984+0000","last_fullsized":"2024-05-14T09:31:30.188984+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:01:35.713857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,4],"acting":[8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173271+0000","last_change":"2024-05-14T09:30:56.799165+0000","last_active":"2024-05-14T09:31:30.173271+0000","last_peered":"2024-05-14T09:31:30.173271+0000","last_clean":"2024-05-14T09:31:30.173271+0000","last_became_active":"2024-05-14T09:30:56.798750+0000","last_became_peered":"2024-05-14T09:30:56.798750+0000","last_unstale":"2024-05-14T09:31:30.173271+0000","last_undegraded":"2024-05-14T09:31:30.173271+0000","last_fullsized":"2024-05-14T09:31:30.173271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:51:28.479007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.170804+0000","last_change":"2024-05-14T09:31:08.115740+0000","last_active":"2024-05-14T09:31:30.170804+0000","last_peered":"2024-05-14T09:31:30.170804+0000","last_clean":"2024-05-14T09:31:30.170804+0000","last_became_active":"2024-05-14T09:31:08.115202+0000","last_became_peered":"2024-05-14T09:31:08.115202+0000","last_unstale":"2024-05-14T09:31:30.170804+0000","last_undegraded":"2024-05-14T09:31:30.170804+0000","last_fullsized":"2024-05-14T09:31:30.170804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:44.519280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":74,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:32:21.279798+0000","last_change":"2024-05-14T09:30:53.770869+0000","last_active":"2024-05-14T09:32:21.279798+0000","last_peered":"2024-05-14T09:32:21.279798+0000","last_clean":"2024-05-14T09:32:21.279798+0000","last_became_active":"2024-05-14T09:30:53.770425+0000","last_became_peered":"2024-05-14T09:30:53.770425+0000","last_unstale":"2024-05-14T09:32:21.279798+0000","last_undegraded":"2024-05-14T09:32:21.279798+0000","last_fullsized":"2024-05-14T09:32:21.279798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:03:33.768120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166858+0000","last_change":"2024-05-14T09:30:56.789267+0000","last_active":"2024-05-14T09:31:30.166858+0000","last_peered":"2024-05-14T09:31:30.166858+0000","last_clean":"2024-05-14T09:31:30.166858+0000","last_became_active":"2024-05-14T09:30:56.788883+0000","last_became_peered":"2024-05-14T09:30:56.788883+0000","last_unstale":"2024-05-14T09:31:30.166858+0000","last_undegraded":"2024-05-14T09:31:30.166858+0000","last_fullsized":"2024-05-14T09:31:30.166858+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:09.818693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.170314+0000","last_change":"2024-05-14T09:31:08.115193+0000","last_active":"2024-05-14T09:31:30.170314+0000","last_peered":"2024-05-14T09:31:30.170314+0000","last_clean":"2024-05-14T09:31:30.170314+0000","last_became_active":"2024-05-14T09:31:08.114701+0000","last_became_peered":"2024-05-14T09:31:08.114701+0000","last_unstale":"2024-05-14T09:31:30.170314+0000","last_undegraded":"2024-05-14T09:31:30.170314+0000","last_fullsized":"2024-05-14T09:31:30.170314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:47:48.487922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.720597+0000","last_change":"2024-05-14T09:31:30.174760+0000","last_active":"2024-05-14T09:31:55.720597+0000","last_peered":"2024-05-14T09:31:55.720597+0000","last_clean":"2024-05-14T09:31:55.720597+0000","last_became_active":"2024-05-14T09:31:30.174253+0000","last_became_peered":"2024-05-14T09:31:30.174253+0000","last_unstale":"2024-05-14T09:31:55.720597+0000","last_undegraded":"2024-05-14T09:31:55.720597+0000","last_fullsized":"2024-05-14T09:31:55.720597+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:11:17.163436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173142+0000","last_change":"2024-05-14T09:30:56.794654+0000","last_active":"2024-05-14T09:31:30.173142+0000","last_peered":"2024-05-14T09:31:30.173142+0000","last_clean":"2024-05-14T09:31:30.173142+0000","last_became_active":"2024-05-14T09:30:56.794282+0000","last_became_peered":"2024-05-14T09:30:56.794282+0000","last_unstale":"2024-05-14T09:31:30.173142+0000","last_undegraded":"2024-05-14T09:31:30.173142+0000","last_fullsized":"2024-05-14T09:31:30.173142+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:55:41.478707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165960+0000","last_change":"2024-05-14T09:31:08.119965+0000","last_active":"2024-05-14T09:31:30.165960+0000","last_peered":"2024-05-14T09:31:30.165960+0000","last_clean":"2024-05-14T09:31:30.165960+0000","last_became_active":"2024-05-14T09:31:08.119502+0000","last_became_peered":"2024-05-14T09:31:08.119502+0000","last_unstale":"2024-05-14T09:31:30.165960+0000","last_undegraded":"2024-05-14T09:31:30.165960+0000","last_fullsized":"2024-05-14T09:31:30.165960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:01:23.612036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.720018+0000","last_change":"2024-05-14T09:30:53.766495+0000","last_active":"2024-05-14T09:31:55.720018+0000","last_peered":"2024-05-14T09:31:55.720018+0000","last_clean":"2024-05-14T09:31:55.720018+0000","last_became_active":"2024-05-14T09:30:53.766122+0000","last_became_peered":"2024-05-14T09:30:53.766122+0000","last_unstale":"2024-05-14T09:31:55.720018+0000","last_undegraded":"2024-05-14T09:31:55.720018+0000","last_fullsized":"2024-05-14T09:31:55.720018+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:43:13.344209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167185+0000","last_change":"2024-05-14T09:30:56.791938+0000","last_active":"2024-05-14T09:31:30.167185+0000","last_peered":"2024-05-14T09:31:30.167185+0000","last_clean":"2024-05-14T09:31:30.167185+0000","last_became_active":"2024-05-14T09:30:56.791525+0000","last_became_peered":"2024-05-14T09:30:56.791525+0000","last_unstale":"2024-05-14T09:31:30.167185+0000","last_undegraded":"2024-05-14T09:31:30.167185+0000","last_fullsized":"2024-05-14T09:31:30.167185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:18.178208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295717+0000","last_change":"2024-05-14T09:31:08.113548+0000","last_active":"2024-05-14T09:31:30.295717+0000","last_peered":"2024-05-14T09:31:30.295717+0000","last_clean":"2024-05-14T09:31:30.295717+0000","last_became_active":"2024-05-14T09:31:08.113117+0000","last_became_peered":"2024-05-14T09:31:08.113117+0000","last_unstale":"2024-05-14T09:31:30.295717+0000","last_undegraded":"2024-05-14T09:31:30.295717+0000","last_fullsized":"2024-05-14T09:31:30.295717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:25:15.483761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171131+0000","last_change":"2024-05-14T09:30:53.770987+0000","last_active":"2024-05-14T09:31:30.171131+0000","last_peered":"2024-05-14T09:31:30.171131+0000","last_clean":"2024-05-14T09:31:30.171131+0000","last_became_active":"2024-05-14T09:30:53.770609+0000","last_became_peered":"2024-05-14T09:30:53.770609+0000","last_unstale":"2024-05-14T09:31:30.171131+0000","last_undegraded":"2024-05-14T09:31:30.171131+0000","last_fullsized":"2024-05-14T09:31:30.171131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:26:22.020642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.163851+0000","last_change":"2024-05-14T09:30:56.789219+0000","last_active":"2024-05-14T09:31:30.163851+0000","last_peered":"2024-05-14T09:31:30.163851+0000","last_clean":"2024-05-14T09:31:30.163851+0000","last_became_active":"2024-05-14T09:30:56.788871+0000","last_became_peered":"2024-05-14T09:30:56.788871+0000","last_unstale":"2024-05-14T09:31:30.163851+0000","last_undegraded":"2024-05-14T09:31:30.163851+0000","last_fullsized":"2024-05-14T09:31:30.163851+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:10:50.774267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.174708+0000","last_change":"2024-05-14T09:31:30.174708+0000","last_active":"2024-05-14T09:31:30.174708+0000","last_peered":"2024-05-14T09:31:30.174708+0000","last_clean":"2024-05-14T09:31:30.174708+0000","last_became_active":"2024-05-14T09:31:30.174199+0000","last_became_peered":"2024-05-14T09:31:30.174199+0000","last_unstale":"2024-05-14T09:31:30.174708+0000","last_undegraded":"2024-05-14T09:31:30.174708+0000","last_fullsized":"2024-05-14T09:31:30.174708+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:23:55.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:32:21.278852+0000","last_change":"2024-05-14T09:30:53.770898+0000","last_active":"2024-05-14T09:32:21.278852+0000","last_peered":"2024-05-14T09:32:21.278852+0000","last_clean":"2024-05-14T09:32:21.278852+0000","last_became_active":"2024-05-14T09:30:53.770483+0000","last_became_peered":"2024-05-14T09:30:53.770483+0000","last_unstale":"2024-05-14T09:32:21.278852+0000","last_undegraded":"2024-05-14T09:32:21.278852+0000","last_fullsized":"2024-05-14T09:32:21.278852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:07.764348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173069+0000","last_change":"2024-05-14T09:30:56.799049+0000","last_active":"2024-05-14T09:31:30.173069+0000","last_peered":"2024-05-14T09:31:30.173069+0000","last_clean":"2024-05-14T09:31:30.173069+0000","last_became_active":"2024-05-14T09:30:56.798643+0000","last_became_peered":"2024-05-14T09:30:56.798643+0000","last_unstale":"2024-05-14T09:31:30.173069+0000","last_undegraded":"2024-05-14T09:31:30.173069+0000","last_fullsized":"2024-05-14T09:31:30.173069+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:12.378373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.163806+0000","last_change":"2024-05-14T09:31:08.109876+0000","last_active":"2024-05-14T09:31:30.163806+0000","last_peered":"2024-05-14T09:31:30.163806+0000","last_clean":"2024-05-14T09:31:30.163806+0000","last_became_active":"2024-05-14T09:31:08.109470+0000","last_became_peered":"2024-05-14T09:31:08.109470+0000","last_unstale":"2024-05-14T09:31:30.163806+0000","last_undegraded":"2024-05-14T09:31:30.163806+0000","last_fullsized":"2024-05-14T09:31:30.163806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:07.704153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.711814+0000","last_change":"2024-05-14T09:31:30.179086+0000","last_active":"2024-05-14T09:31:54.711814+0000","last_peered":"2024-05-14T09:31:54.711814+0000","last_clean":"2024-05-14T09:31:54.711814+0000","last_became_active":"2024-05-14T09:31:30.178604+0000","last_became_peered":"2024-05-14T09:31:30.178604+0000","last_unstale":"2024-05-14T09:31:54.711814+0000","last_undegraded":"2024-05-14T09:31:54.711814+0000","last_fullsized":"2024-05-14T09:31:54.711814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:39.712395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295613+0000","last_change":"2024-05-14T09:30:56.793109+0000","last_active":"2024-05-14T09:31:30.295613+0000","last_peered":"2024-05-14T09:31:30.295613+0000","last_clean":"2024-05-14T09:31:30.295613+0000","last_became_active":"2024-05-14T09:30:56.792761+0000","last_became_peered":"2024-05-14T09:30:56.792761+0000","last_unstale":"2024-05-14T09:31:30.295613+0000","last_undegraded":"2024-05-14T09:31:30.295613+0000","last_fullsized":"2024-05-14T09:31:30.295613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:25:33.216281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168519+0000","last_change":"2024-05-14T09:31:08.120978+0000","last_active":"2024-05-14T09:31:30.168519+0000","last_peered":"2024-05-14T09:31:30.168519+0000","last_clean":"2024-05-14T09:31:30.168519+0000","last_became_active":"2024-05-14T09:31:08.120227+0000","last_became_peered":"2024-05-14T09:31:08.120227+0000","last_unstale":"2024-05-14T09:31:30.168519+0000","last_undegraded":"2024-05-14T09:31:30.168519+0000","last_fullsized":"2024-05-14T09:31:30.168519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:06.630390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.710669+0000","last_change":"2024-05-14T09:30:54.017382+0000","last_active":"2024-05-14T09:31:54.710669+0000","last_peered":"2024-05-14T09:31:54.710669+0000","last_clean":"2024-05-14T09:31:54.710669+0000","last_became_active":"2024-05-14T09:30:54.016887+0000","last_became_peered":"2024-05-14T09:30:54.016887+0000","last_unstale":"2024-05-14T09:31:54.710669+0000","last_undegraded":"2024-05-14T09:31:54.710669+0000","last_fullsized":"2024-05-14T09:31:54.710669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:28.823659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171462+0000","last_change":"2024-05-14T09:30:56.800626+0000","last_active":"2024-05-14T09:31:30.171462+0000","last_peered":"2024-05-14T09:31:30.171462+0000","last_clean":"2024-05-14T09:31:30.171462+0000","last_became_active":"2024-05-14T09:30:56.800258+0000","last_became_peered":"2024-05-14T09:30:56.800258+0000","last_unstale":"2024-05-14T09:31:30.171462+0000","last_undegraded":"2024-05-14T09:31:30.171462+0000","last_fullsized":"2024-05-14T09:31:30.171462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:47:26.369633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.163943+0000","last_change":"2024-05-14T09:31:08.115456+0000","last_active":"2024-05-14T09:31:30.163943+0000","last_peered":"2024-05-14T09:31:30.163943+0000","last_clean":"2024-05-14T09:31:30.163943+0000","last_became_active":"2024-05-14T09:31:08.114705+0000","last_became_peered":"2024-05-14T09:31:08.114705+0000","last_unstale":"2024-05-14T09:31:30.163943+0000","last_undegraded":"2024-05-14T09:31:30.163943+0000","last_fullsized":"2024-05-14T09:31:30.163943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:50.096747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:32:22.280367+0000","last_change":"2024-05-14T09:31:30.186202+0000","last_active":"2024-05-14T09:32:22.280367+0000","last_peered":"2024-05-14T09:32:22.280367+0000","last_clean":"2024-05-14T09:32:22.280367+0000","last_became_active":"2024-05-14T09:31:30.174248+0000","last_became_peered":"2024-05-14T09:31:30.174248+0000","last_unstale":"2024-05-14T09:32:22.280367+0000","last_undegraded":"2024-05-14T09:32:22.280367+0000","last_fullsized":"2024-05-14T09:32:22.280367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:45:03.131934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58657,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":58,"num_read_kb":86,"num_write":32,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166988+0000","last_change":"2024-05-14T09:30:56.791099+0000","last_active":"2024-05-14T09:31:30.166988+0000","last_peered":"2024-05-14T09:31:30.166988+0000","last_clean":"2024-05-14T09:31:30.166988+0000","last_became_active":"2024-05-14T09:30:56.790676+0000","last_became_peered":"2024-05-14T09:30:56.790676+0000","last_unstale":"2024-05-14T09:31:30.166988+0000","last_undegraded":"2024-05-14T09:31:30.166988+0000","last_fullsized":"2024-05-14T09:31:30.166988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:09.851357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173530+0000","last_change":"2024-05-14T09:31:08.120289+0000","last_active":"2024-05-14T09:31:30.173530+0000","last_peered":"2024-05-14T09:31:30.173530+0000","last_clean":"2024-05-14T09:31:30.173530+0000","last_became_active":"2024-05-14T09:31:08.118574+0000","last_became_peered":"2024-05-14T09:31:08.118574+0000","last_unstale":"2024-05-14T09:31:30.173530+0000","last_undegraded":"2024-05-14T09:31:30.173530+0000","last_fullsized":"2024-05-14T09:31:30.173530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:01.723442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.717856+0000","last_change":"2024-05-14T09:30:53.766578+0000","last_active":"2024-05-14T09:31:55.717856+0000","last_peered":"2024-05-14T09:31:55.717856+0000","last_clean":"2024-05-14T09:31:55.717856+0000","last_became_active":"2024-05-14T09:30:53.765928+0000","last_became_peered":"2024-05-14T09:30:53.765928+0000","last_unstale":"2024-05-14T09:31:55.717856+0000","last_undegraded":"2024-05-14T09:31:55.717856+0000","last_fullsized":"2024-05-14T09:31:55.717856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:19:08.690013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.170200+0000","last_change":"2024-05-14T09:30:56.785507+0000","last_active":"2024-05-14T09:31:30.170200+0000","last_peered":"2024-05-14T09:31:30.170200+0000","last_clean":"2024-05-14T09:31:30.170200+0000","last_became_active":"2024-05-14T09:30:56.784800+0000","last_became_peered":"2024-05-14T09:30:56.784800+0000","last_unstale":"2024-05-14T09:31:30.170200+0000","last_undegraded":"2024-05-14T09:31:30.170200+0000","last_fullsized":"2024-05-14T09:31:30.170200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:36:05.440222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165117+0000","last_change":"2024-05-14T09:31:08.119051+0000","last_active":"2024-05-14T09:31:30.165117+0000","last_peered":"2024-05-14T09:31:30.165117+0000","last_clean":"2024-05-14T09:31:30.165117+0000","last_became_active":"2024-05-14T09:31:08.118569+0000","last_became_peered":"2024-05-14T09:31:08.118569+0000","last_unstale":"2024-05-14T09:31:30.165117+0000","last_undegraded":"2024-05-14T09:31:30.165117+0000","last_fullsized":"2024-05-14T09:31:30.165117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:12:39.643740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.710378+0000","last_change":"2024-05-14T09:30:53.772832+0000","last_active":"2024-05-14T09:31:54.710378+0000","last_peered":"2024-05-14T09:31:54.710378+0000","last_clean":"2024-05-14T09:31:54.710378+0000","last_became_active":"2024-05-14T09:30:53.772228+0000","last_became_peered":"2024-05-14T09:30:53.772228+0000","last_unstale":"2024-05-14T09:31:54.710378+0000","last_undegraded":"2024-05-14T09:31:54.710378+0000","last_fullsized":"2024-05-14T09:31:54.710378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:14:43.350330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.175161+0000","last_change":"2024-05-14T09:30:56.782831+0000","last_active":"2024-05-14T09:31:30.175161+0000","last_peered":"2024-05-14T09:31:30.175161+0000","last_clean":"2024-05-14T09:31:30.175161+0000","last_became_active":"2024-05-14T09:30:56.782402+0000","last_became_peered":"2024-05-14T09:30:56.782402+0000","last_unstale":"2024-05-14T09:31:30.175161+0000","last_undegraded":"2024-05-14T09:31:30.175161+0000","last_fullsized":"2024-05-14T09:31:30.175161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:54.603524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.297353+0000","last_change":"2024-05-14T09:31:30.296957+0000","last_active":"2024-05-14T09:31:30.297353+0000","last_peered":"2024-05-14T09:31:30.297353+0000","last_clean":"2024-05-14T09:31:30.297353+0000","last_became_active":"2024-05-14T09:31:30.296554+0000","last_became_peered":"2024-05-14T09:31:30.296554+0000","last_unstale":"2024-05-14T09:31:30.297353+0000","last_undegraded":"2024-05-14T09:31:30.297353+0000","last_fullsized":"2024-05-14T09:31:30.297353+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:28.963042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:32:20.277741+0000","last_change":"2024-05-14T09:30:53.772731+0000","last_active":"2024-05-14T09:32:20.277741+0000","last_peered":"2024-05-14T09:32:20.277741+0000","last_clean":"2024-05-14T09:32:20.277741+0000","last_became_active":"2024-05-14T09:30:53.771986+0000","last_became_peered":"2024-05-14T09:30:53.771986+0000","last_unstale":"2024-05-14T09:32:20.277741+0000","last_undegraded":"2024-05-14T09:32:20.277741+0000","last_fullsized":"2024-05-14T09:32:20.277741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:50:00.810937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173861+0000","last_change":"2024-05-14T09:30:56.790503+0000","last_active":"2024-05-14T09:31:30.173861+0000","last_peered":"2024-05-14T09:31:30.173861+0000","last_clean":"2024-05-14T09:31:30.173861+0000","last_became_active":"2024-05-14T09:30:56.790103+0000","last_became_peered":"2024-05-14T09:30:56.790103+0000","last_unstale":"2024-05-14T09:31:30.173861+0000","last_undegraded":"2024-05-14T09:31:30.173861+0000","last_fullsized":"2024-05-14T09:31:30.173861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:00.950794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169425+0000","last_change":"2024-05-14T09:31:08.116475+0000","last_active":"2024-05-14T09:31:30.169425+0000","last_peered":"2024-05-14T09:31:30.169425+0000","last_clean":"2024-05-14T09:31:30.169425+0000","last_became_active":"2024-05-14T09:31:08.116061+0000","last_became_peered":"2024-05-14T09:31:08.116061+0000","last_unstale":"2024-05-14T09:31:30.169425+0000","last_undegraded":"2024-05-14T09:31:30.169425+0000","last_fullsized":"2024-05-14T09:31:30.169425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:07:02.281853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.721199+0000","last_change":"2024-05-14T09:30:54.016570+0000","last_active":"2024-05-14T09:31:55.721199+0000","last_peered":"2024-05-14T09:31:55.721199+0000","last_clean":"2024-05-14T09:31:55.721199+0000","last_became_active":"2024-05-14T09:30:54.015987+0000","last_became_peered":"2024-05-14T09:30:54.015987+0000","last_unstale":"2024-05-14T09:31:55.721199+0000","last_undegraded":"2024-05-14T09:31:55.721199+0000","last_fullsized":"2024-05-14T09:31:55.721199+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:18:28.694488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171816+0000","last_change":"2024-05-14T09:30:56.800727+0000","last_active":"2024-05-14T09:31:30.171816+0000","last_peered":"2024-05-14T09:31:30.171816+0000","last_clean":"2024-05-14T09:31:30.171816+0000","last_became_active":"2024-05-14T09:30:56.800354+0000","last_became_peered":"2024-05-14T09:30:56.800354+0000","last_unstale":"2024-05-14T09:31:30.171816+0000","last_undegraded":"2024-05-14T09:31:30.171816+0000","last_fullsized":"2024-05-14T09:31:30.171816+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:13:07.227577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171643+0000","last_change":"2024-05-14T09:31:08.115685+0000","last_active":"2024-05-14T09:31:30.171643+0000","last_peered":"2024-05-14T09:31:30.171643+0000","last_clean":"2024-05-14T09:31:30.171643+0000","last_became_active":"2024-05-14T09:31:08.114948+0000","last_became_peered":"2024-05-14T09:31:08.114948+0000","last_unstale":"2024-05-14T09:31:30.171643+0000","last_undegraded":"2024-05-14T09:31:30.171643+0000","last_fullsized":"2024-05-14T09:31:30.171643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:33:07.286823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:56.714258+0000","last_change":"2024-05-14T09:30:53.769949+0000","last_active":"2024-05-14T09:31:56.714258+0000","last_peered":"2024-05-14T09:31:56.714258+0000","last_clean":"2024-05-14T09:31:56.714258+0000","last_became_active":"2024-05-14T09:30:53.769327+0000","last_became_peered":"2024-05-14T09:30:53.769327+0000","last_unstale":"2024-05-14T09:31:56.714258+0000","last_undegraded":"2024-05-14T09:31:56.714258+0000","last_fullsized":"2024-05-14T09:31:56.714258+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:35:11.850991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165834+0000","last_change":"2024-05-14T09:30:56.796590+0000","last_active":"2024-05-14T09:31:30.165834+0000","last_peered":"2024-05-14T09:31:30.165834+0000","last_clean":"2024-05-14T09:31:30.165834+0000","last_became_active":"2024-05-14T09:30:56.796185+0000","last_became_peered":"2024-05-14T09:30:56.796185+0000","last_unstale":"2024-05-14T09:31:30.165834+0000","last_undegraded":"2024-05-14T09:31:30.165834+0000","last_fullsized":"2024-05-14T09:31:30.165834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:48:58.388665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166814+0000","last_change":"2024-05-14T09:31:08.387316+0000","last_active":"2024-05-14T09:31:30.166814+0000","last_peered":"2024-05-14T09:31:30.166814+0000","last_clean":"2024-05-14T09:31:30.166814+0000","last_became_active":"2024-05-14T09:31:08.386739+0000","last_became_peered":"2024-05-14T09:31:08.386739+0000","last_unstale":"2024-05-14T09:31:30.166814+0000","last_undegraded":"2024-05-14T09:31:30.166814+0000","last_fullsized":"2024-05-14T09:31:30.166814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:17.870255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.164455+0000","last_change":"2024-05-14T09:30:54.017756+0000","last_active":"2024-05-14T09:31:30.164455+0000","last_peered":"2024-05-14T09:31:30.164455+0000","last_clean":"2024-05-14T09:31:30.164455+0000","last_became_active":"2024-05-14T09:30:54.017277+0000","last_became_peered":"2024-05-14T09:30:54.017277+0000","last_unstale":"2024-05-14T09:31:30.164455+0000","last_undegraded":"2024-05-14T09:31:30.164455+0000","last_fullsized":"2024-05-14T09:31:30.164455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:27:36.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166185+0000","last_change":"2024-05-14T09:30:56.788408+0000","last_active":"2024-05-14T09:31:30.166185+0000","last_peered":"2024-05-14T09:31:30.166185+0000","last_clean":"2024-05-14T09:31:30.166185+0000","last_became_active":"2024-05-14T09:30:56.787955+0000","last_became_peered":"2024-05-14T09:30:56.787955+0000","last_unstale":"2024-05-14T09:31:30.166185+0000","last_undegraded":"2024-05-14T09:31:30.166185+0000","last_fullsized":"2024-05-14T09:31:30.166185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:09.777069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169255+0000","last_change":"2024-05-14T09:31:08.119078+0000","last_active":"2024-05-14T09:31:30.169255+0000","last_peered":"2024-05-14T09:31:30.169255+0000","last_clean":"2024-05-14T09:31:30.169255+0000","last_became_active":"2024-05-14T09:31:08.117939+0000","last_became_peered":"2024-05-14T09:31:08.117939+0000","last_unstale":"2024-05-14T09:31:30.169255+0000","last_undegraded":"2024-05-14T09:31:30.169255+0000","last_fullsized":"2024-05-14T09:31:30.169255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:48:37.363503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.174543+0000","last_change":"2024-05-14T09:30:53.767843+0000","last_active":"2024-05-14T09:31:30.174543+0000","last_peered":"2024-05-14T09:31:30.174543+0000","last_clean":"2024-05-14T09:31:30.174543+0000","last_became_active":"2024-05-14T09:30:53.767419+0000","last_became_peered":"2024-05-14T09:30:53.767419+0000","last_unstale":"2024-05-14T09:31:30.174543+0000","last_undegraded":"2024-05-14T09:31:30.174543+0000","last_fullsized":"2024-05-14T09:31:30.174543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:49:06.441600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165701+0000","last_change":"2024-05-14T09:30:56.796001+0000","last_active":"2024-05-14T09:31:30.165701+0000","last_peered":"2024-05-14T09:31:30.165701+0000","last_clean":"2024-05-14T09:31:30.165701+0000","last_became_active":"2024-05-14T09:30:56.794517+0000","last_became_peered":"2024-05-14T09:30:56.794517+0000","last_unstale":"2024-05-14T09:31:30.165701+0000","last_undegraded":"2024-05-14T09:31:30.165701+0000","last_fullsized":"2024-05-14T09:31:30.165701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:20:37.514679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169553+0000","last_change":"2024-05-14T09:31:08.387570+0000","last_active":"2024-05-14T09:31:30.169553+0000","last_peered":"2024-05-14T09:31:30.169553+0000","last_clean":"2024-05-14T09:31:30.169553+0000","last_became_active":"2024-05-14T09:31:08.386745+0000","last_became_peered":"2024-05-14T09:31:08.386745+0000","last_unstale":"2024-05-14T09:31:30.169553+0000","last_undegraded":"2024-05-14T09:31:30.169553+0000","last_fullsized":"2024-05-14T09:31:30.169553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:44:01.024784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171385+0000","last_change":"2024-05-14T09:30:56.795532+0000","last_active":"2024-05-14T09:31:30.171385+0000","last_peered":"2024-05-14T09:31:30.171385+0000","last_clean":"2024-05-14T09:31:30.171385+0000","last_became_active":"2024-05-14T09:30:56.795171+0000","last_became_peered":"2024-05-14T09:30:56.795171+0000","last_unstale":"2024-05-14T09:31:30.171385+0000","last_undegraded":"2024-05-14T09:31:30.171385+0000","last_fullsized":"2024-05-14T09:31:30.171385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:09:40.252489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:32:21.278805+0000","last_change":"2024-05-14T09:30:54.015629+0000","last_active":"2024-05-14T09:32:21.278805+0000","last_peered":"2024-05-14T09:32:21.278805+0000","last_clean":"2024-05-14T09:32:21.278805+0000","last_became_active":"2024-05-14T09:30:54.015288+0000","last_became_peered":"2024-05-14T09:30:54.015288+0000","last_unstale":"2024-05-14T09:32:21.278805+0000","last_undegraded":"2024-05-14T09:32:21.278805+0000","last_fullsized":"2024-05-14T09:32:21.278805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:50.316125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.180012+0000","last_change":"2024-05-14T09:31:30.179056+0000","last_active":"2024-05-14T09:31:30.180012+0000","last_peered":"2024-05-14T09:31:30.180012+0000","last_clean":"2024-05-14T09:31:30.180012+0000","last_became_active":"2024-05-14T09:31:30.178398+0000","last_became_peered":"2024-05-14T09:31:30.178398+0000","last_unstale":"2024-05-14T09:31:30.180012+0000","last_undegraded":"2024-05-14T09:31:30.180012+0000","last_fullsized":"2024-05-14T09:31:30.180012+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:15:55.369801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167393+0000","last_change":"2024-05-14T09:30:56.793642+0000","last_active":"2024-05-14T09:31:30.167393+0000","last_peered":"2024-05-14T09:31:30.167393+0000","last_clean":"2024-05-14T09:31:30.167393+0000","last_became_active":"2024-05-14T09:30:56.793229+0000","last_became_peered":"2024-05-14T09:30:56.793229+0000","last_unstale":"2024-05-14T09:31:30.167393+0000","last_undegraded":"2024-05-14T09:31:30.167393+0000","last_fullsized":"2024-05-14T09:31:30.167393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:17.940674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:58.239238+0000","last_change":"2024-05-14T09:30:53.769582+0000","last_active":"2024-05-14T09:31:58.239238+0000","last_peered":"2024-05-14T09:31:58.239238+0000","last_clean":"2024-05-14T09:31:58.239238+0000","last_became_active":"2024-05-14T09:30:53.769113+0000","last_became_peered":"2024-05-14T09:30:53.769113+0000","last_unstale":"2024-05-14T09:31:58.239238+0000","last_undegraded":"2024-05-14T09:31:58.239238+0000","last_fullsized":"2024-05-14T09:31:58.239238+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:48.755113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":114,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173554+0000","last_change":"2024-05-14T09:29:30.767389+0000","last_active":"2024-05-14T09:31:30.173554+0000","last_peered":"2024-05-14T09:31:30.173554+0000","last_clean":"2024-05-14T09:31:30.173554+0000","last_became_active":"2024-05-14T09:29:30.761827+0000","last_became_peered":"2024-05-14T09:29:30.761827+0000","last_unstale":"2024-05-14T09:31:30.173554+0000","last_undegraded":"2024-05-14T09:31:30.173554+0000","last_fullsized":"2024-05-14T09:31:30.173554+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:26:03.741821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:26:03.741821+0000","last_clean_scrub_stamp":"2024-05-14T09:26:03.741821+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:52:11.421658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.172554+0000","last_change":"2024-05-14T09:31:08.387819+0000","last_active":"2024-05-14T09:31:30.172554+0000","last_peered":"2024-05-14T09:31:30.172554+0000","last_clean":"2024-05-14T09:31:30.172554+0000","last_became_active":"2024-05-14T09:31:08.387275+0000","last_became_peered":"2024-05-14T09:31:08.387275+0000","last_unstale":"2024-05-14T09:31:30.172554+0000","last_undegraded":"2024-05-14T09:31:30.172554+0000","last_fullsized":"2024-05-14T09:31:30.172554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:46.627389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168669+0000","last_change":"2024-05-14T09:30:56.788159+0000","last_active":"2024-05-14T09:31:30.168669+0000","last_peered":"2024-05-14T09:31:30.168669+0000","last_clean":"2024-05-14T09:31:30.168669+0000","last_became_active":"2024-05-14T09:30:56.787744+0000","last_became_peered":"2024-05-14T09:30:56.787744+0000","last_unstale":"2024-05-14T09:31:30.168669+0000","last_undegraded":"2024-05-14T09:31:30.168669+0000","last_fullsized":"2024-05-14T09:31:30.168669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:45:25.243471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.719840+0000","last_change":"2024-05-14T09:30:53.774345+0000","last_active":"2024-05-14T09:31:55.719840+0000","last_peered":"2024-05-14T09:31:55.719840+0000","last_clean":"2024-05-14T09:31:55.719840+0000","last_became_active":"2024-05-14T09:30:53.773853+0000","last_became_peered":"2024-05-14T09:30:53.773853+0000","last_unstale":"2024-05-14T09:31:55.719840+0000","last_undegraded":"2024-05-14T09:31:55.719840+0000","last_fullsized":"2024-05-14T09:31:55.719840+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:35:31.938442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165350+0000","last_change":"2024-05-14T09:31:08.122876+0000","last_active":"2024-05-14T09:31:30.165350+0000","last_peered":"2024-05-14T09:31:30.165350+0000","last_clean":"2024-05-14T09:31:30.165350+0000","last_became_active":"2024-05-14T09:31:08.122182+0000","last_became_peered":"2024-05-14T09:31:08.122182+0000","last_unstale":"2024-05-14T09:31:30.165350+0000","last_undegraded":"2024-05-14T09:31:30.165350+0000","last_fullsized":"2024-05-14T09:31:30.165350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:47:30.735057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.707799+0000","last_change":"2024-05-14T09:30:54.018168+0000","last_active":"2024-05-14T09:31:54.707799+0000","last_peered":"2024-05-14T09:31:54.707799+0000","last_clean":"2024-05-14T09:31:54.707799+0000","last_became_active":"2024-05-14T09:30:54.017699+0000","last_became_peered":"2024-05-14T09:30:54.017699+0000","last_unstale":"2024-05-14T09:31:54.707799+0000","last_undegraded":"2024-05-14T09:31:54.707799+0000","last_fullsized":"2024-05-14T09:31:54.707799+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:31:01.162000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169427+0000","last_change":"2024-05-14T09:30:56.787156+0000","last_active":"2024-05-14T09:31:30.169427+0000","last_peered":"2024-05-14T09:31:30.169427+0000","last_clean":"2024-05-14T09:31:30.169427+0000","last_became_active":"2024-05-14T09:30:56.786832+0000","last_became_peered":"2024-05-14T09:30:56.786832+0000","last_unstale":"2024-05-14T09:31:30.169427+0000","last_undegraded":"2024-05-14T09:31:30.169427+0000","last_fullsized":"2024-05-14T09:31:30.169427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:00:41.719641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167185+0000","last_change":"2024-05-14T09:31:08.387907+0000","last_active":"2024-05-14T09:31:30.167185+0000","last_peered":"2024-05-14T09:31:30.167185+0000","last_clean":"2024-05-14T09:31:30.167185+0000","last_became_active":"2024-05-14T09:31:08.386765+0000","last_became_peered":"2024-05-14T09:31:08.386765+0000","last_unstale":"2024-05-14T09:31:30.167185+0000","last_undegraded":"2024-05-14T09:31:30.167185+0000","last_fullsized":"2024-05-14T09:31:30.167185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:24:56.890353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:32:18.803807+0000","last_change":"2024-05-14T09:30:53.767508+0000","last_active":"2024-05-14T09:32:18.803807+0000","last_peered":"2024-05-14T09:32:18.803807+0000","last_clean":"2024-05-14T09:32:18.803807+0000","last_became_active":"2024-05-14T09:30:53.767169+0000","last_became_peered":"2024-05-14T09:30:53.767169+0000","last_unstale":"2024-05-14T09:32:18.803807+0000","last_undegraded":"2024-05-14T09:32:18.803807+0000","last_fullsized":"2024-05-14T09:32:18.803807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:04:23.621686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167705+0000","last_change":"2024-05-14T09:30:56.790398+0000","last_active":"2024-05-14T09:31:30.167705+0000","last_peered":"2024-05-14T09:31:30.167705+0000","last_clean":"2024-05-14T09:31:30.167705+0000","last_became_active":"2024-05-14T09:30:56.790055+0000","last_became_peered":"2024-05-14T09:30:56.790055+0000","last_unstale":"2024-05-14T09:31:30.167705+0000","last_undegraded":"2024-05-14T09:31:30.167705+0000","last_fullsized":"2024-05-14T09:31:30.167705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:12:37.688647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169218+0000","last_change":"2024-05-14T09:31:08.111533+0000","last_active":"2024-05-14T09:31:30.169218+0000","last_peered":"2024-05-14T09:31:30.169218+0000","last_clean":"2024-05-14T09:31:30.169218+0000","last_became_active":"2024-05-14T09:31:08.110448+0000","last_became_peered":"2024-05-14T09:31:08.110448+0000","last_unstale":"2024-05-14T09:31:30.169218+0000","last_undegraded":"2024-05-14T09:31:30.169218+0000","last_fullsized":"2024-05-14T09:31:30.169218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:43:45.822382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.709126+0000","last_change":"2024-05-14T09:30:54.069180+0000","last_active":"2024-05-14T09:31:54.709126+0000","last_peered":"2024-05-14T09:31:54.709126+0000","last_clean":"2024-05-14T09:31:54.709126+0000","last_became_active":"2024-05-14T09:30:54.068846+0000","last_became_peered":"2024-05-14T09:30:54.068846+0000","last_unstale":"2024-05-14T09:31:54.709126+0000","last_undegraded":"2024-05-14T09:31:54.709126+0000","last_fullsized":"2024-05-14T09:31:54.709126+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:22.541720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295970+0000","last_change":"2024-05-14T09:30:56.788159+0000","last_active":"2024-05-14T09:31:30.295970+0000","last_peered":"2024-05-14T09:31:30.295970+0000","last_clean":"2024-05-14T09:31:30.295970+0000","last_became_active":"2024-05-14T09:30:56.787786+0000","last_became_peered":"2024-05-14T09:30:56.787786+0000","last_unstale":"2024-05-14T09:31:30.295970+0000","last_undegraded":"2024-05-14T09:31:30.295970+0000","last_fullsized":"2024-05-14T09:31:30.295970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:24:51.785543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168278+0000","last_change":"2024-05-14T09:31:08.388201+0000","last_active":"2024-05-14T09:31:30.168278+0000","last_peered":"2024-05-14T09:31:30.168278+0000","last_clean":"2024-05-14T09:31:30.168278+0000","last_became_active":"2024-05-14T09:31:08.387572+0000","last_became_peered":"2024-05-14T09:31:08.387572+0000","last_unstale":"2024-05-14T09:31:30.168278+0000","last_undegraded":"2024-05-14T09:31:30.168278+0000","last_fullsized":"2024-05-14T09:31:30.168278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:25:13.171350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.708354+0000","last_change":"2024-05-14T09:30:54.016524+0000","last_active":"2024-05-14T09:31:54.708354+0000","last_peered":"2024-05-14T09:31:54.708354+0000","last_clean":"2024-05-14T09:31:54.708354+0000","last_became_active":"2024-05-14T09:30:54.016154+0000","last_became_peered":"2024-05-14T09:30:54.016154+0000","last_unstale":"2024-05-14T09:31:54.708354+0000","last_undegraded":"2024-05-14T09:31:54.708354+0000","last_fullsized":"2024-05-14T09:31:54.708354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:43:16.671053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166563+0000","last_change":"2024-05-14T09:30:56.796748+0000","last_active":"2024-05-14T09:31:30.166563+0000","last_peered":"2024-05-14T09:31:30.166563+0000","last_clean":"2024-05-14T09:31:30.166563+0000","last_became_active":"2024-05-14T09:30:56.796354+0000","last_became_peered":"2024-05-14T09:30:56.796354+0000","last_unstale":"2024-05-14T09:31:30.166563+0000","last_undegraded":"2024-05-14T09:31:30.166563+0000","last_fullsized":"2024-05-14T09:31:30.166563+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:30:15.820677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.164298+0000","last_change":"2024-05-14T09:31:08.117266+0000","last_active":"2024-05-14T09:31:30.164298+0000","last_peered":"2024-05-14T09:31:30.164298+0000","last_clean":"2024-05-14T09:31:30.164298+0000","last_became_active":"2024-05-14T09:31:08.115393+0000","last_became_peered":"2024-05-14T09:31:08.115393+0000","last_unstale":"2024-05-14T09:31:30.164298+0000","last_undegraded":"2024-05-14T09:31:30.164298+0000","last_fullsized":"2024-05-14T09:31:30.164298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:29:26.348818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.709345+0000","last_change":"2024-05-14T09:30:53.772417+0000","last_active":"2024-05-14T09:31:54.709345+0000","last_peered":"2024-05-14T09:31:54.709345+0000","last_clean":"2024-05-14T09:31:54.709345+0000","last_became_active":"2024-05-14T09:30:53.772028+0000","last_became_peered":"2024-05-14T09:30:53.772028+0000","last_unstale":"2024-05-14T09:31:54.709345+0000","last_undegraded":"2024-05-14T09:31:54.709345+0000","last_fullsized":"2024-05-14T09:31:54.709345+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:16:16.603957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.164993+0000","last_change":"2024-05-14T09:30:56.804750+0000","last_active":"2024-05-14T09:31:30.164993+0000","last_peered":"2024-05-14T09:31:30.164993+0000","last_clean":"2024-05-14T09:31:30.164993+0000","last_became_active":"2024-05-14T09:30:56.803990+0000","last_became_peered":"2024-05-14T09:30:56.803990+0000","last_unstale":"2024-05-14T09:31:30.164993+0000","last_undegraded":"2024-05-14T09:31:30.164993+0000","last_fullsized":"2024-05-14T09:31:30.164993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:42:00.661033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168707+0000","last_change":"2024-05-14T09:31:08.116061+0000","last_active":"2024-05-14T09:31:30.168707+0000","last_peered":"2024-05-14T09:31:30.168707+0000","last_clean":"2024-05-14T09:31:30.168707+0000","last_became_active":"2024-05-14T09:31:08.115578+0000","last_became_peered":"2024-05-14T09:31:08.115578+0000","last_unstale":"2024-05-14T09:31:30.168707+0000","last_undegraded":"2024-05-14T09:31:30.168707+0000","last_fullsized":"2024-05-14T09:31:30.168707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:18.931035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.170206+0000","last_change":"2024-05-14T09:30:53.768988+0000","last_active":"2024-05-14T09:31:30.170206+0000","last_peered":"2024-05-14T09:31:30.170206+0000","last_clean":"2024-05-14T09:31:30.170206+0000","last_became_active":"2024-05-14T09:30:53.768529+0000","last_became_peered":"2024-05-14T09:30:53.768529+0000","last_unstale":"2024-05-14T09:31:30.170206+0000","last_undegraded":"2024-05-14T09:31:30.170206+0000","last_fullsized":"2024-05-14T09:31:30.170206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:17:02.381447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167877+0000","last_change":"2024-05-14T09:30:56.797110+0000","last_active":"2024-05-14T09:31:30.167877+0000","last_peered":"2024-05-14T09:31:30.167877+0000","last_clean":"2024-05-14T09:31:30.167877+0000","last_became_active":"2024-05-14T09:30:56.796015+0000","last_became_peered":"2024-05-14T09:30:56.796015+0000","last_unstale":"2024-05-14T09:31:30.167877+0000","last_undegraded":"2024-05-14T09:31:30.167877+0000","last_fullsized":"2024-05-14T09:31:30.167877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:37:44.674607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167730+0000","last_change":"2024-05-14T09:31:08.387240+0000","last_active":"2024-05-14T09:31:30.167730+0000","last_peered":"2024-05-14T09:31:30.167730+0000","last_clean":"2024-05-14T09:31:30.167730+0000","last_became_active":"2024-05-14T09:31:08.386653+0000","last_became_peered":"2024-05-14T09:31:08.386653+0000","last_unstale":"2024-05-14T09:31:30.167730+0000","last_undegraded":"2024-05-14T09:31:30.167730+0000","last_fullsized":"2024-05-14T09:31:30.167730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:18:43.408030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.710918+0000","last_change":"2024-05-14T09:30:53.772723+0000","last_active":"2024-05-14T09:31:54.710918+0000","last_peered":"2024-05-14T09:31:54.710918+0000","last_clean":"2024-05-14T09:31:54.710918+0000","last_became_active":"2024-05-14T09:30:53.772056+0000","last_became_peered":"2024-05-14T09:30:53.772056+0000","last_unstale":"2024-05-14T09:31:54.710918+0000","last_undegraded":"2024-05-14T09:31:54.710918+0000","last_fullsized":"2024-05-14T09:31:54.710918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:37.941282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.174622+0000","last_change":"2024-05-14T09:30:56.799223+0000","last_active":"2024-05-14T09:31:30.174622+0000","last_peered":"2024-05-14T09:31:30.174622+0000","last_clean":"2024-05-14T09:31:30.174622+0000","last_became_active":"2024-05-14T09:30:56.796092+0000","last_became_peered":"2024-05-14T09:30:56.796092+0000","last_unstale":"2024-05-14T09:31:30.174622+0000","last_undegraded":"2024-05-14T09:31:30.174622+0000","last_fullsized":"2024-05-14T09:31:30.174622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:43:49.746797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.172036+0000","last_change":"2024-05-14T09:31:08.115053+0000","last_active":"2024-05-14T09:31:30.172036+0000","last_peered":"2024-05-14T09:31:30.172036+0000","last_clean":"2024-05-14T09:31:30.172036+0000","last_became_active":"2024-05-14T09:31:08.114578+0000","last_became_peered":"2024-05-14T09:31:08.114578+0000","last_unstale":"2024-05-14T09:31:30.172036+0000","last_undegraded":"2024-05-14T09:31:30.172036+0000","last_fullsized":"2024-05-14T09:31:30.172036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:47.788962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.719248+0000","last_change":"2024-05-14T09:30:54.069055+0000","last_active":"2024-05-14T09:31:55.719248+0000","last_peered":"2024-05-14T09:31:55.719248+0000","last_clean":"2024-05-14T09:31:55.719248+0000","last_became_active":"2024-05-14T09:30:54.068479+0000","last_became_peered":"2024-05-14T09:30:54.068479+0000","last_unstale":"2024-05-14T09:31:55.719248+0000","last_undegraded":"2024-05-14T09:31:55.719248+0000","last_fullsized":"2024-05-14T09:31:55.719248+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:34.092472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.175965+0000","last_change":"2024-05-14T09:30:56.798580+0000","last_active":"2024-05-14T09:31:30.175965+0000","last_peered":"2024-05-14T09:31:30.175965+0000","last_clean":"2024-05-14T09:31:30.175965+0000","last_became_active":"2024-05-14T09:30:56.795617+0000","last_became_peered":"2024-05-14T09:30:56.795617+0000","last_unstale":"2024-05-14T09:31:30.175965+0000","last_undegraded":"2024-05-14T09:31:30.175965+0000","last_fullsized":"2024-05-14T09:31:30.175965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:30:35.913497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295529+0000","last_change":"2024-05-14T09:31:08.111375+0000","last_active":"2024-05-14T09:31:30.295529+0000","last_peered":"2024-05-14T09:31:30.295529+0000","last_clean":"2024-05-14T09:31:30.295529+0000","last_became_active":"2024-05-14T09:31:08.110931+0000","last_became_peered":"2024-05-14T09:31:08.110931+0000","last_unstale":"2024-05-14T09:31:30.295529+0000","last_undegraded":"2024-05-14T09:31:30.295529+0000","last_fullsized":"2024-05-14T09:31:30.295529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:08:30.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173185+0000","last_change":"2024-05-14T09:30:53.772385+0000","last_active":"2024-05-14T09:31:30.173185+0000","last_peered":"2024-05-14T09:31:30.173185+0000","last_clean":"2024-05-14T09:31:30.173185+0000","last_became_active":"2024-05-14T09:30:53.771928+0000","last_became_peered":"2024-05-14T09:30:53.771928+0000","last_unstale":"2024-05-14T09:31:30.173185+0000","last_undegraded":"2024-05-14T09:31:30.173185+0000","last_fullsized":"2024-05-14T09:31:30.173185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:06:21.357341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.176063+0000","last_change":"2024-05-14T09:30:56.789527+0000","last_active":"2024-05-14T09:31:30.176063+0000","last_peered":"2024-05-14T09:31:30.176063+0000","last_clean":"2024-05-14T09:31:30.176063+0000","last_became_active":"2024-05-14T09:30:56.789075+0000","last_became_peered":"2024-05-14T09:30:56.789075+0000","last_unstale":"2024-05-14T09:31:30.176063+0000","last_undegraded":"2024-05-14T09:31:30.176063+0000","last_fullsized":"2024-05-14T09:31:30.176063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:07.140087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.170686+0000","last_change":"2024-05-14T09:31:08.119624+0000","last_active":"2024-05-14T09:31:30.170686+0000","last_peered":"2024-05-14T09:31:30.170686+0000","last_clean":"2024-05-14T09:31:30.170686+0000","last_became_active":"2024-05-14T09:31:08.118102+0000","last_became_peered":"2024-05-14T09:31:08.118102+0000","last_unstale":"2024-05-14T09:31:30.170686+0000","last_undegraded":"2024-05-14T09:31:30.170686+0000","last_fullsized":"2024-05-14T09:31:30.170686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:48:17.806976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169513+0000","last_change":"2024-05-14T09:30:53.763903+0000","last_active":"2024-05-14T09:31:30.169513+0000","last_peered":"2024-05-14T09:31:30.169513+0000","last_clean":"2024-05-14T09:31:30.169513+0000","last_became_active":"2024-05-14T09:30:53.763517+0000","last_became_peered":"2024-05-14T09:30:53.763517+0000","last_unstale":"2024-05-14T09:31:30.169513+0000","last_undegraded":"2024-05-14T09:31:30.169513+0000","last_fullsized":"2024-05-14T09:31:30.169513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:57.144392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.176599+0000","last_change":"2024-05-14T09:30:56.791612+0000","last_active":"2024-05-14T09:31:30.176599+0000","last_peered":"2024-05-14T09:31:30.176599+0000","last_clean":"2024-05-14T09:31:30.176599+0000","last_became_active":"2024-05-14T09:30:56.791178+0000","last_became_peered":"2024-05-14T09:30:56.791178+0000","last_unstale":"2024-05-14T09:31:30.176599+0000","last_undegraded":"2024-05-14T09:31:30.176599+0000","last_fullsized":"2024-05-14T09:31:30.176599+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:11:19.058139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165093+0000","last_change":"2024-05-14T09:30:56.804437+0000","last_active":"2024-05-14T09:31:30.165093+0000","last_peered":"2024-05-14T09:31:30.165093+0000","last_clean":"2024-05-14T09:31:30.165093+0000","last_became_active":"2024-05-14T09:30:56.804048+0000","last_became_peered":"2024-05-14T09:30:56.804048+0000","last_unstale":"2024-05-14T09:31:30.165093+0000","last_undegraded":"2024-05-14T09:31:30.165093+0000","last_fullsized":"2024-05-14T09:31:30.165093+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:33.788903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.170173+0000","last_change":"2024-05-14T09:30:56.794309+0000","last_active":"2024-05-14T09:31:30.170173+0000","last_peered":"2024-05-14T09:31:30.170173+0000","last_clean":"2024-05-14T09:31:30.170173+0000","last_became_active":"2024-05-14T09:30:56.793317+0000","last_became_peered":"2024-05-14T09:30:56.793317+0000","last_unstale":"2024-05-14T09:31:30.170173+0000","last_undegraded":"2024-05-14T09:31:30.170173+0000","last_fullsized":"2024-05-14T09:31:30.170173+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:40:26.878746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169958+0000","last_change":"2024-05-14T09:30:56.793618+0000","last_active":"2024-05-14T09:31:30.169958+0000","last_peered":"2024-05-14T09:31:30.169958+0000","last_clean":"2024-05-14T09:31:30.169958+0000","last_became_active":"2024-05-14T09:30:56.792819+0000","last_became_peered":"2024-05-14T09:30:56.792819+0000","last_unstale":"2024-05-14T09:31:30.169958+0000","last_undegraded":"2024-05-14T09:31:30.169958+0000","last_fullsized":"2024-05-14T09:31:30.169958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:29:38.191617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295810+0000","last_change":"2024-05-14T09:30:56.794268+0000","last_active":"2024-05-14T09:31:30.295810+0000","last_peered":"2024-05-14T09:31:30.295810+0000","last_clean":"2024-05-14T09:31:30.295810+0000","last_became_active":"2024-05-14T09:30:56.793898+0000","last_became_peered":"2024-05-14T09:30:56.793898+0000","last_unstale":"2024-05-14T09:31:30.295810+0000","last_undegraded":"2024-05-14T09:31:30.295810+0000","last_fullsized":"2024-05-14T09:31:30.295810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:25:35.022236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169392+0000","last_change":"2024-05-14T09:30:56.783366+0000","last_active":"2024-05-14T09:31:30.169392+0000","last_peered":"2024-05-14T09:31:30.169392+0000","last_clean":"2024-05-14T09:31:30.169392+0000","last_became_active":"2024-05-14T09:30:56.782980+0000","last_became_peered":"2024-05-14T09:30:56.782980+0000","last_unstale":"2024-05-14T09:31:30.169392+0000","last_undegraded":"2024-05-14T09:31:30.169392+0000","last_fullsized":"2024-05-14T09:31:30.169392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:27.744902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165790+0000","last_change":"2024-05-14T09:30:56.795578+0000","last_active":"2024-05-14T09:31:30.165790+0000","last_peered":"2024-05-14T09:31:30.165790+0000","last_clean":"2024-05-14T09:31:30.165790+0000","last_became_active":"2024-05-14T09:30:56.795141+0000","last_became_peered":"2024-05-14T09:30:56.795141+0000","last_unstale":"2024-05-14T09:31:30.165790+0000","last_undegraded":"2024-05-14T09:31:30.165790+0000","last_fullsized":"2024-05-14T09:31:30.165790+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:05:25.012646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171520+0000","last_change":"2024-05-14T09:30:56.795872+0000","last_active":"2024-05-14T09:31:30.171520+0000","last_peered":"2024-05-14T09:31:30.171520+0000","last_clean":"2024-05-14T09:31:30.171520+0000","last_became_active":"2024-05-14T09:30:56.795492+0000","last_became_peered":"2024-05-14T09:30:56.795492+0000","last_unstale":"2024-05-14T09:31:30.171520+0000","last_undegraded":"2024-05-14T09:31:30.171520+0000","last_fullsized":"2024-05-14T09:31:30.171520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:26:29.226575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168024+0000","last_change":"2024-05-14T09:30:56.797078+0000","last_active":"2024-05-14T09:31:30.168024+0000","last_peered":"2024-05-14T09:31:30.168024+0000","last_clean":"2024-05-14T09:31:30.168024+0000","last_became_active":"2024-05-14T09:30:56.796703+0000","last_became_peered":"2024-05-14T09:30:56.796703+0000","last_unstale":"2024-05-14T09:31:30.168024+0000","last_undegraded":"2024-05-14T09:31:30.168024+0000","last_fullsized":"2024-05-14T09:31:30.168024+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:06:16.651877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.172918+0000","last_change":"2024-05-14T09:30:56.792814+0000","last_active":"2024-05-14T09:31:30.172918+0000","last_peered":"2024-05-14T09:31:30.172918+0000","last_clean":"2024-05-14T09:31:30.172918+0000","last_became_active":"2024-05-14T09:30:56.792282+0000","last_became_peered":"2024-05-14T09:30:56.792282+0000","last_unstale":"2024-05-14T09:31:30.172918+0000","last_undegraded":"2024-05-14T09:31:30.172918+0000","last_fullsized":"2024-05-14T09:31:30.172918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:52:35.319479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168151+0000","last_change":"2024-05-14T09:30:56.791575+0000","last_active":"2024-05-14T09:31:30.168151+0000","last_peered":"2024-05-14T09:31:30.168151+0000","last_clean":"2024-05-14T09:31:30.168151+0000","last_became_active":"2024-05-14T09:30:56.791230+0000","last_became_peered":"2024-05-14T09:30:56.791230+0000","last_unstale":"2024-05-14T09:31:30.168151+0000","last_undegraded":"2024-05-14T09:31:30.168151+0000","last_fullsized":"2024-05-14T09:31:30.168151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:40.510062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.172446+0000","last_change":"2024-05-14T09:30:56.796996+0000","last_active":"2024-05-14T09:31:30.172446+0000","last_peered":"2024-05-14T09:31:30.172446+0000","last_clean":"2024-05-14T09:31:30.172446+0000","last_became_active":"2024-05-14T09:30:56.796333+0000","last_became_peered":"2024-05-14T09:30:56.796333+0000","last_unstale":"2024-05-14T09:31:30.172446+0000","last_undegraded":"2024-05-14T09:31:30.172446+0000","last_fullsized":"2024-05-14T09:31:30.172446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:29:34.799088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169183+0000","last_change":"2024-05-14T09:30:56.785740+0000","last_active":"2024-05-14T09:31:30.169183+0000","last_peered":"2024-05-14T09:31:30.169183+0000","last_clean":"2024-05-14T09:31:30.169183+0000","last_became_active":"2024-05-14T09:30:56.785394+0000","last_became_peered":"2024-05-14T09:30:56.785394+0000","last_unstale":"2024-05-14T09:31:30.169183+0000","last_undegraded":"2024-05-14T09:31:30.169183+0000","last_fullsized":"2024-05-14T09:31:30.169183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:17.819523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165516+0000","last_change":"2024-05-14T09:30:56.796957+0000","last_active":"2024-05-14T09:31:30.165516+0000","last_peered":"2024-05-14T09:31:30.165516+0000","last_clean":"2024-05-14T09:31:30.165516+0000","last_became_active":"2024-05-14T09:30:56.795071+0000","last_became_peered":"2024-05-14T09:30:56.795071+0000","last_unstale":"2024-05-14T09:31:30.165516+0000","last_undegraded":"2024-05-14T09:31:30.165516+0000","last_fullsized":"2024-05-14T09:31:30.165516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:44:31.480207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245291,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62658,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":176,"num_read_kb":155,"num_write":165,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":2194,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":187974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678047,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":499802,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874277,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":1149241,"data_compressed":62023,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037803,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":499456,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1597,"internal_metadata":36174275},"hb_peers":[0,1,2,3,4,5,6,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234031,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":498794,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430261,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":1094703,"data_compressed":62023,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593786,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":495214,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":553199,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":496020,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":496283,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":494826,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953541,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":557939,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790016,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1090873,"data_compressed":62023,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33883,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33883,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":59301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T09:32:25.586 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T09:32:25.841 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:26.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:26 smithi002 ceph-mon[31557]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:32:26.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:26 smithi002 ceph-mon[31557]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-05-14T09:32:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:26 smithi029 ceph-mon[36249]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:32:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:26 smithi029 ceph-mon[36249]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-05-14T09:32:26.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:26 smithi026 ceph-mon[36056]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:32:26.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:26 smithi026 ceph-mon[36056]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-05-14T09:32:27.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.685+0000 7ff8c022e700 1 -- 172.21.15.2:0/3523636679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b8109850 msgr2=0x7ff8b81111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:27.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.685+0000 7ff8c022e700 1 --2- 172.21.15.2:0/3523636679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b8109850 0x7ff8b81111b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000b0d0 tx=0x7ff8b000b3e0 comp rx=0 tx=0).stop 2024-05-14T09:32:27.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.685+0000 7ff8c022e700 1 -- 172.21.15.2:0/3523636679 shutdown_connections 2024-05-14T09:32:27.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.685+0000 7ff8c022e700 1 --2- 172.21.15.2:0/3523636679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b8109850 0x7ff8b81111b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:27.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.685+0000 7ff8c022e700 1 --2- 172.21.15.2:0/3523636679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8b81045d0 0x7ff8b81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:27.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.685+0000 7ff8c022e700 1 --2- 172.21.15.2:0/3523636679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b8108f00 0x7ff8b8109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:27.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.685+0000 7ff8c022e700 1 -- 172.21.15.2:0/3523636679 >> 172.21.15.2:0/3523636679 conn(0x7ff8b80ffea0 msgr2=0x7ff8b81022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:27.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.685+0000 7ff8c022e700 1 -- 172.21.15.2:0/3523636679 shutdown_connections 2024-05-14T09:32:27.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.685+0000 7ff8c022e700 1 -- 172.21.15.2:0/3523636679 wait complete. 2024-05-14T09:32:27.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.686+0000 7ff8c022e700 1 Processor -- start 2024-05-14T09:32:27.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.686+0000 7ff8c022e700 1 -- start start 2024-05-14T09:32:27.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.686+0000 7ff8c022e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b81045d0 0x7ff8b819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:27.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.686+0000 7ff8c022e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b8108f00 0x7ff8b819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:27.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.686+0000 7ff8c022e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8b8109850 0x7ff8b819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:27.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.686+0000 7ff8c022e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b81144d0 con 0x7ff8b8108f00 2024-05-14T09:32:27.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.686+0000 7ff8c022e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b8114350 con 0x7ff8b81045d0 2024-05-14T09:32:27.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.686+0000 7ff8c022e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b8114650 con 0x7ff8b8109850 2024-05-14T09:32:27.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.686+0000 7ff8bd7c9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b8108f00 0x7ff8b819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:27.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.686+0000 7ff8bd7c9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b8108f00 0x7ff8b819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47708/0 (socket says 172.21.15.2:47708) 2024-05-14T09:32:27.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.686+0000 7ff8bd7c9700 1 -- 172.21.15.2:0/3416098255 learned_addr learned my addr 172.21.15.2:0/3416098255 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:27.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.687+0000 7ff8be7cb700 1 --2- 172.21.15.2:0/3416098255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8b8109850 0x7ff8b819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:27.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.687+0000 7ff8bdfca700 1 --2- 172.21.15.2:0/3416098255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b81045d0 0x7ff8b819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:27.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.687+0000 7ff8bd7c9700 1 -- 172.21.15.2:0/3416098255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8b8109850 msgr2=0x7ff8b819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:27.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.687+0000 7ff8bd7c9700 1 --2- 172.21.15.2:0/3416098255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8b8109850 0x7ff8b819f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:27.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.687+0000 7ff8bd7c9700 1 -- 172.21.15.2:0/3416098255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b81045d0 msgr2=0x7ff8b819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:27.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.687+0000 7ff8bd7c9700 1 --2- 172.21.15.2:0/3416098255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b81045d0 0x7ff8b819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:27.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.687+0000 7ff8bd7c9700 1 -- 172.21.15.2:0/3416098255 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b819fc00 con 0x7ff8b8108f00 2024-05-14T09:32:27.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.687+0000 7ff8bdfca700 1 --2- 172.21.15.2:0/3416098255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b81045d0 0x7ff8b819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:32:27.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.687+0000 7ff8bd7c9700 1 --2- 172.21.15.2:0/3416098255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b8108f00 0x7ff8b819b380 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800c9e0 tx=0x7ff8a800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:27.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.687+0000 7ff8aeffd700 1 -- 172.21.15.2:0/3416098255 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8007aa0 con 0x7ff8b8108f00 2024-05-14T09:32:27.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.687+0000 7ff8aeffd700 1 -- 172.21.15.2:0/3416098255 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a8007c40 con 0x7ff8b8108f00 2024-05-14T09:32:27.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.687+0000 7ff8c022e700 1 -- 172.21.15.2:0/3416098255 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b819fef0 con 0x7ff8b8108f00 2024-05-14T09:32:27.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.687+0000 7ff8aeffd700 1 -- 172.21.15.2:0/3416098255 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8018660 con 0x7ff8b8108f00 2024-05-14T09:32:27.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.687+0000 7ff8c022e700 1 -- 172.21.15.2:0/3416098255 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b81a0430 con 0x7ff8b8108f00 2024-05-14T09:32:27.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.689+0000 7ff8c022e700 1 -- 172.21.15.2:0/3416098255 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b80512c0 con 0x7ff8b8108f00 2024-05-14T09:32:27.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.690+0000 7ff8aeffd700 1 -- 172.21.15.2:0/3416098255 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7ff8a8018800 con 0x7ff8b8108f00 2024-05-14T09:32:27.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.691+0000 7ff8aeffd700 1 --2- 172.21.15.2:0/3416098255 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff8a40b2910 0x7ff8a40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:27.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.691+0000 7ff8bdfca700 1 --2- 172.21.15.2:0/3416098255 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff8a40b2910 0x7ff8a40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:27.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.691+0000 7ff8aeffd700 1 -- 172.21.15.2:0/3416098255 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7ff8a80e2680 con 0x7ff8b8108f00 2024-05-14T09:32:27.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.691+0000 7ff8bdfca700 1 --2- 172.21.15.2:0/3416098255 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff8a40b2910 0x7ff8a40b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4005950 tx=0x7ff8b4005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:27.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.695+0000 7ff8aeffd700 1 -- 172.21.15.2:0/3416098255 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff8a8014030 con 0x7ff8b8108f00 2024-05-14T09:32:27.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.911+0000 7ff8c022e700 1 -- 172.21.15.2:0/3416098255 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8b8063c00 con 0x7ff8a40b2910 2024-05-14T09:32:27.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.923+0000 7ff8aeffd700 1 -- 172.21.15.2:0/3416098255 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340610 (secure 0 0 0) 0x7ff8b8063c00 con 0x7ff8a40b2910 2024-05-14T09:32:27.929 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:32:27.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.926+0000 7ff8c022e700 1 -- 172.21.15.2:0/3416098255 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff8a40b2910 msgr2=0x7ff8a40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:27.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.926+0000 7ff8c022e700 1 --2- 172.21.15.2:0/3416098255 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff8a40b2910 0x7ff8a40b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4005950 tx=0x7ff8b4005f50 comp rx=0 tx=0).stop 2024-05-14T09:32:27.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.926+0000 7ff8c022e700 1 -- 172.21.15.2:0/3416098255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b8108f00 msgr2=0x7ff8b819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:27.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.926+0000 7ff8c022e700 1 --2- 172.21.15.2:0/3416098255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b8108f00 0x7ff8b819b380 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800c9e0 tx=0x7ff8a800ccf0 comp rx=0 tx=0).stop 2024-05-14T09:32:27.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.926+0000 7ff8c022e700 1 -- 172.21.15.2:0/3416098255 shutdown_connections 2024-05-14T09:32:27.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.926+0000 7ff8c022e700 1 --2- 172.21.15.2:0/3416098255 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff8a40b2910 0x7ff8a40b4e20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:27.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.926+0000 7ff8c022e700 1 --2- 172.21.15.2:0/3416098255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8b81045d0 0x7ff8b819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:27.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.926+0000 7ff8c022e700 1 --2- 172.21.15.2:0/3416098255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff8b8109850 0x7ff8b819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:27.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.926+0000 7ff8c022e700 1 --2- 172.21.15.2:0/3416098255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b8108f00 0x7ff8b819b380 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:27.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.926+0000 7ff8c022e700 1 -- 172.21.15.2:0/3416098255 >> 172.21.15.2:0/3416098255 conn(0x7ff8b80ffea0 msgr2=0x7ff8b8108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:27.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.926+0000 7ff8c022e700 1 -- 172.21.15.2:0/3416098255 shutdown_connections 2024-05-14T09:32:27.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:27.926+0000 7ff8c022e700 1 -- 172.21.15.2:0/3416098255 wait complete. 2024-05-14T09:32:27.932 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-05-14T09:32:28.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:28 smithi002 ceph-mon[31557]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-14T09:32:28.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:28 smithi029 ceph-mon[36249]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-14T09:32:28.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:28 smithi026 ceph-mon[36056]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-14T09:32:28.722 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-05-14T09:32:27.062163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653260,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":257,"num_read_kb":219,"num_write":290,"num_write_kb":2262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2954034,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":394208,"kb_used_data":11776,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124547616,"statfs":{"total":1151940427776,"available":1151536758784,"internally_reserved":0,"allocated":12058624,"data_stored":7937156,"data_compressed":424731,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19140,"internal_metadata":391492924},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29567,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":68,"num_read_kb":60,"num_write":21,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003476"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166300+0000","last_change":"2024-05-14T09:30:56.797051+0000","last_active":"2024-05-14T09:31:30.166300+0000","last_peered":"2024-05-14T09:31:30.166300+0000","last_clean":"2024-05-14T09:31:30.166300+0000","last_became_active":"2024-05-14T09:30:56.796517+0000","last_became_peered":"2024-05-14T09:30:56.796517+0000","last_unstale":"2024-05-14T09:31:30.166300+0000","last_undegraded":"2024-05-14T09:31:30.166300+0000","last_fullsized":"2024-05-14T09:31:30.166300+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:58:22.170187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295855+0000","last_change":"2024-05-14T09:30:56.795325+0000","last_active":"2024-05-14T09:31:30.295855+0000","last_peered":"2024-05-14T09:31:30.295855+0000","last_clean":"2024-05-14T09:31:30.295855+0000","last_became_active":"2024-05-14T09:30:56.794962+0000","last_became_peered":"2024-05-14T09:30:56.794962+0000","last_unstale":"2024-05-14T09:31:30.295855+0000","last_undegraded":"2024-05-14T09:31:30.295855+0000","last_fullsized":"2024-05-14T09:31:30.295855+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:15:28.890565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295685+0000","last_change":"2024-05-14T09:30:56.793200+0000","last_active":"2024-05-14T09:31:30.295685+0000","last_peered":"2024-05-14T09:31:30.295685+0000","last_clean":"2024-05-14T09:31:30.295685+0000","last_became_active":"2024-05-14T09:30:56.792835+0000","last_became_peered":"2024-05-14T09:30:56.792835+0000","last_unstale":"2024-05-14T09:31:30.295685+0000","last_undegraded":"2024-05-14T09:31:30.295685+0000","last_fullsized":"2024-05-14T09:31:30.295685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:19.851276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168183+0000","last_change":"2024-05-14T09:30:56.790238+0000","last_active":"2024-05-14T09:31:30.168183+0000","last_peered":"2024-05-14T09:31:30.168183+0000","last_clean":"2024-05-14T09:31:30.168183+0000","last_became_active":"2024-05-14T09:30:56.789826+0000","last_became_peered":"2024-05-14T09:30:56.789826+0000","last_unstale":"2024-05-14T09:31:30.168183+0000","last_undegraded":"2024-05-14T09:31:30.168183+0000","last_fullsized":"2024-05-14T09:31:30.168183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:19:19.853931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166364+0000","last_change":"2024-05-14T09:30:56.788021+0000","last_active":"2024-05-14T09:31:30.166364+0000","last_peered":"2024-05-14T09:31:30.166364+0000","last_clean":"2024-05-14T09:31:30.166364+0000","last_became_active":"2024-05-14T09:30:56.787657+0000","last_became_peered":"2024-05-14T09:30:56.787657+0000","last_unstale":"2024-05-14T09:31:30.166364+0000","last_undegraded":"2024-05-14T09:31:30.166364+0000","last_fullsized":"2024-05-14T09:31:30.166364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:37:03.232137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165471+0000","last_change":"2024-05-14T09:30:56.795985+0000","last_active":"2024-05-14T09:31:30.165471+0000","last_peered":"2024-05-14T09:31:30.165471+0000","last_clean":"2024-05-14T09:31:30.165471+0000","last_became_active":"2024-05-14T09:30:56.795575+0000","last_became_peered":"2024-05-14T09:30:56.795575+0000","last_unstale":"2024-05-14T09:31:30.165471+0000","last_undegraded":"2024-05-14T09:31:30.165471+0000","last_fullsized":"2024-05-14T09:31:30.165471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:42.923901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.175394+0000","last_change":"2024-05-14T09:30:56.795647+0000","last_active":"2024-05-14T09:31:30.175394+0000","last_peered":"2024-05-14T09:31:30.175394+0000","last_clean":"2024-05-14T09:31:30.175394+0000","last_became_active":"2024-05-14T09:30:56.795186+0000","last_became_peered":"2024-05-14T09:30:56.795186+0000","last_unstale":"2024-05-14T09:31:30.175394+0000","last_undegraded":"2024-05-14T09:31:30.175394+0000","last_fullsized":"2024-05-14T09:31:30.175394+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:04:53.471305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167059+0000","last_change":"2024-05-14T09:30:56.796834+0000","last_active":"2024-05-14T09:31:30.167059+0000","last_peered":"2024-05-14T09:31:30.167059+0000","last_clean":"2024-05-14T09:31:30.167059+0000","last_became_active":"2024-05-14T09:30:56.796492+0000","last_became_peered":"2024-05-14T09:30:56.796492+0000","last_unstale":"2024-05-14T09:31:30.167059+0000","last_undegraded":"2024-05-14T09:31:30.167059+0000","last_fullsized":"2024-05-14T09:31:30.167059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:46.606129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.172227+0000","last_change":"2024-05-14T09:30:56.795020+0000","last_active":"2024-05-14T09:31:30.172227+0000","last_peered":"2024-05-14T09:31:30.172227+0000","last_clean":"2024-05-14T09:31:30.172227+0000","last_became_active":"2024-05-14T09:30:56.794625+0000","last_became_peered":"2024-05-14T09:30:56.794625+0000","last_unstale":"2024-05-14T09:31:30.172227+0000","last_undegraded":"2024-05-14T09:31:30.172227+0000","last_fullsized":"2024-05-14T09:31:30.172227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:09:38.597296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166641+0000","last_change":"2024-05-14T09:30:56.796439+0000","last_active":"2024-05-14T09:31:30.166641+0000","last_peered":"2024-05-14T09:31:30.166641+0000","last_clean":"2024-05-14T09:31:30.166641+0000","last_became_active":"2024-05-14T09:30:56.795168+0000","last_became_peered":"2024-05-14T09:30:56.795168+0000","last_unstale":"2024-05-14T09:31:30.166641+0000","last_undegraded":"2024-05-14T09:31:30.166641+0000","last_fullsized":"2024-05-14T09:31:30.166641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:47.660286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.175736+0000","last_change":"2024-05-14T09:30:56.795994+0000","last_active":"2024-05-14T09:31:30.175736+0000","last_peered":"2024-05-14T09:31:30.175736+0000","last_clean":"2024-05-14T09:31:30.175736+0000","last_became_active":"2024-05-14T09:30:56.795079+0000","last_became_peered":"2024-05-14T09:30:56.795079+0000","last_unstale":"2024-05-14T09:31:30.175736+0000","last_undegraded":"2024-05-14T09:31:30.175736+0000","last_fullsized":"2024-05-14T09:31:30.175736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:37:42.930195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168036+0000","last_change":"2024-05-14T09:30:56.792997+0000","last_active":"2024-05-14T09:31:30.168036+0000","last_peered":"2024-05-14T09:31:30.168036+0000","last_clean":"2024-05-14T09:31:30.168036+0000","last_became_active":"2024-05-14T09:30:56.792302+0000","last_became_peered":"2024-05-14T09:30:56.792302+0000","last_unstale":"2024-05-14T09:31:30.168036+0000","last_undegraded":"2024-05-14T09:31:30.168036+0000","last_fullsized":"2024-05-14T09:31:30.168036+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:52.986662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169028+0000","last_change":"2024-05-14T09:30:56.795903+0000","last_active":"2024-05-14T09:31:30.169028+0000","last_peered":"2024-05-14T09:31:30.169028+0000","last_clean":"2024-05-14T09:31:30.169028+0000","last_became_active":"2024-05-14T09:30:56.795466+0000","last_became_peered":"2024-05-14T09:30:56.795466+0000","last_unstale":"2024-05-14T09:31:30.169028+0000","last_undegraded":"2024-05-14T09:31:30.169028+0000","last_fullsized":"2024-05-14T09:31:30.169028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:02:10.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295513+0000","last_change":"2024-05-14T09:30:56.795911+0000","last_active":"2024-05-14T09:31:30.295513+0000","last_peered":"2024-05-14T09:31:30.295513+0000","last_clean":"2024-05-14T09:31:30.295513+0000","last_became_active":"2024-05-14T09:30:56.795547+0000","last_became_peered":"2024-05-14T09:30:56.795547+0000","last_unstale":"2024-05-14T09:31:30.295513+0000","last_undegraded":"2024-05-14T09:31:30.295513+0000","last_fullsized":"2024-05-14T09:31:30.295513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:56:29.621783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295455+0000","last_change":"2024-05-14T09:30:56.787745+0000","last_active":"2024-05-14T09:31:30.295455+0000","last_peered":"2024-05-14T09:31:30.295455+0000","last_clean":"2024-05-14T09:31:30.295455+0000","last_became_active":"2024-05-14T09:30:56.787385+0000","last_became_peered":"2024-05-14T09:30:56.787385+0000","last_unstale":"2024-05-14T09:31:30.295455+0000","last_undegraded":"2024-05-14T09:31:30.295455+0000","last_fullsized":"2024-05-14T09:31:30.295455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:11.256775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171640+0000","last_change":"2024-05-14T09:30:56.795785+0000","last_active":"2024-05-14T09:31:30.171640+0000","last_peered":"2024-05-14T09:31:30.171640+0000","last_clean":"2024-05-14T09:31:30.171640+0000","last_became_active":"2024-05-14T09:30:56.795436+0000","last_became_peered":"2024-05-14T09:30:56.795436+0000","last_unstale":"2024-05-14T09:31:30.171640+0000","last_undegraded":"2024-05-14T09:31:30.171640+0000","last_fullsized":"2024-05-14T09:31:30.171640+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:12:09.221929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166125+0000","last_change":"2024-05-14T09:30:56.800688+0000","last_active":"2024-05-14T09:31:30.166125+0000","last_peered":"2024-05-14T09:31:30.166125+0000","last_clean":"2024-05-14T09:31:30.166125+0000","last_became_active":"2024-05-14T09:30:56.800204+0000","last_became_peered":"2024-05-14T09:30:56.800204+0000","last_unstale":"2024-05-14T09:31:30.166125+0000","last_undegraded":"2024-05-14T09:31:30.166125+0000","last_fullsized":"2024-05-14T09:31:30.166125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:32:27.057688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.164256+0000","last_change":"2024-05-14T09:30:56.795604+0000","last_active":"2024-05-14T09:31:30.164256+0000","last_peered":"2024-05-14T09:31:30.164256+0000","last_clean":"2024-05-14T09:31:30.164256+0000","last_became_active":"2024-05-14T09:30:56.795245+0000","last_became_peered":"2024-05-14T09:30:56.795245+0000","last_unstale":"2024-05-14T09:31:30.164256+0000","last_undegraded":"2024-05-14T09:31:30.164256+0000","last_fullsized":"2024-05-14T09:31:30.164256+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:14.072584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171084+0000","last_change":"2024-05-14T09:30:56.794815+0000","last_active":"2024-05-14T09:31:30.171084+0000","last_peered":"2024-05-14T09:31:30.171084+0000","last_clean":"2024-05-14T09:31:30.171084+0000","last_became_active":"2024-05-14T09:30:56.794429+0000","last_became_peered":"2024-05-14T09:30:56.794429+0000","last_unstale":"2024-05-14T09:31:30.171084+0000","last_undegraded":"2024-05-14T09:31:30.171084+0000","last_fullsized":"2024-05-14T09:31:30.171084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:32:08.367046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.170256+0000","last_change":"2024-05-14T09:31:08.107895+0000","last_active":"2024-05-14T09:31:30.170256+0000","last_peered":"2024-05-14T09:31:30.170256+0000","last_clean":"2024-05-14T09:31:30.170256+0000","last_became_active":"2024-05-14T09:31:08.107458+0000","last_became_peered":"2024-05-14T09:31:08.107458+0000","last_unstale":"2024-05-14T09:31:30.170256+0000","last_undegraded":"2024-05-14T09:31:30.170256+0000","last_fullsized":"2024-05-14T09:31:30.170256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:13:11.717753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":64,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.720901+0000","last_change":"2024-05-14T09:31:30.178197+0000","last_active":"2024-05-14T09:31:55.720901+0000","last_peered":"2024-05-14T09:31:55.720901+0000","last_clean":"2024-05-14T09:31:55.720901+0000","last_became_active":"2024-05-14T09:31:30.171154+0000","last_became_peered":"2024-05-14T09:31:30.171154+0000","last_unstale":"2024-05-14T09:31:55.720901+0000","last_undegraded":"2024-05-14T09:31:55.720901+0000","last_fullsized":"2024-05-14T09:31:55.720901+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:43:46.208439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167609+0000","last_change":"2024-05-14T09:30:56.800526+0000","last_active":"2024-05-14T09:31:30.167609+0000","last_peered":"2024-05-14T09:31:30.167609+0000","last_clean":"2024-05-14T09:31:30.167609+0000","last_became_active":"2024-05-14T09:30:56.797692+0000","last_became_peered":"2024-05-14T09:30:56.797692+0000","last_unstale":"2024-05-14T09:31:30.167609+0000","last_undegraded":"2024-05-14T09:31:30.167609+0000","last_fullsized":"2024-05-14T09:31:30.167609+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:09:14.471173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173810+0000","last_change":"2024-05-14T09:31:30.173810+0000","last_active":"2024-05-14T09:31:30.173810+0000","last_peered":"2024-05-14T09:31:30.173810+0000","last_clean":"2024-05-14T09:31:30.173810+0000","last_became_active":"2024-05-14T09:31:30.173242+0000","last_became_peered":"2024-05-14T09:31:30.173242+0000","last_unstale":"2024-05-14T09:31:30.173810+0000","last_undegraded":"2024-05-14T09:31:30.173810+0000","last_fullsized":"2024-05-14T09:31:30.173810+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:37:15.991805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168122+0000","last_change":"2024-05-14T09:30:54.015636+0000","last_active":"2024-05-14T09:31:30.168122+0000","last_peered":"2024-05-14T09:31:30.168122+0000","last_clean":"2024-05-14T09:31:30.168122+0000","last_became_active":"2024-05-14T09:30:54.015290+0000","last_became_peered":"2024-05-14T09:30:54.015290+0000","last_unstale":"2024-05-14T09:31:30.168122+0000","last_undegraded":"2024-05-14T09:31:30.168122+0000","last_fullsized":"2024-05-14T09:31:30.168122+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:29:17.548817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295746+0000","last_change":"2024-05-14T09:30:56.799506+0000","last_active":"2024-05-14T09:31:30.295746+0000","last_peered":"2024-05-14T09:31:30.295746+0000","last_clean":"2024-05-14T09:31:30.295746+0000","last_became_active":"2024-05-14T09:30:56.799153+0000","last_became_peered":"2024-05-14T09:30:56.799153+0000","last_unstale":"2024-05-14T09:31:30.295746+0000","last_undegraded":"2024-05-14T09:31:30.295746+0000","last_fullsized":"2024-05-14T09:31:30.295746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:04:08.217337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168111+0000","last_change":"2024-05-14T09:31:08.115533+0000","last_active":"2024-05-14T09:31:30.168111+0000","last_peered":"2024-05-14T09:31:30.168111+0000","last_clean":"2024-05-14T09:31:30.168111+0000","last_became_active":"2024-05-14T09:31:08.115033+0000","last_became_peered":"2024-05-14T09:31:08.115033+0000","last_unstale":"2024-05-14T09:31:30.168111+0000","last_undegraded":"2024-05-14T09:31:30.168111+0000","last_fullsized":"2024-05-14T09:31:30.168111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:15.668357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.719855+0000","last_change":"2024-05-14T09:30:53.770547+0000","last_active":"2024-05-14T09:31:55.719855+0000","last_peered":"2024-05-14T09:31:55.719855+0000","last_clean":"2024-05-14T09:31:55.719855+0000","last_became_active":"2024-05-14T09:30:53.770110+0000","last_became_peered":"2024-05-14T09:30:53.770110+0000","last_unstale":"2024-05-14T09:31:55.719855+0000","last_undegraded":"2024-05-14T09:31:55.719855+0000","last_fullsized":"2024-05-14T09:31:55.719855+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:18.371318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171215+0000","last_change":"2024-05-14T09:30:56.799427+0000","last_active":"2024-05-14T09:31:30.171215+0000","last_peered":"2024-05-14T09:31:30.171215+0000","last_clean":"2024-05-14T09:31:30.171215+0000","last_became_active":"2024-05-14T09:30:56.799018+0000","last_became_peered":"2024-05-14T09:30:56.799018+0000","last_unstale":"2024-05-14T09:31:30.171215+0000","last_undegraded":"2024-05-14T09:31:30.171215+0000","last_fullsized":"2024-05-14T09:31:30.171215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:13:31.837985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167589+0000","last_change":"2024-05-14T09:31:08.116258+0000","last_active":"2024-05-14T09:31:30.167589+0000","last_peered":"2024-05-14T09:31:30.167589+0000","last_clean":"2024-05-14T09:31:30.167589+0000","last_became_active":"2024-05-14T09:31:08.115040+0000","last_became_peered":"2024-05-14T09:31:08.115040+0000","last_unstale":"2024-05-14T09:31:30.167589+0000","last_undegraded":"2024-05-14T09:31:30.167589+0000","last_fullsized":"2024-05-14T09:31:30.167589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:57:54.839203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.188984+0000","last_change":"2024-05-14T09:31:30.188984+0000","last_active":"2024-05-14T09:31:30.188984+0000","last_peered":"2024-05-14T09:31:30.188984+0000","last_clean":"2024-05-14T09:31:30.188984+0000","last_became_active":"2024-05-14T09:31:30.174410+0000","last_became_peered":"2024-05-14T09:31:30.174410+0000","last_unstale":"2024-05-14T09:31:30.188984+0000","last_undegraded":"2024-05-14T09:31:30.188984+0000","last_fullsized":"2024-05-14T09:31:30.188984+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:01:35.713857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,4],"acting":[8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173271+0000","last_change":"2024-05-14T09:30:56.799165+0000","last_active":"2024-05-14T09:31:30.173271+0000","last_peered":"2024-05-14T09:31:30.173271+0000","last_clean":"2024-05-14T09:31:30.173271+0000","last_became_active":"2024-05-14T09:30:56.798750+0000","last_became_peered":"2024-05-14T09:30:56.798750+0000","last_unstale":"2024-05-14T09:31:30.173271+0000","last_undegraded":"2024-05-14T09:31:30.173271+0000","last_fullsized":"2024-05-14T09:31:30.173271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:51:28.479007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.170804+0000","last_change":"2024-05-14T09:31:08.115740+0000","last_active":"2024-05-14T09:31:30.170804+0000","last_peered":"2024-05-14T09:31:30.170804+0000","last_clean":"2024-05-14T09:31:30.170804+0000","last_became_active":"2024-05-14T09:31:08.115202+0000","last_became_peered":"2024-05-14T09:31:08.115202+0000","last_unstale":"2024-05-14T09:31:30.170804+0000","last_undegraded":"2024-05-14T09:31:30.170804+0000","last_fullsized":"2024-05-14T09:31:30.170804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:44.519280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":74,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:32:21.279798+0000","last_change":"2024-05-14T09:30:53.770869+0000","last_active":"2024-05-14T09:32:21.279798+0000","last_peered":"2024-05-14T09:32:21.279798+0000","last_clean":"2024-05-14T09:32:21.279798+0000","last_became_active":"2024-05-14T09:30:53.770425+0000","last_became_peered":"2024-05-14T09:30:53.770425+0000","last_unstale":"2024-05-14T09:32:21.279798+0000","last_undegraded":"2024-05-14T09:32:21.279798+0000","last_fullsized":"2024-05-14T09:32:21.279798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:03:33.768120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166858+0000","last_change":"2024-05-14T09:30:56.789267+0000","last_active":"2024-05-14T09:31:30.166858+0000","last_peered":"2024-05-14T09:31:30.166858+0000","last_clean":"2024-05-14T09:31:30.166858+0000","last_became_active":"2024-05-14T09:30:56.788883+0000","last_became_peered":"2024-05-14T09:30:56.788883+0000","last_unstale":"2024-05-14T09:31:30.166858+0000","last_undegraded":"2024-05-14T09:31:30.166858+0000","last_fullsized":"2024-05-14T09:31:30.166858+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:09.818693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.170314+0000","last_change":"2024-05-14T09:31:08.115193+0000","last_active":"2024-05-14T09:31:30.170314+0000","last_peered":"2024-05-14T09:31:30.170314+0000","last_clean":"2024-05-14T09:31:30.170314+0000","last_became_active":"2024-05-14T09:31:08.114701+0000","last_became_peered":"2024-05-14T09:31:08.114701+0000","last_unstale":"2024-05-14T09:31:30.170314+0000","last_undegraded":"2024-05-14T09:31:30.170314+0000","last_fullsized":"2024-05-14T09:31:30.170314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:47:48.487922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.720597+0000","last_change":"2024-05-14T09:31:30.174760+0000","last_active":"2024-05-14T09:31:55.720597+0000","last_peered":"2024-05-14T09:31:55.720597+0000","last_clean":"2024-05-14T09:31:55.720597+0000","last_became_active":"2024-05-14T09:31:30.174253+0000","last_became_peered":"2024-05-14T09:31:30.174253+0000","last_unstale":"2024-05-14T09:31:55.720597+0000","last_undegraded":"2024-05-14T09:31:55.720597+0000","last_fullsized":"2024-05-14T09:31:55.720597+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:11:17.163436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173142+0000","last_change":"2024-05-14T09:30:56.794654+0000","last_active":"2024-05-14T09:31:30.173142+0000","last_peered":"2024-05-14T09:31:30.173142+0000","last_clean":"2024-05-14T09:31:30.173142+0000","last_became_active":"2024-05-14T09:30:56.794282+0000","last_became_peered":"2024-05-14T09:30:56.794282+0000","last_unstale":"2024-05-14T09:31:30.173142+0000","last_undegraded":"2024-05-14T09:31:30.173142+0000","last_fullsized":"2024-05-14T09:31:30.173142+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:55:41.478707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165960+0000","last_change":"2024-05-14T09:31:08.119965+0000","last_active":"2024-05-14T09:31:30.165960+0000","last_peered":"2024-05-14T09:31:30.165960+0000","last_clean":"2024-05-14T09:31:30.165960+0000","last_became_active":"2024-05-14T09:31:08.119502+0000","last_became_peered":"2024-05-14T09:31:08.119502+0000","last_unstale":"2024-05-14T09:31:30.165960+0000","last_undegraded":"2024-05-14T09:31:30.165960+0000","last_fullsized":"2024-05-14T09:31:30.165960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:01:23.612036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.720018+0000","last_change":"2024-05-14T09:30:53.766495+0000","last_active":"2024-05-14T09:31:55.720018+0000","last_peered":"2024-05-14T09:31:55.720018+0000","last_clean":"2024-05-14T09:31:55.720018+0000","last_became_active":"2024-05-14T09:30:53.766122+0000","last_became_peered":"2024-05-14T09:30:53.766122+0000","last_unstale":"2024-05-14T09:31:55.720018+0000","last_undegraded":"2024-05-14T09:31:55.720018+0000","last_fullsized":"2024-05-14T09:31:55.720018+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:43:13.344209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167185+0000","last_change":"2024-05-14T09:30:56.791938+0000","last_active":"2024-05-14T09:31:30.167185+0000","last_peered":"2024-05-14T09:31:30.167185+0000","last_clean":"2024-05-14T09:31:30.167185+0000","last_became_active":"2024-05-14T09:30:56.791525+0000","last_became_peered":"2024-05-14T09:30:56.791525+0000","last_unstale":"2024-05-14T09:31:30.167185+0000","last_undegraded":"2024-05-14T09:31:30.167185+0000","last_fullsized":"2024-05-14T09:31:30.167185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:18.178208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295717+0000","last_change":"2024-05-14T09:31:08.113548+0000","last_active":"2024-05-14T09:31:30.295717+0000","last_peered":"2024-05-14T09:31:30.295717+0000","last_clean":"2024-05-14T09:31:30.295717+0000","last_became_active":"2024-05-14T09:31:08.113117+0000","last_became_peered":"2024-05-14T09:31:08.113117+0000","last_unstale":"2024-05-14T09:31:30.295717+0000","last_undegraded":"2024-05-14T09:31:30.295717+0000","last_fullsized":"2024-05-14T09:31:30.295717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:25:15.483761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171131+0000","last_change":"2024-05-14T09:30:53.770987+0000","last_active":"2024-05-14T09:31:30.171131+0000","last_peered":"2024-05-14T09:31:30.171131+0000","last_clean":"2024-05-14T09:31:30.171131+0000","last_became_active":"2024-05-14T09:30:53.770609+0000","last_became_peered":"2024-05-14T09:30:53.770609+0000","last_unstale":"2024-05-14T09:31:30.171131+0000","last_undegraded":"2024-05-14T09:31:30.171131+0000","last_fullsized":"2024-05-14T09:31:30.171131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:26:22.020642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.163851+0000","last_change":"2024-05-14T09:30:56.789219+0000","last_active":"2024-05-14T09:31:30.163851+0000","last_peered":"2024-05-14T09:31:30.163851+0000","last_clean":"2024-05-14T09:31:30.163851+0000","last_became_active":"2024-05-14T09:30:56.788871+0000","last_became_peered":"2024-05-14T09:30:56.788871+0000","last_unstale":"2024-05-14T09:31:30.163851+0000","last_undegraded":"2024-05-14T09:31:30.163851+0000","last_fullsized":"2024-05-14T09:31:30.163851+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:10:50.774267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.174708+0000","last_change":"2024-05-14T09:31:30.174708+0000","last_active":"2024-05-14T09:31:30.174708+0000","last_peered":"2024-05-14T09:31:30.174708+0000","last_clean":"2024-05-14T09:31:30.174708+0000","last_became_active":"2024-05-14T09:31:30.174199+0000","last_became_peered":"2024-05-14T09:31:30.174199+0000","last_unstale":"2024-05-14T09:31:30.174708+0000","last_undegraded":"2024-05-14T09:31:30.174708+0000","last_fullsized":"2024-05-14T09:31:30.174708+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:23:55.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:32:21.278852+0000","last_change":"2024-05-14T09:30:53.770898+0000","last_active":"2024-05-14T09:32:21.278852+0000","last_peered":"2024-05-14T09:32:21.278852+0000","last_clean":"2024-05-14T09:32:21.278852+0000","last_became_active":"2024-05-14T09:30:53.770483+0000","last_became_peered":"2024-05-14T09:30:53.770483+0000","last_unstale":"2024-05-14T09:32:21.278852+0000","last_undegraded":"2024-05-14T09:32:21.278852+0000","last_fullsized":"2024-05-14T09:32:21.278852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:07.764348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173069+0000","last_change":"2024-05-14T09:30:56.799049+0000","last_active":"2024-05-14T09:31:30.173069+0000","last_peered":"2024-05-14T09:31:30.173069+0000","last_clean":"2024-05-14T09:31:30.173069+0000","last_became_active":"2024-05-14T09:30:56.798643+0000","last_became_peered":"2024-05-14T09:30:56.798643+0000","last_unstale":"2024-05-14T09:31:30.173069+0000","last_undegraded":"2024-05-14T09:31:30.173069+0000","last_fullsized":"2024-05-14T09:31:30.173069+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:12.378373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.163806+0000","last_change":"2024-05-14T09:31:08.109876+0000","last_active":"2024-05-14T09:31:30.163806+0000","last_peered":"2024-05-14T09:31:30.163806+0000","last_clean":"2024-05-14T09:31:30.163806+0000","last_became_active":"2024-05-14T09:31:08.109470+0000","last_became_peered":"2024-05-14T09:31:08.109470+0000","last_unstale":"2024-05-14T09:31:30.163806+0000","last_undegraded":"2024-05-14T09:31:30.163806+0000","last_fullsized":"2024-05-14T09:31:30.163806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:07.704153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.711814+0000","last_change":"2024-05-14T09:31:30.179086+0000","last_active":"2024-05-14T09:31:54.711814+0000","last_peered":"2024-05-14T09:31:54.711814+0000","last_clean":"2024-05-14T09:31:54.711814+0000","last_became_active":"2024-05-14T09:31:30.178604+0000","last_became_peered":"2024-05-14T09:31:30.178604+0000","last_unstale":"2024-05-14T09:31:54.711814+0000","last_undegraded":"2024-05-14T09:31:54.711814+0000","last_fullsized":"2024-05-14T09:31:54.711814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:39.712395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295613+0000","last_change":"2024-05-14T09:30:56.793109+0000","last_active":"2024-05-14T09:31:30.295613+0000","last_peered":"2024-05-14T09:31:30.295613+0000","last_clean":"2024-05-14T09:31:30.295613+0000","last_became_active":"2024-05-14T09:30:56.792761+0000","last_became_peered":"2024-05-14T09:30:56.792761+0000","last_unstale":"2024-05-14T09:31:30.295613+0000","last_undegraded":"2024-05-14T09:31:30.295613+0000","last_fullsized":"2024-05-14T09:31:30.295613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:25:33.216281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168519+0000","last_change":"2024-05-14T09:31:08.120978+0000","last_active":"2024-05-14T09:31:30.168519+0000","last_peered":"2024-05-14T09:31:30.168519+0000","last_clean":"2024-05-14T09:31:30.168519+0000","last_became_active":"2024-05-14T09:31:08.120227+0000","last_became_peered":"2024-05-14T09:31:08.120227+0000","last_unstale":"2024-05-14T09:31:30.168519+0000","last_undegraded":"2024-05-14T09:31:30.168519+0000","last_fullsized":"2024-05-14T09:31:30.168519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:06.630390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.710669+0000","last_change":"2024-05-14T09:30:54.017382+0000","last_active":"2024-05-14T09:31:54.710669+0000","last_peered":"2024-05-14T09:31:54.710669+0000","last_clean":"2024-05-14T09:31:54.710669+0000","last_became_active":"2024-05-14T09:30:54.016887+0000","last_became_peered":"2024-05-14T09:30:54.016887+0000","last_unstale":"2024-05-14T09:31:54.710669+0000","last_undegraded":"2024-05-14T09:31:54.710669+0000","last_fullsized":"2024-05-14T09:31:54.710669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:28.823659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171462+0000","last_change":"2024-05-14T09:30:56.800626+0000","last_active":"2024-05-14T09:31:30.171462+0000","last_peered":"2024-05-14T09:31:30.171462+0000","last_clean":"2024-05-14T09:31:30.171462+0000","last_became_active":"2024-05-14T09:30:56.800258+0000","last_became_peered":"2024-05-14T09:30:56.800258+0000","last_unstale":"2024-05-14T09:31:30.171462+0000","last_undegraded":"2024-05-14T09:31:30.171462+0000","last_fullsized":"2024-05-14T09:31:30.171462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:47:26.369633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.163943+0000","last_change":"2024-05-14T09:31:08.115456+0000","last_active":"2024-05-14T09:31:30.163943+0000","last_peered":"2024-05-14T09:31:30.163943+0000","last_clean":"2024-05-14T09:31:30.163943+0000","last_became_active":"2024-05-14T09:31:08.114705+0000","last_became_peered":"2024-05-14T09:31:08.114705+0000","last_unstale":"2024-05-14T09:31:30.163943+0000","last_undegraded":"2024-05-14T09:31:30.163943+0000","last_fullsized":"2024-05-14T09:31:30.163943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:50.096747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:32:22.280367+0000","last_change":"2024-05-14T09:31:30.186202+0000","last_active":"2024-05-14T09:32:22.280367+0000","last_peered":"2024-05-14T09:32:22.280367+0000","last_clean":"2024-05-14T09:32:22.280367+0000","last_became_active":"2024-05-14T09:31:30.174248+0000","last_became_peered":"2024-05-14T09:31:30.174248+0000","last_unstale":"2024-05-14T09:32:22.280367+0000","last_undegraded":"2024-05-14T09:32:22.280367+0000","last_fullsized":"2024-05-14T09:32:22.280367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:45:03.131934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58657,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":58,"num_read_kb":86,"num_write":32,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166988+0000","last_change":"2024-05-14T09:30:56.791099+0000","last_active":"2024-05-14T09:31:30.166988+0000","last_peered":"2024-05-14T09:31:30.166988+0000","last_clean":"2024-05-14T09:31:30.166988+0000","last_became_active":"2024-05-14T09:30:56.790676+0000","last_became_peered":"2024-05-14T09:30:56.790676+0000","last_unstale":"2024-05-14T09:31:30.166988+0000","last_undegraded":"2024-05-14T09:31:30.166988+0000","last_fullsized":"2024-05-14T09:31:30.166988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:09.851357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173530+0000","last_change":"2024-05-14T09:31:08.120289+0000","last_active":"2024-05-14T09:31:30.173530+0000","last_peered":"2024-05-14T09:31:30.173530+0000","last_clean":"2024-05-14T09:31:30.173530+0000","last_became_active":"2024-05-14T09:31:08.118574+0000","last_became_peered":"2024-05-14T09:31:08.118574+0000","last_unstale":"2024-05-14T09:31:30.173530+0000","last_undegraded":"2024-05-14T09:31:30.173530+0000","last_fullsized":"2024-05-14T09:31:30.173530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:01.723442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.717856+0000","last_change":"2024-05-14T09:30:53.766578+0000","last_active":"2024-05-14T09:31:55.717856+0000","last_peered":"2024-05-14T09:31:55.717856+0000","last_clean":"2024-05-14T09:31:55.717856+0000","last_became_active":"2024-05-14T09:30:53.765928+0000","last_became_peered":"2024-05-14T09:30:53.765928+0000","last_unstale":"2024-05-14T09:31:55.717856+0000","last_undegraded":"2024-05-14T09:31:55.717856+0000","last_fullsized":"2024-05-14T09:31:55.717856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:19:08.690013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.170200+0000","last_change":"2024-05-14T09:30:56.785507+0000","last_active":"2024-05-14T09:31:30.170200+0000","last_peered":"2024-05-14T09:31:30.170200+0000","last_clean":"2024-05-14T09:31:30.170200+0000","last_became_active":"2024-05-14T09:30:56.784800+0000","last_became_peered":"2024-05-14T09:30:56.784800+0000","last_unstale":"2024-05-14T09:31:30.170200+0000","last_undegraded":"2024-05-14T09:31:30.170200+0000","last_fullsized":"2024-05-14T09:31:30.170200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:36:05.440222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165117+0000","last_change":"2024-05-14T09:31:08.119051+0000","last_active":"2024-05-14T09:31:30.165117+0000","last_peered":"2024-05-14T09:31:30.165117+0000","last_clean":"2024-05-14T09:31:30.165117+0000","last_became_active":"2024-05-14T09:31:08.118569+0000","last_became_peered":"2024-05-14T09:31:08.118569+0000","last_unstale":"2024-05-14T09:31:30.165117+0000","last_undegraded":"2024-05-14T09:31:30.165117+0000","last_fullsized":"2024-05-14T09:31:30.165117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:12:39.643740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.710378+0000","last_change":"2024-05-14T09:30:53.772832+0000","last_active":"2024-05-14T09:31:54.710378+0000","last_peered":"2024-05-14T09:31:54.710378+0000","last_clean":"2024-05-14T09:31:54.710378+0000","last_became_active":"2024-05-14T09:30:53.772228+0000","last_became_peered":"2024-05-14T09:30:53.772228+0000","last_unstale":"2024-05-14T09:31:54.710378+0000","last_undegraded":"2024-05-14T09:31:54.710378+0000","last_fullsized":"2024-05-14T09:31:54.710378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:14:43.350330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.175161+0000","last_change":"2024-05-14T09:30:56.782831+0000","last_active":"2024-05-14T09:31:30.175161+0000","last_peered":"2024-05-14T09:31:30.175161+0000","last_clean":"2024-05-14T09:31:30.175161+0000","last_became_active":"2024-05-14T09:30:56.782402+0000","last_became_peered":"2024-05-14T09:30:56.782402+0000","last_unstale":"2024-05-14T09:31:30.175161+0000","last_undegraded":"2024-05-14T09:31:30.175161+0000","last_fullsized":"2024-05-14T09:31:30.175161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:54.603524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.297353+0000","last_change":"2024-05-14T09:31:30.296957+0000","last_active":"2024-05-14T09:31:30.297353+0000","last_peered":"2024-05-14T09:31:30.297353+0000","last_clean":"2024-05-14T09:31:30.297353+0000","last_became_active":"2024-05-14T09:31:30.296554+0000","last_became_peered":"2024-05-14T09:31:30.296554+0000","last_unstale":"2024-05-14T09:31:30.297353+0000","last_undegraded":"2024-05-14T09:31:30.297353+0000","last_fullsized":"2024-05-14T09:31:30.297353+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:28.963042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":74,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:32:25.285985+0000","last_change":"2024-05-14T09:30:53.772731+0000","last_active":"2024-05-14T09:32:25.285985+0000","last_peered":"2024-05-14T09:32:25.285985+0000","last_clean":"2024-05-14T09:32:25.285985+0000","last_became_active":"2024-05-14T09:30:53.771986+0000","last_became_peered":"2024-05-14T09:30:53.771986+0000","last_unstale":"2024-05-14T09:32:25.285985+0000","last_undegraded":"2024-05-14T09:32:25.285985+0000","last_fullsized":"2024-05-14T09:32:25.285985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:50:00.810937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173861+0000","last_change":"2024-05-14T09:30:56.790503+0000","last_active":"2024-05-14T09:31:30.173861+0000","last_peered":"2024-05-14T09:31:30.173861+0000","last_clean":"2024-05-14T09:31:30.173861+0000","last_became_active":"2024-05-14T09:30:56.790103+0000","last_became_peered":"2024-05-14T09:30:56.790103+0000","last_unstale":"2024-05-14T09:31:30.173861+0000","last_undegraded":"2024-05-14T09:31:30.173861+0000","last_fullsized":"2024-05-14T09:31:30.173861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:00.950794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169425+0000","last_change":"2024-05-14T09:31:08.116475+0000","last_active":"2024-05-14T09:31:30.169425+0000","last_peered":"2024-05-14T09:31:30.169425+0000","last_clean":"2024-05-14T09:31:30.169425+0000","last_became_active":"2024-05-14T09:31:08.116061+0000","last_became_peered":"2024-05-14T09:31:08.116061+0000","last_unstale":"2024-05-14T09:31:30.169425+0000","last_undegraded":"2024-05-14T09:31:30.169425+0000","last_fullsized":"2024-05-14T09:31:30.169425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:07:02.281853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.721199+0000","last_change":"2024-05-14T09:30:54.016570+0000","last_active":"2024-05-14T09:31:55.721199+0000","last_peered":"2024-05-14T09:31:55.721199+0000","last_clean":"2024-05-14T09:31:55.721199+0000","last_became_active":"2024-05-14T09:30:54.015987+0000","last_became_peered":"2024-05-14T09:30:54.015987+0000","last_unstale":"2024-05-14T09:31:55.721199+0000","last_undegraded":"2024-05-14T09:31:55.721199+0000","last_fullsized":"2024-05-14T09:31:55.721199+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:18:28.694488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171816+0000","last_change":"2024-05-14T09:30:56.800727+0000","last_active":"2024-05-14T09:31:30.171816+0000","last_peered":"2024-05-14T09:31:30.171816+0000","last_clean":"2024-05-14T09:31:30.171816+0000","last_became_active":"2024-05-14T09:30:56.800354+0000","last_became_peered":"2024-05-14T09:30:56.800354+0000","last_unstale":"2024-05-14T09:31:30.171816+0000","last_undegraded":"2024-05-14T09:31:30.171816+0000","last_fullsized":"2024-05-14T09:31:30.171816+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:13:07.227577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171643+0000","last_change":"2024-05-14T09:31:08.115685+0000","last_active":"2024-05-14T09:31:30.171643+0000","last_peered":"2024-05-14T09:31:30.171643+0000","last_clean":"2024-05-14T09:31:30.171643+0000","last_became_active":"2024-05-14T09:31:08.114948+0000","last_became_peered":"2024-05-14T09:31:08.114948+0000","last_unstale":"2024-05-14T09:31:30.171643+0000","last_undegraded":"2024-05-14T09:31:30.171643+0000","last_fullsized":"2024-05-14T09:31:30.171643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:33:07.286823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:56.714258+0000","last_change":"2024-05-14T09:30:53.769949+0000","last_active":"2024-05-14T09:31:56.714258+0000","last_peered":"2024-05-14T09:31:56.714258+0000","last_clean":"2024-05-14T09:31:56.714258+0000","last_became_active":"2024-05-14T09:30:53.769327+0000","last_became_peered":"2024-05-14T09:30:53.769327+0000","last_unstale":"2024-05-14T09:31:56.714258+0000","last_undegraded":"2024-05-14T09:31:56.714258+0000","last_fullsized":"2024-05-14T09:31:56.714258+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:35:11.850991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165834+0000","last_change":"2024-05-14T09:30:56.796590+0000","last_active":"2024-05-14T09:31:30.165834+0000","last_peered":"2024-05-14T09:31:30.165834+0000","last_clean":"2024-05-14T09:31:30.165834+0000","last_became_active":"2024-05-14T09:30:56.796185+0000","last_became_peered":"2024-05-14T09:30:56.796185+0000","last_unstale":"2024-05-14T09:31:30.165834+0000","last_undegraded":"2024-05-14T09:31:30.165834+0000","last_fullsized":"2024-05-14T09:31:30.165834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:48:58.388665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166814+0000","last_change":"2024-05-14T09:31:08.387316+0000","last_active":"2024-05-14T09:31:30.166814+0000","last_peered":"2024-05-14T09:31:30.166814+0000","last_clean":"2024-05-14T09:31:30.166814+0000","last_became_active":"2024-05-14T09:31:08.386739+0000","last_became_peered":"2024-05-14T09:31:08.386739+0000","last_unstale":"2024-05-14T09:31:30.166814+0000","last_undegraded":"2024-05-14T09:31:30.166814+0000","last_fullsized":"2024-05-14T09:31:30.166814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:17.870255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.164455+0000","last_change":"2024-05-14T09:30:54.017756+0000","last_active":"2024-05-14T09:31:30.164455+0000","last_peered":"2024-05-14T09:31:30.164455+0000","last_clean":"2024-05-14T09:31:30.164455+0000","last_became_active":"2024-05-14T09:30:54.017277+0000","last_became_peered":"2024-05-14T09:30:54.017277+0000","last_unstale":"2024-05-14T09:31:30.164455+0000","last_undegraded":"2024-05-14T09:31:30.164455+0000","last_fullsized":"2024-05-14T09:31:30.164455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:27:36.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166185+0000","last_change":"2024-05-14T09:30:56.788408+0000","last_active":"2024-05-14T09:31:30.166185+0000","last_peered":"2024-05-14T09:31:30.166185+0000","last_clean":"2024-05-14T09:31:30.166185+0000","last_became_active":"2024-05-14T09:30:56.787955+0000","last_became_peered":"2024-05-14T09:30:56.787955+0000","last_unstale":"2024-05-14T09:31:30.166185+0000","last_undegraded":"2024-05-14T09:31:30.166185+0000","last_fullsized":"2024-05-14T09:31:30.166185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:09.777069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169255+0000","last_change":"2024-05-14T09:31:08.119078+0000","last_active":"2024-05-14T09:31:30.169255+0000","last_peered":"2024-05-14T09:31:30.169255+0000","last_clean":"2024-05-14T09:31:30.169255+0000","last_became_active":"2024-05-14T09:31:08.117939+0000","last_became_peered":"2024-05-14T09:31:08.117939+0000","last_unstale":"2024-05-14T09:31:30.169255+0000","last_undegraded":"2024-05-14T09:31:30.169255+0000","last_fullsized":"2024-05-14T09:31:30.169255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:48:37.363503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.174543+0000","last_change":"2024-05-14T09:30:53.767843+0000","last_active":"2024-05-14T09:31:30.174543+0000","last_peered":"2024-05-14T09:31:30.174543+0000","last_clean":"2024-05-14T09:31:30.174543+0000","last_became_active":"2024-05-14T09:30:53.767419+0000","last_became_peered":"2024-05-14T09:30:53.767419+0000","last_unstale":"2024-05-14T09:31:30.174543+0000","last_undegraded":"2024-05-14T09:31:30.174543+0000","last_fullsized":"2024-05-14T09:31:30.174543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:49:06.441600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165701+0000","last_change":"2024-05-14T09:30:56.796001+0000","last_active":"2024-05-14T09:31:30.165701+0000","last_peered":"2024-05-14T09:31:30.165701+0000","last_clean":"2024-05-14T09:31:30.165701+0000","last_became_active":"2024-05-14T09:30:56.794517+0000","last_became_peered":"2024-05-14T09:30:56.794517+0000","last_unstale":"2024-05-14T09:31:30.165701+0000","last_undegraded":"2024-05-14T09:31:30.165701+0000","last_fullsized":"2024-05-14T09:31:30.165701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:20:37.514679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169553+0000","last_change":"2024-05-14T09:31:08.387570+0000","last_active":"2024-05-14T09:31:30.169553+0000","last_peered":"2024-05-14T09:31:30.169553+0000","last_clean":"2024-05-14T09:31:30.169553+0000","last_became_active":"2024-05-14T09:31:08.386745+0000","last_became_peered":"2024-05-14T09:31:08.386745+0000","last_unstale":"2024-05-14T09:31:30.169553+0000","last_undegraded":"2024-05-14T09:31:30.169553+0000","last_fullsized":"2024-05-14T09:31:30.169553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:44:01.024784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171385+0000","last_change":"2024-05-14T09:30:56.795532+0000","last_active":"2024-05-14T09:31:30.171385+0000","last_peered":"2024-05-14T09:31:30.171385+0000","last_clean":"2024-05-14T09:31:30.171385+0000","last_became_active":"2024-05-14T09:30:56.795171+0000","last_became_peered":"2024-05-14T09:30:56.795171+0000","last_unstale":"2024-05-14T09:31:30.171385+0000","last_undegraded":"2024-05-14T09:31:30.171385+0000","last_fullsized":"2024-05-14T09:31:30.171385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:09:40.252489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:32:21.278805+0000","last_change":"2024-05-14T09:30:54.015629+0000","last_active":"2024-05-14T09:32:21.278805+0000","last_peered":"2024-05-14T09:32:21.278805+0000","last_clean":"2024-05-14T09:32:21.278805+0000","last_became_active":"2024-05-14T09:30:54.015288+0000","last_became_peered":"2024-05-14T09:30:54.015288+0000","last_unstale":"2024-05-14T09:32:21.278805+0000","last_undegraded":"2024-05-14T09:32:21.278805+0000","last_fullsized":"2024-05-14T09:32:21.278805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:50.316125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.180012+0000","last_change":"2024-05-14T09:31:30.179056+0000","last_active":"2024-05-14T09:31:30.180012+0000","last_peered":"2024-05-14T09:31:30.180012+0000","last_clean":"2024-05-14T09:31:30.180012+0000","last_became_active":"2024-05-14T09:31:30.178398+0000","last_became_peered":"2024-05-14T09:31:30.178398+0000","last_unstale":"2024-05-14T09:31:30.180012+0000","last_undegraded":"2024-05-14T09:31:30.180012+0000","last_fullsized":"2024-05-14T09:31:30.180012+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:15:55.369801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167393+0000","last_change":"2024-05-14T09:30:56.793642+0000","last_active":"2024-05-14T09:31:30.167393+0000","last_peered":"2024-05-14T09:31:30.167393+0000","last_clean":"2024-05-14T09:31:30.167393+0000","last_became_active":"2024-05-14T09:30:56.793229+0000","last_became_peered":"2024-05-14T09:30:56.793229+0000","last_unstale":"2024-05-14T09:31:30.167393+0000","last_undegraded":"2024-05-14T09:31:30.167393+0000","last_fullsized":"2024-05-14T09:31:30.167393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:17.940674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:58.239238+0000","last_change":"2024-05-14T09:30:53.769582+0000","last_active":"2024-05-14T09:31:58.239238+0000","last_peered":"2024-05-14T09:31:58.239238+0000","last_clean":"2024-05-14T09:31:58.239238+0000","last_became_active":"2024-05-14T09:30:53.769113+0000","last_became_peered":"2024-05-14T09:30:53.769113+0000","last_unstale":"2024-05-14T09:31:58.239238+0000","last_undegraded":"2024-05-14T09:31:58.239238+0000","last_fullsized":"2024-05-14T09:31:58.239238+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:48.755113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":114,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173554+0000","last_change":"2024-05-14T09:29:30.767389+0000","last_active":"2024-05-14T09:31:30.173554+0000","last_peered":"2024-05-14T09:31:30.173554+0000","last_clean":"2024-05-14T09:31:30.173554+0000","last_became_active":"2024-05-14T09:29:30.761827+0000","last_became_peered":"2024-05-14T09:29:30.761827+0000","last_unstale":"2024-05-14T09:31:30.173554+0000","last_undegraded":"2024-05-14T09:31:30.173554+0000","last_fullsized":"2024-05-14T09:31:30.173554+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:26:03.741821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:26:03.741821+0000","last_clean_scrub_stamp":"2024-05-14T09:26:03.741821+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:52:11.421658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.172554+0000","last_change":"2024-05-14T09:31:08.387819+0000","last_active":"2024-05-14T09:31:30.172554+0000","last_peered":"2024-05-14T09:31:30.172554+0000","last_clean":"2024-05-14T09:31:30.172554+0000","last_became_active":"2024-05-14T09:31:08.387275+0000","last_became_peered":"2024-05-14T09:31:08.387275+0000","last_unstale":"2024-05-14T09:31:30.172554+0000","last_undegraded":"2024-05-14T09:31:30.172554+0000","last_fullsized":"2024-05-14T09:31:30.172554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:46.627389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168669+0000","last_change":"2024-05-14T09:30:56.788159+0000","last_active":"2024-05-14T09:31:30.168669+0000","last_peered":"2024-05-14T09:31:30.168669+0000","last_clean":"2024-05-14T09:31:30.168669+0000","last_became_active":"2024-05-14T09:30:56.787744+0000","last_became_peered":"2024-05-14T09:30:56.787744+0000","last_unstale":"2024-05-14T09:31:30.168669+0000","last_undegraded":"2024-05-14T09:31:30.168669+0000","last_fullsized":"2024-05-14T09:31:30.168669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:45:25.243471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.719840+0000","last_change":"2024-05-14T09:30:53.774345+0000","last_active":"2024-05-14T09:31:55.719840+0000","last_peered":"2024-05-14T09:31:55.719840+0000","last_clean":"2024-05-14T09:31:55.719840+0000","last_became_active":"2024-05-14T09:30:53.773853+0000","last_became_peered":"2024-05-14T09:30:53.773853+0000","last_unstale":"2024-05-14T09:31:55.719840+0000","last_undegraded":"2024-05-14T09:31:55.719840+0000","last_fullsized":"2024-05-14T09:31:55.719840+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:35:31.938442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165350+0000","last_change":"2024-05-14T09:31:08.122876+0000","last_active":"2024-05-14T09:31:30.165350+0000","last_peered":"2024-05-14T09:31:30.165350+0000","last_clean":"2024-05-14T09:31:30.165350+0000","last_became_active":"2024-05-14T09:31:08.122182+0000","last_became_peered":"2024-05-14T09:31:08.122182+0000","last_unstale":"2024-05-14T09:31:30.165350+0000","last_undegraded":"2024-05-14T09:31:30.165350+0000","last_fullsized":"2024-05-14T09:31:30.165350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:47:30.735057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.707799+0000","last_change":"2024-05-14T09:30:54.018168+0000","last_active":"2024-05-14T09:31:54.707799+0000","last_peered":"2024-05-14T09:31:54.707799+0000","last_clean":"2024-05-14T09:31:54.707799+0000","last_became_active":"2024-05-14T09:30:54.017699+0000","last_became_peered":"2024-05-14T09:30:54.017699+0000","last_unstale":"2024-05-14T09:31:54.707799+0000","last_undegraded":"2024-05-14T09:31:54.707799+0000","last_fullsized":"2024-05-14T09:31:54.707799+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:31:01.162000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169427+0000","last_change":"2024-05-14T09:30:56.787156+0000","last_active":"2024-05-14T09:31:30.169427+0000","last_peered":"2024-05-14T09:31:30.169427+0000","last_clean":"2024-05-14T09:31:30.169427+0000","last_became_active":"2024-05-14T09:30:56.786832+0000","last_became_peered":"2024-05-14T09:30:56.786832+0000","last_unstale":"2024-05-14T09:31:30.169427+0000","last_undegraded":"2024-05-14T09:31:30.169427+0000","last_fullsized":"2024-05-14T09:31:30.169427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:00:41.719641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167185+0000","last_change":"2024-05-14T09:31:08.387907+0000","last_active":"2024-05-14T09:31:30.167185+0000","last_peered":"2024-05-14T09:31:30.167185+0000","last_clean":"2024-05-14T09:31:30.167185+0000","last_became_active":"2024-05-14T09:31:08.386765+0000","last_became_peered":"2024-05-14T09:31:08.386765+0000","last_unstale":"2024-05-14T09:31:30.167185+0000","last_undegraded":"2024-05-14T09:31:30.167185+0000","last_fullsized":"2024-05-14T09:31:30.167185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:24:56.890353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:32:18.803807+0000","last_change":"2024-05-14T09:30:53.767508+0000","last_active":"2024-05-14T09:32:18.803807+0000","last_peered":"2024-05-14T09:32:18.803807+0000","last_clean":"2024-05-14T09:32:18.803807+0000","last_became_active":"2024-05-14T09:30:53.767169+0000","last_became_peered":"2024-05-14T09:30:53.767169+0000","last_unstale":"2024-05-14T09:32:18.803807+0000","last_undegraded":"2024-05-14T09:32:18.803807+0000","last_fullsized":"2024-05-14T09:32:18.803807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:04:23.621686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167705+0000","last_change":"2024-05-14T09:30:56.790398+0000","last_active":"2024-05-14T09:31:30.167705+0000","last_peered":"2024-05-14T09:31:30.167705+0000","last_clean":"2024-05-14T09:31:30.167705+0000","last_became_active":"2024-05-14T09:30:56.790055+0000","last_became_peered":"2024-05-14T09:30:56.790055+0000","last_unstale":"2024-05-14T09:31:30.167705+0000","last_undegraded":"2024-05-14T09:31:30.167705+0000","last_fullsized":"2024-05-14T09:31:30.167705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:12:37.688647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169218+0000","last_change":"2024-05-14T09:31:08.111533+0000","last_active":"2024-05-14T09:31:30.169218+0000","last_peered":"2024-05-14T09:31:30.169218+0000","last_clean":"2024-05-14T09:31:30.169218+0000","last_became_active":"2024-05-14T09:31:08.110448+0000","last_became_peered":"2024-05-14T09:31:08.110448+0000","last_unstale":"2024-05-14T09:31:30.169218+0000","last_undegraded":"2024-05-14T09:31:30.169218+0000","last_fullsized":"2024-05-14T09:31:30.169218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:43:45.822382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.709126+0000","last_change":"2024-05-14T09:30:54.069180+0000","last_active":"2024-05-14T09:31:54.709126+0000","last_peered":"2024-05-14T09:31:54.709126+0000","last_clean":"2024-05-14T09:31:54.709126+0000","last_became_active":"2024-05-14T09:30:54.068846+0000","last_became_peered":"2024-05-14T09:30:54.068846+0000","last_unstale":"2024-05-14T09:31:54.709126+0000","last_undegraded":"2024-05-14T09:31:54.709126+0000","last_fullsized":"2024-05-14T09:31:54.709126+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:22.541720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295970+0000","last_change":"2024-05-14T09:30:56.788159+0000","last_active":"2024-05-14T09:31:30.295970+0000","last_peered":"2024-05-14T09:31:30.295970+0000","last_clean":"2024-05-14T09:31:30.295970+0000","last_became_active":"2024-05-14T09:30:56.787786+0000","last_became_peered":"2024-05-14T09:30:56.787786+0000","last_unstale":"2024-05-14T09:31:30.295970+0000","last_undegraded":"2024-05-14T09:31:30.295970+0000","last_fullsized":"2024-05-14T09:31:30.295970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:24:51.785543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168278+0000","last_change":"2024-05-14T09:31:08.388201+0000","last_active":"2024-05-14T09:31:30.168278+0000","last_peered":"2024-05-14T09:31:30.168278+0000","last_clean":"2024-05-14T09:31:30.168278+0000","last_became_active":"2024-05-14T09:31:08.387572+0000","last_became_peered":"2024-05-14T09:31:08.387572+0000","last_unstale":"2024-05-14T09:31:30.168278+0000","last_undegraded":"2024-05-14T09:31:30.168278+0000","last_fullsized":"2024-05-14T09:31:30.168278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:25:13.171350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.708354+0000","last_change":"2024-05-14T09:30:54.016524+0000","last_active":"2024-05-14T09:31:54.708354+0000","last_peered":"2024-05-14T09:31:54.708354+0000","last_clean":"2024-05-14T09:31:54.708354+0000","last_became_active":"2024-05-14T09:30:54.016154+0000","last_became_peered":"2024-05-14T09:30:54.016154+0000","last_unstale":"2024-05-14T09:31:54.708354+0000","last_undegraded":"2024-05-14T09:31:54.708354+0000","last_fullsized":"2024-05-14T09:31:54.708354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:43:16.671053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.166563+0000","last_change":"2024-05-14T09:30:56.796748+0000","last_active":"2024-05-14T09:31:30.166563+0000","last_peered":"2024-05-14T09:31:30.166563+0000","last_clean":"2024-05-14T09:31:30.166563+0000","last_became_active":"2024-05-14T09:30:56.796354+0000","last_became_peered":"2024-05-14T09:30:56.796354+0000","last_unstale":"2024-05-14T09:31:30.166563+0000","last_undegraded":"2024-05-14T09:31:30.166563+0000","last_fullsized":"2024-05-14T09:31:30.166563+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:30:15.820677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.164298+0000","last_change":"2024-05-14T09:31:08.117266+0000","last_active":"2024-05-14T09:31:30.164298+0000","last_peered":"2024-05-14T09:31:30.164298+0000","last_clean":"2024-05-14T09:31:30.164298+0000","last_became_active":"2024-05-14T09:31:08.115393+0000","last_became_peered":"2024-05-14T09:31:08.115393+0000","last_unstale":"2024-05-14T09:31:30.164298+0000","last_undegraded":"2024-05-14T09:31:30.164298+0000","last_fullsized":"2024-05-14T09:31:30.164298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:29:26.348818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.709345+0000","last_change":"2024-05-14T09:30:53.772417+0000","last_active":"2024-05-14T09:31:54.709345+0000","last_peered":"2024-05-14T09:31:54.709345+0000","last_clean":"2024-05-14T09:31:54.709345+0000","last_became_active":"2024-05-14T09:30:53.772028+0000","last_became_peered":"2024-05-14T09:30:53.772028+0000","last_unstale":"2024-05-14T09:31:54.709345+0000","last_undegraded":"2024-05-14T09:31:54.709345+0000","last_fullsized":"2024-05-14T09:31:54.709345+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:16:16.603957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.164993+0000","last_change":"2024-05-14T09:30:56.804750+0000","last_active":"2024-05-14T09:31:30.164993+0000","last_peered":"2024-05-14T09:31:30.164993+0000","last_clean":"2024-05-14T09:31:30.164993+0000","last_became_active":"2024-05-14T09:30:56.803990+0000","last_became_peered":"2024-05-14T09:30:56.803990+0000","last_unstale":"2024-05-14T09:31:30.164993+0000","last_undegraded":"2024-05-14T09:31:30.164993+0000","last_fullsized":"2024-05-14T09:31:30.164993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:42:00.661033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168707+0000","last_change":"2024-05-14T09:31:08.116061+0000","last_active":"2024-05-14T09:31:30.168707+0000","last_peered":"2024-05-14T09:31:30.168707+0000","last_clean":"2024-05-14T09:31:30.168707+0000","last_became_active":"2024-05-14T09:31:08.115578+0000","last_became_peered":"2024-05-14T09:31:08.115578+0000","last_unstale":"2024-05-14T09:31:30.168707+0000","last_undegraded":"2024-05-14T09:31:30.168707+0000","last_fullsized":"2024-05-14T09:31:30.168707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:18.931035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.170206+0000","last_change":"2024-05-14T09:30:53.768988+0000","last_active":"2024-05-14T09:31:30.170206+0000","last_peered":"2024-05-14T09:31:30.170206+0000","last_clean":"2024-05-14T09:31:30.170206+0000","last_became_active":"2024-05-14T09:30:53.768529+0000","last_became_peered":"2024-05-14T09:30:53.768529+0000","last_unstale":"2024-05-14T09:31:30.170206+0000","last_undegraded":"2024-05-14T09:31:30.170206+0000","last_fullsized":"2024-05-14T09:31:30.170206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:17:02.381447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167877+0000","last_change":"2024-05-14T09:30:56.797110+0000","last_active":"2024-05-14T09:31:30.167877+0000","last_peered":"2024-05-14T09:31:30.167877+0000","last_clean":"2024-05-14T09:31:30.167877+0000","last_became_active":"2024-05-14T09:30:56.796015+0000","last_became_peered":"2024-05-14T09:30:56.796015+0000","last_unstale":"2024-05-14T09:31:30.167877+0000","last_undegraded":"2024-05-14T09:31:30.167877+0000","last_fullsized":"2024-05-14T09:31:30.167877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:37:44.674607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.167730+0000","last_change":"2024-05-14T09:31:08.387240+0000","last_active":"2024-05-14T09:31:30.167730+0000","last_peered":"2024-05-14T09:31:30.167730+0000","last_clean":"2024-05-14T09:31:30.167730+0000","last_became_active":"2024-05-14T09:31:08.386653+0000","last_became_peered":"2024-05-14T09:31:08.386653+0000","last_unstale":"2024-05-14T09:31:30.167730+0000","last_undegraded":"2024-05-14T09:31:30.167730+0000","last_fullsized":"2024-05-14T09:31:30.167730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:18:43.408030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:54.710918+0000","last_change":"2024-05-14T09:30:53.772723+0000","last_active":"2024-05-14T09:31:54.710918+0000","last_peered":"2024-05-14T09:31:54.710918+0000","last_clean":"2024-05-14T09:31:54.710918+0000","last_became_active":"2024-05-14T09:30:53.772056+0000","last_became_peered":"2024-05-14T09:30:53.772056+0000","last_unstale":"2024-05-14T09:31:54.710918+0000","last_undegraded":"2024-05-14T09:31:54.710918+0000","last_fullsized":"2024-05-14T09:31:54.710918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:37.941282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.174622+0000","last_change":"2024-05-14T09:30:56.799223+0000","last_active":"2024-05-14T09:31:30.174622+0000","last_peered":"2024-05-14T09:31:30.174622+0000","last_clean":"2024-05-14T09:31:30.174622+0000","last_became_active":"2024-05-14T09:30:56.796092+0000","last_became_peered":"2024-05-14T09:30:56.796092+0000","last_unstale":"2024-05-14T09:31:30.174622+0000","last_undegraded":"2024-05-14T09:31:30.174622+0000","last_fullsized":"2024-05-14T09:31:30.174622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:43:49.746797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.172036+0000","last_change":"2024-05-14T09:31:08.115053+0000","last_active":"2024-05-14T09:31:30.172036+0000","last_peered":"2024-05-14T09:31:30.172036+0000","last_clean":"2024-05-14T09:31:30.172036+0000","last_became_active":"2024-05-14T09:31:08.114578+0000","last_became_peered":"2024-05-14T09:31:08.114578+0000","last_unstale":"2024-05-14T09:31:30.172036+0000","last_undegraded":"2024-05-14T09:31:30.172036+0000","last_fullsized":"2024-05-14T09:31:30.172036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:47.788962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:55.719248+0000","last_change":"2024-05-14T09:30:54.069055+0000","last_active":"2024-05-14T09:31:55.719248+0000","last_peered":"2024-05-14T09:31:55.719248+0000","last_clean":"2024-05-14T09:31:55.719248+0000","last_became_active":"2024-05-14T09:30:54.068479+0000","last_became_peered":"2024-05-14T09:30:54.068479+0000","last_unstale":"2024-05-14T09:31:55.719248+0000","last_undegraded":"2024-05-14T09:31:55.719248+0000","last_fullsized":"2024-05-14T09:31:55.719248+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:34.092472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.175965+0000","last_change":"2024-05-14T09:30:56.798580+0000","last_active":"2024-05-14T09:31:30.175965+0000","last_peered":"2024-05-14T09:31:30.175965+0000","last_clean":"2024-05-14T09:31:30.175965+0000","last_became_active":"2024-05-14T09:30:56.795617+0000","last_became_peered":"2024-05-14T09:30:56.795617+0000","last_unstale":"2024-05-14T09:31:30.175965+0000","last_undegraded":"2024-05-14T09:31:30.175965+0000","last_fullsized":"2024-05-14T09:31:30.175965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:30:35.913497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295529+0000","last_change":"2024-05-14T09:31:08.111375+0000","last_active":"2024-05-14T09:31:30.295529+0000","last_peered":"2024-05-14T09:31:30.295529+0000","last_clean":"2024-05-14T09:31:30.295529+0000","last_became_active":"2024-05-14T09:31:08.110931+0000","last_became_peered":"2024-05-14T09:31:08.110931+0000","last_unstale":"2024-05-14T09:31:30.295529+0000","last_undegraded":"2024-05-14T09:31:30.295529+0000","last_fullsized":"2024-05-14T09:31:30.295529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:08:30.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.173185+0000","last_change":"2024-05-14T09:30:53.772385+0000","last_active":"2024-05-14T09:31:30.173185+0000","last_peered":"2024-05-14T09:31:30.173185+0000","last_clean":"2024-05-14T09:31:30.173185+0000","last_became_active":"2024-05-14T09:30:53.771928+0000","last_became_peered":"2024-05-14T09:30:53.771928+0000","last_unstale":"2024-05-14T09:31:30.173185+0000","last_undegraded":"2024-05-14T09:31:30.173185+0000","last_fullsized":"2024-05-14T09:31:30.173185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:06:21.357341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.176063+0000","last_change":"2024-05-14T09:30:56.789527+0000","last_active":"2024-05-14T09:31:30.176063+0000","last_peered":"2024-05-14T09:31:30.176063+0000","last_clean":"2024-05-14T09:31:30.176063+0000","last_became_active":"2024-05-14T09:30:56.789075+0000","last_became_peered":"2024-05-14T09:30:56.789075+0000","last_unstale":"2024-05-14T09:31:30.176063+0000","last_undegraded":"2024-05-14T09:31:30.176063+0000","last_fullsized":"2024-05-14T09:31:30.176063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:07.140087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.170686+0000","last_change":"2024-05-14T09:31:08.119624+0000","last_active":"2024-05-14T09:31:30.170686+0000","last_peered":"2024-05-14T09:31:30.170686+0000","last_clean":"2024-05-14T09:31:30.170686+0000","last_became_active":"2024-05-14T09:31:08.118102+0000","last_became_peered":"2024-05-14T09:31:08.118102+0000","last_unstale":"2024-05-14T09:31:30.170686+0000","last_undegraded":"2024-05-14T09:31:30.170686+0000","last_fullsized":"2024-05-14T09:31:30.170686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:48:17.806976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169513+0000","last_change":"2024-05-14T09:30:53.763903+0000","last_active":"2024-05-14T09:31:30.169513+0000","last_peered":"2024-05-14T09:31:30.169513+0000","last_clean":"2024-05-14T09:31:30.169513+0000","last_became_active":"2024-05-14T09:30:53.763517+0000","last_became_peered":"2024-05-14T09:30:53.763517+0000","last_unstale":"2024-05-14T09:31:30.169513+0000","last_undegraded":"2024-05-14T09:31:30.169513+0000","last_fullsized":"2024-05-14T09:31:30.169513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:57.144392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.176599+0000","last_change":"2024-05-14T09:30:56.791612+0000","last_active":"2024-05-14T09:31:30.176599+0000","last_peered":"2024-05-14T09:31:30.176599+0000","last_clean":"2024-05-14T09:31:30.176599+0000","last_became_active":"2024-05-14T09:30:56.791178+0000","last_became_peered":"2024-05-14T09:30:56.791178+0000","last_unstale":"2024-05-14T09:31:30.176599+0000","last_undegraded":"2024-05-14T09:31:30.176599+0000","last_fullsized":"2024-05-14T09:31:30.176599+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:11:19.058139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165093+0000","last_change":"2024-05-14T09:30:56.804437+0000","last_active":"2024-05-14T09:31:30.165093+0000","last_peered":"2024-05-14T09:31:30.165093+0000","last_clean":"2024-05-14T09:31:30.165093+0000","last_became_active":"2024-05-14T09:30:56.804048+0000","last_became_peered":"2024-05-14T09:30:56.804048+0000","last_unstale":"2024-05-14T09:31:30.165093+0000","last_undegraded":"2024-05-14T09:31:30.165093+0000","last_fullsized":"2024-05-14T09:31:30.165093+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:33.788903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.170173+0000","last_change":"2024-05-14T09:30:56.794309+0000","last_active":"2024-05-14T09:31:30.170173+0000","last_peered":"2024-05-14T09:31:30.170173+0000","last_clean":"2024-05-14T09:31:30.170173+0000","last_became_active":"2024-05-14T09:30:56.793317+0000","last_became_peered":"2024-05-14T09:30:56.793317+0000","last_unstale":"2024-05-14T09:31:30.170173+0000","last_undegraded":"2024-05-14T09:31:30.170173+0000","last_fullsized":"2024-05-14T09:31:30.170173+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:40:26.878746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169958+0000","last_change":"2024-05-14T09:30:56.793618+0000","last_active":"2024-05-14T09:31:30.169958+0000","last_peered":"2024-05-14T09:31:30.169958+0000","last_clean":"2024-05-14T09:31:30.169958+0000","last_became_active":"2024-05-14T09:30:56.792819+0000","last_became_peered":"2024-05-14T09:30:56.792819+0000","last_unstale":"2024-05-14T09:31:30.169958+0000","last_undegraded":"2024-05-14T09:31:30.169958+0000","last_fullsized":"2024-05-14T09:31:30.169958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:29:38.191617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.295810+0000","last_change":"2024-05-14T09:30:56.794268+0000","last_active":"2024-05-14T09:31:30.295810+0000","last_peered":"2024-05-14T09:31:30.295810+0000","last_clean":"2024-05-14T09:31:30.295810+0000","last_became_active":"2024-05-14T09:30:56.793898+0000","last_became_peered":"2024-05-14T09:30:56.793898+0000","last_unstale":"2024-05-14T09:31:30.295810+0000","last_undegraded":"2024-05-14T09:31:30.295810+0000","last_fullsized":"2024-05-14T09:31:30.295810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:25:35.022236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169392+0000","last_change":"2024-05-14T09:30:56.783366+0000","last_active":"2024-05-14T09:31:30.169392+0000","last_peered":"2024-05-14T09:31:30.169392+0000","last_clean":"2024-05-14T09:31:30.169392+0000","last_became_active":"2024-05-14T09:30:56.782980+0000","last_became_peered":"2024-05-14T09:30:56.782980+0000","last_unstale":"2024-05-14T09:31:30.169392+0000","last_undegraded":"2024-05-14T09:31:30.169392+0000","last_fullsized":"2024-05-14T09:31:30.169392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:27.744902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165790+0000","last_change":"2024-05-14T09:30:56.795578+0000","last_active":"2024-05-14T09:31:30.165790+0000","last_peered":"2024-05-14T09:31:30.165790+0000","last_clean":"2024-05-14T09:31:30.165790+0000","last_became_active":"2024-05-14T09:30:56.795141+0000","last_became_peered":"2024-05-14T09:30:56.795141+0000","last_unstale":"2024-05-14T09:31:30.165790+0000","last_undegraded":"2024-05-14T09:31:30.165790+0000","last_fullsized":"2024-05-14T09:31:30.165790+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:05:25.012646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.171520+0000","last_change":"2024-05-14T09:30:56.795872+0000","last_active":"2024-05-14T09:31:30.171520+0000","last_peered":"2024-05-14T09:31:30.171520+0000","last_clean":"2024-05-14T09:31:30.171520+0000","last_became_active":"2024-05-14T09:30:56.795492+0000","last_became_peered":"2024-05-14T09:30:56.795492+0000","last_unstale":"2024-05-14T09:31:30.171520+0000","last_undegraded":"2024-05-14T09:31:30.171520+0000","last_fullsized":"2024-05-14T09:31:30.171520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:26:29.226575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168024+0000","last_change":"2024-05-14T09:30:56.797078+0000","last_active":"2024-05-14T09:31:30.168024+0000","last_peered":"2024-05-14T09:31:30.168024+0000","last_clean":"2024-05-14T09:31:30.168024+0000","last_became_active":"2024-05-14T09:30:56.796703+0000","last_became_peered":"2024-05-14T09:30:56.796703+0000","last_unstale":"2024-05-14T09:31:30.168024+0000","last_undegraded":"2024-05-14T09:31:30.168024+0000","last_fullsized":"2024-05-14T09:31:30.168024+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:06:16.651877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.172918+0000","last_change":"2024-05-14T09:30:56.792814+0000","last_active":"2024-05-14T09:31:30.172918+0000","last_peered":"2024-05-14T09:31:30.172918+0000","last_clean":"2024-05-14T09:31:30.172918+0000","last_became_active":"2024-05-14T09:30:56.792282+0000","last_became_peered":"2024-05-14T09:30:56.792282+0000","last_unstale":"2024-05-14T09:31:30.172918+0000","last_undegraded":"2024-05-14T09:31:30.172918+0000","last_fullsized":"2024-05-14T09:31:30.172918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:52:35.319479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.168151+0000","last_change":"2024-05-14T09:30:56.791575+0000","last_active":"2024-05-14T09:31:30.168151+0000","last_peered":"2024-05-14T09:31:30.168151+0000","last_clean":"2024-05-14T09:31:30.168151+0000","last_became_active":"2024-05-14T09:30:56.791230+0000","last_became_peered":"2024-05-14T09:30:56.791230+0000","last_unstale":"2024-05-14T09:31:30.168151+0000","last_undegraded":"2024-05-14T09:31:30.168151+0000","last_fullsized":"2024-05-14T09:31:30.168151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:40.510062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.172446+0000","last_change":"2024-05-14T09:30:56.796996+0000","last_active":"2024-05-14T09:31:30.172446+0000","last_peered":"2024-05-14T09:31:30.172446+0000","last_clean":"2024-05-14T09:31:30.172446+0000","last_became_active":"2024-05-14T09:30:56.796333+0000","last_became_peered":"2024-05-14T09:30:56.796333+0000","last_unstale":"2024-05-14T09:31:30.172446+0000","last_undegraded":"2024-05-14T09:31:30.172446+0000","last_fullsized":"2024-05-14T09:31:30.172446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:29:34.799088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.169183+0000","last_change":"2024-05-14T09:30:56.785740+0000","last_active":"2024-05-14T09:31:30.169183+0000","last_peered":"2024-05-14T09:31:30.169183+0000","last_clean":"2024-05-14T09:31:30.169183+0000","last_became_active":"2024-05-14T09:30:56.785394+0000","last_became_peered":"2024-05-14T09:30:56.785394+0000","last_unstale":"2024-05-14T09:31:30.169183+0000","last_undegraded":"2024-05-14T09:31:30.169183+0000","last_fullsized":"2024-05-14T09:31:30.169183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:17.819523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T09:31:30.165516+0000","last_change":"2024-05-14T09:30:56.796957+0000","last_active":"2024-05-14T09:31:30.165516+0000","last_peered":"2024-05-14T09:31:30.165516+0000","last_clean":"2024-05-14T09:31:30.165516+0000","last_became_active":"2024-05-14T09:30:56.795071+0000","last_became_peered":"2024-05-14T09:30:56.795071+0000","last_unstale":"2024-05-14T09:31:30.165516+0000","last_undegraded":"2024-05-14T09:31:30.165516+0000","last_fullsized":"2024-05-14T09:31:30.165516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:44:31.480207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245291,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62658,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":181,"num_read_kb":155,"num_write":165,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":2194,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":198780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678048,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":499802,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874277,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":1149241,"data_compressed":62023,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037803,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":499456,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1597,"internal_metadata":36174275},"hb_peers":[0,1,2,3,4,5,6,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234032,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":498794,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430261,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":1094703,"data_compressed":62023,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593786,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":495214,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":558602,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":496020,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":496283,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149769,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":494826,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":563342,"data_compressed":26518,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790016,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1090873,"data_compressed":62023,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33883,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33883,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T09:32:28.727 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-14T09:32:28.728 INFO:tasks.cephadm:Setting up client nodes... 2024-05-14T09:32:28.728 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-14T09:32:28.984 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:29 smithi002 ceph-mon[31557]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:32:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-05-14T09:32:29.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [5, 11]}]: dispatch 2024-05-14T09:32:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9, 4, 7]}]: dispatch 2024-05-14T09:32:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-05-14T09:32:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-14T09:32:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-05-14T09:32:29.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:29 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-14T09:32:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:29 smithi029 ceph-mon[36249]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:32:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-05-14T09:32:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [5, 11]}]: dispatch 2024-05-14T09:32:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9, 4, 7]}]: dispatch 2024-05-14T09:32:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-05-14T09:32:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-14T09:32:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-05-14T09:32:29.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:29 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-14T09:32:29.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:29 smithi026 ceph-mon[36056]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:32:29.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-05-14T09:32:29.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [5, 11]}]: dispatch 2024-05-14T09:32:29.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9, 4, 7]}]: dispatch 2024-05-14T09:32:29.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-05-14T09:32:29.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-14T09:32:29.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-05-14T09:32:29.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:29 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-14T09:32:30.386 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:30 smithi002 ceph-mon[31557]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-14T09:32:30.387 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-05-14T09:32:30.387 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [5, 11]}]': finished 2024-05-14T09:32:30.387 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9, 4, 7]}]': finished 2024-05-14T09:32:30.387 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-05-14T09:32:30.387 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-14T09:32:30.387 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-05-14T09:32:30.387 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-14T09:32:30.387 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:30 smithi002 ceph-mon[31557]: osdmap e89: 12 total, 12 up, 12 in 2024-05-14T09:32:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:30 smithi029 ceph-mon[36249]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-14T09:32:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-05-14T09:32:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [5, 11]}]': finished 2024-05-14T09:32:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9, 4, 7]}]': finished 2024-05-14T09:32:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-05-14T09:32:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-14T09:32:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-05-14T09:32:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-14T09:32:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:30 smithi029 ceph-mon[36249]: osdmap e89: 12 total, 12 up, 12 in 2024-05-14T09:32:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:30 smithi026 ceph-mon[36056]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-14T09:32:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-05-14T09:32:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.d", "id": [5, 11]}]': finished 2024-05-14T09:32:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9, 4, 7]}]': finished 2024-05-14T09:32:30.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-05-14T09:32:30.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-14T09:32:30.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-05-14T09:32:30.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-14T09:32:30.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:30 smithi026 ceph-mon[36056]: osdmap e89: 12 total, 12 up, 12 in 2024-05-14T09:32:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.815+0000 7fe6868f1700 1 -- 172.21.15.2:0/758358827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe680105e10 msgr2=0x7fe68010c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.815+0000 7fe6868f1700 1 --2- 172.21.15.2:0/758358827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe680105e10 0x7fe68010c6c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe67800b0a0 tx=0x7fe67800b3b0 comp rx=0 tx=0).stop 2024-05-14T09:32:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.815+0000 7fe6868f1700 1 -- 172.21.15.2:0/758358827 shutdown_connections 2024-05-14T09:32:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.815+0000 7fe6868f1700 1 --2- 172.21.15.2:0/758358827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe680105e10 0x7fe68010c6c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.815+0000 7fe6868f1700 1 --2- 172.21.15.2:0/758358827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe680104340 0x7fe680104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.815+0000 7fe6868f1700 1 --2- 172.21.15.2:0/758358827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe680105470 0x7fe680105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.815+0000 7fe6868f1700 1 -- 172.21.15.2:0/758358827 >> 172.21.15.2:0/758358827 conn(0x7fe6800ff980 msgr2=0x7fe680101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.815+0000 7fe6868f1700 1 -- 172.21.15.2:0/758358827 shutdown_connections 2024-05-14T09:32:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.815+0000 7fe6868f1700 1 -- 172.21.15.2:0/758358827 wait complete. 2024-05-14T09:32:30.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe6868f1700 1 Processor -- start 2024-05-14T09:32:30.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe6868f1700 1 -- start start 2024-05-14T09:32:30.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe6868f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe680104340 0x7fe68019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:30.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe6868f1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe680105470 0x7fe68019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:30.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe6868f1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe680105e10 0x7fe6801a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:30.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe6868f1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe68010fe20 con 0x7fe680105470 2024-05-14T09:32:30.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe6868f1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe68010fca0 con 0x7fe680104340 2024-05-14T09:32:30.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe6868f1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe68010ffa0 con 0x7fe680105e10 2024-05-14T09:32:30.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe677fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe680105470 0x7fe68019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:30.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe677fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe680105470 0x7fe68019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35470/0 (socket says 172.21.15.2:35470) 2024-05-14T09:32:30.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe677fff700 1 -- 172.21.15.2:0/1474924376 learned_addr learned my addr 172.21.15.2:0/1474924376 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:30.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe684e8e700 1 --2- 172.21.15.2:0/1474924376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe680105e10 0x7fe6801a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:30.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe67ffff700 1 --2- 172.21.15.2:0/1474924376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe680104340 0x7fe68019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:30.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe677fff700 1 -- 172.21.15.2:0/1474924376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe680105e10 msgr2=0x7fe6801a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:30.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe677fff700 1 --2- 172.21.15.2:0/1474924376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe680105e10 0x7fe6801a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:30.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe677fff700 1 -- 172.21.15.2:0/1474924376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe680104340 msgr2=0x7fe68019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:30.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe677fff700 1 --2- 172.21.15.2:0/1474924376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe680104340 0x7fe68019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:30.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.816+0000 7fe677fff700 1 -- 172.21.15.2:0/1474924376 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6801a2430 con 0x7fe680105470 2024-05-14T09:32:30.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.817+0000 7fe677fff700 1 --2- 172.21.15.2:0/1474924376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe680105470 0x7fe68019ae10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe66800b530 tx=0x7fe66800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:30.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.817+0000 7fe67dffb700 1 -- 172.21.15.2:0/1474924376 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668015020 con 0x7fe680105470 2024-05-14T09:32:30.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.817+0000 7fe67dffb700 1 -- 172.21.15.2:0/1474924376 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe668011ce0 con 0x7fe680105470 2024-05-14T09:32:30.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.817+0000 7fe67dffb700 1 -- 172.21.15.2:0/1474924376 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668010e40 con 0x7fe680105470 2024-05-14T09:32:30.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.817+0000 7fe6868f1700 1 -- 172.21.15.2:0/1474924376 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6801a2720 con 0x7fe680105470 2024-05-14T09:32:30.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.817+0000 7fe6868f1700 1 -- 172.21.15.2:0/1474924376 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6801a2d30 con 0x7fe680105470 2024-05-14T09:32:30.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.818+0000 7fe6868f1700 1 -- 172.21.15.2:0/1474924376 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6800512c0 con 0x7fe680105470 2024-05-14T09:32:30.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.819+0000 7fe67dffb700 1 -- 172.21.15.2:0/1474924376 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fe668018020 con 0x7fe680105470 2024-05-14T09:32:30.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.820+0000 7fe67dffb700 1 --2- 172.21.15.2:0/1474924376 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe66c0b2990 0x7fe66c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:30.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.820+0000 7fe67ffff700 1 --2- 172.21.15.2:0/1474924376 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe66c0b2990 0x7fe66c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:30.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.820+0000 7fe67dffb700 1 -- 172.21.15.2:0/1474924376 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fe6680e2480 con 0x7fe680105470 2024-05-14T09:32:30.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.821+0000 7fe67ffff700 1 --2- 172.21.15.2:0/1474924376 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe66c0b2990 0x7fe66c0b4ea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe670005950 tx=0x7fe6700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:30.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:30.824+0000 7fe67dffb700 1 -- 172.21.15.2:0/1474924376 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe66801f020 con 0x7fe680105470 2024-05-14T09:32:31.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:31.127+0000 7fe6868f1700 1 -- 172.21.15.2:0/1474924376 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fe68002e5f0 con 0x7fe680105470 2024-05-14T09:32:31.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:31.130+0000 7fe67dffb700 1 -- 172.21.15.2:0/1474924376 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fe6680a9cd0 con 0x7fe680105470 2024-05-14T09:32:31.131 INFO:teuthology.orchestra.run.smithi002.stdout:[client.0] 2024-05-14T09:32:31.131 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCvL0NmOAS6BxAAm22zesmv1tA2KiRQL+CzWg== 2024-05-14T09:32:31.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:31.132+0000 7fe6868f1700 1 -- 172.21.15.2:0/1474924376 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe66c0b2990 msgr2=0x7fe66c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:31.132+0000 7fe6868f1700 1 --2- 172.21.15.2:0/1474924376 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe66c0b2990 0x7fe66c0b4ea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe670005950 tx=0x7fe6700058e0 comp rx=0 tx=0).stop 2024-05-14T09:32:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:31.132+0000 7fe6868f1700 1 -- 172.21.15.2:0/1474924376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe680105470 msgr2=0x7fe68019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:31.132+0000 7fe6868f1700 1 --2- 172.21.15.2:0/1474924376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe680105470 0x7fe68019ae10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe66800b530 tx=0x7fe66800b840 comp rx=0 tx=0).stop 2024-05-14T09:32:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:31.133+0000 7fe6868f1700 1 -- 172.21.15.2:0/1474924376 shutdown_connections 2024-05-14T09:32:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:31.133+0000 7fe6868f1700 1 --2- 172.21.15.2:0/1474924376 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe66c0b2990 0x7fe66c0b4ea0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:31.133+0000 7fe6868f1700 1 --2- 172.21.15.2:0/1474924376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe680104340 0x7fe68019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:31.133+0000 7fe6868f1700 1 --2- 172.21.15.2:0/1474924376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe680105e10 0x7fe6801a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:31.133+0000 7fe6868f1700 1 --2- 172.21.15.2:0/1474924376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe680105470 0x7fe68019ae10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:31.133+0000 7fe6868f1700 1 -- 172.21.15.2:0/1474924376 >> 172.21.15.2:0/1474924376 conn(0x7fe6800ff980 msgr2=0x7fe6801086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:31.133+0000 7fe6868f1700 1 -- 172.21.15.2:0/1474924376 shutdown_connections 2024-05-14T09:32:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:31.133+0000 7fe6868f1700 1 -- 172.21.15.2:0/1474924376 wait complete. 2024-05-14T09:32:31.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:31 smithi002 ceph-mon[31557]: osdmap e90: 12 total, 12 up, 12 in 2024-05-14T09:32:31.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:31 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1474924376' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T09:32:31.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:31 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1474924376' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-14T09:32:31.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:31 smithi029 ceph-mon[36249]: osdmap e90: 12 total, 12 up, 12 in 2024-05-14T09:32:31.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:31 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1474924376' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T09:32:31.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:31 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1474924376' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-14T09:32:31.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:31 smithi026 ceph-mon[36056]: osdmap e90: 12 total, 12 up, 12 in 2024-05-14T09:32:31.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:31 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1474924376' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T09:32:31.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:31 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1474924376' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-14T09:32:31.920 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:32:31.921 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-14T09:32:31.921 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-14T09:32:31.963 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-14T09:32:31.963 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-14T09:32:31.964 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-14T09:32:32.254 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:32.601 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:32 smithi002 ceph-mon[31557]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 4 op/s 2024-05-14T09:32:32.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:32 smithi029 ceph-mon[36249]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 4 op/s 2024-05-14T09:32:32.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:32 smithi026 ceph-mon[36056]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 4 op/s 2024-05-14T09:32:34.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.010+0000 7f9937b3e700 1 -- 172.21.15.2:0/3487350339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99301034c0 msgr2=0x7f99301038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:34.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.010+0000 7f9937b3e700 1 --2- 172.21.15.2:0/3487350339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99301034c0 0x7f99301038d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f992c0099c0 tx=0x7f992c009cd0 comp rx=0 tx=0).stop 2024-05-14T09:32:34.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.010+0000 7f9937b3e700 1 -- 172.21.15.2:0/3487350339 shutdown_connections 2024-05-14T09:32:34.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.010+0000 7f9937b3e700 1 --2- 172.21.15.2:0/3487350339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9930108740 0x7f99301100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:34.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.010+0000 7f9937b3e700 1 --2- 172.21.15.2:0/3487350339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99301034c0 0x7f99301038d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:34.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.010+0000 7f9937b3e700 1 --2- 172.21.15.2:0/3487350339 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9930107df0 0x7f9930108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:34.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.010+0000 7f9937b3e700 1 -- 172.21.15.2:0/3487350339 >> 172.21.15.2:0/3487350339 conn(0x7f99300fedf0 msgr2=0x7f9930101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:34.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.010+0000 7f9937b3e700 1 -- 172.21.15.2:0/3487350339 shutdown_connections 2024-05-14T09:32:34.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.010+0000 7f9937b3e700 1 -- 172.21.15.2:0/3487350339 wait complete. 2024-05-14T09:32:34.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.010+0000 7f9937b3e700 1 Processor -- start 2024-05-14T09:32:34.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f9937b3e700 1 -- start start 2024-05-14T09:32:34.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f9937b3e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99301034c0 0x7f993019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:34.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f9937b3e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9930107df0 0x7f993019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:34.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f9937b3e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9930108740 0x7f993019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:34.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f9937b3e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f99301133c0 con 0x7f9930108740 2024-05-14T09:32:34.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f9937b3e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9930113240 con 0x7f9930107df0 2024-05-14T09:32:34.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f9937b3e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9930113540 con 0x7f99301034c0 2024-05-14T09:32:34.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f99360db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9930108740 0x7f993019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:34.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f99360db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9930108740 0x7f993019f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35476/0 (socket says 172.21.15.2:35476) 2024-05-14T09:32:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f99360db700 1 -- 172.21.15.2:0/2090406587 learned_addr learned my addr 172.21.15.2:0/2090406587 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f99358da700 1 --2- 172.21.15.2:0/2090406587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99301034c0 0x7f993019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f99350d9700 1 --2- 172.21.15.2:0/2090406587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9930107df0 0x7f993019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f99360db700 1 -- 172.21.15.2:0/2090406587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99301034c0 msgr2=0x7f993019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f99360db700 1 --2- 172.21.15.2:0/2090406587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99301034c0 0x7f993019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f99360db700 1 -- 172.21.15.2:0/2090406587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9930107df0 msgr2=0x7f993019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f99360db700 1 --2- 172.21.15.2:0/2090406587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9930107df0 0x7f993019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.011+0000 7f99360db700 1 -- 172.21.15.2:0/2090406587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993019fcd0 con 0x7f9930108740 2024-05-14T09:32:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.012+0000 7f99360db700 1 --2- 172.21.15.2:0/2090406587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9930108740 0x7f993019f630 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9928007ee0 tx=0x7f992800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.012+0000 7f9926ffd700 1 -- 172.21.15.2:0/2090406587 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992800dbd0 con 0x7f9930108740 2024-05-14T09:32:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.012+0000 7f9926ffd700 1 -- 172.21.15.2:0/2090406587 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f992800dd70 con 0x7f9930108740 2024-05-14T09:32:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.012+0000 7f9926ffd700 1 -- 172.21.15.2:0/2090406587 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928005990 con 0x7f9930108740 2024-05-14T09:32:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.012+0000 7f9937b3e700 1 -- 172.21.15.2:0/2090406587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993019ffc0 con 0x7f9930108740 2024-05-14T09:32:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.012+0000 7f9937b3e700 1 -- 172.21.15.2:0/2090406587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99301a78e0 con 0x7f9930108740 2024-05-14T09:32:34.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.013+0000 7f9937b3e700 1 -- 172.21.15.2:0/2090406587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99300512c0 con 0x7f9930108740 2024-05-14T09:32:34.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.016+0000 7f9926ffd700 1 -- 172.21.15.2:0/2090406587 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f992801a020 con 0x7f9930108740 2024-05-14T09:32:34.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.016+0000 7f9926ffd700 1 --2- 172.21.15.2:0/2090406587 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f991c0b2990 0x7f991c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:34.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.016+0000 7f99358da700 1 --2- 172.21.15.2:0/2090406587 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f991c0b2990 0x7f991c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:34.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.016+0000 7f9926ffd700 1 -- 172.21.15.2:0/2090406587 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f99280e1e70 con 0x7f9930108740 2024-05-14T09:32:34.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.016+0000 7f99358da700 1 --2- 172.21.15.2:0/2090406587 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f991c0b2990 0x7f991c0b4ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f992c00b580 tx=0x7f992c0036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:34.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.021+0000 7f9926ffd700 1 -- 172.21.15.2:0/2090406587 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9928010070 con 0x7f9930108740 2024-05-14T09:32:34.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.306+0000 7f9937b3e700 1 -- 172.21.15.2:0/2090406587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f993002e5f0 con 0x7f9930108740 2024-05-14T09:32:34.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.312+0000 7f9926ffd700 1 -- 172.21.15.2:0/2090406587 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288635 (secure 0 0 0) 0x7f99280a9610 con 0x7f9930108740 2024-05-14T09:32:34.316 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:32:34.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.315+0000 7f9937b3e700 1 -- 172.21.15.2:0/2090406587 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f991c0b2990 msgr2=0x7f991c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.315+0000 7f9937b3e700 1 --2- 172.21.15.2:0/2090406587 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f991c0b2990 0x7f991c0b4ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f992c00b580 tx=0x7f992c0036e0 comp rx=0 tx=0).stop 2024-05-14T09:32:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.315+0000 7f9937b3e700 1 -- 172.21.15.2:0/2090406587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9930108740 msgr2=0x7f993019f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.315+0000 7f9937b3e700 1 --2- 172.21.15.2:0/2090406587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9930108740 0x7f993019f630 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9928007ee0 tx=0x7f992800d3b0 comp rx=0 tx=0).stop 2024-05-14T09:32:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.315+0000 7f9937b3e700 1 -- 172.21.15.2:0/2090406587 shutdown_connections 2024-05-14T09:32:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.315+0000 7f9937b3e700 1 --2- 172.21.15.2:0/2090406587 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f991c0b2990 0x7f991c0b4ea0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.315+0000 7f9937b3e700 1 --2- 172.21.15.2:0/2090406587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9930108740 0x7f993019f630 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.315+0000 7f9937b3e700 1 --2- 172.21.15.2:0/2090406587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f99301034c0 0x7f993019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.315+0000 7f9937b3e700 1 --2- 172.21.15.2:0/2090406587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9930107df0 0x7f993019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.315+0000 7f9937b3e700 1 -- 172.21.15.2:0/2090406587 >> 172.21.15.2:0/2090406587 conn(0x7f99300fedf0 msgr2=0x7f9930075ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.315+0000 7f9937b3e700 1 -- 172.21.15.2:0/2090406587 shutdown_connections 2024-05-14T09:32:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:34.315+0000 7f9937b3e700 1 -- 172.21.15.2:0/2090406587 wait complete. 2024-05-14T09:32:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:34 smithi002 ceph-mon[31557]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s; 81 B/s, 0 objects/s recovering 2024-05-14T09:32:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:34 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2090406587' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-14T09:32:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:34 smithi029 ceph-mon[36249]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s; 81 B/s, 0 objects/s recovering 2024-05-14T09:32:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:34 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2090406587' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-14T09:32:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:34 smithi026 ceph-mon[36056]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s; 81 B/s, 0 objects/s recovering 2024-05-14T09:32:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:34 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2090406587' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-14T09:32:35.136 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3050449557},{"type":"v1","addr":"172.21.15.2:6801","nonce":3050449557}]},"active_addr":"172.21.15.2:6801/3050449557","active_change":"2024-05-14T09:23:28.940862+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.2:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":1733118536}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":865198306}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":939147374}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":3572577192}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":797340944}]}]} 2024-05-14T09:32:35.142 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-14T09:32:35.142 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-14T09:32:35.146 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T09:32:35.399 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:36.529 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:36 smithi002 ceph-mon[31557]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s; 81 B/s, 0 objects/s recovering 2024-05-14T09:32:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:36 smithi029 ceph-mon[36249]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s; 81 B/s, 0 objects/s recovering 2024-05-14T09:32:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:36 smithi026 ceph-mon[36056]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s; 81 B/s, 0 objects/s recovering 2024-05-14T09:32:37.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.111+0000 7f73b0a7c700 1 -- 172.21.15.2:0/1243655153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73ac107e10 msgr2=0x7f73ac108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:37.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.111+0000 7f73b0a7c700 1 --2- 172.21.15.2:0/1243655153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73ac107e10 0x7f73ac108220 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f73940099c0 tx=0x7f7394009cd0 comp rx=0 tx=0).stop 2024-05-14T09:32:37.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.111+0000 7f73b0a7c700 1 -- 172.21.15.2:0/1243655153 shutdown_connections 2024-05-14T09:32:37.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.111+0000 7f73b0a7c700 1 --2- 172.21.15.2:0/1243655153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73ac1034e0 0x7f73ac1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:37.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.111+0000 7f73b0a7c700 1 --2- 172.21.15.2:0/1243655153 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73ac108760 0x7f73ac1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:37.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.111+0000 7f73b0a7c700 1 --2- 172.21.15.2:0/1243655153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73ac107e10 0x7f73ac108220 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:37.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.111+0000 7f73b0a7c700 1 -- 172.21.15.2:0/1243655153 >> 172.21.15.2:0/1243655153 conn(0x7f73ac0fef50 msgr2=0x7f73ac101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:37.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.111+0000 7f73b0a7c700 1 -- 172.21.15.2:0/1243655153 shutdown_connections 2024-05-14T09:32:37.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.111+0000 7f73b0a7c700 1 -- 172.21.15.2:0/1243655153 wait complete. 2024-05-14T09:32:37.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.112+0000 7f73b0a7c700 1 Processor -- start 2024-05-14T09:32:37.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.112+0000 7f73b0a7c700 1 -- start start 2024-05-14T09:32:37.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73b0a7c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73ac1034e0 0x7f73ac199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:37.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73b0a7c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73ac107e10 0x7f73ac19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:37.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73b0a7c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73ac108760 0x7f73ac19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:37.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73b0a7c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac113390 con 0x7f73ac1034e0 2024-05-14T09:32:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73b0a7c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac113210 con 0x7f73ac107e10 2024-05-14T09:32:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73b0a7c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac113510 con 0x7f73ac108760 2024-05-14T09:32:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73aa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73ac1034e0 0x7f73ac199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73aa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73ac1034e0 0x7f73ac199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35500/0 (socket says 172.21.15.2:35500) 2024-05-14T09:32:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73aa59c700 1 -- 172.21.15.2:0/152187228 learned_addr learned my addr 172.21.15.2:0/152187228 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73aad9d700 1 --2- 172.21.15.2:0/152187228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73ac108760 0x7f73ac19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73a1d9b700 1 --2- 172.21.15.2:0/152187228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73ac107e10 0x7f73ac19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73aa59c700 1 -- 172.21.15.2:0/152187228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73ac108760 msgr2=0x7f73ac19e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73aa59c700 1 --2- 172.21.15.2:0/152187228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73ac108760 0x7f73ac19e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73aa59c700 1 -- 172.21.15.2:0/152187228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73ac107e10 msgr2=0x7f73ac19a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73aa59c700 1 --2- 172.21.15.2:0/152187228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73ac107e10 0x7f73ac19a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73aa59c700 1 -- 172.21.15.2:0/152187228 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ac19ec10 con 0x7f73ac1034e0 2024-05-14T09:32:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.113+0000 7f73a1d9b700 1 --2- 172.21.15.2:0/152187228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73ac107e10 0x7f73ac19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:32:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.114+0000 7f73aa59c700 1 --2- 172.21.15.2:0/152187228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73ac1034e0 0x7f73ac199d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f739c00e9a0 tx=0x7f739c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.114+0000 7f73a3fff700 1 -- 172.21.15.2:0/152187228 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c00cbd0 con 0x7f73ac1034e0 2024-05-14T09:32:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.114+0000 7f73b0a7c700 1 -- 172.21.15.2:0/152187228 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ac19ef00 con 0x7f73ac1034e0 2024-05-14T09:32:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.114+0000 7f73a3fff700 1 -- 172.21.15.2:0/152187228 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f739c00cd70 con 0x7f73ac1034e0 2024-05-14T09:32:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.114+0000 7f73a3fff700 1 -- 172.21.15.2:0/152187228 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c018680 con 0x7f73ac1034e0 2024-05-14T09:32:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.114+0000 7f73b0a7c700 1 -- 172.21.15.2:0/152187228 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ac1a6820 con 0x7f73ac1034e0 2024-05-14T09:32:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.115+0000 7f73b0a7c700 1 -- 172.21.15.2:0/152187228 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ac0512c0 con 0x7f73ac1034e0 2024-05-14T09:32:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.117+0000 7f73a3fff700 1 -- 172.21.15.2:0/152187228 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f739c010430 con 0x7f73ac1034e0 2024-05-14T09:32:37.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.118+0000 7f73a3fff700 1 --2- 172.21.15.2:0/152187228 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f73980b2910 0x7f73980b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:37.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.118+0000 7f73a1d9b700 1 --2- 172.21.15.2:0/152187228 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f73980b2910 0x7f73980b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:37.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.118+0000 7f73a3fff700 1 -- 172.21.15.2:0/152187228 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f739c0e2a60 con 0x7f73ac1034e0 2024-05-14T09:32:37.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.118+0000 7f73a1d9b700 1 --2- 172.21.15.2:0/152187228 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f73980b2910 0x7f73980b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7394000c00 tx=0x7f739401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:37.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.123+0000 7f73a3fff700 1 -- 172.21.15.2:0/152187228 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f739c0aa200 con 0x7f73ac1034e0 2024-05-14T09:32:37.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.342+0000 7f73b0a7c700 1 -- 172.21.15.2:0/152187228 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73ac19b0a0 con 0x7f73ac1034e0 2024-05-14T09:32:37.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.344+0000 7f73a3fff700 1 -- 172.21.15.2:0/152187228 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24740 (secure 0 0 0) 0x7f739c0b2280 con 0x7f73ac1034e0 2024-05-14T09:32:37.344 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:32:37.344 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":90,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","created":"2024-05-14T09:22:34.937047+0000","modified":"2024-05-14T09:32:30.306958+0000","last_up_change":"2024-05-14T09:29:55.157569+0000","last_in_change":"2024-05-14T09:29:34.922562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T09:26:03.046454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T09:30:51.814350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T09:30:55.500000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T09:31:06.099087+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10d34289-414e-4b1b-bc7c-ba1dd24c7297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6803","nonce":2622184634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6805","nonce":2622184634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6809","nonce":2622184634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6807","nonce":2622184634}]},"public_addr":"172.21.15.2:6803/2622184634","cluster_addr":"172.21.15.2:6805/2622184634","heartbeat_back_addr":"172.21.15.2:6809/2622184634","heartbeat_front_addr":"172.21.15.2:6807/2622184634","state":["exists","up"]},{"osd":1,"uuid":"bc4aefc2-706e-4faf-9042-64b87ad097b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6801","nonce":3084747226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6803","nonce":3084747226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6807","nonce":3084747226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6805","nonce":3084747226}]},"public_addr":"172.21.15.26:6801/3084747226","cluster_addr":"172.21.15.26:6803/3084747226","heartbeat_back_addr":"172.21.15.26:6807/3084747226","heartbeat_front_addr":"172.21.15.26:6805/3084747226","state":["exists","up"]},{"osd":2,"uuid":"8c5bb589-56e2-469f-9a6f-6035b3ddf4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6801","nonce":814048086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6803","nonce":814048086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6807","nonce":814048086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6805","nonce":814048086}]},"public_addr":"172.21.15.29:6801/814048086","cluster_addr":"172.21.15.29:6803/814048086","heartbeat_back_addr":"172.21.15.29:6807/814048086","heartbeat_front_addr":"172.21.15.29:6805/814048086","state":["exists","up"]},{"osd":3,"uuid":"8fbfc078-85e7-4dc2-acda-5e7b0ab927e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6811","nonce":879639451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6813","nonce":879639451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6817","nonce":879639451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6815","nonce":879639451}]},"public_addr":"172.21.15.2:6811/879639451","cluster_addr":"172.21.15.2:6813/879639451","heartbeat_back_addr":"172.21.15.2:6817/879639451","heartbeat_front_addr":"172.21.15.2:6815/879639451","state":["exists","up"]},{"osd":4,"uuid":"f2515448-e9c1-48a2-b270-1e5e781a2fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6809","nonce":457100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6811","nonce":457100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6815","nonce":457100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6813","nonce":457100752}]},"public_addr":"172.21.15.26:6809/457100752","cluster_addr":"172.21.15.26:6811/457100752","heartbeat_back_addr":"172.21.15.26:6815/457100752","heartbeat_front_addr":"172.21.15.26:6813/457100752","state":["exists","up"]},{"osd":5,"uuid":"20030857-a1e6-4b0a-b186-1cfa44f2e4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6809","nonce":1554045093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6811","nonce":1554045093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6815","nonce":1554045093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6813","nonce":1554045093}]},"public_addr":"172.21.15.29:6809/1554045093","cluster_addr":"172.21.15.29:6811/1554045093","heartbeat_back_addr":"172.21.15.29:6815/1554045093","heartbeat_front_addr":"172.21.15.29:6813/1554045093","state":["exists","up"]},{"osd":6,"uuid":"d757c72a-29f0-4f14-9f13-a2e61eb5e6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6819","nonce":2749897595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6821","nonce":2749897595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6825","nonce":2749897595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6823","nonce":2749897595}]},"public_addr":"172.21.15.2:6819/2749897595","cluster_addr":"172.21.15.2:6821/2749897595","heartbeat_back_addr":"172.21.15.2:6825/2749897595","heartbeat_front_addr":"172.21.15.2:6823/2749897595","state":["exists","up"]},{"osd":7,"uuid":"22ddef80-3633-44c5-bc0e-edc75b546ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6817","nonce":1180401611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6819","nonce":1180401611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6823","nonce":1180401611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6821","nonce":1180401611}]},"public_addr":"172.21.15.26:6817/1180401611","cluster_addr":"172.21.15.26:6819/1180401611","heartbeat_back_addr":"172.21.15.26:6823/1180401611","heartbeat_front_addr":"172.21.15.26:6821/1180401611","state":["exists","up"]},{"osd":8,"uuid":"466e7ac4-3ff7-4254-a3fd-8096420b1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6817","nonce":2858408446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6819","nonce":2858408446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6823","nonce":2858408446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6821","nonce":2858408446}]},"public_addr":"172.21.15.29:6817/2858408446","cluster_addr":"172.21.15.29:6819/2858408446","heartbeat_back_addr":"172.21.15.29:6823/2858408446","heartbeat_front_addr":"172.21.15.29:6821/2858408446","state":["exists","up"]},{"osd":9,"uuid":"119eeac9-29f7-4086-a1ad-04a0bc154b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6827","nonce":176898226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6829","nonce":176898226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6833","nonce":176898226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6831","nonce":176898226}]},"public_addr":"172.21.15.2:6827/176898226","cluster_addr":"172.21.15.2:6829/176898226","heartbeat_back_addr":"172.21.15.2:6833/176898226","heartbeat_front_addr":"172.21.15.2:6831/176898226","state":["exists","up"]},{"osd":10,"uuid":"55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6825","nonce":4065159515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6827","nonce":4065159515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6831","nonce":4065159515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6829","nonce":4065159515}]},"public_addr":"172.21.15.26:6825/4065159515","cluster_addr":"172.21.15.26:6827/4065159515","heartbeat_back_addr":"172.21.15.26:6831/4065159515","heartbeat_front_addr":"172.21.15.26:6829/4065159515","state":["exists","up"]},{"osd":11,"uuid":"c5246763-c4f6-4ae3-8a93-5929c541ac5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6825","nonce":378800417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6827","nonce":378800417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6831","nonce":378800417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6829","nonce":378800417}]},"public_addr":"172.21.15.29:6825/378800417","cluster_addr":"172.21.15.29:6827/378800417","heartbeat_back_addr":"172.21.15.29:6831/378800417","heartbeat_front_addr":"172.21.15.29:6829/378800417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:09.342387+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:34.156359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:58.860653+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:24.009323+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:49.057146+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:15.007172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:41.085748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:06.674923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:33.409128+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:00.106186+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:26.803487+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:53.668283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11},{"from":8,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9},{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4051922409":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/2353790181":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/4268171474":"2024-05-15T09:23:28.940670+0000","172.21.15.2:6801/1423742578":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/969850441":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3252991268":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6801/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3789067476":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/1423742578":"2024-05-15T09:23:28.940670+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T09:32:37.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.346+0000 7f73b0a7c700 1 -- 172.21.15.2:0/152187228 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f73980b2910 msgr2=0x7f73980b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:37.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.346+0000 7f73b0a7c700 1 --2- 172.21.15.2:0/152187228 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f73980b2910 0x7f73980b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7394000c00 tx=0x7f739401a040 comp rx=0 tx=0).stop 2024-05-14T09:32:37.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.346+0000 7f73b0a7c700 1 -- 172.21.15.2:0/152187228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73ac1034e0 msgr2=0x7f73ac199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:37.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.346+0000 7f73b0a7c700 1 --2- 172.21.15.2:0/152187228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73ac1034e0 0x7f73ac199d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f739c00e9a0 tx=0x7f739c00ecb0 comp rx=0 tx=0).stop 2024-05-14T09:32:37.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.346+0000 7f73b0a7c700 1 -- 172.21.15.2:0/152187228 shutdown_connections 2024-05-14T09:32:37.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.346+0000 7f73b0a7c700 1 --2- 172.21.15.2:0/152187228 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f73980b2910 0x7f73980b4e20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:37.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.346+0000 7f73b0a7c700 1 --2- 172.21.15.2:0/152187228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f73ac108760 0x7f73ac19e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:37.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.346+0000 7f73b0a7c700 1 --2- 172.21.15.2:0/152187228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73ac1034e0 0x7f73ac199d40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:37.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.346+0000 7f73b0a7c700 1 --2- 172.21.15.2:0/152187228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73ac107e10 0x7f73ac19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:37.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.346+0000 7f73b0a7c700 1 -- 172.21.15.2:0/152187228 >> 172.21.15.2:0/152187228 conn(0x7f73ac0fef50 msgr2=0x7f73ac107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:37.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.346+0000 7f73b0a7c700 1 -- 172.21.15.2:0/152187228 shutdown_connections 2024-05-14T09:32:37.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:37.346+0000 7f73b0a7c700 1 -- 172.21.15.2:0/152187228 wait complete. 2024-05-14T09:32:38.129 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-14T09:32:38.130 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T09:32:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:38.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:38 smithi002 ceph-mon[31557]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s; 81 B/s, 0 objects/s recovering 2024-05-14T09:32:38.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:38 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/152187228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:32:38.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:38 smithi029 ceph-mon[36249]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s; 81 B/s, 0 objects/s recovering 2024-05-14T09:32:38.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:38 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/152187228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:32:38.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:38 smithi026 ceph-mon[36056]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s; 81 B/s, 0 objects/s recovering 2024-05-14T09:32:38.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:38 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/152187228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:32:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.258+0000 7f2cabfde700 1 -- 172.21.15.2:0/3134401368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ca41045f0 msgr2=0x7f2ca4104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.258+0000 7f2cabfde700 1 --2- 172.21.15.2:0/3134401368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ca41045f0 0x7f2ca4104a00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00099c0 tx=0x7f2ca0009cd0 comp rx=0 tx=0).stop 2024-05-14T09:32:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.258+0000 7f2cabfde700 1 -- 172.21.15.2:0/3134401368 shutdown_connections 2024-05-14T09:32:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.258+0000 7f2cabfde700 1 --2- 172.21.15.2:0/3134401368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca4109870 0x7f2ca41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.258+0000 7f2cabfde700 1 --2- 172.21.15.2:0/3134401368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ca41045f0 0x7f2ca4104a00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.259+0000 7f2cabfde700 1 --2- 172.21.15.2:0/3134401368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca4108f20 0x7f2ca4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.259+0000 7f2cabfde700 1 -- 172.21.15.2:0/3134401368 >> 172.21.15.2:0/3134401368 conn(0x7f2ca4100040 msgr2=0x7f2ca4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.259+0000 7f2cabfde700 1 -- 172.21.15.2:0/3134401368 shutdown_connections 2024-05-14T09:32:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.259+0000 7f2cabfde700 1 -- 172.21.15.2:0/3134401368 wait complete. 2024-05-14T09:32:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.259+0000 7f2cabfde700 1 Processor -- start 2024-05-14T09:32:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2cabfde700 1 -- start start 2024-05-14T09:32:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2cabfde700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca41045f0 0x7f2ca4075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2cabfde700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca4108f20 0x7f2ca4076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2cabfde700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ca4109870 0x7f2ca4076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2cabfde700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41144a0 con 0x7f2ca4109870 2024-05-14T09:32:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2cabfde700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca4114320 con 0x7f2ca41045f0 2024-05-14T09:32:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2cabfde700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca4114620 con 0x7f2ca4108f20 2024-05-14T09:32:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2caa57b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ca4109870 0x7f2ca4076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2ca9d7a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca41045f0 0x7f2ca4075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2caa57b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ca4109870 0x7f2ca4076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37616/0 (socket says 172.21.15.2:37616) 2024-05-14T09:32:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2ca9d7a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca41045f0 0x7f2ca4075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:44904/0 (socket says 172.21.15.2:44904) 2024-05-14T09:32:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2caa57b700 1 -- 172.21.15.2:0/2720766557 learned_addr learned my addr 172.21.15.2:0/2720766557 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2ca9579700 1 --2- 172.21.15.2:0/2720766557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca4108f20 0x7f2ca4076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2caa57b700 1 -- 172.21.15.2:0/2720766557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca4108f20 msgr2=0x7f2ca4076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2caa57b700 1 --2- 172.21.15.2:0/2720766557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca4108f20 0x7f2ca4076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2caa57b700 1 -- 172.21.15.2:0/2720766557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca41045f0 msgr2=0x7f2ca4075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.260+0000 7f2caa57b700 1 --2- 172.21.15.2:0/2720766557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca41045f0 0x7f2ca4075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.261+0000 7f2caa57b700 1 -- 172.21.15.2:0/2720766557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca41a9210 con 0x7f2ca4109870 2024-05-14T09:32:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.261+0000 7f2ca9d7a700 1 --2- 172.21.15.2:0/2720766557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca41045f0 0x7f2ca4075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:32:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.261+0000 7f2caa57b700 1 --2- 172.21.15.2:0/2720766557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ca4109870 0x7f2ca4076540 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00ef30 tx=0x7f2c9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.261+0000 7f2c9affd700 1 -- 172.21.15.2:0/2720766557 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0092e0 con 0x7f2ca4109870 2024-05-14T09:32:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.261+0000 7f2cabfde700 1 -- 172.21.15.2:0/2720766557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca41a94a0 con 0x7f2ca4109870 2024-05-14T09:32:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.261+0000 7f2c9affd700 1 -- 172.21.15.2:0/2720766557 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c9c004bb0 con 0x7f2ca4109870 2024-05-14T09:32:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.261+0000 7f2c9affd700 1 -- 172.21.15.2:0/2720766557 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c004020 con 0x7f2ca4109870 2024-05-14T09:32:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.261+0000 7f2cabfde700 1 -- 172.21.15.2:0/2720766557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca41a9990 con 0x7f2ca4109870 2024-05-14T09:32:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.262+0000 7f2cabfde700 1 -- 172.21.15.2:0/2720766557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca40512c0 con 0x7f2ca4109870 2024-05-14T09:32:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.264+0000 7f2c9affd700 1 -- 172.21.15.2:0/2720766557 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f2c9c01b030 con 0x7f2ca4109870 2024-05-14T09:32:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.265+0000 7f2c9affd700 1 --2- 172.21.15.2:0/2720766557 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2c900b2910 0x7f2c900b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.265+0000 7f2ca9d7a700 1 --2- 172.21.15.2:0/2720766557 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2c900b2910 0x7f2c900b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.265+0000 7f2c9affd700 1 -- 172.21.15.2:0/2720766557 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f2c9c0e2be0 con 0x7f2ca4109870 2024-05-14T09:32:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.265+0000 7f2ca9d7a700 1 --2- 172.21.15.2:0/2720766557 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2c900b2910 0x7f2c900b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0000c00 tx=0x7f2ca001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:40.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.268+0000 7f2c9affd700 1 -- 172.21.15.2:0/2720766557 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c9c022020 con 0x7f2ca4109870 2024-05-14T09:32:40.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.486+0000 7f2cabfde700 1 -- 172.21.15.2:0/2720766557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ca402e7b0 con 0x7f2ca4109870 2024-05-14T09:32:40.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.488+0000 7f2c9affd700 1 -- 172.21.15.2:0/2720766557 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24740 (secure 0 0 0) 0x7f2c9c010040 con 0x7f2ca4109870 2024-05-14T09:32:40.488 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:32:40.488 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":90,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","created":"2024-05-14T09:22:34.937047+0000","modified":"2024-05-14T09:32:30.306958+0000","last_up_change":"2024-05-14T09:29:55.157569+0000","last_in_change":"2024-05-14T09:29:34.922562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T09:26:03.046454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T09:30:51.814350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T09:30:55.500000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T09:31:06.099087+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10d34289-414e-4b1b-bc7c-ba1dd24c7297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6803","nonce":2622184634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6805","nonce":2622184634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6809","nonce":2622184634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6807","nonce":2622184634}]},"public_addr":"172.21.15.2:6803/2622184634","cluster_addr":"172.21.15.2:6805/2622184634","heartbeat_back_addr":"172.21.15.2:6809/2622184634","heartbeat_front_addr":"172.21.15.2:6807/2622184634","state":["exists","up"]},{"osd":1,"uuid":"bc4aefc2-706e-4faf-9042-64b87ad097b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6801","nonce":3084747226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6803","nonce":3084747226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6807","nonce":3084747226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6805","nonce":3084747226}]},"public_addr":"172.21.15.26:6801/3084747226","cluster_addr":"172.21.15.26:6803/3084747226","heartbeat_back_addr":"172.21.15.26:6807/3084747226","heartbeat_front_addr":"172.21.15.26:6805/3084747226","state":["exists","up"]},{"osd":2,"uuid":"8c5bb589-56e2-469f-9a6f-6035b3ddf4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6801","nonce":814048086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6803","nonce":814048086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6807","nonce":814048086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6805","nonce":814048086}]},"public_addr":"172.21.15.29:6801/814048086","cluster_addr":"172.21.15.29:6803/814048086","heartbeat_back_addr":"172.21.15.29:6807/814048086","heartbeat_front_addr":"172.21.15.29:6805/814048086","state":["exists","up"]},{"osd":3,"uuid":"8fbfc078-85e7-4dc2-acda-5e7b0ab927e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6811","nonce":879639451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6813","nonce":879639451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6817","nonce":879639451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6815","nonce":879639451}]},"public_addr":"172.21.15.2:6811/879639451","cluster_addr":"172.21.15.2:6813/879639451","heartbeat_back_addr":"172.21.15.2:6817/879639451","heartbeat_front_addr":"172.21.15.2:6815/879639451","state":["exists","up"]},{"osd":4,"uuid":"f2515448-e9c1-48a2-b270-1e5e781a2fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6809","nonce":457100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6811","nonce":457100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6815","nonce":457100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6813","nonce":457100752}]},"public_addr":"172.21.15.26:6809/457100752","cluster_addr":"172.21.15.26:6811/457100752","heartbeat_back_addr":"172.21.15.26:6815/457100752","heartbeat_front_addr":"172.21.15.26:6813/457100752","state":["exists","up"]},{"osd":5,"uuid":"20030857-a1e6-4b0a-b186-1cfa44f2e4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6809","nonce":1554045093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6811","nonce":1554045093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6815","nonce":1554045093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6813","nonce":1554045093}]},"public_addr":"172.21.15.29:6809/1554045093","cluster_addr":"172.21.15.29:6811/1554045093","heartbeat_back_addr":"172.21.15.29:6815/1554045093","heartbeat_front_addr":"172.21.15.29:6813/1554045093","state":["exists","up"]},{"osd":6,"uuid":"d757c72a-29f0-4f14-9f13-a2e61eb5e6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6819","nonce":2749897595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6821","nonce":2749897595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6825","nonce":2749897595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6823","nonce":2749897595}]},"public_addr":"172.21.15.2:6819/2749897595","cluster_addr":"172.21.15.2:6821/2749897595","heartbeat_back_addr":"172.21.15.2:6825/2749897595","heartbeat_front_addr":"172.21.15.2:6823/2749897595","state":["exists","up"]},{"osd":7,"uuid":"22ddef80-3633-44c5-bc0e-edc75b546ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6817","nonce":1180401611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6819","nonce":1180401611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6823","nonce":1180401611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6821","nonce":1180401611}]},"public_addr":"172.21.15.26:6817/1180401611","cluster_addr":"172.21.15.26:6819/1180401611","heartbeat_back_addr":"172.21.15.26:6823/1180401611","heartbeat_front_addr":"172.21.15.26:6821/1180401611","state":["exists","up"]},{"osd":8,"uuid":"466e7ac4-3ff7-4254-a3fd-8096420b1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6817","nonce":2858408446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6819","nonce":2858408446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6823","nonce":2858408446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6821","nonce":2858408446}]},"public_addr":"172.21.15.29:6817/2858408446","cluster_addr":"172.21.15.29:6819/2858408446","heartbeat_back_addr":"172.21.15.29:6823/2858408446","heartbeat_front_addr":"172.21.15.29:6821/2858408446","state":["exists","up"]},{"osd":9,"uuid":"119eeac9-29f7-4086-a1ad-04a0bc154b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6827","nonce":176898226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6829","nonce":176898226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6833","nonce":176898226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6831","nonce":176898226}]},"public_addr":"172.21.15.2:6827/176898226","cluster_addr":"172.21.15.2:6829/176898226","heartbeat_back_addr":"172.21.15.2:6833/176898226","heartbeat_front_addr":"172.21.15.2:6831/176898226","state":["exists","up"]},{"osd":10,"uuid":"55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6825","nonce":4065159515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6827","nonce":4065159515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6831","nonce":4065159515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6829","nonce":4065159515}]},"public_addr":"172.21.15.26:6825/4065159515","cluster_addr":"172.21.15.26:6827/4065159515","heartbeat_back_addr":"172.21.15.26:6831/4065159515","heartbeat_front_addr":"172.21.15.26:6829/4065159515","state":["exists","up"]},{"osd":11,"uuid":"c5246763-c4f6-4ae3-8a93-5929c541ac5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6825","nonce":378800417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6827","nonce":378800417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6831","nonce":378800417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6829","nonce":378800417}]},"public_addr":"172.21.15.29:6825/378800417","cluster_addr":"172.21.15.29:6827/378800417","heartbeat_back_addr":"172.21.15.29:6831/378800417","heartbeat_front_addr":"172.21.15.29:6829/378800417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:09.342387+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:34.156359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:58.860653+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:24.009323+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:49.057146+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:15.007172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:41.085748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:06.674923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:33.409128+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:00.106186+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:26.803487+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:53.668283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11},{"from":8,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9},{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4051922409":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/2353790181":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/4268171474":"2024-05-15T09:23:28.940670+0000","172.21.15.2:6801/1423742578":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/969850441":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3252991268":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6801/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3789067476":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/1423742578":"2024-05-15T09:23:28.940670+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T09:32:40.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.490+0000 7f2cabfde700 1 -- 172.21.15.2:0/2720766557 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2c900b2910 msgr2=0x7f2c900b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:40.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.490+0000 7f2cabfde700 1 --2- 172.21.15.2:0/2720766557 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2c900b2910 0x7f2c900b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0000c00 tx=0x7f2ca001a040 comp rx=0 tx=0).stop 2024-05-14T09:32:40.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.490+0000 7f2cabfde700 1 -- 172.21.15.2:0/2720766557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ca4109870 msgr2=0x7f2ca4076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:40.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.490+0000 7f2cabfde700 1 --2- 172.21.15.2:0/2720766557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ca4109870 0x7f2ca4076540 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00ef30 tx=0x7f2c9c00c3a0 comp rx=0 tx=0).stop 2024-05-14T09:32:40.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.490+0000 7f2cabfde700 1 -- 172.21.15.2:0/2720766557 shutdown_connections 2024-05-14T09:32:40.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.490+0000 7f2cabfde700 1 --2- 172.21.15.2:0/2720766557 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2c900b2910 0x7f2c900b4e20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:40.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.490+0000 7f2cabfde700 1 --2- 172.21.15.2:0/2720766557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ca41045f0 0x7f2ca4075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:40.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.490+0000 7f2cabfde700 1 --2- 172.21.15.2:0/2720766557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ca4109870 0x7f2ca4076540 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:40.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.490+0000 7f2cabfde700 1 --2- 172.21.15.2:0/2720766557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2ca4108f20 0x7f2ca4076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:40.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.490+0000 7f2cabfde700 1 -- 172.21.15.2:0/2720766557 >> 172.21.15.2:0/2720766557 conn(0x7f2ca4100040 msgr2=0x7f2ca4108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:40.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.491+0000 7f2cabfde700 1 -- 172.21.15.2:0/2720766557 shutdown_connections 2024-05-14T09:32:40.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:40.491+0000 7f2cabfde700 1 -- 172.21.15.2:0/2720766557 wait complete. 2024-05-14T09:32:40.582 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:40 smithi002 ceph-mon[31557]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 66 B/s, 0 objects/s recovering 2024-05-14T09:32:40.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:40 smithi029 ceph-mon[36249]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 66 B/s, 0 objects/s recovering 2024-05-14T09:32:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:40 smithi026 ceph-mon[36056]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 66 B/s, 0 objects/s recovering 2024-05-14T09:32:41.218 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-14T09:32:41.218 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-14T09:32:41.219 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-14T09:32:41.219 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-14T09:32:41.219 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-14T09:32:41.220 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-14T09:32:41.220 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-14T09:32:41.221 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-14T09:32:41.221 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-14T09:32:41.222 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-14T09:32:41.222 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-14T09:32:41.222 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-14T09:32:41.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:41 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2720766557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:32:41.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:41 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2720766557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:32:41.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:41 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2720766557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:32:41.954 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:41.955 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:41.958 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:41.959 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:41.963 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:41.969 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:41.977 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:41.978 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:41.986 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:42.002 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:42.008 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:42.049 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:32:42.391 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:42 smithi002 ceph-mon[31557]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 60 B/s, 0 objects/s recovering 2024-05-14T09:32:42.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:42 smithi029 ceph-mon[36249]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 60 B/s, 0 objects/s recovering 2024-05-14T09:32:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:42 smithi026 ceph-mon[36056]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 60 B/s, 0 objects/s recovering 2024-05-14T09:32:44.634 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:44 smithi002 ceph-mon[31557]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 54 B/s, 0 objects/s recovering 2024-05-14T09:32:44.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:44 smithi029 ceph-mon[36249]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 54 B/s, 0 objects/s recovering 2024-05-14T09:32:44.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:44 smithi026 ceph-mon[36056]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 54 B/s, 0 objects/s recovering 2024-05-14T09:32:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:46 smithi002 ceph-mon[31557]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:32:46.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:46 smithi029 ceph-mon[36249]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:32:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:46 smithi026 ceph-mon[36056]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:32:48.627 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:48 smithi002 ceph-mon[31557]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:32:48.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:48 smithi029 ceph-mon[36249]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:32:48.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:48 smithi026 ceph-mon[36056]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:32:50.661 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:50 smithi002 ceph-mon[31557]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:32:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:50 smithi029 ceph-mon[36249]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:32:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:50 smithi026 ceph-mon[36056]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:32:52.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:52 smithi002 ceph-mon[31557]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:32:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:52 smithi029 ceph-mon[36249]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:32:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:52 smithi026 ceph-mon[36056]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:32:53.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.209+0000 7ff55d8d3700 1 -- 172.21.15.2:0/1443882417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff558107df0 msgr2=0x7ff558108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:53.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.209+0000 7ff55d8d3700 1 --2- 172.21.15.2:0/1443882417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff558107df0 0x7ff558108200 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff5400099c0 tx=0x7ff540009cd0 comp rx=0 tx=0).stop 2024-05-14T09:32:53.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.209+0000 7ff55d8d3700 1 -- 172.21.15.2:0/1443882417 shutdown_connections 2024-05-14T09:32:53.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.209+0000 7ff55d8d3700 1 --2- 172.21.15.2:0/1443882417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5581034c0 0x7ff5581038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:53.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.209+0000 7ff55d8d3700 1 --2- 172.21.15.2:0/1443882417 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff558108740 0x7ff5581100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:53.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.209+0000 7ff55d8d3700 1 --2- 172.21.15.2:0/1443882417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff558107df0 0x7ff558108200 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:53.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.209+0000 7ff55d8d3700 1 -- 172.21.15.2:0/1443882417 >> 172.21.15.2:0/1443882417 conn(0x7ff5580fedf0 msgr2=0x7ff558101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:53.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.209+0000 7ff55d8d3700 1 -- 172.21.15.2:0/1443882417 shutdown_connections 2024-05-14T09:32:53.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.209+0000 7ff55d8d3700 1 -- 172.21.15.2:0/1443882417 wait complete. 2024-05-14T09:32:53.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.209+0000 7ff55d8d3700 1 Processor -- start 2024-05-14T09:32:53.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff55d8d3700 1 -- start start 2024-05-14T09:32:53.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff55d8d3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5581034c0 0x7ff55819bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:53.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff55d8d3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff558107df0 0x7ff55819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:53.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff55d8d3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff558108740 0x7ff55819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:53.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff55d8d3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff5581133a0 con 0x7ff5581034c0 2024-05-14T09:32:53.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff55d8d3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff558113220 con 0x7ff558108740 2024-05-14T09:32:53.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff55d8d3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff558113520 con 0x7ff558107df0 2024-05-14T09:32:53.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff556ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5581034c0 0x7ff55819bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:53.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff556ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5581034c0 0x7ff55819bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45694/0 (socket says 172.21.15.2:45694) 2024-05-14T09:32:53.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff556ffd700 1 -- 172.21.15.2:0/709858905 learned_addr learned my addr 172.21.15.2:0/709858905 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:53.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff5577fe700 1 --2- 172.21.15.2:0/709858905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff558108740 0x7ff55819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:53.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff54e7fc700 1 --2- 172.21.15.2:0/709858905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff558107df0 0x7ff55819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:53.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff556ffd700 1 -- 172.21.15.2:0/709858905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff558107df0 msgr2=0x7ff55819c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:53.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff556ffd700 1 --2- 172.21.15.2:0/709858905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff558107df0 0x7ff55819c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:53.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff556ffd700 1 -- 172.21.15.2:0/709858905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff558108740 msgr2=0x7ff55819c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff556ffd700 1 --2- 172.21.15.2:0/709858905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff558108740 0x7ff55819c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff556ffd700 1 -- 172.21.15.2:0/709858905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5581a7010 con 0x7ff5581034c0 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.210+0000 7ff54e7fc700 1 --2- 172.21.15.2:0/709858905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff558107df0 0x7ff55819c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.211+0000 7ff556ffd700 1 --2- 172.21.15.2:0/709858905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5581034c0 0x7ff55819bca0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff54800d950 tx=0x7ff54800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.211+0000 7ff55c8d1700 1 -- 172.21.15.2:0/709858905 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff548016070 con 0x7ff5581034c0 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.211+0000 7ff55d8d3700 1 -- 172.21.15.2:0/709858905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5581a7300 con 0x7ff5581034c0 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.211+0000 7ff55c8d1700 1 -- 172.21.15.2:0/709858905 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff548010460 con 0x7ff5581034c0 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.211+0000 7ff55c8d1700 1 -- 172.21.15.2:0/709858905 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54800f620 con 0x7ff5581034c0 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.211+0000 7ff55d8d3700 1 -- 172.21.15.2:0/709858905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5581a7890 con 0x7ff5581034c0 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.212+0000 7ff55d8d3700 1 -- 172.21.15.2:0/709858905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff558063e90 con 0x7ff5581034c0 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.213+0000 7ff55c8d1700 1 -- 172.21.15.2:0/709858905 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7ff548019070 con 0x7ff5581034c0 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.214+0000 7ff55c8d1700 1 --2- 172.21.15.2:0/709858905 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff5440b2910 0x7ff5440b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.214+0000 7ff55c8d1700 1 -- 172.21.15.2:0/709858905 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7ff5480e2980 con 0x7ff5581034c0 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.214+0000 7ff54e7fc700 1 --2- 172.21.15.2:0/709858905 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff5440b2910 0x7ff5440b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.215+0000 7ff55c8d1700 1 --2- 172.21.15.2:0/709858905 >> [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446] conn(0x7ff5440bddc0 0x7ff5440c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:53.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.215+0000 7ff55c8d1700 1 -- 172.21.15.2:0/709858905 --> [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5440c0980 con 0x7ff5440bddc0 2024-05-14T09:32:53.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.215+0000 7ff55c8d1700 1 -- 172.21.15.2:0/709858905 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff5480e2dd0 con 0x7ff5581034c0 2024-05-14T09:32:53.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.215+0000 7ff54e7fc700 1 --2- 172.21.15.2:0/709858905 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff5440b2910 0x7ff5440b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff540003800 tx=0x7ff540000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:53.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.215+0000 7ff5577fe700 1 --2- 172.21.15.2:0/709858905 >> [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446] conn(0x7ff5440bddc0 0x7ff5440c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:53.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.215+0000 7ff5577fe700 1 --2- 172.21.15.2:0/709858905 >> [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446] conn(0x7ff5440bddc0 0x7ff5440c02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:53.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.216+0000 7ff55c8d1700 1 -- 172.21.15.2:0/709858905 <== osd.8 v2:172.21.15.29:6816/2858408446 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff5440c0980 con 0x7ff5440bddc0 2024-05-14T09:32:53.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.233+0000 7ff55d8d3700 1 -- 172.21.15.2:0/709858905 --> [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff5580642c0 con 0x7ff5440bddc0 2024-05-14T09:32:53.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.234+0000 7ff55c8d1700 1 -- 172.21.15.2:0/709858905 <== osd.8 v2:172.21.15.29:6816/2858408446 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff5580642c0 con 0x7ff5440bddc0 2024-05-14T09:32:53.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.234+0000 7ff55d8d3700 1 -- 172.21.15.2:0/709858905 >> [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446] conn(0x7ff5440bddc0 msgr2=0x7ff5440c02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:53.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.234+0000 7ff55d8d3700 1 --2- 172.21.15.2:0/709858905 >> [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446] conn(0x7ff5440bddc0 0x7ff5440c02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:53.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.234+0000 7ff55d8d3700 1 -- 172.21.15.2:0/709858905 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff5440b2910 msgr2=0x7ff5440b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:53.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.234+0000 7ff55d8d3700 1 --2- 172.21.15.2:0/709858905 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff5440b2910 0x7ff5440b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff540003800 tx=0x7ff540000bc0 comp rx=0 tx=0).stop 2024-05-14T09:32:53.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.234+0000 7ff55d8d3700 1 -- 172.21.15.2:0/709858905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5581034c0 msgr2=0x7ff55819bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:53.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.234+0000 7ff55d8d3700 1 --2- 172.21.15.2:0/709858905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5581034c0 0x7ff55819bca0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff54800d950 tx=0x7ff54800dd10 comp rx=0 tx=0).stop 2024-05-14T09:32:53.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.235+0000 7ff5577fe700 1 -- 172.21.15.2:0/709858905 reap_dead start 2024-05-14T09:32:53.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.235+0000 7ff55d8d3700 1 -- 172.21.15.2:0/709858905 shutdown_connections 2024-05-14T09:32:53.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.235+0000 7ff55d8d3700 1 -- 172.21.15.2:0/709858905 >> 172.21.15.2:0/709858905 conn(0x7ff5580fedf0 msgr2=0x7ff5581096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:53.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.235+0000 7ff55d8d3700 1 -- 172.21.15.2:0/709858905 shutdown_connections 2024-05-14T09:32:53.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:50.235+0000 7ff55d8d3700 1 -- 172.21.15.2:0/709858905 wait complete. 2024-05-14T09:32:53.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.990+0000 7fcb4199d700 1 -- 172.21.15.2:0/2846212037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb3c1034c0 msgr2=0x7fcb3c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:53.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.990+0000 7fcb4199d700 1 --2- 172.21.15.2:0/2846212037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb3c1034c0 0x7fcb3c1038d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0099c0 tx=0x7fcb2c009cd0 comp rx=0 tx=0).stop 2024-05-14T09:32:53.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.990+0000 7fcb4199d700 1 -- 172.21.15.2:0/2846212037 shutdown_connections 2024-05-14T09:32:53.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.990+0000 7fcb4199d700 1 --2- 172.21.15.2:0/2846212037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c108740 0x7fcb3c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:53.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.990+0000 7fcb4199d700 1 --2- 172.21.15.2:0/2846212037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb3c1034c0 0x7fcb3c1038d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:53.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.990+0000 7fcb4199d700 1 --2- 172.21.15.2:0/2846212037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb3c107df0 0x7fcb3c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:53.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.990+0000 7fcb4199d700 1 -- 172.21.15.2:0/2846212037 >> 172.21.15.2:0/2846212037 conn(0x7fcb3c0fedf0 msgr2=0x7fcb3c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:53.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.990+0000 7fcb4199d700 1 -- 172.21.15.2:0/2846212037 shutdown_connections 2024-05-14T09:32:53.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.990+0000 7fcb4199d700 1 -- 172.21.15.2:0/2846212037 wait complete. 2024-05-14T09:32:53.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.991+0000 7fcb4199d700 1 Processor -- start 2024-05-14T09:32:53.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.991+0000 7fcb4199d700 1 -- start start 2024-05-14T09:32:53.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.991+0000 7fcb4199d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb3c1034c0 0x7fcb3c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:53.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb4199d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb3c107df0 0x7fcb3c19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:53.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb4199d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c108740 0x7fcb3c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:53.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb4199d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c113370 con 0x7fcb3c1034c0 2024-05-14T09:32:53.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb4199d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c1131f0 con 0x7fcb3c108740 2024-05-14T09:32:53.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb4199d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c1134f0 con 0x7fcb3c107df0 2024-05-14T09:32:53.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb3affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb3c1034c0 0x7fcb3c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:53.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb3affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb3c1034c0 0x7fcb3c19ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45726/0 (socket says 172.21.15.2:45726) 2024-05-14T09:32:53.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb3affd700 1 -- 172.21.15.2:0/3643664749 learned_addr learned my addr 172.21.15.2:0/3643664749 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:53.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb327fc700 1 --2- 172.21.15.2:0/3643664749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb3c107df0 0x7fcb3c19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:53.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb3b7fe700 1 --2- 172.21.15.2:0/3643664749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c108740 0x7fcb3c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:53.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb3affd700 1 -- 172.21.15.2:0/3643664749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb3c107df0 msgr2=0x7fcb3c19b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:53.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb3affd700 1 --2- 172.21.15.2:0/3643664749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcb3c107df0 0x7fcb3c19b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:53.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb3affd700 1 -- 172.21.15.2:0/3643664749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c108740 msgr2=0x7fcb3c19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:53.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb3affd700 1 --2- 172.21.15.2:0/3643664749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb3c108740 0x7fcb3c19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:53.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb3affd700 1 -- 172.21.15.2:0/3643664749 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3c19fc50 con 0x7fcb3c1034c0 2024-05-14T09:32:53.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.992+0000 7fcb3affd700 1 --2- 172.21.15.2:0/3643664749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb3c1034c0 0x7fcb3c19ae30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0098e0 tx=0x7fcb2c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:53.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.993+0000 7fcb4099b700 1 -- 172.21.15.2:0/3643664749 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c027070 con 0x7fcb3c1034c0 2024-05-14T09:32:53.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.993+0000 7fcb4099b700 1 -- 172.21.15.2:0/3643664749 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb2c003f20 con 0x7fcb3c1034c0 2024-05-14T09:32:53.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.993+0000 7fcb4099b700 1 -- 172.21.15.2:0/3643664749 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c022ca0 con 0x7fcb3c1034c0 2024-05-14T09:32:53.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.993+0000 7fcb4199d700 1 -- 172.21.15.2:0/3643664749 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3c19fee0 con 0x7fcb3c1034c0 2024-05-14T09:32:53.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.993+0000 7fcb4199d700 1 -- 172.21.15.2:0/3643664749 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3c1a03c0 con 0x7fcb3c1034c0 2024-05-14T09:32:53.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.994+0000 7fcb4199d700 1 -- 172.21.15.2:0/3643664749 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcb3c063f50 con 0x7fcb3c1034c0 2024-05-14T09:32:53.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.995+0000 7fcb4099b700 1 -- 172.21.15.2:0/3643664749 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fcb2c02a070 con 0x7fcb3c1034c0 2024-05-14T09:32:53.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.996+0000 7fcb4099b700 1 --2- 172.21.15.2:0/3643664749 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcb280b2910 0x7fcb280b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:53.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.996+0000 7fcb327fc700 1 --2- 172.21.15.2:0/3643664749 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcb280b2910 0x7fcb280b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:53.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.996+0000 7fcb4099b700 1 -- 172.21.15.2:0/3643664749 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fcb2c0e43d0 con 0x7fcb3c1034c0 2024-05-14T09:32:53.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.996+0000 7fcb4099b700 1 --2- 172.21.15.2:0/3643664749 >> [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611] conn(0x7fcb280bddc0 0x7fcb280c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:53.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.996+0000 7fcb4099b700 1 -- 172.21.15.2:0/3643664749 --> [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcb280c0980 con 0x7fcb280bddc0 2024-05-14T09:32:53.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.996+0000 7fcb4099b700 1 -- 172.21.15.2:0/3643664749 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fcb2c0e4780 con 0x7fcb3c1034c0 2024-05-14T09:32:53.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.997+0000 7fcb327fc700 1 --2- 172.21.15.2:0/3643664749 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcb280b2910 0x7fcb280b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c109a50 tx=0x7fcb2400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:53.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.997+0000 7fcb3b7fe700 1 --2- 172.21.15.2:0/3643664749 >> [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611] conn(0x7fcb280bddc0 0x7fcb280c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:53.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.997+0000 7fcb3b7fe700 1 --2- 172.21.15.2:0/3643664749 >> [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611] conn(0x7fcb280bddc0 0x7fcb280c02b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:53.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:53.998+0000 7fcb4099b700 1 -- 172.21.15.2:0/3643664749 <== osd.7 v2:172.21.15.26:6816/1180401611 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcb280c0980 con 0x7fcb280bddc0 2024-05-14T09:32:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:54.015+0000 7fcb4199d700 1 -- 172.21.15.2:0/3643664749 --> [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcb3c064350 con 0x7fcb280bddc0 2024-05-14T09:32:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:54.016+0000 7fcb4099b700 1 -- 172.21.15.2:0/3643664749 <== osd.7 v2:172.21.15.26:6816/1180401611 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcb3c064350 con 0x7fcb280bddc0 2024-05-14T09:32:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:54.016+0000 7fcb4199d700 1 -- 172.21.15.2:0/3643664749 >> [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611] conn(0x7fcb280bddc0 msgr2=0x7fcb280c02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:54.016+0000 7fcb4199d700 1 --2- 172.21.15.2:0/3643664749 >> [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611] conn(0x7fcb280bddc0 0x7fcb280c02b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:54.016+0000 7fcb4199d700 1 -- 172.21.15.2:0/3643664749 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcb280b2910 msgr2=0x7fcb280b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:54.016+0000 7fcb4199d700 1 --2- 172.21.15.2:0/3643664749 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcb280b2910 0x7fcb280b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c109a50 tx=0x7fcb2400a380 comp rx=0 tx=0).stop 2024-05-14T09:32:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:54.016+0000 7fcb4199d700 1 -- 172.21.15.2:0/3643664749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb3c1034c0 msgr2=0x7fcb3c19ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:54.016+0000 7fcb4199d700 1 --2- 172.21.15.2:0/3643664749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb3c1034c0 0x7fcb3c19ae30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0098e0 tx=0x7fcb2c0057c0 comp rx=0 tx=0).stop 2024-05-14T09:32:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:54.016+0000 7fcb3b7fe700 1 -- 172.21.15.2:0/3643664749 reap_dead start 2024-05-14T09:32:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:54.016+0000 7fcb4199d700 1 -- 172.21.15.2:0/3643664749 shutdown_connections 2024-05-14T09:32:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:54.016+0000 7fcb4199d700 1 -- 172.21.15.2:0/3643664749 >> 172.21.15.2:0/3643664749 conn(0x7fcb3c0fedf0 msgr2=0x7fcb3c075d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:54.016+0000 7fcb4199d700 1 -- 172.21.15.2:0/3643664749 shutdown_connections 2024-05-14T09:32:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:54.016+0000 7fcb4199d700 1 -- 172.21.15.2:0/3643664749 wait complete. 2024-05-14T09:32:54.085 INFO:teuthology.orchestra.run.smithi002.stdout:231928234037 2024-05-14T09:32:54.085 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-14T09:32:54.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:54 smithi002 ceph-mon[31557]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:32:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:54 smithi029 ceph-mon[36249]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:32:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:54 smithi026 ceph-mon[36056]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:32:54.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.413+0000 7f3c81a0e700 1 -- 172.21.15.2:0/4236858786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c7c10e8f0 msgr2=0x7f3c7c110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:54.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.413+0000 7f3c81a0e700 1 --2- 172.21.15.2:0/4236858786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c7c10e8f0 0x7f3c7c110dd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400b0a0 tx=0x7f3c7400b3b0 comp rx=0 tx=0).stop 2024-05-14T09:32:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.413+0000 7f3c81a0e700 1 -- 172.21.15.2:0/4236858786 shutdown_connections 2024-05-14T09:32:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.413+0000 7f3c81a0e700 1 --2- 172.21.15.2:0/4236858786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c7c105480 0x7f3c7c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.413+0000 7f3c81a0e700 1 --2- 172.21.15.2:0/4236858786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c7c10e8f0 0x7f3c7c110dd0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.413+0000 7f3c81a0e700 1 --2- 172.21.15.2:0/4236858786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c7c105dd0 0x7f3c7c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.413+0000 7f3c81a0e700 1 -- 172.21.15.2:0/4236858786 >> 172.21.15.2:0/4236858786 conn(0x7f3c7c0fef30 msgr2=0x7f3c7c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.413+0000 7f3c81a0e700 1 -- 172.21.15.2:0/4236858786 shutdown_connections 2024-05-14T09:32:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.413+0000 7f3c81a0e700 1 -- 172.21.15.2:0/4236858786 wait complete. 2024-05-14T09:32:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.414+0000 7f3c81a0e700 1 Processor -- start 2024-05-14T09:32:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.414+0000 7f3c81a0e700 1 -- start start 2024-05-14T09:32:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.414+0000 7f3c81a0e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c7c105480 0x7f3c7c103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.414+0000 7f3c81a0e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c7c105dd0 0x7f3c7c1036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.414+0000 7f3c81a0e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c7c10e8f0 0x7f3c7c103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.414+0000 7f3c81a0e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1144d0 con 0x7f3c7c105dd0 2024-05-14T09:32:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.414+0000 7f3c81a0e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c114350 con 0x7f3c7c105480 2024-05-14T09:32:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.414+0000 7f3c81a0e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c114650 con 0x7f3c7c10e8f0 2024-05-14T09:32:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.414+0000 7f3c7a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c7c105dd0 0x7f3c7c1036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.414+0000 7f3c7a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c7c105dd0 0x7f3c7c1036b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45716/0 (socket says 172.21.15.2:45716) 2024-05-14T09:32:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.414+0000 7f3c7a7fc700 1 -- 172.21.15.2:0/145955972 learned_addr learned my addr 172.21.15.2:0/145955972 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.414+0000 7f3c7b7fe700 1 --2- 172.21.15.2:0/145955972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c7c10e8f0 0x7f3c7c103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.415+0000 7f3c7affd700 1 --2- 172.21.15.2:0/145955972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c7c105480 0x7f3c7c103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.415+0000 7f3c7a7fc700 1 -- 172.21.15.2:0/145955972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c7c10e8f0 msgr2=0x7f3c7c103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.415+0000 7f3c7a7fc700 1 --2- 172.21.15.2:0/145955972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3c7c10e8f0 0x7f3c7c103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.415+0000 7f3c7a7fc700 1 -- 172.21.15.2:0/145955972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c7c105480 msgr2=0x7f3c7c103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.415+0000 7f3c7a7fc700 1 --2- 172.21.15.2:0/145955972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c7c105480 0x7f3c7c103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.415+0000 7f3c7a7fc700 1 -- 172.21.15.2:0/145955972 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7c1a91e0 con 0x7f3c7c105dd0 2024-05-14T09:32:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.415+0000 7f3c7a7fc700 1 --2- 172.21.15.2:0/145955972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c7c105dd0 0x7f3c7c1036b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00b530 tx=0x7f3c6c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.415+0000 7f3c80a0c700 1 -- 172.21.15.2:0/145955972 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c015020 con 0x7f3c7c105dd0 2024-05-14T09:32:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.415+0000 7f3c81a0e700 1 -- 172.21.15.2:0/145955972 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7c1a94d0 con 0x7f3c7c105dd0 2024-05-14T09:32:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.415+0000 7f3c80a0c700 1 -- 172.21.15.2:0/145955972 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c6c011ce0 con 0x7f3c7c105dd0 2024-05-14T09:32:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.415+0000 7f3c80a0c700 1 -- 172.21.15.2:0/145955972 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c010dd0 con 0x7f3c7c105dd0 2024-05-14T09:32:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.415+0000 7f3c81a0e700 1 -- 172.21.15.2:0/145955972 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7c1a9a60 con 0x7f3c7c105dd0 2024-05-14T09:32:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.416+0000 7f3c81a0e700 1 -- 172.21.15.2:0/145955972 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3c7c063ed0 con 0x7f3c7c105dd0 2024-05-14T09:32:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.419+0000 7f3c80a0c700 1 -- 172.21.15.2:0/145955972 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f3c6c018020 con 0x7f3c7c105dd0 2024-05-14T09:32:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.420+0000 7f3c80a0c700 1 --2- 172.21.15.2:0/145955972 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3c680b2910 0x7f3c680b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.420+0000 7f3c7affd700 1 --2- 172.21.15.2:0/145955972 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3c680b2910 0x7f3c680b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.420+0000 7f3c80a0c700 1 -- 172.21.15.2:0/145955972 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3c6c0e2430 con 0x7f3c7c105dd0 2024-05-14T09:32:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.420+0000 7f3c80a0c700 1 --2- 172.21.15.2:0/145955972 >> [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093] conn(0x7f3c680bddc0 0x7f3c680c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.421+0000 7f3c80a0c700 1 -- 172.21.15.2:0/145955972 --> [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c680c0980 con 0x7f3c680bddc0 2024-05-14T09:32:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.421+0000 7f3c80a0c700 1 -- 172.21.15.2:0/145955972 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3c6c0e2880 con 0x7f3c7c105dd0 2024-05-14T09:32:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.421+0000 7f3c7affd700 1 --2- 172.21.15.2:0/145955972 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3c680b2910 0x7f3c680b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3c64007be0 tx=0x7f3c640073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.421+0000 7f3c7b7fe700 1 --2- 172.21.15.2:0/145955972 >> [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093] conn(0x7f3c680bddc0 0x7f3c680c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.421+0000 7f3c7b7fe700 1 --2- 172.21.15.2:0/145955972 >> [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093] conn(0x7f3c680bddc0 0x7f3c680c02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.422+0000 7f3c80a0c700 1 -- 172.21.15.2:0/145955972 <== osd.5 v2:172.21.15.29:6808/1554045093 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3c680c0980 con 0x7f3c680bddc0 2024-05-14T09:32:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.440+0000 7f3c81a0e700 1 -- 172.21.15.2:0/145955972 --> [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3c7c0642d0 con 0x7f3c680bddc0 2024-05-14T09:32:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.441+0000 7f3c80a0c700 1 -- 172.21.15.2:0/145955972 <== osd.5 v2:172.21.15.29:6808/1554045093 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3c7c0642d0 con 0x7f3c680bddc0 2024-05-14T09:32:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.441+0000 7f3c81a0e700 1 -- 172.21.15.2:0/145955972 >> [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093] conn(0x7f3c680bddc0 msgr2=0x7f3c680c02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.441+0000 7f3c81a0e700 1 --2- 172.21.15.2:0/145955972 >> [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093] conn(0x7f3c680bddc0 0x7f3c680c02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.441+0000 7f3c81a0e700 1 -- 172.21.15.2:0/145955972 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3c680b2910 msgr2=0x7f3c680b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.441+0000 7f3c81a0e700 1 --2- 172.21.15.2:0/145955972 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3c680b2910 0x7f3c680b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3c64007be0 tx=0x7f3c640073d0 comp rx=0 tx=0).stop 2024-05-14T09:32:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.441+0000 7f3c81a0e700 1 -- 172.21.15.2:0/145955972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c7c105dd0 msgr2=0x7f3c7c1036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.441+0000 7f3c81a0e700 1 --2- 172.21.15.2:0/145955972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c7c105dd0 0x7f3c7c1036b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00b530 tx=0x7f3c6c00b840 comp rx=0 tx=0).stop 2024-05-14T09:32:54.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.441+0000 7f3c7b7fe700 1 -- 172.21.15.2:0/145955972 reap_dead start 2024-05-14T09:32:54.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.441+0000 7f3c81a0e700 1 -- 172.21.15.2:0/145955972 shutdown_connections 2024-05-14T09:32:54.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.441+0000 7f3c81a0e700 1 -- 172.21.15.2:0/145955972 >> 172.21.15.2:0/145955972 conn(0x7f3c7c0fef30 msgr2=0x7f3c7c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:54.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.441+0000 7f3c81a0e700 1 -- 172.21.15.2:0/145955972 shutdown_connections 2024-05-14T09:32:54.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:52.441+0000 7f3c81a0e700 1 -- 172.21.15.2:0/145955972 wait complete. 2024-05-14T09:32:56.629 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:56 smithi002 ceph-mon[31557]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:32:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:56 smithi029 ceph-mon[36249]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:32:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:56 smithi026 ceph-mon[36056]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:32:56.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.854+0000 7f2d9fd23700 1 -- 172.21.15.2:0/3554951273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d981034c0 msgr2=0x7f2d981038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:56.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.854+0000 7f2d9fd23700 1 --2- 172.21.15.2:0/3554951273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d981034c0 0x7f2d981038d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2d940099c0 tx=0x7f2d94009cd0 comp rx=0 tx=0).stop 2024-05-14T09:32:56.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.854+0000 7f2d9fd23700 1 -- 172.21.15.2:0/3554951273 shutdown_connections 2024-05-14T09:32:56.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.854+0000 7f2d9fd23700 1 --2- 172.21.15.2:0/3554951273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d98108740 0x7f2d981100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:56.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.854+0000 7f2d9fd23700 1 --2- 172.21.15.2:0/3554951273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d981034c0 0x7f2d981038d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:56.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.854+0000 7f2d9fd23700 1 --2- 172.21.15.2:0/3554951273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d98107df0 0x7f2d98108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:56.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.854+0000 7f2d9fd23700 1 -- 172.21.15.2:0/3554951273 >> 172.21.15.2:0/3554951273 conn(0x7f2d980fedf0 msgr2=0x7f2d98101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:56.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.854+0000 7f2d9fd23700 1 -- 172.21.15.2:0/3554951273 shutdown_connections 2024-05-14T09:32:56.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.854+0000 7f2d9fd23700 1 -- 172.21.15.2:0/3554951273 wait complete. 2024-05-14T09:32:56.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.855+0000 7f2d9fd23700 1 Processor -- start 2024-05-14T09:32:56.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.855+0000 7f2d9fd23700 1 -- start start 2024-05-14T09:32:56.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.855+0000 7f2d9fd23700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d981034c0 0x7f2d980ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:56.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.855+0000 7f2d9fd23700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d98107df0 0x7f2d980ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:56.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.855+0000 7f2d9fd23700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d98108740 0x7f2d980ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:56.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.855+0000 7f2d9fd23700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98113370 con 0x7f2d98108740 2024-05-14T09:32:56.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.855+0000 7f2d9fd23700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2d981131f0 con 0x7f2d981034c0 2024-05-14T09:32:56.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.855+0000 7f2d9fd23700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2d981134f0 con 0x7f2d98107df0 2024-05-14T09:32:56.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.855+0000 7f2d9d2be700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d98107df0 0x7f2d980ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:56.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.855+0000 7f2d9dabf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d981034c0 0x7f2d980ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:56.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.855+0000 7f2d9e2c0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d98108740 0x7f2d980ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.855+0000 7f2d9d2be700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d98107df0 0x7f2d980ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:46182/0 (socket says 172.21.15.2:46182) 2024-05-14T09:32:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.855+0000 7f2d9dabf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d981034c0 0x7f2d980ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:44228/0 (socket says 172.21.15.2:44228) 2024-05-14T09:32:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.855+0000 7f2d9d2be700 1 -- 172.21.15.2:0/4151350836 learned_addr learned my addr 172.21.15.2:0/4151350836 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:32:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.856+0000 7f2d9d2be700 1 -- 172.21.15.2:0/4151350836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d981034c0 msgr2=0x7f2d980ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.856+0000 7f2d9d2be700 1 --2- 172.21.15.2:0/4151350836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d981034c0 0x7f2d980ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.856+0000 7f2d9d2be700 1 -- 172.21.15.2:0/4151350836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d98108740 msgr2=0x7f2d980ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.856+0000 7f2d9d2be700 1 --2- 172.21.15.2:0/4151350836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d98108740 0x7f2d980ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.856+0000 7f2d9d2be700 1 -- 172.21.15.2:0/4151350836 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d981a91d0 con 0x7f2d98107df0 2024-05-14T09:32:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.856+0000 7f2d9e2c0700 1 --2- 172.21.15.2:0/4151350836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d98108740 0x7f2d980ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:32:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.856+0000 7f2d9d2be700 1 --2- 172.21.15.2:0/4151350836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d98107df0 0x7f2d980ff810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800e970 tx=0x7f2d8800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.856+0000 7f2d8effd700 1 -- 172.21.15.2:0/4151350836 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88004a30 con 0x7f2d98107df0 2024-05-14T09:32:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.856+0000 7f2d8effd700 1 -- 172.21.15.2:0/4151350836 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d8800cc90 con 0x7f2d98107df0 2024-05-14T09:32:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.856+0000 7f2d9fd23700 1 -- 172.21.15.2:0/4151350836 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d981a94c0 con 0x7f2d98107df0 2024-05-14T09:32:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.856+0000 7f2d8effd700 1 -- 172.21.15.2:0/4151350836 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88005610 con 0x7f2d98107df0 2024-05-14T09:32:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.856+0000 7f2d9fd23700 1 -- 172.21.15.2:0/4151350836 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d981a9a00 con 0x7f2d98107df0 2024-05-14T09:32:56.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.857+0000 7f2d9fd23700 1 -- 172.21.15.2:0/4151350836 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d98063ed0 con 0x7f2d98107df0 2024-05-14T09:32:56.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.859+0000 7f2d8effd700 1 -- 172.21.15.2:0/4151350836 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f2d8800c7e0 con 0x7f2d98107df0 2024-05-14T09:32:56.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.860+0000 7f2d8effd700 1 --2- 172.21.15.2:0/4151350836 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2d840b2910 0x7f2d840b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:56.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.860+0000 7f2d9dabf700 1 --2- 172.21.15.2:0/4151350836 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2d840b2910 0x7f2d840b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:56.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.860+0000 7f2d8effd700 1 -- 172.21.15.2:0/4151350836 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f2d880e33b0 con 0x7f2d98107df0 2024-05-14T09:32:56.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.860+0000 7f2d8effd700 1 --2- 172.21.15.2:0/4151350836 >> [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595] conn(0x7f2d840bddc0 0x7f2d840c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:32:56.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.860+0000 7f2d8effd700 1 -- 172.21.15.2:0/4151350836 --> [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d8801ee50 con 0x7f2d840bddc0 2024-05-14T09:32:56.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.860+0000 7f2d8effd700 1 -- 172.21.15.2:0/4151350836 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f2d880e37f0 con 0x7f2d98107df0 2024-05-14T09:32:56.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.860+0000 7f2d9e2c0700 1 --2- 172.21.15.2:0/4151350836 >> [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595] conn(0x7f2d840bddc0 0x7f2d840c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:32:56.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.861+0000 7f2d9dabf700 1 --2- 172.21.15.2:0/4151350836 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2d840b2910 0x7f2d840b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000c00 tx=0x7f2d9401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:56.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.861+0000 7f2d9e2c0700 1 --2- 172.21.15.2:0/4151350836 >> [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595] conn(0x7f2d840bddc0 0x7f2d840c02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:32:56.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.862+0000 7f2d8effd700 1 -- 172.21.15.2:0/4151350836 <== osd.6 v2:172.21.15.2:6818/2749897595 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d8801ee50 con 0x7f2d840bddc0 2024-05-14T09:32:56.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.879+0000 7f2d9fd23700 1 -- 172.21.15.2:0/4151350836 --> [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d98064350 con 0x7f2d840bddc0 2024-05-14T09:32:56.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.879+0000 7f2d8effd700 1 -- 172.21.15.2:0/4151350836 <== osd.6 v2:172.21.15.2:6818/2749897595 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2d98064350 con 0x7f2d840bddc0 2024-05-14T09:32:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.880+0000 7f2d9fd23700 1 -- 172.21.15.2:0/4151350836 >> [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595] conn(0x7f2d840bddc0 msgr2=0x7f2d840c02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.880+0000 7f2d9fd23700 1 --2- 172.21.15.2:0/4151350836 >> [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595] conn(0x7f2d840bddc0 0x7f2d840c02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:32:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.880+0000 7f2d9fd23700 1 -- 172.21.15.2:0/4151350836 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2d840b2910 msgr2=0x7f2d840b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.880+0000 7f2d9fd23700 1 --2- 172.21.15.2:0/4151350836 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2d840b2910 0x7f2d840b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000c00 tx=0x7f2d9401a040 comp rx=0 tx=0).stop 2024-05-14T09:32:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.880+0000 7f2d9fd23700 1 -- 172.21.15.2:0/4151350836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d98107df0 msgr2=0x7f2d980ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:32:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.880+0000 7f2d9fd23700 1 --2- 172.21.15.2:0/4151350836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d98107df0 0x7f2d980ff810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800e970 tx=0x7f2d8800ec80 comp rx=0 tx=0).stop 2024-05-14T09:32:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.880+0000 7f2d9e2c0700 1 -- 172.21.15.2:0/4151350836 reap_dead start 2024-05-14T09:32:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.880+0000 7f2d9fd23700 1 -- 172.21.15.2:0/4151350836 shutdown_connections 2024-05-14T09:32:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.880+0000 7f2d9fd23700 1 -- 172.21.15.2:0/4151350836 >> 172.21.15.2:0/4151350836 conn(0x7f2d980fedf0 msgr2=0x7f2d98075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:32:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.880+0000 7f2d9fd23700 1 -- 172.21.15.2:0/4151350836 shutdown_connections 2024-05-14T09:32:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:32:56.880+0000 7f2d9fd23700 1 -- 172.21.15.2:0/4151350836 wait complete. 2024-05-14T09:32:57.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:32:57.648 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:32:57.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:32:58.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:58 smithi002 ceph-mon[31557]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T09:32:58.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:58 smithi029 ceph-mon[36249]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T09:32:58.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:58 smithi026 ceph-mon[36056]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T09:32:59.362 INFO:teuthology.orchestra.run.smithi002.stdout:184683593793 2024-05-14T09:32:59.362 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-14T09:32:59.581 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:32:59.581 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:32:59.581 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:32:59.582 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:32:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:32:59.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:32:59.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:32:59.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:32:59.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:32:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:32:59.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:32:59.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:32:59.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:32:59.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:32:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:33:00.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.042+0000 7f03f131c700 1 -- 172.21.15.2:0/3168987965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03ec1089f0 msgr2=0x7f03ec108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:00.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.042+0000 7f03f131c700 1 --2- 172.21.15.2:0/3168987965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03ec1089f0 0x7f03ec108e00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00aaa0 tx=0x7f03dc00adb0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.042+0000 7f03f131c700 1 -- 172.21.15.2:0/3168987965 shutdown_connections 2024-05-14T09:33:00.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.042+0000 7f03f131c700 1 --2- 172.21.15.2:0/3168987965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ec103310 0x7f03ec10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.042+0000 7f03f131c700 1 --2- 172.21.15.2:0/3168987965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03ec1089f0 0x7f03ec108e00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.042+0000 7f03f131c700 1 --2- 172.21.15.2:0/3168987965 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03ec1029c0 0x7f03ec102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.042+0000 7f03f131c700 1 -- 172.21.15.2:0/3168987965 >> 172.21.15.2:0/3168987965 conn(0x7f03ec0fe6f0 msgr2=0x7f03ec100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:00.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.042+0000 7f03f131c700 1 -- 172.21.15.2:0/3168987965 shutdown_connections 2024-05-14T09:33:00.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.042+0000 7f03f131c700 1 -- 172.21.15.2:0/3168987965 wait complete. 2024-05-14T09:33:00.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.042+0000 7f03f131c700 1 Processor -- start 2024-05-14T09:33:00.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.043+0000 7f03f131c700 1 -- start start 2024-05-14T09:33:00.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.043+0000 7f03f131c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03ec1029c0 0x7f03ec19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:00.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.043+0000 7f03f131c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03ec103310 0x7f03ec19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:00.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.043+0000 7f03f131c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ec1089f0 0x7f03ec198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:00.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.043+0000 7f03f131c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec1122f0 con 0x7f03ec1029c0 2024-05-14T09:33:00.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.043+0000 7f03f131c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec112170 con 0x7f03ec1089f0 2024-05-14T09:33:00.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.043+0000 7f03f131c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec112470 con 0x7f03ec103310 2024-05-14T09:33:00.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.043+0000 7f03eaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03ec1029c0 0x7f03ec19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:00.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.043+0000 7f03eaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03ec1029c0 0x7f03ec19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38780/0 (socket says 172.21.15.2:38780) 2024-05-14T09:33:00.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.043+0000 7f03eaffd700 1 -- 172.21.15.2:0/2799430472 learned_addr learned my addr 172.21.15.2:0/2799430472 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:00.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.043+0000 7f03ea7fc700 1 --2- 172.21.15.2:0/2799430472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03ec103310 0x7f03ec19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:00.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.043+0000 7f03eb7fe700 1 --2- 172.21.15.2:0/2799430472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ec1089f0 0x7f03ec198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:00.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.044+0000 7f03eaffd700 1 -- 172.21.15.2:0/2799430472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03ec103310 msgr2=0x7f03ec19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:00.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.044+0000 7f03eaffd700 1 --2- 172.21.15.2:0/2799430472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03ec103310 0x7f03ec19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.044+0000 7f03eaffd700 1 -- 172.21.15.2:0/2799430472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ec1089f0 msgr2=0x7f03ec198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:00.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.044+0000 7f03eaffd700 1 --2- 172.21.15.2:0/2799430472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ec1089f0 0x7f03ec198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.044+0000 7f03eaffd700 1 -- 172.21.15.2:0/2799430472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ec199140 con 0x7f03ec1029c0 2024-05-14T09:33:00.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.044+0000 7f03eaffd700 1 --2- 172.21.15.2:0/2799430472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03ec1029c0 0x7f03ec19e870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f03dc000c00 tx=0x7f03dc004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:00.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.044+0000 7f03e3fff700 1 -- 172.21.15.2:0/2799430472 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc024020 con 0x7f03ec1029c0 2024-05-14T09:33:00.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.044+0000 7f03f131c700 1 -- 172.21.15.2:0/2799430472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ec1993d0 con 0x7f03ec1029c0 2024-05-14T09:33:00.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.044+0000 7f03e3fff700 1 -- 172.21.15.2:0/2799430472 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03dc00e450 con 0x7f03ec1029c0 2024-05-14T09:33:00.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.044+0000 7f03f131c700 1 -- 172.21.15.2:0/2799430472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ec1a5700 con 0x7f03ec1029c0 2024-05-14T09:33:00.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.044+0000 7f03e3fff700 1 -- 172.21.15.2:0/2799430472 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc01b040 con 0x7f03ec1029c0 2024-05-14T09:33:00.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.045+0000 7f03f131c700 1 -- 172.21.15.2:0/2799430472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f03ec063f50 con 0x7f03ec1029c0 2024-05-14T09:33:00.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.047+0000 7f03e3fff700 1 -- 172.21.15.2:0/2799430472 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f03dc027020 con 0x7f03ec1029c0 2024-05-14T09:33:00.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.047+0000 7f03e3fff700 1 --2- 172.21.15.2:0/2799430472 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f03d80b2910 0x7f03d80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:00.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.048+0000 7f03e3fff700 1 -- 172.21.15.2:0/2799430472 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f03dc0e55c0 con 0x7f03ec1029c0 2024-05-14T09:33:00.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.048+0000 7f03ea7fc700 1 --2- 172.21.15.2:0/2799430472 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f03d80b2910 0x7f03d80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:00.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.048+0000 7f03e3fff700 1 --2- 172.21.15.2:0/2799430472 >> [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515] conn(0x7f03d80bddc0 0x7f03d80c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:00.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.048+0000 7f03e3fff700 1 -- 172.21.15.2:0/2799430472 --> [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03d80c0980 con 0x7f03d80bddc0 2024-05-14T09:33:00.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.048+0000 7f03e3fff700 1 -- 172.21.15.2:0/2799430472 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f03dc0e5a10 con 0x7f03ec1029c0 2024-05-14T09:33:00.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.048+0000 7f03eb7fe700 1 --2- 172.21.15.2:0/2799430472 >> [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515] conn(0x7f03d80bddc0 0x7f03d80c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:00.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.048+0000 7f03ea7fc700 1 --2- 172.21.15.2:0/2799430472 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f03d80b2910 0x7f03d80b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f03ec1003c0 tx=0x7f03d400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:00.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.049+0000 7f03eb7fe700 1 --2- 172.21.15.2:0/2799430472 >> [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515] conn(0x7f03d80bddc0 0x7f03d80c02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:00.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.050+0000 7f03e3fff700 1 -- 172.21.15.2:0/2799430472 <== osd.10 v2:172.21.15.26:6824/4065159515 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03d80c0980 con 0x7f03d80bddc0 2024-05-14T09:33:00.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.082+0000 7f03f131c700 1 -- 172.21.15.2:0/2799430472 --> [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f03ec064350 con 0x7f03d80bddc0 2024-05-14T09:33:00.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.083+0000 7f03e3fff700 1 -- 172.21.15.2:0/2799430472 <== osd.10 v2:172.21.15.26:6824/4065159515 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f03ec064350 con 0x7f03d80bddc0 2024-05-14T09:33:00.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.083+0000 7f03f131c700 1 -- 172.21.15.2:0/2799430472 >> [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515] conn(0x7f03d80bddc0 msgr2=0x7f03d80c02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:00.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.083+0000 7f03f131c700 1 --2- 172.21.15.2:0/2799430472 >> [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515] conn(0x7f03d80bddc0 0x7f03d80c02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.083+0000 7f03f131c700 1 -- 172.21.15.2:0/2799430472 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f03d80b2910 msgr2=0x7f03d80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:00.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.083+0000 7f03f131c700 1 --2- 172.21.15.2:0/2799430472 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f03d80b2910 0x7f03d80b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f03ec1003c0 tx=0x7f03d400b3c0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.083+0000 7f03f131c700 1 -- 172.21.15.2:0/2799430472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03ec1029c0 msgr2=0x7f03ec19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:00.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.083+0000 7f03f131c700 1 --2- 172.21.15.2:0/2799430472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03ec1029c0 0x7f03ec19e870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f03dc000c00 tx=0x7f03dc004710 comp rx=0 tx=0).stop 2024-05-14T09:33:00.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.083+0000 7f03eb7fe700 1 -- 172.21.15.2:0/2799430472 reap_dead start 2024-05-14T09:33:00.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.083+0000 7f03f131c700 1 -- 172.21.15.2:0/2799430472 shutdown_connections 2024-05-14T09:33:00.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.083+0000 7f03f131c700 1 -- 172.21.15.2:0/2799430472 >> 172.21.15.2:0/2799430472 conn(0x7f03ec0fe6f0 msgr2=0x7f03ec100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:00.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.083+0000 7f03f131c700 1 -- 172.21.15.2:0/2799430472 shutdown_connections 2024-05-14T09:33:00.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.084+0000 7f03f131c700 1 -- 172.21.15.2:0/2799430472 wait complete. 2024-05-14T09:33:00.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.673+0000 7fb01c13a700 1 -- 172.21.15.2:0/712339463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb014107df0 msgr2=0x7fb014108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:00.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.673+0000 7fb01c13a700 1 --2- 172.21.15.2:0/712339463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb014107df0 0x7fb014108200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb0040099c0 tx=0x7fb004009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.673 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:00 smithi002 ceph-mon[31557]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:00.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.673+0000 7fb01c13a700 1 -- 172.21.15.2:0/712339463 shutdown_connections 2024-05-14T09:33:00.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.673+0000 7fb01c13a700 1 --2- 172.21.15.2:0/712339463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0141034c0 0x7fb0141038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.673+0000 7fb01c13a700 1 --2- 172.21.15.2:0/712339463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb014108740 0x7fb0141100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.673+0000 7fb01c13a700 1 --2- 172.21.15.2:0/712339463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb014107df0 0x7fb014108200 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.673+0000 7fb01c13a700 1 -- 172.21.15.2:0/712339463 >> 172.21.15.2:0/712339463 conn(0x7fb0140fedc0 msgr2=0x7fb0141011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:00.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.673+0000 7fb01c13a700 1 -- 172.21.15.2:0/712339463 shutdown_connections 2024-05-14T09:33:00.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.673+0000 7fb01c13a700 1 -- 172.21.15.2:0/712339463 wait complete. 2024-05-14T09:33:00.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.674+0000 7fb01c13a700 1 Processor -- start 2024-05-14T09:33:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.674+0000 7fb01c13a700 1 -- start start 2024-05-14T09:33:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.675+0000 7fb01c13a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0141034c0 0x7fb0141128b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.675+0000 7fb01c13a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb014108740 0x7fb014110d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.675+0000 7fb01c13a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0141112f0 0x7fb0141117c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.675+0000 7fb01c13a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb014114450 con 0x7fb0141112f0 2024-05-14T09:33:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.675+0000 7fb01c13a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb0141142d0 con 0x7fb0141034c0 2024-05-14T09:33:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.675+0000 7fb01c13a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb0141145d0 con 0x7fb014108740 2024-05-14T09:33:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.675+0000 7fb019ed6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0141034c0 0x7fb0141128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.675+0000 7fb01a6d7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0141112f0 0x7fb0141117c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.675+0000 7fb019ed6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0141034c0 0x7fb0141128b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:34588/0 (socket says 172.21.15.2:34588) 2024-05-14T09:33:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.675+0000 7fb019ed6700 1 -- 172.21.15.2:0/3643735640 learned_addr learned my addr 172.21.15.2:0/3643735640 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.675+0000 7fb0196d5700 1 --2- 172.21.15.2:0/3643735640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb014108740 0x7fb014110d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.676+0000 7fb01a6d7700 1 -- 172.21.15.2:0/3643735640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb014108740 msgr2=0x7fb014110d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.676+0000 7fb01a6d7700 1 --2- 172.21.15.2:0/3643735640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb014108740 0x7fb014110d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.676+0000 7fb01a6d7700 1 -- 172.21.15.2:0/3643735640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0141034c0 msgr2=0x7fb0141128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.676+0000 7fb01a6d7700 1 --2- 172.21.15.2:0/3643735640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0141034c0 0x7fb0141128b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.676+0000 7fb01a6d7700 1 -- 172.21.15.2:0/3643735640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0141adb60 con 0x7fb0141112f0 2024-05-14T09:33:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.676+0000 7fb019ed6700 1 --2- 172.21.15.2:0/3643735640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0141034c0 0x7fb0141128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:33:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.676+0000 7fb01a6d7700 1 --2- 172.21.15.2:0/3643735640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0141112f0 0x7fb0141117c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb00c007f10 tx=0x7fb00c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.676+0000 7fb00affd700 1 -- 172.21.15.2:0/3643735640 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c004db0 con 0x7fb0141112f0 2024-05-14T09:33:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.676+0000 7fb01c13a700 1 -- 172.21.15.2:0/3643735640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0141ade50 con 0x7fb0141112f0 2024-05-14T09:33:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.676+0000 7fb00affd700 1 -- 172.21.15.2:0/3643735640 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb00c009e20 con 0x7fb0141112f0 2024-05-14T09:33:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.676+0000 7fb00affd700 1 -- 172.21.15.2:0/3643735640 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c015400 con 0x7fb0141112f0 2024-05-14T09:33:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.676+0000 7fb01c13a700 1 -- 172.21.15.2:0/3643735640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0141ae2d0 con 0x7fb0141112f0 2024-05-14T09:33:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.677+0000 7fb01c13a700 1 -- 172.21.15.2:0/3643735640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb014063ed0 con 0x7fb0141112f0 2024-05-14T09:33:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.679+0000 7fb00affd700 1 -- 172.21.15.2:0/3643735640 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fb00c005260 con 0x7fb0141112f0 2024-05-14T09:33:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.680+0000 7fb00affd700 1 --2- 172.21.15.2:0/3643735640 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb0000b2910 0x7fb0000b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.680+0000 7fb00affd700 1 -- 172.21.15.2:0/3643735640 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fb00c0e2c70 con 0x7fb0141112f0 2024-05-14T09:33:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.680+0000 7fb019ed6700 1 --2- 172.21.15.2:0/3643735640 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb0000b2910 0x7fb0000b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.680+0000 7fb00affd700 1 --2- 172.21.15.2:0/3643735640 >> [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634] conn(0x7fb0000bddc0 0x7fb0000c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.680+0000 7fb00affd700 1 -- 172.21.15.2:0/3643735640 --> [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb00c012ec0 con 0x7fb0000bddc0 2024-05-14T09:33:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.680+0000 7fb00affd700 1 -- 172.21.15.2:0/3643735640 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb00c0e30c0 con 0x7fb0141112f0 2024-05-14T09:33:00.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.680+0000 7fb0196d5700 1 --2- 172.21.15.2:0/3643735640 >> [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634] conn(0x7fb0000bddc0 0x7fb0000c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:00.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.681+0000 7fb019ed6700 1 --2- 172.21.15.2:0/3643735640 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb0000b2910 0x7fb0000b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb010005950 tx=0x7fb01000b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:00.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.681+0000 7fb0196d5700 1 --2- 172.21.15.2:0/3643735640 >> [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634] conn(0x7fb0000bddc0 0x7fb0000c02b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:00.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.682+0000 7fb00affd700 1 -- 172.21.15.2:0/3643735640 <== osd.0 v2:172.21.15.2:6802/2622184634 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb00c012ec0 con 0x7fb0000bddc0 2024-05-14T09:33:00.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.699+0000 7fb01c13a700 1 -- 172.21.15.2:0/3643735640 --> [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb0140642d0 con 0x7fb0000bddc0 2024-05-14T09:33:00.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.699+0000 7fb00affd700 1 -- 172.21.15.2:0/3643735640 <== osd.0 v2:172.21.15.2:6802/2622184634 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb0140642d0 con 0x7fb0000bddc0 2024-05-14T09:33:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.700+0000 7fb01c13a700 1 -- 172.21.15.2:0/3643735640 >> [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634] conn(0x7fb0000bddc0 msgr2=0x7fb0000c02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.700+0000 7fb01c13a700 1 --2- 172.21.15.2:0/3643735640 >> [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634] conn(0x7fb0000bddc0 0x7fb0000c02b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.700+0000 7fb01c13a700 1 -- 172.21.15.2:0/3643735640 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb0000b2910 msgr2=0x7fb0000b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.700+0000 7fb01c13a700 1 --2- 172.21.15.2:0/3643735640 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb0000b2910 0x7fb0000b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb010005950 tx=0x7fb01000b500 comp rx=0 tx=0).stop 2024-05-14T09:33:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.700+0000 7fb01c13a700 1 -- 172.21.15.2:0/3643735640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0141112f0 msgr2=0x7fb0141117c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.700+0000 7fb01c13a700 1 --2- 172.21.15.2:0/3643735640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0141112f0 0x7fb0141117c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb00c007f10 tx=0x7fb00c00d3b0 comp rx=0 tx=0).stop 2024-05-14T09:33:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.700+0000 7fb01a6d7700 1 -- 172.21.15.2:0/3643735640 reap_dead start 2024-05-14T09:33:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.700+0000 7fb01c13a700 1 -- 172.21.15.2:0/3643735640 shutdown_connections 2024-05-14T09:33:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.700+0000 7fb01c13a700 1 -- 172.21.15.2:0/3643735640 >> 172.21.15.2:0/3643735640 conn(0x7fb0140fedc0 msgr2=0x7fb0141011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.700+0000 7fb01c13a700 1 -- 172.21.15.2:0/3643735640 shutdown_connections 2024-05-14T09:33:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:00.700+0000 7fb01c13a700 1 -- 172.21.15.2:0/3643735640 wait complete. 2024-05-14T09:33:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:00 smithi029 ceph-mon[36249]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:00 smithi026 ceph-mon[36056]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:01.141 INFO:teuthology.orchestra.run.smithi002.stdout:279172874285 2024-05-14T09:33:01.142 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-14T09:33:01.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.459+0000 7fec54da6700 1 -- 172.21.15.2:0/381995684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5010e8f0 msgr2=0x7fec50110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:01.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.459+0000 7fec54da6700 1 --2- 172.21.15.2:0/381995684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5010e8f0 0x7fec50110dd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fec48009f90 tx=0x7fec4800b330 comp rx=0 tx=0).stop 2024-05-14T09:33:01.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.459+0000 7fec54da6700 1 -- 172.21.15.2:0/381995684 shutdown_connections 2024-05-14T09:33:01.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.459+0000 7fec54da6700 1 --2- 172.21.15.2:0/381995684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec50105480 0x7fec50105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:01.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.459+0000 7fec54da6700 1 --2- 172.21.15.2:0/381995684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec5010e8f0 0x7fec50110dd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:01.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.459+0000 7fec54da6700 1 --2- 172.21.15.2:0/381995684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec50105dd0 0x7fec5010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:01.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.459+0000 7fec54da6700 1 -- 172.21.15.2:0/381995684 >> 172.21.15.2:0/381995684 conn(0x7fec500fef30 msgr2=0x7fec50101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:01.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.460+0000 7fec54da6700 1 -- 172.21.15.2:0/381995684 shutdown_connections 2024-05-14T09:33:01.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.460+0000 7fec54da6700 1 -- 172.21.15.2:0/381995684 wait complete. 2024-05-14T09:33:01.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.461+0000 7fec54da6700 1 Processor -- start 2024-05-14T09:33:01.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.461+0000 7fec54da6700 1 -- start start 2024-05-14T09:33:01.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.461+0000 7fec54da6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec50105480 0x7fec5019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.461+0000 7fec54da6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec50105dd0 0x7fec5019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.461+0000 7fec54da6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5010e8f0 0x7fec5019f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.461+0000 7fec54da6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fec50114450 con 0x7fec50105480 2024-05-14T09:33:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.462+0000 7fec4e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec50105480 0x7fec5019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.462+0000 7fec4e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec50105480 0x7fec5019ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38818/0 (socket says 172.21.15.2:38818) 2024-05-14T09:33:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.462+0000 7fec4e59c700 1 -- 172.21.15.2:0/2282910227 learned_addr learned my addr 172.21.15.2:0/2282910227 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.462+0000 7fec4dd9b700 1 --2- 172.21.15.2:0/2282910227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec50105dd0 0x7fec5019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.462+0000 7fec54da6700 1 -- 172.21.15.2:0/2282910227 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fec501142d0 con 0x7fec5010e8f0 2024-05-14T09:33:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.462+0000 7fec54da6700 1 -- 172.21.15.2:0/2282910227 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fec501145d0 con 0x7fec50105dd0 2024-05-14T09:33:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.462+0000 7fec4e59c700 1 -- 172.21.15.2:0/2282910227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec50105dd0 msgr2=0x7fec5019b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.462+0000 7fec4e59c700 1 --2- 172.21.15.2:0/2282910227 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fec50105dd0 0x7fec5019b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.462+0000 7fec4e59c700 1 -- 172.21.15.2:0/2282910227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5010e8f0 msgr2=0x7fec5019f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.462+0000 7fec4e59c700 1 --2- 172.21.15.2:0/2282910227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5010e8f0 0x7fec5019f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.462+0000 7fec4e59c700 1 -- 172.21.15.2:0/2282910227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5019fc70 con 0x7fec50105480 2024-05-14T09:33:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.463+0000 7fec4e59c700 1 --2- 172.21.15.2:0/2282910227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec50105480 0x7fec5019ada0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fec4000e9a0 tx=0x7fec4000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:01.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.464+0000 7fec477fe700 1 -- 172.21.15.2:0/2282910227 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec40020070 con 0x7fec50105480 2024-05-14T09:33:01.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.464+0000 7fec477fe700 1 -- 172.21.15.2:0/2282910227 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec4000cc70 con 0x7fec50105480 2024-05-14T09:33:01.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.465+0000 7fec54da6700 1 -- 172.21.15.2:0/2282910227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5019ff60 con 0x7fec50105480 2024-05-14T09:33:01.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.465+0000 7fec477fe700 1 -- 172.21.15.2:0/2282910227 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4001b860 con 0x7fec50105480 2024-05-14T09:33:01.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.465+0000 7fec54da6700 1 -- 172.21.15.2:0/2282910227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec501030a0 con 0x7fec50105480 2024-05-14T09:33:01.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.466+0000 7fec54da6700 1 -- 172.21.15.2:0/2282910227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec50063f50 con 0x7fec50105480 2024-05-14T09:33:01.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.468+0000 7fec477fe700 1 -- 172.21.15.2:0/2282910227 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fec40023070 con 0x7fec50105480 2024-05-14T09:33:01.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.469+0000 7fec477fe700 1 --2- 172.21.15.2:0/2282910227 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fec3c0b2990 0x7fec3c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:01.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.469+0000 7fec4dd9b700 1 --2- 172.21.15.2:0/2282910227 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fec3c0b2990 0x7fec3c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:01.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.469+0000 7fec477fe700 1 -- 172.21.15.2:0/2282910227 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fec400e7200 con 0x7fec50105480 2024-05-14T09:33:01.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.470+0000 7fec477fe700 1 --2- 172.21.15.2:0/2282910227 >> [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451] conn(0x7fec3c0bde40 0x7fec3c0c0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:01.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.470+0000 7fec477fe700 1 -- 172.21.15.2:0/2282910227 --> [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec40009030 con 0x7fec3c0bde40 2024-05-14T09:33:01.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.470+0000 7fec477fe700 1 -- 172.21.15.2:0/2282910227 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fec400e7650 con 0x7fec50105480 2024-05-14T09:33:01.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.470+0000 7fec4ed9d700 1 --2- 172.21.15.2:0/2282910227 >> [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451] conn(0x7fec3c0bde40 0x7fec3c0c0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:01.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.470+0000 7fec4dd9b700 1 --2- 172.21.15.2:0/2282910227 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fec3c0b2990 0x7fec3c0b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fec501022e0 tx=0x7fec38009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:01.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.470+0000 7fec4ed9d700 1 --2- 172.21.15.2:0/2282910227 >> [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451] conn(0x7fec3c0bde40 0x7fec3c0c0330 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:01.471 INFO:teuthology.orchestra.run.smithi002.stdout:206158430268 2024-05-14T09:33:01.471 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-14T09:33:01.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.471+0000 7fec477fe700 1 -- 172.21.15.2:0/2282910227 <== osd.3 v2:172.21.15.2:6810/879639451 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec40009030 con 0x7fec3c0bde40 2024-05-14T09:33:01.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.489+0000 7fec54da6700 1 -- 172.21.15.2:0/2282910227 --> [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec50064380 con 0x7fec3c0bde40 2024-05-14T09:33:01.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.490+0000 7fec477fe700 1 -- 172.21.15.2:0/2282910227 <== osd.3 v2:172.21.15.2:6810/879639451 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fec50064380 con 0x7fec3c0bde40 2024-05-14T09:33:01.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.490+0000 7fec54da6700 1 -- 172.21.15.2:0/2282910227 >> [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451] conn(0x7fec3c0bde40 msgr2=0x7fec3c0c0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:01.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.490+0000 7fec54da6700 1 --2- 172.21.15.2:0/2282910227 >> [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451] conn(0x7fec3c0bde40 0x7fec3c0c0330 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:01.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.490+0000 7fec54da6700 1 -- 172.21.15.2:0/2282910227 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fec3c0b2990 msgr2=0x7fec3c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:01.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.490+0000 7fec54da6700 1 --2- 172.21.15.2:0/2282910227 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fec3c0b2990 0x7fec3c0b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fec501022e0 tx=0x7fec38009310 comp rx=0 tx=0).stop 2024-05-14T09:33:01.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.490+0000 7fec54da6700 1 -- 172.21.15.2:0/2282910227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec50105480 msgr2=0x7fec5019ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:01.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.490+0000 7fec54da6700 1 --2- 172.21.15.2:0/2282910227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec50105480 0x7fec5019ada0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fec4000e9a0 tx=0x7fec4000ed60 comp rx=0 tx=0).stop 2024-05-14T09:33:01.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.490+0000 7fec4ed9d700 1 -- 172.21.15.2:0/2282910227 reap_dead start 2024-05-14T09:33:01.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.490+0000 7fec54da6700 1 -- 172.21.15.2:0/2282910227 shutdown_connections 2024-05-14T09:33:01.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.490+0000 7fec54da6700 1 -- 172.21.15.2:0/2282910227 >> 172.21.15.2:0/2282910227 conn(0x7fec500fef30 msgr2=0x7fec500ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:01.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.490+0000 7fec54da6700 1 -- 172.21.15.2:0/2282910227 shutdown_connections 2024-05-14T09:33:01.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:01.490+0000 7fec54da6700 1 -- 172.21.15.2:0/2282910227 wait complete. 2024-05-14T09:33:02.248 INFO:teuthology.orchestra.run.smithi002.stdout:34359738465 2024-05-14T09:33:02.248 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-14T09:33:02.565 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:02 smithi002 ceph-mon[31557]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T09:33:02.901 INFO:teuthology.orchestra.run.smithi002.stdout:111669149777 2024-05-14T09:33:02.902 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-14T09:33:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:02 smithi029 ceph-mon[36249]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T09:33:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:02 smithi026 ceph-mon[36056]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T09:33:04.569 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:04 smithi002 ceph-mon[31557]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:04.606 INFO:teuthology.orchestra.run.smithi002.stdout:158913790022 2024-05-14T09:33:04.606 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-14T09:33:04.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.219+0000 7ff9c343d700 1 -- 172.21.15.2:0/4147763490 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9bc107df0 msgr2=0x7ff9bc108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:04.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.219+0000 7ff9c343d700 1 --2- 172.21.15.2:0/4147763490 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9bc107df0 0x7ff9bc108200 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0099c0 tx=0x7ff9ac009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:04.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.219+0000 7ff9c343d700 1 -- 172.21.15.2:0/4147763490 shutdown_connections 2024-05-14T09:33:04.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.219+0000 7ff9c343d700 1 --2- 172.21.15.2:0/4147763490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9bc1034c0 0x7ff9bc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:04.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.219+0000 7ff9c343d700 1 --2- 172.21.15.2:0/4147763490 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9bc108740 0x7ff9bc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:04.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.219+0000 7ff9c343d700 1 --2- 172.21.15.2:0/4147763490 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9bc107df0 0x7ff9bc108200 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:04.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.219+0000 7ff9c343d700 1 -- 172.21.15.2:0/4147763490 >> 172.21.15.2:0/4147763490 conn(0x7ff9bc0fedf0 msgr2=0x7ff9bc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:04.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.219+0000 7ff9c343d700 1 -- 172.21.15.2:0/4147763490 shutdown_connections 2024-05-14T09:33:04.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.219+0000 7ff9c343d700 1 -- 172.21.15.2:0/4147763490 wait complete. 2024-05-14T09:33:04.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.220+0000 7ff9c343d700 1 Processor -- start 2024-05-14T09:33:04.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.220+0000 7ff9c343d700 1 -- start start 2024-05-14T09:33:04.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.220+0000 7ff9c343d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9bc1034c0 0x7ff9bc19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:04.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.220+0000 7ff9c343d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9bc107df0 0x7ff9bc19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:04.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.220+0000 7ff9c343d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9bc108740 0x7ff9bc19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:04.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.220+0000 7ff9c343d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc113370 con 0x7ff9bc108740 2024-05-14T09:33:04.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.220+0000 7ff9c343d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc1131f0 con 0x7ff9bc107df0 2024-05-14T09:33:04.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.220+0000 7ff9c343d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc1134f0 con 0x7ff9bc1034c0 2024-05-14T09:33:04.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.220+0000 7ff9c19da700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9bc108740 0x7ff9bc19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:04.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.221+0000 7ff9c19da700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9bc108740 0x7ff9bc19f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38840/0 (socket says 172.21.15.2:38840) 2024-05-14T09:33:04.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.221+0000 7ff9c19da700 1 -- 172.21.15.2:0/2588583034 learned_addr learned my addr 172.21.15.2:0/2588583034 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:04.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.221+0000 7ff9c09d8700 1 --2- 172.21.15.2:0/2588583034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9bc107df0 0x7ff9bc19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:04.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.221+0000 7ff9c11d9700 1 --2- 172.21.15.2:0/2588583034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9bc1034c0 0x7ff9bc19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:04.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.221+0000 7ff9c19da700 1 -- 172.21.15.2:0/2588583034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9bc1034c0 msgr2=0x7ff9bc19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:04.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.221+0000 7ff9c19da700 1 --2- 172.21.15.2:0/2588583034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff9bc1034c0 0x7ff9bc19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:04.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.221+0000 7ff9c19da700 1 -- 172.21.15.2:0/2588583034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9bc107df0 msgr2=0x7ff9bc19b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:04.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.221+0000 7ff9c19da700 1 --2- 172.21.15.2:0/2588583034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9bc107df0 0x7ff9bc19b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:04.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.221+0000 7ff9c19da700 1 -- 172.21.15.2:0/2588583034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9bc19fd00 con 0x7ff9bc108740 2024-05-14T09:33:04.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.221+0000 7ff9c19da700 1 --2- 172.21.15.2:0/2588583034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9bc108740 0x7ff9bc19f660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4007ee0 tx=0x7ff9b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:04.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.221+0000 7ff9b27fc700 1 -- 172.21.15.2:0/2588583034 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b400dbd0 con 0x7ff9bc108740 2024-05-14T09:33:04.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.221+0000 7ff9c343d700 1 -- 172.21.15.2:0/2588583034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9bc19fff0 con 0x7ff9bc108740 2024-05-14T09:33:04.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.221+0000 7ff9b27fc700 1 -- 172.21.15.2:0/2588583034 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9b400dd70 con 0x7ff9bc108740 2024-05-14T09:33:04.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.221+0000 7ff9b27fc700 1 -- 172.21.15.2:0/2588583034 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4005990 con 0x7ff9bc108740 2024-05-14T09:33:04.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.221+0000 7ff9c343d700 1 -- 172.21.15.2:0/2588583034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9bc1a7910 con 0x7ff9bc108740 2024-05-14T09:33:04.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.222+0000 7ff9c343d700 1 -- 172.21.15.2:0/2588583034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff9bc063f10 con 0x7ff9bc108740 2024-05-14T09:33:04.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.224+0000 7ff9b27fc700 1 -- 172.21.15.2:0/2588583034 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7ff9b401a020 con 0x7ff9bc108740 2024-05-14T09:33:04.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.225+0000 7ff9b27fc700 1 --2- 172.21.15.2:0/2588583034 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff9a80b2910 0x7ff9a80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:04.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.225+0000 7ff9c11d9700 1 --2- 172.21.15.2:0/2588583034 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff9a80b2910 0x7ff9a80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:04.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.225+0000 7ff9b27fc700 1 -- 172.21.15.2:0/2588583034 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7ff9b40e28d0 con 0x7ff9bc108740 2024-05-14T09:33:04.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.225+0000 7ff9b27fc700 1 --2- 172.21.15.2:0/2588583034 >> [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226] conn(0x7ff9a80bddc0 0x7ff9a80c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:04.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.225+0000 7ff9b27fc700 1 -- 172.21.15.2:0/2588583034 --> [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9b401fe50 con 0x7ff9a80bddc0 2024-05-14T09:33:04.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.225+0000 7ff9b27fc700 1 -- 172.21.15.2:0/2588583034 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff9b40e2d20 con 0x7ff9bc108740 2024-05-14T09:33:04.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.225+0000 7ff9c11d9700 1 --2- 172.21.15.2:0/2588583034 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff9a80b2910 0x7ff9a80b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005950 tx=0x7ff9b80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:04.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.225+0000 7ff9c09d8700 1 --2- 172.21.15.2:0/2588583034 >> [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226] conn(0x7ff9a80bddc0 0x7ff9a80c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:04.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.226+0000 7ff9c09d8700 1 --2- 172.21.15.2:0/2588583034 >> [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226] conn(0x7ff9a80bddc0 0x7ff9a80c02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:04.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.227+0000 7ff9b27fc700 1 -- 172.21.15.2:0/2588583034 <== osd.1 v2:172.21.15.26:6800/3084747226 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9b401fe50 con 0x7ff9a80bddc0 2024-05-14T09:33:04.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.243+0000 7ff9c343d700 1 -- 172.21.15.2:0/2588583034 --> [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff9bc064310 con 0x7ff9a80bddc0 2024-05-14T09:33:04.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.244+0000 7ff9b27fc700 1 -- 172.21.15.2:0/2588583034 <== osd.1 v2:172.21.15.26:6800/3084747226 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff9bc064310 con 0x7ff9a80bddc0 2024-05-14T09:33:04.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.245+0000 7ff9c343d700 1 -- 172.21.15.2:0/2588583034 >> [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226] conn(0x7ff9a80bddc0 msgr2=0x7ff9a80c02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:04.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.245+0000 7ff9c343d700 1 --2- 172.21.15.2:0/2588583034 >> [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226] conn(0x7ff9a80bddc0 0x7ff9a80c02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:04.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.245+0000 7ff9c343d700 1 -- 172.21.15.2:0/2588583034 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff9a80b2910 msgr2=0x7ff9a80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:04.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.245+0000 7ff9c343d700 1 --2- 172.21.15.2:0/2588583034 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff9a80b2910 0x7ff9a80b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005950 tx=0x7ff9b80058e0 comp rx=0 tx=0).stop 2024-05-14T09:33:04.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.245+0000 7ff9c343d700 1 -- 172.21.15.2:0/2588583034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9bc108740 msgr2=0x7ff9bc19f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:04.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.245+0000 7ff9c343d700 1 --2- 172.21.15.2:0/2588583034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9bc108740 0x7ff9bc19f660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4007ee0 tx=0x7ff9b400d3b0 comp rx=0 tx=0).stop 2024-05-14T09:33:04.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.245+0000 7ff9c19da700 1 -- 172.21.15.2:0/2588583034 reap_dead start 2024-05-14T09:33:04.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.245+0000 7ff9c343d700 1 -- 172.21.15.2:0/2588583034 shutdown_connections 2024-05-14T09:33:04.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.245+0000 7ff9c343d700 1 -- 172.21.15.2:0/2588583034 >> 172.21.15.2:0/2588583034 conn(0x7ff9bc0fedf0 msgr2=0x7ff9bc075da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:04.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.245+0000 7ff9c343d700 1 -- 172.21.15.2:0/2588583034 shutdown_connections 2024-05-14T09:33:04.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.245+0000 7ff9c343d700 1 -- 172.21.15.2:0/2588583034 wait complete. 2024-05-14T09:33:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:04 smithi029 ceph-mon[36249]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:04 smithi026 ceph-mon[36056]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:05.424 INFO:teuthology.orchestra.run.smithi002.stdout:55834574941 2024-05-14T09:33:05.424 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-14T09:33:05.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.956+0000 7f9832fdd700 1 -- 172.21.15.2:0/4122747420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f982c1045e0 msgr2=0x7f982c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:05.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.956+0000 7f9832fdd700 1 --2- 172.21.15.2:0/4122747420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f982c1045e0 0x7f982c1049f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f981c0099c0 tx=0x7f981c009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:05.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.956+0000 7f9832fdd700 1 -- 172.21.15.2:0/4122747420 shutdown_connections 2024-05-14T09:33:05.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.956+0000 7f9832fdd700 1 --2- 172.21.15.2:0/4122747420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c109860 0x7f982c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:05.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.956+0000 7f9832fdd700 1 --2- 172.21.15.2:0/4122747420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f982c1045e0 0x7f982c1049f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:05.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.956+0000 7f9832fdd700 1 --2- 172.21.15.2:0/4122747420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f982c108f10 0x7f982c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:05.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.956+0000 7f9832fdd700 1 -- 172.21.15.2:0/4122747420 >> 172.21.15.2:0/4122747420 conn(0x7f982c0ffed0 msgr2=0x7f982c102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:05.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.956+0000 7f9832fdd700 1 -- 172.21.15.2:0/4122747420 shutdown_connections 2024-05-14T09:33:05.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.956+0000 7f9832fdd700 1 -- 172.21.15.2:0/4122747420 wait complete. 2024-05-14T09:33:05.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f9832fdd700 1 Processor -- start 2024-05-14T09:33:05.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f9832fdd700 1 -- start start 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f9832fdd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c1045e0 0x7f982c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f9832fdd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f982c108f10 0x7f982c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f9832fdd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f982c109860 0x7f982c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f9832fdd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f982c114500 con 0x7f982c108f10 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f9832fdd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f982c114380 con 0x7f982c1045e0 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f9832fdd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f982c114680 con 0x7f982c109860 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f982bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f982c108f10 0x7f982c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f982bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f982c108f10 0x7f982c19b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38860/0 (socket says 172.21.15.2:38860) 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f982bfff700 1 -- 172.21.15.2:0/2901306268 learned_addr learned my addr 172.21.15.2:0/2901306268 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f9830d79700 1 --2- 172.21.15.2:0/2901306268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c1045e0 0x7f982c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f983157a700 1 --2- 172.21.15.2:0/2901306268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f982c109860 0x7f982c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f982bfff700 1 -- 172.21.15.2:0/2901306268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f982c109860 msgr2=0x7f982c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f982bfff700 1 --2- 172.21.15.2:0/2901306268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f982c109860 0x7f982c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f982bfff700 1 -- 172.21.15.2:0/2901306268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c1045e0 msgr2=0x7f982c19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f982bfff700 1 --2- 172.21.15.2:0/2901306268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c1045e0 0x7f982c19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f982bfff700 1 -- 172.21.15.2:0/2901306268 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c19fd20 con 0x7f982c108f10 2024-05-14T09:33:05.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.957+0000 7f983157a700 1 --2- 172.21.15.2:0/2901306268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f982c109860 0x7f982c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:33:05.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.958+0000 7f982bfff700 1 --2- 172.21.15.2:0/2901306268 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f982c108f10 0x7f982c19b3f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9814007ee0 tx=0x7f981400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:05.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.958+0000 7f9829ffb700 1 -- 172.21.15.2:0/2901306268 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981400ddd0 con 0x7f982c108f10 2024-05-14T09:33:05.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.958+0000 7f9829ffb700 1 -- 172.21.15.2:0/2901306268 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9814004bb0 con 0x7f982c108f10 2024-05-14T09:33:05.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.958+0000 7f9829ffb700 1 -- 172.21.15.2:0/2901306268 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9814015400 con 0x7f982c108f10 2024-05-14T09:33:05.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.958+0000 7f9832fdd700 1 -- 172.21.15.2:0/2901306268 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982c1a0010 con 0x7f982c108f10 2024-05-14T09:33:05.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.958+0000 7f9832fdd700 1 -- 172.21.15.2:0/2901306268 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982c1a7930 con 0x7f982c108f10 2024-05-14T09:33:05.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.959+0000 7f9832fdd700 1 -- 172.21.15.2:0/2901306268 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f982c063f50 con 0x7f982c108f10 2024-05-14T09:33:05.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.961+0000 7f9829ffb700 1 -- 172.21.15.2:0/2901306268 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f9814004d60 con 0x7f982c108f10 2024-05-14T09:33:05.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.961+0000 7f9829ffb700 1 --2- 172.21.15.2:0/2901306268 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f98180b2910 0x7f98180b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:05.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.962+0000 7f9830d79700 1 --2- 172.21.15.2:0/2901306268 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f98180b2910 0x7f98180b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:05.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.962+0000 7f9829ffb700 1 -- 172.21.15.2:0/2901306268 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f98140e2e50 con 0x7f982c108f10 2024-05-14T09:33:05.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.962+0000 7f9829ffb700 1 --2- 172.21.15.2:0/2901306268 >> [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226] conn(0x7f98180bddc0 0x7f98180c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:05.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.962+0000 7f9829ffb700 1 -- 172.21.15.2:0/2901306268 --> [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9814011ec0 con 0x7f98180bddc0 2024-05-14T09:33:05.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.962+0000 7f9829ffb700 1 -- 172.21.15.2:0/2901306268 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f98140e32a0 con 0x7f982c108f10 2024-05-14T09:33:05.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.962+0000 7f983157a700 1 --2- 172.21.15.2:0/2901306268 >> [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226] conn(0x7f98180bddc0 0x7f98180c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:05.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.962+0000 7f9830d79700 1 --2- 172.21.15.2:0/2901306268 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f98180b2910 0x7f98180b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f981c009990 tx=0x7f981c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:05.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.962+0000 7f983157a700 1 --2- 172.21.15.2:0/2901306268 >> [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226] conn(0x7f98180bddc0 0x7f98180c02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:05.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.963+0000 7f9829ffb700 1 -- 172.21.15.2:0/2901306268 <== osd.9 v2:172.21.15.2:6826/176898226 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9814011ec0 con 0x7f98180bddc0 2024-05-14T09:33:05.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.981+0000 7f9832fdd700 1 -- 172.21.15.2:0/2901306268 --> [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f982c0643d0 con 0x7f98180bddc0 2024-05-14T09:33:05.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.982+0000 7f9829ffb700 1 -- 172.21.15.2:0/2901306268 <== osd.9 v2:172.21.15.2:6826/176898226 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f982c0643d0 con 0x7f98180bddc0 2024-05-14T09:33:05.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.982+0000 7f9832fdd700 1 -- 172.21.15.2:0/2901306268 >> [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226] conn(0x7f98180bddc0 msgr2=0x7f98180c02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:05.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.982+0000 7f9832fdd700 1 --2- 172.21.15.2:0/2901306268 >> [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226] conn(0x7f98180bddc0 0x7f98180c02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:05.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.982+0000 7f9832fdd700 1 -- 172.21.15.2:0/2901306268 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f98180b2910 msgr2=0x7f98180b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:05.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.982+0000 7f9832fdd700 1 --2- 172.21.15.2:0/2901306268 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f98180b2910 0x7f98180b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f981c009990 tx=0x7f981c000bc0 comp rx=0 tx=0).stop 2024-05-14T09:33:05.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.982+0000 7f9832fdd700 1 -- 172.21.15.2:0/2901306268 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f982c108f10 msgr2=0x7f982c19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:05.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.982+0000 7f9832fdd700 1 --2- 172.21.15.2:0/2901306268 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f982c108f10 0x7f982c19b3f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9814007ee0 tx=0x7f981400d3b0 comp rx=0 tx=0).stop 2024-05-14T09:33:05.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.982+0000 7f983157a700 1 -- 172.21.15.2:0/2901306268 reap_dead start 2024-05-14T09:33:05.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.983+0000 7f9832fdd700 1 -- 172.21.15.2:0/2901306268 shutdown_connections 2024-05-14T09:33:05.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.983+0000 7f9832fdd700 1 -- 172.21.15.2:0/2901306268 >> 172.21.15.2:0/2901306268 conn(0x7f982c0ffed0 msgr2=0x7f982c108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:05.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.983+0000 7f9832fdd700 1 -- 172.21.15.2:0/2901306268 shutdown_connections 2024-05-14T09:33:05.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:05.983+0000 7f9832fdd700 1 -- 172.21.15.2:0/2901306268 wait complete. 2024-05-14T09:33:06.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.443+0000 7fa34912e700 1 -- 172.21.15.2:0/3512061194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa344109870 msgr2=0x7fa3441111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:06.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.443+0000 7fa34912e700 1 --2- 172.21.15.2:0/3512061194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa344109870 0x7fa3441111d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00b0d0 tx=0x7fa33c00b3e0 comp rx=0 tx=0).stop 2024-05-14T09:33:06.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.443+0000 7fa34912e700 1 -- 172.21.15.2:0/3512061194 shutdown_connections 2024-05-14T09:33:06.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.443+0000 7fa34912e700 1 --2- 172.21.15.2:0/3512061194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3441045f0 0x7fa344104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:06.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.443+0000 7fa34912e700 1 --2- 172.21.15.2:0/3512061194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa344109870 0x7fa3441111d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:06.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.443+0000 7fa34912e700 1 --2- 172.21.15.2:0/3512061194 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa344108f20 0x7fa344109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:06.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.443+0000 7fa34912e700 1 -- 172.21.15.2:0/3512061194 >> 172.21.15.2:0/3512061194 conn(0x7fa344100040 msgr2=0x7fa344102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:06.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.443+0000 7fa34912e700 1 -- 172.21.15.2:0/3512061194 shutdown_connections 2024-05-14T09:33:06.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.443+0000 7fa34912e700 1 -- 172.21.15.2:0/3512061194 wait complete. 2024-05-14T09:33:06.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.444+0000 7fa34912e700 1 Processor -- start 2024-05-14T09:33:06.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.444+0000 7fa34912e700 1 -- start start 2024-05-14T09:33:06.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.444+0000 7fa34912e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3441045f0 0x7fa34419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:06.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.444+0000 7fa34912e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa344108f20 0x7fa34419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:06.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.444+0000 7fa34912e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa344109870 0x7fa34419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:06.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.444+0000 7fa34912e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441144a0 con 0x7fa344108f20 2024-05-14T09:33:06.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.444+0000 7fa34912e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa344114320 con 0x7fa3441045f0 2024-05-14T09:33:06.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.444+0000 7fa34912e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa344114620 con 0x7fa344109870 2024-05-14T09:33:06.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.444+0000 7fa34259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa344108f20 0x7fa34419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:06.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.444+0000 7fa34359e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa344109870 0x7fa34419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:06.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.444+0000 7fa34259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa344108f20 0x7fa34419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38866/0 (socket says 172.21.15.2:38866) 2024-05-14T09:33:06.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.444+0000 7fa34259c700 1 -- 172.21.15.2:0/1476881650 learned_addr learned my addr 172.21.15.2:0/1476881650 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:06.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.444+0000 7fa342d9d700 1 --2- 172.21.15.2:0/1476881650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3441045f0 0x7fa34419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:06.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.445+0000 7fa34259c700 1 -- 172.21.15.2:0/1476881650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa344109870 msgr2=0x7fa34419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:06.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.445+0000 7fa34259c700 1 --2- 172.21.15.2:0/1476881650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa344109870 0x7fa34419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:06.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.445+0000 7fa34259c700 1 -- 172.21.15.2:0/1476881650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3441045f0 msgr2=0x7fa34419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:06.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.445+0000 7fa34259c700 1 --2- 172.21.15.2:0/1476881650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3441045f0 0x7fa34419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:06.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.445+0000 7fa34259c700 1 -- 172.21.15.2:0/1476881650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34419fc80 con 0x7fa344108f20 2024-05-14T09:33:06.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.445+0000 7fa342d9d700 1 --2- 172.21.15.2:0/1476881650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3441045f0 0x7fa34419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:33:06.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.445+0000 7fa34359e700 1 --2- 172.21.15.2:0/1476881650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa344109870 0x7fa34419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:33:06.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.445+0000 7fa34259c700 1 --2- 172.21.15.2:0/1476881650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa344108f20 0x7fa34419b350 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa33800d980 tx=0x7fa33800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:06.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.445+0000 7fa32bfff700 1 -- 172.21.15.2:0/1476881650 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338017070 con 0x7fa344108f20 2024-05-14T09:33:06.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.445+0000 7fa32bfff700 1 -- 172.21.15.2:0/1476881650 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa33800fd40 con 0x7fa344108f20 2024-05-14T09:33:06.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.445+0000 7fa34912e700 1 -- 172.21.15.2:0/1476881650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34419ff70 con 0x7fa344108f20 2024-05-14T09:33:06.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.446+0000 7fa32bfff700 1 -- 172.21.15.2:0/1476881650 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33800e3b0 con 0x7fa344108f20 2024-05-14T09:33:06.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.446+0000 7fa34912e700 1 -- 172.21.15.2:0/1476881650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3441a7890 con 0x7fa344108f20 2024-05-14T09:33:06.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.446+0000 7fa34912e700 1 -- 172.21.15.2:0/1476881650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa344063f50 con 0x7fa344108f20 2024-05-14T09:33:06.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.448+0000 7fa32bfff700 1 -- 172.21.15.2:0/1476881650 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fa338010610 con 0x7fa344108f20 2024-05-14T09:33:06.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.449+0000 7fa32bfff700 1 --2- 172.21.15.2:0/1476881650 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa32c0b2910 0x7fa32c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:06.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.449+0000 7fa342d9d700 1 --2- 172.21.15.2:0/1476881650 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa32c0b2910 0x7fa32c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:06.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.449+0000 7fa32bfff700 1 -- 172.21.15.2:0/1476881650 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fa3380e2530 con 0x7fa344108f20 2024-05-14T09:33:06.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.449+0000 7fa32bfff700 1 --2- 172.21.15.2:0/1476881650 >> [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752] conn(0x7fa32c0bddc0 0x7fa32c0c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:06.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.449+0000 7fa32bfff700 1 -- 172.21.15.2:0/1476881650 --> [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa338016ec0 con 0x7fa32c0bddc0 2024-05-14T09:33:06.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.449+0000 7fa32bfff700 1 -- 172.21.15.2:0/1476881650 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fa3380e2980 con 0x7fa344108f20 2024-05-14T09:33:06.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.449+0000 7fa342d9d700 1 --2- 172.21.15.2:0/1476881650 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa32c0b2910 0x7fa32c0b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa334007fd0 tx=0x7fa3340073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:06.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.450+0000 7fa34359e700 1 --2- 172.21.15.2:0/1476881650 >> [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752] conn(0x7fa32c0bddc0 0x7fa32c0c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:06.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.450+0000 7fa34359e700 1 --2- 172.21.15.2:0/1476881650 >> [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752] conn(0x7fa32c0bddc0 0x7fa32c0c02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:06.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.451+0000 7fa32bfff700 1 -- 172.21.15.2:0/1476881650 <== osd.4 v2:172.21.15.26:6808/457100752 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa338016ec0 con 0x7fa32c0bddc0 2024-05-14T09:33:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.468+0000 7fa34912e700 1 -- 172.21.15.2:0/1476881650 --> [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa344064350 con 0x7fa32c0bddc0 2024-05-14T09:33:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.469+0000 7fa32bfff700 1 -- 172.21.15.2:0/1476881650 <== osd.4 v2:172.21.15.26:6808/457100752 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa344064350 con 0x7fa32c0bddc0 2024-05-14T09:33:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.469+0000 7fa34912e700 1 -- 172.21.15.2:0/1476881650 >> [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752] conn(0x7fa32c0bddc0 msgr2=0x7fa32c0c02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.469+0000 7fa34912e700 1 --2- 172.21.15.2:0/1476881650 >> [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752] conn(0x7fa32c0bddc0 0x7fa32c0c02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.469+0000 7fa34912e700 1 -- 172.21.15.2:0/1476881650 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa32c0b2910 msgr2=0x7fa32c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.469+0000 7fa34912e700 1 --2- 172.21.15.2:0/1476881650 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa32c0b2910 0x7fa32c0b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa334007fd0 tx=0x7fa3340073d0 comp rx=0 tx=0).stop 2024-05-14T09:33:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.469+0000 7fa34912e700 1 -- 172.21.15.2:0/1476881650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa344108f20 msgr2=0x7fa34419b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.469+0000 7fa34912e700 1 --2- 172.21.15.2:0/1476881650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa344108f20 0x7fa34419b350 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa33800d980 tx=0x7fa33800dc90 comp rx=0 tx=0).stop 2024-05-14T09:33:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.469+0000 7fa34359e700 1 -- 172.21.15.2:0/1476881650 reap_dead start 2024-05-14T09:33:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.469+0000 7fa34912e700 1 -- 172.21.15.2:0/1476881650 shutdown_connections 2024-05-14T09:33:06.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.469+0000 7fa34912e700 1 -- 172.21.15.2:0/1476881650 >> 172.21.15.2:0/1476881650 conn(0x7fa344100040 msgr2=0x7fa344108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:06.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.469+0000 7fa34912e700 1 -- 172.21.15.2:0/1476881650 shutdown_connections 2024-05-14T09:33:06.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:06.469+0000 7fa34912e700 1 -- 172.21.15.2:0/1476881650 wait complete. 2024-05-14T09:33:06.480 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:06 smithi002 ceph-mon[31557]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:06.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.426+0000 7fd9ff527700 1 -- 172.21.15.2:0/1810967934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9f8107df0 msgr2=0x7fd9f8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:06.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.426+0000 7fd9ff527700 1 --2- 172.21.15.2:0/1810967934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9f8107df0 0x7fd9f8108200 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80099c0 tx=0x7fd9e8009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:06.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.427+0000 7fd9ff527700 1 -- 172.21.15.2:0/1810967934 shutdown_connections 2024-05-14T09:33:06.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.427+0000 7fd9ff527700 1 --2- 172.21.15.2:0/1810967934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8108740 0x7fd9f81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:06.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.427+0000 7fd9ff527700 1 --2- 172.21.15.2:0/1810967934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9f81034c0 0x7fd9f81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:06.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.427+0000 7fd9ff527700 1 --2- 172.21.15.2:0/1810967934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9f8107df0 0x7fd9f8108200 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:06.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.427+0000 7fd9ff527700 1 -- 172.21.15.2:0/1810967934 >> 172.21.15.2:0/1810967934 conn(0x7fd9f80fedc0 msgr2=0x7fd9f81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:06.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.427+0000 7fd9ff527700 1 -- 172.21.15.2:0/1810967934 shutdown_connections 2024-05-14T09:33:06.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.427+0000 7fd9ff527700 1 -- 172.21.15.2:0/1810967934 wait complete. 2024-05-14T09:33:06.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.427+0000 7fd9ff527700 1 Processor -- start 2024-05-14T09:33:06.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.427+0000 7fd9ff527700 1 -- start start 2024-05-14T09:33:06.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9ff527700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9f81034c0 0x7fd9f819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:06.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9ff527700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8107df0 0x7fd9f819c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:06.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9ff527700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9f8108740 0x7fd9f819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:06.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9ff527700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f8113350 con 0x7fd9f8108740 2024-05-14T09:33:06.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9ff527700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f81131d0 con 0x7fd9f8107df0 2024-05-14T09:33:06.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9ff527700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f81134d0 con 0x7fd9f81034c0 2024-05-14T09:33:06.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9fdac4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9f8108740 0x7fd9f819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:06.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9fdac4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9f8108740 0x7fd9f819c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38846/0 (socket says 172.21.15.2:38846) 2024-05-14T09:33:06.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9fdac4700 1 -- 172.21.15.2:0/3600190112 learned_addr learned my addr 172.21.15.2:0/3600190112 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:06.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9fd2c3700 1 --2- 172.21.15.2:0/3600190112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9f81034c0 0x7fd9f819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:06.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9fcac2700 1 --2- 172.21.15.2:0/3600190112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8107df0 0x7fd9f819c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:06.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9fdac4700 1 -- 172.21.15.2:0/3600190112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9f81034c0 msgr2=0x7fd9f819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:06.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9fdac4700 1 --2- 172.21.15.2:0/3600190112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9f81034c0 0x7fd9f819bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:06.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9fdac4700 1 -- 172.21.15.2:0/3600190112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8107df0 msgr2=0x7fd9f819c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:06.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9fdac4700 1 --2- 172.21.15.2:0/3600190112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8107df0 0x7fd9f819c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:06.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9fdac4700 1 -- 172.21.15.2:0/3600190112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f81a6f70 con 0x7fd9f8108740 2024-05-14T09:33:06.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9fdac4700 1 --2- 172.21.15.2:0/3600190112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9f8108740 0x7fd9f819c750 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0007d70 tx=0x7fd9f000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:06.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.428+0000 7fd9ee7fc700 1 -- 172.21.15.2:0/3600190112 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0016020 con 0x7fd9f8108740 2024-05-14T09:33:06.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.429+0000 7fd9ee7fc700 1 -- 172.21.15.2:0/3600190112 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9f000ad40 con 0x7fd9f8108740 2024-05-14T09:33:06.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.429+0000 7fd9ff527700 1 -- 172.21.15.2:0/3600190112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f81a7260 con 0x7fd9f8108740 2024-05-14T09:33:06.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.429+0000 7fd9ee7fc700 1 -- 172.21.15.2:0/3600190112 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00115e0 con 0x7fd9f8108740 2024-05-14T09:33:06.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.429+0000 7fd9ff527700 1 -- 172.21.15.2:0/3600190112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f81a7720 con 0x7fd9f8108740 2024-05-14T09:33:06.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.429+0000 7fd9ff527700 1 -- 172.21.15.2:0/3600190112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd9f8063e90 con 0x7fd9f8108740 2024-05-14T09:33:06.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.431+0000 7fd9ee7fc700 1 -- 172.21.15.2:0/3600190112 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fd9f0019020 con 0x7fd9f8108740 2024-05-14T09:33:06.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.432+0000 7fd9ee7fc700 1 --2- 172.21.15.2:0/3600190112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9e40b2910 0x7fd9e40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:06.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.432+0000 7fd9ee7fc700 1 -- 172.21.15.2:0/3600190112 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd9f00e2cd0 con 0x7fd9f8108740 2024-05-14T09:33:06.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.432+0000 7fd9fd2c3700 1 --2- 172.21.15.2:0/3600190112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9e40b2910 0x7fd9e40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:06.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.432+0000 7fd9ee7fc700 1 --2- 172.21.15.2:0/3600190112 >> [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417] conn(0x7fd9e40bddc0 0x7fd9e40c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:06.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.432+0000 7fd9ee7fc700 1 -- 172.21.15.2:0/3600190112 --> [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd9f001ce50 con 0x7fd9e40bddc0 2024-05-14T09:33:06.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.432+0000 7fd9ee7fc700 1 -- 172.21.15.2:0/3600190112 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fd9f00e3080 con 0x7fd9f8108740 2024-05-14T09:33:06.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.432+0000 7fd9fd2c3700 1 --2- 172.21.15.2:0/3600190112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9e40b2910 0x7fd9e40b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4005950 tx=0x7fd9f400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:06.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.433+0000 7fd9fcac2700 1 --2- 172.21.15.2:0/3600190112 >> [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417] conn(0x7fd9e40bddc0 0x7fd9e40c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:06.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.433+0000 7fd9fcac2700 1 --2- 172.21.15.2:0/3600190112 >> [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417] conn(0x7fd9e40bddc0 0x7fd9e40c02b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:06.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.434+0000 7fd9ee7fc700 1 -- 172.21.15.2:0/3600190112 <== osd.11 v2:172.21.15.29:6824/378800417 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9f001ce50 con 0x7fd9e40bddc0 2024-05-14T09:33:06.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.451+0000 7fd9ff527700 1 -- 172.21.15.2:0/3600190112 --> [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd9f8064310 con 0x7fd9e40bddc0 2024-05-14T09:33:06.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.452+0000 7fd9ee7fc700 1 -- 172.21.15.2:0/3600190112 <== osd.11 v2:172.21.15.29:6824/378800417 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd9f8064310 con 0x7fd9e40bddc0 2024-05-14T09:33:06.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.452+0000 7fd9ff527700 1 -- 172.21.15.2:0/3600190112 >> [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417] conn(0x7fd9e40bddc0 msgr2=0x7fd9e40c02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:06.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.452+0000 7fd9ff527700 1 --2- 172.21.15.2:0/3600190112 >> [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417] conn(0x7fd9e40bddc0 0x7fd9e40c02b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:06.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.452+0000 7fd9ff527700 1 -- 172.21.15.2:0/3600190112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9e40b2910 msgr2=0x7fd9e40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:06.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.452+0000 7fd9ff527700 1 --2- 172.21.15.2:0/3600190112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9e40b2910 0x7fd9e40b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4005950 tx=0x7fd9f400b410 comp rx=0 tx=0).stop 2024-05-14T09:33:06.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.452+0000 7fd9ff527700 1 -- 172.21.15.2:0/3600190112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9f8108740 msgr2=0x7fd9f819c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:06.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.452+0000 7fd9ff527700 1 --2- 172.21.15.2:0/3600190112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9f8108740 0x7fd9f819c750 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0007d70 tx=0x7fd9f000a340 comp rx=0 tx=0).stop 2024-05-14T09:33:06.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.452+0000 7fd9fdac4700 1 -- 172.21.15.2:0/3600190112 reap_dead start 2024-05-14T09:33:06.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.452+0000 7fd9ff527700 1 -- 172.21.15.2:0/3600190112 shutdown_connections 2024-05-14T09:33:06.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.452+0000 7fd9ff527700 1 -- 172.21.15.2:0/3600190112 >> 172.21.15.2:0/3600190112 conn(0x7fd9f80fedc0 msgr2=0x7fd9f81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:06.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.452+0000 7fd9ff527700 1 -- 172.21.15.2:0/3600190112 shutdown_connections 2024-05-14T09:33:06.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:03.452+0000 7fd9ff527700 1 -- 172.21.15.2:0/3600190112 wait complete. 2024-05-14T09:33:06.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:06 smithi029 ceph-mon[36249]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:06.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:06 smithi026 ceph-mon[36056]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:07.171 INFO:teuthology.orchestra.run.smithi002.stdout:304942678056 2024-05-14T09:33:07.171 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-14T09:33:07.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.402+0000 7fe8f680d700 1 -- 172.21.15.2:0/1934798422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8f01034c0 msgr2=0x7fe8f01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:07.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.402+0000 7fe8f680d700 1 --2- 172.21.15.2:0/1934798422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8f01034c0 0x7fe8f01038d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00099c0 tx=0x7fe8e0009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:07.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.402+0000 7fe8f680d700 1 -- 172.21.15.2:0/1934798422 shutdown_connections 2024-05-14T09:33:07.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.402+0000 7fe8f680d700 1 --2- 172.21.15.2:0/1934798422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f0108740 0x7fe8f01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:07.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.402+0000 7fe8f680d700 1 --2- 172.21.15.2:0/1934798422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8f01034c0 0x7fe8f01038d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:07.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.402+0000 7fe8f680d700 1 --2- 172.21.15.2:0/1934798422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8f0107df0 0x7fe8f0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:07.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.402+0000 7fe8f680d700 1 -- 172.21.15.2:0/1934798422 >> 172.21.15.2:0/1934798422 conn(0x7fe8f00fedc0 msgr2=0x7fe8f01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:07.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.402+0000 7fe8f680d700 1 -- 172.21.15.2:0/1934798422 shutdown_connections 2024-05-14T09:33:07.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.402+0000 7fe8f680d700 1 -- 172.21.15.2:0/1934798422 wait complete. 2024-05-14T09:33:07.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.402+0000 7fe8f680d700 1 Processor -- start 2024-05-14T09:33:07.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8f680d700 1 -- start start 2024-05-14T09:33:07.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8f680d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8f01034c0 0x7fe8f019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:07.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8f680d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8f0107df0 0x7fe8f019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:07.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8f680d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f0108740 0x7fe8f0199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:07.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8f680d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f0113370 con 0x7fe8f0107df0 2024-05-14T09:33:07.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8f680d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01131f0 con 0x7fe8f01034c0 2024-05-14T09:33:07.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8f680d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01134f0 con 0x7fe8f0108740 2024-05-14T09:33:07.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8e7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8f0107df0 0x7fe8f019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:07.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8e7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8f0107df0 0x7fe8f019fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38882/0 (socket says 172.21.15.2:38882) 2024-05-14T09:33:07.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8e7fff700 1 -- 172.21.15.2:0/3181923616 learned_addr learned my addr 172.21.15.2:0/3181923616 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:07.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8effff700 1 --2- 172.21.15.2:0/3181923616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8f01034c0 0x7fe8f019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:07.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8f4daa700 1 --2- 172.21.15.2:0/3181923616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f0108740 0x7fe8f0199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:07.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8e7fff700 1 -- 172.21.15.2:0/3181923616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f0108740 msgr2=0x7fe8f0199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:07.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8e7fff700 1 --2- 172.21.15.2:0/3181923616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f0108740 0x7fe8f0199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:07.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8e7fff700 1 -- 172.21.15.2:0/3181923616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8f01034c0 msgr2=0x7fe8f019f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:07.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8e7fff700 1 --2- 172.21.15.2:0/3181923616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8f01034c0 0x7fe8f019f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:07.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.403+0000 7fe8e7fff700 1 -- 172.21.15.2:0/3181923616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f019a0f0 con 0x7fe8f0107df0 2024-05-14T09:33:07.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.404+0000 7fe8e7fff700 1 --2- 172.21.15.2:0/3181923616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8f0107df0 0x7fe8f019fdd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800c9e0 tx=0x7fe8d800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:07.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.404+0000 7fe8edffb700 1 -- 172.21.15.2:0/3181923616 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8007aa0 con 0x7fe8f0107df0 2024-05-14T09:33:07.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.404+0000 7fe8edffb700 1 -- 172.21.15.2:0/3181923616 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8d8007c40 con 0x7fe8f0107df0 2024-05-14T09:33:07.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.404+0000 7fe8f680d700 1 -- 172.21.15.2:0/3181923616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f019a3e0 con 0x7fe8f0107df0 2024-05-14T09:33:07.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.404+0000 7fe8edffb700 1 -- 172.21.15.2:0/3181923616 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d800fe60 con 0x7fe8f0107df0 2024-05-14T09:33:07.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.404+0000 7fe8f680d700 1 -- 172.21.15.2:0/3181923616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f01a6720 con 0x7fe8f0107df0 2024-05-14T09:33:07.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.405+0000 7fe8f680d700 1 -- 172.21.15.2:0/3181923616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe8f0063f50 con 0x7fe8f0107df0 2024-05-14T09:33:07.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.406+0000 7fe8edffb700 1 -- 172.21.15.2:0/3181923616 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fe8d800f450 con 0x7fe8f0107df0 2024-05-14T09:33:07.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.407+0000 7fe8edffb700 1 --2- 172.21.15.2:0/3181923616 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8dc0b2910 0x7fe8dc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:07.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.407+0000 7fe8effff700 1 --2- 172.21.15.2:0/3181923616 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8dc0b2910 0x7fe8dc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:07.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.407+0000 7fe8edffb700 1 -- 172.21.15.2:0/3181923616 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fe8d80e2b30 con 0x7fe8f0107df0 2024-05-14T09:33:07.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.408+0000 7fe8edffb700 1 --2- 172.21.15.2:0/3181923616 >> [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086] conn(0x7fe8dc0bddc0 0x7fe8dc0c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:07.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.408+0000 7fe8edffb700 1 -- 172.21.15.2:0/3181923616 --> [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe8dc0c0980 con 0x7fe8dc0bddc0 2024-05-14T09:33:07.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.408+0000 7fe8edffb700 1 -- 172.21.15.2:0/3181923616 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fe8d80e2f80 con 0x7fe8f0107df0 2024-05-14T09:33:07.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.408+0000 7fe8effff700 1 --2- 172.21.15.2:0/3181923616 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8dc0b2910 0x7fe8dc0b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0006010 tx=0x7fe8e0009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:07.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.408+0000 7fe8f4daa700 1 --2- 172.21.15.2:0/3181923616 >> [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086] conn(0x7fe8dc0bddc0 0x7fe8dc0c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:07.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.408+0000 7fe8f4daa700 1 --2- 172.21.15.2:0/3181923616 >> [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086] conn(0x7fe8dc0bddc0 0x7fe8dc0c02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:07.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.409+0000 7fe8edffb700 1 -- 172.21.15.2:0/3181923616 <== osd.2 v2:172.21.15.29:6800/814048086 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe8dc0c0980 con 0x7fe8dc0bddc0 2024-05-14T09:33:07.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.426+0000 7fe8f680d700 1 -- 172.21.15.2:0/3181923616 --> [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe8f00643d0 con 0x7fe8dc0bddc0 2024-05-14T09:33:07.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.427+0000 7fe8edffb700 1 -- 172.21.15.2:0/3181923616 <== osd.2 v2:172.21.15.29:6800/814048086 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe8f00643d0 con 0x7fe8dc0bddc0 2024-05-14T09:33:07.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.427+0000 7fe8f680d700 1 -- 172.21.15.2:0/3181923616 >> [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086] conn(0x7fe8dc0bddc0 msgr2=0x7fe8dc0c02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:07.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.427+0000 7fe8f680d700 1 --2- 172.21.15.2:0/3181923616 >> [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086] conn(0x7fe8dc0bddc0 0x7fe8dc0c02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:07.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.427+0000 7fe8f680d700 1 -- 172.21.15.2:0/3181923616 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8dc0b2910 msgr2=0x7fe8dc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:07.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.427+0000 7fe8f680d700 1 --2- 172.21.15.2:0/3181923616 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8dc0b2910 0x7fe8dc0b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0006010 tx=0x7fe8e0009f10 comp rx=0 tx=0).stop 2024-05-14T09:33:07.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.427+0000 7fe8f680d700 1 -- 172.21.15.2:0/3181923616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8f0107df0 msgr2=0x7fe8f019fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:07.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.427+0000 7fe8f680d700 1 --2- 172.21.15.2:0/3181923616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8f0107df0 0x7fe8f019fdd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800c9e0 tx=0x7fe8d800ccf0 comp rx=0 tx=0).stop 2024-05-14T09:33:07.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.427+0000 7fe8f4daa700 1 -- 172.21.15.2:0/3181923616 reap_dead start 2024-05-14T09:33:07.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.427+0000 7fe8f680d700 1 -- 172.21.15.2:0/3181923616 shutdown_connections 2024-05-14T09:33:07.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.427+0000 7fe8f680d700 1 -- 172.21.15.2:0/3181923616 >> 172.21.15.2:0/3181923616 conn(0x7fe8f00fedc0 msgr2=0x7fe8f0107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:07.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.427+0000 7fe8f680d700 1 -- 172.21.15.2:0/3181923616 shutdown_connections 2024-05-14T09:33:07.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:07.428+0000 7fe8f680d700 1 -- 172.21.15.2:0/3181923616 wait complete. 2024-05-14T09:33:08.068 INFO:teuthology.orchestra.run.smithi002.stdout:137438953550 2024-05-14T09:33:08.069 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-14T09:33:08.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:08 smithi002 ceph-mon[31557]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:33:08.739 INFO:teuthology.orchestra.run.smithi002.stdout:77309411417 2024-05-14T09:33:08.739 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-14T09:33:08.740 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:08 smithi029 ceph-mon[36249]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:33:08.753 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:08 smithi026 ceph-mon[36056]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:33:09.394 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:09.394 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:09.431 INFO:teuthology.orchestra.run.smithi002.stdout:257698037812 2024-05-14T09:33:09.431 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-14T09:33:09.442 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:09.453 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:09.458 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:09.458 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:10 smithi002 ceph-mon[31557]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T09:33:10.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:10 smithi029 ceph-mon[36249]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T09:33:10.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:10 smithi026 ceph-mon[36056]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T09:33:12.807 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:12 smithi002 ceph-mon[31557]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:12.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:12 smithi029 ceph-mon[36249]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:12.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:12 smithi026 ceph-mon[36056]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:14.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:14 smithi002 ceph-mon[31557]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T09:33:14.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:14 smithi029 ceph-mon[36249]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T09:33:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:14 smithi026 ceph-mon[36056]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-14T09:33:15.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.142+0000 7fbcbd2ee700 1 -- 172.21.15.2:0/4027827315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcb8108740 msgr2=0x7fbcb81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:15.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.142+0000 7fbcbd2ee700 1 --2- 172.21.15.2:0/4027827315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcb8108740 0x7fbcb81100a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000b0d0 tx=0x7fbcb000b3e0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.142+0000 7fbcbd2ee700 1 -- 172.21.15.2:0/4027827315 shutdown_connections 2024-05-14T09:33:15.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.142+0000 7fbcbd2ee700 1 --2- 172.21.15.2:0/4027827315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcb8108740 0x7fbcb81100a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.142+0000 7fbcbd2ee700 1 --2- 172.21.15.2:0/4027827315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbcb81034c0 0x7fbcb81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.142+0000 7fbcbd2ee700 1 --2- 172.21.15.2:0/4027827315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb8107df0 0x7fbcb8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.142+0000 7fbcbd2ee700 1 -- 172.21.15.2:0/4027827315 >> 172.21.15.2:0/4027827315 conn(0x7fbcb80fedc0 msgr2=0x7fbcb81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:15.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.142+0000 7fbcbd2ee700 1 -- 172.21.15.2:0/4027827315 shutdown_connections 2024-05-14T09:33:15.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.142+0000 7fbcbd2ee700 1 -- 172.21.15.2:0/4027827315 wait complete. 2024-05-14T09:33:15.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.143+0000 7fbcbd2ee700 1 Processor -- start 2024-05-14T09:33:15.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.143+0000 7fbcbd2ee700 1 -- start start 2024-05-14T09:33:15.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.143+0000 7fbcbd2ee700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbcb81034c0 0x7fbcb819bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:15.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.143+0000 7fbcbd2ee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb8107df0 0x7fbcb819c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:15.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.143+0000 7fbcbd2ee700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcb8108740 0x7fbcb81a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:15.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.143+0000 7fbcbd2ee700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb81133c0 con 0x7fbcb8108740 2024-05-14T09:33:15.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.143+0000 7fbcbd2ee700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8113240 con 0x7fbcb8107df0 2024-05-14T09:33:15.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.143+0000 7fbcbd2ee700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8113540 con 0x7fbcb81034c0 2024-05-14T09:33:15.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbcb77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcb8108740 0x7fbcb81a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:15.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbcb77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcb8108740 0x7fbcb81a6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32802/0 (socket says 172.21.15.2:32802) 2024-05-14T09:33:15.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbcb77fe700 1 -- 172.21.15.2:0/3404448662 learned_addr learned my addr 172.21.15.2:0/3404448662 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:15.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbcb67fc700 1 --2- 172.21.15.2:0/3404448662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb8107df0 0x7fbcb819c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:15.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbcb6ffd700 1 --2- 172.21.15.2:0/3404448662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbcb81034c0 0x7fbcb819bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:15.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbcb77fe700 1 -- 172.21.15.2:0/3404448662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbcb81034c0 msgr2=0x7fbcb819bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:15.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbcb77fe700 1 --2- 172.21.15.2:0/3404448662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbcb81034c0 0x7fbcb819bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbcb77fe700 1 -- 172.21.15.2:0/3404448662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb8107df0 msgr2=0x7fbcb819c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:15.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbcb77fe700 1 --2- 172.21.15.2:0/3404448662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb8107df0 0x7fbcb819c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbcb77fe700 1 -- 172.21.15.2:0/3404448662 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb819c980 con 0x7fbcb8108740 2024-05-14T09:33:15.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbcb6ffd700 1 --2- 172.21.15.2:0/3404448662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbcb81034c0 0x7fbcb819bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:33:15.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbcb77fe700 1 --2- 172.21.15.2:0/3404448662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcb8108740 0x7fbcb81a6ca0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d010 tx=0x7fbcb00079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:15.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbc9ffff700 1 -- 172.21.15.2:0/3404448662 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb001f070 con 0x7fbcb8108740 2024-05-14T09:33:15.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbc9ffff700 1 -- 172.21.15.2:0/3404448662 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcb0003c00 con 0x7fbcb8108740 2024-05-14T09:33:15.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbcbd2ee700 1 -- 172.21.15.2:0/3404448662 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb81a71e0 con 0x7fbcb8108740 2024-05-14T09:33:15.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.144+0000 7fbc9ffff700 1 -- 172.21.15.2:0/3404448662 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb001b6a0 con 0x7fbcb8108740 2024-05-14T09:33:15.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.145+0000 7fbcbd2ee700 1 -- 172.21.15.2:0/3404448662 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb81a7630 con 0x7fbcb8108740 2024-05-14T09:33:15.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.146+0000 7fbcbd2ee700 1 -- 172.21.15.2:0/3404448662 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb80512c0 con 0x7fbcb8108740 2024-05-14T09:33:15.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.147+0000 7fbc9ffff700 1 -- 172.21.15.2:0/3404448662 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fbcb0022070 con 0x7fbcb8108740 2024-05-14T09:33:15.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.148+0000 7fbc9ffff700 1 --2- 172.21.15.2:0/3404448662 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbca00b2910 0x7fbca00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:15.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.148+0000 7fbcb6ffd700 1 --2- 172.21.15.2:0/3404448662 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbca00b2910 0x7fbca00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:15.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.148+0000 7fbc9ffff700 1 -- 172.21.15.2:0/3404448662 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fbcb000f020 con 0x7fbcb8108740 2024-05-14T09:33:15.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.149+0000 7fbcb6ffd700 1 --2- 172.21.15.2:0/3404448662 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbca00b2910 0x7fbca00b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbca8005950 tx=0x7fbca80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:15.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.152+0000 7fbc9ffff700 1 -- 172.21.15.2:0/3404448662 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbcb0029020 con 0x7fbcb8108740 2024-05-14T09:33:15.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.353+0000 7f494e9e7700 1 -- 172.21.15.2:0/577606539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948102b50 msgr2=0x7f4948102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:15.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.353+0000 7f494e9e7700 1 --2- 172.21.15.2:0/577606539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948102b50 0x7f4948102f60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f49380099c0 tx=0x7f4938009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.353+0000 7f494e9e7700 1 -- 172.21.15.2:0/577606539 shutdown_connections 2024-05-14T09:33:15.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.353+0000 7f494e9e7700 1 --2- 172.21.15.2:0/577606539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4948109850 0x7f49481111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.353+0000 7f494e9e7700 1 --2- 172.21.15.2:0/577606539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948102b50 0x7f4948102f60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.353+0000 7f494e9e7700 1 --2- 172.21.15.2:0/577606539 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4948108f00 0x7f4948109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.353+0000 7f494e9e7700 1 -- 172.21.15.2:0/577606539 >> 172.21.15.2:0/577606539 conn(0x7f49480fe6f0 msgr2=0x7f4948100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:15.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.353+0000 7f494e9e7700 1 -- 172.21.15.2:0/577606539 shutdown_connections 2024-05-14T09:33:15.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.353+0000 7f494e9e7700 1 -- 172.21.15.2:0/577606539 wait complete. 2024-05-14T09:33:15.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.354+0000 7f494e9e7700 1 Processor -- start 2024-05-14T09:33:15.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.354+0000 7f494e9e7700 1 -- start start 2024-05-14T09:33:15.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.354+0000 7f494e9e7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4948102b50 0x7f4948103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:15.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.354+0000 7f494e9e7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4948108f00 0x7f4948103fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:15.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.354+0000 7f494e9e7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948109850 0x7f4948104520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:15.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.354+0000 7f494e9e7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f49481144d0 con 0x7f4948109850 2024-05-14T09:33:15.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.354+0000 7f494e9e7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4948114350 con 0x7f4948102b50 2024-05-14T09:33:15.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.354+0000 7f494e9e7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4948114650 con 0x7f4948108f00 2024-05-14T09:33:15.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.354+0000 7f494cf84700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948109850 0x7f4948104520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:15.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.354+0000 7f494cf84700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948109850 0x7f4948104520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32820/0 (socket says 172.21.15.2:32820) 2024-05-14T09:33:15.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.354+0000 7f494cf84700 1 -- 172.21.15.2:0/483240551 learned_addr learned my addr 172.21.15.2:0/483240551 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:15.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.355+0000 7f4947fff700 1 --2- 172.21.15.2:0/483240551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4948102b50 0x7f4948103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:15.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.355+0000 7f494cf84700 1 -- 172.21.15.2:0/483240551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4948108f00 msgr2=0x7f4948103fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:15.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.355+0000 7f494cf84700 1 --2- 172.21.15.2:0/483240551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4948108f00 0x7f4948103fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.355+0000 7f494cf84700 1 -- 172.21.15.2:0/483240551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4948102b50 msgr2=0x7f4948103aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:15.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.355+0000 7f494cf84700 1 --2- 172.21.15.2:0/483240551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4948102b50 0x7f4948103aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.355+0000 7f494cf84700 1 -- 172.21.15.2:0/483240551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49481a9280 con 0x7f4948109850 2024-05-14T09:33:15.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.355+0000 7f4947fff700 1 --2- 172.21.15.2:0/483240551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4948102b50 0x7f4948103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:33:15.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.355+0000 7f494cf84700 1 --2- 172.21.15.2:0/483240551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948109850 0x7f4948104520 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f494000ef30 tx=0x7f494000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:15.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.356+0000 7f49457fa700 1 -- 172.21.15.2:0/483240551 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49400092e0 con 0x7f4948109850 2024-05-14T09:33:15.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.356+0000 7f494e9e7700 1 -- 172.21.15.2:0/483240551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49481a9510 con 0x7f4948109850 2024-05-14T09:33:15.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.356+0000 7f49457fa700 1 -- 172.21.15.2:0/483240551 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4940004bb0 con 0x7f4948109850 2024-05-14T09:33:15.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.356+0000 7f49457fa700 1 -- 172.21.15.2:0/483240551 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940004280 con 0x7f4948109850 2024-05-14T09:33:15.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.356+0000 7f494e9e7700 1 -- 172.21.15.2:0/483240551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49481a9a50 con 0x7f4948109850 2024-05-14T09:33:15.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.357+0000 7f494e9e7700 1 -- 172.21.15.2:0/483240551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49480512c0 con 0x7f4948109850 2024-05-14T09:33:15.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.364+0000 7f49457fa700 1 -- 172.21.15.2:0/483240551 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f494001b030 con 0x7f4948109850 2024-05-14T09:33:15.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.365+0000 7f49457fa700 1 --2- 172.21.15.2:0/483240551 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f49300b2a10 0x7f49300b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:15.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.366+0000 7f49457fa700 1 -- 172.21.15.2:0/483240551 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f49400e2c30 con 0x7f4948109850 2024-05-14T09:33:15.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.366+0000 7f4947fff700 1 --2- 172.21.15.2:0/483240551 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f49300b2a10 0x7f49300b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:15.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.367+0000 7f4947fff700 1 --2- 172.21.15.2:0/483240551 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f49300b2a10 0x7f49300b4f20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4938000c00 tx=0x7f493801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:15.368 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:15.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.369+0000 7f49457fa700 1 -- 172.21.15.2:0/483240551 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4940022020 con 0x7f4948109850 2024-05-14T09:33:15.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.399+0000 7fbcbd2ee700 1 -- 172.21.15.2:0/3404448662 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbcb802e850 con 0x7fbcb8108740 2024-05-14T09:33:15.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.400+0000 7fbc9ffff700 1 -- 172.21.15.2:0/3404448662 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbcb00ab5a0 con 0x7fbcb8108740 2024-05-14T09:33:15.401 INFO:teuthology.orchestra.run.smithi002.stdout:184683593797 2024-05-14T09:33:15.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.403+0000 7fbcbd2ee700 1 -- 172.21.15.2:0/3404448662 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbca00b2910 msgr2=0x7fbca00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:15.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.403+0000 7fbcbd2ee700 1 --2- 172.21.15.2:0/3404448662 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbca00b2910 0x7fbca00b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbca8005950 tx=0x7fbca80058e0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.403+0000 7fbcbd2ee700 1 -- 172.21.15.2:0/3404448662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcb8108740 msgr2=0x7fbcb81a6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:15.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.403+0000 7fbcbd2ee700 1 --2- 172.21.15.2:0/3404448662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcb8108740 0x7fbcb81a6ca0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d010 tx=0x7fbcb00079a0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.403+0000 7fbcbd2ee700 1 -- 172.21.15.2:0/3404448662 shutdown_connections 2024-05-14T09:33:15.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.403+0000 7fbcbd2ee700 1 --2- 172.21.15.2:0/3404448662 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbca00b2910 0x7fbca00b4e20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.403+0000 7fbcbd2ee700 1 --2- 172.21.15.2:0/3404448662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcb8108740 0x7fbcb81a6ca0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.403+0000 7fbcbd2ee700 1 --2- 172.21.15.2:0/3404448662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbcb81034c0 0x7fbcb819bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.403+0000 7fbcbd2ee700 1 --2- 172.21.15.2:0/3404448662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb8107df0 0x7fbcb819c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.403+0000 7fbcbd2ee700 1 -- 172.21.15.2:0/3404448662 >> 172.21.15.2:0/3404448662 conn(0x7fbcb80fedc0 msgr2=0x7fbcb8107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:15.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.404+0000 7fbcbd2ee700 1 -- 172.21.15.2:0/3404448662 shutdown_connections 2024-05-14T09:33:15.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.404+0000 7fbcbd2ee700 1 -- 172.21.15.2:0/3404448662 wait complete. 2024-05-14T09:33:15.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.607+0000 7f494e9e7700 1 -- 172.21.15.2:0/483240551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f494802e5f0 con 0x7f4948109850 2024-05-14T09:33:15.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.607+0000 7f49457fa700 1 -- 172.21.15.2:0/483240551 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4940010040 con 0x7f4948109850 2024-05-14T09:33:15.608 INFO:teuthology.orchestra.run.smithi002.stdout:111669149779 2024-05-14T09:33:15.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.609+0000 7f494e9e7700 1 -- 172.21.15.2:0/483240551 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f49300b2a10 msgr2=0x7f49300b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:15.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.609+0000 7f494e9e7700 1 --2- 172.21.15.2:0/483240551 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f49300b2a10 0x7f49300b4f20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4938000c00 tx=0x7f493801a040 comp rx=0 tx=0).stop 2024-05-14T09:33:15.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.609+0000 7f494e9e7700 1 -- 172.21.15.2:0/483240551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948109850 msgr2=0x7f4948104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:15.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.609+0000 7f494e9e7700 1 --2- 172.21.15.2:0/483240551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948109850 0x7f4948104520 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f494000ef30 tx=0x7f494000c3a0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.609+0000 7f494e9e7700 1 -- 172.21.15.2:0/483240551 shutdown_connections 2024-05-14T09:33:15.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.609+0000 7f494e9e7700 1 --2- 172.21.15.2:0/483240551 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f49300b2a10 0x7f49300b4f20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.609+0000 7f494e9e7700 1 --2- 172.21.15.2:0/483240551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4948102b50 0x7f4948103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.609+0000 7f494e9e7700 1 --2- 172.21.15.2:0/483240551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948109850 0x7f4948104520 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.610+0000 7f494e9e7700 1 --2- 172.21.15.2:0/483240551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4948108f00 0x7f4948103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:15.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.610+0000 7f494e9e7700 1 -- 172.21.15.2:0/483240551 >> 172.21.15.2:0/483240551 conn(0x7f49480fe6f0 msgr2=0x7f4948108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:15.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.610+0000 7f494e9e7700 1 -- 172.21.15.2:0/483240551 shutdown_connections 2024-05-14T09:33:15.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:15.610+0000 7f494e9e7700 1 -- 172.21.15.2:0/483240551 wait complete. 2024-05-14T09:33:15.622 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:15 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3404448662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-14T09:33:15.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:15 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3404448662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-14T09:33:15.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:15 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3404448662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-14T09:33:15.993 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:16.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.005+0000 7f7b04544700 1 -- 172.21.15.2:0/2735283228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7afc103130 msgr2=0x7f7afc111120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:16.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.005+0000 7f7b04544700 1 --2- 172.21.15.2:0/2735283228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7afc103130 0x7f7afc111120 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7af400b0a0 tx=0x7f7af400b3b0 comp rx=0 tx=0).stop 2024-05-14T09:33:16.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.005+0000 7f7b04544700 1 -- 172.21.15.2:0/2735283228 shutdown_connections 2024-05-14T09:33:16.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.005+0000 7f7b04544700 1 --2- 172.21.15.2:0/2735283228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7afc103130 0x7f7afc111120 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:16.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.005+0000 7f7b04544700 1 --2- 172.21.15.2:0/2735283228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7afc10a3e0 0x7f7afc10a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:16.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.005+0000 7f7b04544700 1 --2- 172.21.15.2:0/2735283228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7afc1027e0 0x7f7afc102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:16.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.005+0000 7f7b04544700 1 -- 172.21.15.2:0/2735283228 >> 172.21.15.2:0/2735283228 conn(0x7f7afc0fe6f0 msgr2=0x7f7afc100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:16.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.005+0000 7f7b04544700 1 -- 172.21.15.2:0/2735283228 shutdown_connections 2024-05-14T09:33:16.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.005+0000 7f7b04544700 1 -- 172.21.15.2:0/2735283228 wait complete. 2024-05-14T09:33:16.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.006+0000 7f7b04544700 1 Processor -- start 2024-05-14T09:33:16.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.006+0000 7f7b04544700 1 -- start start 2024-05-14T09:33:16.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.006+0000 7f7b04544700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7afc1027e0 0x7f7afc19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:16.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b04544700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7afc103130 0x7f7afc19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:16.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b04544700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7afc10a3e0 0x7f7afc1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:16.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b04544700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7afc1122e0 con 0x7f7afc103130 2024-05-14T09:33:16.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b04544700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7afc112160 con 0x7f7afc10a3e0 2024-05-14T09:33:16.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b04544700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7afc112460 con 0x7f7afc1027e0 2024-05-14T09:33:16.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b01adf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7afc103130 0x7f7afc19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:16.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b01adf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7afc103130 0x7f7afc19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32826/0 (socket says 172.21.15.2:32826) 2024-05-14T09:33:16.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b01adf700 1 -- 172.21.15.2:0/2841333210 learned_addr learned my addr 172.21.15.2:0/2841333210 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:16.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b022e0700 1 --2- 172.21.15.2:0/2841333210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7afc1027e0 0x7f7afc19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:16.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b01adf700 1 -- 172.21.15.2:0/2841333210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7afc1027e0 msgr2=0x7f7afc19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:16.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b01adf700 1 --2- 172.21.15.2:0/2841333210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7afc1027e0 0x7f7afc19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:16.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b01adf700 1 -- 172.21.15.2:0/2841333210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7afc10a3e0 msgr2=0x7f7afc1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:16.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b02ae1700 1 --2- 172.21.15.2:0/2841333210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7afc10a3e0 0x7f7afc1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:16.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b01adf700 1 --2- 172.21.15.2:0/2841333210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7afc10a3e0 0x7f7afc1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:16.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b01adf700 1 -- 172.21.15.2:0/2841333210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7afc199050 con 0x7f7afc103130 2024-05-14T09:33:16.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b02ae1700 1 --2- 172.21.15.2:0/2841333210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7afc10a3e0 0x7f7afc1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:33:16.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.007+0000 7f7b01adf700 1 --2- 172.21.15.2:0/2841333210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7afc103130 0x7f7afc19ecc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00b560 tx=0x7f7aec00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:16.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.008+0000 7f7af37fe700 1 -- 172.21.15.2:0/2841333210 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec004830 con 0x7f7afc103130 2024-05-14T09:33:16.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.008+0000 7f7b04544700 1 -- 172.21.15.2:0/2841333210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7afc199340 con 0x7f7afc103130 2024-05-14T09:33:16.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.008+0000 7f7af37fe700 1 -- 172.21.15.2:0/2841333210 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aec005020 con 0x7f7afc103130 2024-05-14T09:33:16.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.008+0000 7f7af37fe700 1 -- 172.21.15.2:0/2841333210 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec01b700 con 0x7f7afc103130 2024-05-14T09:33:16.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.008+0000 7f7b04544700 1 -- 172.21.15.2:0/2841333210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7afc1a5610 con 0x7f7afc103130 2024-05-14T09:33:16.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.009+0000 7f7b04544700 1 -- 172.21.15.2:0/2841333210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7afc0512c0 con 0x7f7afc103130 2024-05-14T09:33:16.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.012+0000 7f7af37fe700 1 -- 172.21.15.2:0/2841333210 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f7aec0049d0 con 0x7f7afc103130 2024-05-14T09:33:16.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.013+0000 7f7af37fe700 1 --2- 172.21.15.2:0/2841333210 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ae80b2990 0x7f7ae80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:16.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.013+0000 7f7b022e0700 1 --2- 172.21.15.2:0/2841333210 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ae80b2990 0x7f7ae80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:16.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.014+0000 7f7af37fe700 1 -- 172.21.15.2:0/2841333210 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f7aec0eaf80 con 0x7f7afc103130 2024-05-14T09:33:16.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.014+0000 7f7b022e0700 1 --2- 172.21.15.2:0/2841333210 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ae80b2990 0x7f7ae80b4ea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7afc199ae0 tx=0x7f7af8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:16.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.018+0000 7f7af37fe700 1 -- 172.21.15.2:0/2841333210 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7aec0b2720 con 0x7f7afc103130 2024-05-14T09:33:16.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.239+0000 7f7b04544700 1 -- 172.21.15.2:0/2841333210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f7afc199ed0 con 0x7f7afc103130 2024-05-14T09:33:16.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.239+0000 7f7af37fe700 1 -- 172.21.15.2:0/2841333210 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f7aec0b2020 con 0x7f7afc103130 2024-05-14T09:33:16.240 INFO:teuthology.orchestra.run.smithi002.stdout:279172874288 2024-05-14T09:33:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.242+0000 7f7b04544700 1 -- 172.21.15.2:0/2841333210 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ae80b2990 msgr2=0x7f7ae80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.242+0000 7f7b04544700 1 --2- 172.21.15.2:0/2841333210 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ae80b2990 0x7f7ae80b4ea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7afc199ae0 tx=0x7f7af8009260 comp rx=0 tx=0).stop 2024-05-14T09:33:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.242+0000 7f7b04544700 1 -- 172.21.15.2:0/2841333210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7afc103130 msgr2=0x7f7afc19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.242+0000 7f7b04544700 1 --2- 172.21.15.2:0/2841333210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7afc103130 0x7f7afc19ecc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00b560 tx=0x7f7aec00b920 comp rx=0 tx=0).stop 2024-05-14T09:33:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.242+0000 7f7b04544700 1 -- 172.21.15.2:0/2841333210 shutdown_connections 2024-05-14T09:33:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.242+0000 7f7b04544700 1 --2- 172.21.15.2:0/2841333210 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ae80b2990 0x7f7ae80b4ea0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.242+0000 7f7b04544700 1 --2- 172.21.15.2:0/2841333210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7afc10a3e0 0x7f7afc1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.242+0000 7f7b04544700 1 --2- 172.21.15.2:0/2841333210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7afc1027e0 0x7f7afc19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.242+0000 7f7b04544700 1 --2- 172.21.15.2:0/2841333210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7afc103130 0x7f7afc19ecc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.242+0000 7f7b04544700 1 -- 172.21.15.2:0/2841333210 >> 172.21.15.2:0/2841333210 conn(0x7f7afc0fe6f0 msgr2=0x7f7afc100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.242+0000 7f7b04544700 1 -- 172.21.15.2:0/2841333210 shutdown_connections 2024-05-14T09:33:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:16.242+0000 7f7b04544700 1 -- 172.21.15.2:0/2841333210 wait complete. 2024-05-14T09:33:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:16 smithi002 ceph-mon[31557]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:16 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/483240551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-14T09:33:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:16 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2841333210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-14T09:33:16.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:16 smithi029 ceph-mon[36249]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:16.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:16 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/483240551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-14T09:33:16.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:16 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2841333210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-14T09:33:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:16 smithi026 ceph-mon[36056]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:16 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/483240551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-14T09:33:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:16 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2841333210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-14T09:33:18.355 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149777 got 111669149779 for osd.3 2024-05-14T09:33:18.355 DEBUG:teuthology.parallel:result is None 2024-05-14T09:33:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.548+0000 7f4b68ce0700 1 -- 172.21.15.2:0/2712859468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b641034c0 msgr2=0x7f4b641038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.548+0000 7f4b68ce0700 1 --2- 172.21.15.2:0/2712859468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b641034c0 0x7f4b641038d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4b540099c0 tx=0x7f4b54009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.548+0000 7f4b68ce0700 1 -- 172.21.15.2:0/2712859468 shutdown_connections 2024-05-14T09:33:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.548+0000 7f4b68ce0700 1 --2- 172.21.15.2:0/2712859468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b64108740 0x7f4b641100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.548+0000 7f4b68ce0700 1 --2- 172.21.15.2:0/2712859468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b641034c0 0x7f4b641038d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.548+0000 7f4b68ce0700 1 --2- 172.21.15.2:0/2712859468 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b64107df0 0x7f4b64108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.548+0000 7f4b68ce0700 1 -- 172.21.15.2:0/2712859468 >> 172.21.15.2:0/2712859468 conn(0x7f4b640fedc0 msgr2=0x7f4b641011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.548+0000 7f4b68ce0700 1 -- 172.21.15.2:0/2712859468 shutdown_connections 2024-05-14T09:33:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.548+0000 7f4b68ce0700 1 -- 172.21.15.2:0/2712859468 wait complete. 2024-05-14T09:33:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.549+0000 7f4b68ce0700 1 Processor -- start 2024-05-14T09:33:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.549+0000 7f4b68ce0700 1 -- start start 2024-05-14T09:33:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.549+0000 7f4b68ce0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b641034c0 0x7f4b6419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.549+0000 7f4b68ce0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b64107df0 0x7f4b6419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.549+0000 7f4b68ce0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b64108740 0x7f4b641a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.549+0000 7f4b68ce0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4b641144c0 con 0x7f4b64107df0 2024-05-14T09:33:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.549+0000 7f4b68ce0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4b64114340 con 0x7f4b64108740 2024-05-14T09:33:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.549+0000 7f4b68ce0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4b64114640 con 0x7f4b641034c0 2024-05-14T09:33:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.549+0000 7f4b61d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b64107df0 0x7f4b6419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.549+0000 7f4b61d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b64107df0 0x7f4b6419d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45466/0 (socket says 172.21.15.2:45466) 2024-05-14T09:33:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.549+0000 7f4b61d9b700 1 -- 172.21.15.2:0/3689060270 learned_addr learned my addr 172.21.15.2:0/3689060270 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.549+0000 7f4b6259c700 1 --2- 172.21.15.2:0/3689060270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b641034c0 0x7f4b6419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.550+0000 7f4b62d9d700 1 --2- 172.21.15.2:0/3689060270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b64108740 0x7f4b641a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.550+0000 7f4b61d9b700 1 -- 172.21.15.2:0/3689060270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b641034c0 msgr2=0x7f4b6419cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.550+0000 7f4b61d9b700 1 --2- 172.21.15.2:0/3689060270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b641034c0 0x7f4b6419cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.550+0000 7f4b61d9b700 1 -- 172.21.15.2:0/3689060270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b64108740 msgr2=0x7f4b641a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.550+0000 7f4b61d9b700 1 --2- 172.21.15.2:0/3689060270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b64108740 0x7f4b641a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.550+0000 7f4b61d9b700 1 -- 172.21.15.2:0/3689060270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b641a1ec0 con 0x7f4b64107df0 2024-05-14T09:33:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.550+0000 7f4b62d9d700 1 --2- 172.21.15.2:0/3689060270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b64108740 0x7f4b641a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:33:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.550+0000 7f4b61d9b700 1 --2- 172.21.15.2:0/3689060270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b64107df0 0x7f4b6419d590 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800e970 tx=0x7f4b5800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.550+0000 7f4b537fe700 1 -- 172.21.15.2:0/3689060270 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800cbd0 con 0x7f4b64107df0 2024-05-14T09:33:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.550+0000 7f4b537fe700 1 -- 172.21.15.2:0/3689060270 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b5800cd70 con 0x7f4b64107df0 2024-05-14T09:33:18.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.550+0000 7f4b68ce0700 1 -- 172.21.15.2:0/3689060270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b641a21b0 con 0x7f4b64107df0 2024-05-14T09:33:18.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.550+0000 7f4b537fe700 1 -- 172.21.15.2:0/3689060270 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58005680 con 0x7f4b64107df0 2024-05-14T09:33:18.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.550+0000 7f4b68ce0700 1 -- 172.21.15.2:0/3689060270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b641a9ad0 con 0x7f4b64107df0 2024-05-14T09:33:18.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.551+0000 7f4b68ce0700 1 -- 172.21.15.2:0/3689060270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b640512c0 con 0x7f4b64107df0 2024-05-14T09:33:18.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.553+0000 7f4b537fe700 1 -- 172.21.15.2:0/3689060270 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f4b58004750 con 0x7f4b64107df0 2024-05-14T09:33:18.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.554+0000 7f4b537fe700 1 --2- 172.21.15.2:0/3689060270 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4b4c0b2910 0x7f4b4c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:18.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.554+0000 7f4b6259c700 1 --2- 172.21.15.2:0/3689060270 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4b4c0b2910 0x7f4b4c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:18.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.555+0000 7f4b537fe700 1 -- 172.21.15.2:0/3689060270 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f4b580e2c40 con 0x7f4b64107df0 2024-05-14T09:33:18.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.555+0000 7f4b6259c700 1 --2- 172.21.15.2:0/3689060270 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4b4c0b2910 0x7f4b4c0b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b54000c00 tx=0x7f4b5401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:18.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.559+0000 7f4b537fe700 1 -- 172.21.15.2:0/3689060270 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4b58054df0 con 0x7f4b64107df0 2024-05-14T09:33:18.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:18 smithi002 ceph-mon[31557]: pgmap v374: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:33:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.777+0000 7f4b68ce0700 1 -- 172.21.15.2:0/3689060270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f4b641a9d00 con 0x7f4b64107df0 2024-05-14T09:33:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.777+0000 7f4b537fe700 1 -- 172.21.15.2:0/3689060270 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4b580aa380 con 0x7f4b64107df0 2024-05-14T09:33:18.778 INFO:teuthology.orchestra.run.smithi002.stdout:231928234043 2024-05-14T09:33:18.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.780+0000 7f4b68ce0700 1 -- 172.21.15.2:0/3689060270 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4b4c0b2910 msgr2=0x7f4b4c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:18.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.780+0000 7f4b68ce0700 1 --2- 172.21.15.2:0/3689060270 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4b4c0b2910 0x7f4b4c0b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b54000c00 tx=0x7f4b5401a040 comp rx=0 tx=0).stop 2024-05-14T09:33:18.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.780+0000 7f4b68ce0700 1 -- 172.21.15.2:0/3689060270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b64107df0 msgr2=0x7f4b6419d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:18.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.780+0000 7f4b68ce0700 1 --2- 172.21.15.2:0/3689060270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b64107df0 0x7f4b6419d590 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800e970 tx=0x7f4b5800ec80 comp rx=0 tx=0).stop 2024-05-14T09:33:18.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.780+0000 7f4b68ce0700 1 -- 172.21.15.2:0/3689060270 shutdown_connections 2024-05-14T09:33:18.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.780+0000 7f4b68ce0700 1 --2- 172.21.15.2:0/3689060270 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4b4c0b2910 0x7f4b4c0b4e20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:18.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.780+0000 7f4b68ce0700 1 --2- 172.21.15.2:0/3689060270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b64108740 0x7f4b641a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:18.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.780+0000 7f4b68ce0700 1 --2- 172.21.15.2:0/3689060270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b641034c0 0x7f4b6419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:18.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.780+0000 7f4b68ce0700 1 --2- 172.21.15.2:0/3689060270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b64107df0 0x7f4b6419d590 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:18.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.780+0000 7f4b68ce0700 1 -- 172.21.15.2:0/3689060270 >> 172.21.15.2:0/3689060270 conn(0x7f4b640fedc0 msgr2=0x7f4b6410f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:18.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.780+0000 7f4b68ce0700 1 -- 172.21.15.2:0/3689060270 shutdown_connections 2024-05-14T09:33:18.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:18.780+0000 7f4b68ce0700 1 -- 172.21.15.2:0/3689060270 wait complete. 2024-05-14T09:33:18.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:18 smithi029 ceph-mon[36249]: pgmap v374: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:33:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:18 smithi026 ceph-mon[36056]: pgmap v374: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:33:19.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:19 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3689060270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-14T09:33:19.874 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234037 got 231928234043 for osd.8 2024-05-14T09:33:19.874 DEBUG:teuthology.parallel:result is None 2024-05-14T09:33:19.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:19 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3689060270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-14T09:33:19.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:19 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3689060270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-14T09:33:20.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:20 smithi002 ceph-mon[31557]: pgmap v375: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:20 smithi029 ceph-mon[36249]: pgmap v375: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:20.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:20 smithi026 ceph-mon[36056]: pgmap v375: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:21.036 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874285 got 279172874288 for osd.10 2024-05-14T09:33:21.036 DEBUG:teuthology.parallel:result is None 2024-05-14T09:33:21.969 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593793 got 184683593797 for osd.6 2024-05-14T09:33:21.970 DEBUG:teuthology.parallel:result is None 2024-05-14T09:33:22.767 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:22 smithi002 ceph-mon[31557]: pgmap v376: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:22.777 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:22.778 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:22.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:22 smithi029 ceph-mon[36249]: pgmap v376: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:22.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:22 smithi026 ceph-mon[36056]: pgmap v376: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:24.505 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:24.599 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:24 smithi002 ceph-mon[31557]: pgmap v377: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:24.599 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:24 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1157692293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-14T09:33:24.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.398+0000 7f57198ed700 1 -- 172.21.15.2:0/1906512116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5714104650 msgr2=0x7f5714104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:24.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.398+0000 7f57198ed700 1 --2- 172.21.15.2:0/1906512116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5714104650 0x7f5714104a60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f57040099c0 tx=0x7f5704009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:24.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.398+0000 7f57198ed700 1 -- 172.21.15.2:0/1906512116 shutdown_connections 2024-05-14T09:33:24.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.398+0000 7f57198ed700 1 --2- 172.21.15.2:0/1906512116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5714109880 0x7f57141111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:24.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.398+0000 7f57198ed700 1 --2- 172.21.15.2:0/1906512116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5714104650 0x7f5714104a60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:24.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.398+0000 7f57198ed700 1 --2- 172.21.15.2:0/1906512116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5714108f30 0x7f5714109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:24.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.398+0000 7f57198ed700 1 -- 172.21.15.2:0/1906512116 >> 172.21.15.2:0/1906512116 conn(0x7f5714079f00 msgr2=0x7f571407a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:24.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.398+0000 7f57198ed700 1 -- 172.21.15.2:0/1906512116 shutdown_connections 2024-05-14T09:33:24.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.398+0000 7f57198ed700 1 -- 172.21.15.2:0/1906512116 wait complete. 2024-05-14T09:33:24.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.399+0000 7f57198ed700 1 Processor -- start 2024-05-14T09:33:24.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.399+0000 7f57198ed700 1 -- start start 2024-05-14T09:33:24.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.399+0000 7f57198ed700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5714104650 0x7f571419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:24.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.399+0000 7f57198ed700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5714108f30 0x7f571419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:24.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.399+0000 7f57198ed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5714109880 0x7f571419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:24.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.399+0000 7f57198ed700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f57141144d0 con 0x7f5714108f30 2024-05-14T09:33:24.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.399+0000 7f57198ed700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5714114350 con 0x7f5714109880 2024-05-14T09:33:24.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.399+0000 7f57198ed700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5714114650 con 0x7f5714104650 2024-05-14T09:33:24.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.399+0000 7f57127fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5714108f30 0x7f571419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:24.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.400+0000 7f57127fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5714108f30 0x7f571419b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45508/0 (socket says 172.21.15.2:45508) 2024-05-14T09:33:24.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.400+0000 7f57127fc700 1 -- 172.21.15.2:0/740526956 learned_addr learned my addr 172.21.15.2:0/740526956 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:24.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.400+0000 7f57137fe700 1 --2- 172.21.15.2:0/740526956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5714109880 0x7f571419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.400+0000 7f5712ffd700 1 --2- 172.21.15.2:0/740526956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5714104650 0x7f571419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.400+0000 7f57127fc700 1 -- 172.21.15.2:0/740526956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5714104650 msgr2=0x7f571419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.400+0000 7f57127fc700 1 --2- 172.21.15.2:0/740526956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5714104650 0x7f571419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.400+0000 7f57127fc700 1 -- 172.21.15.2:0/740526956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5714109880 msgr2=0x7f571419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.400+0000 7f57127fc700 1 --2- 172.21.15.2:0/740526956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5714109880 0x7f571419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.400+0000 7f57127fc700 1 -- 172.21.15.2:0/740526956 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571419fcf0 con 0x7f5714108f30 2024-05-14T09:33:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.400+0000 7f57127fc700 1 --2- 172.21.15.2:0/740526956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5714108f30 0x7f571419b3c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f570800e9f0 tx=0x7f570800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.400+0000 7f57188eb700 1 -- 172.21.15.2:0/740526956 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570800cbd0 con 0x7f5714108f30 2024-05-14T09:33:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.400+0000 7f57188eb700 1 -- 172.21.15.2:0/740526956 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f570800cd70 con 0x7f5714108f30 2024-05-14T09:33:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.400+0000 7f57188eb700 1 -- 172.21.15.2:0/740526956 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708005650 con 0x7f5714108f30 2024-05-14T09:33:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.400+0000 7f57198ed700 1 -- 172.21.15.2:0/740526956 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571419ffe0 con 0x7f5714108f30 2024-05-14T09:33:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.400+0000 7f57198ed700 1 -- 172.21.15.2:0/740526956 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57141a7900 con 0x7f5714108f30 2024-05-14T09:33:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.401+0000 7f57198ed700 1 -- 172.21.15.2:0/740526956 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57140512c0 con 0x7f5714108f30 2024-05-14T09:33:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.403+0000 7f57188eb700 1 -- 172.21.15.2:0/740526956 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f5708004750 con 0x7f5714108f30 2024-05-14T09:33:24.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.404+0000 7f57188eb700 1 --2- 172.21.15.2:0/740526956 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f56fc0b2910 0x7f56fc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:24.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.404+0000 7f5712ffd700 1 --2- 172.21.15.2:0/740526956 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f56fc0b2910 0x7f56fc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:24.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.404+0000 7f57188eb700 1 -- 172.21.15.2:0/740526956 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f57080e2d60 con 0x7f5714108f30 2024-05-14T09:33:24.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.405+0000 7f5712ffd700 1 --2- 172.21.15.2:0/740526956 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f56fc0b2910 0x7f56fc0b4e20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5704003760 tx=0x7f570401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:24.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.407+0000 7f57188eb700 1 -- 172.21.15.2:0/740526956 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5708054df0 con 0x7f5714108f30 2024-05-14T09:33:24.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.628+0000 7f57198ed700 1 -- 172.21.15.2:0/740526956 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f571419c180 con 0x7f5714108f30 2024-05-14T09:33:24.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.628+0000 7f57188eb700 1 -- 172.21.15.2:0/740526956 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f57080aa4a0 con 0x7f5714108f30 2024-05-14T09:33:24.813 INFO:teuthology.orchestra.run.smithi002.stdout:158913790029 2024-05-14T09:33:24.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.631+0000 7f57198ed700 1 -- 172.21.15.2:0/740526956 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f56fc0b2910 msgr2=0x7f56fc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:24.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.631+0000 7f57198ed700 1 --2- 172.21.15.2:0/740526956 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f56fc0b2910 0x7f56fc0b4e20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5704003760 tx=0x7f570401a040 comp rx=0 tx=0).stop 2024-05-14T09:33:24.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.631+0000 7f57198ed700 1 -- 172.21.15.2:0/740526956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5714108f30 msgr2=0x7f571419b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:24.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.631+0000 7f57198ed700 1 --2- 172.21.15.2:0/740526956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5714108f30 0x7f571419b3c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f570800e9f0 tx=0x7f570800ed00 comp rx=0 tx=0).stop 2024-05-14T09:33:24.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.631+0000 7f57198ed700 1 -- 172.21.15.2:0/740526956 shutdown_connections 2024-05-14T09:33:24.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.631+0000 7f57198ed700 1 --2- 172.21.15.2:0/740526956 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f56fc0b2910 0x7f56fc0b4e20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:24.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.631+0000 7f57198ed700 1 --2- 172.21.15.2:0/740526956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5714109880 0x7f571419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:24.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.631+0000 7f57198ed700 1 --2- 172.21.15.2:0/740526956 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5714104650 0x7f571419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:24.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.631+0000 7f57198ed700 1 --2- 172.21.15.2:0/740526956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5714108f30 0x7f571419b3c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:24.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.631+0000 7f57198ed700 1 -- 172.21.15.2:0/740526956 >> 172.21.15.2:0/740526956 conn(0x7f5714079f00 msgr2=0x7f5714102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:24.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.631+0000 7f57198ed700 1 -- 172.21.15.2:0/740526956 shutdown_connections 2024-05-14T09:33:24.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:24.631+0000 7f57198ed700 1 -- 172.21.15.2:0/740526956 wait complete. 2024-05-14T09:33:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:24 smithi029 ceph-mon[36249]: pgmap v377: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:24 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1157692293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-14T09:33:24.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:24 smithi026 ceph-mon[36056]: pgmap v377: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:24.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:24 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1157692293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-14T09:33:25.436 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790022 got 158913790029 for osd.5 2024-05-14T09:33:25.436 DEBUG:teuthology.parallel:result is None 2024-05-14T09:33:25.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:25 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/740526956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-14T09:33:25.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:25 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/740526956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-14T09:33:25.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:25 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/740526956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-14T09:33:26.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.443+0000 7f7e2eb7e700 1 -- 172.21.15.2:0/3127999599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28108f20 msgr2=0x7f7e28109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.443+0000 7f7e2eb7e700 1 --2- 172.21.15.2:0/3127999599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28108f20 0x7f7e28109330 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7e140099c0 tx=0x7f7e14009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.443+0000 7f7e2eb7e700 1 -- 172.21.15.2:0/3127999599 shutdown_connections 2024-05-14T09:33:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.443+0000 7f7e2eb7e700 1 --2- 172.21.15.2:0/3127999599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28109870 0x7f7e281111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.443+0000 7f7e2eb7e700 1 --2- 172.21.15.2:0/3127999599 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e281045f0 0x7f7e28104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.443+0000 7f7e2eb7e700 1 --2- 172.21.15.2:0/3127999599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28108f20 0x7f7e28109330 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.443+0000 7f7e2eb7e700 1 -- 172.21.15.2:0/3127999599 >> 172.21.15.2:0/3127999599 conn(0x7f7e28100040 msgr2=0x7f7e28102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.443+0000 7f7e2eb7e700 1 -- 172.21.15.2:0/3127999599 shutdown_connections 2024-05-14T09:33:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.443+0000 7f7e2eb7e700 1 -- 172.21.15.2:0/3127999599 wait complete. 2024-05-14T09:33:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.444+0000 7f7e2eb7e700 1 Processor -- start 2024-05-14T09:33:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.444+0000 7f7e2eb7e700 1 -- start start 2024-05-14T09:33:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.444+0000 7f7e2eb7e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e281045f0 0x7f7e2819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.444+0000 7f7e2eb7e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28108f20 0x7f7e2819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.444+0000 7f7e2eb7e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28109870 0x7f7e2819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.444+0000 7f7e2eb7e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7e281144d0 con 0x7f7e28109870 2024-05-14T09:33:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.444+0000 7f7e2eb7e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7e28114350 con 0x7f7e28108f20 2024-05-14T09:33:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.444+0000 7f7e2eb7e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7e28114650 con 0x7f7e281045f0 2024-05-14T09:33:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.445+0000 7f7e2d11b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28109870 0x7f7e2819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.445+0000 7f7e2d11b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28109870 0x7f7e2819f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45490/0 (socket says 172.21.15.2:45490) 2024-05-14T09:33:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.445+0000 7f7e2d11b700 1 -- 172.21.15.2:0/1157692293 learned_addr learned my addr 172.21.15.2:0/1157692293 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.445+0000 7f7e27fff700 1 --2- 172.21.15.2:0/1157692293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28108f20 0x7f7e2819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.445+0000 7f7e2c91a700 1 --2- 172.21.15.2:0/1157692293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e281045f0 0x7f7e2819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.445+0000 7f7e2d11b700 1 -- 172.21.15.2:0/1157692293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e281045f0 msgr2=0x7f7e2819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.445+0000 7f7e2d11b700 1 --2- 172.21.15.2:0/1157692293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e281045f0 0x7f7e2819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.445+0000 7f7e2d11b700 1 -- 172.21.15.2:0/1157692293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28108f20 msgr2=0x7f7e2819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.445+0000 7f7e2d11b700 1 --2- 172.21.15.2:0/1157692293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28108f20 0x7f7e2819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:26.071 INFO:teuthology.orchestra.run.smithi002.stdout:34359738470 2024-05-14T09:33:26.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.445+0000 7f7e2d11b700 1 -- 172.21.15.2:0/1157692293 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2819fd30 con 0x7f7e28109870 2024-05-14T09:33:26.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.445+0000 7f7e2d11b700 1 --2- 172.21.15.2:0/1157692293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28109870 0x7f7e2819f690 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000ef30 tx=0x7f7e2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:26.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.445+0000 7f7e25ffb700 1 -- 172.21.15.2:0/1157692293 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e200092e0 con 0x7f7e28109870 2024-05-14T09:33:26.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.445+0000 7f7e2eb7e700 1 -- 172.21.15.2:0/1157692293 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2819ffc0 con 0x7f7e28109870 2024-05-14T09:33:26.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.445+0000 7f7e25ffb700 1 -- 172.21.15.2:0/1157692293 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e20004bb0 con 0x7f7e28109870 2024-05-14T09:33:26.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.445+0000 7f7e25ffb700 1 -- 172.21.15.2:0/1157692293 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20004020 con 0x7f7e28109870 2024-05-14T09:33:26.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.446+0000 7f7e2eb7e700 1 -- 172.21.15.2:0/1157692293 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e281a04d0 con 0x7f7e28109870 2024-05-14T09:33:26.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.447+0000 7f7e2eb7e700 1 -- 172.21.15.2:0/1157692293 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e280512c0 con 0x7f7e28109870 2024-05-14T09:33:26.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.448+0000 7f7e25ffb700 1 -- 172.21.15.2:0/1157692293 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f7e2001b030 con 0x7f7e28109870 2024-05-14T09:33:26.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.449+0000 7f7e25ffb700 1 --2- 172.21.15.2:0/1157692293 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7e180b2910 0x7f7e180b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.449+0000 7f7e2c91a700 1 --2- 172.21.15.2:0/1157692293 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7e180b2910 0x7f7e180b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.449+0000 7f7e25ffb700 1 -- 172.21.15.2:0/1157692293 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f7e200e2210 con 0x7f7e28109870 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.450+0000 7f7e2c91a700 1 --2- 172.21.15.2:0/1157692293 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7e180b2910 0x7f7e180b4e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c007be0 tx=0x7f7e1c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.454+0000 7f7e25ffb700 1 -- 172.21.15.2:0/1157692293 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e20022020 con 0x7f7e28109870 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.668+0000 7f7e2eb7e700 1 -- 172.21.15.2:0/1157692293 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7e2802e5f0 con 0x7f7e28109870 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.669+0000 7f7e25ffb700 1 -- 172.21.15.2:0/1157692293 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7e20010040 con 0x7f7e28109870 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.671+0000 7f7e2eb7e700 1 -- 172.21.15.2:0/1157692293 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7e180b2910 msgr2=0x7f7e180b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.671+0000 7f7e2eb7e700 1 --2- 172.21.15.2:0/1157692293 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7e180b2910 0x7f7e180b4e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c007be0 tx=0x7f7e1c0073d0 comp rx=0 tx=0).stop 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.671+0000 7f7e2eb7e700 1 -- 172.21.15.2:0/1157692293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28109870 msgr2=0x7f7e2819f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.671+0000 7f7e2eb7e700 1 --2- 172.21.15.2:0/1157692293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28109870 0x7f7e2819f690 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000ef30 tx=0x7f7e2000c3a0 comp rx=0 tx=0).stop 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.672+0000 7f7e2eb7e700 1 -- 172.21.15.2:0/1157692293 shutdown_connections 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.672+0000 7f7e2eb7e700 1 --2- 172.21.15.2:0/1157692293 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7e180b2910 0x7f7e180b4e20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.672+0000 7f7e2eb7e700 1 --2- 172.21.15.2:0/1157692293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28109870 0x7f7e2819f690 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.672+0000 7f7e2eb7e700 1 --2- 172.21.15.2:0/1157692293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e281045f0 0x7f7e2819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.672+0000 7f7e2eb7e700 1 --2- 172.21.15.2:0/1157692293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28108f20 0x7f7e2819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.672+0000 7f7e2eb7e700 1 -- 172.21.15.2:0/1157692293 >> 172.21.15.2:0/1157692293 conn(0x7f7e28100040 msgr2=0x7f7e2810a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:26.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.672+0000 7f7e2eb7e700 1 -- 172.21.15.2:0/1157692293 shutdown_connections 2024-05-14T09:33:26.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:23.672+0000 7f7e2eb7e700 1 -- 172.21.15.2:0/1157692293 wait complete. 2024-05-14T09:33:26.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:26 smithi002 ceph-mon[31557]: pgmap v378: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:26.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:26 smithi029 ceph-mon[36249]: pgmap v378: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:26 smithi026 ceph-mon[36056]: pgmap v378: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:27.280 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738465 got 34359738470 for osd.0 2024-05-14T09:33:27.280 DEBUG:teuthology.parallel:result is None 2024-05-14T09:33:28.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:28 smithi002 ceph-mon[31557]: pgmap v379: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:33:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:28 smithi029 ceph-mon[36249]: pgmap v379: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:33:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:28 smithi026 ceph-mon[36056]: pgmap v379: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:33:30.546 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:30 smithi002 ceph-mon[31557]: pgmap v380: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:30.546 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:30 smithi002 ceph-mon[31557]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-14T09:33:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.902+0000 7f027e34b700 1 -- 172.21.15.2:0/1001696492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0278107df0 msgr2=0x7f0278108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.902+0000 7f027e34b700 1 --2- 172.21.15.2:0/1001696492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0278107df0 0x7f0278108200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f026c0099c0 tx=0x7f026c009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.902+0000 7f027e34b700 1 -- 172.21.15.2:0/1001696492 shutdown_connections 2024-05-14T09:33:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.902+0000 7f027e34b700 1 --2- 172.21.15.2:0/1001696492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02781034c0 0x7f02781038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.902+0000 7f027e34b700 1 --2- 172.21.15.2:0/1001696492 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0278108740 0x7f02781100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:30.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.902+0000 7f027e34b700 1 --2- 172.21.15.2:0/1001696492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0278107df0 0x7f0278108200 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:30.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.902+0000 7f027e34b700 1 -- 172.21.15.2:0/1001696492 >> 172.21.15.2:0/1001696492 conn(0x7f02780fedc0 msgr2=0x7f02781011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:30.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.902+0000 7f027e34b700 1 -- 172.21.15.2:0/1001696492 shutdown_connections 2024-05-14T09:33:30.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.902+0000 7f027e34b700 1 -- 172.21.15.2:0/1001696492 wait complete. 2024-05-14T09:33:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.903+0000 7f027e34b700 1 Processor -- start 2024-05-14T09:33:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.903+0000 7f027e34b700 1 -- start start 2024-05-14T09:33:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f027e34b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02781034c0 0x7f027819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f027e34b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0278107df0 0x7f027819c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f027e34b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0278108740 0x7f027819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f027e34b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0278113350 con 0x7f02781034c0 2024-05-14T09:33:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f027e34b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f02781131d0 con 0x7f0278108740 2024-05-14T09:33:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f027e34b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f02781134d0 con 0x7f0278107df0 2024-05-14T09:33:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f0277fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02781034c0 0x7f027819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f0277fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02781034c0 0x7f027819bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40878/0 (socket says 172.21.15.2:40878) 2024-05-14T09:33:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f0277fff700 1 -- 172.21.15.2:0/2245543315 learned_addr learned my addr 172.21.15.2:0/2245543315 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f027c8e8700 1 --2- 172.21.15.2:0/2245543315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0278108740 0x7f027819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f02777fe700 1 --2- 172.21.15.2:0/2245543315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0278107df0 0x7f027819c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f0277fff700 1 -- 172.21.15.2:0/2245543315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0278107df0 msgr2=0x7f027819c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f0277fff700 1 --2- 172.21.15.2:0/2245543315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0278107df0 0x7f027819c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f0277fff700 1 -- 172.21.15.2:0/2245543315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0278108740 msgr2=0x7f027819c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f0277fff700 1 --2- 172.21.15.2:0/2245543315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0278108740 0x7f027819c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f0277fff700 1 -- 172.21.15.2:0/2245543315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02781a6fc0 con 0x7f02781034c0 2024-05-14T09:33:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.904+0000 7f027c8e8700 1 --2- 172.21.15.2:0/2245543315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0278108740 0x7f027819c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:33:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.905+0000 7f0277fff700 1 --2- 172.21.15.2:0/2245543315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02781034c0 0x7f027819bc50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f026800e9a0 tx=0x7f026800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.905+0000 7f02757fa700 1 -- 172.21.15.2:0/2245543315 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026800ca10 con 0x7f02781034c0 2024-05-14T09:33:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.905+0000 7f02757fa700 1 -- 172.21.15.2:0/2245543315 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f026800cbb0 con 0x7f02781034c0 2024-05-14T09:33:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.905+0000 7f027e34b700 1 -- 172.21.15.2:0/2245543315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02781a72b0 con 0x7f02781034c0 2024-05-14T09:33:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.905+0000 7f02757fa700 1 -- 172.21.15.2:0/2245543315 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268018680 con 0x7f02781034c0 2024-05-14T09:33:30.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.905+0000 7f027e34b700 1 -- 172.21.15.2:0/2245543315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02781a7810 con 0x7f02781034c0 2024-05-14T09:33:30.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.906+0000 7f027e34b700 1 -- 172.21.15.2:0/2245543315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02780512c0 con 0x7f02781034c0 2024-05-14T09:33:30.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.908+0000 7f02757fa700 1 -- 172.21.15.2:0/2245543315 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0268010430 con 0x7f02781034c0 2024-05-14T09:33:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.908+0000 7f02757fa700 1 --2- 172.21.15.2:0/2245543315 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f02600b2900 0x7f02600b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.908+0000 7f02777fe700 1 --2- 172.21.15.2:0/2245543315 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f02600b2900 0x7f02600b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.908+0000 7f02757fa700 1 -- 172.21.15.2:0/2245543315 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f02680e2a60 con 0x7f02781034c0 2024-05-14T09:33:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.909+0000 7f02777fe700 1 --2- 172.21.15.2:0/2245543315 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f02600b2900 0x7f02600b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f026c000c00 tx=0x7f026c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:30.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:30.912+0000 7f02757fa700 1 -- 172.21.15.2:0/2245543315 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02680aa360 con 0x7f02781034c0 2024-05-14T09:33:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:30 smithi029 ceph-mon[36249]: pgmap v380: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:30 smithi029 ceph-mon[36249]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-14T09:33:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:30 smithi026 ceph-mon[36056]: pgmap v380: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:30 smithi026 ceph-mon[36056]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-14T09:33:31.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.137+0000 7f027e34b700 1 -- 172.21.15.2:0/2245543315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f027802e710 con 0x7f02781034c0 2024-05-14T09:33:31.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.137+0000 7f02757fa700 1 -- 172.21.15.2:0/2245543315 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f02680b23e0 con 0x7f02781034c0 2024-05-14T09:33:31.138 INFO:teuthology.orchestra.run.smithi002.stdout:257698037817 2024-05-14T09:33:31.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.140+0000 7f027e34b700 1 -- 172.21.15.2:0/2245543315 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f02600b2900 msgr2=0x7f02600b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:31.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.140+0000 7f027e34b700 1 --2- 172.21.15.2:0/2245543315 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f02600b2900 0x7f02600b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f026c000c00 tx=0x7f026c01a040 comp rx=0 tx=0).stop 2024-05-14T09:33:31.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.140+0000 7f027e34b700 1 -- 172.21.15.2:0/2245543315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02781034c0 msgr2=0x7f027819bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:31.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.140+0000 7f027e34b700 1 --2- 172.21.15.2:0/2245543315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02781034c0 0x7f027819bc50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f026800e9a0 tx=0x7f026800ecb0 comp rx=0 tx=0).stop 2024-05-14T09:33:31.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.140+0000 7f027e34b700 1 -- 172.21.15.2:0/2245543315 shutdown_connections 2024-05-14T09:33:31.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.140+0000 7f027e34b700 1 --2- 172.21.15.2:0/2245543315 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f02600b2900 0x7f02600b4e10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:31.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.140+0000 7f027e34b700 1 --2- 172.21.15.2:0/2245543315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0278108740 0x7f027819c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:31.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.140+0000 7f027e34b700 1 --2- 172.21.15.2:0/2245543315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02781034c0 0x7f027819bc50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:31.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.140+0000 7f027e34b700 1 --2- 172.21.15.2:0/2245543315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0278107df0 0x7f027819c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:31.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.140+0000 7f027e34b700 1 -- 172.21.15.2:0/2245543315 >> 172.21.15.2:0/2245543315 conn(0x7f02780fedc0 msgr2=0x7f02781096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:31.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.141+0000 7f027e34b700 1 -- 172.21.15.2:0/2245543315 shutdown_connections 2024-05-14T09:33:31.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.141+0000 7f027e34b700 1 -- 172.21.15.2:0/2245543315 wait complete. 2024-05-14T09:33:31.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.557+0000 7fc0834cc700 1 -- 172.21.15.2:0/3164183247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc07c108740 msgr2=0x7fc07c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:31.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.557+0000 7fc0834cc700 1 --2- 172.21.15.2:0/3164183247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc07c108740 0x7fc07c1100a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc07400b0d0 tx=0x7fc07400b3e0 comp rx=0 tx=0).stop 2024-05-14T09:33:31.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.557+0000 7fc0834cc700 1 -- 172.21.15.2:0/3164183247 shutdown_connections 2024-05-14T09:33:31.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.557+0000 7fc0834cc700 1 --2- 172.21.15.2:0/3164183247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc07c1034c0 0x7fc07c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:31.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.557+0000 7fc0834cc700 1 --2- 172.21.15.2:0/3164183247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc07c108740 0x7fc07c1100a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:31.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.557+0000 7fc0834cc700 1 --2- 172.21.15.2:0/3164183247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc07c107df0 0x7fc07c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:31.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.557+0000 7fc0834cc700 1 -- 172.21.15.2:0/3164183247 >> 172.21.15.2:0/3164183247 conn(0x7fc07c0fedc0 msgr2=0x7fc07c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:31.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.557+0000 7fc0834cc700 1 -- 172.21.15.2:0/3164183247 shutdown_connections 2024-05-14T09:33:31.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.557+0000 7fc0834cc700 1 -- 172.21.15.2:0/3164183247 wait complete. 2024-05-14T09:33:31.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc0834cc700 1 Processor -- start 2024-05-14T09:33:31.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc0834cc700 1 -- start start 2024-05-14T09:33:31.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc0834cc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc07c1034c0 0x7fc07c199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:31.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc0834cc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc07c107df0 0x7fc07c19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:31.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc0834cc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc07c108740 0x7fc07c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:31.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc0834cc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c113460 con 0x7fc07c108740 2024-05-14T09:33:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc0834cc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c1132e0 con 0x7fc07c107df0 2024-05-14T09:33:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc0834cc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c1135e0 con 0x7fc07c1034c0 2024-05-14T09:33:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc081a69700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc07c108740 0x7fc07c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc081a69700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc07c108740 0x7fc07c19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40928/0 (socket says 172.21.15.2:40928) 2024-05-14T09:33:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc081a69700 1 -- 172.21.15.2:0/3364560209 learned_addr learned my addr 172.21.15.2:0/3364560209 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc081268700 1 --2- 172.21.15.2:0/3364560209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc07c1034c0 0x7fc07c199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc080a67700 1 --2- 172.21.15.2:0/3364560209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc07c107df0 0x7fc07c19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc081a69700 1 -- 172.21.15.2:0/3364560209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc07c1034c0 msgr2=0x7fc07c199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc081a69700 1 --2- 172.21.15.2:0/3364560209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc07c1034c0 0x7fc07c199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc081a69700 1 -- 172.21.15.2:0/3364560209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc07c107df0 msgr2=0x7fc07c19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc081a69700 1 --2- 172.21.15.2:0/3364560209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc07c107df0 0x7fc07c19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.558+0000 7fc081a69700 1 -- 172.21.15.2:0/3364560209 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07c19ecc0 con 0x7fc07c108740 2024-05-14T09:33:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.559+0000 7fc081a69700 1 --2- 172.21.15.2:0/3364560209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc07c108740 0x7fc07c19e620 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc074009fd0 tx=0x7fc074007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.559+0000 7fc0727fc700 1 -- 172.21.15.2:0/3364560209 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07401f070 con 0x7fc07c108740 2024-05-14T09:33:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.559+0000 7fc0727fc700 1 -- 172.21.15.2:0/3364560209 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc074007e50 con 0x7fc07c108740 2024-05-14T09:33:31.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.559+0000 7fc0834cc700 1 -- 172.21.15.2:0/3364560209 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07c19ef50 con 0x7fc07c108740 2024-05-14T09:33:31.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.559+0000 7fc0727fc700 1 -- 172.21.15.2:0/3364560209 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07401b6a0 con 0x7fc07c108740 2024-05-14T09:33:31.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.559+0000 7fc0834cc700 1 -- 172.21.15.2:0/3364560209 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07c19f430 con 0x7fc07c108740 2024-05-14T09:33:31.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.560+0000 7fc0834cc700 1 -- 172.21.15.2:0/3364560209 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc07c0512c0 con 0x7fc07c108740 2024-05-14T09:33:31.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.563+0000 7fc0727fc700 1 -- 172.21.15.2:0/3364560209 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc074022070 con 0x7fc07c108740 2024-05-14T09:33:31.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.563+0000 7fc0727fc700 1 --2- 172.21.15.2:0/3364560209 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0680b2880 0x7fc0680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:31.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.564+0000 7fc081268700 1 --2- 172.21.15.2:0/3364560209 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0680b2880 0x7fc0680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:31.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.564+0000 7fc0727fc700 1 -- 172.21.15.2:0/3364560209 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fc07400f020 con 0x7fc07c108740 2024-05-14T09:33:31.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.564+0000 7fc081268700 1 --2- 172.21.15.2:0/3364560209 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0680b2880 0x7fc0680b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc078009a10 tx=0x7fc078009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:31.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.567+0000 7fc0727fc700 1 -- 172.21.15.2:0/3364560209 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc074029020 con 0x7fc07c108740 2024-05-14T09:33:31.641 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:31 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2245543315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-14T09:33:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.796+0000 7fc0834cc700 1 -- 172.21.15.2:0/3364560209 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fc07c10b1d0 con 0x7fc07c108740 2024-05-14T09:33:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.797+0000 7fc0727fc700 1 -- 172.21.15.2:0/3364560209 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc0740ab5f0 con 0x7fc07c108740 2024-05-14T09:33:31.797 INFO:teuthology.orchestra.run.smithi002.stdout:304942678061 2024-05-14T09:33:31.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.799+0000 7fc0834cc700 1 -- 172.21.15.2:0/3364560209 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0680b2880 msgr2=0x7fc0680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:31.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.799+0000 7fc0834cc700 1 --2- 172.21.15.2:0/3364560209 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0680b2880 0x7fc0680b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc078009a10 tx=0x7fc078009190 comp rx=0 tx=0).stop 2024-05-14T09:33:31.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.799+0000 7fc0834cc700 1 -- 172.21.15.2:0/3364560209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc07c108740 msgr2=0x7fc07c19e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:31.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.799+0000 7fc0834cc700 1 --2- 172.21.15.2:0/3364560209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc07c108740 0x7fc07c19e620 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc074009fd0 tx=0x7fc074007a20 comp rx=0 tx=0).stop 2024-05-14T09:33:31.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.799+0000 7fc0834cc700 1 -- 172.21.15.2:0/3364560209 shutdown_connections 2024-05-14T09:33:31.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.799+0000 7fc0834cc700 1 --2- 172.21.15.2:0/3364560209 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0680b2880 0x7fc0680b4d90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:31.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.799+0000 7fc0834cc700 1 --2- 172.21.15.2:0/3364560209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc07c108740 0x7fc07c19e620 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:31.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.799+0000 7fc0834cc700 1 --2- 172.21.15.2:0/3364560209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc07c1034c0 0x7fc07c199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:31.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.799+0000 7fc0834cc700 1 --2- 172.21.15.2:0/3364560209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc07c107df0 0x7fc07c19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:31.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.799+0000 7fc0834cc700 1 -- 172.21.15.2:0/3364560209 >> 172.21.15.2:0/3364560209 conn(0x7fc07c0fedc0 msgr2=0x7fc07c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:31.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.799+0000 7fc0834cc700 1 -- 172.21.15.2:0/3364560209 shutdown_connections 2024-05-14T09:33:31.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.799+0000 7fc0834cc700 1 -- 172.21.15.2:0/3364560209 wait complete. 2024-05-14T09:33:31.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:31 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2245543315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-14T09:33:31.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:31 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2245543315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-14T09:33:32.452 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678056 got 304942678061 for osd.11 2024-05-14T09:33:32.452 DEBUG:teuthology.parallel:result is None 2024-05-14T09:33:32.533 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:32 smithi002 ceph-mon[31557]: pgmap v381: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:32.534 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:32 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1202768972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-14T09:33:32.534 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:32 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3364560209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-14T09:33:32.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.651+0000 7fac9bbc1700 1 -- 172.21.15.2:0/297529936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac94110c30 msgr2=0x7fac94113050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:32.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.651+0000 7fac9bbc1700 1 --2- 172.21.15.2:0/297529936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac94110c30 0x7fac94113050 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00b0a0 tx=0x7fac8c00b3b0 comp rx=0 tx=0).stop 2024-05-14T09:33:32.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.651+0000 7fac9bbc1700 1 -- 172.21.15.2:0/297529936 shutdown_connections 2024-05-14T09:33:32.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.651+0000 7fac9bbc1700 1 --2- 172.21.15.2:0/297529936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac94110c30 0x7fac94113050 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:32.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.651+0000 7fac9bbc1700 1 --2- 172.21.15.2:0/297529936 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac94103080 0x7fac94103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:32.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.651+0000 7fac9bbc1700 1 --2- 172.21.15.2:0/297529936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac94103a60 0x7fac941106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:32.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.651+0000 7fac9bbc1700 1 -- 172.21.15.2:0/297529936 >> 172.21.15.2:0/297529936 conn(0x7fac940fedc0 msgr2=0x7fac941011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:32.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.652+0000 7fac9bbc1700 1 -- 172.21.15.2:0/297529936 shutdown_connections 2024-05-14T09:33:32.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.652+0000 7fac9bbc1700 1 -- 172.21.15.2:0/297529936 wait complete. 2024-05-14T09:33:32.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.652+0000 7fac9bbc1700 1 Processor -- start 2024-05-14T09:33:32.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.652+0000 7fac9bbc1700 1 -- start start 2024-05-14T09:33:32.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9bbc1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac94103080 0x7fac9419d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:32.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9bbc1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac94103a60 0x7fac9419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:32.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9bbc1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac94110c30 0x7fac941a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:32.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9bbc1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac94116690 con 0x7fac94103080 2024-05-14T09:33:32.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9bbc1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fac94116510 con 0x7fac94110c30 2024-05-14T09:33:32.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9bbc1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac94116810 con 0x7fac94103a60 2024-05-14T09:33:32.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9995d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac94103080 0x7fac9419d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:32.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9995d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac94103080 0x7fac9419d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40946/0 (socket says 172.21.15.2:40946) 2024-05-14T09:33:32.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9995d700 1 -- 172.21.15.2:0/2858469162 learned_addr learned my addr 172.21.15.2:0/2858469162 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:32.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9a15e700 1 --2- 172.21.15.2:0/2858469162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac94110c30 0x7fac941a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:32.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9915c700 1 --2- 172.21.15.2:0/2858469162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac94103a60 0x7fac9419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:32.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9995d700 1 -- 172.21.15.2:0/2858469162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac94103a60 msgr2=0x7fac9419d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:32.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9995d700 1 --2- 172.21.15.2:0/2858469162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac94103a60 0x7fac9419d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:32.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9995d700 1 -- 172.21.15.2:0/2858469162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac94110c30 msgr2=0x7fac941a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:32.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9995d700 1 --2- 172.21.15.2:0/2858469162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac94110c30 0x7fac941a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:32.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9995d700 1 -- 172.21.15.2:0/2858469162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac941a1ef0 con 0x7fac94103080 2024-05-14T09:33:32.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9915c700 1 --2- 172.21.15.2:0/2858469162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac94103a60 0x7fac9419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:33:32.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.653+0000 7fac9995d700 1 --2- 172.21.15.2:0/2858469162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac94103080 0x7fac9419d020 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fac9000e9a0 tx=0x7fac9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:32.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.654+0000 7fac8affd700 1 -- 172.21.15.2:0/2858469162 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9000cbd0 con 0x7fac94103080 2024-05-14T09:33:32.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.654+0000 7fac8affd700 1 -- 172.21.15.2:0/2858469162 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac9000cd70 con 0x7fac94103080 2024-05-14T09:33:32.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.654+0000 7fac8affd700 1 -- 172.21.15.2:0/2858469162 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90018680 con 0x7fac94103080 2024-05-14T09:33:32.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.654+0000 7fac9bbc1700 1 -- 172.21.15.2:0/2858469162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac941a21e0 con 0x7fac94103080 2024-05-14T09:33:32.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.654+0000 7fac9bbc1700 1 -- 172.21.15.2:0/2858469162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac941a9b00 con 0x7fac94103080 2024-05-14T09:33:32.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.655+0000 7fac9bbc1700 1 -- 172.21.15.2:0/2858469162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac940512c0 con 0x7fac94103080 2024-05-14T09:33:32.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.657+0000 7fac8affd700 1 -- 172.21.15.2:0/2858469162 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fac90010430 con 0x7fac94103080 2024-05-14T09:33:32.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.658+0000 7fac8affd700 1 --2- 172.21.15.2:0/2858469162 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fac800b2900 0x7fac800b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:32.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.658+0000 7fac9915c700 1 --2- 172.21.15.2:0/2858469162 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fac800b2900 0x7fac800b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:32.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.658+0000 7fac8affd700 1 -- 172.21.15.2:0/2858469162 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fac900e2ce0 con 0x7fac94103080 2024-05-14T09:33:32.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.659+0000 7fac9915c700 1 --2- 172.21.15.2:0/2858469162 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fac800b2900 0x7fac800b4e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac941121a0 tx=0x7fac8400b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:32.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.662+0000 7fac8affd700 1 -- 172.21.15.2:0/2858469162 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac900aa5e0 con 0x7fac94103080 2024-05-14T09:33:32.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.884+0000 7fac9bbc1700 1 -- 172.21.15.2:0/2858469162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fac9402e5f0 con 0x7fac94103080 2024-05-14T09:33:32.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.885+0000 7fac8affd700 1 -- 172.21.15.2:0/2858469162 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fac900a9ee0 con 0x7fac94103080 2024-05-14T09:33:32.885 INFO:teuthology.orchestra.run.smithi002.stdout:55834574946 2024-05-14T09:33:32.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.887+0000 7fac9bbc1700 1 -- 172.21.15.2:0/2858469162 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fac800b2900 msgr2=0x7fac800b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:32.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.887+0000 7fac9bbc1700 1 --2- 172.21.15.2:0/2858469162 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fac800b2900 0x7fac800b4e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac941121a0 tx=0x7fac8400b5f0 comp rx=0 tx=0).stop 2024-05-14T09:33:32.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.887+0000 7fac9bbc1700 1 -- 172.21.15.2:0/2858469162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac94103080 msgr2=0x7fac9419d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:32.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.887+0000 7fac9bbc1700 1 --2- 172.21.15.2:0/2858469162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac94103080 0x7fac9419d020 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fac9000e9a0 tx=0x7fac9000ecb0 comp rx=0 tx=0).stop 2024-05-14T09:33:32.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.887+0000 7fac9bbc1700 1 -- 172.21.15.2:0/2858469162 shutdown_connections 2024-05-14T09:33:32.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.887+0000 7fac9bbc1700 1 --2- 172.21.15.2:0/2858469162 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fac800b2900 0x7fac800b4e10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:32.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.887+0000 7fac9bbc1700 1 --2- 172.21.15.2:0/2858469162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac94110c30 0x7fac941a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:32.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.887+0000 7fac9bbc1700 1 --2- 172.21.15.2:0/2858469162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac94103080 0x7fac9419d020 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:32.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.887+0000 7fac9bbc1700 1 --2- 172.21.15.2:0/2858469162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac94103a60 0x7fac9419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:32.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.887+0000 7fac9bbc1700 1 -- 172.21.15.2:0/2858469162 >> 172.21.15.2:0/2858469162 conn(0x7fac940fedc0 msgr2=0x7fac941126a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:32.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.887+0000 7fac9bbc1700 1 -- 172.21.15.2:0/2858469162 shutdown_connections 2024-05-14T09:33:32.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:32.887+0000 7fac9bbc1700 1 -- 172.21.15.2:0/2858469162 wait complete. 2024-05-14T09:33:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:32 smithi029 ceph-mon[36249]: pgmap v381: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:32 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1202768972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-14T09:33:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:32 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3364560209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-14T09:33:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:32 smithi026 ceph-mon[36056]: pgmap v381: 129 pgs: 129 active+clean; 643 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:32 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1202768972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-14T09:33:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:32 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3364560209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-14T09:33:33.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.100+0000 7f0f13c41700 1 -- 172.21.15.2:0/3733842332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f0c1034c0 msgr2=0x7f0f0c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:33.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.100+0000 7f0f13c41700 1 --2- 172.21.15.2:0/3733842332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f0c1034c0 0x7f0f0c1038d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00b0d0 tx=0x7f0efc00b3e0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.100+0000 7f0f13c41700 1 -- 172.21.15.2:0/3733842332 shutdown_connections 2024-05-14T09:33:33.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.100+0000 7f0f13c41700 1 --2- 172.21.15.2:0/3733842332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f0c108740 0x7f0f0c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.100+0000 7f0f13c41700 1 --2- 172.21.15.2:0/3733842332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f0c1034c0 0x7f0f0c1038d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.100+0000 7f0f13c41700 1 --2- 172.21.15.2:0/3733842332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c107df0 0x7f0f0c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.100+0000 7f0f13c41700 1 -- 172.21.15.2:0/3733842332 >> 172.21.15.2:0/3733842332 conn(0x7f0f0c0fedc0 msgr2=0x7f0f0c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:33.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.100+0000 7f0f13c41700 1 -- 172.21.15.2:0/3733842332 shutdown_connections 2024-05-14T09:33:33.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.100+0000 7f0f13c41700 1 -- 172.21.15.2:0/3733842332 wait complete. 2024-05-14T09:33:33.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.100+0000 7f0f13c41700 1 Processor -- start 2024-05-14T09:33:33.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f13c41700 1 -- start start 2024-05-14T09:33:33.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f13c41700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c1034c0 0x7f0f0c19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:33.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f13c41700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f0c107df0 0x7f0f0c19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:33.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f13c41700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f0c108740 0x7f0f0c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:33.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f13c41700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c113370 con 0x7f0f0c108740 2024-05-14T09:33:33.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f13c41700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1131f0 con 0x7f0f0c1034c0 2024-05-14T09:33:33.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f13c41700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1134f0 con 0x7f0f0c107df0 2024-05-14T09:33:33.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f121de700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f0c108740 0x7f0f0c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:33.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f121de700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f0c108740 0x7f0f0c19c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40968/0 (socket says 172.21.15.2:40968) 2024-05-14T09:33:33.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f121de700 1 -- 172.21.15.2:0/2039693916 learned_addr learned my addr 172.21.15.2:0/2039693916 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:33.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f119dd700 1 --2- 172.21.15.2:0/2039693916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c1034c0 0x7f0f0c19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:33.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f111dc700 1 --2- 172.21.15.2:0/2039693916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f0c107df0 0x7f0f0c19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:33.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f121de700 1 -- 172.21.15.2:0/2039693916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f0c107df0 msgr2=0x7f0f0c19c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:33.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f121de700 1 --2- 172.21.15.2:0/2039693916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f0c107df0 0x7f0f0c19c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f121de700 1 -- 172.21.15.2:0/2039693916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c1034c0 msgr2=0x7f0f0c19bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:33.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f121de700 1 --2- 172.21.15.2:0/2039693916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c1034c0 0x7f0f0c19bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f121de700 1 -- 172.21.15.2:0/2039693916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0c1a6f10 con 0x7f0f0c108740 2024-05-14T09:33:33.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.101+0000 7f0f121de700 1 --2- 172.21.15.2:0/2039693916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f0c108740 0x7f0f0c19c6f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400c900 tx=0x7f0f0400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:33.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.102+0000 7f0f0affd700 1 -- 172.21.15.2:0/2039693916 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04007b00 con 0x7f0f0c108740 2024-05-14T09:33:33.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.102+0000 7f0f13c41700 1 -- 172.21.15.2:0/2039693916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0c1a7200 con 0x7f0f0c108740 2024-05-14T09:33:33.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.102+0000 7f0f0affd700 1 -- 172.21.15.2:0/2039693916 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f0400ce50 con 0x7f0f0c108740 2024-05-14T09:33:33.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.102+0000 7f0f0affd700 1 -- 172.21.15.2:0/2039693916 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04018660 con 0x7f0f0c108740 2024-05-14T09:33:33.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.102+0000 7f0f13c41700 1 -- 172.21.15.2:0/2039693916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f0c1a7740 con 0x7f0f0c108740 2024-05-14T09:33:33.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.103+0000 7f0f13c41700 1 -- 172.21.15.2:0/2039693916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f0c0512c0 con 0x7f0f0c108740 2024-05-14T09:33:33.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.104+0000 7f0f0affd700 1 -- 172.21.15.2:0/2039693916 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0f0400f450 con 0x7f0f0c108740 2024-05-14T09:33:33.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.105+0000 7f0f0affd700 1 --2- 172.21.15.2:0/2039693916 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0ef40b2880 0x7f0ef40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:33.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.106+0000 7f0f0affd700 1 -- 172.21.15.2:0/2039693916 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f0f040e2530 con 0x7f0f0c108740 2024-05-14T09:33:33.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.106+0000 7f0f119dd700 1 --2- 172.21.15.2:0/2039693916 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0ef40b2880 0x7f0ef40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:33.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.106+0000 7f0f119dd700 1 --2- 172.21.15.2:0/2039693916 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0ef40b2880 0x7f0ef40b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009fd0 tx=0x7f0efc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:33.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.109+0000 7f0f0affd700 1 -- 172.21.15.2:0/2039693916 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f04014030 con 0x7f0f0c108740 2024-05-14T09:33:33.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.331+0000 7f0f13c41700 1 -- 172.21.15.2:0/2039693916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0f0c06bc00 con 0x7f0f0c108740 2024-05-14T09:33:33.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.331+0000 7f0f0affd700 1 -- 172.21.15.2:0/2039693916 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0f040a9ec0 con 0x7f0f0c108740 2024-05-14T09:33:33.331 INFO:teuthology.orchestra.run.smithi002.stdout:206158430275 2024-05-14T09:33:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.334+0000 7f0f13c41700 1 -- 172.21.15.2:0/2039693916 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0ef40b2880 msgr2=0x7f0ef40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.334+0000 7f0f13c41700 1 --2- 172.21.15.2:0/2039693916 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0ef40b2880 0x7f0ef40b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009fd0 tx=0x7f0efc00b040 comp rx=0 tx=0).stop 2024-05-14T09:33:33.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.334+0000 7f0f13c41700 1 -- 172.21.15.2:0/2039693916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f0c108740 msgr2=0x7f0f0c19c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:33.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.334+0000 7f0f13c41700 1 --2- 172.21.15.2:0/2039693916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f0c108740 0x7f0f0c19c6f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400c900 tx=0x7f0f0400cc10 comp rx=0 tx=0).stop 2024-05-14T09:33:33.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.334+0000 7f0f13c41700 1 -- 172.21.15.2:0/2039693916 shutdown_connections 2024-05-14T09:33:33.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.334+0000 7f0f13c41700 1 --2- 172.21.15.2:0/2039693916 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0ef40b2880 0x7f0ef40b4d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.334+0000 7f0f13c41700 1 --2- 172.21.15.2:0/2039693916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0c1034c0 0x7f0f0c19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.334+0000 7f0f13c41700 1 --2- 172.21.15.2:0/2039693916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f0c108740 0x7f0f0c19c6f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.334+0000 7f0f13c41700 1 --2- 172.21.15.2:0/2039693916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f0c107df0 0x7f0f0c19c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.334+0000 7f0f13c41700 1 -- 172.21.15.2:0/2039693916 >> 172.21.15.2:0/2039693916 conn(0x7f0f0c0fedc0 msgr2=0x7f0f0c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:33.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.334+0000 7f0f13c41700 1 -- 172.21.15.2:0/2039693916 shutdown_connections 2024-05-14T09:33:33.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.334+0000 7f0f13c41700 1 -- 172.21.15.2:0/2039693916 wait complete. 2024-05-14T09:33:33.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.565+0000 7f8eeec7a700 1 -- 172.21.15.2:0/3401320796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8107df0 msgr2=0x7f8ee8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:33.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.565+0000 7f8eeec7a700 1 --2- 172.21.15.2:0/3401320796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8107df0 0x7f8ee8108200 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40099c0 tx=0x7f8ed4009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.565+0000 7f8eeec7a700 1 -- 172.21.15.2:0/3401320796 shutdown_connections 2024-05-14T09:33:33.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.565+0000 7f8eeec7a700 1 --2- 172.21.15.2:0/3401320796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee8108740 0x7f8ee81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.565+0000 7f8eeec7a700 1 --2- 172.21.15.2:0/3401320796 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ee81034c0 0x7f8ee81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.565+0000 7f8eeec7a700 1 --2- 172.21.15.2:0/3401320796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8107df0 0x7f8ee8108200 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.565+0000 7f8eeec7a700 1 -- 172.21.15.2:0/3401320796 >> 172.21.15.2:0/3401320796 conn(0x7f8ee80fedf0 msgr2=0x7f8ee8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:33.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.565+0000 7f8eeec7a700 1 -- 172.21.15.2:0/3401320796 shutdown_connections 2024-05-14T09:33:33.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.565+0000 7f8eeec7a700 1 -- 172.21.15.2:0/3401320796 wait complete. 2024-05-14T09:33:33.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.566+0000 7f8eeec7a700 1 Processor -- start 2024-05-14T09:33:33.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.566+0000 7f8eeec7a700 1 -- start start 2024-05-14T09:33:33.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.566+0000 7f8eeec7a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ee81034c0 0x7f8ee8199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:33.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.566+0000 7f8eeec7a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee8107df0 0x7f8ee819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:33.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.566+0000 7f8eeec7a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8108740 0x7f8ee819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:33.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.566+0000 7f8eeec7a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81133c0 con 0x7f8ee8108740 2024-05-14T09:33:33.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.566+0000 7f8eeec7a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8113240 con 0x7f8ee8107df0 2024-05-14T09:33:33.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.566+0000 7f8eeec7a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8113540 con 0x7f8ee81034c0 2024-05-14T09:33:33.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.566+0000 7f8eed217700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8108740 0x7f8ee819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:33.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.566+0000 7f8eed217700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8108740 0x7f8ee819e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40984/0 (socket says 172.21.15.2:40984) 2024-05-14T09:33:33.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.566+0000 7f8eed217700 1 -- 172.21.15.2:0/3615140372 learned_addr learned my addr 172.21.15.2:0/3615140372 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:33.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.566+0000 7f8eeca16700 1 --2- 172.21.15.2:0/3615140372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ee81034c0 0x7f8ee8199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:33.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.566+0000 7f8ee7fff700 1 --2- 172.21.15.2:0/3615140372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee8107df0 0x7f8ee819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:33.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.567+0000 7f8eed217700 1 -- 172.21.15.2:0/3615140372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ee81034c0 msgr2=0x7f8ee8199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:33.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.567+0000 7f8eed217700 1 --2- 172.21.15.2:0/3615140372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ee81034c0 0x7f8ee8199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.567+0000 7f8eed217700 1 -- 172.21.15.2:0/3615140372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee8107df0 msgr2=0x7f8ee819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:33.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.567+0000 7f8eed217700 1 --2- 172.21.15.2:0/3615140372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee8107df0 0x7f8ee819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.567+0000 7f8eed217700 1 -- 172.21.15.2:0/3615140372 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee819ec40 con 0x7f8ee8108740 2024-05-14T09:33:33.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.567+0000 7f8ee7fff700 1 --2- 172.21.15.2:0/3615140372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee8107df0 0x7f8ee819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:33:33.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.567+0000 7f8eed217700 1 --2- 172.21.15.2:0/3615140372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8108740 0x7f8ee819e5a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000b970 tx=0x7f8ee000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:33.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.567+0000 7f8ee5ffb700 1 -- 172.21.15.2:0/3615140372 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0019070 con 0x7f8ee8108740 2024-05-14T09:33:33.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.567+0000 7f8eeec7a700 1 -- 172.21.15.2:0/3615140372 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee819ef30 con 0x7f8ee8108740 2024-05-14T09:33:33.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.567+0000 7f8ee5ffb700 1 -- 172.21.15.2:0/3615140372 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee000cde0 con 0x7f8ee8108740 2024-05-14T09:33:33.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.567+0000 7f8ee5ffb700 1 -- 172.21.15.2:0/3615140372 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0007370 con 0x7f8ee8108740 2024-05-14T09:33:33.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.567+0000 7f8eeec7a700 1 -- 172.21.15.2:0/3615140372 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee81a6850 con 0x7f8ee8108740 2024-05-14T09:33:33.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.568+0000 7f8eeec7a700 1 -- 172.21.15.2:0/3615140372 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee80512c0 con 0x7f8ee8108740 2024-05-14T09:33:33.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.570+0000 7f8ee5ffb700 1 -- 172.21.15.2:0/3615140372 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8ee001c070 con 0x7f8ee8108740 2024-05-14T09:33:33.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.571+0000 7f8ee5ffb700 1 --2- 172.21.15.2:0/3615140372 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8ed80b2880 0x7f8ed80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:33.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.571+0000 7f8eeca16700 1 --2- 172.21.15.2:0/3615140372 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8ed80b2880 0x7f8ed80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:33.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.571+0000 7f8ee5ffb700 1 -- 172.21.15.2:0/3615140372 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f8ee00e1c90 con 0x7f8ee8108740 2024-05-14T09:33:33.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.571+0000 7f8eeca16700 1 --2- 172.21.15.2:0/3615140372 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8ed80b2880 0x7f8ed80b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8edc005950 tx=0x7f8edc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:33.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.574+0000 7f8ee5ffb700 1 -- 172.21.15.2:0/3615140372 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ee00a9530 con 0x7f8ee8108740 2024-05-14T09:33:33.782 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:33 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2858469162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-14T09:33:33.782 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:33 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2039693916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-14T09:33:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.790+0000 7f8eeec7a700 1 -- 172.21.15.2:0/3615140372 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8ee802e5f0 con 0x7f8ee8108740 2024-05-14T09:33:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.791+0000 7f8ee5ffb700 1 -- 172.21.15.2:0/3615140372 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8ee00a8e30 con 0x7f8ee8108740 2024-05-14T09:33:33.790 INFO:teuthology.orchestra.run.smithi002.stdout:77309411422 2024-05-14T09:33:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.793+0000 7f8eeec7a700 1 -- 172.21.15.2:0/3615140372 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8ed80b2880 msgr2=0x7f8ed80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.793+0000 7f8eeec7a700 1 --2- 172.21.15.2:0/3615140372 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8ed80b2880 0x7f8ed80b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8edc005950 tx=0x7f8edc00b410 comp rx=0 tx=0).stop 2024-05-14T09:33:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.793+0000 7f8eeec7a700 1 -- 172.21.15.2:0/3615140372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8108740 msgr2=0x7f8ee819e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.793+0000 7f8eeec7a700 1 --2- 172.21.15.2:0/3615140372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8108740 0x7f8ee819e5a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000b970 tx=0x7f8ee000bc80 comp rx=0 tx=0).stop 2024-05-14T09:33:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.793+0000 7f8eeec7a700 1 -- 172.21.15.2:0/3615140372 shutdown_connections 2024-05-14T09:33:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.793+0000 7f8eeec7a700 1 --2- 172.21.15.2:0/3615140372 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8ed80b2880 0x7f8ed80b4d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.793+0000 7f8eeec7a700 1 --2- 172.21.15.2:0/3615140372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8108740 0x7f8ee819e5a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.793+0000 7f8eeec7a700 1 --2- 172.21.15.2:0/3615140372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ee81034c0 0x7f8ee8199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.793+0000 7f8eeec7a700 1 --2- 172.21.15.2:0/3615140372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee8107df0 0x7f8ee819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.793+0000 7f8eeec7a700 1 -- 172.21.15.2:0/3615140372 >> 172.21.15.2:0/3615140372 conn(0x7f8ee80fedf0 msgr2=0x7f8ee81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.793+0000 7f8eeec7a700 1 -- 172.21.15.2:0/3615140372 shutdown_connections 2024-05-14T09:33:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:33.793+0000 7f8eeec7a700 1 -- 172.21.15.2:0/3615140372 wait complete. 2024-05-14T09:33:33.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:33 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2858469162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-14T09:33:33.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:33 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2039693916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-14T09:33:33.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:33 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2858469162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-14T09:33:33.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:33 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2039693916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-14T09:33:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:34 smithi002 ceph-mon[31557]: pgmap v382: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-14T09:33:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:34 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3615140372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-14T09:33:34.803 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411417 got 77309411422 for osd.2 2024-05-14T09:33:34.803 DEBUG:teuthology.parallel:result is None 2024-05-14T09:33:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:34 smithi029 ceph-mon[36249]: pgmap v382: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-14T09:33:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:34 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3615140372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-14T09:33:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:34 smithi026 ceph-mon[36056]: pgmap v382: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-14T09:33:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:34 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3615140372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-14T09:33:35.265 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574941 got 55834574946 for osd.1 2024-05-14T09:33:35.265 DEBUG:teuthology.parallel:result is None 2024-05-14T09:33:35.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.385+0000 7f6ff500f700 1 -- 172.21.15.2:0/126504750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ff0107e10 msgr2=0x7f6ff0108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:35.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.385+0000 7f6ff500f700 1 --2- 172.21.15.2:0/126504750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ff0107e10 0x7f6ff0108220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40099c0 tx=0x7f6fe4009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:35.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.385+0000 7f6ff500f700 1 -- 172.21.15.2:0/126504750 shutdown_connections 2024-05-14T09:33:35.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.385+0000 7f6ff500f700 1 --2- 172.21.15.2:0/126504750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff0108760 0x7f6ff01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:35.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.385+0000 7f6ff500f700 1 --2- 172.21.15.2:0/126504750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ff01034e0 0x7f6ff01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:35.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.385+0000 7f6ff500f700 1 --2- 172.21.15.2:0/126504750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ff0107e10 0x7f6ff0108220 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:35.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.385+0000 7f6ff500f700 1 -- 172.21.15.2:0/126504750 >> 172.21.15.2:0/126504750 conn(0x7f6ff00fef30 msgr2=0x7f6ff0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:35.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.385+0000 7f6ff500f700 1 -- 172.21.15.2:0/126504750 shutdown_connections 2024-05-14T09:33:35.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.385+0000 7f6ff500f700 1 -- 172.21.15.2:0/126504750 wait complete. 2024-05-14T09:33:35.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.386+0000 7f6ff500f700 1 Processor -- start 2024-05-14T09:33:35.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6ff500f700 1 -- start start 2024-05-14T09:33:35.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6ff500f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ff01034e0 0x7f6ff019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:35.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6ff500f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ff0107e10 0x7f6ff019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:35.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6ff500f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff0108760 0x7f6ff019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:35.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6ff500f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff0113370 con 0x7f6ff0107e10 2024-05-14T09:33:35.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6ff500f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff01131f0 con 0x7f6ff0108760 2024-05-14T09:33:35.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6ff500f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff01134f0 con 0x7f6ff01034e0 2024-05-14T09:33:35.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6fee59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ff0107e10 0x7f6ff019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:35.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6fee59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ff0107e10 0x7f6ff019b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40908/0 (socket says 172.21.15.2:40908) 2024-05-14T09:33:35.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6fee59c700 1 -- 172.21.15.2:0/1202768972 learned_addr learned my addr 172.21.15.2:0/1202768972 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:35.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6feed9d700 1 --2- 172.21.15.2:0/1202768972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ff01034e0 0x7f6ff019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:35.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6fef59e700 1 --2- 172.21.15.2:0/1202768972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff0108760 0x7f6ff019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:35.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6fee59c700 1 -- 172.21.15.2:0/1202768972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ff01034e0 msgr2=0x7f6ff019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:35.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6fee59c700 1 --2- 172.21.15.2:0/1202768972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ff01034e0 0x7f6ff019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:35.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6fee59c700 1 -- 172.21.15.2:0/1202768972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff0108760 msgr2=0x7f6ff019f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:35.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.387+0000 7f6fee59c700 1 --2- 172.21.15.2:0/1202768972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff0108760 0x7f6ff019f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:35.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.388+0000 7f6fee59c700 1 -- 172.21.15.2:0/1202768972 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff019fc70 con 0x7f6ff0107e10 2024-05-14T09:33:35.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.388+0000 7f6fee59c700 1 --2- 172.21.15.2:0/1202768972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ff0107e10 0x7f6ff019b3f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40045e0 tx=0x7f6fe4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:35.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.388+0000 7f6fd7fff700 1 -- 172.21.15.2:0/1202768972 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4026070 con 0x7f6ff0107e10 2024-05-14T09:33:35.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.388+0000 7f6ff500f700 1 -- 172.21.15.2:0/1202768972 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff019ff00 con 0x7f6ff0107e10 2024-05-14T09:33:35.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.388+0000 7f6fd7fff700 1 -- 172.21.15.2:0/1202768972 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe400f460 con 0x7f6ff0107e10 2024-05-14T09:33:35.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.388+0000 7f6fd7fff700 1 -- 172.21.15.2:0/1202768972 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4009e60 con 0x7f6ff0107e10 2024-05-14T09:33:35.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.388+0000 7f6ff500f700 1 -- 172.21.15.2:0/1202768972 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff01a0430 con 0x7f6ff0107e10 2024-05-14T09:33:35.400 INFO:teuthology.orchestra.run.smithi002.stdout:137438953555 2024-05-14T09:33:35.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.389+0000 7f6ff500f700 1 -- 172.21.15.2:0/1202768972 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff00512c0 con 0x7f6ff0107e10 2024-05-14T09:33:35.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.391+0000 7f6fd7fff700 1 -- 172.21.15.2:0/1202768972 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6fe4029070 con 0x7f6ff0107e10 2024-05-14T09:33:35.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.391+0000 7f6fd7fff700 1 --2- 172.21.15.2:0/1202768972 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6fd80b2880 0x7f6fd80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:35.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.392+0000 7f6feed9d700 1 --2- 172.21.15.2:0/1202768972 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6fd80b2880 0x7f6fd80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:35.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.392+0000 7f6fd7fff700 1 -- 172.21.15.2:0/1202768972 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f6fe40e30b0 con 0x7f6ff0107e10 2024-05-14T09:33:35.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.392+0000 7f6feed9d700 1 --2- 172.21.15.2:0/1202768972 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6fd80b2880 0x7f6fd80b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009c60 tx=0x7f6fe0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:35.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.395+0000 7f6fd7fff700 1 -- 172.21.15.2:0/1202768972 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6fe4013070 con 0x7f6ff0107e10 2024-05-14T09:33:35.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.626+0000 7f6ff500f700 1 -- 172.21.15.2:0/1202768972 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6ff019c1b0 con 0x7f6ff0107e10 2024-05-14T09:33:35.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.626+0000 7f6fd7fff700 1 -- 172.21.15.2:0/1202768972 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6fe40aaa40 con 0x7f6ff0107e10 2024-05-14T09:33:35.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.629+0000 7f6ff500f700 1 -- 172.21.15.2:0/1202768972 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6fd80b2880 msgr2=0x7f6fd80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:35.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.629+0000 7f6ff500f700 1 --2- 172.21.15.2:0/1202768972 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6fd80b2880 0x7f6fd80b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009c60 tx=0x7f6fe0009260 comp rx=0 tx=0).stop 2024-05-14T09:33:35.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.629+0000 7f6ff500f700 1 -- 172.21.15.2:0/1202768972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ff0107e10 msgr2=0x7f6ff019b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:35.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.629+0000 7f6ff500f700 1 --2- 172.21.15.2:0/1202768972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ff0107e10 0x7f6ff019b3f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40045e0 tx=0x7f6fe4004610 comp rx=0 tx=0).stop 2024-05-14T09:33:35.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.629+0000 7f6ff500f700 1 -- 172.21.15.2:0/1202768972 shutdown_connections 2024-05-14T09:33:35.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.629+0000 7f6ff500f700 1 --2- 172.21.15.2:0/1202768972 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6fd80b2880 0x7f6fd80b4d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:35.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.629+0000 7f6ff500f700 1 --2- 172.21.15.2:0/1202768972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff0108760 0x7f6ff019f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:35.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.629+0000 7f6ff500f700 1 --2- 172.21.15.2:0/1202768972 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ff01034e0 0x7f6ff019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:35.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.629+0000 7f6ff500f700 1 --2- 172.21.15.2:0/1202768972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ff0107e10 0x7f6ff019b3f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:35.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.629+0000 7f6ff500f700 1 -- 172.21.15.2:0/1202768972 >> 172.21.15.2:0/1202768972 conn(0x7f6ff00fef30 msgr2=0x7f6ff006c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:35.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.629+0000 7f6ff500f700 1 -- 172.21.15.2:0/1202768972 shutdown_connections 2024-05-14T09:33:35.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:31.629+0000 7f6ff500f700 1 -- 172.21.15.2:0/1202768972 wait complete. 2024-05-14T09:33:36.222 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430268 got 206158430275 for osd.7 2024-05-14T09:33:36.222 DEBUG:teuthology.parallel:result is None 2024-05-14T09:33:36.667 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037812 got 257698037817 for osd.9 2024-05-14T09:33:36.667 DEBUG:teuthology.parallel:result is None 2024-05-14T09:33:36.833 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:36 smithi002 ceph-mon[31557]: pgmap v383: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T09:33:36.833 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:33:36.833 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:33:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:36 smithi029 ceph-mon[36249]: pgmap v383: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T09:33:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:33:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:33:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:36 smithi026 ceph-mon[36056]: pgmap v383: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T09:33:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:33:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:33:37.127 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953550 got 137438953555 for osd.4 2024-05-14T09:33:37.127 DEBUG:teuthology.parallel:result is None 2024-05-14T09:33:37.127 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-14T09:33:37.127 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T09:33:37.385 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:37.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:37 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:33:37.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:37 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:33:37.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:37 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:33:37.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:37 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:33:37.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:37 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:33:37.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:37 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:33:37.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:37 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:33:37.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:37 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:33:37.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:37 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:33:38.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:38 smithi002 ceph-mon[31557]: pgmap v384: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T09:33:38.839 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:38 smithi029 ceph-mon[36249]: pgmap v384: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T09:33:38.854 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:38 smithi026 ceph-mon[36056]: pgmap v384: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T09:33:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.166+0000 7fc2764ea700 1 -- 172.21.15.2:0/3384477260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc270108740 msgr2=0x7fc2701100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.166+0000 7fc2764ea700 1 --2- 172.21.15.2:0/3384477260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc270108740 0x7fc2701100a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc26800b0d0 tx=0x7fc26800b3e0 comp rx=0 tx=0).stop 2024-05-14T09:33:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.166+0000 7fc2764ea700 1 -- 172.21.15.2:0/3384477260 shutdown_connections 2024-05-14T09:33:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.166+0000 7fc2764ea700 1 --2- 172.21.15.2:0/3384477260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc270108740 0x7fc2701100a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.166+0000 7fc2764ea700 1 --2- 172.21.15.2:0/3384477260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc2701034c0 0x7fc2701038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.166+0000 7fc2764ea700 1 --2- 172.21.15.2:0/3384477260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc270107df0 0x7fc270108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.166+0000 7fc2764ea700 1 -- 172.21.15.2:0/3384477260 >> 172.21.15.2:0/3384477260 conn(0x7fc2700fedc0 msgr2=0x7fc2701011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.166+0000 7fc2764ea700 1 -- 172.21.15.2:0/3384477260 shutdown_connections 2024-05-14T09:33:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.166+0000 7fc2764ea700 1 -- 172.21.15.2:0/3384477260 wait complete. 2024-05-14T09:33:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.167+0000 7fc2764ea700 1 Processor -- start 2024-05-14T09:33:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.167+0000 7fc2764ea700 1 -- start start 2024-05-14T09:33:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.167+0000 7fc2764ea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2701034c0 0x7fc2700ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.167+0000 7fc2764ea700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc270107df0 0x7fc2700ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.167+0000 7fc2764ea700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc270108740 0x7fc2700ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.167+0000 7fc2764ea700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc270113370 con 0x7fc270107df0 2024-05-14T09:33:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc2764ea700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc2701131f0 con 0x7fc2701034c0 2024-05-14T09:33:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc2764ea700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc2701134f0 con 0x7fc270108740 2024-05-14T09:33:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc26f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc270107df0 0x7fc2700ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc26f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc270107df0 0x7fc2700ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57138/0 (socket says 172.21.15.2:57138) 2024-05-14T09:33:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc26f7fe700 1 -- 172.21.15.2:0/2617396419 learned_addr learned my addr 172.21.15.2:0/2617396419 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc26ffff700 1 --2- 172.21.15.2:0/2617396419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2701034c0 0x7fc2700ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc274a87700 1 --2- 172.21.15.2:0/2617396419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc270108740 0x7fc2700ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc26f7fe700 1 -- 172.21.15.2:0/2617396419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc270108740 msgr2=0x7fc2700ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc26f7fe700 1 --2- 172.21.15.2:0/2617396419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc270108740 0x7fc2700ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc26f7fe700 1 -- 172.21.15.2:0/2617396419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2701034c0 msgr2=0x7fc2700ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc26f7fe700 1 --2- 172.21.15.2:0/2617396419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2701034c0 0x7fc2700ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc26f7fe700 1 -- 172.21.15.2:0/2617396419 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2701a9150 con 0x7fc270107df0 2024-05-14T09:33:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc274a87700 1 --2- 172.21.15.2:0/2617396419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc270108740 0x7fc2700ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:33:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc26f7fe700 1 --2- 172.21.15.2:0/2617396419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc270107df0 0x7fc2700ff7e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc26400c8f0 tx=0x7fc26400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc26d7fa700 1 -- 172.21.15.2:0/2617396419 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264007940 con 0x7fc270107df0 2024-05-14T09:33:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.168+0000 7fc2764ea700 1 -- 172.21.15.2:0/2617396419 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2701a9440 con 0x7fc270107df0 2024-05-14T09:33:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.169+0000 7fc26d7fa700 1 -- 172.21.15.2:0/2617396419 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc26400f450 con 0x7fc270107df0 2024-05-14T09:33:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.169+0000 7fc26d7fa700 1 -- 172.21.15.2:0/2617396419 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264018660 con 0x7fc270107df0 2024-05-14T09:33:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.169+0000 7fc2764ea700 1 -- 172.21.15.2:0/2617396419 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2701a99d0 con 0x7fc270107df0 2024-05-14T09:33:39.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.170+0000 7fc2764ea700 1 -- 172.21.15.2:0/2617396419 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2700512c0 con 0x7fc270107df0 2024-05-14T09:33:39.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.171+0000 7fc26d7fa700 1 -- 172.21.15.2:0/2617396419 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc26400f890 con 0x7fc270107df0 2024-05-14T09:33:39.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.172+0000 7fc26d7fa700 1 --2- 172.21.15.2:0/2617396419 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc2580b2880 0x7fc2580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.172+0000 7fc26ffff700 1 --2- 172.21.15.2:0/2617396419 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc2580b2880 0x7fc2580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.172+0000 7fc26d7fa700 1 -- 172.21.15.2:0/2617396419 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fc2640e2550 con 0x7fc270107df0 2024-05-14T09:33:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.172+0000 7fc26ffff700 1 --2- 172.21.15.2:0/2617396419 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc2580b2880 0x7fc2580b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc260005950 tx=0x7fc26000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:39.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.175+0000 7fc26d7fa700 1 -- 172.21.15.2:0/2617396419 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc264014030 con 0x7fc270107df0 2024-05-14T09:33:39.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.384+0000 7fc2764ea700 1 -- 172.21.15.2:0/2617396419 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc2701013e0 con 0x7fc2580b2880 2024-05-14T09:33:39.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.394+0000 7fc26d7fa700 1 -- 172.21.15.2:0/2617396419 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340738 (secure 0 0 0) 0x7fc2701013e0 con 0x7fc2580b2880 2024-05-14T09:33:39.398 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:33:39.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.397+0000 7fc2764ea700 1 -- 172.21.15.2:0/2617396419 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc2580b2880 msgr2=0x7fc2580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:39.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.397+0000 7fc2764ea700 1 --2- 172.21.15.2:0/2617396419 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc2580b2880 0x7fc2580b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc260005950 tx=0x7fc26000a300 comp rx=0 tx=0).stop 2024-05-14T09:33:39.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.397+0000 7fc2764ea700 1 -- 172.21.15.2:0/2617396419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc270107df0 msgr2=0x7fc2700ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:39.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.397+0000 7fc2764ea700 1 --2- 172.21.15.2:0/2617396419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc270107df0 0x7fc2700ff7e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc26400c8f0 tx=0x7fc26400cc00 comp rx=0 tx=0).stop 2024-05-14T09:33:39.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.398+0000 7fc2764ea700 1 -- 172.21.15.2:0/2617396419 shutdown_connections 2024-05-14T09:33:39.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.398+0000 7fc2764ea700 1 --2- 172.21.15.2:0/2617396419 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc2580b2880 0x7fc2580b4d90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:39.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.398+0000 7fc2764ea700 1 --2- 172.21.15.2:0/2617396419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2701034c0 0x7fc2700ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:39.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.398+0000 7fc2764ea700 1 --2- 172.21.15.2:0/2617396419 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc270108740 0x7fc2700ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:39.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.398+0000 7fc2764ea700 1 --2- 172.21.15.2:0/2617396419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc270107df0 0x7fc2700ff7e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:39.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.398+0000 7fc2764ea700 1 -- 172.21.15.2:0/2617396419 >> 172.21.15.2:0/2617396419 conn(0x7fc2700fedc0 msgr2=0x7fc270107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:39.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.398+0000 7fc2764ea700 1 -- 172.21.15.2:0/2617396419 shutdown_connections 2024-05-14T09:33:39.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:39.398+0000 7fc2764ea700 1 -- 172.21.15.2:0/2617396419 wait complete. 2024-05-14T09:33:39.401 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-05-14T09:33:40.221 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-05-14T09:33:39.084352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659053,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":703,"num_read_kb":450,"num_write":300,"num_write_kb":2275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2954686,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":744,"ondisk_log_size":744,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":394500,"kb_used_data":12068,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124547324,"statfs":{"total":1151940427776,"available":1151536459776,"internally_reserved":0,"allocated":12357632,"data_stored":8174309,"data_compressed":496431,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19140,"internal_metadata":391492924},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":47,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003562"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087224+0000","last_change":"2024-05-14T09:30:56.797051+0000","last_active":"2024-05-14T09:33:29.087224+0000","last_peered":"2024-05-14T09:33:29.087224+0000","last_clean":"2024-05-14T09:33:29.087224+0000","last_became_active":"2024-05-14T09:30:56.796517+0000","last_became_peered":"2024-05-14T09:30:56.796517+0000","last_unstale":"2024-05-14T09:33:29.087224+0000","last_undegraded":"2024-05-14T09:33:29.087224+0000","last_fullsized":"2024-05-14T09:33:29.087224+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:58:22.170187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.073126+0000","last_change":"2024-05-14T09:30:56.795325+0000","last_active":"2024-05-14T09:33:29.073126+0000","last_peered":"2024-05-14T09:33:29.073126+0000","last_clean":"2024-05-14T09:33:29.073126+0000","last_became_active":"2024-05-14T09:30:56.794962+0000","last_became_peered":"2024-05-14T09:30:56.794962+0000","last_unstale":"2024-05-14T09:33:29.073126+0000","last_undegraded":"2024-05-14T09:33:29.073126+0000","last_fullsized":"2024-05-14T09:33:29.073126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:15:28.890565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084179+0000","last_change":"2024-05-14T09:30:56.793200+0000","last_active":"2024-05-14T09:33:29.084179+0000","last_peered":"2024-05-14T09:33:29.084179+0000","last_clean":"2024-05-14T09:33:29.084179+0000","last_became_active":"2024-05-14T09:30:56.792835+0000","last_became_peered":"2024-05-14T09:30:56.792835+0000","last_unstale":"2024-05-14T09:33:29.084179+0000","last_undegraded":"2024-05-14T09:33:29.084179+0000","last_fullsized":"2024-05-14T09:33:29.084179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:19.851276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.070997+0000","last_change":"2024-05-14T09:30:56.790238+0000","last_active":"2024-05-14T09:33:29.070997+0000","last_peered":"2024-05-14T09:33:29.070997+0000","last_clean":"2024-05-14T09:33:29.070997+0000","last_became_active":"2024-05-14T09:30:56.789826+0000","last_became_peered":"2024-05-14T09:30:56.789826+0000","last_unstale":"2024-05-14T09:33:29.070997+0000","last_undegraded":"2024-05-14T09:33:29.070997+0000","last_fullsized":"2024-05-14T09:33:29.070997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:19:19.853931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080306+0000","last_change":"2024-05-14T09:30:56.788021+0000","last_active":"2024-05-14T09:33:29.080306+0000","last_peered":"2024-05-14T09:33:29.080306+0000","last_clean":"2024-05-14T09:33:29.080306+0000","last_became_active":"2024-05-14T09:30:56.787657+0000","last_became_peered":"2024-05-14T09:30:56.787657+0000","last_unstale":"2024-05-14T09:33:29.080306+0000","last_undegraded":"2024-05-14T09:33:29.080306+0000","last_fullsized":"2024-05-14T09:33:29.080306+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:37:03.232137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.082547+0000","last_change":"2024-05-14T09:32:30.525175+0000","last_active":"2024-05-14T09:33:29.082547+0000","last_peered":"2024-05-14T09:33:29.082547+0000","last_clean":"2024-05-14T09:33:29.082547+0000","last_became_active":"2024-05-14T09:32:30.524861+0000","last_became_peered":"2024-05-14T09:32:30.524861+0000","last_unstale":"2024-05-14T09:33:29.082547+0000","last_undegraded":"2024-05-14T09:33:29.082547+0000","last_fullsized":"2024-05-14T09:33:29.082547+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:22:21.063843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.066050+0000","last_change":"2024-05-14T09:30:56.795647+0000","last_active":"2024-05-14T09:33:29.066050+0000","last_peered":"2024-05-14T09:33:29.066050+0000","last_clean":"2024-05-14T09:33:29.066050+0000","last_became_active":"2024-05-14T09:30:56.795186+0000","last_became_peered":"2024-05-14T09:30:56.795186+0000","last_unstale":"2024-05-14T09:33:29.066050+0000","last_undegraded":"2024-05-14T09:33:29.066050+0000","last_fullsized":"2024-05-14T09:33:29.066050+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:04:53.471305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081501+0000","last_change":"2024-05-14T09:30:56.796834+0000","last_active":"2024-05-14T09:33:29.081501+0000","last_peered":"2024-05-14T09:33:29.081501+0000","last_clean":"2024-05-14T09:33:29.081501+0000","last_became_active":"2024-05-14T09:30:56.796492+0000","last_became_peered":"2024-05-14T09:30:56.796492+0000","last_unstale":"2024-05-14T09:33:29.081501+0000","last_undegraded":"2024-05-14T09:33:29.081501+0000","last_fullsized":"2024-05-14T09:33:29.081501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:46.606129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.086070+0000","last_change":"2024-05-14T09:30:56.795020+0000","last_active":"2024-05-14T09:33:29.086070+0000","last_peered":"2024-05-14T09:33:29.086070+0000","last_clean":"2024-05-14T09:33:29.086070+0000","last_became_active":"2024-05-14T09:30:56.794625+0000","last_became_peered":"2024-05-14T09:30:56.794625+0000","last_unstale":"2024-05-14T09:33:29.086070+0000","last_undegraded":"2024-05-14T09:33:29.086070+0000","last_fullsized":"2024-05-14T09:33:29.086070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:09:38.597296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.082128+0000","last_change":"2024-05-14T09:30:56.796439+0000","last_active":"2024-05-14T09:33:29.082128+0000","last_peered":"2024-05-14T09:33:29.082128+0000","last_clean":"2024-05-14T09:33:29.082128+0000","last_became_active":"2024-05-14T09:30:56.795168+0000","last_became_peered":"2024-05-14T09:30:56.795168+0000","last_unstale":"2024-05-14T09:33:29.082128+0000","last_undegraded":"2024-05-14T09:33:29.082128+0000","last_fullsized":"2024-05-14T09:33:29.082128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:47.660286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087219+0000","last_change":"2024-05-14T09:30:56.795994+0000","last_active":"2024-05-14T09:33:29.087219+0000","last_peered":"2024-05-14T09:33:29.087219+0000","last_clean":"2024-05-14T09:33:29.087219+0000","last_became_active":"2024-05-14T09:30:56.795079+0000","last_became_peered":"2024-05-14T09:30:56.795079+0000","last_unstale":"2024-05-14T09:33:29.087219+0000","last_undegraded":"2024-05-14T09:33:29.087219+0000","last_fullsized":"2024-05-14T09:33:29.087219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:37:42.930195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.072741+0000","last_change":"2024-05-14T09:30:56.792997+0000","last_active":"2024-05-14T09:33:29.072741+0000","last_peered":"2024-05-14T09:33:29.072741+0000","last_clean":"2024-05-14T09:33:29.072741+0000","last_became_active":"2024-05-14T09:30:56.792302+0000","last_became_peered":"2024-05-14T09:30:56.792302+0000","last_unstale":"2024-05-14T09:33:29.072741+0000","last_undegraded":"2024-05-14T09:33:29.072741+0000","last_fullsized":"2024-05-14T09:33:29.072741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:52.986662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077933+0000","last_change":"2024-05-14T09:30:56.795903+0000","last_active":"2024-05-14T09:33:29.077933+0000","last_peered":"2024-05-14T09:33:29.077933+0000","last_clean":"2024-05-14T09:33:29.077933+0000","last_became_active":"2024-05-14T09:30:56.795466+0000","last_became_peered":"2024-05-14T09:30:56.795466+0000","last_unstale":"2024-05-14T09:33:29.077933+0000","last_undegraded":"2024-05-14T09:33:29.077933+0000","last_fullsized":"2024-05-14T09:33:29.077933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:02:10.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.070939+0000","last_change":"2024-05-14T09:30:56.795911+0000","last_active":"2024-05-14T09:33:29.070939+0000","last_peered":"2024-05-14T09:33:29.070939+0000","last_clean":"2024-05-14T09:33:29.070939+0000","last_became_active":"2024-05-14T09:30:56.795547+0000","last_became_peered":"2024-05-14T09:30:56.795547+0000","last_unstale":"2024-05-14T09:33:29.070939+0000","last_undegraded":"2024-05-14T09:33:29.070939+0000","last_fullsized":"2024-05-14T09:33:29.070939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:56:29.621783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.067755+0000","last_change":"2024-05-14T09:30:56.787745+0000","last_active":"2024-05-14T09:33:29.067755+0000","last_peered":"2024-05-14T09:33:29.067755+0000","last_clean":"2024-05-14T09:33:29.067755+0000","last_became_active":"2024-05-14T09:30:56.787385+0000","last_became_peered":"2024-05-14T09:30:56.787385+0000","last_unstale":"2024-05-14T09:33:29.067755+0000","last_undegraded":"2024-05-14T09:33:29.067755+0000","last_fullsized":"2024-05-14T09:33:29.067755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:11.256775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087830+0000","last_change":"2024-05-14T09:30:56.795785+0000","last_active":"2024-05-14T09:33:29.087830+0000","last_peered":"2024-05-14T09:33:29.087830+0000","last_clean":"2024-05-14T09:33:29.087830+0000","last_became_active":"2024-05-14T09:30:56.795436+0000","last_became_peered":"2024-05-14T09:30:56.795436+0000","last_unstale":"2024-05-14T09:33:29.087830+0000","last_undegraded":"2024-05-14T09:33:29.087830+0000","last_fullsized":"2024-05-14T09:33:29.087830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:12:09.221929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087321+0000","last_change":"2024-05-14T09:30:56.800688+0000","last_active":"2024-05-14T09:33:29.087321+0000","last_peered":"2024-05-14T09:33:29.087321+0000","last_clean":"2024-05-14T09:33:29.087321+0000","last_became_active":"2024-05-14T09:30:56.800204+0000","last_became_peered":"2024-05-14T09:30:56.800204+0000","last_unstale":"2024-05-14T09:33:29.087321+0000","last_undegraded":"2024-05-14T09:33:29.087321+0000","last_fullsized":"2024-05-14T09:33:29.087321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:32:27.057688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.075054+0000","last_change":"2024-05-14T09:30:56.795604+0000","last_active":"2024-05-14T09:33:29.075054+0000","last_peered":"2024-05-14T09:33:29.075054+0000","last_clean":"2024-05-14T09:33:29.075054+0000","last_became_active":"2024-05-14T09:30:56.795245+0000","last_became_peered":"2024-05-14T09:30:56.795245+0000","last_unstale":"2024-05-14T09:33:29.075054+0000","last_undegraded":"2024-05-14T09:33:29.075054+0000","last_fullsized":"2024-05-14T09:33:29.075054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:14.072584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080216+0000","last_change":"2024-05-14T09:30:56.794815+0000","last_active":"2024-05-14T09:33:29.080216+0000","last_peered":"2024-05-14T09:33:29.080216+0000","last_clean":"2024-05-14T09:33:29.080216+0000","last_became_active":"2024-05-14T09:30:56.794429+0000","last_became_peered":"2024-05-14T09:30:56.794429+0000","last_unstale":"2024-05-14T09:33:29.080216+0000","last_undegraded":"2024-05-14T09:33:29.080216+0000","last_fullsized":"2024-05-14T09:33:29.080216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:32:08.367046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319506+0000","last_change":"2024-05-14T09:31:08.107895+0000","last_active":"2024-05-14T09:32:30.319506+0000","last_peered":"2024-05-14T09:32:30.319506+0000","last_clean":"2024-05-14T09:32:30.319506+0000","last_became_active":"2024-05-14T09:31:08.107458+0000","last_became_peered":"2024-05-14T09:31:08.107458+0000","last_unstale":"2024-05-14T09:32:30.319506+0000","last_undegraded":"2024-05-14T09:32:30.319506+0000","last_fullsized":"2024-05-14T09:32:30.319506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:13:11.717753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":69,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319382+0000","last_change":"2024-05-14T09:31:30.178197+0000","last_active":"2024-05-14T09:32:30.319382+0000","last_peered":"2024-05-14T09:32:30.319382+0000","last_clean":"2024-05-14T09:32:30.319382+0000","last_became_active":"2024-05-14T09:31:30.171154+0000","last_became_peered":"2024-05-14T09:31:30.171154+0000","last_unstale":"2024-05-14T09:32:30.319382+0000","last_undegraded":"2024-05-14T09:32:30.319382+0000","last_fullsized":"2024-05-14T09:32:30.319382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:43:46.208439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077181+0000","last_change":"2024-05-14T09:30:56.800526+0000","last_active":"2024-05-14T09:33:29.077181+0000","last_peered":"2024-05-14T09:33:29.077181+0000","last_clean":"2024-05-14T09:33:29.077181+0000","last_became_active":"2024-05-14T09:30:56.797692+0000","last_became_peered":"2024-05-14T09:30:56.797692+0000","last_unstale":"2024-05-14T09:33:29.077181+0000","last_undegraded":"2024-05-14T09:33:29.077181+0000","last_fullsized":"2024-05-14T09:33:29.077181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:09:14.471173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522407+0000","last_change":"2024-05-14T09:31:30.173810+0000","last_active":"2024-05-14T09:32:30.522407+0000","last_peered":"2024-05-14T09:32:30.522407+0000","last_clean":"2024-05-14T09:32:30.522407+0000","last_became_active":"2024-05-14T09:31:30.173242+0000","last_became_peered":"2024-05-14T09:31:30.173242+0000","last_unstale":"2024-05-14T09:32:30.522407+0000","last_undegraded":"2024-05-14T09:32:30.522407+0000","last_fullsized":"2024-05-14T09:32:30.522407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:37:15.991805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.322883+0000","last_change":"2024-05-14T09:30:54.015636+0000","last_active":"2024-05-14T09:32:30.322883+0000","last_peered":"2024-05-14T09:32:30.322883+0000","last_clean":"2024-05-14T09:32:30.322883+0000","last_became_active":"2024-05-14T09:30:54.015290+0000","last_became_peered":"2024-05-14T09:30:54.015290+0000","last_unstale":"2024-05-14T09:32:30.322883+0000","last_undegraded":"2024-05-14T09:32:30.322883+0000","last_fullsized":"2024-05-14T09:32:30.322883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:29:17.548817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.088634+0000","last_change":"2024-05-14T09:30:56.799506+0000","last_active":"2024-05-14T09:33:29.088634+0000","last_peered":"2024-05-14T09:33:29.088634+0000","last_clean":"2024-05-14T09:33:29.088634+0000","last_became_active":"2024-05-14T09:30:56.799153+0000","last_became_peered":"2024-05-14T09:30:56.799153+0000","last_unstale":"2024-05-14T09:33:29.088634+0000","last_undegraded":"2024-05-14T09:33:29.088634+0000","last_fullsized":"2024-05-14T09:33:29.088634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:04:08.217337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522612+0000","last_change":"2024-05-14T09:31:08.115533+0000","last_active":"2024-05-14T09:32:30.522612+0000","last_peered":"2024-05-14T09:32:30.522612+0000","last_clean":"2024-05-14T09:32:30.522612+0000","last_became_active":"2024-05-14T09:31:08.115033+0000","last_became_peered":"2024-05-14T09:31:08.115033+0000","last_unstale":"2024-05-14T09:32:30.522612+0000","last_undegraded":"2024-05-14T09:32:30.522612+0000","last_fullsized":"2024-05-14T09:32:30.522612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:15.668357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524484+0000","last_change":"2024-05-14T09:30:53.770547+0000","last_active":"2024-05-14T09:32:30.524484+0000","last_peered":"2024-05-14T09:32:30.524484+0000","last_clean":"2024-05-14T09:32:30.524484+0000","last_became_active":"2024-05-14T09:30:53.770110+0000","last_became_peered":"2024-05-14T09:30:53.770110+0000","last_unstale":"2024-05-14T09:32:30.524484+0000","last_undegraded":"2024-05-14T09:32:30.524484+0000","last_fullsized":"2024-05-14T09:32:30.524484+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:18.371318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084457+0000","last_change":"2024-05-14T09:30:56.799427+0000","last_active":"2024-05-14T09:33:29.084457+0000","last_peered":"2024-05-14T09:33:29.084457+0000","last_clean":"2024-05-14T09:33:29.084457+0000","last_became_active":"2024-05-14T09:30:56.799018+0000","last_became_peered":"2024-05-14T09:30:56.799018+0000","last_unstale":"2024-05-14T09:33:29.084457+0000","last_undegraded":"2024-05-14T09:33:29.084457+0000","last_fullsized":"2024-05-14T09:33:29.084457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:13:31.837985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522470+0000","last_change":"2024-05-14T09:31:08.116258+0000","last_active":"2024-05-14T09:32:30.522470+0000","last_peered":"2024-05-14T09:32:30.522470+0000","last_clean":"2024-05-14T09:32:30.522470+0000","last_became_active":"2024-05-14T09:31:08.115040+0000","last_became_peered":"2024-05-14T09:31:08.115040+0000","last_unstale":"2024-05-14T09:32:30.522470+0000","last_undegraded":"2024-05-14T09:32:30.522470+0000","last_fullsized":"2024-05-14T09:32:30.522470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:57:54.839203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523793+0000","last_change":"2024-05-14T09:31:30.188984+0000","last_active":"2024-05-14T09:32:30.523793+0000","last_peered":"2024-05-14T09:32:30.523793+0000","last_clean":"2024-05-14T09:32:30.523793+0000","last_became_active":"2024-05-14T09:31:30.174410+0000","last_became_peered":"2024-05-14T09:31:30.174410+0000","last_unstale":"2024-05-14T09:32:30.523793+0000","last_undegraded":"2024-05-14T09:32:30.523793+0000","last_fullsized":"2024-05-14T09:32:30.523793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:01:35.713857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,4],"acting":[8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081749+0000","last_change":"2024-05-14T09:32:30.319390+0000","last_active":"2024-05-14T09:33:29.081749+0000","last_peered":"2024-05-14T09:33:29.081749+0000","last_clean":"2024-05-14T09:33:29.081749+0000","last_became_active":"2024-05-14T09:32:30.318645+0000","last_became_peered":"2024-05-14T09:32:30.318645+0000","last_unstale":"2024-05-14T09:33:29.081749+0000","last_undegraded":"2024-05-14T09:33:29.081749+0000","last_fullsized":"2024-05-14T09:33:29.081749+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:37:49.866996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320617+0000","last_change":"2024-05-14T09:31:08.115740+0000","last_active":"2024-05-14T09:32:30.320617+0000","last_peered":"2024-05-14T09:32:30.320617+0000","last_clean":"2024-05-14T09:32:30.320617+0000","last_became_active":"2024-05-14T09:31:08.115202+0000","last_became_peered":"2024-05-14T09:31:08.115202+0000","last_unstale":"2024-05-14T09:32:30.320617+0000","last_undegraded":"2024-05-14T09:32:30.320617+0000","last_fullsized":"2024-05-14T09:32:30.320617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:44.519280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":156,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:36.394647+0000","last_change":"2024-05-14T09:30:53.770869+0000","last_active":"2024-05-14T09:33:36.394647+0000","last_peered":"2024-05-14T09:33:36.394647+0000","last_clean":"2024-05-14T09:33:36.394647+0000","last_became_active":"2024-05-14T09:30:53.770425+0000","last_became_peered":"2024-05-14T09:30:53.770425+0000","last_unstale":"2024-05-14T09:33:36.394647+0000","last_undegraded":"2024-05-14T09:33:36.394647+0000","last_fullsized":"2024-05-14T09:33:36.394647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:03:33.768120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.059353+0000","last_change":"2024-05-14T09:30:56.789267+0000","last_active":"2024-05-14T09:33:29.059353+0000","last_peered":"2024-05-14T09:33:29.059353+0000","last_clean":"2024-05-14T09:33:29.059353+0000","last_became_active":"2024-05-14T09:30:56.788883+0000","last_became_peered":"2024-05-14T09:30:56.788883+0000","last_unstale":"2024-05-14T09:33:29.059353+0000","last_undegraded":"2024-05-14T09:33:29.059353+0000","last_fullsized":"2024-05-14T09:33:29.059353+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:09.818693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320509+0000","last_change":"2024-05-14T09:31:08.115193+0000","last_active":"2024-05-14T09:32:30.320509+0000","last_peered":"2024-05-14T09:32:30.320509+0000","last_clean":"2024-05-14T09:32:30.320509+0000","last_became_active":"2024-05-14T09:31:08.114701+0000","last_became_peered":"2024-05-14T09:31:08.114701+0000","last_unstale":"2024-05-14T09:32:30.320509+0000","last_undegraded":"2024-05-14T09:32:30.320509+0000","last_fullsized":"2024-05-14T09:32:30.320509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:47:48.487922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319686+0000","last_change":"2024-05-14T09:31:30.174760+0000","last_active":"2024-05-14T09:32:30.319686+0000","last_peered":"2024-05-14T09:32:30.319686+0000","last_clean":"2024-05-14T09:32:30.319686+0000","last_became_active":"2024-05-14T09:31:30.174253+0000","last_became_peered":"2024-05-14T09:31:30.174253+0000","last_unstale":"2024-05-14T09:32:30.319686+0000","last_undegraded":"2024-05-14T09:32:30.319686+0000","last_fullsized":"2024-05-14T09:32:30.319686+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:11:17.163436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081148+0000","last_change":"2024-05-14T09:30:56.794654+0000","last_active":"2024-05-14T09:33:29.081148+0000","last_peered":"2024-05-14T09:33:29.081148+0000","last_clean":"2024-05-14T09:33:29.081148+0000","last_became_active":"2024-05-14T09:30:56.794282+0000","last_became_peered":"2024-05-14T09:30:56.794282+0000","last_unstale":"2024-05-14T09:33:29.081148+0000","last_undegraded":"2024-05-14T09:33:29.081148+0000","last_fullsized":"2024-05-14T09:33:29.081148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:55:41.478707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320586+0000","last_change":"2024-05-14T09:31:08.119965+0000","last_active":"2024-05-14T09:32:30.320586+0000","last_peered":"2024-05-14T09:32:30.320586+0000","last_clean":"2024-05-14T09:32:30.320586+0000","last_became_active":"2024-05-14T09:31:08.119502+0000","last_became_peered":"2024-05-14T09:31:08.119502+0000","last_unstale":"2024-05-14T09:32:30.320586+0000","last_undegraded":"2024-05-14T09:32:30.320586+0000","last_fullsized":"2024-05-14T09:32:30.320586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:01:23.612036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.314804+0000","last_change":"2024-05-14T09:30:53.766495+0000","last_active":"2024-05-14T09:32:30.314804+0000","last_peered":"2024-05-14T09:32:30.314804+0000","last_clean":"2024-05-14T09:32:30.314804+0000","last_became_active":"2024-05-14T09:30:53.766122+0000","last_became_peered":"2024-05-14T09:30:53.766122+0000","last_unstale":"2024-05-14T09:32:30.314804+0000","last_undegraded":"2024-05-14T09:32:30.314804+0000","last_fullsized":"2024-05-14T09:32:30.314804+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:43:13.344209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.073417+0000","last_change":"2024-05-14T09:30:56.791938+0000","last_active":"2024-05-14T09:33:29.073417+0000","last_peered":"2024-05-14T09:33:29.073417+0000","last_clean":"2024-05-14T09:33:29.073417+0000","last_became_active":"2024-05-14T09:30:56.791525+0000","last_became_peered":"2024-05-14T09:30:56.791525+0000","last_unstale":"2024-05-14T09:33:29.073417+0000","last_undegraded":"2024-05-14T09:33:29.073417+0000","last_fullsized":"2024-05-14T09:33:29.073417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:18.178208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320104+0000","last_change":"2024-05-14T09:31:08.113548+0000","last_active":"2024-05-14T09:32:30.320104+0000","last_peered":"2024-05-14T09:32:30.320104+0000","last_clean":"2024-05-14T09:32:30.320104+0000","last_became_active":"2024-05-14T09:31:08.113117+0000","last_became_peered":"2024-05-14T09:31:08.113117+0000","last_unstale":"2024-05-14T09:32:30.320104+0000","last_undegraded":"2024-05-14T09:32:30.320104+0000","last_fullsized":"2024-05-14T09:32:30.320104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:25:15.483761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315966+0000","last_change":"2024-05-14T09:30:53.770987+0000","last_active":"2024-05-14T09:32:30.315966+0000","last_peered":"2024-05-14T09:32:30.315966+0000","last_clean":"2024-05-14T09:32:30.315966+0000","last_became_active":"2024-05-14T09:30:53.770609+0000","last_became_peered":"2024-05-14T09:30:53.770609+0000","last_unstale":"2024-05-14T09:32:30.315966+0000","last_undegraded":"2024-05-14T09:32:30.315966+0000","last_fullsized":"2024-05-14T09:32:30.315966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:26:22.020642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081149+0000","last_change":"2024-05-14T09:30:56.789219+0000","last_active":"2024-05-14T09:33:29.081149+0000","last_peered":"2024-05-14T09:33:29.081149+0000","last_clean":"2024-05-14T09:33:29.081149+0000","last_became_active":"2024-05-14T09:30:56.788871+0000","last_became_peered":"2024-05-14T09:30:56.788871+0000","last_unstale":"2024-05-14T09:33:29.081149+0000","last_undegraded":"2024-05-14T09:33:29.081149+0000","last_fullsized":"2024-05-14T09:33:29.081149+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:10:50.774267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.322531+0000","last_change":"2024-05-14T09:31:30.174708+0000","last_active":"2024-05-14T09:32:30.322531+0000","last_peered":"2024-05-14T09:32:30.322531+0000","last_clean":"2024-05-14T09:32:30.322531+0000","last_became_active":"2024-05-14T09:31:30.174199+0000","last_became_peered":"2024-05-14T09:31:30.174199+0000","last_unstale":"2024-05-14T09:32:30.322531+0000","last_undegraded":"2024-05-14T09:32:30.322531+0000","last_fullsized":"2024-05-14T09:32:30.322531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:23:55.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":149,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:36.394034+0000","last_change":"2024-05-14T09:30:53.770898+0000","last_active":"2024-05-14T09:33:36.394034+0000","last_peered":"2024-05-14T09:33:36.394034+0000","last_clean":"2024-05-14T09:33:36.394034+0000","last_became_active":"2024-05-14T09:30:53.770483+0000","last_became_peered":"2024-05-14T09:30:53.770483+0000","last_unstale":"2024-05-14T09:33:36.394034+0000","last_undegraded":"2024-05-14T09:33:36.394034+0000","last_fullsized":"2024-05-14T09:33:36.394034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:07.764348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084404+0000","last_change":"2024-05-14T09:30:56.799049+0000","last_active":"2024-05-14T09:33:29.084404+0000","last_peered":"2024-05-14T09:33:29.084404+0000","last_clean":"2024-05-14T09:33:29.084404+0000","last_became_active":"2024-05-14T09:30:56.798643+0000","last_became_peered":"2024-05-14T09:30:56.798643+0000","last_unstale":"2024-05-14T09:33:29.084404+0000","last_undegraded":"2024-05-14T09:33:29.084404+0000","last_fullsized":"2024-05-14T09:33:29.084404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:12.378373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.318070+0000","last_change":"2024-05-14T09:31:08.109876+0000","last_active":"2024-05-14T09:32:30.318070+0000","last_peered":"2024-05-14T09:32:30.318070+0000","last_clean":"2024-05-14T09:32:30.318070+0000","last_became_active":"2024-05-14T09:31:08.109470+0000","last_became_peered":"2024-05-14T09:31:08.109470+0000","last_unstale":"2024-05-14T09:32:30.318070+0000","last_undegraded":"2024-05-14T09:32:30.318070+0000","last_fullsized":"2024-05-14T09:32:30.318070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:07.704153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.015003+0000","last_change":"2024-05-14T09:31:30.179086+0000","last_active":"2024-05-14T09:32:31.015003+0000","last_peered":"2024-05-14T09:32:31.015003+0000","last_clean":"2024-05-14T09:32:31.015003+0000","last_became_active":"2024-05-14T09:31:30.178604+0000","last_became_peered":"2024-05-14T09:31:30.178604+0000","last_unstale":"2024-05-14T09:32:31.015003+0000","last_undegraded":"2024-05-14T09:32:31.015003+0000","last_fullsized":"2024-05-14T09:32:31.015003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:39.712395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.073315+0000","last_change":"2024-05-14T09:30:56.793109+0000","last_active":"2024-05-14T09:33:29.073315+0000","last_peered":"2024-05-14T09:33:29.073315+0000","last_clean":"2024-05-14T09:33:29.073315+0000","last_became_active":"2024-05-14T09:30:56.792761+0000","last_became_peered":"2024-05-14T09:30:56.792761+0000","last_unstale":"2024-05-14T09:33:29.073315+0000","last_undegraded":"2024-05-14T09:33:29.073315+0000","last_fullsized":"2024-05-14T09:33:29.073315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:25:33.216281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522916+0000","last_change":"2024-05-14T09:31:08.120978+0000","last_active":"2024-05-14T09:32:30.522916+0000","last_peered":"2024-05-14T09:32:30.522916+0000","last_clean":"2024-05-14T09:32:30.522916+0000","last_became_active":"2024-05-14T09:31:08.120227+0000","last_became_peered":"2024-05-14T09:31:08.120227+0000","last_unstale":"2024-05-14T09:32:30.522916+0000","last_undegraded":"2024-05-14T09:32:30.522916+0000","last_fullsized":"2024-05-14T09:32:30.522916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:06.630390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316348+0000","last_change":"2024-05-14T09:30:54.017382+0000","last_active":"2024-05-14T09:32:30.316348+0000","last_peered":"2024-05-14T09:32:30.316348+0000","last_clean":"2024-05-14T09:32:30.316348+0000","last_became_active":"2024-05-14T09:30:54.016887+0000","last_became_peered":"2024-05-14T09:30:54.016887+0000","last_unstale":"2024-05-14T09:32:30.316348+0000","last_undegraded":"2024-05-14T09:32:30.316348+0000","last_fullsized":"2024-05-14T09:32:30.316348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:28.823659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.085392+0000","last_change":"2024-05-14T09:30:56.800626+0000","last_active":"2024-05-14T09:33:29.085392+0000","last_peered":"2024-05-14T09:33:29.085392+0000","last_clean":"2024-05-14T09:33:29.085392+0000","last_became_active":"2024-05-14T09:30:56.800258+0000","last_became_peered":"2024-05-14T09:30:56.800258+0000","last_unstale":"2024-05-14T09:33:29.085392+0000","last_undegraded":"2024-05-14T09:33:29.085392+0000","last_fullsized":"2024-05-14T09:33:29.085392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:47:26.369633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.318135+0000","last_change":"2024-05-14T09:31:08.115456+0000","last_active":"2024-05-14T09:32:30.318135+0000","last_peered":"2024-05-14T09:32:30.318135+0000","last_clean":"2024-05-14T09:32:30.318135+0000","last_became_active":"2024-05-14T09:31:08.114705+0000","last_became_peered":"2024-05-14T09:31:08.114705+0000","last_unstale":"2024-05-14T09:32:30.318135+0000","last_undegraded":"2024-05-14T09:32:30.318135+0000","last_fullsized":"2024-05-14T09:32:30.318135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:50.096747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":308,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:37.395294+0000","last_change":"2024-05-14T09:31:30.186202+0000","last_active":"2024-05-14T09:33:37.395294+0000","last_peered":"2024-05-14T09:33:37.395294+0000","last_clean":"2024-05-14T09:33:37.395294+0000","last_became_active":"2024-05-14T09:31:30.174248+0000","last_became_peered":"2024-05-14T09:31:30.174248+0000","last_unstale":"2024-05-14T09:33:37.395294+0000","last_undegraded":"2024-05-14T09:33:37.395294+0000","last_fullsized":"2024-05-14T09:33:37.395294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:45:03.131934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64309,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":210,"num_read_kb":168,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077083+0000","last_change":"2024-05-14T09:30:56.791099+0000","last_active":"2024-05-14T09:33:29.077083+0000","last_peered":"2024-05-14T09:33:29.077083+0000","last_clean":"2024-05-14T09:33:29.077083+0000","last_became_active":"2024-05-14T09:30:56.790676+0000","last_became_peered":"2024-05-14T09:30:56.790676+0000","last_unstale":"2024-05-14T09:33:29.077083+0000","last_undegraded":"2024-05-14T09:33:29.077083+0000","last_fullsized":"2024-05-14T09:33:29.077083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:09.851357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316986+0000","last_change":"2024-05-14T09:31:08.120289+0000","last_active":"2024-05-14T09:32:30.316986+0000","last_peered":"2024-05-14T09:32:30.316986+0000","last_clean":"2024-05-14T09:32:30.316986+0000","last_became_active":"2024-05-14T09:31:08.118574+0000","last_became_peered":"2024-05-14T09:31:08.118574+0000","last_unstale":"2024-05-14T09:32:30.316986+0000","last_undegraded":"2024-05-14T09:32:30.316986+0000","last_fullsized":"2024-05-14T09:32:30.316986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:01.723442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319990+0000","last_change":"2024-05-14T09:30:53.766578+0000","last_active":"2024-05-14T09:32:30.319990+0000","last_peered":"2024-05-14T09:32:30.319990+0000","last_clean":"2024-05-14T09:32:30.319990+0000","last_became_active":"2024-05-14T09:30:53.765928+0000","last_became_peered":"2024-05-14T09:30:53.765928+0000","last_unstale":"2024-05-14T09:32:30.319990+0000","last_undegraded":"2024-05-14T09:32:30.319990+0000","last_fullsized":"2024-05-14T09:32:30.319990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:19:08.690013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.083536+0000","last_change":"2024-05-14T09:30:56.785507+0000","last_active":"2024-05-14T09:33:29.083536+0000","last_peered":"2024-05-14T09:33:29.083536+0000","last_clean":"2024-05-14T09:33:29.083536+0000","last_became_active":"2024-05-14T09:30:56.784800+0000","last_became_peered":"2024-05-14T09:30:56.784800+0000","last_unstale":"2024-05-14T09:33:29.083536+0000","last_undegraded":"2024-05-14T09:33:29.083536+0000","last_fullsized":"2024-05-14T09:33:29.083536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:36:05.440222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.321298+0000","last_change":"2024-05-14T09:31:08.119051+0000","last_active":"2024-05-14T09:32:30.321298+0000","last_peered":"2024-05-14T09:32:30.321298+0000","last_clean":"2024-05-14T09:32:30.321298+0000","last_became_active":"2024-05-14T09:31:08.118569+0000","last_became_peered":"2024-05-14T09:31:08.118569+0000","last_unstale":"2024-05-14T09:32:30.321298+0000","last_undegraded":"2024-05-14T09:32:30.321298+0000","last_fullsized":"2024-05-14T09:32:30.321298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:12:39.643740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316079+0000","last_change":"2024-05-14T09:30:53.772832+0000","last_active":"2024-05-14T09:32:30.316079+0000","last_peered":"2024-05-14T09:32:30.316079+0000","last_clean":"2024-05-14T09:32:30.316079+0000","last_became_active":"2024-05-14T09:30:53.772228+0000","last_became_peered":"2024-05-14T09:30:53.772228+0000","last_unstale":"2024-05-14T09:32:30.316079+0000","last_undegraded":"2024-05-14T09:32:30.316079+0000","last_fullsized":"2024-05-14T09:32:30.316079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:14:43.350330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084806+0000","last_change":"2024-05-14T09:30:56.782831+0000","last_active":"2024-05-14T09:33:29.084806+0000","last_peered":"2024-05-14T09:33:29.084806+0000","last_clean":"2024-05-14T09:33:29.084806+0000","last_became_active":"2024-05-14T09:30:56.782402+0000","last_became_peered":"2024-05-14T09:30:56.782402+0000","last_unstale":"2024-05-14T09:33:29.084806+0000","last_undegraded":"2024-05-14T09:33:29.084806+0000","last_fullsized":"2024-05-14T09:33:29.084806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:54.603524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523969+0000","last_change":"2024-05-14T09:31:30.296957+0000","last_active":"2024-05-14T09:32:30.523969+0000","last_peered":"2024-05-14T09:32:30.523969+0000","last_clean":"2024-05-14T09:32:30.523969+0000","last_became_active":"2024-05-14T09:31:30.296554+0000","last_became_peered":"2024-05-14T09:31:30.296554+0000","last_unstale":"2024-05-14T09:32:30.523969+0000","last_undegraded":"2024-05-14T09:32:30.523969+0000","last_fullsized":"2024-05-14T09:32:30.523969+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:28.963042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":149,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:35.392945+0000","last_change":"2024-05-14T09:30:53.772731+0000","last_active":"2024-05-14T09:33:35.392945+0000","last_peered":"2024-05-14T09:33:35.392945+0000","last_clean":"2024-05-14T09:33:35.392945+0000","last_became_active":"2024-05-14T09:30:53.771986+0000","last_became_peered":"2024-05-14T09:30:53.771986+0000","last_unstale":"2024-05-14T09:33:35.392945+0000","last_undegraded":"2024-05-14T09:33:35.392945+0000","last_fullsized":"2024-05-14T09:33:35.392945+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:50:00.810937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077545+0000","last_change":"2024-05-14T09:30:56.790503+0000","last_active":"2024-05-14T09:33:29.077545+0000","last_peered":"2024-05-14T09:33:29.077545+0000","last_clean":"2024-05-14T09:33:29.077545+0000","last_became_active":"2024-05-14T09:30:56.790103+0000","last_became_peered":"2024-05-14T09:30:56.790103+0000","last_unstale":"2024-05-14T09:33:29.077545+0000","last_undegraded":"2024-05-14T09:33:29.077545+0000","last_fullsized":"2024-05-14T09:33:29.077545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:00.950794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315462+0000","last_change":"2024-05-14T09:31:08.116475+0000","last_active":"2024-05-14T09:32:30.315462+0000","last_peered":"2024-05-14T09:32:30.315462+0000","last_clean":"2024-05-14T09:32:30.315462+0000","last_became_active":"2024-05-14T09:31:08.116061+0000","last_became_peered":"2024-05-14T09:31:08.116061+0000","last_unstale":"2024-05-14T09:32:30.315462+0000","last_undegraded":"2024-05-14T09:32:30.315462+0000","last_fullsized":"2024-05-14T09:32:30.315462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:07:02.281853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523311+0000","last_change":"2024-05-14T09:30:54.016570+0000","last_active":"2024-05-14T09:32:30.523311+0000","last_peered":"2024-05-14T09:32:30.523311+0000","last_clean":"2024-05-14T09:32:30.523311+0000","last_became_active":"2024-05-14T09:30:54.015987+0000","last_became_peered":"2024-05-14T09:30:54.015987+0000","last_unstale":"2024-05-14T09:32:30.523311+0000","last_undegraded":"2024-05-14T09:32:30.523311+0000","last_fullsized":"2024-05-14T09:32:30.523311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:18:28.694488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081072+0000","last_change":"2024-05-14T09:30:56.800727+0000","last_active":"2024-05-14T09:33:29.081072+0000","last_peered":"2024-05-14T09:33:29.081072+0000","last_clean":"2024-05-14T09:33:29.081072+0000","last_became_active":"2024-05-14T09:30:56.800354+0000","last_became_peered":"2024-05-14T09:30:56.800354+0000","last_unstale":"2024-05-14T09:33:29.081072+0000","last_undegraded":"2024-05-14T09:33:29.081072+0000","last_fullsized":"2024-05-14T09:33:29.081072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:13:07.227577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315777+0000","last_change":"2024-05-14T09:31:08.115685+0000","last_active":"2024-05-14T09:32:30.315777+0000","last_peered":"2024-05-14T09:32:30.315777+0000","last_clean":"2024-05-14T09:32:30.315777+0000","last_became_active":"2024-05-14T09:31:08.114948+0000","last_became_peered":"2024-05-14T09:31:08.114948+0000","last_unstale":"2024-05-14T09:32:30.315777+0000","last_undegraded":"2024-05-14T09:32:30.315777+0000","last_fullsized":"2024-05-14T09:32:30.315777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:33:07.286823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524227+0000","last_change":"2024-05-14T09:30:53.769949+0000","last_active":"2024-05-14T09:32:30.524227+0000","last_peered":"2024-05-14T09:32:30.524227+0000","last_clean":"2024-05-14T09:32:30.524227+0000","last_became_active":"2024-05-14T09:30:53.769327+0000","last_became_peered":"2024-05-14T09:30:53.769327+0000","last_unstale":"2024-05-14T09:32:30.524227+0000","last_undegraded":"2024-05-14T09:32:30.524227+0000","last_fullsized":"2024-05-14T09:32:30.524227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:35:11.850991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077817+0000","last_change":"2024-05-14T09:32:31.015994+0000","last_active":"2024-05-14T09:33:29.077817+0000","last_peered":"2024-05-14T09:33:29.077817+0000","last_clean":"2024-05-14T09:33:29.077817+0000","last_became_active":"2024-05-14T09:32:31.015515+0000","last_became_peered":"2024-05-14T09:32:31.015515+0000","last_unstale":"2024-05-14T09:33:29.077817+0000","last_undegraded":"2024-05-14T09:33:29.077817+0000","last_fullsized":"2024-05-14T09:33:29.077817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:21:49.811521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7],"acting":[11,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524005+0000","last_change":"2024-05-14T09:31:08.387316+0000","last_active":"2024-05-14T09:32:30.524005+0000","last_peered":"2024-05-14T09:32:30.524005+0000","last_clean":"2024-05-14T09:32:30.524005+0000","last_became_active":"2024-05-14T09:31:08.386739+0000","last_became_peered":"2024-05-14T09:31:08.386739+0000","last_unstale":"2024-05-14T09:32:30.524005+0000","last_undegraded":"2024-05-14T09:32:30.524005+0000","last_fullsized":"2024-05-14T09:32:30.524005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:17.870255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.318704+0000","last_change":"2024-05-14T09:30:54.017756+0000","last_active":"2024-05-14T09:32:30.318704+0000","last_peered":"2024-05-14T09:32:30.318704+0000","last_clean":"2024-05-14T09:32:30.318704+0000","last_became_active":"2024-05-14T09:30:54.017277+0000","last_became_peered":"2024-05-14T09:30:54.017277+0000","last_unstale":"2024-05-14T09:32:30.318704+0000","last_undegraded":"2024-05-14T09:32:30.318704+0000","last_fullsized":"2024-05-14T09:32:30.318704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:27:36.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.072775+0000","last_change":"2024-05-14T09:30:56.788408+0000","last_active":"2024-05-14T09:33:29.072775+0000","last_peered":"2024-05-14T09:33:29.072775+0000","last_clean":"2024-05-14T09:33:29.072775+0000","last_became_active":"2024-05-14T09:30:56.787955+0000","last_became_peered":"2024-05-14T09:30:56.787955+0000","last_unstale":"2024-05-14T09:33:29.072775+0000","last_undegraded":"2024-05-14T09:33:29.072775+0000","last_fullsized":"2024-05-14T09:33:29.072775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:09.777069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315137+0000","last_change":"2024-05-14T09:31:08.119078+0000","last_active":"2024-05-14T09:32:30.315137+0000","last_peered":"2024-05-14T09:32:30.315137+0000","last_clean":"2024-05-14T09:32:30.315137+0000","last_became_active":"2024-05-14T09:31:08.117939+0000","last_became_peered":"2024-05-14T09:31:08.117939+0000","last_unstale":"2024-05-14T09:32:30.315137+0000","last_undegraded":"2024-05-14T09:32:30.315137+0000","last_fullsized":"2024-05-14T09:32:30.315137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:48:37.363503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014384+0000","last_change":"2024-05-14T09:30:53.767843+0000","last_active":"2024-05-14T09:32:31.014384+0000","last_peered":"2024-05-14T09:32:31.014384+0000","last_clean":"2024-05-14T09:32:31.014384+0000","last_became_active":"2024-05-14T09:30:53.767419+0000","last_became_peered":"2024-05-14T09:30:53.767419+0000","last_unstale":"2024-05-14T09:32:31.014384+0000","last_undegraded":"2024-05-14T09:32:31.014384+0000","last_fullsized":"2024-05-14T09:32:31.014384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:49:06.441600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077073+0000","last_change":"2024-05-14T09:30:56.796001+0000","last_active":"2024-05-14T09:33:29.077073+0000","last_peered":"2024-05-14T09:33:29.077073+0000","last_clean":"2024-05-14T09:33:29.077073+0000","last_became_active":"2024-05-14T09:30:56.794517+0000","last_became_peered":"2024-05-14T09:30:56.794517+0000","last_unstale":"2024-05-14T09:33:29.077073+0000","last_undegraded":"2024-05-14T09:33:29.077073+0000","last_fullsized":"2024-05-14T09:33:29.077073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:20:37.514679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315309+0000","last_change":"2024-05-14T09:31:08.387570+0000","last_active":"2024-05-14T09:32:30.315309+0000","last_peered":"2024-05-14T09:32:30.315309+0000","last_clean":"2024-05-14T09:32:30.315309+0000","last_became_active":"2024-05-14T09:31:08.386745+0000","last_became_peered":"2024-05-14T09:31:08.386745+0000","last_unstale":"2024-05-14T09:32:30.315309+0000","last_undegraded":"2024-05-14T09:32:30.315309+0000","last_fullsized":"2024-05-14T09:32:30.315309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:44:01.024784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.060037+0000","last_change":"2024-05-14T09:30:56.795532+0000","last_active":"2024-05-14T09:33:29.060037+0000","last_peered":"2024-05-14T09:33:29.060037+0000","last_clean":"2024-05-14T09:33:29.060037+0000","last_became_active":"2024-05-14T09:30:56.795171+0000","last_became_peered":"2024-05-14T09:30:56.795171+0000","last_unstale":"2024-05-14T09:33:29.060037+0000","last_undegraded":"2024-05-14T09:33:29.060037+0000","last_fullsized":"2024-05-14T09:33:29.060037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:09:40.252489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":149,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:36.393909+0000","last_change":"2024-05-14T09:30:54.015629+0000","last_active":"2024-05-14T09:33:36.393909+0000","last_peered":"2024-05-14T09:33:36.393909+0000","last_clean":"2024-05-14T09:33:36.393909+0000","last_became_active":"2024-05-14T09:30:54.015288+0000","last_became_peered":"2024-05-14T09:30:54.015288+0000","last_unstale":"2024-05-14T09:33:36.393909+0000","last_undegraded":"2024-05-14T09:33:36.393909+0000","last_fullsized":"2024-05-14T09:33:36.393909+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:50.316125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014777+0000","last_change":"2024-05-14T09:31:30.179056+0000","last_active":"2024-05-14T09:32:31.014777+0000","last_peered":"2024-05-14T09:32:31.014777+0000","last_clean":"2024-05-14T09:32:31.014777+0000","last_became_active":"2024-05-14T09:31:30.178398+0000","last_became_peered":"2024-05-14T09:31:30.178398+0000","last_unstale":"2024-05-14T09:32:31.014777+0000","last_undegraded":"2024-05-14T09:32:31.014777+0000","last_fullsized":"2024-05-14T09:32:31.014777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:15:55.369801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077759+0000","last_change":"2024-05-14T09:30:56.793642+0000","last_active":"2024-05-14T09:33:29.077759+0000","last_peered":"2024-05-14T09:33:29.077759+0000","last_clean":"2024-05-14T09:33:29.077759+0000","last_became_active":"2024-05-14T09:30:56.793229+0000","last_became_peered":"2024-05-14T09:30:56.793229+0000","last_unstale":"2024-05-14T09:33:29.077759+0000","last_undegraded":"2024-05-14T09:33:29.077759+0000","last_fullsized":"2024-05-14T09:33:29.077759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:17.940674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522946+0000","last_change":"2024-05-14T09:30:53.769582+0000","last_active":"2024-05-14T09:32:30.522946+0000","last_peered":"2024-05-14T09:32:30.522946+0000","last_clean":"2024-05-14T09:32:30.522946+0000","last_became_active":"2024-05-14T09:30:53.769113+0000","last_became_peered":"2024-05-14T09:30:53.769113+0000","last_unstale":"2024-05-14T09:32:30.522946+0000","last_undegraded":"2024-05-14T09:32:30.522946+0000","last_fullsized":"2024-05-14T09:32:30.522946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:48.755113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014072+0000","last_change":"2024-05-14T09:29:30.767389+0000","last_active":"2024-05-14T09:32:31.014072+0000","last_peered":"2024-05-14T09:32:31.014072+0000","last_clean":"2024-05-14T09:32:31.014072+0000","last_became_active":"2024-05-14T09:29:30.761827+0000","last_became_peered":"2024-05-14T09:29:30.761827+0000","last_unstale":"2024-05-14T09:32:31.014072+0000","last_undegraded":"2024-05-14T09:32:31.014072+0000","last_fullsized":"2024-05-14T09:32:31.014072+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:26:03.741821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:26:03.741821+0000","last_clean_scrub_stamp":"2024-05-14T09:26:03.741821+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:52:11.421658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316444+0000","last_change":"2024-05-14T09:31:08.387819+0000","last_active":"2024-05-14T09:32:30.316444+0000","last_peered":"2024-05-14T09:32:30.316444+0000","last_clean":"2024-05-14T09:32:30.316444+0000","last_became_active":"2024-05-14T09:31:08.387275+0000","last_became_peered":"2024-05-14T09:31:08.387275+0000","last_unstale":"2024-05-14T09:32:30.316444+0000","last_undegraded":"2024-05-14T09:32:30.316444+0000","last_fullsized":"2024-05-14T09:32:30.316444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:46.627389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.078055+0000","last_change":"2024-05-14T09:30:56.788159+0000","last_active":"2024-05-14T09:33:29.078055+0000","last_peered":"2024-05-14T09:33:29.078055+0000","last_clean":"2024-05-14T09:33:29.078055+0000","last_became_active":"2024-05-14T09:30:56.787744+0000","last_became_peered":"2024-05-14T09:30:56.787744+0000","last_unstale":"2024-05-14T09:33:29.078055+0000","last_undegraded":"2024-05-14T09:33:29.078055+0000","last_fullsized":"2024-05-14T09:33:29.078055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:45:25.243471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014462+0000","last_change":"2024-05-14T09:30:53.774345+0000","last_active":"2024-05-14T09:32:31.014462+0000","last_peered":"2024-05-14T09:32:31.014462+0000","last_clean":"2024-05-14T09:32:31.014462+0000","last_became_active":"2024-05-14T09:30:53.773853+0000","last_became_peered":"2024-05-14T09:30:53.773853+0000","last_unstale":"2024-05-14T09:32:31.014462+0000","last_undegraded":"2024-05-14T09:32:31.014462+0000","last_fullsized":"2024-05-14T09:32:31.014462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:35:31.938442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320387+0000","last_change":"2024-05-14T09:31:08.122876+0000","last_active":"2024-05-14T09:32:30.320387+0000","last_peered":"2024-05-14T09:32:30.320387+0000","last_clean":"2024-05-14T09:32:30.320387+0000","last_became_active":"2024-05-14T09:31:08.122182+0000","last_became_peered":"2024-05-14T09:31:08.122182+0000","last_unstale":"2024-05-14T09:32:30.320387+0000","last_undegraded":"2024-05-14T09:32:30.320387+0000","last_fullsized":"2024-05-14T09:32:30.320387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:47:30.735057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524333+0000","last_change":"2024-05-14T09:30:54.018168+0000","last_active":"2024-05-14T09:32:30.524333+0000","last_peered":"2024-05-14T09:32:30.524333+0000","last_clean":"2024-05-14T09:32:30.524333+0000","last_became_active":"2024-05-14T09:30:54.017699+0000","last_became_peered":"2024-05-14T09:30:54.017699+0000","last_unstale":"2024-05-14T09:32:30.524333+0000","last_undegraded":"2024-05-14T09:32:30.524333+0000","last_fullsized":"2024-05-14T09:32:30.524333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:31:01.162000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.070935+0000","last_change":"2024-05-14T09:30:56.787156+0000","last_active":"2024-05-14T09:33:29.070935+0000","last_peered":"2024-05-14T09:33:29.070935+0000","last_clean":"2024-05-14T09:33:29.070935+0000","last_became_active":"2024-05-14T09:30:56.786832+0000","last_became_peered":"2024-05-14T09:30:56.786832+0000","last_unstale":"2024-05-14T09:33:29.070935+0000","last_undegraded":"2024-05-14T09:33:29.070935+0000","last_fullsized":"2024-05-14T09:33:29.070935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:00:41.719641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.321913+0000","last_change":"2024-05-14T09:31:08.387907+0000","last_active":"2024-05-14T09:32:30.321913+0000","last_peered":"2024-05-14T09:32:30.321913+0000","last_clean":"2024-05-14T09:32:30.321913+0000","last_became_active":"2024-05-14T09:31:08.386765+0000","last_became_peered":"2024-05-14T09:31:08.386765+0000","last_unstale":"2024-05-14T09:32:30.321913+0000","last_undegraded":"2024-05-14T09:32:30.321913+0000","last_fullsized":"2024-05-14T09:32:30.321913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:24:56.890353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'12","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:05.982882+0000","last_change":"2024-05-14T09:30:53.767508+0000","last_active":"2024-05-14T09:33:05.982882+0000","last_peered":"2024-05-14T09:33:05.982882+0000","last_clean":"2024-05-14T09:33:05.982882+0000","last_became_active":"2024-05-14T09:30:53.767169+0000","last_became_peered":"2024-05-14T09:30:53.767169+0000","last_unstale":"2024-05-14T09:33:05.982882+0000","last_undegraded":"2024-05-14T09:33:05.982882+0000","last_fullsized":"2024-05-14T09:33:05.982882+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:04:23.621686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076780+0000","last_change":"2024-05-14T09:30:56.790398+0000","last_active":"2024-05-14T09:33:29.076780+0000","last_peered":"2024-05-14T09:33:29.076780+0000","last_clean":"2024-05-14T09:33:29.076780+0000","last_became_active":"2024-05-14T09:30:56.790055+0000","last_became_peered":"2024-05-14T09:30:56.790055+0000","last_unstale":"2024-05-14T09:33:29.076780+0000","last_undegraded":"2024-05-14T09:33:29.076780+0000","last_fullsized":"2024-05-14T09:33:29.076780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:12:37.688647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319239+0000","last_change":"2024-05-14T09:31:08.111533+0000","last_active":"2024-05-14T09:32:30.319239+0000","last_peered":"2024-05-14T09:32:30.319239+0000","last_clean":"2024-05-14T09:32:30.319239+0000","last_became_active":"2024-05-14T09:31:08.110448+0000","last_became_peered":"2024-05-14T09:31:08.110448+0000","last_unstale":"2024-05-14T09:32:30.319239+0000","last_undegraded":"2024-05-14T09:32:30.319239+0000","last_fullsized":"2024-05-14T09:32:30.319239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:43:45.822382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523626+0000","last_change":"2024-05-14T09:30:54.069180+0000","last_active":"2024-05-14T09:32:30.523626+0000","last_peered":"2024-05-14T09:32:30.523626+0000","last_clean":"2024-05-14T09:32:30.523626+0000","last_became_active":"2024-05-14T09:30:54.068846+0000","last_became_peered":"2024-05-14T09:30:54.068846+0000","last_unstale":"2024-05-14T09:32:30.523626+0000","last_undegraded":"2024-05-14T09:32:30.523626+0000","last_fullsized":"2024-05-14T09:32:30.523626+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:22.541720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080401+0000","last_change":"2024-05-14T09:30:56.788159+0000","last_active":"2024-05-14T09:33:29.080401+0000","last_peered":"2024-05-14T09:33:29.080401+0000","last_clean":"2024-05-14T09:33:29.080401+0000","last_became_active":"2024-05-14T09:30:56.787786+0000","last_became_peered":"2024-05-14T09:30:56.787786+0000","last_unstale":"2024-05-14T09:33:29.080401+0000","last_undegraded":"2024-05-14T09:33:29.080401+0000","last_fullsized":"2024-05-14T09:33:29.080401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:24:51.785543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523393+0000","last_change":"2024-05-14T09:31:08.388201+0000","last_active":"2024-05-14T09:32:30.523393+0000","last_peered":"2024-05-14T09:32:30.523393+0000","last_clean":"2024-05-14T09:32:30.523393+0000","last_became_active":"2024-05-14T09:31:08.387572+0000","last_became_peered":"2024-05-14T09:31:08.387572+0000","last_unstale":"2024-05-14T09:32:30.523393+0000","last_undegraded":"2024-05-14T09:32:30.523393+0000","last_fullsized":"2024-05-14T09:32:30.523393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:25:13.171350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316020+0000","last_change":"2024-05-14T09:30:54.016524+0000","last_active":"2024-05-14T09:32:30.316020+0000","last_peered":"2024-05-14T09:32:30.316020+0000","last_clean":"2024-05-14T09:32:30.316020+0000","last_became_active":"2024-05-14T09:30:54.016154+0000","last_became_peered":"2024-05-14T09:30:54.016154+0000","last_unstale":"2024-05-14T09:32:30.316020+0000","last_undegraded":"2024-05-14T09:32:30.316020+0000","last_fullsized":"2024-05-14T09:32:30.316020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:43:16.671053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.035477+0000","last_change":"2024-05-14T09:30:56.796748+0000","last_active":"2024-05-14T09:33:29.035477+0000","last_peered":"2024-05-14T09:33:29.035477+0000","last_clean":"2024-05-14T09:33:29.035477+0000","last_became_active":"2024-05-14T09:30:56.796354+0000","last_became_peered":"2024-05-14T09:30:56.796354+0000","last_unstale":"2024-05-14T09:33:29.035477+0000","last_undegraded":"2024-05-14T09:33:29.035477+0000","last_fullsized":"2024-05-14T09:33:29.035477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:30:15.820677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319112+0000","last_change":"2024-05-14T09:31:08.117266+0000","last_active":"2024-05-14T09:32:30.319112+0000","last_peered":"2024-05-14T09:32:30.319112+0000","last_clean":"2024-05-14T09:32:30.319112+0000","last_became_active":"2024-05-14T09:31:08.115393+0000","last_became_peered":"2024-05-14T09:31:08.115393+0000","last_unstale":"2024-05-14T09:32:30.319112+0000","last_undegraded":"2024-05-14T09:32:30.319112+0000","last_fullsized":"2024-05-14T09:32:30.319112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:29:26.348818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.322687+0000","last_change":"2024-05-14T09:30:53.772417+0000","last_active":"2024-05-14T09:32:30.322687+0000","last_peered":"2024-05-14T09:32:30.322687+0000","last_clean":"2024-05-14T09:32:30.322687+0000","last_became_active":"2024-05-14T09:30:53.772028+0000","last_became_peered":"2024-05-14T09:30:53.772028+0000","last_unstale":"2024-05-14T09:32:30.322687+0000","last_undegraded":"2024-05-14T09:32:30.322687+0000","last_fullsized":"2024-05-14T09:32:30.322687+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:16:16.603957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077309+0000","last_change":"2024-05-14T09:30:56.804750+0000","last_active":"2024-05-14T09:33:29.077309+0000","last_peered":"2024-05-14T09:33:29.077309+0000","last_clean":"2024-05-14T09:33:29.077309+0000","last_became_active":"2024-05-14T09:30:56.803990+0000","last_became_peered":"2024-05-14T09:30:56.803990+0000","last_unstale":"2024-05-14T09:33:29.077309+0000","last_undegraded":"2024-05-14T09:33:29.077309+0000","last_fullsized":"2024-05-14T09:33:29.077309+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:42:00.661033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523484+0000","last_change":"2024-05-14T09:31:08.116061+0000","last_active":"2024-05-14T09:32:30.523484+0000","last_peered":"2024-05-14T09:32:30.523484+0000","last_clean":"2024-05-14T09:32:30.523484+0000","last_became_active":"2024-05-14T09:31:08.115578+0000","last_became_peered":"2024-05-14T09:31:08.115578+0000","last_unstale":"2024-05-14T09:32:30.523484+0000","last_undegraded":"2024-05-14T09:32:30.523484+0000","last_fullsized":"2024-05-14T09:32:30.523484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:18.931035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319214+0000","last_change":"2024-05-14T09:30:53.768988+0000","last_active":"2024-05-14T09:32:30.319214+0000","last_peered":"2024-05-14T09:32:30.319214+0000","last_clean":"2024-05-14T09:32:30.319214+0000","last_became_active":"2024-05-14T09:30:53.768529+0000","last_became_peered":"2024-05-14T09:30:53.768529+0000","last_unstale":"2024-05-14T09:32:30.319214+0000","last_undegraded":"2024-05-14T09:32:30.319214+0000","last_fullsized":"2024-05-14T09:32:30.319214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:17:02.381447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.079500+0000","last_change":"2024-05-14T09:30:56.797110+0000","last_active":"2024-05-14T09:33:29.079500+0000","last_peered":"2024-05-14T09:33:29.079500+0000","last_clean":"2024-05-14T09:33:29.079500+0000","last_became_active":"2024-05-14T09:30:56.796015+0000","last_became_peered":"2024-05-14T09:30:56.796015+0000","last_unstale":"2024-05-14T09:33:29.079500+0000","last_undegraded":"2024-05-14T09:33:29.079500+0000","last_fullsized":"2024-05-14T09:33:29.079500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:37:44.674607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523289+0000","last_change":"2024-05-14T09:31:08.387240+0000","last_active":"2024-05-14T09:32:30.523289+0000","last_peered":"2024-05-14T09:32:30.523289+0000","last_clean":"2024-05-14T09:32:30.523289+0000","last_became_active":"2024-05-14T09:31:08.386653+0000","last_became_peered":"2024-05-14T09:31:08.386653+0000","last_unstale":"2024-05-14T09:32:30.523289+0000","last_undegraded":"2024-05-14T09:32:30.523289+0000","last_fullsized":"2024-05-14T09:32:30.523289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:18:43.408030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523577+0000","last_change":"2024-05-14T09:30:53.772723+0000","last_active":"2024-05-14T09:32:30.523577+0000","last_peered":"2024-05-14T09:32:30.523577+0000","last_clean":"2024-05-14T09:32:30.523577+0000","last_became_active":"2024-05-14T09:30:53.772056+0000","last_became_peered":"2024-05-14T09:30:53.772056+0000","last_unstale":"2024-05-14T09:32:30.523577+0000","last_undegraded":"2024-05-14T09:32:30.523577+0000","last_fullsized":"2024-05-14T09:32:30.523577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:37.941282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087926+0000","last_change":"2024-05-14T09:30:56.799223+0000","last_active":"2024-05-14T09:33:29.087926+0000","last_peered":"2024-05-14T09:33:29.087926+0000","last_clean":"2024-05-14T09:33:29.087926+0000","last_became_active":"2024-05-14T09:30:56.796092+0000","last_became_peered":"2024-05-14T09:30:56.796092+0000","last_unstale":"2024-05-14T09:33:29.087926+0000","last_undegraded":"2024-05-14T09:33:29.087926+0000","last_fullsized":"2024-05-14T09:33:29.087926+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:43:49.746797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316061+0000","last_change":"2024-05-14T09:31:08.115053+0000","last_active":"2024-05-14T09:32:30.316061+0000","last_peered":"2024-05-14T09:32:30.316061+0000","last_clean":"2024-05-14T09:32:30.316061+0000","last_became_active":"2024-05-14T09:31:08.114578+0000","last_became_peered":"2024-05-14T09:31:08.114578+0000","last_unstale":"2024-05-14T09:32:30.316061+0000","last_undegraded":"2024-05-14T09:32:30.316061+0000","last_fullsized":"2024-05-14T09:32:30.316061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:47.788962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.327052+0000","last_change":"2024-05-14T09:32:30.326580+0000","last_active":"2024-05-14T09:32:30.327052+0000","last_peered":"2024-05-14T09:32:30.327052+0000","last_clean":"2024-05-14T09:32:30.327052+0000","last_became_active":"2024-05-14T09:32:30.322052+0000","last_became_peered":"2024-05-14T09:32:30.322052+0000","last_unstale":"2024-05-14T09:32:30.327052+0000","last_undegraded":"2024-05-14T09:32:30.327052+0000","last_fullsized":"2024-05-14T09:32:30.327052+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:04.991674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087235+0000","last_change":"2024-05-14T09:30:56.798580+0000","last_active":"2024-05-14T09:33:29.087235+0000","last_peered":"2024-05-14T09:33:29.087235+0000","last_clean":"2024-05-14T09:33:29.087235+0000","last_became_active":"2024-05-14T09:30:56.795617+0000","last_became_peered":"2024-05-14T09:30:56.795617+0000","last_unstale":"2024-05-14T09:33:29.087235+0000","last_undegraded":"2024-05-14T09:33:29.087235+0000","last_fullsized":"2024-05-14T09:33:29.087235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:30:35.913497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319424+0000","last_change":"2024-05-14T09:31:08.111375+0000","last_active":"2024-05-14T09:32:30.319424+0000","last_peered":"2024-05-14T09:32:30.319424+0000","last_clean":"2024-05-14T09:32:30.319424+0000","last_became_active":"2024-05-14T09:31:08.110931+0000","last_became_peered":"2024-05-14T09:31:08.110931+0000","last_unstale":"2024-05-14T09:32:30.319424+0000","last_undegraded":"2024-05-14T09:32:30.319424+0000","last_fullsized":"2024-05-14T09:32:30.319424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:08:30.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.327490+0000","last_change":"2024-05-14T09:32:30.326979+0000","last_active":"2024-05-14T09:32:30.327490+0000","last_peered":"2024-05-14T09:32:30.327490+0000","last_clean":"2024-05-14T09:32:30.327490+0000","last_became_active":"2024-05-14T09:32:30.323545+0000","last_became_peered":"2024-05-14T09:32:30.323545+0000","last_unstale":"2024-05-14T09:32:30.327490+0000","last_undegraded":"2024-05-14T09:32:30.327490+0000","last_fullsized":"2024-05-14T09:32:30.327490+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:00:21.493910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":70,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.064380+0000","last_change":"2024-05-14T09:30:56.789527+0000","last_active":"2024-05-14T09:33:29.064380+0000","last_peered":"2024-05-14T09:33:29.064380+0000","last_clean":"2024-05-14T09:33:29.064380+0000","last_became_active":"2024-05-14T09:30:56.789075+0000","last_became_peered":"2024-05-14T09:30:56.789075+0000","last_unstale":"2024-05-14T09:33:29.064380+0000","last_undegraded":"2024-05-14T09:33:29.064380+0000","last_fullsized":"2024-05-14T09:33:29.064380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:07.140087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315692+0000","last_change":"2024-05-14T09:31:08.119624+0000","last_active":"2024-05-14T09:32:30.315692+0000","last_peered":"2024-05-14T09:32:30.315692+0000","last_clean":"2024-05-14T09:32:30.315692+0000","last_became_active":"2024-05-14T09:31:08.118102+0000","last_became_peered":"2024-05-14T09:31:08.118102+0000","last_unstale":"2024-05-14T09:32:30.315692+0000","last_undegraded":"2024-05-14T09:32:30.315692+0000","last_fullsized":"2024-05-14T09:32:30.315692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:48:17.806976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315472+0000","last_change":"2024-05-14T09:30:53.763903+0000","last_active":"2024-05-14T09:32:30.315472+0000","last_peered":"2024-05-14T09:32:30.315472+0000","last_clean":"2024-05-14T09:32:30.315472+0000","last_became_active":"2024-05-14T09:30:53.763517+0000","last_became_peered":"2024-05-14T09:30:53.763517+0000","last_unstale":"2024-05-14T09:32:30.315472+0000","last_undegraded":"2024-05-14T09:32:30.315472+0000","last_fullsized":"2024-05-14T09:32:30.315472+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:57.144392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081279+0000","last_change":"2024-05-14T09:30:56.791612+0000","last_active":"2024-05-14T09:33:29.081279+0000","last_peered":"2024-05-14T09:33:29.081279+0000","last_clean":"2024-05-14T09:33:29.081279+0000","last_became_active":"2024-05-14T09:30:56.791178+0000","last_became_peered":"2024-05-14T09:30:56.791178+0000","last_unstale":"2024-05-14T09:33:29.081279+0000","last_undegraded":"2024-05-14T09:33:29.081279+0000","last_fullsized":"2024-05-14T09:33:29.081279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:11:19.058139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076827+0000","last_change":"2024-05-14T09:30:56.804437+0000","last_active":"2024-05-14T09:33:29.076827+0000","last_peered":"2024-05-14T09:33:29.076827+0000","last_clean":"2024-05-14T09:33:29.076827+0000","last_became_active":"2024-05-14T09:30:56.804048+0000","last_became_peered":"2024-05-14T09:30:56.804048+0000","last_unstale":"2024-05-14T09:33:29.076827+0000","last_undegraded":"2024-05-14T09:33:29.076827+0000","last_fullsized":"2024-05-14T09:33:29.076827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:33.788903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.058110+0000","last_change":"2024-05-14T09:30:56.794309+0000","last_active":"2024-05-14T09:33:29.058110+0000","last_peered":"2024-05-14T09:33:29.058110+0000","last_clean":"2024-05-14T09:33:29.058110+0000","last_became_active":"2024-05-14T09:30:56.793317+0000","last_became_peered":"2024-05-14T09:30:56.793317+0000","last_unstale":"2024-05-14T09:33:29.058110+0000","last_undegraded":"2024-05-14T09:33:29.058110+0000","last_fullsized":"2024-05-14T09:33:29.058110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:40:26.878746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080059+0000","last_change":"2024-05-14T09:30:56.793618+0000","last_active":"2024-05-14T09:33:29.080059+0000","last_peered":"2024-05-14T09:33:29.080059+0000","last_clean":"2024-05-14T09:33:29.080059+0000","last_became_active":"2024-05-14T09:30:56.792819+0000","last_became_peered":"2024-05-14T09:30:56.792819+0000","last_unstale":"2024-05-14T09:33:29.080059+0000","last_undegraded":"2024-05-14T09:33:29.080059+0000","last_fullsized":"2024-05-14T09:33:29.080059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:29:38.191617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.072960+0000","last_change":"2024-05-14T09:30:56.794268+0000","last_active":"2024-05-14T09:33:29.072960+0000","last_peered":"2024-05-14T09:33:29.072960+0000","last_clean":"2024-05-14T09:33:29.072960+0000","last_became_active":"2024-05-14T09:30:56.793898+0000","last_became_peered":"2024-05-14T09:30:56.793898+0000","last_unstale":"2024-05-14T09:33:29.072960+0000","last_undegraded":"2024-05-14T09:33:29.072960+0000","last_fullsized":"2024-05-14T09:33:29.072960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:25:35.022236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.078917+0000","last_change":"2024-05-14T09:30:56.783366+0000","last_active":"2024-05-14T09:33:29.078917+0000","last_peered":"2024-05-14T09:33:29.078917+0000","last_clean":"2024-05-14T09:33:29.078917+0000","last_became_active":"2024-05-14T09:30:56.782980+0000","last_became_peered":"2024-05-14T09:30:56.782980+0000","last_unstale":"2024-05-14T09:33:29.078917+0000","last_undegraded":"2024-05-14T09:33:29.078917+0000","last_fullsized":"2024-05-14T09:33:29.078917+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:27.744902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076941+0000","last_change":"2024-05-14T09:32:30.524390+0000","last_active":"2024-05-14T09:33:29.076941+0000","last_peered":"2024-05-14T09:33:29.076941+0000","last_clean":"2024-05-14T09:33:29.076941+0000","last_became_active":"2024-05-14T09:32:30.523926+0000","last_became_peered":"2024-05-14T09:32:30.523926+0000","last_unstale":"2024-05-14T09:33:29.076941+0000","last_undegraded":"2024-05-14T09:33:29.076941+0000","last_fullsized":"2024-05-14T09:33:29.076941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:58:47.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.074545+0000","last_change":"2024-05-14T09:30:56.795872+0000","last_active":"2024-05-14T09:33:29.074545+0000","last_peered":"2024-05-14T09:33:29.074545+0000","last_clean":"2024-05-14T09:33:29.074545+0000","last_became_active":"2024-05-14T09:30:56.795492+0000","last_became_peered":"2024-05-14T09:30:56.795492+0000","last_unstale":"2024-05-14T09:33:29.074545+0000","last_undegraded":"2024-05-14T09:33:29.074545+0000","last_fullsized":"2024-05-14T09:33:29.074545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:26:29.226575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.062634+0000","last_change":"2024-05-14T09:32:30.524189+0000","last_active":"2024-05-14T09:33:29.062634+0000","last_peered":"2024-05-14T09:33:29.062634+0000","last_clean":"2024-05-14T09:33:29.062634+0000","last_became_active":"2024-05-14T09:32:30.523791+0000","last_became_peered":"2024-05-14T09:32:30.523791+0000","last_unstale":"2024-05-14T09:33:29.062634+0000","last_undegraded":"2024-05-14T09:33:29.062634+0000","last_fullsized":"2024-05-14T09:33:29.062634+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:52.441130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080195+0000","last_change":"2024-05-14T09:30:56.792814+0000","last_active":"2024-05-14T09:33:29.080195+0000","last_peered":"2024-05-14T09:33:29.080195+0000","last_clean":"2024-05-14T09:33:29.080195+0000","last_became_active":"2024-05-14T09:30:56.792282+0000","last_became_peered":"2024-05-14T09:30:56.792282+0000","last_unstale":"2024-05-14T09:33:29.080195+0000","last_undegraded":"2024-05-14T09:33:29.080195+0000","last_fullsized":"2024-05-14T09:33:29.080195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:52:35.319479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076831+0000","last_change":"2024-05-14T09:30:56.791575+0000","last_active":"2024-05-14T09:33:29.076831+0000","last_peered":"2024-05-14T09:33:29.076831+0000","last_clean":"2024-05-14T09:33:29.076831+0000","last_became_active":"2024-05-14T09:30:56.791230+0000","last_became_peered":"2024-05-14T09:30:56.791230+0000","last_unstale":"2024-05-14T09:33:29.076831+0000","last_undegraded":"2024-05-14T09:33:29.076831+0000","last_fullsized":"2024-05-14T09:33:29.076831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:40.510062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.088030+0000","last_change":"2024-05-14T09:30:56.796996+0000","last_active":"2024-05-14T09:33:29.088030+0000","last_peered":"2024-05-14T09:33:29.088030+0000","last_clean":"2024-05-14T09:33:29.088030+0000","last_became_active":"2024-05-14T09:30:56.796333+0000","last_became_peered":"2024-05-14T09:30:56.796333+0000","last_unstale":"2024-05-14T09:33:29.088030+0000","last_undegraded":"2024-05-14T09:33:29.088030+0000","last_fullsized":"2024-05-14T09:33:29.088030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:29:34.799088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.083938+0000","last_change":"2024-05-14T09:30:56.785740+0000","last_active":"2024-05-14T09:33:29.083938+0000","last_peered":"2024-05-14T09:33:29.083938+0000","last_clean":"2024-05-14T09:33:29.083938+0000","last_became_active":"2024-05-14T09:30:56.785394+0000","last_became_peered":"2024-05-14T09:30:56.785394+0000","last_unstale":"2024-05-14T09:33:29.083938+0000","last_undegraded":"2024-05-14T09:33:29.083938+0000","last_fullsized":"2024-05-14T09:33:29.083938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:17.819523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.079641+0000","last_change":"2024-05-14T09:30:56.796957+0000","last_active":"2024-05-14T09:33:29.079641+0000","last_peered":"2024-05-14T09:33:29.079641+0000","last_clean":"2024-05-14T09:33:29.079641+0000","last_became_active":"2024-05-14T09:30:56.795071+0000","last_became_peered":"2024-05-14T09:30:56.795071+0000","last_unstale":"2024-05-14T09:33:29.079641+0000","last_undegraded":"2024-05-14T09:33:29.079641+0000","last_fullsized":"2024-05-14T09:33:29.079641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:44:31.480207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245291,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68451,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":627,"num_read_kb":386,"num_write":175,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2846,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678063,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":519087,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874293,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":1174108,"data_compressed":67998,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037819,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":518671,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1597,"internal_metadata":36174275},"hb_peers":[0,1,2,3,4,5,6,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234047,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":518009,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430277,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":1114059,"data_compressed":67998,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593802,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":514429,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":578648,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":515235,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":515057,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149784,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":514041,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953557,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":582806,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790032,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1110159,"data_compressed":67998,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33883,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33883,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":64891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T09:33:40.226 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T09:33:40.484 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:40.863 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:40 smithi002 ceph-mon[31557]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T09:33:40.863 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:40 smithi002 ceph-mon[31557]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:33:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:40 smithi029 ceph-mon[36249]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T09:33:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:40 smithi029 ceph-mon[36249]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:33:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:40 smithi026 ceph-mon[36056]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T09:33:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:40 smithi026 ceph-mon[36056]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:33:42.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.248+0000 7f6abd59f700 1 -- 172.21.15.2:0/1179426204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ab8107df0 msgr2=0x7f6ab8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.248+0000 7f6abd59f700 1 --2- 172.21.15.2:0/1179426204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ab8107df0 0x7f6ab8108200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0099c0 tx=0x7f6aac009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.248+0000 7f6abd59f700 1 -- 172.21.15.2:0/1179426204 shutdown_connections 2024-05-14T09:33:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.248+0000 7f6abd59f700 1 --2- 172.21.15.2:0/1179426204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab8108740 0x7f6ab81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.248+0000 7f6abd59f700 1 --2- 172.21.15.2:0/1179426204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ab81034c0 0x7f6ab81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.248+0000 7f6abd59f700 1 --2- 172.21.15.2:0/1179426204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ab8107df0 0x7f6ab8108200 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.248+0000 7f6abd59f700 1 -- 172.21.15.2:0/1179426204 >> 172.21.15.2:0/1179426204 conn(0x7f6ab80fedf0 msgr2=0x7f6ab8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.248+0000 7f6abd59f700 1 -- 172.21.15.2:0/1179426204 shutdown_connections 2024-05-14T09:33:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.248+0000 7f6abd59f700 1 -- 172.21.15.2:0/1179426204 wait complete. 2024-05-14T09:33:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.249+0000 7f6abd59f700 1 Processor -- start 2024-05-14T09:33:42.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.249+0000 7f6abd59f700 1 -- start start 2024-05-14T09:33:42.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.249+0000 7f6abd59f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab81034c0 0x7f6ab819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:42.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.249+0000 7f6abd59f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ab8107df0 0x7f6ab819c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:42.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.249+0000 7f6abd59f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ab8108740 0x7f6ab81a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:42.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.249+0000 7f6abd59f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab8113350 con 0x7f6ab8107df0 2024-05-14T09:33:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.249+0000 7f6abd59f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81131d0 con 0x7f6ab81034c0 2024-05-14T09:33:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.249+0000 7f6abd59f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81134d0 con 0x7f6ab8108740 2024-05-14T09:33:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6ab67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ab8107df0 0x7f6ab819c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6ab67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ab8107df0 0x7f6ab819c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57156/0 (socket says 172.21.15.2:57156) 2024-05-14T09:33:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6ab67fc700 1 -- 172.21.15.2:0/2888479440 learned_addr learned my addr 172.21.15.2:0/2888479440 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6ab6ffd700 1 --2- 172.21.15.2:0/2888479440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab81034c0 0x7f6ab819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6ab77fe700 1 --2- 172.21.15.2:0/2888479440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ab8108740 0x7f6ab81a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6ab67fc700 1 -- 172.21.15.2:0/2888479440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ab8108740 msgr2=0x7f6ab81a6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6ab67fc700 1 --2- 172.21.15.2:0/2888479440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ab8108740 0x7f6ab81a6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6ab67fc700 1 -- 172.21.15.2:0/2888479440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab81034c0 msgr2=0x7f6ab819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6ab67fc700 1 --2- 172.21.15.2:0/2888479440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab81034c0 0x7f6ab819bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6ab67fc700 1 -- 172.21.15.2:0/2888479440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab819c9b0 con 0x7f6ab8107df0 2024-05-14T09:33:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6ab77fe700 1 --2- 172.21.15.2:0/2888479440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ab8108740 0x7f6ab81a6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:33:42.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6ab67fc700 1 --2- 172.21.15.2:0/2888479440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ab8107df0 0x7f6ab819c190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009f50 tx=0x7f6aac0037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:42.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6a9ffff700 1 -- 172.21.15.2:0/2888479440 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac027070 con 0x7f6ab8107df0 2024-05-14T09:33:42.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6abd59f700 1 -- 172.21.15.2:0/2888479440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab81a7210 con 0x7f6ab8107df0 2024-05-14T09:33:42.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6a9ffff700 1 -- 172.21.15.2:0/2888479440 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aac004090 con 0x7f6ab8107df0 2024-05-14T09:33:42.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6a9ffff700 1 -- 172.21.15.2:0/2888479440 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac022e60 con 0x7f6ab8107df0 2024-05-14T09:33:42.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.250+0000 7f6abd59f700 1 -- 172.21.15.2:0/2888479440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab81a76c0 con 0x7f6ab8107df0 2024-05-14T09:33:42.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.251+0000 7f6abd59f700 1 -- 172.21.15.2:0/2888479440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab80512c0 con 0x7f6ab8107df0 2024-05-14T09:33:42.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.253+0000 7f6a9ffff700 1 -- 172.21.15.2:0/2888479440 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6aac02a070 con 0x7f6ab8107df0 2024-05-14T09:33:42.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.254+0000 7f6a9ffff700 1 --2- 172.21.15.2:0/2888479440 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6aa00b2880 0x7f6aa00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:42.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.254+0000 7f6ab6ffd700 1 --2- 172.21.15.2:0/2888479440 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6aa00b2880 0x7f6aa00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:42.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.254+0000 7f6a9ffff700 1 -- 172.21.15.2:0/2888479440 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f6aac0e3600 con 0x7f6ab8107df0 2024-05-14T09:33:42.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.254+0000 7f6ab6ffd700 1 --2- 172.21.15.2:0/2888479440 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6aa00b2880 0x7f6aa00b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800ac50 tx=0x7f6aa800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:42.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.257+0000 7f6a9ffff700 1 -- 172.21.15.2:0/2888479440 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6aac013070 con 0x7f6ab8107df0 2024-05-14T09:33:42.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.466+0000 7f6abd59f700 1 -- 172.21.15.2:0/2888479440 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ab8063c00 con 0x7f6aa00b2880 2024-05-14T09:33:42.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.476+0000 7f6a9ffff700 1 -- 172.21.15.2:0/2888479440 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340738 (secure 0 0 0) 0x7f6ab8063c00 con 0x7f6aa00b2880 2024-05-14T09:33:42.480 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:33:42.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.479+0000 7f6abd59f700 1 -- 172.21.15.2:0/2888479440 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6aa00b2880 msgr2=0x7f6aa00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:42.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.479+0000 7f6abd59f700 1 --2- 172.21.15.2:0/2888479440 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6aa00b2880 0x7f6aa00b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800ac50 tx=0x7f6aa800a380 comp rx=0 tx=0).stop 2024-05-14T09:33:42.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.479+0000 7f6abd59f700 1 -- 172.21.15.2:0/2888479440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ab8107df0 msgr2=0x7f6ab819c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:42.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.479+0000 7f6abd59f700 1 --2- 172.21.15.2:0/2888479440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ab8107df0 0x7f6ab819c190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009f50 tx=0x7f6aac0037c0 comp rx=0 tx=0).stop 2024-05-14T09:33:42.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.479+0000 7f6abd59f700 1 -- 172.21.15.2:0/2888479440 shutdown_connections 2024-05-14T09:33:42.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.479+0000 7f6abd59f700 1 --2- 172.21.15.2:0/2888479440 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6aa00b2880 0x7f6aa00b4d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:42.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.479+0000 7f6abd59f700 1 --2- 172.21.15.2:0/2888479440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab81034c0 0x7f6ab819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:42.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.479+0000 7f6abd59f700 1 --2- 172.21.15.2:0/2888479440 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ab8108740 0x7f6ab81a6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:42.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.479+0000 7f6abd59f700 1 --2- 172.21.15.2:0/2888479440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ab8107df0 0x7f6ab819c190 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:42.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.479+0000 7f6abd59f700 1 -- 172.21.15.2:0/2888479440 >> 172.21.15.2:0/2888479440 conn(0x7f6ab80fedf0 msgr2=0x7f6ab81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:42.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.480+0000 7f6abd59f700 1 -- 172.21.15.2:0/2888479440 shutdown_connections 2024-05-14T09:33:42.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:42.480+0000 7f6abd59f700 1 -- 172.21.15.2:0/2888479440 wait complete. 2024-05-14T09:33:42.483 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-05-14T09:33:42.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:42 smithi002 ceph-mon[31557]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T09:33:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:42 smithi029 ceph-mon[36249]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T09:33:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:42 smithi026 ceph-mon[36056]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T09:33:43.139 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-05-14T09:33:41.085144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659053,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":703,"num_read_kb":450,"num_write":300,"num_write_kb":2275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2954686,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":744,"ondisk_log_size":744,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":394500,"kb_used_data":12068,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124547324,"statfs":{"total":1151940427776,"available":1151536459776,"internally_reserved":0,"allocated":12357632,"data_stored":8174309,"data_compressed":496431,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19140,"internal_metadata":391492924},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":32,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003739"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087224+0000","last_change":"2024-05-14T09:30:56.797051+0000","last_active":"2024-05-14T09:33:29.087224+0000","last_peered":"2024-05-14T09:33:29.087224+0000","last_clean":"2024-05-14T09:33:29.087224+0000","last_became_active":"2024-05-14T09:30:56.796517+0000","last_became_peered":"2024-05-14T09:30:56.796517+0000","last_unstale":"2024-05-14T09:33:29.087224+0000","last_undegraded":"2024-05-14T09:33:29.087224+0000","last_fullsized":"2024-05-14T09:33:29.087224+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:58:22.170187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.073126+0000","last_change":"2024-05-14T09:30:56.795325+0000","last_active":"2024-05-14T09:33:29.073126+0000","last_peered":"2024-05-14T09:33:29.073126+0000","last_clean":"2024-05-14T09:33:29.073126+0000","last_became_active":"2024-05-14T09:30:56.794962+0000","last_became_peered":"2024-05-14T09:30:56.794962+0000","last_unstale":"2024-05-14T09:33:29.073126+0000","last_undegraded":"2024-05-14T09:33:29.073126+0000","last_fullsized":"2024-05-14T09:33:29.073126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:15:28.890565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084179+0000","last_change":"2024-05-14T09:30:56.793200+0000","last_active":"2024-05-14T09:33:29.084179+0000","last_peered":"2024-05-14T09:33:29.084179+0000","last_clean":"2024-05-14T09:33:29.084179+0000","last_became_active":"2024-05-14T09:30:56.792835+0000","last_became_peered":"2024-05-14T09:30:56.792835+0000","last_unstale":"2024-05-14T09:33:29.084179+0000","last_undegraded":"2024-05-14T09:33:29.084179+0000","last_fullsized":"2024-05-14T09:33:29.084179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:19.851276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.070997+0000","last_change":"2024-05-14T09:30:56.790238+0000","last_active":"2024-05-14T09:33:29.070997+0000","last_peered":"2024-05-14T09:33:29.070997+0000","last_clean":"2024-05-14T09:33:29.070997+0000","last_became_active":"2024-05-14T09:30:56.789826+0000","last_became_peered":"2024-05-14T09:30:56.789826+0000","last_unstale":"2024-05-14T09:33:29.070997+0000","last_undegraded":"2024-05-14T09:33:29.070997+0000","last_fullsized":"2024-05-14T09:33:29.070997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:19:19.853931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080306+0000","last_change":"2024-05-14T09:30:56.788021+0000","last_active":"2024-05-14T09:33:29.080306+0000","last_peered":"2024-05-14T09:33:29.080306+0000","last_clean":"2024-05-14T09:33:29.080306+0000","last_became_active":"2024-05-14T09:30:56.787657+0000","last_became_peered":"2024-05-14T09:30:56.787657+0000","last_unstale":"2024-05-14T09:33:29.080306+0000","last_undegraded":"2024-05-14T09:33:29.080306+0000","last_fullsized":"2024-05-14T09:33:29.080306+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:37:03.232137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.082547+0000","last_change":"2024-05-14T09:32:30.525175+0000","last_active":"2024-05-14T09:33:29.082547+0000","last_peered":"2024-05-14T09:33:29.082547+0000","last_clean":"2024-05-14T09:33:29.082547+0000","last_became_active":"2024-05-14T09:32:30.524861+0000","last_became_peered":"2024-05-14T09:32:30.524861+0000","last_unstale":"2024-05-14T09:33:29.082547+0000","last_undegraded":"2024-05-14T09:33:29.082547+0000","last_fullsized":"2024-05-14T09:33:29.082547+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:22:21.063843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.066050+0000","last_change":"2024-05-14T09:30:56.795647+0000","last_active":"2024-05-14T09:33:29.066050+0000","last_peered":"2024-05-14T09:33:29.066050+0000","last_clean":"2024-05-14T09:33:29.066050+0000","last_became_active":"2024-05-14T09:30:56.795186+0000","last_became_peered":"2024-05-14T09:30:56.795186+0000","last_unstale":"2024-05-14T09:33:29.066050+0000","last_undegraded":"2024-05-14T09:33:29.066050+0000","last_fullsized":"2024-05-14T09:33:29.066050+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:04:53.471305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081501+0000","last_change":"2024-05-14T09:30:56.796834+0000","last_active":"2024-05-14T09:33:29.081501+0000","last_peered":"2024-05-14T09:33:29.081501+0000","last_clean":"2024-05-14T09:33:29.081501+0000","last_became_active":"2024-05-14T09:30:56.796492+0000","last_became_peered":"2024-05-14T09:30:56.796492+0000","last_unstale":"2024-05-14T09:33:29.081501+0000","last_undegraded":"2024-05-14T09:33:29.081501+0000","last_fullsized":"2024-05-14T09:33:29.081501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:46.606129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.086070+0000","last_change":"2024-05-14T09:30:56.795020+0000","last_active":"2024-05-14T09:33:29.086070+0000","last_peered":"2024-05-14T09:33:29.086070+0000","last_clean":"2024-05-14T09:33:29.086070+0000","last_became_active":"2024-05-14T09:30:56.794625+0000","last_became_peered":"2024-05-14T09:30:56.794625+0000","last_unstale":"2024-05-14T09:33:29.086070+0000","last_undegraded":"2024-05-14T09:33:29.086070+0000","last_fullsized":"2024-05-14T09:33:29.086070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:09:38.597296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.082128+0000","last_change":"2024-05-14T09:30:56.796439+0000","last_active":"2024-05-14T09:33:29.082128+0000","last_peered":"2024-05-14T09:33:29.082128+0000","last_clean":"2024-05-14T09:33:29.082128+0000","last_became_active":"2024-05-14T09:30:56.795168+0000","last_became_peered":"2024-05-14T09:30:56.795168+0000","last_unstale":"2024-05-14T09:33:29.082128+0000","last_undegraded":"2024-05-14T09:33:29.082128+0000","last_fullsized":"2024-05-14T09:33:29.082128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:47.660286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087219+0000","last_change":"2024-05-14T09:30:56.795994+0000","last_active":"2024-05-14T09:33:29.087219+0000","last_peered":"2024-05-14T09:33:29.087219+0000","last_clean":"2024-05-14T09:33:29.087219+0000","last_became_active":"2024-05-14T09:30:56.795079+0000","last_became_peered":"2024-05-14T09:30:56.795079+0000","last_unstale":"2024-05-14T09:33:29.087219+0000","last_undegraded":"2024-05-14T09:33:29.087219+0000","last_fullsized":"2024-05-14T09:33:29.087219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:37:42.930195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.072741+0000","last_change":"2024-05-14T09:30:56.792997+0000","last_active":"2024-05-14T09:33:29.072741+0000","last_peered":"2024-05-14T09:33:29.072741+0000","last_clean":"2024-05-14T09:33:29.072741+0000","last_became_active":"2024-05-14T09:30:56.792302+0000","last_became_peered":"2024-05-14T09:30:56.792302+0000","last_unstale":"2024-05-14T09:33:29.072741+0000","last_undegraded":"2024-05-14T09:33:29.072741+0000","last_fullsized":"2024-05-14T09:33:29.072741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:52.986662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077933+0000","last_change":"2024-05-14T09:30:56.795903+0000","last_active":"2024-05-14T09:33:29.077933+0000","last_peered":"2024-05-14T09:33:29.077933+0000","last_clean":"2024-05-14T09:33:29.077933+0000","last_became_active":"2024-05-14T09:30:56.795466+0000","last_became_peered":"2024-05-14T09:30:56.795466+0000","last_unstale":"2024-05-14T09:33:29.077933+0000","last_undegraded":"2024-05-14T09:33:29.077933+0000","last_fullsized":"2024-05-14T09:33:29.077933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:02:10.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.070939+0000","last_change":"2024-05-14T09:30:56.795911+0000","last_active":"2024-05-14T09:33:29.070939+0000","last_peered":"2024-05-14T09:33:29.070939+0000","last_clean":"2024-05-14T09:33:29.070939+0000","last_became_active":"2024-05-14T09:30:56.795547+0000","last_became_peered":"2024-05-14T09:30:56.795547+0000","last_unstale":"2024-05-14T09:33:29.070939+0000","last_undegraded":"2024-05-14T09:33:29.070939+0000","last_fullsized":"2024-05-14T09:33:29.070939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:56:29.621783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.067755+0000","last_change":"2024-05-14T09:30:56.787745+0000","last_active":"2024-05-14T09:33:29.067755+0000","last_peered":"2024-05-14T09:33:29.067755+0000","last_clean":"2024-05-14T09:33:29.067755+0000","last_became_active":"2024-05-14T09:30:56.787385+0000","last_became_peered":"2024-05-14T09:30:56.787385+0000","last_unstale":"2024-05-14T09:33:29.067755+0000","last_undegraded":"2024-05-14T09:33:29.067755+0000","last_fullsized":"2024-05-14T09:33:29.067755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:11.256775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087830+0000","last_change":"2024-05-14T09:30:56.795785+0000","last_active":"2024-05-14T09:33:29.087830+0000","last_peered":"2024-05-14T09:33:29.087830+0000","last_clean":"2024-05-14T09:33:29.087830+0000","last_became_active":"2024-05-14T09:30:56.795436+0000","last_became_peered":"2024-05-14T09:30:56.795436+0000","last_unstale":"2024-05-14T09:33:29.087830+0000","last_undegraded":"2024-05-14T09:33:29.087830+0000","last_fullsized":"2024-05-14T09:33:29.087830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:12:09.221929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087321+0000","last_change":"2024-05-14T09:30:56.800688+0000","last_active":"2024-05-14T09:33:29.087321+0000","last_peered":"2024-05-14T09:33:29.087321+0000","last_clean":"2024-05-14T09:33:29.087321+0000","last_became_active":"2024-05-14T09:30:56.800204+0000","last_became_peered":"2024-05-14T09:30:56.800204+0000","last_unstale":"2024-05-14T09:33:29.087321+0000","last_undegraded":"2024-05-14T09:33:29.087321+0000","last_fullsized":"2024-05-14T09:33:29.087321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:32:27.057688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.075054+0000","last_change":"2024-05-14T09:30:56.795604+0000","last_active":"2024-05-14T09:33:29.075054+0000","last_peered":"2024-05-14T09:33:29.075054+0000","last_clean":"2024-05-14T09:33:29.075054+0000","last_became_active":"2024-05-14T09:30:56.795245+0000","last_became_peered":"2024-05-14T09:30:56.795245+0000","last_unstale":"2024-05-14T09:33:29.075054+0000","last_undegraded":"2024-05-14T09:33:29.075054+0000","last_fullsized":"2024-05-14T09:33:29.075054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:14.072584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080216+0000","last_change":"2024-05-14T09:30:56.794815+0000","last_active":"2024-05-14T09:33:29.080216+0000","last_peered":"2024-05-14T09:33:29.080216+0000","last_clean":"2024-05-14T09:33:29.080216+0000","last_became_active":"2024-05-14T09:30:56.794429+0000","last_became_peered":"2024-05-14T09:30:56.794429+0000","last_unstale":"2024-05-14T09:33:29.080216+0000","last_undegraded":"2024-05-14T09:33:29.080216+0000","last_fullsized":"2024-05-14T09:33:29.080216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:32:08.367046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319506+0000","last_change":"2024-05-14T09:31:08.107895+0000","last_active":"2024-05-14T09:32:30.319506+0000","last_peered":"2024-05-14T09:32:30.319506+0000","last_clean":"2024-05-14T09:32:30.319506+0000","last_became_active":"2024-05-14T09:31:08.107458+0000","last_became_peered":"2024-05-14T09:31:08.107458+0000","last_unstale":"2024-05-14T09:32:30.319506+0000","last_undegraded":"2024-05-14T09:32:30.319506+0000","last_fullsized":"2024-05-14T09:32:30.319506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:13:11.717753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":69,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319382+0000","last_change":"2024-05-14T09:31:30.178197+0000","last_active":"2024-05-14T09:32:30.319382+0000","last_peered":"2024-05-14T09:32:30.319382+0000","last_clean":"2024-05-14T09:32:30.319382+0000","last_became_active":"2024-05-14T09:31:30.171154+0000","last_became_peered":"2024-05-14T09:31:30.171154+0000","last_unstale":"2024-05-14T09:32:30.319382+0000","last_undegraded":"2024-05-14T09:32:30.319382+0000","last_fullsized":"2024-05-14T09:32:30.319382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:43:46.208439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077181+0000","last_change":"2024-05-14T09:30:56.800526+0000","last_active":"2024-05-14T09:33:29.077181+0000","last_peered":"2024-05-14T09:33:29.077181+0000","last_clean":"2024-05-14T09:33:29.077181+0000","last_became_active":"2024-05-14T09:30:56.797692+0000","last_became_peered":"2024-05-14T09:30:56.797692+0000","last_unstale":"2024-05-14T09:33:29.077181+0000","last_undegraded":"2024-05-14T09:33:29.077181+0000","last_fullsized":"2024-05-14T09:33:29.077181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:09:14.471173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522407+0000","last_change":"2024-05-14T09:31:30.173810+0000","last_active":"2024-05-14T09:32:30.522407+0000","last_peered":"2024-05-14T09:32:30.522407+0000","last_clean":"2024-05-14T09:32:30.522407+0000","last_became_active":"2024-05-14T09:31:30.173242+0000","last_became_peered":"2024-05-14T09:31:30.173242+0000","last_unstale":"2024-05-14T09:32:30.522407+0000","last_undegraded":"2024-05-14T09:32:30.522407+0000","last_fullsized":"2024-05-14T09:32:30.522407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:37:15.991805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.322883+0000","last_change":"2024-05-14T09:30:54.015636+0000","last_active":"2024-05-14T09:32:30.322883+0000","last_peered":"2024-05-14T09:32:30.322883+0000","last_clean":"2024-05-14T09:32:30.322883+0000","last_became_active":"2024-05-14T09:30:54.015290+0000","last_became_peered":"2024-05-14T09:30:54.015290+0000","last_unstale":"2024-05-14T09:32:30.322883+0000","last_undegraded":"2024-05-14T09:32:30.322883+0000","last_fullsized":"2024-05-14T09:32:30.322883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:29:17.548817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.088634+0000","last_change":"2024-05-14T09:30:56.799506+0000","last_active":"2024-05-14T09:33:29.088634+0000","last_peered":"2024-05-14T09:33:29.088634+0000","last_clean":"2024-05-14T09:33:29.088634+0000","last_became_active":"2024-05-14T09:30:56.799153+0000","last_became_peered":"2024-05-14T09:30:56.799153+0000","last_unstale":"2024-05-14T09:33:29.088634+0000","last_undegraded":"2024-05-14T09:33:29.088634+0000","last_fullsized":"2024-05-14T09:33:29.088634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:04:08.217337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522612+0000","last_change":"2024-05-14T09:31:08.115533+0000","last_active":"2024-05-14T09:32:30.522612+0000","last_peered":"2024-05-14T09:32:30.522612+0000","last_clean":"2024-05-14T09:32:30.522612+0000","last_became_active":"2024-05-14T09:31:08.115033+0000","last_became_peered":"2024-05-14T09:31:08.115033+0000","last_unstale":"2024-05-14T09:32:30.522612+0000","last_undegraded":"2024-05-14T09:32:30.522612+0000","last_fullsized":"2024-05-14T09:32:30.522612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:15.668357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524484+0000","last_change":"2024-05-14T09:30:53.770547+0000","last_active":"2024-05-14T09:32:30.524484+0000","last_peered":"2024-05-14T09:32:30.524484+0000","last_clean":"2024-05-14T09:32:30.524484+0000","last_became_active":"2024-05-14T09:30:53.770110+0000","last_became_peered":"2024-05-14T09:30:53.770110+0000","last_unstale":"2024-05-14T09:32:30.524484+0000","last_undegraded":"2024-05-14T09:32:30.524484+0000","last_fullsized":"2024-05-14T09:32:30.524484+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:18.371318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084457+0000","last_change":"2024-05-14T09:30:56.799427+0000","last_active":"2024-05-14T09:33:29.084457+0000","last_peered":"2024-05-14T09:33:29.084457+0000","last_clean":"2024-05-14T09:33:29.084457+0000","last_became_active":"2024-05-14T09:30:56.799018+0000","last_became_peered":"2024-05-14T09:30:56.799018+0000","last_unstale":"2024-05-14T09:33:29.084457+0000","last_undegraded":"2024-05-14T09:33:29.084457+0000","last_fullsized":"2024-05-14T09:33:29.084457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:13:31.837985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522470+0000","last_change":"2024-05-14T09:31:08.116258+0000","last_active":"2024-05-14T09:32:30.522470+0000","last_peered":"2024-05-14T09:32:30.522470+0000","last_clean":"2024-05-14T09:32:30.522470+0000","last_became_active":"2024-05-14T09:31:08.115040+0000","last_became_peered":"2024-05-14T09:31:08.115040+0000","last_unstale":"2024-05-14T09:32:30.522470+0000","last_undegraded":"2024-05-14T09:32:30.522470+0000","last_fullsized":"2024-05-14T09:32:30.522470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:57:54.839203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523793+0000","last_change":"2024-05-14T09:31:30.188984+0000","last_active":"2024-05-14T09:32:30.523793+0000","last_peered":"2024-05-14T09:32:30.523793+0000","last_clean":"2024-05-14T09:32:30.523793+0000","last_became_active":"2024-05-14T09:31:30.174410+0000","last_became_peered":"2024-05-14T09:31:30.174410+0000","last_unstale":"2024-05-14T09:32:30.523793+0000","last_undegraded":"2024-05-14T09:32:30.523793+0000","last_fullsized":"2024-05-14T09:32:30.523793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:01:35.713857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,4],"acting":[8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081749+0000","last_change":"2024-05-14T09:32:30.319390+0000","last_active":"2024-05-14T09:33:29.081749+0000","last_peered":"2024-05-14T09:33:29.081749+0000","last_clean":"2024-05-14T09:33:29.081749+0000","last_became_active":"2024-05-14T09:32:30.318645+0000","last_became_peered":"2024-05-14T09:32:30.318645+0000","last_unstale":"2024-05-14T09:33:29.081749+0000","last_undegraded":"2024-05-14T09:33:29.081749+0000","last_fullsized":"2024-05-14T09:33:29.081749+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:37:49.866996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320617+0000","last_change":"2024-05-14T09:31:08.115740+0000","last_active":"2024-05-14T09:32:30.320617+0000","last_peered":"2024-05-14T09:32:30.320617+0000","last_clean":"2024-05-14T09:32:30.320617+0000","last_became_active":"2024-05-14T09:31:08.115202+0000","last_became_peered":"2024-05-14T09:31:08.115202+0000","last_unstale":"2024-05-14T09:32:30.320617+0000","last_undegraded":"2024-05-14T09:32:30.320617+0000","last_fullsized":"2024-05-14T09:32:30.320617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:44.519280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":156,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:36.394647+0000","last_change":"2024-05-14T09:30:53.770869+0000","last_active":"2024-05-14T09:33:36.394647+0000","last_peered":"2024-05-14T09:33:36.394647+0000","last_clean":"2024-05-14T09:33:36.394647+0000","last_became_active":"2024-05-14T09:30:53.770425+0000","last_became_peered":"2024-05-14T09:30:53.770425+0000","last_unstale":"2024-05-14T09:33:36.394647+0000","last_undegraded":"2024-05-14T09:33:36.394647+0000","last_fullsized":"2024-05-14T09:33:36.394647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:03:33.768120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.059353+0000","last_change":"2024-05-14T09:30:56.789267+0000","last_active":"2024-05-14T09:33:29.059353+0000","last_peered":"2024-05-14T09:33:29.059353+0000","last_clean":"2024-05-14T09:33:29.059353+0000","last_became_active":"2024-05-14T09:30:56.788883+0000","last_became_peered":"2024-05-14T09:30:56.788883+0000","last_unstale":"2024-05-14T09:33:29.059353+0000","last_undegraded":"2024-05-14T09:33:29.059353+0000","last_fullsized":"2024-05-14T09:33:29.059353+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:09.818693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320509+0000","last_change":"2024-05-14T09:31:08.115193+0000","last_active":"2024-05-14T09:32:30.320509+0000","last_peered":"2024-05-14T09:32:30.320509+0000","last_clean":"2024-05-14T09:32:30.320509+0000","last_became_active":"2024-05-14T09:31:08.114701+0000","last_became_peered":"2024-05-14T09:31:08.114701+0000","last_unstale":"2024-05-14T09:32:30.320509+0000","last_undegraded":"2024-05-14T09:32:30.320509+0000","last_fullsized":"2024-05-14T09:32:30.320509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:47:48.487922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319686+0000","last_change":"2024-05-14T09:31:30.174760+0000","last_active":"2024-05-14T09:32:30.319686+0000","last_peered":"2024-05-14T09:32:30.319686+0000","last_clean":"2024-05-14T09:32:30.319686+0000","last_became_active":"2024-05-14T09:31:30.174253+0000","last_became_peered":"2024-05-14T09:31:30.174253+0000","last_unstale":"2024-05-14T09:32:30.319686+0000","last_undegraded":"2024-05-14T09:32:30.319686+0000","last_fullsized":"2024-05-14T09:32:30.319686+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:11:17.163436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081148+0000","last_change":"2024-05-14T09:30:56.794654+0000","last_active":"2024-05-14T09:33:29.081148+0000","last_peered":"2024-05-14T09:33:29.081148+0000","last_clean":"2024-05-14T09:33:29.081148+0000","last_became_active":"2024-05-14T09:30:56.794282+0000","last_became_peered":"2024-05-14T09:30:56.794282+0000","last_unstale":"2024-05-14T09:33:29.081148+0000","last_undegraded":"2024-05-14T09:33:29.081148+0000","last_fullsized":"2024-05-14T09:33:29.081148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:55:41.478707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320586+0000","last_change":"2024-05-14T09:31:08.119965+0000","last_active":"2024-05-14T09:32:30.320586+0000","last_peered":"2024-05-14T09:32:30.320586+0000","last_clean":"2024-05-14T09:32:30.320586+0000","last_became_active":"2024-05-14T09:31:08.119502+0000","last_became_peered":"2024-05-14T09:31:08.119502+0000","last_unstale":"2024-05-14T09:32:30.320586+0000","last_undegraded":"2024-05-14T09:32:30.320586+0000","last_fullsized":"2024-05-14T09:32:30.320586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:01:23.612036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.314804+0000","last_change":"2024-05-14T09:30:53.766495+0000","last_active":"2024-05-14T09:32:30.314804+0000","last_peered":"2024-05-14T09:32:30.314804+0000","last_clean":"2024-05-14T09:32:30.314804+0000","last_became_active":"2024-05-14T09:30:53.766122+0000","last_became_peered":"2024-05-14T09:30:53.766122+0000","last_unstale":"2024-05-14T09:32:30.314804+0000","last_undegraded":"2024-05-14T09:32:30.314804+0000","last_fullsized":"2024-05-14T09:32:30.314804+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:43:13.344209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.073417+0000","last_change":"2024-05-14T09:30:56.791938+0000","last_active":"2024-05-14T09:33:29.073417+0000","last_peered":"2024-05-14T09:33:29.073417+0000","last_clean":"2024-05-14T09:33:29.073417+0000","last_became_active":"2024-05-14T09:30:56.791525+0000","last_became_peered":"2024-05-14T09:30:56.791525+0000","last_unstale":"2024-05-14T09:33:29.073417+0000","last_undegraded":"2024-05-14T09:33:29.073417+0000","last_fullsized":"2024-05-14T09:33:29.073417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:18.178208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320104+0000","last_change":"2024-05-14T09:31:08.113548+0000","last_active":"2024-05-14T09:32:30.320104+0000","last_peered":"2024-05-14T09:32:30.320104+0000","last_clean":"2024-05-14T09:32:30.320104+0000","last_became_active":"2024-05-14T09:31:08.113117+0000","last_became_peered":"2024-05-14T09:31:08.113117+0000","last_unstale":"2024-05-14T09:32:30.320104+0000","last_undegraded":"2024-05-14T09:32:30.320104+0000","last_fullsized":"2024-05-14T09:32:30.320104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:25:15.483761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315966+0000","last_change":"2024-05-14T09:30:53.770987+0000","last_active":"2024-05-14T09:32:30.315966+0000","last_peered":"2024-05-14T09:32:30.315966+0000","last_clean":"2024-05-14T09:32:30.315966+0000","last_became_active":"2024-05-14T09:30:53.770609+0000","last_became_peered":"2024-05-14T09:30:53.770609+0000","last_unstale":"2024-05-14T09:32:30.315966+0000","last_undegraded":"2024-05-14T09:32:30.315966+0000","last_fullsized":"2024-05-14T09:32:30.315966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:26:22.020642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081149+0000","last_change":"2024-05-14T09:30:56.789219+0000","last_active":"2024-05-14T09:33:29.081149+0000","last_peered":"2024-05-14T09:33:29.081149+0000","last_clean":"2024-05-14T09:33:29.081149+0000","last_became_active":"2024-05-14T09:30:56.788871+0000","last_became_peered":"2024-05-14T09:30:56.788871+0000","last_unstale":"2024-05-14T09:33:29.081149+0000","last_undegraded":"2024-05-14T09:33:29.081149+0000","last_fullsized":"2024-05-14T09:33:29.081149+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:10:50.774267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.322531+0000","last_change":"2024-05-14T09:31:30.174708+0000","last_active":"2024-05-14T09:32:30.322531+0000","last_peered":"2024-05-14T09:32:30.322531+0000","last_clean":"2024-05-14T09:32:30.322531+0000","last_became_active":"2024-05-14T09:31:30.174199+0000","last_became_peered":"2024-05-14T09:31:30.174199+0000","last_unstale":"2024-05-14T09:32:30.322531+0000","last_undegraded":"2024-05-14T09:32:30.322531+0000","last_fullsized":"2024-05-14T09:32:30.322531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:23:55.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":149,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:36.394034+0000","last_change":"2024-05-14T09:30:53.770898+0000","last_active":"2024-05-14T09:33:36.394034+0000","last_peered":"2024-05-14T09:33:36.394034+0000","last_clean":"2024-05-14T09:33:36.394034+0000","last_became_active":"2024-05-14T09:30:53.770483+0000","last_became_peered":"2024-05-14T09:30:53.770483+0000","last_unstale":"2024-05-14T09:33:36.394034+0000","last_undegraded":"2024-05-14T09:33:36.394034+0000","last_fullsized":"2024-05-14T09:33:36.394034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:07.764348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084404+0000","last_change":"2024-05-14T09:30:56.799049+0000","last_active":"2024-05-14T09:33:29.084404+0000","last_peered":"2024-05-14T09:33:29.084404+0000","last_clean":"2024-05-14T09:33:29.084404+0000","last_became_active":"2024-05-14T09:30:56.798643+0000","last_became_peered":"2024-05-14T09:30:56.798643+0000","last_unstale":"2024-05-14T09:33:29.084404+0000","last_undegraded":"2024-05-14T09:33:29.084404+0000","last_fullsized":"2024-05-14T09:33:29.084404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:12.378373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.318070+0000","last_change":"2024-05-14T09:31:08.109876+0000","last_active":"2024-05-14T09:32:30.318070+0000","last_peered":"2024-05-14T09:32:30.318070+0000","last_clean":"2024-05-14T09:32:30.318070+0000","last_became_active":"2024-05-14T09:31:08.109470+0000","last_became_peered":"2024-05-14T09:31:08.109470+0000","last_unstale":"2024-05-14T09:32:30.318070+0000","last_undegraded":"2024-05-14T09:32:30.318070+0000","last_fullsized":"2024-05-14T09:32:30.318070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:07.704153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.015003+0000","last_change":"2024-05-14T09:31:30.179086+0000","last_active":"2024-05-14T09:32:31.015003+0000","last_peered":"2024-05-14T09:32:31.015003+0000","last_clean":"2024-05-14T09:32:31.015003+0000","last_became_active":"2024-05-14T09:31:30.178604+0000","last_became_peered":"2024-05-14T09:31:30.178604+0000","last_unstale":"2024-05-14T09:32:31.015003+0000","last_undegraded":"2024-05-14T09:32:31.015003+0000","last_fullsized":"2024-05-14T09:32:31.015003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:39.712395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.073315+0000","last_change":"2024-05-14T09:30:56.793109+0000","last_active":"2024-05-14T09:33:29.073315+0000","last_peered":"2024-05-14T09:33:29.073315+0000","last_clean":"2024-05-14T09:33:29.073315+0000","last_became_active":"2024-05-14T09:30:56.792761+0000","last_became_peered":"2024-05-14T09:30:56.792761+0000","last_unstale":"2024-05-14T09:33:29.073315+0000","last_undegraded":"2024-05-14T09:33:29.073315+0000","last_fullsized":"2024-05-14T09:33:29.073315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:25:33.216281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522916+0000","last_change":"2024-05-14T09:31:08.120978+0000","last_active":"2024-05-14T09:32:30.522916+0000","last_peered":"2024-05-14T09:32:30.522916+0000","last_clean":"2024-05-14T09:32:30.522916+0000","last_became_active":"2024-05-14T09:31:08.120227+0000","last_became_peered":"2024-05-14T09:31:08.120227+0000","last_unstale":"2024-05-14T09:32:30.522916+0000","last_undegraded":"2024-05-14T09:32:30.522916+0000","last_fullsized":"2024-05-14T09:32:30.522916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:06.630390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316348+0000","last_change":"2024-05-14T09:30:54.017382+0000","last_active":"2024-05-14T09:32:30.316348+0000","last_peered":"2024-05-14T09:32:30.316348+0000","last_clean":"2024-05-14T09:32:30.316348+0000","last_became_active":"2024-05-14T09:30:54.016887+0000","last_became_peered":"2024-05-14T09:30:54.016887+0000","last_unstale":"2024-05-14T09:32:30.316348+0000","last_undegraded":"2024-05-14T09:32:30.316348+0000","last_fullsized":"2024-05-14T09:32:30.316348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:28.823659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.085392+0000","last_change":"2024-05-14T09:30:56.800626+0000","last_active":"2024-05-14T09:33:29.085392+0000","last_peered":"2024-05-14T09:33:29.085392+0000","last_clean":"2024-05-14T09:33:29.085392+0000","last_became_active":"2024-05-14T09:30:56.800258+0000","last_became_peered":"2024-05-14T09:30:56.800258+0000","last_unstale":"2024-05-14T09:33:29.085392+0000","last_undegraded":"2024-05-14T09:33:29.085392+0000","last_fullsized":"2024-05-14T09:33:29.085392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:47:26.369633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.318135+0000","last_change":"2024-05-14T09:31:08.115456+0000","last_active":"2024-05-14T09:32:30.318135+0000","last_peered":"2024-05-14T09:32:30.318135+0000","last_clean":"2024-05-14T09:32:30.318135+0000","last_became_active":"2024-05-14T09:31:08.114705+0000","last_became_peered":"2024-05-14T09:31:08.114705+0000","last_unstale":"2024-05-14T09:32:30.318135+0000","last_undegraded":"2024-05-14T09:32:30.318135+0000","last_fullsized":"2024-05-14T09:32:30.318135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:50.096747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":308,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:37.395294+0000","last_change":"2024-05-14T09:31:30.186202+0000","last_active":"2024-05-14T09:33:37.395294+0000","last_peered":"2024-05-14T09:33:37.395294+0000","last_clean":"2024-05-14T09:33:37.395294+0000","last_became_active":"2024-05-14T09:31:30.174248+0000","last_became_peered":"2024-05-14T09:31:30.174248+0000","last_unstale":"2024-05-14T09:33:37.395294+0000","last_undegraded":"2024-05-14T09:33:37.395294+0000","last_fullsized":"2024-05-14T09:33:37.395294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:45:03.131934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64309,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":210,"num_read_kb":168,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077083+0000","last_change":"2024-05-14T09:30:56.791099+0000","last_active":"2024-05-14T09:33:29.077083+0000","last_peered":"2024-05-14T09:33:29.077083+0000","last_clean":"2024-05-14T09:33:29.077083+0000","last_became_active":"2024-05-14T09:30:56.790676+0000","last_became_peered":"2024-05-14T09:30:56.790676+0000","last_unstale":"2024-05-14T09:33:29.077083+0000","last_undegraded":"2024-05-14T09:33:29.077083+0000","last_fullsized":"2024-05-14T09:33:29.077083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:09.851357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316986+0000","last_change":"2024-05-14T09:31:08.120289+0000","last_active":"2024-05-14T09:32:30.316986+0000","last_peered":"2024-05-14T09:32:30.316986+0000","last_clean":"2024-05-14T09:32:30.316986+0000","last_became_active":"2024-05-14T09:31:08.118574+0000","last_became_peered":"2024-05-14T09:31:08.118574+0000","last_unstale":"2024-05-14T09:32:30.316986+0000","last_undegraded":"2024-05-14T09:32:30.316986+0000","last_fullsized":"2024-05-14T09:32:30.316986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:01.723442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319990+0000","last_change":"2024-05-14T09:30:53.766578+0000","last_active":"2024-05-14T09:32:30.319990+0000","last_peered":"2024-05-14T09:32:30.319990+0000","last_clean":"2024-05-14T09:32:30.319990+0000","last_became_active":"2024-05-14T09:30:53.765928+0000","last_became_peered":"2024-05-14T09:30:53.765928+0000","last_unstale":"2024-05-14T09:32:30.319990+0000","last_undegraded":"2024-05-14T09:32:30.319990+0000","last_fullsized":"2024-05-14T09:32:30.319990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:19:08.690013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.083536+0000","last_change":"2024-05-14T09:30:56.785507+0000","last_active":"2024-05-14T09:33:29.083536+0000","last_peered":"2024-05-14T09:33:29.083536+0000","last_clean":"2024-05-14T09:33:29.083536+0000","last_became_active":"2024-05-14T09:30:56.784800+0000","last_became_peered":"2024-05-14T09:30:56.784800+0000","last_unstale":"2024-05-14T09:33:29.083536+0000","last_undegraded":"2024-05-14T09:33:29.083536+0000","last_fullsized":"2024-05-14T09:33:29.083536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:36:05.440222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.321298+0000","last_change":"2024-05-14T09:31:08.119051+0000","last_active":"2024-05-14T09:32:30.321298+0000","last_peered":"2024-05-14T09:32:30.321298+0000","last_clean":"2024-05-14T09:32:30.321298+0000","last_became_active":"2024-05-14T09:31:08.118569+0000","last_became_peered":"2024-05-14T09:31:08.118569+0000","last_unstale":"2024-05-14T09:32:30.321298+0000","last_undegraded":"2024-05-14T09:32:30.321298+0000","last_fullsized":"2024-05-14T09:32:30.321298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:12:39.643740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316079+0000","last_change":"2024-05-14T09:30:53.772832+0000","last_active":"2024-05-14T09:32:30.316079+0000","last_peered":"2024-05-14T09:32:30.316079+0000","last_clean":"2024-05-14T09:32:30.316079+0000","last_became_active":"2024-05-14T09:30:53.772228+0000","last_became_peered":"2024-05-14T09:30:53.772228+0000","last_unstale":"2024-05-14T09:32:30.316079+0000","last_undegraded":"2024-05-14T09:32:30.316079+0000","last_fullsized":"2024-05-14T09:32:30.316079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:14:43.350330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084806+0000","last_change":"2024-05-14T09:30:56.782831+0000","last_active":"2024-05-14T09:33:29.084806+0000","last_peered":"2024-05-14T09:33:29.084806+0000","last_clean":"2024-05-14T09:33:29.084806+0000","last_became_active":"2024-05-14T09:30:56.782402+0000","last_became_peered":"2024-05-14T09:30:56.782402+0000","last_unstale":"2024-05-14T09:33:29.084806+0000","last_undegraded":"2024-05-14T09:33:29.084806+0000","last_fullsized":"2024-05-14T09:33:29.084806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:54.603524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523969+0000","last_change":"2024-05-14T09:31:30.296957+0000","last_active":"2024-05-14T09:32:30.523969+0000","last_peered":"2024-05-14T09:32:30.523969+0000","last_clean":"2024-05-14T09:32:30.523969+0000","last_became_active":"2024-05-14T09:31:30.296554+0000","last_became_peered":"2024-05-14T09:31:30.296554+0000","last_unstale":"2024-05-14T09:32:30.523969+0000","last_undegraded":"2024-05-14T09:32:30.523969+0000","last_fullsized":"2024-05-14T09:32:30.523969+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:28.963042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":149,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:35.392945+0000","last_change":"2024-05-14T09:30:53.772731+0000","last_active":"2024-05-14T09:33:35.392945+0000","last_peered":"2024-05-14T09:33:35.392945+0000","last_clean":"2024-05-14T09:33:35.392945+0000","last_became_active":"2024-05-14T09:30:53.771986+0000","last_became_peered":"2024-05-14T09:30:53.771986+0000","last_unstale":"2024-05-14T09:33:35.392945+0000","last_undegraded":"2024-05-14T09:33:35.392945+0000","last_fullsized":"2024-05-14T09:33:35.392945+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:50:00.810937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077545+0000","last_change":"2024-05-14T09:30:56.790503+0000","last_active":"2024-05-14T09:33:29.077545+0000","last_peered":"2024-05-14T09:33:29.077545+0000","last_clean":"2024-05-14T09:33:29.077545+0000","last_became_active":"2024-05-14T09:30:56.790103+0000","last_became_peered":"2024-05-14T09:30:56.790103+0000","last_unstale":"2024-05-14T09:33:29.077545+0000","last_undegraded":"2024-05-14T09:33:29.077545+0000","last_fullsized":"2024-05-14T09:33:29.077545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:00.950794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315462+0000","last_change":"2024-05-14T09:31:08.116475+0000","last_active":"2024-05-14T09:32:30.315462+0000","last_peered":"2024-05-14T09:32:30.315462+0000","last_clean":"2024-05-14T09:32:30.315462+0000","last_became_active":"2024-05-14T09:31:08.116061+0000","last_became_peered":"2024-05-14T09:31:08.116061+0000","last_unstale":"2024-05-14T09:32:30.315462+0000","last_undegraded":"2024-05-14T09:32:30.315462+0000","last_fullsized":"2024-05-14T09:32:30.315462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:07:02.281853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523311+0000","last_change":"2024-05-14T09:30:54.016570+0000","last_active":"2024-05-14T09:32:30.523311+0000","last_peered":"2024-05-14T09:32:30.523311+0000","last_clean":"2024-05-14T09:32:30.523311+0000","last_became_active":"2024-05-14T09:30:54.015987+0000","last_became_peered":"2024-05-14T09:30:54.015987+0000","last_unstale":"2024-05-14T09:32:30.523311+0000","last_undegraded":"2024-05-14T09:32:30.523311+0000","last_fullsized":"2024-05-14T09:32:30.523311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:18:28.694488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081072+0000","last_change":"2024-05-14T09:30:56.800727+0000","last_active":"2024-05-14T09:33:29.081072+0000","last_peered":"2024-05-14T09:33:29.081072+0000","last_clean":"2024-05-14T09:33:29.081072+0000","last_became_active":"2024-05-14T09:30:56.800354+0000","last_became_peered":"2024-05-14T09:30:56.800354+0000","last_unstale":"2024-05-14T09:33:29.081072+0000","last_undegraded":"2024-05-14T09:33:29.081072+0000","last_fullsized":"2024-05-14T09:33:29.081072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:13:07.227577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315777+0000","last_change":"2024-05-14T09:31:08.115685+0000","last_active":"2024-05-14T09:32:30.315777+0000","last_peered":"2024-05-14T09:32:30.315777+0000","last_clean":"2024-05-14T09:32:30.315777+0000","last_became_active":"2024-05-14T09:31:08.114948+0000","last_became_peered":"2024-05-14T09:31:08.114948+0000","last_unstale":"2024-05-14T09:32:30.315777+0000","last_undegraded":"2024-05-14T09:32:30.315777+0000","last_fullsized":"2024-05-14T09:32:30.315777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:33:07.286823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524227+0000","last_change":"2024-05-14T09:30:53.769949+0000","last_active":"2024-05-14T09:32:30.524227+0000","last_peered":"2024-05-14T09:32:30.524227+0000","last_clean":"2024-05-14T09:32:30.524227+0000","last_became_active":"2024-05-14T09:30:53.769327+0000","last_became_peered":"2024-05-14T09:30:53.769327+0000","last_unstale":"2024-05-14T09:32:30.524227+0000","last_undegraded":"2024-05-14T09:32:30.524227+0000","last_fullsized":"2024-05-14T09:32:30.524227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:35:11.850991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077817+0000","last_change":"2024-05-14T09:32:31.015994+0000","last_active":"2024-05-14T09:33:29.077817+0000","last_peered":"2024-05-14T09:33:29.077817+0000","last_clean":"2024-05-14T09:33:29.077817+0000","last_became_active":"2024-05-14T09:32:31.015515+0000","last_became_peered":"2024-05-14T09:32:31.015515+0000","last_unstale":"2024-05-14T09:33:29.077817+0000","last_undegraded":"2024-05-14T09:33:29.077817+0000","last_fullsized":"2024-05-14T09:33:29.077817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:21:49.811521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7],"acting":[11,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524005+0000","last_change":"2024-05-14T09:31:08.387316+0000","last_active":"2024-05-14T09:32:30.524005+0000","last_peered":"2024-05-14T09:32:30.524005+0000","last_clean":"2024-05-14T09:32:30.524005+0000","last_became_active":"2024-05-14T09:31:08.386739+0000","last_became_peered":"2024-05-14T09:31:08.386739+0000","last_unstale":"2024-05-14T09:32:30.524005+0000","last_undegraded":"2024-05-14T09:32:30.524005+0000","last_fullsized":"2024-05-14T09:32:30.524005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:17.870255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.318704+0000","last_change":"2024-05-14T09:30:54.017756+0000","last_active":"2024-05-14T09:32:30.318704+0000","last_peered":"2024-05-14T09:32:30.318704+0000","last_clean":"2024-05-14T09:32:30.318704+0000","last_became_active":"2024-05-14T09:30:54.017277+0000","last_became_peered":"2024-05-14T09:30:54.017277+0000","last_unstale":"2024-05-14T09:32:30.318704+0000","last_undegraded":"2024-05-14T09:32:30.318704+0000","last_fullsized":"2024-05-14T09:32:30.318704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:27:36.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.072775+0000","last_change":"2024-05-14T09:30:56.788408+0000","last_active":"2024-05-14T09:33:29.072775+0000","last_peered":"2024-05-14T09:33:29.072775+0000","last_clean":"2024-05-14T09:33:29.072775+0000","last_became_active":"2024-05-14T09:30:56.787955+0000","last_became_peered":"2024-05-14T09:30:56.787955+0000","last_unstale":"2024-05-14T09:33:29.072775+0000","last_undegraded":"2024-05-14T09:33:29.072775+0000","last_fullsized":"2024-05-14T09:33:29.072775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:09.777069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315137+0000","last_change":"2024-05-14T09:31:08.119078+0000","last_active":"2024-05-14T09:32:30.315137+0000","last_peered":"2024-05-14T09:32:30.315137+0000","last_clean":"2024-05-14T09:32:30.315137+0000","last_became_active":"2024-05-14T09:31:08.117939+0000","last_became_peered":"2024-05-14T09:31:08.117939+0000","last_unstale":"2024-05-14T09:32:30.315137+0000","last_undegraded":"2024-05-14T09:32:30.315137+0000","last_fullsized":"2024-05-14T09:32:30.315137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:48:37.363503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014384+0000","last_change":"2024-05-14T09:30:53.767843+0000","last_active":"2024-05-14T09:32:31.014384+0000","last_peered":"2024-05-14T09:32:31.014384+0000","last_clean":"2024-05-14T09:32:31.014384+0000","last_became_active":"2024-05-14T09:30:53.767419+0000","last_became_peered":"2024-05-14T09:30:53.767419+0000","last_unstale":"2024-05-14T09:32:31.014384+0000","last_undegraded":"2024-05-14T09:32:31.014384+0000","last_fullsized":"2024-05-14T09:32:31.014384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:49:06.441600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077073+0000","last_change":"2024-05-14T09:30:56.796001+0000","last_active":"2024-05-14T09:33:29.077073+0000","last_peered":"2024-05-14T09:33:29.077073+0000","last_clean":"2024-05-14T09:33:29.077073+0000","last_became_active":"2024-05-14T09:30:56.794517+0000","last_became_peered":"2024-05-14T09:30:56.794517+0000","last_unstale":"2024-05-14T09:33:29.077073+0000","last_undegraded":"2024-05-14T09:33:29.077073+0000","last_fullsized":"2024-05-14T09:33:29.077073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:20:37.514679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315309+0000","last_change":"2024-05-14T09:31:08.387570+0000","last_active":"2024-05-14T09:32:30.315309+0000","last_peered":"2024-05-14T09:32:30.315309+0000","last_clean":"2024-05-14T09:32:30.315309+0000","last_became_active":"2024-05-14T09:31:08.386745+0000","last_became_peered":"2024-05-14T09:31:08.386745+0000","last_unstale":"2024-05-14T09:32:30.315309+0000","last_undegraded":"2024-05-14T09:32:30.315309+0000","last_fullsized":"2024-05-14T09:32:30.315309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:44:01.024784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.060037+0000","last_change":"2024-05-14T09:30:56.795532+0000","last_active":"2024-05-14T09:33:29.060037+0000","last_peered":"2024-05-14T09:33:29.060037+0000","last_clean":"2024-05-14T09:33:29.060037+0000","last_became_active":"2024-05-14T09:30:56.795171+0000","last_became_peered":"2024-05-14T09:30:56.795171+0000","last_unstale":"2024-05-14T09:33:29.060037+0000","last_undegraded":"2024-05-14T09:33:29.060037+0000","last_fullsized":"2024-05-14T09:33:29.060037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:09:40.252489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":149,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:36.393909+0000","last_change":"2024-05-14T09:30:54.015629+0000","last_active":"2024-05-14T09:33:36.393909+0000","last_peered":"2024-05-14T09:33:36.393909+0000","last_clean":"2024-05-14T09:33:36.393909+0000","last_became_active":"2024-05-14T09:30:54.015288+0000","last_became_peered":"2024-05-14T09:30:54.015288+0000","last_unstale":"2024-05-14T09:33:36.393909+0000","last_undegraded":"2024-05-14T09:33:36.393909+0000","last_fullsized":"2024-05-14T09:33:36.393909+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:50.316125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014777+0000","last_change":"2024-05-14T09:31:30.179056+0000","last_active":"2024-05-14T09:32:31.014777+0000","last_peered":"2024-05-14T09:32:31.014777+0000","last_clean":"2024-05-14T09:32:31.014777+0000","last_became_active":"2024-05-14T09:31:30.178398+0000","last_became_peered":"2024-05-14T09:31:30.178398+0000","last_unstale":"2024-05-14T09:32:31.014777+0000","last_undegraded":"2024-05-14T09:32:31.014777+0000","last_fullsized":"2024-05-14T09:32:31.014777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:15:55.369801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077759+0000","last_change":"2024-05-14T09:30:56.793642+0000","last_active":"2024-05-14T09:33:29.077759+0000","last_peered":"2024-05-14T09:33:29.077759+0000","last_clean":"2024-05-14T09:33:29.077759+0000","last_became_active":"2024-05-14T09:30:56.793229+0000","last_became_peered":"2024-05-14T09:30:56.793229+0000","last_unstale":"2024-05-14T09:33:29.077759+0000","last_undegraded":"2024-05-14T09:33:29.077759+0000","last_fullsized":"2024-05-14T09:33:29.077759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:17.940674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522946+0000","last_change":"2024-05-14T09:30:53.769582+0000","last_active":"2024-05-14T09:32:30.522946+0000","last_peered":"2024-05-14T09:32:30.522946+0000","last_clean":"2024-05-14T09:32:30.522946+0000","last_became_active":"2024-05-14T09:30:53.769113+0000","last_became_peered":"2024-05-14T09:30:53.769113+0000","last_unstale":"2024-05-14T09:32:30.522946+0000","last_undegraded":"2024-05-14T09:32:30.522946+0000","last_fullsized":"2024-05-14T09:32:30.522946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:48.755113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014072+0000","last_change":"2024-05-14T09:29:30.767389+0000","last_active":"2024-05-14T09:32:31.014072+0000","last_peered":"2024-05-14T09:32:31.014072+0000","last_clean":"2024-05-14T09:32:31.014072+0000","last_became_active":"2024-05-14T09:29:30.761827+0000","last_became_peered":"2024-05-14T09:29:30.761827+0000","last_unstale":"2024-05-14T09:32:31.014072+0000","last_undegraded":"2024-05-14T09:32:31.014072+0000","last_fullsized":"2024-05-14T09:32:31.014072+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:26:03.741821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:26:03.741821+0000","last_clean_scrub_stamp":"2024-05-14T09:26:03.741821+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:52:11.421658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316444+0000","last_change":"2024-05-14T09:31:08.387819+0000","last_active":"2024-05-14T09:32:30.316444+0000","last_peered":"2024-05-14T09:32:30.316444+0000","last_clean":"2024-05-14T09:32:30.316444+0000","last_became_active":"2024-05-14T09:31:08.387275+0000","last_became_peered":"2024-05-14T09:31:08.387275+0000","last_unstale":"2024-05-14T09:32:30.316444+0000","last_undegraded":"2024-05-14T09:32:30.316444+0000","last_fullsized":"2024-05-14T09:32:30.316444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:46.627389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.078055+0000","last_change":"2024-05-14T09:30:56.788159+0000","last_active":"2024-05-14T09:33:29.078055+0000","last_peered":"2024-05-14T09:33:29.078055+0000","last_clean":"2024-05-14T09:33:29.078055+0000","last_became_active":"2024-05-14T09:30:56.787744+0000","last_became_peered":"2024-05-14T09:30:56.787744+0000","last_unstale":"2024-05-14T09:33:29.078055+0000","last_undegraded":"2024-05-14T09:33:29.078055+0000","last_fullsized":"2024-05-14T09:33:29.078055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:45:25.243471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014462+0000","last_change":"2024-05-14T09:30:53.774345+0000","last_active":"2024-05-14T09:32:31.014462+0000","last_peered":"2024-05-14T09:32:31.014462+0000","last_clean":"2024-05-14T09:32:31.014462+0000","last_became_active":"2024-05-14T09:30:53.773853+0000","last_became_peered":"2024-05-14T09:30:53.773853+0000","last_unstale":"2024-05-14T09:32:31.014462+0000","last_undegraded":"2024-05-14T09:32:31.014462+0000","last_fullsized":"2024-05-14T09:32:31.014462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:35:31.938442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320387+0000","last_change":"2024-05-14T09:31:08.122876+0000","last_active":"2024-05-14T09:32:30.320387+0000","last_peered":"2024-05-14T09:32:30.320387+0000","last_clean":"2024-05-14T09:32:30.320387+0000","last_became_active":"2024-05-14T09:31:08.122182+0000","last_became_peered":"2024-05-14T09:31:08.122182+0000","last_unstale":"2024-05-14T09:32:30.320387+0000","last_undegraded":"2024-05-14T09:32:30.320387+0000","last_fullsized":"2024-05-14T09:32:30.320387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:47:30.735057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524333+0000","last_change":"2024-05-14T09:30:54.018168+0000","last_active":"2024-05-14T09:32:30.524333+0000","last_peered":"2024-05-14T09:32:30.524333+0000","last_clean":"2024-05-14T09:32:30.524333+0000","last_became_active":"2024-05-14T09:30:54.017699+0000","last_became_peered":"2024-05-14T09:30:54.017699+0000","last_unstale":"2024-05-14T09:32:30.524333+0000","last_undegraded":"2024-05-14T09:32:30.524333+0000","last_fullsized":"2024-05-14T09:32:30.524333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:31:01.162000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.070935+0000","last_change":"2024-05-14T09:30:56.787156+0000","last_active":"2024-05-14T09:33:29.070935+0000","last_peered":"2024-05-14T09:33:29.070935+0000","last_clean":"2024-05-14T09:33:29.070935+0000","last_became_active":"2024-05-14T09:30:56.786832+0000","last_became_peered":"2024-05-14T09:30:56.786832+0000","last_unstale":"2024-05-14T09:33:29.070935+0000","last_undegraded":"2024-05-14T09:33:29.070935+0000","last_fullsized":"2024-05-14T09:33:29.070935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:00:41.719641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.321913+0000","last_change":"2024-05-14T09:31:08.387907+0000","last_active":"2024-05-14T09:32:30.321913+0000","last_peered":"2024-05-14T09:32:30.321913+0000","last_clean":"2024-05-14T09:32:30.321913+0000","last_became_active":"2024-05-14T09:31:08.386765+0000","last_became_peered":"2024-05-14T09:31:08.386765+0000","last_unstale":"2024-05-14T09:32:30.321913+0000","last_undegraded":"2024-05-14T09:32:30.321913+0000","last_fullsized":"2024-05-14T09:32:30.321913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:24:56.890353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'12","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:05.982882+0000","last_change":"2024-05-14T09:30:53.767508+0000","last_active":"2024-05-14T09:33:05.982882+0000","last_peered":"2024-05-14T09:33:05.982882+0000","last_clean":"2024-05-14T09:33:05.982882+0000","last_became_active":"2024-05-14T09:30:53.767169+0000","last_became_peered":"2024-05-14T09:30:53.767169+0000","last_unstale":"2024-05-14T09:33:05.982882+0000","last_undegraded":"2024-05-14T09:33:05.982882+0000","last_fullsized":"2024-05-14T09:33:05.982882+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:04:23.621686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076780+0000","last_change":"2024-05-14T09:30:56.790398+0000","last_active":"2024-05-14T09:33:29.076780+0000","last_peered":"2024-05-14T09:33:29.076780+0000","last_clean":"2024-05-14T09:33:29.076780+0000","last_became_active":"2024-05-14T09:30:56.790055+0000","last_became_peered":"2024-05-14T09:30:56.790055+0000","last_unstale":"2024-05-14T09:33:29.076780+0000","last_undegraded":"2024-05-14T09:33:29.076780+0000","last_fullsized":"2024-05-14T09:33:29.076780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:12:37.688647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319239+0000","last_change":"2024-05-14T09:31:08.111533+0000","last_active":"2024-05-14T09:32:30.319239+0000","last_peered":"2024-05-14T09:32:30.319239+0000","last_clean":"2024-05-14T09:32:30.319239+0000","last_became_active":"2024-05-14T09:31:08.110448+0000","last_became_peered":"2024-05-14T09:31:08.110448+0000","last_unstale":"2024-05-14T09:32:30.319239+0000","last_undegraded":"2024-05-14T09:32:30.319239+0000","last_fullsized":"2024-05-14T09:32:30.319239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:43:45.822382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523626+0000","last_change":"2024-05-14T09:30:54.069180+0000","last_active":"2024-05-14T09:32:30.523626+0000","last_peered":"2024-05-14T09:32:30.523626+0000","last_clean":"2024-05-14T09:32:30.523626+0000","last_became_active":"2024-05-14T09:30:54.068846+0000","last_became_peered":"2024-05-14T09:30:54.068846+0000","last_unstale":"2024-05-14T09:32:30.523626+0000","last_undegraded":"2024-05-14T09:32:30.523626+0000","last_fullsized":"2024-05-14T09:32:30.523626+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:22.541720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080401+0000","last_change":"2024-05-14T09:30:56.788159+0000","last_active":"2024-05-14T09:33:29.080401+0000","last_peered":"2024-05-14T09:33:29.080401+0000","last_clean":"2024-05-14T09:33:29.080401+0000","last_became_active":"2024-05-14T09:30:56.787786+0000","last_became_peered":"2024-05-14T09:30:56.787786+0000","last_unstale":"2024-05-14T09:33:29.080401+0000","last_undegraded":"2024-05-14T09:33:29.080401+0000","last_fullsized":"2024-05-14T09:33:29.080401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:24:51.785543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523393+0000","last_change":"2024-05-14T09:31:08.388201+0000","last_active":"2024-05-14T09:32:30.523393+0000","last_peered":"2024-05-14T09:32:30.523393+0000","last_clean":"2024-05-14T09:32:30.523393+0000","last_became_active":"2024-05-14T09:31:08.387572+0000","last_became_peered":"2024-05-14T09:31:08.387572+0000","last_unstale":"2024-05-14T09:32:30.523393+0000","last_undegraded":"2024-05-14T09:32:30.523393+0000","last_fullsized":"2024-05-14T09:32:30.523393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:25:13.171350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316020+0000","last_change":"2024-05-14T09:30:54.016524+0000","last_active":"2024-05-14T09:32:30.316020+0000","last_peered":"2024-05-14T09:32:30.316020+0000","last_clean":"2024-05-14T09:32:30.316020+0000","last_became_active":"2024-05-14T09:30:54.016154+0000","last_became_peered":"2024-05-14T09:30:54.016154+0000","last_unstale":"2024-05-14T09:32:30.316020+0000","last_undegraded":"2024-05-14T09:32:30.316020+0000","last_fullsized":"2024-05-14T09:32:30.316020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:43:16.671053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.035477+0000","last_change":"2024-05-14T09:30:56.796748+0000","last_active":"2024-05-14T09:33:29.035477+0000","last_peered":"2024-05-14T09:33:29.035477+0000","last_clean":"2024-05-14T09:33:29.035477+0000","last_became_active":"2024-05-14T09:30:56.796354+0000","last_became_peered":"2024-05-14T09:30:56.796354+0000","last_unstale":"2024-05-14T09:33:29.035477+0000","last_undegraded":"2024-05-14T09:33:29.035477+0000","last_fullsized":"2024-05-14T09:33:29.035477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:30:15.820677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319112+0000","last_change":"2024-05-14T09:31:08.117266+0000","last_active":"2024-05-14T09:32:30.319112+0000","last_peered":"2024-05-14T09:32:30.319112+0000","last_clean":"2024-05-14T09:32:30.319112+0000","last_became_active":"2024-05-14T09:31:08.115393+0000","last_became_peered":"2024-05-14T09:31:08.115393+0000","last_unstale":"2024-05-14T09:32:30.319112+0000","last_undegraded":"2024-05-14T09:32:30.319112+0000","last_fullsized":"2024-05-14T09:32:30.319112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:29:26.348818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.322687+0000","last_change":"2024-05-14T09:30:53.772417+0000","last_active":"2024-05-14T09:32:30.322687+0000","last_peered":"2024-05-14T09:32:30.322687+0000","last_clean":"2024-05-14T09:32:30.322687+0000","last_became_active":"2024-05-14T09:30:53.772028+0000","last_became_peered":"2024-05-14T09:30:53.772028+0000","last_unstale":"2024-05-14T09:32:30.322687+0000","last_undegraded":"2024-05-14T09:32:30.322687+0000","last_fullsized":"2024-05-14T09:32:30.322687+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:16:16.603957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077309+0000","last_change":"2024-05-14T09:30:56.804750+0000","last_active":"2024-05-14T09:33:29.077309+0000","last_peered":"2024-05-14T09:33:29.077309+0000","last_clean":"2024-05-14T09:33:29.077309+0000","last_became_active":"2024-05-14T09:30:56.803990+0000","last_became_peered":"2024-05-14T09:30:56.803990+0000","last_unstale":"2024-05-14T09:33:29.077309+0000","last_undegraded":"2024-05-14T09:33:29.077309+0000","last_fullsized":"2024-05-14T09:33:29.077309+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:42:00.661033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523484+0000","last_change":"2024-05-14T09:31:08.116061+0000","last_active":"2024-05-14T09:32:30.523484+0000","last_peered":"2024-05-14T09:32:30.523484+0000","last_clean":"2024-05-14T09:32:30.523484+0000","last_became_active":"2024-05-14T09:31:08.115578+0000","last_became_peered":"2024-05-14T09:31:08.115578+0000","last_unstale":"2024-05-14T09:32:30.523484+0000","last_undegraded":"2024-05-14T09:32:30.523484+0000","last_fullsized":"2024-05-14T09:32:30.523484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:18.931035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319214+0000","last_change":"2024-05-14T09:30:53.768988+0000","last_active":"2024-05-14T09:32:30.319214+0000","last_peered":"2024-05-14T09:32:30.319214+0000","last_clean":"2024-05-14T09:32:30.319214+0000","last_became_active":"2024-05-14T09:30:53.768529+0000","last_became_peered":"2024-05-14T09:30:53.768529+0000","last_unstale":"2024-05-14T09:32:30.319214+0000","last_undegraded":"2024-05-14T09:32:30.319214+0000","last_fullsized":"2024-05-14T09:32:30.319214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:17:02.381447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.079500+0000","last_change":"2024-05-14T09:30:56.797110+0000","last_active":"2024-05-14T09:33:29.079500+0000","last_peered":"2024-05-14T09:33:29.079500+0000","last_clean":"2024-05-14T09:33:29.079500+0000","last_became_active":"2024-05-14T09:30:56.796015+0000","last_became_peered":"2024-05-14T09:30:56.796015+0000","last_unstale":"2024-05-14T09:33:29.079500+0000","last_undegraded":"2024-05-14T09:33:29.079500+0000","last_fullsized":"2024-05-14T09:33:29.079500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:37:44.674607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523289+0000","last_change":"2024-05-14T09:31:08.387240+0000","last_active":"2024-05-14T09:32:30.523289+0000","last_peered":"2024-05-14T09:32:30.523289+0000","last_clean":"2024-05-14T09:32:30.523289+0000","last_became_active":"2024-05-14T09:31:08.386653+0000","last_became_peered":"2024-05-14T09:31:08.386653+0000","last_unstale":"2024-05-14T09:32:30.523289+0000","last_undegraded":"2024-05-14T09:32:30.523289+0000","last_fullsized":"2024-05-14T09:32:30.523289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:18:43.408030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523577+0000","last_change":"2024-05-14T09:30:53.772723+0000","last_active":"2024-05-14T09:32:30.523577+0000","last_peered":"2024-05-14T09:32:30.523577+0000","last_clean":"2024-05-14T09:32:30.523577+0000","last_became_active":"2024-05-14T09:30:53.772056+0000","last_became_peered":"2024-05-14T09:30:53.772056+0000","last_unstale":"2024-05-14T09:32:30.523577+0000","last_undegraded":"2024-05-14T09:32:30.523577+0000","last_fullsized":"2024-05-14T09:32:30.523577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:37.941282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087926+0000","last_change":"2024-05-14T09:30:56.799223+0000","last_active":"2024-05-14T09:33:29.087926+0000","last_peered":"2024-05-14T09:33:29.087926+0000","last_clean":"2024-05-14T09:33:29.087926+0000","last_became_active":"2024-05-14T09:30:56.796092+0000","last_became_peered":"2024-05-14T09:30:56.796092+0000","last_unstale":"2024-05-14T09:33:29.087926+0000","last_undegraded":"2024-05-14T09:33:29.087926+0000","last_fullsized":"2024-05-14T09:33:29.087926+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:43:49.746797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316061+0000","last_change":"2024-05-14T09:31:08.115053+0000","last_active":"2024-05-14T09:32:30.316061+0000","last_peered":"2024-05-14T09:32:30.316061+0000","last_clean":"2024-05-14T09:32:30.316061+0000","last_became_active":"2024-05-14T09:31:08.114578+0000","last_became_peered":"2024-05-14T09:31:08.114578+0000","last_unstale":"2024-05-14T09:32:30.316061+0000","last_undegraded":"2024-05-14T09:32:30.316061+0000","last_fullsized":"2024-05-14T09:32:30.316061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:47.788962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.327052+0000","last_change":"2024-05-14T09:32:30.326580+0000","last_active":"2024-05-14T09:32:30.327052+0000","last_peered":"2024-05-14T09:32:30.327052+0000","last_clean":"2024-05-14T09:32:30.327052+0000","last_became_active":"2024-05-14T09:32:30.322052+0000","last_became_peered":"2024-05-14T09:32:30.322052+0000","last_unstale":"2024-05-14T09:32:30.327052+0000","last_undegraded":"2024-05-14T09:32:30.327052+0000","last_fullsized":"2024-05-14T09:32:30.327052+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:04.991674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087235+0000","last_change":"2024-05-14T09:30:56.798580+0000","last_active":"2024-05-14T09:33:29.087235+0000","last_peered":"2024-05-14T09:33:29.087235+0000","last_clean":"2024-05-14T09:33:29.087235+0000","last_became_active":"2024-05-14T09:30:56.795617+0000","last_became_peered":"2024-05-14T09:30:56.795617+0000","last_unstale":"2024-05-14T09:33:29.087235+0000","last_undegraded":"2024-05-14T09:33:29.087235+0000","last_fullsized":"2024-05-14T09:33:29.087235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:30:35.913497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319424+0000","last_change":"2024-05-14T09:31:08.111375+0000","last_active":"2024-05-14T09:32:30.319424+0000","last_peered":"2024-05-14T09:32:30.319424+0000","last_clean":"2024-05-14T09:32:30.319424+0000","last_became_active":"2024-05-14T09:31:08.110931+0000","last_became_peered":"2024-05-14T09:31:08.110931+0000","last_unstale":"2024-05-14T09:32:30.319424+0000","last_undegraded":"2024-05-14T09:32:30.319424+0000","last_fullsized":"2024-05-14T09:32:30.319424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:08:30.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.327490+0000","last_change":"2024-05-14T09:32:30.326979+0000","last_active":"2024-05-14T09:32:30.327490+0000","last_peered":"2024-05-14T09:32:30.327490+0000","last_clean":"2024-05-14T09:32:30.327490+0000","last_became_active":"2024-05-14T09:32:30.323545+0000","last_became_peered":"2024-05-14T09:32:30.323545+0000","last_unstale":"2024-05-14T09:32:30.327490+0000","last_undegraded":"2024-05-14T09:32:30.327490+0000","last_fullsized":"2024-05-14T09:32:30.327490+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:00:21.493910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":70,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.064380+0000","last_change":"2024-05-14T09:30:56.789527+0000","last_active":"2024-05-14T09:33:29.064380+0000","last_peered":"2024-05-14T09:33:29.064380+0000","last_clean":"2024-05-14T09:33:29.064380+0000","last_became_active":"2024-05-14T09:30:56.789075+0000","last_became_peered":"2024-05-14T09:30:56.789075+0000","last_unstale":"2024-05-14T09:33:29.064380+0000","last_undegraded":"2024-05-14T09:33:29.064380+0000","last_fullsized":"2024-05-14T09:33:29.064380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:07.140087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315692+0000","last_change":"2024-05-14T09:31:08.119624+0000","last_active":"2024-05-14T09:32:30.315692+0000","last_peered":"2024-05-14T09:32:30.315692+0000","last_clean":"2024-05-14T09:32:30.315692+0000","last_became_active":"2024-05-14T09:31:08.118102+0000","last_became_peered":"2024-05-14T09:31:08.118102+0000","last_unstale":"2024-05-14T09:32:30.315692+0000","last_undegraded":"2024-05-14T09:32:30.315692+0000","last_fullsized":"2024-05-14T09:32:30.315692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:48:17.806976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315472+0000","last_change":"2024-05-14T09:30:53.763903+0000","last_active":"2024-05-14T09:32:30.315472+0000","last_peered":"2024-05-14T09:32:30.315472+0000","last_clean":"2024-05-14T09:32:30.315472+0000","last_became_active":"2024-05-14T09:30:53.763517+0000","last_became_peered":"2024-05-14T09:30:53.763517+0000","last_unstale":"2024-05-14T09:32:30.315472+0000","last_undegraded":"2024-05-14T09:32:30.315472+0000","last_fullsized":"2024-05-14T09:32:30.315472+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:57.144392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081279+0000","last_change":"2024-05-14T09:30:56.791612+0000","last_active":"2024-05-14T09:33:29.081279+0000","last_peered":"2024-05-14T09:33:29.081279+0000","last_clean":"2024-05-14T09:33:29.081279+0000","last_became_active":"2024-05-14T09:30:56.791178+0000","last_became_peered":"2024-05-14T09:30:56.791178+0000","last_unstale":"2024-05-14T09:33:29.081279+0000","last_undegraded":"2024-05-14T09:33:29.081279+0000","last_fullsized":"2024-05-14T09:33:29.081279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:11:19.058139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076827+0000","last_change":"2024-05-14T09:30:56.804437+0000","last_active":"2024-05-14T09:33:29.076827+0000","last_peered":"2024-05-14T09:33:29.076827+0000","last_clean":"2024-05-14T09:33:29.076827+0000","last_became_active":"2024-05-14T09:30:56.804048+0000","last_became_peered":"2024-05-14T09:30:56.804048+0000","last_unstale":"2024-05-14T09:33:29.076827+0000","last_undegraded":"2024-05-14T09:33:29.076827+0000","last_fullsized":"2024-05-14T09:33:29.076827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:33.788903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.058110+0000","last_change":"2024-05-14T09:30:56.794309+0000","last_active":"2024-05-14T09:33:29.058110+0000","last_peered":"2024-05-14T09:33:29.058110+0000","last_clean":"2024-05-14T09:33:29.058110+0000","last_became_active":"2024-05-14T09:30:56.793317+0000","last_became_peered":"2024-05-14T09:30:56.793317+0000","last_unstale":"2024-05-14T09:33:29.058110+0000","last_undegraded":"2024-05-14T09:33:29.058110+0000","last_fullsized":"2024-05-14T09:33:29.058110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:40:26.878746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080059+0000","last_change":"2024-05-14T09:30:56.793618+0000","last_active":"2024-05-14T09:33:29.080059+0000","last_peered":"2024-05-14T09:33:29.080059+0000","last_clean":"2024-05-14T09:33:29.080059+0000","last_became_active":"2024-05-14T09:30:56.792819+0000","last_became_peered":"2024-05-14T09:30:56.792819+0000","last_unstale":"2024-05-14T09:33:29.080059+0000","last_undegraded":"2024-05-14T09:33:29.080059+0000","last_fullsized":"2024-05-14T09:33:29.080059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:29:38.191617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.072960+0000","last_change":"2024-05-14T09:30:56.794268+0000","last_active":"2024-05-14T09:33:29.072960+0000","last_peered":"2024-05-14T09:33:29.072960+0000","last_clean":"2024-05-14T09:33:29.072960+0000","last_became_active":"2024-05-14T09:30:56.793898+0000","last_became_peered":"2024-05-14T09:30:56.793898+0000","last_unstale":"2024-05-14T09:33:29.072960+0000","last_undegraded":"2024-05-14T09:33:29.072960+0000","last_fullsized":"2024-05-14T09:33:29.072960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:25:35.022236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.078917+0000","last_change":"2024-05-14T09:30:56.783366+0000","last_active":"2024-05-14T09:33:29.078917+0000","last_peered":"2024-05-14T09:33:29.078917+0000","last_clean":"2024-05-14T09:33:29.078917+0000","last_became_active":"2024-05-14T09:30:56.782980+0000","last_became_peered":"2024-05-14T09:30:56.782980+0000","last_unstale":"2024-05-14T09:33:29.078917+0000","last_undegraded":"2024-05-14T09:33:29.078917+0000","last_fullsized":"2024-05-14T09:33:29.078917+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:27.744902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076941+0000","last_change":"2024-05-14T09:32:30.524390+0000","last_active":"2024-05-14T09:33:29.076941+0000","last_peered":"2024-05-14T09:33:29.076941+0000","last_clean":"2024-05-14T09:33:29.076941+0000","last_became_active":"2024-05-14T09:32:30.523926+0000","last_became_peered":"2024-05-14T09:32:30.523926+0000","last_unstale":"2024-05-14T09:33:29.076941+0000","last_undegraded":"2024-05-14T09:33:29.076941+0000","last_fullsized":"2024-05-14T09:33:29.076941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:58:47.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.074545+0000","last_change":"2024-05-14T09:30:56.795872+0000","last_active":"2024-05-14T09:33:29.074545+0000","last_peered":"2024-05-14T09:33:29.074545+0000","last_clean":"2024-05-14T09:33:29.074545+0000","last_became_active":"2024-05-14T09:30:56.795492+0000","last_became_peered":"2024-05-14T09:30:56.795492+0000","last_unstale":"2024-05-14T09:33:29.074545+0000","last_undegraded":"2024-05-14T09:33:29.074545+0000","last_fullsized":"2024-05-14T09:33:29.074545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:26:29.226575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.062634+0000","last_change":"2024-05-14T09:32:30.524189+0000","last_active":"2024-05-14T09:33:29.062634+0000","last_peered":"2024-05-14T09:33:29.062634+0000","last_clean":"2024-05-14T09:33:29.062634+0000","last_became_active":"2024-05-14T09:32:30.523791+0000","last_became_peered":"2024-05-14T09:32:30.523791+0000","last_unstale":"2024-05-14T09:33:29.062634+0000","last_undegraded":"2024-05-14T09:33:29.062634+0000","last_fullsized":"2024-05-14T09:33:29.062634+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:52.441130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080195+0000","last_change":"2024-05-14T09:30:56.792814+0000","last_active":"2024-05-14T09:33:29.080195+0000","last_peered":"2024-05-14T09:33:29.080195+0000","last_clean":"2024-05-14T09:33:29.080195+0000","last_became_active":"2024-05-14T09:30:56.792282+0000","last_became_peered":"2024-05-14T09:30:56.792282+0000","last_unstale":"2024-05-14T09:33:29.080195+0000","last_undegraded":"2024-05-14T09:33:29.080195+0000","last_fullsized":"2024-05-14T09:33:29.080195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:52:35.319479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076831+0000","last_change":"2024-05-14T09:30:56.791575+0000","last_active":"2024-05-14T09:33:29.076831+0000","last_peered":"2024-05-14T09:33:29.076831+0000","last_clean":"2024-05-14T09:33:29.076831+0000","last_became_active":"2024-05-14T09:30:56.791230+0000","last_became_peered":"2024-05-14T09:30:56.791230+0000","last_unstale":"2024-05-14T09:33:29.076831+0000","last_undegraded":"2024-05-14T09:33:29.076831+0000","last_fullsized":"2024-05-14T09:33:29.076831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:40.510062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.088030+0000","last_change":"2024-05-14T09:30:56.796996+0000","last_active":"2024-05-14T09:33:29.088030+0000","last_peered":"2024-05-14T09:33:29.088030+0000","last_clean":"2024-05-14T09:33:29.088030+0000","last_became_active":"2024-05-14T09:30:56.796333+0000","last_became_peered":"2024-05-14T09:30:56.796333+0000","last_unstale":"2024-05-14T09:33:29.088030+0000","last_undegraded":"2024-05-14T09:33:29.088030+0000","last_fullsized":"2024-05-14T09:33:29.088030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:29:34.799088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.083938+0000","last_change":"2024-05-14T09:30:56.785740+0000","last_active":"2024-05-14T09:33:29.083938+0000","last_peered":"2024-05-14T09:33:29.083938+0000","last_clean":"2024-05-14T09:33:29.083938+0000","last_became_active":"2024-05-14T09:30:56.785394+0000","last_became_peered":"2024-05-14T09:30:56.785394+0000","last_unstale":"2024-05-14T09:33:29.083938+0000","last_undegraded":"2024-05-14T09:33:29.083938+0000","last_fullsized":"2024-05-14T09:33:29.083938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:17.819523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.079641+0000","last_change":"2024-05-14T09:30:56.796957+0000","last_active":"2024-05-14T09:33:29.079641+0000","last_peered":"2024-05-14T09:33:29.079641+0000","last_clean":"2024-05-14T09:33:29.079641+0000","last_became_active":"2024-05-14T09:30:56.795071+0000","last_became_peered":"2024-05-14T09:30:56.795071+0000","last_unstale":"2024-05-14T09:33:29.079641+0000","last_undegraded":"2024-05-14T09:33:29.079641+0000","last_fullsized":"2024-05-14T09:33:29.079641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:44:31.480207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245291,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68451,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":627,"num_read_kb":386,"num_write":175,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2846,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678063,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":519087,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874293,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":1174108,"data_compressed":67998,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037819,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":518671,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1597,"internal_metadata":36174275},"hb_peers":[0,1,2,3,4,5,6,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234048,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":518009,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430277,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":1114059,"data_compressed":67998,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593802,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":514429,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":578648,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":515235,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":515057,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149785,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":514041,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953557,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":582806,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790032,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1110159,"data_compressed":67998,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33883,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33883,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":64891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T09:33:43.144 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-14T09:33:43.144 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-14T09:33:43.144 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-14T09:33:43.144 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-14T09:33:43.394 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:43.780 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:43 smithi002 ceph-mon[31557]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:33:43.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:43 smithi029 ceph-mon[36249]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:33:43.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:43 smithi026 ceph-mon[36056]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:33:44.908 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:44 smithi002 ceph-mon[31557]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-14T09:33:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:44 smithi029 ceph-mon[36249]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-14T09:33:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:44 smithi026 ceph-mon[36056]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-14T09:33:45.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.154+0000 7fd2b917b700 1 -- 172.21.15.2:0/4136667520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2b4108740 msgr2=0x7fd2b41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:45.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.154+0000 7fd2b917b700 1 --2- 172.21.15.2:0/4136667520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2b4108740 0x7fd2b41100a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00b0d0 tx=0x7fd2ac00b3e0 comp rx=0 tx=0).stop 2024-05-14T09:33:45.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.154+0000 7fd2b917b700 1 -- 172.21.15.2:0/4136667520 shutdown_connections 2024-05-14T09:33:45.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.154+0000 7fd2b917b700 1 --2- 172.21.15.2:0/4136667520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2b4108740 0x7fd2b41100a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:45.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.154+0000 7fd2b917b700 1 --2- 172.21.15.2:0/4136667520 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2b41034c0 0x7fd2b41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:45.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.154+0000 7fd2b917b700 1 --2- 172.21.15.2:0/4136667520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b4107df0 0x7fd2b4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:45.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.154+0000 7fd2b917b700 1 -- 172.21.15.2:0/4136667520 >> 172.21.15.2:0/4136667520 conn(0x7fd2b40fedf0 msgr2=0x7fd2b4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:45.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.154+0000 7fd2b917b700 1 -- 172.21.15.2:0/4136667520 shutdown_connections 2024-05-14T09:33:45.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.154+0000 7fd2b917b700 1 -- 172.21.15.2:0/4136667520 wait complete. 2024-05-14T09:33:45.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.155+0000 7fd2b917b700 1 Processor -- start 2024-05-14T09:33:45.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.155+0000 7fd2b917b700 1 -- start start 2024-05-14T09:33:45.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.155+0000 7fd2b917b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2b41034c0 0x7fd2b40ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:45.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.155+0000 7fd2b917b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b4107df0 0x7fd2b40ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:45.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.155+0000 7fd2b917b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2b4108740 0x7fd2b40ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:45.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b917b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b4113370 con 0x7fd2b4108740 2024-05-14T09:33:45.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b917b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b41131f0 con 0x7fd2b4107df0 2024-05-14T09:33:45.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b917b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b41134f0 con 0x7fd2b41034c0 2024-05-14T09:33:45.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b359e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2b4108740 0x7fd2b40ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:45.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b359e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2b4108740 0x7fd2b40ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57180/0 (socket says 172.21.15.2:57180) 2024-05-14T09:33:45.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b359e700 1 -- 172.21.15.2:0/372897441 learned_addr learned my addr 172.21.15.2:0/372897441 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:45.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b259c700 1 --2- 172.21.15.2:0/372897441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b4107df0 0x7fd2b40ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:45.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b2d9d700 1 --2- 172.21.15.2:0/372897441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2b41034c0 0x7fd2b40ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:45.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b359e700 1 -- 172.21.15.2:0/372897441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2b41034c0 msgr2=0x7fd2b40ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:45.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b359e700 1 --2- 172.21.15.2:0/372897441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2b41034c0 0x7fd2b40ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:45.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b359e700 1 -- 172.21.15.2:0/372897441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b4107df0 msgr2=0x7fd2b40ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:45.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b359e700 1 --2- 172.21.15.2:0/372897441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b4107df0 0x7fd2b40ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:45.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b359e700 1 -- 172.21.15.2:0/372897441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b41a9130 con 0x7fd2b4108740 2024-05-14T09:33:45.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b2d9d700 1 --2- 172.21.15.2:0/372897441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2b41034c0 0x7fd2b40ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:33:45.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b359e700 1 --2- 172.21.15.2:0/372897441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2b4108740 0x7fd2b40ffd50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0095c0 tx=0x7fd2ac007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:45.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd29bfff700 1 -- 172.21.15.2:0/372897441 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac01f070 con 0x7fd2b4108740 2024-05-14T09:33:45.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd2b917b700 1 -- 172.21.15.2:0/372897441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b41a93c0 con 0x7fd2b4108740 2024-05-14T09:33:45.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.156+0000 7fd29bfff700 1 -- 172.21.15.2:0/372897441 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2ac003c00 con 0x7fd2b4108740 2024-05-14T09:33:45.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.157+0000 7fd29bfff700 1 -- 172.21.15.2:0/372897441 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac01b6a0 con 0x7fd2b4108740 2024-05-14T09:33:45.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.157+0000 7fd2b917b700 1 -- 172.21.15.2:0/372897441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b41a9870 con 0x7fd2b4108740 2024-05-14T09:33:45.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.158+0000 7fd2b917b700 1 -- 172.21.15.2:0/372897441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b40512c0 con 0x7fd2b4108740 2024-05-14T09:33:45.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.159+0000 7fd29bfff700 1 -- 172.21.15.2:0/372897441 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd2ac022070 con 0x7fd2b4108740 2024-05-14T09:33:45.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.160+0000 7fd29bfff700 1 --2- 172.21.15.2:0/372897441 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd29c0b2880 0x7fd29c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:45.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.160+0000 7fd29bfff700 1 -- 172.21.15.2:0/372897441 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd2ac00f020 con 0x7fd2b4108740 2024-05-14T09:33:45.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.160+0000 7fd2b2d9d700 1 --2- 172.21.15.2:0/372897441 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd29c0b2880 0x7fd29c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:45.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.161+0000 7fd2b2d9d700 1 --2- 172.21.15.2:0/372897441 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd29c0b2880 0x7fd29c0b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4005950 tx=0x7fd2a40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:45.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.165+0000 7fd29bfff700 1 -- 172.21.15.2:0/372897441 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd2ac029020 con 0x7fd2b4108740 2024-05-14T09:33:45.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.425+0000 7fd2b917b700 1 -- 172.21.15.2:0/372897441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd2b402e5f0 con 0x7fd2b4108740 2024-05-14T09:33:45.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.425+0000 7fd29bfff700 1 -- 172.21.15.2:0/372897441 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd2ac0ab260 con 0x7fd2b4108740 2024-05-14T09:33:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:33:45.426 INFO:teuthology.orchestra.run.smithi002.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-14T09:33:45.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.427+0000 7fd2b917b700 1 -- 172.21.15.2:0/372897441 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd29c0b2880 msgr2=0x7fd29c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:45.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.427+0000 7fd2b917b700 1 --2- 172.21.15.2:0/372897441 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd29c0b2880 0x7fd29c0b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4005950 tx=0x7fd2a40058e0 comp rx=0 tx=0).stop 2024-05-14T09:33:45.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.428+0000 7fd2b917b700 1 -- 172.21.15.2:0/372897441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2b4108740 msgr2=0x7fd2b40ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:45.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.428+0000 7fd2b917b700 1 --2- 172.21.15.2:0/372897441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2b4108740 0x7fd2b40ffd50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0095c0 tx=0x7fd2ac007a20 comp rx=0 tx=0).stop 2024-05-14T09:33:45.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.428+0000 7fd2b917b700 1 -- 172.21.15.2:0/372897441 shutdown_connections 2024-05-14T09:33:45.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.428+0000 7fd2b917b700 1 --2- 172.21.15.2:0/372897441 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd29c0b2880 0x7fd29c0b4d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:45.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.428+0000 7fd2b917b700 1 --2- 172.21.15.2:0/372897441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2b4108740 0x7fd2b40ffd50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:45.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.428+0000 7fd2b917b700 1 --2- 172.21.15.2:0/372897441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd2b41034c0 0x7fd2b40ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:45.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.428+0000 7fd2b917b700 1 --2- 172.21.15.2:0/372897441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b4107df0 0x7fd2b40ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:45.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.428+0000 7fd2b917b700 1 -- 172.21.15.2:0/372897441 >> 172.21.15.2:0/372897441 conn(0x7fd2b40fedf0 msgr2=0x7fd2b4075c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:45.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.428+0000 7fd2b917b700 1 -- 172.21.15.2:0/372897441 shutdown_connections 2024-05-14T09:33:45.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:45.428+0000 7fd2b917b700 1 -- 172.21.15.2:0/372897441 wait complete. 2024-05-14T09:33:45.788 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:45 smithi002 ceph-mon[31557]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:45.788 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:45 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/372897441' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-14T09:33:45.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:45 smithi029 ceph-mon[36249]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:45.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:45 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/372897441' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-14T09:33:45.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:45 smithi026 ceph-mon[36056]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:45.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:45 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/372897441' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-14T09:33:46.153 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-14T09:33:46.153 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:33:46.412 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:48.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.375+0000 7f6cb36e0700 1 -- 172.21.15.2:0/1693265287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cac108f10 msgr2=0x7f6cac109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:48.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.376+0000 7f6cb36e0700 1 --2- 172.21.15.2:0/1693265287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cac108f10 0x7f6cac109320 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0099c0 tx=0x7f6c9c009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:48.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.376+0000 7f6cb36e0700 1 -- 172.21.15.2:0/1693265287 shutdown_connections 2024-05-14T09:33:48.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.376+0000 7f6cb36e0700 1 --2- 172.21.15.2:0/1693265287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac1045e0 0x7f6cac1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:48.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.376+0000 7f6cb36e0700 1 --2- 172.21.15.2:0/1693265287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cac109860 0x7f6cac1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:48.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.376+0000 7f6cb36e0700 1 --2- 172.21.15.2:0/1693265287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cac108f10 0x7f6cac109320 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:48.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.376+0000 7f6cb36e0700 1 -- 172.21.15.2:0/1693265287 >> 172.21.15.2:0/1693265287 conn(0x7f6cac0ffed0 msgr2=0x7f6cac102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:48.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.376+0000 7f6cb36e0700 1 -- 172.21.15.2:0/1693265287 shutdown_connections 2024-05-14T09:33:48.377 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:48 smithi002 ceph-mon[31557]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:33:48.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.376+0000 7f6cb36e0700 1 -- 172.21.15.2:0/1693265287 wait complete. 2024-05-14T09:33:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.377+0000 7f6cb36e0700 1 Processor -- start 2024-05-14T09:33:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.377+0000 7f6cb36e0700 1 -- start start 2024-05-14T09:33:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.377+0000 7f6cb36e0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cac1045e0 0x7f6cac076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.377+0000 7f6cb36e0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cac108f10 0x7f6cac077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.377+0000 7f6cb36e0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac109860 0x7f6cac077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.377+0000 7f6cb36e0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1144c0 con 0x7f6cac108f10 2024-05-14T09:33:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.377+0000 7f6cb36e0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac114340 con 0x7f6cac109860 2024-05-14T09:33:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.377+0000 7f6cb36e0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac114640 con 0x7f6cac1045e0 2024-05-14T09:33:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.377+0000 7f6cb0c7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cac108f10 0x7f6cac077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.377+0000 7f6cb147c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cac1045e0 0x7f6cac076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.377+0000 7f6cb0c7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cac108f10 0x7f6cac077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57512/0 (socket says 172.21.15.2:57512) 2024-05-14T09:33:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.377+0000 7f6cb0c7b700 1 -- 172.21.15.2:0/2505739535 learned_addr learned my addr 172.21.15.2:0/2505739535 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:48.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.378+0000 7f6cb1c7d700 1 --2- 172.21.15.2:0/2505739535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac109860 0x7f6cac077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.378+0000 7f6cb147c700 1 -- 172.21.15.2:0/2505739535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac109860 msgr2=0x7f6cac077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.378+0000 7f6cb147c700 1 --2- 172.21.15.2:0/2505739535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac109860 0x7f6cac077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.378+0000 7f6cb147c700 1 -- 172.21.15.2:0/2505739535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cac108f10 msgr2=0x7f6cac077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.378+0000 7f6cb147c700 1 --2- 172.21.15.2:0/2505739535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cac108f10 0x7f6cac077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.378+0000 7f6cb147c700 1 -- 172.21.15.2:0/2505739535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cac1a91f0 con 0x7f6cac1045e0 2024-05-14T09:33:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.378+0000 7f6cb0c7b700 1 --2- 172.21.15.2:0/2505739535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cac108f10 0x7f6cac077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:33:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.378+0000 7f6cb147c700 1 --2- 172.21.15.2:0/2505739535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cac1045e0 0x7f6cac076bd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8007ec0 tx=0x7f6ca800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.378+0000 7f6ca27fc700 1 -- 172.21.15.2:0/2505739535 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8004db0 con 0x7f6cac1045e0 2024-05-14T09:33:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.378+0000 7f6cb36e0700 1 -- 172.21.15.2:0/2505739535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cac1a94e0 con 0x7f6cac1045e0 2024-05-14T09:33:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.378+0000 7f6ca27fc700 1 -- 172.21.15.2:0/2505739535 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca8005020 con 0x7f6cac1045e0 2024-05-14T09:33:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.379+0000 7f6ca27fc700 1 -- 172.21.15.2:0/2505739535 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8015660 con 0x7f6cac1045e0 2024-05-14T09:33:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.379+0000 7f6cb36e0700 1 -- 172.21.15.2:0/2505739535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cac1a9a70 con 0x7f6cac1045e0 2024-05-14T09:33:48.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.379+0000 7f6cb36e0700 1 -- 172.21.15.2:0/2505739535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cac0512c0 con 0x7f6cac1045e0 2024-05-14T09:33:48.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.381+0000 7f6ca27fc700 1 -- 172.21.15.2:0/2505739535 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6ca80054d0 con 0x7f6cac1045e0 2024-05-14T09:33:48.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.382+0000 7f6ca27fc700 1 --2- 172.21.15.2:0/2505739535 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6c980b2880 0x7f6c980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:48.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.382+0000 7f6cb0c7b700 1 --2- 172.21.15.2:0/2505739535 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6c980b2880 0x7f6c980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:48.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.382+0000 7f6ca27fc700 1 -- 172.21.15.2:0/2505739535 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f6ca80e2820 con 0x7f6cac1045e0 2024-05-14T09:33:48.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.383+0000 7f6cb0c7b700 1 --2- 172.21.15.2:0/2505739535 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6c980b2880 0x7f6c980b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6cac1077e0 tx=0x7f6c9c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:48.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.386+0000 7f6ca27fc700 1 -- 172.21.15.2:0/2505739535 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ca8010070 con 0x7f6cac1045e0 2024-05-14T09:33:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:48 smithi029 ceph-mon[36249]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:33:48.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:48 smithi026 ceph-mon[36056]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:33:48.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.646+0000 7f6cb36e0700 1 -- 172.21.15.2:0/2505739535 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cac02e5f0 con 0x7f6cac1045e0 2024-05-14T09:33:48.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.647+0000 7f6ca27fc700 1 -- 172.21.15.2:0/2505739535 <== mon.1 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11843 (secure 0 0 0) 0x7f6ca80aa0c0 con 0x7f6cac1045e0 2024-05-14T09:33:48.647 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:33:48.647 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T09:32:01.167752+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24497,24524,24487]},"id":1}]} 2024-05-14T09:33:48.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.649+0000 7f6cb36e0700 1 -- 172.21.15.2:0/2505739535 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6c980b2880 msgr2=0x7f6c980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:48.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.649+0000 7f6cb36e0700 1 --2- 172.21.15.2:0/2505739535 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6c980b2880 0x7f6c980b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6cac1077e0 tx=0x7f6c9c01a040 comp rx=0 tx=0).stop 2024-05-14T09:33:48.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.649+0000 7f6cb36e0700 1 -- 172.21.15.2:0/2505739535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cac1045e0 msgr2=0x7f6cac076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:48.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.649+0000 7f6cb36e0700 1 --2- 172.21.15.2:0/2505739535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cac1045e0 0x7f6cac076bd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8007ec0 tx=0x7f6ca800d3b0 comp rx=0 tx=0).stop 2024-05-14T09:33:48.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.649+0000 7f6cb36e0700 1 -- 172.21.15.2:0/2505739535 shutdown_connections 2024-05-14T09:33:48.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.649+0000 7f6cb36e0700 1 --2- 172.21.15.2:0/2505739535 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6c980b2880 0x7f6c980b4d90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:48.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.649+0000 7f6cb36e0700 1 --2- 172.21.15.2:0/2505739535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac109860 0x7f6cac077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:48.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.649+0000 7f6cb36e0700 1 --2- 172.21.15.2:0/2505739535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6cac1045e0 0x7f6cac076bd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:48.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.649+0000 7f6cb36e0700 1 --2- 172.21.15.2:0/2505739535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cac108f10 0x7f6cac077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:48.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.649+0000 7f6cb36e0700 1 -- 172.21.15.2:0/2505739535 >> 172.21.15.2:0/2505739535 conn(0x7f6cac0ffed0 msgr2=0x7f6cac10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:48.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.649+0000 7f6cb36e0700 1 -- 172.21.15.2:0/2505739535 shutdown_connections 2024-05-14T09:33:48.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:48.649+0000 7f6cb36e0700 1 -- 172.21.15.2:0/2505739535 wait complete. 2024-05-14T09:33:48.651 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-05-14T09:33:49.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:49 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2505739535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:33:49.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:49 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2505739535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:33:49.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:49 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2505739535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:33:49.494 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:33:49.750 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:50 smithi002 ceph-mon[31557]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:50.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:50 smithi029 ceph-mon[36249]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:50.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:50 smithi026 ceph-mon[36056]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:51.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.539+0000 7f5cb0e43700 1 -- 172.21.15.2:0/2028437629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cac108970 msgr2=0x7f5cac1102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:51.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.539+0000 7f5cb0e43700 1 --2- 172.21.15.2:0/2028437629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cac108970 0x7f5cac1102d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40099c0 tx=0x7f5ca4009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:51.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.539+0000 7f5cb0e43700 1 -- 172.21.15.2:0/2028437629 shutdown_connections 2024-05-14T09:33:51.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.539+0000 7f5cb0e43700 1 --2- 172.21.15.2:0/2028437629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cac103c10 0x7f5cac104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.539+0000 7f5cb0e43700 1 --2- 172.21.15.2:0/2028437629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cac108970 0x7f5cac1102d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.539+0000 7f5cb0e43700 1 --2- 172.21.15.2:0/2028437629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cac108020 0x7f5cac108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.539+0000 7f5cb0e43700 1 -- 172.21.15.2:0/2028437629 >> 172.21.15.2:0/2028437629 conn(0x7f5cac0fe540 msgr2=0x7f5cac100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.539+0000 7f5cb0e43700 1 -- 172.21.15.2:0/2028437629 shutdown_connections 2024-05-14T09:33:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.539+0000 7f5cb0e43700 1 -- 172.21.15.2:0/2028437629 wait complete. 2024-05-14T09:33:51.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.540+0000 7f5cb0e43700 1 Processor -- start 2024-05-14T09:33:51.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.540+0000 7f5cb0e43700 1 -- start start 2024-05-14T09:33:51.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.540+0000 7f5cb0e43700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cac103c10 0x7f5cac19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:51.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.540+0000 7f5cb0e43700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cac108020 0x7f5cac19c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:51.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.540+0000 7f5cb0e43700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cac108970 0x7f5cac19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:51.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.540+0000 7f5cb0e43700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac1135a0 con 0x7f5cac108970 2024-05-14T09:33:51.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.540+0000 7f5cb0e43700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac113420 con 0x7f5cac103c10 2024-05-14T09:33:51.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.540+0000 7f5cb0e43700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac113720 con 0x7f5cac108020 2024-05-14T09:33:51.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.540+0000 7f5caad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cac108970 0x7f5cac19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:51.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.540+0000 7f5caa59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cac103c10 0x7f5cac19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:51.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.540+0000 7f5caad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cac108970 0x7f5cac19c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57534/0 (socket says 172.21.15.2:57534) 2024-05-14T09:33:51.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.540+0000 7f5caad9d700 1 -- 172.21.15.2:0/4131408547 learned_addr learned my addr 172.21.15.2:0/4131408547 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:51.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.540+0000 7f5ca9d9b700 1 --2- 172.21.15.2:0/4131408547 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cac108020 0x7f5cac19c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:51.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.541+0000 7f5caad9d700 1 -- 172.21.15.2:0/4131408547 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cac108020 msgr2=0x7f5cac19c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:51.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.541+0000 7f5caad9d700 1 --2- 172.21.15.2:0/4131408547 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cac108020 0x7f5cac19c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:51.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.541+0000 7f5caad9d700 1 -- 172.21.15.2:0/4131408547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cac103c10 msgr2=0x7f5cac19be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:51.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.541+0000 7f5caad9d700 1 --2- 172.21.15.2:0/4131408547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cac103c10 0x7f5cac19be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:51.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.541+0000 7f5caad9d700 1 -- 172.21.15.2:0/4131408547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac1a71b0 con 0x7f5cac108970 2024-05-14T09:33:51.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.541+0000 7f5caad9d700 1 --2- 172.21.15.2:0/4131408547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cac108970 0x7f5cac19c900 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40059d0 tx=0x7f5ca40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:51.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.541+0000 7f5c9b7fe700 1 -- 172.21.15.2:0/4131408547 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4027070 con 0x7f5cac108970 2024-05-14T09:33:51.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.541+0000 7f5cb0e43700 1 -- 172.21.15.2:0/4131408547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cac1a7440 con 0x7f5cac108970 2024-05-14T09:33:51.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.541+0000 7f5c9b7fe700 1 -- 172.21.15.2:0/4131408547 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca4003fe0 con 0x7f5cac108970 2024-05-14T09:33:51.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.541+0000 7f5c9b7fe700 1 -- 172.21.15.2:0/4131408547 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4022b50 con 0x7f5cac108970 2024-05-14T09:33:51.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.541+0000 7f5cb0e43700 1 -- 172.21.15.2:0/4131408547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cac1a78f0 con 0x7f5cac108970 2024-05-14T09:33:51.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.542+0000 7f5cb0e43700 1 -- 172.21.15.2:0/4131408547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cac0512c0 con 0x7f5cac108970 2024-05-14T09:33:51.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.544+0000 7f5c9b7fe700 1 -- 172.21.15.2:0/4131408547 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5ca402a070 con 0x7f5cac108970 2024-05-14T09:33:51.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.545+0000 7f5c9b7fe700 1 --2- 172.21.15.2:0/4131408547 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5c940b2880 0x7f5c940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:51.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.545+0000 7f5caa59c700 1 --2- 172.21.15.2:0/4131408547 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5c940b2880 0x7f5c940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:51.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.545+0000 7f5c9b7fe700 1 -- 172.21.15.2:0/4131408547 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f5ca40e3ea0 con 0x7f5cac108970 2024-05-14T09:33:51.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.545+0000 7f5caa59c700 1 --2- 172.21.15.2:0/4131408547 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5c940b2880 0x7f5c940b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c007be0 tx=0x7f5c9c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:51.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.549+0000 7f5c9b7fe700 1 -- 172.21.15.2:0/4131408547 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ca4031080 con 0x7f5cac108970 2024-05-14T09:33:51.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.809+0000 7f5cb0e43700 1 -- 172.21.15.2:0/4131408547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cac06c160 con 0x7f5cac108970 2024-05-14T09:33:51.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.810+0000 7f5c9b7fe700 1 -- 172.21.15.2:0/4131408547 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11843 (secure 0 0 0) 0x7f5ca4013070 con 0x7f5cac108970 2024-05-14T09:33:51.810 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:33:51.810 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T09:32:01.167752+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24497,24524,24487]},"id":1}]} 2024-05-14T09:33:51.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.813+0000 7f5cb0e43700 1 -- 172.21.15.2:0/4131408547 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5c940b2880 msgr2=0x7f5c940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:51.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.813+0000 7f5cb0e43700 1 --2- 172.21.15.2:0/4131408547 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5c940b2880 0x7f5c940b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c007be0 tx=0x7f5c9c0073d0 comp rx=0 tx=0).stop 2024-05-14T09:33:51.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.813+0000 7f5cb0e43700 1 -- 172.21.15.2:0/4131408547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cac108970 msgr2=0x7f5cac19c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:51.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.813+0000 7f5cb0e43700 1 --2- 172.21.15.2:0/4131408547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cac108970 0x7f5cac19c900 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40059d0 tx=0x7f5ca40057c0 comp rx=0 tx=0).stop 2024-05-14T09:33:51.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.813+0000 7f5cb0e43700 1 -- 172.21.15.2:0/4131408547 shutdown_connections 2024-05-14T09:33:51.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.813+0000 7f5cb0e43700 1 --2- 172.21.15.2:0/4131408547 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5c940b2880 0x7f5c940b4d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:51.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.813+0000 7f5cb0e43700 1 --2- 172.21.15.2:0/4131408547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cac103c10 0x7f5cac19be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:51.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.813+0000 7f5cb0e43700 1 --2- 172.21.15.2:0/4131408547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cac108970 0x7f5cac19c900 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:51.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.813+0000 7f5cb0e43700 1 --2- 172.21.15.2:0/4131408547 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cac108020 0x7f5cac19c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:51.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.813+0000 7f5cb0e43700 1 -- 172.21.15.2:0/4131408547 >> 172.21.15.2:0/4131408547 conn(0x7f5cac0fe540 msgr2=0x7f5cac106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:51.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.813+0000 7f5cb0e43700 1 -- 172.21.15.2:0/4131408547 shutdown_connections 2024-05-14T09:33:51.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:51.813+0000 7f5cb0e43700 1 -- 172.21.15.2:0/4131408547 wait complete. 2024-05-14T09:33:51.814 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-05-14T09:33:52.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:52 smithi002 ceph-mon[31557]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:52.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:52 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/4131408547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:33:52.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:52 smithi029 ceph-mon[36249]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:52.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:52 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/4131408547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:33:52.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:52 smithi026 ceph-mon[36056]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:52.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:52 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/4131408547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:33:52.577 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T09:33:52.835 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:54.284 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:54 smithi002 ceph-mon[31557]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:54.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:54 smithi029 ceph-mon[36249]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:54.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:54 smithi026 ceph-mon[36056]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:33:54.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.657+0000 7f334237b700 1 -- 172.21.15.2:0/2008365314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f333c108f30 msgr2=0x7f333c109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:54.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.657+0000 7f334237b700 1 --2- 172.21.15.2:0/2008365314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f333c108f30 0x7f333c109340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f33240099c0 tx=0x7f3324009cd0 comp rx=0 tx=0).stop 2024-05-14T09:33:54.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.657+0000 7f334237b700 1 -- 172.21.15.2:0/2008365314 shutdown_connections 2024-05-14T09:33:54.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.657+0000 7f334237b700 1 --2- 172.21.15.2:0/2008365314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f333c104650 0x7f333c104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:54.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.657+0000 7f334237b700 1 --2- 172.21.15.2:0/2008365314 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f333c109880 0x7f333c1111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:54.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.657+0000 7f334237b700 1 --2- 172.21.15.2:0/2008365314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f333c108f30 0x7f333c109340 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.657+0000 7f334237b700 1 -- 172.21.15.2:0/2008365314 >> 172.21.15.2:0/2008365314 conn(0x7f333c079f00 msgr2=0x7f333c07a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.657+0000 7f334237b700 1 -- 172.21.15.2:0/2008365314 shutdown_connections 2024-05-14T09:33:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.657+0000 7f334237b700 1 -- 172.21.15.2:0/2008365314 wait complete. 2024-05-14T09:33:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.657+0000 7f334237b700 1 Processor -- start 2024-05-14T09:33:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.657+0000 7f334237b700 1 -- start start 2024-05-14T09:33:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f334237b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f333c104650 0x7f333c19af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f334237b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f333c108f30 0x7f333c19b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f334237b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f333c109880 0x7f333c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f334237b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f333c1144b0 con 0x7f333c108f30 2024-05-14T09:33:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f334237b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f333c114330 con 0x7f333c109880 2024-05-14T09:33:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f334237b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f333c114630 con 0x7f333c104650 2024-05-14T09:33:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f3333fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f333c108f30 0x7f333c19b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f3333fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f333c108f30 0x7f333c19b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57544/0 (socket says 172.21.15.2:57544) 2024-05-14T09:33:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f3333fff700 1 -- 172.21.15.2:0/1367232281 learned_addr learned my addr 172.21.15.2:0/1367232281 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f333bfff700 1 --2- 172.21.15.2:0/1367232281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f333c104650 0x7f333c19af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f3340918700 1 --2- 172.21.15.2:0/1367232281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f333c109880 0x7f333c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f3333fff700 1 -- 172.21.15.2:0/1367232281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f333c104650 msgr2=0x7f333c19af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f3333fff700 1 --2- 172.21.15.2:0/1367232281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f333c104650 0x7f333c19af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f3333fff700 1 -- 172.21.15.2:0/1367232281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f333c109880 msgr2=0x7f333c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f3333fff700 1 --2- 172.21.15.2:0/1367232281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f333c109880 0x7f333c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f3333fff700 1 -- 172.21.15.2:0/1367232281 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c19fe30 con 0x7f333c108f30 2024-05-14T09:33:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.658+0000 7f3340918700 1 --2- 172.21.15.2:0/1367232281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f333c109880 0x7f333c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:33:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.659+0000 7f3333fff700 1 --2- 172.21.15.2:0/1367232281 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f333c108f30 0x7f333c19b500 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3324009f50 tx=0x7f33240037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.659+0000 7f3339ffb700 1 -- 172.21.15.2:0/1367232281 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324027070 con 0x7f333c108f30 2024-05-14T09:33:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.659+0000 7f3339ffb700 1 -- 172.21.15.2:0/1367232281 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3324004090 con 0x7f333c108f30 2024-05-14T09:33:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.659+0000 7f3339ffb700 1 -- 172.21.15.2:0/1367232281 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324022e60 con 0x7f333c108f30 2024-05-14T09:33:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.659+0000 7f334237b700 1 -- 172.21.15.2:0/1367232281 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333c1a00c0 con 0x7f333c108f30 2024-05-14T09:33:54.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.659+0000 7f334237b700 1 -- 172.21.15.2:0/1367232281 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f333c1a05a0 con 0x7f333c108f30 2024-05-14T09:33:54.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.660+0000 7f334237b700 1 -- 172.21.15.2:0/1367232281 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333c0512c0 con 0x7f333c108f30 2024-05-14T09:33:54.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.661+0000 7f3339ffb700 1 -- 172.21.15.2:0/1367232281 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f332402a070 con 0x7f333c108f30 2024-05-14T09:33:54.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.662+0000 7f3339ffb700 1 --2- 172.21.15.2:0/1367232281 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f33280b2880 0x7f33280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:54.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.662+0000 7f333bfff700 1 --2- 172.21.15.2:0/1367232281 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f33280b2880 0x7f33280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:54.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.662+0000 7f3339ffb700 1 -- 172.21.15.2:0/1367232281 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f33240e38c0 con 0x7f333c108f30 2024-05-14T09:33:54.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.663+0000 7f333bfff700 1 --2- 172.21.15.2:0/1367232281 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f33280b2880 0x7f33280b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f332c005950 tx=0x7f332c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:54.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.666+0000 7f3339ffb700 1 -- 172.21.15.2:0/1367232281 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3324031080 con 0x7f333c108f30 2024-05-14T09:33:54.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.879+0000 7f334237b700 1 -- 172.21.15.2:0/1367232281 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f333c02e5f0 con 0x7f333c108f30 2024-05-14T09:33:54.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.881+0000 7f3339ffb700 1 -- 172.21.15.2:0/1367232281 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24740 (secure 0 0 0) 0x7f3324013070 con 0x7f333c108f30 2024-05-14T09:33:54.881 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:33:54.882 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":90,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","created":"2024-05-14T09:22:34.937047+0000","modified":"2024-05-14T09:32:30.306958+0000","last_up_change":"2024-05-14T09:29:55.157569+0000","last_in_change":"2024-05-14T09:29:34.922562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T09:26:03.046454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T09:30:51.814350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T09:30:55.500000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T09:31:06.099087+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10d34289-414e-4b1b-bc7c-ba1dd24c7297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6803","nonce":2622184634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6805","nonce":2622184634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6809","nonce":2622184634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6807","nonce":2622184634}]},"public_addr":"172.21.15.2:6803/2622184634","cluster_addr":"172.21.15.2:6805/2622184634","heartbeat_back_addr":"172.21.15.2:6809/2622184634","heartbeat_front_addr":"172.21.15.2:6807/2622184634","state":["exists","up"]},{"osd":1,"uuid":"bc4aefc2-706e-4faf-9042-64b87ad097b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6801","nonce":3084747226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6803","nonce":3084747226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6807","nonce":3084747226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6805","nonce":3084747226}]},"public_addr":"172.21.15.26:6801/3084747226","cluster_addr":"172.21.15.26:6803/3084747226","heartbeat_back_addr":"172.21.15.26:6807/3084747226","heartbeat_front_addr":"172.21.15.26:6805/3084747226","state":["exists","up"]},{"osd":2,"uuid":"8c5bb589-56e2-469f-9a6f-6035b3ddf4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6801","nonce":814048086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6803","nonce":814048086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6807","nonce":814048086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6805","nonce":814048086}]},"public_addr":"172.21.15.29:6801/814048086","cluster_addr":"172.21.15.29:6803/814048086","heartbeat_back_addr":"172.21.15.29:6807/814048086","heartbeat_front_addr":"172.21.15.29:6805/814048086","state":["exists","up"]},{"osd":3,"uuid":"8fbfc078-85e7-4dc2-acda-5e7b0ab927e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6811","nonce":879639451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6813","nonce":879639451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6817","nonce":879639451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6815","nonce":879639451}]},"public_addr":"172.21.15.2:6811/879639451","cluster_addr":"172.21.15.2:6813/879639451","heartbeat_back_addr":"172.21.15.2:6817/879639451","heartbeat_front_addr":"172.21.15.2:6815/879639451","state":["exists","up"]},{"osd":4,"uuid":"f2515448-e9c1-48a2-b270-1e5e781a2fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6809","nonce":457100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6811","nonce":457100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6815","nonce":457100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6813","nonce":457100752}]},"public_addr":"172.21.15.26:6809/457100752","cluster_addr":"172.21.15.26:6811/457100752","heartbeat_back_addr":"172.21.15.26:6815/457100752","heartbeat_front_addr":"172.21.15.26:6813/457100752","state":["exists","up"]},{"osd":5,"uuid":"20030857-a1e6-4b0a-b186-1cfa44f2e4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6809","nonce":1554045093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6811","nonce":1554045093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6815","nonce":1554045093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6813","nonce":1554045093}]},"public_addr":"172.21.15.29:6809/1554045093","cluster_addr":"172.21.15.29:6811/1554045093","heartbeat_back_addr":"172.21.15.29:6815/1554045093","heartbeat_front_addr":"172.21.15.29:6813/1554045093","state":["exists","up"]},{"osd":6,"uuid":"d757c72a-29f0-4f14-9f13-a2e61eb5e6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6819","nonce":2749897595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6821","nonce":2749897595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6825","nonce":2749897595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6823","nonce":2749897595}]},"public_addr":"172.21.15.2:6819/2749897595","cluster_addr":"172.21.15.2:6821/2749897595","heartbeat_back_addr":"172.21.15.2:6825/2749897595","heartbeat_front_addr":"172.21.15.2:6823/2749897595","state":["exists","up"]},{"osd":7,"uuid":"22ddef80-3633-44c5-bc0e-edc75b546ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6817","nonce":1180401611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6819","nonce":1180401611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6823","nonce":1180401611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6821","nonce":1180401611}]},"public_addr":"172.21.15.26:6817/1180401611","cluster_addr":"172.21.15.26:6819/1180401611","heartbeat_back_addr":"172.21.15.26:6823/1180401611","heartbeat_front_addr":"172.21.15.26:6821/1180401611","state":["exists","up"]},{"osd":8,"uuid":"466e7ac4-3ff7-4254-a3fd-8096420b1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6817","nonce":2858408446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6819","nonce":2858408446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6823","nonce":2858408446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6821","nonce":2858408446}]},"public_addr":"172.21.15.29:6817/2858408446","cluster_addr":"172.21.15.29:6819/2858408446","heartbeat_back_addr":"172.21.15.29:6823/2858408446","heartbeat_front_addr":"172.21.15.29:6821/2858408446","state":["exists","up"]},{"osd":9,"uuid":"119eeac9-29f7-4086-a1ad-04a0bc154b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6827","nonce":176898226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6829","nonce":176898226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6833","nonce":176898226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6831","nonce":176898226}]},"public_addr":"172.21.15.2:6827/176898226","cluster_addr":"172.21.15.2:6829/176898226","heartbeat_back_addr":"172.21.15.2:6833/176898226","heartbeat_front_addr":"172.21.15.2:6831/176898226","state":["exists","up"]},{"osd":10,"uuid":"55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6825","nonce":4065159515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6827","nonce":4065159515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6831","nonce":4065159515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6829","nonce":4065159515}]},"public_addr":"172.21.15.26:6825/4065159515","cluster_addr":"172.21.15.26:6827/4065159515","heartbeat_back_addr":"172.21.15.26:6831/4065159515","heartbeat_front_addr":"172.21.15.26:6829/4065159515","state":["exists","up"]},{"osd":11,"uuid":"c5246763-c4f6-4ae3-8a93-5929c541ac5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6825","nonce":378800417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6827","nonce":378800417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6831","nonce":378800417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6829","nonce":378800417}]},"public_addr":"172.21.15.29:6825/378800417","cluster_addr":"172.21.15.29:6827/378800417","heartbeat_back_addr":"172.21.15.29:6831/378800417","heartbeat_front_addr":"172.21.15.29:6829/378800417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:09.342387+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:34.156359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:58.860653+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:24.009323+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:49.057146+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:15.007172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:41.085748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:06.674923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:33.409128+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:00.106186+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:26.803487+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:53.668283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11},{"from":8,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9},{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4051922409":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/2353790181":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/4268171474":"2024-05-15T09:23:28.940670+0000","172.21.15.2:6801/1423742578":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/969850441":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3252991268":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6801/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3789067476":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/1423742578":"2024-05-15T09:23:28.940670+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T09:33:54.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.883+0000 7f334237b700 1 -- 172.21.15.2:0/1367232281 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f33280b2880 msgr2=0x7f33280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:54.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.883+0000 7f334237b700 1 --2- 172.21.15.2:0/1367232281 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f33280b2880 0x7f33280b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f332c005950 tx=0x7f332c00a300 comp rx=0 tx=0).stop 2024-05-14T09:33:54.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.883+0000 7f334237b700 1 -- 172.21.15.2:0/1367232281 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f333c108f30 msgr2=0x7f333c19b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:54.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.883+0000 7f334237b700 1 --2- 172.21.15.2:0/1367232281 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f333c108f30 0x7f333c19b500 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3324009f50 tx=0x7f33240037c0 comp rx=0 tx=0).stop 2024-05-14T09:33:54.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.883+0000 7f334237b700 1 -- 172.21.15.2:0/1367232281 shutdown_connections 2024-05-14T09:33:54.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.883+0000 7f334237b700 1 --2- 172.21.15.2:0/1367232281 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f33280b2880 0x7f33280b4d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:54.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.883+0000 7f334237b700 1 --2- 172.21.15.2:0/1367232281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f333c109880 0x7f333c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:54.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.883+0000 7f334237b700 1 --2- 172.21.15.2:0/1367232281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f333c104650 0x7f333c19af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:54.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.883+0000 7f334237b700 1 --2- 172.21.15.2:0/1367232281 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f333c108f30 0x7f333c19b500 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:54.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.883+0000 7f334237b700 1 -- 172.21.15.2:0/1367232281 >> 172.21.15.2:0/1367232281 conn(0x7f333c079f00 msgr2=0x7f333c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:54.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.883+0000 7f334237b700 1 -- 172.21.15.2:0/1367232281 shutdown_connections 2024-05-14T09:33:54.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:54.883+0000 7f334237b700 1 -- 172.21.15.2:0/1367232281 wait complete. 2024-05-14T09:33:55.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:55 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1367232281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:33:55.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:55 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1367232281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:33:55.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:55 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1367232281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:33:55.588 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:33:55.844 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:33:56.220 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:56 smithi002 ceph-mon[31557]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:56 smithi029 ceph-mon[36249]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:56 smithi026 ceph-mon[36056]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:33:57.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.688+0000 7f4ae57e8700 1 -- 172.21.15.2:0/3628812672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae010a3e0 msgr2=0x7f4ae010a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:57.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.688+0000 7f4ae57e8700 1 --2- 172.21.15.2:0/3628812672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae010a3e0 0x7f4ae010a7f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000b0a0 tx=0x7f4ad000b3b0 comp rx=0 tx=0).stop 2024-05-14T09:33:57.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.688+0000 7f4ae57e8700 1 -- 172.21.15.2:0/3628812672 shutdown_connections 2024-05-14T09:33:57.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.688+0000 7f4ae57e8700 1 --2- 172.21.15.2:0/3628812672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ae0103130 0x7f4ae0111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:57.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.688+0000 7f4ae57e8700 1 --2- 172.21.15.2:0/3628812672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae010a3e0 0x7f4ae010a7f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:57.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.688+0000 7f4ae57e8700 1 --2- 172.21.15.2:0/3628812672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae01027e0 0x7f4ae0102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:57.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.688+0000 7f4ae57e8700 1 -- 172.21.15.2:0/3628812672 >> 172.21.15.2:0/3628812672 conn(0x7f4ae00fe6f0 msgr2=0x7f4ae0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:57.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.688+0000 7f4ae57e8700 1 -- 172.21.15.2:0/3628812672 shutdown_connections 2024-05-14T09:33:57.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.688+0000 7f4ae57e8700 1 -- 172.21.15.2:0/3628812672 wait complete. 2024-05-14T09:33:57.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.689+0000 7f4ae57e8700 1 Processor -- start 2024-05-14T09:33:57.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.689+0000 7f4ae57e8700 1 -- start start 2024-05-14T09:33:57.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.689+0000 7f4ae57e8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae01027e0 0x7f4ae019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:57.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.689+0000 7f4ae57e8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ae0103130 0x7f4ae019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:57.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.689+0000 7f4ae57e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae010a3e0 0x7f4ae019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:57.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.690+0000 7f4ae57e8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae01144d0 con 0x7f4ae010a3e0 2024-05-14T09:33:57.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.690+0000 7f4ae57e8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae0114350 con 0x7f4ae01027e0 2024-05-14T09:33:57.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.690+0000 7f4ae57e8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae0114650 con 0x7f4ae0103130 2024-05-14T09:33:57.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.690+0000 7f4adeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae01027e0 0x7f4ae019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.690+0000 7f4adf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae010a3e0 0x7f4ae019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.690+0000 7f4adeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae01027e0 0x7f4ae019ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:46100/0 (socket says 172.21.15.2:46100) 2024-05-14T09:33:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.690+0000 7f4adeffd700 1 -- 172.21.15.2:0/358923668 learned_addr learned my addr 172.21.15.2:0/358923668 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:33:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.690+0000 7f4ade7fc700 1 --2- 172.21.15.2:0/358923668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ae0103130 0x7f4ae019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.690+0000 7f4adf7fe700 1 -- 172.21.15.2:0/358923668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ae0103130 msgr2=0x7f4ae019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.690+0000 7f4adf7fe700 1 --2- 172.21.15.2:0/358923668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ae0103130 0x7f4ae019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.690+0000 7f4adf7fe700 1 -- 172.21.15.2:0/358923668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae01027e0 msgr2=0x7f4ae019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.690+0000 7f4adf7fe700 1 --2- 172.21.15.2:0/358923668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae01027e0 0x7f4ae019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.690+0000 7f4adf7fe700 1 -- 172.21.15.2:0/358923668 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae019fd30 con 0x7f4ae010a3e0 2024-05-14T09:33:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.690+0000 7f4adf7fe700 1 --2- 172.21.15.2:0/358923668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae010a3e0 0x7f4ae019f690 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c7a0 tx=0x7f4ad800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.690+0000 7f4ac7fff700 1 -- 172.21.15.2:0/358923668 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8004a30 con 0x7f4ae010a3e0 2024-05-14T09:33:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.691+0000 7f4ac7fff700 1 -- 172.21.15.2:0/358923668 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ad80054d0 con 0x7f4ae010a3e0 2024-05-14T09:33:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.691+0000 7f4ae57e8700 1 -- 172.21.15.2:0/358923668 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae01a0020 con 0x7f4ae010a3e0 2024-05-14T09:33:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.691+0000 7f4ac7fff700 1 -- 172.21.15.2:0/358923668 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800ec20 con 0x7f4ae010a3e0 2024-05-14T09:33:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.691+0000 7f4ae57e8700 1 -- 172.21.15.2:0/358923668 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae01a7940 con 0x7f4ae010a3e0 2024-05-14T09:33:57.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.692+0000 7f4ae57e8700 1 -- 172.21.15.2:0/358923668 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae010c850 con 0x7f4ae010a3e0 2024-05-14T09:33:57.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.693+0000 7f4ac7fff700 1 -- 172.21.15.2:0/358923668 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4ad8005000 con 0x7f4ae010a3e0 2024-05-14T09:33:57.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.694+0000 7f4ac7fff700 1 --2- 172.21.15.2:0/358923668 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4ac80b2880 0x7f4ac80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:33:57.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.694+0000 7f4adeffd700 1 --2- 172.21.15.2:0/358923668 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4ac80b2880 0x7f4ac80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:33:57.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.694+0000 7f4ac7fff700 1 -- 172.21.15.2:0/358923668 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f4ad80e2450 con 0x7f4ae010a3e0 2024-05-14T09:33:57.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.695+0000 7f4adeffd700 1 --2- 172.21.15.2:0/358923668 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4ac80b2880 0x7f4ac80b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009e60 tx=0x7f4ad0006230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:33:57.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.698+0000 7f4ac7fff700 1 -- 172.21.15.2:0/358923668 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4ad8014030 con 0x7f4ae010a3e0 2024-05-14T09:33:57.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.966+0000 7f4ae57e8700 1 -- 172.21.15.2:0/358923668 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ae00512c0 con 0x7f4ae010a3e0 2024-05-14T09:33:57.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.967+0000 7f4ac7fff700 1 -- 172.21.15.2:0/358923668 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11843 (secure 0 0 0) 0x7f4ad80052f0 con 0x7f4ae010a3e0 2024-05-14T09:33:57.968 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:33:57.968 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T09:32:01.167752+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24497,24524,24487]},"id":1}]} 2024-05-14T09:33:57.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.969+0000 7f4ae57e8700 1 -- 172.21.15.2:0/358923668 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4ac80b2880 msgr2=0x7f4ac80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:57.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.969+0000 7f4ae57e8700 1 --2- 172.21.15.2:0/358923668 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4ac80b2880 0x7f4ac80b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009e60 tx=0x7f4ad0006230 comp rx=0 tx=0).stop 2024-05-14T09:33:57.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.969+0000 7f4ae57e8700 1 -- 172.21.15.2:0/358923668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae010a3e0 msgr2=0x7f4ae019f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:33:57.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.969+0000 7f4ae57e8700 1 --2- 172.21.15.2:0/358923668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae010a3e0 0x7f4ae019f690 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c7a0 tx=0x7f4ad800cab0 comp rx=0 tx=0).stop 2024-05-14T09:33:57.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.969+0000 7f4ae57e8700 1 -- 172.21.15.2:0/358923668 shutdown_connections 2024-05-14T09:33:57.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.969+0000 7f4ae57e8700 1 --2- 172.21.15.2:0/358923668 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4ac80b2880 0x7f4ac80b4d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:57.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.969+0000 7f4ae57e8700 1 --2- 172.21.15.2:0/358923668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae01027e0 0x7f4ae019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:57.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.969+0000 7f4ae57e8700 1 --2- 172.21.15.2:0/358923668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae010a3e0 0x7f4ae019f690 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:57.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.969+0000 7f4ae57e8700 1 --2- 172.21.15.2:0/358923668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4ae0103130 0x7f4ae019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:33:57.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.969+0000 7f4ae57e8700 1 -- 172.21.15.2:0/358923668 >> 172.21.15.2:0/358923668 conn(0x7f4ae00fe6f0 msgr2=0x7f4ae0100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:33:57.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.969+0000 7f4ae57e8700 1 -- 172.21.15.2:0/358923668 shutdown_connections 2024-05-14T09:33:57.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:33:57.969+0000 7f4ae57e8700 1 -- 172.21.15.2:0/358923668 wait complete. 2024-05-14T09:33:57.971 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-05-14T09:33:58.338 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:58 smithi002 ceph-mon[31557]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:33:58.338 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:33:58 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/358923668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:33:58.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:58 smithi029 ceph-mon[36249]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:33:58.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:33:58 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/358923668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:33:58.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:58 smithi026 ceph-mon[36056]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:33:58.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:33:58 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/358923668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:33:58.796 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-14T09:30:58.644918+0000', 'modified': '2024-05-14T09:32:01.167752+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24524, 'mds_1': 24497, 'mds_2': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.29:6835/3647517494', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6834', 'nonce': 3647517494}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 3647517494}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.29:6837/1333834114', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6836', 'nonce': 1333834114}, {'type': 'v1', 'addr': '172.21.15.29:6837', 'nonce': 1333834114}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6835/2529273890', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 2529273890}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 2529273890}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.29:6839/3625063120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6838', 'nonce': 3625063120}, {'type': 'v1', 'addr': '172.21.15.29:6839', 'nonce': 3625063120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.26:6837/3301773578', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6836', 'nonce': 3301773578}, {'type': 'v1', 'addr': '172.21.15.26:6837', 'nonce': 3301773578}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.26:6839/903985766', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 903985766}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 903985766}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24524, 'qdb_cluster': [24497, 24524, 24487]} 2024-05-14T09:33:58.796 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-14T09:33:58.797 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-14T09:33:59.053 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:00.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:00 smithi002 ceph-mon[31557]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:00.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:00 smithi029 ceph-mon[36249]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:00.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:00 smithi026 ceph-mon[36056]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.896+0000 7f9f7dc88700 1 -- 172.21.15.2:0/4203052290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f78108f00 msgr2=0x7f9f78109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.896+0000 7f9f7dc88700 1 --2- 172.21.15.2:0/4203052290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f78108f00 0x7f9f78109310 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0099c0 tx=0x7f9f6c009cd0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.896+0000 7f9f7dc88700 1 -- 172.21.15.2:0/4203052290 shutdown_connections 2024-05-14T09:34:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.896+0000 7f9f7dc88700 1 --2- 172.21.15.2:0/4203052290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f78109850 0x7f9f781111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.896+0000 7f9f7dc88700 1 --2- 172.21.15.2:0/4203052290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f781045d0 0x7f9f781049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.896+0000 7f9f7dc88700 1 --2- 172.21.15.2:0/4203052290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f78108f00 0x7f9f78109310 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.896+0000 7f9f7dc88700 1 -- 172.21.15.2:0/4203052290 >> 172.21.15.2:0/4203052290 conn(0x7f9f780ffea0 msgr2=0x7f9f781022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.896+0000 7f9f7dc88700 1 -- 172.21.15.2:0/4203052290 shutdown_connections 2024-05-14T09:34:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.896+0000 7f9f7dc88700 1 -- 172.21.15.2:0/4203052290 wait complete. 2024-05-14T09:34:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.897+0000 7f9f7dc88700 1 Processor -- start 2024-05-14T09:34:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.897+0000 7f9f7dc88700 1 -- start start 2024-05-14T09:34:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.897+0000 7f9f7dc88700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f781045d0 0x7f9f78076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.897+0000 7f9f7dc88700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f78108f00 0x7f9f78077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.897+0000 7f9f7dc88700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f78109850 0x7f9f78077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.897+0000 7f9f7dc88700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781144d0 con 0x7f9f78109850 2024-05-14T09:34:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.897+0000 7f9f7dc88700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9f78114350 con 0x7f9f781045d0 2024-05-14T09:34:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.897+0000 7f9f7dc88700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9f78114650 con 0x7f9f78108f00 2024-05-14T09:34:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.897+0000 7f9f77fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f78109850 0x7f9f78077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.897+0000 7f9f77fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f78109850 0x7f9f78077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51026/0 (socket says 172.21.15.2:51026) 2024-05-14T09:34:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.897+0000 7f9f77fff700 1 -- 172.21.15.2:0/3122317530 learned_addr learned my addr 172.21.15.2:0/3122317530 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.897+0000 7f9f76ffd700 1 --2- 172.21.15.2:0/3122317530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f78108f00 0x7f9f78077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.897+0000 7f9f777fe700 1 --2- 172.21.15.2:0/3122317530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f781045d0 0x7f9f78076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.898+0000 7f9f77fff700 1 -- 172.21.15.2:0/3122317530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f78108f00 msgr2=0x7f9f78077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.898+0000 7f9f77fff700 1 --2- 172.21.15.2:0/3122317530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f78108f00 0x7f9f78077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.898+0000 7f9f77fff700 1 -- 172.21.15.2:0/3122317530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f781045d0 msgr2=0x7f9f78076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.898+0000 7f9f77fff700 1 --2- 172.21.15.2:0/3122317530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f781045d0 0x7f9f78076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.898+0000 7f9f77fff700 1 -- 172.21.15.2:0/3122317530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f781a91e0 con 0x7f9f78109850 2024-05-14T09:34:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.898+0000 7f9f77fff700 1 --2- 172.21.15.2:0/3122317530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f78109850 0x7f9f78077650 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9f70007d70 tx=0x7f9f7000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.898+0000 7f9f74ff9700 1 -- 172.21.15.2:0/3122317530 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70016020 con 0x7f9f78109850 2024-05-14T09:34:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.898+0000 7f9f74ff9700 1 -- 172.21.15.2:0/3122317530 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f7000ac90 con 0x7f9f78109850 2024-05-14T09:34:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.898+0000 7f9f7dc88700 1 -- 172.21.15.2:0/3122317530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f781a94d0 con 0x7f9f78109850 2024-05-14T09:34:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.898+0000 7f9f74ff9700 1 -- 172.21.15.2:0/3122317530 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f700115e0 con 0x7f9f78109850 2024-05-14T09:34:00.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.898+0000 7f9f7dc88700 1 -- 172.21.15.2:0/3122317530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f781a9990 con 0x7f9f78109850 2024-05-14T09:34:00.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.901+0000 7f9f74ff9700 1 -- 172.21.15.2:0/3122317530 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9f70019020 con 0x7f9f78109850 2024-05-14T09:34:00.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.902+0000 7f9f74ff9700 1 --2- 172.21.15.2:0/3122317530 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9f600b2880 0x7f9f600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.902+0000 7f9f8c69cb80 1 Processor -- start 2024-05-14T09:34:00.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.902+0000 7f9f777fe700 1 --2- 172.21.15.2:0/3122317530 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9f600b2880 0x7f9f600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.902+0000 7f9f74ff9700 1 -- 172.21.15.2:0/3122317530 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f9f700e25c0 con 0x7f9f78109850 2024-05-14T09:34:00.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.902+0000 7f9f8c69cb80 1 -- start start 2024-05-14T09:34:00.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.902+0000 7f9f8c69cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa336d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.902+0000 7f9f8c69cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa33c10 0x56011fa37ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.902+0000 7f9f8c69cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa38420 0x56011fa38830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.902+0000 7f9f8c69cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56011f955810 con 0x56011fa38420 2024-05-14T09:34:00.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.902+0000 7f9f8c69cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56011f8f4770 con 0x56011f8b5df0 2024-05-14T09:34:00.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.902+0000 7f9f8c69cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56011f9517d0 con 0x56011fa33c10 2024-05-14T09:34:00.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.902+0000 7f9f777fe700 1 --2- 172.21.15.2:0/3122317530 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9f600b2880 0x7f9f600b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009a10 tx=0x7f9f68009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:00.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f77fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa38420 0x56011fa38830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f777fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa336d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f77fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa38420 0x56011fa38830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51028/0 (socket says 172.21.15.2:51028) 2024-05-14T09:34:00.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f77fff700 1 -- 172.21.15.2:0/3538267443 learned_addr learned my addr 172.21.15.2:0/3538267443 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:00.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f777fe700 1 --2- 172.21.15.2:0/3538267443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa336d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:55836/0 (socket says 172.21.15.2:55836) 2024-05-14T09:34:00.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f76ffd700 1 --2- 172.21.15.2:0/3538267443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa33c10 0x56011fa37ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f77fff700 1 -- 172.21.15.2:0/3538267443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa33c10 msgr2=0x56011fa37ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:00.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f77fff700 1 --2- 172.21.15.2:0/3538267443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa33c10 0x56011fa37ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f77fff700 1 -- 172.21.15.2:0/3538267443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 msgr2=0x56011fa336d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f77fff700 1 --2- 172.21.15.2:0/3538267443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa336d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f77fff700 1 -- 172.21.15.2:0/3538267443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56011f8a2880 con 0x56011fa38420 2024-05-14T09:34:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f777fe700 1 --2- 172.21.15.2:0/3538267443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa336d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f77fff700 1 --2- 172.21.15.2:0/3538267443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa38420 0x56011fa38830 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9f700a94c0 tx=0x7f9f700129f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202a82650d04047d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f5dffb700 1 -- 172.21.15.2:0/3538267443 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f700a92d0 con 0x56011fa38420 2024-05-14T09:34:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f5dffb700 1 -- 172.21.15.2:0/3538267443 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f700cc0f0 con 0x56011fa38420 2024-05-14T09:34:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.903+0000 7f9f5dffb700 1 -- 172.21.15.2:0/3538267443 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f700b2190 con 0x56011fa38420 2024-05-14T09:34:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.904+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/3538267443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa38420 msgr2=0x56011fa38830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.904+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/3538267443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa38420 0x56011fa38830 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9f700a94c0 tx=0x7f9f700129f0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.904+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/3538267443 shutdown_connections 2024-05-14T09:34:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.904+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/3538267443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa336d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.904+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/3538267443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa38420 0x56011fa38830 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.904+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/3538267443 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa33c10 0x56011fa37ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.904+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/3538267443 >> 172.21.15.2:0/3538267443 conn(0x56011f8fd5b0 msgr2=0x56011fa2e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.904+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/3538267443 shutdown_connections 2024-05-14T09:34:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.904+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/3538267443 wait complete. 2024-05-14T09:34:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.905+0000 7f9f8c69cb80 1 -- start start 2024-05-14T09:34:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.905+0000 7f9f8c69cb80 1 Processor -- start 2024-05-14T09:34:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.905+0000 7f9f8c69cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa3d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.905+0000 7f9f8c69cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa3d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.905+0000 7f9f8c69cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 0x56011fa3dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.905+0000 7f9f8c69cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56011fa403d0 con 0x56011fa33c10 2024-05-14T09:34:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.905+0000 7f9f8c69cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56011f8bbbe0 con 0x56011f8b5df0 2024-05-14T09:34:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.905+0000 7f9f8c69cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56011fa40550 con 0x56011fa38420 2024-05-14T09:34:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.905+0000 7f9f8c69cb80 10 client.0 _ll_register_callbacks cb 0x56011f8ed8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:34:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f76ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa3d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f76ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa3d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51034/0 (socket says 172.21.15.2:51034) 2024-05-14T09:34:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f76ffd700 1 -- 172.21.15.2:0/2079951344 learned_addr learned my addr 172.21.15.2:0/2079951344 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f777fe700 1 --2- 172.21.15.2:0/2079951344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa3d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f77fff700 1 --2- 172.21.15.2:0/2079951344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 0x56011fa3dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f76ffd700 1 -- 172.21.15.2:0/2079951344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 msgr2=0x56011fa3dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f76ffd700 1 --2- 172.21.15.2:0/2079951344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 0x56011fa3dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f76ffd700 1 -- 172.21.15.2:0/2079951344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 msgr2=0x56011fa3d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f76ffd700 1 --2- 172.21.15.2:0/2079951344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa3d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f76ffd700 1 -- 172.21.15.2:0/2079951344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56011f8f1640 con 0x56011fa33c10 2024-05-14T09:34:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f77fff700 1 --2- 172.21.15.2:0/2079951344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 0x56011fa3dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:34:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f777fe700 1 --2- 172.21.15.2:0/2079951344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa3d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f76ffd700 1 --2- 172.21.15.2:0/2079951344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa3d6a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9f7806c170 tx=0x7f9f6c0036e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98886761e458b8a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:34:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f56ffd700 1 -- 172.21.15.2:0/2079951344 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c005680 con 0x56011fa33c10 2024-05-14T09:34:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f56ffd700 1 -- 172.21.15.2:0/2079951344 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f9f6c00b790 con 0x56011fa33c10 2024-05-14T09:34:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.906+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/2079951344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f700cc0f0 con 0x56011fa33c10 2024-05-14T09:34:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.907+0000 7f9f56ffd700 1 -- 172.21.15.2:0/2079951344 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f6c031050 con 0x56011fa33c10 2024-05-14T09:34:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.907+0000 7f9f8c69cb80 10 client.15084 fetch_fsmap learned FSMap version 26 2024-05-14T09:34:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.907+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/2079951344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56011fa4b080 con 0x56011fa33c10 2024-05-14T09:34:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.907+0000 7f9f56ffd700 1 -- 172.21.15.2:0/2079951344 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f700cc0f0 con 0x56011fa33c10 2024-05-14T09:34:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.907+0000 7f9f56ffd700 1 -- 172.21.15.2:0/2079951344 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9f6c023b10 con 0x56011fa33c10 2024-05-14T09:34:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.907+0000 7f9f8c69cb80 10 client.15084 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T09:34:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.907+0000 7f9f8c69cb80 10 client.15084 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T09:34:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.907+0000 7f9f8c69cb80 20 client.15084 populate_metadata read hostname 'smithi002' 2024-05-14T09:34:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.907+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/2079951344 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x56011fa4d770 0x56011fa4ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.907+0000 7f9f8c69cb80 4 client.15084 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:34:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.907+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/2079951344 --> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56011f9631f0 con 0x56011fa4d770 2024-05-14T09:34:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.907+0000 7f9f777fe700 1 --2- 172.21.15.2:0/2079951344 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x56011fa4d770 0x56011fa4ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.908+0000 7f9f777fe700 1 --2- 172.21.15.2:0/2079951344 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x56011fa4d770 0x56011fa4ebe0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fc7292b79a58819 server_cookie=e33d9842b4bbc5f3 in_seq=0 out_seq=0 2024-05-14T09:34:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.908+0000 7f9f56ffd700 10 client.15084 ms_handle_connect on v2:172.21.15.29:6838/3625063120 2024-05-14T09:34:00.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.909+0000 7f9f56ffd700 1 -- 172.21.15.2:0/2079951344 <== mds.2 v2:172.21.15.29:6838/3625063120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56011f9631f0 con 0x56011fa4d770 2024-05-14T09:34:00.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.909+0000 7f9f56ffd700 10 client.15084 handle_command_reply: tid=0 2024-05-14T09:34:00.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.909+0000 7f9f8c69cb80 1 client.15084 shutdown 2024-05-14T09:34:00.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.909+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/2079951344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 msgr2=0x56011fa3d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:00.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.909+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/2079951344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa3d6a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9f7806c170 tx=0x7f9f6c0036e0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.909+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/2079951344 shutdown_connections 2024-05-14T09:34:00.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.909+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/2079951344 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x56011fa4d770 0x56011fa4ebe0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.909+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/2079951344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa3d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.909+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/2079951344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 0x56011fa3dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.909+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/2079951344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa3d6a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.909+0000 7f9f56ffd700 0 client.15084 ms_handle_reset on v2:172.21.15.29:6838/3625063120 2024-05-14T09:34:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.909+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/2079951344 >> 172.21.15.2:0/2079951344 conn(0x56011f8fd5b0 msgr2=0x56011fa32400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.909+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/2079951344 shutdown_connections 2024-05-14T09:34:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.909+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/2079951344 wait complete. 2024-05-14T09:34:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.909+0000 7f9f8c69cb80 20 client.15084 trim_cache size 0 max 16384 2024-05-14T09:34:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f8c69cb80 1 Processor -- start 2024-05-14T09:34:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f8c69cb80 1 -- start start 2024-05-14T09:34:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f8c69cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa3fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f8c69cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa49820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f8c69cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 0x56011fa3ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f8c69cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56011fa4cf20 con 0x56011fa33c10 2024-05-14T09:34:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f8c69cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56011fa4d5f0 con 0x56011f8b5df0 2024-05-14T09:34:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f8c69cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56011fa4dcc0 con 0x56011fa38420 2024-05-14T09:34:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f76ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa49820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f76ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa49820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51042/0 (socket says 172.21.15.2:51042) 2024-05-14T09:34:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f76ffd700 1 -- 172.21.15.2:0/3068913013 learned_addr learned my addr 172.21.15.2:0/3068913013 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f77fff700 1 --2- 172.21.15.2:0/3068913013 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 0x56011fa3ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f777fe700 1 --2- 172.21.15.2:0/3068913013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa3fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f76ffd700 1 -- 172.21.15.2:0/3068913013 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 msgr2=0x56011fa3ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f76ffd700 1 --2- 172.21.15.2:0/3068913013 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 0x56011fa3ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f76ffd700 1 -- 172.21.15.2:0/3068913013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 msgr2=0x56011fa3fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f76ffd700 1 --2- 172.21.15.2:0/3068913013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa3fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f76ffd700 1 -- 172.21.15.2:0/3068913013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56011fa3b6d0 con 0x56011fa33c10 2024-05-14T09:34:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.931+0000 7f9f777fe700 1 --2- 172.21.15.2:0/3068913013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa3fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:34:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.932+0000 7f9f76ffd700 1 --2- 172.21.15.2:0/3068913013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa49820 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c003800 tx=0x7f9f6c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa8dd62901d1d87 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.932+0000 7f9f5d7fa700 1 -- 172.21.15.2:0/3068913013 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c039070 con 0x56011fa33c10 2024-05-14T09:34:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.932+0000 7f9f5d7fa700 1 -- 172.21.15.2:0/3068913013 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f6c028070 con 0x56011fa33c10 2024-05-14T09:34:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.932+0000 7f9f5d7fa700 1 -- 172.21.15.2:0/3068913013 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c0223d0 con 0x56011fa33c10 2024-05-14T09:34:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.933+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/3068913013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 msgr2=0x56011fa49820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.933+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/3068913013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa49820 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c003800 tx=0x7f9f6c009310 comp rx=0 tx=0).stop 2024-05-14T09:34:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.933+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/3068913013 shutdown_connections 2024-05-14T09:34:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.933+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/3068913013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa3fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.933+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/3068913013 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 0x56011fa3ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.933+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/3068913013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa49820 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.933+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/3068913013 >> 172.21.15.2:0/3068913013 conn(0x56011f8fd5b0 msgr2=0x56011fa28130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.933+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/3068913013 shutdown_connections 2024-05-14T09:34:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.933+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/3068913013 wait complete. 2024-05-14T09:34:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.933+0000 7f9f8c69cb80 1 -- start start 2024-05-14T09:34:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.933+0000 7f9f8c69cb80 1 Processor -- start 2024-05-14T09:34:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f8c69cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa58340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f8c69cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa3c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f8c69cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 0x56011fa3cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f8c69cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56011fa4ea60 con 0x56011fa33c10 2024-05-14T09:34:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f8c69cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56011fa4e390 con 0x56011f8b5df0 2024-05-14T09:34:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f8c69cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56011fa4f130 con 0x56011fa38420 2024-05-14T09:34:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f8c69cb80 10 client.0 _ll_register_callbacks cb 0x56011f8ed8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:34:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f76ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa3c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f777fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa58340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f76ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa3c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51054/0 (socket says 172.21.15.2:51054) 2024-05-14T09:34:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f76ffd700 1 -- 172.21.15.2:0/1904195321 learned_addr learned my addr 172.21.15.2:0/1904195321 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f77fff700 1 --2- 172.21.15.2:0/1904195321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 0x56011fa3cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f777fe700 1 -- 172.21.15.2:0/1904195321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 msgr2=0x56011fa3cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f777fe700 1 --2- 172.21.15.2:0/1904195321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 0x56011fa3cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f777fe700 1 -- 172.21.15.2:0/1904195321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 msgr2=0x56011fa3c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f777fe700 1 --2- 172.21.15.2:0/1904195321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa3c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f777fe700 1 -- 172.21.15.2:0/1904195321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56011fa3d220 con 0x56011f8b5df0 2024-05-14T09:34:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f777fe700 1 --2- 172.21.15.2:0/1904195321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa58340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x56011fa3fb40 tx=0x7f9f68004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=273f76cf5224f064 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.934+0000 7f9f56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:34:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.935+0000 7f9f56ffd700 1 -- 172.21.15.2:0/1904195321 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6802d070 con 0x56011f8b5df0 2024-05-14T09:34:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.935+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/1904195321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f6c028070 con 0x56011f8b5df0 2024-05-14T09:34:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.935+0000 7f9f56ffd700 1 -- 172.21.15.2:0/1904195321 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f9f68029250 con 0x56011f8b5df0 2024-05-14T09:34:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.935+0000 7f9f56ffd700 1 -- 172.21.15.2:0/1904195321 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f68039050 con 0x56011f8b5df0 2024-05-14T09:34:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.935+0000 7f9f8c69cb80 10 client.24866 fetch_fsmap learned FSMap version 26 2024-05-14T09:34:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.935+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/1904195321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56011fa57cd0 con 0x56011f8b5df0 2024-05-14T09:34:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.935+0000 7f9f56ffd700 1 -- 172.21.15.2:0/1904195321 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f6c028070 con 0x56011f8b5df0 2024-05-14T09:34:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.935+0000 7f9f56ffd700 1 -- 172.21.15.2:0/1904195321 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9f68015ce0 con 0x56011f8b5df0 2024-05-14T09:34:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.935+0000 7f9f8c69cb80 10 client.24866 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T09:34:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.936+0000 7f9f8c69cb80 10 client.24866 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T09:34:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.936+0000 7f9f8c69cb80 20 client.24866 populate_metadata read hostname 'smithi002' 2024-05-14T09:34:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.936+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/1904195321 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x56011fababf0 0x56011fabc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.936+0000 7f9f8c69cb80 4 client.24866 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T09:34:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.936+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/1904195321 --> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] -- command(tid 0: {"prefix": "status"}) -- 0x56011f95d290 con 0x56011fababf0 2024-05-14T09:34:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.936+0000 7f9f76ffd700 1 --2- 172.21.15.2:0/1904195321 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x56011fababf0 0x56011fabc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.936+0000 7f9f76ffd700 1 --2- 172.21.15.2:0/1904195321 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x56011fababf0 0x56011fabc060 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e073e2362e711c server_cookie=8b2bc2a79f46e5c in_seq=0 out_seq=0 2024-05-14T09:34:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.936+0000 7f9f56ffd700 10 client.24866 ms_handle_connect on v2:172.21.15.29:6838/3625063120 2024-05-14T09:34:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.936+0000 7f9f56ffd700 1 -- 172.21.15.2:0/1904195321 <== mds.2 v2:172.21.15.29:6838/3625063120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x56011f95d290 con 0x56011fababf0 2024-05-14T09:34:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.936+0000 7f9f56ffd700 10 client.24866 handle_command_reply: tid=0 2024-05-14T09:34:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.936+0000 7f9f8c69cb80 1 client.24866 shutdown 2024-05-14T09:34:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/1904195321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 msgr2=0x56011fa58340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/1904195321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa58340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x56011fa3fb40 tx=0x7f9f68004750 comp rx=0 tx=0).stop 2024-05-14T09:34:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/1904195321 shutdown_connections 2024-05-14T09:34:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/1904195321 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x56011fababf0 0x56011fabc060 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/1904195321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56011f8b5df0 0x56011fa58340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f56ffd700 0 client.24866 ms_handle_reset on v2:172.21.15.29:6838/3625063120 2024-05-14T09:34:00.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/1904195321 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56011fa38420 0x56011fa3cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f8c69cb80 1 --2- 172.21.15.2:0/1904195321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56011fa33c10 0x56011fa3c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/1904195321 >> 172.21.15.2:0/1904195321 conn(0x56011f8fd5b0 msgr2=0x56011fa3ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:00.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/1904195321 shutdown_connections 2024-05-14T09:34:00.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f8c69cb80 1 -- 172.21.15.2:0/1904195321 wait complete. 2024-05-14T09:34:00.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f8c69cb80 20 client.24866 trim_cache size 0 max 16384 2024-05-14T09:34:00.942 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:34:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "50d5aebe-11d3-11ef-bc98-c7b262605968", 2024-05-14T09:34:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 2, 2024-05-14T09:34:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24487, 2024-05-14T09:34:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T09:34:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T09:34:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T09:34:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 125.228987098, 2024-05-14T09:34:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 26, 2024-05-14T09:34:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 90, 2024-05-14T09:34:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 88, 2024-05-14T09:34:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 206.78874019 2024-05-14T09:34:00.943 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:34:00.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f7dc88700 1 -- 172.21.15.2:0/3122317530 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9f600b2880 msgr2=0x7f9f600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:00.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f7dc88700 1 --2- 172.21.15.2:0/3122317530 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9f600b2880 0x7f9f600b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009a10 tx=0x7f9f68009190 comp rx=0 tx=0).stop 2024-05-14T09:34:00.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f7dc88700 1 -- 172.21.15.2:0/3122317530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f78109850 msgr2=0x7f9f78077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:00.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f7dc88700 1 --2- 172.21.15.2:0/3122317530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f78109850 0x7f9f78077650 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9f70007d70 tx=0x7f9f7000a340 comp rx=0 tx=0).stop 2024-05-14T09:34:00.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f7dc88700 1 -- 172.21.15.2:0/3122317530 shutdown_connections 2024-05-14T09:34:00.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.937+0000 7f9f7dc88700 1 --2- 172.21.15.2:0/3122317530 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9f600b2880 0x7f9f600b4d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.938+0000 7f9f7dc88700 1 --2- 172.21.15.2:0/3122317530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f781045d0 0x7f9f78076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.938+0000 7f9f7dc88700 1 --2- 172.21.15.2:0/3122317530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f78109850 0x7f9f78077650 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.938+0000 7f9f7dc88700 1 --2- 172.21.15.2:0/3122317530 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f78108f00 0x7f9f78077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:00.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.938+0000 7f9f7dc88700 1 -- 172.21.15.2:0/3122317530 >> 172.21.15.2:0/3122317530 conn(0x7f9f780ffea0 msgr2=0x7f9f7810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:00.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.938+0000 7f9f7dc88700 1 -- 172.21.15.2:0/3122317530 shutdown_connections 2024-05-14T09:34:00.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:00.938+0000 7f9f7dc88700 1 -- 172.21.15.2:0/3122317530 wait complete. 2024-05-14T09:34:01.791 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-14T09:34:02.049 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:02.420 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:02 smithi002 ceph-mon[31557]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:02.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:02 smithi029 ceph-mon[36249]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:02.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:02 smithi026 ceph-mon[36056]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:03.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.889+0000 7f31d65f6700 1 -- 172.21.15.2:0/4232892261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31d01034c0 msgr2=0x7f31d01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:03.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.889+0000 7f31d65f6700 1 --2- 172.21.15.2:0/4232892261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31d01034c0 0x7f31d01038d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f31c00099c0 tx=0x7f31c0009cd0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.889+0000 7f31d65f6700 1 -- 172.21.15.2:0/4232892261 shutdown_connections 2024-05-14T09:34:03.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.889+0000 7f31d65f6700 1 --2- 172.21.15.2:0/4232892261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31d0108740 0x7f31d01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.889+0000 7f31d65f6700 1 --2- 172.21.15.2:0/4232892261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31d01034c0 0x7f31d01038d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.889+0000 7f31d65f6700 1 --2- 172.21.15.2:0/4232892261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31d0107df0 0x7f31d0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.889+0000 7f31d65f6700 1 -- 172.21.15.2:0/4232892261 >> 172.21.15.2:0/4232892261 conn(0x7f31d00fedf0 msgr2=0x7f31d0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:03.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.889+0000 7f31d65f6700 1 -- 172.21.15.2:0/4232892261 shutdown_connections 2024-05-14T09:34:03.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.889+0000 7f31d65f6700 1 -- 172.21.15.2:0/4232892261 wait complete. 2024-05-14T09:34:03.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.890+0000 7f31d65f6700 1 Processor -- start 2024-05-14T09:34:03.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.890+0000 7f31d65f6700 1 -- start start 2024-05-14T09:34:03.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.890+0000 7f31d65f6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31d01034c0 0x7f31d019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.890+0000 7f31d65f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31d0107df0 0x7f31d019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.890+0000 7f31d65f6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31d0108740 0x7f31d019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.890+0000 7f31d65f6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f31d0113370 con 0x7f31d0108740 2024-05-14T09:34:03.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.890+0000 7f31d65f6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f31d01131f0 con 0x7f31d0107df0 2024-05-14T09:34:03.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.890+0000 7f31d65f6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f31d01134f0 con 0x7f31d01034c0 2024-05-14T09:34:03.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.890+0000 7f31d4b93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31d0108740 0x7f31d019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.890+0000 7f31cffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31d01034c0 0x7f31d019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.890+0000 7f31cffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31d01034c0 0x7f31d019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:39838/0 (socket says 172.21.15.2:39838) 2024-05-14T09:34:03.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.890+0000 7f31cffff700 1 -- 172.21.15.2:0/1384966118 learned_addr learned my addr 172.21.15.2:0/1384966118 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:03.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.890+0000 7f31c7fff700 1 --2- 172.21.15.2:0/1384966118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31d0107df0 0x7f31d019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.891+0000 7f31d4b93700 1 -- 172.21.15.2:0/1384966118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31d01034c0 msgr2=0x7f31d019bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:03.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.891+0000 7f31d4b93700 1 --2- 172.21.15.2:0/1384966118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31d01034c0 0x7f31d019bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.891+0000 7f31d4b93700 1 -- 172.21.15.2:0/1384966118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31d0107df0 msgr2=0x7f31d019c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:03.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.891+0000 7f31d4b93700 1 --2- 172.21.15.2:0/1384966118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31d0107df0 0x7f31d019c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.891+0000 7f31d4b93700 1 -- 172.21.15.2:0/1384966118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d01a6fa0 con 0x7f31d0108740 2024-05-14T09:34:03.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.891+0000 7f31d4b93700 1 --2- 172.21.15.2:0/1384966118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31d0108740 0x7f31d019c6f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f31c8007f10 tx=0x7f31c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:03.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.891+0000 7f31cdffb700 1 -- 172.21.15.2:0/1384966118 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8004db0 con 0x7f31d0108740 2024-05-14T09:34:03.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.891+0000 7f31d65f6700 1 -- 172.21.15.2:0/1384966118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d01a7290 con 0x7f31d0108740 2024-05-14T09:34:03.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.891+0000 7f31cdffb700 1 -- 172.21.15.2:0/1384966118 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c8009e20 con 0x7f31d0108740 2024-05-14T09:34:03.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.891+0000 7f31cdffb700 1 -- 172.21.15.2:0/1384966118 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8015400 con 0x7f31d0108740 2024-05-14T09:34:03.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.891+0000 7f31d65f6700 1 -- 172.21.15.2:0/1384966118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d01a77d0 con 0x7f31d0108740 2024-05-14T09:34:03.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.894+0000 7f31cdffb700 1 -- 172.21.15.2:0/1384966118 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f31c8005260 con 0x7f31d0108740 2024-05-14T09:34:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.895+0000 7f31cdffb700 1 --2- 172.21.15.2:0/1384966118 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f31bc0b2880 0x7f31bc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.896+0000 7f31cffff700 1 --2- 172.21.15.2:0/1384966118 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f31bc0b2880 0x7f31bc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.896+0000 7f31cdffb700 1 -- 172.21.15.2:0/1384966118 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f31c80e1f30 con 0x7f31d0108740 2024-05-14T09:34:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.896+0000 7f31e500ab80 1 Processor -- start 2024-05-14T09:34:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.896+0000 7f31e500ab80 1 -- start start 2024-05-14T09:34:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.896+0000 7f31cffff700 1 --2- 172.21.15.2:0/1384966118 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f31bc0b2880 0x7f31bc0b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f31c0003800 tx=0x7f31c0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.896+0000 7f31e500ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2d22df0 0x55d8f2eaab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.896+0000 7f31e500ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2eab090 0x55d8f2ea4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.896+0000 7f31e500ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2ea51d0 0x55d8f2ea55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.896+0000 7f31e500ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d8f2dc28f0 con 0x55d8f2d22df0 2024-05-14T09:34:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.896+0000 7f31e500ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d8f2d27d50 con 0x55d8f2ea51d0 2024-05-14T09:34:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.896+0000 7f31e500ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d8f2dbe8b0 con 0x55d8f2eab090 2024-05-14T09:34:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.896+0000 7f31cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2d22df0 0x55d8f2eaab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.896+0000 7f31c7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2eab090 0x55d8f2ea4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.896+0000 7f31cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2d22df0 0x55d8f2eaab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51070/0 (socket says 172.21.15.2:51070) 2024-05-14T09:34:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.896+0000 7f31cffff700 1 -- 172.21.15.2:0/1216833925 learned_addr learned my addr 172.21.15.2:0/1216833925 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.897+0000 7f31d4b93700 1 --2- 172.21.15.2:0/1216833925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2ea51d0 0x55d8f2ea55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.897+0000 7f31c7fff700 1 -- 172.21.15.2:0/1216833925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2ea51d0 msgr2=0x55d8f2ea55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.897+0000 7f31c7fff700 1 --2- 172.21.15.2:0/1216833925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2ea51d0 0x55d8f2ea55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.897+0000 7f31c7fff700 1 -- 172.21.15.2:0/1216833925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2d22df0 msgr2=0x55d8f2eaab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.897+0000 7f31c7fff700 1 --2- 172.21.15.2:0/1216833925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2d22df0 0x55d8f2eaab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.897+0000 7f31c7fff700 1 -- 172.21.15.2:0/1216833925 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f2d0f880 con 0x55d8f2eab090 2024-05-14T09:34:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.897+0000 7f31c7fff700 1 --2- 172.21.15.2:0/1216833925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2eab090 0x55d8f2ea4c90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f31d0076b40 tx=0x7f31b8007ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d81d0f9557636fe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.897+0000 7f31c67fc700 1 -- 172.21.15.2:0/1216833925 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8004310 con 0x55d8f2eab090 2024-05-14T09:34:03.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.897+0000 7f31c67fc700 1 -- 172.21.15.2:0/1216833925 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31b8015070 con 0x55d8f2eab090 2024-05-14T09:34:03.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.897+0000 7f31c67fc700 1 -- 172.21.15.2:0/1216833925 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8004e00 con 0x55d8f2eab090 2024-05-14T09:34:03.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.898+0000 7f31e500ab80 1 -- 172.21.15.2:0/1216833925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2eab090 msgr2=0x55d8f2ea4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:03.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.898+0000 7f31e500ab80 1 --2- 172.21.15.2:0/1216833925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2eab090 0x55d8f2ea4c90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f31d0076b40 tx=0x7f31b8007ea0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.898+0000 7f31e500ab80 1 -- 172.21.15.2:0/1216833925 shutdown_connections 2024-05-14T09:34:03.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.898+0000 7f31e500ab80 1 --2- 172.21.15.2:0/1216833925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2ea51d0 0x55d8f2ea55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.898+0000 7f31e500ab80 1 --2- 172.21.15.2:0/1216833925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2d22df0 0x55d8f2eaab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.898+0000 7f31e500ab80 1 --2- 172.21.15.2:0/1216833925 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2eab090 0x55d8f2ea4c90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.898+0000 7f31e500ab80 1 -- 172.21.15.2:0/1216833925 >> 172.21.15.2:0/1216833925 conn(0x55d8f2d6a5e0 msgr2=0x55d8f2e9b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:03.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.898+0000 7f31e500ab80 1 -- 172.21.15.2:0/1216833925 shutdown_connections 2024-05-14T09:34:03.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.898+0000 7f31e500ab80 1 -- 172.21.15.2:0/1216833925 wait complete. 2024-05-14T09:34:03.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.899+0000 7f31e500ab80 1 -- start start 2024-05-14T09:34:03.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.899+0000 7f31e500ab80 1 Processor -- start 2024-05-14T09:34:03.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.899+0000 7f31e500ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2d22df0 0x55d8f2eae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.899+0000 7f31e500ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2ea51d0 0x55d8f2eaeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.899+0000 7f31e500ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2eab090 0x55d8f2eaf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.899+0000 7f31e500ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d8f2eb1b90 con 0x55d8f2d22df0 2024-05-14T09:34:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.899+0000 7f31e500ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d8f2d7be80 con 0x55d8f2ea51d0 2024-05-14T09:34:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.899+0000 7f31e500ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d8f2eb1d10 con 0x55d8f2eab090 2024-05-14T09:34:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.899+0000 7f31e500ab80 10 client.0 _ll_register_callbacks cb 0x55d8f2d5a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:34:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.899+0000 7f31cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2d22df0 0x55d8f2eae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.899+0000 7f31cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2d22df0 0x55d8f2eae9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51078/0 (socket says 172.21.15.2:51078) 2024-05-14T09:34:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.899+0000 7f31cffff700 1 -- 172.21.15.2:0/2436540766 learned_addr learned my addr 172.21.15.2:0/2436540766 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.899+0000 7f31d4b93700 1 --2- 172.21.15.2:0/2436540766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2eab090 0x55d8f2eaf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.899+0000 7f31c7fff700 1 --2- 172.21.15.2:0/2436540766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2ea51d0 0x55d8f2eaeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.900+0000 7f31cffff700 1 -- 172.21.15.2:0/2436540766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2eab090 msgr2=0x55d8f2eaf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.900+0000 7f31cffff700 1 --2- 172.21.15.2:0/2436540766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2eab090 0x55d8f2eaf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.900+0000 7f31cffff700 1 -- 172.21.15.2:0/2436540766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2ea51d0 msgr2=0x55d8f2eaeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.900+0000 7f31cffff700 1 --2- 172.21.15.2:0/2436540766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2ea51d0 0x55d8f2eaeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.900+0000 7f31cffff700 1 -- 172.21.15.2:0/2436540766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8f2d79710 con 0x55d8f2d22df0 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.900+0000 7f31d4b93700 1 --2- 172.21.15.2:0/2436540766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2eab090 0x55d8f2eaf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.900+0000 7f31cffff700 1 --2- 172.21.15.2:0/2436540766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2d22df0 0x55d8f2eae9a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f31c00052d0 tx=0x7f31c001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2fb31f1a766650 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.900+0000 7f31af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.900+0000 7f31af7fe700 1 -- 172.21.15.2:0/2436540766 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c00056a0 con 0x55d8f2d22df0 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.900+0000 7f31af7fe700 1 -- 172.21.15.2:0/2436540766 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f31c000f460 con 0x55d8f2d22df0 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.900+0000 7f31e500ab80 1 -- 172.21.15.2:0/2436540766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31b8015070 con 0x55d8f2d22df0 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.900+0000 7f31af7fe700 1 -- 172.21.15.2:0/2436540766 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f31c003d050 con 0x55d8f2d22df0 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.900+0000 7f31e500ab80 10 client.15102 fetch_fsmap learned FSMap version 26 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.900+0000 7f31e500ab80 1 -- 172.21.15.2:0/2436540766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d8f2ebc860 con 0x55d8f2d22df0 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.901+0000 7f31af7fe700 1 -- 172.21.15.2:0/2436540766 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31b8015070 con 0x55d8f2d22df0 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.901+0000 7f31af7fe700 1 -- 172.21.15.2:0/2436540766 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f31c0022070 con 0x55d8f2d22df0 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.901+0000 7f31e500ab80 10 client.15102 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.901+0000 7f31e500ab80 10 client.15102 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.901+0000 7f31e500ab80 20 client.15102 populate_metadata read hostname 'smithi002' 2024-05-14T09:34:03.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.901+0000 7f31e500ab80 1 --2- 172.21.15.2:0/2436540766 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55d8f2ebef30 0x55d8f2ec03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.901+0000 7f31e500ab80 4 client.15102 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:34:03.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.901+0000 7f31e500ab80 1 -- 172.21.15.2:0/2436540766 --> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8f2dd02d0 con 0x55d8f2ebef30 2024-05-14T09:34:03.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.901+0000 7f31c7fff700 1 --2- 172.21.15.2:0/2436540766 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55d8f2ebef30 0x55d8f2ec03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.902+0000 7f31c7fff700 1 --2- 172.21.15.2:0/2436540766 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55d8f2ebef30 0x55d8f2ec03a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e14ca51dd8a98c43 server_cookie=e2a8343d175e0b43 in_seq=0 out_seq=0 2024-05-14T09:34:03.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.902+0000 7f31af7fe700 10 client.15102 ms_handle_connect on v2:172.21.15.26:6836/3301773578 2024-05-14T09:34:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.903+0000 7f31af7fe700 1 -- 172.21.15.2:0/2436540766 <== mds.1 v2:172.21.15.26:6836/3301773578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d8f2dd02d0 con 0x55d8f2ebef30 2024-05-14T09:34:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.903+0000 7f31af7fe700 10 client.15102 handle_command_reply: tid=0 2024-05-14T09:34:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.903+0000 7f31e500ab80 1 client.15102 shutdown 2024-05-14T09:34:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.903+0000 7f31e500ab80 1 -- 172.21.15.2:0/2436540766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2d22df0 msgr2=0x55d8f2eae9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.903+0000 7f31e500ab80 1 --2- 172.21.15.2:0/2436540766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2d22df0 0x55d8f2eae9a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f31c00052d0 tx=0x7f31c001a050 comp rx=0 tx=0).stop 2024-05-14T09:34:03.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.903+0000 7f31e500ab80 1 -- 172.21.15.2:0/2436540766 shutdown_connections 2024-05-14T09:34:03.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.903+0000 7f31e500ab80 1 --2- 172.21.15.2:0/2436540766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2eab090 0x55d8f2eaf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.903+0000 7f31e500ab80 1 --2- 172.21.15.2:0/2436540766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2d22df0 0x55d8f2eae9a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.903+0000 7f31e500ab80 1 --2- 172.21.15.2:0/2436540766 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55d8f2ebef30 0x55d8f2ec03a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.903+0000 7f31e500ab80 1 --2- 172.21.15.2:0/2436540766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2ea51d0 0x55d8f2eaeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.903+0000 7f31e500ab80 1 -- 172.21.15.2:0/2436540766 >> 172.21.15.2:0/2436540766 conn(0x55d8f2d6a5e0 msgr2=0x55d8f2e9f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:03.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.903+0000 7f31af7fe700 0 client.15102 ms_handle_reset on v2:172.21.15.26:6836/3301773578 2024-05-14T09:34:03.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.903+0000 7f31e500ab80 1 -- 172.21.15.2:0/2436540766 shutdown_connections 2024-05-14T09:34:03.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.903+0000 7f31e500ab80 1 -- 172.21.15.2:0/2436540766 wait complete. 2024-05-14T09:34:03.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.903+0000 7f31e500ab80 20 client.15102 trim_cache size 0 max 16384 2024-05-14T09:34:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31e500ab80 1 Processor -- start 2024-05-14T09:34:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31e500ab80 1 -- start start 2024-05-14T09:34:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31e500ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2d22df0 0x55d8f2eaf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31e500ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2ea51d0 0x55d8f2eb0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31e500ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2eab090 0x55d8f2ec9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31e500ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d8f2ebe6e0 con 0x55d8f2eab090 2024-05-14T09:34:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31e500ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d8f2ebedb0 con 0x55d8f2d22df0 2024-05-14T09:34:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31e500ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d8f2ebf480 con 0x55d8f2ea51d0 2024-05-14T09:34:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31d4b93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2eab090 0x55d8f2ec9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31d4b93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2eab090 0x55d8f2ec9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51092/0 (socket says 172.21.15.2:51092) 2024-05-14T09:34:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31d4b93700 1 -- 172.21.15.2:0/2722895656 learned_addr learned my addr 172.21.15.2:0/2722895656 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31c7fff700 1 --2- 172.21.15.2:0/2722895656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2ea51d0 0x55d8f2eb0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31cffff700 1 --2- 172.21.15.2:0/2722895656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2d22df0 0x55d8f2eaf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31d4b93700 1 -- 172.21.15.2:0/2722895656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2ea51d0 msgr2=0x55d8f2eb0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31d4b93700 1 --2- 172.21.15.2:0/2722895656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2ea51d0 0x55d8f2eb0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31d4b93700 1 -- 172.21.15.2:0/2722895656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2d22df0 msgr2=0x55d8f2eaf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:03.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31d4b93700 1 --2- 172.21.15.2:0/2722895656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2d22df0 0x55d8f2eaf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31d4b93700 1 -- 172.21.15.2:0/2722895656 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f2ebebd0 con 0x55d8f2eab090 2024-05-14T09:34:03.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31c7fff700 1 --2- 172.21.15.2:0/2722895656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2ea51d0 0x55d8f2eb0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:03.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31d4b93700 1 --2- 172.21.15.2:0/2722895656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2eab090 0x55d8f2ec9800 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009710 tx=0x7f31c8004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18c89a65d131301 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:03.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31c5ffb700 1 -- 172.21.15.2:0/2722895656 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c80a9130 con 0x55d8f2eab090 2024-05-14T09:34:03.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31c5ffb700 1 -- 172.21.15.2:0/2722895656 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c80ad4e0 con 0x55d8f2eab090 2024-05-14T09:34:03.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.926+0000 7f31c5ffb700 1 -- 172.21.15.2:0/2722895656 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c800db20 con 0x55d8f2eab090 2024-05-14T09:34:03.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.927+0000 7f31e500ab80 1 -- 172.21.15.2:0/2722895656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2eab090 msgr2=0x55d8f2ec9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:03.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.927+0000 7f31e500ab80 1 --2- 172.21.15.2:0/2722895656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2eab090 0x55d8f2ec9800 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009710 tx=0x7f31c8004f50 comp rx=0 tx=0).stop 2024-05-14T09:34:03.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.927+0000 7f31e500ab80 1 -- 172.21.15.2:0/2722895656 shutdown_connections 2024-05-14T09:34:03.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.927+0000 7f31e500ab80 1 --2- 172.21.15.2:0/2722895656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2d22df0 0x55d8f2eaf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.927+0000 7f31e500ab80 1 --2- 172.21.15.2:0/2722895656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2eab090 0x55d8f2ec9800 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.927+0000 7f31e500ab80 1 --2- 172.21.15.2:0/2722895656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2ea51d0 0x55d8f2eb0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.927+0000 7f31e500ab80 1 -- 172.21.15.2:0/2722895656 >> 172.21.15.2:0/2722895656 conn(0x55d8f2d6a5e0 msgr2=0x55d8f2ead290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:03.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.927+0000 7f31e500ab80 1 -- 172.21.15.2:0/2722895656 shutdown_connections 2024-05-14T09:34:03.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.927+0000 7f31e500ab80 1 -- 172.21.15.2:0/2722895656 wait complete. 2024-05-14T09:34:03.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.928+0000 7f31e500ab80 1 -- start start 2024-05-14T09:34:03.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.928+0000 7f31e500ab80 1 Processor -- start 2024-05-14T09:34:03.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.928+0000 7f31e500ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2d22df0 0x55d8f2eae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.928+0000 7f31e500ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2ea51d0 0x55d8f2eae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.928+0000 7f31e500ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2eab090 0x55d8f2ec8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.928+0000 7f31e500ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d8f2ec0220 con 0x55d8f2eab090 2024-05-14T09:34:03.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.928+0000 7f31e500ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d8f2ebfb50 con 0x55d8f2ea51d0 2024-05-14T09:34:03.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.928+0000 7f31e500ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d8f2ec08f0 con 0x55d8f2d22df0 2024-05-14T09:34:03.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.928+0000 7f31e500ab80 10 client.0 _ll_register_callbacks cb 0x55d8f2d5a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:34:03.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.928+0000 7f31d4b93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2eab090 0x55d8f2ec8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.928+0000 7f31d4b93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2eab090 0x55d8f2ec8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51100/0 (socket says 172.21.15.2:51100) 2024-05-14T09:34:03.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31d4b93700 1 -- 172.21.15.2:0/1979345192 learned_addr learned my addr 172.21.15.2:0/1979345192 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:03.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31c7fff700 1 --2- 172.21.15.2:0/1979345192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2ea51d0 0x55d8f2eae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31cffff700 1 --2- 172.21.15.2:0/1979345192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2d22df0 0x55d8f2eae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31d4b93700 1 -- 172.21.15.2:0/1979345192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2d22df0 msgr2=0x55d8f2eae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:03.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31d4b93700 1 --2- 172.21.15.2:0/1979345192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2d22df0 0x55d8f2eae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31d4b93700 1 -- 172.21.15.2:0/1979345192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2ea51d0 msgr2=0x55d8f2eae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:03.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31d4b93700 1 --2- 172.21.15.2:0/1979345192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2ea51d0 0x55d8f2eae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31d4b93700 1 -- 172.21.15.2:0/1979345192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8f2ec8d50 con 0x55d8f2eab090 2024-05-14T09:34:03.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31d4b93700 1 --2- 172.21.15.2:0/1979345192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2eab090 0x55d8f2ec8810 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009fd0 tx=0x7f31c80b2090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6a47168f1ce407 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:03.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:34:03.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31af7fe700 1 -- 172.21.15.2:0/1979345192 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c80b1d10 con 0x55d8f2eab090 2024-05-14T09:34:03.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31af7fe700 1 -- 172.21.15.2:0/1979345192 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f31c8068ba0 con 0x55d8f2eab090 2024-05-14T09:34:03.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31e500ab80 1 -- 172.21.15.2:0/1979345192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c80ad4e0 con 0x55d8f2eab090 2024-05-14T09:34:03.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31af7fe700 1 -- 172.21.15.2:0/1979345192 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f31c80ca1a0 con 0x55d8f2eab090 2024-05-14T09:34:03.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31e500ab80 10 client.15108 fetch_fsmap learned FSMap version 26 2024-05-14T09:34:03.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.929+0000 7f31e500ab80 1 -- 172.21.15.2:0/1979345192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d8f2ec9fd0 con 0x55d8f2eab090 2024-05-14T09:34:03.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.930+0000 7f31af7fe700 1 -- 172.21.15.2:0/1979345192 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c80ad4e0 con 0x55d8f2eab090 2024-05-14T09:34:03.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.930+0000 7f31af7fe700 1 -- 172.21.15.2:0/1979345192 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f31c80a4aa0 con 0x55d8f2eab090 2024-05-14T09:34:03.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.930+0000 7f31e500ab80 10 client.15108 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T09:34:03.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.930+0000 7f31e500ab80 10 client.15108 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T09:34:03.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.930+0000 7f31e500ab80 20 client.15108 populate_metadata read hostname 'smithi002' 2024-05-14T09:34:03.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.930+0000 7f31e500ab80 1 --2- 172.21.15.2:0/1979345192 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55d8f2f2bd70 0x55d8f2f2d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:03.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.930+0000 7f31e500ab80 4 client.15108 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T09:34:03.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.930+0000 7f31e500ab80 1 -- 172.21.15.2:0/1979345192 --> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] -- command(tid 0: {"prefix": "status"}) -- 0x55d8f2dca370 con 0x55d8f2f2bd70 2024-05-14T09:34:03.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.930+0000 7f31cffff700 1 --2- 172.21.15.2:0/1979345192 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55d8f2f2bd70 0x55d8f2f2d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:03.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31af7fe700 10 client.15108 ms_handle_connect on v2:172.21.15.26:6836/3301773578 2024-05-14T09:34:03.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31cffff700 1 --2- 172.21.15.2:0/1979345192 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55d8f2f2bd70 0x55d8f2f2d1e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fdecc49f0f124b35 server_cookie=da7de01812127f8b in_seq=0 out_seq=0 2024-05-14T09:34:03.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31af7fe700 1 -- 172.21.15.2:0/1979345192 <== mds.1 v2:172.21.15.26:6836/3301773578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d8f2dca370 con 0x55d8f2f2bd70 2024-05-14T09:34:03.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31af7fe700 10 client.15108 handle_command_reply: tid=0 2024-05-14T09:34:03.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31e500ab80 1 client.15108 shutdown 2024-05-14T09:34:03.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31e500ab80 1 -- 172.21.15.2:0/1979345192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2eab090 msgr2=0x55d8f2ec8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:03.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31e500ab80 1 --2- 172.21.15.2:0/1979345192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2eab090 0x55d8f2ec8810 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009fd0 tx=0x7f31c80b2090 comp rx=0 tx=0).stop 2024-05-14T09:34:03.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31e500ab80 1 -- 172.21.15.2:0/1979345192 shutdown_connections 2024-05-14T09:34:03.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31e500ab80 1 --2- 172.21.15.2:0/1979345192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8f2eab090 0x55d8f2ec8810 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31e500ab80 1 --2- 172.21.15.2:0/1979345192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8f2d22df0 0x55d8f2eae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31e500ab80 1 --2- 172.21.15.2:0/1979345192 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55d8f2f2bd70 0x55d8f2f2d1e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31e500ab80 1 --2- 172.21.15.2:0/1979345192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8f2ea51d0 0x55d8f2eae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31e500ab80 1 -- 172.21.15.2:0/1979345192 >> 172.21.15.2:0/1979345192 conn(0x55d8f2d6a5e0 msgr2=0x55d8f2ea9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:03.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31af7fe700 0 client.15108 ms_handle_reset on v2:172.21.15.26:6836/3301773578 2024-05-14T09:34:03.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31e500ab80 1 -- 172.21.15.2:0/1979345192 shutdown_connections 2024-05-14T09:34:03.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.931+0000 7f31e500ab80 1 -- 172.21.15.2:0/1979345192 wait complete. 2024-05-14T09:34:03.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.932+0000 7f31e500ab80 20 client.15108 trim_cache size 0 max 16384 2024-05-14T09:34:03.938 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:34:03.938 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "50d5aebe-11d3-11ef-bc98-c7b262605968", 2024-05-14T09:34:03.938 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 1, 2024-05-14T09:34:03.938 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24497, 2024-05-14T09:34:03.938 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T09:34:03.938 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T09:34:03.938 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T09:34:03.938 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 129.23443167400001, 2024-05-14T09:34:03.938 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 26, 2024-05-14T09:34:03.938 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 90, 2024-05-14T09:34:03.938 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 88, 2024-05-14T09:34:03.939 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 212.68518657600001 2024-05-14T09:34:03.939 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:34:03.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.932+0000 7f31d65f6700 1 -- 172.21.15.2:0/1384966118 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f31bc0b2880 msgr2=0x7f31bc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:03.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.932+0000 7f31d65f6700 1 --2- 172.21.15.2:0/1384966118 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f31bc0b2880 0x7f31bc0b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f31c0003800 tx=0x7f31c0005fb0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.932+0000 7f31d65f6700 1 -- 172.21.15.2:0/1384966118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31d0108740 msgr2=0x7f31d019c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:03.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.932+0000 7f31d65f6700 1 --2- 172.21.15.2:0/1384966118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31d0108740 0x7f31d019c6f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f31c8007f10 tx=0x7f31c800d3b0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.932+0000 7f31d65f6700 1 -- 172.21.15.2:0/1384966118 shutdown_connections 2024-05-14T09:34:03.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.932+0000 7f31d65f6700 1 --2- 172.21.15.2:0/1384966118 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f31bc0b2880 0x7f31bc0b4d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.932+0000 7f31d65f6700 1 --2- 172.21.15.2:0/1384966118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31d0108740 0x7f31d019c6f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.932+0000 7f31d65f6700 1 --2- 172.21.15.2:0/1384966118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f31d01034c0 0x7f31d019bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.932+0000 7f31d65f6700 1 --2- 172.21.15.2:0/1384966118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31d0107df0 0x7f31d019c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:03.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.932+0000 7f31d65f6700 1 -- 172.21.15.2:0/1384966118 >> 172.21.15.2:0/1384966118 conn(0x7f31d00fedf0 msgr2=0x7f31d0101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:03.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.932+0000 7f31d65f6700 1 -- 172.21.15.2:0/1384966118 shutdown_connections 2024-05-14T09:34:03.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:03.932+0000 7f31d65f6700 1 -- 172.21.15.2:0/1384966118 wait complete. 2024-05-14T09:34:04.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:04 smithi002 ceph-mon[31557]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:04.436 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:04 smithi029 ceph-mon[36249]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:04.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:04 smithi026 ceph-mon[36056]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:04.665 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-14T09:34:04.919 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:06.343 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:06 smithi002 ceph-mon[31557]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:06.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:06 smithi029 ceph-mon[36249]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:06.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:06 smithi026 ceph-mon[36056]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.596+0000 7fde916c0700 1 -- 172.21.15.2:0/2107500860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde8c1034c0 msgr2=0x7fde8c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.596+0000 7fde916c0700 1 --2- 172.21.15.2:0/2107500860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde8c1034c0 0x7fde8c1038d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0099c0 tx=0x7fde7c009cd0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.596+0000 7fde916c0700 1 -- 172.21.15.2:0/2107500860 shutdown_connections 2024-05-14T09:34:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.596+0000 7fde916c0700 1 --2- 172.21.15.2:0/2107500860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde8c108740 0x7fde8c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.596+0000 7fde916c0700 1 --2- 172.21.15.2:0/2107500860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde8c1034c0 0x7fde8c1038d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.596+0000 7fde916c0700 1 --2- 172.21.15.2:0/2107500860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c107df0 0x7fde8c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.596+0000 7fde916c0700 1 -- 172.21.15.2:0/2107500860 >> 172.21.15.2:0/2107500860 conn(0x7fde8c0fedf0 msgr2=0x7fde8c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.596+0000 7fde916c0700 1 -- 172.21.15.2:0/2107500860 shutdown_connections 2024-05-14T09:34:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.596+0000 7fde916c0700 1 -- 172.21.15.2:0/2107500860 wait complete. 2024-05-14T09:34:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.597+0000 7fde916c0700 1 Processor -- start 2024-05-14T09:34:06.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.597+0000 7fde916c0700 1 -- start start 2024-05-14T09:34:06.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.598+0000 7fde916c0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c1034c0 0x7fde8c19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.598+0000 7fde916c0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde8c107df0 0x7fde8c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.598+0000 7fde916c0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde8c108740 0x7fde8c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.598+0000 7fde916c0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c1144c0 con 0x7fde8c108740 2024-05-14T09:34:06.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.598+0000 7fde916c0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c114340 con 0x7fde8c1034c0 2024-05-14T09:34:06.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.598+0000 7fde916c0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c114640 con 0x7fde8c107df0 2024-05-14T09:34:06.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.598+0000 7fde8b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde8c108740 0x7fde8c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.598+0000 7fde8b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde8c108740 0x7fde8c19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51118/0 (socket says 172.21.15.2:51118) 2024-05-14T09:34:06.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.598+0000 7fde8b7fe700 1 -- 172.21.15.2:0/1649557579 learned_addr learned my addr 172.21.15.2:0/1649557579 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:06.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.598+0000 7fde8affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c1034c0 0x7fde8c19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.598+0000 7fde8a7fc700 1 --2- 172.21.15.2:0/1649557579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde8c107df0 0x7fde8c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.598+0000 7fde8b7fe700 1 -- 172.21.15.2:0/1649557579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde8c107df0 msgr2=0x7fde8c19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:06.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.598+0000 7fde8b7fe700 1 --2- 172.21.15.2:0/1649557579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde8c107df0 0x7fde8c19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.598+0000 7fde8b7fe700 1 -- 172.21.15.2:0/1649557579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c1034c0 msgr2=0x7fde8c19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:06.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.598+0000 7fde8b7fe700 1 --2- 172.21.15.2:0/1649557579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c1034c0 0x7fde8c19ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.599+0000 7fde8b7fe700 1 -- 172.21.15.2:0/1649557579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8c19fd60 con 0x7fde8c108740 2024-05-14T09:34:06.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.599+0000 7fde8affd700 1 --2- 172.21.15.2:0/1649557579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c1034c0 0x7fde8c19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:06.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.599+0000 7fde8a7fc700 1 --2- 172.21.15.2:0/1649557579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde8c107df0 0x7fde8c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:06.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.599+0000 7fde8b7fe700 1 --2- 172.21.15.2:0/1649557579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde8c108740 0x7fde8c19f6c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fde8400ef30 tx=0x7fde8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:06.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.599+0000 7fde73fff700 1 -- 172.21.15.2:0/1649557579 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde840092e0 con 0x7fde8c108740 2024-05-14T09:34:06.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.599+0000 7fde916c0700 1 -- 172.21.15.2:0/1649557579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde8c19fff0 con 0x7fde8c108740 2024-05-14T09:34:06.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.599+0000 7fde73fff700 1 -- 172.21.15.2:0/1649557579 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde84004bb0 con 0x7fde8c108740 2024-05-14T09:34:06.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.599+0000 7fde73fff700 1 -- 172.21.15.2:0/1649557579 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84004020 con 0x7fde8c108740 2024-05-14T09:34:06.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.599+0000 7fde916c0700 1 -- 172.21.15.2:0/1649557579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde8c1a7970 con 0x7fde8c108740 2024-05-14T09:34:06.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.602+0000 7fde73fff700 1 -- 172.21.15.2:0/1649557579 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fde8401b030 con 0x7fde8c108740 2024-05-14T09:34:06.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.602+0000 7fde73fff700 1 --2- 172.21.15.2:0/1649557579 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fde740b2880 0x7fde740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.603+0000 7fde8affd700 1 --2- 172.21.15.2:0/1649557579 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fde740b2880 0x7fde740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.603+0000 7fde73fff700 1 -- 172.21.15.2:0/1649557579 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fde840e2060 con 0x7fde8c108740 2024-05-14T09:34:06.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.603+0000 7fdea00d4b80 1 Processor -- start 2024-05-14T09:34:06.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.603+0000 7fdea00d4b80 1 -- start start 2024-05-14T09:34:06.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.603+0000 7fde8affd700 1 --2- 172.21.15.2:0/1649557579 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fde740b2880 0x7fde740b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fde7c000c00 tx=0x7fde7c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:06.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.603+0000 7fdea00d4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9eb97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.603+0000 7fdea00d4b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9eb3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.603+0000 7fdea00d4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9eb41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.603+0000 7fdea00d4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f6d9dd58f0 con 0x55f6d9d35df0 2024-05-14T09:34:06.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.603+0000 7fdea00d4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f6d9d3ad50 con 0x55f6d9eb3dc0 2024-05-14T09:34:06.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.603+0000 7fdea00d4b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f6d9dd18b0 con 0x55f6d9eb9d30 2024-05-14T09:34:06.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.604+0000 7fde8affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9eb97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.604+0000 7fde8affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9eb97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51120/0 (socket says 172.21.15.2:51120) 2024-05-14T09:34:06.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.604+0000 7fde8affd700 1 -- 172.21.15.2:0/1898519007 learned_addr learned my addr 172.21.15.2:0/1898519007 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:06.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.604+0000 7fde8b7fe700 1 --2- 172.21.15.2:0/1898519007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9eb41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.604+0000 7fde8a7fc700 1 --2- 172.21.15.2:0/1898519007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9eb3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.604+0000 7fde8affd700 1 -- 172.21.15.2:0/1898519007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 msgr2=0x55f6d9eb3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:06.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.604+0000 7fde8affd700 1 --2- 172.21.15.2:0/1898519007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9eb3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.604+0000 7fde8affd700 1 -- 172.21.15.2:0/1898519007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9eb3dc0 msgr2=0x55f6d9eb41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:06.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.604+0000 7fde8affd700 1 --2- 172.21.15.2:0/1898519007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9eb41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.604+0000 7fde8affd700 1 -- 172.21.15.2:0/1898519007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6d9d22880 con 0x55f6d9d35df0 2024-05-14T09:34:06.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.604+0000 7fde8b7fe700 1 --2- 172.21.15.2:0/1898519007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9eb41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:06.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.604+0000 7fde8affd700 1 --2- 172.21.15.2:0/1898519007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9eb97f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00a010 tx=0x7fde7c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695b86c1ca6bacbf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:06.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.604+0000 7fde717fa700 1 -- 172.21.15.2:0/1898519007 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c0037c0 con 0x55f6d9d35df0 2024-05-14T09:34:06.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.604+0000 7fde717fa700 1 -- 172.21.15.2:0/1898519007 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7c003e10 con 0x55f6d9d35df0 2024-05-14T09:34:06.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.604+0000 7fde717fa700 1 -- 172.21.15.2:0/1898519007 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c030db0 con 0x55f6d9d35df0 2024-05-14T09:34:06.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.605+0000 7fdea00d4b80 1 -- 172.21.15.2:0/1898519007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 msgr2=0x55f6d9eb97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:06.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.605+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/1898519007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9eb97f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00a010 tx=0x7fde7c022000 comp rx=0 tx=0).stop 2024-05-14T09:34:06.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.605+0000 7fdea00d4b80 1 -- 172.21.15.2:0/1898519007 shutdown_connections 2024-05-14T09:34:06.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.605+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/1898519007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9eb41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.605+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/1898519007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9eb97f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.605+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/1898519007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9eb3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.605+0000 7fdea00d4b80 1 -- 172.21.15.2:0/1898519007 >> 172.21.15.2:0/1898519007 conn(0x55f6d9d7d5e0 msgr2=0x55f6d9eae9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:06.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.605+0000 7fdea00d4b80 1 -- 172.21.15.2:0/1898519007 shutdown_connections 2024-05-14T09:34:06.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.605+0000 7fdea00d4b80 1 -- 172.21.15.2:0/1898519007 wait complete. 2024-05-14T09:34:06.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.606+0000 7fdea00d4b80 1 -- start start 2024-05-14T09:34:06.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.606+0000 7fdea00d4b80 1 Processor -- start 2024-05-14T09:34:06.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.606+0000 7fdea00d4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9d35df0 0x55f6d9ebd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.606+0000 7fdea00d4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9ebd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.606+0000 7fdea00d4b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9ebdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.606+0000 7fdea00d4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f6d9ec0770 con 0x55f6d9eb3dc0 2024-05-14T09:34:06.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.606+0000 7fdea00d4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f6d9d8ee80 con 0x55f6d9d35df0 2024-05-14T09:34:06.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.606+0000 7fdea00d4b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f6d9ec08f0 con 0x55f6d9eb9d30 2024-05-14T09:34:06.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.606+0000 7fdea00d4b80 10 client.0 _ll_register_callbacks cb 0x55f6d9d6d900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:34:06.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.606+0000 7fde8a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9ebd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.606+0000 7fde8a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9ebd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51130/0 (socket says 172.21.15.2:51130) 2024-05-14T09:34:06.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.607+0000 7fde8a7fc700 1 -- 172.21.15.2:0/3529496135 learned_addr learned my addr 172.21.15.2:0/3529496135 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:06.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.607+0000 7fde8affd700 1 --2- 172.21.15.2:0/3529496135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9d35df0 0x55f6d9ebd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.607+0000 7fde8b7fe700 1 --2- 172.21.15.2:0/3529496135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9ebdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.607+0000 7fde8a7fc700 1 -- 172.21.15.2:0/3529496135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 msgr2=0x55f6d9ebdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:06.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.607+0000 7fde8a7fc700 1 --2- 172.21.15.2:0/3529496135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9ebdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.607+0000 7fde8a7fc700 1 -- 172.21.15.2:0/3529496135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9d35df0 msgr2=0x55f6d9ebd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.607+0000 7fde8a7fc700 1 --2- 172.21.15.2:0/3529496135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9d35df0 0x55f6d9ebd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.607+0000 7fde8a7fc700 1 -- 172.21.15.2:0/3529496135 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6d9d8c710 con 0x55f6d9eb3dc0 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.607+0000 7fde8a7fc700 1 --2- 172.21.15.2:0/3529496135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9ebd9d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fde800117e0 tx=0x7fde80011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfc28ce1fc6dc25 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.607+0000 7fde6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.607+0000 7fde6eff5700 1 -- 172.21.15.2:0/3529496135 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80004020 con 0x55f6d9eb3dc0 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.607+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3529496135 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde7c003e10 con 0x55f6d9eb3dc0 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.607+0000 7fde6eff5700 1 -- 172.21.15.2:0/3529496135 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fde800041c0 con 0x55f6d9eb3dc0 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.608+0000 7fde6eff5700 1 -- 172.21.15.2:0/3529496135 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fde80024050 con 0x55f6d9eb3dc0 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.608+0000 7fdea00d4b80 10 client.15120 fetch_fsmap learned FSMap version 26 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.608+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3529496135 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f6d9ecb390 con 0x55f6d9eb3dc0 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.608+0000 7fde6eff5700 1 -- 172.21.15.2:0/3529496135 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7c003e10 con 0x55f6d9eb3dc0 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.608+0000 7fde6eff5700 1 -- 172.21.15.2:0/3529496135 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fde800038b0 con 0x55f6d9eb3dc0 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.608+0000 7fdea00d4b80 10 client.15120 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.608+0000 7fdea00d4b80 10 client.15120 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.608+0000 7fdea00d4b80 20 client.15120 populate_metadata read hostname 'smithi002' 2024-05-14T09:34:06.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.608+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/3529496135 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f6d9ecd9e0 0x55f6d9ecee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.608+0000 7fdea00d4b80 4 client.15120 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:34:06.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.608+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3529496135 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6d9de32d0 con 0x55f6d9ecd9e0 2024-05-14T09:34:06.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.608+0000 7fde8affd700 1 --2- 172.21.15.2:0/3529496135 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f6d9ecd9e0 0x55f6d9ecee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.609+0000 7fde8affd700 1 --2- 172.21.15.2:0/3529496135 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f6d9ecd9e0 0x55f6d9ecee50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2feddaf891cf29cb server_cookie=15230658085f34c5 in_seq=0 out_seq=0 2024-05-14T09:34:06.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.609+0000 7fde6eff5700 10 client.15120 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:34:06.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.610+0000 7fde6eff5700 1 -- 172.21.15.2:0/3529496135 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f6d9de32d0 con 0x55f6d9ecd9e0 2024-05-14T09:34:06.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.610+0000 7fde6eff5700 10 client.15120 handle_command_reply: tid=0 2024-05-14T09:34:06.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.610+0000 7fdea00d4b80 1 client.15120 shutdown 2024-05-14T09:34:06.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.610+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3529496135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9eb3dc0 msgr2=0x55f6d9ebd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:06.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.610+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/3529496135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9ebd9d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fde800117e0 tx=0x7fde80011af0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.610+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3529496135 shutdown_connections 2024-05-14T09:34:06.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.610+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/3529496135 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f6d9ecd9e0 0x55f6d9ecee50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.610+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/3529496135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9d35df0 0x55f6d9ebd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.610+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/3529496135 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9ebdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.610+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/3529496135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9ebd9d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.610+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3529496135 >> 172.21.15.2:0/3529496135 conn(0x55f6d9d7d5e0 msgr2=0x55f6d9eb2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:06.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.610+0000 7fde6eff5700 0 client.15120 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:34:06.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.610+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3529496135 shutdown_connections 2024-05-14T09:34:06.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.610+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3529496135 wait complete. 2024-05-14T09:34:06.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.610+0000 7fdea00d4b80 20 client.15120 trim_cache size 0 max 16384 2024-05-14T09:34:06.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fdea00d4b80 1 Processor -- start 2024-05-14T09:34:06.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fdea00d4b80 1 -- start start 2024-05-14T09:34:06.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fdea00d4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9eb3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fdea00d4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9eb41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fdea00d4b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9ec9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fdea00d4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f6d9ecd190 con 0x55f6d9d35df0 2024-05-14T09:34:06.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fdea00d4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f6d9ecd860 con 0x55f6d9eb3dc0 2024-05-14T09:34:06.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fdea00d4b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f6d9ecdf30 con 0x55f6d9eb9d30 2024-05-14T09:34:06.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fde8affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9eb3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fde8affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9eb3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51134/0 (socket says 172.21.15.2:51134) 2024-05-14T09:34:06.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fde8affd700 1 -- 172.21.15.2:0/1539200908 learned_addr learned my addr 172.21.15.2:0/1539200908 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:06.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fde8affd700 1 -- 172.21.15.2:0/1539200908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 msgr2=0x55f6d9ec9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:06.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fde8affd700 1 --2- 172.21.15.2:0/1539200908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9ec9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fde8affd700 1 -- 172.21.15.2:0/1539200908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9eb3dc0 msgr2=0x55f6d9eb41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:06.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fde8affd700 1 --2- 172.21.15.2:0/1539200908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9eb41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.632+0000 7fde8affd700 1 -- 172.21.15.2:0/1539200908 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6d9eaf6d0 con 0x55f6d9d35df0 2024-05-14T09:34:06.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.633+0000 7fde8affd700 1 --2- 172.21.15.2:0/1539200908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9eb3880 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fde7c005b40 tx=0x7fde7c005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a322dd3203399a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:06.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.633+0000 7fde70ff9700 1 -- 172.21.15.2:0/1539200908 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c0039c0 con 0x55f6d9d35df0 2024-05-14T09:34:06.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.633+0000 7fde70ff9700 1 -- 172.21.15.2:0/1539200908 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7c003b60 con 0x55f6d9d35df0 2024-05-14T09:34:06.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.633+0000 7fde70ff9700 1 -- 172.21.15.2:0/1539200908 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00fe60 con 0x55f6d9d35df0 2024-05-14T09:34:06.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.633+0000 7fdea00d4b80 1 -- 172.21.15.2:0/1539200908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 msgr2=0x55f6d9eb3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:06.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.633+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/1539200908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9eb3880 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fde7c005b40 tx=0x7fde7c005ad0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 1 -- 172.21.15.2:0/1539200908 shutdown_connections 2024-05-14T09:34:06.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/1539200908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9ec9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/1539200908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9eb3880 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/1539200908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9eb41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 1 -- 172.21.15.2:0/1539200908 >> 172.21.15.2:0/1539200908 conn(0x55f6d9d7d5e0 msgr2=0x55f6d9eb8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:06.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 1 -- 172.21.15.2:0/1539200908 shutdown_connections 2024-05-14T09:34:06.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 1 -- 172.21.15.2:0/1539200908 wait complete. 2024-05-14T09:34:06.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 1 -- start start 2024-05-14T09:34:06.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 1 Processor -- start 2024-05-14T09:34:06.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9ec8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9ec90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9ec9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f6d9ececd0 con 0x55f6d9d35df0 2024-05-14T09:34:06.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f6d9ece600 con 0x55f6d9eb3dc0 2024-05-14T09:34:06.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f6d9ecf3a0 con 0x55f6d9eb9d30 2024-05-14T09:34:06.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.634+0000 7fdea00d4b80 10 client.0 _ll_register_callbacks cb 0x55f6d9d6d900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:34:06.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fde8affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9ec8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fde8affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9ec8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51136/0 (socket says 172.21.15.2:51136) 2024-05-14T09:34:06.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fde8affd700 1 -- 172.21.15.2:0/3603795106 learned_addr learned my addr 172.21.15.2:0/3603795106 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:06.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fde8b7fe700 1 --2- 172.21.15.2:0/3603795106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9ec9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fde8a7fc700 1 --2- 172.21.15.2:0/3603795106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9ec90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fde8affd700 1 -- 172.21.15.2:0/3603795106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 msgr2=0x55f6d9ec9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:06.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fde8affd700 1 --2- 172.21.15.2:0/3603795106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9ec9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fde8affd700 1 -- 172.21.15.2:0/3603795106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9eb3dc0 msgr2=0x55f6d9ec90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:06.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fde8affd700 1 --2- 172.21.15.2:0/3603795106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9ec90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fde8affd700 1 -- 172.21.15.2:0/3603795106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6d9ed73c0 con 0x55f6d9d35df0 2024-05-14T09:34:06.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fde8b7fe700 1 --2- 172.21.15.2:0/3603795106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9ec9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:34:06.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fde8affd700 1 --2- 172.21.15.2:0/3603795106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9ec8b80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00b600 tx=0x7fde7c003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d338f29ab29e994f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:06.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fde6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:34:06.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fde6eff5700 1 -- 172.21.15.2:0/3603795106 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c04b060 con 0x55f6d9d35df0 2024-05-14T09:34:06.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fde6eff5700 1 -- 172.21.15.2:0/3603795106 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fde7c00f500 con 0x55f6d9d35df0 2024-05-14T09:34:06.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.635+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3603795106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde7c003b60 con 0x55f6d9d35df0 2024-05-14T09:34:06.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.636+0000 7fde6eff5700 1 -- 172.21.15.2:0/3603795106 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fde7c00f700 con 0x55f6d9d35df0 2024-05-14T09:34:06.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.636+0000 7fdea00d4b80 10 client.15126 fetch_fsmap learned FSMap version 26 2024-05-14T09:34:06.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.636+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3603795106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f6d9ed8620 con 0x55f6d9d35df0 2024-05-14T09:34:06.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.636+0000 7fde6eff5700 1 -- 172.21.15.2:0/3603795106 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7c003b60 con 0x55f6d9d35df0 2024-05-14T09:34:06.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.636+0000 7fde6eff5700 1 -- 172.21.15.2:0/3603795106 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fde7c045070 con 0x55f6d9d35df0 2024-05-14T09:34:06.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.636+0000 7fdea00d4b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T09:34:06.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.636+0000 7fdea00d4b80 10 client.15126 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T09:34:06.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.636+0000 7fdea00d4b80 20 client.15126 populate_metadata read hostname 'smithi002' 2024-05-14T09:34:06.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.636+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/3603795106 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f6d9f3a470 0x55f6d9f3b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:06.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.636+0000 7fdea00d4b80 4 client.15126 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T09:34:06.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.636+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3603795106 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "status"}) -- 0x55f6d9ddd370 con 0x55f6d9f3a470 2024-05-14T09:34:06.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.636+0000 7fde8a7fc700 1 --2- 172.21.15.2:0/3603795106 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f6d9f3a470 0x55f6d9f3b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:06.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.637+0000 7fde6eff5700 10 client.15126 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:34:06.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.637+0000 7fde8a7fc700 1 --2- 172.21.15.2:0/3603795106 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f6d9f3a470 0x55f6d9f3b8e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6b74c0c3ce97bca server_cookie=fc2ff05d6ddf2858 in_seq=0 out_seq=0 2024-05-14T09:34:06.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.637+0000 7fde6eff5700 1 -- 172.21.15.2:0/3603795106 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55f6d9ddd370 con 0x55f6d9f3a470 2024-05-14T09:34:06.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.637+0000 7fde6eff5700 10 client.15126 handle_command_reply: tid=0 2024-05-14T09:34:06.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.637+0000 7fdea00d4b80 1 client.15126 shutdown 2024-05-14T09:34:06.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.637+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3603795106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 msgr2=0x55f6d9ec8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:06.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.637+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/3603795106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9ec8b80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00b600 tx=0x7fde7c003dc0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.637+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3603795106 shutdown_connections 2024-05-14T09:34:06.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.637+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/3603795106 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f6d9f3a470 0x55f6d9f3b8e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.637+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/3603795106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f6d9eb9d30 0x55f6d9ec9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.637+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/3603795106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6d9d35df0 0x55f6d9ec8b80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.637+0000 7fdea00d4b80 1 --2- 172.21.15.2:0/3603795106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6d9eb3dc0 0x55f6d9ec90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.637+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3603795106 >> 172.21.15.2:0/3603795106 conn(0x55f6d9d7d5e0 msgr2=0x55f6d9ea8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:06.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.637+0000 7fde6eff5700 0 client.15126 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:34:06.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.637+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3603795106 shutdown_connections 2024-05-14T09:34:06.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.638+0000 7fdea00d4b80 1 -- 172.21.15.2:0/3603795106 wait complete. 2024-05-14T09:34:06.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.638+0000 7fdea00d4b80 20 client.15126 trim_cache size 0 max 16384 2024-05-14T09:34:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:34:06.642 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "50d5aebe-11d3-11ef-bc98-c7b262605968", 2024-05-14T09:34:06.642 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2024-05-14T09:34:06.642 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24524, 2024-05-14T09:34:06.642 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T09:34:06.642 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T09:34:06.642 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T09:34:06.642 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 187.851968715, 2024-05-14T09:34:06.642 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 26, 2024-05-14T09:34:06.643 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 90, 2024-05-14T09:34:06.643 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 88, 2024-05-14T09:34:06.643 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 202.942388155 2024-05-14T09:34:06.643 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:34:06.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.638+0000 7fde916c0700 1 -- 172.21.15.2:0/1649557579 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fde740b2880 msgr2=0x7fde740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:06.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.638+0000 7fde916c0700 1 --2- 172.21.15.2:0/1649557579 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fde740b2880 0x7fde740b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fde7c000c00 tx=0x7fde7c01a040 comp rx=0 tx=0).stop 2024-05-14T09:34:06.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.638+0000 7fde916c0700 1 -- 172.21.15.2:0/1649557579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde8c108740 msgr2=0x7fde8c19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:06.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.638+0000 7fde916c0700 1 --2- 172.21.15.2:0/1649557579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde8c108740 0x7fde8c19f6c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fde8400ef30 tx=0x7fde8400c3a0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.638+0000 7fde916c0700 1 -- 172.21.15.2:0/1649557579 shutdown_connections 2024-05-14T09:34:06.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.638+0000 7fde916c0700 1 --2- 172.21.15.2:0/1649557579 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fde740b2880 0x7fde740b4d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.638+0000 7fde916c0700 1 --2- 172.21.15.2:0/1649557579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde8c1034c0 0x7fde8c19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.638+0000 7fde916c0700 1 --2- 172.21.15.2:0/1649557579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde8c108740 0x7fde8c19f6c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.638+0000 7fde916c0700 1 --2- 172.21.15.2:0/1649557579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fde8c107df0 0x7fde8c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:06.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.638+0000 7fde916c0700 1 -- 172.21.15.2:0/1649557579 >> 172.21.15.2:0/1649557579 conn(0x7fde8c0fedf0 msgr2=0x7fde8c1075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:06.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.638+0000 7fde916c0700 1 -- 172.21.15.2:0/1649557579 shutdown_connections 2024-05-14T09:34:06.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:06.638+0000 7fde916c0700 1 -- 172.21.15.2:0/1649557579 wait complete. 2024-05-14T09:34:07.440 INFO:tasks.cephadm:Setup complete, yielding 2024-05-14T09:34:07.441 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-14T09:34:07.453 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T09:34:07.454 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-14T09:34:07.709 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:08.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:08 smithi002 ceph-mon[31557]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:34:08.436 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:08 smithi029 ceph-mon[36249]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:34:08.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:08 smithi026 ceph-mon[36056]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:34:09.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.334+0000 7f5cc3c7f700 1 -- 172.21.15.2:0/1598544641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cbc104370 msgr2=0x7f5cbc104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:09.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.334+0000 7f5cc3c7f700 1 --2- 172.21.15.2:0/1598544641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cbc104370 0x7f5cbc104780 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009990 tx=0x7f5cb8009ca0 comp rx=0 tx=0).stop 2024-05-14T09:34:09.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.334+0000 7f5cc3c7f700 1 -- 172.21.15.2:0/1598544641 shutdown_connections 2024-05-14T09:34:09.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.334+0000 7f5cc3c7f700 1 --2- 172.21.15.2:0/1598544641 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cbc105e40 0x7f5cbc10c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:09.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.334+0000 7f5cc3c7f700 1 --2- 172.21.15.2:0/1598544641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cbc104370 0x7f5cbc104780 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:09.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.334+0000 7f5cc3c7f700 1 --2- 172.21.15.2:0/1598544641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc1054a0 0x7f5cbc1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:09.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.334+0000 7f5cc3c7f700 1 -- 172.21.15.2:0/1598544641 >> 172.21.15.2:0/1598544641 conn(0x7f5cbc0ff9d0 msgr2=0x7f5cbc101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:09.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.334+0000 7f5cc3c7f700 1 -- 172.21.15.2:0/1598544641 shutdown_connections 2024-05-14T09:34:09.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.334+0000 7f5cc3c7f700 1 -- 172.21.15.2:0/1598544641 wait complete. 2024-05-14T09:34:09.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.334+0000 7f5cc3c7f700 1 Processor -- start 2024-05-14T09:34:09.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.334+0000 7f5cc3c7f700 1 -- start start 2024-05-14T09:34:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc3c7f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cbc104370 0x7f5cbc19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc3c7f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc1054a0 0x7f5cbc19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc3c7f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cbc105e40 0x7f5cbc1a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc3c7f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc10fdb0 con 0x7f5cbc104370 2024-05-14T09:34:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc3c7f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc10fc30 con 0x7f5cbc1054a0 2024-05-14T09:34:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc3c7f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc10ff30 con 0x7f5cbc105e40 2024-05-14T09:34:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc1a1b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cbc104370 0x7f5cbc19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc1a1b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cbc104370 0x7f5cbc19a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57986/0 (socket says 172.21.15.2:57986) 2024-05-14T09:34:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc1a1b700 1 -- 172.21.15.2:0/3479090437 learned_addr learned my addr 172.21.15.2:0/3479090437 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc221c700 1 --2- 172.21.15.2:0/3479090437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cbc105e40 0x7f5cbc1a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc121a700 1 --2- 172.21.15.2:0/3479090437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc1054a0 0x7f5cbc19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:09.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc1a1b700 1 -- 172.21.15.2:0/3479090437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cbc105e40 msgr2=0x7f5cbc1a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:09.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc1a1b700 1 --2- 172.21.15.2:0/3479090437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cbc105e40 0x7f5cbc1a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:09.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc1a1b700 1 -- 172.21.15.2:0/3479090437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc1054a0 msgr2=0x7f5cbc19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:09.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc1a1b700 1 --2- 172.21.15.2:0/3479090437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc1054a0 0x7f5cbc19ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:09.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc1a1b700 1 -- 172.21.15.2:0/3479090437 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cbc1a24b0 con 0x7f5cbc104370 2024-05-14T09:34:09.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.335+0000 7f5cc1a1b700 1 --2- 172.21.15.2:0/3479090437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cbc104370 0x7f5cbc19a950 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8000c00 tx=0x7f5cb80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:09.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.336+0000 7f5cb2ffd700 1 -- 172.21.15.2:0/3479090437 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8004830 con 0x7f5cbc104370 2024-05-14T09:34:09.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.336+0000 7f5cb2ffd700 1 -- 172.21.15.2:0/3479090437 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb8003c00 con 0x7f5cbc104370 2024-05-14T09:34:09.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.336+0000 7f5cb2ffd700 1 -- 172.21.15.2:0/3479090437 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8005360 con 0x7f5cbc104370 2024-05-14T09:34:09.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.336+0000 7f5cc3c7f700 1 -- 172.21.15.2:0/3479090437 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cbc1a2740 con 0x7f5cbc104370 2024-05-14T09:34:09.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.336+0000 7f5cc3c7f700 1 -- 172.21.15.2:0/3479090437 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cbc1a2b90 con 0x7f5cbc104370 2024-05-14T09:34:09.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.337+0000 7f5cc3c7f700 1 -- 172.21.15.2:0/3479090437 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cbc0512c0 con 0x7f5cbc104370 2024-05-14T09:34:09.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.338+0000 7f5cb2ffd700 1 -- 172.21.15.2:0/3479090437 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5cb8005500 con 0x7f5cbc104370 2024-05-14T09:34:09.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.339+0000 7f5cb2ffd700 1 --2- 172.21.15.2:0/3479090437 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5ca80b2900 0x7f5ca80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:09.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.340+0000 7f5cc121a700 1 --2- 172.21.15.2:0/3479090437 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5ca80b2900 0x7f5ca80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:09.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.340+0000 7f5cb2ffd700 1 -- 172.21.15.2:0/3479090437 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f5cb80e3510 con 0x7f5cbc104370 2024-05-14T09:34:09.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.340+0000 7f5cc121a700 1 --2- 172.21.15.2:0/3479090437 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5ca80b2900 0x7f5ca80b4e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5cac005950 tx=0x7f5cac0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.345+0000 7f5cb2ffd700 1 -- 172.21.15.2:0/3479090437 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5cb8013070 con 0x7f5cbc104370 2024-05-14T09:34:09.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.574+0000 7f5cc3c7f700 1 -- 172.21.15.2:0/3479090437 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f5cbc063cc0 con 0x7f5ca80b2900 2024-05-14T09:34:09.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.575+0000 7f5cb2ffd700 1 -- 172.21.15.2:0/3479090437 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f5cbc063cc0 con 0x7f5ca80b2900 2024-05-14T09:34:09.575 INFO:teuthology.orchestra.run.smithi002.stdout:Backend: cephadm 2024-05-14T09:34:09.575 INFO:teuthology.orchestra.run.smithi002.stdout:Available: Yes 2024-05-14T09:34:09.575 INFO:teuthology.orchestra.run.smithi002.stdout:Paused: No 2024-05-14T09:34:09.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.578+0000 7f5cc3c7f700 1 -- 172.21.15.2:0/3479090437 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5ca80b2900 msgr2=0x7f5ca80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:09.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.578+0000 7f5cc3c7f700 1 --2- 172.21.15.2:0/3479090437 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5ca80b2900 0x7f5ca80b4e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5cac005950 tx=0x7f5cac0058e0 comp rx=0 tx=0).stop 2024-05-14T09:34:09.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.578+0000 7f5cc3c7f700 1 -- 172.21.15.2:0/3479090437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cbc104370 msgr2=0x7f5cbc19a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:09.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.578+0000 7f5cc3c7f700 1 --2- 172.21.15.2:0/3479090437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cbc104370 0x7f5cbc19a950 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8000c00 tx=0x7f5cb80047b0 comp rx=0 tx=0).stop 2024-05-14T09:34:09.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.578+0000 7f5cc3c7f700 1 -- 172.21.15.2:0/3479090437 shutdown_connections 2024-05-14T09:34:09.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.578+0000 7f5cc3c7f700 1 --2- 172.21.15.2:0/3479090437 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5ca80b2900 0x7f5ca80b4e10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:09.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.578+0000 7f5cc3c7f700 1 --2- 172.21.15.2:0/3479090437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cbc105e40 0x7f5cbc1a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:09.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.578+0000 7f5cc3c7f700 1 --2- 172.21.15.2:0/3479090437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cbc104370 0x7f5cbc19a950 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:09.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.578+0000 7f5cc3c7f700 1 --2- 172.21.15.2:0/3479090437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc1054a0 0x7f5cbc19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:09.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.578+0000 7f5cc3c7f700 1 -- 172.21.15.2:0/3479090437 >> 172.21.15.2:0/3479090437 conn(0x7f5cbc0ff9d0 msgr2=0x7f5cbc1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:09.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.578+0000 7f5cc3c7f700 1 -- 172.21.15.2:0/3479090437 shutdown_connections 2024-05-14T09:34:09.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:09.578+0000 7f5cc3c7f700 1 -- 172.21.15.2:0/3479090437 wait complete. 2024-05-14T09:34:10.275 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:10 smithi002 ceph-mon[31557]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:10.305 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-14T09:34:10.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:10 smithi029 ceph-mon[36249]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:10.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:10 smithi026 ceph-mon[36056]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:10.558 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:11.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:11 smithi002 ceph-mon[31557]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:34:11.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:11 smithi029 ceph-mon[36249]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:34:11.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:11 smithi026 ceph-mon[36056]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:34:12.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.287+0000 7f4534810700 1 -- 172.21.15.2:0/1415011615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530104d10 msgr2=0x7f453010b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:12.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.287+0000 7f4534810700 1 --2- 172.21.15.2:0/1415011615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530104d10 0x7f453010b5c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4528009990 tx=0x7f4528009ca0 comp rx=0 tx=0).stop 2024-05-14T09:34:12.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.287+0000 7f4534810700 1 -- 172.21.15.2:0/1415011615 shutdown_connections 2024-05-14T09:34:12.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.287+0000 7f4534810700 1 --2- 172.21.15.2:0/1415011615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4530103240 0x7f4530103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:12.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.287+0000 7f4534810700 1 --2- 172.21.15.2:0/1415011615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530104d10 0x7f453010b5c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:12.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.287+0000 7f4534810700 1 --2- 172.21.15.2:0/1415011615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4530104370 0x7f4530104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:12.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.287+0000 7f4534810700 1 -- 172.21.15.2:0/1415011615 >> 172.21.15.2:0/1415011615 conn(0x7f45300fe880 msgr2=0x7f4530100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:12.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.288+0000 7f4534810700 1 -- 172.21.15.2:0/1415011615 shutdown_connections 2024-05-14T09:34:12.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.288+0000 7f4534810700 1 -- 172.21.15.2:0/1415011615 wait complete. 2024-05-14T09:34:12.289 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:12 smithi002 ceph-mon[31557]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:12.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.288+0000 7f4534810700 1 Processor -- start 2024-05-14T09:34:12.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f4534810700 1 -- start start 2024-05-14T09:34:12.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f4534810700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530103240 0x7f4530199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:12.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f4534810700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4530104370 0x7f4530199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:12.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f4534810700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4530104d10 0x7f45301a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:12.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f4534810700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f453010ed20 con 0x7f4530103240 2024-05-14T09:34:12.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f4534810700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f453010eba0 con 0x7f4530104370 2024-05-14T09:34:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f4534810700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f453010eea0 con 0x7f4530104d10 2024-05-14T09:34:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f452e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530103240 0x7f4530199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f452e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530103240 0x7f4530199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58000/0 (socket says 172.21.15.2:58000) 2024-05-14T09:34:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f452e59c700 1 -- 172.21.15.2:0/2040390301 learned_addr learned my addr 172.21.15.2:0/2040390301 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f452dd9b700 1 --2- 172.21.15.2:0/2040390301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4530104370 0x7f4530199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f452ed9d700 1 --2- 172.21.15.2:0/2040390301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4530104d10 0x7f45301a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f452e59c700 1 -- 172.21.15.2:0/2040390301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4530104d10 msgr2=0x7f45301a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f452e59c700 1 --2- 172.21.15.2:0/2040390301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4530104d10 0x7f45301a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f452e59c700 1 -- 172.21.15.2:0/2040390301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4530104370 msgr2=0x7f4530199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f452e59c700 1 --2- 172.21.15.2:0/2040390301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4530104370 0x7f4530199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f452e59c700 1 -- 172.21.15.2:0/2040390301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453006b6c0 con 0x7f4530103240 2024-05-14T09:34:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.289+0000 7f452ed9d700 1 --2- 172.21.15.2:0/2040390301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4530104d10 0x7f45301a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:34:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.290+0000 7f452e59c700 1 --2- 172.21.15.2:0/2040390301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530103240 0x7f4530199830 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f452000ada0 tx=0x7f452000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.290+0000 7f45277fe700 1 -- 172.21.15.2:0/2040390301 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520017020 con 0x7f4530103240 2024-05-14T09:34:12.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.290+0000 7f45277fe700 1 -- 172.21.15.2:0/2040390301 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f452000a0d0 con 0x7f4530103240 2024-05-14T09:34:12.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.290+0000 7f45277fe700 1 -- 172.21.15.2:0/2040390301 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520004280 con 0x7f4530103240 2024-05-14T09:34:12.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.290+0000 7f4534810700 1 -- 172.21.15.2:0/2040390301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453006b9b0 con 0x7f4530103240 2024-05-14T09:34:12.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.290+0000 7f4534810700 1 -- 172.21.15.2:0/2040390301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f453006bf20 con 0x7f4530103240 2024-05-14T09:34:12.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.292+0000 7f4534810700 1 -- 172.21.15.2:0/2040390301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45300512c0 con 0x7f4530103240 2024-05-14T09:34:12.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.293+0000 7f45277fe700 1 -- 172.21.15.2:0/2040390301 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4520003a20 con 0x7f4530103240 2024-05-14T09:34:12.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.294+0000 7f45277fe700 1 --2- 172.21.15.2:0/2040390301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f451c0b2900 0x7f451c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:12.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.294+0000 7f452dd9b700 1 --2- 172.21.15.2:0/2040390301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f451c0b2900 0x7f451c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:12.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.294+0000 7f45277fe700 1 -- 172.21.15.2:0/2040390301 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f45200e1ce0 con 0x7f4530103240 2024-05-14T09:34:12.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.294+0000 7f452dd9b700 1 --2- 172.21.15.2:0/2040390301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f451c0b2900 0x7f451c0b4e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f453019a8e0 tx=0x7f4518009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:12.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.298+0000 7f45277fe700 1 -- 172.21.15.2:0/2040390301 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4520021020 con 0x7f4530103240 2024-05-14T09:34:12.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:12 smithi029 ceph-mon[36249]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:12.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:12 smithi026 ceph-mon[36056]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:12.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.523+0000 7f4534810700 1 -- 172.21.15.2:0/2040390301 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f4530063c00 con 0x7f451c0b2900 2024-05-14T09:34:12.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.537+0000 7f45277fe700 1 -- 172.21.15.2:0/2040390301 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f4530063c00 con 0x7f451c0b2900 2024-05-14T09:34:12.537 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-14T09:34:12.537 INFO:teuthology.orchestra.run.smithi002.stdout:mds.a smithi002 running (3m) 36s ago 3m 84.5M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 73d41c4e0cf7 2024-05-14T09:34:12.537 INFO:teuthology.orchestra.run.smithi002.stdout:mds.b smithi026 running (3m) 74s ago 3m 93.7M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 968bdd4c0b63 2024-05-14T09:34:12.537 INFO:teuthology.orchestra.run.smithi002.stdout:mds.c smithi029 running (3m) 74s ago 3m 87.4M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 1128ca721878 2024-05-14T09:34:12.537 INFO:teuthology.orchestra.run.smithi002.stdout:mds.d smithi002 running (3m) 36s ago 3m 81.1M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 038ad3835cf8 2024-05-14T09:34:12.537 INFO:teuthology.orchestra.run.smithi002.stdout:mds.e smithi026 running (3m) 74s ago 3m 86.1M - 19.0.0-3588-ge2b755a6 bc14c6192b1a eb79eacbc98a 2024-05-14T09:34:12.538 INFO:teuthology.orchestra.run.smithi002.stdout:mds.f smithi029 running (3m) 74s ago 3m 83.9M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 56f5739413b1 2024-05-14T09:34:12.538 INFO:teuthology.orchestra.run.smithi002.stdout:mds.g smithi002 running (3m) 36s ago 3m 83.0M - 19.0.0-3588-ge2b755a6 bc14c6192b1a f3fcf57575ab 2024-05-14T09:34:12.538 INFO:teuthology.orchestra.run.smithi002.stdout:mds.h smithi026 running (3m) 74s ago 3m 85.4M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 58bf2a4d3411 2024-05-14T09:34:12.538 INFO:teuthology.orchestra.run.smithi002.stdout:mds.i smithi029 running (3m) 74s ago 3m 88.3M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 9e4f54c74b3e 2024-05-14T09:34:12.538 INFO:teuthology.orchestra.run.smithi002.stdout:mds.j smithi002 running (3m) 36s ago 3m 81.3M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 810e8ad44c3b 2024-05-14T09:34:12.538 INFO:teuthology.orchestra.run.smithi002.stdout:mds.k smithi026 running (4m) 74s ago 4m 84.3M - 19.0.0-3588-ge2b755a6 bc14c6192b1a d4e5220632df 2024-05-14T09:34:12.538 INFO:teuthology.orchestra.run.smithi002.stdout:mds.l smithi029 running (4m) 74s ago 4m 81.5M - 19.0.0-3588-ge2b755a6 bc14c6192b1a e34a19116bb8 2024-05-14T09:34:12.538 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.x smithi002 *:9283,8765 running (11m) 36s ago 11m 598M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 50319b871636 2024-05-14T09:34:12.538 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.y smithi026 *:8443,8765 running (9m) 74s ago 9m 537M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 3a307e8b7787 2024-05-14T09:34:12.538 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.z smithi029 *:8443,8765 running (9m) 74s ago 9m 534M - 19.0.0-3588-ge2b755a6 bc14c6192b1a aeefc0356c8a 2024-05-14T09:34:12.538 INFO:teuthology.orchestra.run.smithi002.stdout:mon.a smithi002 running (11m) 36s ago 11m 119M 2048M 19.0.0-3588-ge2b755a6 bc14c6192b1a 8434b6d71ae2 2024-05-14T09:34:12.538 INFO:teuthology.orchestra.run.smithi002.stdout:mon.b smithi026 running (9m) 74s ago 9m 108M 2048M 19.0.0-3588-ge2b755a6 bc14c6192b1a a44b30303c5a 2024-05-14T09:34:12.539 INFO:teuthology.orchestra.run.smithi002.stdout:mon.c smithi029 running (9m) 74s ago 9m 107M 2048M 19.0.0-3588-ge2b755a6 bc14c6192b1a d42860fea015 2024-05-14T09:34:12.539 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (9m) 36s ago 9m 111M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a 907f0fb0714e 2024-05-14T09:34:12.539 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi026 running (8m) 74s ago 8m 127M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a 38b8eef45a4c 2024-05-14T09:34:12.539 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi029 running (8m) 74s ago 8m 127M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a a13f7cc56956 2024-05-14T09:34:12.539 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi002 running (7m) 36s ago 7m 123M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a e545f915774a 2024-05-14T09:34:12.539 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (7m) 74s ago 7m 111M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a 7847e042ca07 2024-05-14T09:34:12.539 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi029 running (7m) 74s ago 7m 112M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a ab73c1f396bc 2024-05-14T09:34:12.539 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6 smithi002 running (6m) 36s ago 6m 109M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a c65e6f5092ab 2024-05-14T09:34:12.539 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 smithi026 running (6m) 74s ago 6m 123M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a 3e89494dd660 2024-05-14T09:34:12.539 INFO:teuthology.orchestra.run.smithi002.stdout:osd.8 smithi029 running (5m) 74s ago 5m 121M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a 862829554ce4 2024-05-14T09:34:12.539 INFO:teuthology.orchestra.run.smithi002.stdout:osd.9 smithi002 running (5m) 36s ago 5m 122M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a 2e4b54fb7754 2024-05-14T09:34:12.539 INFO:teuthology.orchestra.run.smithi002.stdout:osd.10 smithi026 running (4m) 74s ago 4m 122M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a d702f2a68103 2024-05-14T09:34:12.539 INFO:teuthology.orchestra.run.smithi002.stdout:osd.11 smithi029 running (4m) 74s ago 4m 109M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a a47901cf6ce0 2024-05-14T09:34:12.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.539+0000 7f4534810700 1 -- 172.21.15.2:0/2040390301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f451c0b2900 msgr2=0x7f451c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:12.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.539+0000 7f4534810700 1 --2- 172.21.15.2:0/2040390301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f451c0b2900 0x7f451c0b4e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f453019a8e0 tx=0x7f4518009190 comp rx=0 tx=0).stop 2024-05-14T09:34:12.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.539+0000 7f4534810700 1 -- 172.21.15.2:0/2040390301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530103240 msgr2=0x7f4530199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:12.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.539+0000 7f4534810700 1 --2- 172.21.15.2:0/2040390301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530103240 0x7f4530199830 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f452000ada0 tx=0x7f452000c330 comp rx=0 tx=0).stop 2024-05-14T09:34:12.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.539+0000 7f4534810700 1 -- 172.21.15.2:0/2040390301 shutdown_connections 2024-05-14T09:34:12.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.539+0000 7f4534810700 1 --2- 172.21.15.2:0/2040390301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f451c0b2900 0x7f451c0b4e10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:12.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.540+0000 7f4534810700 1 --2- 172.21.15.2:0/2040390301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4530104d10 0x7f45301a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:12.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.540+0000 7f4534810700 1 --2- 172.21.15.2:0/2040390301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530103240 0x7f4530199830 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:12.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.540+0000 7f4534810700 1 --2- 172.21.15.2:0/2040390301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4530104370 0x7f4530199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:12.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.540+0000 7f4534810700 1 -- 172.21.15.2:0/2040390301 >> 172.21.15.2:0/2040390301 conn(0x7f45300fe880 msgr2=0x7f45301075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:12.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.540+0000 7f4534810700 1 -- 172.21.15.2:0/2040390301 shutdown_connections 2024-05-14T09:34:12.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:12.540+0000 7f4534810700 1 -- 172.21.15.2:0/2040390301 wait complete. 2024-05-14T09:34:13.264 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-14T09:34:13.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:13 smithi026 ceph-mon[36056]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:34:13.516 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:13.638 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:13 smithi002 ceph-mon[31557]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:34:13.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:13 smithi029 ceph-mon[36249]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:34:14.351 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:14 smithi002 ceph-mon[31557]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:14.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:14 smithi026 ceph-mon[36056]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:14.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:14 smithi029 ceph-mon[36249]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:15.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.184+0000 7f779802e700 1 -- 172.21.15.2:0/710583087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7790104d20 msgr2=0x7f779010b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:15.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.184+0000 7f779802e700 1 --2- 172.21.15.2:0/710583087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7790104d20 0x7f779010b5d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f778800b0a0 tx=0x7f778800b3b0 comp rx=0 tx=0).stop 2024-05-14T09:34:15.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.184+0000 7f779802e700 1 -- 172.21.15.2:0/710583087 shutdown_connections 2024-05-14T09:34:15.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.184+0000 7f779802e700 1 --2- 172.21.15.2:0/710583087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7790103250 0x7f7790103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:15.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.184+0000 7f779802e700 1 --2- 172.21.15.2:0/710583087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7790104d20 0x7f779010b5d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:15.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.184+0000 7f779802e700 1 --2- 172.21.15.2:0/710583087 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7790104380 0x7f7790104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:15.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.184+0000 7f779802e700 1 -- 172.21.15.2:0/710583087 >> 172.21.15.2:0/710583087 conn(0x7f77900fe890 msgr2=0x7f7790100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:15.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.184+0000 7f779802e700 1 -- 172.21.15.2:0/710583087 shutdown_connections 2024-05-14T09:34:15.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.184+0000 7f779802e700 1 -- 172.21.15.2:0/710583087 wait complete. 2024-05-14T09:34:15.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.185+0000 7f779802e700 1 Processor -- start 2024-05-14T09:34:15.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.185+0000 7f779802e700 1 -- start start 2024-05-14T09:34:15.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.185+0000 7f779802e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7790103250 0x7f779019a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:15.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.185+0000 7f779802e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7790104380 0x7f779019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:15.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.185+0000 7f779802e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7790104d20 0x7f77901a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:15.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.185+0000 7f779802e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f779010ecb0 con 0x7f7790104380 2024-05-14T09:34:15.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.185+0000 7f779802e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f779010eb30 con 0x7f7790103250 2024-05-14T09:34:15.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.185+0000 7f779802e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f779010ee30 con 0x7f7790104d20 2024-05-14T09:34:15.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.185+0000 7f77955c9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7790104380 0x7f779019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:15.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.185+0000 7f77955c9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7790104380 0x7f779019adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58016/0 (socket says 172.21.15.2:58016) 2024-05-14T09:34:15.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.185+0000 7f77955c9700 1 -- 172.21.15.2:0/3957647236 learned_addr learned my addr 172.21.15.2:0/3957647236 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:15.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.186+0000 7f7795dca700 1 --2- 172.21.15.2:0/3957647236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7790103250 0x7f779019a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:15.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.186+0000 7f77965cb700 1 --2- 172.21.15.2:0/3957647236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7790104d20 0x7f77901a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:15.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.186+0000 7f77955c9700 1 -- 172.21.15.2:0/3957647236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7790104d20 msgr2=0x7f77901a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:15.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.186+0000 7f77955c9700 1 --2- 172.21.15.2:0/3957647236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7790104d20 0x7f77901a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:15.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.186+0000 7f77955c9700 1 -- 172.21.15.2:0/3957647236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7790103250 msgr2=0x7f779019a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:15.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.186+0000 7f77955c9700 1 --2- 172.21.15.2:0/3957647236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7790103250 0x7f779019a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:15.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.186+0000 7f77955c9700 1 -- 172.21.15.2:0/3957647236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77901a23e0 con 0x7f7790104380 2024-05-14T09:34:15.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.186+0000 7f77955c9700 1 --2- 172.21.15.2:0/3957647236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7790104380 0x7f779019adc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f778000d950 tx=0x7f778000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:15.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.186+0000 7f7786ffd700 1 -- 172.21.15.2:0/3957647236 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780016070 con 0x7f7790104380 2024-05-14T09:34:15.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.186+0000 7f7786ffd700 1 -- 172.21.15.2:0/3957647236 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7780009df0 con 0x7f7790104380 2024-05-14T09:34:15.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.186+0000 7f779802e700 1 -- 172.21.15.2:0/3957647236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77901a26d0 con 0x7f7790104380 2024-05-14T09:34:15.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.186+0000 7f7786ffd700 1 -- 172.21.15.2:0/3957647236 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778000f3c0 con 0x7f7790104380 2024-05-14T09:34:15.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.186+0000 7f779802e700 1 -- 172.21.15.2:0/3957647236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77901a2c10 con 0x7f7790104380 2024-05-14T09:34:15.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.187+0000 7f779802e700 1 -- 172.21.15.2:0/3957647236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77900512c0 con 0x7f7790104380 2024-05-14T09:34:15.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.189+0000 7f7786ffd700 1 -- 172.21.15.2:0/3957647236 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f7780019070 con 0x7f7790104380 2024-05-14T09:34:15.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.190+0000 7f7786ffd700 1 --2- 172.21.15.2:0/3957647236 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f777c0b2880 0x7f777c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:15.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.190+0000 7f7795dca700 1 --2- 172.21.15.2:0/3957647236 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f777c0b2880 0x7f777c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:15.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.190+0000 7f7786ffd700 1 -- 172.21.15.2:0/3957647236 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f77800e2220 con 0x7f7790104380 2024-05-14T09:34:15.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.190+0000 7f7795dca700 1 --2- 172.21.15.2:0/3957647236 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f777c0b2880 0x7f777c0b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7790104280 tx=0x7f778c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:15.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.193+0000 7f7786ffd700 1 -- 172.21.15.2:0/3957647236 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f77800a9b20 con 0x7f7790104380 2024-05-14T09:34:15.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.417+0000 7f779802e700 1 -- 172.21.15.2:0/3957647236 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f7790100880 con 0x7f777c0b2880 2024-05-14T09:34:15.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.423+0000 7f7786ffd700 1 -- 172.21.15.2:0/3957647236 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f7790100880 con 0x7f777c0b2880 2024-05-14T09:34:15.423 INFO:teuthology.orchestra.run.smithi002.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-14T09:34:15.423 INFO:teuthology.orchestra.run.smithi002.stdout:mds.all 12/12 77s ago 4m smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi029=c;smithi029=f;smithi029=i;smithi029=l;count:12 2024-05-14T09:34:15.423 INFO:teuthology.orchestra.run.smithi002.stdout:mgr 3/3 77s ago 9m smithi002=x;smithi026=y;smithi029=z;count:3 2024-05-14T09:34:15.423 INFO:teuthology.orchestra.run.smithi002.stdout:mon 3/3 77s ago 10m smithi002:172.21.15.2=a;smithi026:172.21.15.26=b;smithi029:172.21.15.29=c;count:3 2024-05-14T09:34:15.423 INFO:teuthology.orchestra.run.smithi002.stdout:osd 12 77s ago - 2024-05-14T09:34:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.426+0000 7f779802e700 1 -- 172.21.15.2:0/3957647236 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f777c0b2880 msgr2=0x7f777c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.426+0000 7f779802e700 1 --2- 172.21.15.2:0/3957647236 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f777c0b2880 0x7f777c0b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7790104280 tx=0x7f778c00b410 comp rx=0 tx=0).stop 2024-05-14T09:34:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.426+0000 7f779802e700 1 -- 172.21.15.2:0/3957647236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7790104380 msgr2=0x7f779019adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.426+0000 7f779802e700 1 --2- 172.21.15.2:0/3957647236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7790104380 0x7f779019adc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f778000d950 tx=0x7f778000dc60 comp rx=0 tx=0).stop 2024-05-14T09:34:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.426+0000 7f779802e700 1 -- 172.21.15.2:0/3957647236 shutdown_connections 2024-05-14T09:34:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.426+0000 7f779802e700 1 --2- 172.21.15.2:0/3957647236 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f777c0b2880 0x7f777c0b4d90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.426+0000 7f779802e700 1 --2- 172.21.15.2:0/3957647236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7790103250 0x7f779019a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.426+0000 7f779802e700 1 --2- 172.21.15.2:0/3957647236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7790104d20 0x7f77901a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.426+0000 7f779802e700 1 --2- 172.21.15.2:0/3957647236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7790104380 0x7f779019adc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.426+0000 7f779802e700 1 -- 172.21.15.2:0/3957647236 >> 172.21.15.2:0/3957647236 conn(0x7f77900fe890 msgr2=0x7f7790100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.426+0000 7f779802e700 1 -- 172.21.15.2:0/3957647236 shutdown_connections 2024-05-14T09:34:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:15.426+0000 7f779802e700 1 -- 172.21.15.2:0/3957647236 wait complete. 2024-05-14T09:34:16.157 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-14T09:34:16.412 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:16.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:16 smithi002 ceph-mon[31557]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:16.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:16 smithi002 ceph-mon[31557]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:34:16.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:16 smithi026 ceph-mon[36056]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:16.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:16 smithi026 ceph-mon[36056]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:34:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:16 smithi029 ceph-mon[36249]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:16 smithi029 ceph-mon[36249]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:34:18.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.110+0000 7f28b9761700 1 -- 172.21.15.2:0/2048087863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28b4107810 msgr2=0x7f28b4109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:18.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.110+0000 7f28b9761700 1 --2- 172.21.15.2:0/2048087863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28b4107810 0x7f28b4109c30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009990 tx=0x7f28a8009ca0 comp rx=0 tx=0).stop 2024-05-14T09:34:18.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.110+0000 7f28b9761700 1 -- 172.21.15.2:0/2048087863 shutdown_connections 2024-05-14T09:34:18.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.110+0000 7f28b9761700 1 --2- 172.21.15.2:0/2048087863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b4102770 0x7f28b4102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:18.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.110+0000 7f28b9761700 1 --2- 172.21.15.2:0/2048087863 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28b410a170 0x7f28b410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:18.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.110+0000 7f28b9761700 1 --2- 172.21.15.2:0/2048087863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28b4107810 0x7f28b4109c30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:18.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.110+0000 7f28b9761700 1 -- 172.21.15.2:0/2048087863 >> 172.21.15.2:0/2048087863 conn(0x7f28b40fe020 msgr2=0x7f28b41004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:18.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.110+0000 7f28b9761700 1 -- 172.21.15.2:0/2048087863 shutdown_connections 2024-05-14T09:34:18.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.110+0000 7f28b9761700 1 -- 172.21.15.2:0/2048087863 wait complete. 2024-05-14T09:34:18.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.111+0000 7f28b9761700 1 Processor -- start 2024-05-14T09:34:18.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.111+0000 7f28b9761700 1 -- start start 2024-05-14T09:34:18.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.111+0000 7f28b9761700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28b4102770 0x7f28b419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:18.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b9761700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28b4107810 0x7f28b419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:18.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b9761700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b410a170 0x7f28b41a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:18.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b9761700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f28b410fe50 con 0x7f28b4107810 2024-05-14T09:34:18.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b9761700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f28b410fcd0 con 0x7f28b410a170 2024-05-14T09:34:18.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b9761700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f28b410ffd0 con 0x7f28b4102770 2024-05-14T09:34:18.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b27fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28b4107810 0x7f28b419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:18.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b27fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28b4107810 0x7f28b419ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58416/0 (socket says 172.21.15.2:58416) 2024-05-14T09:34:18.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b27fc700 1 -- 172.21.15.2:0/155091579 learned_addr learned my addr 172.21.15.2:0/155091579 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:18.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b2ffd700 1 --2- 172.21.15.2:0/155091579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28b4102770 0x7f28b419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:18.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b37fe700 1 --2- 172.21.15.2:0/155091579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b410a170 0x7f28b41a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:18.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b27fc700 1 -- 172.21.15.2:0/155091579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28b4102770 msgr2=0x7f28b419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:18.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b27fc700 1 --2- 172.21.15.2:0/155091579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28b4102770 0x7f28b419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:18.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b27fc700 1 -- 172.21.15.2:0/155091579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b410a170 msgr2=0x7f28b41a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:18.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b27fc700 1 --2- 172.21.15.2:0/155091579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b410a170 0x7f28b41a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:18.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b27fc700 1 -- 172.21.15.2:0/155091579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b41a24a0 con 0x7f28b4107810 2024-05-14T09:34:18.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b2ffd700 1 --2- 172.21.15.2:0/155091579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28b4102770 0x7f28b419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:18.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.112+0000 7f28b27fc700 1 --2- 172.21.15.2:0/155091579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28b4107810 0x7f28b419ae80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f28a8006010 tx=0x7f28a80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:18.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.113+0000 7f289bfff700 1 -- 172.21.15.2:0/155091579 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8027070 con 0x7f28b4107810 2024-05-14T09:34:18.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.113+0000 7f28b9761700 1 -- 172.21.15.2:0/155091579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b41a2730 con 0x7f28b4107810 2024-05-14T09:34:18.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.113+0000 7f289bfff700 1 -- 172.21.15.2:0/155091579 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28a8004000 con 0x7f28b4107810 2024-05-14T09:34:18.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.113+0000 7f289bfff700 1 -- 172.21.15.2:0/155091579 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8022d60 con 0x7f28b4107810 2024-05-14T09:34:18.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.113+0000 7f28b9761700 1 -- 172.21.15.2:0/155091579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b41a2be0 con 0x7f28b4107810 2024-05-14T09:34:18.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.114+0000 7f2899ffb700 1 -- 172.21.15.2:0/155091579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b40512c0 con 0x7f28b4107810 2024-05-14T09:34:18.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.116+0000 7f289bfff700 1 -- 172.21.15.2:0/155091579 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f28a802a060 con 0x7f28b4107810 2024-05-14T09:34:18.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.117+0000 7f289bfff700 1 --2- 172.21.15.2:0/155091579 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f289c0b2880 0x7f289c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:18.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.117+0000 7f28b2ffd700 1 --2- 172.21.15.2:0/155091579 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f289c0b2880 0x7f289c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:18.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.117+0000 7f289bfff700 1 -- 172.21.15.2:0/155091579 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f28a80e3c20 con 0x7f28b4107810 2024-05-14T09:34:18.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.117+0000 7f28b2ffd700 1 --2- 172.21.15.2:0/155091579 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f289c0b2880 0x7f289c0b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f28a4005950 tx=0x7f28a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:18.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.120+0000 7f289bfff700 1 -- 172.21.15.2:0/155091579 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28a8030020 con 0x7f28b4107810 2024-05-14T09:34:18.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.342+0000 7f2899ffb700 1 -- 172.21.15.2:0/155091579 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f28b4063c00 con 0x7f289c0b2880 2024-05-14T09:34:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.343+0000 7f289bfff700 1 -- 172.21.15.2:0/155091579 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7f28b4063c00 con 0x7f289c0b2880 2024-05-14T09:34:18.343 INFO:teuthology.orchestra.run.smithi002.stdout:HOST ADDR LABELS STATUS 2024-05-14T09:34:18.343 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002 172.21.15.2 2024-05-14T09:34:18.343 INFO:teuthology.orchestra.run.smithi002.stdout:smithi026 172.21.15.26 2024-05-14T09:34:18.343 INFO:teuthology.orchestra.run.smithi002.stdout:smithi029 172.21.15.29 2024-05-14T09:34:18.343 INFO:teuthology.orchestra.run.smithi002.stdout:3 hosts in cluster 2024-05-14T09:34:18.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.345+0000 7f2899ffb700 1 -- 172.21.15.2:0/155091579 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f289c0b2880 msgr2=0x7f289c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:18.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.345+0000 7f2899ffb700 1 --2- 172.21.15.2:0/155091579 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f289c0b2880 0x7f289c0b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f28a4005950 tx=0x7f28a400b410 comp rx=0 tx=0).stop 2024-05-14T09:34:18.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.345+0000 7f2899ffb700 1 -- 172.21.15.2:0/155091579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28b4107810 msgr2=0x7f28b419ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.345+0000 7f2899ffb700 1 --2- 172.21.15.2:0/155091579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28b4107810 0x7f28b419ae80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f28a8006010 tx=0x7f28a80057c0 comp rx=0 tx=0).stop 2024-05-14T09:34:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.345+0000 7f2899ffb700 1 -- 172.21.15.2:0/155091579 shutdown_connections 2024-05-14T09:34:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.345+0000 7f2899ffb700 1 --2- 172.21.15.2:0/155091579 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f289c0b2880 0x7f289c0b4d90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.345+0000 7f2899ffb700 1 --2- 172.21.15.2:0/155091579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b410a170 0x7f28b41a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.345+0000 7f2899ffb700 1 --2- 172.21.15.2:0/155091579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f28b4102770 0x7f28b419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.345+0000 7f2899ffb700 1 --2- 172.21.15.2:0/155091579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28b4107810 0x7f28b419ae80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.345+0000 7f2899ffb700 1 -- 172.21.15.2:0/155091579 >> 172.21.15.2:0/155091579 conn(0x7f28b40fe020 msgr2=0x7f28b4108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.346+0000 7f2899ffb700 1 -- 172.21.15.2:0/155091579 shutdown_connections 2024-05-14T09:34:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:18.346+0000 7f2899ffb700 1 -- 172.21.15.2:0/155091579 wait complete. 2024-05-14T09:34:18.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:18 smithi002 ceph-mon[31557]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:34:18.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:18 smithi026 ceph-mon[36056]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:34:18.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:18 smithi029 ceph-mon[36249]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:34:19.130 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-14T09:34:19.385 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:19.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:19 smithi002 ceph-mon[31557]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:34:19.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:19 smithi029 ceph-mon[36249]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:34:19.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:19 smithi026 ceph-mon[36056]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:34:20.500 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:20 smithi002 ceph-mon[31557]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:20 smithi029 ceph-mon[36249]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:20 smithi026 ceph-mon[36056]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:21.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.074+0000 7fbe1f186700 1 -- 172.21.15.2:0/3381569671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe18104370 msgr2=0x7fbe18104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:21.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.074+0000 7fbe1f186700 1 --2- 172.21.15.2:0/3381569671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe18104370 0x7fbe18104780 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009990 tx=0x7fbe08009ca0 comp rx=0 tx=0).stop 2024-05-14T09:34:21.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.074+0000 7fbe1f186700 1 -- 172.21.15.2:0/3381569671 shutdown_connections 2024-05-14T09:34:21.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.074+0000 7fbe1f186700 1 --2- 172.21.15.2:0/3381569671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe18105e40 0x7fbe1810c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:21.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.074+0000 7fbe1f186700 1 --2- 172.21.15.2:0/3381569671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe18104370 0x7fbe18104780 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:21.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.074+0000 7fbe1f186700 1 --2- 172.21.15.2:0/3381569671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe181054a0 0x7fbe181058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.074+0000 7fbe1f186700 1 -- 172.21.15.2:0/3381569671 >> 172.21.15.2:0/3381569671 conn(0x7fbe180ff9d0 msgr2=0x7fbe18101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.074+0000 7fbe1f186700 1 -- 172.21.15.2:0/3381569671 shutdown_connections 2024-05-14T09:34:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.074+0000 7fbe1f186700 1 -- 172.21.15.2:0/3381569671 wait complete. 2024-05-14T09:34:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.075+0000 7fbe1f186700 1 Processor -- start 2024-05-14T09:34:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.075+0000 7fbe1f186700 1 -- start start 2024-05-14T09:34:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.075+0000 7fbe1f186700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe18104370 0x7fbe1819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.075+0000 7fbe1f186700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe181054a0 0x7fbe1819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.075+0000 7fbe1f186700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe18105e40 0x7fbe181a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.075+0000 7fbe1f186700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1810fe30 con 0x7fbe18104370 2024-05-14T09:34:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.075+0000 7fbe1f186700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1810fcb0 con 0x7fbe18105e40 2024-05-14T09:34:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.075+0000 7fbe1f186700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1810ffb0 con 0x7fbe181054a0 2024-05-14T09:34:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.075+0000 7fbe1cf22700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe18104370 0x7fbe1819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.075+0000 7fbe1cf22700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe18104370 0x7fbe1819a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58422/0 (socket says 172.21.15.2:58422) 2024-05-14T09:34:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.075+0000 7fbe1cf22700 1 -- 172.21.15.2:0/3251486836 learned_addr learned my addr 172.21.15.2:0/3251486836 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.076+0000 7fbe17fff700 1 --2- 172.21.15.2:0/3251486836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe181054a0 0x7fbe1819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.076+0000 7fbe1d723700 1 --2- 172.21.15.2:0/3251486836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe18105e40 0x7fbe181a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.076+0000 7fbe1cf22700 1 -- 172.21.15.2:0/3251486836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe181054a0 msgr2=0x7fbe1819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.076+0000 7fbe1cf22700 1 --2- 172.21.15.2:0/3251486836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe181054a0 0x7fbe1819aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.076+0000 7fbe1cf22700 1 -- 172.21.15.2:0/3251486836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe18105e40 msgr2=0x7fbe181a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.076+0000 7fbe1cf22700 1 --2- 172.21.15.2:0/3251486836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe18105e40 0x7fbe181a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.076+0000 7fbe1cf22700 1 -- 172.21.15.2:0/3251486836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe181a24e0 con 0x7fbe18104370 2024-05-14T09:34:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.076+0000 7fbe1d723700 1 --2- 172.21.15.2:0/3251486836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe18105e40 0x7fbe181a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:34:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.076+0000 7fbe1cf22700 1 --2- 172.21.15.2:0/3251486836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe18104370 0x7fbe1819a980 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbe08000c00 tx=0x7fbe080057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.076+0000 7fbe15ffb700 1 -- 172.21.15.2:0/3251486836 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08026070 con 0x7fbe18104370 2024-05-14T09:34:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.076+0000 7fbe15ffb700 1 -- 172.21.15.2:0/3251486836 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe08004140 con 0x7fbe18104370 2024-05-14T09:34:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.076+0000 7fbe15ffb700 1 -- 172.21.15.2:0/3251486836 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08021410 con 0x7fbe18104370 2024-05-14T09:34:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.076+0000 7fbe1f186700 1 -- 172.21.15.2:0/3251486836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe181a2770 con 0x7fbe18104370 2024-05-14T09:34:21.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.076+0000 7fbe1f186700 1 -- 172.21.15.2:0/3251486836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe181a2b70 con 0x7fbe18104370 2024-05-14T09:34:21.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.078+0000 7fbe1f186700 1 -- 172.21.15.2:0/3251486836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe180512c0 con 0x7fbe18104370 2024-05-14T09:34:21.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.079+0000 7fbe15ffb700 1 -- 172.21.15.2:0/3251486836 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fbe08029070 con 0x7fbe18104370 2024-05-14T09:34:21.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.080+0000 7fbe15ffb700 1 --2- 172.21.15.2:0/3251486836 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbe040b2880 0x7fbe040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:21.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.081+0000 7fbe17fff700 1 --2- 172.21.15.2:0/3251486836 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbe040b2880 0x7fbe040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:21.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.081+0000 7fbe15ffb700 1 -- 172.21.15.2:0/3251486836 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fbe080e37c0 con 0x7fbe18104370 2024-05-14T09:34:21.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.081+0000 7fbe17fff700 1 --2- 172.21.15.2:0/3251486836 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbe040b2880 0x7fbe040b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbe000060b0 tx=0x7fbe000077b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.084+0000 7fbe15ffb700 1 -- 172.21.15.2:0/3251486836 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe08013070 con 0x7fbe18104370 2024-05-14T09:34:21.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.307+0000 7fbe1f186700 1 -- 172.21.15.2:0/3251486836 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fbe18063c00 con 0x7fbe040b2880 2024-05-14T09:34:21.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.309+0000 7fbe15ffb700 1 -- 172.21.15.2:0/3251486836 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fbe18063c00 con 0x7fbe040b2880 2024-05-14T09:34:21.308 INFO:teuthology.orchestra.run.smithi002.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-14T09:34:21.309 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EV480DGN 447G No 5m ago Has a FileSystem, LVM detected 2024-05-14T09:34:21.309 INFO:teuthology.orchestra.run.smithi002.stdout:smithi026 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760020480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-05-14T09:34:21.309 INFO:teuthology.orchestra.run.smithi002.stdout:smithi029 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7512009B480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-05-14T09:34:21.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.311+0000 7fbe1f186700 1 -- 172.21.15.2:0/3251486836 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbe040b2880 msgr2=0x7fbe040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:21.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.311+0000 7fbe1f186700 1 --2- 172.21.15.2:0/3251486836 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbe040b2880 0x7fbe040b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbe000060b0 tx=0x7fbe000077b0 comp rx=0 tx=0).stop 2024-05-14T09:34:21.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.311+0000 7fbe1f186700 1 -- 172.21.15.2:0/3251486836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe18104370 msgr2=0x7fbe1819a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:21.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.311+0000 7fbe1f186700 1 --2- 172.21.15.2:0/3251486836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe18104370 0x7fbe1819a980 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbe08000c00 tx=0x7fbe080057c0 comp rx=0 tx=0).stop 2024-05-14T09:34:21.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.311+0000 7fbe1f186700 1 -- 172.21.15.2:0/3251486836 shutdown_connections 2024-05-14T09:34:21.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.311+0000 7fbe1f186700 1 --2- 172.21.15.2:0/3251486836 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbe040b2880 0x7fbe040b4d90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:21.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.311+0000 7fbe1f186700 1 --2- 172.21.15.2:0/3251486836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe18105e40 0x7fbe181a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:21.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.311+0000 7fbe1f186700 1 --2- 172.21.15.2:0/3251486836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe18104370 0x7fbe1819a980 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:21.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.311+0000 7fbe1f186700 1 --2- 172.21.15.2:0/3251486836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbe181054a0 0x7fbe1819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:21.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.311+0000 7fbe1f186700 1 -- 172.21.15.2:0/3251486836 >> 172.21.15.2:0/3251486836 conn(0x7fbe180ff9d0 msgr2=0x7fbe181086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:21.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.311+0000 7fbe1f186700 1 -- 172.21.15.2:0/3251486836 shutdown_connections 2024-05-14T09:34:21.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:21.311+0000 7fbe1f186700 1 -- 172.21.15.2:0/3251486836 wait complete. 2024-05-14T09:34:21.979 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-14T09:34:21.988 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T09:34:21.989 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-14T09:34:22.241 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:22.364 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:22 smithi002 ceph-mon[31557]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:22.364 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:22 smithi002 ceph-mon[31557]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:34:22.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:22 smithi029 ceph-mon[36249]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:22.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:22 smithi029 ceph-mon[36249]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:34:22.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:22 smithi026 ceph-mon[36056]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:22.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:22 smithi026 ceph-mon[36056]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T09:34:23.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.955+0000 7ffaba28f700 1 -- 172.21.15.2:0/2849655887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffab4104d20 msgr2=0x7ffab410b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:23.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.955+0000 7ffaba28f700 1 --2- 172.21.15.2:0/2849655887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffab4104d20 0x7ffab410b5d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00b0a0 tx=0x7ffaac00b3b0 comp rx=0 tx=0).stop 2024-05-14T09:34:23.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.955+0000 7ffaba28f700 1 -- 172.21.15.2:0/2849655887 shutdown_connections 2024-05-14T09:34:23.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.955+0000 7ffaba28f700 1 --2- 172.21.15.2:0/2849655887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4103250 0x7ffab4103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:23.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.955+0000 7ffaba28f700 1 --2- 172.21.15.2:0/2849655887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffab4104d20 0x7ffab410b5d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:23.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.955+0000 7ffaba28f700 1 --2- 172.21.15.2:0/2849655887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab4104380 0x7ffab4104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:23.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.955+0000 7ffaba28f700 1 -- 172.21.15.2:0/2849655887 >> 172.21.15.2:0/2849655887 conn(0x7ffab40fe890 msgr2=0x7ffab4100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:23.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.955+0000 7ffaba28f700 1 -- 172.21.15.2:0/2849655887 shutdown_connections 2024-05-14T09:34:23.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.955+0000 7ffaba28f700 1 -- 172.21.15.2:0/2849655887 wait complete. 2024-05-14T09:34:23.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.955+0000 7ffaba28f700 1 Processor -- start 2024-05-14T09:34:23.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.955+0000 7ffaba28f700 1 -- start start 2024-05-14T09:34:23.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.956+0000 7ffaba28f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab4103250 0x7ffab419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:23.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.956+0000 7ffaba28f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4104380 0x7ffab419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:23.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.956+0000 7ffaba28f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffab4104d20 0x7ffab41a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:23.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.956+0000 7ffaba28f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffab410ecb0 con 0x7ffab4104d20 2024-05-14T09:34:23.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.956+0000 7ffaba28f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffab410eb30 con 0x7ffab4104380 2024-05-14T09:34:23.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.956+0000 7ffaba28f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffab410ee30 con 0x7ffab4103250 2024-05-14T09:34:23.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.956+0000 7ffab882c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffab4104d20 0x7ffab41a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:23.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.956+0000 7ffab882c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffab4104d20 0x7ffab41a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58436/0 (socket says 172.21.15.2:58436) 2024-05-14T09:34:23.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.956+0000 7ffab882c700 1 -- 172.21.15.2:0/1638409780 learned_addr learned my addr 172.21.15.2:0/1638409780 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:23.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.956+0000 7ffab3fff700 1 --2- 172.21.15.2:0/1638409780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab4103250 0x7ffab419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:23.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.956+0000 7ffab37fe700 1 --2- 172.21.15.2:0/1638409780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4104380 0x7ffab419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:23.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.956+0000 7ffab882c700 1 -- 172.21.15.2:0/1638409780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab4103250 msgr2=0x7ffab419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:23.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.956+0000 7ffab882c700 1 --2- 172.21.15.2:0/1638409780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab4103250 0x7ffab419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:23.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.956+0000 7ffab882c700 1 -- 172.21.15.2:0/1638409780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4104380 msgr2=0x7ffab419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:23.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.956+0000 7ffab882c700 1 --2- 172.21.15.2:0/1638409780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4104380 0x7ffab419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:23.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.957+0000 7ffab882c700 1 -- 172.21.15.2:0/1638409780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab41a23e0 con 0x7ffab4104d20 2024-05-14T09:34:23.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.957+0000 7ffab882c700 1 --2- 172.21.15.2:0/1638409780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffab4104d20 0x7ffab41a1d60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00d010 tx=0x7ffaac004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:23.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.957+0000 7ffab17fa700 1 -- 172.21.15.2:0/1638409780 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac021070 con 0x7ffab4104d20 2024-05-14T09:34:23.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.957+0000 7ffab17fa700 1 -- 172.21.15.2:0/1638409780 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaac007c00 con 0x7ffab4104d20 2024-05-14T09:34:23.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.957+0000 7ffaba28f700 1 -- 172.21.15.2:0/1638409780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab41a2670 con 0x7ffab4104d20 2024-05-14T09:34:23.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.957+0000 7ffab17fa700 1 -- 172.21.15.2:0/1638409780 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0052a0 con 0x7ffab4104d20 2024-05-14T09:34:23.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.957+0000 7ffaba28f700 1 -- 172.21.15.2:0/1638409780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab41a2bb0 con 0x7ffab4104d20 2024-05-14T09:34:23.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.958+0000 7ffaba28f700 1 -- 172.21.15.2:0/1638409780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab40512c0 con 0x7ffab4104d20 2024-05-14T09:34:23.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.960+0000 7ffab17fa700 1 -- 172.21.15.2:0/1638409780 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ffaac024070 con 0x7ffab4104d20 2024-05-14T09:34:23.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.960+0000 7ffab17fa700 1 --2- 172.21.15.2:0/1638409780 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffa9c0b2880 0x7ffa9c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:23.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.960+0000 7ffab3fff700 1 --2- 172.21.15.2:0/1638409780 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffa9c0b2880 0x7ffa9c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:23.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.961+0000 7ffab17fa700 1 -- 172.21.15.2:0/1638409780 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7ffaac00f020 con 0x7ffab4104d20 2024-05-14T09:34:23.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.961+0000 7ffab3fff700 1 --2- 172.21.15.2:0/1638409780 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffa9c0b2880 0x7ffa9c0b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffab4104280 tx=0x7ffaa4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:23.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:23.965+0000 7ffab17fa700 1 -- 172.21.15.2:0/1638409780 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffaac02b020 con 0x7ffab4104d20 2024-05-14T09:34:24.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:24.228+0000 7ffaba28f700 1 -- 172.21.15.2:0/1638409780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ffab41a33c0 con 0x7ffab4104d20 2024-05-14T09:34:24.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:24.229+0000 7ffab17fa700 1 -- 172.21.15.2:0/1638409780 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3032 (secure 0 0 0) 0x7ffaac0ab620 con 0x7ffab4104d20 2024-05-14T09:34:24.229 INFO:teuthology.orchestra.run.smithi002.stdout:e26 2024-05-14T09:34:24.229 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T09:34:24.229 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T09:34:24.229 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-05-14T09:34:24.230 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:34:24.230 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-05-14T09:34:24.230 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-05-14T09:34:24.230 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 26 2024-05-14T09:34:24.230 INFO:teuthology.orchestra.run.smithi002.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-14T09:34:24.230 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-05-14T09:30:58.644918+0000 2024-05-14T09:34:24.230 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-05-14T09:32:01.167752+0000 2024-05-14T09:34:24.230 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-05-14T09:34:24.230 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-05-14T09:34:24.230 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 300 2024-05-14T09:34:24.230 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-05-14T09:34:24.230 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-05-14T09:34:24.230 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-05-14T09:34:24.230 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-05-14T09:34:24.231 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-05-14T09:34:24.231 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 0 2024-05-14T09:34:24.231 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T09:34:24.231 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 3 2024-05-14T09:34:24.231 INFO:teuthology.orchestra.run.smithi002.stdout:in 0,1,2 2024-05-14T09:34:24.231 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24524,1=24497,2=24487} 2024-05-14T09:34:24.231 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-05-14T09:34:24.231 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-05-14T09:34:24.231 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-05-14T09:34:24.231 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3,4] 2024-05-14T09:34:24.231 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-05-14T09:34:24.231 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data disabled 2024-05-14T09:34:24.231 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-05-14T09:34:24.231 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-05-14T09:34:24.232 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 0 2024-05-14T09:34:24.232 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 24524 members: 24497,24524,24487 2024-05-14T09:34:24.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.b{0:24524} state up:active seq 5 addr [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T09:34:24.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.e{0:24473} state up:standby-replay seq 1 addr [v2:172.21.15.26:6834/2529273890,v1:172.21.15.26:6835/2529273890] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T09:34:24.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.h{1:24497} state up:active seq 22 addr [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T09:34:24.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.c{1:24469} state up:standby-replay seq 1 addr [v2:172.21.15.29:6836/1333834114,v1:172.21.15.29:6837/1333834114] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T09:34:24.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.i{2:24487} state up:active seq 22 addr [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T09:34:24.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.f{2:24463} state up:standby-replay seq 1 addr [v2:172.21.15.29:6834/3647517494,v1:172.21.15.29:6835/3647517494] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T09:34:24.232 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:34:24.232 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:34:24.232 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-05-14T09:34:24.232 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:34:24.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.j{-1:14568} state up:standby seq 1 addr [v2:172.21.15.2:6834/3590687009,v1:172.21.15.2:6835/3590687009] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T09:34:24.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.d{-1:14580} state up:standby seq 1 addr [v2:172.21.15.2:6836/2302731082,v1:172.21.15.2:6837/2302731082] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T09:34:24.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.g{-1:14622} state up:standby seq 1 addr [v2:172.21.15.2:6838/2887812543,v1:172.21.15.2:6839/2887812543] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T09:34:24.233 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.a{-1:14634} state up:standby seq 1 addr [v2:172.21.15.2:6840/4114057120,v1:172.21.15.2:6841/4114057120] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T09:34:24.233 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.l{-1:24421} state up:standby seq 1 addr [v2:172.21.15.29:6832/2153165231,v1:172.21.15.29:6833/2153165231] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T09:34:24.233 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.k{-1:24446} state up:standby seq 1 addr [v2:172.21.15.26:6832/39434912,v1:172.21.15.26:6833/39434912] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T09:34:24.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:24.231+0000 7ffaba28f700 1 -- 172.21.15.2:0/1638409780 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffa9c0b2880 msgr2=0x7ffa9c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:24.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:24.231+0000 7ffaba28f700 1 --2- 172.21.15.2:0/1638409780 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffa9c0b2880 0x7ffa9c0b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffab4104280 tx=0x7ffaa4009260 comp rx=0 tx=0).stop 2024-05-14T09:34:24.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:24.231+0000 7ffaba28f700 1 -- 172.21.15.2:0/1638409780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffab4104d20 msgr2=0x7ffab41a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:24.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:24.231+0000 7ffaba28f700 1 --2- 172.21.15.2:0/1638409780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffab4104d20 0x7ffab41a1d60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00d010 tx=0x7ffaac004db0 comp rx=0 tx=0).stop 2024-05-14T09:34:24.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:24.231+0000 7ffaba28f700 1 -- 172.21.15.2:0/1638409780 shutdown_connections 2024-05-14T09:34:24.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:24.231+0000 7ffaba28f700 1 --2- 172.21.15.2:0/1638409780 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffa9c0b2880 0x7ffa9c0b4d90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:24.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:24.231+0000 7ffaba28f700 1 --2- 172.21.15.2:0/1638409780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffab4104d20 0x7ffab41a1d60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:24.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:24.231+0000 7ffaba28f700 1 --2- 172.21.15.2:0/1638409780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffab4103250 0x7ffab419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:24.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:24.231+0000 7ffaba28f700 1 --2- 172.21.15.2:0/1638409780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4104380 0x7ffab419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:24.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:24.231+0000 7ffaba28f700 1 -- 172.21.15.2:0/1638409780 >> 172.21.15.2:0/1638409780 conn(0x7ffab40fe890 msgr2=0x7ffab4100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:24.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:24.231+0000 7ffaba28f700 1 -- 172.21.15.2:0/1638409780 shutdown_connections 2024-05-14T09:34:24.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:24.231+0000 7ffaba28f700 1 -- 172.21.15.2:0/1638409780 wait complete. 2024-05-14T09:34:24.234 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-05-14T09:34:24.615 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:24 smithi002 ceph-mon[31557]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:24.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:24 smithi029 ceph-mon[36249]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:24.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:24 smithi026 ceph-mon[36056]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:24.938 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-14T09:34:25.190 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:25.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:25 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1638409780' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-14T09:34:25.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:25 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1638409780' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-14T09:34:25.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:25 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1638409780' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-14T09:34:26.497 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:26 smithi002 ceph-mon[31557]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:26 smithi029 ceph-mon[36249]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:26.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:26 smithi026 ceph-mon[36056]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.848+0000 7f216d509700 1 -- 172.21.15.2:0/2214816345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f216810a110 msgr2=0x7f216810c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.848+0000 7f216d509700 1 --2- 172.21.15.2:0/2214816345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f216810a110 0x7f216810c530 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f216000b0a0 tx=0x7f216000b3b0 comp rx=0 tx=0).stop 2024-05-14T09:34:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.848+0000 7f216d509700 1 -- 172.21.15.2:0/2214816345 shutdown_connections 2024-05-14T09:34:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.848+0000 7f216d509700 1 --2- 172.21.15.2:0/2214816345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f216810a110 0x7f216810c530 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.848+0000 7f216d509700 1 --2- 172.21.15.2:0/2214816345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2168104e50 0x7f2168107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.848+0000 7f216d509700 1 --2- 172.21.15.2:0/2214816345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21681077b0 0x7f2168109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.848+0000 7f216d509700 1 -- 172.21.15.2:0/2214816345 >> 172.21.15.2:0/2214816345 conn(0x7f21680fe890 msgr2=0x7f2168100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.848+0000 7f216d509700 1 -- 172.21.15.2:0/2214816345 shutdown_connections 2024-05-14T09:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.848+0000 7f216d509700 1 -- 172.21.15.2:0/2214816345 wait complete. 2024-05-14T09:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.849+0000 7f216d509700 1 Processor -- start 2024-05-14T09:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.849+0000 7f216d509700 1 -- start start 2024-05-14T09:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.849+0000 7f216d509700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2168104e50 0x7f216819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.849+0000 7f216d509700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21681077b0 0x7f216819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.849+0000 7f216d509700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216810a110 0x7f21681a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.849+0000 7f216d509700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f216810fdb0 con 0x7f2168104e50 2024-05-14T09:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f216d509700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f216810fc30 con 0x7f21681077b0 2024-05-14T09:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f216d509700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f216810ff30 con 0x7f216810a110 2024-05-14T09:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f2166ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2168104e50 0x7f216819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f2166ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2168104e50 0x7f216819a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58450/0 (socket says 172.21.15.2:58450) 2024-05-14T09:34:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f2166ffd700 1 -- 172.21.15.2:0/1454519919 learned_addr learned my addr 172.21.15.2:0/1454519919 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f21667fc700 1 --2- 172.21.15.2:0/1454519919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21681077b0 0x7f216819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f21677fe700 1 --2- 172.21.15.2:0/1454519919 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216810a110 0x7f21681a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:26.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f2166ffd700 1 -- 172.21.15.2:0/1454519919 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216810a110 msgr2=0x7f21681a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:26.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f2166ffd700 1 --2- 172.21.15.2:0/1454519919 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216810a110 0x7f21681a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:26.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f2166ffd700 1 -- 172.21.15.2:0/1454519919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21681077b0 msgr2=0x7f216819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:26.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f2166ffd700 1 --2- 172.21.15.2:0/1454519919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21681077b0 0x7f216819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:26.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f2166ffd700 1 -- 172.21.15.2:0/1454519919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21681a2400 con 0x7f2168104e50 2024-05-14T09:34:26.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f2166ffd700 1 --2- 172.21.15.2:0/1454519919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2168104e50 0x7f216819a8a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f215800e970 tx=0x7f215800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:26.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f214ffff700 1 -- 172.21.15.2:0/1454519919 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215800cbd0 con 0x7f2168104e50 2024-05-14T09:34:26.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f216d509700 1 -- 172.21.15.2:0/1454519919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21681a26f0 con 0x7f2168104e50 2024-05-14T09:34:26.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f214ffff700 1 -- 172.21.15.2:0/1454519919 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f215800cd70 con 0x7f2168104e50 2024-05-14T09:34:26.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.850+0000 7f214ffff700 1 -- 172.21.15.2:0/1454519919 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2158005680 con 0x7f2168104e50 2024-05-14T09:34:26.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.851+0000 7f216d509700 1 -- 172.21.15.2:0/1454519919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21681a2c80 con 0x7f2168104e50 2024-05-14T09:34:26.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.851+0000 7f216d509700 1 -- 172.21.15.2:0/1454519919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21680512c0 con 0x7f2168104e50 2024-05-14T09:34:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.853+0000 7f214ffff700 1 -- 172.21.15.2:0/1454519919 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2158004750 con 0x7f2168104e50 2024-05-14T09:34:26.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.854+0000 7f214ffff700 1 --2- 172.21.15.2:0/1454519919 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f21500b2880 0x7f21500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:26.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.854+0000 7f21667fc700 1 --2- 172.21.15.2:0/1454519919 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f21500b2880 0x7f21500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:26.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.854+0000 7f214ffff700 1 -- 172.21.15.2:0/1454519919 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f21580e2ad0 con 0x7f2168104e50 2024-05-14T09:34:26.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.854+0000 7f21667fc700 1 --2- 172.21.15.2:0/1454519919 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f21500b2880 0x7f21500b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f216819b950 tx=0x7f215c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:26.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:26.857+0000 7f214ffff700 1 -- 172.21.15.2:0/1454519919 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2158054df0 con 0x7f2168104e50 2024-05-14T09:34:27.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:27.073+0000 7f216d509700 1 -- 172.21.15.2:0/1454519919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f216802e5f0 con 0x7f2168104e50 2024-05-14T09:34:27.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:27.074+0000 7f214ffff700 1 -- 172.21.15.2:0/1454519919 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5701 (secure 0 0 0) 0x7f21580aa370 con 0x7f2168104e50 2024-05-14T09:34:27.074 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 90 2024-05-14T09:34:27.074 INFO:teuthology.orchestra.run.smithi002.stdout:fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T09:34:27.074 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-05-14T09:22:34.937047+0000 2024-05-14T09:34:27.075 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-05-14T09:32:30.306958+0000 2024-05-14T09:34:27.075 INFO:teuthology.orchestra.run.smithi002.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-14T09:34:27.075 INFO:teuthology.orchestra.run.smithi002.stdout:crush_version 27 2024-05-14T09:34:27.075 INFO:teuthology.orchestra.run.smithi002.stdout:full_ratio 0.95 2024-05-14T09:34:27.075 INFO:teuthology.orchestra.run.smithi002.stdout:backfillfull_ratio 0.9 2024-05-14T09:34:27.075 INFO:teuthology.orchestra.run.smithi002.stdout:nearfull_ratio 0.85 2024-05-14T09:34:27.075 INFO:teuthology.orchestra.run.smithi002.stdout:require_min_compat_client luminous 2024-05-14T09:34:27.075 INFO:teuthology.orchestra.run.smithi002.stdout:min_compat_client luminous 2024-05-14T09:34:27.075 INFO:teuthology.orchestra.run.smithi002.stdout:require_osd_release squid 2024-05-14T09:34:27.075 INFO:teuthology.orchestra.run.smithi002.stdout:stretch_mode_enabled false 2024-05-14T09:34:27.075 INFO:teuthology.orchestra.run.smithi002.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-14T09:34:27.075 INFO:teuthology.orchestra.run.smithi002.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-05-14T09:34:27.075 INFO:teuthology.orchestra.run.smithi002.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-14T09:34:27.075 INFO:teuthology.orchestra.run.smithi002.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-14T09:34:27.076 INFO:teuthology.orchestra.run.smithi002.stdout:max_osd 12 2024-05-14T09:34:27.076 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6802/2622184634,v1:172.21.15.2:6803/2622184634] [v2:172.21.15.2:6804/2622184634,v1:172.21.15.2:6805/2622184634] exists,up 10d34289-414e-4b1b-bc7c-ba1dd24c7297 2024-05-14T09:34:27.076 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6800/3084747226,v1:172.21.15.26:6801/3084747226] [v2:172.21.15.26:6802/3084747226,v1:172.21.15.26:6803/3084747226] exists,up bc4aefc2-706e-4faf-9042-64b87ad097b5 2024-05-14T09:34:27.076 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 up in weight 1 up_from 18 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6800/814048086,v1:172.21.15.29:6801/814048086] [v2:172.21.15.29:6802/814048086,v1:172.21.15.29:6803/814048086] exists,up 8c5bb589-56e2-469f-9a6f-6035b3ddf4bd 2024-05-14T09:34:27.076 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 up in weight 1 up_from 26 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6810/879639451,v1:172.21.15.2:6811/879639451] [v2:172.21.15.2:6812/879639451,v1:172.21.15.2:6813/879639451] exists,up 8fbfc078-85e7-4dc2-acda-5e7b0ab927e1 2024-05-14T09:34:27.076 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 up in weight 1 up_from 32 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6808/457100752,v1:172.21.15.26:6809/457100752] [v2:172.21.15.26:6810/457100752,v1:172.21.15.26:6811/457100752] exists,up f2515448-e9c1-48a2-b270-1e5e781a2fad 2024-05-14T09:34:27.076 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 up in weight 1 up_from 37 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6808/1554045093,v1:172.21.15.29:6809/1554045093] [v2:172.21.15.29:6810/1554045093,v1:172.21.15.29:6811/1554045093] exists,up 20030857-a1e6-4b0a-b186-1cfa44f2e4ef 2024-05-14T09:34:27.076 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6 up in weight 1 up_from 43 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6818/2749897595,v1:172.21.15.2:6819/2749897595] [v2:172.21.15.2:6820/2749897595,v1:172.21.15.2:6821/2749897595] exists,up d757c72a-29f0-4f14-9f13-a2e61eb5e6a2 2024-05-14T09:34:27.076 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 up in weight 1 up_from 48 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6816/1180401611,v1:172.21.15.26:6817/1180401611] [v2:172.21.15.26:6818/1180401611,v1:172.21.15.26:6819/1180401611] exists,up 22ddef80-3633-44c5-bc0e-edc75b546ea9 2024-05-14T09:34:27.076 INFO:teuthology.orchestra.run.smithi002.stdout:osd.8 up in weight 1 up_from 54 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6816/2858408446,v1:172.21.15.29:6817/2858408446] [v2:172.21.15.29:6818/2858408446,v1:172.21.15.29:6819/2858408446] exists,up 466e7ac4-3ff7-4254-a3fd-8096420b1397 2024-05-14T09:34:27.076 INFO:teuthology.orchestra.run.smithi002.stdout:osd.9 up in weight 1 up_from 60 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6826/176898226,v1:172.21.15.2:6827/176898226] [v2:172.21.15.2:6828/176898226,v1:172.21.15.2:6829/176898226] exists,up 119eeac9-29f7-4086-a1ad-04a0bc154b0e 2024-05-14T09:34:27.076 INFO:teuthology.orchestra.run.smithi002.stdout:osd.10 up in weight 1 up_from 65 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6824/4065159515,v1:172.21.15.26:6825/4065159515] [v2:172.21.15.26:6826/4065159515,v1:172.21.15.26:6827/4065159515] exists,up 55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce 2024-05-14T09:34:27.076 INFO:teuthology.orchestra.run.smithi002.stdout:osd.11 up in weight 1 up_from 71 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6824/378800417,v1:172.21.15.29:6825/378800417] [v2:172.21.15.29:6826/378800417,v1:172.21.15.29:6827/378800417] exists,up c5246763-c4f6-4ae3-8a93-5929c541ac5d 2024-05-14T09:34:27.076 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.c [2,1] 2024-05-14T09:34:27.076 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.d [5,11] 2024-05-14T09:34:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.14 [2,4] 2024-05-14T09:34:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.16 [2,4] 2024-05-14T09:34:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.1a [8,1] 2024-05-14T09:34:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.1c [5,4] 2024-05-14T09:34:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.1f [7,11,8,6] 2024-05-14T09:34:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.e [5,9,4,7] 2024-05-14T09:34:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.1d [4,6] 2024-05-14T09:34:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.25 [5,9] 2024-05-14T09:34:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.27 [3,9] 2024-05-14T09:34:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.3a [4,8] 2024-05-14T09:34:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.5 [9,10] 2024-05-14T09:34:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.11 [5,8] 2024-05-14T09:34:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.17 [9,8] 2024-05-14T09:34:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.18 [9,8] 2024-05-14T09:34:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/4051922409 expires 2024-05-15T09:23:28.940670+0000 2024-05-14T09:34:27.078 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/2353790181 expires 2024-05-15T09:23:28.940670+0000 2024-05-14T09:34:27.078 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/4268171474 expires 2024-05-15T09:23:28.940670+0000 2024-05-14T09:34:27.078 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6801/1423742578 expires 2024-05-15T09:23:28.940670+0000 2024-05-14T09:34:27.078 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/969850441 expires 2024-05-15T09:23:02.498446+0000 2024-05-14T09:34:27.078 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/3252991268 expires 2024-05-15T09:23:02.498446+0000 2024-05-14T09:34:27.078 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6800/3593583257 expires 2024-05-15T09:23:02.498446+0000 2024-05-14T09:34:27.078 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6801/3593583257 expires 2024-05-15T09:23:02.498446+0000 2024-05-14T09:34:27.078 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/3789067476 expires 2024-05-15T09:23:02.498446+0000 2024-05-14T09:34:27.078 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6800/1423742578 expires 2024-05-15T09:23:28.940670+0000 2024-05-14T09:34:27.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:27.077+0000 7f216d509700 1 -- 172.21.15.2:0/1454519919 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f21500b2880 msgr2=0x7f21500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:27.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:27.077+0000 7f216d509700 1 --2- 172.21.15.2:0/1454519919 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f21500b2880 0x7f21500b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f216819b950 tx=0x7f215c00b410 comp rx=0 tx=0).stop 2024-05-14T09:34:27.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:27.077+0000 7f216d509700 1 -- 172.21.15.2:0/1454519919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2168104e50 msgr2=0x7f216819a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:27.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:27.077+0000 7f216d509700 1 --2- 172.21.15.2:0/1454519919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2168104e50 0x7f216819a8a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f215800e970 tx=0x7f215800ec80 comp rx=0 tx=0).stop 2024-05-14T09:34:27.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:27.077+0000 7f216d509700 1 -- 172.21.15.2:0/1454519919 shutdown_connections 2024-05-14T09:34:27.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:27.077+0000 7f216d509700 1 --2- 172.21.15.2:0/1454519919 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f21500b2880 0x7f21500b4d90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:27.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:27.077+0000 7f216d509700 1 --2- 172.21.15.2:0/1454519919 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f216810a110 0x7f21681a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:27.077+0000 7f216d509700 1 --2- 172.21.15.2:0/1454519919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2168104e50 0x7f216819a8a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:27.077+0000 7f216d509700 1 --2- 172.21.15.2:0/1454519919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21681077b0 0x7f216819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:27.077+0000 7f216d509700 1 -- 172.21.15.2:0/1454519919 >> 172.21.15.2:0/1454519919 conn(0x7f21680fe890 msgr2=0x7f2168105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:27.077+0000 7f216d509700 1 -- 172.21.15.2:0/1454519919 shutdown_connections 2024-05-14T09:34:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:27.077+0000 7f216d509700 1 -- 172.21.15.2:0/1454519919 wait complete. 2024-05-14T09:34:27.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:27 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1454519919' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-14T09:34:27.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:27 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1454519919' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-14T09:34:27.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:27 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1454519919' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-14T09:34:27.752 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-14T09:34:27.766 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:34:28.020 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:28.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:28 smithi002 ceph-mon[31557]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:34:28.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:28 smithi029 ceph-mon[36249]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:34:28.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:28 smithi026 ceph-mon[36056]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:34:29.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.826+0000 7f8dbee21700 1 -- 172.21.15.2:0/2647765563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8db8108740 msgr2=0x7f8db81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.826+0000 7f8dbee21700 1 --2- 172.21.15.2:0/2647765563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8db8108740 0x7f8db81100a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8db000b0d0 tx=0x7f8db000b3e0 comp rx=0 tx=0).stop 2024-05-14T09:34:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.826+0000 7f8dbee21700 1 -- 172.21.15.2:0/2647765563 shutdown_connections 2024-05-14T09:34:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.826+0000 7f8dbee21700 1 --2- 172.21.15.2:0/2647765563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8db8108740 0x7f8db81100a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.826+0000 7f8dbee21700 1 --2- 172.21.15.2:0/2647765563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8db81034c0 0x7f8db81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.826+0000 7f8dbee21700 1 --2- 172.21.15.2:0/2647765563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8db8107df0 0x7f8db8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.826+0000 7f8dbee21700 1 -- 172.21.15.2:0/2647765563 >> 172.21.15.2:0/2647765563 conn(0x7f8db80fedc0 msgr2=0x7f8db81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.826+0000 7f8dbee21700 1 -- 172.21.15.2:0/2647765563 shutdown_connections 2024-05-14T09:34:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.826+0000 7f8dbee21700 1 -- 172.21.15.2:0/2647765563 wait complete. 2024-05-14T09:34:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.827+0000 7f8dbee21700 1 Processor -- start 2024-05-14T09:34:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.827+0000 7f8dbee21700 1 -- start start 2024-05-14T09:34:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.827+0000 7f8dbee21700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8db81034c0 0x7f8db80ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.827+0000 7f8dbee21700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8db8107df0 0x7f8db80ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.827+0000 7f8dbee21700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8db8108740 0x7f8db80ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.827+0000 7f8dbee21700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8db8113370 con 0x7f8db8107df0 2024-05-14T09:34:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.827+0000 7f8dbee21700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8db81131f0 con 0x7f8db81034c0 2024-05-14T09:34:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.827+0000 7f8dbee21700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8db81134f0 con 0x7f8db8108740 2024-05-14T09:34:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.827+0000 7f8db7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8db8107df0 0x7f8db80ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.828+0000 7f8db7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8db8107df0 0x7f8db80ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46734/0 (socket says 172.21.15.2:46734) 2024-05-14T09:34:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.828+0000 7f8db7fff700 1 -- 172.21.15.2:0/4280898533 learned_addr learned my addr 172.21.15.2:0/4280898533 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.828+0000 7f8dbd3be700 1 --2- 172.21.15.2:0/4280898533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8db8108740 0x7f8db80ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.828+0000 7f8dbcbbd700 1 --2- 172.21.15.2:0/4280898533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8db81034c0 0x7f8db80ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.828+0000 7f8db7fff700 1 -- 172.21.15.2:0/4280898533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8db8108740 msgr2=0x7f8db80ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.828+0000 7f8db7fff700 1 --2- 172.21.15.2:0/4280898533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8db8108740 0x7f8db80ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.828+0000 7f8db7fff700 1 -- 172.21.15.2:0/4280898533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8db81034c0 msgr2=0x7f8db80ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.828+0000 7f8db7fff700 1 --2- 172.21.15.2:0/4280898533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8db81034c0 0x7f8db80ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.828+0000 7f8db7fff700 1 -- 172.21.15.2:0/4280898533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db81a9170 con 0x7f8db8107df0 2024-05-14T09:34:29.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.828+0000 7f8db7fff700 1 --2- 172.21.15.2:0/4280898533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8db8107df0 0x7f8db80ff7e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8da400e970 tx=0x7f8da400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:29.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.828+0000 7f8db5ffb700 1 -- 172.21.15.2:0/4280898533 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4004a30 con 0x7f8db8107df0 2024-05-14T09:34:29.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.828+0000 7f8dbee21700 1 -- 172.21.15.2:0/4280898533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db81a9460 con 0x7f8db8107df0 2024-05-14T09:34:29.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.828+0000 7f8db5ffb700 1 -- 172.21.15.2:0/4280898533 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8da400cc90 con 0x7f8db8107df0 2024-05-14T09:34:29.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.829+0000 7f8db5ffb700 1 -- 172.21.15.2:0/4280898533 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da40054f0 con 0x7f8db8107df0 2024-05-14T09:34:29.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.829+0000 7f8dbee21700 1 -- 172.21.15.2:0/4280898533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db81a99a0 con 0x7f8db8107df0 2024-05-14T09:34:29.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.830+0000 7f8dbee21700 1 -- 172.21.15.2:0/4280898533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db80512c0 con 0x7f8db8107df0 2024-05-14T09:34:29.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.832+0000 7f8db5ffb700 1 -- 172.21.15.2:0/4280898533 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8da4004bd0 con 0x7f8db8107df0 2024-05-14T09:34:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.832+0000 7f8db5ffb700 1 --2- 172.21.15.2:0/4280898533 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8da80b2880 0x7f8da80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.833+0000 7f8db5ffb700 1 -- 172.21.15.2:0/4280898533 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f8da40e2910 con 0x7f8db8107df0 2024-05-14T09:34:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.833+0000 7f8dbcbbd700 1 --2- 172.21.15.2:0/4280898533 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8da80b2880 0x7f8da80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:29.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.833+0000 7f8dbcbbd700 1 --2- 172.21.15.2:0/4280898533 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8da80b2880 0x7f8da80b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8dac005950 tx=0x7f8dac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:29.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:29.836+0000 7f8db5ffb700 1 -- 172.21.15.2:0/4280898533 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8da4054df0 con 0x7f8db8107df0 2024-05-14T09:34:30.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:30.099+0000 7f8dbee21700 1 -- 172.21.15.2:0/4280898533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8db81007d0 con 0x7f8db8107df0 2024-05-14T09:34:30.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:30.100+0000 7f8db5ffb700 1 -- 172.21.15.2:0/4280898533 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11843 (secure 0 0 0) 0x7f8da40aa210 con 0x7f8db8107df0 2024-05-14T09:34:30.100 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:34:30.100 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T09:32:01.167752+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24497,24524,24487]},"id":1}]} 2024-05-14T09:34:30.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:30.102+0000 7f8dbee21700 1 -- 172.21.15.2:0/4280898533 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8da80b2880 msgr2=0x7f8da80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:30.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:30.102+0000 7f8dbee21700 1 --2- 172.21.15.2:0/4280898533 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8da80b2880 0x7f8da80b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8dac005950 tx=0x7f8dac00b410 comp rx=0 tx=0).stop 2024-05-14T09:34:30.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:30.102+0000 7f8dbee21700 1 -- 172.21.15.2:0/4280898533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8db8107df0 msgr2=0x7f8db80ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:30.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:30.102+0000 7f8dbee21700 1 --2- 172.21.15.2:0/4280898533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8db8107df0 0x7f8db80ff7e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8da400e970 tx=0x7f8da400ec80 comp rx=0 tx=0).stop 2024-05-14T09:34:30.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:30.102+0000 7f8dbee21700 1 -- 172.21.15.2:0/4280898533 shutdown_connections 2024-05-14T09:34:30.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:30.102+0000 7f8dbee21700 1 --2- 172.21.15.2:0/4280898533 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8da80b2880 0x7f8da80b4d90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:30.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:30.102+0000 7f8dbee21700 1 --2- 172.21.15.2:0/4280898533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8db81034c0 0x7f8db80ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:30.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:30.102+0000 7f8dbee21700 1 --2- 172.21.15.2:0/4280898533 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8db8108740 0x7f8db80ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:30.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:30.102+0000 7f8dbee21700 1 --2- 172.21.15.2:0/4280898533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8db8107df0 0x7f8db80ff7e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:30.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:30.102+0000 7f8dbee21700 1 -- 172.21.15.2:0/4280898533 >> 172.21.15.2:0/4280898533 conn(0x7f8db80fedc0 msgr2=0x7f8db8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:30.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:30.102+0000 7f8dbee21700 1 -- 172.21.15.2:0/4280898533 shutdown_connections 2024-05-14T09:34:30.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:30.102+0000 7f8dbee21700 1 -- 172.21.15.2:0/4280898533 wait complete. 2024-05-14T09:34:30.103 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-05-14T09:34:30.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:30 smithi002 ceph-mon[31557]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:30.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:30 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/4280898533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:34:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:30 smithi029 ceph-mon[36249]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:30 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/4280898533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:34:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:30 smithi026 ceph-mon[36056]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:30 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/4280898533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:34:30.789 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:34:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:32.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:32 smithi002 ceph-mon[31557]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:32.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.647+0000 7f411fb5d700 1 -- 172.21.15.2:0/3895380385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118108f30 msgr2=0x7f4118109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:32.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.647+0000 7f411fb5d700 1 --2- 172.21.15.2:0/3895380385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118108f30 0x7f4118109340 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f41080099c0 tx=0x7f4108009cd0 comp rx=0 tx=0).stop 2024-05-14T09:34:32.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.647+0000 7f411fb5d700 1 -- 172.21.15.2:0/3895380385 shutdown_connections 2024-05-14T09:34:32.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.647+0000 7f411fb5d700 1 --2- 172.21.15.2:0/3895380385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4118104650 0x7f4118104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:32.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.647+0000 7f411fb5d700 1 --2- 172.21.15.2:0/3895380385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4118109880 0x7f41181111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:32.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.647+0000 7f411fb5d700 1 --2- 172.21.15.2:0/3895380385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118108f30 0x7f4118109340 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:32.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.647+0000 7f411fb5d700 1 -- 172.21.15.2:0/3895380385 >> 172.21.15.2:0/3895380385 conn(0x7f4118079f00 msgr2=0x7f411807a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:32.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.647+0000 7f411fb5d700 1 -- 172.21.15.2:0/3895380385 shutdown_connections 2024-05-14T09:34:32.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.647+0000 7f411fb5d700 1 -- 172.21.15.2:0/3895380385 wait complete. 2024-05-14T09:34:32.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.648+0000 7f411fb5d700 1 Processor -- start 2024-05-14T09:34:32.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.648+0000 7f411fb5d700 1 -- start start 2024-05-14T09:34:32.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.648+0000 7f411fb5d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4118104650 0x7f411819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:32.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.648+0000 7f411fb5d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118108f30 0x7f411819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:32.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.648+0000 7f411fb5d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4118109880 0x7f411819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:32.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.648+0000 7f411fb5d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f41181144b0 con 0x7f4118108f30 2024-05-14T09:34:32.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.648+0000 7f411fb5d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4118114330 con 0x7f4118109880 2024-05-14T09:34:32.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.648+0000 7f411fb5d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4118114630 con 0x7f4118104650 2024-05-14T09:34:32.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.648+0000 7f411d0f8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118108f30 0x7f411819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:32.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.649+0000 7f411d0f8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118108f30 0x7f411819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46754/0 (socket says 172.21.15.2:46754) 2024-05-14T09:34:32.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.649+0000 7f411d0f8700 1 -- 172.21.15.2:0/3844967302 learned_addr learned my addr 172.21.15.2:0/3844967302 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:32.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.649+0000 7f411d8f9700 1 --2- 172.21.15.2:0/3844967302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4118104650 0x7f411819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:32.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.649+0000 7f411e0fa700 1 --2- 172.21.15.2:0/3844967302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4118109880 0x7f411819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:32.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.649+0000 7f411d0f8700 1 -- 172.21.15.2:0/3844967302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4118104650 msgr2=0x7f411819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:32.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.649+0000 7f411d0f8700 1 --2- 172.21.15.2:0/3844967302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4118104650 0x7f411819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:32.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.649+0000 7f411d0f8700 1 -- 172.21.15.2:0/3844967302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4118109880 msgr2=0x7f411819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:32.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.649+0000 7f411d0f8700 1 --2- 172.21.15.2:0/3844967302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4118109880 0x7f411819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:32.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.649+0000 7f411d0f8700 1 -- 172.21.15.2:0/3844967302 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411819fc80 con 0x7f4118108f30 2024-05-14T09:34:32.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.649+0000 7f411d0f8700 1 --2- 172.21.15.2:0/3844967302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118108f30 0x7f411819b400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4108009f50 tx=0x7f41080037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:32.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.649+0000 7f410effd700 1 -- 172.21.15.2:0/3844967302 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108027070 con 0x7f4118108f30 2024-05-14T09:34:32.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.649+0000 7f411fb5d700 1 -- 172.21.15.2:0/3844967302 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411819ff10 con 0x7f4118108f30 2024-05-14T09:34:32.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.649+0000 7f410effd700 1 -- 172.21.15.2:0/3844967302 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4108004090 con 0x7f4118108f30 2024-05-14T09:34:32.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.649+0000 7f410effd700 1 -- 172.21.15.2:0/3844967302 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108022e60 con 0x7f4118108f30 2024-05-14T09:34:32.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.650+0000 7f411fb5d700 1 -- 172.21.15.2:0/3844967302 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41181a03f0 con 0x7f4118108f30 2024-05-14T09:34:32.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.650+0000 7f411fb5d700 1 -- 172.21.15.2:0/3844967302 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41180512c0 con 0x7f4118108f30 2024-05-14T09:34:32.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.652+0000 7f410effd700 1 -- 172.21.15.2:0/3844967302 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f410802a070 con 0x7f4118108f30 2024-05-14T09:34:32.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.653+0000 7f410effd700 1 --2- 172.21.15.2:0/3844967302 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f41040b2880 0x7f41040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:32.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.653+0000 7f411d8f9700 1 --2- 172.21.15.2:0/3844967302 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f41040b2880 0x7f41040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:32.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.653+0000 7f410effd700 1 -- 172.21.15.2:0/3844967302 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f41080e38c0 con 0x7f4118108f30 2024-05-14T09:34:32.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.653+0000 7f411d8f9700 1 --2- 172.21.15.2:0/3844967302 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f41040b2880 0x7f41040b4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f411400ac50 tx=0x7f411400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:32.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.656+0000 7f410effd700 1 -- 172.21.15.2:0/3844967302 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4108031080 con 0x7f4118108f30 2024-05-14T09:34:32.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:32 smithi029 ceph-mon[36249]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:32.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:32 smithi026 ceph-mon[36056]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.921+0000 7f411fb5d700 1 -- 172.21.15.2:0/3844967302 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f411802e5f0 con 0x7f4118108f30 2024-05-14T09:34:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.922+0000 7f410effd700 1 -- 172.21.15.2:0/3844967302 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11843 (secure 0 0 0) 0x7f4108013070 con 0x7f4118108f30 2024-05-14T09:34:32.922 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:34:32.922 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T09:32:01.167752+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24497,24524,24487]},"id":1}]} 2024-05-14T09:34:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.924+0000 7f411fb5d700 1 -- 172.21.15.2:0/3844967302 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f41040b2880 msgr2=0x7f41040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.924+0000 7f411fb5d700 1 --2- 172.21.15.2:0/3844967302 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f41040b2880 0x7f41040b4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f411400ac50 tx=0x7f411400a380 comp rx=0 tx=0).stop 2024-05-14T09:34:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.924+0000 7f411fb5d700 1 -- 172.21.15.2:0/3844967302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118108f30 msgr2=0x7f411819b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.924+0000 7f411fb5d700 1 --2- 172.21.15.2:0/3844967302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118108f30 0x7f411819b400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4108009f50 tx=0x7f41080037c0 comp rx=0 tx=0).stop 2024-05-14T09:34:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.924+0000 7f411fb5d700 1 -- 172.21.15.2:0/3844967302 shutdown_connections 2024-05-14T09:34:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.924+0000 7f411fb5d700 1 --2- 172.21.15.2:0/3844967302 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f41040b2880 0x7f41040b4d90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.924+0000 7f411fb5d700 1 --2- 172.21.15.2:0/3844967302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4118109880 0x7f411819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.924+0000 7f411fb5d700 1 --2- 172.21.15.2:0/3844967302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4118104650 0x7f411819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.924+0000 7f411fb5d700 1 --2- 172.21.15.2:0/3844967302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118108f30 0x7f411819b400 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.924+0000 7f411fb5d700 1 -- 172.21.15.2:0/3844967302 >> 172.21.15.2:0/3844967302 conn(0x7f4118079f00 msgr2=0x7f4118102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.924+0000 7f411fb5d700 1 -- 172.21.15.2:0/3844967302 shutdown_connections 2024-05-14T09:34:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:32.924+0000 7f411fb5d700 1 -- 172.21.15.2:0/3844967302 wait complete. 2024-05-14T09:34:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-05-14T09:34:33.544 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:33 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3844967302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:34:33.574 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T09:34:33.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:33 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3844967302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:34:33.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:33 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3844967302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:34:33.827 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:34 smithi002 ceph-mon[31557]: pgmap v412: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:34 smithi029 ceph-mon[36249]: pgmap v412: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:34 smithi026 ceph-mon[36056]: pgmap v412: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:35.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.626+0000 7fd78925d700 1 -- 172.21.15.2:0/2586102155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784108f30 msgr2=0x7fd784109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:35.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.626+0000 7fd78925d700 1 --2- 172.21.15.2:0/2586102155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784108f30 0x7fd784109340 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0099c0 tx=0x7fd76c009cd0 comp rx=0 tx=0).stop 2024-05-14T09:34:35.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.627+0000 7fd78925d700 1 -- 172.21.15.2:0/2586102155 shutdown_connections 2024-05-14T09:34:35.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.627+0000 7fd78925d700 1 --2- 172.21.15.2:0/2586102155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd784109880 0x7fd7841111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:35.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.627+0000 7fd78925d700 1 --2- 172.21.15.2:0/2586102155 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd784104650 0x7fd784104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:35.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.627+0000 7fd78925d700 1 --2- 172.21.15.2:0/2586102155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784108f30 0x7fd784109340 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:35.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.627+0000 7fd78925d700 1 -- 172.21.15.2:0/2586102155 >> 172.21.15.2:0/2586102155 conn(0x7fd784079f00 msgr2=0x7fd78407a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:35.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.627+0000 7fd78925d700 1 -- 172.21.15.2:0/2586102155 shutdown_connections 2024-05-14T09:34:35.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.627+0000 7fd78925d700 1 -- 172.21.15.2:0/2586102155 wait complete. 2024-05-14T09:34:35.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.627+0000 7fd78925d700 1 Processor -- start 2024-05-14T09:34:35.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd78925d700 1 -- start start 2024-05-14T09:34:35.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd78925d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd784104650 0x7fd78419d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:35.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd78925d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784108f30 0x7fd78419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:35.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd78925d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd784109880 0x7fd7841a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:35.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd78925d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd784114490 con 0x7fd784108f30 2024-05-14T09:34:35.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd78925d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd784114310 con 0x7fd784109880 2024-05-14T09:34:35.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd78925d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd784114610 con 0x7fd784104650 2024-05-14T09:34:35.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd77a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784108f30 0x7fd78419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:35.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd77a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784108f30 0x7fd78419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46772/0 (socket says 172.21.15.2:46772) 2024-05-14T09:34:35.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd77a59c700 1 -- 172.21.15.2:0/3702186493 learned_addr learned my addr 172.21.15.2:0/3702186493 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:35.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd782d9d700 1 --2- 172.21.15.2:0/3702186493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd784104650 0x7fd78419d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:35.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd78359e700 1 --2- 172.21.15.2:0/3702186493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd784109880 0x7fd7841a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:35.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd77a59c700 1 -- 172.21.15.2:0/3702186493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd784104650 msgr2=0x7fd78419d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:35.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd77a59c700 1 --2- 172.21.15.2:0/3702186493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd784104650 0x7fd78419d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:35.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd77a59c700 1 -- 172.21.15.2:0/3702186493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd784109880 msgr2=0x7fd7841a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:35.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd77a59c700 1 --2- 172.21.15.2:0/3702186493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd784109880 0x7fd7841a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:35.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd77a59c700 1 -- 172.21.15.2:0/3702186493 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7841a1f50 con 0x7fd784108f30 2024-05-14T09:34:35.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.628+0000 7fd782d9d700 1 --2- 172.21.15.2:0/3702186493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd784104650 0x7fd78419d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:34:35.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.629+0000 7fd77a59c700 1 --2- 172.21.15.2:0/3702186493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784108f30 0x7fd78419d620 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd76c006010 tx=0x7fd76c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:35.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.629+0000 7fd780d99700 1 -- 172.21.15.2:0/3702186493 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c027070 con 0x7fd784108f30 2024-05-14T09:34:35.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.629+0000 7fd780d99700 1 -- 172.21.15.2:0/3702186493 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd76c003fa0 con 0x7fd784108f30 2024-05-14T09:34:35.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.629+0000 7fd78925d700 1 -- 172.21.15.2:0/3702186493 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7841a21e0 con 0x7fd784108f30 2024-05-14T09:34:35.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.629+0000 7fd780d99700 1 -- 172.21.15.2:0/3702186493 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c022dd0 con 0x7fd784108f30 2024-05-14T09:34:35.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.629+0000 7fd78925d700 1 -- 172.21.15.2:0/3702186493 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7841a26c0 con 0x7fd784108f30 2024-05-14T09:34:35.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.630+0000 7fd78925d700 1 -- 172.21.15.2:0/3702186493 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7840512c0 con 0x7fd784108f30 2024-05-14T09:34:35.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.632+0000 7fd780d99700 1 -- 172.21.15.2:0/3702186493 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd76c02a070 con 0x7fd784108f30 2024-05-14T09:34:35.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.633+0000 7fd780d99700 1 --2- 172.21.15.2:0/3702186493 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd7700b2880 0x7fd7700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:35.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.633+0000 7fd782d9d700 1 --2- 172.21.15.2:0/3702186493 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd7700b2880 0x7fd7700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:35.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.633+0000 7fd780d99700 1 -- 172.21.15.2:0/3702186493 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd76c0e37b0 con 0x7fd784108f30 2024-05-14T09:34:35.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.633+0000 7fd782d9d700 1 --2- 172.21.15.2:0/3702186493 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd7700b2880 0x7fd7700b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd774005950 tx=0x7fd7740058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:35.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.637+0000 7fd780d99700 1 -- 172.21.15.2:0/3702186493 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd76c031080 con 0x7fd784108f30 2024-05-14T09:34:35.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.857+0000 7fd78925d700 1 -- 172.21.15.2:0/3702186493 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7841a9b60 con 0x7fd784108f30 2024-05-14T09:34:35.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.859+0000 7fd780d99700 1 -- 172.21.15.2:0/3702186493 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24740 (secure 0 0 0) 0x7fd76c013070 con 0x7fd784108f30 2024-05-14T09:34:35.859 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:34:35.859 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":90,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","created":"2024-05-14T09:22:34.937047+0000","modified":"2024-05-14T09:32:30.306958+0000","last_up_change":"2024-05-14T09:29:55.157569+0000","last_in_change":"2024-05-14T09:29:34.922562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T09:26:03.046454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T09:30:51.814350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T09:30:55.500000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T09:31:06.099087+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10d34289-414e-4b1b-bc7c-ba1dd24c7297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6803","nonce":2622184634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6805","nonce":2622184634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6809","nonce":2622184634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6807","nonce":2622184634}]},"public_addr":"172.21.15.2:6803/2622184634","cluster_addr":"172.21.15.2:6805/2622184634","heartbeat_back_addr":"172.21.15.2:6809/2622184634","heartbeat_front_addr":"172.21.15.2:6807/2622184634","state":["exists","up"]},{"osd":1,"uuid":"bc4aefc2-706e-4faf-9042-64b87ad097b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6801","nonce":3084747226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6803","nonce":3084747226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6807","nonce":3084747226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6805","nonce":3084747226}]},"public_addr":"172.21.15.26:6801/3084747226","cluster_addr":"172.21.15.26:6803/3084747226","heartbeat_back_addr":"172.21.15.26:6807/3084747226","heartbeat_front_addr":"172.21.15.26:6805/3084747226","state":["exists","up"]},{"osd":2,"uuid":"8c5bb589-56e2-469f-9a6f-6035b3ddf4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6801","nonce":814048086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6803","nonce":814048086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6807","nonce":814048086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6805","nonce":814048086}]},"public_addr":"172.21.15.29:6801/814048086","cluster_addr":"172.21.15.29:6803/814048086","heartbeat_back_addr":"172.21.15.29:6807/814048086","heartbeat_front_addr":"172.21.15.29:6805/814048086","state":["exists","up"]},{"osd":3,"uuid":"8fbfc078-85e7-4dc2-acda-5e7b0ab927e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6811","nonce":879639451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6813","nonce":879639451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6817","nonce":879639451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6815","nonce":879639451}]},"public_addr":"172.21.15.2:6811/879639451","cluster_addr":"172.21.15.2:6813/879639451","heartbeat_back_addr":"172.21.15.2:6817/879639451","heartbeat_front_addr":"172.21.15.2:6815/879639451","state":["exists","up"]},{"osd":4,"uuid":"f2515448-e9c1-48a2-b270-1e5e781a2fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6809","nonce":457100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6811","nonce":457100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6815","nonce":457100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6813","nonce":457100752}]},"public_addr":"172.21.15.26:6809/457100752","cluster_addr":"172.21.15.26:6811/457100752","heartbeat_back_addr":"172.21.15.26:6815/457100752","heartbeat_front_addr":"172.21.15.26:6813/457100752","state":["exists","up"]},{"osd":5,"uuid":"20030857-a1e6-4b0a-b186-1cfa44f2e4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6809","nonce":1554045093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6811","nonce":1554045093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6815","nonce":1554045093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6813","nonce":1554045093}]},"public_addr":"172.21.15.29:6809/1554045093","cluster_addr":"172.21.15.29:6811/1554045093","heartbeat_back_addr":"172.21.15.29:6815/1554045093","heartbeat_front_addr":"172.21.15.29:6813/1554045093","state":["exists","up"]},{"osd":6,"uuid":"d757c72a-29f0-4f14-9f13-a2e61eb5e6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6819","nonce":2749897595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6821","nonce":2749897595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6825","nonce":2749897595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6823","nonce":2749897595}]},"public_addr":"172.21.15.2:6819/2749897595","cluster_addr":"172.21.15.2:6821/2749897595","heartbeat_back_addr":"172.21.15.2:6825/2749897595","heartbeat_front_addr":"172.21.15.2:6823/2749897595","state":["exists","up"]},{"osd":7,"uuid":"22ddef80-3633-44c5-bc0e-edc75b546ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6817","nonce":1180401611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6819","nonce":1180401611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6823","nonce":1180401611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6821","nonce":1180401611}]},"public_addr":"172.21.15.26:6817/1180401611","cluster_addr":"172.21.15.26:6819/1180401611","heartbeat_back_addr":"172.21.15.26:6823/1180401611","heartbeat_front_addr":"172.21.15.26:6821/1180401611","state":["exists","up"]},{"osd":8,"uuid":"466e7ac4-3ff7-4254-a3fd-8096420b1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6817","nonce":2858408446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6819","nonce":2858408446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6823","nonce":2858408446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6821","nonce":2858408446}]},"public_addr":"172.21.15.29:6817/2858408446","cluster_addr":"172.21.15.29:6819/2858408446","heartbeat_back_addr":"172.21.15.29:6823/2858408446","heartbeat_front_addr":"172.21.15.29:6821/2858408446","state":["exists","up"]},{"osd":9,"uuid":"119eeac9-29f7-4086-a1ad-04a0bc154b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6827","nonce":176898226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6829","nonce":176898226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6833","nonce":176898226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6831","nonce":176898226}]},"public_addr":"172.21.15.2:6827/176898226","cluster_addr":"172.21.15.2:6829/176898226","heartbeat_back_addr":"172.21.15.2:6833/176898226","heartbeat_front_addr":"172.21.15.2:6831/176898226","state":["exists","up"]},{"osd":10,"uuid":"55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6825","nonce":4065159515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6827","nonce":4065159515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6831","nonce":4065159515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6829","nonce":4065159515}]},"public_addr":"172.21.15.26:6825/4065159515","cluster_addr":"172.21.15.26:6827/4065159515","heartbeat_back_addr":"172.21.15.26:6831/4065159515","heartbeat_front_addr":"172.21.15.26:6829/4065159515","state":["exists","up"]},{"osd":11,"uuid":"c5246763-c4f6-4ae3-8a93-5929c541ac5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6825","nonce":378800417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6827","nonce":378800417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6831","nonce":378800417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6829","nonce":378800417}]},"public_addr":"172.21.15.29:6825/378800417","cluster_addr":"172.21.15.29:6827/378800417","heartbeat_back_addr":"172.21.15.29:6831/378800417","heartbeat_front_addr":"172.21.15.29:6829/378800417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:09.342387+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:34.156359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:58.860653+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:24.009323+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:49.057146+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:15.007172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:41.085748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:06.674923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:33.409128+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:00.106186+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:26.803487+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:53.668283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11},{"from":8,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9},{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4051922409":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/2353790181":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/4268171474":"2024-05-15T09:23:28.940670+0000","172.21.15.2:6801/1423742578":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/969850441":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3252991268":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6801/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3789067476":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/1423742578":"2024-05-15T09:23:28.940670+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T09:34:35.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.861+0000 7fd78925d700 1 -- 172.21.15.2:0/3702186493 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd7700b2880 msgr2=0x7fd7700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:35.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.861+0000 7fd78925d700 1 --2- 172.21.15.2:0/3702186493 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd7700b2880 0x7fd7700b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd774005950 tx=0x7fd7740058e0 comp rx=0 tx=0).stop 2024-05-14T09:34:35.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.861+0000 7fd78925d700 1 -- 172.21.15.2:0/3702186493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784108f30 msgr2=0x7fd78419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:35.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.861+0000 7fd78925d700 1 --2- 172.21.15.2:0/3702186493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784108f30 0x7fd78419d620 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd76c006010 tx=0x7fd76c0047b0 comp rx=0 tx=0).stop 2024-05-14T09:34:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.861+0000 7fd78925d700 1 -- 172.21.15.2:0/3702186493 shutdown_connections 2024-05-14T09:34:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.861+0000 7fd78925d700 1 --2- 172.21.15.2:0/3702186493 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd7700b2880 0x7fd7700b4d90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.861+0000 7fd78925d700 1 --2- 172.21.15.2:0/3702186493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd784109880 0x7fd7841a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.861+0000 7fd78925d700 1 --2- 172.21.15.2:0/3702186493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd784104650 0x7fd78419d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.861+0000 7fd78925d700 1 --2- 172.21.15.2:0/3702186493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784108f30 0x7fd78419d620 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.861+0000 7fd78925d700 1 -- 172.21.15.2:0/3702186493 >> 172.21.15.2:0/3702186493 conn(0x7fd784079f00 msgr2=0x7fd78410f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.861+0000 7fd78925d700 1 -- 172.21.15.2:0/3702186493 shutdown_connections 2024-05-14T09:34:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:35.861+0000 7fd78925d700 1 -- 172.21.15.2:0/3702186493 wait complete. 2024-05-14T09:34:36.468 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:36 smithi002 ceph-mon[31557]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:36.468 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:36 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3702186493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:34:36.498 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-14T09:30:58.644918+0000', 'modified': '2024-05-14T09:32:01.167752+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24524, 'mds_1': 24497, 'mds_2': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.29:6835/3647517494', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6834', 'nonce': 3647517494}, {'type': 'v1', 'addr': '172.21.15.29:6835', 'nonce': 3647517494}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.29:6837/1333834114', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6836', 'nonce': 1333834114}, {'type': 'v1', 'addr': '172.21.15.29:6837', 'nonce': 1333834114}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6835/2529273890', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 2529273890}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 2529273890}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.29:6839/3625063120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6838', 'nonce': 3625063120}, {'type': 'v1', 'addr': '172.21.15.29:6839', 'nonce': 3625063120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.26:6837/3301773578', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6836', 'nonce': 3301773578}, {'type': 'v1', 'addr': '172.21.15.26:6837', 'nonce': 3301773578}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.26:6839/903985766', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 903985766}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 903985766}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24524, 'qdb_cluster': [24497, 24524, 24487]} 2024-05-14T09:34:36.499 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-14T09:34:36.499 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-14T09:34:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:36 smithi029 ceph-mon[36249]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:36 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3702186493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:34:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:36 smithi026 ceph-mon[36056]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:36 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3702186493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:34:36.752 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:38.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.484+0000 7f71e4956700 1 -- 172.21.15.2:0/2908677208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71e0108740 msgr2=0x7f71e01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:38.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.484+0000 7f71e4956700 1 --2- 172.21.15.2:0/2908677208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71e0108740 0x7f71e01100a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f71d800b0d0 tx=0x7f71d800b3e0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.484+0000 7f71e4956700 1 -- 172.21.15.2:0/2908677208 shutdown_connections 2024-05-14T09:34:38.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.484+0000 7f71e4956700 1 --2- 172.21.15.2:0/2908677208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71e0108740 0x7f71e01100a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.484+0000 7f71e4956700 1 --2- 172.21.15.2:0/2908677208 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71e01034c0 0x7f71e01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.484+0000 7f71e4956700 1 --2- 172.21.15.2:0/2908677208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e0107df0 0x7f71e0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.484+0000 7f71e4956700 1 -- 172.21.15.2:0/2908677208 >> 172.21.15.2:0/2908677208 conn(0x7f71e00fedc0 msgr2=0x7f71e01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.484+0000 7f71e4956700 1 -- 172.21.15.2:0/2908677208 shutdown_connections 2024-05-14T09:34:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.484+0000 7f71e4956700 1 -- 172.21.15.2:0/2908677208 wait complete. 2024-05-14T09:34:38.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.485+0000 7f71e4956700 1 Processor -- start 2024-05-14T09:34:38.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.485+0000 7f71e4956700 1 -- start start 2024-05-14T09:34:38.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71e4956700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71e01034c0 0x7f71e019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71e4956700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71e0107df0 0x7f71e019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71e4956700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e0108740 0x7f71e019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71e4956700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f71e0113370 con 0x7f71e01034c0 2024-05-14T09:34:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71e4956700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01131f0 con 0x7f71e0108740 2024-05-14T09:34:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71e4956700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01134f0 con 0x7f71e0107df0 2024-05-14T09:34:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71de59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71e01034c0 0x7f71e019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71de59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71e01034c0 0x7f71e019bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35592/0 (socket says 172.21.15.2:35592) 2024-05-14T09:34:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71de59c700 1 -- 172.21.15.2:0/3913798672 learned_addr learned my addr 172.21.15.2:0/3913798672 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71d5d9b700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71e0107df0 0x7f71e019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71ded9d700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e0108740 0x7f71e019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71de59c700 1 -- 172.21.15.2:0/3913798672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71e0107df0 msgr2=0x7f71e019c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71de59c700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71e0107df0 0x7f71e019c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71de59c700 1 -- 172.21.15.2:0/3913798672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e0108740 msgr2=0x7f71e019c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71de59c700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e0108740 0x7f71e019c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71de59c700 1 -- 172.21.15.2:0/3913798672 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e01a6f60 con 0x7f71e01034c0 2024-05-14T09:34:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71d5d9b700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71e0107df0 0x7f71e019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71ded9d700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e0108740 0x7f71e019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.486+0000 7f71de59c700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71e01034c0 0x7f71e019bc10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f71d000e9a0 tx=0x7f71d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.487+0000 7f71d7fff700 1 -- 172.21.15.2:0/3913798672 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d000ca10 con 0x7f71e01034c0 2024-05-14T09:34:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.487+0000 7f71e4956700 1 -- 172.21.15.2:0/3913798672 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e01a7250 con 0x7f71e01034c0 2024-05-14T09:34:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.487+0000 7f71d7fff700 1 -- 172.21.15.2:0/3913798672 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d000cbb0 con 0x7f71e01034c0 2024-05-14T09:34:38.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.487+0000 7f71e4956700 1 -- 172.21.15.2:0/3913798672 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e01a7790 con 0x7f71e01034c0 2024-05-14T09:34:38.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.487+0000 7f71d7fff700 1 -- 172.21.15.2:0/3913798672 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0018680 con 0x7f71e01034c0 2024-05-14T09:34:38.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.490+0000 7f71d7fff700 1 -- 172.21.15.2:0/3913798672 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f71d0010430 con 0x7f71e01034c0 2024-05-14T09:34:38.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.490+0000 7f71d7fff700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f71cc0b2880 0x7f71cc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.490+0000 7f71d7fff700 1 -- 172.21.15.2:0/3913798672 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f71d00e2780 con 0x7f71e01034c0 2024-05-14T09:34:38.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.490+0000 7f71d5d9b700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f71cc0b2880 0x7f71cc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.491+0000 7f71d5d9b700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f71cc0b2880 0x7f71cc0b4d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71e0102360 tx=0x7f71c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:38.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.491+0000 7f71f336ab80 1 Processor -- start 2024-05-14T09:34:38.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.491+0000 7f71f336ab80 1 -- start start 2024-05-14T09:34:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71f336ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca03ffdf0 0x563ca057d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71f336ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca057dc10 0x563ca0581ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71f336ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca0582420 0x563ca0582830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71f336ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563ca049f810 con 0x563ca057dc10 2024-05-14T09:34:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71f336ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563ca043e770 con 0x563ca0582420 2024-05-14T09:34:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71f336ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563ca049b7d0 con 0x563ca03ffdf0 2024-05-14T09:34:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71d5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca057dc10 0x563ca0581ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71d5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca057dc10 0x563ca0581ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35604/0 (socket says 172.21.15.2:35604) 2024-05-14T09:34:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71d5d9b700 1 -- 172.21.15.2:0/891024111 learned_addr learned my addr 172.21.15.2:0/891024111 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71de59c700 1 --2- 172.21.15.2:0/891024111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca03ffdf0 0x563ca057d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71ded9d700 1 --2- 172.21.15.2:0/891024111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca0582420 0x563ca0582830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71de59c700 1 -- 172.21.15.2:0/891024111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca0582420 msgr2=0x563ca0582830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:38.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71de59c700 1 --2- 172.21.15.2:0/891024111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca0582420 0x563ca0582830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71de59c700 1 -- 172.21.15.2:0/891024111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca057dc10 msgr2=0x563ca0581ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:38.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71de59c700 1 --2- 172.21.15.2:0/891024111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca057dc10 0x563ca0581ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71de59c700 1 -- 172.21.15.2:0/891024111 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca03ec880 con 0x563ca03ffdf0 2024-05-14T09:34:38.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.492+0000 7f71de59c700 1 --2- 172.21.15.2:0/891024111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca03ffdf0 0x563ca057d6d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f71d00dfe60 tx=0x7f71d00b2210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53434e36027b999d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:38.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.493+0000 7f71d4d99700 1 -- 172.21.15.2:0/891024111 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d00aa080 con 0x563ca03ffdf0 2024-05-14T09:34:38.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.493+0000 7f71d4d99700 1 -- 172.21.15.2:0/891024111 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d00a9980 con 0x563ca03ffdf0 2024-05-14T09:34:38.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.493+0000 7f71d4d99700 1 -- 172.21.15.2:0/891024111 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d00b2350 con 0x563ca03ffdf0 2024-05-14T09:34:38.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.494+0000 7f71f336ab80 1 -- 172.21.15.2:0/891024111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca03ffdf0 msgr2=0x563ca057d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:38.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.494+0000 7f71f336ab80 1 --2- 172.21.15.2:0/891024111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca03ffdf0 0x563ca057d6d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f71d00dfe60 tx=0x7f71d00b2210 comp rx=0 tx=0).stop 2024-05-14T09:34:38.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.494+0000 7f71f336ab80 1 -- 172.21.15.2:0/891024111 shutdown_connections 2024-05-14T09:34:38.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.494+0000 7f71f336ab80 1 --2- 172.21.15.2:0/891024111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca0582420 0x563ca0582830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.494+0000 7f71f336ab80 1 --2- 172.21.15.2:0/891024111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca03ffdf0 0x563ca057d6d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.494+0000 7f71f336ab80 1 --2- 172.21.15.2:0/891024111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca057dc10 0x563ca0581ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.494+0000 7f71f336ab80 1 -- 172.21.15.2:0/891024111 >> 172.21.15.2:0/891024111 conn(0x563ca04475b0 msgr2=0x563ca0578460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:38.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.494+0000 7f71f336ab80 1 -- 172.21.15.2:0/891024111 shutdown_connections 2024-05-14T09:34:38.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.494+0000 7f71f336ab80 1 -- 172.21.15.2:0/891024111 wait complete. 2024-05-14T09:34:38.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.494+0000 7f71f336ab80 1 -- start start 2024-05-14T09:34:38.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.494+0000 7f71f336ab80 1 Processor -- start 2024-05-14T09:34:38.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71f336ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca03ffdf0 0x563ca05871a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71f336ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca057dc10 0x563ca05876e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71f336ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca0582420 0x563ca0587c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71f336ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563ca058a3d0 con 0x563ca03ffdf0 2024-05-14T09:34:38.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71f336ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563ca0405be0 con 0x563ca057dc10 2024-05-14T09:34:38.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71f336ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563ca058a550 con 0x563ca0582420 2024-05-14T09:34:38.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71de59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca03ffdf0 0x563ca05871a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71f336ab80 10 client.0 _ll_register_callbacks cb 0x563ca04378d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:34:38.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71de59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca03ffdf0 0x563ca05871a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35616/0 (socket says 172.21.15.2:35616) 2024-05-14T09:34:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71de59c700 1 -- 172.21.15.2:0/4122405884 learned_addr learned my addr 172.21.15.2:0/4122405884 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71d5d9b700 1 --2- 172.21.15.2:0/4122405884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca057dc10 0x563ca05876e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71ded9d700 1 --2- 172.21.15.2:0/4122405884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca0582420 0x563ca0587c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71de59c700 1 -- 172.21.15.2:0/4122405884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca0582420 msgr2=0x563ca0587c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71de59c700 1 --2- 172.21.15.2:0/4122405884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca0582420 0x563ca0587c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71de59c700 1 -- 172.21.15.2:0/4122405884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca057dc10 msgr2=0x563ca05876e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71de59c700 1 --2- 172.21.15.2:0/4122405884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca057dc10 0x563ca05876e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71de59c700 1 -- 172.21.15.2:0/4122405884 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca043b640 con 0x563ca03ffdf0 2024-05-14T09:34:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71d5d9b700 1 --2- 172.21.15.2:0/4122405884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca057dc10 0x563ca05876e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.495+0000 7f71ded9d700 1 --2- 172.21.15.2:0/4122405884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca0582420 0x563ca0587c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.496+0000 7f71de59c700 1 --2- 172.21.15.2:0/4122405884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca03ffdf0 0x563ca05871a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f71d00a9680 tx=0x7f71d00addb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bbfa4dcf6fbfa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.496+0000 7f71c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:34:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.496+0000 7f71c5ffb700 1 -- 172.21.15.2:0/4122405884 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d006e060 con 0x563ca03ffdf0 2024-05-14T09:34:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.496+0000 7f71c5ffb700 1 -- 172.21.15.2:0/4122405884 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f71d00b2350 con 0x563ca03ffdf0 2024-05-14T09:34:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.496+0000 7f71f336ab80 1 -- 172.21.15.2:0/4122405884 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d00a9980 con 0x563ca03ffdf0 2024-05-14T09:34:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.496+0000 7f71c5ffb700 1 -- 172.21.15.2:0/4122405884 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f71d00cc2e0 con 0x563ca03ffdf0 2024-05-14T09:34:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.496+0000 7f71f336ab80 10 client.15198 fetch_fsmap learned FSMap version 26 2024-05-14T09:34:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.496+0000 7f71f336ab80 1 -- 172.21.15.2:0/4122405884 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ca0595080 con 0x563ca03ffdf0 2024-05-14T09:34:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.497+0000 7f71c5ffb700 1 -- 172.21.15.2:0/4122405884 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d00a9980 con 0x563ca03ffdf0 2024-05-14T09:34:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.497+0000 7f71c5ffb700 1 -- 172.21.15.2:0/4122405884 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f71d00a52f0 con 0x563ca03ffdf0 2024-05-14T09:34:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.497+0000 7f71f336ab80 10 client.15198 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T09:34:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.497+0000 7f71f336ab80 10 client.15198 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T09:34:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.497+0000 7f71f336ab80 20 client.15198 populate_metadata read hostname 'smithi002' 2024-05-14T09:34:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.497+0000 7f71f336ab80 1 --2- 172.21.15.2:0/4122405884 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x563ca05963d0 0x563ca0598520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.497+0000 7f71f336ab80 4 client.15198 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:34:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.497+0000 7f71f336ab80 1 -- 172.21.15.2:0/4122405884 --> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ca04ad1f0 con 0x563ca05963d0 2024-05-14T09:34:38.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.497+0000 7f71d5d9b700 1 --2- 172.21.15.2:0/4122405884 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x563ca05963d0 0x563ca0598520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.498+0000 7f71c5ffb700 10 client.15198 ms_handle_connect on v2:172.21.15.29:6838/3625063120 2024-05-14T09:34:38.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.498+0000 7f71d5d9b700 1 --2- 172.21.15.2:0/4122405884 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x563ca05963d0 0x563ca0598520 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44284d3ad32c8eeb server_cookie=df3decd4381aa868 in_seq=0 out_seq=0 2024-05-14T09:34:38.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.498+0000 7f71c5ffb700 1 -- 172.21.15.2:0/4122405884 <== mds.2 v2:172.21.15.29:6838/3625063120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563ca04ad1f0 con 0x563ca05963d0 2024-05-14T09:34:38.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.498+0000 7f71c5ffb700 10 client.15198 handle_command_reply: tid=0 2024-05-14T09:34:38.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.499+0000 7f71f336ab80 1 client.15198 shutdown 2024-05-14T09:34:38.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:38 smithi002 ceph-mon[31557]: pgmap v414: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:34:38.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:34:38.506 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:38.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:38.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:38.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:38.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.499+0000 7f71f336ab80 1 -- 172.21.15.2:0/4122405884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca03ffdf0 msgr2=0x563ca05871a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:38.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.499+0000 7f71f336ab80 1 --2- 172.21.15.2:0/4122405884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca03ffdf0 0x563ca05871a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f71d00a9680 tx=0x7f71d00addb0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.499+0000 7f71f336ab80 1 -- 172.21.15.2:0/4122405884 shutdown_connections 2024-05-14T09:34:38.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.499+0000 7f71f336ab80 1 --2- 172.21.15.2:0/4122405884 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x563ca05963d0 0x563ca0598520 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.499+0000 7f71f336ab80 1 --2- 172.21.15.2:0/4122405884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca0582420 0x563ca0587c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.499+0000 7f71c5ffb700 0 client.15198 ms_handle_reset on v2:172.21.15.29:6838/3625063120 2024-05-14T09:34:38.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.499+0000 7f71f336ab80 1 --2- 172.21.15.2:0/4122405884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca03ffdf0 0x563ca05871a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.499+0000 7f71f336ab80 1 --2- 172.21.15.2:0/4122405884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca057dc10 0x563ca05876e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.499+0000 7f71f336ab80 1 -- 172.21.15.2:0/4122405884 >> 172.21.15.2:0/4122405884 conn(0x563ca04475b0 msgr2=0x563ca0582830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:38.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.499+0000 7f71f336ab80 1 -- 172.21.15.2:0/4122405884 shutdown_connections 2024-05-14T09:34:38.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.499+0000 7f71f336ab80 1 -- 172.21.15.2:0/4122405884 wait complete. 2024-05-14T09:34:38.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.499+0000 7f71f336ab80 20 client.15198 trim_cache size 0 max 16384 2024-05-14T09:34:38.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:38 smithi026 ceph-mon[36056]: pgmap v414: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:34:38.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:34:38.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:38.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:38.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:38.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:38.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.522+0000 7f71f336ab80 1 Processor -- start 2024-05-14T09:34:38.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.522+0000 7f71f336ab80 1 -- start start 2024-05-14T09:34:38.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.522+0000 7f71f336ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca03ffdf0 0x563ca05864c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.522+0000 7f71f336ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca057dc10 0x563ca0593820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.522+0000 7f71f336ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca0582420 0x563ca0587a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.522+0000 7f71f336ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563ca0596860 con 0x563ca057dc10 2024-05-14T09:34:38.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.522+0000 7f71f336ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563ca0596f30 con 0x563ca0582420 2024-05-14T09:34:38.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.522+0000 7f71f336ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563ca0597600 con 0x563ca03ffdf0 2024-05-14T09:34:38.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.522+0000 7f71d5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca057dc10 0x563ca0593820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.522+0000 7f71de59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca03ffdf0 0x563ca05864c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.522+0000 7f71d5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca057dc10 0x563ca0593820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35622/0 (socket says 172.21.15.2:35622) 2024-05-14T09:34:38.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.522+0000 7f71d5d9b700 1 -- 172.21.15.2:0/2251021211 learned_addr learned my addr 172.21.15.2:0/2251021211 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:38.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.522+0000 7f71de59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca03ffdf0 0x563ca05864c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:34136/0 (socket says 172.21.15.2:34136) 2024-05-14T09:34:38.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.522+0000 7f71ded9d700 1 --2- 172.21.15.2:0/2251021211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca0582420 0x563ca0587a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.523+0000 7f71d5d9b700 1 -- 172.21.15.2:0/2251021211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca03ffdf0 msgr2=0x563ca05864c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:38.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.523+0000 7f71d5d9b700 1 --2- 172.21.15.2:0/2251021211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca03ffdf0 0x563ca05864c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.523+0000 7f71d5d9b700 1 -- 172.21.15.2:0/2251021211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca0582420 msgr2=0x563ca0587a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:38.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.523+0000 7f71d5d9b700 1 --2- 172.21.15.2:0/2251021211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca0582420 0x563ca0587a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.523+0000 7f71d5d9b700 1 -- 172.21.15.2:0/2251021211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca0589f60 con 0x563ca057dc10 2024-05-14T09:34:38.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.523+0000 7f71de59c700 1 --2- 172.21.15.2:0/2251021211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca03ffdf0 0x563ca05864c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:38.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.523+0000 7f71ded9d700 1 --2- 172.21.15.2:0/2251021211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca0582420 0x563ca0587a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:38.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.523+0000 7f71d5d9b700 1 --2- 172.21.15.2:0/2251021211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca057dc10 0x563ca0593820 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x563ca0579b20 tx=0x7f71c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9953856412d879 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:38.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.523+0000 7f71c7fff700 1 -- 172.21.15.2:0/2251021211 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c801ea90 con 0x563ca057dc10 2024-05-14T09:34:38.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.523+0000 7f71c7fff700 1 -- 172.21.15.2:0/2251021211 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c802f070 con 0x563ca057dc10 2024-05-14T09:34:38.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.523+0000 7f71c7fff700 1 -- 172.21.15.2:0/2251021211 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c800f7e0 con 0x563ca057dc10 2024-05-14T09:34:38.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.524+0000 7f71f336ab80 1 -- 172.21.15.2:0/2251021211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca057dc10 msgr2=0x563ca0593820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:38.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.524+0000 7f71f336ab80 1 --2- 172.21.15.2:0/2251021211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca057dc10 0x563ca0593820 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x563ca0579b20 tx=0x7f71c8014000 comp rx=0 tx=0).stop 2024-05-14T09:34:38.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.524+0000 7f71f336ab80 1 -- 172.21.15.2:0/2251021211 shutdown_connections 2024-05-14T09:34:38.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.524+0000 7f71f336ab80 1 --2- 172.21.15.2:0/2251021211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca0582420 0x563ca0587a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.524+0000 7f71f336ab80 1 --2- 172.21.15.2:0/2251021211 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca03ffdf0 0x563ca05864c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.524+0000 7f71f336ab80 1 --2- 172.21.15.2:0/2251021211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca057dc10 0x563ca0593820 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.524+0000 7f71f336ab80 1 -- 172.21.15.2:0/2251021211 >> 172.21.15.2:0/2251021211 conn(0x563ca04475b0 msgr2=0x563ca0577b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:38.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.524+0000 7f71f336ab80 1 -- 172.21.15.2:0/2251021211 shutdown_connections 2024-05-14T09:34:38.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.524+0000 7f71f336ab80 1 -- 172.21.15.2:0/2251021211 wait complete. 2024-05-14T09:34:38.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.525+0000 7f71f336ab80 1 -- start start 2024-05-14T09:34:38.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.525+0000 7f71f336ab80 1 Processor -- start 2024-05-14T09:34:38.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.525+0000 7f71f336ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca03ffdf0 0x563ca05870b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.525+0000 7f71f336ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca057dc10 0x563ca05875f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.525+0000 7f71f336ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca0582420 0x563ca0587b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.525+0000 7f71f336ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563ca05983a0 con 0x563ca0582420 2024-05-14T09:34:38.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.525+0000 7f71f336ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563ca0597cd0 con 0x563ca03ffdf0 2024-05-14T09:34:38.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.525+0000 7f71f336ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563ca0598a70 con 0x563ca057dc10 2024-05-14T09:34:38.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.525+0000 7f71f336ab80 10 client.0 _ll_register_callbacks cb 0x563ca04378d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:34:38.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71de59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca03ffdf0 0x563ca05870b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71ded9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca0582420 0x563ca0587b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71d5d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca057dc10 0x563ca05875f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71de59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca03ffdf0 0x563ca05870b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:41590/0 (socket says 172.21.15.2:41590) 2024-05-14T09:34:38.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71de59c700 1 -- 172.21.15.2:0/1776042784 learned_addr learned my addr 172.21.15.2:0/1776042784 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:38.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71de59c700 1 -- 172.21.15.2:0/1776042784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca057dc10 msgr2=0x563ca05875f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:38.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71de59c700 1 --2- 172.21.15.2:0/1776042784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca057dc10 0x563ca05875f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71de59c700 1 -- 172.21.15.2:0/1776042784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca0582420 msgr2=0x563ca0587b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:38.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71de59c700 1 --2- 172.21.15.2:0/1776042784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca0582420 0x563ca0587b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71de59c700 1 -- 172.21.15.2:0/1776042784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca057e020 con 0x563ca03ffdf0 2024-05-14T09:34:38.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71de59c700 1 --2- 172.21.15.2:0/1776042784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca03ffdf0 0x563ca05870b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f71d000c2d0 tx=0x7f71d0010930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a18401913d000b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:38.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:34:38.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71c5ffb700 1 -- 172.21.15.2:0/1776042784 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d006e040 con 0x563ca03ffdf0 2024-05-14T09:34:38.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71f336ab80 1 -- 172.21.15.2:0/1776042784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c802f070 con 0x563ca03ffdf0 2024-05-14T09:34:38.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71c5ffb700 1 -- 172.21.15.2:0/1776042784 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f71d00a8790 con 0x563ca03ffdf0 2024-05-14T09:34:38.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.526+0000 7f71c5ffb700 1 -- 172.21.15.2:0/1776042784 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f71d00b2120 con 0x563ca03ffdf0 2024-05-14T09:34:38.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.527+0000 7f71f336ab80 10 client.24962 fetch_fsmap learned FSMap version 26 2024-05-14T09:34:38.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.527+0000 7f71f336ab80 1 -- 172.21.15.2:0/1776042784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ca0602770 con 0x563ca03ffdf0 2024-05-14T09:34:38.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.527+0000 7f71c5ffb700 1 -- 172.21.15.2:0/1776042784 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c802f070 con 0x563ca03ffdf0 2024-05-14T09:34:38.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.527+0000 7f71c5ffb700 1 -- 172.21.15.2:0/1776042784 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f71d00b2350 con 0x563ca03ffdf0 2024-05-14T09:34:38.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.527+0000 7f71f336ab80 10 client.24962 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T09:34:38.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.527+0000 7f71f336ab80 10 client.24962 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T09:34:38.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.527+0000 7f71f336ab80 20 client.24962 populate_metadata read hostname 'smithi002' 2024-05-14T09:34:38.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.527+0000 7f71f336ab80 1 --2- 172.21.15.2:0/1776042784 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x563ca05963d0 0x563ca0605cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:38.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.527+0000 7f71f336ab80 4 client.24962 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T09:34:38.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.527+0000 7f71f336ab80 1 -- 172.21.15.2:0/1776042784 --> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] -- command(tid 0: {"prefix": "status"}) -- 0x563ca04a7290 con 0x563ca05963d0 2024-05-14T09:34:38.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.527+0000 7f71d5d9b700 1 --2- 172.21.15.2:0/1776042784 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x563ca05963d0 0x563ca0605cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:38.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71d5d9b700 1 --2- 172.21.15.2:0/1776042784 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x563ca05963d0 0x563ca0605cd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef882a61eb102e2a server_cookie=9ec92ffb59b7ff9e in_seq=0 out_seq=0 2024-05-14T09:34:38.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71c5ffb700 10 client.24962 ms_handle_connect on v2:172.21.15.29:6838/3625063120 2024-05-14T09:34:38.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71c5ffb700 1 -- 172.21.15.2:0/1776042784 <== mds.2 v2:172.21.15.29:6838/3625063120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563ca04a7290 con 0x563ca05963d0 2024-05-14T09:34:38.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71c5ffb700 10 client.24962 handle_command_reply: tid=0 2024-05-14T09:34:38.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71f336ab80 1 client.24962 shutdown 2024-05-14T09:34:38.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71f336ab80 1 -- 172.21.15.2:0/1776042784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca03ffdf0 msgr2=0x563ca05870b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:38.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71f336ab80 1 --2- 172.21.15.2:0/1776042784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca03ffdf0 0x563ca05870b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f71d000c2d0 tx=0x7f71d0010930 comp rx=0 tx=0).stop 2024-05-14T09:34:38.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71f336ab80 1 -- 172.21.15.2:0/1776042784 shutdown_connections 2024-05-14T09:34:38.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71f336ab80 1 --2- 172.21.15.2:0/1776042784 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x563ca05963d0 0x563ca0605cd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71f336ab80 1 --2- 172.21.15.2:0/1776042784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563ca03ffdf0 0x563ca05870b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71f336ab80 1 --2- 172.21.15.2:0/1776042784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ca0582420 0x563ca0587b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71f336ab80 1 --2- 172.21.15.2:0/1776042784 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563ca057dc10 0x563ca05875f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71f336ab80 1 -- 172.21.15.2:0/1776042784 >> 172.21.15.2:0/1776042784 conn(0x563ca04475b0 msgr2=0x563ca0586220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:38.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71c5ffb700 0 client.24962 ms_handle_reset on v2:172.21.15.29:6838/3625063120 2024-05-14T09:34:38.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71f336ab80 1 -- 172.21.15.2:0/1776042784 shutdown_connections 2024-05-14T09:34:38.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.528+0000 7f71f336ab80 1 -- 172.21.15.2:0/1776042784 wait complete. 2024-05-14T09:34:38.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.529+0000 7f71f336ab80 20 client.24962 trim_cache size 0 max 16384 2024-05-14T09:34:38.533 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:34:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "50d5aebe-11d3-11ef-bc98-c7b262605968", 2024-05-14T09:34:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 2, 2024-05-14T09:34:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24487, 2024-05-14T09:34:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T09:34:38.533 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T09:34:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T09:34:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 162.81962499799999, 2024-05-14T09:34:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 26, 2024-05-14T09:34:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 90, 2024-05-14T09:34:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 88, 2024-05-14T09:34:38.534 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 244.37937874100001 2024-05-14T09:34:38.534 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:34:38.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.529+0000 7f71e4956700 1 -- 172.21.15.2:0/3913798672 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f71cc0b2880 msgr2=0x7f71cc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:38.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.529+0000 7f71e4956700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f71cc0b2880 0x7f71cc0b4d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71e0102360 tx=0x7f71c8009260 comp rx=0 tx=0).stop 2024-05-14T09:34:38.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.529+0000 7f71e4956700 1 -- 172.21.15.2:0/3913798672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71e01034c0 msgr2=0x7f71e019bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:38.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.529+0000 7f71e4956700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71e01034c0 0x7f71e019bc10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f71d000e9a0 tx=0x7f71d000ecb0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.529+0000 7f71e4956700 1 -- 172.21.15.2:0/3913798672 shutdown_connections 2024-05-14T09:34:38.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.529+0000 7f71e4956700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f71cc0b2880 0x7f71cc0b4d90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.529+0000 7f71e4956700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e0108740 0x7f71e019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.529+0000 7f71e4956700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71e01034c0 0x7f71e019bc10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.529+0000 7f71e4956700 1 --2- 172.21.15.2:0/3913798672 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71e0107df0 0x7f71e019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:38.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.529+0000 7f71e4956700 1 -- 172.21.15.2:0/3913798672 >> 172.21.15.2:0/3913798672 conn(0x7f71e00fedc0 msgr2=0x7f71e0107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:38.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.530+0000 7f71e4956700 1 -- 172.21.15.2:0/3913798672 shutdown_connections 2024-05-14T09:34:38.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:38.530+0000 7f71e4956700 1 -- 172.21.15.2:0/3913798672 wait complete. 2024-05-14T09:34:38.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:38 smithi029 ceph-mon[36249]: pgmap v414: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:34:38.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:34:38.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:38.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:38.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:38.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:39.247 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-14T09:34:39.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:39 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:39.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:39 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:39.509 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:39.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:39 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:39.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:39 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:39.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:39 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:39.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:39 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:40.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:40 smithi002 ceph-mon[31557]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:40.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:34:40.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:34:40.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:40 smithi029 ceph-mon[36249]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:34:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:34:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:40 smithi026 ceph-mon[36056]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:34:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:34:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:34:41.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.181+0000 7f45734f6700 1 -- 172.21.15.2:0/376909810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c108f10 msgr2=0x7f456c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:41.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.181+0000 7f45734f6700 1 --2- 172.21.15.2:0/376909810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c108f10 0x7f456c109320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f455c0099c0 tx=0x7f455c009cd0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.181+0000 7f45734f6700 1 -- 172.21.15.2:0/376909810 shutdown_connections 2024-05-14T09:34:41.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.181+0000 7f45734f6700 1 --2- 172.21.15.2:0/376909810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456c109860 0x7f456c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.181+0000 7f45734f6700 1 --2- 172.21.15.2:0/376909810 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f456c1045e0 0x7f456c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.181+0000 7f45734f6700 1 --2- 172.21.15.2:0/376909810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c108f10 0x7f456c109320 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.181+0000 7f45734f6700 1 -- 172.21.15.2:0/376909810 >> 172.21.15.2:0/376909810 conn(0x7f456c0ffed0 msgr2=0x7f456c102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:41.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.182+0000 7f45734f6700 1 -- 172.21.15.2:0/376909810 shutdown_connections 2024-05-14T09:34:41.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.182+0000 7f45734f6700 1 -- 172.21.15.2:0/376909810 wait complete. 2024-05-14T09:34:41.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.182+0000 7f45734f6700 1 Processor -- start 2024-05-14T09:34:41.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.182+0000 7f45734f6700 1 -- start start 2024-05-14T09:34:41.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f45734f6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c1045e0 0x7f456c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f45734f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456c108f10 0x7f456c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f45734f6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f456c109860 0x7f456c19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f45734f6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f456c114470 con 0x7f456c1045e0 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f45734f6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1142f0 con 0x7f456c108f10 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f45734f6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1145f0 con 0x7f456c109860 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f4571292700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c1045e0 0x7f456c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f4571292700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c1045e0 0x7f456c19ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35648/0 (socket says 172.21.15.2:35648) 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f4571292700 1 -- 172.21.15.2:0/4058501059 learned_addr learned my addr 172.21.15.2:0/4058501059 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f4570a91700 1 --2- 172.21.15.2:0/4058501059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456c108f10 0x7f456c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f4571a93700 1 --2- 172.21.15.2:0/4058501059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f456c109860 0x7f456c19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f4571292700 1 -- 172.21.15.2:0/4058501059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f456c109860 msgr2=0x7f456c19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f4571292700 1 --2- 172.21.15.2:0/4058501059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f456c109860 0x7f456c19f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f4571292700 1 -- 172.21.15.2:0/4058501059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456c108f10 msgr2=0x7f456c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f4571292700 1 --2- 172.21.15.2:0/4058501059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456c108f10 0x7f456c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.183+0000 7f4571292700 1 -- 172.21.15.2:0/4058501059 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c19fbc0 con 0x7f456c1045e0 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.184+0000 7f4571292700 1 --2- 172.21.15.2:0/4058501059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c1045e0 0x7f456c19ada0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f456800c9b0 tx=0x7f456800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.184+0000 7f45627fc700 1 -- 172.21.15.2:0/4058501059 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568007b00 con 0x7f456c1045e0 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.184+0000 7f45627fc700 1 -- 172.21.15.2:0/4058501059 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f456800ce50 con 0x7f456c1045e0 2024-05-14T09:34:41.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.184+0000 7f45734f6700 1 -- 172.21.15.2:0/4058501059 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456c19feb0 con 0x7f456c1045e0 2024-05-14T09:34:41.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.184+0000 7f45627fc700 1 -- 172.21.15.2:0/4058501059 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568018660 con 0x7f456c1045e0 2024-05-14T09:34:41.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.184+0000 7f45734f6700 1 -- 172.21.15.2:0/4058501059 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456c1a03c0 con 0x7f456c1045e0 2024-05-14T09:34:41.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.187+0000 7f45627fc700 1 -- 172.21.15.2:0/4058501059 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4568004830 con 0x7f456c1045e0 2024-05-14T09:34:41.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f4581f0ab80 1 Processor -- start 2024-05-14T09:34:41.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f45627fc700 1 --2- 172.21.15.2:0/4058501059 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f45580b2880 0x7f45580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f4581f0ab80 1 -- start start 2024-05-14T09:34:41.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f4571a93700 1 --2- 172.21.15.2:0/4058501059 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f45580b2880 0x7f45580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f45627fc700 1 -- 172.21.15.2:0/4058501059 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f45680e2570 con 0x7f456c1045e0 2024-05-14T09:34:41.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f4581f0ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab0f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f4581f0ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab13ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f4581f0ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab14420 0x55cf2ab14830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f4581f0ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cf2aa31810 con 0x55cf2ab14420 2024-05-14T09:34:41.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f4581f0ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cf2a9d0770 con 0x55cf2ab0fc10 2024-05-14T09:34:41.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f4581f0ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cf2aa2d7d0 con 0x55cf2a991df0 2024-05-14T09:34:41.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f4571a93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab14420 0x55cf2ab14830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f4571292700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab0f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f4571a93700 1 --2- 172.21.15.2:0/4058501059 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f45580b2880 0x7f45580b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f456400f3c0 tx=0x7f4564007950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:41.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f4571292700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab0f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:34170/0 (socket says 172.21.15.2:34170) 2024-05-14T09:34:41.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f4570a91700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab13ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.188+0000 7f4571292700 1 -- 172.21.15.2:0/2053421410 learned_addr learned my addr 172.21.15.2:0/2053421410 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:41.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.189+0000 7f4571292700 1 -- 172.21.15.2:0/2053421410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab0fc10 msgr2=0x55cf2ab13ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:41.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.189+0000 7f4571292700 1 --2- 172.21.15.2:0/2053421410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab13ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.189+0000 7f4571292700 1 -- 172.21.15.2:0/2053421410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab14420 msgr2=0x55cf2ab14830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:41.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.189+0000 7f4571292700 1 --2- 172.21.15.2:0/2053421410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab14420 0x55cf2ab14830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.189+0000 7f4571292700 1 -- 172.21.15.2:0/2053421410 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf2a97e880 con 0x55cf2a991df0 2024-05-14T09:34:41.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.189+0000 7f4570a91700 1 --2- 172.21.15.2:0/2053421410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab13ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:41.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.189+0000 7f4571a93700 1 --2- 172.21.15.2:0/2053421410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab14420 0x55cf2ab14830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:41.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.189+0000 7f4571292700 1 --2- 172.21.15.2:0/2053421410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab0f6d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f45680a69e0 tx=0x7f45680a9190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36d917d9f1f62e93 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:41.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.189+0000 7f45577fe700 1 -- 172.21.15.2:0/2053421410 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680c3070 con 0x55cf2a991df0 2024-05-14T09:34:41.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.189+0000 7f45577fe700 1 -- 172.21.15.2:0/2053421410 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45680cc0b0 con 0x55cf2a991df0 2024-05-14T09:34:41.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.189+0000 7f45577fe700 1 -- 172.21.15.2:0/2053421410 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680e2210 con 0x55cf2a991df0 2024-05-14T09:34:41.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.190+0000 7f4581f0ab80 1 -- 172.21.15.2:0/2053421410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 msgr2=0x55cf2ab0f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:41.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.190+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/2053421410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab0f6d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f45680a69e0 tx=0x7f45680a9190 comp rx=0 tx=0).stop 2024-05-14T09:34:41.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.190+0000 7f4581f0ab80 1 -- 172.21.15.2:0/2053421410 shutdown_connections 2024-05-14T09:34:41.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.190+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/2053421410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab14420 0x55cf2ab14830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.190+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/2053421410 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab0f6d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.190+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/2053421410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab13ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.190+0000 7f4581f0ab80 1 -- 172.21.15.2:0/2053421410 >> 172.21.15.2:0/2053421410 conn(0x55cf2a9d95b0 msgr2=0x55cf2ab0a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:41.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.190+0000 7f4581f0ab80 1 -- 172.21.15.2:0/2053421410 shutdown_connections 2024-05-14T09:34:41.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.190+0000 7f4581f0ab80 1 -- 172.21.15.2:0/2053421410 wait complete. 2024-05-14T09:34:41.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.191+0000 7f4581f0ab80 1 -- start start 2024-05-14T09:34:41.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.191+0000 7f4581f0ab80 1 Processor -- start 2024-05-14T09:34:41.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.191+0000 7f4581f0ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab19140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.191+0000 7f4581f0ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab19680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.191+0000 7f4581f0ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab14420 0x55cf2ab19bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.191+0000 7f4581f0ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cf2ab1c3d0 con 0x55cf2ab0fc10 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.191+0000 7f4581f0ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cf2a997be0 con 0x55cf2ab14420 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.191+0000 7f4581f0ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cf2ab1c550 con 0x55cf2a991df0 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.191+0000 7f4581f0ab80 10 client.0 _ll_register_callbacks cb 0x55cf2a9c98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.191+0000 7f4570a91700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab19680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.192+0000 7f4571292700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab19140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.192+0000 7f4570a91700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab19680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35662/0 (socket says 172.21.15.2:35662) 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.192+0000 7f4570a91700 1 -- 172.21.15.2:0/1681837896 learned_addr learned my addr 172.21.15.2:0/1681837896 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.192+0000 7f4571a93700 1 --2- 172.21.15.2:0/1681837896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab14420 0x55cf2ab19bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.192+0000 7f4571292700 1 -- 172.21.15.2:0/1681837896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab14420 msgr2=0x55cf2ab19bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.192+0000 7f4571292700 1 --2- 172.21.15.2:0/1681837896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab14420 0x55cf2ab19bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.192+0000 7f4571292700 1 -- 172.21.15.2:0/1681837896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab0fc10 msgr2=0x55cf2ab19680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.192+0000 7f4571292700 1 --2- 172.21.15.2:0/1681837896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab19680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.192+0000 7f4571292700 1 -- 172.21.15.2:0/1681837896 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf2a9cd640 con 0x55cf2a991df0 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.192+0000 7f4570a91700 1 --2- 172.21.15.2:0/1681837896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab19680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.192+0000 7f4571292700 1 --2- 172.21.15.2:0/1681837896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab19140 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f45680a69e0 tx=0x7f45680047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b5f5efb372d8b86 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.192+0000 7f4554ff9700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:34:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.192+0000 7f4554ff9700 1 -- 172.21.15.2:0/1681837896 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680c3070 con 0x55cf2a991df0 2024-05-14T09:34:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.192+0000 7f4581f0ab80 1 -- 172.21.15.2:0/1681837896 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cf2aa3f1f0 con 0x55cf2a991df0 2024-05-14T09:34:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.192+0000 7f4554ff9700 1 -- 172.21.15.2:0/1681837896 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f45680a5d60 con 0x55cf2a991df0 2024-05-14T09:34:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.193+0000 7f4554ff9700 1 -- 172.21.15.2:0/1681837896 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45680adb20 con 0x55cf2a991df0 2024-05-14T09:34:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.193+0000 7f4581f0ab80 10 client.24985 fetch_fsmap learned FSMap version 26 2024-05-14T09:34:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.193+0000 7f4581f0ab80 1 -- 172.21.15.2:0/1681837896 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cf2ab1c3d0 con 0x55cf2a991df0 2024-05-14T09:34:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.193+0000 7f4554ff9700 1 -- 172.21.15.2:0/1681837896 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cf2aa3f1f0 con 0x55cf2a991df0 2024-05-14T09:34:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.193+0000 7f4554ff9700 1 -- 172.21.15.2:0/1681837896 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f456808fce0 con 0x55cf2a991df0 2024-05-14T09:34:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.193+0000 7f4581f0ab80 10 client.24985 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T09:34:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.193+0000 7f4581f0ab80 10 client.24985 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T09:34:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.193+0000 7f4581f0ab80 20 client.24985 populate_metadata read hostname 'smithi002' 2024-05-14T09:34:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.193+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/1681837896 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55cf2ab29100 0x55cf2ab2a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.193+0000 7f4581f0ab80 4 client.24985 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:34:41.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.193+0000 7f4581f0ab80 1 -- 172.21.15.2:0/1681837896 --> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf2aa39290 con 0x55cf2ab29100 2024-05-14T09:34:41.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.194+0000 7f4570a91700 1 --2- 172.21.15.2:0/1681837896 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55cf2ab29100 0x55cf2ab2a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.194+0000 7f4570a91700 1 --2- 172.21.15.2:0/1681837896 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55cf2ab29100 0x55cf2ab2a570 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=568ea2b782804eed server_cookie=fd1546308b14f141 in_seq=0 out_seq=0 2024-05-14T09:34:41.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.194+0000 7f4554ff9700 10 client.24985 ms_handle_connect on v2:172.21.15.26:6836/3301773578 2024-05-14T09:34:41.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.195+0000 7f4554ff9700 1 -- 172.21.15.2:0/1681837896 <== mds.1 v2:172.21.15.26:6836/3301773578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cf2aa39290 con 0x55cf2ab29100 2024-05-14T09:34:41.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.195+0000 7f4554ff9700 10 client.24985 handle_command_reply: tid=0 2024-05-14T09:34:41.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.195+0000 7f4581f0ab80 1 client.24985 shutdown 2024-05-14T09:34:41.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.195+0000 7f4581f0ab80 1 -- 172.21.15.2:0/1681837896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 msgr2=0x55cf2ab19140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:41.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.195+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/1681837896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab19140 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f45680a69e0 tx=0x7f45680047c0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.195+0000 7f4581f0ab80 1 -- 172.21.15.2:0/1681837896 shutdown_connections 2024-05-14T09:34:41.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.195+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/1681837896 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55cf2ab29100 0x55cf2ab2a570 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.195+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/1681837896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab14420 0x55cf2ab19bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.195+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/1681837896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab19140 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.195+0000 7f4554ff9700 0 client.24985 ms_handle_reset on v2:172.21.15.26:6836/3301773578 2024-05-14T09:34:41.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.195+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/1681837896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab19680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.195+0000 7f4581f0ab80 1 -- 172.21.15.2:0/1681837896 >> 172.21.15.2:0/1681837896 conn(0x55cf2a9d95b0 msgr2=0x55cf2ab0e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:41.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.195+0000 7f4581f0ab80 1 -- 172.21.15.2:0/1681837896 shutdown_connections 2024-05-14T09:34:41.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.196+0000 7f4581f0ab80 1 -- 172.21.15.2:0/1681837896 wait complete. 2024-05-14T09:34:41.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.196+0000 7f4581f0ab80 20 client.24985 trim_cache size 0 max 16384 2024-05-14T09:34:41.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.217+0000 7f4581f0ab80 1 Processor -- start 2024-05-14T09:34:41.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.217+0000 7f4581f0ab80 1 -- start start 2024-05-14T09:34:41.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.217+0000 7f4581f0ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2a991df0 0x55cf2ab0a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.217+0000 7f4581f0ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab25820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.217+0000 7f4581f0ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab14420 0x55cf2ab13d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.217+0000 7f4581f0ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cf2ab288b0 con 0x55cf2ab14420 2024-05-14T09:34:41.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.217+0000 7f4581f0ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cf2ab28f80 con 0x55cf2a991df0 2024-05-14T09:34:41.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.217+0000 7f4581f0ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cf2ab29650 con 0x55cf2ab0fc10 2024-05-14T09:34:41.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.217+0000 7f4571a93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab14420 0x55cf2ab13d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.217+0000 7f4571a93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab14420 0x55cf2ab13d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35674/0 (socket says 172.21.15.2:35674) 2024-05-14T09:34:41.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.217+0000 7f4571a93700 1 -- 172.21.15.2:0/3765107612 learned_addr learned my addr 172.21.15.2:0/3765107612 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:41.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.218+0000 7f4570a91700 1 --2- 172.21.15.2:0/3765107612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab25820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.218+0000 7f4571292700 1 --2- 172.21.15.2:0/3765107612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2a991df0 0x55cf2ab0a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.218+0000 7f4571a93700 1 -- 172.21.15.2:0/3765107612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2ab0fc10 msgr2=0x55cf2ab25820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:41.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.218+0000 7f4571a93700 1 --2- 172.21.15.2:0/3765107612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab25820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.218+0000 7f4571a93700 1 -- 172.21.15.2:0/3765107612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2a991df0 msgr2=0x55cf2ab0a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:41.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.218+0000 7f4571a93700 1 --2- 172.21.15.2:0/3765107612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2a991df0 0x55cf2ab0a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.218+0000 7f4571a93700 1 -- 172.21.15.2:0/3765107612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf2ab06a90 con 0x55cf2ab14420 2024-05-14T09:34:41.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.218+0000 7f4571a93700 1 --2- 172.21.15.2:0/3765107612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab14420 0x55cf2ab13d70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4564014a00 tx=0x7f4564012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f9f6002126ccb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:41.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.218+0000 7f4556ffd700 1 -- 172.21.15.2:0/3765107612 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456401b040 con 0x55cf2ab14420 2024-05-14T09:34:41.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.218+0000 7f4556ffd700 1 -- 172.21.15.2:0/3765107612 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f456401c040 con 0x55cf2ab14420 2024-05-14T09:34:41.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.218+0000 7f4556ffd700 1 -- 172.21.15.2:0/3765107612 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564015960 con 0x55cf2ab14420 2024-05-14T09:34:41.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.219+0000 7f4581f0ab80 1 -- 172.21.15.2:0/3765107612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab14420 msgr2=0x55cf2ab13d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:41.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.219+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/3765107612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab14420 0x55cf2ab13d70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4564014a00 tx=0x7f4564012000 comp rx=0 tx=0).stop 2024-05-14T09:34:41.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.219+0000 7f4581f0ab80 1 -- 172.21.15.2:0/3765107612 shutdown_connections 2024-05-14T09:34:41.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.219+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/3765107612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2a991df0 0x55cf2ab0a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.219+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/3765107612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab14420 0x55cf2ab13d70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.219+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/3765107612 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab25820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.219+0000 7f4581f0ab80 1 -- 172.21.15.2:0/3765107612 >> 172.21.15.2:0/3765107612 conn(0x55cf2a9d95b0 msgr2=0x55cf2ab0e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:41.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.220+0000 7f4581f0ab80 1 -- 172.21.15.2:0/3765107612 shutdown_connections 2024-05-14T09:34:41.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.220+0000 7f4581f0ab80 1 -- 172.21.15.2:0/3765107612 wait complete. 2024-05-14T09:34:41.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.220+0000 7f4581f0ab80 1 -- start start 2024-05-14T09:34:41.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.220+0000 7f4581f0ab80 1 Processor -- start 2024-05-14T09:34:41.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4581f0ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab14830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4581f0ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab14d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4581f0ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab14420 0x55cf2ab329e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4581f0ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cf2ab2a3f0 con 0x55cf2ab0fc10 2024-05-14T09:34:41.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4581f0ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cf2ab29d20 con 0x55cf2ab14420 2024-05-14T09:34:41.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4581f0ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cf2ab2aac0 con 0x55cf2a991df0 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4581f0ab80 10 client.0 _ll_register_callbacks cb 0x55cf2a9c98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4570a91700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab14d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4570a91700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab14d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35690/0 (socket says 172.21.15.2:35690) 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4570a91700 1 -- 172.21.15.2:0/28477685 learned_addr learned my addr 172.21.15.2:0/28477685 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4571a93700 1 --2- 172.21.15.2:0/28477685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab14420 0x55cf2ab329e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4571292700 1 --2- 172.21.15.2:0/28477685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab14830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4570a91700 1 -- 172.21.15.2:0/28477685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 msgr2=0x55cf2ab14830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4570a91700 1 --2- 172.21.15.2:0/28477685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab14830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4570a91700 1 -- 172.21.15.2:0/28477685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab14420 msgr2=0x55cf2ab329e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4570a91700 1 --2- 172.21.15.2:0/28477685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab14420 0x55cf2ab329e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4570a91700 1 -- 172.21.15.2:0/28477685 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf2ab32f20 con 0x55cf2ab0fc10 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4571292700 1 --2- 172.21.15.2:0/28477685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab14830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4570a91700 1 --2- 172.21.15.2:0/28477685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab14d70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f455c005010 tx=0x7f455c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f1a4202ad7906c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.221+0000 7f4554ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.222+0000 7f4554ff9700 1 -- 172.21.15.2:0/28477685 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c00b790 con 0x55cf2ab0fc10 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.222+0000 7f4554ff9700 1 -- 172.21.15.2:0/28477685 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f455c00b930 con 0x55cf2ab0fc10 2024-05-14T09:34:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.222+0000 7f4581f0ab80 1 -- 172.21.15.2:0/28477685 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f456401c040 con 0x55cf2ab0fc10 2024-05-14T09:34:41.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.222+0000 7f4554ff9700 1 -- 172.21.15.2:0/28477685 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f455c039050 con 0x55cf2ab0fc10 2024-05-14T09:34:41.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.222+0000 7f4581f0ab80 10 client.15222 fetch_fsmap learned FSMap version 26 2024-05-14T09:34:41.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.222+0000 7f4581f0ab80 1 -- 172.21.15.2:0/28477685 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cf2ab341e0 con 0x55cf2ab0fc10 2024-05-14T09:34:41.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.222+0000 7f4554ff9700 1 -- 172.21.15.2:0/28477685 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f456401c040 con 0x55cf2ab0fc10 2024-05-14T09:34:41.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.222+0000 7f4554ff9700 1 -- 172.21.15.2:0/28477685 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f455c029880 con 0x55cf2ab0fc10 2024-05-14T09:34:41.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.222+0000 7f4581f0ab80 10 client.15222 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T09:34:41.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.222+0000 7f4581f0ab80 10 client.15222 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T09:34:41.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.222+0000 7f4581f0ab80 20 client.15222 populate_metadata read hostname 'smithi002' 2024-05-14T09:34:41.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.222+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/28477685 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55cf2ab96490 0x55cf2ab97900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:41.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.222+0000 7f4581f0ab80 4 client.15222 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T09:34:41.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.222+0000 7f4581f0ab80 1 -- 172.21.15.2:0/28477685 --> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] -- command(tid 0: {"prefix": "status"}) -- 0x55cf2a8b1200 con 0x55cf2ab96490 2024-05-14T09:34:41.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.223+0000 7f4571292700 1 --2- 172.21.15.2:0/28477685 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55cf2ab96490 0x55cf2ab97900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:41.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.223+0000 7f4554ff9700 10 client.15222 ms_handle_connect on v2:172.21.15.26:6836/3301773578 2024-05-14T09:34:41.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.223+0000 7f4571292700 1 --2- 172.21.15.2:0/28477685 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55cf2ab96490 0x55cf2ab97900 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2158d678196e0c31 server_cookie=209ea9ba9eba731 in_seq=0 out_seq=0 2024-05-14T09:34:41.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.223+0000 7f4554ff9700 1 -- 172.21.15.2:0/28477685 <== mds.1 v2:172.21.15.26:6836/3301773578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55cf2a8b1200 con 0x55cf2ab96490 2024-05-14T09:34:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.223+0000 7f4554ff9700 10 client.15222 handle_command_reply: tid=0 2024-05-14T09:34:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.223+0000 7f4581f0ab80 1 client.15222 shutdown 2024-05-14T09:34:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.224+0000 7f4581f0ab80 1 -- 172.21.15.2:0/28477685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab0fc10 msgr2=0x55cf2ab14d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.224+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/28477685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab14d70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f455c005010 tx=0x7f455c0050b0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.224+0000 7f4581f0ab80 1 -- 172.21.15.2:0/28477685 shutdown_connections 2024-05-14T09:34:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.224+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/28477685 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x55cf2ab96490 0x55cf2ab97900 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.224+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/28477685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cf2ab14420 0x55cf2ab329e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.224+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/28477685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cf2a991df0 0x55cf2ab14830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.224+0000 7f4581f0ab80 1 --2- 172.21.15.2:0/28477685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf2ab0fc10 0x55cf2ab14d70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.224+0000 7f4554ff9700 0 client.15222 ms_handle_reset on v2:172.21.15.26:6836/3301773578 2024-05-14T09:34:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.224+0000 7f4581f0ab80 1 -- 172.21.15.2:0/28477685 >> 172.21.15.2:0/28477685 conn(0x55cf2a9d95b0 msgr2=0x55cf2ab0a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.224+0000 7f4581f0ab80 1 -- 172.21.15.2:0/28477685 shutdown_connections 2024-05-14T09:34:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.224+0000 7f4581f0ab80 1 -- 172.21.15.2:0/28477685 wait complete. 2024-05-14T09:34:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.224+0000 7f4581f0ab80 20 client.15222 trim_cache size 0 max 16384 2024-05-14T09:34:41.228 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:34:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "50d5aebe-11d3-11ef-bc98-c7b262605968", 2024-05-14T09:34:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 1, 2024-05-14T09:34:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24497, 2024-05-14T09:34:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T09:34:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T09:34:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T09:34:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 166.52588525799999, 2024-05-14T09:34:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 26, 2024-05-14T09:34:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 90, 2024-05-14T09:34:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 88, 2024-05-14T09:34:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 249.97664000500001 2024-05-14T09:34:41.228 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:34:41.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.225+0000 7f45734f6700 1 -- 172.21.15.2:0/4058501059 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f45580b2880 msgr2=0x7f45580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:41.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.225+0000 7f45734f6700 1 --2- 172.21.15.2:0/4058501059 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f45580b2880 0x7f45580b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f456400f3c0 tx=0x7f4564007950 comp rx=0 tx=0).stop 2024-05-14T09:34:41.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.225+0000 7f45734f6700 1 -- 172.21.15.2:0/4058501059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c1045e0 msgr2=0x7f456c19ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:41.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.225+0000 7f45734f6700 1 --2- 172.21.15.2:0/4058501059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c1045e0 0x7f456c19ada0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f456800c9b0 tx=0x7f456800ccc0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.225+0000 7f45734f6700 1 -- 172.21.15.2:0/4058501059 shutdown_connections 2024-05-14T09:34:41.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.225+0000 7f45734f6700 1 --2- 172.21.15.2:0/4058501059 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f45580b2880 0x7f45580b4d90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.225+0000 7f45734f6700 1 --2- 172.21.15.2:0/4058501059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f456c109860 0x7f456c19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.225+0000 7f45734f6700 1 --2- 172.21.15.2:0/4058501059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c1045e0 0x7f456c19ada0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.225+0000 7f45734f6700 1 --2- 172.21.15.2:0/4058501059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f456c108f10 0x7f456c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:41.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.225+0000 7f45734f6700 1 -- 172.21.15.2:0/4058501059 >> 172.21.15.2:0/4058501059 conn(0x7f456c0ffed0 msgr2=0x7f456c10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:41.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.225+0000 7f45734f6700 1 -- 172.21.15.2:0/4058501059 shutdown_connections 2024-05-14T09:34:41.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:41.225+0000 7f45734f6700 1 -- 172.21.15.2:0/4058501059 wait complete. 2024-05-14T09:34:41.914 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-14T09:34:42.167 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:42.622 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:42 smithi002 ceph-mon[31557]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:42.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:42 smithi029 ceph-mon[36249]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:42 smithi026 ceph-mon[36056]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:43.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.880+0000 7f1c2b1fd700 1 -- 172.21.15.2:0/2750009660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c241034c0 msgr2=0x7f1c241038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:43.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.880+0000 7f1c2b1fd700 1 --2- 172.21.15.2:0/2750009660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c241034c0 0x7f1c241038d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0099c0 tx=0x7f1c0c009cd0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.880+0000 7f1c2b1fd700 1 -- 172.21.15.2:0/2750009660 shutdown_connections 2024-05-14T09:34:43.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.880+0000 7f1c2b1fd700 1 --2- 172.21.15.2:0/2750009660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c24108740 0x7f1c241100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.880+0000 7f1c2b1fd700 1 --2- 172.21.15.2:0/2750009660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c241034c0 0x7f1c241038d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.880+0000 7f1c2b1fd700 1 --2- 172.21.15.2:0/2750009660 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c24107df0 0x7f1c24108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.880+0000 7f1c2b1fd700 1 -- 172.21.15.2:0/2750009660 >> 172.21.15.2:0/2750009660 conn(0x7f1c240fedf0 msgr2=0x7f1c24101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:43.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.880+0000 7f1c2b1fd700 1 -- 172.21.15.2:0/2750009660 shutdown_connections 2024-05-14T09:34:43.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.880+0000 7f1c2b1fd700 1 -- 172.21.15.2:0/2750009660 wait complete. 2024-05-14T09:34:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.881+0000 7f1c2b1fd700 1 Processor -- start 2024-05-14T09:34:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.881+0000 7f1c2b1fd700 1 -- start start 2024-05-14T09:34:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.881+0000 7f1c2b1fd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c241034c0 0x7f1c24199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.881+0000 7f1c2b1fd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c24107df0 0x7f1c2419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.882+0000 7f1c2b1fd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c24108740 0x7f1c2419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.882+0000 7f1c2b1fd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1c24113390 con 0x7f1c24107df0 2024-05-14T09:34:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.882+0000 7f1c2b1fd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c24113210 con 0x7f1c24108740 2024-05-14T09:34:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.882+0000 7f1c2b1fd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1c24113510 con 0x7f1c241034c0 2024-05-14T09:34:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.882+0000 7f1c23fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c24107df0 0x7f1c2419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.882+0000 7f1c28f99700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c241034c0 0x7f1c24199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.882+0000 7f1c23fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c24107df0 0x7f1c2419a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35700/0 (socket says 172.21.15.2:35700) 2024-05-14T09:34:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.882+0000 7f1c2979a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c24108740 0x7f1c2419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.882+0000 7f1c23fff700 1 -- 172.21.15.2:0/2397643843 learned_addr learned my addr 172.21.15.2:0/2397643843 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.882+0000 7f1c23fff700 1 -- 172.21.15.2:0/2397643843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c241034c0 msgr2=0x7f1c24199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.882+0000 7f1c23fff700 1 --2- 172.21.15.2:0/2397643843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c241034c0 0x7f1c24199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.882+0000 7f1c23fff700 1 -- 172.21.15.2:0/2397643843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c24108740 msgr2=0x7f1c2419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.882+0000 7f1c23fff700 1 --2- 172.21.15.2:0/2397643843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c24108740 0x7f1c2419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.882+0000 7f1c23fff700 1 -- 172.21.15.2:0/2397643843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2419ec50 con 0x7f1c24107df0 2024-05-14T09:34:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.882+0000 7f1c23fff700 1 --2- 172.21.15.2:0/2397643843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c24107df0 0x7f1c2419a320 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400ada0 tx=0x7f1c1400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:43.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.883+0000 7f1c21ffb700 1 -- 172.21.15.2:0/2397643843 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14017020 con 0x7f1c24107df0 2024-05-14T09:34:43.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.883+0000 7f1c21ffb700 1 -- 172.21.15.2:0/2397643843 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c1400cd80 con 0x7f1c24107df0 2024-05-14T09:34:43.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.883+0000 7f1c21ffb700 1 -- 172.21.15.2:0/2397643843 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14005480 con 0x7f1c24107df0 2024-05-14T09:34:43.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.883+0000 7f1c2b1fd700 1 -- 172.21.15.2:0/2397643843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2419ef40 con 0x7f1c24107df0 2024-05-14T09:34:43.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.883+0000 7f1c2b1fd700 1 -- 172.21.15.2:0/2397643843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c240759f0 con 0x7f1c24107df0 2024-05-14T09:34:43.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.886+0000 7f1c21ffb700 1 -- 172.21.15.2:0/2397643843 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1c14004ad0 con 0x7f1c24107df0 2024-05-14T09:34:43.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.886+0000 7f1c21ffb700 1 --2- 172.21.15.2:0/2397643843 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c100b2880 0x7f1c100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c39c11b80 1 Processor -- start 2024-05-14T09:34:43.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c28f99700 1 --2- 172.21.15.2:0/2397643843 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c100b2880 0x7f1c100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c21ffb700 1 -- 172.21.15.2:0/2397643843 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1c140e1d60 con 0x7f1c24107df0 2024-05-14T09:34:43.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c39c11b80 1 -- start start 2024-05-14T09:34:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c39c11b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700c47df0 0x564700dcb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c39c11b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dcbd30 0x564700dc5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c39c11b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dc5dc0 0x564700dc61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c39c11b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564700ce78f0 con 0x564700c47df0 2024-05-14T09:34:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c39c11b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564700c4cd50 con 0x564700dcbd30 2024-05-14T09:34:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c39c11b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564700ce38b0 con 0x564700dc5dc0 2024-05-14T09:34:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c28f99700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700c47df0 0x564700dcb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c28f99700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700c47df0 0x564700dcb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35714/0 (socket says 172.21.15.2:35714) 2024-05-14T09:34:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c28f99700 1 -- 172.21.15.2:0/3584512538 learned_addr learned my addr 172.21.15.2:0/3584512538 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c28f99700 1 --2- 172.21.15.2:0/2397643843 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c100b2880 0x7f1c100b4d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009990 tx=0x7f1c0c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c2979a700 1 --2- 172.21.15.2:0/3584512538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dc5dc0 0x564700dc61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.887+0000 7f1c23fff700 1 --2- 172.21.15.2:0/3584512538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dcbd30 0x564700dc5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.888+0000 7f1c28f99700 1 -- 172.21.15.2:0/3584512538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dc5dc0 msgr2=0x564700dc61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.888+0000 7f1c28f99700 1 --2- 172.21.15.2:0/3584512538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dc5dc0 0x564700dc61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.888+0000 7f1c28f99700 1 -- 172.21.15.2:0/3584512538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dcbd30 msgr2=0x564700dc5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.888+0000 7f1c28f99700 1 --2- 172.21.15.2:0/3584512538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dcbd30 0x564700dc5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.888+0000 7f1c28f99700 1 -- 172.21.15.2:0/3584512538 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564700c34880 con 0x564700c47df0 2024-05-14T09:34:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.888+0000 7f1c28f99700 1 --2- 172.21.15.2:0/3584512538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700c47df0 0x564700dcb7f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c015000 tx=0x7f1c0c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3773aa683b10c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.888+0000 7f1c1affd700 1 -- 172.21.15.2:0/3584512538 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c003dc0 con 0x564700c47df0 2024-05-14T09:34:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.888+0000 7f1c1affd700 1 -- 172.21.15.2:0/3584512538 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0c00bc40 con 0x564700c47df0 2024-05-14T09:34:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.888+0000 7f1c1affd700 1 -- 172.21.15.2:0/3584512538 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c00fbe0 con 0x564700c47df0 2024-05-14T09:34:43.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.889+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3584512538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700c47df0 msgr2=0x564700dcb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:43.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.889+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/3584512538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700c47df0 0x564700dcb7f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c015000 tx=0x7f1c0c01e000 comp rx=0 tx=0).stop 2024-05-14T09:34:43.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.889+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3584512538 shutdown_connections 2024-05-14T09:34:43.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.889+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/3584512538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dc5dc0 0x564700dc61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.889+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/3584512538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700c47df0 0x564700dcb7f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.889+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/3584512538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dcbd30 0x564700dc5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.889+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3584512538 >> 172.21.15.2:0/3584512538 conn(0x564700c8f5e0 msgr2=0x564700dc09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:43.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.889+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3584512538 shutdown_connections 2024-05-14T09:34:43.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.889+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3584512538 wait complete. 2024-05-14T09:34:43.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.889+0000 7f1c39c11b80 1 -- start start 2024-05-14T09:34:43.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.890+0000 7f1c39c11b80 1 Processor -- start 2024-05-14T09:34:43.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.890+0000 7f1c39c11b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700c47df0 0x564700dcf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.890+0000 7f1c39c11b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dc5dc0 0x564700dcfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.890+0000 7f1c39c11b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dcbd30 0x564700dcff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.890+0000 7f1c39c11b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564700dd2770 con 0x564700c47df0 2024-05-14T09:34:43.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.890+0000 7f1c39c11b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564700ca0e80 con 0x564700dc5dc0 2024-05-14T09:34:43.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.890+0000 7f1c39c11b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564700dd28f0 con 0x564700dcbd30 2024-05-14T09:34:43.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.890+0000 7f1c28f99700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700c47df0 0x564700dcf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.890+0000 7f1c39c11b80 10 client.0 _ll_register_callbacks cb 0x564700c7f900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:34:43.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.890+0000 7f1c28f99700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700c47df0 0x564700dcf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35726/0 (socket says 172.21.15.2:35726) 2024-05-14T09:34:43.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.890+0000 7f1c28f99700 1 -- 172.21.15.2:0/354775035 learned_addr learned my addr 172.21.15.2:0/354775035 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:43.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.890+0000 7f1c2979a700 1 --2- 172.21.15.2:0/354775035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dcbd30 0x564700dcff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.890+0000 7f1c23fff700 1 --2- 172.21.15.2:0/354775035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dc5dc0 0x564700dcfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.891+0000 7f1c28f99700 1 -- 172.21.15.2:0/354775035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dcbd30 msgr2=0x564700dcff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:43.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.891+0000 7f1c28f99700 1 --2- 172.21.15.2:0/354775035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dcbd30 0x564700dcff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.891+0000 7f1c28f99700 1 -- 172.21.15.2:0/354775035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dc5dc0 msgr2=0x564700dcfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:43.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.891+0000 7f1c28f99700 1 --2- 172.21.15.2:0/354775035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dc5dc0 0x564700dcfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.891+0000 7f1c28f99700 1 -- 172.21.15.2:0/354775035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564700c9e710 con 0x564700c47df0 2024-05-14T09:34:43.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.891+0000 7f1c28f99700 1 --2- 172.21.15.2:0/354775035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700c47df0 0x564700dcf500 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0099c0 tx=0x7f1c0c00bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a9e511115444dd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:43.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.891+0000 7f1c03fff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:34:43.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.891+0000 7f1c03fff700 1 -- 172.21.15.2:0/354775035 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c033030 con 0x564700c47df0 2024-05-14T09:34:43.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.891+0000 7f1c03fff700 1 -- 172.21.15.2:0/354775035 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1c0c02b050 con 0x564700c47df0 2024-05-14T09:34:43.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.891+0000 7f1c39c11b80 1 -- 172.21.15.2:0/354775035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c0c00bc40 con 0x564700c47df0 2024-05-14T09:34:43.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.891+0000 7f1c03fff700 1 -- 172.21.15.2:0/354775035 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c0c040050 con 0x564700c47df0 2024-05-14T09:34:43.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.891+0000 7f1c39c11b80 10 client.15234 fetch_fsmap learned FSMap version 26 2024-05-14T09:34:43.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.891+0000 7f1c39c11b80 1 -- 172.21.15.2:0/354775035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564700de1820 con 0x564700c47df0 2024-05-14T09:34:43.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.891+0000 7f1c03fff700 1 -- 172.21.15.2:0/354775035 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0c00bc40 con 0x564700c47df0 2024-05-14T09:34:43.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.892+0000 7f1c03fff700 1 -- 172.21.15.2:0/354775035 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f1c0c022070 con 0x564700c47df0 2024-05-14T09:34:43.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.892+0000 7f1c39c11b80 10 client.15234 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T09:34:43.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.892+0000 7f1c39c11b80 10 client.15234 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T09:34:43.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.892+0000 7f1c39c11b80 20 client.15234 populate_metadata read hostname 'smithi002' 2024-05-14T09:34:43.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.892+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/354775035 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564700de2b70 0x564700de4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.892+0000 7f1c39c11b80 4 client.15234 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:34:43.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.892+0000 7f1c39c11b80 1 -- 172.21.15.2:0/354775035 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564700cf52d0 con 0x564700de2b70 2024-05-14T09:34:43.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.892+0000 7f1c23fff700 1 --2- 172.21.15.2:0/354775035 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564700de2b70 0x564700de4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.893+0000 7f1c23fff700 1 --2- 172.21.15.2:0/354775035 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564700de2b70 0x564700de4ce0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ffe0dd30085052c server_cookie=2f781d4b2fb68cab in_seq=0 out_seq=0 2024-05-14T09:34:43.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.893+0000 7f1c03fff700 10 client.15234 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:34:43.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.893+0000 7f1c03fff700 1 -- 172.21.15.2:0/354775035 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564700cf52d0 con 0x564700de2b70 2024-05-14T09:34:43.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.893+0000 7f1c03fff700 10 client.15234 handle_command_reply: tid=0 2024-05-14T09:34:43.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.893+0000 7f1c39c11b80 1 client.15234 shutdown 2024-05-14T09:34:43.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.893+0000 7f1c39c11b80 1 -- 172.21.15.2:0/354775035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700c47df0 msgr2=0x564700dcf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:43.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.893+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/354775035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700c47df0 0x564700dcf500 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0099c0 tx=0x7f1c0c00bb80 comp rx=0 tx=0).stop 2024-05-14T09:34:43.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.894+0000 7f1c39c11b80 1 -- 172.21.15.2:0/354775035 shutdown_connections 2024-05-14T09:34:43.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.894+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/354775035 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564700de2b70 0x564700de4ce0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.894+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/354775035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dcbd30 0x564700dcff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.894+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/354775035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700c47df0 0x564700dcf500 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.894+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/354775035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dc5dc0 0x564700dcfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.894+0000 7f1c39c11b80 1 -- 172.21.15.2:0/354775035 >> 172.21.15.2:0/354775035 conn(0x564700c8f5e0 msgr2=0x564700dc4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:43.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.894+0000 7f1c03fff700 0 client.15234 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:34:43.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.894+0000 7f1c39c11b80 1 -- 172.21.15.2:0/354775035 shutdown_connections 2024-05-14T09:34:43.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.894+0000 7f1c39c11b80 1 -- 172.21.15.2:0/354775035 wait complete. 2024-05-14T09:34:43.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.894+0000 7f1c39c11b80 20 client.15234 trim_cache size 0 max 16384 2024-05-14T09:34:43.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.915+0000 7f1c39c11b80 1 Processor -- start 2024-05-14T09:34:43.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.915+0000 7f1c39c11b80 1 -- start start 2024-05-14T09:34:43.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.915+0000 7f1c39c11b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700c47df0 0x564700dc5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.915+0000 7f1c39c11b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700dc5dc0 0x564700dcd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.915+0000 7f1c39c11b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dcbd30 0x564700dcfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.915+0000 7f1c39c11b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564700de3020 con 0x564700dc5dc0 2024-05-14T09:34:43.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.915+0000 7f1c39c11b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564700de36f0 con 0x564700c47df0 2024-05-14T09:34:43.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.915+0000 7f1c39c11b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564700de3dc0 con 0x564700dcbd30 2024-05-14T09:34:43.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.915+0000 7f1c23fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700dc5dc0 0x564700dcd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.915+0000 7f1c23fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700dc5dc0 0x564700dcd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35728/0 (socket says 172.21.15.2:35728) 2024-05-14T09:34:43.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.915+0000 7f1c23fff700 1 -- 172.21.15.2:0/3231641861 learned_addr learned my addr 172.21.15.2:0/3231641861 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:43.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.916+0000 7f1c28f99700 1 --2- 172.21.15.2:0/3231641861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700c47df0 0x564700dc5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.916+0000 7f1c2979a700 1 --2- 172.21.15.2:0/3231641861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dcbd30 0x564700dcfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.916+0000 7f1c23fff700 1 -- 172.21.15.2:0/3231641861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dcbd30 msgr2=0x564700dcfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:43.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.916+0000 7f1c23fff700 1 --2- 172.21.15.2:0/3231641861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dcbd30 0x564700dcfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.916+0000 7f1c23fff700 1 -- 172.21.15.2:0/3231641861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700c47df0 msgr2=0x564700dc5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:43.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.916+0000 7f1c23fff700 1 --2- 172.21.15.2:0/3231641861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700c47df0 0x564700dc5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.916+0000 7f1c23fff700 1 -- 172.21.15.2:0/3231641861 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564700dd2300 con 0x564700dc5dc0 2024-05-14T09:34:43.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.916+0000 7f1c2979a700 1 --2- 172.21.15.2:0/3231641861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dcbd30 0x564700dcfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:34:43.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.916+0000 7f1c23fff700 1 --2- 172.21.15.2:0/3231641861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700dc5dc0 0x564700dcd150 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1c140df7b0 tx=0x7f1c140a4fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4d78df3d41cc6a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:43.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.916+0000 7f1c1a7fc700 1 -- 172.21.15.2:0/3231641861 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c140df550 con 0x564700dc5dc0 2024-05-14T09:34:43.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.916+0000 7f1c1a7fc700 1 -- 172.21.15.2:0/3231641861 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c1400a0a0 con 0x564700dc5dc0 2024-05-14T09:34:43.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.916+0000 7f1c1a7fc700 1 -- 172.21.15.2:0/3231641861 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c140a4960 con 0x564700dc5dc0 2024-05-14T09:34:43.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.917+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3231641861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700dc5dc0 msgr2=0x564700dcd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:43.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.917+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/3231641861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700dc5dc0 0x564700dcd150 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1c140df7b0 tx=0x7f1c140a4fe0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.917+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3231641861 shutdown_connections 2024-05-14T09:34:43.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.917+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/3231641861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700c47df0 0x564700dc5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.917+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/3231641861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700dcbd30 0x564700dcfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.917+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/3231641861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700dc5dc0 0x564700dcd150 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.917+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3231641861 >> 172.21.15.2:0/3231641861 conn(0x564700c8f5e0 msgr2=0x564700dca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:43.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.917+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3231641861 shutdown_connections 2024-05-14T09:34:43.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.917+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3231641861 wait complete. 2024-05-14T09:34:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.918+0000 7f1c39c11b80 1 -- start start 2024-05-14T09:34:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.918+0000 7f1c39c11b80 1 Processor -- start 2024-05-14T09:34:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.918+0000 7f1c39c11b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700c47df0 0x564700dcc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.918+0000 7f1c39c11b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dc5dc0 0x564700dcc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.918+0000 7f1c39c11b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700dcbd30 0x564700dccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.918+0000 7f1c39c11b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564700de4b60 con 0x564700dcbd30 2024-05-14T09:34:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.918+0000 7f1c39c11b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564700de4490 con 0x564700dc5dc0 2024-05-14T09:34:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.918+0000 7f1c39c11b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564700de5230 con 0x564700c47df0 2024-05-14T09:34:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.918+0000 7f1c39c11b80 10 client.0 _ll_register_callbacks cb 0x564700c7f900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:34:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.918+0000 7f1c2979a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700dcbd30 0x564700dccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.918+0000 7f1c28f99700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700c47df0 0x564700dcc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.918+0000 7f1c23fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dc5dc0 0x564700dcc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.918+0000 7f1c28f99700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700c47df0 0x564700dcc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:34246/0 (socket says 172.21.15.2:34246) 2024-05-14T09:34:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.918+0000 7f1c2979a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700dcbd30 0x564700dccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35736/0 (socket says 172.21.15.2:35736) 2024-05-14T09:34:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.918+0000 7f1c28f99700 1 -- 172.21.15.2:0/3387527673 learned_addr learned my addr 172.21.15.2:0/3387527673 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.919+0000 7f1c23fff700 1 -- 172.21.15.2:0/3387527673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700c47df0 msgr2=0x564700dcc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.919+0000 7f1c23fff700 1 --2- 172.21.15.2:0/3387527673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700c47df0 0x564700dcc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.919+0000 7f1c23fff700 1 -- 172.21.15.2:0/3387527673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700dcbd30 msgr2=0x564700dccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:34:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.919+0000 7f1c23fff700 1 --2- 172.21.15.2:0/3387527673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700dcbd30 0x564700dccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.919+0000 7f1c23fff700 1 -- 172.21.15.2:0/3387527673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564700ded250 con 0x564700dc5dc0 2024-05-14T09:34:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.919+0000 7f1c23fff700 1 --2- 172.21.15.2:0/3387527673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dc5dc0 0x564700dcc680 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1c140060b0 tx=0x7f1c14004e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecbbf99b1f143cc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.919+0000 7f1c03fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:34:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.919+0000 7f1c03fff700 1 -- 172.21.15.2:0/3387527673 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c140a9250 con 0x564700dc5dc0 2024-05-14T09:34:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.919+0000 7f1c03fff700 1 -- 172.21.15.2:0/3387527673 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1c140a93f0 con 0x564700dc5dc0 2024-05-14T09:34:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.919+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3387527673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c1400a0a0 con 0x564700dc5dc0 2024-05-14T09:34:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.919+0000 7f1c03fff700 1 -- 172.21.15.2:0/3387527673 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c140a8250 con 0x564700dc5dc0 2024-05-14T09:34:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.919+0000 7f1c39c11b80 10 client.24992 fetch_fsmap learned FSMap version 26 2024-05-14T09:34:43.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.919+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3387527673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564700dee4b0 con 0x564700dc5dc0 2024-05-14T09:34:43.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.920+0000 7f1c03fff700 1 -- 172.21.15.2:0/3387527673 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c1400a0a0 con 0x564700dc5dc0 2024-05-14T09:34:43.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.920+0000 7f1c03fff700 1 -- 172.21.15.2:0/3387527673 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f1c14012070 con 0x564700dc5dc0 2024-05-14T09:34:43.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.920+0000 7f1c39c11b80 10 client.24992 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T09:34:43.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.920+0000 7f1c39c11b80 10 client.24992 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T09:34:43.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.920+0000 7f1c39c11b80 20 client.24992 populate_metadata read hostname 'smithi002' 2024-05-14T09:34:43.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.920+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/3387527673 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564700e502b0 0x564700e51720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:43.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.920+0000 7f1c39c11b80 4 client.24992 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T09:34:43.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.920+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3387527673 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "status"}) -- 0x564700cef370 con 0x564700e502b0 2024-05-14T09:34:43.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.920+0000 7f1c28f99700 1 --2- 172.21.15.2:0/3387527673 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564700e502b0 0x564700e51720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:43.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.920+0000 7f1c03fff700 10 client.24992 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:34:43.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.920+0000 7f1c28f99700 1 --2- 172.21.15.2:0/3387527673 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564700e502b0 0x564700e51720 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e2ab9ffb58c4f7 server_cookie=c5b21932c8637b5f in_seq=0 out_seq=0 2024-05-14T09:34:43.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.921+0000 7f1c03fff700 1 -- 172.21.15.2:0/3387527673 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564700cef370 con 0x564700e502b0 2024-05-14T09:34:43.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.921+0000 7f1c03fff700 10 client.24992 handle_command_reply: tid=0 2024-05-14T09:34:43.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.921+0000 7f1c39c11b80 1 client.24992 shutdown 2024-05-14T09:34:43.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.921+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3387527673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dc5dc0 msgr2=0x564700dcc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:43.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.921+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/3387527673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dc5dc0 0x564700dcc680 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1c140060b0 tx=0x7f1c14004e40 comp rx=0 tx=0).stop 2024-05-14T09:34:43.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.921+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3387527673 shutdown_connections 2024-05-14T09:34:43.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.921+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/3387527673 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564700e502b0 0x564700e51720 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.921+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/3387527673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564700dcbd30 0x564700dccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.921+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/3387527673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564700c47df0 0x564700dcc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.921+0000 7f1c03fff700 0 client.24992 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:34:43.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.921+0000 7f1c39c11b80 1 --2- 172.21.15.2:0/3387527673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564700dc5dc0 0x564700dcc680 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.921+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3387527673 >> 172.21.15.2:0/3387527673 conn(0x564700c8f5e0 msgr2=0x564700dc0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:43.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.921+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3387527673 shutdown_connections 2024-05-14T09:34:43.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.921+0000 7f1c39c11b80 1 -- 172.21.15.2:0/3387527673 wait complete. 2024-05-14T09:34:43.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.921+0000 7f1c39c11b80 20 client.24992 trim_cache size 0 max 16384 2024-05-14T09:34:43.928 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:34:43.928 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "50d5aebe-11d3-11ef-bc98-c7b262605968", 2024-05-14T09:34:43.928 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2024-05-14T09:34:43.928 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24524, 2024-05-14T09:34:43.928 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T09:34:43.928 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T09:34:43.928 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T09:34:43.928 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 225.13653340400001, 2024-05-14T09:34:43.928 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 26, 2024-05-14T09:34:43.928 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 90, 2024-05-14T09:34:43.928 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 88, 2024-05-14T09:34:43.929 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 240.226952592 2024-05-14T09:34:43.929 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:34:43.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.922+0000 7f1c2b1fd700 1 -- 172.21.15.2:0/2397643843 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c100b2880 msgr2=0x7f1c100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:43.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.922+0000 7f1c2b1fd700 1 --2- 172.21.15.2:0/2397643843 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c100b2880 0x7f1c100b4d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009990 tx=0x7f1c0c000bc0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.922+0000 7f1c2b1fd700 1 -- 172.21.15.2:0/2397643843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c24107df0 msgr2=0x7f1c2419a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:43.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.922+0000 7f1c2b1fd700 1 --2- 172.21.15.2:0/2397643843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c24107df0 0x7f1c2419a320 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400ada0 tx=0x7f1c1400c330 comp rx=0 tx=0).stop 2024-05-14T09:34:43.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.922+0000 7f1c2b1fd700 1 -- 172.21.15.2:0/2397643843 shutdown_connections 2024-05-14T09:34:43.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.922+0000 7f1c2b1fd700 1 --2- 172.21.15.2:0/2397643843 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c100b2880 0x7f1c100b4d90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.922+0000 7f1c2b1fd700 1 --2- 172.21.15.2:0/2397643843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c24108740 0x7f1c2419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.922+0000 7f1c2b1fd700 1 --2- 172.21.15.2:0/2397643843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c241034c0 0x7f1c24199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.922+0000 7f1c2b1fd700 1 --2- 172.21.15.2:0/2397643843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c24107df0 0x7f1c2419a320 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:43.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.922+0000 7f1c2b1fd700 1 -- 172.21.15.2:0/2397643843 >> 172.21.15.2:0/2397643843 conn(0x7f1c240fedf0 msgr2=0x7f1c24101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:43.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.922+0000 7f1c2b1fd700 1 -- 172.21.15.2:0/2397643843 shutdown_connections 2024-05-14T09:34:43.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:43.922+0000 7f1c2b1fd700 1 -- 172.21.15.2:0/2397643843 wait complete. 2024-05-14T09:34:44.593 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:44 smithi002 ceph-mon[31557]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:44.622 INFO:teuthology.run_tasks:Running task sequential... 2024-05-14T09:34:44.636 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-14T09:34:44.636 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-14T09:34:44.637 INFO:teuthology.task.print:Enabling mgr modules 2024-05-14T09:34:44.637 INFO:teuthology.run_tasks:Running task kclient... 2024-05-14T09:34:44.659 INFO:tasks.kclient:Mounting kernel clients... 2024-05-14T09:34:44.659 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-14T09:34:44.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:34:44.659 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-05-14T09:34:44.677 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-14T09:34:44.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:34:44.678 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T09:34:44.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:44 smithi029 ceph-mon[36249]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:44.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:44 smithi026 ceph-mon[36056]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:44.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:34:44.746 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2024-05-14T09:34:44.851 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-14T09:34:44.851 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-14T09:34:44.851 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T09:34:44.851 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-14T09:34:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:46.501 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:46 smithi002 ceph-mon[31557]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:46 smithi029 ceph-mon[36249]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:46 smithi026 ceph-mon[36056]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:46.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.757+0000 7fc62908d700 1 -- 172.21.15.2:0/4247399706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc624107df0 msgr2=0x7fc624108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:46.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.757+0000 7fc62908d700 1 --2- 172.21.15.2:0/4247399706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc624107df0 0x7fc624108200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc6180099c0 tx=0x7fc618009cd0 comp rx=0 tx=0).stop 2024-05-14T09:34:46.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.757+0000 7fc62908d700 1 -- 172.21.15.2:0/4247399706 shutdown_connections 2024-05-14T09:34:46.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.757+0000 7fc62908d700 1 --2- 172.21.15.2:0/4247399706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6241034c0 0x7fc6241038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:46.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.757+0000 7fc62908d700 1 --2- 172.21.15.2:0/4247399706 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc624108740 0x7fc6241100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:46.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.757+0000 7fc62908d700 1 --2- 172.21.15.2:0/4247399706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc624107df0 0x7fc624108200 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:46.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.757+0000 7fc62908d700 1 -- 172.21.15.2:0/4247399706 >> 172.21.15.2:0/4247399706 conn(0x7fc6240fedc0 msgr2=0x7fc6241011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:46.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.757+0000 7fc62908d700 1 -- 172.21.15.2:0/4247399706 shutdown_connections 2024-05-14T09:34:46.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.757+0000 7fc62908d700 1 -- 172.21.15.2:0/4247399706 wait complete. 2024-05-14T09:34:46.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.758+0000 7fc62908d700 1 Processor -- start 2024-05-14T09:34:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62908d700 1 -- start start 2024-05-14T09:34:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62908d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6241034c0 0x7fc6240ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62908d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc624107df0 0x7fc6240ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62908d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc624108740 0x7fc6240ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62908d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc624113350 con 0x7fc624107df0 2024-05-14T09:34:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62908d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc6241131d0 con 0x7fc624108740 2024-05-14T09:34:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62908d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc6241134d0 con 0x7fc6241034c0 2024-05-14T09:34:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc624107df0 0x7fc6240ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc624107df0 0x7fc6240ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35766/0 (socket says 172.21.15.2:35766) 2024-05-14T09:34:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62259c700 1 -- 172.21.15.2:0/19627089 learned_addr learned my addr 172.21.15.2:0/19627089 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62359e700 1 --2- 172.21.15.2:0/19627089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc624108740 0x7fc6240ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc622d9d700 1 --2- 172.21.15.2:0/19627089 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6241034c0 0x7fc6240ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62259c700 1 -- 172.21.15.2:0/19627089 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6241034c0 msgr2=0x7fc6240ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62259c700 1 --2- 172.21.15.2:0/19627089 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6241034c0 0x7fc6240ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62259c700 1 -- 172.21.15.2:0/19627089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc624108740 msgr2=0x7fc6240ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62259c700 1 --2- 172.21.15.2:0/19627089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc624108740 0x7fc6240ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.759+0000 7fc62259c700 1 -- 172.21.15.2:0/19627089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6241a91d0 con 0x7fc624107df0 2024-05-14T09:34:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.760+0000 7fc62259c700 1 --2- 172.21.15.2:0/19627089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc624107df0 0x7fc6240ff7e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc618006010 tx=0x7fc6180055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.760+0000 7fc60bfff700 1 -- 172.21.15.2:0/19627089 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618026070 con 0x7fc624107df0 2024-05-14T09:34:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.760+0000 7fc62908d700 1 -- 172.21.15.2:0/19627089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6241a9460 con 0x7fc624107df0 2024-05-14T09:34:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.760+0000 7fc60bfff700 1 -- 172.21.15.2:0/19627089 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc61800f460 con 0x7fc624107df0 2024-05-14T09:34:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.760+0000 7fc62908d700 1 -- 172.21.15.2:0/19627089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6241a9910 con 0x7fc624107df0 2024-05-14T09:34:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.760+0000 7fc60bfff700 1 -- 172.21.15.2:0/19627089 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618009e60 con 0x7fc624107df0 2024-05-14T09:34:46.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.761+0000 7fc62908d700 1 -- 172.21.15.2:0/19627089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc610005330 con 0x7fc624107df0 2024-05-14T09:34:46.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.763+0000 7fc60bfff700 1 -- 172.21.15.2:0/19627089 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc618029070 con 0x7fc624107df0 2024-05-14T09:34:46.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.763+0000 7fc60bfff700 1 --2- 172.21.15.2:0/19627089 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc60c0b2880 0x7fc60c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:46.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.764+0000 7fc622d9d700 1 --2- 172.21.15.2:0/19627089 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc60c0b2880 0x7fc60c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:46.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.764+0000 7fc60bfff700 1 -- 172.21.15.2:0/19627089 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fc6180e2e60 con 0x7fc624107df0 2024-05-14T09:34:46.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.764+0000 7fc622d9d700 1 --2- 172.21.15.2:0/19627089 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc60c0b2880 0x7fc60c0b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc61400f540 tx=0x7fc61400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:46.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:46.769+0000 7fc60bfff700 1 -- 172.21.15.2:0/19627089 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc618013070 con 0x7fc624107df0 2024-05-14T09:34:47.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47.029+0000 7fc62908d700 1 -- 172.21.15.2:0/19627089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc610005e30 con 0x7fc624107df0 2024-05-14T09:34:47.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47.030+0000 7fc60bfff700 1 -- 172.21.15.2:0/19627089 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7fc6180aa7f0 con 0x7fc624107df0 2024-05-14T09:34:47.031 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-14T09:34:47.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47.032+0000 7fc62908d700 1 -- 172.21.15.2:0/19627089 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc60c0b2880 msgr2=0x7fc60c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:47.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47.032+0000 7fc62908d700 1 --2- 172.21.15.2:0/19627089 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc60c0b2880 0x7fc60c0b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc61400f540 tx=0x7fc61400b040 comp rx=0 tx=0).stop 2024-05-14T09:34:47.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47.032+0000 7fc62908d700 1 -- 172.21.15.2:0/19627089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc624107df0 msgr2=0x7fc6240ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:47.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47.032+0000 7fc62908d700 1 --2- 172.21.15.2:0/19627089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc624107df0 0x7fc6240ff7e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc618006010 tx=0x7fc6180055f0 comp rx=0 tx=0).stop 2024-05-14T09:34:47.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47.032+0000 7fc62908d700 1 -- 172.21.15.2:0/19627089 shutdown_connections 2024-05-14T09:34:47.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47.032+0000 7fc62908d700 1 --2- 172.21.15.2:0/19627089 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc60c0b2880 0x7fc60c0b4d90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:47.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47.032+0000 7fc62908d700 1 --2- 172.21.15.2:0/19627089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc624108740 0x7fc6240ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:47.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47.032+0000 7fc62908d700 1 --2- 172.21.15.2:0/19627089 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6241034c0 0x7fc6240ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:47.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47.032+0000 7fc62908d700 1 --2- 172.21.15.2:0/19627089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc624107df0 0x7fc6240ff7e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:47.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47.032+0000 7fc62908d700 1 -- 172.21.15.2:0/19627089 >> 172.21.15.2:0/19627089 conn(0x7fc6240fedc0 msgr2=0x7fc62406b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:47.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47.032+0000 7fc62908d700 1 -- 172.21.15.2:0/19627089 shutdown_connections 2024-05-14T09:34:47.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47.032+0000 7fc62908d700 1 -- 172.21.15.2:0/19627089 wait complete. 2024-05-14T09:34:47.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:47 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/19627089' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T09:34:47.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:47 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/19627089' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T09:34:47.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:47 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/19627089' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T09:34:47.755 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T09:34:47.755 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T09:34:47.755 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi002.front.sepia.ceph.com 2024-05-14T09:34:47.755 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T09:34:47.755 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T09:34:47.755 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T09:34:47.755 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T09:34:47.755 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T09:34:47.755 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T09:34:47.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:34:47.756 DEBUG:teuthology.orchestra.run.smithi002:> ip addr 2024-05-14T09:34:47.775 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T09:34:47.775 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T09:34:47.776 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T09:34:47.776 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-05-14T09:34:47.776 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2024-05-14T09:34:47.776 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-05-14T09:34:47.776 INFO:teuthology.orchestra.run.smithi002.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T09:34:47.776 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:02 brd ff:ff:ff:ff:ff:ff 2024-05-14T09:34:47.776 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f0 2024-05-14T09:34:47.776 INFO:teuthology.orchestra.run.smithi002.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T09:34:47.776 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:03 brd ff:ff:ff:ff:ff:ff 2024-05-14T09:34:47.776 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f1 2024-05-14T09:34:47.776 INFO:teuthology.orchestra.run.smithi002.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T09:34:47.776 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:20 brd ff:ff:ff:ff:ff:ff 2024-05-14T09:34:47.776 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f0 2024-05-14T09:34:47.776 INFO:teuthology.orchestra.run.smithi002.stdout: inet 172.21.15.2/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-14T09:34:47.776 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft 41926sec preferred_lft 41926sec 2024-05-14T09:34:47.777 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::ec4:7aff:febd:1920/64 scope link noprefixroute 2024-05-14T09:34:47.777 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-05-14T09:34:47.777 INFO:teuthology.orchestra.run.smithi002.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T09:34:47.777 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:21 brd ff:ff:ff:ff:ff:ff 2024-05-14T09:34:47.777 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f1 2024-05-14T09:34:47.777 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-14T09:34:47.777 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T09:34:47.777 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-05-14T09:34:47.777 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add name ceph-brx type bridge 2024-05-14T09:34:47.777 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr flush dev ceph-brx 2024-05-14T09:34:47.778 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set ceph-brx up 2024-05-14T09:34:47.778 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-14T09:34:47.778 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-05-14T09:34:47.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T09:34:47.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:34:47.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:34:47.929 DEBUG:teuthology.orchestra.run.smithi002:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T09:34:47.955 INFO:teuthology.orchestra.run.smithi002.stdout:1 2024-05-14T09:34:47.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:34:47.957 DEBUG:teuthology.orchestra.run.smithi002:> route 2024-05-14T09:34:48.013 INFO:teuthology.orchestra.run.smithi002.stdout:Kernel IP routing table 2024-05-14T09:34:48.013 INFO:teuthology.orchestra.run.smithi002.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T09:34:48.013 INFO:teuthology.orchestra.run.smithi002.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-14T09:34:48.014 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-14T09:34:48.014 INFO:teuthology.orchestra.run.smithi002.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T09:34:48.014 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T09:34:48.014 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-05-14T09:34:48.014 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-14T09:34:48.014 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-14T09:34:48.014 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-14T09:34:48.014 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-05-14T09:34:48.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T09:34:48.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:34:48.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:34:48.144 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-05-14T09:34:48.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:34:48.160 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list-id 2024-05-14T09:34:48.216 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T09:34:48.216 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-05-14T09:34:48.216 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T09:34:48.216 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-14T09:34:48.216 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-05-14T09:34:48.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T09:34:48.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:34:48.337 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-14T09:34:48.337 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T09:34:48.337 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-05-14T09:34:48.337 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-14T09:34:48.338 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-14T09:34:48.338 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-14T09:34:48.338 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-14T09:34:48.338 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-14T09:34:48.338 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-05-14T09:34:48.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T09:34:48.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:34:48.457 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T09:34:48.458 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-05-14T09:34:48.458 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set brx.0 up 2024-05-14T09:34:48.458 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set dev brx.0 master ceph-brx 2024-05-14T09:34:48.458 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-05-14T09:34:48.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T09:34:48.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:34:48.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T09:34:48.540 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T09:34:48.558 INFO:teuthology.orchestra.run.smithi002.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T09:34:48.558 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T09:34:48.558 DEBUG:teuthology.orchestra.run.smithi002:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T09:34:48.615 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-14T09:34:48.615 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-14T09:34:48.615 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 2024-05-14T09:34:48.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:34:48.615 DEBUG:teuthology.orchestra.run.smithi002:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-05-14T09:34:48.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:48 smithi002 ceph-mon[31557]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:34:48.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:48 smithi029 ceph-mon[36249]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:34:48.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:48 smithi026 ceph-mon[36056]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T09:34:48.710 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-05-14T09:34:48.714 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-05-14T09:34:48.714 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2024-05-14T09:34:48.840 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-05-14T09:34:48.840 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-05-14T09:34:48.840 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2024-05-14T09:34:48.840 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:6789,172.21.15.26:6789,172.21.15.29:6789" 2024-05-14T09:34:48.840 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.2:6789,172.21.15.26:6789,172.21.15.29:6789:/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 2024-05-14T09:34:48.840 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,key=0,fsid=50d5aebe-11d3-11ef-bc98-c7b262605968" will pass to kernel 2024-05-14T09:34:48.840 INFO:teuthology.orchestra.run.smithi002.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-14T09:34:48.841 INFO:teuthology.orchestra.run.smithi002.stdout: You just mounted a file system that supports labels which does not 2024-05-14T09:34:48.841 INFO:teuthology.orchestra.run.smithi002.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-14T09:34:48.841 INFO:teuthology.orchestra.run.smithi002.stdout: applications will generate AVC messages and not be allowed access to 2024-05-14T09:34:48.841 INFO:teuthology.orchestra.run.smithi002.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-14T09:34:48.843 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-14T09:34:48.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T09:34:48.843 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T09:34:48.914 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-14T09:34:49.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T09:34:49.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:34:49.015 INFO:teuthology.orchestra.run.smithi002.stdout:client25001 2024-05-14T09:34:49.015 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:34:49.015 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T09:34:49.033 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-14T09:34:49.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T09:34:49.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:34:49.131 INFO:teuthology.orchestra.run.smithi002.stdout:client25001 2024-05-14T09:34:49.131 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-14T09:34:49.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T09:34:49.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:34:49.190 INFO:teuthology.orchestra.run.smithi002.stdout:client25001 2024-05-14T09:34:49.190 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968.client25001/status') 2024-05-14T09:34:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T09:34:49.251 INFO:teuthology.orchestra.run.smithi002.stdout:instance: client.25001 (3)172.21.15.2:0/676989162 2024-05-14T09:34:49.251 INFO:teuthology.orchestra.run.smithi002.stdout:blocklisted: false 2024-05-14T09:34:49.252 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records in 2024-05-14T09:34:49.252 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records out 2024-05-14T09:34:49.252 INFO:teuthology.orchestra.run.smithi002.stderr:69 bytes copied, 2.4276e-05 s, 2.8 MB/s 2024-05-14T09:34:49.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T09:34:49.259 INFO:teuthology.run_tasks:Running task exec... 2024-05-14T09:34:49.270 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T09:34:49.270 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T09:34:49.270 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-14T09:34:49.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.387+0000 7fc21dcd9640 1 -- 172.21.15.2:0/4216864113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc218103d90 msgr2=0x7fc2181041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:49.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.387+0000 7fc21dcd9640 1 --2- 172.21.15.2:0/4216864113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc218103d90 0x7fc2181041a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc200009900 tx=0x7fc20002a410 comp rx=0 tx=0).stop 2024-05-14T09:34:49.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.387+0000 7fc21dcd9640 1 -- 172.21.15.2:0/4216864113 shutdown_connections 2024-05-14T09:34:49.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.387+0000 7fc21dcd9640 1 --2- 172.21.15.2:0/4216864113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc218108a90 0x7fc21810aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:49.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.387+0000 7fc21dcd9640 1 --2- 172.21.15.2:0/4216864113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc218103d90 0x7fc2181041a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:49.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.387+0000 7fc21dcd9640 1 --2- 172.21.15.2:0/4216864113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc218102b50 0x7fc218103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:49.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.387+0000 7fc21dcd9640 1 -- 172.21.15.2:0/4216864113 >> 172.21.15.2:0/4216864113 conn(0x7fc2180fdd40 msgr2=0x7fc2181001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:49.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.387+0000 7fc21dcd9640 1 -- 172.21.15.2:0/4216864113 shutdown_connections 2024-05-14T09:34:49.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.387+0000 7fc21dcd9640 1 -- 172.21.15.2:0/4216864113 wait complete. 2024-05-14T09:34:49.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.388+0000 7fc21dcd9640 1 Processor -- start 2024-05-14T09:34:49.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.388+0000 7fc21dcd9640 1 -- start start 2024-05-14T09:34:49.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.388+0000 7fc21dcd9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc218102b50 0x7fc21806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:49.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.388+0000 7fc21dcd9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc218103d90 0x7fc2181a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:49.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.388+0000 7fc21dcd9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc218108a90 0x7fc2181a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:49.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.388+0000 7fc21dcd9640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc21810e680 con 0x7fc218102b50 2024-05-14T09:34:49.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.388+0000 7fc21dcd9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc21810e500 con 0x7fc218108a90 2024-05-14T09:34:49.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.388+0000 7fc21dcd9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc21810e800 con 0x7fc218103d90 2024-05-14T09:34:49.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.388+0000 7fc2177fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc218102b50 0x7fc21806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:49.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.388+0000 7fc2177fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc218102b50 0x7fc21806b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54504/0 (socket says 172.21.15.2:54504) 2024-05-14T09:34:49.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.388+0000 7fc2177fe640 1 -- 172.21.15.2:0/1527280560 learned_addr learned my addr 172.21.15.2:0/1527280560 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:49.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.388+0000 7fc20effd640 1 --2- 172.21.15.2:0/1527280560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc218103d90 0x7fc2181a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:49.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.389+0000 7fc217fff640 1 --2- 172.21.15.2:0/1527280560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc218108a90 0x7fc2181a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:49.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.389+0000 7fc2177fe640 1 -- 172.21.15.2:0/1527280560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc218103d90 msgr2=0x7fc2181a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:49.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.389+0000 7fc2177fe640 1 --2- 172.21.15.2:0/1527280560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc218103d90 0x7fc2181a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:49.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.389+0000 7fc2177fe640 1 -- 172.21.15.2:0/1527280560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc218108a90 msgr2=0x7fc2181a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:49.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.389+0000 7fc2177fe640 1 --2- 172.21.15.2:0/1527280560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc218108a90 0x7fc2181a3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:49.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.389+0000 7fc2177fe640 1 -- 172.21.15.2:0/1527280560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2181a3ef0 con 0x7fc218102b50 2024-05-14T09:34:49.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.389+0000 7fc217fff640 1 --2- 172.21.15.2:0/1527280560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc218108a90 0x7fc2181a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:34:49.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.389+0000 7fc2177fe640 1 --2- 172.21.15.2:0/1527280560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc218102b50 0x7fc21806b0d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc20800c970 tx=0x7fc20800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:49.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.389+0000 7fc2157fa640 1 -- 172.21.15.2:0/1527280560 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208007c70 con 0x7fc218102b50 2024-05-14T09:34:49.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.389+0000 7fc21dcd9640 1 -- 172.21.15.2:0/1527280560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2181a41e0 con 0x7fc218102b50 2024-05-14T09:34:49.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.389+0000 7fc2157fa640 1 -- 172.21.15.2:0/1527280560 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc208007e10 con 0x7fc218102b50 2024-05-14T09:34:49.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.389+0000 7fc2157fa640 1 -- 172.21.15.2:0/1527280560 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208018760 con 0x7fc218102b50 2024-05-14T09:34:49.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.389+0000 7fc21dcd9640 1 -- 172.21.15.2:0/1527280560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2181a4780 con 0x7fc218102b50 2024-05-14T09:34:49.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.390+0000 7fc21dcd9640 1 -- 172.21.15.2:0/1527280560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2181078a0 con 0x7fc218102b50 2024-05-14T09:34:49.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.392+0000 7fc2157fa640 1 -- 172.21.15.2:0/1527280560 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc20800fab0 con 0x7fc218102b50 2024-05-14T09:34:49.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.393+0000 7fc2157fa640 1 --2- 172.21.15.2:0/1527280560 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc2040b28a0 0x7fc2040b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:49.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.393+0000 7fc20effd640 1 --2- 172.21.15.2:0/1527280560 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc2040b28a0 0x7fc2040b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:49.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.393+0000 7fc2157fa640 1 -- 172.21.15.2:0/1527280560 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fc2080e2820 con 0x7fc218102b50 2024-05-14T09:34:49.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.393+0000 7fc20effd640 1 --2- 172.21.15.2:0/1527280560 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc2040b28a0 0x7fc2040b4db0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc200009230 tx=0x7fc200031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:49.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.396+0000 7fc2157fa640 1 -- 172.21.15.2:0/1527280560 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc208014030 con 0x7fc218102b50 2024-05-14T09:34:49.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:49.631+0000 7fc21dcd9640 1 -- 172.21.15.2:0/1527280560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fc21802dc30 con 0x7fc218102b50 2024-05-14T09:34:50.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.412+0000 7fc2157fa640 1 -- 172.21.15.2:0/1527280560 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7fc2080aa150 con 0x7fc218102b50 2024-05-14T09:34:50.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.415+0000 7fc21dcd9640 1 -- 172.21.15.2:0/1527280560 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc2040b28a0 msgr2=0x7fc2040b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:50.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.415+0000 7fc21dcd9640 1 --2- 172.21.15.2:0/1527280560 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc2040b28a0 0x7fc2040b4db0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc200009230 tx=0x7fc200031400 comp rx=0 tx=0).stop 2024-05-14T09:34:50.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.415+0000 7fc21dcd9640 1 -- 172.21.15.2:0/1527280560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc218102b50 msgr2=0x7fc21806b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:50.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.415+0000 7fc21dcd9640 1 --2- 172.21.15.2:0/1527280560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc218102b50 0x7fc21806b0d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc20800c970 tx=0x7fc20800ce30 comp rx=0 tx=0).stop 2024-05-14T09:34:50.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.415+0000 7fc21dcd9640 1 -- 172.21.15.2:0/1527280560 shutdown_connections 2024-05-14T09:34:50.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.415+0000 7fc21dcd9640 1 --2- 172.21.15.2:0/1527280560 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc2040b28a0 0x7fc2040b4db0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.415+0000 7fc21dcd9640 1 --2- 172.21.15.2:0/1527280560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc218108a90 0x7fc2181a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.415+0000 7fc21dcd9640 1 --2- 172.21.15.2:0/1527280560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc218103d90 0x7fc2181a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.415+0000 7fc21dcd9640 1 --2- 172.21.15.2:0/1527280560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc218102b50 0x7fc21806b0d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.415+0000 7fc21dcd9640 1 -- 172.21.15.2:0/1527280560 >> 172.21.15.2:0/1527280560 conn(0x7fc2180fdd40 msgr2=0x7fc218104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:50.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.415+0000 7fc21dcd9640 1 -- 172.21.15.2:0/1527280560 shutdown_connections 2024-05-14T09:34:50.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.415+0000 7fc21dcd9640 1 -- 172.21.15.2:0/1527280560 wait complete. 2024-05-14T09:34:50.429 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-14T09:34:50.442 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-14T09:34:50.453 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T09:34:50.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.576+0000 7f32ec7a8640 1 -- 172.21.15.2:0/4027868616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32e4102b60 msgr2=0x7f32e4103860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:50.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.576+0000 7f32ec7a8640 1 --2- 172.21.15.2:0/4027868616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32e4102b60 0x7f32e4103860 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009ca0 tx=0x7f32e002a8c0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.577+0000 7f32ec7a8640 1 -- 172.21.15.2:0/4027868616 shutdown_connections 2024-05-14T09:34:50.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.577+0000 7f32ec7a8640 1 --2- 172.21.15.2:0/4027868616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e4108aa0 0x7f32e410aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.577+0000 7f32ec7a8640 1 --2- 172.21.15.2:0/4027868616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32e4103da0 0x7f32e41041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.577+0000 7f32ec7a8640 1 --2- 172.21.15.2:0/4027868616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32e4102b60 0x7f32e4103860 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.577+0000 7f32ec7a8640 1 -- 172.21.15.2:0/4027868616 >> 172.21.15.2:0/4027868616 conn(0x7f32e40fdd70 msgr2=0x7f32e4100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:50.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.577+0000 7f32ec7a8640 1 -- 172.21.15.2:0/4027868616 shutdown_connections 2024-05-14T09:34:50.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.577+0000 7f32ec7a8640 1 -- 172.21.15.2:0/4027868616 wait complete. 2024-05-14T09:34:50.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.577+0000 7f32ec7a8640 1 Processor -- start 2024-05-14T09:34:50.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.577+0000 7f32ec7a8640 1 -- start start 2024-05-14T09:34:50.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32ec7a8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e4102b60 0x7f32e406aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:50.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32ec7a8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32e4103da0 0x7f32e419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:50.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32ec7a8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32e4108aa0 0x7f32e41a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:50.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32ec7a8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f32e410e5f0 con 0x7f32e4103da0 2024-05-14T09:34:50.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32ec7a8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32e410e470 con 0x7f32e4102b60 2024-05-14T09:34:50.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32ec7a8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f32e410e770 con 0x7f32e4108aa0 2024-05-14T09:34:50.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32e9d43640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32e4103da0 0x7f32e419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:50.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32e9d43640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32e4103da0 0x7f32e419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54526/0 (socket says 172.21.15.2:54526) 2024-05-14T09:34:50.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32e9d43640 1 -- 172.21.15.2:0/2758826511 learned_addr learned my addr 172.21.15.2:0/2758826511 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:50.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32ea544640 1 --2- 172.21.15.2:0/2758826511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e4102b60 0x7f32e406aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:50.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32ead45640 1 --2- 172.21.15.2:0/2758826511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32e4108aa0 0x7f32e41a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:50.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32e9d43640 1 -- 172.21.15.2:0/2758826511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32e4108aa0 msgr2=0x7f32e41a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:50.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32e9d43640 1 --2- 172.21.15.2:0/2758826511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32e4108aa0 0x7f32e41a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32e9d43640 1 -- 172.21.15.2:0/2758826511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e4102b60 msgr2=0x7f32e406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:50.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32e9d43640 1 --2- 172.21.15.2:0/2758826511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e4102b60 0x7f32e406aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32e9d43640 1 -- 172.21.15.2:0/2758826511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e41a1db0 con 0x7f32e4103da0 2024-05-14T09:34:50.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32ead45640 1 --2- 172.21.15.2:0/2758826511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32e4108aa0 0x7f32e41a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:34:50.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32e9d43640 1 --2- 172.21.15.2:0/2758826511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32e4103da0 0x7f32e419f040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f32d400d770 tx=0x7f32d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:50.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32db7fe640 1 -- 172.21.15.2:0/2758826511 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4017070 con 0x7f32e4103da0 2024-05-14T09:34:50.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32db7fe640 1 -- 172.21.15.2:0/2758826511 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32d40054d0 con 0x7f32e4103da0 2024-05-14T09:34:50.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32ec7a8640 1 -- 172.21.15.2:0/2758826511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e41a20a0 con 0x7f32e4103da0 2024-05-14T09:34:50.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.578+0000 7f32db7fe640 1 -- 172.21.15.2:0/2758826511 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4004c30 con 0x7f32e4103da0 2024-05-14T09:34:50.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.579+0000 7f32ec7a8640 1 -- 172.21.15.2:0/2758826511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e41a25f0 con 0x7f32e4103da0 2024-05-14T09:34:50.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.579+0000 7f32ec7a8640 1 -- 172.21.15.2:0/2758826511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e41078b0 con 0x7f32e4103da0 2024-05-14T09:34:50.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.581+0000 7f32db7fe640 1 -- 172.21.15.2:0/2758826511 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f32d4004dd0 con 0x7f32e4103da0 2024-05-14T09:34:50.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.582+0000 7f32db7fe640 1 --2- 172.21.15.2:0/2758826511 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f32d00b2920 0x7f32d00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:50.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.582+0000 7f32ea544640 1 --2- 172.21.15.2:0/2758826511 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f32d00b2920 0x7f32d00b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:50.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.582+0000 7f32db7fe640 1 -- 172.21.15.2:0/2758826511 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f32d40e2400 con 0x7f32e4103da0 2024-05-14T09:34:50.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.582+0000 7f32ea544640 1 --2- 172.21.15.2:0/2758826511 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f32d00b2920 0x7f32d00b4e30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f32e0000f80 tx=0x7f32e0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:50.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.585+0000 7f32db7fe640 1 -- 172.21.15.2:0/2758826511 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f32d40a9e20 con 0x7f32e4103da0 2024-05-14T09:34:50.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:50 smithi002 ceph-mon[31557]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:50.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:50 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1527280560' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-14T09:34:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:50 smithi029 ceph-mon[36249]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:50 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1527280560' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-14T09:34:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:50 smithi026 ceph-mon[36056]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T09:34:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:50 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1527280560' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-14T09:34:50.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.780+0000 7f32ec7a8640 1 -- 172.21.15.2:0/2758826511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32e402de00 con 0x7f32e4103da0 2024-05-14T09:34:50.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.781+0000 7f32db7fe640 1 -- 172.21.15.2:0/2758826511 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24740 (secure 0 0 0) 0x7f32d40a9720 con 0x7f32e4103da0 2024-05-14T09:34:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:34:50.782 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":90,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","created":"2024-05-14T09:22:34.937047+0000","modified":"2024-05-14T09:32:30.306958+0000","last_up_change":"2024-05-14T09:29:55.157569+0000","last_in_change":"2024-05-14T09:29:34.922562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T09:26:03.046454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T09:30:51.814350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T09:30:55.500000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T09:31:06.099087+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10d34289-414e-4b1b-bc7c-ba1dd24c7297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6803","nonce":2622184634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6805","nonce":2622184634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6809","nonce":2622184634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6807","nonce":2622184634}]},"public_addr":"172.21.15.2:6803/2622184634","cluster_addr":"172.21.15.2:6805/2622184634","heartbeat_back_addr":"172.21.15.2:6809/2622184634","heartbeat_front_addr":"172.21.15.2:6807/2622184634","state":["exists","up"]},{"osd":1,"uuid":"bc4aefc2-706e-4faf-9042-64b87ad097b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6801","nonce":3084747226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6803","nonce":3084747226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6807","nonce":3084747226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6805","nonce":3084747226}]},"public_addr":"172.21.15.26:6801/3084747226","cluster_addr":"172.21.15.26:6803/3084747226","heartbeat_back_addr":"172.21.15.26:6807/3084747226","heartbeat_front_addr":"172.21.15.26:6805/3084747226","state":["exists","up"]},{"osd":2,"uuid":"8c5bb589-56e2-469f-9a6f-6035b3ddf4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6801","nonce":814048086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6803","nonce":814048086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6807","nonce":814048086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6805","nonce":814048086}]},"public_addr":"172.21.15.29:6801/814048086","cluster_addr":"172.21.15.29:6803/814048086","heartbeat_back_addr":"172.21.15.29:6807/814048086","heartbeat_front_addr":"172.21.15.29:6805/814048086","state":["exists","up"]},{"osd":3,"uuid":"8fbfc078-85e7-4dc2-acda-5e7b0ab927e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6811","nonce":879639451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6813","nonce":879639451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6817","nonce":879639451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6815","nonce":879639451}]},"public_addr":"172.21.15.2:6811/879639451","cluster_addr":"172.21.15.2:6813/879639451","heartbeat_back_addr":"172.21.15.2:6817/879639451","heartbeat_front_addr":"172.21.15.2:6815/879639451","state":["exists","up"]},{"osd":4,"uuid":"f2515448-e9c1-48a2-b270-1e5e781a2fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6809","nonce":457100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6811","nonce":457100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6815","nonce":457100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6813","nonce":457100752}]},"public_addr":"172.21.15.26:6809/457100752","cluster_addr":"172.21.15.26:6811/457100752","heartbeat_back_addr":"172.21.15.26:6815/457100752","heartbeat_front_addr":"172.21.15.26:6813/457100752","state":["exists","up"]},{"osd":5,"uuid":"20030857-a1e6-4b0a-b186-1cfa44f2e4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6809","nonce":1554045093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6811","nonce":1554045093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6815","nonce":1554045093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6813","nonce":1554045093}]},"public_addr":"172.21.15.29:6809/1554045093","cluster_addr":"172.21.15.29:6811/1554045093","heartbeat_back_addr":"172.21.15.29:6815/1554045093","heartbeat_front_addr":"172.21.15.29:6813/1554045093","state":["exists","up"]},{"osd":6,"uuid":"d757c72a-29f0-4f14-9f13-a2e61eb5e6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6819","nonce":2749897595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6821","nonce":2749897595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6825","nonce":2749897595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6823","nonce":2749897595}]},"public_addr":"172.21.15.2:6819/2749897595","cluster_addr":"172.21.15.2:6821/2749897595","heartbeat_back_addr":"172.21.15.2:6825/2749897595","heartbeat_front_addr":"172.21.15.2:6823/2749897595","state":["exists","up"]},{"osd":7,"uuid":"22ddef80-3633-44c5-bc0e-edc75b546ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6817","nonce":1180401611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6819","nonce":1180401611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6823","nonce":1180401611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6821","nonce":1180401611}]},"public_addr":"172.21.15.26:6817/1180401611","cluster_addr":"172.21.15.26:6819/1180401611","heartbeat_back_addr":"172.21.15.26:6823/1180401611","heartbeat_front_addr":"172.21.15.26:6821/1180401611","state":["exists","up"]},{"osd":8,"uuid":"466e7ac4-3ff7-4254-a3fd-8096420b1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6817","nonce":2858408446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6819","nonce":2858408446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6823","nonce":2858408446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6821","nonce":2858408446}]},"public_addr":"172.21.15.29:6817/2858408446","cluster_addr":"172.21.15.29:6819/2858408446","heartbeat_back_addr":"172.21.15.29:6823/2858408446","heartbeat_front_addr":"172.21.15.29:6821/2858408446","state":["exists","up"]},{"osd":9,"uuid":"119eeac9-29f7-4086-a1ad-04a0bc154b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6827","nonce":176898226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6829","nonce":176898226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6833","nonce":176898226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6831","nonce":176898226}]},"public_addr":"172.21.15.2:6827/176898226","cluster_addr":"172.21.15.2:6829/176898226","heartbeat_back_addr":"172.21.15.2:6833/176898226","heartbeat_front_addr":"172.21.15.2:6831/176898226","state":["exists","up"]},{"osd":10,"uuid":"55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6825","nonce":4065159515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6827","nonce":4065159515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6831","nonce":4065159515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6829","nonce":4065159515}]},"public_addr":"172.21.15.26:6825/4065159515","cluster_addr":"172.21.15.26:6827/4065159515","heartbeat_back_addr":"172.21.15.26:6831/4065159515","heartbeat_front_addr":"172.21.15.26:6829/4065159515","state":["exists","up"]},{"osd":11,"uuid":"c5246763-c4f6-4ae3-8a93-5929c541ac5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6825","nonce":378800417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6827","nonce":378800417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6831","nonce":378800417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6829","nonce":378800417}]},"public_addr":"172.21.15.29:6825/378800417","cluster_addr":"172.21.15.29:6827/378800417","heartbeat_back_addr":"172.21.15.29:6831/378800417","heartbeat_front_addr":"172.21.15.29:6829/378800417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:09.342387+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:34.156359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:58.860653+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:24.009323+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:49.057146+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:15.007172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:41.085748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:06.674923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:33.409128+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:00.106186+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:26.803487+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:53.668283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11},{"from":8,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9},{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4051922409":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/2353790181":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/4268171474":"2024-05-15T09:23:28.940670+0000","172.21.15.2:6801/1423742578":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/969850441":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3252991268":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6801/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3789067476":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/1423742578":"2024-05-15T09:23:28.940670+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T09:34:50.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.783+0000 7f32ec7a8640 1 -- 172.21.15.2:0/2758826511 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f32d00b2920 msgr2=0x7f32d00b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.783+0000 7f32ec7a8640 1 --2- 172.21.15.2:0/2758826511 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f32d00b2920 0x7f32d00b4e30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f32e0000f80 tx=0x7f32e0004a40 comp rx=0 tx=0).stop 2024-05-14T09:34:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.783+0000 7f32ec7a8640 1 -- 172.21.15.2:0/2758826511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32e4103da0 msgr2=0x7f32e419f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.783+0000 7f32ec7a8640 1 --2- 172.21.15.2:0/2758826511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32e4103da0 0x7f32e419f040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f32d400d770 tx=0x7f32d400dc30 comp rx=0 tx=0).stop 2024-05-14T09:34:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.784+0000 7f32ec7a8640 1 -- 172.21.15.2:0/2758826511 shutdown_connections 2024-05-14T09:34:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.784+0000 7f32ec7a8640 1 --2- 172.21.15.2:0/2758826511 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f32d00b2920 0x7f32d00b4e30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.784+0000 7f32ec7a8640 1 --2- 172.21.15.2:0/2758826511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f32e4108aa0 0x7f32e41a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.784+0000 7f32ec7a8640 1 --2- 172.21.15.2:0/2758826511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32e4103da0 0x7f32e419f040 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.784+0000 7f32ec7a8640 1 --2- 172.21.15.2:0/2758826511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e4102b60 0x7f32e406aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.784+0000 7f32ec7a8640 1 -- 172.21.15.2:0/2758826511 >> 172.21.15.2:0/2758826511 conn(0x7f32e40fdd70 msgr2=0x7f32e4104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.784+0000 7f32ec7a8640 1 -- 172.21.15.2:0/2758826511 shutdown_connections 2024-05-14T09:34:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.784+0000 7f32ec7a8640 1 -- 172.21.15.2:0/2758826511 wait complete. 2024-05-14T09:34:50.795 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T09:26:03.046454+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-14T09:30:51.814350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-14T09:30:55.500000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-14T09:31:06.099087+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-14T09:34:50.796 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-14T09:34:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.914+0000 7fa9cf2e2640 1 -- 172.21.15.2:0/1431431420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9c8102b40 msgr2=0x7fa9c8103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.914+0000 7fa9cf2e2640 1 --2- 172.21.15.2:0/1431431420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9c8102b40 0x7fa9c8103840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009900 tx=0x7fa9c002a410 comp rx=0 tx=0).stop 2024-05-14T09:34:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.914+0000 7fa9cf2e2640 1 -- 172.21.15.2:0/1431431420 shutdown_connections 2024-05-14T09:34:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.914+0000 7fa9cf2e2640 1 --2- 172.21.15.2:0/1431431420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9c8108a80 0x7fa9c810aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.914+0000 7fa9cf2e2640 1 --2- 172.21.15.2:0/1431431420 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9c8103d80 0x7fa9c8104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.914+0000 7fa9cf2e2640 1 --2- 172.21.15.2:0/1431431420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9c8102b40 0x7fa9c8103840 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.914+0000 7fa9cf2e2640 1 -- 172.21.15.2:0/1431431420 >> 172.21.15.2:0/1431431420 conn(0x7fa9c80fdd30 msgr2=0x7fa9c81001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.914+0000 7fa9cf2e2640 1 -- 172.21.15.2:0/1431431420 shutdown_connections 2024-05-14T09:34:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.914+0000 7fa9cf2e2640 1 -- 172.21.15.2:0/1431431420 wait complete. 2024-05-14T09:34:50.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.914+0000 7fa9cf2e2640 1 Processor -- start 2024-05-14T09:34:50.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cf2e2640 1 -- start start 2024-05-14T09:34:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cf2e2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9c8102b40 0x7fa9c819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cf2e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9c8103d80 0x7fa9c819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cf2e2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9c8108a80 0x7fa9c81a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cf2e2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c810e620 con 0x7fa9c8108a80 2024-05-14T09:34:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cf2e2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c810e4a0 con 0x7fa9c8103d80 2024-05-14T09:34:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cf2e2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c810e7a0 con 0x7fa9c8102b40 2024-05-14T09:34:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cd87f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9c8108a80 0x7fa9c81a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cd87f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9c8108a80 0x7fa9c81a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54558/0 (socket says 172.21.15.2:54558) 2024-05-14T09:34:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cd87f640 1 -- 172.21.15.2:0/3861139709 learned_addr learned my addr 172.21.15.2:0/3861139709 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cd07e640 1 --2- 172.21.15.2:0/3861139709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9c8102b40 0x7fa9c819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cc87d640 1 --2- 172.21.15.2:0/3861139709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9c8103d80 0x7fa9c819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cd87f640 1 -- 172.21.15.2:0/3861139709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9c8102b40 msgr2=0x7fa9c819a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cd87f640 1 --2- 172.21.15.2:0/3861139709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9c8102b40 0x7fa9c819a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cd87f640 1 -- 172.21.15.2:0/3861139709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9c8103d80 msgr2=0x7fa9c819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cd87f640 1 --2- 172.21.15.2:0/3861139709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9c8103d80 0x7fa9c819a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cd87f640 1 -- 172.21.15.2:0/3861139709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c81a1ef0 con 0x7fa9c8108a80 2024-05-14T09:34:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.915+0000 7fa9cd87f640 1 --2- 172.21.15.2:0/3861139709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9c8108a80 0x7fa9c81a1880 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800eb10 tx=0x7fa9b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.916+0000 7fa9c67fc640 1 -- 172.21.15.2:0/3861139709 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8004b20 con 0x7fa9c8108a80 2024-05-14T09:34:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.916+0000 7fa9c67fc640 1 -- 172.21.15.2:0/3861139709 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b8004cc0 con 0x7fa9c8108a80 2024-05-14T09:34:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.916+0000 7fa9cf2e2640 1 -- 172.21.15.2:0/3861139709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c81a21e0 con 0x7fa9c8108a80 2024-05-14T09:34:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.916+0000 7fa9c67fc640 1 -- 172.21.15.2:0/3861139709 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8010670 con 0x7fa9c8108a80 2024-05-14T09:34:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.916+0000 7fa9cf2e2640 1 -- 172.21.15.2:0/3861139709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c81a26e0 con 0x7fa9c8108a80 2024-05-14T09:34:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.917+0000 7fa9cf2e2640 1 -- 172.21.15.2:0/3861139709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c81a29a0 con 0x7fa9c8108a80 2024-05-14T09:34:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.918+0000 7fa9c67fc640 1 -- 172.21.15.2:0/3861139709 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa9b8005000 con 0x7fa9c8108a80 2024-05-14T09:34:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.919+0000 7fa9c67fc640 1 --2- 172.21.15.2:0/3861139709 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa9b00b27f0 0x7fa9b00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.919+0000 7fa9cd07e640 1 --2- 172.21.15.2:0/3861139709 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa9b00b27f0 0x7fa9b00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.919+0000 7fa9c67fc640 1 -- 172.21.15.2:0/3861139709 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fa9b80e2f60 con 0x7fa9c8108a80 2024-05-14T09:34:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.920+0000 7fa9cd07e640 1 --2- 172.21.15.2:0/3861139709 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa9b00b27f0 0x7fa9b00b4d00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009230 tx=0x7fa9c0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:50.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:50.922+0000 7fa9c67fc640 1 -- 172.21.15.2:0/3861139709 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9b8054df0 con 0x7fa9c8108a80 2024-05-14T09:34:51.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.114+0000 7fa9cf2e2640 1 -- 172.21.15.2:0/3861139709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa9c81a2b70 con 0x7fa9c8108a80 2024-05-14T09:34:51.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.115+0000 7fa9c67fc640 1 -- 172.21.15.2:0/3861139709 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fa9b80aa860 con 0x7fa9c8108a80 2024-05-14T09:34:51.115 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-05-14T09:34:51.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.117+0000 7fa9cf2e2640 1 -- 172.21.15.2:0/3861139709 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa9b00b27f0 msgr2=0x7fa9b00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:51.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.117+0000 7fa9cf2e2640 1 --2- 172.21.15.2:0/3861139709 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa9b00b27f0 0x7fa9b00b4d00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009230 tx=0x7fa9c0031400 comp rx=0 tx=0).stop 2024-05-14T09:34:51.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.117+0000 7fa9cf2e2640 1 -- 172.21.15.2:0/3861139709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9c8108a80 msgr2=0x7fa9c81a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:51.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.117+0000 7fa9cf2e2640 1 --2- 172.21.15.2:0/3861139709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9c8108a80 0x7fa9c81a1880 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800eb10 tx=0x7fa9b800c2d0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.117+0000 7fa9cf2e2640 1 -- 172.21.15.2:0/3861139709 shutdown_connections 2024-05-14T09:34:51.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.117+0000 7fa9cf2e2640 1 --2- 172.21.15.2:0/3861139709 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa9b00b27f0 0x7fa9b00b4d00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.117+0000 7fa9cf2e2640 1 --2- 172.21.15.2:0/3861139709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9c8108a80 0x7fa9c81a1880 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.117+0000 7fa9cf2e2640 1 --2- 172.21.15.2:0/3861139709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9c8103d80 0x7fa9c819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.117+0000 7fa9cf2e2640 1 --2- 172.21.15.2:0/3861139709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9c8102b40 0x7fa9c819a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.117+0000 7fa9cf2e2640 1 -- 172.21.15.2:0/3861139709 >> 172.21.15.2:0/3861139709 conn(0x7fa9c80fdd30 msgr2=0x7fa9c810a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:51.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.117+0000 7fa9cf2e2640 1 -- 172.21.15.2:0/3861139709 shutdown_connections 2024-05-14T09:34:51.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.117+0000 7fa9cf2e2640 1 -- 172.21.15.2:0/3861139709 wait complete. 2024-05-14T09:34:51.129 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-14T09:34:51.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.247+0000 7fed0b642640 1 -- 172.21.15.2:0/3710602951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed04109ba0 msgr2=0x7fed0410bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:51.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.247+0000 7fed0b642640 1 --2- 172.21.15.2:0/3710602951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed04109ba0 0x7fed0410bfc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009ca0 tx=0x7fecfc02a8c0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.247+0000 7fed0b642640 1 -- 172.21.15.2:0/3710602951 shutdown_connections 2024-05-14T09:34:51.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.247+0000 7fed0b642640 1 --2- 172.21.15.2:0/3710602951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed04109ba0 0x7fed0410bfc0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.247+0000 7fed0b642640 1 --2- 172.21.15.2:0/3710602951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed04103960 0x7fed04103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.247+0000 7fed0b642640 1 --2- 172.21.15.2:0/3710602951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed041052a0 0x7fed04103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.247+0000 7fed0b642640 1 -- 172.21.15.2:0/3710602951 >> 172.21.15.2:0/3710602951 conn(0x7fed040fee70 msgr2=0x7fed04101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:51.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.247+0000 7fed0b642640 1 -- 172.21.15.2:0/3710602951 shutdown_connections 2024-05-14T09:34:51.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.247+0000 7fed0b642640 1 -- 172.21.15.2:0/3710602951 wait complete. 2024-05-14T09:34:51.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed0b642640 1 Processor -- start 2024-05-14T09:34:51.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed0b642640 1 -- start start 2024-05-14T09:34:51.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed0b642640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed04103960 0x7fed0419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:51.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed0b642640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed041052a0 0x7fed0419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:51.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed0b642640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed04109ba0 0x7fed041a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:51.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed0b642640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fed0410f8a0 con 0x7fed041052a0 2024-05-14T09:34:51.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed0b642640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fed0410f720 con 0x7fed04109ba0 2024-05-14T09:34:51.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed0b642640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fed0410fa20 con 0x7fed04103960 2024-05-14T09:34:51.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed08bdd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed041052a0 0x7fed0419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:51.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed08bdd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed041052a0 0x7fed0419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54580/0 (socket says 172.21.15.2:54580) 2024-05-14T09:34:51.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed08bdd640 1 -- 172.21.15.2:0/3853797184 learned_addr learned my addr 172.21.15.2:0/3853797184 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:51.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed09bdf640 1 --2- 172.21.15.2:0/3853797184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed04109ba0 0x7fed041a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:51.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed093de640 1 --2- 172.21.15.2:0/3853797184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed04103960 0x7fed0419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:51.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed08bdd640 1 -- 172.21.15.2:0/3853797184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed04103960 msgr2=0x7fed0419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:51.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed08bdd640 1 --2- 172.21.15.2:0/3853797184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed04103960 0x7fed0419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed08bdd640 1 -- 172.21.15.2:0/3853797184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed04109ba0 msgr2=0x7fed041a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:51.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed08bdd640 1 --2- 172.21.15.2:0/3853797184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed04109ba0 0x7fed041a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.248+0000 7fed08bdd640 1 -- 172.21.15.2:0/3853797184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed041a1fc0 con 0x7fed041052a0 2024-05-14T09:34:51.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.249+0000 7fed08bdd640 1 --2- 172.21.15.2:0/3853797184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed041052a0 0x7fed0419a9c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fecf400d950 tx=0x7fecf400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:51.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.249+0000 7fecfa7fc640 1 -- 172.21.15.2:0/3853797184 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4017030 con 0x7fed041052a0 2024-05-14T09:34:51.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.249+0000 7fecfa7fc640 1 -- 172.21.15.2:0/3853797184 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecf4004020 con 0x7fed041052a0 2024-05-14T09:34:51.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.249+0000 7fed0b642640 1 -- 172.21.15.2:0/3853797184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed041a22b0 con 0x7fed041052a0 2024-05-14T09:34:51.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.249+0000 7fecfa7fc640 1 -- 172.21.15.2:0/3853797184 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf40047a0 con 0x7fed041052a0 2024-05-14T09:34:51.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.249+0000 7fed0b642640 1 -- 172.21.15.2:0/3853797184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed041a2800 con 0x7fed041052a0 2024-05-14T09:34:51.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.250+0000 7fed0b642640 1 -- 172.21.15.2:0/3853797184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed0406a340 con 0x7fed041052a0 2024-05-14T09:34:51.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.251+0000 7fecfa7fc640 1 -- 172.21.15.2:0/3853797184 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fecf4009a30 con 0x7fed041052a0 2024-05-14T09:34:51.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.252+0000 7fecfa7fc640 1 --2- 172.21.15.2:0/3853797184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fecf00b27f0 0x7fecf00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:51.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.252+0000 7fecfa7fc640 1 -- 172.21.15.2:0/3853797184 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fecf40e25f0 con 0x7fed041052a0 2024-05-14T09:34:51.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.252+0000 7fed093de640 1 --2- 172.21.15.2:0/3853797184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fecf00b27f0 0x7fecf00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:51.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.253+0000 7fed093de640 1 --2- 172.21.15.2:0/3853797184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fecf00b27f0 0x7fecf00b4d00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fed04104860 tx=0x7fed00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:51.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.256+0000 7fecfa7fc640 1 -- 172.21.15.2:0/3853797184 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fecf40a9ef0 con 0x7fed041052a0 2024-05-14T09:34:51.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.449+0000 7fed0b642640 1 -- 172.21.15.2:0/3853797184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fed040638d0 con 0x7fed041052a0 2024-05-14T09:34:51.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.450+0000 7fecfa7fc640 1 -- 172.21.15.2:0/3853797184 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fecf40a97f0 con 0x7fed041052a0 2024-05-14T09:34:51.450 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-05-14T09:34:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.452+0000 7fed0b642640 1 -- 172.21.15.2:0/3853797184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fecf00b27f0 msgr2=0x7fecf00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.452+0000 7fed0b642640 1 --2- 172.21.15.2:0/3853797184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fecf00b27f0 0x7fecf00b4d00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fed04104860 tx=0x7fed00005f90 comp rx=0 tx=0).stop 2024-05-14T09:34:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.452+0000 7fed0b642640 1 -- 172.21.15.2:0/3853797184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed041052a0 msgr2=0x7fed0419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.452+0000 7fed0b642640 1 --2- 172.21.15.2:0/3853797184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed041052a0 0x7fed0419a9c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fecf400d950 tx=0x7fecf400de10 comp rx=0 tx=0).stop 2024-05-14T09:34:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.452+0000 7fed0b642640 1 -- 172.21.15.2:0/3853797184 shutdown_connections 2024-05-14T09:34:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.452+0000 7fed0b642640 1 --2- 172.21.15.2:0/3853797184 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fecf00b27f0 0x7fecf00b4d00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.452+0000 7fed0b642640 1 --2- 172.21.15.2:0/3853797184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed04109ba0 0x7fed041a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.452+0000 7fed0b642640 1 --2- 172.21.15.2:0/3853797184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed041052a0 0x7fed0419a9c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.452+0000 7fed0b642640 1 --2- 172.21.15.2:0/3853797184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fed04103960 0x7fed0419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.452+0000 7fed0b642640 1 -- 172.21.15.2:0/3853797184 >> 172.21.15.2:0/3853797184 conn(0x7fed040fee70 msgr2=0x7fed04105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.452+0000 7fed0b642640 1 -- 172.21.15.2:0/3853797184 shutdown_connections 2024-05-14T09:34:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.452+0000 7fed0b642640 1 -- 172.21.15.2:0/3853797184 wait complete. 2024-05-14T09:34:51.464 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-14T09:34:51.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.582+0000 7fa80ad0d640 1 -- 172.21.15.2:0/1275167895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa804103d40 msgr2=0x7fa804104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:51.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.582+0000 7fa80ad0d640 1 --2- 172.21.15.2:0/1275167895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa804103d40 0x7fa804104150 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009900 tx=0x7fa7f802a410 comp rx=0 tx=0).stop 2024-05-14T09:34:51.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.582+0000 7fa80ad0d640 1 -- 172.21.15.2:0/1275167895 shutdown_connections 2024-05-14T09:34:51.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.582+0000 7fa80ad0d640 1 --2- 172.21.15.2:0/1275167895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa804108a40 0x7fa80410ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.582+0000 7fa80ad0d640 1 --2- 172.21.15.2:0/1275167895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa804103d40 0x7fa804104150 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.582+0000 7fa80ad0d640 1 --2- 172.21.15.2:0/1275167895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa804102b00 0x7fa804103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.582+0000 7fa80ad0d640 1 -- 172.21.15.2:0/1275167895 >> 172.21.15.2:0/1275167895 conn(0x7fa8040fdcf0 msgr2=0x7fa8041001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.582+0000 7fa80ad0d640 1 -- 172.21.15.2:0/1275167895 shutdown_connections 2024-05-14T09:34:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.582+0000 7fa80ad0d640 1 -- 172.21.15.2:0/1275167895 wait complete. 2024-05-14T09:34:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa80ad0d640 1 Processor -- start 2024-05-14T09:34:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa80ad0d640 1 -- start start 2024-05-14T09:34:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa80ad0d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa804102b00 0x7fa80419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa80ad0d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa804103d40 0x7fa80419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa80ad0d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa804108a40 0x7fa8041a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa80ad0d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa80410f790 con 0x7fa804103d40 2024-05-14T09:34:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa80ad0d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa80410f610 con 0x7fa804102b00 2024-05-14T09:34:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa80ad0d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa80410f910 con 0x7fa804108a40 2024-05-14T09:34:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa803fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa804103d40 0x7fa80419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa803fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa804103d40 0x7fa80419a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54596/0 (socket says 172.21.15.2:54596) 2024-05-14T09:34:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa803fff640 1 -- 172.21.15.2:0/4053206459 learned_addr learned my addr 172.21.15.2:0/4053206459 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:51.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa8092aa640 1 --2- 172.21.15.2:0/4053206459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa804108a40 0x7fa8041a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa808aa9640 1 --2- 172.21.15.2:0/4053206459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa804102b00 0x7fa80419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa803fff640 1 -- 172.21.15.2:0/4053206459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa804108a40 msgr2=0x7fa8041a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa803fff640 1 --2- 172.21.15.2:0/4053206459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa804108a40 0x7fa8041a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa803fff640 1 -- 172.21.15.2:0/4053206459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa804102b00 msgr2=0x7fa80419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa803fff640 1 --2- 172.21.15.2:0/4053206459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa804102b00 0x7fa80419a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.583+0000 7fa803fff640 1 -- 172.21.15.2:0/4053206459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8041a1eb0 con 0x7fa804103d40 2024-05-14T09:34:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.584+0000 7fa803fff640 1 --2- 172.21.15.2:0/4053206459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa804103d40 0x7fa80419a8b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8006010 tx=0x7fa7f8004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.584+0000 7fa801ffb640 1 -- 172.21.15.2:0/4053206459 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f803f030 con 0x7fa804103d40 2024-05-14T09:34:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.584+0000 7fa80ad0d640 1 -- 172.21.15.2:0/4053206459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8041a2140 con 0x7fa804103d40 2024-05-14T09:34:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.584+0000 7fa801ffb640 1 -- 172.21.15.2:0/4053206459 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7f8003cb0 con 0x7fa804103d40 2024-05-14T09:34:51.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.584+0000 7fa801ffb640 1 -- 172.21.15.2:0/4053206459 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8035070 con 0x7fa804103d40 2024-05-14T09:34:51.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.584+0000 7fa80ad0d640 1 -- 172.21.15.2:0/4053206459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8041a2680 con 0x7fa804103d40 2024-05-14T09:34:51.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.585+0000 7fa80ad0d640 1 -- 172.21.15.2:0/4053206459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa804107850 con 0x7fa804103d40 2024-05-14T09:34:51.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.587+0000 7fa801ffb640 1 -- 172.21.15.2:0/4053206459 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa7f8042030 con 0x7fa804103d40 2024-05-14T09:34:51.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.587+0000 7fa801ffb640 1 --2- 172.21.15.2:0/4053206459 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa7ec0b27f0 0x7fa7ec0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:51.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.588+0000 7fa808aa9640 1 --2- 172.21.15.2:0/4053206459 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa7ec0b27f0 0x7fa7ec0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:51.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.588+0000 7fa801ffb640 1 -- 172.21.15.2:0/4053206459 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fa7f8103440 con 0x7fa804103d40 2024-05-14T09:34:51.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.588+0000 7fa808aa9640 1 --2- 172.21.15.2:0/4053206459 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa7ec0b27f0 0x7fa7ec0b4d00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa804103700 tx=0x7fa7f4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:51.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.591+0000 7fa801ffb640 1 -- 172.21.15.2:0/4053206459 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7f80cace0 con 0x7fa804103d40 2024-05-14T09:34:51.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:51 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1527280560' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-14T09:34:51.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:51 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:34:51.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:51 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2758826511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:34:51.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:51 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3861139709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T09:34:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:51 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1527280560' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-14T09:34:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:51 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:34:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:51 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2758826511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:34:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:51 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3861139709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T09:34:51.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.781+0000 7fa80ad0d640 1 -- 172.21.15.2:0/4053206459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa80402ddc0 con 0x7fa804103d40 2024-05-14T09:34:51.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.782+0000 7fa801ffb640 1 -- 172.21.15.2:0/4053206459 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fa7f80ca5e0 con 0x7fa804103d40 2024-05-14T09:34:51.781 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-05-14T09:34:51.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.784+0000 7fa80ad0d640 1 -- 172.21.15.2:0/4053206459 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa7ec0b27f0 msgr2=0x7fa7ec0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:51.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.784+0000 7fa80ad0d640 1 --2- 172.21.15.2:0/4053206459 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa7ec0b27f0 0x7fa7ec0b4d00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa804103700 tx=0x7fa7f4008040 comp rx=0 tx=0).stop 2024-05-14T09:34:51.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.784+0000 7fa80ad0d640 1 -- 172.21.15.2:0/4053206459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa804103d40 msgr2=0x7fa80419a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:51.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.784+0000 7fa80ad0d640 1 --2- 172.21.15.2:0/4053206459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa804103d40 0x7fa80419a8b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8006010 tx=0x7fa7f8004340 comp rx=0 tx=0).stop 2024-05-14T09:34:51.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.784+0000 7fa80ad0d640 1 -- 172.21.15.2:0/4053206459 shutdown_connections 2024-05-14T09:34:51.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.784+0000 7fa80ad0d640 1 --2- 172.21.15.2:0/4053206459 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa7ec0b27f0 0x7fa7ec0b4d00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.784+0000 7fa80ad0d640 1 --2- 172.21.15.2:0/4053206459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa804108a40 0x7fa8041a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.784+0000 7fa80ad0d640 1 --2- 172.21.15.2:0/4053206459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa804103d40 0x7fa80419a8b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.784+0000 7fa80ad0d640 1 --2- 172.21.15.2:0/4053206459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa804102b00 0x7fa80419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.784+0000 7fa80ad0d640 1 -- 172.21.15.2:0/4053206459 >> 172.21.15.2:0/4053206459 conn(0x7fa8040fdcf0 msgr2=0x7fa804104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:51.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.784+0000 7fa80ad0d640 1 -- 172.21.15.2:0/4053206459 shutdown_connections 2024-05-14T09:34:51.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.784+0000 7fa80ad0d640 1 -- 172.21.15.2:0/4053206459 wait complete. 2024-05-14T09:34:51.794 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:51 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1527280560' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-14T09:34:51.794 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:51 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:34:51.794 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:51 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2758826511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:34:51.794 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:51 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3861139709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T09:34:51.795 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-14T09:34:51.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.913+0000 7fb2ef776640 1 -- 172.21.15.2:0/28803973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2e8104ea0 msgr2=0x7fb2e81052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:51.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.913+0000 7fb2ef776640 1 --2- 172.21.15.2:0/28803973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2e8104ea0 0x7fb2e81052b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009900 tx=0x7fb2d802a410 comp rx=0 tx=0).stop 2024-05-14T09:34:51.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.913+0000 7fb2ef776640 1 -- 172.21.15.2:0/28803973 shutdown_connections 2024-05-14T09:34:51.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.913+0000 7fb2ef776640 1 --2- 172.21.15.2:0/28803973 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e8109ba0 0x7fb2e810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.913+0000 7fb2ef776640 1 --2- 172.21.15.2:0/28803973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2e8104ea0 0x7fb2e81052b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.913+0000 7fb2ef776640 1 --2- 172.21.15.2:0/28803973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e8103c60 0x7fb2e8104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.913+0000 7fb2ef776640 1 -- 172.21.15.2:0/28803973 >> 172.21.15.2:0/28803973 conn(0x7fb2e80fee70 msgr2=0x7fb2e8101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:51.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.913+0000 7fb2ef776640 1 -- 172.21.15.2:0/28803973 shutdown_connections 2024-05-14T09:34:51.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.913+0000 7fb2ef776640 1 -- 172.21.15.2:0/28803973 wait complete. 2024-05-14T09:34:51.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.914+0000 7fb2ef776640 1 Processor -- start 2024-05-14T09:34:51.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.914+0000 7fb2ef776640 1 -- start start 2024-05-14T09:34:51.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.914+0000 7fb2ef776640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e8103c60 0x7fb2e819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:51.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.914+0000 7fb2ef776640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e8104ea0 0x7fb2e819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:51.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.914+0000 7fb2ef776640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2e8109ba0 0x7fb2e81a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:51.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.914+0000 7fb2ef776640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e810f7e0 con 0x7fb2e8109ba0 2024-05-14T09:34:51.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.914+0000 7fb2ef776640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e810f660 con 0x7fb2e8103c60 2024-05-14T09:34:51.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.914+0000 7fb2ef776640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e810f960 con 0x7fb2e8104ea0 2024-05-14T09:34:51.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.914+0000 7fb2edd13640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2e8109ba0 0x7fb2e81a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:51.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.915+0000 7fb2edd13640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2e8109ba0 0x7fb2e81a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54614/0 (socket says 172.21.15.2:54614) 2024-05-14T09:34:51.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.915+0000 7fb2edd13640 1 -- 172.21.15.2:0/1692112742 learned_addr learned my addr 172.21.15.2:0/1692112742 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:51.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.915+0000 7fb2ed512640 1 --2- 172.21.15.2:0/1692112742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e8103c60 0x7fb2e819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:51.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.915+0000 7fb2ecd11640 1 --2- 172.21.15.2:0/1692112742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e8104ea0 0x7fb2e819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:51.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.915+0000 7fb2edd13640 1 -- 172.21.15.2:0/1692112742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e8104ea0 msgr2=0x7fb2e819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:51.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.915+0000 7fb2edd13640 1 --2- 172.21.15.2:0/1692112742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e8104ea0 0x7fb2e819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.915+0000 7fb2edd13640 1 -- 172.21.15.2:0/1692112742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e8103c60 msgr2=0x7fb2e819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:51.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.915+0000 7fb2edd13640 1 --2- 172.21.15.2:0/1692112742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e8103c60 0x7fb2e819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:51.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.915+0000 7fb2edd13640 1 -- 172.21.15.2:0/1692112742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e81a1f20 con 0x7fb2e8109ba0 2024-05-14T09:34:51.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.915+0000 7fb2edd13640 1 --2- 172.21.15.2:0/1692112742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2e8109ba0 0x7fb2e81a18b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000eaa0 tx=0x7fb2e000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:51.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.915+0000 7fb2de7fc640 1 -- 172.21.15.2:0/1692112742 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0004d20 con 0x7fb2e8109ba0 2024-05-14T09:34:51.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.916+0000 7fb2de7fc640 1 -- 172.21.15.2:0/1692112742 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e0005020 con 0x7fb2e8109ba0 2024-05-14T09:34:51.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.916+0000 7fb2ef776640 1 -- 172.21.15.2:0/1692112742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e81a2210 con 0x7fb2e8109ba0 2024-05-14T09:34:51.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.916+0000 7fb2de7fc640 1 -- 172.21.15.2:0/1692112742 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00116b0 con 0x7fb2e8109ba0 2024-05-14T09:34:51.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.916+0000 7fb2ef776640 1 -- 172.21.15.2:0/1692112742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e81a27e0 con 0x7fb2e8109ba0 2024-05-14T09:34:51.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.917+0000 7fb2ef776640 1 -- 172.21.15.2:0/1692112742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e81089b0 con 0x7fb2e8109ba0 2024-05-14T09:34:51.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.918+0000 7fb2de7fc640 1 -- 172.21.15.2:0/1692112742 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fb2e00051d0 con 0x7fb2e8109ba0 2024-05-14T09:34:51.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.919+0000 7fb2de7fc640 1 --2- 172.21.15.2:0/1692112742 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2d40b2870 0x7fb2d40b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:51.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.919+0000 7fb2ed512640 1 --2- 172.21.15.2:0/1692112742 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2d40b2870 0x7fb2d40b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:51.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.919+0000 7fb2de7fc640 1 -- 172.21.15.2:0/1692112742 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fb2e00e2800 con 0x7fb2e8109ba0 2024-05-14T09:34:51.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.919+0000 7fb2ed512640 1 --2- 172.21.15.2:0/1692112742 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2d40b2870 0x7fb2d40b4d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8104860 tx=0x7fb2e4009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:51.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:51.924+0000 7fb2de7fc640 1 -- 172.21.15.2:0/1692112742 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2e0016030 con 0x7fb2e8109ba0 2024-05-14T09:34:52.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:52.131+0000 7fb2ef776640 1 -- 172.21.15.2:0/1692112742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb2e80638d0 con 0x7fb2e8109ba0 2024-05-14T09:34:52.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:52.132+0000 7fb2de7fc640 1 -- 172.21.15.2:0/1692112742 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7fb2e00aa100 con 0x7fb2e8109ba0 2024-05-14T09:34:52.131 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-05-14T09:34:52.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:52.134+0000 7fb2ef776640 1 -- 172.21.15.2:0/1692112742 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2d40b2870 msgr2=0x7fb2d40b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:52.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:52.134+0000 7fb2ef776640 1 --2- 172.21.15.2:0/1692112742 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2d40b2870 0x7fb2d40b4d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8104860 tx=0x7fb2e4009110 comp rx=0 tx=0).stop 2024-05-14T09:34:52.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:52.134+0000 7fb2ef776640 1 -- 172.21.15.2:0/1692112742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2e8109ba0 msgr2=0x7fb2e81a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:52.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:52.134+0000 7fb2ef776640 1 --2- 172.21.15.2:0/1692112742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2e8109ba0 0x7fb2e81a18b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000eaa0 tx=0x7fb2e000eea0 comp rx=0 tx=0).stop 2024-05-14T09:34:52.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:52.134+0000 7fb2ef776640 1 -- 172.21.15.2:0/1692112742 shutdown_connections 2024-05-14T09:34:52.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:52.134+0000 7fb2ef776640 1 --2- 172.21.15.2:0/1692112742 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2d40b2870 0x7fb2d40b4d80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:52.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:52.134+0000 7fb2ef776640 1 --2- 172.21.15.2:0/1692112742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2e8109ba0 0x7fb2e81a18b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:52.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:52.134+0000 7fb2ef776640 1 --2- 172.21.15.2:0/1692112742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2e8104ea0 0x7fb2e819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:52.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:52.134+0000 7fb2ef776640 1 --2- 172.21.15.2:0/1692112742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2e8103c60 0x7fb2e819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:52.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:52.134+0000 7fb2ef776640 1 -- 172.21.15.2:0/1692112742 >> 172.21.15.2:0/1692112742 conn(0x7fb2e80fee70 msgr2=0x7fb2e8105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:52.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:52.134+0000 7fb2ef776640 1 -- 172.21.15.2:0/1692112742 shutdown_connections 2024-05-14T09:34:52.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:52.134+0000 7fb2ef776640 1 -- 172.21.15.2:0/1692112742 wait complete. 2024-05-14T09:34:52.145 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-14T09:34:52.145 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:34:52.396 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:52.677 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:52 smithi002 ceph-mon[31557]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:52.678 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:52 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3853797184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T09:34:52.678 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:52 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/4053206459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T09:34:52.678 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:52 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1692112742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-14T09:34:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:52 smithi029 ceph-mon[36249]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:52 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3853797184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T09:34:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:52 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/4053206459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T09:34:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:52 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1692112742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-14T09:34:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:52 smithi026 ceph-mon[36056]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:34:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:52 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3853797184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T09:34:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:52 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/4053206459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T09:34:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:52 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1692112742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-14T09:34:54.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.150+0000 7fded6acf700 1 -- 172.21.15.2:0/3629322908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fded01045e0 msgr2=0x7fded01049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:54.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.150+0000 7fded6acf700 1 --2- 172.21.15.2:0/3629322908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fded01045e0 0x7fded01049f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdec40099c0 tx=0x7fdec4009cd0 comp rx=0 tx=0).stop 2024-05-14T09:34:54.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.150+0000 7fded6acf700 1 -- 172.21.15.2:0/3629322908 shutdown_connections 2024-05-14T09:34:54.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.150+0000 7fded6acf700 1 --2- 172.21.15.2:0/3629322908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fded0109860 0x7fded01111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:54.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.150+0000 7fded6acf700 1 --2- 172.21.15.2:0/3629322908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fded01045e0 0x7fded01049f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:54.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.150+0000 7fded6acf700 1 --2- 172.21.15.2:0/3629322908 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fded0108f10 0x7fded0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:54.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.150+0000 7fded6acf700 1 -- 172.21.15.2:0/3629322908 >> 172.21.15.2:0/3629322908 conn(0x7fded00ffed0 msgr2=0x7fded0102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.150+0000 7fded6acf700 1 -- 172.21.15.2:0/3629322908 shutdown_connections 2024-05-14T09:34:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.150+0000 7fded6acf700 1 -- 172.21.15.2:0/3629322908 wait complete. 2024-05-14T09:34:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fded6acf700 1 Processor -- start 2024-05-14T09:34:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fded6acf700 1 -- start start 2024-05-14T09:34:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fded6acf700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fded01045e0 0x7fded019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fded6acf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fded0108f10 0x7fded019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fded6acf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fded0109860 0x7fded019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fded6acf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fded0114490 con 0x7fded0108f10 2024-05-14T09:34:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fded6acf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fded0114310 con 0x7fded0109860 2024-05-14T09:34:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fded6acf700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fded0114610 con 0x7fded01045e0 2024-05-14T09:34:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fdecffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fded0108f10 0x7fded019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fdecffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fded0108f10 0x7fded019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54622/0 (socket says 172.21.15.2:54622) 2024-05-14T09:34:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fdecffff700 1 -- 172.21.15.2:0/3684057596 learned_addr learned my addr 172.21.15.2:0/3684057596 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fded486b700 1 --2- 172.21.15.2:0/3684057596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fded01045e0 0x7fded019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fded506c700 1 --2- 172.21.15.2:0/3684057596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fded0109860 0x7fded019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fdecffff700 1 -- 172.21.15.2:0/3684057596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fded01045e0 msgr2=0x7fded019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fdecffff700 1 --2- 172.21.15.2:0/3684057596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fded01045e0 0x7fded019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fdecffff700 1 -- 172.21.15.2:0/3684057596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fded0109860 msgr2=0x7fded019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fdecffff700 1 --2- 172.21.15.2:0/3684057596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fded0109860 0x7fded019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fdecffff700 1 -- 172.21.15.2:0/3684057596 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded019fbe0 con 0x7fded0108f10 2024-05-14T09:34:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.151+0000 7fded506c700 1 --2- 172.21.15.2:0/3684057596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fded0109860 0x7fded019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:34:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.152+0000 7fdecffff700 1 --2- 172.21.15.2:0/3684057596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fded0108f10 0x7fded019b360 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00c9e0 tx=0x7fdebc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.152+0000 7fdecdffb700 1 -- 172.21.15.2:0/3684057596 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc007aa0 con 0x7fded0108f10 2024-05-14T09:34:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.152+0000 7fdecdffb700 1 -- 172.21.15.2:0/3684057596 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdebc007c40 con 0x7fded0108f10 2024-05-14T09:34:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.152+0000 7fdecdffb700 1 -- 172.21.15.2:0/3684057596 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc018660 con 0x7fded0108f10 2024-05-14T09:34:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.152+0000 7fded6acf700 1 -- 172.21.15.2:0/3684057596 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded019fed0 con 0x7fded0108f10 2024-05-14T09:34:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.152+0000 7fded6acf700 1 -- 172.21.15.2:0/3684057596 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded0076b00 con 0x7fded0108f10 2024-05-14T09:34:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.153+0000 7fded6acf700 1 -- 172.21.15.2:0/3684057596 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded0105a90 con 0x7fded0108f10 2024-05-14T09:34:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.155+0000 7fdecdffb700 1 -- 172.21.15.2:0/3684057596 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fdebc018800 con 0x7fded0108f10 2024-05-14T09:34:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.155+0000 7fdecdffb700 1 --2- 172.21.15.2:0/3684057596 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdec00b2900 0x7fdec00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.155+0000 7fded486b700 1 --2- 172.21.15.2:0/3684057596 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdec00b2900 0x7fdec00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.155+0000 7fdecdffb700 1 -- 172.21.15.2:0/3684057596 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fdebc0e2800 con 0x7fded0108f10 2024-05-14T09:34:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.156+0000 7fded486b700 1 --2- 172.21.15.2:0/3684057596 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdec00b2900 0x7fdec00b4e10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009990 tx=0x7fdec4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.159+0000 7fdecdffb700 1 -- 172.21.15.2:0/3684057596 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdebc014030 con 0x7fded0108f10 2024-05-14T09:34:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.419+0000 7fded6acf700 1 -- 172.21.15.2:0/3684057596 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fded019c120 con 0x7fded0108f10 2024-05-14T09:34:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.419+0000 7fdecdffb700 1 -- 172.21.15.2:0/3684057596 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11843 (secure 0 0 0) 0x7fdebc0b2020 con 0x7fded0108f10 2024-05-14T09:34:54.419 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:34:54.420 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T09:34:49.632659+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24487,24497,24524]},"id":1}]} 2024-05-14T09:34:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.422+0000 7fded6acf700 1 -- 172.21.15.2:0/3684057596 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdec00b2900 msgr2=0x7fdec00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.422+0000 7fded6acf700 1 --2- 172.21.15.2:0/3684057596 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdec00b2900 0x7fdec00b4e10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009990 tx=0x7fdec4000bc0 comp rx=0 tx=0).stop 2024-05-14T09:34:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.422+0000 7fded6acf700 1 -- 172.21.15.2:0/3684057596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fded0108f10 msgr2=0x7fded019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.422+0000 7fded6acf700 1 --2- 172.21.15.2:0/3684057596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fded0108f10 0x7fded019b360 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00c9e0 tx=0x7fdebc00ccf0 comp rx=0 tx=0).stop 2024-05-14T09:34:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.422+0000 7fded6acf700 1 -- 172.21.15.2:0/3684057596 shutdown_connections 2024-05-14T09:34:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.422+0000 7fded6acf700 1 --2- 172.21.15.2:0/3684057596 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdec00b2900 0x7fdec00b4e10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.422+0000 7fded6acf700 1 --2- 172.21.15.2:0/3684057596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fded0109860 0x7fded019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.422+0000 7fded6acf700 1 --2- 172.21.15.2:0/3684057596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fded01045e0 0x7fded019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.422+0000 7fded6acf700 1 --2- 172.21.15.2:0/3684057596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fded0108f10 0x7fded019b360 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.422+0000 7fded6acf700 1 -- 172.21.15.2:0/3684057596 >> 172.21.15.2:0/3684057596 conn(0x7fded00ffed0 msgr2=0x7fded01081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:54.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.422+0000 7fded6acf700 1 -- 172.21.15.2:0/3684057596 shutdown_connections 2024-05-14T09:34:54.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:54.422+0000 7fded6acf700 1 -- 172.21.15.2:0/3684057596 wait complete. 2024-05-14T09:34:54.423 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 27 2024-05-14T09:34:54.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:54 smithi002 ceph-mon[31557]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-14T09:34:54.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:54 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3684057596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:34:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:54 smithi029 ceph-mon[36249]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-14T09:34:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:54 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3684057596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:34:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:54 smithi026 ceph-mon[36056]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-14T09:34:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:54 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3684057596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:34:55.169 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-14T09:34:55.169 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-14T09:34:55.169 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T09:34:55.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.292+0000 7ff129661640 1 -- 172.21.15.2:0/3386276962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1241070b0 msgr2=0x7ff1241094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:55.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.292+0000 7ff129661640 1 --2- 172.21.15.2:0/3386276962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1241070b0 0x7ff1241094d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009900 tx=0x7ff10c02a410 comp rx=0 tx=0).stop 2024-05-14T09:34:55.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.292+0000 7ff129661640 1 -- 172.21.15.2:0/3386276962 shutdown_connections 2024-05-14T09:34:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.292+0000 7ff129661640 1 --2- 172.21.15.2:0/3386276962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff124109a10 0x7ff12410be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.292+0000 7ff129661640 1 --2- 172.21.15.2:0/3386276962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1241070b0 0x7ff1241094d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.292+0000 7ff129661640 1 --2- 172.21.15.2:0/3386276962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff124104750 0x7ff124106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.292+0000 7ff129661640 1 -- 172.21.15.2:0/3386276962 >> 172.21.15.2:0/3386276962 conn(0x7ff1240fdd70 msgr2=0x7ff124100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.292+0000 7ff129661640 1 -- 172.21.15.2:0/3386276962 shutdown_connections 2024-05-14T09:34:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.292+0000 7ff129661640 1 -- 172.21.15.2:0/3386276962 wait complete. 2024-05-14T09:34:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.293+0000 7ff129661640 1 Processor -- start 2024-05-14T09:34:55.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.293+0000 7ff129661640 1 -- start start 2024-05-14T09:34:55.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.293+0000 7ff129661640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff124104750 0x7ff12419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:55.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.293+0000 7ff129661640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1241070b0 0x7ff12419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.293+0000 7ff129661640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff124109a10 0x7ff1241a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.293+0000 7ff129661640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff12410f700 con 0x7ff124109a10 2024-05-14T09:34:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.293+0000 7ff129661640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff12410f580 con 0x7ff1241070b0 2024-05-14T09:34:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.293+0000 7ff129661640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff12410f880 con 0x7ff124104750 2024-05-14T09:34:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.293+0000 7ff1237fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff124109a10 0x7ff1241a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.293+0000 7ff1237fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff124109a10 0x7ff1241a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54636/0 (socket says 172.21.15.2:54636) 2024-05-14T09:34:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.293+0000 7ff1237fe640 1 -- 172.21.15.2:0/2915370269 learned_addr learned my addr 172.21.15.2:0/2915370269 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.293+0000 7ff1227fc640 1 --2- 172.21.15.2:0/2915370269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1241070b0 0x7ff12419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.294+0000 7ff122ffd640 1 --2- 172.21.15.2:0/2915370269 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff124104750 0x7ff12419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.294+0000 7ff1237fe640 1 -- 172.21.15.2:0/2915370269 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff124104750 msgr2=0x7ff12419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.294+0000 7ff1237fe640 1 --2- 172.21.15.2:0/2915370269 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff124104750 0x7ff12419a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.294+0000 7ff1237fe640 1 -- 172.21.15.2:0/2915370269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1241070b0 msgr2=0x7ff12419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.294+0000 7ff1237fe640 1 --2- 172.21.15.2:0/2915370269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1241070b0 0x7ff12419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.294+0000 7ff1237fe640 1 -- 172.21.15.2:0/2915370269 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1241a1d60 con 0x7ff124109a10 2024-05-14T09:34:55.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.294+0000 7ff122ffd640 1 --2- 172.21.15.2:0/2915370269 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff124104750 0x7ff12419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:34:55.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.294+0000 7ff1237fe640 1 --2- 172.21.15.2:0/2915370269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff124109a10 0x7ff1241a16f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff11400e970 tx=0x7ff11400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:55.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.294+0000 7ff11bfff640 1 -- 172.21.15.2:0/2915370269 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11400ccc0 con 0x7ff124109a10 2024-05-14T09:34:55.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.294+0000 7ff11bfff640 1 -- 172.21.15.2:0/2915370269 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1140054d0 con 0x7ff124109a10 2024-05-14T09:34:55.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.294+0000 7ff129661640 1 -- 172.21.15.2:0/2915370269 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1241a1ff0 con 0x7ff124109a10 2024-05-14T09:34:55.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.294+0000 7ff11bfff640 1 -- 172.21.15.2:0/2915370269 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114010430 con 0x7ff124109a10 2024-05-14T09:34:55.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.294+0000 7ff129661640 1 -- 172.21.15.2:0/2915370269 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1241a24b0 con 0x7ff124109a10 2024-05-14T09:34:55.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.295+0000 7ff129661640 1 -- 172.21.15.2:0/2915370269 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1240ff890 con 0x7ff124109a10 2024-05-14T09:34:55.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.297+0000 7ff11bfff640 1 -- 172.21.15.2:0/2915370269 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff114004fe0 con 0x7ff124109a10 2024-05-14T09:34:55.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.298+0000 7ff11bfff640 1 --2- 172.21.15.2:0/2915370269 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff1100b2920 0x7ff1100b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:55.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.298+0000 7ff11bfff640 1 -- 172.21.15.2:0/2915370269 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7ff1140e2c20 con 0x7ff124109a10 2024-05-14T09:34:55.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.298+0000 7ff122ffd640 1 --2- 172.21.15.2:0/2915370269 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff1100b2920 0x7ff1100b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:55.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.298+0000 7ff122ffd640 1 --2- 172.21.15.2:0/2915370269 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff1100b2920 0x7ff1100b4e30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff12406ab50 tx=0x7ff11c009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:55.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.301+0000 7ff11bfff640 1 -- 172.21.15.2:0/2915370269 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff114054df0 con 0x7ff124109a10 2024-05-14T09:34:55.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.489+0000 7ff129661640 1 -- 172.21.15.2:0/2915370269 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff124063110 con 0x7ff1100b2920 2024-05-14T09:34:55.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.498+0000 7ff11bfff640 1 -- 172.21.15.2:0/2915370269 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340740 (secure 0 0 0) 0x7ff124063110 con 0x7ff1100b2920 2024-05-14T09:34:55.503 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:34:55.505 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-05-14T09:34:55.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.502+0000 7ff129661640 1 -- 172.21.15.2:0/2915370269 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff1100b2920 msgr2=0x7ff1100b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:55.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.502+0000 7ff129661640 1 --2- 172.21.15.2:0/2915370269 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff1100b2920 0x7ff1100b4e30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff12406ab50 tx=0x7ff11c009520 comp rx=0 tx=0).stop 2024-05-14T09:34:55.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.502+0000 7ff129661640 1 -- 172.21.15.2:0/2915370269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff124109a10 msgr2=0x7ff1241a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:55.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.502+0000 7ff129661640 1 --2- 172.21.15.2:0/2915370269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff124109a10 0x7ff1241a16f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff11400e970 tx=0x7ff11400ee30 comp rx=0 tx=0).stop 2024-05-14T09:34:55.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.502+0000 7ff129661640 1 -- 172.21.15.2:0/2915370269 shutdown_connections 2024-05-14T09:34:55.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.502+0000 7ff129661640 1 --2- 172.21.15.2:0/2915370269 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff1100b2920 0x7ff1100b4e30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.502+0000 7ff129661640 1 --2- 172.21.15.2:0/2915370269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff124109a10 0x7ff1241a16f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.502+0000 7ff129661640 1 --2- 172.21.15.2:0/2915370269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1241070b0 0x7ff12419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.502+0000 7ff129661640 1 --2- 172.21.15.2:0/2915370269 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff124104750 0x7ff12419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.502+0000 7ff129661640 1 -- 172.21.15.2:0/2915370269 >> 172.21.15.2:0/2915370269 conn(0x7ff1240fdd70 msgr2=0x7ff124105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:55.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.503+0000 7ff129661640 1 -- 172.21.15.2:0/2915370269 shutdown_connections 2024-05-14T09:34:55.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.503+0000 7ff129661640 1 -- 172.21.15.2:0/2915370269 wait complete. 2024-05-14T09:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-05-14T09:34:55.107700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659669,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1155,"num_read_kb":677,"num_write":308,"num_write_kb":2281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2954686,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":750,"ondisk_log_size":750,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":394500,"kb_used_data":12068,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124547324,"statfs":{"total":1151940427776,"available":1151536459776,"internally_reserved":0,"allocated":12357632,"data_stored":8176157,"data_compressed":496431,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19140,"internal_metadata":391492924},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":32,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003660"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087224+0000","last_change":"2024-05-14T09:30:56.797051+0000","last_active":"2024-05-14T09:33:29.087224+0000","last_peered":"2024-05-14T09:33:29.087224+0000","last_clean":"2024-05-14T09:33:29.087224+0000","last_became_active":"2024-05-14T09:30:56.796517+0000","last_became_peered":"2024-05-14T09:30:56.796517+0000","last_unstale":"2024-05-14T09:33:29.087224+0000","last_undegraded":"2024-05-14T09:33:29.087224+0000","last_fullsized":"2024-05-14T09:33:29.087224+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:58:22.170187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.073126+0000","last_change":"2024-05-14T09:30:56.795325+0000","last_active":"2024-05-14T09:33:29.073126+0000","last_peered":"2024-05-14T09:33:29.073126+0000","last_clean":"2024-05-14T09:33:29.073126+0000","last_became_active":"2024-05-14T09:30:56.794962+0000","last_became_peered":"2024-05-14T09:30:56.794962+0000","last_unstale":"2024-05-14T09:33:29.073126+0000","last_undegraded":"2024-05-14T09:33:29.073126+0000","last_fullsized":"2024-05-14T09:33:29.073126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:15:28.890565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084179+0000","last_change":"2024-05-14T09:30:56.793200+0000","last_active":"2024-05-14T09:33:29.084179+0000","last_peered":"2024-05-14T09:33:29.084179+0000","last_clean":"2024-05-14T09:33:29.084179+0000","last_became_active":"2024-05-14T09:30:56.792835+0000","last_became_peered":"2024-05-14T09:30:56.792835+0000","last_unstale":"2024-05-14T09:33:29.084179+0000","last_undegraded":"2024-05-14T09:33:29.084179+0000","last_fullsized":"2024-05-14T09:33:29.084179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:19.851276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.070997+0000","last_change":"2024-05-14T09:30:56.790238+0000","last_active":"2024-05-14T09:33:29.070997+0000","last_peered":"2024-05-14T09:33:29.070997+0000","last_clean":"2024-05-14T09:33:29.070997+0000","last_became_active":"2024-05-14T09:30:56.789826+0000","last_became_peered":"2024-05-14T09:30:56.789826+0000","last_unstale":"2024-05-14T09:33:29.070997+0000","last_undegraded":"2024-05-14T09:33:29.070997+0000","last_fullsized":"2024-05-14T09:33:29.070997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:19:19.853931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080306+0000","last_change":"2024-05-14T09:30:56.788021+0000","last_active":"2024-05-14T09:33:29.080306+0000","last_peered":"2024-05-14T09:33:29.080306+0000","last_clean":"2024-05-14T09:33:29.080306+0000","last_became_active":"2024-05-14T09:30:56.787657+0000","last_became_peered":"2024-05-14T09:30:56.787657+0000","last_unstale":"2024-05-14T09:33:29.080306+0000","last_undegraded":"2024-05-14T09:33:29.080306+0000","last_fullsized":"2024-05-14T09:33:29.080306+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:37:03.232137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.082547+0000","last_change":"2024-05-14T09:32:30.525175+0000","last_active":"2024-05-14T09:33:29.082547+0000","last_peered":"2024-05-14T09:33:29.082547+0000","last_clean":"2024-05-14T09:33:29.082547+0000","last_became_active":"2024-05-14T09:32:30.524861+0000","last_became_peered":"2024-05-14T09:32:30.524861+0000","last_unstale":"2024-05-14T09:33:29.082547+0000","last_undegraded":"2024-05-14T09:33:29.082547+0000","last_fullsized":"2024-05-14T09:33:29.082547+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:22:21.063843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.066050+0000","last_change":"2024-05-14T09:30:56.795647+0000","last_active":"2024-05-14T09:33:29.066050+0000","last_peered":"2024-05-14T09:33:29.066050+0000","last_clean":"2024-05-14T09:33:29.066050+0000","last_became_active":"2024-05-14T09:30:56.795186+0000","last_became_peered":"2024-05-14T09:30:56.795186+0000","last_unstale":"2024-05-14T09:33:29.066050+0000","last_undegraded":"2024-05-14T09:33:29.066050+0000","last_fullsized":"2024-05-14T09:33:29.066050+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:04:53.471305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081501+0000","last_change":"2024-05-14T09:30:56.796834+0000","last_active":"2024-05-14T09:33:29.081501+0000","last_peered":"2024-05-14T09:33:29.081501+0000","last_clean":"2024-05-14T09:33:29.081501+0000","last_became_active":"2024-05-14T09:30:56.796492+0000","last_became_peered":"2024-05-14T09:30:56.796492+0000","last_unstale":"2024-05-14T09:33:29.081501+0000","last_undegraded":"2024-05-14T09:33:29.081501+0000","last_fullsized":"2024-05-14T09:33:29.081501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:46.606129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.086070+0000","last_change":"2024-05-14T09:30:56.795020+0000","last_active":"2024-05-14T09:33:29.086070+0000","last_peered":"2024-05-14T09:33:29.086070+0000","last_clean":"2024-05-14T09:33:29.086070+0000","last_became_active":"2024-05-14T09:30:56.794625+0000","last_became_peered":"2024-05-14T09:30:56.794625+0000","last_unstale":"2024-05-14T09:33:29.086070+0000","last_undegraded":"2024-05-14T09:33:29.086070+0000","last_fullsized":"2024-05-14T09:33:29.086070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:09:38.597296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.082128+0000","last_change":"2024-05-14T09:30:56.796439+0000","last_active":"2024-05-14T09:33:29.082128+0000","last_peered":"2024-05-14T09:33:29.082128+0000","last_clean":"2024-05-14T09:33:29.082128+0000","last_became_active":"2024-05-14T09:30:56.795168+0000","last_became_peered":"2024-05-14T09:30:56.795168+0000","last_unstale":"2024-05-14T09:33:29.082128+0000","last_undegraded":"2024-05-14T09:33:29.082128+0000","last_fullsized":"2024-05-14T09:33:29.082128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:47.660286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087219+0000","last_change":"2024-05-14T09:30:56.795994+0000","last_active":"2024-05-14T09:33:29.087219+0000","last_peered":"2024-05-14T09:33:29.087219+0000","last_clean":"2024-05-14T09:33:29.087219+0000","last_became_active":"2024-05-14T09:30:56.795079+0000","last_became_peered":"2024-05-14T09:30:56.795079+0000","last_unstale":"2024-05-14T09:33:29.087219+0000","last_undegraded":"2024-05-14T09:33:29.087219+0000","last_fullsized":"2024-05-14T09:33:29.087219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:37:42.930195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.072741+0000","last_change":"2024-05-14T09:30:56.792997+0000","last_active":"2024-05-14T09:33:29.072741+0000","last_peered":"2024-05-14T09:33:29.072741+0000","last_clean":"2024-05-14T09:33:29.072741+0000","last_became_active":"2024-05-14T09:30:56.792302+0000","last_became_peered":"2024-05-14T09:30:56.792302+0000","last_unstale":"2024-05-14T09:33:29.072741+0000","last_undegraded":"2024-05-14T09:33:29.072741+0000","last_fullsized":"2024-05-14T09:33:29.072741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:52.986662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077933+0000","last_change":"2024-05-14T09:30:56.795903+0000","last_active":"2024-05-14T09:33:29.077933+0000","last_peered":"2024-05-14T09:33:29.077933+0000","last_clean":"2024-05-14T09:33:29.077933+0000","last_became_active":"2024-05-14T09:30:56.795466+0000","last_became_peered":"2024-05-14T09:30:56.795466+0000","last_unstale":"2024-05-14T09:33:29.077933+0000","last_undegraded":"2024-05-14T09:33:29.077933+0000","last_fullsized":"2024-05-14T09:33:29.077933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:02:10.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.070939+0000","last_change":"2024-05-14T09:30:56.795911+0000","last_active":"2024-05-14T09:33:29.070939+0000","last_peered":"2024-05-14T09:33:29.070939+0000","last_clean":"2024-05-14T09:33:29.070939+0000","last_became_active":"2024-05-14T09:30:56.795547+0000","last_became_peered":"2024-05-14T09:30:56.795547+0000","last_unstale":"2024-05-14T09:33:29.070939+0000","last_undegraded":"2024-05-14T09:33:29.070939+0000","last_fullsized":"2024-05-14T09:33:29.070939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:56:29.621783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.067755+0000","last_change":"2024-05-14T09:30:56.787745+0000","last_active":"2024-05-14T09:33:29.067755+0000","last_peered":"2024-05-14T09:33:29.067755+0000","last_clean":"2024-05-14T09:33:29.067755+0000","last_became_active":"2024-05-14T09:30:56.787385+0000","last_became_peered":"2024-05-14T09:30:56.787385+0000","last_unstale":"2024-05-14T09:33:29.067755+0000","last_undegraded":"2024-05-14T09:33:29.067755+0000","last_fullsized":"2024-05-14T09:33:29.067755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:11.256775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087830+0000","last_change":"2024-05-14T09:30:56.795785+0000","last_active":"2024-05-14T09:33:29.087830+0000","last_peered":"2024-05-14T09:33:29.087830+0000","last_clean":"2024-05-14T09:33:29.087830+0000","last_became_active":"2024-05-14T09:30:56.795436+0000","last_became_peered":"2024-05-14T09:30:56.795436+0000","last_unstale":"2024-05-14T09:33:29.087830+0000","last_undegraded":"2024-05-14T09:33:29.087830+0000","last_fullsized":"2024-05-14T09:33:29.087830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:12:09.221929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087321+0000","last_change":"2024-05-14T09:30:56.800688+0000","last_active":"2024-05-14T09:33:29.087321+0000","last_peered":"2024-05-14T09:33:29.087321+0000","last_clean":"2024-05-14T09:33:29.087321+0000","last_became_active":"2024-05-14T09:30:56.800204+0000","last_became_peered":"2024-05-14T09:30:56.800204+0000","last_unstale":"2024-05-14T09:33:29.087321+0000","last_undegraded":"2024-05-14T09:33:29.087321+0000","last_fullsized":"2024-05-14T09:33:29.087321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:32:27.057688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.075054+0000","last_change":"2024-05-14T09:30:56.795604+0000","last_active":"2024-05-14T09:33:29.075054+0000","last_peered":"2024-05-14T09:33:29.075054+0000","last_clean":"2024-05-14T09:33:29.075054+0000","last_became_active":"2024-05-14T09:30:56.795245+0000","last_became_peered":"2024-05-14T09:30:56.795245+0000","last_unstale":"2024-05-14T09:33:29.075054+0000","last_undegraded":"2024-05-14T09:33:29.075054+0000","last_fullsized":"2024-05-14T09:33:29.075054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:14.072584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080216+0000","last_change":"2024-05-14T09:30:56.794815+0000","last_active":"2024-05-14T09:33:29.080216+0000","last_peered":"2024-05-14T09:33:29.080216+0000","last_clean":"2024-05-14T09:33:29.080216+0000","last_became_active":"2024-05-14T09:30:56.794429+0000","last_became_peered":"2024-05-14T09:30:56.794429+0000","last_unstale":"2024-05-14T09:33:29.080216+0000","last_undegraded":"2024-05-14T09:33:29.080216+0000","last_fullsized":"2024-05-14T09:33:29.080216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:32:08.367046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319506+0000","last_change":"2024-05-14T09:31:08.107895+0000","last_active":"2024-05-14T09:32:30.319506+0000","last_peered":"2024-05-14T09:32:30.319506+0000","last_clean":"2024-05-14T09:32:30.319506+0000","last_became_active":"2024-05-14T09:31:08.107458+0000","last_became_peered":"2024-05-14T09:31:08.107458+0000","last_unstale":"2024-05-14T09:32:30.319506+0000","last_undegraded":"2024-05-14T09:32:30.319506+0000","last_fullsized":"2024-05-14T09:32:30.319506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:13:11.717753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":69,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319382+0000","last_change":"2024-05-14T09:31:30.178197+0000","last_active":"2024-05-14T09:32:30.319382+0000","last_peered":"2024-05-14T09:32:30.319382+0000","last_clean":"2024-05-14T09:32:30.319382+0000","last_became_active":"2024-05-14T09:31:30.171154+0000","last_became_peered":"2024-05-14T09:31:30.171154+0000","last_unstale":"2024-05-14T09:32:30.319382+0000","last_undegraded":"2024-05-14T09:32:30.319382+0000","last_fullsized":"2024-05-14T09:32:30.319382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:43:46.208439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077181+0000","last_change":"2024-05-14T09:30:56.800526+0000","last_active":"2024-05-14T09:33:29.077181+0000","last_peered":"2024-05-14T09:33:29.077181+0000","last_clean":"2024-05-14T09:33:29.077181+0000","last_became_active":"2024-05-14T09:30:56.797692+0000","last_became_peered":"2024-05-14T09:30:56.797692+0000","last_unstale":"2024-05-14T09:33:29.077181+0000","last_undegraded":"2024-05-14T09:33:29.077181+0000","last_fullsized":"2024-05-14T09:33:29.077181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:09:14.471173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522407+0000","last_change":"2024-05-14T09:31:30.173810+0000","last_active":"2024-05-14T09:32:30.522407+0000","last_peered":"2024-05-14T09:32:30.522407+0000","last_clean":"2024-05-14T09:32:30.522407+0000","last_became_active":"2024-05-14T09:31:30.173242+0000","last_became_peered":"2024-05-14T09:31:30.173242+0000","last_unstale":"2024-05-14T09:32:30.522407+0000","last_undegraded":"2024-05-14T09:32:30.522407+0000","last_fullsized":"2024-05-14T09:32:30.522407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:37:15.991805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.322883+0000","last_change":"2024-05-14T09:30:54.015636+0000","last_active":"2024-05-14T09:32:30.322883+0000","last_peered":"2024-05-14T09:32:30.322883+0000","last_clean":"2024-05-14T09:32:30.322883+0000","last_became_active":"2024-05-14T09:30:54.015290+0000","last_became_peered":"2024-05-14T09:30:54.015290+0000","last_unstale":"2024-05-14T09:32:30.322883+0000","last_undegraded":"2024-05-14T09:32:30.322883+0000","last_fullsized":"2024-05-14T09:32:30.322883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:29:17.548817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.088634+0000","last_change":"2024-05-14T09:30:56.799506+0000","last_active":"2024-05-14T09:33:29.088634+0000","last_peered":"2024-05-14T09:33:29.088634+0000","last_clean":"2024-05-14T09:33:29.088634+0000","last_became_active":"2024-05-14T09:30:56.799153+0000","last_became_peered":"2024-05-14T09:30:56.799153+0000","last_unstale":"2024-05-14T09:33:29.088634+0000","last_undegraded":"2024-05-14T09:33:29.088634+0000","last_fullsized":"2024-05-14T09:33:29.088634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:04:08.217337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522681+0000","last_change":"2024-05-14T09:31:08.115533+0000","last_active":"2024-05-14T09:32:30.522681+0000","last_peered":"2024-05-14T09:32:30.522681+0000","last_clean":"2024-05-14T09:32:30.522681+0000","last_became_active":"2024-05-14T09:31:08.115033+0000","last_became_peered":"2024-05-14T09:31:08.115033+0000","last_unstale":"2024-05-14T09:32:30.522681+0000","last_undegraded":"2024-05-14T09:32:30.522681+0000","last_fullsized":"2024-05-14T09:32:30.522681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:15.668357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524484+0000","last_change":"2024-05-14T09:30:53.770547+0000","last_active":"2024-05-14T09:32:30.524484+0000","last_peered":"2024-05-14T09:32:30.524484+0000","last_clean":"2024-05-14T09:32:30.524484+0000","last_became_active":"2024-05-14T09:30:53.770110+0000","last_became_peered":"2024-05-14T09:30:53.770110+0000","last_unstale":"2024-05-14T09:32:30.524484+0000","last_undegraded":"2024-05-14T09:32:30.524484+0000","last_fullsized":"2024-05-14T09:32:30.524484+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:18.371318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084457+0000","last_change":"2024-05-14T09:30:56.799427+0000","last_active":"2024-05-14T09:33:29.084457+0000","last_peered":"2024-05-14T09:33:29.084457+0000","last_clean":"2024-05-14T09:33:29.084457+0000","last_became_active":"2024-05-14T09:30:56.799018+0000","last_became_peered":"2024-05-14T09:30:56.799018+0000","last_unstale":"2024-05-14T09:33:29.084457+0000","last_undegraded":"2024-05-14T09:33:29.084457+0000","last_fullsized":"2024-05-14T09:33:29.084457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:13:31.837985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522635+0000","last_change":"2024-05-14T09:31:08.116258+0000","last_active":"2024-05-14T09:32:30.522635+0000","last_peered":"2024-05-14T09:32:30.522635+0000","last_clean":"2024-05-14T09:32:30.522635+0000","last_became_active":"2024-05-14T09:31:08.115040+0000","last_became_peered":"2024-05-14T09:31:08.115040+0000","last_unstale":"2024-05-14T09:32:30.522635+0000","last_undegraded":"2024-05-14T09:32:30.522635+0000","last_fullsized":"2024-05-14T09:32:30.522635+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:57:54.839203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523793+0000","last_change":"2024-05-14T09:31:30.188984+0000","last_active":"2024-05-14T09:32:30.523793+0000","last_peered":"2024-05-14T09:32:30.523793+0000","last_clean":"2024-05-14T09:32:30.523793+0000","last_became_active":"2024-05-14T09:31:30.174410+0000","last_became_peered":"2024-05-14T09:31:30.174410+0000","last_unstale":"2024-05-14T09:32:30.523793+0000","last_undegraded":"2024-05-14T09:32:30.523793+0000","last_fullsized":"2024-05-14T09:32:30.523793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:01:35.713857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,4],"acting":[8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081749+0000","last_change":"2024-05-14T09:32:30.319390+0000","last_active":"2024-05-14T09:33:29.081749+0000","last_peered":"2024-05-14T09:33:29.081749+0000","last_clean":"2024-05-14T09:33:29.081749+0000","last_became_active":"2024-05-14T09:32:30.318645+0000","last_became_peered":"2024-05-14T09:32:30.318645+0000","last_unstale":"2024-05-14T09:33:29.081749+0000","last_undegraded":"2024-05-14T09:33:29.081749+0000","last_fullsized":"2024-05-14T09:33:29.081749+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:37:49.866996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320705+0000","last_change":"2024-05-14T09:31:08.115740+0000","last_active":"2024-05-14T09:32:30.320705+0000","last_peered":"2024-05-14T09:32:30.320705+0000","last_clean":"2024-05-14T09:32:30.320705+0000","last_became_active":"2024-05-14T09:31:08.115202+0000","last_became_peered":"2024-05-14T09:31:08.115202+0000","last_unstale":"2024-05-14T09:32:30.320705+0000","last_undegraded":"2024-05-14T09:32:30.320705+0000","last_fullsized":"2024-05-14T09:32:30.320705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:44.519280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":233,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:34:51.508971+0000","last_change":"2024-05-14T09:30:53.770869+0000","last_active":"2024-05-14T09:34:51.508971+0000","last_peered":"2024-05-14T09:34:51.508971+0000","last_clean":"2024-05-14T09:34:51.508971+0000","last_became_active":"2024-05-14T09:30:53.770425+0000","last_became_peered":"2024-05-14T09:30:53.770425+0000","last_unstale":"2024-05-14T09:34:51.508971+0000","last_undegraded":"2024-05-14T09:34:51.508971+0000","last_fullsized":"2024-05-14T09:34:51.508971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:03:33.768120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":180,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.059353+0000","last_change":"2024-05-14T09:30:56.789267+0000","last_active":"2024-05-14T09:33:29.059353+0000","last_peered":"2024-05-14T09:33:29.059353+0000","last_clean":"2024-05-14T09:33:29.059353+0000","last_became_active":"2024-05-14T09:30:56.788883+0000","last_became_peered":"2024-05-14T09:30:56.788883+0000","last_unstale":"2024-05-14T09:33:29.059353+0000","last_undegraded":"2024-05-14T09:33:29.059353+0000","last_fullsized":"2024-05-14T09:33:29.059353+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:09.818693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320595+0000","last_change":"2024-05-14T09:31:08.115193+0000","last_active":"2024-05-14T09:32:30.320595+0000","last_peered":"2024-05-14T09:32:30.320595+0000","last_clean":"2024-05-14T09:32:30.320595+0000","last_became_active":"2024-05-14T09:31:08.114701+0000","last_became_peered":"2024-05-14T09:31:08.114701+0000","last_unstale":"2024-05-14T09:32:30.320595+0000","last_undegraded":"2024-05-14T09:32:30.320595+0000","last_fullsized":"2024-05-14T09:32:30.320595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:47:48.487922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319686+0000","last_change":"2024-05-14T09:31:30.174760+0000","last_active":"2024-05-14T09:32:30.319686+0000","last_peered":"2024-05-14T09:32:30.319686+0000","last_clean":"2024-05-14T09:32:30.319686+0000","last_became_active":"2024-05-14T09:31:30.174253+0000","last_became_peered":"2024-05-14T09:31:30.174253+0000","last_unstale":"2024-05-14T09:32:30.319686+0000","last_undegraded":"2024-05-14T09:32:30.319686+0000","last_fullsized":"2024-05-14T09:32:30.319686+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:11:17.163436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081148+0000","last_change":"2024-05-14T09:30:56.794654+0000","last_active":"2024-05-14T09:33:29.081148+0000","last_peered":"2024-05-14T09:33:29.081148+0000","last_clean":"2024-05-14T09:33:29.081148+0000","last_became_active":"2024-05-14T09:30:56.794282+0000","last_became_peered":"2024-05-14T09:30:56.794282+0000","last_unstale":"2024-05-14T09:33:29.081148+0000","last_undegraded":"2024-05-14T09:33:29.081148+0000","last_fullsized":"2024-05-14T09:33:29.081148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:55:41.478707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320586+0000","last_change":"2024-05-14T09:31:08.119965+0000","last_active":"2024-05-14T09:32:30.320586+0000","last_peered":"2024-05-14T09:32:30.320586+0000","last_clean":"2024-05-14T09:32:30.320586+0000","last_became_active":"2024-05-14T09:31:08.119502+0000","last_became_peered":"2024-05-14T09:31:08.119502+0000","last_unstale":"2024-05-14T09:32:30.320586+0000","last_undegraded":"2024-05-14T09:32:30.320586+0000","last_fullsized":"2024-05-14T09:32:30.320586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:01:23.612036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.314804+0000","last_change":"2024-05-14T09:30:53.766495+0000","last_active":"2024-05-14T09:32:30.314804+0000","last_peered":"2024-05-14T09:32:30.314804+0000","last_clean":"2024-05-14T09:32:30.314804+0000","last_became_active":"2024-05-14T09:30:53.766122+0000","last_became_peered":"2024-05-14T09:30:53.766122+0000","last_unstale":"2024-05-14T09:32:30.314804+0000","last_undegraded":"2024-05-14T09:32:30.314804+0000","last_fullsized":"2024-05-14T09:32:30.314804+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:43:13.344209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.073417+0000","last_change":"2024-05-14T09:30:56.791938+0000","last_active":"2024-05-14T09:33:29.073417+0000","last_peered":"2024-05-14T09:33:29.073417+0000","last_clean":"2024-05-14T09:33:29.073417+0000","last_became_active":"2024-05-14T09:30:56.791525+0000","last_became_peered":"2024-05-14T09:30:56.791525+0000","last_unstale":"2024-05-14T09:33:29.073417+0000","last_undegraded":"2024-05-14T09:33:29.073417+0000","last_fullsized":"2024-05-14T09:33:29.073417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:18.178208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320104+0000","last_change":"2024-05-14T09:31:08.113548+0000","last_active":"2024-05-14T09:32:30.320104+0000","last_peered":"2024-05-14T09:32:30.320104+0000","last_clean":"2024-05-14T09:32:30.320104+0000","last_became_active":"2024-05-14T09:31:08.113117+0000","last_became_peered":"2024-05-14T09:31:08.113117+0000","last_unstale":"2024-05-14T09:32:30.320104+0000","last_undegraded":"2024-05-14T09:32:30.320104+0000","last_fullsized":"2024-05-14T09:32:30.320104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:25:15.483761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315966+0000","last_change":"2024-05-14T09:30:53.770987+0000","last_active":"2024-05-14T09:32:30.315966+0000","last_peered":"2024-05-14T09:32:30.315966+0000","last_clean":"2024-05-14T09:32:30.315966+0000","last_became_active":"2024-05-14T09:30:53.770609+0000","last_became_peered":"2024-05-14T09:30:53.770609+0000","last_unstale":"2024-05-14T09:32:30.315966+0000","last_undegraded":"2024-05-14T09:32:30.315966+0000","last_fullsized":"2024-05-14T09:32:30.315966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:26:22.020642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081149+0000","last_change":"2024-05-14T09:30:56.789219+0000","last_active":"2024-05-14T09:33:29.081149+0000","last_peered":"2024-05-14T09:33:29.081149+0000","last_clean":"2024-05-14T09:33:29.081149+0000","last_became_active":"2024-05-14T09:30:56.788871+0000","last_became_peered":"2024-05-14T09:30:56.788871+0000","last_unstale":"2024-05-14T09:33:29.081149+0000","last_undegraded":"2024-05-14T09:33:29.081149+0000","last_fullsized":"2024-05-14T09:33:29.081149+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:10:50.774267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.322531+0000","last_change":"2024-05-14T09:31:30.174708+0000","last_active":"2024-05-14T09:32:30.322531+0000","last_peered":"2024-05-14T09:32:30.322531+0000","last_clean":"2024-05-14T09:32:30.322531+0000","last_became_active":"2024-05-14T09:31:30.174199+0000","last_became_peered":"2024-05-14T09:31:30.174199+0000","last_unstale":"2024-05-14T09:32:30.322531+0000","last_undegraded":"2024-05-14T09:32:30.322531+0000","last_fullsized":"2024-05-14T09:32:30.322531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:23:55.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":225,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:34:51.508350+0000","last_change":"2024-05-14T09:30:53.770898+0000","last_active":"2024-05-14T09:34:51.508350+0000","last_peered":"2024-05-14T09:34:51.508350+0000","last_clean":"2024-05-14T09:34:51.508350+0000","last_became_active":"2024-05-14T09:30:53.770483+0000","last_became_peered":"2024-05-14T09:30:53.770483+0000","last_unstale":"2024-05-14T09:34:51.508350+0000","last_undegraded":"2024-05-14T09:34:51.508350+0000","last_fullsized":"2024-05-14T09:34:51.508350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:07.764348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084404+0000","last_change":"2024-05-14T09:30:56.799049+0000","last_active":"2024-05-14T09:33:29.084404+0000","last_peered":"2024-05-14T09:33:29.084404+0000","last_clean":"2024-05-14T09:33:29.084404+0000","last_became_active":"2024-05-14T09:30:56.798643+0000","last_became_peered":"2024-05-14T09:30:56.798643+0000","last_unstale":"2024-05-14T09:33:29.084404+0000","last_undegraded":"2024-05-14T09:33:29.084404+0000","last_fullsized":"2024-05-14T09:33:29.084404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:12.378373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.318070+0000","last_change":"2024-05-14T09:31:08.109876+0000","last_active":"2024-05-14T09:32:30.318070+0000","last_peered":"2024-05-14T09:32:30.318070+0000","last_clean":"2024-05-14T09:32:30.318070+0000","last_became_active":"2024-05-14T09:31:08.109470+0000","last_became_peered":"2024-05-14T09:31:08.109470+0000","last_unstale":"2024-05-14T09:32:30.318070+0000","last_undegraded":"2024-05-14T09:32:30.318070+0000","last_fullsized":"2024-05-14T09:32:30.318070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:07.704153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.015003+0000","last_change":"2024-05-14T09:31:30.179086+0000","last_active":"2024-05-14T09:32:31.015003+0000","last_peered":"2024-05-14T09:32:31.015003+0000","last_clean":"2024-05-14T09:32:31.015003+0000","last_became_active":"2024-05-14T09:31:30.178604+0000","last_became_peered":"2024-05-14T09:31:30.178604+0000","last_unstale":"2024-05-14T09:32:31.015003+0000","last_undegraded":"2024-05-14T09:32:31.015003+0000","last_fullsized":"2024-05-14T09:32:31.015003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:39.712395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.073315+0000","last_change":"2024-05-14T09:30:56.793109+0000","last_active":"2024-05-14T09:33:29.073315+0000","last_peered":"2024-05-14T09:33:29.073315+0000","last_clean":"2024-05-14T09:33:29.073315+0000","last_became_active":"2024-05-14T09:30:56.792761+0000","last_became_peered":"2024-05-14T09:30:56.792761+0000","last_unstale":"2024-05-14T09:33:29.073315+0000","last_undegraded":"2024-05-14T09:33:29.073315+0000","last_fullsized":"2024-05-14T09:33:29.073315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:25:33.216281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522916+0000","last_change":"2024-05-14T09:31:08.120978+0000","last_active":"2024-05-14T09:32:30.522916+0000","last_peered":"2024-05-14T09:32:30.522916+0000","last_clean":"2024-05-14T09:32:30.522916+0000","last_became_active":"2024-05-14T09:31:08.120227+0000","last_became_peered":"2024-05-14T09:31:08.120227+0000","last_unstale":"2024-05-14T09:32:30.522916+0000","last_undegraded":"2024-05-14T09:32:30.522916+0000","last_fullsized":"2024-05-14T09:32:30.522916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:06.630390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316348+0000","last_change":"2024-05-14T09:30:54.017382+0000","last_active":"2024-05-14T09:32:30.316348+0000","last_peered":"2024-05-14T09:32:30.316348+0000","last_clean":"2024-05-14T09:32:30.316348+0000","last_became_active":"2024-05-14T09:30:54.016887+0000","last_became_peered":"2024-05-14T09:30:54.016887+0000","last_unstale":"2024-05-14T09:32:30.316348+0000","last_undegraded":"2024-05-14T09:32:30.316348+0000","last_fullsized":"2024-05-14T09:32:30.316348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:28.823659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.085392+0000","last_change":"2024-05-14T09:30:56.800626+0000","last_active":"2024-05-14T09:33:29.085392+0000","last_peered":"2024-05-14T09:33:29.085392+0000","last_clean":"2024-05-14T09:33:29.085392+0000","last_became_active":"2024-05-14T09:30:56.800258+0000","last_became_peered":"2024-05-14T09:30:56.800258+0000","last_unstale":"2024-05-14T09:33:29.085392+0000","last_undegraded":"2024-05-14T09:33:29.085392+0000","last_fullsized":"2024-05-14T09:33:29.085392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:47:26.369633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.318135+0000","last_change":"2024-05-14T09:31:08.115456+0000","last_active":"2024-05-14T09:32:30.318135+0000","last_peered":"2024-05-14T09:32:30.318135+0000","last_clean":"2024-05-14T09:32:30.318135+0000","last_became_active":"2024-05-14T09:31:08.114705+0000","last_became_peered":"2024-05-14T09:31:08.114705+0000","last_unstale":"2024-05-14T09:32:30.318135+0000","last_undegraded":"2024-05-14T09:32:30.318135+0000","last_fullsized":"2024-05-14T09:32:30.318135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:50.096747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'39","reported_seq":461,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:34:52.509793+0000","last_change":"2024-05-14T09:31:30.186202+0000","last_active":"2024-05-14T09:34:52.509793+0000","last_peered":"2024-05-14T09:34:52.509793+0000","last_clean":"2024-05-14T09:34:52.509793+0000","last_became_active":"2024-05-14T09:31:30.174248+0000","last_became_peered":"2024-05-14T09:31:30.174248+0000","last_unstale":"2024-05-14T09:34:52.509793+0000","last_undegraded":"2024-05-14T09:34:52.509793+0000","last_fullsized":"2024-05-14T09:34:52.509793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:45:03.131934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64617,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":361,"num_read_kb":244,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077083+0000","last_change":"2024-05-14T09:30:56.791099+0000","last_active":"2024-05-14T09:33:29.077083+0000","last_peered":"2024-05-14T09:33:29.077083+0000","last_clean":"2024-05-14T09:33:29.077083+0000","last_became_active":"2024-05-14T09:30:56.790676+0000","last_became_peered":"2024-05-14T09:30:56.790676+0000","last_unstale":"2024-05-14T09:33:29.077083+0000","last_undegraded":"2024-05-14T09:33:29.077083+0000","last_fullsized":"2024-05-14T09:33:29.077083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:09.851357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316986+0000","last_change":"2024-05-14T09:31:08.120289+0000","last_active":"2024-05-14T09:32:30.316986+0000","last_peered":"2024-05-14T09:32:30.316986+0000","last_clean":"2024-05-14T09:32:30.316986+0000","last_became_active":"2024-05-14T09:31:08.118574+0000","last_became_peered":"2024-05-14T09:31:08.118574+0000","last_unstale":"2024-05-14T09:32:30.316986+0000","last_undegraded":"2024-05-14T09:32:30.316986+0000","last_fullsized":"2024-05-14T09:32:30.316986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:01.723442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319990+0000","last_change":"2024-05-14T09:30:53.766578+0000","last_active":"2024-05-14T09:32:30.319990+0000","last_peered":"2024-05-14T09:32:30.319990+0000","last_clean":"2024-05-14T09:32:30.319990+0000","last_became_active":"2024-05-14T09:30:53.765928+0000","last_became_peered":"2024-05-14T09:30:53.765928+0000","last_unstale":"2024-05-14T09:32:30.319990+0000","last_undegraded":"2024-05-14T09:32:30.319990+0000","last_fullsized":"2024-05-14T09:32:30.319990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:19:08.690013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.083536+0000","last_change":"2024-05-14T09:30:56.785507+0000","last_active":"2024-05-14T09:33:29.083536+0000","last_peered":"2024-05-14T09:33:29.083536+0000","last_clean":"2024-05-14T09:33:29.083536+0000","last_became_active":"2024-05-14T09:30:56.784800+0000","last_became_peered":"2024-05-14T09:30:56.784800+0000","last_unstale":"2024-05-14T09:33:29.083536+0000","last_undegraded":"2024-05-14T09:33:29.083536+0000","last_fullsized":"2024-05-14T09:33:29.083536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:36:05.440222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.321298+0000","last_change":"2024-05-14T09:31:08.119051+0000","last_active":"2024-05-14T09:32:30.321298+0000","last_peered":"2024-05-14T09:32:30.321298+0000","last_clean":"2024-05-14T09:32:30.321298+0000","last_became_active":"2024-05-14T09:31:08.118569+0000","last_became_peered":"2024-05-14T09:31:08.118569+0000","last_unstale":"2024-05-14T09:32:30.321298+0000","last_undegraded":"2024-05-14T09:32:30.321298+0000","last_fullsized":"2024-05-14T09:32:30.321298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:12:39.643740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316079+0000","last_change":"2024-05-14T09:30:53.772832+0000","last_active":"2024-05-14T09:32:30.316079+0000","last_peered":"2024-05-14T09:32:30.316079+0000","last_clean":"2024-05-14T09:32:30.316079+0000","last_became_active":"2024-05-14T09:30:53.772228+0000","last_became_peered":"2024-05-14T09:30:53.772228+0000","last_unstale":"2024-05-14T09:32:30.316079+0000","last_undegraded":"2024-05-14T09:32:30.316079+0000","last_fullsized":"2024-05-14T09:32:30.316079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:14:43.350330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084806+0000","last_change":"2024-05-14T09:30:56.782831+0000","last_active":"2024-05-14T09:33:29.084806+0000","last_peered":"2024-05-14T09:33:29.084806+0000","last_clean":"2024-05-14T09:33:29.084806+0000","last_became_active":"2024-05-14T09:30:56.782402+0000","last_became_peered":"2024-05-14T09:30:56.782402+0000","last_unstale":"2024-05-14T09:33:29.084806+0000","last_undegraded":"2024-05-14T09:33:29.084806+0000","last_fullsized":"2024-05-14T09:33:29.084806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:54.603524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524041+0000","last_change":"2024-05-14T09:31:30.296957+0000","last_active":"2024-05-14T09:32:30.524041+0000","last_peered":"2024-05-14T09:32:30.524041+0000","last_clean":"2024-05-14T09:32:30.524041+0000","last_became_active":"2024-05-14T09:31:30.296554+0000","last_became_peered":"2024-05-14T09:31:30.296554+0000","last_unstale":"2024-05-14T09:32:30.524041+0000","last_undegraded":"2024-05-14T09:32:30.524041+0000","last_fullsized":"2024-05-14T09:32:30.524041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:28.963042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":224,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:34:50.507285+0000","last_change":"2024-05-14T09:30:53.772731+0000","last_active":"2024-05-14T09:34:50.507285+0000","last_peered":"2024-05-14T09:34:50.507285+0000","last_clean":"2024-05-14T09:34:50.507285+0000","last_became_active":"2024-05-14T09:30:53.771986+0000","last_became_peered":"2024-05-14T09:30:53.771986+0000","last_unstale":"2024-05-14T09:34:50.507285+0000","last_undegraded":"2024-05-14T09:34:50.507285+0000","last_fullsized":"2024-05-14T09:34:50.507285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:50:00.810937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077545+0000","last_change":"2024-05-14T09:30:56.790503+0000","last_active":"2024-05-14T09:33:29.077545+0000","last_peered":"2024-05-14T09:33:29.077545+0000","last_clean":"2024-05-14T09:33:29.077545+0000","last_became_active":"2024-05-14T09:30:56.790103+0000","last_became_peered":"2024-05-14T09:30:56.790103+0000","last_unstale":"2024-05-14T09:33:29.077545+0000","last_undegraded":"2024-05-14T09:33:29.077545+0000","last_fullsized":"2024-05-14T09:33:29.077545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:00.950794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315462+0000","last_change":"2024-05-14T09:31:08.116475+0000","last_active":"2024-05-14T09:32:30.315462+0000","last_peered":"2024-05-14T09:32:30.315462+0000","last_clean":"2024-05-14T09:32:30.315462+0000","last_became_active":"2024-05-14T09:31:08.116061+0000","last_became_peered":"2024-05-14T09:31:08.116061+0000","last_unstale":"2024-05-14T09:32:30.315462+0000","last_undegraded":"2024-05-14T09:32:30.315462+0000","last_fullsized":"2024-05-14T09:32:30.315462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:07:02.281853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523311+0000","last_change":"2024-05-14T09:30:54.016570+0000","last_active":"2024-05-14T09:32:30.523311+0000","last_peered":"2024-05-14T09:32:30.523311+0000","last_clean":"2024-05-14T09:32:30.523311+0000","last_became_active":"2024-05-14T09:30:54.015987+0000","last_became_peered":"2024-05-14T09:30:54.015987+0000","last_unstale":"2024-05-14T09:32:30.523311+0000","last_undegraded":"2024-05-14T09:32:30.523311+0000","last_fullsized":"2024-05-14T09:32:30.523311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:18:28.694488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081072+0000","last_change":"2024-05-14T09:30:56.800727+0000","last_active":"2024-05-14T09:33:29.081072+0000","last_peered":"2024-05-14T09:33:29.081072+0000","last_clean":"2024-05-14T09:33:29.081072+0000","last_became_active":"2024-05-14T09:30:56.800354+0000","last_became_peered":"2024-05-14T09:30:56.800354+0000","last_unstale":"2024-05-14T09:33:29.081072+0000","last_undegraded":"2024-05-14T09:33:29.081072+0000","last_fullsized":"2024-05-14T09:33:29.081072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:13:07.227577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315777+0000","last_change":"2024-05-14T09:31:08.115685+0000","last_active":"2024-05-14T09:32:30.315777+0000","last_peered":"2024-05-14T09:32:30.315777+0000","last_clean":"2024-05-14T09:32:30.315777+0000","last_became_active":"2024-05-14T09:31:08.114948+0000","last_became_peered":"2024-05-14T09:31:08.114948+0000","last_unstale":"2024-05-14T09:32:30.315777+0000","last_undegraded":"2024-05-14T09:32:30.315777+0000","last_fullsized":"2024-05-14T09:32:30.315777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:33:07.286823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524227+0000","last_change":"2024-05-14T09:30:53.769949+0000","last_active":"2024-05-14T09:32:30.524227+0000","last_peered":"2024-05-14T09:32:30.524227+0000","last_clean":"2024-05-14T09:32:30.524227+0000","last_became_active":"2024-05-14T09:30:53.769327+0000","last_became_peered":"2024-05-14T09:30:53.769327+0000","last_unstale":"2024-05-14T09:32:30.524227+0000","last_undegraded":"2024-05-14T09:32:30.524227+0000","last_fullsized":"2024-05-14T09:32:30.524227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:35:11.850991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077817+0000","last_change":"2024-05-14T09:32:31.015994+0000","last_active":"2024-05-14T09:33:29.077817+0000","last_peered":"2024-05-14T09:33:29.077817+0000","last_clean":"2024-05-14T09:33:29.077817+0000","last_became_active":"2024-05-14T09:32:31.015515+0000","last_became_peered":"2024-05-14T09:32:31.015515+0000","last_unstale":"2024-05-14T09:33:29.077817+0000","last_undegraded":"2024-05-14T09:33:29.077817+0000","last_fullsized":"2024-05-14T09:33:29.077817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:21:49.811521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7],"acting":[11,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524005+0000","last_change":"2024-05-14T09:31:08.387316+0000","last_active":"2024-05-14T09:32:30.524005+0000","last_peered":"2024-05-14T09:32:30.524005+0000","last_clean":"2024-05-14T09:32:30.524005+0000","last_became_active":"2024-05-14T09:31:08.386739+0000","last_became_peered":"2024-05-14T09:31:08.386739+0000","last_unstale":"2024-05-14T09:32:30.524005+0000","last_undegraded":"2024-05-14T09:32:30.524005+0000","last_fullsized":"2024-05-14T09:32:30.524005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:17.870255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.318704+0000","last_change":"2024-05-14T09:30:54.017756+0000","last_active":"2024-05-14T09:32:30.318704+0000","last_peered":"2024-05-14T09:32:30.318704+0000","last_clean":"2024-05-14T09:32:30.318704+0000","last_became_active":"2024-05-14T09:30:54.017277+0000","last_became_peered":"2024-05-14T09:30:54.017277+0000","last_unstale":"2024-05-14T09:32:30.318704+0000","last_undegraded":"2024-05-14T09:32:30.318704+0000","last_fullsized":"2024-05-14T09:32:30.318704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:27:36.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.072775+0000","last_change":"2024-05-14T09:30:56.788408+0000","last_active":"2024-05-14T09:33:29.072775+0000","last_peered":"2024-05-14T09:33:29.072775+0000","last_clean":"2024-05-14T09:33:29.072775+0000","last_became_active":"2024-05-14T09:30:56.787955+0000","last_became_peered":"2024-05-14T09:30:56.787955+0000","last_unstale":"2024-05-14T09:33:29.072775+0000","last_undegraded":"2024-05-14T09:33:29.072775+0000","last_fullsized":"2024-05-14T09:33:29.072775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:09.777069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315137+0000","last_change":"2024-05-14T09:31:08.119078+0000","last_active":"2024-05-14T09:32:30.315137+0000","last_peered":"2024-05-14T09:32:30.315137+0000","last_clean":"2024-05-14T09:32:30.315137+0000","last_became_active":"2024-05-14T09:31:08.117939+0000","last_became_peered":"2024-05-14T09:31:08.117939+0000","last_unstale":"2024-05-14T09:32:30.315137+0000","last_undegraded":"2024-05-14T09:32:30.315137+0000","last_fullsized":"2024-05-14T09:32:30.315137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:48:37.363503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014384+0000","last_change":"2024-05-14T09:30:53.767843+0000","last_active":"2024-05-14T09:32:31.014384+0000","last_peered":"2024-05-14T09:32:31.014384+0000","last_clean":"2024-05-14T09:32:31.014384+0000","last_became_active":"2024-05-14T09:30:53.767419+0000","last_became_peered":"2024-05-14T09:30:53.767419+0000","last_unstale":"2024-05-14T09:32:31.014384+0000","last_undegraded":"2024-05-14T09:32:31.014384+0000","last_fullsized":"2024-05-14T09:32:31.014384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:49:06.441600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077073+0000","last_change":"2024-05-14T09:30:56.796001+0000","last_active":"2024-05-14T09:33:29.077073+0000","last_peered":"2024-05-14T09:33:29.077073+0000","last_clean":"2024-05-14T09:33:29.077073+0000","last_became_active":"2024-05-14T09:30:56.794517+0000","last_became_peered":"2024-05-14T09:30:56.794517+0000","last_unstale":"2024-05-14T09:33:29.077073+0000","last_undegraded":"2024-05-14T09:33:29.077073+0000","last_fullsized":"2024-05-14T09:33:29.077073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:20:37.514679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315309+0000","last_change":"2024-05-14T09:31:08.387570+0000","last_active":"2024-05-14T09:32:30.315309+0000","last_peered":"2024-05-14T09:32:30.315309+0000","last_clean":"2024-05-14T09:32:30.315309+0000","last_became_active":"2024-05-14T09:31:08.386745+0000","last_became_peered":"2024-05-14T09:31:08.386745+0000","last_unstale":"2024-05-14T09:32:30.315309+0000","last_undegraded":"2024-05-14T09:32:30.315309+0000","last_fullsized":"2024-05-14T09:32:30.315309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:44:01.024784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.060037+0000","last_change":"2024-05-14T09:30:56.795532+0000","last_active":"2024-05-14T09:33:29.060037+0000","last_peered":"2024-05-14T09:33:29.060037+0000","last_clean":"2024-05-14T09:33:29.060037+0000","last_became_active":"2024-05-14T09:30:56.795171+0000","last_became_peered":"2024-05-14T09:30:56.795171+0000","last_unstale":"2024-05-14T09:33:29.060037+0000","last_undegraded":"2024-05-14T09:33:29.060037+0000","last_fullsized":"2024-05-14T09:33:29.060037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:09:40.252489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":224,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:34:51.508100+0000","last_change":"2024-05-14T09:30:54.015629+0000","last_active":"2024-05-14T09:34:51.508100+0000","last_peered":"2024-05-14T09:34:51.508100+0000","last_clean":"2024-05-14T09:34:51.508100+0000","last_became_active":"2024-05-14T09:30:54.015288+0000","last_became_peered":"2024-05-14T09:30:54.015288+0000","last_unstale":"2024-05-14T09:34:51.508100+0000","last_undegraded":"2024-05-14T09:34:51.508100+0000","last_fullsized":"2024-05-14T09:34:51.508100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:50.316125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014777+0000","last_change":"2024-05-14T09:31:30.179056+0000","last_active":"2024-05-14T09:32:31.014777+0000","last_peered":"2024-05-14T09:32:31.014777+0000","last_clean":"2024-05-14T09:32:31.014777+0000","last_became_active":"2024-05-14T09:31:30.178398+0000","last_became_peered":"2024-05-14T09:31:30.178398+0000","last_unstale":"2024-05-14T09:32:31.014777+0000","last_undegraded":"2024-05-14T09:32:31.014777+0000","last_fullsized":"2024-05-14T09:32:31.014777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:15:55.369801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077759+0000","last_change":"2024-05-14T09:30:56.793642+0000","last_active":"2024-05-14T09:33:29.077759+0000","last_peered":"2024-05-14T09:33:29.077759+0000","last_clean":"2024-05-14T09:33:29.077759+0000","last_became_active":"2024-05-14T09:30:56.793229+0000","last_became_peered":"2024-05-14T09:30:56.793229+0000","last_unstale":"2024-05-14T09:33:29.077759+0000","last_undegraded":"2024-05-14T09:33:29.077759+0000","last_fullsized":"2024-05-14T09:33:29.077759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:17.940674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522946+0000","last_change":"2024-05-14T09:30:53.769582+0000","last_active":"2024-05-14T09:32:30.522946+0000","last_peered":"2024-05-14T09:32:30.522946+0000","last_clean":"2024-05-14T09:32:30.522946+0000","last_became_active":"2024-05-14T09:30:53.769113+0000","last_became_peered":"2024-05-14T09:30:53.769113+0000","last_unstale":"2024-05-14T09:32:30.522946+0000","last_undegraded":"2024-05-14T09:32:30.522946+0000","last_fullsized":"2024-05-14T09:32:30.522946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:48.755113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":119,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014145+0000","last_change":"2024-05-14T09:29:30.767389+0000","last_active":"2024-05-14T09:32:31.014145+0000","last_peered":"2024-05-14T09:32:31.014145+0000","last_clean":"2024-05-14T09:32:31.014145+0000","last_became_active":"2024-05-14T09:29:30.761827+0000","last_became_peered":"2024-05-14T09:29:30.761827+0000","last_unstale":"2024-05-14T09:32:31.014145+0000","last_undegraded":"2024-05-14T09:32:31.014145+0000","last_fullsized":"2024-05-14T09:32:31.014145+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:26:03.741821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:26:03.741821+0000","last_clean_scrub_stamp":"2024-05-14T09:26:03.741821+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:52:11.421658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316444+0000","last_change":"2024-05-14T09:31:08.387819+0000","last_active":"2024-05-14T09:32:30.316444+0000","last_peered":"2024-05-14T09:32:30.316444+0000","last_clean":"2024-05-14T09:32:30.316444+0000","last_became_active":"2024-05-14T09:31:08.387275+0000","last_became_peered":"2024-05-14T09:31:08.387275+0000","last_unstale":"2024-05-14T09:32:30.316444+0000","last_undegraded":"2024-05-14T09:32:30.316444+0000","last_fullsized":"2024-05-14T09:32:30.316444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:46.627389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.078055+0000","last_change":"2024-05-14T09:30:56.788159+0000","last_active":"2024-05-14T09:33:29.078055+0000","last_peered":"2024-05-14T09:33:29.078055+0000","last_clean":"2024-05-14T09:33:29.078055+0000","last_became_active":"2024-05-14T09:30:56.787744+0000","last_became_peered":"2024-05-14T09:30:56.787744+0000","last_unstale":"2024-05-14T09:33:29.078055+0000","last_undegraded":"2024-05-14T09:33:29.078055+0000","last_fullsized":"2024-05-14T09:33:29.078055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:45:25.243471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:52.843555+0000","last_change":"2024-05-14T09:30:53.774345+0000","last_active":"2024-05-14T09:32:52.843555+0000","last_peered":"2024-05-14T09:32:52.843555+0000","last_clean":"2024-05-14T09:32:52.843555+0000","last_became_active":"2024-05-14T09:30:53.773853+0000","last_became_peered":"2024-05-14T09:30:53.773853+0000","last_unstale":"2024-05-14T09:32:52.843555+0000","last_undegraded":"2024-05-14T09:32:52.843555+0000","last_fullsized":"2024-05-14T09:32:52.843555+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:35:31.938442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320387+0000","last_change":"2024-05-14T09:31:08.122876+0000","last_active":"2024-05-14T09:32:30.320387+0000","last_peered":"2024-05-14T09:32:30.320387+0000","last_clean":"2024-05-14T09:32:30.320387+0000","last_became_active":"2024-05-14T09:31:08.122182+0000","last_became_peered":"2024-05-14T09:31:08.122182+0000","last_unstale":"2024-05-14T09:32:30.320387+0000","last_undegraded":"2024-05-14T09:32:30.320387+0000","last_fullsized":"2024-05-14T09:32:30.320387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:47:30.735057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524333+0000","last_change":"2024-05-14T09:30:54.018168+0000","last_active":"2024-05-14T09:32:30.524333+0000","last_peered":"2024-05-14T09:32:30.524333+0000","last_clean":"2024-05-14T09:32:30.524333+0000","last_became_active":"2024-05-14T09:30:54.017699+0000","last_became_peered":"2024-05-14T09:30:54.017699+0000","last_unstale":"2024-05-14T09:32:30.524333+0000","last_undegraded":"2024-05-14T09:32:30.524333+0000","last_fullsized":"2024-05-14T09:32:30.524333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:31:01.162000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.070935+0000","last_change":"2024-05-14T09:30:56.787156+0000","last_active":"2024-05-14T09:33:29.070935+0000","last_peered":"2024-05-14T09:33:29.070935+0000","last_clean":"2024-05-14T09:33:29.070935+0000","last_became_active":"2024-05-14T09:30:56.786832+0000","last_became_peered":"2024-05-14T09:30:56.786832+0000","last_unstale":"2024-05-14T09:33:29.070935+0000","last_undegraded":"2024-05-14T09:33:29.070935+0000","last_fullsized":"2024-05-14T09:33:29.070935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:00:41.719641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.321913+0000","last_change":"2024-05-14T09:31:08.387907+0000","last_active":"2024-05-14T09:32:30.321913+0000","last_peered":"2024-05-14T09:32:30.321913+0000","last_clean":"2024-05-14T09:32:30.321913+0000","last_became_active":"2024-05-14T09:31:08.386765+0000","last_became_peered":"2024-05-14T09:31:08.386765+0000","last_unstale":"2024-05-14T09:32:30.321913+0000","last_undegraded":"2024-05-14T09:32:30.321913+0000","last_fullsized":"2024-05-14T09:32:30.321913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:24:56.890353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'13","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.814407+0000","last_change":"2024-05-14T09:30:53.767508+0000","last_active":"2024-05-14T09:33:29.814407+0000","last_peered":"2024-05-14T09:33:29.814407+0000","last_clean":"2024-05-14T09:33:29.814407+0000","last_became_active":"2024-05-14T09:30:53.767169+0000","last_became_peered":"2024-05-14T09:30:53.767169+0000","last_unstale":"2024-05-14T09:33:29.814407+0000","last_undegraded":"2024-05-14T09:33:29.814407+0000","last_fullsized":"2024-05-14T09:33:29.814407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:04:23.621686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076780+0000","last_change":"2024-05-14T09:30:56.790398+0000","last_active":"2024-05-14T09:33:29.076780+0000","last_peered":"2024-05-14T09:33:29.076780+0000","last_clean":"2024-05-14T09:33:29.076780+0000","last_became_active":"2024-05-14T09:30:56.790055+0000","last_became_peered":"2024-05-14T09:30:56.790055+0000","last_unstale":"2024-05-14T09:33:29.076780+0000","last_undegraded":"2024-05-14T09:33:29.076780+0000","last_fullsized":"2024-05-14T09:33:29.076780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:12:37.688647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319239+0000","last_change":"2024-05-14T09:31:08.111533+0000","last_active":"2024-05-14T09:32:30.319239+0000","last_peered":"2024-05-14T09:32:30.319239+0000","last_clean":"2024-05-14T09:32:30.319239+0000","last_became_active":"2024-05-14T09:31:08.110448+0000","last_became_peered":"2024-05-14T09:31:08.110448+0000","last_unstale":"2024-05-14T09:32:30.319239+0000","last_undegraded":"2024-05-14T09:32:30.319239+0000","last_fullsized":"2024-05-14T09:32:30.319239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:43:45.822382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523626+0000","last_change":"2024-05-14T09:30:54.069180+0000","last_active":"2024-05-14T09:32:30.523626+0000","last_peered":"2024-05-14T09:32:30.523626+0000","last_clean":"2024-05-14T09:32:30.523626+0000","last_became_active":"2024-05-14T09:30:54.068846+0000","last_became_peered":"2024-05-14T09:30:54.068846+0000","last_unstale":"2024-05-14T09:32:30.523626+0000","last_undegraded":"2024-05-14T09:32:30.523626+0000","last_fullsized":"2024-05-14T09:32:30.523626+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:22.541720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080401+0000","last_change":"2024-05-14T09:30:56.788159+0000","last_active":"2024-05-14T09:33:29.080401+0000","last_peered":"2024-05-14T09:33:29.080401+0000","last_clean":"2024-05-14T09:33:29.080401+0000","last_became_active":"2024-05-14T09:30:56.787786+0000","last_became_peered":"2024-05-14T09:30:56.787786+0000","last_unstale":"2024-05-14T09:33:29.080401+0000","last_undegraded":"2024-05-14T09:33:29.080401+0000","last_fullsized":"2024-05-14T09:33:29.080401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:24:51.785543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523393+0000","last_change":"2024-05-14T09:31:08.388201+0000","last_active":"2024-05-14T09:32:30.523393+0000","last_peered":"2024-05-14T09:32:30.523393+0000","last_clean":"2024-05-14T09:32:30.523393+0000","last_became_active":"2024-05-14T09:31:08.387572+0000","last_became_peered":"2024-05-14T09:31:08.387572+0000","last_unstale":"2024-05-14T09:32:30.523393+0000","last_undegraded":"2024-05-14T09:32:30.523393+0000","last_fullsized":"2024-05-14T09:32:30.523393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:25:13.171350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316020+0000","last_change":"2024-05-14T09:30:54.016524+0000","last_active":"2024-05-14T09:32:30.316020+0000","last_peered":"2024-05-14T09:32:30.316020+0000","last_clean":"2024-05-14T09:32:30.316020+0000","last_became_active":"2024-05-14T09:30:54.016154+0000","last_became_peered":"2024-05-14T09:30:54.016154+0000","last_unstale":"2024-05-14T09:32:30.316020+0000","last_undegraded":"2024-05-14T09:32:30.316020+0000","last_fullsized":"2024-05-14T09:32:30.316020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:43:16.671053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.035477+0000","last_change":"2024-05-14T09:30:56.796748+0000","last_active":"2024-05-14T09:33:29.035477+0000","last_peered":"2024-05-14T09:33:29.035477+0000","last_clean":"2024-05-14T09:33:29.035477+0000","last_became_active":"2024-05-14T09:30:56.796354+0000","last_became_peered":"2024-05-14T09:30:56.796354+0000","last_unstale":"2024-05-14T09:33:29.035477+0000","last_undegraded":"2024-05-14T09:33:29.035477+0000","last_fullsized":"2024-05-14T09:33:29.035477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:30:15.820677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319112+0000","last_change":"2024-05-14T09:31:08.117266+0000","last_active":"2024-05-14T09:32:30.319112+0000","last_peered":"2024-05-14T09:32:30.319112+0000","last_clean":"2024-05-14T09:32:30.319112+0000","last_became_active":"2024-05-14T09:31:08.115393+0000","last_became_peered":"2024-05-14T09:31:08.115393+0000","last_unstale":"2024-05-14T09:32:30.319112+0000","last_undegraded":"2024-05-14T09:32:30.319112+0000","last_fullsized":"2024-05-14T09:32:30.319112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:29:26.348818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.322687+0000","last_change":"2024-05-14T09:30:53.772417+0000","last_active":"2024-05-14T09:32:30.322687+0000","last_peered":"2024-05-14T09:32:30.322687+0000","last_clean":"2024-05-14T09:32:30.322687+0000","last_became_active":"2024-05-14T09:30:53.772028+0000","last_became_peered":"2024-05-14T09:30:53.772028+0000","last_unstale":"2024-05-14T09:32:30.322687+0000","last_undegraded":"2024-05-14T09:32:30.322687+0000","last_fullsized":"2024-05-14T09:32:30.322687+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:16:16.603957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077309+0000","last_change":"2024-05-14T09:30:56.804750+0000","last_active":"2024-05-14T09:33:29.077309+0000","last_peered":"2024-05-14T09:33:29.077309+0000","last_clean":"2024-05-14T09:33:29.077309+0000","last_became_active":"2024-05-14T09:30:56.803990+0000","last_became_peered":"2024-05-14T09:30:56.803990+0000","last_unstale":"2024-05-14T09:33:29.077309+0000","last_undegraded":"2024-05-14T09:33:29.077309+0000","last_fullsized":"2024-05-14T09:33:29.077309+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:42:00.661033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523484+0000","last_change":"2024-05-14T09:31:08.116061+0000","last_active":"2024-05-14T09:32:30.523484+0000","last_peered":"2024-05-14T09:32:30.523484+0000","last_clean":"2024-05-14T09:32:30.523484+0000","last_became_active":"2024-05-14T09:31:08.115578+0000","last_became_peered":"2024-05-14T09:31:08.115578+0000","last_unstale":"2024-05-14T09:32:30.523484+0000","last_undegraded":"2024-05-14T09:32:30.523484+0000","last_fullsized":"2024-05-14T09:32:30.523484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:18.931035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:06.782240+0000","last_change":"2024-05-14T09:30:53.768988+0000","last_active":"2024-05-14T09:33:06.782240+0000","last_peered":"2024-05-14T09:33:06.782240+0000","last_clean":"2024-05-14T09:33:06.782240+0000","last_became_active":"2024-05-14T09:30:53.768529+0000","last_became_peered":"2024-05-14T09:30:53.768529+0000","last_unstale":"2024-05-14T09:33:06.782240+0000","last_undegraded":"2024-05-14T09:33:06.782240+0000","last_fullsized":"2024-05-14T09:33:06.782240+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:17:02.381447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.079500+0000","last_change":"2024-05-14T09:30:56.797110+0000","last_active":"2024-05-14T09:33:29.079500+0000","last_peered":"2024-05-14T09:33:29.079500+0000","last_clean":"2024-05-14T09:33:29.079500+0000","last_became_active":"2024-05-14T09:30:56.796015+0000","last_became_peered":"2024-05-14T09:30:56.796015+0000","last_unstale":"2024-05-14T09:33:29.079500+0000","last_undegraded":"2024-05-14T09:33:29.079500+0000","last_fullsized":"2024-05-14T09:33:29.079500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:37:44.674607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523289+0000","last_change":"2024-05-14T09:31:08.387240+0000","last_active":"2024-05-14T09:32:30.523289+0000","last_peered":"2024-05-14T09:32:30.523289+0000","last_clean":"2024-05-14T09:32:30.523289+0000","last_became_active":"2024-05-14T09:31:08.386653+0000","last_became_peered":"2024-05-14T09:31:08.386653+0000","last_unstale":"2024-05-14T09:32:30.523289+0000","last_undegraded":"2024-05-14T09:32:30.523289+0000","last_fullsized":"2024-05-14T09:32:30.523289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:18:43.408030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523577+0000","last_change":"2024-05-14T09:30:53.772723+0000","last_active":"2024-05-14T09:32:30.523577+0000","last_peered":"2024-05-14T09:32:30.523577+0000","last_clean":"2024-05-14T09:32:30.523577+0000","last_became_active":"2024-05-14T09:30:53.772056+0000","last_became_peered":"2024-05-14T09:30:53.772056+0000","last_unstale":"2024-05-14T09:32:30.523577+0000","last_undegraded":"2024-05-14T09:32:30.523577+0000","last_fullsized":"2024-05-14T09:32:30.523577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:37.941282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087926+0000","last_change":"2024-05-14T09:30:56.799223+0000","last_active":"2024-05-14T09:33:29.087926+0000","last_peered":"2024-05-14T09:33:29.087926+0000","last_clean":"2024-05-14T09:33:29.087926+0000","last_became_active":"2024-05-14T09:30:56.796092+0000","last_became_peered":"2024-05-14T09:30:56.796092+0000","last_unstale":"2024-05-14T09:33:29.087926+0000","last_undegraded":"2024-05-14T09:33:29.087926+0000","last_fullsized":"2024-05-14T09:33:29.087926+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:43:49.746797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316061+0000","last_change":"2024-05-14T09:31:08.115053+0000","last_active":"2024-05-14T09:32:30.316061+0000","last_peered":"2024-05-14T09:32:30.316061+0000","last_clean":"2024-05-14T09:32:30.316061+0000","last_became_active":"2024-05-14T09:31:08.114578+0000","last_became_peered":"2024-05-14T09:31:08.114578+0000","last_unstale":"2024-05-14T09:32:30.316061+0000","last_undegraded":"2024-05-14T09:32:30.316061+0000","last_fullsized":"2024-05-14T09:32:30.316061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:47.788962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:32.003842+0000","last_change":"2024-05-14T09:32:30.326580+0000","last_active":"2024-05-14T09:32:32.003842+0000","last_peered":"2024-05-14T09:32:32.003842+0000","last_clean":"2024-05-14T09:32:32.003842+0000","last_became_active":"2024-05-14T09:32:30.322052+0000","last_became_peered":"2024-05-14T09:32:30.322052+0000","last_unstale":"2024-05-14T09:32:32.003842+0000","last_undegraded":"2024-05-14T09:32:32.003842+0000","last_fullsized":"2024-05-14T09:32:32.003842+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:04.991674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087235+0000","last_change":"2024-05-14T09:30:56.798580+0000","last_active":"2024-05-14T09:33:29.087235+0000","last_peered":"2024-05-14T09:33:29.087235+0000","last_clean":"2024-05-14T09:33:29.087235+0000","last_became_active":"2024-05-14T09:30:56.795617+0000","last_became_peered":"2024-05-14T09:30:56.795617+0000","last_unstale":"2024-05-14T09:33:29.087235+0000","last_undegraded":"2024-05-14T09:33:29.087235+0000","last_fullsized":"2024-05-14T09:33:29.087235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:30:35.913497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319424+0000","last_change":"2024-05-14T09:31:08.111375+0000","last_active":"2024-05-14T09:32:30.319424+0000","last_peered":"2024-05-14T09:32:30.319424+0000","last_clean":"2024-05-14T09:32:30.319424+0000","last_became_active":"2024-05-14T09:31:08.110931+0000","last_became_peered":"2024-05-14T09:31:08.110931+0000","last_unstale":"2024-05-14T09:32:30.319424+0000","last_undegraded":"2024-05-14T09:32:30.319424+0000","last_fullsized":"2024-05-14T09:32:30.319424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:08:30.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.176828+0000","last_change":"2024-05-14T09:32:30.326979+0000","last_active":"2024-05-14T09:32:31.176828+0000","last_peered":"2024-05-14T09:32:31.176828+0000","last_clean":"2024-05-14T09:32:31.176828+0000","last_became_active":"2024-05-14T09:32:30.323545+0000","last_became_peered":"2024-05-14T09:32:30.323545+0000","last_unstale":"2024-05-14T09:32:31.176828+0000","last_undegraded":"2024-05-14T09:32:31.176828+0000","last_fullsized":"2024-05-14T09:32:31.176828+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:00:21.493910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":70,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.064380+0000","last_change":"2024-05-14T09:30:56.789527+0000","last_active":"2024-05-14T09:33:29.064380+0000","last_peered":"2024-05-14T09:33:29.064380+0000","last_clean":"2024-05-14T09:33:29.064380+0000","last_became_active":"2024-05-14T09:30:56.789075+0000","last_became_peered":"2024-05-14T09:30:56.789075+0000","last_unstale":"2024-05-14T09:33:29.064380+0000","last_undegraded":"2024-05-14T09:33:29.064380+0000","last_fullsized":"2024-05-14T09:33:29.064380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:07.140087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315692+0000","last_change":"2024-05-14T09:31:08.119624+0000","last_active":"2024-05-14T09:32:30.315692+0000","last_peered":"2024-05-14T09:32:30.315692+0000","last_clean":"2024-05-14T09:32:30.315692+0000","last_became_active":"2024-05-14T09:31:08.118102+0000","last_became_peered":"2024-05-14T09:31:08.118102+0000","last_unstale":"2024-05-14T09:32:30.315692+0000","last_undegraded":"2024-05-14T09:32:30.315692+0000","last_fullsized":"2024-05-14T09:32:30.315692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:48:17.806976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315472+0000","last_change":"2024-05-14T09:30:53.763903+0000","last_active":"2024-05-14T09:32:30.315472+0000","last_peered":"2024-05-14T09:32:30.315472+0000","last_clean":"2024-05-14T09:32:30.315472+0000","last_became_active":"2024-05-14T09:30:53.763517+0000","last_became_peered":"2024-05-14T09:30:53.763517+0000","last_unstale":"2024-05-14T09:32:30.315472+0000","last_undegraded":"2024-05-14T09:32:30.315472+0000","last_fullsized":"2024-05-14T09:32:30.315472+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:57.144392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081279+0000","last_change":"2024-05-14T09:30:56.791612+0000","last_active":"2024-05-14T09:33:29.081279+0000","last_peered":"2024-05-14T09:33:29.081279+0000","last_clean":"2024-05-14T09:33:29.081279+0000","last_became_active":"2024-05-14T09:30:56.791178+0000","last_became_peered":"2024-05-14T09:30:56.791178+0000","last_unstale":"2024-05-14T09:33:29.081279+0000","last_undegraded":"2024-05-14T09:33:29.081279+0000","last_fullsized":"2024-05-14T09:33:29.081279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:11:19.058139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076827+0000","last_change":"2024-05-14T09:30:56.804437+0000","last_active":"2024-05-14T09:33:29.076827+0000","last_peered":"2024-05-14T09:33:29.076827+0000","last_clean":"2024-05-14T09:33:29.076827+0000","last_became_active":"2024-05-14T09:30:56.804048+0000","last_became_peered":"2024-05-14T09:30:56.804048+0000","last_unstale":"2024-05-14T09:33:29.076827+0000","last_undegraded":"2024-05-14T09:33:29.076827+0000","last_fullsized":"2024-05-14T09:33:29.076827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:33.788903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.058110+0000","last_change":"2024-05-14T09:30:56.794309+0000","last_active":"2024-05-14T09:33:29.058110+0000","last_peered":"2024-05-14T09:33:29.058110+0000","last_clean":"2024-05-14T09:33:29.058110+0000","last_became_active":"2024-05-14T09:30:56.793317+0000","last_became_peered":"2024-05-14T09:30:56.793317+0000","last_unstale":"2024-05-14T09:33:29.058110+0000","last_undegraded":"2024-05-14T09:33:29.058110+0000","last_fullsized":"2024-05-14T09:33:29.058110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:40:26.878746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080059+0000","last_change":"2024-05-14T09:30:56.793618+0000","last_active":"2024-05-14T09:33:29.080059+0000","last_peered":"2024-05-14T09:33:29.080059+0000","last_clean":"2024-05-14T09:33:29.080059+0000","last_became_active":"2024-05-14T09:30:56.792819+0000","last_became_peered":"2024-05-14T09:30:56.792819+0000","last_unstale":"2024-05-14T09:33:29.080059+0000","last_undegraded":"2024-05-14T09:33:29.080059+0000","last_fullsized":"2024-05-14T09:33:29.080059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:29:38.191617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.072960+0000","last_change":"2024-05-14T09:30:56.794268+0000","last_active":"2024-05-14T09:33:29.072960+0000","last_peered":"2024-05-14T09:33:29.072960+0000","last_clean":"2024-05-14T09:33:29.072960+0000","last_became_active":"2024-05-14T09:30:56.793898+0000","last_became_peered":"2024-05-14T09:30:56.793898+0000","last_unstale":"2024-05-14T09:33:29.072960+0000","last_undegraded":"2024-05-14T09:33:29.072960+0000","last_fullsized":"2024-05-14T09:33:29.072960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:25:35.022236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.078917+0000","last_change":"2024-05-14T09:30:56.783366+0000","last_active":"2024-05-14T09:33:29.078917+0000","last_peered":"2024-05-14T09:33:29.078917+0000","last_clean":"2024-05-14T09:33:29.078917+0000","last_became_active":"2024-05-14T09:30:56.782980+0000","last_became_peered":"2024-05-14T09:30:56.782980+0000","last_unstale":"2024-05-14T09:33:29.078917+0000","last_undegraded":"2024-05-14T09:33:29.078917+0000","last_fullsized":"2024-05-14T09:33:29.078917+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:27.744902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076941+0000","last_change":"2024-05-14T09:32:30.524390+0000","last_active":"2024-05-14T09:33:29.076941+0000","last_peered":"2024-05-14T09:33:29.076941+0000","last_clean":"2024-05-14T09:33:29.076941+0000","last_became_active":"2024-05-14T09:32:30.523926+0000","last_became_peered":"2024-05-14T09:32:30.523926+0000","last_unstale":"2024-05-14T09:33:29.076941+0000","last_undegraded":"2024-05-14T09:33:29.076941+0000","last_fullsized":"2024-05-14T09:33:29.076941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:58:47.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.074545+0000","last_change":"2024-05-14T09:30:56.795872+0000","last_active":"2024-05-14T09:33:29.074545+0000","last_peered":"2024-05-14T09:33:29.074545+0000","last_clean":"2024-05-14T09:33:29.074545+0000","last_became_active":"2024-05-14T09:30:56.795492+0000","last_became_peered":"2024-05-14T09:30:56.795492+0000","last_unstale":"2024-05-14T09:33:29.074545+0000","last_undegraded":"2024-05-14T09:33:29.074545+0000","last_fullsized":"2024-05-14T09:33:29.074545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:26:29.226575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.062863+0000","last_change":"2024-05-14T09:32:30.524189+0000","last_active":"2024-05-14T09:33:29.062863+0000","last_peered":"2024-05-14T09:33:29.062863+0000","last_clean":"2024-05-14T09:33:29.062863+0000","last_became_active":"2024-05-14T09:32:30.523791+0000","last_became_peered":"2024-05-14T09:32:30.523791+0000","last_unstale":"2024-05-14T09:33:29.062863+0000","last_undegraded":"2024-05-14T09:33:29.062863+0000","last_fullsized":"2024-05-14T09:33:29.062863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:52.441130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080195+0000","last_change":"2024-05-14T09:30:56.792814+0000","last_active":"2024-05-14T09:33:29.080195+0000","last_peered":"2024-05-14T09:33:29.080195+0000","last_clean":"2024-05-14T09:33:29.080195+0000","last_became_active":"2024-05-14T09:30:56.792282+0000","last_became_peered":"2024-05-14T09:30:56.792282+0000","last_unstale":"2024-05-14T09:33:29.080195+0000","last_undegraded":"2024-05-14T09:33:29.080195+0000","last_fullsized":"2024-05-14T09:33:29.080195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:52:35.319479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076831+0000","last_change":"2024-05-14T09:30:56.791575+0000","last_active":"2024-05-14T09:33:29.076831+0000","last_peered":"2024-05-14T09:33:29.076831+0000","last_clean":"2024-05-14T09:33:29.076831+0000","last_became_active":"2024-05-14T09:30:56.791230+0000","last_became_peered":"2024-05-14T09:30:56.791230+0000","last_unstale":"2024-05-14T09:33:29.076831+0000","last_undegraded":"2024-05-14T09:33:29.076831+0000","last_fullsized":"2024-05-14T09:33:29.076831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:40.510062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.088030+0000","last_change":"2024-05-14T09:30:56.796996+0000","last_active":"2024-05-14T09:33:29.088030+0000","last_peered":"2024-05-14T09:33:29.088030+0000","last_clean":"2024-05-14T09:33:29.088030+0000","last_became_active":"2024-05-14T09:30:56.796333+0000","last_became_peered":"2024-05-14T09:30:56.796333+0000","last_unstale":"2024-05-14T09:33:29.088030+0000","last_undegraded":"2024-05-14T09:33:29.088030+0000","last_fullsized":"2024-05-14T09:33:29.088030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:29:34.799088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.083938+0000","last_change":"2024-05-14T09:30:56.785740+0000","last_active":"2024-05-14T09:33:29.083938+0000","last_peered":"2024-05-14T09:33:29.083938+0000","last_clean":"2024-05-14T09:33:29.083938+0000","last_became_active":"2024-05-14T09:30:56.785394+0000","last_became_peered":"2024-05-14T09:30:56.785394+0000","last_unstale":"2024-05-14T09:33:29.083938+0000","last_undegraded":"2024-05-14T09:33:29.083938+0000","last_fullsized":"2024-05-14T09:33:29.083938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:17.819523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.079641+0000","last_change":"2024-05-14T09:30:56.796957+0000","last_active":"2024-05-14T09:33:29.079641+0000","last_peered":"2024-05-14T09:33:29.079641+0000","last_clean":"2024-05-14T09:33:29.079641+0000","last_became_active":"2024-05-14T09:30:56.795071+0000","last_became_peered":"2024-05-14T09:30:56.795071+0000","last_unstale":"2024-05-14T09:33:29.079641+0000","last_undegraded":"2024-05-14T09:33:29.079641+0000","last_fullsized":"2024-05-14T09:33:29.079641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:44:31.480207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245291,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69067,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1079,"num_read_kb":613,"num_write":183,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2846,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":207201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678078,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":519087,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874308,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":1174416,"data_compressed":67998,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037834,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":518671,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1597,"internal_metadata":36174275},"hb_peers":[0,1,2,3,4,5,6,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234062,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":518009,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430292,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":1114367,"data_compressed":67998,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593817,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":514429,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":578956,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":515235,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":515365,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149799,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":514041,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953572,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":583114,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790047,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1110467,"data_compressed":67998,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33883,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33883,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":65199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T09:34:55.520 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T09:34:55.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.670+0000 7ff39aa15640 1 -- 172.21.15.2:0/1133926344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff394104780 msgr2=0x7ff394106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:55.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.670+0000 7ff39aa15640 1 --2- 172.21.15.2:0/1133926344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff394104780 0x7ff394106ba0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff384009900 tx=0x7ff38402a410 comp rx=0 tx=0).stop 2024-05-14T09:34:55.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.670+0000 7ff39aa15640 1 -- 172.21.15.2:0/1133926344 shutdown_connections 2024-05-14T09:34:55.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.670+0000 7ff39aa15640 1 --2- 172.21.15.2:0/1133926344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff394109a40 0x7ff39410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.670+0000 7ff39aa15640 1 --2- 172.21.15.2:0/1133926344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3941070e0 0x7ff394109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.671+0000 7ff39aa15640 1 --2- 172.21.15.2:0/1133926344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff394104780 0x7ff394106ba0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.671+0000 7ff39aa15640 1 -- 172.21.15.2:0/1133926344 >> 172.21.15.2:0/1133926344 conn(0x7ff3940fdda0 msgr2=0x7ff394100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:55.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.671+0000 7ff39aa15640 1 -- 172.21.15.2:0/1133926344 shutdown_connections 2024-05-14T09:34:55.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.671+0000 7ff39aa15640 1 -- 172.21.15.2:0/1133926344 wait complete. 2024-05-14T09:34:55.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.671+0000 7ff39aa15640 1 Processor -- start 2024-05-14T09:34:55.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.671+0000 7ff39aa15640 1 -- start start 2024-05-14T09:34:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.671+0000 7ff39aa15640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff394104780 0x7ff39419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.671+0000 7ff39aa15640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3941070e0 0x7ff39419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff39aa15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff394109a40 0x7ff3941a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff39aa15640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410f720 con 0x7ff394104780 2024-05-14T09:34:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff39aa15640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410f5a0 con 0x7ff394109a40 2024-05-14T09:34:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff39aa15640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410f8a0 con 0x7ff3941070e0 2024-05-14T09:34:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff393fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff394104780 0x7ff39419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff393fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff394104780 0x7ff39419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54654/0 (socket says 172.21.15.2:54654) 2024-05-14T09:34:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff393fff640 1 -- 172.21.15.2:0/2511431123 learned_addr learned my addr 172.21.15.2:0/2511431123 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff398fb2640 1 --2- 172.21.15.2:0/2511431123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff394109a40 0x7ff3941a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff38bfff640 1 --2- 172.21.15.2:0/2511431123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3941070e0 0x7ff39419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff393fff640 1 -- 172.21.15.2:0/2511431123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3941070e0 msgr2=0x7ff39419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:55.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff393fff640 1 --2- 172.21.15.2:0/2511431123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3941070e0 0x7ff39419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff393fff640 1 -- 172.21.15.2:0/2511431123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff394109a40 msgr2=0x7ff3941a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:55.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff393fff640 1 --2- 172.21.15.2:0/2511431123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff394109a40 0x7ff3941a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff393fff640 1 -- 172.21.15.2:0/2511431123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3941a1d80 con 0x7ff394104780 2024-05-14T09:34:55.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff393fff640 1 --2- 172.21.15.2:0/2511431123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff394104780 0x7ff39419a2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff384006010 tx=0x7ff384031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:55.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff391ffb640 1 -- 172.21.15.2:0/2511431123 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38403f030 con 0x7ff394104780 2024-05-14T09:34:55.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff391ffb640 1 -- 172.21.15.2:0/2511431123 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff384003aa0 con 0x7ff394104780 2024-05-14T09:34:55.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff39aa15640 1 -- 172.21.15.2:0/2511431123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3941a2010 con 0x7ff394104780 2024-05-14T09:34:55.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.672+0000 7ff391ffb640 1 -- 172.21.15.2:0/2511431123 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384035070 con 0x7ff394104780 2024-05-14T09:34:55.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.673+0000 7ff39aa15640 1 -- 172.21.15.2:0/2511431123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3941a2550 con 0x7ff394104780 2024-05-14T09:34:55.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.674+0000 7ff39aa15640 1 -- 172.21.15.2:0/2511431123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3940ff8c0 con 0x7ff394104780 2024-05-14T09:34:55.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.675+0000 7ff391ffb640 1 -- 172.21.15.2:0/2511431123 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff384042030 con 0x7ff394104780 2024-05-14T09:34:55.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.676+0000 7ff391ffb640 1 --2- 172.21.15.2:0/2511431123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3800b2920 0x7ff3800b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:55.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.676+0000 7ff38bfff640 1 --2- 172.21.15.2:0/2511431123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3800b2920 0x7ff3800b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:55.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.676+0000 7ff391ffb640 1 -- 172.21.15.2:0/2511431123 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7ff384102ff0 con 0x7ff394104780 2024-05-14T09:34:55.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.677+0000 7ff38bfff640 1 --2- 172.21.15.2:0/2511431123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3800b2920 0x7ff3800b4e30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff37c005e10 tx=0x7ff37c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:55.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.679+0000 7ff391ffb640 1 -- 172.21.15.2:0/2511431123 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff3840ca8d0 con 0x7ff394104780 2024-05-14T09:34:55.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.868+0000 7ff39aa15640 1 -- 172.21.15.2:0/2511431123 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff394063110 con 0x7ff3800b2920 2024-05-14T09:34:55.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.878+0000 7ff391ffb640 1 -- 172.21.15.2:0/2511431123 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340740 (secure 0 0 0) 0x7ff394063110 con 0x7ff3800b2920 2024-05-14T09:34:55.881 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:34:55.883 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-05-14T09:34:55.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.882+0000 7ff39aa15640 1 -- 172.21.15.2:0/2511431123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3800b2920 msgr2=0x7ff3800b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:55.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.882+0000 7ff39aa15640 1 --2- 172.21.15.2:0/2511431123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3800b2920 0x7ff3800b4e30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff37c005e10 tx=0x7ff37c005da0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.882+0000 7ff39aa15640 1 -- 172.21.15.2:0/2511431123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff394104780 msgr2=0x7ff39419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:55.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.882+0000 7ff39aa15640 1 --2- 172.21.15.2:0/2511431123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff394104780 0x7ff39419a2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff384006010 tx=0x7ff384031a50 comp rx=0 tx=0).stop 2024-05-14T09:34:55.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.882+0000 7ff39aa15640 1 -- 172.21.15.2:0/2511431123 shutdown_connections 2024-05-14T09:34:55.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.882+0000 7ff39aa15640 1 --2- 172.21.15.2:0/2511431123 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3800b2920 0x7ff3800b4e30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.882+0000 7ff39aa15640 1 --2- 172.21.15.2:0/2511431123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff394109a40 0x7ff3941a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.882+0000 7ff39aa15640 1 --2- 172.21.15.2:0/2511431123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3941070e0 0x7ff39419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.882+0000 7ff39aa15640 1 --2- 172.21.15.2:0/2511431123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff394104780 0x7ff39419a2f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:55.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.882+0000 7ff39aa15640 1 -- 172.21.15.2:0/2511431123 >> 172.21.15.2:0/2511431123 conn(0x7ff3940fdda0 msgr2=0x7ff394105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:55.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.882+0000 7ff39aa15640 1 -- 172.21.15.2:0/2511431123 shutdown_connections 2024-05-14T09:34:55.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:55.882+0000 7ff39aa15640 1 -- 172.21.15.2:0/2511431123 wait complete. 2024-05-14T09:34:55.893 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-05-14T09:34:55.107700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659669,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1155,"num_read_kb":677,"num_write":308,"num_write_kb":2281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2954686,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":750,"ondisk_log_size":750,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":394500,"kb_used_data":12068,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124547324,"statfs":{"total":1151940427776,"available":1151536459776,"internally_reserved":0,"allocated":12357632,"data_stored":8176157,"data_compressed":496431,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19140,"internal_metadata":391492924},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":32,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003660"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087224+0000","last_change":"2024-05-14T09:30:56.797051+0000","last_active":"2024-05-14T09:33:29.087224+0000","last_peered":"2024-05-14T09:33:29.087224+0000","last_clean":"2024-05-14T09:33:29.087224+0000","last_became_active":"2024-05-14T09:30:56.796517+0000","last_became_peered":"2024-05-14T09:30:56.796517+0000","last_unstale":"2024-05-14T09:33:29.087224+0000","last_undegraded":"2024-05-14T09:33:29.087224+0000","last_fullsized":"2024-05-14T09:33:29.087224+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:58:22.170187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.073126+0000","last_change":"2024-05-14T09:30:56.795325+0000","last_active":"2024-05-14T09:33:29.073126+0000","last_peered":"2024-05-14T09:33:29.073126+0000","last_clean":"2024-05-14T09:33:29.073126+0000","last_became_active":"2024-05-14T09:30:56.794962+0000","last_became_peered":"2024-05-14T09:30:56.794962+0000","last_unstale":"2024-05-14T09:33:29.073126+0000","last_undegraded":"2024-05-14T09:33:29.073126+0000","last_fullsized":"2024-05-14T09:33:29.073126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:15:28.890565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084179+0000","last_change":"2024-05-14T09:30:56.793200+0000","last_active":"2024-05-14T09:33:29.084179+0000","last_peered":"2024-05-14T09:33:29.084179+0000","last_clean":"2024-05-14T09:33:29.084179+0000","last_became_active":"2024-05-14T09:30:56.792835+0000","last_became_peered":"2024-05-14T09:30:56.792835+0000","last_unstale":"2024-05-14T09:33:29.084179+0000","last_undegraded":"2024-05-14T09:33:29.084179+0000","last_fullsized":"2024-05-14T09:33:29.084179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:19.851276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.070997+0000","last_change":"2024-05-14T09:30:56.790238+0000","last_active":"2024-05-14T09:33:29.070997+0000","last_peered":"2024-05-14T09:33:29.070997+0000","last_clean":"2024-05-14T09:33:29.070997+0000","last_became_active":"2024-05-14T09:30:56.789826+0000","last_became_peered":"2024-05-14T09:30:56.789826+0000","last_unstale":"2024-05-14T09:33:29.070997+0000","last_undegraded":"2024-05-14T09:33:29.070997+0000","last_fullsized":"2024-05-14T09:33:29.070997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:19:19.853931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080306+0000","last_change":"2024-05-14T09:30:56.788021+0000","last_active":"2024-05-14T09:33:29.080306+0000","last_peered":"2024-05-14T09:33:29.080306+0000","last_clean":"2024-05-14T09:33:29.080306+0000","last_became_active":"2024-05-14T09:30:56.787657+0000","last_became_peered":"2024-05-14T09:30:56.787657+0000","last_unstale":"2024-05-14T09:33:29.080306+0000","last_undegraded":"2024-05-14T09:33:29.080306+0000","last_fullsized":"2024-05-14T09:33:29.080306+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:37:03.232137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.082547+0000","last_change":"2024-05-14T09:32:30.525175+0000","last_active":"2024-05-14T09:33:29.082547+0000","last_peered":"2024-05-14T09:33:29.082547+0000","last_clean":"2024-05-14T09:33:29.082547+0000","last_became_active":"2024-05-14T09:32:30.524861+0000","last_became_peered":"2024-05-14T09:32:30.524861+0000","last_unstale":"2024-05-14T09:33:29.082547+0000","last_undegraded":"2024-05-14T09:33:29.082547+0000","last_fullsized":"2024-05-14T09:33:29.082547+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:22:21.063843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.066050+0000","last_change":"2024-05-14T09:30:56.795647+0000","last_active":"2024-05-14T09:33:29.066050+0000","last_peered":"2024-05-14T09:33:29.066050+0000","last_clean":"2024-05-14T09:33:29.066050+0000","last_became_active":"2024-05-14T09:30:56.795186+0000","last_became_peered":"2024-05-14T09:30:56.795186+0000","last_unstale":"2024-05-14T09:33:29.066050+0000","last_undegraded":"2024-05-14T09:33:29.066050+0000","last_fullsized":"2024-05-14T09:33:29.066050+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:04:53.471305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081501+0000","last_change":"2024-05-14T09:30:56.796834+0000","last_active":"2024-05-14T09:33:29.081501+0000","last_peered":"2024-05-14T09:33:29.081501+0000","last_clean":"2024-05-14T09:33:29.081501+0000","last_became_active":"2024-05-14T09:30:56.796492+0000","last_became_peered":"2024-05-14T09:30:56.796492+0000","last_unstale":"2024-05-14T09:33:29.081501+0000","last_undegraded":"2024-05-14T09:33:29.081501+0000","last_fullsized":"2024-05-14T09:33:29.081501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:46.606129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.086070+0000","last_change":"2024-05-14T09:30:56.795020+0000","last_active":"2024-05-14T09:33:29.086070+0000","last_peered":"2024-05-14T09:33:29.086070+0000","last_clean":"2024-05-14T09:33:29.086070+0000","last_became_active":"2024-05-14T09:30:56.794625+0000","last_became_peered":"2024-05-14T09:30:56.794625+0000","last_unstale":"2024-05-14T09:33:29.086070+0000","last_undegraded":"2024-05-14T09:33:29.086070+0000","last_fullsized":"2024-05-14T09:33:29.086070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:09:38.597296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.082128+0000","last_change":"2024-05-14T09:30:56.796439+0000","last_active":"2024-05-14T09:33:29.082128+0000","last_peered":"2024-05-14T09:33:29.082128+0000","last_clean":"2024-05-14T09:33:29.082128+0000","last_became_active":"2024-05-14T09:30:56.795168+0000","last_became_peered":"2024-05-14T09:30:56.795168+0000","last_unstale":"2024-05-14T09:33:29.082128+0000","last_undegraded":"2024-05-14T09:33:29.082128+0000","last_fullsized":"2024-05-14T09:33:29.082128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:47.660286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087219+0000","last_change":"2024-05-14T09:30:56.795994+0000","last_active":"2024-05-14T09:33:29.087219+0000","last_peered":"2024-05-14T09:33:29.087219+0000","last_clean":"2024-05-14T09:33:29.087219+0000","last_became_active":"2024-05-14T09:30:56.795079+0000","last_became_peered":"2024-05-14T09:30:56.795079+0000","last_unstale":"2024-05-14T09:33:29.087219+0000","last_undegraded":"2024-05-14T09:33:29.087219+0000","last_fullsized":"2024-05-14T09:33:29.087219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:37:42.930195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.072741+0000","last_change":"2024-05-14T09:30:56.792997+0000","last_active":"2024-05-14T09:33:29.072741+0000","last_peered":"2024-05-14T09:33:29.072741+0000","last_clean":"2024-05-14T09:33:29.072741+0000","last_became_active":"2024-05-14T09:30:56.792302+0000","last_became_peered":"2024-05-14T09:30:56.792302+0000","last_unstale":"2024-05-14T09:33:29.072741+0000","last_undegraded":"2024-05-14T09:33:29.072741+0000","last_fullsized":"2024-05-14T09:33:29.072741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:52.986662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077933+0000","last_change":"2024-05-14T09:30:56.795903+0000","last_active":"2024-05-14T09:33:29.077933+0000","last_peered":"2024-05-14T09:33:29.077933+0000","last_clean":"2024-05-14T09:33:29.077933+0000","last_became_active":"2024-05-14T09:30:56.795466+0000","last_became_peered":"2024-05-14T09:30:56.795466+0000","last_unstale":"2024-05-14T09:33:29.077933+0000","last_undegraded":"2024-05-14T09:33:29.077933+0000","last_fullsized":"2024-05-14T09:33:29.077933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:02:10.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.070939+0000","last_change":"2024-05-14T09:30:56.795911+0000","last_active":"2024-05-14T09:33:29.070939+0000","last_peered":"2024-05-14T09:33:29.070939+0000","last_clean":"2024-05-14T09:33:29.070939+0000","last_became_active":"2024-05-14T09:30:56.795547+0000","last_became_peered":"2024-05-14T09:30:56.795547+0000","last_unstale":"2024-05-14T09:33:29.070939+0000","last_undegraded":"2024-05-14T09:33:29.070939+0000","last_fullsized":"2024-05-14T09:33:29.070939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:56:29.621783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.067755+0000","last_change":"2024-05-14T09:30:56.787745+0000","last_active":"2024-05-14T09:33:29.067755+0000","last_peered":"2024-05-14T09:33:29.067755+0000","last_clean":"2024-05-14T09:33:29.067755+0000","last_became_active":"2024-05-14T09:30:56.787385+0000","last_became_peered":"2024-05-14T09:30:56.787385+0000","last_unstale":"2024-05-14T09:33:29.067755+0000","last_undegraded":"2024-05-14T09:33:29.067755+0000","last_fullsized":"2024-05-14T09:33:29.067755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:11.256775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087830+0000","last_change":"2024-05-14T09:30:56.795785+0000","last_active":"2024-05-14T09:33:29.087830+0000","last_peered":"2024-05-14T09:33:29.087830+0000","last_clean":"2024-05-14T09:33:29.087830+0000","last_became_active":"2024-05-14T09:30:56.795436+0000","last_became_peered":"2024-05-14T09:30:56.795436+0000","last_unstale":"2024-05-14T09:33:29.087830+0000","last_undegraded":"2024-05-14T09:33:29.087830+0000","last_fullsized":"2024-05-14T09:33:29.087830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:12:09.221929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087321+0000","last_change":"2024-05-14T09:30:56.800688+0000","last_active":"2024-05-14T09:33:29.087321+0000","last_peered":"2024-05-14T09:33:29.087321+0000","last_clean":"2024-05-14T09:33:29.087321+0000","last_became_active":"2024-05-14T09:30:56.800204+0000","last_became_peered":"2024-05-14T09:30:56.800204+0000","last_unstale":"2024-05-14T09:33:29.087321+0000","last_undegraded":"2024-05-14T09:33:29.087321+0000","last_fullsized":"2024-05-14T09:33:29.087321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:32:27.057688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.075054+0000","last_change":"2024-05-14T09:30:56.795604+0000","last_active":"2024-05-14T09:33:29.075054+0000","last_peered":"2024-05-14T09:33:29.075054+0000","last_clean":"2024-05-14T09:33:29.075054+0000","last_became_active":"2024-05-14T09:30:56.795245+0000","last_became_peered":"2024-05-14T09:30:56.795245+0000","last_unstale":"2024-05-14T09:33:29.075054+0000","last_undegraded":"2024-05-14T09:33:29.075054+0000","last_fullsized":"2024-05-14T09:33:29.075054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:14.072584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080216+0000","last_change":"2024-05-14T09:30:56.794815+0000","last_active":"2024-05-14T09:33:29.080216+0000","last_peered":"2024-05-14T09:33:29.080216+0000","last_clean":"2024-05-14T09:33:29.080216+0000","last_became_active":"2024-05-14T09:30:56.794429+0000","last_became_peered":"2024-05-14T09:30:56.794429+0000","last_unstale":"2024-05-14T09:33:29.080216+0000","last_undegraded":"2024-05-14T09:33:29.080216+0000","last_fullsized":"2024-05-14T09:33:29.080216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:32:08.367046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319506+0000","last_change":"2024-05-14T09:31:08.107895+0000","last_active":"2024-05-14T09:32:30.319506+0000","last_peered":"2024-05-14T09:32:30.319506+0000","last_clean":"2024-05-14T09:32:30.319506+0000","last_became_active":"2024-05-14T09:31:08.107458+0000","last_became_peered":"2024-05-14T09:31:08.107458+0000","last_unstale":"2024-05-14T09:32:30.319506+0000","last_undegraded":"2024-05-14T09:32:30.319506+0000","last_fullsized":"2024-05-14T09:32:30.319506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:13:11.717753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":69,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319382+0000","last_change":"2024-05-14T09:31:30.178197+0000","last_active":"2024-05-14T09:32:30.319382+0000","last_peered":"2024-05-14T09:32:30.319382+0000","last_clean":"2024-05-14T09:32:30.319382+0000","last_became_active":"2024-05-14T09:31:30.171154+0000","last_became_peered":"2024-05-14T09:31:30.171154+0000","last_unstale":"2024-05-14T09:32:30.319382+0000","last_undegraded":"2024-05-14T09:32:30.319382+0000","last_fullsized":"2024-05-14T09:32:30.319382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:43:46.208439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077181+0000","last_change":"2024-05-14T09:30:56.800526+0000","last_active":"2024-05-14T09:33:29.077181+0000","last_peered":"2024-05-14T09:33:29.077181+0000","last_clean":"2024-05-14T09:33:29.077181+0000","last_became_active":"2024-05-14T09:30:56.797692+0000","last_became_peered":"2024-05-14T09:30:56.797692+0000","last_unstale":"2024-05-14T09:33:29.077181+0000","last_undegraded":"2024-05-14T09:33:29.077181+0000","last_fullsized":"2024-05-14T09:33:29.077181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:09:14.471173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522407+0000","last_change":"2024-05-14T09:31:30.173810+0000","last_active":"2024-05-14T09:32:30.522407+0000","last_peered":"2024-05-14T09:32:30.522407+0000","last_clean":"2024-05-14T09:32:30.522407+0000","last_became_active":"2024-05-14T09:31:30.173242+0000","last_became_peered":"2024-05-14T09:31:30.173242+0000","last_unstale":"2024-05-14T09:32:30.522407+0000","last_undegraded":"2024-05-14T09:32:30.522407+0000","last_fullsized":"2024-05-14T09:32:30.522407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:37:15.991805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.322883+0000","last_change":"2024-05-14T09:30:54.015636+0000","last_active":"2024-05-14T09:32:30.322883+0000","last_peered":"2024-05-14T09:32:30.322883+0000","last_clean":"2024-05-14T09:32:30.322883+0000","last_became_active":"2024-05-14T09:30:54.015290+0000","last_became_peered":"2024-05-14T09:30:54.015290+0000","last_unstale":"2024-05-14T09:32:30.322883+0000","last_undegraded":"2024-05-14T09:32:30.322883+0000","last_fullsized":"2024-05-14T09:32:30.322883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:29:17.548817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.088634+0000","last_change":"2024-05-14T09:30:56.799506+0000","last_active":"2024-05-14T09:33:29.088634+0000","last_peered":"2024-05-14T09:33:29.088634+0000","last_clean":"2024-05-14T09:33:29.088634+0000","last_became_active":"2024-05-14T09:30:56.799153+0000","last_became_peered":"2024-05-14T09:30:56.799153+0000","last_unstale":"2024-05-14T09:33:29.088634+0000","last_undegraded":"2024-05-14T09:33:29.088634+0000","last_fullsized":"2024-05-14T09:33:29.088634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:04:08.217337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522681+0000","last_change":"2024-05-14T09:31:08.115533+0000","last_active":"2024-05-14T09:32:30.522681+0000","last_peered":"2024-05-14T09:32:30.522681+0000","last_clean":"2024-05-14T09:32:30.522681+0000","last_became_active":"2024-05-14T09:31:08.115033+0000","last_became_peered":"2024-05-14T09:31:08.115033+0000","last_unstale":"2024-05-14T09:32:30.522681+0000","last_undegraded":"2024-05-14T09:32:30.522681+0000","last_fullsized":"2024-05-14T09:32:30.522681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:15.668357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524484+0000","last_change":"2024-05-14T09:30:53.770547+0000","last_active":"2024-05-14T09:32:30.524484+0000","last_peered":"2024-05-14T09:32:30.524484+0000","last_clean":"2024-05-14T09:32:30.524484+0000","last_became_active":"2024-05-14T09:30:53.770110+0000","last_became_peered":"2024-05-14T09:30:53.770110+0000","last_unstale":"2024-05-14T09:32:30.524484+0000","last_undegraded":"2024-05-14T09:32:30.524484+0000","last_fullsized":"2024-05-14T09:32:30.524484+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:18.371318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084457+0000","last_change":"2024-05-14T09:30:56.799427+0000","last_active":"2024-05-14T09:33:29.084457+0000","last_peered":"2024-05-14T09:33:29.084457+0000","last_clean":"2024-05-14T09:33:29.084457+0000","last_became_active":"2024-05-14T09:30:56.799018+0000","last_became_peered":"2024-05-14T09:30:56.799018+0000","last_unstale":"2024-05-14T09:33:29.084457+0000","last_undegraded":"2024-05-14T09:33:29.084457+0000","last_fullsized":"2024-05-14T09:33:29.084457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:13:31.837985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522635+0000","last_change":"2024-05-14T09:31:08.116258+0000","last_active":"2024-05-14T09:32:30.522635+0000","last_peered":"2024-05-14T09:32:30.522635+0000","last_clean":"2024-05-14T09:32:30.522635+0000","last_became_active":"2024-05-14T09:31:08.115040+0000","last_became_peered":"2024-05-14T09:31:08.115040+0000","last_unstale":"2024-05-14T09:32:30.522635+0000","last_undegraded":"2024-05-14T09:32:30.522635+0000","last_fullsized":"2024-05-14T09:32:30.522635+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:57:54.839203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523793+0000","last_change":"2024-05-14T09:31:30.188984+0000","last_active":"2024-05-14T09:32:30.523793+0000","last_peered":"2024-05-14T09:32:30.523793+0000","last_clean":"2024-05-14T09:32:30.523793+0000","last_became_active":"2024-05-14T09:31:30.174410+0000","last_became_peered":"2024-05-14T09:31:30.174410+0000","last_unstale":"2024-05-14T09:32:30.523793+0000","last_undegraded":"2024-05-14T09:32:30.523793+0000","last_fullsized":"2024-05-14T09:32:30.523793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:01:35.713857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,4],"acting":[8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081749+0000","last_change":"2024-05-14T09:32:30.319390+0000","last_active":"2024-05-14T09:33:29.081749+0000","last_peered":"2024-05-14T09:33:29.081749+0000","last_clean":"2024-05-14T09:33:29.081749+0000","last_became_active":"2024-05-14T09:32:30.318645+0000","last_became_peered":"2024-05-14T09:32:30.318645+0000","last_unstale":"2024-05-14T09:33:29.081749+0000","last_undegraded":"2024-05-14T09:33:29.081749+0000","last_fullsized":"2024-05-14T09:33:29.081749+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:37:49.866996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320705+0000","last_change":"2024-05-14T09:31:08.115740+0000","last_active":"2024-05-14T09:32:30.320705+0000","last_peered":"2024-05-14T09:32:30.320705+0000","last_clean":"2024-05-14T09:32:30.320705+0000","last_became_active":"2024-05-14T09:31:08.115202+0000","last_became_peered":"2024-05-14T09:31:08.115202+0000","last_unstale":"2024-05-14T09:32:30.320705+0000","last_undegraded":"2024-05-14T09:32:30.320705+0000","last_fullsized":"2024-05-14T09:32:30.320705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:44.519280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":233,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:34:51.508971+0000","last_change":"2024-05-14T09:30:53.770869+0000","last_active":"2024-05-14T09:34:51.508971+0000","last_peered":"2024-05-14T09:34:51.508971+0000","last_clean":"2024-05-14T09:34:51.508971+0000","last_became_active":"2024-05-14T09:30:53.770425+0000","last_became_peered":"2024-05-14T09:30:53.770425+0000","last_unstale":"2024-05-14T09:34:51.508971+0000","last_undegraded":"2024-05-14T09:34:51.508971+0000","last_fullsized":"2024-05-14T09:34:51.508971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:03:33.768120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":180,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.059353+0000","last_change":"2024-05-14T09:30:56.789267+0000","last_active":"2024-05-14T09:33:29.059353+0000","last_peered":"2024-05-14T09:33:29.059353+0000","last_clean":"2024-05-14T09:33:29.059353+0000","last_became_active":"2024-05-14T09:30:56.788883+0000","last_became_peered":"2024-05-14T09:30:56.788883+0000","last_unstale":"2024-05-14T09:33:29.059353+0000","last_undegraded":"2024-05-14T09:33:29.059353+0000","last_fullsized":"2024-05-14T09:33:29.059353+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:09.818693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320595+0000","last_change":"2024-05-14T09:31:08.115193+0000","last_active":"2024-05-14T09:32:30.320595+0000","last_peered":"2024-05-14T09:32:30.320595+0000","last_clean":"2024-05-14T09:32:30.320595+0000","last_became_active":"2024-05-14T09:31:08.114701+0000","last_became_peered":"2024-05-14T09:31:08.114701+0000","last_unstale":"2024-05-14T09:32:30.320595+0000","last_undegraded":"2024-05-14T09:32:30.320595+0000","last_fullsized":"2024-05-14T09:32:30.320595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:47:48.487922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319686+0000","last_change":"2024-05-14T09:31:30.174760+0000","last_active":"2024-05-14T09:32:30.319686+0000","last_peered":"2024-05-14T09:32:30.319686+0000","last_clean":"2024-05-14T09:32:30.319686+0000","last_became_active":"2024-05-14T09:31:30.174253+0000","last_became_peered":"2024-05-14T09:31:30.174253+0000","last_unstale":"2024-05-14T09:32:30.319686+0000","last_undegraded":"2024-05-14T09:32:30.319686+0000","last_fullsized":"2024-05-14T09:32:30.319686+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:11:17.163436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081148+0000","last_change":"2024-05-14T09:30:56.794654+0000","last_active":"2024-05-14T09:33:29.081148+0000","last_peered":"2024-05-14T09:33:29.081148+0000","last_clean":"2024-05-14T09:33:29.081148+0000","last_became_active":"2024-05-14T09:30:56.794282+0000","last_became_peered":"2024-05-14T09:30:56.794282+0000","last_unstale":"2024-05-14T09:33:29.081148+0000","last_undegraded":"2024-05-14T09:33:29.081148+0000","last_fullsized":"2024-05-14T09:33:29.081148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:55:41.478707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320586+0000","last_change":"2024-05-14T09:31:08.119965+0000","last_active":"2024-05-14T09:32:30.320586+0000","last_peered":"2024-05-14T09:32:30.320586+0000","last_clean":"2024-05-14T09:32:30.320586+0000","last_became_active":"2024-05-14T09:31:08.119502+0000","last_became_peered":"2024-05-14T09:31:08.119502+0000","last_unstale":"2024-05-14T09:32:30.320586+0000","last_undegraded":"2024-05-14T09:32:30.320586+0000","last_fullsized":"2024-05-14T09:32:30.320586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:01:23.612036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.314804+0000","last_change":"2024-05-14T09:30:53.766495+0000","last_active":"2024-05-14T09:32:30.314804+0000","last_peered":"2024-05-14T09:32:30.314804+0000","last_clean":"2024-05-14T09:32:30.314804+0000","last_became_active":"2024-05-14T09:30:53.766122+0000","last_became_peered":"2024-05-14T09:30:53.766122+0000","last_unstale":"2024-05-14T09:32:30.314804+0000","last_undegraded":"2024-05-14T09:32:30.314804+0000","last_fullsized":"2024-05-14T09:32:30.314804+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:43:13.344209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.073417+0000","last_change":"2024-05-14T09:30:56.791938+0000","last_active":"2024-05-14T09:33:29.073417+0000","last_peered":"2024-05-14T09:33:29.073417+0000","last_clean":"2024-05-14T09:33:29.073417+0000","last_became_active":"2024-05-14T09:30:56.791525+0000","last_became_peered":"2024-05-14T09:30:56.791525+0000","last_unstale":"2024-05-14T09:33:29.073417+0000","last_undegraded":"2024-05-14T09:33:29.073417+0000","last_fullsized":"2024-05-14T09:33:29.073417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:18.178208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320104+0000","last_change":"2024-05-14T09:31:08.113548+0000","last_active":"2024-05-14T09:32:30.320104+0000","last_peered":"2024-05-14T09:32:30.320104+0000","last_clean":"2024-05-14T09:32:30.320104+0000","last_became_active":"2024-05-14T09:31:08.113117+0000","last_became_peered":"2024-05-14T09:31:08.113117+0000","last_unstale":"2024-05-14T09:32:30.320104+0000","last_undegraded":"2024-05-14T09:32:30.320104+0000","last_fullsized":"2024-05-14T09:32:30.320104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:25:15.483761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315966+0000","last_change":"2024-05-14T09:30:53.770987+0000","last_active":"2024-05-14T09:32:30.315966+0000","last_peered":"2024-05-14T09:32:30.315966+0000","last_clean":"2024-05-14T09:32:30.315966+0000","last_became_active":"2024-05-14T09:30:53.770609+0000","last_became_peered":"2024-05-14T09:30:53.770609+0000","last_unstale":"2024-05-14T09:32:30.315966+0000","last_undegraded":"2024-05-14T09:32:30.315966+0000","last_fullsized":"2024-05-14T09:32:30.315966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:26:22.020642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081149+0000","last_change":"2024-05-14T09:30:56.789219+0000","last_active":"2024-05-14T09:33:29.081149+0000","last_peered":"2024-05-14T09:33:29.081149+0000","last_clean":"2024-05-14T09:33:29.081149+0000","last_became_active":"2024-05-14T09:30:56.788871+0000","last_became_peered":"2024-05-14T09:30:56.788871+0000","last_unstale":"2024-05-14T09:33:29.081149+0000","last_undegraded":"2024-05-14T09:33:29.081149+0000","last_fullsized":"2024-05-14T09:33:29.081149+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:10:50.774267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.322531+0000","last_change":"2024-05-14T09:31:30.174708+0000","last_active":"2024-05-14T09:32:30.322531+0000","last_peered":"2024-05-14T09:32:30.322531+0000","last_clean":"2024-05-14T09:32:30.322531+0000","last_became_active":"2024-05-14T09:31:30.174199+0000","last_became_peered":"2024-05-14T09:31:30.174199+0000","last_unstale":"2024-05-14T09:32:30.322531+0000","last_undegraded":"2024-05-14T09:32:30.322531+0000","last_fullsized":"2024-05-14T09:32:30.322531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:23:55.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":225,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:34:51.508350+0000","last_change":"2024-05-14T09:30:53.770898+0000","last_active":"2024-05-14T09:34:51.508350+0000","last_peered":"2024-05-14T09:34:51.508350+0000","last_clean":"2024-05-14T09:34:51.508350+0000","last_became_active":"2024-05-14T09:30:53.770483+0000","last_became_peered":"2024-05-14T09:30:53.770483+0000","last_unstale":"2024-05-14T09:34:51.508350+0000","last_undegraded":"2024-05-14T09:34:51.508350+0000","last_fullsized":"2024-05-14T09:34:51.508350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:07.764348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084404+0000","last_change":"2024-05-14T09:30:56.799049+0000","last_active":"2024-05-14T09:33:29.084404+0000","last_peered":"2024-05-14T09:33:29.084404+0000","last_clean":"2024-05-14T09:33:29.084404+0000","last_became_active":"2024-05-14T09:30:56.798643+0000","last_became_peered":"2024-05-14T09:30:56.798643+0000","last_unstale":"2024-05-14T09:33:29.084404+0000","last_undegraded":"2024-05-14T09:33:29.084404+0000","last_fullsized":"2024-05-14T09:33:29.084404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:12.378373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.318070+0000","last_change":"2024-05-14T09:31:08.109876+0000","last_active":"2024-05-14T09:32:30.318070+0000","last_peered":"2024-05-14T09:32:30.318070+0000","last_clean":"2024-05-14T09:32:30.318070+0000","last_became_active":"2024-05-14T09:31:08.109470+0000","last_became_peered":"2024-05-14T09:31:08.109470+0000","last_unstale":"2024-05-14T09:32:30.318070+0000","last_undegraded":"2024-05-14T09:32:30.318070+0000","last_fullsized":"2024-05-14T09:32:30.318070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:07.704153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.015003+0000","last_change":"2024-05-14T09:31:30.179086+0000","last_active":"2024-05-14T09:32:31.015003+0000","last_peered":"2024-05-14T09:32:31.015003+0000","last_clean":"2024-05-14T09:32:31.015003+0000","last_became_active":"2024-05-14T09:31:30.178604+0000","last_became_peered":"2024-05-14T09:31:30.178604+0000","last_unstale":"2024-05-14T09:32:31.015003+0000","last_undegraded":"2024-05-14T09:32:31.015003+0000","last_fullsized":"2024-05-14T09:32:31.015003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:39.712395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.073315+0000","last_change":"2024-05-14T09:30:56.793109+0000","last_active":"2024-05-14T09:33:29.073315+0000","last_peered":"2024-05-14T09:33:29.073315+0000","last_clean":"2024-05-14T09:33:29.073315+0000","last_became_active":"2024-05-14T09:30:56.792761+0000","last_became_peered":"2024-05-14T09:30:56.792761+0000","last_unstale":"2024-05-14T09:33:29.073315+0000","last_undegraded":"2024-05-14T09:33:29.073315+0000","last_fullsized":"2024-05-14T09:33:29.073315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:25:33.216281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522916+0000","last_change":"2024-05-14T09:31:08.120978+0000","last_active":"2024-05-14T09:32:30.522916+0000","last_peered":"2024-05-14T09:32:30.522916+0000","last_clean":"2024-05-14T09:32:30.522916+0000","last_became_active":"2024-05-14T09:31:08.120227+0000","last_became_peered":"2024-05-14T09:31:08.120227+0000","last_unstale":"2024-05-14T09:32:30.522916+0000","last_undegraded":"2024-05-14T09:32:30.522916+0000","last_fullsized":"2024-05-14T09:32:30.522916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:06.630390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316348+0000","last_change":"2024-05-14T09:30:54.017382+0000","last_active":"2024-05-14T09:32:30.316348+0000","last_peered":"2024-05-14T09:32:30.316348+0000","last_clean":"2024-05-14T09:32:30.316348+0000","last_became_active":"2024-05-14T09:30:54.016887+0000","last_became_peered":"2024-05-14T09:30:54.016887+0000","last_unstale":"2024-05-14T09:32:30.316348+0000","last_undegraded":"2024-05-14T09:32:30.316348+0000","last_fullsized":"2024-05-14T09:32:30.316348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:28.823659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.085392+0000","last_change":"2024-05-14T09:30:56.800626+0000","last_active":"2024-05-14T09:33:29.085392+0000","last_peered":"2024-05-14T09:33:29.085392+0000","last_clean":"2024-05-14T09:33:29.085392+0000","last_became_active":"2024-05-14T09:30:56.800258+0000","last_became_peered":"2024-05-14T09:30:56.800258+0000","last_unstale":"2024-05-14T09:33:29.085392+0000","last_undegraded":"2024-05-14T09:33:29.085392+0000","last_fullsized":"2024-05-14T09:33:29.085392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:47:26.369633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.318135+0000","last_change":"2024-05-14T09:31:08.115456+0000","last_active":"2024-05-14T09:32:30.318135+0000","last_peered":"2024-05-14T09:32:30.318135+0000","last_clean":"2024-05-14T09:32:30.318135+0000","last_became_active":"2024-05-14T09:31:08.114705+0000","last_became_peered":"2024-05-14T09:31:08.114705+0000","last_unstale":"2024-05-14T09:32:30.318135+0000","last_undegraded":"2024-05-14T09:32:30.318135+0000","last_fullsized":"2024-05-14T09:32:30.318135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:50.096747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'39","reported_seq":461,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:34:52.509793+0000","last_change":"2024-05-14T09:31:30.186202+0000","last_active":"2024-05-14T09:34:52.509793+0000","last_peered":"2024-05-14T09:34:52.509793+0000","last_clean":"2024-05-14T09:34:52.509793+0000","last_became_active":"2024-05-14T09:31:30.174248+0000","last_became_peered":"2024-05-14T09:31:30.174248+0000","last_unstale":"2024-05-14T09:34:52.509793+0000","last_undegraded":"2024-05-14T09:34:52.509793+0000","last_fullsized":"2024-05-14T09:34:52.509793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:45:03.131934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64617,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":361,"num_read_kb":244,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077083+0000","last_change":"2024-05-14T09:30:56.791099+0000","last_active":"2024-05-14T09:33:29.077083+0000","last_peered":"2024-05-14T09:33:29.077083+0000","last_clean":"2024-05-14T09:33:29.077083+0000","last_became_active":"2024-05-14T09:30:56.790676+0000","last_became_peered":"2024-05-14T09:30:56.790676+0000","last_unstale":"2024-05-14T09:33:29.077083+0000","last_undegraded":"2024-05-14T09:33:29.077083+0000","last_fullsized":"2024-05-14T09:33:29.077083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:09.851357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316986+0000","last_change":"2024-05-14T09:31:08.120289+0000","last_active":"2024-05-14T09:32:30.316986+0000","last_peered":"2024-05-14T09:32:30.316986+0000","last_clean":"2024-05-14T09:32:30.316986+0000","last_became_active":"2024-05-14T09:31:08.118574+0000","last_became_peered":"2024-05-14T09:31:08.118574+0000","last_unstale":"2024-05-14T09:32:30.316986+0000","last_undegraded":"2024-05-14T09:32:30.316986+0000","last_fullsized":"2024-05-14T09:32:30.316986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:01.723442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319990+0000","last_change":"2024-05-14T09:30:53.766578+0000","last_active":"2024-05-14T09:32:30.319990+0000","last_peered":"2024-05-14T09:32:30.319990+0000","last_clean":"2024-05-14T09:32:30.319990+0000","last_became_active":"2024-05-14T09:30:53.765928+0000","last_became_peered":"2024-05-14T09:30:53.765928+0000","last_unstale":"2024-05-14T09:32:30.319990+0000","last_undegraded":"2024-05-14T09:32:30.319990+0000","last_fullsized":"2024-05-14T09:32:30.319990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:19:08.690013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.083536+0000","last_change":"2024-05-14T09:30:56.785507+0000","last_active":"2024-05-14T09:33:29.083536+0000","last_peered":"2024-05-14T09:33:29.083536+0000","last_clean":"2024-05-14T09:33:29.083536+0000","last_became_active":"2024-05-14T09:30:56.784800+0000","last_became_peered":"2024-05-14T09:30:56.784800+0000","last_unstale":"2024-05-14T09:33:29.083536+0000","last_undegraded":"2024-05-14T09:33:29.083536+0000","last_fullsized":"2024-05-14T09:33:29.083536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:36:05.440222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.321298+0000","last_change":"2024-05-14T09:31:08.119051+0000","last_active":"2024-05-14T09:32:30.321298+0000","last_peered":"2024-05-14T09:32:30.321298+0000","last_clean":"2024-05-14T09:32:30.321298+0000","last_became_active":"2024-05-14T09:31:08.118569+0000","last_became_peered":"2024-05-14T09:31:08.118569+0000","last_unstale":"2024-05-14T09:32:30.321298+0000","last_undegraded":"2024-05-14T09:32:30.321298+0000","last_fullsized":"2024-05-14T09:32:30.321298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:12:39.643740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316079+0000","last_change":"2024-05-14T09:30:53.772832+0000","last_active":"2024-05-14T09:32:30.316079+0000","last_peered":"2024-05-14T09:32:30.316079+0000","last_clean":"2024-05-14T09:32:30.316079+0000","last_became_active":"2024-05-14T09:30:53.772228+0000","last_became_peered":"2024-05-14T09:30:53.772228+0000","last_unstale":"2024-05-14T09:32:30.316079+0000","last_undegraded":"2024-05-14T09:32:30.316079+0000","last_fullsized":"2024-05-14T09:32:30.316079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:14:43.350330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084806+0000","last_change":"2024-05-14T09:30:56.782831+0000","last_active":"2024-05-14T09:33:29.084806+0000","last_peered":"2024-05-14T09:33:29.084806+0000","last_clean":"2024-05-14T09:33:29.084806+0000","last_became_active":"2024-05-14T09:30:56.782402+0000","last_became_peered":"2024-05-14T09:30:56.782402+0000","last_unstale":"2024-05-14T09:33:29.084806+0000","last_undegraded":"2024-05-14T09:33:29.084806+0000","last_fullsized":"2024-05-14T09:33:29.084806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:54.603524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524041+0000","last_change":"2024-05-14T09:31:30.296957+0000","last_active":"2024-05-14T09:32:30.524041+0000","last_peered":"2024-05-14T09:32:30.524041+0000","last_clean":"2024-05-14T09:32:30.524041+0000","last_became_active":"2024-05-14T09:31:30.296554+0000","last_became_peered":"2024-05-14T09:31:30.296554+0000","last_unstale":"2024-05-14T09:32:30.524041+0000","last_undegraded":"2024-05-14T09:32:30.524041+0000","last_fullsized":"2024-05-14T09:32:30.524041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:28.963042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":224,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:34:50.507285+0000","last_change":"2024-05-14T09:30:53.772731+0000","last_active":"2024-05-14T09:34:50.507285+0000","last_peered":"2024-05-14T09:34:50.507285+0000","last_clean":"2024-05-14T09:34:50.507285+0000","last_became_active":"2024-05-14T09:30:53.771986+0000","last_became_peered":"2024-05-14T09:30:53.771986+0000","last_unstale":"2024-05-14T09:34:50.507285+0000","last_undegraded":"2024-05-14T09:34:50.507285+0000","last_fullsized":"2024-05-14T09:34:50.507285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:50:00.810937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077545+0000","last_change":"2024-05-14T09:30:56.790503+0000","last_active":"2024-05-14T09:33:29.077545+0000","last_peered":"2024-05-14T09:33:29.077545+0000","last_clean":"2024-05-14T09:33:29.077545+0000","last_became_active":"2024-05-14T09:30:56.790103+0000","last_became_peered":"2024-05-14T09:30:56.790103+0000","last_unstale":"2024-05-14T09:33:29.077545+0000","last_undegraded":"2024-05-14T09:33:29.077545+0000","last_fullsized":"2024-05-14T09:33:29.077545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:00.950794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315462+0000","last_change":"2024-05-14T09:31:08.116475+0000","last_active":"2024-05-14T09:32:30.315462+0000","last_peered":"2024-05-14T09:32:30.315462+0000","last_clean":"2024-05-14T09:32:30.315462+0000","last_became_active":"2024-05-14T09:31:08.116061+0000","last_became_peered":"2024-05-14T09:31:08.116061+0000","last_unstale":"2024-05-14T09:32:30.315462+0000","last_undegraded":"2024-05-14T09:32:30.315462+0000","last_fullsized":"2024-05-14T09:32:30.315462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:07:02.281853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523311+0000","last_change":"2024-05-14T09:30:54.016570+0000","last_active":"2024-05-14T09:32:30.523311+0000","last_peered":"2024-05-14T09:32:30.523311+0000","last_clean":"2024-05-14T09:32:30.523311+0000","last_became_active":"2024-05-14T09:30:54.015987+0000","last_became_peered":"2024-05-14T09:30:54.015987+0000","last_unstale":"2024-05-14T09:32:30.523311+0000","last_undegraded":"2024-05-14T09:32:30.523311+0000","last_fullsized":"2024-05-14T09:32:30.523311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:18:28.694488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081072+0000","last_change":"2024-05-14T09:30:56.800727+0000","last_active":"2024-05-14T09:33:29.081072+0000","last_peered":"2024-05-14T09:33:29.081072+0000","last_clean":"2024-05-14T09:33:29.081072+0000","last_became_active":"2024-05-14T09:30:56.800354+0000","last_became_peered":"2024-05-14T09:30:56.800354+0000","last_unstale":"2024-05-14T09:33:29.081072+0000","last_undegraded":"2024-05-14T09:33:29.081072+0000","last_fullsized":"2024-05-14T09:33:29.081072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:13:07.227577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315777+0000","last_change":"2024-05-14T09:31:08.115685+0000","last_active":"2024-05-14T09:32:30.315777+0000","last_peered":"2024-05-14T09:32:30.315777+0000","last_clean":"2024-05-14T09:32:30.315777+0000","last_became_active":"2024-05-14T09:31:08.114948+0000","last_became_peered":"2024-05-14T09:31:08.114948+0000","last_unstale":"2024-05-14T09:32:30.315777+0000","last_undegraded":"2024-05-14T09:32:30.315777+0000","last_fullsized":"2024-05-14T09:32:30.315777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:33:07.286823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524227+0000","last_change":"2024-05-14T09:30:53.769949+0000","last_active":"2024-05-14T09:32:30.524227+0000","last_peered":"2024-05-14T09:32:30.524227+0000","last_clean":"2024-05-14T09:32:30.524227+0000","last_became_active":"2024-05-14T09:30:53.769327+0000","last_became_peered":"2024-05-14T09:30:53.769327+0000","last_unstale":"2024-05-14T09:32:30.524227+0000","last_undegraded":"2024-05-14T09:32:30.524227+0000","last_fullsized":"2024-05-14T09:32:30.524227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:35:11.850991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077817+0000","last_change":"2024-05-14T09:32:31.015994+0000","last_active":"2024-05-14T09:33:29.077817+0000","last_peered":"2024-05-14T09:33:29.077817+0000","last_clean":"2024-05-14T09:33:29.077817+0000","last_became_active":"2024-05-14T09:32:31.015515+0000","last_became_peered":"2024-05-14T09:32:31.015515+0000","last_unstale":"2024-05-14T09:33:29.077817+0000","last_undegraded":"2024-05-14T09:33:29.077817+0000","last_fullsized":"2024-05-14T09:33:29.077817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:21:49.811521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7],"acting":[11,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524005+0000","last_change":"2024-05-14T09:31:08.387316+0000","last_active":"2024-05-14T09:32:30.524005+0000","last_peered":"2024-05-14T09:32:30.524005+0000","last_clean":"2024-05-14T09:32:30.524005+0000","last_became_active":"2024-05-14T09:31:08.386739+0000","last_became_peered":"2024-05-14T09:31:08.386739+0000","last_unstale":"2024-05-14T09:32:30.524005+0000","last_undegraded":"2024-05-14T09:32:30.524005+0000","last_fullsized":"2024-05-14T09:32:30.524005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:17.870255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.318704+0000","last_change":"2024-05-14T09:30:54.017756+0000","last_active":"2024-05-14T09:32:30.318704+0000","last_peered":"2024-05-14T09:32:30.318704+0000","last_clean":"2024-05-14T09:32:30.318704+0000","last_became_active":"2024-05-14T09:30:54.017277+0000","last_became_peered":"2024-05-14T09:30:54.017277+0000","last_unstale":"2024-05-14T09:32:30.318704+0000","last_undegraded":"2024-05-14T09:32:30.318704+0000","last_fullsized":"2024-05-14T09:32:30.318704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:27:36.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.072775+0000","last_change":"2024-05-14T09:30:56.788408+0000","last_active":"2024-05-14T09:33:29.072775+0000","last_peered":"2024-05-14T09:33:29.072775+0000","last_clean":"2024-05-14T09:33:29.072775+0000","last_became_active":"2024-05-14T09:30:56.787955+0000","last_became_peered":"2024-05-14T09:30:56.787955+0000","last_unstale":"2024-05-14T09:33:29.072775+0000","last_undegraded":"2024-05-14T09:33:29.072775+0000","last_fullsized":"2024-05-14T09:33:29.072775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:09.777069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315137+0000","last_change":"2024-05-14T09:31:08.119078+0000","last_active":"2024-05-14T09:32:30.315137+0000","last_peered":"2024-05-14T09:32:30.315137+0000","last_clean":"2024-05-14T09:32:30.315137+0000","last_became_active":"2024-05-14T09:31:08.117939+0000","last_became_peered":"2024-05-14T09:31:08.117939+0000","last_unstale":"2024-05-14T09:32:30.315137+0000","last_undegraded":"2024-05-14T09:32:30.315137+0000","last_fullsized":"2024-05-14T09:32:30.315137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:48:37.363503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014384+0000","last_change":"2024-05-14T09:30:53.767843+0000","last_active":"2024-05-14T09:32:31.014384+0000","last_peered":"2024-05-14T09:32:31.014384+0000","last_clean":"2024-05-14T09:32:31.014384+0000","last_became_active":"2024-05-14T09:30:53.767419+0000","last_became_peered":"2024-05-14T09:30:53.767419+0000","last_unstale":"2024-05-14T09:32:31.014384+0000","last_undegraded":"2024-05-14T09:32:31.014384+0000","last_fullsized":"2024-05-14T09:32:31.014384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:49:06.441600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077073+0000","last_change":"2024-05-14T09:30:56.796001+0000","last_active":"2024-05-14T09:33:29.077073+0000","last_peered":"2024-05-14T09:33:29.077073+0000","last_clean":"2024-05-14T09:33:29.077073+0000","last_became_active":"2024-05-14T09:30:56.794517+0000","last_became_peered":"2024-05-14T09:30:56.794517+0000","last_unstale":"2024-05-14T09:33:29.077073+0000","last_undegraded":"2024-05-14T09:33:29.077073+0000","last_fullsized":"2024-05-14T09:33:29.077073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:20:37.514679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315309+0000","last_change":"2024-05-14T09:31:08.387570+0000","last_active":"2024-05-14T09:32:30.315309+0000","last_peered":"2024-05-14T09:32:30.315309+0000","last_clean":"2024-05-14T09:32:30.315309+0000","last_became_active":"2024-05-14T09:31:08.386745+0000","last_became_peered":"2024-05-14T09:31:08.386745+0000","last_unstale":"2024-05-14T09:32:30.315309+0000","last_undegraded":"2024-05-14T09:32:30.315309+0000","last_fullsized":"2024-05-14T09:32:30.315309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:44:01.024784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.060037+0000","last_change":"2024-05-14T09:30:56.795532+0000","last_active":"2024-05-14T09:33:29.060037+0000","last_peered":"2024-05-14T09:33:29.060037+0000","last_clean":"2024-05-14T09:33:29.060037+0000","last_became_active":"2024-05-14T09:30:56.795171+0000","last_became_peered":"2024-05-14T09:30:56.795171+0000","last_unstale":"2024-05-14T09:33:29.060037+0000","last_undegraded":"2024-05-14T09:33:29.060037+0000","last_fullsized":"2024-05-14T09:33:29.060037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:09:40.252489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":224,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:34:51.508100+0000","last_change":"2024-05-14T09:30:54.015629+0000","last_active":"2024-05-14T09:34:51.508100+0000","last_peered":"2024-05-14T09:34:51.508100+0000","last_clean":"2024-05-14T09:34:51.508100+0000","last_became_active":"2024-05-14T09:30:54.015288+0000","last_became_peered":"2024-05-14T09:30:54.015288+0000","last_unstale":"2024-05-14T09:34:51.508100+0000","last_undegraded":"2024-05-14T09:34:51.508100+0000","last_fullsized":"2024-05-14T09:34:51.508100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:50.316125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014777+0000","last_change":"2024-05-14T09:31:30.179056+0000","last_active":"2024-05-14T09:32:31.014777+0000","last_peered":"2024-05-14T09:32:31.014777+0000","last_clean":"2024-05-14T09:32:31.014777+0000","last_became_active":"2024-05-14T09:31:30.178398+0000","last_became_peered":"2024-05-14T09:31:30.178398+0000","last_unstale":"2024-05-14T09:32:31.014777+0000","last_undegraded":"2024-05-14T09:32:31.014777+0000","last_fullsized":"2024-05-14T09:32:31.014777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:15:55.369801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077759+0000","last_change":"2024-05-14T09:30:56.793642+0000","last_active":"2024-05-14T09:33:29.077759+0000","last_peered":"2024-05-14T09:33:29.077759+0000","last_clean":"2024-05-14T09:33:29.077759+0000","last_became_active":"2024-05-14T09:30:56.793229+0000","last_became_peered":"2024-05-14T09:30:56.793229+0000","last_unstale":"2024-05-14T09:33:29.077759+0000","last_undegraded":"2024-05-14T09:33:29.077759+0000","last_fullsized":"2024-05-14T09:33:29.077759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:17.940674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522946+0000","last_change":"2024-05-14T09:30:53.769582+0000","last_active":"2024-05-14T09:32:30.522946+0000","last_peered":"2024-05-14T09:32:30.522946+0000","last_clean":"2024-05-14T09:32:30.522946+0000","last_became_active":"2024-05-14T09:30:53.769113+0000","last_became_peered":"2024-05-14T09:30:53.769113+0000","last_unstale":"2024-05-14T09:32:30.522946+0000","last_undegraded":"2024-05-14T09:32:30.522946+0000","last_fullsized":"2024-05-14T09:32:30.522946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:48.755113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":119,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014145+0000","last_change":"2024-05-14T09:29:30.767389+0000","last_active":"2024-05-14T09:32:31.014145+0000","last_peered":"2024-05-14T09:32:31.014145+0000","last_clean":"2024-05-14T09:32:31.014145+0000","last_became_active":"2024-05-14T09:29:30.761827+0000","last_became_peered":"2024-05-14T09:29:30.761827+0000","last_unstale":"2024-05-14T09:32:31.014145+0000","last_undegraded":"2024-05-14T09:32:31.014145+0000","last_fullsized":"2024-05-14T09:32:31.014145+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:26:03.741821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:26:03.741821+0000","last_clean_scrub_stamp":"2024-05-14T09:26:03.741821+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:52:11.421658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316444+0000","last_change":"2024-05-14T09:31:08.387819+0000","last_active":"2024-05-14T09:32:30.316444+0000","last_peered":"2024-05-14T09:32:30.316444+0000","last_clean":"2024-05-14T09:32:30.316444+0000","last_became_active":"2024-05-14T09:31:08.387275+0000","last_became_peered":"2024-05-14T09:31:08.387275+0000","last_unstale":"2024-05-14T09:32:30.316444+0000","last_undegraded":"2024-05-14T09:32:30.316444+0000","last_fullsized":"2024-05-14T09:32:30.316444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:46.627389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.078055+0000","last_change":"2024-05-14T09:30:56.788159+0000","last_active":"2024-05-14T09:33:29.078055+0000","last_peered":"2024-05-14T09:33:29.078055+0000","last_clean":"2024-05-14T09:33:29.078055+0000","last_became_active":"2024-05-14T09:30:56.787744+0000","last_became_peered":"2024-05-14T09:30:56.787744+0000","last_unstale":"2024-05-14T09:33:29.078055+0000","last_undegraded":"2024-05-14T09:33:29.078055+0000","last_fullsized":"2024-05-14T09:33:29.078055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:45:25.243471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:52.843555+0000","last_change":"2024-05-14T09:30:53.774345+0000","last_active":"2024-05-14T09:32:52.843555+0000","last_peered":"2024-05-14T09:32:52.843555+0000","last_clean":"2024-05-14T09:32:52.843555+0000","last_became_active":"2024-05-14T09:30:53.773853+0000","last_became_peered":"2024-05-14T09:30:53.773853+0000","last_unstale":"2024-05-14T09:32:52.843555+0000","last_undegraded":"2024-05-14T09:32:52.843555+0000","last_fullsized":"2024-05-14T09:32:52.843555+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:35:31.938442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320387+0000","last_change":"2024-05-14T09:31:08.122876+0000","last_active":"2024-05-14T09:32:30.320387+0000","last_peered":"2024-05-14T09:32:30.320387+0000","last_clean":"2024-05-14T09:32:30.320387+0000","last_became_active":"2024-05-14T09:31:08.122182+0000","last_became_peered":"2024-05-14T09:31:08.122182+0000","last_unstale":"2024-05-14T09:32:30.320387+0000","last_undegraded":"2024-05-14T09:32:30.320387+0000","last_fullsized":"2024-05-14T09:32:30.320387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:47:30.735057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524333+0000","last_change":"2024-05-14T09:30:54.018168+0000","last_active":"2024-05-14T09:32:30.524333+0000","last_peered":"2024-05-14T09:32:30.524333+0000","last_clean":"2024-05-14T09:32:30.524333+0000","last_became_active":"2024-05-14T09:30:54.017699+0000","last_became_peered":"2024-05-14T09:30:54.017699+0000","last_unstale":"2024-05-14T09:32:30.524333+0000","last_undegraded":"2024-05-14T09:32:30.524333+0000","last_fullsized":"2024-05-14T09:32:30.524333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:31:01.162000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.070935+0000","last_change":"2024-05-14T09:30:56.787156+0000","last_active":"2024-05-14T09:33:29.070935+0000","last_peered":"2024-05-14T09:33:29.070935+0000","last_clean":"2024-05-14T09:33:29.070935+0000","last_became_active":"2024-05-14T09:30:56.786832+0000","last_became_peered":"2024-05-14T09:30:56.786832+0000","last_unstale":"2024-05-14T09:33:29.070935+0000","last_undegraded":"2024-05-14T09:33:29.070935+0000","last_fullsized":"2024-05-14T09:33:29.070935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:00:41.719641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.321913+0000","last_change":"2024-05-14T09:31:08.387907+0000","last_active":"2024-05-14T09:32:30.321913+0000","last_peered":"2024-05-14T09:32:30.321913+0000","last_clean":"2024-05-14T09:32:30.321913+0000","last_became_active":"2024-05-14T09:31:08.386765+0000","last_became_peered":"2024-05-14T09:31:08.386765+0000","last_unstale":"2024-05-14T09:32:30.321913+0000","last_undegraded":"2024-05-14T09:32:30.321913+0000","last_fullsized":"2024-05-14T09:32:30.321913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:24:56.890353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'13","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.814407+0000","last_change":"2024-05-14T09:30:53.767508+0000","last_active":"2024-05-14T09:33:29.814407+0000","last_peered":"2024-05-14T09:33:29.814407+0000","last_clean":"2024-05-14T09:33:29.814407+0000","last_became_active":"2024-05-14T09:30:53.767169+0000","last_became_peered":"2024-05-14T09:30:53.767169+0000","last_unstale":"2024-05-14T09:33:29.814407+0000","last_undegraded":"2024-05-14T09:33:29.814407+0000","last_fullsized":"2024-05-14T09:33:29.814407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:04:23.621686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076780+0000","last_change":"2024-05-14T09:30:56.790398+0000","last_active":"2024-05-14T09:33:29.076780+0000","last_peered":"2024-05-14T09:33:29.076780+0000","last_clean":"2024-05-14T09:33:29.076780+0000","last_became_active":"2024-05-14T09:30:56.790055+0000","last_became_peered":"2024-05-14T09:30:56.790055+0000","last_unstale":"2024-05-14T09:33:29.076780+0000","last_undegraded":"2024-05-14T09:33:29.076780+0000","last_fullsized":"2024-05-14T09:33:29.076780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:12:37.688647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319239+0000","last_change":"2024-05-14T09:31:08.111533+0000","last_active":"2024-05-14T09:32:30.319239+0000","last_peered":"2024-05-14T09:32:30.319239+0000","last_clean":"2024-05-14T09:32:30.319239+0000","last_became_active":"2024-05-14T09:31:08.110448+0000","last_became_peered":"2024-05-14T09:31:08.110448+0000","last_unstale":"2024-05-14T09:32:30.319239+0000","last_undegraded":"2024-05-14T09:32:30.319239+0000","last_fullsized":"2024-05-14T09:32:30.319239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:43:45.822382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523626+0000","last_change":"2024-05-14T09:30:54.069180+0000","last_active":"2024-05-14T09:32:30.523626+0000","last_peered":"2024-05-14T09:32:30.523626+0000","last_clean":"2024-05-14T09:32:30.523626+0000","last_became_active":"2024-05-14T09:30:54.068846+0000","last_became_peered":"2024-05-14T09:30:54.068846+0000","last_unstale":"2024-05-14T09:32:30.523626+0000","last_undegraded":"2024-05-14T09:32:30.523626+0000","last_fullsized":"2024-05-14T09:32:30.523626+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:22.541720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080401+0000","last_change":"2024-05-14T09:30:56.788159+0000","last_active":"2024-05-14T09:33:29.080401+0000","last_peered":"2024-05-14T09:33:29.080401+0000","last_clean":"2024-05-14T09:33:29.080401+0000","last_became_active":"2024-05-14T09:30:56.787786+0000","last_became_peered":"2024-05-14T09:30:56.787786+0000","last_unstale":"2024-05-14T09:33:29.080401+0000","last_undegraded":"2024-05-14T09:33:29.080401+0000","last_fullsized":"2024-05-14T09:33:29.080401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:24:51.785543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523393+0000","last_change":"2024-05-14T09:31:08.388201+0000","last_active":"2024-05-14T09:32:30.523393+0000","last_peered":"2024-05-14T09:32:30.523393+0000","last_clean":"2024-05-14T09:32:30.523393+0000","last_became_active":"2024-05-14T09:31:08.387572+0000","last_became_peered":"2024-05-14T09:31:08.387572+0000","last_unstale":"2024-05-14T09:32:30.523393+0000","last_undegraded":"2024-05-14T09:32:30.523393+0000","last_fullsized":"2024-05-14T09:32:30.523393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:25:13.171350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316020+0000","last_change":"2024-05-14T09:30:54.016524+0000","last_active":"2024-05-14T09:32:30.316020+0000","last_peered":"2024-05-14T09:32:30.316020+0000","last_clean":"2024-05-14T09:32:30.316020+0000","last_became_active":"2024-05-14T09:30:54.016154+0000","last_became_peered":"2024-05-14T09:30:54.016154+0000","last_unstale":"2024-05-14T09:32:30.316020+0000","last_undegraded":"2024-05-14T09:32:30.316020+0000","last_fullsized":"2024-05-14T09:32:30.316020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:43:16.671053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.035477+0000","last_change":"2024-05-14T09:30:56.796748+0000","last_active":"2024-05-14T09:33:29.035477+0000","last_peered":"2024-05-14T09:33:29.035477+0000","last_clean":"2024-05-14T09:33:29.035477+0000","last_became_active":"2024-05-14T09:30:56.796354+0000","last_became_peered":"2024-05-14T09:30:56.796354+0000","last_unstale":"2024-05-14T09:33:29.035477+0000","last_undegraded":"2024-05-14T09:33:29.035477+0000","last_fullsized":"2024-05-14T09:33:29.035477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:30:15.820677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319112+0000","last_change":"2024-05-14T09:31:08.117266+0000","last_active":"2024-05-14T09:32:30.319112+0000","last_peered":"2024-05-14T09:32:30.319112+0000","last_clean":"2024-05-14T09:32:30.319112+0000","last_became_active":"2024-05-14T09:31:08.115393+0000","last_became_peered":"2024-05-14T09:31:08.115393+0000","last_unstale":"2024-05-14T09:32:30.319112+0000","last_undegraded":"2024-05-14T09:32:30.319112+0000","last_fullsized":"2024-05-14T09:32:30.319112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:29:26.348818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.322687+0000","last_change":"2024-05-14T09:30:53.772417+0000","last_active":"2024-05-14T09:32:30.322687+0000","last_peered":"2024-05-14T09:32:30.322687+0000","last_clean":"2024-05-14T09:32:30.322687+0000","last_became_active":"2024-05-14T09:30:53.772028+0000","last_became_peered":"2024-05-14T09:30:53.772028+0000","last_unstale":"2024-05-14T09:32:30.322687+0000","last_undegraded":"2024-05-14T09:32:30.322687+0000","last_fullsized":"2024-05-14T09:32:30.322687+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:16:16.603957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077309+0000","last_change":"2024-05-14T09:30:56.804750+0000","last_active":"2024-05-14T09:33:29.077309+0000","last_peered":"2024-05-14T09:33:29.077309+0000","last_clean":"2024-05-14T09:33:29.077309+0000","last_became_active":"2024-05-14T09:30:56.803990+0000","last_became_peered":"2024-05-14T09:30:56.803990+0000","last_unstale":"2024-05-14T09:33:29.077309+0000","last_undegraded":"2024-05-14T09:33:29.077309+0000","last_fullsized":"2024-05-14T09:33:29.077309+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:42:00.661033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523484+0000","last_change":"2024-05-14T09:31:08.116061+0000","last_active":"2024-05-14T09:32:30.523484+0000","last_peered":"2024-05-14T09:32:30.523484+0000","last_clean":"2024-05-14T09:32:30.523484+0000","last_became_active":"2024-05-14T09:31:08.115578+0000","last_became_peered":"2024-05-14T09:31:08.115578+0000","last_unstale":"2024-05-14T09:32:30.523484+0000","last_undegraded":"2024-05-14T09:32:30.523484+0000","last_fullsized":"2024-05-14T09:32:30.523484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:18.931035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:06.782240+0000","last_change":"2024-05-14T09:30:53.768988+0000","last_active":"2024-05-14T09:33:06.782240+0000","last_peered":"2024-05-14T09:33:06.782240+0000","last_clean":"2024-05-14T09:33:06.782240+0000","last_became_active":"2024-05-14T09:30:53.768529+0000","last_became_peered":"2024-05-14T09:30:53.768529+0000","last_unstale":"2024-05-14T09:33:06.782240+0000","last_undegraded":"2024-05-14T09:33:06.782240+0000","last_fullsized":"2024-05-14T09:33:06.782240+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:17:02.381447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.079500+0000","last_change":"2024-05-14T09:30:56.797110+0000","last_active":"2024-05-14T09:33:29.079500+0000","last_peered":"2024-05-14T09:33:29.079500+0000","last_clean":"2024-05-14T09:33:29.079500+0000","last_became_active":"2024-05-14T09:30:56.796015+0000","last_became_peered":"2024-05-14T09:30:56.796015+0000","last_unstale":"2024-05-14T09:33:29.079500+0000","last_undegraded":"2024-05-14T09:33:29.079500+0000","last_fullsized":"2024-05-14T09:33:29.079500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:37:44.674607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523289+0000","last_change":"2024-05-14T09:31:08.387240+0000","last_active":"2024-05-14T09:32:30.523289+0000","last_peered":"2024-05-14T09:32:30.523289+0000","last_clean":"2024-05-14T09:32:30.523289+0000","last_became_active":"2024-05-14T09:31:08.386653+0000","last_became_peered":"2024-05-14T09:31:08.386653+0000","last_unstale":"2024-05-14T09:32:30.523289+0000","last_undegraded":"2024-05-14T09:32:30.523289+0000","last_fullsized":"2024-05-14T09:32:30.523289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:18:43.408030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523577+0000","last_change":"2024-05-14T09:30:53.772723+0000","last_active":"2024-05-14T09:32:30.523577+0000","last_peered":"2024-05-14T09:32:30.523577+0000","last_clean":"2024-05-14T09:32:30.523577+0000","last_became_active":"2024-05-14T09:30:53.772056+0000","last_became_peered":"2024-05-14T09:30:53.772056+0000","last_unstale":"2024-05-14T09:32:30.523577+0000","last_undegraded":"2024-05-14T09:32:30.523577+0000","last_fullsized":"2024-05-14T09:32:30.523577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:37.941282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087926+0000","last_change":"2024-05-14T09:30:56.799223+0000","last_active":"2024-05-14T09:33:29.087926+0000","last_peered":"2024-05-14T09:33:29.087926+0000","last_clean":"2024-05-14T09:33:29.087926+0000","last_became_active":"2024-05-14T09:30:56.796092+0000","last_became_peered":"2024-05-14T09:30:56.796092+0000","last_unstale":"2024-05-14T09:33:29.087926+0000","last_undegraded":"2024-05-14T09:33:29.087926+0000","last_fullsized":"2024-05-14T09:33:29.087926+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:43:49.746797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316061+0000","last_change":"2024-05-14T09:31:08.115053+0000","last_active":"2024-05-14T09:32:30.316061+0000","last_peered":"2024-05-14T09:32:30.316061+0000","last_clean":"2024-05-14T09:32:30.316061+0000","last_became_active":"2024-05-14T09:31:08.114578+0000","last_became_peered":"2024-05-14T09:31:08.114578+0000","last_unstale":"2024-05-14T09:32:30.316061+0000","last_undegraded":"2024-05-14T09:32:30.316061+0000","last_fullsized":"2024-05-14T09:32:30.316061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:47.788962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:32.003842+0000","last_change":"2024-05-14T09:32:30.326580+0000","last_active":"2024-05-14T09:32:32.003842+0000","last_peered":"2024-05-14T09:32:32.003842+0000","last_clean":"2024-05-14T09:32:32.003842+0000","last_became_active":"2024-05-14T09:32:30.322052+0000","last_became_peered":"2024-05-14T09:32:30.322052+0000","last_unstale":"2024-05-14T09:32:32.003842+0000","last_undegraded":"2024-05-14T09:32:32.003842+0000","last_fullsized":"2024-05-14T09:32:32.003842+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:04.991674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087235+0000","last_change":"2024-05-14T09:30:56.798580+0000","last_active":"2024-05-14T09:33:29.087235+0000","last_peered":"2024-05-14T09:33:29.087235+0000","last_clean":"2024-05-14T09:33:29.087235+0000","last_became_active":"2024-05-14T09:30:56.795617+0000","last_became_peered":"2024-05-14T09:30:56.795617+0000","last_unstale":"2024-05-14T09:33:29.087235+0000","last_undegraded":"2024-05-14T09:33:29.087235+0000","last_fullsized":"2024-05-14T09:33:29.087235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:30:35.913497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319424+0000","last_change":"2024-05-14T09:31:08.111375+0000","last_active":"2024-05-14T09:32:30.319424+0000","last_peered":"2024-05-14T09:32:30.319424+0000","last_clean":"2024-05-14T09:32:30.319424+0000","last_became_active":"2024-05-14T09:31:08.110931+0000","last_became_peered":"2024-05-14T09:31:08.110931+0000","last_unstale":"2024-05-14T09:32:30.319424+0000","last_undegraded":"2024-05-14T09:32:30.319424+0000","last_fullsized":"2024-05-14T09:32:30.319424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:08:30.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.176828+0000","last_change":"2024-05-14T09:32:30.326979+0000","last_active":"2024-05-14T09:32:31.176828+0000","last_peered":"2024-05-14T09:32:31.176828+0000","last_clean":"2024-05-14T09:32:31.176828+0000","last_became_active":"2024-05-14T09:32:30.323545+0000","last_became_peered":"2024-05-14T09:32:30.323545+0000","last_unstale":"2024-05-14T09:32:31.176828+0000","last_undegraded":"2024-05-14T09:32:31.176828+0000","last_fullsized":"2024-05-14T09:32:31.176828+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:00:21.493910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":70,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.064380+0000","last_change":"2024-05-14T09:30:56.789527+0000","last_active":"2024-05-14T09:33:29.064380+0000","last_peered":"2024-05-14T09:33:29.064380+0000","last_clean":"2024-05-14T09:33:29.064380+0000","last_became_active":"2024-05-14T09:30:56.789075+0000","last_became_peered":"2024-05-14T09:30:56.789075+0000","last_unstale":"2024-05-14T09:33:29.064380+0000","last_undegraded":"2024-05-14T09:33:29.064380+0000","last_fullsized":"2024-05-14T09:33:29.064380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:07.140087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315692+0000","last_change":"2024-05-14T09:31:08.119624+0000","last_active":"2024-05-14T09:32:30.315692+0000","last_peered":"2024-05-14T09:32:30.315692+0000","last_clean":"2024-05-14T09:32:30.315692+0000","last_became_active":"2024-05-14T09:31:08.118102+0000","last_became_peered":"2024-05-14T09:31:08.118102+0000","last_unstale":"2024-05-14T09:32:30.315692+0000","last_undegraded":"2024-05-14T09:32:30.315692+0000","last_fullsized":"2024-05-14T09:32:30.315692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:48:17.806976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315472+0000","last_change":"2024-05-14T09:30:53.763903+0000","last_active":"2024-05-14T09:32:30.315472+0000","last_peered":"2024-05-14T09:32:30.315472+0000","last_clean":"2024-05-14T09:32:30.315472+0000","last_became_active":"2024-05-14T09:30:53.763517+0000","last_became_peered":"2024-05-14T09:30:53.763517+0000","last_unstale":"2024-05-14T09:32:30.315472+0000","last_undegraded":"2024-05-14T09:32:30.315472+0000","last_fullsized":"2024-05-14T09:32:30.315472+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:57.144392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081279+0000","last_change":"2024-05-14T09:30:56.791612+0000","last_active":"2024-05-14T09:33:29.081279+0000","last_peered":"2024-05-14T09:33:29.081279+0000","last_clean":"2024-05-14T09:33:29.081279+0000","last_became_active":"2024-05-14T09:30:56.791178+0000","last_became_peered":"2024-05-14T09:30:56.791178+0000","last_unstale":"2024-05-14T09:33:29.081279+0000","last_undegraded":"2024-05-14T09:33:29.081279+0000","last_fullsized":"2024-05-14T09:33:29.081279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:11:19.058139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076827+0000","last_change":"2024-05-14T09:30:56.804437+0000","last_active":"2024-05-14T09:33:29.076827+0000","last_peered":"2024-05-14T09:33:29.076827+0000","last_clean":"2024-05-14T09:33:29.076827+0000","last_became_active":"2024-05-14T09:30:56.804048+0000","last_became_peered":"2024-05-14T09:30:56.804048+0000","last_unstale":"2024-05-14T09:33:29.076827+0000","last_undegraded":"2024-05-14T09:33:29.076827+0000","last_fullsized":"2024-05-14T09:33:29.076827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:33.788903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.058110+0000","last_change":"2024-05-14T09:30:56.794309+0000","last_active":"2024-05-14T09:33:29.058110+0000","last_peered":"2024-05-14T09:33:29.058110+0000","last_clean":"2024-05-14T09:33:29.058110+0000","last_became_active":"2024-05-14T09:30:56.793317+0000","last_became_peered":"2024-05-14T09:30:56.793317+0000","last_unstale":"2024-05-14T09:33:29.058110+0000","last_undegraded":"2024-05-14T09:33:29.058110+0000","last_fullsized":"2024-05-14T09:33:29.058110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:40:26.878746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080059+0000","last_change":"2024-05-14T09:30:56.793618+0000","last_active":"2024-05-14T09:33:29.080059+0000","last_peered":"2024-05-14T09:33:29.080059+0000","last_clean":"2024-05-14T09:33:29.080059+0000","last_became_active":"2024-05-14T09:30:56.792819+0000","last_became_peered":"2024-05-14T09:30:56.792819+0000","last_unstale":"2024-05-14T09:33:29.080059+0000","last_undegraded":"2024-05-14T09:33:29.080059+0000","last_fullsized":"2024-05-14T09:33:29.080059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:29:38.191617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.072960+0000","last_change":"2024-05-14T09:30:56.794268+0000","last_active":"2024-05-14T09:33:29.072960+0000","last_peered":"2024-05-14T09:33:29.072960+0000","last_clean":"2024-05-14T09:33:29.072960+0000","last_became_active":"2024-05-14T09:30:56.793898+0000","last_became_peered":"2024-05-14T09:30:56.793898+0000","last_unstale":"2024-05-14T09:33:29.072960+0000","last_undegraded":"2024-05-14T09:33:29.072960+0000","last_fullsized":"2024-05-14T09:33:29.072960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:25:35.022236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.078917+0000","last_change":"2024-05-14T09:30:56.783366+0000","last_active":"2024-05-14T09:33:29.078917+0000","last_peered":"2024-05-14T09:33:29.078917+0000","last_clean":"2024-05-14T09:33:29.078917+0000","last_became_active":"2024-05-14T09:30:56.782980+0000","last_became_peered":"2024-05-14T09:30:56.782980+0000","last_unstale":"2024-05-14T09:33:29.078917+0000","last_undegraded":"2024-05-14T09:33:29.078917+0000","last_fullsized":"2024-05-14T09:33:29.078917+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:27.744902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076941+0000","last_change":"2024-05-14T09:32:30.524390+0000","last_active":"2024-05-14T09:33:29.076941+0000","last_peered":"2024-05-14T09:33:29.076941+0000","last_clean":"2024-05-14T09:33:29.076941+0000","last_became_active":"2024-05-14T09:32:30.523926+0000","last_became_peered":"2024-05-14T09:32:30.523926+0000","last_unstale":"2024-05-14T09:33:29.076941+0000","last_undegraded":"2024-05-14T09:33:29.076941+0000","last_fullsized":"2024-05-14T09:33:29.076941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:58:47.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.074545+0000","last_change":"2024-05-14T09:30:56.795872+0000","last_active":"2024-05-14T09:33:29.074545+0000","last_peered":"2024-05-14T09:33:29.074545+0000","last_clean":"2024-05-14T09:33:29.074545+0000","last_became_active":"2024-05-14T09:30:56.795492+0000","last_became_peered":"2024-05-14T09:30:56.795492+0000","last_unstale":"2024-05-14T09:33:29.074545+0000","last_undegraded":"2024-05-14T09:33:29.074545+0000","last_fullsized":"2024-05-14T09:33:29.074545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:26:29.226575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.062863+0000","last_change":"2024-05-14T09:32:30.524189+0000","last_active":"2024-05-14T09:33:29.062863+0000","last_peered":"2024-05-14T09:33:29.062863+0000","last_clean":"2024-05-14T09:33:29.062863+0000","last_became_active":"2024-05-14T09:32:30.523791+0000","last_became_peered":"2024-05-14T09:32:30.523791+0000","last_unstale":"2024-05-14T09:33:29.062863+0000","last_undegraded":"2024-05-14T09:33:29.062863+0000","last_fullsized":"2024-05-14T09:33:29.062863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:52.441130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080195+0000","last_change":"2024-05-14T09:30:56.792814+0000","last_active":"2024-05-14T09:33:29.080195+0000","last_peered":"2024-05-14T09:33:29.080195+0000","last_clean":"2024-05-14T09:33:29.080195+0000","last_became_active":"2024-05-14T09:30:56.792282+0000","last_became_peered":"2024-05-14T09:30:56.792282+0000","last_unstale":"2024-05-14T09:33:29.080195+0000","last_undegraded":"2024-05-14T09:33:29.080195+0000","last_fullsized":"2024-05-14T09:33:29.080195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:52:35.319479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076831+0000","last_change":"2024-05-14T09:30:56.791575+0000","last_active":"2024-05-14T09:33:29.076831+0000","last_peered":"2024-05-14T09:33:29.076831+0000","last_clean":"2024-05-14T09:33:29.076831+0000","last_became_active":"2024-05-14T09:30:56.791230+0000","last_became_peered":"2024-05-14T09:30:56.791230+0000","last_unstale":"2024-05-14T09:33:29.076831+0000","last_undegraded":"2024-05-14T09:33:29.076831+0000","last_fullsized":"2024-05-14T09:33:29.076831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:40.510062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.088030+0000","last_change":"2024-05-14T09:30:56.796996+0000","last_active":"2024-05-14T09:33:29.088030+0000","last_peered":"2024-05-14T09:33:29.088030+0000","last_clean":"2024-05-14T09:33:29.088030+0000","last_became_active":"2024-05-14T09:30:56.796333+0000","last_became_peered":"2024-05-14T09:30:56.796333+0000","last_unstale":"2024-05-14T09:33:29.088030+0000","last_undegraded":"2024-05-14T09:33:29.088030+0000","last_fullsized":"2024-05-14T09:33:29.088030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:29:34.799088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.083938+0000","last_change":"2024-05-14T09:30:56.785740+0000","last_active":"2024-05-14T09:33:29.083938+0000","last_peered":"2024-05-14T09:33:29.083938+0000","last_clean":"2024-05-14T09:33:29.083938+0000","last_became_active":"2024-05-14T09:30:56.785394+0000","last_became_peered":"2024-05-14T09:30:56.785394+0000","last_unstale":"2024-05-14T09:33:29.083938+0000","last_undegraded":"2024-05-14T09:33:29.083938+0000","last_fullsized":"2024-05-14T09:33:29.083938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:17.819523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.079641+0000","last_change":"2024-05-14T09:30:56.796957+0000","last_active":"2024-05-14T09:33:29.079641+0000","last_peered":"2024-05-14T09:33:29.079641+0000","last_clean":"2024-05-14T09:33:29.079641+0000","last_became_active":"2024-05-14T09:30:56.795071+0000","last_became_peered":"2024-05-14T09:30:56.795071+0000","last_unstale":"2024-05-14T09:33:29.079641+0000","last_undegraded":"2024-05-14T09:33:29.079641+0000","last_fullsized":"2024-05-14T09:33:29.079641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:44:31.480207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245291,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69067,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1079,"num_read_kb":613,"num_write":183,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2846,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":207201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678078,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":519087,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874308,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":1174416,"data_compressed":67998,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037834,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":518671,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1597,"internal_metadata":36174275},"hb_peers":[0,1,2,3,4,5,6,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234062,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":518009,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430292,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":1114367,"data_compressed":67998,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593817,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":514429,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":578956,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":515235,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":515365,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149799,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":514041,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953572,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":583114,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790047,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1110467,"data_compressed":67998,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33883,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33883,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":65199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T09:34:55.898 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-14T09:34:55.898 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T09:34:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.048+0000 7fd9e4757640 1 -- 172.21.15.2:0/1368632161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9dc103810 msgr2=0x7fd9dc103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.048+0000 7fd9e4757640 1 --2- 172.21.15.2:0/1368632161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9dc103810 0x7fd9dc103c20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009cd0 tx=0x7fd9d402a8e0 comp rx=0 tx=0).stop 2024-05-14T09:34:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.049+0000 7fd9e4757640 1 -- 172.21.15.2:0/1368632161 shutdown_connections 2024-05-14T09:34:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.049+0000 7fd9e4757640 1 --2- 172.21.15.2:0/1368632161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9dc103810 0x7fd9dc103c20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.049+0000 7fd9e4757640 1 --2- 172.21.15.2:0/1368632161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9dc102ec0 0x7fd9dc1032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.049+0000 7fd9e4757640 1 --2- 172.21.15.2:0/1368632161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9dc102570 0x7fd9dc102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.049+0000 7fd9e4757640 1 -- 172.21.15.2:0/1368632161 >> 172.21.15.2:0/1368632161 conn(0x7fd9dc0fdda0 msgr2=0x7fd9dc100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.049+0000 7fd9e4757640 1 -- 172.21.15.2:0/1368632161 shutdown_connections 2024-05-14T09:34:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.049+0000 7fd9e4757640 1 -- 172.21.15.2:0/1368632161 wait complete. 2024-05-14T09:34:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.049+0000 7fd9e4757640 1 Processor -- start 2024-05-14T09:34:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.049+0000 7fd9e4757640 1 -- start start 2024-05-14T09:34:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.049+0000 7fd9e4757640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9dc102570 0x7fd9dc19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.049+0000 7fd9e4757640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9dc102ec0 0x7fd9dc19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.049+0000 7fd9e4757640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9dc103810 0x7fd9dc1a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.049+0000 7fd9e4757640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc111940 con 0x7fd9dc102ec0 2024-05-14T09:34:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.049+0000 7fd9e4757640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc1117c0 con 0x7fd9dc102570 2024-05-14T09:34:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.049+0000 7fd9e4757640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc111ac0 con 0x7fd9dc103810 2024-05-14T09:34:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9e1cf2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9dc102ec0 0x7fd9dc19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9e1cf2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9dc102ec0 0x7fd9dc19c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54680/0 (socket says 172.21.15.2:54680) 2024-05-14T09:34:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9e1cf2640 1 -- 172.21.15.2:0/261445045 learned_addr learned my addr 172.21.15.2:0/261445045 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9e2cf4640 1 --2- 172.21.15.2:0/261445045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9dc103810 0x7fd9dc1a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9e24f3640 1 --2- 172.21.15.2:0/261445045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9dc102570 0x7fd9dc19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9e1cf2640 1 -- 172.21.15.2:0/261445045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9dc103810 msgr2=0x7fd9dc1a3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9e1cf2640 1 --2- 172.21.15.2:0/261445045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9dc103810 0x7fd9dc1a3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9e1cf2640 1 -- 172.21.15.2:0/261445045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9dc102570 msgr2=0x7fd9dc19c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9e1cf2640 1 --2- 172.21.15.2:0/261445045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9dc102570 0x7fd9dc19c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9e1cf2640 1 -- 172.21.15.2:0/261445045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc1a3ff0 con 0x7fd9dc102ec0 2024-05-14T09:34:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9e1cf2640 1 --2- 172.21.15.2:0/261445045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9dc102ec0 0x7fd9dc19c9f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00d950 tx=0x7fd9cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9d37fe640 1 -- 172.21.15.2:0/261445045 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc017070 con 0x7fd9dc102ec0 2024-05-14T09:34:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9d37fe640 1 -- 172.21.15.2:0/261445045 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9cc009c30 con 0x7fd9dc102ec0 2024-05-14T09:34:56.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9e4757640 1 -- 172.21.15.2:0/261445045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9dc1a42e0 con 0x7fd9dc102ec0 2024-05-14T09:34:56.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9d37fe640 1 -- 172.21.15.2:0/261445045 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00f620 con 0x7fd9dc102ec0 2024-05-14T09:34:56.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.050+0000 7fd9e4757640 1 -- 172.21.15.2:0/261445045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9dc1a4830 con 0x7fd9dc102ec0 2024-05-14T09:34:56.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.051+0000 7fd9e4757640 1 -- 172.21.15.2:0/261445045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9dc10d5b0 con 0x7fd9dc102ec0 2024-05-14T09:34:56.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.054+0000 7fd9d37fe640 1 -- 172.21.15.2:0/261445045 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd9cc010460 con 0x7fd9dc102ec0 2024-05-14T09:34:56.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.055+0000 7fd9d37fe640 1 --2- 172.21.15.2:0/261445045 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9c80b28a0 0x7fd9c80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:56.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.055+0000 7fd9e24f3640 1 --2- 172.21.15.2:0/261445045 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9c80b28a0 0x7fd9c80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:56.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.055+0000 7fd9d37fe640 1 -- 172.21.15.2:0/261445045 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd9cc0e21f0 con 0x7fd9dc102ec0 2024-05-14T09:34:56.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.055+0000 7fd9e24f3640 1 --2- 172.21.15.2:0/261445045 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9c80b28a0 0x7fd9c80b4db0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc075920 tx=0x7fd9d800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:56.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.058+0000 7fd9d37fe640 1 -- 172.21.15.2:0/261445045 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd9cc009de0 con 0x7fd9dc102ec0 2024-05-14T09:34:56.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.245+0000 7fd9e4757640 1 -- 172.21.15.2:0/261445045 --> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd9dc063110 con 0x7fd9c80b28a0 2024-05-14T09:34:56.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.254+0000 7fd9d37fe640 1 -- 172.21.15.2:0/261445045 <== mgr.14152 v2:172.21.15.2:6800/3050449557 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340740 (secure 0 0 0) 0x7fd9dc063110 con 0x7fd9c80b28a0 2024-05-14T09:34:56.258 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:34:56.260 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-05-14T09:34:56.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.258+0000 7fd9e4757640 1 -- 172.21.15.2:0/261445045 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9c80b28a0 msgr2=0x7fd9c80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:56.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.258+0000 7fd9e4757640 1 --2- 172.21.15.2:0/261445045 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9c80b28a0 0x7fd9c80b4db0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc075920 tx=0x7fd9d800b390 comp rx=0 tx=0).stop 2024-05-14T09:34:56.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.258+0000 7fd9e4757640 1 -- 172.21.15.2:0/261445045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9dc102ec0 msgr2=0x7fd9dc19c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:56.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.258+0000 7fd9e4757640 1 --2- 172.21.15.2:0/261445045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9dc102ec0 0x7fd9dc19c9f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00d950 tx=0x7fd9cc00de10 comp rx=0 tx=0).stop 2024-05-14T09:34:56.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.258+0000 7fd9e4757640 1 -- 172.21.15.2:0/261445045 shutdown_connections 2024-05-14T09:34:56.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.258+0000 7fd9e4757640 1 --2- 172.21.15.2:0/261445045 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9c80b28a0 0x7fd9c80b4db0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:56.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.258+0000 7fd9e4757640 1 --2- 172.21.15.2:0/261445045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9dc103810 0x7fd9dc1a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:56.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.258+0000 7fd9e4757640 1 --2- 172.21.15.2:0/261445045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9dc102ec0 0x7fd9dc19c9f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:56.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.258+0000 7fd9e4757640 1 --2- 172.21.15.2:0/261445045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9dc102570 0x7fd9dc19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:56.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.258+0000 7fd9e4757640 1 -- 172.21.15.2:0/261445045 >> 172.21.15.2:0/261445045 conn(0x7fd9dc0fdda0 msgr2=0x7fd9dc10a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:56.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.258+0000 7fd9e4757640 1 -- 172.21.15.2:0/261445045 shutdown_connections 2024-05-14T09:34:56.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:56.258+0000 7fd9e4757640 1 -- 172.21.15.2:0/261445045 wait complete. 2024-05-14T09:34:56.270 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-05-14T09:34:55.107700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659669,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1155,"num_read_kb":677,"num_write":308,"num_write_kb":2281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2954686,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":750,"ondisk_log_size":750,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":394500,"kb_used_data":12068,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124547324,"statfs":{"total":1151940427776,"available":1151536459776,"internally_reserved":0,"allocated":12357632,"data_stored":8176157,"data_compressed":496431,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19140,"internal_metadata":391492924},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":32,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003660"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087224+0000","last_change":"2024-05-14T09:30:56.797051+0000","last_active":"2024-05-14T09:33:29.087224+0000","last_peered":"2024-05-14T09:33:29.087224+0000","last_clean":"2024-05-14T09:33:29.087224+0000","last_became_active":"2024-05-14T09:30:56.796517+0000","last_became_peered":"2024-05-14T09:30:56.796517+0000","last_unstale":"2024-05-14T09:33:29.087224+0000","last_undegraded":"2024-05-14T09:33:29.087224+0000","last_fullsized":"2024-05-14T09:33:29.087224+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:58:22.170187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.073126+0000","last_change":"2024-05-14T09:30:56.795325+0000","last_active":"2024-05-14T09:33:29.073126+0000","last_peered":"2024-05-14T09:33:29.073126+0000","last_clean":"2024-05-14T09:33:29.073126+0000","last_became_active":"2024-05-14T09:30:56.794962+0000","last_became_peered":"2024-05-14T09:30:56.794962+0000","last_unstale":"2024-05-14T09:33:29.073126+0000","last_undegraded":"2024-05-14T09:33:29.073126+0000","last_fullsized":"2024-05-14T09:33:29.073126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:15:28.890565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084179+0000","last_change":"2024-05-14T09:30:56.793200+0000","last_active":"2024-05-14T09:33:29.084179+0000","last_peered":"2024-05-14T09:33:29.084179+0000","last_clean":"2024-05-14T09:33:29.084179+0000","last_became_active":"2024-05-14T09:30:56.792835+0000","last_became_peered":"2024-05-14T09:30:56.792835+0000","last_unstale":"2024-05-14T09:33:29.084179+0000","last_undegraded":"2024-05-14T09:33:29.084179+0000","last_fullsized":"2024-05-14T09:33:29.084179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:19.851276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.070997+0000","last_change":"2024-05-14T09:30:56.790238+0000","last_active":"2024-05-14T09:33:29.070997+0000","last_peered":"2024-05-14T09:33:29.070997+0000","last_clean":"2024-05-14T09:33:29.070997+0000","last_became_active":"2024-05-14T09:30:56.789826+0000","last_became_peered":"2024-05-14T09:30:56.789826+0000","last_unstale":"2024-05-14T09:33:29.070997+0000","last_undegraded":"2024-05-14T09:33:29.070997+0000","last_fullsized":"2024-05-14T09:33:29.070997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:19:19.853931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080306+0000","last_change":"2024-05-14T09:30:56.788021+0000","last_active":"2024-05-14T09:33:29.080306+0000","last_peered":"2024-05-14T09:33:29.080306+0000","last_clean":"2024-05-14T09:33:29.080306+0000","last_became_active":"2024-05-14T09:30:56.787657+0000","last_became_peered":"2024-05-14T09:30:56.787657+0000","last_unstale":"2024-05-14T09:33:29.080306+0000","last_undegraded":"2024-05-14T09:33:29.080306+0000","last_fullsized":"2024-05-14T09:33:29.080306+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:37:03.232137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.082547+0000","last_change":"2024-05-14T09:32:30.525175+0000","last_active":"2024-05-14T09:33:29.082547+0000","last_peered":"2024-05-14T09:33:29.082547+0000","last_clean":"2024-05-14T09:33:29.082547+0000","last_became_active":"2024-05-14T09:32:30.524861+0000","last_became_peered":"2024-05-14T09:32:30.524861+0000","last_unstale":"2024-05-14T09:33:29.082547+0000","last_undegraded":"2024-05-14T09:33:29.082547+0000","last_fullsized":"2024-05-14T09:33:29.082547+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:22:21.063843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.066050+0000","last_change":"2024-05-14T09:30:56.795647+0000","last_active":"2024-05-14T09:33:29.066050+0000","last_peered":"2024-05-14T09:33:29.066050+0000","last_clean":"2024-05-14T09:33:29.066050+0000","last_became_active":"2024-05-14T09:30:56.795186+0000","last_became_peered":"2024-05-14T09:30:56.795186+0000","last_unstale":"2024-05-14T09:33:29.066050+0000","last_undegraded":"2024-05-14T09:33:29.066050+0000","last_fullsized":"2024-05-14T09:33:29.066050+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:04:53.471305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081501+0000","last_change":"2024-05-14T09:30:56.796834+0000","last_active":"2024-05-14T09:33:29.081501+0000","last_peered":"2024-05-14T09:33:29.081501+0000","last_clean":"2024-05-14T09:33:29.081501+0000","last_became_active":"2024-05-14T09:30:56.796492+0000","last_became_peered":"2024-05-14T09:30:56.796492+0000","last_unstale":"2024-05-14T09:33:29.081501+0000","last_undegraded":"2024-05-14T09:33:29.081501+0000","last_fullsized":"2024-05-14T09:33:29.081501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:46.606129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.086070+0000","last_change":"2024-05-14T09:30:56.795020+0000","last_active":"2024-05-14T09:33:29.086070+0000","last_peered":"2024-05-14T09:33:29.086070+0000","last_clean":"2024-05-14T09:33:29.086070+0000","last_became_active":"2024-05-14T09:30:56.794625+0000","last_became_peered":"2024-05-14T09:30:56.794625+0000","last_unstale":"2024-05-14T09:33:29.086070+0000","last_undegraded":"2024-05-14T09:33:29.086070+0000","last_fullsized":"2024-05-14T09:33:29.086070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:09:38.597296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.082128+0000","last_change":"2024-05-14T09:30:56.796439+0000","last_active":"2024-05-14T09:33:29.082128+0000","last_peered":"2024-05-14T09:33:29.082128+0000","last_clean":"2024-05-14T09:33:29.082128+0000","last_became_active":"2024-05-14T09:30:56.795168+0000","last_became_peered":"2024-05-14T09:30:56.795168+0000","last_unstale":"2024-05-14T09:33:29.082128+0000","last_undegraded":"2024-05-14T09:33:29.082128+0000","last_fullsized":"2024-05-14T09:33:29.082128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:47.660286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087219+0000","last_change":"2024-05-14T09:30:56.795994+0000","last_active":"2024-05-14T09:33:29.087219+0000","last_peered":"2024-05-14T09:33:29.087219+0000","last_clean":"2024-05-14T09:33:29.087219+0000","last_became_active":"2024-05-14T09:30:56.795079+0000","last_became_peered":"2024-05-14T09:30:56.795079+0000","last_unstale":"2024-05-14T09:33:29.087219+0000","last_undegraded":"2024-05-14T09:33:29.087219+0000","last_fullsized":"2024-05-14T09:33:29.087219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:37:42.930195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.072741+0000","last_change":"2024-05-14T09:30:56.792997+0000","last_active":"2024-05-14T09:33:29.072741+0000","last_peered":"2024-05-14T09:33:29.072741+0000","last_clean":"2024-05-14T09:33:29.072741+0000","last_became_active":"2024-05-14T09:30:56.792302+0000","last_became_peered":"2024-05-14T09:30:56.792302+0000","last_unstale":"2024-05-14T09:33:29.072741+0000","last_undegraded":"2024-05-14T09:33:29.072741+0000","last_fullsized":"2024-05-14T09:33:29.072741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:52.986662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077933+0000","last_change":"2024-05-14T09:30:56.795903+0000","last_active":"2024-05-14T09:33:29.077933+0000","last_peered":"2024-05-14T09:33:29.077933+0000","last_clean":"2024-05-14T09:33:29.077933+0000","last_became_active":"2024-05-14T09:30:56.795466+0000","last_became_peered":"2024-05-14T09:30:56.795466+0000","last_unstale":"2024-05-14T09:33:29.077933+0000","last_undegraded":"2024-05-14T09:33:29.077933+0000","last_fullsized":"2024-05-14T09:33:29.077933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:02:10.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.070939+0000","last_change":"2024-05-14T09:30:56.795911+0000","last_active":"2024-05-14T09:33:29.070939+0000","last_peered":"2024-05-14T09:33:29.070939+0000","last_clean":"2024-05-14T09:33:29.070939+0000","last_became_active":"2024-05-14T09:30:56.795547+0000","last_became_peered":"2024-05-14T09:30:56.795547+0000","last_unstale":"2024-05-14T09:33:29.070939+0000","last_undegraded":"2024-05-14T09:33:29.070939+0000","last_fullsized":"2024-05-14T09:33:29.070939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:56:29.621783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.067755+0000","last_change":"2024-05-14T09:30:56.787745+0000","last_active":"2024-05-14T09:33:29.067755+0000","last_peered":"2024-05-14T09:33:29.067755+0000","last_clean":"2024-05-14T09:33:29.067755+0000","last_became_active":"2024-05-14T09:30:56.787385+0000","last_became_peered":"2024-05-14T09:30:56.787385+0000","last_unstale":"2024-05-14T09:33:29.067755+0000","last_undegraded":"2024-05-14T09:33:29.067755+0000","last_fullsized":"2024-05-14T09:33:29.067755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:11.256775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087830+0000","last_change":"2024-05-14T09:30:56.795785+0000","last_active":"2024-05-14T09:33:29.087830+0000","last_peered":"2024-05-14T09:33:29.087830+0000","last_clean":"2024-05-14T09:33:29.087830+0000","last_became_active":"2024-05-14T09:30:56.795436+0000","last_became_peered":"2024-05-14T09:30:56.795436+0000","last_unstale":"2024-05-14T09:33:29.087830+0000","last_undegraded":"2024-05-14T09:33:29.087830+0000","last_fullsized":"2024-05-14T09:33:29.087830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:12:09.221929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087321+0000","last_change":"2024-05-14T09:30:56.800688+0000","last_active":"2024-05-14T09:33:29.087321+0000","last_peered":"2024-05-14T09:33:29.087321+0000","last_clean":"2024-05-14T09:33:29.087321+0000","last_became_active":"2024-05-14T09:30:56.800204+0000","last_became_peered":"2024-05-14T09:30:56.800204+0000","last_unstale":"2024-05-14T09:33:29.087321+0000","last_undegraded":"2024-05-14T09:33:29.087321+0000","last_fullsized":"2024-05-14T09:33:29.087321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:32:27.057688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.075054+0000","last_change":"2024-05-14T09:30:56.795604+0000","last_active":"2024-05-14T09:33:29.075054+0000","last_peered":"2024-05-14T09:33:29.075054+0000","last_clean":"2024-05-14T09:33:29.075054+0000","last_became_active":"2024-05-14T09:30:56.795245+0000","last_became_peered":"2024-05-14T09:30:56.795245+0000","last_unstale":"2024-05-14T09:33:29.075054+0000","last_undegraded":"2024-05-14T09:33:29.075054+0000","last_fullsized":"2024-05-14T09:33:29.075054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:14.072584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080216+0000","last_change":"2024-05-14T09:30:56.794815+0000","last_active":"2024-05-14T09:33:29.080216+0000","last_peered":"2024-05-14T09:33:29.080216+0000","last_clean":"2024-05-14T09:33:29.080216+0000","last_became_active":"2024-05-14T09:30:56.794429+0000","last_became_peered":"2024-05-14T09:30:56.794429+0000","last_unstale":"2024-05-14T09:33:29.080216+0000","last_undegraded":"2024-05-14T09:33:29.080216+0000","last_fullsized":"2024-05-14T09:33:29.080216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:32:08.367046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319506+0000","last_change":"2024-05-14T09:31:08.107895+0000","last_active":"2024-05-14T09:32:30.319506+0000","last_peered":"2024-05-14T09:32:30.319506+0000","last_clean":"2024-05-14T09:32:30.319506+0000","last_became_active":"2024-05-14T09:31:08.107458+0000","last_became_peered":"2024-05-14T09:31:08.107458+0000","last_unstale":"2024-05-14T09:32:30.319506+0000","last_undegraded":"2024-05-14T09:32:30.319506+0000","last_fullsized":"2024-05-14T09:32:30.319506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:13:11.717753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":69,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319382+0000","last_change":"2024-05-14T09:31:30.178197+0000","last_active":"2024-05-14T09:32:30.319382+0000","last_peered":"2024-05-14T09:32:30.319382+0000","last_clean":"2024-05-14T09:32:30.319382+0000","last_became_active":"2024-05-14T09:31:30.171154+0000","last_became_peered":"2024-05-14T09:31:30.171154+0000","last_unstale":"2024-05-14T09:32:30.319382+0000","last_undegraded":"2024-05-14T09:32:30.319382+0000","last_fullsized":"2024-05-14T09:32:30.319382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:43:46.208439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077181+0000","last_change":"2024-05-14T09:30:56.800526+0000","last_active":"2024-05-14T09:33:29.077181+0000","last_peered":"2024-05-14T09:33:29.077181+0000","last_clean":"2024-05-14T09:33:29.077181+0000","last_became_active":"2024-05-14T09:30:56.797692+0000","last_became_peered":"2024-05-14T09:30:56.797692+0000","last_unstale":"2024-05-14T09:33:29.077181+0000","last_undegraded":"2024-05-14T09:33:29.077181+0000","last_fullsized":"2024-05-14T09:33:29.077181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:09:14.471173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522407+0000","last_change":"2024-05-14T09:31:30.173810+0000","last_active":"2024-05-14T09:32:30.522407+0000","last_peered":"2024-05-14T09:32:30.522407+0000","last_clean":"2024-05-14T09:32:30.522407+0000","last_became_active":"2024-05-14T09:31:30.173242+0000","last_became_peered":"2024-05-14T09:31:30.173242+0000","last_unstale":"2024-05-14T09:32:30.522407+0000","last_undegraded":"2024-05-14T09:32:30.522407+0000","last_fullsized":"2024-05-14T09:32:30.522407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:37:15.991805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.322883+0000","last_change":"2024-05-14T09:30:54.015636+0000","last_active":"2024-05-14T09:32:30.322883+0000","last_peered":"2024-05-14T09:32:30.322883+0000","last_clean":"2024-05-14T09:32:30.322883+0000","last_became_active":"2024-05-14T09:30:54.015290+0000","last_became_peered":"2024-05-14T09:30:54.015290+0000","last_unstale":"2024-05-14T09:32:30.322883+0000","last_undegraded":"2024-05-14T09:32:30.322883+0000","last_fullsized":"2024-05-14T09:32:30.322883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:29:17.548817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.088634+0000","last_change":"2024-05-14T09:30:56.799506+0000","last_active":"2024-05-14T09:33:29.088634+0000","last_peered":"2024-05-14T09:33:29.088634+0000","last_clean":"2024-05-14T09:33:29.088634+0000","last_became_active":"2024-05-14T09:30:56.799153+0000","last_became_peered":"2024-05-14T09:30:56.799153+0000","last_unstale":"2024-05-14T09:33:29.088634+0000","last_undegraded":"2024-05-14T09:33:29.088634+0000","last_fullsized":"2024-05-14T09:33:29.088634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:04:08.217337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522681+0000","last_change":"2024-05-14T09:31:08.115533+0000","last_active":"2024-05-14T09:32:30.522681+0000","last_peered":"2024-05-14T09:32:30.522681+0000","last_clean":"2024-05-14T09:32:30.522681+0000","last_became_active":"2024-05-14T09:31:08.115033+0000","last_became_peered":"2024-05-14T09:31:08.115033+0000","last_unstale":"2024-05-14T09:32:30.522681+0000","last_undegraded":"2024-05-14T09:32:30.522681+0000","last_fullsized":"2024-05-14T09:32:30.522681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:15.668357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524484+0000","last_change":"2024-05-14T09:30:53.770547+0000","last_active":"2024-05-14T09:32:30.524484+0000","last_peered":"2024-05-14T09:32:30.524484+0000","last_clean":"2024-05-14T09:32:30.524484+0000","last_became_active":"2024-05-14T09:30:53.770110+0000","last_became_peered":"2024-05-14T09:30:53.770110+0000","last_unstale":"2024-05-14T09:32:30.524484+0000","last_undegraded":"2024-05-14T09:32:30.524484+0000","last_fullsized":"2024-05-14T09:32:30.524484+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:18.371318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084457+0000","last_change":"2024-05-14T09:30:56.799427+0000","last_active":"2024-05-14T09:33:29.084457+0000","last_peered":"2024-05-14T09:33:29.084457+0000","last_clean":"2024-05-14T09:33:29.084457+0000","last_became_active":"2024-05-14T09:30:56.799018+0000","last_became_peered":"2024-05-14T09:30:56.799018+0000","last_unstale":"2024-05-14T09:33:29.084457+0000","last_undegraded":"2024-05-14T09:33:29.084457+0000","last_fullsized":"2024-05-14T09:33:29.084457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:13:31.837985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522635+0000","last_change":"2024-05-14T09:31:08.116258+0000","last_active":"2024-05-14T09:32:30.522635+0000","last_peered":"2024-05-14T09:32:30.522635+0000","last_clean":"2024-05-14T09:32:30.522635+0000","last_became_active":"2024-05-14T09:31:08.115040+0000","last_became_peered":"2024-05-14T09:31:08.115040+0000","last_unstale":"2024-05-14T09:32:30.522635+0000","last_undegraded":"2024-05-14T09:32:30.522635+0000","last_fullsized":"2024-05-14T09:32:30.522635+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:57:54.839203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523793+0000","last_change":"2024-05-14T09:31:30.188984+0000","last_active":"2024-05-14T09:32:30.523793+0000","last_peered":"2024-05-14T09:32:30.523793+0000","last_clean":"2024-05-14T09:32:30.523793+0000","last_became_active":"2024-05-14T09:31:30.174410+0000","last_became_peered":"2024-05-14T09:31:30.174410+0000","last_unstale":"2024-05-14T09:32:30.523793+0000","last_undegraded":"2024-05-14T09:32:30.523793+0000","last_fullsized":"2024-05-14T09:32:30.523793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:01:35.713857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,4],"acting":[8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081749+0000","last_change":"2024-05-14T09:32:30.319390+0000","last_active":"2024-05-14T09:33:29.081749+0000","last_peered":"2024-05-14T09:33:29.081749+0000","last_clean":"2024-05-14T09:33:29.081749+0000","last_became_active":"2024-05-14T09:32:30.318645+0000","last_became_peered":"2024-05-14T09:32:30.318645+0000","last_unstale":"2024-05-14T09:33:29.081749+0000","last_undegraded":"2024-05-14T09:33:29.081749+0000","last_fullsized":"2024-05-14T09:33:29.081749+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:37:49.866996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320705+0000","last_change":"2024-05-14T09:31:08.115740+0000","last_active":"2024-05-14T09:32:30.320705+0000","last_peered":"2024-05-14T09:32:30.320705+0000","last_clean":"2024-05-14T09:32:30.320705+0000","last_became_active":"2024-05-14T09:31:08.115202+0000","last_became_peered":"2024-05-14T09:31:08.115202+0000","last_unstale":"2024-05-14T09:32:30.320705+0000","last_undegraded":"2024-05-14T09:32:30.320705+0000","last_fullsized":"2024-05-14T09:32:30.320705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:44.519280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":233,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:34:51.508971+0000","last_change":"2024-05-14T09:30:53.770869+0000","last_active":"2024-05-14T09:34:51.508971+0000","last_peered":"2024-05-14T09:34:51.508971+0000","last_clean":"2024-05-14T09:34:51.508971+0000","last_became_active":"2024-05-14T09:30:53.770425+0000","last_became_peered":"2024-05-14T09:30:53.770425+0000","last_unstale":"2024-05-14T09:34:51.508971+0000","last_undegraded":"2024-05-14T09:34:51.508971+0000","last_fullsized":"2024-05-14T09:34:51.508971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:03:33.768120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":180,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.059353+0000","last_change":"2024-05-14T09:30:56.789267+0000","last_active":"2024-05-14T09:33:29.059353+0000","last_peered":"2024-05-14T09:33:29.059353+0000","last_clean":"2024-05-14T09:33:29.059353+0000","last_became_active":"2024-05-14T09:30:56.788883+0000","last_became_peered":"2024-05-14T09:30:56.788883+0000","last_unstale":"2024-05-14T09:33:29.059353+0000","last_undegraded":"2024-05-14T09:33:29.059353+0000","last_fullsized":"2024-05-14T09:33:29.059353+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:09.818693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320595+0000","last_change":"2024-05-14T09:31:08.115193+0000","last_active":"2024-05-14T09:32:30.320595+0000","last_peered":"2024-05-14T09:32:30.320595+0000","last_clean":"2024-05-14T09:32:30.320595+0000","last_became_active":"2024-05-14T09:31:08.114701+0000","last_became_peered":"2024-05-14T09:31:08.114701+0000","last_unstale":"2024-05-14T09:32:30.320595+0000","last_undegraded":"2024-05-14T09:32:30.320595+0000","last_fullsized":"2024-05-14T09:32:30.320595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:47:48.487922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319686+0000","last_change":"2024-05-14T09:31:30.174760+0000","last_active":"2024-05-14T09:32:30.319686+0000","last_peered":"2024-05-14T09:32:30.319686+0000","last_clean":"2024-05-14T09:32:30.319686+0000","last_became_active":"2024-05-14T09:31:30.174253+0000","last_became_peered":"2024-05-14T09:31:30.174253+0000","last_unstale":"2024-05-14T09:32:30.319686+0000","last_undegraded":"2024-05-14T09:32:30.319686+0000","last_fullsized":"2024-05-14T09:32:30.319686+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:11:17.163436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081148+0000","last_change":"2024-05-14T09:30:56.794654+0000","last_active":"2024-05-14T09:33:29.081148+0000","last_peered":"2024-05-14T09:33:29.081148+0000","last_clean":"2024-05-14T09:33:29.081148+0000","last_became_active":"2024-05-14T09:30:56.794282+0000","last_became_peered":"2024-05-14T09:30:56.794282+0000","last_unstale":"2024-05-14T09:33:29.081148+0000","last_undegraded":"2024-05-14T09:33:29.081148+0000","last_fullsized":"2024-05-14T09:33:29.081148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:55:41.478707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320586+0000","last_change":"2024-05-14T09:31:08.119965+0000","last_active":"2024-05-14T09:32:30.320586+0000","last_peered":"2024-05-14T09:32:30.320586+0000","last_clean":"2024-05-14T09:32:30.320586+0000","last_became_active":"2024-05-14T09:31:08.119502+0000","last_became_peered":"2024-05-14T09:31:08.119502+0000","last_unstale":"2024-05-14T09:32:30.320586+0000","last_undegraded":"2024-05-14T09:32:30.320586+0000","last_fullsized":"2024-05-14T09:32:30.320586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:01:23.612036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.314804+0000","last_change":"2024-05-14T09:30:53.766495+0000","last_active":"2024-05-14T09:32:30.314804+0000","last_peered":"2024-05-14T09:32:30.314804+0000","last_clean":"2024-05-14T09:32:30.314804+0000","last_became_active":"2024-05-14T09:30:53.766122+0000","last_became_peered":"2024-05-14T09:30:53.766122+0000","last_unstale":"2024-05-14T09:32:30.314804+0000","last_undegraded":"2024-05-14T09:32:30.314804+0000","last_fullsized":"2024-05-14T09:32:30.314804+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:43:13.344209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.073417+0000","last_change":"2024-05-14T09:30:56.791938+0000","last_active":"2024-05-14T09:33:29.073417+0000","last_peered":"2024-05-14T09:33:29.073417+0000","last_clean":"2024-05-14T09:33:29.073417+0000","last_became_active":"2024-05-14T09:30:56.791525+0000","last_became_peered":"2024-05-14T09:30:56.791525+0000","last_unstale":"2024-05-14T09:33:29.073417+0000","last_undegraded":"2024-05-14T09:33:29.073417+0000","last_fullsized":"2024-05-14T09:33:29.073417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:18.178208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320104+0000","last_change":"2024-05-14T09:31:08.113548+0000","last_active":"2024-05-14T09:32:30.320104+0000","last_peered":"2024-05-14T09:32:30.320104+0000","last_clean":"2024-05-14T09:32:30.320104+0000","last_became_active":"2024-05-14T09:31:08.113117+0000","last_became_peered":"2024-05-14T09:31:08.113117+0000","last_unstale":"2024-05-14T09:32:30.320104+0000","last_undegraded":"2024-05-14T09:32:30.320104+0000","last_fullsized":"2024-05-14T09:32:30.320104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:25:15.483761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315966+0000","last_change":"2024-05-14T09:30:53.770987+0000","last_active":"2024-05-14T09:32:30.315966+0000","last_peered":"2024-05-14T09:32:30.315966+0000","last_clean":"2024-05-14T09:32:30.315966+0000","last_became_active":"2024-05-14T09:30:53.770609+0000","last_became_peered":"2024-05-14T09:30:53.770609+0000","last_unstale":"2024-05-14T09:32:30.315966+0000","last_undegraded":"2024-05-14T09:32:30.315966+0000","last_fullsized":"2024-05-14T09:32:30.315966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:26:22.020642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081149+0000","last_change":"2024-05-14T09:30:56.789219+0000","last_active":"2024-05-14T09:33:29.081149+0000","last_peered":"2024-05-14T09:33:29.081149+0000","last_clean":"2024-05-14T09:33:29.081149+0000","last_became_active":"2024-05-14T09:30:56.788871+0000","last_became_peered":"2024-05-14T09:30:56.788871+0000","last_unstale":"2024-05-14T09:33:29.081149+0000","last_undegraded":"2024-05-14T09:33:29.081149+0000","last_fullsized":"2024-05-14T09:33:29.081149+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:10:50.774267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.322531+0000","last_change":"2024-05-14T09:31:30.174708+0000","last_active":"2024-05-14T09:32:30.322531+0000","last_peered":"2024-05-14T09:32:30.322531+0000","last_clean":"2024-05-14T09:32:30.322531+0000","last_became_active":"2024-05-14T09:31:30.174199+0000","last_became_peered":"2024-05-14T09:31:30.174199+0000","last_unstale":"2024-05-14T09:32:30.322531+0000","last_undegraded":"2024-05-14T09:32:30.322531+0000","last_fullsized":"2024-05-14T09:32:30.322531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:23:55.939302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":225,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:34:51.508350+0000","last_change":"2024-05-14T09:30:53.770898+0000","last_active":"2024-05-14T09:34:51.508350+0000","last_peered":"2024-05-14T09:34:51.508350+0000","last_clean":"2024-05-14T09:34:51.508350+0000","last_became_active":"2024-05-14T09:30:53.770483+0000","last_became_peered":"2024-05-14T09:30:53.770483+0000","last_unstale":"2024-05-14T09:34:51.508350+0000","last_undegraded":"2024-05-14T09:34:51.508350+0000","last_fullsized":"2024-05-14T09:34:51.508350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:07.764348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084404+0000","last_change":"2024-05-14T09:30:56.799049+0000","last_active":"2024-05-14T09:33:29.084404+0000","last_peered":"2024-05-14T09:33:29.084404+0000","last_clean":"2024-05-14T09:33:29.084404+0000","last_became_active":"2024-05-14T09:30:56.798643+0000","last_became_peered":"2024-05-14T09:30:56.798643+0000","last_unstale":"2024-05-14T09:33:29.084404+0000","last_undegraded":"2024-05-14T09:33:29.084404+0000","last_fullsized":"2024-05-14T09:33:29.084404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:12.378373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.318070+0000","last_change":"2024-05-14T09:31:08.109876+0000","last_active":"2024-05-14T09:32:30.318070+0000","last_peered":"2024-05-14T09:32:30.318070+0000","last_clean":"2024-05-14T09:32:30.318070+0000","last_became_active":"2024-05-14T09:31:08.109470+0000","last_became_peered":"2024-05-14T09:31:08.109470+0000","last_unstale":"2024-05-14T09:32:30.318070+0000","last_undegraded":"2024-05-14T09:32:30.318070+0000","last_fullsized":"2024-05-14T09:32:30.318070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:07.704153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.015003+0000","last_change":"2024-05-14T09:31:30.179086+0000","last_active":"2024-05-14T09:32:31.015003+0000","last_peered":"2024-05-14T09:32:31.015003+0000","last_clean":"2024-05-14T09:32:31.015003+0000","last_became_active":"2024-05-14T09:31:30.178604+0000","last_became_peered":"2024-05-14T09:31:30.178604+0000","last_unstale":"2024-05-14T09:32:31.015003+0000","last_undegraded":"2024-05-14T09:32:31.015003+0000","last_fullsized":"2024-05-14T09:32:31.015003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:39.712395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.073315+0000","last_change":"2024-05-14T09:30:56.793109+0000","last_active":"2024-05-14T09:33:29.073315+0000","last_peered":"2024-05-14T09:33:29.073315+0000","last_clean":"2024-05-14T09:33:29.073315+0000","last_became_active":"2024-05-14T09:30:56.792761+0000","last_became_peered":"2024-05-14T09:30:56.792761+0000","last_unstale":"2024-05-14T09:33:29.073315+0000","last_undegraded":"2024-05-14T09:33:29.073315+0000","last_fullsized":"2024-05-14T09:33:29.073315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:25:33.216281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522916+0000","last_change":"2024-05-14T09:31:08.120978+0000","last_active":"2024-05-14T09:32:30.522916+0000","last_peered":"2024-05-14T09:32:30.522916+0000","last_clean":"2024-05-14T09:32:30.522916+0000","last_became_active":"2024-05-14T09:31:08.120227+0000","last_became_peered":"2024-05-14T09:31:08.120227+0000","last_unstale":"2024-05-14T09:32:30.522916+0000","last_undegraded":"2024-05-14T09:32:30.522916+0000","last_fullsized":"2024-05-14T09:32:30.522916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:06.630390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316348+0000","last_change":"2024-05-14T09:30:54.017382+0000","last_active":"2024-05-14T09:32:30.316348+0000","last_peered":"2024-05-14T09:32:30.316348+0000","last_clean":"2024-05-14T09:32:30.316348+0000","last_became_active":"2024-05-14T09:30:54.016887+0000","last_became_peered":"2024-05-14T09:30:54.016887+0000","last_unstale":"2024-05-14T09:32:30.316348+0000","last_undegraded":"2024-05-14T09:32:30.316348+0000","last_fullsized":"2024-05-14T09:32:30.316348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:28.823659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.085392+0000","last_change":"2024-05-14T09:30:56.800626+0000","last_active":"2024-05-14T09:33:29.085392+0000","last_peered":"2024-05-14T09:33:29.085392+0000","last_clean":"2024-05-14T09:33:29.085392+0000","last_became_active":"2024-05-14T09:30:56.800258+0000","last_became_peered":"2024-05-14T09:30:56.800258+0000","last_unstale":"2024-05-14T09:33:29.085392+0000","last_undegraded":"2024-05-14T09:33:29.085392+0000","last_fullsized":"2024-05-14T09:33:29.085392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:47:26.369633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.318135+0000","last_change":"2024-05-14T09:31:08.115456+0000","last_active":"2024-05-14T09:32:30.318135+0000","last_peered":"2024-05-14T09:32:30.318135+0000","last_clean":"2024-05-14T09:32:30.318135+0000","last_became_active":"2024-05-14T09:31:08.114705+0000","last_became_peered":"2024-05-14T09:31:08.114705+0000","last_unstale":"2024-05-14T09:32:30.318135+0000","last_undegraded":"2024-05-14T09:32:30.318135+0000","last_fullsized":"2024-05-14T09:32:30.318135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:50.096747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'39","reported_seq":461,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:34:52.509793+0000","last_change":"2024-05-14T09:31:30.186202+0000","last_active":"2024-05-14T09:34:52.509793+0000","last_peered":"2024-05-14T09:34:52.509793+0000","last_clean":"2024-05-14T09:34:52.509793+0000","last_became_active":"2024-05-14T09:31:30.174248+0000","last_became_peered":"2024-05-14T09:31:30.174248+0000","last_unstale":"2024-05-14T09:34:52.509793+0000","last_undegraded":"2024-05-14T09:34:52.509793+0000","last_fullsized":"2024-05-14T09:34:52.509793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:45:03.131934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64617,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":361,"num_read_kb":244,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077083+0000","last_change":"2024-05-14T09:30:56.791099+0000","last_active":"2024-05-14T09:33:29.077083+0000","last_peered":"2024-05-14T09:33:29.077083+0000","last_clean":"2024-05-14T09:33:29.077083+0000","last_became_active":"2024-05-14T09:30:56.790676+0000","last_became_peered":"2024-05-14T09:30:56.790676+0000","last_unstale":"2024-05-14T09:33:29.077083+0000","last_undegraded":"2024-05-14T09:33:29.077083+0000","last_fullsized":"2024-05-14T09:33:29.077083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:09.851357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316986+0000","last_change":"2024-05-14T09:31:08.120289+0000","last_active":"2024-05-14T09:32:30.316986+0000","last_peered":"2024-05-14T09:32:30.316986+0000","last_clean":"2024-05-14T09:32:30.316986+0000","last_became_active":"2024-05-14T09:31:08.118574+0000","last_became_peered":"2024-05-14T09:31:08.118574+0000","last_unstale":"2024-05-14T09:32:30.316986+0000","last_undegraded":"2024-05-14T09:32:30.316986+0000","last_fullsized":"2024-05-14T09:32:30.316986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:01.723442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319990+0000","last_change":"2024-05-14T09:30:53.766578+0000","last_active":"2024-05-14T09:32:30.319990+0000","last_peered":"2024-05-14T09:32:30.319990+0000","last_clean":"2024-05-14T09:32:30.319990+0000","last_became_active":"2024-05-14T09:30:53.765928+0000","last_became_peered":"2024-05-14T09:30:53.765928+0000","last_unstale":"2024-05-14T09:32:30.319990+0000","last_undegraded":"2024-05-14T09:32:30.319990+0000","last_fullsized":"2024-05-14T09:32:30.319990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:19:08.690013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.083536+0000","last_change":"2024-05-14T09:30:56.785507+0000","last_active":"2024-05-14T09:33:29.083536+0000","last_peered":"2024-05-14T09:33:29.083536+0000","last_clean":"2024-05-14T09:33:29.083536+0000","last_became_active":"2024-05-14T09:30:56.784800+0000","last_became_peered":"2024-05-14T09:30:56.784800+0000","last_unstale":"2024-05-14T09:33:29.083536+0000","last_undegraded":"2024-05-14T09:33:29.083536+0000","last_fullsized":"2024-05-14T09:33:29.083536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:36:05.440222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.321298+0000","last_change":"2024-05-14T09:31:08.119051+0000","last_active":"2024-05-14T09:32:30.321298+0000","last_peered":"2024-05-14T09:32:30.321298+0000","last_clean":"2024-05-14T09:32:30.321298+0000","last_became_active":"2024-05-14T09:31:08.118569+0000","last_became_peered":"2024-05-14T09:31:08.118569+0000","last_unstale":"2024-05-14T09:32:30.321298+0000","last_undegraded":"2024-05-14T09:32:30.321298+0000","last_fullsized":"2024-05-14T09:32:30.321298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:12:39.643740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316079+0000","last_change":"2024-05-14T09:30:53.772832+0000","last_active":"2024-05-14T09:32:30.316079+0000","last_peered":"2024-05-14T09:32:30.316079+0000","last_clean":"2024-05-14T09:32:30.316079+0000","last_became_active":"2024-05-14T09:30:53.772228+0000","last_became_peered":"2024-05-14T09:30:53.772228+0000","last_unstale":"2024-05-14T09:32:30.316079+0000","last_undegraded":"2024-05-14T09:32:30.316079+0000","last_fullsized":"2024-05-14T09:32:30.316079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:14:43.350330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.084806+0000","last_change":"2024-05-14T09:30:56.782831+0000","last_active":"2024-05-14T09:33:29.084806+0000","last_peered":"2024-05-14T09:33:29.084806+0000","last_clean":"2024-05-14T09:33:29.084806+0000","last_became_active":"2024-05-14T09:30:56.782402+0000","last_became_peered":"2024-05-14T09:30:56.782402+0000","last_unstale":"2024-05-14T09:33:29.084806+0000","last_undegraded":"2024-05-14T09:33:29.084806+0000","last_fullsized":"2024-05-14T09:33:29.084806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:54.603524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524041+0000","last_change":"2024-05-14T09:31:30.296957+0000","last_active":"2024-05-14T09:32:30.524041+0000","last_peered":"2024-05-14T09:32:30.524041+0000","last_clean":"2024-05-14T09:32:30.524041+0000","last_became_active":"2024-05-14T09:31:30.296554+0000","last_became_peered":"2024-05-14T09:31:30.296554+0000","last_unstale":"2024-05-14T09:32:30.524041+0000","last_undegraded":"2024-05-14T09:32:30.524041+0000","last_fullsized":"2024-05-14T09:32:30.524041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:28.963042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":224,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:34:50.507285+0000","last_change":"2024-05-14T09:30:53.772731+0000","last_active":"2024-05-14T09:34:50.507285+0000","last_peered":"2024-05-14T09:34:50.507285+0000","last_clean":"2024-05-14T09:34:50.507285+0000","last_became_active":"2024-05-14T09:30:53.771986+0000","last_became_peered":"2024-05-14T09:30:53.771986+0000","last_unstale":"2024-05-14T09:34:50.507285+0000","last_undegraded":"2024-05-14T09:34:50.507285+0000","last_fullsized":"2024-05-14T09:34:50.507285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:50:00.810937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077545+0000","last_change":"2024-05-14T09:30:56.790503+0000","last_active":"2024-05-14T09:33:29.077545+0000","last_peered":"2024-05-14T09:33:29.077545+0000","last_clean":"2024-05-14T09:33:29.077545+0000","last_became_active":"2024-05-14T09:30:56.790103+0000","last_became_peered":"2024-05-14T09:30:56.790103+0000","last_unstale":"2024-05-14T09:33:29.077545+0000","last_undegraded":"2024-05-14T09:33:29.077545+0000","last_fullsized":"2024-05-14T09:33:29.077545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:00.950794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315462+0000","last_change":"2024-05-14T09:31:08.116475+0000","last_active":"2024-05-14T09:32:30.315462+0000","last_peered":"2024-05-14T09:32:30.315462+0000","last_clean":"2024-05-14T09:32:30.315462+0000","last_became_active":"2024-05-14T09:31:08.116061+0000","last_became_peered":"2024-05-14T09:31:08.116061+0000","last_unstale":"2024-05-14T09:32:30.315462+0000","last_undegraded":"2024-05-14T09:32:30.315462+0000","last_fullsized":"2024-05-14T09:32:30.315462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:07:02.281853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523311+0000","last_change":"2024-05-14T09:30:54.016570+0000","last_active":"2024-05-14T09:32:30.523311+0000","last_peered":"2024-05-14T09:32:30.523311+0000","last_clean":"2024-05-14T09:32:30.523311+0000","last_became_active":"2024-05-14T09:30:54.015987+0000","last_became_peered":"2024-05-14T09:30:54.015987+0000","last_unstale":"2024-05-14T09:32:30.523311+0000","last_undegraded":"2024-05-14T09:32:30.523311+0000","last_fullsized":"2024-05-14T09:32:30.523311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:18:28.694488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081072+0000","last_change":"2024-05-14T09:30:56.800727+0000","last_active":"2024-05-14T09:33:29.081072+0000","last_peered":"2024-05-14T09:33:29.081072+0000","last_clean":"2024-05-14T09:33:29.081072+0000","last_became_active":"2024-05-14T09:30:56.800354+0000","last_became_peered":"2024-05-14T09:30:56.800354+0000","last_unstale":"2024-05-14T09:33:29.081072+0000","last_undegraded":"2024-05-14T09:33:29.081072+0000","last_fullsized":"2024-05-14T09:33:29.081072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:13:07.227577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315777+0000","last_change":"2024-05-14T09:31:08.115685+0000","last_active":"2024-05-14T09:32:30.315777+0000","last_peered":"2024-05-14T09:32:30.315777+0000","last_clean":"2024-05-14T09:32:30.315777+0000","last_became_active":"2024-05-14T09:31:08.114948+0000","last_became_peered":"2024-05-14T09:31:08.114948+0000","last_unstale":"2024-05-14T09:32:30.315777+0000","last_undegraded":"2024-05-14T09:32:30.315777+0000","last_fullsized":"2024-05-14T09:32:30.315777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:33:07.286823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524227+0000","last_change":"2024-05-14T09:30:53.769949+0000","last_active":"2024-05-14T09:32:30.524227+0000","last_peered":"2024-05-14T09:32:30.524227+0000","last_clean":"2024-05-14T09:32:30.524227+0000","last_became_active":"2024-05-14T09:30:53.769327+0000","last_became_peered":"2024-05-14T09:30:53.769327+0000","last_unstale":"2024-05-14T09:32:30.524227+0000","last_undegraded":"2024-05-14T09:32:30.524227+0000","last_fullsized":"2024-05-14T09:32:30.524227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:35:11.850991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077817+0000","last_change":"2024-05-14T09:32:31.015994+0000","last_active":"2024-05-14T09:33:29.077817+0000","last_peered":"2024-05-14T09:33:29.077817+0000","last_clean":"2024-05-14T09:33:29.077817+0000","last_became_active":"2024-05-14T09:32:31.015515+0000","last_became_peered":"2024-05-14T09:32:31.015515+0000","last_unstale":"2024-05-14T09:33:29.077817+0000","last_undegraded":"2024-05-14T09:33:29.077817+0000","last_fullsized":"2024-05-14T09:33:29.077817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:21:49.811521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7],"acting":[11,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524005+0000","last_change":"2024-05-14T09:31:08.387316+0000","last_active":"2024-05-14T09:32:30.524005+0000","last_peered":"2024-05-14T09:32:30.524005+0000","last_clean":"2024-05-14T09:32:30.524005+0000","last_became_active":"2024-05-14T09:31:08.386739+0000","last_became_peered":"2024-05-14T09:31:08.386739+0000","last_unstale":"2024-05-14T09:32:30.524005+0000","last_undegraded":"2024-05-14T09:32:30.524005+0000","last_fullsized":"2024-05-14T09:32:30.524005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:17.870255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.318704+0000","last_change":"2024-05-14T09:30:54.017756+0000","last_active":"2024-05-14T09:32:30.318704+0000","last_peered":"2024-05-14T09:32:30.318704+0000","last_clean":"2024-05-14T09:32:30.318704+0000","last_became_active":"2024-05-14T09:30:54.017277+0000","last_became_peered":"2024-05-14T09:30:54.017277+0000","last_unstale":"2024-05-14T09:32:30.318704+0000","last_undegraded":"2024-05-14T09:32:30.318704+0000","last_fullsized":"2024-05-14T09:32:30.318704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:27:36.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.072775+0000","last_change":"2024-05-14T09:30:56.788408+0000","last_active":"2024-05-14T09:33:29.072775+0000","last_peered":"2024-05-14T09:33:29.072775+0000","last_clean":"2024-05-14T09:33:29.072775+0000","last_became_active":"2024-05-14T09:30:56.787955+0000","last_became_peered":"2024-05-14T09:30:56.787955+0000","last_unstale":"2024-05-14T09:33:29.072775+0000","last_undegraded":"2024-05-14T09:33:29.072775+0000","last_fullsized":"2024-05-14T09:33:29.072775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:09.777069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315137+0000","last_change":"2024-05-14T09:31:08.119078+0000","last_active":"2024-05-14T09:32:30.315137+0000","last_peered":"2024-05-14T09:32:30.315137+0000","last_clean":"2024-05-14T09:32:30.315137+0000","last_became_active":"2024-05-14T09:31:08.117939+0000","last_became_peered":"2024-05-14T09:31:08.117939+0000","last_unstale":"2024-05-14T09:32:30.315137+0000","last_undegraded":"2024-05-14T09:32:30.315137+0000","last_fullsized":"2024-05-14T09:32:30.315137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:48:37.363503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014384+0000","last_change":"2024-05-14T09:30:53.767843+0000","last_active":"2024-05-14T09:32:31.014384+0000","last_peered":"2024-05-14T09:32:31.014384+0000","last_clean":"2024-05-14T09:32:31.014384+0000","last_became_active":"2024-05-14T09:30:53.767419+0000","last_became_peered":"2024-05-14T09:30:53.767419+0000","last_unstale":"2024-05-14T09:32:31.014384+0000","last_undegraded":"2024-05-14T09:32:31.014384+0000","last_fullsized":"2024-05-14T09:32:31.014384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:49:06.441600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077073+0000","last_change":"2024-05-14T09:30:56.796001+0000","last_active":"2024-05-14T09:33:29.077073+0000","last_peered":"2024-05-14T09:33:29.077073+0000","last_clean":"2024-05-14T09:33:29.077073+0000","last_became_active":"2024-05-14T09:30:56.794517+0000","last_became_peered":"2024-05-14T09:30:56.794517+0000","last_unstale":"2024-05-14T09:33:29.077073+0000","last_undegraded":"2024-05-14T09:33:29.077073+0000","last_fullsized":"2024-05-14T09:33:29.077073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:20:37.514679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315309+0000","last_change":"2024-05-14T09:31:08.387570+0000","last_active":"2024-05-14T09:32:30.315309+0000","last_peered":"2024-05-14T09:32:30.315309+0000","last_clean":"2024-05-14T09:32:30.315309+0000","last_became_active":"2024-05-14T09:31:08.386745+0000","last_became_peered":"2024-05-14T09:31:08.386745+0000","last_unstale":"2024-05-14T09:32:30.315309+0000","last_undegraded":"2024-05-14T09:32:30.315309+0000","last_fullsized":"2024-05-14T09:32:30.315309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:44:01.024784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.060037+0000","last_change":"2024-05-14T09:30:56.795532+0000","last_active":"2024-05-14T09:33:29.060037+0000","last_peered":"2024-05-14T09:33:29.060037+0000","last_clean":"2024-05-14T09:33:29.060037+0000","last_became_active":"2024-05-14T09:30:56.795171+0000","last_became_peered":"2024-05-14T09:30:56.795171+0000","last_unstale":"2024-05-14T09:33:29.060037+0000","last_undegraded":"2024-05-14T09:33:29.060037+0000","last_fullsized":"2024-05-14T09:33:29.060037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:09:40.252489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":224,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:34:51.508100+0000","last_change":"2024-05-14T09:30:54.015629+0000","last_active":"2024-05-14T09:34:51.508100+0000","last_peered":"2024-05-14T09:34:51.508100+0000","last_clean":"2024-05-14T09:34:51.508100+0000","last_became_active":"2024-05-14T09:30:54.015288+0000","last_became_peered":"2024-05-14T09:30:54.015288+0000","last_unstale":"2024-05-14T09:34:51.508100+0000","last_undegraded":"2024-05-14T09:34:51.508100+0000","last_fullsized":"2024-05-14T09:34:51.508100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:50.316125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014777+0000","last_change":"2024-05-14T09:31:30.179056+0000","last_active":"2024-05-14T09:32:31.014777+0000","last_peered":"2024-05-14T09:32:31.014777+0000","last_clean":"2024-05-14T09:32:31.014777+0000","last_became_active":"2024-05-14T09:31:30.178398+0000","last_became_peered":"2024-05-14T09:31:30.178398+0000","last_unstale":"2024-05-14T09:32:31.014777+0000","last_undegraded":"2024-05-14T09:32:31.014777+0000","last_fullsized":"2024-05-14T09:32:31.014777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:15:55.369801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077759+0000","last_change":"2024-05-14T09:30:56.793642+0000","last_active":"2024-05-14T09:33:29.077759+0000","last_peered":"2024-05-14T09:33:29.077759+0000","last_clean":"2024-05-14T09:33:29.077759+0000","last_became_active":"2024-05-14T09:30:56.793229+0000","last_became_peered":"2024-05-14T09:30:56.793229+0000","last_unstale":"2024-05-14T09:33:29.077759+0000","last_undegraded":"2024-05-14T09:33:29.077759+0000","last_fullsized":"2024-05-14T09:33:29.077759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:17.940674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.522946+0000","last_change":"2024-05-14T09:30:53.769582+0000","last_active":"2024-05-14T09:32:30.522946+0000","last_peered":"2024-05-14T09:32:30.522946+0000","last_clean":"2024-05-14T09:32:30.522946+0000","last_became_active":"2024-05-14T09:30:53.769113+0000","last_became_peered":"2024-05-14T09:30:53.769113+0000","last_unstale":"2024-05-14T09:32:30.522946+0000","last_undegraded":"2024-05-14T09:32:30.522946+0000","last_fullsized":"2024-05-14T09:32:30.522946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:50:48.755113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":119,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.014145+0000","last_change":"2024-05-14T09:29:30.767389+0000","last_active":"2024-05-14T09:32:31.014145+0000","last_peered":"2024-05-14T09:32:31.014145+0000","last_clean":"2024-05-14T09:32:31.014145+0000","last_became_active":"2024-05-14T09:29:30.761827+0000","last_became_peered":"2024-05-14T09:29:30.761827+0000","last_unstale":"2024-05-14T09:32:31.014145+0000","last_undegraded":"2024-05-14T09:32:31.014145+0000","last_fullsized":"2024-05-14T09:32:31.014145+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:26:03.741821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:26:03.741821+0000","last_clean_scrub_stamp":"2024-05-14T09:26:03.741821+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:52:11.421658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316444+0000","last_change":"2024-05-14T09:31:08.387819+0000","last_active":"2024-05-14T09:32:30.316444+0000","last_peered":"2024-05-14T09:32:30.316444+0000","last_clean":"2024-05-14T09:32:30.316444+0000","last_became_active":"2024-05-14T09:31:08.387275+0000","last_became_peered":"2024-05-14T09:31:08.387275+0000","last_unstale":"2024-05-14T09:32:30.316444+0000","last_undegraded":"2024-05-14T09:32:30.316444+0000","last_fullsized":"2024-05-14T09:32:30.316444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:46.627389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.078055+0000","last_change":"2024-05-14T09:30:56.788159+0000","last_active":"2024-05-14T09:33:29.078055+0000","last_peered":"2024-05-14T09:33:29.078055+0000","last_clean":"2024-05-14T09:33:29.078055+0000","last_became_active":"2024-05-14T09:30:56.787744+0000","last_became_peered":"2024-05-14T09:30:56.787744+0000","last_unstale":"2024-05-14T09:33:29.078055+0000","last_undegraded":"2024-05-14T09:33:29.078055+0000","last_fullsized":"2024-05-14T09:33:29.078055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:45:25.243471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:52.843555+0000","last_change":"2024-05-14T09:30:53.774345+0000","last_active":"2024-05-14T09:32:52.843555+0000","last_peered":"2024-05-14T09:32:52.843555+0000","last_clean":"2024-05-14T09:32:52.843555+0000","last_became_active":"2024-05-14T09:30:53.773853+0000","last_became_peered":"2024-05-14T09:30:53.773853+0000","last_unstale":"2024-05-14T09:32:52.843555+0000","last_undegraded":"2024-05-14T09:32:52.843555+0000","last_fullsized":"2024-05-14T09:32:52.843555+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:35:31.938442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.320387+0000","last_change":"2024-05-14T09:31:08.122876+0000","last_active":"2024-05-14T09:32:30.320387+0000","last_peered":"2024-05-14T09:32:30.320387+0000","last_clean":"2024-05-14T09:32:30.320387+0000","last_became_active":"2024-05-14T09:31:08.122182+0000","last_became_peered":"2024-05-14T09:31:08.122182+0000","last_unstale":"2024-05-14T09:32:30.320387+0000","last_undegraded":"2024-05-14T09:32:30.320387+0000","last_fullsized":"2024-05-14T09:32:30.320387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:47:30.735057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.524333+0000","last_change":"2024-05-14T09:30:54.018168+0000","last_active":"2024-05-14T09:32:30.524333+0000","last_peered":"2024-05-14T09:32:30.524333+0000","last_clean":"2024-05-14T09:32:30.524333+0000","last_became_active":"2024-05-14T09:30:54.017699+0000","last_became_peered":"2024-05-14T09:30:54.017699+0000","last_unstale":"2024-05-14T09:32:30.524333+0000","last_undegraded":"2024-05-14T09:32:30.524333+0000","last_fullsized":"2024-05-14T09:32:30.524333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:31:01.162000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.070935+0000","last_change":"2024-05-14T09:30:56.787156+0000","last_active":"2024-05-14T09:33:29.070935+0000","last_peered":"2024-05-14T09:33:29.070935+0000","last_clean":"2024-05-14T09:33:29.070935+0000","last_became_active":"2024-05-14T09:30:56.786832+0000","last_became_peered":"2024-05-14T09:30:56.786832+0000","last_unstale":"2024-05-14T09:33:29.070935+0000","last_undegraded":"2024-05-14T09:33:29.070935+0000","last_fullsized":"2024-05-14T09:33:29.070935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:00:41.719641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.321913+0000","last_change":"2024-05-14T09:31:08.387907+0000","last_active":"2024-05-14T09:32:30.321913+0000","last_peered":"2024-05-14T09:32:30.321913+0000","last_clean":"2024-05-14T09:32:30.321913+0000","last_became_active":"2024-05-14T09:31:08.386765+0000","last_became_peered":"2024-05-14T09:31:08.386765+0000","last_unstale":"2024-05-14T09:32:30.321913+0000","last_undegraded":"2024-05-14T09:32:30.321913+0000","last_fullsized":"2024-05-14T09:32:30.321913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:24:56.890353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'13","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.814407+0000","last_change":"2024-05-14T09:30:53.767508+0000","last_active":"2024-05-14T09:33:29.814407+0000","last_peered":"2024-05-14T09:33:29.814407+0000","last_clean":"2024-05-14T09:33:29.814407+0000","last_became_active":"2024-05-14T09:30:53.767169+0000","last_became_peered":"2024-05-14T09:30:53.767169+0000","last_unstale":"2024-05-14T09:33:29.814407+0000","last_undegraded":"2024-05-14T09:33:29.814407+0000","last_fullsized":"2024-05-14T09:33:29.814407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:04:23.621686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076780+0000","last_change":"2024-05-14T09:30:56.790398+0000","last_active":"2024-05-14T09:33:29.076780+0000","last_peered":"2024-05-14T09:33:29.076780+0000","last_clean":"2024-05-14T09:33:29.076780+0000","last_became_active":"2024-05-14T09:30:56.790055+0000","last_became_peered":"2024-05-14T09:30:56.790055+0000","last_unstale":"2024-05-14T09:33:29.076780+0000","last_undegraded":"2024-05-14T09:33:29.076780+0000","last_fullsized":"2024-05-14T09:33:29.076780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:12:37.688647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319239+0000","last_change":"2024-05-14T09:31:08.111533+0000","last_active":"2024-05-14T09:32:30.319239+0000","last_peered":"2024-05-14T09:32:30.319239+0000","last_clean":"2024-05-14T09:32:30.319239+0000","last_became_active":"2024-05-14T09:31:08.110448+0000","last_became_peered":"2024-05-14T09:31:08.110448+0000","last_unstale":"2024-05-14T09:32:30.319239+0000","last_undegraded":"2024-05-14T09:32:30.319239+0000","last_fullsized":"2024-05-14T09:32:30.319239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:43:45.822382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523626+0000","last_change":"2024-05-14T09:30:54.069180+0000","last_active":"2024-05-14T09:32:30.523626+0000","last_peered":"2024-05-14T09:32:30.523626+0000","last_clean":"2024-05-14T09:32:30.523626+0000","last_became_active":"2024-05-14T09:30:54.068846+0000","last_became_peered":"2024-05-14T09:30:54.068846+0000","last_unstale":"2024-05-14T09:32:30.523626+0000","last_undegraded":"2024-05-14T09:32:30.523626+0000","last_fullsized":"2024-05-14T09:32:30.523626+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:22.541720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080401+0000","last_change":"2024-05-14T09:30:56.788159+0000","last_active":"2024-05-14T09:33:29.080401+0000","last_peered":"2024-05-14T09:33:29.080401+0000","last_clean":"2024-05-14T09:33:29.080401+0000","last_became_active":"2024-05-14T09:30:56.787786+0000","last_became_peered":"2024-05-14T09:30:56.787786+0000","last_unstale":"2024-05-14T09:33:29.080401+0000","last_undegraded":"2024-05-14T09:33:29.080401+0000","last_fullsized":"2024-05-14T09:33:29.080401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:24:51.785543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523393+0000","last_change":"2024-05-14T09:31:08.388201+0000","last_active":"2024-05-14T09:32:30.523393+0000","last_peered":"2024-05-14T09:32:30.523393+0000","last_clean":"2024-05-14T09:32:30.523393+0000","last_became_active":"2024-05-14T09:31:08.387572+0000","last_became_peered":"2024-05-14T09:31:08.387572+0000","last_unstale":"2024-05-14T09:32:30.523393+0000","last_undegraded":"2024-05-14T09:32:30.523393+0000","last_fullsized":"2024-05-14T09:32:30.523393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:25:13.171350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316020+0000","last_change":"2024-05-14T09:30:54.016524+0000","last_active":"2024-05-14T09:32:30.316020+0000","last_peered":"2024-05-14T09:32:30.316020+0000","last_clean":"2024-05-14T09:32:30.316020+0000","last_became_active":"2024-05-14T09:30:54.016154+0000","last_became_peered":"2024-05-14T09:30:54.016154+0000","last_unstale":"2024-05-14T09:32:30.316020+0000","last_undegraded":"2024-05-14T09:32:30.316020+0000","last_fullsized":"2024-05-14T09:32:30.316020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:43:16.671053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.035477+0000","last_change":"2024-05-14T09:30:56.796748+0000","last_active":"2024-05-14T09:33:29.035477+0000","last_peered":"2024-05-14T09:33:29.035477+0000","last_clean":"2024-05-14T09:33:29.035477+0000","last_became_active":"2024-05-14T09:30:56.796354+0000","last_became_peered":"2024-05-14T09:30:56.796354+0000","last_unstale":"2024-05-14T09:33:29.035477+0000","last_undegraded":"2024-05-14T09:33:29.035477+0000","last_fullsized":"2024-05-14T09:33:29.035477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:30:15.820677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319112+0000","last_change":"2024-05-14T09:31:08.117266+0000","last_active":"2024-05-14T09:32:30.319112+0000","last_peered":"2024-05-14T09:32:30.319112+0000","last_clean":"2024-05-14T09:32:30.319112+0000","last_became_active":"2024-05-14T09:31:08.115393+0000","last_became_peered":"2024-05-14T09:31:08.115393+0000","last_unstale":"2024-05-14T09:32:30.319112+0000","last_undegraded":"2024-05-14T09:32:30.319112+0000","last_fullsized":"2024-05-14T09:32:30.319112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:29:26.348818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.322687+0000","last_change":"2024-05-14T09:30:53.772417+0000","last_active":"2024-05-14T09:32:30.322687+0000","last_peered":"2024-05-14T09:32:30.322687+0000","last_clean":"2024-05-14T09:32:30.322687+0000","last_became_active":"2024-05-14T09:30:53.772028+0000","last_became_peered":"2024-05-14T09:30:53.772028+0000","last_unstale":"2024-05-14T09:32:30.322687+0000","last_undegraded":"2024-05-14T09:32:30.322687+0000","last_fullsized":"2024-05-14T09:32:30.322687+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:16:16.603957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.077309+0000","last_change":"2024-05-14T09:30:56.804750+0000","last_active":"2024-05-14T09:33:29.077309+0000","last_peered":"2024-05-14T09:33:29.077309+0000","last_clean":"2024-05-14T09:33:29.077309+0000","last_became_active":"2024-05-14T09:30:56.803990+0000","last_became_peered":"2024-05-14T09:30:56.803990+0000","last_unstale":"2024-05-14T09:33:29.077309+0000","last_undegraded":"2024-05-14T09:33:29.077309+0000","last_fullsized":"2024-05-14T09:33:29.077309+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:42:00.661033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523484+0000","last_change":"2024-05-14T09:31:08.116061+0000","last_active":"2024-05-14T09:32:30.523484+0000","last_peered":"2024-05-14T09:32:30.523484+0000","last_clean":"2024-05-14T09:32:30.523484+0000","last_became_active":"2024-05-14T09:31:08.115578+0000","last_became_peered":"2024-05-14T09:31:08.115578+0000","last_unstale":"2024-05-14T09:32:30.523484+0000","last_undegraded":"2024-05-14T09:32:30.523484+0000","last_fullsized":"2024-05-14T09:32:30.523484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:18.931035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:06.782240+0000","last_change":"2024-05-14T09:30:53.768988+0000","last_active":"2024-05-14T09:33:06.782240+0000","last_peered":"2024-05-14T09:33:06.782240+0000","last_clean":"2024-05-14T09:33:06.782240+0000","last_became_active":"2024-05-14T09:30:53.768529+0000","last_became_peered":"2024-05-14T09:30:53.768529+0000","last_unstale":"2024-05-14T09:33:06.782240+0000","last_undegraded":"2024-05-14T09:33:06.782240+0000","last_fullsized":"2024-05-14T09:33:06.782240+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:17:02.381447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.079500+0000","last_change":"2024-05-14T09:30:56.797110+0000","last_active":"2024-05-14T09:33:29.079500+0000","last_peered":"2024-05-14T09:33:29.079500+0000","last_clean":"2024-05-14T09:33:29.079500+0000","last_became_active":"2024-05-14T09:30:56.796015+0000","last_became_peered":"2024-05-14T09:30:56.796015+0000","last_unstale":"2024-05-14T09:33:29.079500+0000","last_undegraded":"2024-05-14T09:33:29.079500+0000","last_fullsized":"2024-05-14T09:33:29.079500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:37:44.674607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523289+0000","last_change":"2024-05-14T09:31:08.387240+0000","last_active":"2024-05-14T09:32:30.523289+0000","last_peered":"2024-05-14T09:32:30.523289+0000","last_clean":"2024-05-14T09:32:30.523289+0000","last_became_active":"2024-05-14T09:31:08.386653+0000","last_became_peered":"2024-05-14T09:31:08.386653+0000","last_unstale":"2024-05-14T09:32:30.523289+0000","last_undegraded":"2024-05-14T09:32:30.523289+0000","last_fullsized":"2024-05-14T09:32:30.523289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:18:43.408030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.523577+0000","last_change":"2024-05-14T09:30:53.772723+0000","last_active":"2024-05-14T09:32:30.523577+0000","last_peered":"2024-05-14T09:32:30.523577+0000","last_clean":"2024-05-14T09:32:30.523577+0000","last_became_active":"2024-05-14T09:30:53.772056+0000","last_became_peered":"2024-05-14T09:30:53.772056+0000","last_unstale":"2024-05-14T09:32:30.523577+0000","last_undegraded":"2024-05-14T09:32:30.523577+0000","last_fullsized":"2024-05-14T09:32:30.523577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:37.941282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087926+0000","last_change":"2024-05-14T09:30:56.799223+0000","last_active":"2024-05-14T09:33:29.087926+0000","last_peered":"2024-05-14T09:33:29.087926+0000","last_clean":"2024-05-14T09:33:29.087926+0000","last_became_active":"2024-05-14T09:30:56.796092+0000","last_became_peered":"2024-05-14T09:30:56.796092+0000","last_unstale":"2024-05-14T09:33:29.087926+0000","last_undegraded":"2024-05-14T09:33:29.087926+0000","last_fullsized":"2024-05-14T09:33:29.087926+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:43:49.746797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.316061+0000","last_change":"2024-05-14T09:31:08.115053+0000","last_active":"2024-05-14T09:32:30.316061+0000","last_peered":"2024-05-14T09:32:30.316061+0000","last_clean":"2024-05-14T09:32:30.316061+0000","last_became_active":"2024-05-14T09:31:08.114578+0000","last_became_peered":"2024-05-14T09:31:08.114578+0000","last_unstale":"2024-05-14T09:32:30.316061+0000","last_undegraded":"2024-05-14T09:32:30.316061+0000","last_fullsized":"2024-05-14T09:32:30.316061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:00:47.788962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:32.003842+0000","last_change":"2024-05-14T09:32:30.326580+0000","last_active":"2024-05-14T09:32:32.003842+0000","last_peered":"2024-05-14T09:32:32.003842+0000","last_clean":"2024-05-14T09:32:32.003842+0000","last_became_active":"2024-05-14T09:32:30.322052+0000","last_became_peered":"2024-05-14T09:32:30.322052+0000","last_unstale":"2024-05-14T09:32:32.003842+0000","last_undegraded":"2024-05-14T09:32:32.003842+0000","last_fullsized":"2024-05-14T09:32:32.003842+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:04.991674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.087235+0000","last_change":"2024-05-14T09:30:56.798580+0000","last_active":"2024-05-14T09:33:29.087235+0000","last_peered":"2024-05-14T09:33:29.087235+0000","last_clean":"2024-05-14T09:33:29.087235+0000","last_became_active":"2024-05-14T09:30:56.795617+0000","last_became_peered":"2024-05-14T09:30:56.795617+0000","last_unstale":"2024-05-14T09:33:29.087235+0000","last_undegraded":"2024-05-14T09:33:29.087235+0000","last_fullsized":"2024-05-14T09:33:29.087235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:30:35.913497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.319424+0000","last_change":"2024-05-14T09:31:08.111375+0000","last_active":"2024-05-14T09:32:30.319424+0000","last_peered":"2024-05-14T09:32:30.319424+0000","last_clean":"2024-05-14T09:32:30.319424+0000","last_became_active":"2024-05-14T09:31:08.110931+0000","last_became_peered":"2024-05-14T09:31:08.110931+0000","last_unstale":"2024-05-14T09:32:30.319424+0000","last_undegraded":"2024-05-14T09:32:30.319424+0000","last_fullsized":"2024-05-14T09:32:30.319424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:08:30.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:31.176828+0000","last_change":"2024-05-14T09:32:30.326979+0000","last_active":"2024-05-14T09:32:31.176828+0000","last_peered":"2024-05-14T09:32:31.176828+0000","last_clean":"2024-05-14T09:32:31.176828+0000","last_became_active":"2024-05-14T09:32:30.323545+0000","last_became_peered":"2024-05-14T09:32:30.323545+0000","last_unstale":"2024-05-14T09:32:31.176828+0000","last_undegraded":"2024-05-14T09:32:31.176828+0000","last_fullsized":"2024-05-14T09:32:31.176828+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:00:21.493910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":70,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.064380+0000","last_change":"2024-05-14T09:30:56.789527+0000","last_active":"2024-05-14T09:33:29.064380+0000","last_peered":"2024-05-14T09:33:29.064380+0000","last_clean":"2024-05-14T09:33:29.064380+0000","last_became_active":"2024-05-14T09:30:56.789075+0000","last_became_peered":"2024-05-14T09:30:56.789075+0000","last_unstale":"2024-05-14T09:33:29.064380+0000","last_undegraded":"2024-05-14T09:33:29.064380+0000","last_fullsized":"2024-05-14T09:33:29.064380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:07.140087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315692+0000","last_change":"2024-05-14T09:31:08.119624+0000","last_active":"2024-05-14T09:32:30.315692+0000","last_peered":"2024-05-14T09:32:30.315692+0000","last_clean":"2024-05-14T09:32:30.315692+0000","last_became_active":"2024-05-14T09:31:08.118102+0000","last_became_peered":"2024-05-14T09:31:08.118102+0000","last_unstale":"2024-05-14T09:32:30.315692+0000","last_undegraded":"2024-05-14T09:32:30.315692+0000","last_fullsized":"2024-05-14T09:32:30.315692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:31:07.091700+0000","last_clean_scrub_stamp":"2024-05-14T09:31:07.091700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:48:17.806976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:32:30.315472+0000","last_change":"2024-05-14T09:30:53.763903+0000","last_active":"2024-05-14T09:32:30.315472+0000","last_peered":"2024-05-14T09:32:30.315472+0000","last_clean":"2024-05-14T09:32:30.315472+0000","last_became_active":"2024-05-14T09:30:53.763517+0000","last_became_peered":"2024-05-14T09:30:53.763517+0000","last_unstale":"2024-05-14T09:32:30.315472+0000","last_undegraded":"2024-05-14T09:32:30.315472+0000","last_fullsized":"2024-05-14T09:32:30.315472+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:52.747609+0000","last_clean_scrub_stamp":"2024-05-14T09:30:52.747609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:57.144392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.081279+0000","last_change":"2024-05-14T09:30:56.791612+0000","last_active":"2024-05-14T09:33:29.081279+0000","last_peered":"2024-05-14T09:33:29.081279+0000","last_clean":"2024-05-14T09:33:29.081279+0000","last_became_active":"2024-05-14T09:30:56.791178+0000","last_became_peered":"2024-05-14T09:30:56.791178+0000","last_unstale":"2024-05-14T09:33:29.081279+0000","last_undegraded":"2024-05-14T09:33:29.081279+0000","last_fullsized":"2024-05-14T09:33:29.081279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:11:19.058139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076827+0000","last_change":"2024-05-14T09:30:56.804437+0000","last_active":"2024-05-14T09:33:29.076827+0000","last_peered":"2024-05-14T09:33:29.076827+0000","last_clean":"2024-05-14T09:33:29.076827+0000","last_became_active":"2024-05-14T09:30:56.804048+0000","last_became_peered":"2024-05-14T09:30:56.804048+0000","last_unstale":"2024-05-14T09:33:29.076827+0000","last_undegraded":"2024-05-14T09:33:29.076827+0000","last_fullsized":"2024-05-14T09:33:29.076827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:33.788903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.058110+0000","last_change":"2024-05-14T09:30:56.794309+0000","last_active":"2024-05-14T09:33:29.058110+0000","last_peered":"2024-05-14T09:33:29.058110+0000","last_clean":"2024-05-14T09:33:29.058110+0000","last_became_active":"2024-05-14T09:30:56.793317+0000","last_became_peered":"2024-05-14T09:30:56.793317+0000","last_unstale":"2024-05-14T09:33:29.058110+0000","last_undegraded":"2024-05-14T09:33:29.058110+0000","last_fullsized":"2024-05-14T09:33:29.058110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:40:26.878746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080059+0000","last_change":"2024-05-14T09:30:56.793618+0000","last_active":"2024-05-14T09:33:29.080059+0000","last_peered":"2024-05-14T09:33:29.080059+0000","last_clean":"2024-05-14T09:33:29.080059+0000","last_became_active":"2024-05-14T09:30:56.792819+0000","last_became_peered":"2024-05-14T09:30:56.792819+0000","last_unstale":"2024-05-14T09:33:29.080059+0000","last_undegraded":"2024-05-14T09:33:29.080059+0000","last_fullsized":"2024-05-14T09:33:29.080059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T10:29:38.191617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.072960+0000","last_change":"2024-05-14T09:30:56.794268+0000","last_active":"2024-05-14T09:33:29.072960+0000","last_peered":"2024-05-14T09:33:29.072960+0000","last_clean":"2024-05-14T09:33:29.072960+0000","last_became_active":"2024-05-14T09:30:56.793898+0000","last_became_peered":"2024-05-14T09:30:56.793898+0000","last_unstale":"2024-05-14T09:33:29.072960+0000","last_undegraded":"2024-05-14T09:33:29.072960+0000","last_fullsized":"2024-05-14T09:33:29.072960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:25:35.022236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.078917+0000","last_change":"2024-05-14T09:30:56.783366+0000","last_active":"2024-05-14T09:33:29.078917+0000","last_peered":"2024-05-14T09:33:29.078917+0000","last_clean":"2024-05-14T09:33:29.078917+0000","last_became_active":"2024-05-14T09:30:56.782980+0000","last_became_peered":"2024-05-14T09:30:56.782980+0000","last_unstale":"2024-05-14T09:33:29.078917+0000","last_undegraded":"2024-05-14T09:33:29.078917+0000","last_fullsized":"2024-05-14T09:33:29.078917+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:27.744902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076941+0000","last_change":"2024-05-14T09:32:30.524390+0000","last_active":"2024-05-14T09:33:29.076941+0000","last_peered":"2024-05-14T09:33:29.076941+0000","last_clean":"2024-05-14T09:33:29.076941+0000","last_became_active":"2024-05-14T09:32:30.523926+0000","last_became_peered":"2024-05-14T09:32:30.523926+0000","last_unstale":"2024-05-14T09:33:29.076941+0000","last_undegraded":"2024-05-14T09:33:29.076941+0000","last_fullsized":"2024-05-14T09:33:29.076941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T09:58:47.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.074545+0000","last_change":"2024-05-14T09:30:56.795872+0000","last_active":"2024-05-14T09:33:29.074545+0000","last_peered":"2024-05-14T09:33:29.074545+0000","last_clean":"2024-05-14T09:33:29.074545+0000","last_became_active":"2024-05-14T09:30:56.795492+0000","last_became_peered":"2024-05-14T09:30:56.795492+0000","last_unstale":"2024-05-14T09:33:29.074545+0000","last_undegraded":"2024-05-14T09:33:29.074545+0000","last_fullsized":"2024-05-14T09:33:29.074545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:26:29.226575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.062863+0000","last_change":"2024-05-14T09:32:30.524189+0000","last_active":"2024-05-14T09:33:29.062863+0000","last_peered":"2024-05-14T09:33:29.062863+0000","last_clean":"2024-05-14T09:33:29.062863+0000","last_became_active":"2024-05-14T09:32:30.523791+0000","last_became_peered":"2024-05-14T09:32:30.523791+0000","last_unstale":"2024-05-14T09:33:29.062863+0000","last_undegraded":"2024-05-14T09:33:29.062863+0000","last_fullsized":"2024-05-14T09:33:29.062863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:52.441130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.080195+0000","last_change":"2024-05-14T09:30:56.792814+0000","last_active":"2024-05-14T09:33:29.080195+0000","last_peered":"2024-05-14T09:33:29.080195+0000","last_clean":"2024-05-14T09:33:29.080195+0000","last_became_active":"2024-05-14T09:30:56.792282+0000","last_became_peered":"2024-05-14T09:30:56.792282+0000","last_unstale":"2024-05-14T09:33:29.080195+0000","last_undegraded":"2024-05-14T09:33:29.080195+0000","last_fullsized":"2024-05-14T09:33:29.080195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:52:35.319479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.076831+0000","last_change":"2024-05-14T09:30:56.791575+0000","last_active":"2024-05-14T09:33:29.076831+0000","last_peered":"2024-05-14T09:33:29.076831+0000","last_clean":"2024-05-14T09:33:29.076831+0000","last_became_active":"2024-05-14T09:30:56.791230+0000","last_became_peered":"2024-05-14T09:30:56.791230+0000","last_unstale":"2024-05-14T09:33:29.076831+0000","last_undegraded":"2024-05-14T09:33:29.076831+0000","last_fullsized":"2024-05-14T09:33:29.076831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:40.510062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.088030+0000","last_change":"2024-05-14T09:30:56.796996+0000","last_active":"2024-05-14T09:33:29.088030+0000","last_peered":"2024-05-14T09:33:29.088030+0000","last_clean":"2024-05-14T09:33:29.088030+0000","last_became_active":"2024-05-14T09:30:56.796333+0000","last_became_peered":"2024-05-14T09:30:56.796333+0000","last_unstale":"2024-05-14T09:33:29.088030+0000","last_undegraded":"2024-05-14T09:33:29.088030+0000","last_fullsized":"2024-05-14T09:33:29.088030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:29:34.799088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.083938+0000","last_change":"2024-05-14T09:30:56.785740+0000","last_active":"2024-05-14T09:33:29.083938+0000","last_peered":"2024-05-14T09:33:29.083938+0000","last_clean":"2024-05-14T09:33:29.083938+0000","last_became_active":"2024-05-14T09:30:56.785394+0000","last_became_peered":"2024-05-14T09:30:56.785394+0000","last_unstale":"2024-05-14T09:33:29.083938+0000","last_undegraded":"2024-05-14T09:33:29.083938+0000","last_fullsized":"2024-05-14T09:33:29.083938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:17.819523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T09:33:29.079641+0000","last_change":"2024-05-14T09:30:56.796957+0000","last_active":"2024-05-14T09:33:29.079641+0000","last_peered":"2024-05-14T09:33:29.079641+0000","last_clean":"2024-05-14T09:33:29.079641+0000","last_became_active":"2024-05-14T09:30:56.795071+0000","last_became_peered":"2024-05-14T09:30:56.795071+0000","last_unstale":"2024-05-14T09:33:29.079641+0000","last_undegraded":"2024-05-14T09:33:29.079641+0000","last_fullsized":"2024-05-14T09:33:29.079641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T09:30:55.762099+0000","last_clean_scrub_stamp":"2024-05-14T09:30:55.762099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:44:31.480207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245291,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69067,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1079,"num_read_kb":613,"num_write":183,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2846,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":207201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678078,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":519087,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874308,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":1174416,"data_compressed":67998,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037834,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":518671,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1597,"internal_metadata":36174275},"hb_peers":[0,1,2,3,4,5,6,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234062,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":518009,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430292,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":1114367,"data_compressed":67998,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593817,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":514429,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":578956,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":515235,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":515365,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149799,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":514041,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953572,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":583114,"data_compressed":32493,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790047,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1110467,"data_compressed":67998,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33883,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33883,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35505,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":65199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T09:34:56.275 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:34:56.560 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:56.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:56 smithi002 ceph-mon[31557]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-14T09:34:56.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:56 smithi002 ceph-mon[31557]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:34:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:56 smithi029 ceph-mon[36249]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-14T09:34:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:56 smithi029 ceph-mon[36249]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:34:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:56 smithi026 ceph-mon[36056]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-14T09:34:56.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:56 smithi026 ceph-mon[36056]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:34:57.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:57 smithi029 ceph-mon[36249]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:34:57.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:57 smithi029 ceph-mon[36249]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:34:57.687 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:57 smithi002 ceph-mon[31557]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:34:57.688 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:57 smithi002 ceph-mon[31557]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:34:57.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:57 smithi026 ceph-mon[36056]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:34:57.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:57 smithi026 ceph-mon[36056]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T09:34:58.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.251+0000 7fe70151b700 1 -- 172.21.15.2:0/1709912180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6fc1034c0 msgr2=0x7fe6fc1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:58.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.251+0000 7fe70151b700 1 --2- 172.21.15.2:0/1709912180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6fc1034c0 0x7fe6fc1038d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0099c0 tx=0x7fe6ec009cd0 comp rx=0 tx=0).stop 2024-05-14T09:34:58.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.251+0000 7fe70151b700 1 -- 172.21.15.2:0/1709912180 shutdown_connections 2024-05-14T09:34:58.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.251+0000 7fe70151b700 1 --2- 172.21.15.2:0/1709912180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6fc108740 0x7fe6fc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:58.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.251+0000 7fe70151b700 1 --2- 172.21.15.2:0/1709912180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6fc1034c0 0x7fe6fc1038d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:58.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.251+0000 7fe70151b700 1 --2- 172.21.15.2:0/1709912180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc107df0 0x7fe6fc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:58.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.251+0000 7fe70151b700 1 -- 172.21.15.2:0/1709912180 >> 172.21.15.2:0/1709912180 conn(0x7fe6fc0fedf0 msgr2=0x7fe6fc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:58.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.251+0000 7fe70151b700 1 -- 172.21.15.2:0/1709912180 shutdown_connections 2024-05-14T09:34:58.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.251+0000 7fe70151b700 1 -- 172.21.15.2:0/1709912180 wait complete. 2024-05-14T09:34:58.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.252+0000 7fe70151b700 1 Processor -- start 2024-05-14T09:34:58.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.252+0000 7fe70151b700 1 -- start start 2024-05-14T09:34:58.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.252+0000 7fe70151b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6fc1034c0 0x7fe6fc19d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:58.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.252+0000 7fe70151b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6fc107df0 0x7fe6fc19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:58.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.252+0000 7fe70151b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc108740 0x7fe6fc1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:58.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.252+0000 7fe70151b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc1133c0 con 0x7fe6fc107df0 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.252+0000 7fe70151b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc113240 con 0x7fe6fc108740 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.252+0000 7fe70151b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc113540 con 0x7fe6fc1034c0 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.252+0000 7fe6fa7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6fc107df0 0x7fe6fc19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.253+0000 7fe6fb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc108740 0x7fe6fc1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.253+0000 7fe6faffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6fc1034c0 0x7fe6fc19d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.253+0000 7fe6fb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc108740 0x7fe6fc1a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:48682/0 (socket says 172.21.15.2:48682) 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.253+0000 7fe6fb7fe700 1 -- 172.21.15.2:0/2395812516 learned_addr learned my addr 172.21.15.2:0/2395812516 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.253+0000 7fe6fa7fc700 1 --2- 172.21.15.2:0/2395812516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6fc107df0 0x7fe6fc19d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42410/0 (socket says 172.21.15.2:42410) 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.253+0000 7fe6faffd700 1 -- 172.21.15.2:0/2395812516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc108740 msgr2=0x7fe6fc1a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.253+0000 7fe6faffd700 1 --2- 172.21.15.2:0/2395812516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc108740 0x7fe6fc1a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.253+0000 7fe6faffd700 1 -- 172.21.15.2:0/2395812516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6fc107df0 msgr2=0x7fe6fc19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.253+0000 7fe6faffd700 1 --2- 172.21.15.2:0/2395812516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6fc107df0 0x7fe6fc19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.253+0000 7fe6faffd700 1 -- 172.21.15.2:0/2395812516 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc1a1fc0 con 0x7fe6fc1034c0 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.253+0000 7fe6fb7fe700 1 --2- 172.21.15.2:0/2395812516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc108740 0x7fe6fc1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.253+0000 7fe6faffd700 1 --2- 172.21.15.2:0/2395812516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6fc1034c0 0x7fe6fc19d0f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0098e0 tx=0x7fe6ec0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:58.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.253+0000 7fe6e3fff700 1 -- 172.21.15.2:0/2395812516 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec009e60 con 0x7fe6fc1034c0 2024-05-14T09:34:58.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.254+0000 7fe70151b700 1 -- 172.21.15.2:0/2395812516 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6fc1a2250 con 0x7fe6fc1034c0 2024-05-14T09:34:58.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.254+0000 7fe6e3fff700 1 -- 172.21.15.2:0/2395812516 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6ec005030 con 0x7fe6fc1034c0 2024-05-14T09:34:58.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.254+0000 7fe6e3fff700 1 -- 172.21.15.2:0/2395812516 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec0049f0 con 0x7fe6fc1034c0 2024-05-14T09:34:58.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.254+0000 7fe70151b700 1 -- 172.21.15.2:0/2395812516 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6fc1a2720 con 0x7fe6fc1034c0 2024-05-14T09:34:58.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.255+0000 7fe70151b700 1 -- 172.21.15.2:0/2395812516 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6fc0512c0 con 0x7fe6fc1034c0 2024-05-14T09:34:58.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.257+0000 7fe6e3fff700 1 -- 172.21.15.2:0/2395812516 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe6ec028060 con 0x7fe6fc1034c0 2024-05-14T09:34:58.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.257+0000 7fe6e3fff700 1 --2- 172.21.15.2:0/2395812516 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe6e40b2880 0x7fe6e40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:34:58.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.258+0000 7fe6e3fff700 1 -- 172.21.15.2:0/2395812516 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fe6ec0e2e00 con 0x7fe6fc1034c0 2024-05-14T09:34:58.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.258+0000 7fe6fa7fc700 1 --2- 172.21.15.2:0/2395812516 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe6e40b2880 0x7fe6e40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:34:58.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.258+0000 7fe6fa7fc700 1 --2- 172.21.15.2:0/2395812516 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe6e40b2880 0x7fe6e40b4d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009550 tx=0x7fe6f00091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:34:58.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.261+0000 7fe6e3fff700 1 -- 172.21.15.2:0/2395812516 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe6ec013070 con 0x7fe6fc1034c0 2024-05-14T09:34:58.527 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:58 smithi002 ceph-mon[31557]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-14T09:34:58.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.527+0000 7fe70151b700 1 -- 172.21.15.2:0/2395812516 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6fc02e5f0 con 0x7fe6fc1034c0 2024-05-14T09:34:58.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.528+0000 7fe6e3fff700 1 -- 172.21.15.2:0/2395812516 <== mon.1 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11843 (secure 0 0 0) 0x7fe6ec0aa790 con 0x7fe6fc1034c0 2024-05-14T09:34:58.528 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:34:58.528 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T09:34:49.632659+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24487,24497,24524]},"id":1}]} 2024-05-14T09:34:58.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.530+0000 7fe70151b700 1 -- 172.21.15.2:0/2395812516 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe6e40b2880 msgr2=0x7fe6e40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:58.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.530+0000 7fe70151b700 1 --2- 172.21.15.2:0/2395812516 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe6e40b2880 0x7fe6e40b4d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009550 tx=0x7fe6f00091b0 comp rx=0 tx=0).stop 2024-05-14T09:34:58.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.530+0000 7fe70151b700 1 -- 172.21.15.2:0/2395812516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6fc1034c0 msgr2=0x7fe6fc19d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:34:58.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.530+0000 7fe70151b700 1 --2- 172.21.15.2:0/2395812516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6fc1034c0 0x7fe6fc19d0f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0098e0 tx=0x7fe6ec0045e0 comp rx=0 tx=0).stop 2024-05-14T09:34:58.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.531+0000 7fe70151b700 1 -- 172.21.15.2:0/2395812516 shutdown_connections 2024-05-14T09:34:58.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.531+0000 7fe70151b700 1 --2- 172.21.15.2:0/2395812516 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe6e40b2880 0x7fe6e40b4d90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:58.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.531+0000 7fe70151b700 1 --2- 172.21.15.2:0/2395812516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc108740 0x7fe6fc1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:58.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.531+0000 7fe70151b700 1 --2- 172.21.15.2:0/2395812516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6fc1034c0 0x7fe6fc19d0f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:58.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.531+0000 7fe70151b700 1 --2- 172.21.15.2:0/2395812516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6fc107df0 0x7fe6fc19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:34:58.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.531+0000 7fe70151b700 1 -- 172.21.15.2:0/2395812516 >> 172.21.15.2:0/2395812516 conn(0x7fe6fc0fedf0 msgr2=0x7fe6fc109700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:34:58.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.531+0000 7fe70151b700 1 -- 172.21.15.2:0/2395812516 shutdown_connections 2024-05-14T09:34:58.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:34:58.531+0000 7fe70151b700 1 -- 172.21.15.2:0/2395812516 wait complete. 2024-05-14T09:34:58.532 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 27 2024-05-14T09:34:58.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:58 smithi026 ceph-mon[36056]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-14T09:34:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:58 smithi029 ceph-mon[36249]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-14T09:34:59.300 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T09:34:59.544 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:34:59 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2395812516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:34:59.559 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:34:59.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:34:59 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2395812516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:34:59.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:34:59 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2395812516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:35:00.646 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:00 smithi002 ceph-mon[31557]: pgmap v425: 129 pgs: 129 active+clean; 648 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-14T09:35:00.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:00 smithi026 ceph-mon[36056]: pgmap v425: 129 pgs: 129 active+clean; 648 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-14T09:35:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:00 smithi029 ceph-mon[36249]: pgmap v425: 129 pgs: 129 active+clean; 648 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-14T09:35:01.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.195+0000 7f298b7eb700 1 -- 172.21.15.2:0/454832132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2984110c30 msgr2=0x7f2984113050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:01.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.195+0000 7f298b7eb700 1 --2- 172.21.15.2:0/454832132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2984110c30 0x7f2984113050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f297c009990 tx=0x7f297c009ca0 comp rx=0 tx=0).stop 2024-05-14T09:35:01.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.195+0000 7f298b7eb700 1 -- 172.21.15.2:0/454832132 shutdown_connections 2024-05-14T09:35:01.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.195+0000 7f298b7eb700 1 --2- 172.21.15.2:0/454832132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2984103080 0x7f2984103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:01.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.195+0000 7f298b7eb700 1 --2- 172.21.15.2:0/454832132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2984110c30 0x7f2984113050 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:01.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.195+0000 7f298b7eb700 1 --2- 172.21.15.2:0/454832132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2984103a60 0x7f29841106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:01.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.195+0000 7f298b7eb700 1 -- 172.21.15.2:0/454832132 >> 172.21.15.2:0/454832132 conn(0x7f29840fedc0 msgr2=0x7f29841011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:01.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.195+0000 7f298b7eb700 1 -- 172.21.15.2:0/454832132 shutdown_connections 2024-05-14T09:35:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.195+0000 7f298b7eb700 1 -- 172.21.15.2:0/454832132 wait complete. 2024-05-14T09:35:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.196+0000 7f298b7eb700 1 Processor -- start 2024-05-14T09:35:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.196+0000 7f298b7eb700 1 -- start start 2024-05-14T09:35:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.196+0000 7f298b7eb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2984103080 0x7f298419d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.196+0000 7f298b7eb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2984103a60 0x7f298419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.196+0000 7f298b7eb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2984110c30 0x7f29841a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.196+0000 7f298b7eb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2984116690 con 0x7f2984110c30 2024-05-14T09:35:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.196+0000 7f298b7eb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2984116510 con 0x7f2984103a60 2024-05-14T09:35:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.196+0000 7f298b7eb700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2984116810 con 0x7f2984103080 2024-05-14T09:35:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.196+0000 7f2989d88700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2984110c30 0x7f29841a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.196+0000 7f2989d88700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2984110c30 0x7f29841a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42422/0 (socket says 172.21.15.2:42422) 2024-05-14T09:35:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.196+0000 7f2989d88700 1 -- 172.21.15.2:0/2977255472 learned_addr learned my addr 172.21.15.2:0/2977255472 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.196+0000 7f2989587700 1 --2- 172.21.15.2:0/2977255472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2984103080 0x7f298419d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:01.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.196+0000 7f2988d86700 1 --2- 172.21.15.2:0/2977255472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2984103a60 0x7f298419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.197+0000 7f2989d88700 1 -- 172.21.15.2:0/2977255472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2984103080 msgr2=0x7f298419d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.197+0000 7f2989d88700 1 --2- 172.21.15.2:0/2977255472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2984103080 0x7f298419d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.197+0000 7f2989d88700 1 -- 172.21.15.2:0/2977255472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2984103a60 msgr2=0x7f298419d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.197+0000 7f2989d88700 1 --2- 172.21.15.2:0/2977255472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2984103a60 0x7f298419d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.197+0000 7f2989d88700 1 -- 172.21.15.2:0/2977255472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29841a1ef0 con 0x7f2984110c30 2024-05-14T09:35:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.197+0000 7f2989d88700 1 --2- 172.21.15.2:0/2977255472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2984110c30 0x7f29841a1850 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f297c00c010 tx=0x7f297c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.197+0000 7f297a7fc700 1 -- 172.21.15.2:0/2977255472 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c027070 con 0x7f2984110c30 2024-05-14T09:35:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.197+0000 7f297a7fc700 1 -- 172.21.15.2:0/2977255472 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f297c022470 con 0x7f2984110c30 2024-05-14T09:35:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.197+0000 7f297a7fc700 1 -- 172.21.15.2:0/2977255472 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c017650 con 0x7f2984110c30 2024-05-14T09:35:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.197+0000 7f298b7eb700 1 -- 172.21.15.2:0/2977255472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29841a2180 con 0x7f2984110c30 2024-05-14T09:35:01.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.197+0000 7f298b7eb700 1 -- 172.21.15.2:0/2977255472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29841a2660 con 0x7f2984110c30 2024-05-14T09:35:01.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.198+0000 7f298b7eb700 1 -- 172.21.15.2:0/2977255472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29840512c0 con 0x7f2984110c30 2024-05-14T09:35:01.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.200+0000 7f297a7fc700 1 -- 172.21.15.2:0/2977255472 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f297c02a070 con 0x7f2984110c30 2024-05-14T09:35:01.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.200+0000 7f297a7fc700 1 --2- 172.21.15.2:0/2977255472 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f29700b2880 0x7f29700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:01.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.201+0000 7f2989587700 1 --2- 172.21.15.2:0/2977255472 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f29700b2880 0x7f29700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:01.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.201+0000 7f297a7fc700 1 -- 172.21.15.2:0/2977255472 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f297c0e3370 con 0x7f2984110c30 2024-05-14T09:35:01.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.201+0000 7f2989587700 1 --2- 172.21.15.2:0/2977255472 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f29700b2880 0x7f29700b4d90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2980009550 tx=0x7f29800091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:01.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.206+0000 7f297a7fc700 1 -- 172.21.15.2:0/2977255472 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f297c013070 con 0x7f2984110c30 2024-05-14T09:35:01.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.422+0000 7f298b7eb700 1 -- 172.21.15.2:0/2977255472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29841a9b00 con 0x7f2984110c30 2024-05-14T09:35:01.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.424+0000 7f297a7fc700 1 -- 172.21.15.2:0/2977255472 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24740 (secure 0 0 0) 0x7f297c0aad00 con 0x7f2984110c30 2024-05-14T09:35:01.424 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:35:01.424 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":90,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","created":"2024-05-14T09:22:34.937047+0000","modified":"2024-05-14T09:32:30.306958+0000","last_up_change":"2024-05-14T09:29:55.157569+0000","last_in_change":"2024-05-14T09:29:34.922562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T09:26:03.046454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T09:30:51.814350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T09:30:55.500000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T09:31:06.099087+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10d34289-414e-4b1b-bc7c-ba1dd24c7297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6803","nonce":2622184634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6805","nonce":2622184634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6809","nonce":2622184634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6807","nonce":2622184634}]},"public_addr":"172.21.15.2:6803/2622184634","cluster_addr":"172.21.15.2:6805/2622184634","heartbeat_back_addr":"172.21.15.2:6809/2622184634","heartbeat_front_addr":"172.21.15.2:6807/2622184634","state":["exists","up"]},{"osd":1,"uuid":"bc4aefc2-706e-4faf-9042-64b87ad097b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6801","nonce":3084747226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6803","nonce":3084747226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6807","nonce":3084747226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6805","nonce":3084747226}]},"public_addr":"172.21.15.26:6801/3084747226","cluster_addr":"172.21.15.26:6803/3084747226","heartbeat_back_addr":"172.21.15.26:6807/3084747226","heartbeat_front_addr":"172.21.15.26:6805/3084747226","state":["exists","up"]},{"osd":2,"uuid":"8c5bb589-56e2-469f-9a6f-6035b3ddf4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6801","nonce":814048086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6803","nonce":814048086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6807","nonce":814048086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6805","nonce":814048086}]},"public_addr":"172.21.15.29:6801/814048086","cluster_addr":"172.21.15.29:6803/814048086","heartbeat_back_addr":"172.21.15.29:6807/814048086","heartbeat_front_addr":"172.21.15.29:6805/814048086","state":["exists","up"]},{"osd":3,"uuid":"8fbfc078-85e7-4dc2-acda-5e7b0ab927e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6811","nonce":879639451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6813","nonce":879639451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6817","nonce":879639451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6815","nonce":879639451}]},"public_addr":"172.21.15.2:6811/879639451","cluster_addr":"172.21.15.2:6813/879639451","heartbeat_back_addr":"172.21.15.2:6817/879639451","heartbeat_front_addr":"172.21.15.2:6815/879639451","state":["exists","up"]},{"osd":4,"uuid":"f2515448-e9c1-48a2-b270-1e5e781a2fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6809","nonce":457100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6811","nonce":457100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6815","nonce":457100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6813","nonce":457100752}]},"public_addr":"172.21.15.26:6809/457100752","cluster_addr":"172.21.15.26:6811/457100752","heartbeat_back_addr":"172.21.15.26:6815/457100752","heartbeat_front_addr":"172.21.15.26:6813/457100752","state":["exists","up"]},{"osd":5,"uuid":"20030857-a1e6-4b0a-b186-1cfa44f2e4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6809","nonce":1554045093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6811","nonce":1554045093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6815","nonce":1554045093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6813","nonce":1554045093}]},"public_addr":"172.21.15.29:6809/1554045093","cluster_addr":"172.21.15.29:6811/1554045093","heartbeat_back_addr":"172.21.15.29:6815/1554045093","heartbeat_front_addr":"172.21.15.29:6813/1554045093","state":["exists","up"]},{"osd":6,"uuid":"d757c72a-29f0-4f14-9f13-a2e61eb5e6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6819","nonce":2749897595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6821","nonce":2749897595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6825","nonce":2749897595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6823","nonce":2749897595}]},"public_addr":"172.21.15.2:6819/2749897595","cluster_addr":"172.21.15.2:6821/2749897595","heartbeat_back_addr":"172.21.15.2:6825/2749897595","heartbeat_front_addr":"172.21.15.2:6823/2749897595","state":["exists","up"]},{"osd":7,"uuid":"22ddef80-3633-44c5-bc0e-edc75b546ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6817","nonce":1180401611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6819","nonce":1180401611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6823","nonce":1180401611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6821","nonce":1180401611}]},"public_addr":"172.21.15.26:6817/1180401611","cluster_addr":"172.21.15.26:6819/1180401611","heartbeat_back_addr":"172.21.15.26:6823/1180401611","heartbeat_front_addr":"172.21.15.26:6821/1180401611","state":["exists","up"]},{"osd":8,"uuid":"466e7ac4-3ff7-4254-a3fd-8096420b1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6817","nonce":2858408446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6819","nonce":2858408446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6823","nonce":2858408446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6821","nonce":2858408446}]},"public_addr":"172.21.15.29:6817/2858408446","cluster_addr":"172.21.15.29:6819/2858408446","heartbeat_back_addr":"172.21.15.29:6823/2858408446","heartbeat_front_addr":"172.21.15.29:6821/2858408446","state":["exists","up"]},{"osd":9,"uuid":"119eeac9-29f7-4086-a1ad-04a0bc154b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6827","nonce":176898226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6829","nonce":176898226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6833","nonce":176898226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6831","nonce":176898226}]},"public_addr":"172.21.15.2:6827/176898226","cluster_addr":"172.21.15.2:6829/176898226","heartbeat_back_addr":"172.21.15.2:6833/176898226","heartbeat_front_addr":"172.21.15.2:6831/176898226","state":["exists","up"]},{"osd":10,"uuid":"55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6825","nonce":4065159515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6827","nonce":4065159515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6831","nonce":4065159515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6829","nonce":4065159515}]},"public_addr":"172.21.15.26:6825/4065159515","cluster_addr":"172.21.15.26:6827/4065159515","heartbeat_back_addr":"172.21.15.26:6831/4065159515","heartbeat_front_addr":"172.21.15.26:6829/4065159515","state":["exists","up"]},{"osd":11,"uuid":"c5246763-c4f6-4ae3-8a93-5929c541ac5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6825","nonce":378800417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6827","nonce":378800417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6831","nonce":378800417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6829","nonce":378800417}]},"public_addr":"172.21.15.29:6825/378800417","cluster_addr":"172.21.15.29:6827/378800417","heartbeat_back_addr":"172.21.15.29:6831/378800417","heartbeat_front_addr":"172.21.15.29:6829/378800417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:09.342387+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:34.156359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:58.860653+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:24.009323+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:49.057146+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:15.007172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:41.085748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:06.674923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:33.409128+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:00.106186+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:26.803487+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:53.668283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11},{"from":8,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9},{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4051922409":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/2353790181":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/4268171474":"2024-05-15T09:23:28.940670+0000","172.21.15.2:6801/1423742578":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/969850441":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3252991268":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6801/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3789067476":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/1423742578":"2024-05-15T09:23:28.940670+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T09:35:01.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.427+0000 7f298b7eb700 1 -- 172.21.15.2:0/2977255472 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f29700b2880 msgr2=0x7f29700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:01.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.427+0000 7f298b7eb700 1 --2- 172.21.15.2:0/2977255472 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f29700b2880 0x7f29700b4d90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2980009550 tx=0x7f29800091b0 comp rx=0 tx=0).stop 2024-05-14T09:35:01.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.427+0000 7f298b7eb700 1 -- 172.21.15.2:0/2977255472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2984110c30 msgr2=0x7f29841a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:01.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.427+0000 7f298b7eb700 1 --2- 172.21.15.2:0/2977255472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2984110c30 0x7f29841a1850 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f297c00c010 tx=0x7f297c0047b0 comp rx=0 tx=0).stop 2024-05-14T09:35:01.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.427+0000 7f298b7eb700 1 -- 172.21.15.2:0/2977255472 shutdown_connections 2024-05-14T09:35:01.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.427+0000 7f298b7eb700 1 --2- 172.21.15.2:0/2977255472 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f29700b2880 0x7f29700b4d90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:01.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.427+0000 7f298b7eb700 1 --2- 172.21.15.2:0/2977255472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2984110c30 0x7f29841a1850 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:01.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.427+0000 7f298b7eb700 1 --2- 172.21.15.2:0/2977255472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2984103080 0x7f298419d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:01.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.427+0000 7f298b7eb700 1 --2- 172.21.15.2:0/2977255472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2984103a60 0x7f298419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:01.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.427+0000 7f298b7eb700 1 -- 172.21.15.2:0/2977255472 >> 172.21.15.2:0/2977255472 conn(0x7f29840fedc0 msgr2=0x7f29841124a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:01.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.427+0000 7f298b7eb700 1 -- 172.21.15.2:0/2977255472 shutdown_connections 2024-05-14T09:35:01.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:01.427+0000 7f298b7eb700 1 -- 172.21.15.2:0/2977255472 wait complete. 2024-05-14T09:35:01.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:01 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2977255472' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:35:01.850 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:01 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2977255472' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:35:01.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:01 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2977255472' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T09:35:02.175 DEBUG:tasks.fwd_scrub:Yielding 2024-05-14T09:35:02.175 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-14T09:35:02.186 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T09:35:02.186 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-14T09:35:02.187 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-14T09:35:02.187 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T09:35:02.187 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T09:35:02.189 INFO:teuthology.run_tasks:Running task workunit... 2024-05-14T09:35:02.202 INFO:tasks.workunit:Pulling workunits from ref e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T09:35:02.203 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-14T09:35:02.203 INFO:tasks.workunit:timeout=3h 2024-05-14T09:35:02.203 INFO:tasks.workunit:cleanup=True 2024-05-14T09:35:02.203 DEBUG:teuthology.orchestra.run.smithi002:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-14T09:35:02.221 INFO:teuthology.orchestra.run.smithi002.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-14T09:35:02.221 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-14T09:35:02.221 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6bh/107d Inode: 1099511628283 Links: 2 2024-05-14T09:35:02.221 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T09:35:02.221 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-14T09:35:02.221 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-05-14 09:32:13.005049906 +0000 2024-05-14T09:35:02.221 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-05-14 09:32:13.005049906 +0000 2024-05-14T09:35:02.221 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-05-14 09:34:48.907844383 +0000 2024-05-14T09:35:02.222 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-05-14 09:32:13.005049906 +0000 2024-05-14T09:35:02.222 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-14T09:35:02.222 DEBUG:teuthology.orchestra.run.smithi002:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-14T09:35:02.296 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T09:35:02.353 INFO:tasks.workunit.client.0.smithi002.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-14T09:35:02.454 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:35:02.749 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:02 smithi002 ceph-mon[31557]: pgmap v426: 129 pgs: 129 active+clean; 648 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-14T09:35:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:02 smithi029 ceph-mon[36249]: pgmap v426: 129 pgs: 129 active+clean; 648 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-14T09:35:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:02 smithi026 ceph-mon[36056]: pgmap v426: 129 pgs: 129 active+clean; 648 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-14T09:35:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.188+0000 7fcd0c699700 1 -- 172.21.15.2:0/708953392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd041034c0 msgr2=0x7fcd041038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.188+0000 7fcd0c699700 1 --2- 172.21.15.2:0/708953392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd041034c0 0x7fcd041038d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000b0d0 tx=0x7fcd0000b3e0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.188+0000 7fcd0c699700 1 -- 172.21.15.2:0/708953392 shutdown_connections 2024-05-14T09:35:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.188+0000 7fcd0c699700 1 --2- 172.21.15.2:0/708953392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04108740 0x7fcd041100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.188+0000 7fcd0c699700 1 --2- 172.21.15.2:0/708953392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd041034c0 0x7fcd041038d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.188+0000 7fcd0c699700 1 --2- 172.21.15.2:0/708953392 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd04107df0 0x7fcd04108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.188+0000 7fcd0c699700 1 -- 172.21.15.2:0/708953392 >> 172.21.15.2:0/708953392 conn(0x7fcd040fedf0 msgr2=0x7fcd04101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:04.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.188+0000 7fcd0c699700 1 -- 172.21.15.2:0/708953392 shutdown_connections 2024-05-14T09:35:04.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.188+0000 7fcd0c699700 1 -- 172.21.15.2:0/708953392 wait complete. 2024-05-14T09:35:04.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.189+0000 7fcd0c699700 1 Processor -- start 2024-05-14T09:35:04.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.189+0000 7fcd0c699700 1 -- start start 2024-05-14T09:35:04.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.189+0000 7fcd0c699700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd041034c0 0x7fcd040ffd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.189+0000 7fcd0c699700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd04107df0 0x7fcd04100240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.189+0000 7fcd0c699700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd04108740 0x7fcd04100780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.189+0000 7fcd0c699700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcd04113370 con 0x7fcd04107df0 2024-05-14T09:35:04.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.189+0000 7fcd0c699700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcd041131f0 con 0x7fcd041034c0 2024-05-14T09:35:04.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.189+0000 7fcd0c699700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcd041134f0 con 0x7fcd04108740 2024-05-14T09:35:04.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.189+0000 7fcd09c34700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd04107df0 0x7fcd04100240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.189+0000 7fcd09c34700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd04107df0 0x7fcd04100240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42434/0 (socket says 172.21.15.2:42434) 2024-05-14T09:35:04.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.189+0000 7fcd09c34700 1 -- 172.21.15.2:0/1395412896 learned_addr learned my addr 172.21.15.2:0/1395412896 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:04.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.189+0000 7fcd0a435700 1 --2- 172.21.15.2:0/1395412896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd041034c0 0x7fcd040ffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.189+0000 7fcd0ac36700 1 --2- 172.21.15.2:0/1395412896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd04108740 0x7fcd04100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.190+0000 7fcd09c34700 1 -- 172.21.15.2:0/1395412896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd04108740 msgr2=0x7fcd04100780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:04.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.190+0000 7fcd09c34700 1 --2- 172.21.15.2:0/1395412896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd04108740 0x7fcd04100780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.190+0000 7fcd09c34700 1 -- 172.21.15.2:0/1395412896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd041034c0 msgr2=0x7fcd040ffd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:04.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.190+0000 7fcd09c34700 1 --2- 172.21.15.2:0/1395412896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd041034c0 0x7fcd040ffd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.190+0000 7fcd09c34700 1 -- 172.21.15.2:0/1395412896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd041a9130 con 0x7fcd04107df0 2024-05-14T09:35:04.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.190+0000 7fcd0ac36700 1 --2- 172.21.15.2:0/1395412896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd04108740 0x7fcd04100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:04.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.190+0000 7fcd09c34700 1 --2- 172.21.15.2:0/1395412896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd04107df0 0x7fcd04100240 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fccf400d950 tx=0x7fccf400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:04.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.190+0000 7fccfb7fe700 1 -- 172.21.15.2:0/1395412896 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4016070 con 0x7fcd04107df0 2024-05-14T09:35:04.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.190+0000 7fccfb7fe700 1 -- 172.21.15.2:0/1395412896 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccf4004830 con 0x7fcd04107df0 2024-05-14T09:35:04.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.190+0000 7fcd0c699700 1 -- 172.21.15.2:0/1395412896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd041a9420 con 0x7fcd04107df0 2024-05-14T09:35:04.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.190+0000 7fccfb7fe700 1 -- 172.21.15.2:0/1395412896 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4005440 con 0x7fcd04107df0 2024-05-14T09:35:04.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.190+0000 7fcd0c699700 1 -- 172.21.15.2:0/1395412896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd041a9960 con 0x7fcd04107df0 2024-05-14T09:35:04.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.193+0000 7fccfb7fe700 1 -- 172.21.15.2:0/1395412896 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fccf4004220 con 0x7fcd04107df0 2024-05-14T09:35:04.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.194+0000 7fccfb7fe700 1 --2- 172.21.15.2:0/1395412896 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fccf00b2880 0x7fccf00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.194+0000 7fcd0a435700 1 --2- 172.21.15.2:0/1395412896 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fccf00b2880 0x7fccf00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.194+0000 7fccfb7fe700 1 -- 172.21.15.2:0/1395412896 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fccf40e2400 con 0x7fcd04107df0 2024-05-14T09:35:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.195+0000 7fcd1b0adb80 1 Processor -- start 2024-05-14T09:35:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.195+0000 7fcd0a435700 1 --2- 172.21.15.2:0/1395412896 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fccf00b2880 0x7fccf00b4d90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009900 tx=0x7fcd0000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.195+0000 7fcd1b0adb80 1 -- start start 2024-05-14T09:35:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.195+0000 7fcd1b0adb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d9390bec0 0x555d93a8f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.195+0000 7fcd1b0adb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d93a8fe00 0x555d93a89950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.195+0000 7fcd1b0adb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d93a89e90 0x555d93a8a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.195+0000 7fcd1b0adb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555d939ab9c0 con 0x555d9390bec0 2024-05-14T09:35:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.195+0000 7fcd1b0adb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555d93910e20 con 0x555d93a89e90 2024-05-14T09:35:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.195+0000 7fcd1b0adb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555d939a7980 con 0x555d93a8fe00 2024-05-14T09:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.195+0000 7fcd0a435700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d9390bec0 0x555d93a8f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.195+0000 7fcd0a435700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d9390bec0 0x555d93a8f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42436/0 (socket says 172.21.15.2:42436) 2024-05-14T09:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.195+0000 7fcd0a435700 1 -- 172.21.15.2:0/229119413 learned_addr learned my addr 172.21.15.2:0/229119413 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.196+0000 7fcd09c34700 1 --2- 172.21.15.2:0/229119413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d93a8fe00 0x555d93a89950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.196+0000 7fcd0ac36700 1 --2- 172.21.15.2:0/229119413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d93a89e90 0x555d93a8a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.196+0000 7fcd0a435700 1 -- 172.21.15.2:0/229119413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d93a8fe00 msgr2=0x555d93a89950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.196+0000 7fcd0a435700 1 --2- 172.21.15.2:0/229119413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d93a8fe00 0x555d93a89950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.196+0000 7fcd0a435700 1 -- 172.21.15.2:0/229119413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d93a89e90 msgr2=0x555d93a8a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.196+0000 7fcd0a435700 1 --2- 172.21.15.2:0/229119413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d93a89e90 0x555d93a8a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.196+0000 7fcd0a435700 1 -- 172.21.15.2:0/229119413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d938f8950 con 0x555d9390bec0 2024-05-14T09:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.196+0000 7fcd0a435700 1 --2- 172.21.15.2:0/229119413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d9390bec0 0x555d93a8f8c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000d010 tx=0x7fcd0001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d85f9d8b7a8731 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.196+0000 7fccf8ff9700 1 -- 172.21.15.2:0/229119413 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0002c030 con 0x555d9390bec0 2024-05-14T09:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.196+0000 7fccf8ff9700 1 -- 172.21.15.2:0/229119413 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd00007a70 con 0x555d9390bec0 2024-05-14T09:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.196+0000 7fccf8ff9700 1 -- 172.21.15.2:0/229119413 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00005640 con 0x555d9390bec0 2024-05-14T09:35:04.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.197+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/229119413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d9390bec0 msgr2=0x555d93a8f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:04.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.197+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/229119413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d9390bec0 0x555d93a8f8c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000d010 tx=0x7fcd0001d000 comp rx=0 tx=0).stop 2024-05-14T09:35:04.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.198+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/229119413 shutdown_connections 2024-05-14T09:35:04.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.198+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/229119413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d93a89e90 0x555d93a8a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.198+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/229119413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d9390bec0 0x555d93a8f8c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.198+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/229119413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d93a8fe00 0x555d93a89950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.198+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/229119413 >> 172.21.15.2:0/229119413 conn(0x555d939536b0 msgr2=0x555d93a84a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:04.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.198+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/229119413 shutdown_connections 2024-05-14T09:35:04.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.198+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/229119413 wait complete. 2024-05-14T09:35:04.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.198+0000 7fcd1b0adb80 1 -- start start 2024-05-14T09:35:04.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.198+0000 7fcd1b0adb80 1 Processor -- start 2024-05-14T09:35:04.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd1b0adb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d9390bec0 0x555d93a93520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd1b0adb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d93a89e90 0x555d93a93a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd1b0adb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a93fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd1b0adb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555d93a96840 con 0x555d93a8fe00 2024-05-14T09:35:04.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd1b0adb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555d93964f50 con 0x555d9390bec0 2024-05-14T09:35:04.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd1b0adb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555d93a969c0 con 0x555d93a89e90 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd1b0adb80 10 client.0 _ll_register_callbacks cb 0x555d939439d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd0ac36700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a93fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd0a435700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d9390bec0 0x555d93a93520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd0ac36700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a93fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42450/0 (socket says 172.21.15.2:42450) 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd0ac36700 1 -- 172.21.15.2:0/1222655516 learned_addr learned my addr 172.21.15.2:0/1222655516 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd09c34700 1 --2- 172.21.15.2:0/1222655516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d93a89e90 0x555d93a93a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd0ac36700 1 -- 172.21.15.2:0/1222655516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d93a89e90 msgr2=0x555d93a93a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd0ac36700 1 --2- 172.21.15.2:0/1222655516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d93a89e90 0x555d93a93a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd0ac36700 1 -- 172.21.15.2:0/1222655516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d9390bec0 msgr2=0x555d93a93520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd0ac36700 1 --2- 172.21.15.2:0/1222655516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d9390bec0 0x555d93a93520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd0ac36700 1 -- 172.21.15.2:0/1222655516 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d939627e0 con 0x555d93a8fe00 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd0a435700 1 --2- 172.21.15.2:0/1222655516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d9390bec0 0x555d93a93520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.199+0000 7fcd09c34700 1 --2- 172.21.15.2:0/1222655516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d93a89e90 0x555d93a93a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.200+0000 7fcd0ac36700 1 --2- 172.21.15.2:0/1222655516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a93fa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fccfc010a40 tx=0x7fccfc010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62607007c95ff43 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.200+0000 7fccedffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.200+0000 7fccedffb700 1 -- 172.21.15.2:0/1222655516 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00ec50 con 0x555d93a8fe00 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.200+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/1222655516 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd00007a70 con 0x555d93a8fe00 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.200+0000 7fccedffb700 1 -- 172.21.15.2:0/1222655516 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fccfc00edf0 con 0x555d93a8fe00 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.200+0000 7fccedffb700 1 -- 172.21.15.2:0/1222655516 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fccfc025050 con 0x555d93a8fe00 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.200+0000 7fcd1b0adb80 10 client.15330 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.200+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/1222655516 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d93aa14d0 con 0x555d93a8fe00 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.201+0000 7fccedffb700 1 -- 172.21.15.2:0/1222655516 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd00007a70 con 0x555d93a8fe00 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.201+0000 7fccedffb700 1 -- 172.21.15.2:0/1222655516 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fccfc011440 con 0x555d93a8fe00 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.201+0000 7fcd1b0adb80 10 client.15330 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.201+0000 7fcd1b0adb80 10 client.15330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.201+0000 7fcd1b0adb80 20 client.15330 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.201+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/1222655516 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555d93aa3b10 0x555d93aa5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.201+0000 7fcd1b0adb80 4 client.15330 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.201+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/1222655516 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d939b93a0 con 0x555d93aa3b10 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.201+0000 7fcd0a435700 1 --2- 172.21.15.2:0/1222655516 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555d93aa3b10 0x555d93aa5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.202+0000 7fccedffb700 10 client.15330 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.202+0000 7fcd0a435700 1 --2- 172.21.15.2:0/1222655516 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555d93aa3b10 0x555d93aa5f30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8d870e4b059a296 server_cookie=18eb4e191cfbfb16 in_seq=0 out_seq=0 2024-05-14T09:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.202+0000 7fccedffb700 1 -- 172.21.15.2:0/1222655516 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555d939b93a0 con 0x555d93aa3b10 2024-05-14T09:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.202+0000 7fccedffb700 10 client.15330 handle_command_reply: tid=0 2024-05-14T09:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.203+0000 7fcd1b0adb80 1 client.15330 shutdown 2024-05-14T09:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.203+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/1222655516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 msgr2=0x555d93a93fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.203+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/1222655516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a93fa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fccfc010a40 tx=0x7fccfc010d50 comp rx=0 tx=0).stop 2024-05-14T09:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.203+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/1222655516 shutdown_connections 2024-05-14T09:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.203+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/1222655516 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555d93aa3b10 0x555d93aa5f30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.203+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/1222655516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d9390bec0 0x555d93a93520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.203+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/1222655516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a93fa0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.203+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/1222655516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d93a89e90 0x555d93a93a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.203+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/1222655516 >> 172.21.15.2:0/1222655516 conn(0x555d939536b0 msgr2=0x555d93a88940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:04.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.203+0000 7fccedffb700 0 client.15330 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:04.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.203+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/1222655516 shutdown_connections 2024-05-14T09:35:04.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.204+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/1222655516 wait complete. 2024-05-14T09:35:04.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.204+0000 7fcd1b0adb80 20 client.15330 trim_cache size 0 max 16384 2024-05-14T09:35:04.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.234+0000 7fcd1b0adb80 1 Processor -- start 2024-05-14T09:35:04.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.234+0000 7fcd1b0adb80 1 -- start start 2024-05-14T09:35:04.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.234+0000 7fcd1b0adb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d9390bec0 0x555d93a89950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.234+0000 7fcd1b0adb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d93a89e90 0x555d93a95600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.234+0000 7fcd1b0adb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a93de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.234+0000 7fcd1b0adb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcd041b4e60 con 0x555d93a8fe00 2024-05-14T09:35:04.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.234+0000 7fcd1b0adb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcd041b26f0 con 0x555d93a89e90 2024-05-14T09:35:04.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.234+0000 7fcd1b0adb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcd041b0540 con 0x555d9390bec0 2024-05-14T09:35:04.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.235+0000 7fcd0ac36700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a93de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.235+0000 7fcd0ac36700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a93de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42456/0 (socket says 172.21.15.2:42456) 2024-05-14T09:35:04.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.235+0000 7fcd0ac36700 1 -- 172.21.15.2:0/3938267336 learned_addr learned my addr 172.21.15.2:0/3938267336 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:04.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.235+0000 7fcd0a435700 1 --2- 172.21.15.2:0/3938267336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d9390bec0 0x555d93a89950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.235+0000 7fcd09c34700 1 --2- 172.21.15.2:0/3938267336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d93a89e90 0x555d93a95600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.235+0000 7fcd0ac36700 1 -- 172.21.15.2:0/3938267336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d9390bec0 msgr2=0x555d93a89950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:04.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.235+0000 7fcd0ac36700 1 --2- 172.21.15.2:0/3938267336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d9390bec0 0x555d93a89950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.235+0000 7fcd0ac36700 1 -- 172.21.15.2:0/3938267336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d93a89e90 msgr2=0x555d93a95600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:04.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.235+0000 7fcd0ac36700 1 --2- 172.21.15.2:0/3938267336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d93a89e90 0x555d93a95600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.235+0000 7fcd0ac36700 1 -- 172.21.15.2:0/3938267336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d93a945e0 con 0x555d93a8fe00 2024-05-14T09:35:04.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.235+0000 7fcd0a435700 1 --2- 172.21.15.2:0/3938267336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d9390bec0 0x555d93a89950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:35:04.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.235+0000 7fcd0ac36700 1 --2- 172.21.15.2:0/3938267336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a93de0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fccfc013f70 tx=0x7fccfc013fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2b87ca2b15b4d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:04.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.235+0000 7fcceffff700 1 -- 172.21.15.2:0/3938267336 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00c7a0 con 0x555d93a8fe00 2024-05-14T09:35:04.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.235+0000 7fcceffff700 1 -- 172.21.15.2:0/3938267336 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccfc011440 con 0x555d93a8fe00 2024-05-14T09:35:04.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.235+0000 7fcceffff700 1 -- 172.21.15.2:0/3938267336 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc005330 con 0x555d93a8fe00 2024-05-14T09:35:04.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.236+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/3938267336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 msgr2=0x555d93a93de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:04.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.236+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/3938267336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a93de0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fccfc013f70 tx=0x7fccfc013fa0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.236+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/3938267336 shutdown_connections 2024-05-14T09:35:04.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.236+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/3938267336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a93de0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.236+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/3938267336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d9390bec0 0x555d93a89950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.236+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/3938267336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d93a89e90 0x555d93a95600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.236+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/3938267336 >> 172.21.15.2:0/3938267336 conn(0x555d939536b0 msgr2=0x555d93a7ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:04.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.236+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/3938267336 shutdown_connections 2024-05-14T09:35:04.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.236+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/3938267336 wait complete. 2024-05-14T09:35:04.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.237+0000 7fcd1b0adb80 1 -- start start 2024-05-14T09:35:04.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.237+0000 7fcd1b0adb80 1 Processor -- start 2024-05-14T09:35:04.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.237+0000 7fcd1b0adb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d9390bec0 0x555d93a9bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.237+0000 7fcd1b0adb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d93a89e90 0x555d93a98c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.237+0000 7fcd1b0adb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a99160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.237+0000 7fcd1b0adb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcd041b0bb0 con 0x555d93a8fe00 2024-05-14T09:35:04.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.237+0000 7fcd1b0adb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcd041b2020 con 0x555d93a89e90 2024-05-14T09:35:04.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.237+0000 7fcd1b0adb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcd041b1280 con 0x555d9390bec0 2024-05-14T09:35:04.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.237+0000 7fcd1b0adb80 10 client.0 _ll_register_callbacks cb 0x555d939439d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:04.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.237+0000 7fcd0ac36700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a99160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.237+0000 7fcd0ac36700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a99160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42460/0 (socket says 172.21.15.2:42460) 2024-05-14T09:35:04.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.237+0000 7fcd0ac36700 1 -- 172.21.15.2:0/2761736766 learned_addr learned my addr 172.21.15.2:0/2761736766 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:04.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.237+0000 7fcd0a435700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d9390bec0 0x555d93a9bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.237+0000 7fcd09c34700 1 --2- 172.21.15.2:0/2761736766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d93a89e90 0x555d93a98c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.238+0000 7fcd0ac36700 1 -- 172.21.15.2:0/2761736766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d9390bec0 msgr2=0x555d93a9bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:04.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.238+0000 7fcd0ac36700 1 --2- 172.21.15.2:0/2761736766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d9390bec0 0x555d93a9bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.238+0000 7fcd0ac36700 1 -- 172.21.15.2:0/2761736766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d93a89e90 msgr2=0x555d93a98c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:04.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.238+0000 7fcd0ac36700 1 --2- 172.21.15.2:0/2761736766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d93a89e90 0x555d93a98c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.238+0000 7fcd0ac36700 1 -- 172.21.15.2:0/2761736766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d93a996a0 con 0x555d93a8fe00 2024-05-14T09:35:04.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.238+0000 7fcd0ac36700 1 --2- 172.21.15.2:0/2761736766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a99160 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fccfc000c00 tx=0x7fccfc010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c0413d12f8e4f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:04.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.238+0000 7fccedffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:04.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.238+0000 7fccedffb700 1 -- 172.21.15.2:0/2761736766 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc011a00 con 0x555d93a8fe00 2024-05-14T09:35:04.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.238+0000 7fccedffb700 1 -- 172.21.15.2:0/2761736766 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fccfc011ba0 con 0x555d93a8fe00 2024-05-14T09:35:04.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.238+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/2761736766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccfc011440 con 0x555d93a8fe00 2024-05-14T09:35:04.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.238+0000 7fccedffb700 1 -- 172.21.15.2:0/2761736766 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fccfc03f050 con 0x555d93a8fe00 2024-05-14T09:35:04.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.238+0000 7fcd1b0adb80 10 client.15336 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:04.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.238+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/2761736766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d93b0ef00 con 0x555d93a8fe00 2024-05-14T09:35:04.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.238+0000 7fccedffb700 1 -- 172.21.15.2:0/2761736766 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccfc011440 con 0x555d93a8fe00 2024-05-14T09:35:04.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.239+0000 7fccedffb700 1 -- 172.21.15.2:0/2761736766 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fccfc003840 con 0x555d93a8fe00 2024-05-14T09:35:04.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.239+0000 7fcd1b0adb80 10 client.15336 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:04.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.239+0000 7fcd1b0adb80 10 client.15336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:04.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.239+0000 7fcd1b0adb80 20 client.15336 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:04.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.239+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/2761736766 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555d93b0f4e0 0x555d93b11900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:04.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.239+0000 7fcd1b0adb80 4 client.15336 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T09:35:04.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.239+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/2761736766 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555d939b3440 con 0x555d93b0f4e0 2024-05-14T09:35:04.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.239+0000 7fcd0a435700 1 --2- 172.21.15.2:0/2761736766 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555d93b0f4e0 0x555d93b11900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:04.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.239+0000 7fccedffb700 10 client.15336 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:04.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.239+0000 7fcd0a435700 1 --2- 172.21.15.2:0/2761736766 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555d93b0f4e0 0x555d93b11900 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b46ec10ed4db004 server_cookie=df58e8a68ec42add in_seq=0 out_seq=0 2024-05-14T09:35:04.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.241+0000 7fccedffb700 1 -- 172.21.15.2:0/2761736766 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555d939b3440 con 0x555d93b0f4e0 2024-05-14T09:35:04.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.241+0000 7fccedffb700 10 client.15336 handle_command_reply: tid=0 2024-05-14T09:35:04.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.241+0000 7fcd1b0adb80 1 client.15336 shutdown 2024-05-14T09:35:04.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.241+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/2761736766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 msgr2=0x555d93a99160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:04.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.241+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/2761736766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a99160 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fccfc000c00 tx=0x7fccfc010d50 comp rx=0 tx=0).stop 2024-05-14T09:35:04.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.241+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/2761736766 shutdown_connections 2024-05-14T09:35:04.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.241+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/2761736766 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555d93b0f4e0 0x555d93b11900 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.241+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/2761736766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d93a8fe00 0x555d93a99160 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.241+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/2761736766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555d9390bec0 0x555d93a9bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.241+0000 7fcd1b0adb80 1 --2- 172.21.15.2:0/2761736766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555d93a89e90 0x555d93a98c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.241+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/2761736766 >> 172.21.15.2:0/2761736766 conn(0x555d939536b0 msgr2=0x555d93a7ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:04.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.241+0000 7fccedffb700 0 client.15336 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:04.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.241+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/2761736766 shutdown_connections 2024-05-14T09:35:04.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.241+0000 7fcd1b0adb80 1 -- 172.21.15.2:0/2761736766 wait complete. 2024-05-14T09:35:04.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.241+0000 7fcd1b0adb80 20 client.15336 trim_cache size 0 max 16384 2024-05-14T09:35:04.245 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:35:04.245 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-05-14T09:35:04.245 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "12c4b195-ca28-4e34-b39e-76a450da6fc1", 2024-05-14T09:35:04.245 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-05-14T09:35:04.245 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:35:04.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.242+0000 7fcd0c699700 1 -- 172.21.15.2:0/1395412896 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fccf00b2880 msgr2=0x7fccf00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:04.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.242+0000 7fcd0c699700 1 --2- 172.21.15.2:0/1395412896 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fccf00b2880 0x7fccf00b4d90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009900 tx=0x7fcd0000b040 comp rx=0 tx=0).stop 2024-05-14T09:35:04.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.242+0000 7fcd0c699700 1 -- 172.21.15.2:0/1395412896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd04107df0 msgr2=0x7fcd04100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:04.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.242+0000 7fcd0c699700 1 --2- 172.21.15.2:0/1395412896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd04107df0 0x7fcd04100240 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fccf400d950 tx=0x7fccf400dc60 comp rx=0 tx=0).stop 2024-05-14T09:35:04.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.242+0000 7fcd0c699700 1 -- 172.21.15.2:0/1395412896 shutdown_connections 2024-05-14T09:35:04.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.242+0000 7fcd0c699700 1 --2- 172.21.15.2:0/1395412896 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fccf00b2880 0x7fccf00b4d90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.242+0000 7fcd0c699700 1 --2- 172.21.15.2:0/1395412896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd041034c0 0x7fcd040ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.242+0000 7fcd0c699700 1 --2- 172.21.15.2:0/1395412896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcd04108740 0x7fcd04100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.242+0000 7fcd0c699700 1 --2- 172.21.15.2:0/1395412896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd04107df0 0x7fcd04100240 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:04.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.242+0000 7fcd0c699700 1 -- 172.21.15.2:0/1395412896 >> 172.21.15.2:0/1395412896 conn(0x7fcd040fedf0 msgr2=0x7fcd04101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:04.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.242+0000 7fcd0c699700 1 -- 172.21.15.2:0/1395412896 shutdown_connections 2024-05-14T09:35:04.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:04.242+0000 7fcd0c699700 1 -- 172.21.15.2:0/1395412896 wait complete. 2024-05-14T09:35:04.880 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:04 smithi002 ceph-mon[31557]: pgmap v427: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-14T09:35:04.911 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T09:35:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:04 smithi029 ceph-mon[36249]: pgmap v427: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-14T09:35:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:04 smithi026 ceph-mon[36056]: pgmap v427: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-14T09:35:05.206 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:35:05.634 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:05 smithi002 ceph-mon[31557]: scrub queued for path: / 2024-05-14T09:35:05.634 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:05 smithi002 ceph-mon[31557]: scrub summary: idle+waiting paths [/] 2024-05-14T09:35:05.635 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:05 smithi002 ceph-mon[31557]: scrub summary: active paths [/] 2024-05-14T09:35:05.635 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:05 smithi002 ceph-mon[31557]: scrub summary: idle+waiting paths [/] 2024-05-14T09:35:05.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:05 smithi029 ceph-mon[36249]: scrub queued for path: / 2024-05-14T09:35:05.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:05 smithi029 ceph-mon[36249]: scrub summary: idle+waiting paths [/] 2024-05-14T09:35:05.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:05 smithi029 ceph-mon[36249]: scrub summary: active paths [/] 2024-05-14T09:35:05.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:05 smithi029 ceph-mon[36249]: scrub summary: idle+waiting paths [/] 2024-05-14T09:35:05.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:05 smithi026 ceph-mon[36056]: scrub queued for path: / 2024-05-14T09:35:05.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:05 smithi026 ceph-mon[36056]: scrub summary: idle+waiting paths [/] 2024-05-14T09:35:05.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:05 smithi026 ceph-mon[36056]: scrub summary: active paths [/] 2024-05-14T09:35:05.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:05 smithi026 ceph-mon[36056]: scrub summary: idle+waiting paths [/] 2024-05-14T09:35:06.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.719+0000 7f0f8aad2700 1 -- 172.21.15.2:0/2731230982 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f84108740 msgr2=0x7f0f841100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:06.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.719+0000 7f0f8aad2700 1 --2- 172.21.15.2:0/2731230982 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f84108740 0x7f0f841100a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00b0d0 tx=0x7f0f7c00b3e0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.719+0000 7f0f8aad2700 1 -- 172.21.15.2:0/2731230982 shutdown_connections 2024-05-14T09:35:06.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.719+0000 7f0f8aad2700 1 --2- 172.21.15.2:0/2731230982 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f84108740 0x7f0f841100a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.719+0000 7f0f8aad2700 1 --2- 172.21.15.2:0/2731230982 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f841034c0 0x7f0f841038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.719+0000 7f0f8aad2700 1 --2- 172.21.15.2:0/2731230982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f84107df0 0x7f0f84108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.719+0000 7f0f8aad2700 1 -- 172.21.15.2:0/2731230982 >> 172.21.15.2:0/2731230982 conn(0x7f0f840fedc0 msgr2=0x7f0f841011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:06.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.719+0000 7f0f8aad2700 1 -- 172.21.15.2:0/2731230982 shutdown_connections 2024-05-14T09:35:06.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.719+0000 7f0f8aad2700 1 -- 172.21.15.2:0/2731230982 wait complete. 2024-05-14T09:35:06.720 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:06 smithi002 ceph-mon[31557]: pgmap v428: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-14T09:35:06.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.720+0000 7f0f8aad2700 1 Processor -- start 2024-05-14T09:35:06.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.720+0000 7f0f8aad2700 1 -- start start 2024-05-14T09:35:06.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.720+0000 7f0f8aad2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f841034c0 0x7f0f840ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.720+0000 7f0f8aad2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f84107df0 0x7f0f840ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.720+0000 7f0f8aad2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f84108740 0x7f0f840ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.720+0000 7f0f8aad2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84113370 con 0x7f0f84107df0 2024-05-14T09:35:06.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.720+0000 7f0f8aad2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841131f0 con 0x7f0f841034c0 2024-05-14T09:35:06.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.720+0000 7f0f8aad2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841134f0 con 0x7f0f84108740 2024-05-14T09:35:06.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.720+0000 7f0f83fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f84107df0 0x7f0f840ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.720+0000 7f0f83fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f84107df0 0x7f0f840ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42478/0 (socket says 172.21.15.2:42478) 2024-05-14T09:35:06.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.720+0000 7f0f83fff700 1 -- 172.21.15.2:0/612667257 learned_addr learned my addr 172.21.15.2:0/612667257 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:06.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.720+0000 7f0f8886e700 1 --2- 172.21.15.2:0/612667257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f841034c0 0x7f0f840ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.720+0000 7f0f8906f700 1 --2- 172.21.15.2:0/612667257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f84108740 0x7f0f840ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.721+0000 7f0f83fff700 1 -- 172.21.15.2:0/612667257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f84108740 msgr2=0x7f0f840ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:06.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.721+0000 7f0f83fff700 1 --2- 172.21.15.2:0/612667257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f84108740 0x7f0f840ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.721+0000 7f0f83fff700 1 -- 172.21.15.2:0/612667257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f841034c0 msgr2=0x7f0f840ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:06.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.721+0000 7f0f83fff700 1 --2- 172.21.15.2:0/612667257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f841034c0 0x7f0f840ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.721+0000 7f0f83fff700 1 -- 172.21.15.2:0/612667257 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f841a9170 con 0x7f0f84107df0 2024-05-14T09:35:06.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.721+0000 7f0f83fff700 1 --2- 172.21.15.2:0/612667257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f84107df0 0x7f0f840ff7e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800e970 tx=0x7f0f7800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:06.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.721+0000 7f0f81ffb700 1 -- 172.21.15.2:0/612667257 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78004a30 con 0x7f0f84107df0 2024-05-14T09:35:06.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.721+0000 7f0f8aad2700 1 -- 172.21.15.2:0/612667257 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f841a9460 con 0x7f0f84107df0 2024-05-14T09:35:06.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.721+0000 7f0f81ffb700 1 -- 172.21.15.2:0/612667257 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f7800cc90 con 0x7f0f84107df0 2024-05-14T09:35:06.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.721+0000 7f0f81ffb700 1 -- 172.21.15.2:0/612667257 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78005610 con 0x7f0f84107df0 2024-05-14T09:35:06.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.721+0000 7f0f8aad2700 1 -- 172.21.15.2:0/612667257 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f841a99f0 con 0x7f0f84107df0 2024-05-14T09:35:06.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.724+0000 7f0f81ffb700 1 -- 172.21.15.2:0/612667257 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0f7800c7e0 con 0x7f0f84107df0 2024-05-14T09:35:06.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.725+0000 7f0f81ffb700 1 --2- 172.21.15.2:0/612667257 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0f6c0b2900 0x7f0f6c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.725+0000 7f0f81ffb700 1 -- 172.21.15.2:0/612667257 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f0f780e26d0 con 0x7f0f84107df0 2024-05-14T09:35:06.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.725+0000 7f0f8886e700 1 --2- 172.21.15.2:0/612667257 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0f6c0b2900 0x7f0f6c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.725+0000 7f0f994e6b80 1 Processor -- start 2024-05-14T09:35:06.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.725+0000 7f0f994e6b80 1 -- start start 2024-05-14T09:35:06.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.725+0000 7f0f8886e700 1 --2- 172.21.15.2:0/612667257 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0f6c0b2900 0x7f0f6c0b4e10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0f74005950 tx=0x7f0f7400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:06.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f994e6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a295f9ae40 0x55a296126b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f994e6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961270d0 0x55a296120c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f994e6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a2961211b0 0x55a2961215c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f994e6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a29603a860 con 0x55a2961270d0 2024-05-14T09:35:06.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f994e6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a295fd97c0 con 0x55a295f9ae40 2024-05-14T09:35:06.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f994e6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a296036820 con 0x55a2961211b0 2024-05-14T09:35:06.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f83fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961270d0 0x55a296120c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f83fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961270d0 0x55a296120c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42494/0 (socket says 172.21.15.2:42494) 2024-05-14T09:35:06.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f83fff700 1 -- 172.21.15.2:0/2769604717 learned_addr learned my addr 172.21.15.2:0/2769604717 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:06.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f8886e700 1 --2- 172.21.15.2:0/2769604717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a295f9ae40 0x55a296126b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f8906f700 1 --2- 172.21.15.2:0/2769604717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a2961211b0 0x55a2961215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f83fff700 1 -- 172.21.15.2:0/2769604717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a2961211b0 msgr2=0x55a2961215c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:06.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f83fff700 1 --2- 172.21.15.2:0/2769604717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a2961211b0 0x55a2961215c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f83fff700 1 -- 172.21.15.2:0/2769604717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a295f9ae40 msgr2=0x55a296126b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:06.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f83fff700 1 --2- 172.21.15.2:0/2769604717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a295f9ae40 0x55a296126b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f83fff700 1 -- 172.21.15.2:0/2769604717 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a295f878d0 con 0x55a2961270d0 2024-05-14T09:35:06.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f83fff700 1 --2- 172.21.15.2:0/2769604717 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961270d0 0x55a296120c70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0f780a5950 tx=0x7f0f78005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0553da15730db5f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:06.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.726+0000 7f0f6affd700 1 -- 172.21.15.2:0/2769604717 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f780a5060 con 0x55a2961270d0 2024-05-14T09:35:06.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.727+0000 7f0f6affd700 1 -- 172.21.15.2:0/2769604717 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f7800cad0 con 0x55a2961270d0 2024-05-14T09:35:06.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.727+0000 7f0f6affd700 1 -- 172.21.15.2:0/2769604717 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f780adc80 con 0x55a2961270d0 2024-05-14T09:35:06.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.727+0000 7f0f994e6b80 1 -- 172.21.15.2:0/2769604717 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961270d0 msgr2=0x55a296120c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:06.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.727+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/2769604717 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961270d0 0x55a296120c70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0f780a5950 tx=0x7f0f78005910 comp rx=0 tx=0).stop 2024-05-14T09:35:06.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.727+0000 7f0f994e6b80 1 -- 172.21.15.2:0/2769604717 shutdown_connections 2024-05-14T09:35:06.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.727+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/2769604717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a295f9ae40 0x55a296126b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.727+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/2769604717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a2961211b0 0x55a2961215c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.727+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/2769604717 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961270d0 0x55a296120c70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.727+0000 7f0f994e6b80 1 -- 172.21.15.2:0/2769604717 >> 172.21.15.2:0/2769604717 conn(0x55a295fe2600 msgr2=0x55a2961134b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:06.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.728+0000 7f0f994e6b80 1 -- 172.21.15.2:0/2769604717 shutdown_connections 2024-05-14T09:35:06.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.728+0000 7f0f994e6b80 1 -- 172.21.15.2:0/2769604717 wait complete. 2024-05-14T09:35:06.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.728+0000 7f0f994e6b80 1 -- start start 2024-05-14T09:35:06.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.728+0000 7f0f994e6b80 1 Processor -- start 2024-05-14T09:35:06.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.728+0000 7f0f994e6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a295f9ae40 0x55a29612a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.728+0000 7f0f994e6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961211b0 0x55a29612aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.728+0000 7f0f994e6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961270d0 0x55a29612b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f994e6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a29612db70 con 0x55a2961211b0 2024-05-14T09:35:06.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f994e6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a295fa0c30 con 0x55a2961270d0 2024-05-14T09:35:06.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f994e6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a29612dcf0 con 0x55a295f9ae40 2024-05-14T09:35:06.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f994e6b80 10 client.0 _ll_register_callbacks cb 0x55a295fd2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:06.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f83fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961211b0 0x55a29612aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f83fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961211b0 0x55a29612aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42498/0 (socket says 172.21.15.2:42498) 2024-05-14T09:35:06.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f83fff700 1 -- 172.21.15.2:0/590242017 learned_addr learned my addr 172.21.15.2:0/590242017 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:06.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f8906f700 1 --2- 172.21.15.2:0/590242017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961270d0 0x55a29612b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f8886e700 1 --2- 172.21.15.2:0/590242017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a295f9ae40 0x55a29612a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f8906f700 1 -- 172.21.15.2:0/590242017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a295f9ae40 msgr2=0x55a29612a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:06.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f8906f700 1 --2- 172.21.15.2:0/590242017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a295f9ae40 0x55a29612a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f8906f700 1 -- 172.21.15.2:0/590242017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961211b0 msgr2=0x55a29612aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:06.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f8906f700 1 --2- 172.21.15.2:0/590242017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961211b0 0x55a29612aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f8906f700 1 -- 172.21.15.2:0/590242017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a295fd6690 con 0x55a2961270d0 2024-05-14T09:35:06.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f8906f700 1 --2- 172.21.15.2:0/590242017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961270d0 0x55a29612b400 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00b770 tx=0x7f0f7c00baa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71d0cb6abe744aec server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:06.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f63fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:35:06.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.729+0000 7f0f63fff700 1 -- 172.21.15.2:0/590242017 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00be20 con 0x55a2961270d0 2024-05-14T09:35:06.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.730+0000 7f0f994e6b80 1 -- 172.21.15.2:0/590242017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a296048240 con 0x55a2961270d0 2024-05-14T09:35:06.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.730+0000 7f0f63fff700 1 -- 172.21.15.2:0/590242017 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f0f7c003a00 con 0x55a2961270d0 2024-05-14T09:35:06.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.730+0000 7f0f63fff700 1 -- 172.21.15.2:0/590242017 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f7c02e050 con 0x55a2961270d0 2024-05-14T09:35:06.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.730+0000 7f0f994e6b80 10 client.25076 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:06.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.730+0000 7f0f994e6b80 1 -- 172.21.15.2:0/590242017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a296138810 con 0x55a2961270d0 2024-05-14T09:35:06.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.730+0000 7f0f63fff700 1 -- 172.21.15.2:0/590242017 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a296048240 con 0x55a2961270d0 2024-05-14T09:35:06.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.730+0000 7f0f63fff700 1 -- 172.21.15.2:0/590242017 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0f7c011070 con 0x55a2961270d0 2024-05-14T09:35:06.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.730+0000 7f0f994e6b80 10 client.25076 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:06.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.730+0000 7f0f994e6b80 10 client.25076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:06.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.730+0000 7f0f994e6b80 20 client.25076 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:06.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.730+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/590242017 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a29613ada0 0x55a29613d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.730+0000 7f0f994e6b80 4 client.25076 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:35:06.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.730+0000 7f0f994e6b80 1 -- 172.21.15.2:0/590242017 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2960422e0 con 0x55a29613ada0 2024-05-14T09:35:06.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.730+0000 7f0f8886e700 1 --2- 172.21.15.2:0/590242017 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a29613ada0 0x55a29613d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.731+0000 7f0f63fff700 10 client.25076 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:06.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.731+0000 7f0f8886e700 1 --2- 172.21.15.2:0/590242017 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a29613ada0 0x55a29613d200 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca7d074b02e2dcdf server_cookie=a67f704b5750120 in_seq=0 out_seq=0 2024-05-14T09:35:06.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.732+0000 7f0f63fff700 1 -- 172.21.15.2:0/590242017 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a2960422e0 con 0x55a29613ada0 2024-05-14T09:35:06.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.732+0000 7f0f63fff700 10 client.25076 handle_command_reply: tid=0 2024-05-14T09:35:06.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.732+0000 7f0f994e6b80 1 client.25076 shutdown 2024-05-14T09:35:06.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.732+0000 7f0f994e6b80 1 -- 172.21.15.2:0/590242017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961270d0 msgr2=0x55a29612b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:06.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.732+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/590242017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961270d0 0x55a29612b400 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00b770 tx=0x7f0f7c00baa0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.732+0000 7f0f994e6b80 1 -- 172.21.15.2:0/590242017 shutdown_connections 2024-05-14T09:35:06.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.732+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/590242017 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a29613ada0 0x55a29613d200 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.732+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/590242017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961270d0 0x55a29612b400 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.732+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/590242017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a295f9ae40 0x55a29612a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.732+0000 7f0f63fff700 0 client.25076 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:06.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.732+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/590242017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961211b0 0x55a29612aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.732+0000 7f0f994e6b80 1 -- 172.21.15.2:0/590242017 >> 172.21.15.2:0/590242017 conn(0x55a295fe2600 msgr2=0x55a296117450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:06.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.732+0000 7f0f994e6b80 1 -- 172.21.15.2:0/590242017 shutdown_connections 2024-05-14T09:35:06.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.732+0000 7f0f994e6b80 1 -- 172.21.15.2:0/590242017 wait complete. 2024-05-14T09:35:06.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.732+0000 7f0f994e6b80 20 client.25076 trim_cache size 0 max 16384 2024-05-14T09:35:06.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.752+0000 7f0f994e6b80 1 Processor -- start 2024-05-14T09:35:06.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.752+0000 7f0f994e6b80 1 -- start start 2024-05-14T09:35:06.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f994e6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a295f9ae40 0x55a296113610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f994e6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961211b0 0x55a296117b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f994e6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a2961270d0 0x55a296136c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f994e6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841b0b90 con 0x55a295f9ae40 2024-05-14T09:35:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f994e6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841ae420 con 0x55a2961211b0 2024-05-14T09:35:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f994e6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841ac270 con 0x55a2961270d0 2024-05-14T09:35:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f8886e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a295f9ae40 0x55a296113610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f8886e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a295f9ae40 0x55a296113610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42512/0 (socket says 172.21.15.2:42512) 2024-05-14T09:35:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f8886e700 1 -- 172.21.15.2:0/3495852945 learned_addr learned my addr 172.21.15.2:0/3495852945 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f83fff700 1 --2- 172.21.15.2:0/3495852945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961211b0 0x55a296117b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f8906f700 1 --2- 172.21.15.2:0/3495852945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a2961270d0 0x55a296136c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f8886e700 1 -- 172.21.15.2:0/3495852945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a2961270d0 msgr2=0x55a296136c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f8886e700 1 --2- 172.21.15.2:0/3495852945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a2961270d0 0x55a296136c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f8886e700 1 -- 172.21.15.2:0/3495852945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961211b0 msgr2=0x55a296117b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:06.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f8886e700 1 --2- 172.21.15.2:0/3495852945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961211b0 0x55a296117b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f8886e700 1 -- 172.21.15.2:0/3495852945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a296127560 con 0x55a295f9ae40 2024-05-14T09:35:06.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.753+0000 7f0f83fff700 1 --2- 172.21.15.2:0/3495852945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961211b0 0x55a296117b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:35:06.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.754+0000 7f0f8886e700 1 --2- 172.21.15.2:0/3495852945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a295f9ae40 0x55a296113610 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f74005d90 tx=0x7f0f74013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e080eface4529 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:06.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.754+0000 7f0f6a7fc700 1 -- 172.21.15.2:0/3495852945 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74011e10 con 0x55a295f9ae40 2024-05-14T09:35:06.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.754+0000 7f0f6a7fc700 1 -- 172.21.15.2:0/3495852945 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f74004ce0 con 0x55a295f9ae40 2024-05-14T09:35:06.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.754+0000 7f0f6a7fc700 1 -- 172.21.15.2:0/3495852945 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74005280 con 0x55a295f9ae40 2024-05-14T09:35:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.755+0000 7f0f994e6b80 1 -- 172.21.15.2:0/3495852945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a295f9ae40 msgr2=0x55a296113610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.755+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/3495852945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a295f9ae40 0x55a296113610 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f74005d90 tx=0x7f0f74013000 comp rx=0 tx=0).stop 2024-05-14T09:35:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.755+0000 7f0f994e6b80 1 -- 172.21.15.2:0/3495852945 shutdown_connections 2024-05-14T09:35:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.755+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/3495852945 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a2961270d0 0x55a296136c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.755+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/3495852945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a295f9ae40 0x55a296113610 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.755+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/3495852945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961211b0 0x55a296117b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.755+0000 7f0f994e6b80 1 -- 172.21.15.2:0/3495852945 >> 172.21.15.2:0/3495852945 conn(0x55a295fe2600 msgr2=0x55a296117450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.755+0000 7f0f994e6b80 1 -- 172.21.15.2:0/3495852945 shutdown_connections 2024-05-14T09:35:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.755+0000 7f0f994e6b80 1 -- 172.21.15.2:0/3495852945 wait complete. 2024-05-14T09:35:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.755+0000 7f0f994e6b80 1 -- start start 2024-05-14T09:35:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.755+0000 7f0f994e6b80 1 Processor -- start 2024-05-14T09:35:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f994e6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a295f9ae40 0x55a296133620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f994e6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961211b0 0x55a296130a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f994e6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961270d0 0x55a2961310b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f994e6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841ac8e0 con 0x55a2961270d0 2024-05-14T09:35:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f994e6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841add50 con 0x55a2961211b0 2024-05-14T09:35:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f994e6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841acfb0 con 0x55a295f9ae40 2024-05-14T09:35:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f994e6b80 10 client.0 _ll_register_callbacks cb 0x55a295fd2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f8906f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961270d0 0x55a2961310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f8886e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a295f9ae40 0x55a296133620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f8906f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961270d0 0x55a2961310b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42526/0 (socket says 172.21.15.2:42526) 2024-05-14T09:35:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f83fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961211b0 0x55a296130a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f8906f700 1 -- 172.21.15.2:0/1735893059 learned_addr learned my addr 172.21.15.2:0/1735893059 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:06.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f8906f700 1 -- 172.21.15.2:0/1735893059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a295f9ae40 msgr2=0x55a296133620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:06.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f8906f700 1 --2- 172.21.15.2:0/1735893059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a295f9ae40 0x55a296133620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f8906f700 1 -- 172.21.15.2:0/1735893059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961211b0 msgr2=0x55a296130a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:06.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f8906f700 1 --2- 172.21.15.2:0/1735893059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961211b0 0x55a296130a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f8906f700 1 -- 172.21.15.2:0/1735893059 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a296131850 con 0x55a2961270d0 2024-05-14T09:35:06.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f8906f700 1 --2- 172.21.15.2:0/1735893059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961270d0 0x55a2961310b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0052c0 tx=0x7f0f7c005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3f5d3c8b083fd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:06.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f63fff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:06.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.756+0000 7f0f63fff700 1 -- 172.21.15.2:0/1735893059 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c005550 con 0x55a2961270d0 2024-05-14T09:35:06.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.757+0000 7f0f994e6b80 1 -- 172.21.15.2:0/1735893059 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f74004ce0 con 0x55a2961270d0 2024-05-14T09:35:06.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.757+0000 7f0f63fff700 1 -- 172.21.15.2:0/1735893059 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f0f7c02b070 con 0x55a2961270d0 2024-05-14T09:35:06.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.757+0000 7f0f63fff700 1 -- 172.21.15.2:0/1735893059 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f7c039050 con 0x55a2961270d0 2024-05-14T09:35:06.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.757+0000 7f0f994e6b80 10 client.15354 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:06.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.757+0000 7f0f994e6b80 1 -- 172.21.15.2:0/1735893059 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2961a5620 con 0x55a2961270d0 2024-05-14T09:35:06.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.757+0000 7f0f63fff700 1 -- 172.21.15.2:0/1735893059 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f74004ce0 con 0x55a2961270d0 2024-05-14T09:35:06.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.757+0000 7f0f63fff700 1 -- 172.21.15.2:0/1735893059 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0f7c011040 con 0x55a2961270d0 2024-05-14T09:35:06.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.757+0000 7f0f994e6b80 10 client.15354 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:06.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.757+0000 7f0f994e6b80 10 client.15354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:06.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.757+0000 7f0f994e6b80 20 client.15354 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:06.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.757+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/1735893059 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a2961a5c40 0x55a2961a8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:06.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.757+0000 7f0f994e6b80 4 client.15354 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T09:35:06.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.757+0000 7f0f994e6b80 1 -- 172.21.15.2:0/1735893059 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a295eba250 con 0x55a2961a5c40 2024-05-14T09:35:06.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.757+0000 7f0f8886e700 1 --2- 172.21.15.2:0/1735893059 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a2961a5c40 0x55a2961a8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:06.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f8886e700 1 --2- 172.21.15.2:0/1735893059 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a2961a5c40 0x55a2961a8060 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb914cac92408531 server_cookie=e887a72a21c490cb in_seq=0 out_seq=0 2024-05-14T09:35:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f63fff700 10 client.15354 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f63fff700 1 -- 172.21.15.2:0/1735893059 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a295eba250 con 0x55a2961a5c40 2024-05-14T09:35:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f63fff700 10 client.15354 handle_command_reply: tid=0 2024-05-14T09:35:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f994e6b80 1 client.15354 shutdown 2024-05-14T09:35:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f994e6b80 1 -- 172.21.15.2:0/1735893059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961270d0 msgr2=0x55a2961310b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/1735893059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961270d0 0x55a2961310b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0052c0 tx=0x7f0f7c005a10 comp rx=0 tx=0).stop 2024-05-14T09:35:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f994e6b80 1 -- 172.21.15.2:0/1735893059 shutdown_connections 2024-05-14T09:35:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/1735893059 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a2961a5c40 0x55a2961a8060 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/1735893059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2961270d0 0x55a2961310b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/1735893059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a295f9ae40 0x55a296133620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f994e6b80 1 --2- 172.21.15.2:0/1735893059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2961211b0 0x55a296130a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f994e6b80 1 -- 172.21.15.2:0/1735893059 >> 172.21.15.2:0/1735893059 conn(0x55a295fe2600 msgr2=0x55a296113370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f63fff700 0 client.15354 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f994e6b80 1 -- 172.21.15.2:0/1735893059 shutdown_connections 2024-05-14T09:35:06.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.758+0000 7f0f994e6b80 1 -- 172.21.15.2:0/1735893059 wait complete. 2024-05-14T09:35:06.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.759+0000 7f0f994e6b80 20 client.15354 trim_cache size 0 max 16384 2024-05-14T09:35:06.764 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:35:06.764 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T09:35:06.764 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-05-14T09:35:06.764 INFO:teuthology.orchestra.run.smithi002.stdout: "12c4b195-ca28-4e34-b39e-76a450da6fc1": { 2024-05-14T09:35:06.764 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-05-14T09:35:06.764 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "12c4b195-ca28-4e34-b39e-76a450da6fc1", 2024-05-14T09:35:06.764 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-05-14T09:35:06.765 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T09:35:06.765 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T09:35:06.765 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:35:06.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.759+0000 7f0f8aad2700 1 -- 172.21.15.2:0/612667257 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0f6c0b2900 msgr2=0x7f0f6c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:06.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.759+0000 7f0f8aad2700 1 --2- 172.21.15.2:0/612667257 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0f6c0b2900 0x7f0f6c0b4e10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0f74005950 tx=0x7f0f7400b410 comp rx=0 tx=0).stop 2024-05-14T09:35:06.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.759+0000 7f0f8aad2700 1 -- 172.21.15.2:0/612667257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f84107df0 msgr2=0x7f0f840ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:06.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.759+0000 7f0f8aad2700 1 --2- 172.21.15.2:0/612667257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f84107df0 0x7f0f840ff7e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800e970 tx=0x7f0f7800ec80 comp rx=0 tx=0).stop 2024-05-14T09:35:06.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.759+0000 7f0f8aad2700 1 -- 172.21.15.2:0/612667257 shutdown_connections 2024-05-14T09:35:06.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.759+0000 7f0f8aad2700 1 --2- 172.21.15.2:0/612667257 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0f6c0b2900 0x7f0f6c0b4e10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.759+0000 7f0f8aad2700 1 --2- 172.21.15.2:0/612667257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f841034c0 0x7f0f840ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.759+0000 7f0f8aad2700 1 --2- 172.21.15.2:0/612667257 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0f84108740 0x7f0f840ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.759+0000 7f0f8aad2700 1 --2- 172.21.15.2:0/612667257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f84107df0 0x7f0f840ff7e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:06.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.759+0000 7f0f8aad2700 1 -- 172.21.15.2:0/612667257 >> 172.21.15.2:0/612667257 conn(0x7f0f840fedc0 msgr2=0x7f0f8406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:06.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.759+0000 7f0f8aad2700 1 -- 172.21.15.2:0/612667257 shutdown_connections 2024-05-14T09:35:06.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:06.759+0000 7f0f8aad2700 1 -- 172.21.15.2:0/612667257 wait complete. 2024-05-14T09:35:06.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:06 smithi029 ceph-mon[36249]: pgmap v428: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-14T09:35:06.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:06 smithi026 ceph-mon[36056]: pgmap v428: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-14T09:35:07.487 INFO:tasks.cephfs.filesystem:scrub status for tag:12c4b195-ca28-4e34-b39e-76a450da6fc1 - {'path': '/', 'tag': '12c4b195-ca28-4e34-b39e-76a450da6fc1', 'options': 'recursive,force'} 2024-05-14T09:35:08.739 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:08 smithi029 ceph-mon[36249]: pgmap v429: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-05-14T09:35:08.753 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:08 smithi026 ceph-mon[36056]: pgmap v429: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-05-14T09:35:08.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:08 smithi002 ceph-mon[31557]: pgmap v429: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-05-14T09:35:10.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:10 smithi002 ceph-mon[31557]: pgmap v430: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-14T09:35:10.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:10 smithi029 ceph-mon[36249]: pgmap v430: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-14T09:35:10.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:10 smithi026 ceph-mon[36056]: pgmap v430: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-14T09:35:12.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:12 smithi002 ceph-mon[31557]: pgmap v431: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-14T09:35:12.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:12 smithi029 ceph-mon[36249]: pgmap v431: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-14T09:35:12.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:12 smithi026 ceph-mon[36056]: pgmap v431: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-14T09:35:14.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:14 smithi002 ceph-mon[31557]: pgmap v432: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T09:35:14.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:14 smithi029 ceph-mon[36249]: pgmap v432: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T09:35:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:14 smithi026 ceph-mon[36056]: pgmap v432: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T09:35:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.358+0000 7fbdcb16c640 1 -- 172.21.15.2:0/797234295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc4104ea0 msgr2=0x7fbdc41052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.358+0000 7fbdcb16c640 1 --2- 172.21.15.2:0/797234295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc4104ea0 0x7fbdc41052b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009900 tx=0x7fbdb402a410 comp rx=0 tx=0).stop 2024-05-14T09:35:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.358+0000 7fbdcb16c640 1 -- 172.21.15.2:0/797234295 shutdown_connections 2024-05-14T09:35:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.358+0000 7fbdcb16c640 1 --2- 172.21.15.2:0/797234295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbdc4109ba0 0x7fbdc410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.358+0000 7fbdcb16c640 1 --2- 172.21.15.2:0/797234295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc4104ea0 0x7fbdc41052b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.358+0000 7fbdcb16c640 1 --2- 172.21.15.2:0/797234295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc4103c60 0x7fbdc4104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.358+0000 7fbdcb16c640 1 -- 172.21.15.2:0/797234295 >> 172.21.15.2:0/797234295 conn(0x7fbdc40fee70 msgr2=0x7fbdc4101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.358+0000 7fbdcb16c640 1 -- 172.21.15.2:0/797234295 shutdown_connections 2024-05-14T09:35:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.358+0000 7fbdcb16c640 1 -- 172.21.15.2:0/797234295 wait complete. 2024-05-14T09:35:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdcb16c640 1 Processor -- start 2024-05-14T09:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdcb16c640 1 -- start start 2024-05-14T09:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdcb16c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc4103c60 0x7fbdc4077a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdcb16c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc4104ea0 0x7fbdc4076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdcb16c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbdc4109ba0 0x7fbdc4076670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdcb16c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc410f760 con 0x7fbdc4103c60 2024-05-14T09:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdcb16c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc410f5e0 con 0x7fbdc4104ea0 2024-05-14T09:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdcb16c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc410f8e0 con 0x7fbdc4109ba0 2024-05-14T09:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdc8f08640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc4103c60 0x7fbdc4077a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdc8f08640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc4103c60 0x7fbdc4077a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54878/0 (socket says 172.21.15.2:54878) 2024-05-14T09:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdc8f08640 1 -- 172.21.15.2:0/3302738093 learned_addr learned my addr 172.21.15.2:0/3302738093 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdc9709640 1 --2- 172.21.15.2:0/3302738093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbdc4109ba0 0x7fbdc4076670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdbbfff640 1 --2- 172.21.15.2:0/3302738093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc4104ea0 0x7fbdc4076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdc8f08640 1 -- 172.21.15.2:0/3302738093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbdc4109ba0 msgr2=0x7fbdc4076670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdc8f08640 1 --2- 172.21.15.2:0/3302738093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbdc4109ba0 0x7fbdc4076670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdc8f08640 1 -- 172.21.15.2:0/3302738093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc4104ea0 msgr2=0x7fbdc4076130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.359+0000 7fbdc8f08640 1 --2- 172.21.15.2:0/3302738093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc4104ea0 0x7fbdc4076130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.360+0000 7fbdc8f08640 1 -- 172.21.15.2:0/3302738093 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc4076da0 con 0x7fbdc4103c60 2024-05-14T09:35:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.360+0000 7fbdbbfff640 1 --2- 172.21.15.2:0/3302738093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc4104ea0 0x7fbdc4076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.360+0000 7fbdc8f08640 1 --2- 172.21.15.2:0/3302738093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc4103c60 0x7fbdc4077a90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000e9f0 tx=0x7fbdc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.360+0000 7fbdb9ffb640 1 -- 172.21.15.2:0/3302738093 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc000ccc0 con 0x7fbdc4103c60 2024-05-14T09:35:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.360+0000 7fbdcb16c640 1 -- 172.21.15.2:0/3302738093 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc41a4360 con 0x7fbdc4103c60 2024-05-14T09:35:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.360+0000 7fbdb9ffb640 1 -- 172.21.15.2:0/3302738093 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc00035f0 con 0x7fbdc4103c60 2024-05-14T09:35:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.360+0000 7fbdb9ffb640 1 -- 172.21.15.2:0/3302738093 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc000ce60 con 0x7fbdc4103c60 2024-05-14T09:35:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.360+0000 7fbdcb16c640 1 -- 172.21.15.2:0/3302738093 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc41a4850 con 0x7fbdc4103c60 2024-05-14T09:35:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.363+0000 7fbdb9ffb640 1 -- 172.21.15.2:0/3302738093 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fbdc0003aa0 con 0x7fbdc4103c60 2024-05-14T09:35:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.363+0000 7fbdd201b740 1 Processor -- start 2024-05-14T09:35:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.363+0000 7fbdd201b740 1 -- start start 2024-05-14T09:35:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.363+0000 7fbdd201b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63681b0 0x55c2b6709a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.363+0000 7fbdd201b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b633f2f0 0x55c2b6709fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.363+0000 7fbdd201b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b63474d0 0x55c2b6710ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.363+0000 7fbdd201b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c2b666e5d0 con 0x55c2b63681b0 2024-05-14T09:35:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.363+0000 7fbdd201b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c2b6627890 con 0x55c2b63474d0 2024-05-14T09:35:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.363+0000 7fbdd201b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c2b666a590 con 0x55c2b633f2f0 2024-05-14T09:35:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.363+0000 7fbdc9709640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63681b0 0x55c2b6709a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.363+0000 7fbdc9709640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63681b0 0x55c2b6709a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54888/0 (socket says 172.21.15.2:54888) 2024-05-14T09:35:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.363+0000 7fbdc9709640 1 -- 172.21.15.2:0/370350409 learned_addr learned my addr 172.21.15.2:0/370350409 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.363+0000 7fbdc8f08640 1 --2- 172.21.15.2:0/370350409 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b633f2f0 0x55c2b6709fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.364+0000 7fbdbbfff640 1 --2- 172.21.15.2:0/370350409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b63474d0 0x55c2b6710ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.364+0000 7fbdc9709640 1 -- 172.21.15.2:0/370350409 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b633f2f0 msgr2=0x55c2b6709fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.364+0000 7fbdc9709640 1 --2- 172.21.15.2:0/370350409 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b633f2f0 0x55c2b6709fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.364+0000 7fbdc9709640 1 -- 172.21.15.2:0/370350409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b63474d0 msgr2=0x55c2b6710ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.364+0000 7fbdc9709640 1 --2- 172.21.15.2:0/370350409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b63474d0 0x55c2b6710ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.364+0000 7fbdc9709640 1 -- 172.21.15.2:0/370350409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2b6380210 con 0x55c2b63681b0 2024-05-14T09:35:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.364+0000 7fbdbbfff640 1 --2- 172.21.15.2:0/370350409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b63474d0 0x55c2b6710ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.364+0000 7fbdc8f08640 1 --2- 172.21.15.2:0/370350409 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b633f2f0 0x55c2b6709fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.364+0000 7fbdb9ffb640 1 --2- 172.21.15.2:0/3302738093 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbdac0b28a0 0x7fbdac0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.364+0000 7fbdc9709640 1 --2- 172.21.15.2:0/370350409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63681b0 0x55c2b6709a60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0039f0 tx=0x7fbdbc003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3536c69f6979c27d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.364+0000 7fbdc8f08640 1 --2- 172.21.15.2:0/3302738093 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbdac0b28a0 0x7fbdac0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.364+0000 7fbdb9ffb640 1 -- 172.21.15.2:0/3302738093 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fbdc00e29c0 con 0x7fbdc4103c60 2024-05-14T09:35:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.364+0000 7fbdaaffd640 1 -- 172.21.15.2:0/370350409 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc01a470 con 0x55c2b63681b0 2024-05-14T09:35:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.364+0000 7fbdaaffd640 1 -- 172.21.15.2:0/370350409 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdbc015070 con 0x55c2b63681b0 2024-05-14T09:35:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.364+0000 7fbdaaffd640 1 -- 172.21.15.2:0/370350409 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc0107a0 con 0x55c2b63681b0 2024-05-14T09:35:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.365+0000 7fbdd201b740 1 -- 172.21.15.2:0/370350409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63681b0 msgr2=0x55c2b6709a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.365+0000 7fbdc8f08640 1 --2- 172.21.15.2:0/3302738093 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbdac0b28a0 0x7fbdac0b4db0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55c2b670ae40 tx=0x7fbdc0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.365+0000 7fbdd201b740 1 --2- 172.21.15.2:0/370350409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63681b0 0x55c2b6709a60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0039f0 tx=0x7fbdbc003eb0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.365+0000 7fbdd201b740 1 -- 172.21.15.2:0/370350409 shutdown_connections 2024-05-14T09:35:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.365+0000 7fbdd201b740 1 --2- 172.21.15.2:0/370350409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b63474d0 0x55c2b6710ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.365+0000 7fbdd201b740 1 --2- 172.21.15.2:0/370350409 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b633f2f0 0x55c2b6709fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.365+0000 7fbdd201b740 1 --2- 172.21.15.2:0/370350409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63681b0 0x55c2b6709a60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.365+0000 7fbdd201b740 1 -- 172.21.15.2:0/370350409 >> 172.21.15.2:0/370350409 conn(0x55c2b635c300 msgr2=0x55c2b67053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.365+0000 7fbdd201b740 1 -- 172.21.15.2:0/370350409 shutdown_connections 2024-05-14T09:35:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.365+0000 7fbdd201b740 1 -- 172.21.15.2:0/370350409 wait complete. 2024-05-14T09:35:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.366+0000 7fbdd201b740 1 -- start start 2024-05-14T09:35:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.366+0000 7fbdd201b740 1 Processor -- start 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.366+0000 7fbdd201b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b633f2f0 0x55c2b67136d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.366+0000 7fbdd201b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63474d0 0x55c2b6713c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.366+0000 7fbdd201b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63681b0 0x55c2b671ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.366+0000 7fbdd201b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c2b6711930 con 0x55c2b63681b0 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.366+0000 7fbdd201b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c2b662b370 con 0x55c2b633f2f0 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.366+0000 7fbdd201b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c2b6711ab0 con 0x55c2b63474d0 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.366+0000 7fbdd201b740 10 client.0 _ll_register_callbacks cb 0x55c2b637fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.366+0000 7fbdc9709640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63681b0 0x55c2b671ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.366+0000 7fbdc9709640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63681b0 0x55c2b671ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54890/0 (socket says 172.21.15.2:54890) 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.366+0000 7fbdc9709640 1 -- 172.21.15.2:0/2039518047 learned_addr learned my addr 172.21.15.2:0/2039518047 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.367+0000 7fbdc9709640 1 --2- 172.21.15.2:0/2039518047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b633f2f0 0x55c2b67136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.367+0000 7fbdbbfff640 1 --2- 172.21.15.2:0/2039518047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63474d0 0x55c2b6713c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.367+0000 7fbdc9709640 1 -- 172.21.15.2:0/2039518047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63474d0 msgr2=0x55c2b6713c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.367+0000 7fbdc9709640 1 --2- 172.21.15.2:0/2039518047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63474d0 0x55c2b6713c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.367+0000 7fbdc9709640 1 -- 172.21.15.2:0/2039518047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b633f2f0 msgr2=0x55c2b67136d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.367+0000 7fbdc9709640 1 --2- 172.21.15.2:0/2039518047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b633f2f0 0x55c2b67136d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.367+0000 7fbdc9709640 1 -- 172.21.15.2:0/2039518047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2b644c620 con 0x55c2b63681b0 2024-05-14T09:35:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.367+0000 7fbdc9709640 1 --2- 172.21.15.2:0/2039518047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63681b0 0x55c2b671ab70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009fd0 tx=0x7fbdbc0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd289828739c8b05 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.367+0000 7fbda3fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.367+0000 7fbda3fff640 1 -- 172.21.15.2:0/2039518047 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc023040 con 0x55c2b63681b0 2024-05-14T09:35:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.367+0000 7fbda3fff640 1 -- 172.21.15.2:0/2039518047 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fbdbc004a40 con 0x55c2b63681b0 2024-05-14T09:35:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.367+0000 7fbdd201b740 1 -- 172.21.15.2:0/2039518047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdbc015070 con 0x55c2b63681b0 2024-05-14T09:35:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.367+0000 7fbda3fff640 1 -- 172.21.15.2:0/2039518047 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbdbc031050 con 0x55c2b63681b0 2024-05-14T09:35:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.367+0000 7fbdd201b740 10 client.15366 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.367+0000 7fbdd201b740 1 -- 172.21.15.2:0/2039518047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2b671daa0 con 0x55c2b63681b0 2024-05-14T09:35:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.368+0000 7fbda3fff640 1 -- 172.21.15.2:0/2039518047 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdbc015070 con 0x55c2b63681b0 2024-05-14T09:35:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.368+0000 7fbda3fff640 1 -- 172.21.15.2:0/2039518047 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbdbc01a470 con 0x55c2b63681b0 2024-05-14T09:35:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.368+0000 7fbdd201b740 10 client.15366 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.368+0000 7fbdd201b740 10 client.15366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.368+0000 7fbdd201b740 20 client.15366 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.368+0000 7fbdd201b740 1 --2- 172.21.15.2:0/2039518047 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c2b6326d80 0x55c2b6720fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.368+0000 7fbdd201b740 4 client.15366 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:35:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.368+0000 7fbdd201b740 1 -- 172.21.15.2:0/2039518047 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2b667c1e0 con 0x55c2b6326d80 2024-05-14T09:35:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.368+0000 7fbdbbfff640 1 --2- 172.21.15.2:0/2039518047 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c2b6326d80 0x55c2b6720fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.369+0000 7fbdbbfff640 1 --2- 172.21.15.2:0/2039518047 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c2b6326d80 0x55c2b6720fc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c2adfae54dcbb8d server_cookie=a75fbcb85f416e0f in_seq=0 out_seq=0 2024-05-14T09:35:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.369+0000 7fbda3fff640 10 client.15366 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.369+0000 7fbda3fff640 1 -- 172.21.15.2:0/2039518047 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c2b667c1e0 con 0x55c2b6326d80 2024-05-14T09:35:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.369+0000 7fbda3fff640 10 client.15366 handle_command_reply: tid=0 2024-05-14T09:35:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.370+0000 7fbdd201b740 1 client.15366 shutdown 2024-05-14T09:35:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.370+0000 7fbdd201b740 1 -- 172.21.15.2:0/2039518047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63681b0 msgr2=0x55c2b671ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.370+0000 7fbdd201b740 1 --2- 172.21.15.2:0/2039518047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63681b0 0x55c2b671ab70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009fd0 tx=0x7fbdbc0096a0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.370+0000 7fbdd201b740 1 -- 172.21.15.2:0/2039518047 shutdown_connections 2024-05-14T09:35:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.370+0000 7fbdd201b740 1 --2- 172.21.15.2:0/2039518047 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c2b6326d80 0x55c2b6720fc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.370+0000 7fbdd201b740 1 --2- 172.21.15.2:0/2039518047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63681b0 0x55c2b671ab70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.370+0000 7fbdd201b740 1 --2- 172.21.15.2:0/2039518047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63474d0 0x55c2b6713c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.370+0000 7fbda3fff640 0 client.15366 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.370+0000 7fbdd201b740 1 --2- 172.21.15.2:0/2039518047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b633f2f0 0x55c2b67136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.370+0000 7fbdd201b740 1 -- 172.21.15.2:0/2039518047 >> 172.21.15.2:0/2039518047 conn(0x55c2b635c300 msgr2=0x55c2b67043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.370+0000 7fbdd201b740 1 -- 172.21.15.2:0/2039518047 shutdown_connections 2024-05-14T09:35:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.370+0000 7fbdd201b740 1 -- 172.21.15.2:0/2039518047 wait complete. 2024-05-14T09:35:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.370+0000 7fbdd201b740 20 client.15366 trim_cache size 0 max 16384 2024-05-14T09:35:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.385+0000 7fbdd201b740 1 Processor -- start 2024-05-14T09:35:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.385+0000 7fbdd201b740 1 -- start start 2024-05-14T09:35:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.385+0000 7fbdd201b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b633f2f0 0x55c2b6705400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.385+0000 7fbdd201b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63474d0 0x55c2b671aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.385+0000 7fbdd201b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b63681b0 0x55c2b671afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.385+0000 7fbdd201b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41b1750 con 0x55c2b633f2f0 2024-05-14T09:35:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.385+0000 7fbdd201b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41af020 con 0x55c2b63681b0 2024-05-14T09:35:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.385+0000 7fbdd201b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41ace70 con 0x55c2b63474d0 2024-05-14T09:35:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.385+0000 7fbdc9709640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b633f2f0 0x55c2b6705400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.386+0000 7fbdc9709640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b633f2f0 0x55c2b6705400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54896/0 (socket says 172.21.15.2:54896) 2024-05-14T09:35:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.386+0000 7fbdc9709640 1 -- 172.21.15.2:0/1630378081 learned_addr learned my addr 172.21.15.2:0/1630378081 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.386+0000 7fbdbbfff640 1 --2- 172.21.15.2:0/1630378081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63474d0 0x55c2b671aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.386+0000 7fbdc9709640 1 -- 172.21.15.2:0/1630378081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63474d0 msgr2=0x55c2b671aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.386+0000 7fbdc9709640 1 --2- 172.21.15.2:0/1630378081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63474d0 0x55c2b671aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.386+0000 7fbdc9709640 1 -- 172.21.15.2:0/1630378081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b63681b0 msgr2=0x55c2b671afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.386+0000 7fbdc9709640 1 --2- 172.21.15.2:0/1630378081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b63681b0 0x55c2b671afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.386+0000 7fbdc9709640 1 -- 172.21.15.2:0/1630378081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2b6705940 con 0x55c2b633f2f0 2024-05-14T09:35:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.386+0000 7fbdbbfff640 1 --2- 172.21.15.2:0/1630378081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63474d0 0x55c2b671aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:35:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.386+0000 7fbdc9709640 1 --2- 172.21.15.2:0/1630378081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b633f2f0 0x55c2b6705400 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0104f0 tx=0x7fbdbc01aa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ae9fd68c454d58 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.386+0000 7fbdaa7fc640 1 -- 172.21.15.2:0/1630378081 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc01ab30 con 0x55c2b633f2f0 2024-05-14T09:35:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.386+0000 7fbdaa7fc640 1 -- 172.21.15.2:0/1630378081 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdbc01ae50 con 0x55c2b633f2f0 2024-05-14T09:35:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.386+0000 7fbdaa7fc640 1 -- 172.21.15.2:0/1630378081 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc011a30 con 0x55c2b633f2f0 2024-05-14T09:35:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.387+0000 7fbdd201b740 1 -- 172.21.15.2:0/1630378081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b633f2f0 msgr2=0x55c2b6705400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.387+0000 7fbdd201b740 1 --2- 172.21.15.2:0/1630378081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b633f2f0 0x55c2b6705400 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0104f0 tx=0x7fbdbc01aa30 comp rx=0 tx=0).stop 2024-05-14T09:35:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.387+0000 7fbdd201b740 1 -- 172.21.15.2:0/1630378081 shutdown_connections 2024-05-14T09:35:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.387+0000 7fbdd201b740 1 --2- 172.21.15.2:0/1630378081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b63681b0 0x55c2b671afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.387+0000 7fbdd201b740 1 --2- 172.21.15.2:0/1630378081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63474d0 0x55c2b671aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.387+0000 7fbdd201b740 1 --2- 172.21.15.2:0/1630378081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b633f2f0 0x55c2b6705400 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.387+0000 7fbdd201b740 1 -- 172.21.15.2:0/1630378081 >> 172.21.15.2:0/1630378081 conn(0x55c2b635c300 msgr2=0x55c2b6704cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.387+0000 7fbdd201b740 1 -- 172.21.15.2:0/1630378081 shutdown_connections 2024-05-14T09:35:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.387+0000 7fbdd201b740 1 -- 172.21.15.2:0/1630378081 wait complete. 2024-05-14T09:35:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.387+0000 7fbdd201b740 1 -- start start 2024-05-14T09:35:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.387+0000 7fbdd201b740 1 Processor -- start 2024-05-14T09:35:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdd201b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b633f2f0 0x55c2b6718b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdd201b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63474d0 0x55c2b6719090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdd201b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63681b0 0x55c2b6719fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdd201b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41ad4e0 con 0x55c2b63474d0 2024-05-14T09:35:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdd201b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41ae950 con 0x55c2b633f2f0 2024-05-14T09:35:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdd201b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41adbb0 con 0x55c2b63681b0 2024-05-14T09:35:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdd201b740 10 client.0 _ll_register_callbacks cb 0x7fbdbc009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdbbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63474d0 0x55c2b6719090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdbbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63474d0 0x55c2b6719090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54902/0 (socket says 172.21.15.2:54902) 2024-05-14T09:35:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdbbfff640 1 -- 172.21.15.2:0/3040189113 learned_addr learned my addr 172.21.15.2:0/3040189113 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdc9709640 1 --2- 172.21.15.2:0/3040189113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63681b0 0x55c2b6719fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdc9709640 1 --2- 172.21.15.2:0/3040189113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b633f2f0 0x55c2b6718b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdbbfff640 1 -- 172.21.15.2:0/3040189113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63681b0 msgr2=0x55c2b6719fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdbbfff640 1 --2- 172.21.15.2:0/3040189113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63681b0 0x55c2b6719fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdbbfff640 1 -- 172.21.15.2:0/3040189113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b633f2f0 msgr2=0x55c2b6718b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdbbfff640 1 --2- 172.21.15.2:0/3040189113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b633f2f0 0x55c2b6718b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdbbfff640 1 -- 172.21.15.2:0/3040189113 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2b671a580 con 0x55c2b63474d0 2024-05-14T09:35:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.388+0000 7fbdc9709640 1 --2- 172.21.15.2:0/3040189113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b633f2f0 0x55c2b6718b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbdbbfff640 1 --2- 172.21.15.2:0/3040189113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63474d0 0x55c2b6719090 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4000c00 tx=0x7fbdb4004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5807274c9b2ee281 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbda3fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbda3fff640 1 -- 172.21.15.2:0/3040189113 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4035040 con 0x55c2b63474d0 2024-05-14T09:35:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbdd201b740 1 -- 172.21.15.2:0/3040189113 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdbc01ae50 con 0x55c2b63474d0 2024-05-14T09:35:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbda3fff640 1 -- 172.21.15.2:0/3040189113 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fbdb4005240 con 0x55c2b63474d0 2024-05-14T09:35:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbda3fff640 1 -- 172.21.15.2:0/3040189113 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbdb4051050 con 0x55c2b63474d0 2024-05-14T09:35:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbdd201b740 10 client.15372 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbdd201b740 1 -- 172.21.15.2:0/3040189113 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2b6789350 con 0x55c2b63474d0 2024-05-14T09:35:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbda3fff640 1 -- 172.21.15.2:0/3040189113 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdbc01ae50 con 0x55c2b63474d0 2024-05-14T09:35:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbda3fff640 1 -- 172.21.15.2:0/3040189113 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbdb404c030 con 0x55c2b63474d0 2024-05-14T09:35:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbdd201b740 10 client.15372 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbdd201b740 10 client.15372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbdd201b740 20 client.15372 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbdd201b740 1 --2- 172.21.15.2:0/3040189113 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c2b62fc1c0 0x55c2b678bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbdd201b740 4 client.15372 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:35:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.389+0000 7fbdd201b740 1 -- 172.21.15.2:0/3040189113 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2b6676050 con 0x55c2b62fc1c0 2024-05-14T09:35:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.390+0000 7fbdc9709640 1 --2- 172.21.15.2:0/3040189113 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c2b62fc1c0 0x55c2b678bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.390+0000 7fbda3fff640 10 client.15372 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.390+0000 7fbdc9709640 1 --2- 172.21.15.2:0/3040189113 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c2b62fc1c0 0x55c2b678bd70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5be336bfd4b3785a server_cookie=9bd145c92136b41f in_seq=0 out_seq=0 2024-05-14T09:35:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.405+0000 7fbda3fff640 1 -- 172.21.15.2:0/3040189113 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2b6676050 con 0x55c2b62fc1c0 2024-05-14T09:35:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.405+0000 7fbda3fff640 10 client.15372 handle_command_reply: tid=0 2024-05-14T09:35:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.405+0000 7fbdd201b740 1 client.15372 shutdown 2024-05-14T09:35:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.405+0000 7fbdd201b740 1 -- 172.21.15.2:0/3040189113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63474d0 msgr2=0x55c2b6719090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.405+0000 7fbdd201b740 1 --2- 172.21.15.2:0/3040189113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63474d0 0x55c2b6719090 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4000c00 tx=0x7fbdb4004130 comp rx=0 tx=0).stop 2024-05-14T09:35:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.405+0000 7fbdd201b740 1 -- 172.21.15.2:0/3040189113 shutdown_connections 2024-05-14T09:35:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.405+0000 7fbdd201b740 1 --2- 172.21.15.2:0/3040189113 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c2b62fc1c0 0x55c2b678bd70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.405+0000 7fbdd201b740 1 --2- 172.21.15.2:0/3040189113 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c2b63681b0 0x55c2b6719fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.405+0000 7fbdd201b740 1 --2- 172.21.15.2:0/3040189113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b63474d0 0x55c2b6719090 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.405+0000 7fbdd201b740 1 --2- 172.21.15.2:0/3040189113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2b633f2f0 0x55c2b6718b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.405+0000 7fbda3fff640 0 client.15372 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.405+0000 7fbdd201b740 1 -- 172.21.15.2:0/3040189113 >> 172.21.15.2:0/3040189113 conn(0x55c2b635c300 msgr2=0x55c2b6704cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.406+0000 7fbdd201b740 1 -- 172.21.15.2:0/3040189113 shutdown_connections 2024-05-14T09:35:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.406+0000 7fbdd201b740 1 -- 172.21.15.2:0/3040189113 wait complete. 2024-05-14T09:35:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.406+0000 7fbdd201b740 20 client.15372 trim_cache size 0 max 16384 2024-05-14T09:35:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:35:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:35:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:35:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:35:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.406+0000 7fbdcb16c640 1 -- 172.21.15.2:0/3302738093 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbdac0b28a0 msgr2=0x7fbdac0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.406+0000 7fbdcb16c640 1 --2- 172.21.15.2:0/3302738093 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbdac0b28a0 0x7fbdac0b4db0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55c2b670ae40 tx=0x7fbdc0000bc0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.406+0000 7fbdcb16c640 1 -- 172.21.15.2:0/3302738093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc4103c60 msgr2=0x7fbdc4077a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.406+0000 7fbdcb16c640 1 --2- 172.21.15.2:0/3302738093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc4103c60 0x7fbdc4077a90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000e9f0 tx=0x7fbdc000eeb0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.406+0000 7fbdcb16c640 1 -- 172.21.15.2:0/3302738093 shutdown_connections 2024-05-14T09:35:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.406+0000 7fbdcb16c640 1 --2- 172.21.15.2:0/3302738093 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbdac0b28a0 0x7fbdac0b4db0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.406+0000 7fbdcb16c640 1 --2- 172.21.15.2:0/3302738093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbdc4109ba0 0x7fbdc4076670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.406+0000 7fbdcb16c640 1 --2- 172.21.15.2:0/3302738093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdc4104ea0 0x7fbdc4076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.406+0000 7fbdcb16c640 1 --2- 172.21.15.2:0/3302738093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc4103c60 0x7fbdc4077a90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.406+0000 7fbdcb16c640 1 -- 172.21.15.2:0/3302738093 >> 172.21.15.2:0/3302738093 conn(0x7fbdc40fee70 msgr2=0x7fbdc4105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.407+0000 7fbdcb16c640 1 -- 172.21.15.2:0/3302738093 shutdown_connections 2024-05-14T09:35:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:15.407+0000 7fbdcb16c640 1 -- 172.21.15.2:0/3302738093 wait complete. 2024-05-14T09:35:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:16 smithi002 ceph-mon[31557]: pgmap v433: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-14T09:35:16.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:16 smithi029 ceph-mon[36249]: pgmap v433: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-14T09:35:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:16 smithi026 ceph-mon[36056]: pgmap v433: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-14T09:35:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:18 smithi002 ceph-mon[31557]: pgmap v434: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-14T09:35:18.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:18 smithi029 ceph-mon[36249]: pgmap v434: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-14T09:35:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:18 smithi026 ceph-mon[36056]: pgmap v434: 129 pgs: 129 active+clean; 654 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-14T09:35:19.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:19 smithi002 ceph-mon[31557]: scrub summary: idle 2024-05-14T09:35:19.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:19 smithi029 ceph-mon[36249]: scrub summary: idle 2024-05-14T09:35:19.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:19 smithi026 ceph-mon[36056]: scrub summary: idle 2024-05-14T09:35:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:20 smithi002 ceph-mon[31557]: pgmap v435: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-14T09:35:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:20 smithi029 ceph-mon[36249]: pgmap v435: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-14T09:35:20.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:20 smithi026 ceph-mon[36056]: pgmap v435: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-14T09:35:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:22 smithi002 ceph-mon[31557]: pgmap v436: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-14T09:35:22.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:22 smithi029 ceph-mon[36249]: pgmap v436: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-14T09:35:22.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:22 smithi026 ceph-mon[36056]: pgmap v436: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-14T09:35:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:24 smithi002 ceph-mon[31557]: pgmap v437: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-14T09:35:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:24 smithi029 ceph-mon[36249]: pgmap v437: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-14T09:35:24.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:24 smithi026 ceph-mon[36056]: pgmap v437: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-14T09:35:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:26 smithi002 ceph-mon[31557]: pgmap v438: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-14T09:35:26.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:26 smithi029 ceph-mon[36249]: pgmap v438: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-14T09:35:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:26 smithi026 ceph-mon[36056]: pgmap v438: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-14T09:35:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.520+0000 7fd484c27640 1 -- 172.21.15.2:0/2976790191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd480109ba0 msgr2=0x7fd48010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.520+0000 7fd484c27640 1 --2- 172.21.15.2:0/2976790191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd480109ba0 0x7fd48010bfc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd470009900 tx=0x7fd47002a410 comp rx=0 tx=0).stop 2024-05-14T09:35:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.521+0000 7fd484c27640 1 -- 172.21.15.2:0/2976790191 shutdown_connections 2024-05-14T09:35:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.521+0000 7fd484c27640 1 --2- 172.21.15.2:0/2976790191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd480109ba0 0x7fd48010bfc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.521+0000 7fd484c27640 1 --2- 172.21.15.2:0/2976790191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd480104ea0 0x7fd4801052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.521+0000 7fd484c27640 1 --2- 172.21.15.2:0/2976790191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd480103c60 0x7fd480104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.521+0000 7fd484c27640 1 -- 172.21.15.2:0/2976790191 >> 172.21.15.2:0/2976790191 conn(0x7fd4800fee70 msgr2=0x7fd480101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.521+0000 7fd484c27640 1 -- 172.21.15.2:0/2976790191 shutdown_connections 2024-05-14T09:35:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.521+0000 7fd484c27640 1 -- 172.21.15.2:0/2976790191 wait complete. 2024-05-14T09:35:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.521+0000 7fd484c27640 1 Processor -- start 2024-05-14T09:35:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd484c27640 1 -- start start 2024-05-14T09:35:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd484c27640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd480103c60 0x7fd48019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd484c27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd480104ea0 0x7fd48019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd484c27640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd480109ba0 0x7fd4801a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd484c27640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd48010f790 con 0x7fd480109ba0 2024-05-14T09:35:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd484c27640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd48010f610 con 0x7fd480104ea0 2024-05-14T09:35:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd484c27640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd48010f910 con 0x7fd480103c60 2024-05-14T09:35:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd47ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd480109ba0 0x7fd4801a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd47ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd480109ba0 0x7fd4801a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48856/0 (socket says 172.21.15.2:48856) 2024-05-14T09:35:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd47ed9d640 1 -- 172.21.15.2:0/1860491479 learned_addr learned my addr 172.21.15.2:0/1860491479 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd47dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd480104ea0 0x7fd48019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd47e59c640 1 --2- 172.21.15.2:0/1860491479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd480103c60 0x7fd48019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd47ed9d640 1 -- 172.21.15.2:0/1860491479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd480103c60 msgr2=0x7fd48019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd47ed9d640 1 --2- 172.21.15.2:0/1860491479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd480103c60 0x7fd48019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd47ed9d640 1 -- 172.21.15.2:0/1860491479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd480104ea0 msgr2=0x7fd48019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd47ed9d640 1 --2- 172.21.15.2:0/1860491479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd480104ea0 0x7fd48019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.522+0000 7fd47ed9d640 1 -- 172.21.15.2:0/1860491479 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4801a1dc0 con 0x7fd480109ba0 2024-05-14T09:35:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.523+0000 7fd47ed9d640 1 --2- 172.21.15.2:0/1860491479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd480109ba0 0x7fd4801a1750 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd470009230 tx=0x7fd4700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.523+0000 7fd4777fe640 1 -- 172.21.15.2:0/1860491479 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47003f030 con 0x7fd480109ba0 2024-05-14T09:35:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.523+0000 7fd4777fe640 1 -- 172.21.15.2:0/1860491479 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd470031910 con 0x7fd480109ba0 2024-05-14T09:35:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.523+0000 7fd4777fe640 1 -- 172.21.15.2:0/1860491479 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470035070 con 0x7fd480109ba0 2024-05-14T09:35:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.523+0000 7fd484c27640 1 -- 172.21.15.2:0/1860491479 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4801a2050 con 0x7fd480109ba0 2024-05-14T09:35:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.523+0000 7fd484c27640 1 -- 172.21.15.2:0/1860491479 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4801a2540 con 0x7fd480109ba0 2024-05-14T09:35:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.526+0000 7fd48bbbb740 1 Processor -- start 2024-05-14T09:35:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.526+0000 7fd4777fe640 1 -- 172.21.15.2:0/1860491479 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd470042030 con 0x7fd480109ba0 2024-05-14T09:35:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.526+0000 7fd48bbbb740 1 -- start start 2024-05-14T09:35:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd48bbbb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623ab91b0 0x55b623e5aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd48bbbb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a902f0 0x55b623e5afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd48bbbb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623a984d0 0x55b623e61ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd48bbbb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b623dbf5d0 con 0x55b623a984d0 2024-05-14T09:35:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd48bbbb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b623d78890 con 0x55b623a902f0 2024-05-14T09:35:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd48bbbb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b623dbb590 con 0x55b623ab91b0 2024-05-14T09:35:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd47dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623a984d0 0x55b623e61ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd47dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623a984d0 0x55b623e61ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48870/0 (socket says 172.21.15.2:48870) 2024-05-14T09:35:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd47dd9b640 1 -- 172.21.15.2:0/924708728 learned_addr learned my addr 172.21.15.2:0/924708728 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd47e59c640 1 --2- 172.21.15.2:0/924708728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a902f0 0x55b623e5afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd47dd9b640 1 --2- 172.21.15.2:0/924708728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623ab91b0 0x55b623e5aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd47dd9b640 1 -- 172.21.15.2:0/924708728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623ab91b0 msgr2=0x55b623e5aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd47dd9b640 1 --2- 172.21.15.2:0/924708728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623ab91b0 0x55b623e5aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd47dd9b640 1 -- 172.21.15.2:0/924708728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a902f0 msgr2=0x55b623e5afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd47dd9b640 1 --2- 172.21.15.2:0/924708728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a902f0 0x55b623e5afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd47dd9b640 1 -- 172.21.15.2:0/924708728 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b623ad1210 con 0x55b623a984d0 2024-05-14T09:35:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd4777fe640 1 --2- 172.21.15.2:0/1860491479 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd46c0b28a0 0x7fd46c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.527+0000 7fd47dd9b640 1 --2- 172.21.15.2:0/924708728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623a984d0 0x55b623e61ee0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd46800bfd0 tx=0x7fd46800b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7df70b11d4483ac server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.528+0000 7fd47e59c640 1 --2- 172.21.15.2:0/1860491479 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd46c0b28a0 0x7fd46c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.528+0000 7fd4777fe640 1 -- 172.21.15.2:0/1860491479 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd470102e40 con 0x7fd480109ba0 2024-05-14T09:35:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.528+0000 7fd474ff9640 1 -- 172.21.15.2:0/924708728 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468012d20 con 0x55b623a984d0 2024-05-14T09:35:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.528+0000 7fd474ff9640 1 -- 172.21.15.2:0/924708728 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd468010740 con 0x55b623a984d0 2024-05-14T09:35:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.528+0000 7fd474ff9640 1 -- 172.21.15.2:0/924708728 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468003830 con 0x55b623a984d0 2024-05-14T09:35:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.529+0000 7fd48bbbb740 1 -- 172.21.15.2:0/924708728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623a984d0 msgr2=0x55b623e61ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.529+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/924708728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623a984d0 0x55b623e61ee0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd46800bfd0 tx=0x7fd46800b650 comp rx=0 tx=0).stop 2024-05-14T09:35:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.529+0000 7fd47e59c640 1 --2- 172.21.15.2:0/1860491479 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd46c0b28a0 0x7fd46c0b4db0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55b623e5be40 tx=0x7fd478009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.529+0000 7fd48bbbb740 1 -- 172.21.15.2:0/924708728 shutdown_connections 2024-05-14T09:35:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.529+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/924708728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623a984d0 0x55b623e61ee0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.529+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/924708728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a902f0 0x55b623e5afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.529+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/924708728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623ab91b0 0x55b623e5aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.529+0000 7fd48bbbb740 1 -- 172.21.15.2:0/924708728 >> 172.21.15.2:0/924708728 conn(0x55b623aad300 msgr2=0x55b623e563a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.529+0000 7fd48bbbb740 1 -- 172.21.15.2:0/924708728 shutdown_connections 2024-05-14T09:35:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.529+0000 7fd48bbbb740 1 -- 172.21.15.2:0/924708728 wait complete. 2024-05-14T09:35:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.529+0000 7fd48bbbb740 1 -- start start 2024-05-14T09:35:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.529+0000 7fd48bbbb740 1 Processor -- start 2024-05-14T09:35:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd48bbbb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623a902f0 0x55b623e64760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd48bbbb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a984d0 0x55b623e64ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd48bbbb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623ab91b0 0x55b623e6bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd48bbbb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b623e62930 con 0x55b623ab91b0 2024-05-14T09:35:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd48bbbb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b623d7c370 con 0x55b623a984d0 2024-05-14T09:35:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd48bbbb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b623e62ab0 con 0x55b623a902f0 2024-05-14T09:35:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd48bbbb740 10 client.0 _ll_register_callbacks cb 0x55b623ad0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd47ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623ab91b0 0x55b623e6bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd47ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623ab91b0 0x55b623e6bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48884/0 (socket says 172.21.15.2:48884) 2024-05-14T09:35:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd47ed9d640 1 -- 172.21.15.2:0/2293001634 learned_addr learned my addr 172.21.15.2:0/2293001634 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd47e59c640 1 --2- 172.21.15.2:0/2293001634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623a902f0 0x55b623e64760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd47dd9b640 1 --2- 172.21.15.2:0/2293001634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a984d0 0x55b623e64ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd47ed9d640 1 -- 172.21.15.2:0/2293001634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623a902f0 msgr2=0x55b623e64760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd47ed9d640 1 --2- 172.21.15.2:0/2293001634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623a902f0 0x55b623e64760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd47ed9d640 1 -- 172.21.15.2:0/2293001634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a984d0 msgr2=0x55b623e64ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd47ed9d640 1 --2- 172.21.15.2:0/2293001634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a984d0 0x55b623e64ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd47ed9d640 1 -- 172.21.15.2:0/2293001634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b623b9d620 con 0x55b623ab91b0 2024-05-14T09:35:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.530+0000 7fd47dd9b640 1 --2- 172.21.15.2:0/2293001634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a984d0 0x55b623e64ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd47ed9d640 1 --2- 172.21.15.2:0/2293001634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623ab91b0 0x55b623e6bc00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd4700c9df0 tx=0x7fd4700c9b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1dedbd0550dd5d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd465ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd465ffb640 1 -- 172.21.15.2:0/2293001634 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470049070 con 0x55b623ab91b0 2024-05-14T09:35:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd465ffb640 1 -- 172.21.15.2:0/2293001634 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd4700ce0d0 con 0x55b623ab91b0 2024-05-14T09:35:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd48bbbb740 1 -- 172.21.15.2:0/2293001634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd468010740 con 0x55b623ab91b0 2024-05-14T09:35:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd465ffb640 1 -- 172.21.15.2:0/2293001634 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd4700ce3f0 con 0x55b623ab91b0 2024-05-14T09:35:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd48bbbb740 10 client.15384 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd48bbbb740 1 -- 172.21.15.2:0/2293001634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b623e6eb30 con 0x55b623ab91b0 2024-05-14T09:35:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd465ffb640 1 -- 172.21.15.2:0/2293001634 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd468010740 con 0x55b623ab91b0 2024-05-14T09:35:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd465ffb640 1 -- 172.21.15.2:0/2293001634 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd4700ec050 con 0x55b623ab91b0 2024-05-14T09:35:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd48bbbb740 10 client.15384 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd48bbbb740 10 client.15384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd48bbbb740 20 client.15384 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/2293001634 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b623a77d80 0x55b623e72280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd48bbbb740 4 client.15384 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:35:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.531+0000 7fd48bbbb740 1 -- 172.21.15.2:0/2293001634 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b623dcd1e0 con 0x55b623a77d80 2024-05-14T09:35:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.532+0000 7fd47e59c640 1 --2- 172.21.15.2:0/2293001634 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b623a77d80 0x55b623e72280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.532+0000 7fd47e59c640 1 --2- 172.21.15.2:0/2293001634 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b623a77d80 0x55b623e72280 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=274fd1f035812aff server_cookie=6a9172b7bbc0c519 in_seq=0 out_seq=0 2024-05-14T09:35:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.532+0000 7fd465ffb640 10 client.15384 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.533+0000 7fd465ffb640 1 -- 172.21.15.2:0/2293001634 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b623dcd1e0 con 0x55b623a77d80 2024-05-14T09:35:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.533+0000 7fd465ffb640 10 client.15384 handle_command_reply: tid=0 2024-05-14T09:35:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.533+0000 7fd48bbbb740 1 client.15384 shutdown 2024-05-14T09:35:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.533+0000 7fd48bbbb740 1 -- 172.21.15.2:0/2293001634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623ab91b0 msgr2=0x55b623e6bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.533+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/2293001634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623ab91b0 0x55b623e6bc00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd4700c9df0 tx=0x7fd4700c9b00 comp rx=0 tx=0).stop 2024-05-14T09:35:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.533+0000 7fd48bbbb740 1 -- 172.21.15.2:0/2293001634 shutdown_connections 2024-05-14T09:35:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.533+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/2293001634 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b623a77d80 0x55b623e72280 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.533+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/2293001634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623ab91b0 0x55b623e6bc00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.533+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/2293001634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a984d0 0x55b623e64ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.533+0000 7fd465ffb640 0 client.15384 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.533+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/2293001634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623a902f0 0x55b623e64760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.533+0000 7fd48bbbb740 1 -- 172.21.15.2:0/2293001634 >> 172.21.15.2:0/2293001634 conn(0x55b623aad300 msgr2=0x55b623e59300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.533+0000 7fd48bbbb740 1 -- 172.21.15.2:0/2293001634 shutdown_connections 2024-05-14T09:35:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.533+0000 7fd48bbbb740 1 -- 172.21.15.2:0/2293001634 wait complete. 2024-05-14T09:35:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.533+0000 7fd48bbbb740 20 client.15384 trim_cache size 0 max 16384 2024-05-14T09:35:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd48bbbb740 1 Processor -- start 2024-05-14T09:35:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd48bbbb740 1 -- start start 2024-05-14T09:35:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd48bbbb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623a902f0 0x55b623e50f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd48bbbb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623a984d0 0x55b623e6bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd48bbbb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623ab91b0 0x55b623e55c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd48bbbb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801af480 con 0x55b623a902f0 2024-05-14T09:35:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd48bbbb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801acd50 con 0x55b623ab91b0 2024-05-14T09:35:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd48bbbb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801aaba0 con 0x55b623a984d0 2024-05-14T09:35:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd47e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623a902f0 0x55b623e50f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd47e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623a902f0 0x55b623e50f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48898/0 (socket says 172.21.15.2:48898) 2024-05-14T09:35:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd47e59c640 1 -- 172.21.15.2:0/2292074461 learned_addr learned my addr 172.21.15.2:0/2292074461 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd47e59c640 1 -- 172.21.15.2:0/2292074461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623a984d0 msgr2=0x55b623e6bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd47dd9b640 1 --2- 172.21.15.2:0/2292074461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623a984d0 0x55b623e6bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd47e59c640 1 --2- 172.21.15.2:0/2292074461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623a984d0 0x55b623e6bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd47ed9d640 1 --2- 172.21.15.2:0/2292074461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623ab91b0 0x55b623e55c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd47e59c640 1 -- 172.21.15.2:0/2292074461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623ab91b0 msgr2=0x55b623e55c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd47e59c640 1 --2- 172.21.15.2:0/2292074461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623ab91b0 0x55b623e55c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.549+0000 7fd47e59c640 1 -- 172.21.15.2:0/2292074461 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b623e56d00 con 0x55b623a902f0 2024-05-14T09:35:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.550+0000 7fd47e59c640 1 --2- 172.21.15.2:0/2292074461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623a902f0 0x55b623e50f10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd480103530 tx=0x7fd47801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f64cf2069234bd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.550+0000 7fd467fff640 1 -- 172.21.15.2:0/2292074461 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47802a4e0 con 0x55b623a902f0 2024-05-14T09:35:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.550+0000 7fd467fff640 1 -- 172.21.15.2:0/2292074461 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47802f070 con 0x55b623a902f0 2024-05-14T09:35:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.550+0000 7fd467fff640 1 -- 172.21.15.2:0/2292074461 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478004860 con 0x55b623a902f0 2024-05-14T09:35:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.550+0000 7fd48bbbb740 1 -- 172.21.15.2:0/2292074461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623a902f0 msgr2=0x55b623e50f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.550+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/2292074461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623a902f0 0x55b623e50f10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd480103530 tx=0x7fd47801a000 comp rx=0 tx=0).stop 2024-05-14T09:35:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 1 -- 172.21.15.2:0/2292074461 shutdown_connections 2024-05-14T09:35:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/2292074461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623ab91b0 0x55b623e55c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/2292074461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623a984d0 0x55b623e6bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/2292074461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623a902f0 0x55b623e50f10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 1 -- 172.21.15.2:0/2292074461 >> 172.21.15.2:0/2292074461 conn(0x55b623aad300 msgr2=0x55b623e5eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 1 -- 172.21.15.2:0/2292074461 shutdown_connections 2024-05-14T09:35:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 1 -- 172.21.15.2:0/2292074461 wait complete. 2024-05-14T09:35:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 1 -- start start 2024-05-14T09:35:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 1 Processor -- start 2024-05-14T09:35:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a902f0 0x55b623e69260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623a984d0 0x55b623e697a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623ab91b0 0x55b623e6a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801ab210 con 0x55b623ab91b0 2024-05-14T09:35:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801ac680 con 0x55b623a902f0 2024-05-14T09:35:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801ab8e0 con 0x55b623a984d0 2024-05-14T09:35:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd48bbbb740 10 client.0 _ll_register_callbacks cb 0x7fd47003b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd47ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623ab91b0 0x55b623e6a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd47ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623ab91b0 0x55b623e6a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48912/0 (socket says 172.21.15.2:48912) 2024-05-14T09:35:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd47ed9d640 1 -- 172.21.15.2:0/1238004649 learned_addr learned my addr 172.21.15.2:0/1238004649 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.551+0000 7fd47dd9b640 1 --2- 172.21.15.2:0/1238004649 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623a984d0 0x55b623e697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.552+0000 7fd47e59c640 1 --2- 172.21.15.2:0/1238004649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a902f0 0x55b623e69260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.552+0000 7fd47ed9d640 1 -- 172.21.15.2:0/1238004649 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623a984d0 msgr2=0x55b623e697a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.552+0000 7fd47ed9d640 1 --2- 172.21.15.2:0/1238004649 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623a984d0 0x55b623e697a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.552+0000 7fd47ed9d640 1 -- 172.21.15.2:0/1238004649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a902f0 msgr2=0x55b623e69260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.552+0000 7fd47ed9d640 1 --2- 172.21.15.2:0/1238004649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a902f0 0x55b623e69260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.552+0000 7fd47ed9d640 1 -- 172.21.15.2:0/1238004649 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b623e6ac90 con 0x55b623ab91b0 2024-05-14T09:35:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.552+0000 7fd47ed9d640 1 --2- 172.21.15.2:0/1238004649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623ab91b0 0x55b623e6a6d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd4700c7360 tx=0x7fd470100c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2973318fede5b33 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.552+0000 7fd465ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.552+0000 7fd465ffb640 1 -- 172.21.15.2:0/1238004649 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4700eb030 con 0x55b623ab91b0 2024-05-14T09:35:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.552+0000 7fd465ffb640 1 -- 172.21.15.2:0/1238004649 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd4700c9350 con 0x55b623ab91b0 2024-05-14T09:35:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.552+0000 7fd48bbbb740 1 -- 172.21.15.2:0/1238004649 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd47802f070 con 0x55b623ab91b0 2024-05-14T09:35:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.552+0000 7fd465ffb640 1 -- 172.21.15.2:0/1238004649 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd470049050 con 0x55b623ab91b0 2024-05-14T09:35:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.552+0000 7fd48bbbb740 10 client.15390 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.552+0000 7fd48bbbb740 1 -- 172.21.15.2:0/1238004649 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b623eda630 con 0x55b623ab91b0 2024-05-14T09:35:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.552+0000 7fd465ffb640 1 -- 172.21.15.2:0/1238004649 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47802f070 con 0x55b623ab91b0 2024-05-14T09:35:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.553+0000 7fd465ffb640 1 -- 172.21.15.2:0/1238004649 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd4700ec040 con 0x55b623ab91b0 2024-05-14T09:35:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.553+0000 7fd48bbbb740 10 client.15390 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.553+0000 7fd48bbbb740 10 client.15390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.553+0000 7fd48bbbb740 20 client.15390 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.553+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/1238004649 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b623a4d1c0 0x55b623edcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.553+0000 7fd48bbbb740 4 client.15390 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:35:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.553+0000 7fd48bbbb740 1 -- 172.21.15.2:0/1238004649 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b623dc7050 con 0x55b623a4d1c0 2024-05-14T09:35:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.553+0000 7fd47e59c640 1 --2- 172.21.15.2:0/1238004649 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b623a4d1c0 0x55b623edcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.553+0000 7fd47e59c640 1 --2- 172.21.15.2:0/1238004649 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b623a4d1c0 0x55b623edcce0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e68227513ef3dc server_cookie=a1db398b77d37664 in_seq=0 out_seq=0 2024-05-14T09:35:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.553+0000 7fd465ffb640 10 client.15390 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.557+0000 7fd465ffb640 1 -- 172.21.15.2:0/1238004649 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b623dc7050 con 0x55b623a4d1c0 2024-05-14T09:35:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.557+0000 7fd465ffb640 10 client.15390 handle_command_reply: tid=0 2024-05-14T09:35:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.557+0000 7fd48bbbb740 1 client.15390 shutdown 2024-05-14T09:35:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd48bbbb740 1 -- 172.21.15.2:0/1238004649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623ab91b0 msgr2=0x55b623e6a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/1238004649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623ab91b0 0x55b623e6a6d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd4700c7360 tx=0x7fd470100c10 comp rx=0 tx=0).stop 2024-05-14T09:35:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd48bbbb740 1 -- 172.21.15.2:0/1238004649 shutdown_connections 2024-05-14T09:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/1238004649 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b623a4d1c0 0x55b623edcce0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/1238004649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b623ab91b0 0x55b623e6a6d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/1238004649 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b623a984d0 0x55b623e697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd465ffb640 0 client.15390 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd48bbbb740 1 --2- 172.21.15.2:0/1238004649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b623a902f0 0x55b623e69260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd48bbbb740 1 -- 172.21.15.2:0/1238004649 >> 172.21.15.2:0/1238004649 conn(0x55b623aad300 msgr2=0x55b623e591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd48bbbb740 1 -- 172.21.15.2:0/1238004649 shutdown_connections 2024-05-14T09:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd48bbbb740 1 -- 172.21.15.2:0/1238004649 wait complete. 2024-05-14T09:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd48bbbb740 20 client.15390 trim_cache size 0 max 16384 2024-05-14T09:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:35:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:35:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:35:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:35:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd484c27640 1 -- 172.21.15.2:0/1860491479 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd46c0b28a0 msgr2=0x7fd46c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd484c27640 1 --2- 172.21.15.2:0/1860491479 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd46c0b28a0 0x7fd46c0b4db0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55b623e5be40 tx=0x7fd478009500 comp rx=0 tx=0).stop 2024-05-14T09:35:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd484c27640 1 -- 172.21.15.2:0/1860491479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd480109ba0 msgr2=0x7fd4801a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd484c27640 1 --2- 172.21.15.2:0/1860491479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd480109ba0 0x7fd4801a1750 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd470009230 tx=0x7fd4700042f0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd484c27640 1 -- 172.21.15.2:0/1860491479 shutdown_connections 2024-05-14T09:35:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd484c27640 1 --2- 172.21.15.2:0/1860491479 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd46c0b28a0 0x7fd46c0b4db0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd484c27640 1 --2- 172.21.15.2:0/1860491479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd480109ba0 0x7fd4801a1750 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd484c27640 1 --2- 172.21.15.2:0/1860491479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd480104ea0 0x7fd48019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd484c27640 1 --2- 172.21.15.2:0/1860491479 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd480103c60 0x7fd48019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.558+0000 7fd484c27640 1 -- 172.21.15.2:0/1860491479 >> 172.21.15.2:0/1860491479 conn(0x7fd4800fee70 msgr2=0x7fd480105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.559+0000 7fd484c27640 1 -- 172.21.15.2:0/1860491479 shutdown_connections 2024-05-14T09:35:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:28.559+0000 7fd484c27640 1 -- 172.21.15.2:0/1860491479 wait complete. 2024-05-14T09:35:28.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:28 smithi002 ceph-mon[31557]: pgmap v439: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-14T09:35:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:28 smithi029 ceph-mon[36249]: pgmap v439: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-14T09:35:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:28 smithi026 ceph-mon[36056]: pgmap v439: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-14T09:35:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:30 smithi002 ceph-mon[31557]: pgmap v440: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-14T09:35:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:30 smithi029 ceph-mon[36249]: pgmap v440: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-14T09:35:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:30 smithi026 ceph-mon[36056]: pgmap v440: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-14T09:35:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:32 smithi002 ceph-mon[31557]: pgmap v441: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:35:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:32 smithi029 ceph-mon[36249]: pgmap v441: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:35:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:32 smithi026 ceph-mon[36056]: pgmap v441: 129 pgs: 129 active+clean; 655 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:35:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:34 smithi002 ceph-mon[31557]: pgmap v442: 129 pgs: 129 active+clean; 656 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T09:35:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:34 smithi029 ceph-mon[36249]: pgmap v442: 129 pgs: 129 active+clean; 656 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T09:35:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:34 smithi026 ceph-mon[36056]: pgmap v442: 129 pgs: 129 active+clean; 656 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T09:35:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:36 smithi002 ceph-mon[31557]: pgmap v443: 129 pgs: 129 active+clean; 656 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:35:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:36 smithi029 ceph-mon[36249]: pgmap v443: 129 pgs: 129 active+clean; 656 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:35:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:36 smithi026 ceph-mon[36056]: pgmap v443: 129 pgs: 129 active+clean; 656 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:35:37.488 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T09:35:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:35:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:38 smithi002 ceph-mon[31557]: pgmap v444: 129 pgs: 129 active+clean; 656 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:35:38.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:38 smithi029 ceph-mon[36249]: pgmap v444: 129 pgs: 129 active+clean; 656 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:35:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:38 smithi026 ceph-mon[36056]: pgmap v444: 129 pgs: 129 active+clean; 656 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:35:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.512+0000 7f1435ac7700 1 -- 172.21.15.2:0/3783679284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1430108760 msgr2=0x7f14301100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.512+0000 7f1435ac7700 1 --2- 172.21.15.2:0/3783679284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1430108760 0x7f14301100c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f14280099c0 tx=0x7f1428009cd0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.513+0000 7f1435ac7700 1 -- 172.21.15.2:0/3783679284 shutdown_connections 2024-05-14T09:35:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.513+0000 7f1435ac7700 1 --2- 172.21.15.2:0/3783679284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1430108760 0x7f14301100c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.513+0000 7f1435ac7700 1 --2- 172.21.15.2:0/3783679284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14301034e0 0x7f14301038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.513+0000 7f1435ac7700 1 --2- 172.21.15.2:0/3783679284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1430107e10 0x7f1430108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.513+0000 7f1435ac7700 1 -- 172.21.15.2:0/3783679284 >> 172.21.15.2:0/3783679284 conn(0x7f14300fef30 msgr2=0x7f1430101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.513+0000 7f1435ac7700 1 -- 172.21.15.2:0/3783679284 shutdown_connections 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.513+0000 7f1435ac7700 1 -- 172.21.15.2:0/3783679284 wait complete. 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.513+0000 7f1435ac7700 1 Processor -- start 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.514+0000 7f1435ac7700 1 -- start start 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.514+0000 7f1435ac7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14301034e0 0x7f1430199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.514+0000 7f1435ac7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1430107e10 0x7f143019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.514+0000 7f1435ac7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1430108760 0x7f143019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.514+0000 7f1435ac7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14301133e0 con 0x7f1430108760 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.514+0000 7f1435ac7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1430113260 con 0x7f1430107e10 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.514+0000 7f1435ac7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1430113560 con 0x7f14301034e0 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.514+0000 7f142ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1430108760 0x7f143019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.514+0000 7f142ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1430108760 0x7f143019e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48974/0 (socket says 172.21.15.2:48974) 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.514+0000 7f142ffff700 1 -- 172.21.15.2:0/3059412699 learned_addr learned my addr 172.21.15.2:0/3059412699 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.514+0000 7f142f7fe700 1 --2- 172.21.15.2:0/3059412699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14301034e0 0x7f1430199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.514+0000 7f142effd700 1 --2- 172.21.15.2:0/3059412699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1430107e10 0x7f143019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.515+0000 7f142ffff700 1 -- 172.21.15.2:0/3059412699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14301034e0 msgr2=0x7f1430199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.515+0000 7f142ffff700 1 --2- 172.21.15.2:0/3059412699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14301034e0 0x7f1430199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.515+0000 7f142ffff700 1 -- 172.21.15.2:0/3059412699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1430107e10 msgr2=0x7f143019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.515+0000 7f142ffff700 1 --2- 172.21.15.2:0/3059412699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1430107e10 0x7f143019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.515+0000 7f142ffff700 1 -- 172.21.15.2:0/3059412699 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143019ebf0 con 0x7f1430108760 2024-05-14T09:35:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.515+0000 7f142ffff700 1 --2- 172.21.15.2:0/3059412699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1430108760 0x7f143019e600 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1428009990 tx=0x7f14280057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:39.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.518+0000 7f142cff9700 1 -- 172.21.15.2:0/3059412699 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428029070 con 0x7f1430108760 2024-05-14T09:35:39.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.518+0000 7f142cff9700 1 -- 172.21.15.2:0/3059412699 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1428019cc0 con 0x7f1430108760 2024-05-14T09:35:39.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.518+0000 7f1435ac7700 1 -- 172.21.15.2:0/3059412699 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143019ee80 con 0x7f1430108760 2024-05-14T09:35:39.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.518+0000 7f142cff9700 1 -- 172.21.15.2:0/3059412699 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428004050 con 0x7f1430108760 2024-05-14T09:35:39.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.518+0000 7f1435ac7700 1 -- 172.21.15.2:0/3059412699 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f143019f360 con 0x7f1430108760 2024-05-14T09:35:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.521+0000 7f142cff9700 1 -- 172.21.15.2:0/3059412699 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f142802c070 con 0x7f1430108760 2024-05-14T09:35:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.521+0000 7f142cff9700 1 --2- 172.21.15.2:0/3059412699 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14180b2880 0x7f14180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.522+0000 7f142f7fe700 1 --2- 172.21.15.2:0/3059412699 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14180b2880 0x7f14180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.522+0000 7f142cff9700 1 -- 172.21.15.2:0/3059412699 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f14280e5510 con 0x7f1430108760 2024-05-14T09:35:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.522+0000 7f14444dbb80 1 Processor -- start 2024-05-14T09:35:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.522+0000 7f14444dbb80 1 -- start start 2024-05-14T09:35:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.522+0000 7f142f7fe700 1 --2- 172.21.15.2:0/3059412699 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14180b2880 0x7f14180b4d90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f142000f440 tx=0x7f14200079e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.522+0000 7f14444dbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f352b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.522+0000 7f14444dbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f353070 0x56429f357350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.522+0000 7f14444dbb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f357890 0x56429f357ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.522+0000 7f14444dbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56429f270860 con 0x56429f1d0e40 2024-05-14T09:35:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.522+0000 7f14444dbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56429f20f7c0 con 0x56429f353070 2024-05-14T09:35:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.522+0000 7f14444dbb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56429f26c820 con 0x56429f357890 2024-05-14T09:35:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.522+0000 7f142f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f352b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.523+0000 7f142f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f352b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48986/0 (socket says 172.21.15.2:48986) 2024-05-14T09:35:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.523+0000 7f142f7fe700 1 -- 172.21.15.2:0/2764043577 learned_addr learned my addr 172.21.15.2:0/2764043577 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.523+0000 7f142effd700 1 --2- 172.21.15.2:0/2764043577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f353070 0x56429f357350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.523+0000 7f142ffff700 1 --2- 172.21.15.2:0/2764043577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f357890 0x56429f357ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.523+0000 7f142f7fe700 1 -- 172.21.15.2:0/2764043577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f357890 msgr2=0x56429f357ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.523+0000 7f142f7fe700 1 --2- 172.21.15.2:0/2764043577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f357890 0x56429f357ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.523+0000 7f142f7fe700 1 -- 172.21.15.2:0/2764043577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f353070 msgr2=0x56429f357350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.523+0000 7f142f7fe700 1 --2- 172.21.15.2:0/2764043577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f353070 0x56429f357350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.523+0000 7f142f7fe700 1 -- 172.21.15.2:0/2764043577 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56429f1bd8d0 con 0x56429f1d0e40 2024-05-14T09:35:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.523+0000 7f142effd700 1 --2- 172.21.15.2:0/2764043577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f353070 0x56429f357350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:35:39.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.523+0000 7f142f7fe700 1 --2- 172.21.15.2:0/2764043577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f352b30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1420014770 tx=0x7f1420012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce661c99ea2a76b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:39.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.523+0000 7f1415ffb700 1 -- 172.21.15.2:0/2764043577 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142001b050 con 0x56429f1d0e40 2024-05-14T09:35:39.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.523+0000 7f1415ffb700 1 -- 172.21.15.2:0/2764043577 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1420025070 con 0x56429f1d0e40 2024-05-14T09:35:39.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.523+0000 7f1415ffb700 1 -- 172.21.15.2:0/2764043577 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420015660 con 0x56429f1d0e40 2024-05-14T09:35:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.524+0000 7f14444dbb80 1 -- 172.21.15.2:0/2764043577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 msgr2=0x56429f352b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.524+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2764043577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f352b30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1420014770 tx=0x7f1420012000 comp rx=0 tx=0).stop 2024-05-14T09:35:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.524+0000 7f14444dbb80 1 -- 172.21.15.2:0/2764043577 shutdown_connections 2024-05-14T09:35:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.524+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2764043577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f357890 0x56429f357ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.524+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2764043577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f352b30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.524+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2764043577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f353070 0x56429f357350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.524+0000 7f14444dbb80 1 -- 172.21.15.2:0/2764043577 >> 172.21.15.2:0/2764043577 conn(0x56429f218600 msgr2=0x56429f3494b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.524+0000 7f14444dbb80 1 -- 172.21.15.2:0/2764043577 shutdown_connections 2024-05-14T09:35:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.524+0000 7f14444dbb80 1 -- 172.21.15.2:0/2764043577 wait complete. 2024-05-14T09:35:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.525+0000 7f14444dbb80 1 -- start start 2024-05-14T09:35:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.525+0000 7f14444dbb80 1 Processor -- start 2024-05-14T09:35:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.525+0000 7f14444dbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f35c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.525+0000 7f14444dbb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f353070 0x56429f35ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.525+0000 7f14444dbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f357890 0x56429f35cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f14444dbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56429f35f840 con 0x56429f1d0e40 2024-05-14T09:35:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f14444dbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56429f1d6c30 con 0x56429f357890 2024-05-14T09:35:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f14444dbb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56429f35f9c0 con 0x56429f353070 2024-05-14T09:35:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f14444dbb80 10 client.0 _ll_register_callbacks cb 0x56429f208920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f142f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f35c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f142f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f35c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48998/0 (socket says 172.21.15.2:48998) 2024-05-14T09:35:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f142f7fe700 1 -- 172.21.15.2:0/2633066866 learned_addr learned my addr 172.21.15.2:0/2633066866 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f142effd700 1 --2- 172.21.15.2:0/2633066866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f353070 0x56429f35ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f142ffff700 1 --2- 172.21.15.2:0/2633066866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f357890 0x56429f35cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f142f7fe700 1 -- 172.21.15.2:0/2633066866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f353070 msgr2=0x56429f35ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f142f7fe700 1 --2- 172.21.15.2:0/2633066866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f353070 0x56429f35ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f142f7fe700 1 -- 172.21.15.2:0/2633066866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f357890 msgr2=0x56429f35cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f142f7fe700 1 --2- 172.21.15.2:0/2633066866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f357890 0x56429f35cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f142f7fe700 1 -- 172.21.15.2:0/2633066866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56429f20c690 con 0x56429f1d0e40 2024-05-14T09:35:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f142ffff700 1 --2- 172.21.15.2:0/2633066866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f357890 0x56429f35cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f142f7fe700 1 --2- 172.21.15.2:0/2633066866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f35c540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f142000fb90 tx=0x7f1420020b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1112605756d9780 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f140effd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f140effd700 1 -- 172.21.15.2:0/2633066866 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142002e070 con 0x56429f1d0e40 2024-05-14T09:35:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f140effd700 1 -- 172.21.15.2:0/2633066866 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1420015810 con 0x56429f1d0e40 2024-05-14T09:35:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.526+0000 7f14444dbb80 1 -- 172.21.15.2:0/2633066866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1420025070 con 0x56429f1d0e40 2024-05-14T09:35:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.527+0000 7f140effd700 1 -- 172.21.15.2:0/2633066866 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f142001b020 con 0x56429f1d0e40 2024-05-14T09:35:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.527+0000 7f14444dbb80 10 client.15402 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.527+0000 7f14444dbb80 1 -- 172.21.15.2:0/2633066866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56429f36a3e0 con 0x56429f1d0e40 2024-05-14T09:35:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.527+0000 7f140effd700 1 -- 172.21.15.2:0/2633066866 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1420025070 con 0x56429f1d0e40 2024-05-14T09:35:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.527+0000 7f140effd700 1 -- 172.21.15.2:0/2633066866 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1420033a00 con 0x56429f1d0e40 2024-05-14T09:35:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.527+0000 7f14444dbb80 10 client.15402 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.527+0000 7f14444dbb80 10 client.15402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.527+0000 7f14444dbb80 20 client.15402 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:39.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.527+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2633066866 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56429f36c9c0 0x56429f36ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.527+0000 7f14444dbb80 4 client.15402 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:35:39.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.527+0000 7f14444dbb80 1 -- 172.21.15.2:0/2633066866 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56429f27e240 con 0x56429f36c9c0 2024-05-14T09:35:39.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.527+0000 7f142effd700 1 --2- 172.21.15.2:0/2633066866 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56429f36c9c0 0x56429f36ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.528+0000 7f140effd700 10 client.15402 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:39.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.528+0000 7f142effd700 1 --2- 172.21.15.2:0/2633066866 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56429f36c9c0 0x56429f36ede0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1fdc78e307f61e server_cookie=fd058b58f9eca866 in_seq=0 out_seq=0 2024-05-14T09:35:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.529+0000 7f140effd700 1 -- 172.21.15.2:0/2633066866 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56429f27e240 con 0x56429f36c9c0 2024-05-14T09:35:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.529+0000 7f140effd700 10 client.15402 handle_command_reply: tid=0 2024-05-14T09:35:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.529+0000 7f14444dbb80 1 client.15402 shutdown 2024-05-14T09:35:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.529+0000 7f14444dbb80 1 -- 172.21.15.2:0/2633066866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 msgr2=0x56429f35c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.529+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2633066866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f35c540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f142000fb90 tx=0x7f1420020b60 comp rx=0 tx=0).stop 2024-05-14T09:35:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.529+0000 7f14444dbb80 1 -- 172.21.15.2:0/2633066866 shutdown_connections 2024-05-14T09:35:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.529+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2633066866 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56429f36c9c0 0x56429f36ede0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.529+0000 7f140effd700 0 client.15402 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.529+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2633066866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f357890 0x56429f35cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.529+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2633066866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f35c540 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.529+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2633066866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f353070 0x56429f35ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.529+0000 7f14444dbb80 1 -- 172.21.15.2:0/2633066866 >> 172.21.15.2:0/2633066866 conn(0x56429f218600 msgr2=0x56429f34d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:39.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.529+0000 7f14444dbb80 1 -- 172.21.15.2:0/2633066866 shutdown_connections 2024-05-14T09:35:39.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.529+0000 7f14444dbb80 1 -- 172.21.15.2:0/2633066866 wait complete. 2024-05-14T09:35:39.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.529+0000 7f14444dbb80 20 client.15402 trim_cache size 0 max 16384 2024-05-14T09:35:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.549+0000 7f14444dbb80 1 Processor -- start 2024-05-14T09:35:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.549+0000 7f14444dbb80 1 -- start start 2024-05-14T09:35:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f14444dbb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f1d0e40 0x56429f368c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f14444dbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f353070 0x56429f35ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f14444dbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f357890 0x56429f34db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f14444dbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14301b0b50 con 0x56429f357890 2024-05-14T09:35:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f14444dbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f14301ae3e0 con 0x56429f353070 2024-05-14T09:35:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f14444dbb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f14301ac230 con 0x56429f1d0e40 2024-05-14T09:35:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f142ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f357890 0x56429f34db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f142ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f357890 0x56429f34db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49000/0 (socket says 172.21.15.2:49000) 2024-05-14T09:35:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f142ffff700 1 -- 172.21.15.2:0/667688206 learned_addr learned my addr 172.21.15.2:0/667688206 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f142f7fe700 1 --2- 172.21.15.2:0/667688206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f1d0e40 0x56429f368c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f142effd700 1 --2- 172.21.15.2:0/667688206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f353070 0x56429f35ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f142ffff700 1 -- 172.21.15.2:0/667688206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f1d0e40 msgr2=0x56429f368c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:39.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f142ffff700 1 --2- 172.21.15.2:0/667688206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f1d0e40 0x56429f368c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f142ffff700 1 -- 172.21.15.2:0/667688206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f353070 msgr2=0x56429f35ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:39.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f142ffff700 1 --2- 172.21.15.2:0/667688206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f353070 0x56429f35ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f142ffff700 1 -- 172.21.15.2:0/667688206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56429f349610 con 0x56429f357890 2024-05-14T09:35:39.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.550+0000 7f142ffff700 1 --2- 172.21.15.2:0/667688206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f357890 0x56429f34db60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f14280ac4a0 tx=0x7f1428009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c740c81f3f2512d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:39.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.551+0000 7f14157fa700 1 -- 172.21.15.2:0/667688206 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14280ac2b0 con 0x56429f357890 2024-05-14T09:35:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.551+0000 7f14157fa700 1 -- 172.21.15.2:0/667688206 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14280acea0 con 0x56429f357890 2024-05-14T09:35:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.551+0000 7f14157fa700 1 -- 172.21.15.2:0/667688206 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14280acc60 con 0x56429f357890 2024-05-14T09:35:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.551+0000 7f14444dbb80 1 -- 172.21.15.2:0/667688206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f357890 msgr2=0x56429f34db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.551+0000 7f14444dbb80 1 --2- 172.21.15.2:0/667688206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f357890 0x56429f34db60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f14280ac4a0 tx=0x7f1428009cd0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.552+0000 7f14444dbb80 1 -- 172.21.15.2:0/667688206 shutdown_connections 2024-05-14T09:35:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.552+0000 7f14444dbb80 1 --2- 172.21.15.2:0/667688206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f357890 0x56429f34db60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.552+0000 7f14444dbb80 1 --2- 172.21.15.2:0/667688206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f1d0e40 0x56429f368c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.552+0000 7f14444dbb80 1 --2- 172.21.15.2:0/667688206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f353070 0x56429f35ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.552+0000 7f14444dbb80 1 -- 172.21.15.2:0/667688206 >> 172.21.15.2:0/667688206 conn(0x56429f218600 msgr2=0x56429f34d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.552+0000 7f14444dbb80 1 -- 172.21.15.2:0/667688206 shutdown_connections 2024-05-14T09:35:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.552+0000 7f14444dbb80 1 -- 172.21.15.2:0/667688206 wait complete. 2024-05-14T09:35:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.552+0000 7f14444dbb80 1 -- start start 2024-05-14T09:35:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.552+0000 7f14444dbb80 1 Processor -- start 2024-05-14T09:35:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f14444dbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f36f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f14444dbb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f353070 0x56429f36ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f14444dbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f357890 0x56429f370440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f14444dbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14301ac8a0 con 0x56429f1d0e40 2024-05-14T09:35:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f14444dbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f14301add10 con 0x56429f357890 2024-05-14T09:35:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f14444dbb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f14301acf70 con 0x56429f353070 2024-05-14T09:35:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f14444dbb80 10 client.0 _ll_register_callbacks cb 0x56429f208920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f142f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f36f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f142f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f36f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49006/0 (socket says 172.21.15.2:49006) 2024-05-14T09:35:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f142f7fe700 1 -- 172.21.15.2:0/2790593077 learned_addr learned my addr 172.21.15.2:0/2790593077 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f142ffff700 1 --2- 172.21.15.2:0/2790593077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f357890 0x56429f370440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f142effd700 1 --2- 172.21.15.2:0/2790593077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f353070 0x56429f36ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f142f7fe700 1 -- 172.21.15.2:0/2790593077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f353070 msgr2=0x56429f36ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f142f7fe700 1 --2- 172.21.15.2:0/2790593077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f353070 0x56429f36ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f142f7fe700 1 -- 172.21.15.2:0/2790593077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f357890 msgr2=0x56429f370440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f142f7fe700 1 --2- 172.21.15.2:0/2790593077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f357890 0x56429f370440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f142f7fe700 1 -- 172.21.15.2:0/2790593077 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56429f3d5f10 con 0x56429f1d0e40 2024-05-14T09:35:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f142ffff700 1 --2- 172.21.15.2:0/2790593077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f357890 0x56429f370440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:35:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f142f7fe700 1 --2- 172.21.15.2:0/2790593077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f36f9c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1420033da0 tx=0x7f1420020cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efab7efb0990ed88 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.553+0000 7f140effd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.554+0000 7f140effd700 1 -- 172.21.15.2:0/2790593077 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142002e040 con 0x56429f1d0e40 2024-05-14T09:35:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.554+0000 7f14444dbb80 1 -- 172.21.15.2:0/2790593077 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14280acea0 con 0x56429f1d0e40 2024-05-14T09:35:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.554+0000 7f140effd700 1 -- 172.21.15.2:0/2790593077 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1420004c70 con 0x56429f1d0e40 2024-05-14T09:35:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.554+0000 7f140effd700 1 -- 172.21.15.2:0/2790593077 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14200316f0 con 0x56429f1d0e40 2024-05-14T09:35:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.554+0000 7f14444dbb80 10 client.15408 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.554+0000 7f14444dbb80 1 -- 172.21.15.2:0/2790593077 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56429f3d70e0 con 0x56429f1d0e40 2024-05-14T09:35:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.554+0000 7f140effd700 1 -- 172.21.15.2:0/2790593077 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14280acea0 con 0x56429f1d0e40 2024-05-14T09:35:39.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.554+0000 7f140effd700 1 -- 172.21.15.2:0/2790593077 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f14200337d0 con 0x56429f1d0e40 2024-05-14T09:35:39.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.554+0000 7f14444dbb80 10 client.15408 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:39.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.554+0000 7f14444dbb80 10 client.15408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:39.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.554+0000 7f14444dbb80 20 client.15408 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:39.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.554+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2790593077 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56429f36c9c0 0x56429f3d9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:39.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.554+0000 7f14444dbb80 4 client.15408 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T09:35:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.554+0000 7f14444dbb80 1 -- 172.21.15.2:0/2790593077 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56429f2782e0 con 0x56429f36c9c0 2024-05-14T09:35:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.555+0000 7f142effd700 1 --2- 172.21.15.2:0/2790593077 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56429f36c9c0 0x56429f3d9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.555+0000 7f142effd700 1 --2- 172.21.15.2:0/2790593077 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56429f36c9c0 0x56429f3d9790 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69cd45359fc216bc server_cookie=c1550c6067ee0a8c in_seq=0 out_seq=0 2024-05-14T09:35:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.555+0000 7f140effd700 10 client.15408 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.555+0000 7f140effd700 1 -- 172.21.15.2:0/2790593077 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56429f2782e0 con 0x56429f36c9c0 2024-05-14T09:35:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.555+0000 7f140effd700 10 client.15408 handle_command_reply: tid=0 2024-05-14T09:35:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.555+0000 7f14444dbb80 1 client.15408 shutdown 2024-05-14T09:35:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.555+0000 7f14444dbb80 1 -- 172.21.15.2:0/2790593077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 msgr2=0x56429f36f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.555+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2790593077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f36f9c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1420033da0 tx=0x7f1420020cc0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.555+0000 7f14444dbb80 1 -- 172.21.15.2:0/2790593077 shutdown_connections 2024-05-14T09:35:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.555+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2790593077 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56429f36c9c0 0x56429f3d9790 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.555+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2790593077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56429f357890 0x56429f370440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.555+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2790593077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429f1d0e40 0x56429f36f9c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.555+0000 7f140effd700 0 client.15408 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:39.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.555+0000 7f14444dbb80 1 --2- 172.21.15.2:0/2790593077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56429f353070 0x56429f36ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.555+0000 7f14444dbb80 1 -- 172.21.15.2:0/2790593077 >> 172.21.15.2:0/2790593077 conn(0x56429f218600 msgr2=0x56429f3582e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:39.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.556+0000 7f14444dbb80 1 -- 172.21.15.2:0/2790593077 shutdown_connections 2024-05-14T09:35:39.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.556+0000 7f14444dbb80 1 -- 172.21.15.2:0/2790593077 wait complete. 2024-05-14T09:35:39.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.556+0000 7f14444dbb80 20 client.15408 trim_cache size 0 max 16384 2024-05-14T09:35:39.560 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:35:39.560 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-05-14T09:35:39.560 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-05-14T09:35:39.560 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:35:39.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.556+0000 7f1435ac7700 1 -- 172.21.15.2:0/3059412699 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14180b2880 msgr2=0x7f14180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:39.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.556+0000 7f1435ac7700 1 --2- 172.21.15.2:0/3059412699 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14180b2880 0x7f14180b4d90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f142000f440 tx=0x7f14200079e0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.556+0000 7f1435ac7700 1 -- 172.21.15.2:0/3059412699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1430108760 msgr2=0x7f143019e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:39.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.556+0000 7f1435ac7700 1 --2- 172.21.15.2:0/3059412699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1430108760 0x7f143019e600 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1428009990 tx=0x7f14280057c0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.556+0000 7f1435ac7700 1 -- 172.21.15.2:0/3059412699 shutdown_connections 2024-05-14T09:35:39.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.556+0000 7f1435ac7700 1 --2- 172.21.15.2:0/3059412699 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14180b2880 0x7f14180b4d90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.556+0000 7f1435ac7700 1 --2- 172.21.15.2:0/3059412699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1430108760 0x7f143019e600 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.556+0000 7f1435ac7700 1 --2- 172.21.15.2:0/3059412699 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14301034e0 0x7f1430199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.556+0000 7f1435ac7700 1 --2- 172.21.15.2:0/3059412699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1430107e10 0x7f143019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:39.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.556+0000 7f1435ac7700 1 -- 172.21.15.2:0/3059412699 >> 172.21.15.2:0/3059412699 conn(0x7f14300fef30 msgr2=0x7f1430107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:39.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.557+0000 7f1435ac7700 1 -- 172.21.15.2:0/3059412699 shutdown_connections 2024-05-14T09:35:39.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:39.557+0000 7f1435ac7700 1 -- 172.21.15.2:0/3059412699 wait complete. 2024-05-14T09:35:40.243 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T09:35:40.244 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:35:40.509 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:35:40.634 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:40 smithi002 ceph-mon[31557]: pgmap v445: 129 pgs: 129 active+clean; 665 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-14T09:35:40.634 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:35:40.635 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:35:40.635 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:35:40.635 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:35:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:40 smithi029 ceph-mon[36249]: pgmap v445: 129 pgs: 129 active+clean; 665 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-14T09:35:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:35:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:35:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:35:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:35:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:40 smithi026 ceph-mon[36056]: pgmap v445: 129 pgs: 129 active+clean; 665 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-14T09:35:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:35:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:35:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:35:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:35:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.662+0000 7f5b78430640 1 -- 172.21.15.2:0/1337258412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b70108a70 msgr2=0x7f5b7010ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.662+0000 7f5b78430640 1 --2- 172.21.15.2:0/1337258412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b70108a70 0x7f5b7010ae90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009ca0 tx=0x7f5b6802a8c0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.662+0000 7f5b78430640 1 -- 172.21.15.2:0/1337258412 shutdown_connections 2024-05-14T09:35:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.662+0000 7f5b78430640 1 --2- 172.21.15.2:0/1337258412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b70108a70 0x7f5b7010ae90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.662+0000 7f5b78430640 1 --2- 172.21.15.2:0/1337258412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b70103d70 0x7f5b70104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.662+0000 7f5b78430640 1 --2- 172.21.15.2:0/1337258412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b70102b30 0x7f5b70103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.662+0000 7f5b78430640 1 -- 172.21.15.2:0/1337258412 >> 172.21.15.2:0/1337258412 conn(0x7f5b700fdd20 msgr2=0x7f5b701001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.662+0000 7f5b78430640 1 -- 172.21.15.2:0/1337258412 shutdown_connections 2024-05-14T09:35:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.662+0000 7f5b78430640 1 -- 172.21.15.2:0/1337258412 wait complete. 2024-05-14T09:35:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.663+0000 7f5b78430640 1 Processor -- start 2024-05-14T09:35:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.663+0000 7f5b78430640 1 -- start start 2024-05-14T09:35:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.663+0000 7f5b78430640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b70102b30 0x7f5b700ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.663+0000 7f5b78430640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b70103d70 0x7f5b700fe250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.663+0000 7f5b78430640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b70108a70 0x7f5b700fe790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.663+0000 7f5b78430640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7010e660 con 0x7f5b70108a70 2024-05-14T09:35:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.663+0000 7f5b78430640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7010e4e0 con 0x7f5b70103d70 2024-05-14T09:35:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.663+0000 7f5b78430640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7010e7e0 con 0x7f5b70102b30 2024-05-14T09:35:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.663+0000 7f5b769cd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b70108a70 0x7f5b700fe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.663+0000 7f5b769cd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b70108a70 0x7f5b700fe790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49020/0 (socket says 172.21.15.2:49020) 2024-05-14T09:35:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.663+0000 7f5b769cd640 1 -- 172.21.15.2:0/3653353453 learned_addr learned my addr 172.21.15.2:0/3653353453 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.663+0000 7f5b761cc640 1 --2- 172.21.15.2:0/3653353453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b70102b30 0x7f5b700ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.663+0000 7f5b759cb640 1 --2- 172.21.15.2:0/3653353453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b70103d70 0x7f5b700fe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.664+0000 7f5b769cd640 1 -- 172.21.15.2:0/3653353453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b70102b30 msgr2=0x7f5b700ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.664+0000 7f5b769cd640 1 --2- 172.21.15.2:0/3653353453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b70102b30 0x7f5b700ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.664+0000 7f5b769cd640 1 -- 172.21.15.2:0/3653353453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b70103d70 msgr2=0x7f5b700fe250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.664+0000 7f5b769cd640 1 --2- 172.21.15.2:0/3653353453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b70103d70 0x7f5b700fe250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.664+0000 7f5b769cd640 1 -- 172.21.15.2:0/3653353453 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b700feec0 con 0x7f5b70108a70 2024-05-14T09:35:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.664+0000 7f5b769cd640 1 --2- 172.21.15.2:0/3653353453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b70108a70 0x7f5b700fe790 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5b680097b0 tx=0x7f5b68003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.664+0000 7f5b677fe640 1 -- 172.21.15.2:0/3653353453 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68004240 con 0x7f5b70108a70 2024-05-14T09:35:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.664+0000 7f5b78430640 1 -- 172.21.15.2:0/3653353453 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b701a4350 con 0x7f5b70108a70 2024-05-14T09:35:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.664+0000 7f5b677fe640 1 -- 172.21.15.2:0/3653353453 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b6802f380 con 0x7f5b70108a70 2024-05-14T09:35:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.664+0000 7f5b677fe640 1 -- 172.21.15.2:0/3653353453 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6802fc30 con 0x7f5b70108a70 2024-05-14T09:35:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.664+0000 7f5b78430640 1 -- 172.21.15.2:0/3653353453 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b701a47c0 con 0x7f5b70108a70 2024-05-14T09:35:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.667+0000 7f5b7f3bd740 1 Processor -- start 2024-05-14T09:35:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.667+0000 7f5b677fe640 1 -- 172.21.15.2:0/3653353453 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5b680043e0 con 0x7f5b70108a70 2024-05-14T09:35:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.667+0000 7f5b7f3bd740 1 -- start start 2024-05-14T09:35:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b7f3bd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3c01b0 0x564e5d761a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b7f3bd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d3972f0 0x564e5d761fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b7f3bd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d39f4d0 0x564e5d768ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b7f3bd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564e5d6c65d0 con 0x564e5d3972f0 2024-05-14T09:35:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b7f3bd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564e5d67f890 con 0x564e5d39f4d0 2024-05-14T09:35:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b7f3bd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564e5d6c2590 con 0x564e5d3c01b0 2024-05-14T09:35:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b761cc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d3972f0 0x564e5d761fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b761cc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d3972f0 0x564e5d761fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49030/0 (socket says 172.21.15.2:49030) 2024-05-14T09:35:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b761cc640 1 -- 172.21.15.2:0/2658829413 learned_addr learned my addr 172.21.15.2:0/2658829413 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b759cb640 1 --2- 172.21.15.2:0/2658829413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d39f4d0 0x564e5d768ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b759cb640 1 --2- 172.21.15.2:0/2658829413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3c01b0 0x564e5d761a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b761cc640 1 -- 172.21.15.2:0/2658829413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3c01b0 msgr2=0x564e5d761a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b761cc640 1 --2- 172.21.15.2:0/2658829413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3c01b0 0x564e5d761a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b761cc640 1 -- 172.21.15.2:0/2658829413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d39f4d0 msgr2=0x564e5d768ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b761cc640 1 --2- 172.21.15.2:0/2658829413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d39f4d0 0x564e5d768ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b761cc640 1 -- 172.21.15.2:0/2658829413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e5d3d8210 con 0x564e5d3972f0 2024-05-14T09:35:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b759cb640 1 --2- 172.21.15.2:0/2658829413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d39f4d0 0x564e5d768ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b677fe640 1 --2- 172.21.15.2:0/3653353453 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5b5c0b28a0 0x7f5b5c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b761cc640 1 --2- 172.21.15.2:0/2658829413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d3972f0 0x564e5d761fe0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00b750 tx=0x7f5b6c00bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12412a449f7923a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b759cb640 1 --2- 172.21.15.2:0/3653353453 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5b5c0b28a0 0x7f5b5c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.668+0000 7f5b677fe640 1 -- 172.21.15.2:0/3653353453 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f5b68103f30 con 0x7f5b70108a70 2024-05-14T09:35:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.669+0000 7f5b64bfe640 1 -- 172.21.15.2:0/2658829413 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c00be60 con 0x564e5d3972f0 2024-05-14T09:35:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.669+0000 7f5b64bfe640 1 -- 172.21.15.2:0/2658829413 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b6c015070 con 0x564e5d3972f0 2024-05-14T09:35:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.669+0000 7f5b64bfe640 1 -- 172.21.15.2:0/2658829413 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0106a0 con 0x564e5d3972f0 2024-05-14T09:35:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.669+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/2658829413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d3972f0 msgr2=0x564e5d761fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.669+0000 7f5b759cb640 1 --2- 172.21.15.2:0/3653353453 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5b5c0b28a0 0x7f5b5c0b4db0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005fd0 tx=0x7f5b600046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.669+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/2658829413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d3972f0 0x564e5d761fe0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00b750 tx=0x7f5b6c00bc10 comp rx=0 tx=0).stop 2024-05-14T09:35:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/2658829413 shutdown_connections 2024-05-14T09:35:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/2658829413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d39f4d0 0x564e5d768ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/2658829413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d3972f0 0x564e5d761fe0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/2658829413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3c01b0 0x564e5d761a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/2658829413 >> 172.21.15.2:0/2658829413 conn(0x564e5d3b4300 msgr2=0x564e5d75d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/2658829413 shutdown_connections 2024-05-14T09:35:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/2658829413 wait complete. 2024-05-14T09:35:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 1 -- start start 2024-05-14T09:35:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 1 Processor -- start 2024-05-14T09:35:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3972f0 0x564e5d76b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d39f4d0 0x564e5d76bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d3c01b0 0x564e5d772b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564e5d769930 con 0x564e5d39f4d0 2024-05-14T09:35:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564e5d683370 con 0x564e5d3c01b0 2024-05-14T09:35:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564e5d769ab0 con 0x564e5d3972f0 2024-05-14T09:35:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.670+0000 7f5b7f3bd740 10 client.0 _ll_register_callbacks cb 0x564e5d3d7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.671+0000 7f5b759cb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d39f4d0 0x564e5d76bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.671+0000 7f5b759cb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d39f4d0 0x564e5d76bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49040/0 (socket says 172.21.15.2:49040) 2024-05-14T09:35:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.671+0000 7f5b759cb640 1 -- 172.21.15.2:0/1536401823 learned_addr learned my addr 172.21.15.2:0/1536401823 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.671+0000 7f5b769cd640 1 --2- 172.21.15.2:0/1536401823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d3c01b0 0x564e5d772b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.671+0000 7f5b761cc640 1 --2- 172.21.15.2:0/1536401823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3972f0 0x564e5d76b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.671+0000 7f5b759cb640 1 -- 172.21.15.2:0/1536401823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3972f0 msgr2=0x564e5d76b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.671+0000 7f5b759cb640 1 --2- 172.21.15.2:0/1536401823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3972f0 0x564e5d76b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.671+0000 7f5b759cb640 1 -- 172.21.15.2:0/1536401823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d3c01b0 msgr2=0x564e5d772b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.671+0000 7f5b759cb640 1 --2- 172.21.15.2:0/1536401823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d3c01b0 0x564e5d772b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.671+0000 7f5b759cb640 1 -- 172.21.15.2:0/1536401823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e5d4a4620 con 0x564e5d39f4d0 2024-05-14T09:35:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.671+0000 7f5b759cb640 1 --2- 172.21.15.2:0/1536401823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d39f4d0 0x564e5d76bbe0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5b60010bf0 tx=0x7f5b6000e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351ff846374a91ac server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.671+0000 7f5b59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.671+0000 7f5b59ffb640 1 -- 172.21.15.2:0/1536401823 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60004220 con 0x564e5d39f4d0 2024-05-14T09:35:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.671+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/1536401823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b6c015070 con 0x564e5d39f4d0 2024-05-14T09:35:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.671+0000 7f5b59ffb640 1 -- 172.21.15.2:0/1536401823 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f5b600043c0 con 0x564e5d39f4d0 2024-05-14T09:35:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.672+0000 7f5b59ffb640 1 -- 172.21.15.2:0/1536401823 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b6001c920 con 0x564e5d39f4d0 2024-05-14T09:35:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.672+0000 7f5b7f3bd740 10 client.15420 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.672+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/1536401823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e5d775b40 con 0x564e5d39f4d0 2024-05-14T09:35:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.672+0000 7f5b59ffb640 1 -- 172.21.15.2:0/1536401823 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b6c015070 con 0x564e5d39f4d0 2024-05-14T09:35:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.672+0000 7f5b59ffb640 1 -- 172.21.15.2:0/1536401823 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5b60027430 con 0x564e5d39f4d0 2024-05-14T09:35:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.672+0000 7f5b7f3bd740 10 client.15420 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.672+0000 7f5b7f3bd740 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.672+0000 7f5b7f3bd740 20 client.15420 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.672+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/1536401823 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564e5d37ed80 0x564e5d7792a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.672+0000 7f5b7f3bd740 4 client.15420 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:35:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.672+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/1536401823 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e5d6d41e0 con 0x564e5d37ed80 2024-05-14T09:35:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.672+0000 7f5b761cc640 1 --2- 172.21.15.2:0/1536401823 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564e5d37ed80 0x564e5d7792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.673+0000 7f5b59ffb640 10 client.15420 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.673+0000 7f5b761cc640 1 --2- 172.21.15.2:0/1536401823 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564e5d37ed80 0x564e5d7792a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b88e848347046303 server_cookie=13b39393166d375a in_seq=0 out_seq=0 2024-05-14T09:35:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.674+0000 7f5b59ffb640 1 -- 172.21.15.2:0/1536401823 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564e5d6d41e0 con 0x564e5d37ed80 2024-05-14T09:35:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.674+0000 7f5b59ffb640 10 client.15420 handle_command_reply: tid=0 2024-05-14T09:35:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.674+0000 7f5b7f3bd740 1 client.15420 shutdown 2024-05-14T09:35:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.674+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/1536401823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d39f4d0 msgr2=0x564e5d76bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.674+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/1536401823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d39f4d0 0x564e5d76bbe0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5b60010bf0 tx=0x7f5b6000e3b0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.674+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/1536401823 shutdown_connections 2024-05-14T09:35:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.674+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/1536401823 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564e5d37ed80 0x564e5d7792a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.674+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/1536401823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d3c01b0 0x564e5d772b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.674+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/1536401823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d39f4d0 0x564e5d76bbe0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.674+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/1536401823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3972f0 0x564e5d76b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.674+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/1536401823 >> 172.21.15.2:0/1536401823 conn(0x564e5d3b4300 msgr2=0x564e5d760300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.674+0000 7f5b59ffb640 0 client.15420 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.674+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/1536401823 shutdown_connections 2024-05-14T09:35:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.674+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/1536401823 wait complete. 2024-05-14T09:35:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.674+0000 7f5b7f3bd740 20 client.15420 trim_cache size 0 max 16384 2024-05-14T09:35:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.690+0000 7f5b7f3bd740 1 Processor -- start 2024-05-14T09:35:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.690+0000 7f5b7f3bd740 1 -- start start 2024-05-14T09:35:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.690+0000 7f5b7f3bd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3972f0 0x564e5d769eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b7f3bd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d39f4d0 0x564e5d76a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b7f3bd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d3c01b0 0x564e5d772a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b7f3bd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701b16e0 con 0x564e5d39f4d0 2024-05-14T09:35:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b7f3bd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701aefb0 con 0x564e5d3c01b0 2024-05-14T09:35:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b7f3bd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701ace00 con 0x564e5d3972f0 2024-05-14T09:35:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b759cb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d39f4d0 0x564e5d76a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b759cb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d39f4d0 0x564e5d76a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49056/0 (socket says 172.21.15.2:49056) 2024-05-14T09:35:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b769cd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d3c01b0 0x564e5d772a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b759cb640 1 -- 172.21.15.2:0/1430400688 learned_addr learned my addr 172.21.15.2:0/1430400688 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b761cc640 1 --2- 172.21.15.2:0/1430400688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3972f0 0x564e5d769eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b759cb640 1 -- 172.21.15.2:0/1430400688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3972f0 msgr2=0x564e5d769eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b759cb640 1 --2- 172.21.15.2:0/1430400688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3972f0 0x564e5d769eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b759cb640 1 -- 172.21.15.2:0/1430400688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d3c01b0 msgr2=0x564e5d772a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b759cb640 1 --2- 172.21.15.2:0/1430400688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d3c01b0 0x564e5d772a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b759cb640 1 -- 172.21.15.2:0/1430400688 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e5d768e70 con 0x564e5d39f4d0 2024-05-14T09:35:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b769cd640 1 --2- 172.21.15.2:0/1430400688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d3c01b0 0x564e5d772a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:35:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.691+0000 7f5b759cb640 1 --2- 172.21.15.2:0/1430400688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d39f4d0 0x564e5d76a3f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5b60000c00 tx=0x7f5b60010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5678e4965bec764d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.692+0000 7f5b5bfff640 1 -- 172.21.15.2:0/1430400688 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6001c770 con 0x564e5d39f4d0 2024-05-14T09:35:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.692+0000 7f5b5bfff640 1 -- 172.21.15.2:0/1430400688 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b60027430 con 0x564e5d39f4d0 2024-05-14T09:35:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.692+0000 7f5b5bfff640 1 -- 172.21.15.2:0/1430400688 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60004450 con 0x564e5d39f4d0 2024-05-14T09:35:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.693+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/1430400688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d39f4d0 msgr2=0x564e5d76a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.693+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/1430400688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d39f4d0 0x564e5d76a3f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5b60000c00 tx=0x7f5b60010ef0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.693+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/1430400688 shutdown_connections 2024-05-14T09:35:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.693+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/1430400688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d3c01b0 0x564e5d772a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.693+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/1430400688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d39f4d0 0x564e5d76a3f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.693+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/1430400688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3972f0 0x564e5d769eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.693+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/1430400688 >> 172.21.15.2:0/1430400688 conn(0x564e5d3b4300 msgr2=0x564e5d765ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.693+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/1430400688 shutdown_connections 2024-05-14T09:35:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.693+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/1430400688 wait complete. 2024-05-14T09:35:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.693+0000 7f5b7f3bd740 1 -- start start 2024-05-14T09:35:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b7f3bd740 1 Processor -- start 2024-05-14T09:35:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b7f3bd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d3972f0 0x564e5d773060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b7f3bd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d39f4d0 0x564e5d76fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b7f3bd740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3c01b0 0x564e5d770170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b7f3bd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701ad470 con 0x564e5d3972f0 2024-05-14T09:35:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b7f3bd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701ae8e0 con 0x564e5d39f4d0 2024-05-14T09:35:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b7f3bd740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701adb40 con 0x564e5d3c01b0 2024-05-14T09:35:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b7f3bd740 10 client.0 _ll_register_callbacks cb 0x7f5b6000e580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b761cc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d3972f0 0x564e5d773060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b761cc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d3972f0 0x564e5d773060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49066/0 (socket says 172.21.15.2:49066) 2024-05-14T09:35:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b761cc640 1 -- 172.21.15.2:0/2556805339 learned_addr learned my addr 172.21.15.2:0/2556805339 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b769cd640 1 --2- 172.21.15.2:0/2556805339 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3c01b0 0x564e5d770170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b759cb640 1 --2- 172.21.15.2:0/2556805339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d39f4d0 0x564e5d76fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b761cc640 1 -- 172.21.15.2:0/2556805339 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3c01b0 msgr2=0x564e5d770170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b761cc640 1 --2- 172.21.15.2:0/2556805339 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3c01b0 0x564e5d770170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b761cc640 1 -- 172.21.15.2:0/2556805339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d39f4d0 msgr2=0x564e5d76fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b761cc640 1 --2- 172.21.15.2:0/2556805339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d39f4d0 0x564e5d76fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b761cc640 1 -- 172.21.15.2:0/2556805339 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e5d770780 con 0x564e5d3972f0 2024-05-14T09:35:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.694+0000 7f5b769cd640 1 --2- 172.21.15.2:0/2556805339 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3c01b0 0x564e5d770170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.695+0000 7f5b761cc640 1 --2- 172.21.15.2:0/2556805339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d3972f0 0x564e5d773060 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x564e5d762e40 tx=0x7f5b6c003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69e73b07f041ebe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.695+0000 7f5b59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.695+0000 7f5b59ffb640 1 -- 172.21.15.2:0/2556805339 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c005250 con 0x564e5d3972f0 2024-05-14T09:35:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.695+0000 7f5b59ffb640 1 -- 172.21.15.2:0/2556805339 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f5b6c0053f0 con 0x564e5d3972f0 2024-05-14T09:35:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.695+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/2556805339 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b60027430 con 0x564e5d3972f0 2024-05-14T09:35:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.695+0000 7f5b59ffb640 1 -- 172.21.15.2:0/2556805339 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b6c035050 con 0x564e5d3972f0 2024-05-14T09:35:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.695+0000 7f5b7f3bd740 10 client.15426 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.695+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/2556805339 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e5d7e06a0 con 0x564e5d3972f0 2024-05-14T09:35:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.695+0000 7f5b59ffb640 1 -- 172.21.15.2:0/2556805339 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b60027430 con 0x564e5d3972f0 2024-05-14T09:35:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.695+0000 7f5b59ffb640 1 -- 172.21.15.2:0/2556805339 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5b6c0116d0 con 0x564e5d3972f0 2024-05-14T09:35:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.695+0000 7f5b7f3bd740 10 client.15426 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.695+0000 7f5b7f3bd740 10 client.15426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.695+0000 7f5b7f3bd740 20 client.15426 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.696+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/2556805339 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564e5d3541c0 0x564e5d7e2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.696+0000 7f5b7f3bd740 4 client.15426 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:35:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.696+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/2556805339 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e5d6ce050 con 0x564e5d3541c0 2024-05-14T09:35:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.696+0000 7f5b759cb640 1 --2- 172.21.15.2:0/2556805339 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564e5d3541c0 0x564e5d7e2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.696+0000 7f5b59ffb640 10 client.15426 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.696+0000 7f5b759cb640 1 --2- 172.21.15.2:0/2556805339 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564e5d3541c0 0x564e5d7e2e60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c60f689c22ff6cbe server_cookie=77b2a60a849d218f in_seq=0 out_seq=0 2024-05-14T09:35:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.705+0000 7f5b59ffb640 1 -- 172.21.15.2:0/2556805339 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e5d6ce050 con 0x564e5d3541c0 2024-05-14T09:35:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.705+0000 7f5b59ffb640 10 client.15426 handle_command_reply: tid=0 2024-05-14T09:35:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.705+0000 7f5b7f3bd740 1 client.15426 shutdown 2024-05-14T09:35:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.705+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/2556805339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d3972f0 msgr2=0x564e5d773060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.705+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/2556805339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d3972f0 0x564e5d773060 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x564e5d762e40 tx=0x7f5b6c003dc0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.705+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/2556805339 shutdown_connections 2024-05-14T09:35:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.705+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/2556805339 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564e5d3541c0 0x564e5d7e2e60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.705+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/2556805339 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e5d3c01b0 0x564e5d770170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.705+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/2556805339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e5d39f4d0 0x564e5d76fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.705+0000 7f5b59ffb640 0 client.15426 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.705+0000 7f5b7f3bd740 1 --2- 172.21.15.2:0/2556805339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e5d3972f0 0x564e5d773060 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.705+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/2556805339 >> 172.21.15.2:0/2556805339 conn(0x564e5d3b4300 msgr2=0x564e5d769eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.705+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/2556805339 shutdown_connections 2024-05-14T09:35:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.705+0000 7f5b7f3bd740 1 -- 172.21.15.2:0/2556805339 wait complete. 2024-05-14T09:35:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.705+0000 7f5b7f3bd740 20 client.15426 trim_cache size 0 max 16384 2024-05-14T09:35:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:35:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:35:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:35:41.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:35:41.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.706+0000 7f5b78430640 1 -- 172.21.15.2:0/3653353453 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5b5c0b28a0 msgr2=0x7f5b5c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:41.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.706+0000 7f5b78430640 1 --2- 172.21.15.2:0/3653353453 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5b5c0b28a0 0x7f5b5c0b4db0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005fd0 tx=0x7f5b600046c0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.706+0000 7f5b78430640 1 -- 172.21.15.2:0/3653353453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b70108a70 msgr2=0x7f5b700fe790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.706+0000 7f5b78430640 1 --2- 172.21.15.2:0/3653353453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b70108a70 0x7f5b700fe790 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5b680097b0 tx=0x7f5b68003c20 comp rx=0 tx=0).stop 2024-05-14T09:35:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.706+0000 7f5b78430640 1 -- 172.21.15.2:0/3653353453 shutdown_connections 2024-05-14T09:35:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.706+0000 7f5b78430640 1 --2- 172.21.15.2:0/3653353453 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5b5c0b28a0 0x7f5b5c0b4db0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.706+0000 7f5b78430640 1 --2- 172.21.15.2:0/3653353453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b70108a70 0x7f5b700fe790 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.706+0000 7f5b78430640 1 --2- 172.21.15.2:0/3653353453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b70103d70 0x7f5b700fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.706+0000 7f5b78430640 1 --2- 172.21.15.2:0/3653353453 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5b70102b30 0x7f5b700ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.706+0000 7f5b78430640 1 -- 172.21.15.2:0/3653353453 >> 172.21.15.2:0/3653353453 conn(0x7f5b700fdd20 msgr2=0x7f5b70109ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.706+0000 7f5b78430640 1 -- 172.21.15.2:0/3653353453 shutdown_connections 2024-05-14T09:35:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:41.706+0000 7f5b78430640 1 -- 172.21.15.2:0/3653353453 wait complete. 2024-05-14T09:35:42.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.243+0000 7f620315b700 1 -- 172.21.15.2:0/2059744839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61fc107df0 msgr2=0x7f61fc108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:42.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.243+0000 7f620315b700 1 --2- 172.21.15.2:0/2059744839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61fc107df0 0x7f61fc108200 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f61e400b0d0 tx=0x7f61e400b3e0 comp rx=0 tx=0).stop 2024-05-14T09:35:42.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.244+0000 7f620315b700 1 -- 172.21.15.2:0/2059744839 shutdown_connections 2024-05-14T09:35:42.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.244+0000 7f620315b700 1 --2- 172.21.15.2:0/2059744839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61fc1034c0 0x7f61fc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:42.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.244+0000 7f620315b700 1 --2- 172.21.15.2:0/2059744839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61fc108740 0x7f61fc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:42.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.244+0000 7f620315b700 1 --2- 172.21.15.2:0/2059744839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61fc107df0 0x7f61fc108200 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:42.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.244+0000 7f620315b700 1 -- 172.21.15.2:0/2059744839 >> 172.21.15.2:0/2059744839 conn(0x7f61fc0fedc0 msgr2=0x7f61fc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:42.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.244+0000 7f620315b700 1 -- 172.21.15.2:0/2059744839 shutdown_connections 2024-05-14T09:35:42.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.244+0000 7f620315b700 1 -- 172.21.15.2:0/2059744839 wait complete. 2024-05-14T09:35:42.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.244+0000 7f620315b700 1 Processor -- start 2024-05-14T09:35:42.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.244+0000 7f620315b700 1 -- start start 2024-05-14T09:35:42.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f620315b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61fc1034c0 0x7f61fc19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:42.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f620315b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61fc107df0 0x7f61fc19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:42.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f620315b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61fc108740 0x7f61fc19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:42.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f620315b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc113350 con 0x7f61fc1034c0 2024-05-14T09:35:42.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f620315b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc1131d0 con 0x7f61fc108740 2024-05-14T09:35:42.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f620315b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc1134d0 con 0x7f61fc107df0 2024-05-14T09:35:42.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f6200ef7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61fc1034c0 0x7f61fc19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:42.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f6200ef7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61fc1034c0 0x7f61fc19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49084/0 (socket says 172.21.15.2:49084) 2024-05-14T09:35:42.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f6200ef7700 1 -- 172.21.15.2:0/3103835157 learned_addr learned my addr 172.21.15.2:0/3103835157 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:42.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f62016f8700 1 --2- 172.21.15.2:0/3103835157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61fc108740 0x7f61fc19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:42.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f61fbfff700 1 --2- 172.21.15.2:0/3103835157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61fc107df0 0x7f61fc19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:42.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f6200ef7700 1 -- 172.21.15.2:0/3103835157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61fc107df0 msgr2=0x7f61fc19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:42.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f6200ef7700 1 --2- 172.21.15.2:0/3103835157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61fc107df0 0x7f61fc19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:42.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f6200ef7700 1 -- 172.21.15.2:0/3103835157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61fc108740 msgr2=0x7f61fc19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:42.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f6200ef7700 1 --2- 172.21.15.2:0/3103835157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61fc108740 0x7f61fc19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:42.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.245+0000 7f6200ef7700 1 -- 172.21.15.2:0/3103835157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61fc19fc10 con 0x7f61fc1034c0 2024-05-14T09:35:42.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.246+0000 7f6200ef7700 1 --2- 172.21.15.2:0/3103835157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61fc1034c0 0x7f61fc19adf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00c9b0 tx=0x7f61ec00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:42.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.246+0000 7f61f9ffb700 1 -- 172.21.15.2:0/3103835157 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec007b00 con 0x7f61fc1034c0 2024-05-14T09:35:42.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.246+0000 7f61f9ffb700 1 -- 172.21.15.2:0/3103835157 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61ec00ce50 con 0x7f61fc1034c0 2024-05-14T09:35:42.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.246+0000 7f620315b700 1 -- 172.21.15.2:0/3103835157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61fc19ff00 con 0x7f61fc1034c0 2024-05-14T09:35:42.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.246+0000 7f61f9ffb700 1 -- 172.21.15.2:0/3103835157 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec018660 con 0x7f61fc1034c0 2024-05-14T09:35:42.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.246+0000 7f620315b700 1 -- 172.21.15.2:0/3103835157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61fc0ff1d0 con 0x7f61fc1034c0 2024-05-14T09:35:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.247+0000 7f620315b700 1 -- 172.21.15.2:0/3103835157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61fc104970 con 0x7f61fc1034c0 2024-05-14T09:35:42.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.250+0000 7f61f9ffb700 1 -- 172.21.15.2:0/3103835157 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f61ec00f450 con 0x7f61fc1034c0 2024-05-14T09:35:42.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.251+0000 7f61f9ffb700 1 --2- 172.21.15.2:0/3103835157 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f61e80b2900 0x7f61e80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:42.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.251+0000 7f61fbfff700 1 --2- 172.21.15.2:0/3103835157 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f61e80b2900 0x7f61e80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:42.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.251+0000 7f61f9ffb700 1 -- 172.21.15.2:0/3103835157 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f61ec0e2760 con 0x7f61fc1034c0 2024-05-14T09:35:42.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.251+0000 7f61fbfff700 1 --2- 172.21.15.2:0/3103835157 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f61e80b2900 0x7f61e80b4e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f61fc109970 tx=0x7f61e400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:42.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.254+0000 7f61f9ffb700 1 -- 172.21.15.2:0/3103835157 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61ec014030 con 0x7f61fc1034c0 2024-05-14T09:35:42.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.515+0000 7f620315b700 1 -- 172.21.15.2:0/3103835157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61fc19c150 con 0x7f61fc1034c0 2024-05-14T09:35:42.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.516+0000 7f61f9ffb700 1 -- 172.21.15.2:0/3103835157 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11843 (secure 0 0 0) 0x7f61ec0aa170 con 0x7f61fc1034c0 2024-05-14T09:35:42.517 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:35:42.517 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T09:34:49.632659+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24487,24497,24524]},"id":1}]} 2024-05-14T09:35:42.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.519+0000 7f620315b700 1 -- 172.21.15.2:0/3103835157 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f61e80b2900 msgr2=0x7f61e80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:42.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.519+0000 7f620315b700 1 --2- 172.21.15.2:0/3103835157 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f61e80b2900 0x7f61e80b4e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f61fc109970 tx=0x7f61e400b040 comp rx=0 tx=0).stop 2024-05-14T09:35:42.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.519+0000 7f620315b700 1 -- 172.21.15.2:0/3103835157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61fc1034c0 msgr2=0x7f61fc19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:42.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.519+0000 7f620315b700 1 --2- 172.21.15.2:0/3103835157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61fc1034c0 0x7f61fc19adf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00c9b0 tx=0x7f61ec00ccc0 comp rx=0 tx=0).stop 2024-05-14T09:35:42.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.519+0000 7f620315b700 1 -- 172.21.15.2:0/3103835157 shutdown_connections 2024-05-14T09:35:42.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.519+0000 7f620315b700 1 --2- 172.21.15.2:0/3103835157 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f61e80b2900 0x7f61e80b4e10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:42.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.519+0000 7f620315b700 1 --2- 172.21.15.2:0/3103835157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61fc108740 0x7f61fc19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:42.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.519+0000 7f620315b700 1 --2- 172.21.15.2:0/3103835157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61fc1034c0 0x7f61fc19adf0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:42.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.519+0000 7f620315b700 1 --2- 172.21.15.2:0/3103835157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61fc107df0 0x7f61fc19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:42.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.519+0000 7f620315b700 1 -- 172.21.15.2:0/3103835157 >> 172.21.15.2:0/3103835157 conn(0x7f61fc0fedc0 msgr2=0x7f61fc06b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:42.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.519+0000 7f620315b700 1 -- 172.21.15.2:0/3103835157 shutdown_connections 2024-05-14T09:35:42.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:42.519+0000 7f620315b700 1 -- 172.21.15.2:0/3103835157 wait complete. 2024-05-14T09:35:42.521 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 27 2024-05-14T09:35:42.830 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:42 smithi002 ceph-mon[31557]: pgmap v446: 129 pgs: 129 active+clean; 665 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-14T09:35:42.830 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:42 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3103835157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:35:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:42 smithi029 ceph-mon[36249]: pgmap v446: 129 pgs: 129 active+clean; 665 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-14T09:35:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:42 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3103835157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:35:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:42 smithi026 ceph-mon[36056]: pgmap v446: 129 pgs: 129 active+clean; 665 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-14T09:35:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:42 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3103835157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:35:43.127 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T09:35:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:35:44.829 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:44 smithi002 ceph-mon[31557]: pgmap v447: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-14T09:35:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:44 smithi029 ceph-mon[36249]: pgmap v447: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-14T09:35:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:44 smithi026 ceph-mon[36056]: pgmap v447: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-14T09:35:45.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.231+0000 7fd0247bd700 1 -- 172.21.15.2:0/1390153264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c103960 msgr2=0x7fd01c103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:45.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.231+0000 7fd0247bd700 1 --2- 172.21.15.2:0/1390153264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c103960 0x7fd01c103d70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009990 tx=0x7fd00c009ca0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.231+0000 7fd0247bd700 1 -- 172.21.15.2:0/1390153264 shutdown_connections 2024-05-14T09:35:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.231+0000 7fd0247bd700 1 --2- 172.21.15.2:0/1390153264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd01c108200 0x7fd01c108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.231+0000 7fd0247bd700 1 --2- 172.21.15.2:0/1390153264 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd01c1042b0 0x7fd01c110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.231+0000 7fd0247bd700 1 --2- 172.21.15.2:0/1390153264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c103960 0x7fd01c103d70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.231+0000 7fd0247bd700 1 -- 172.21.15.2:0/1390153264 >> 172.21.15.2:0/1390153264 conn(0x7fd01c0fe520 msgr2=0x7fd01c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.231+0000 7fd0247bd700 1 -- 172.21.15.2:0/1390153264 shutdown_connections 2024-05-14T09:35:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.231+0000 7fd0247bd700 1 -- 172.21.15.2:0/1390153264 wait complete. 2024-05-14T09:35:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.232+0000 7fd0247bd700 1 Processor -- start 2024-05-14T09:35:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.232+0000 7fd0247bd700 1 -- start start 2024-05-14T09:35:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.232+0000 7fd0247bd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd01c103960 0x7fd01c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.232+0000 7fd0247bd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd01c1042b0 0x7fd01c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.232+0000 7fd0247bd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c108200 0x7fd01c19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.232+0000 7fd0247bd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c113380 con 0x7fd01c108200 2024-05-14T09:35:45.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.232+0000 7fd0247bd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c113200 con 0x7fd01c1042b0 2024-05-14T09:35:45.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.232+0000 7fd0247bd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c113500 con 0x7fd01c103960 2024-05-14T09:35:45.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.232+0000 7fd022d5a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c108200 0x7fd01c19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.232+0000 7fd022d5a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c108200 0x7fd01c19f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49098/0 (socket says 172.21.15.2:49098) 2024-05-14T09:35:45.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.232+0000 7fd022d5a700 1 -- 172.21.15.2:0/3031313092 learned_addr learned my addr 172.21.15.2:0/3031313092 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:45.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.233+0000 7fd022559700 1 --2- 172.21.15.2:0/3031313092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd01c103960 0x7fd01c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.233+0000 7fd021d58700 1 --2- 172.21.15.2:0/3031313092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd01c1042b0 0x7fd01c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.233+0000 7fd022d5a700 1 -- 172.21.15.2:0/3031313092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd01c103960 msgr2=0x7fd01c19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:45.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.233+0000 7fd022d5a700 1 --2- 172.21.15.2:0/3031313092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd01c103960 0x7fd01c19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.233+0000 7fd022d5a700 1 -- 172.21.15.2:0/3031313092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd01c1042b0 msgr2=0x7fd01c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:45.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.233+0000 7fd022d5a700 1 --2- 172.21.15.2:0/3031313092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd01c1042b0 0x7fd01c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.233+0000 7fd022d5a700 1 -- 172.21.15.2:0/3031313092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01c19fbc0 con 0x7fd01c108200 2024-05-14T09:35:45.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.233+0000 7fd022d5a700 1 --2- 172.21.15.2:0/3031313092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c108200 0x7fd01c19f5d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd01400ada0 tx=0x7fd01400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:45.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.233+0000 7fd0137fe700 1 -- 172.21.15.2:0/3031313092 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014017020 con 0x7fd01c108200 2024-05-14T09:35:45.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.233+0000 7fd0137fe700 1 -- 172.21.15.2:0/3031313092 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd01400cd80 con 0x7fd01c108200 2024-05-14T09:35:45.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.233+0000 7fd0247bd700 1 -- 172.21.15.2:0/3031313092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01c19feb0 con 0x7fd01c108200 2024-05-14T09:35:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.233+0000 7fd0137fe700 1 -- 172.21.15.2:0/3031313092 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014005260 con 0x7fd01c108200 2024-05-14T09:35:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.233+0000 7fd0247bd700 1 -- 172.21.15.2:0/3031313092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01c1a0370 con 0x7fd01c108200 2024-05-14T09:35:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.236+0000 7fd0137fe700 1 -- 172.21.15.2:0/3031313092 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd01400c8d0 con 0x7fd01c108200 2024-05-14T09:35:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.237+0000 7fd0137fe700 1 --2- 172.21.15.2:0/3031313092 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd0080b2880 0x7fd0080b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.237+0000 7fd0331d1b80 1 Processor -- start 2024-05-14T09:35:45.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.237+0000 7fd022559700 1 --2- 172.21.15.2:0/3031313092 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd0080b2880 0x7fd0080b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.237+0000 7fd0137fe700 1 -- 172.21.15.2:0/3031313092 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd0140e1d90 con 0x7fd01c108200 2024-05-14T09:35:45.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.237+0000 7fd0331d1b80 1 -- start start 2024-05-14T09:35:45.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.237+0000 7fd0331d1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b1520b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.237+0000 7fd0331d1b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b1525350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.237+0000 7fd0331d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b1525ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.237+0000 7fd0331d1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5644b143e860 con 0x5644b139ee40 2024-05-14T09:35:45.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.237+0000 7fd0331d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5644b13dd7c0 con 0x5644b1525890 2024-05-14T09:35:45.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.237+0000 7fd0331d1b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5644b143a820 con 0x5644b1521070 2024-05-14T09:35:45.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd022559700 1 --2- 172.21.15.2:0/3031313092 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd0080b2880 0x7fd0080b4d90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd01c06c2f0 tx=0x7fd01800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:45.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd022559700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b1520b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd022559700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b1520b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49112/0 (socket says 172.21.15.2:49112) 2024-05-14T09:35:45.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd022559700 1 -- 172.21.15.2:0/1245495877 learned_addr learned my addr 172.21.15.2:0/1245495877 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:45.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd022d5a700 1 --2- 172.21.15.2:0/1245495877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b1525ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd021d58700 1 --2- 172.21.15.2:0/1245495877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b1525350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd022559700 1 -- 172.21.15.2:0/1245495877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 msgr2=0x5644b1525350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:45.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd022559700 1 --2- 172.21.15.2:0/1245495877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b1525350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd022559700 1 -- 172.21.15.2:0/1245495877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 msgr2=0x5644b1525ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:45.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd022559700 1 --2- 172.21.15.2:0/1245495877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b1525ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd022559700 1 -- 172.21.15.2:0/1245495877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b138b8d0 con 0x5644b139ee40 2024-05-14T09:35:45.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd022d5a700 1 --2- 172.21.15.2:0/1245495877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b1525ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:35:45.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd022559700 1 --2- 172.21.15.2:0/1245495877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b1520b30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd01800fea0 tx=0x7fd018012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d38899800c13f9f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:45.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd010ff9700 1 -- 172.21.15.2:0/1245495877 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018016070 con 0x5644b139ee40 2024-05-14T09:35:45.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd010ff9700 1 -- 172.21.15.2:0/1245495877 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd01801c420 con 0x5644b139ee40 2024-05-14T09:35:45.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.238+0000 7fd010ff9700 1 -- 172.21.15.2:0/1245495877 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01801b680 con 0x5644b139ee40 2024-05-14T09:35:45.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.239+0000 7fd0331d1b80 1 -- 172.21.15.2:0/1245495877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 msgr2=0x5644b1520b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:45.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.239+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/1245495877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b1520b30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd01800fea0 tx=0x7fd018012000 comp rx=0 tx=0).stop 2024-05-14T09:35:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.239+0000 7fd0331d1b80 1 -- 172.21.15.2:0/1245495877 shutdown_connections 2024-05-14T09:35:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.239+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/1245495877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b1525ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.239+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/1245495877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b1520b30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.239+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/1245495877 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b1525350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.239+0000 7fd0331d1b80 1 -- 172.21.15.2:0/1245495877 >> 172.21.15.2:0/1245495877 conn(0x5644b13e6600 msgr2=0x5644b15174b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.240+0000 7fd0331d1b80 1 -- 172.21.15.2:0/1245495877 shutdown_connections 2024-05-14T09:35:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.240+0000 7fd0331d1b80 1 -- 172.21.15.2:0/1245495877 wait complete. 2024-05-14T09:35:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.240+0000 7fd0331d1b80 1 -- start start 2024-05-14T09:35:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.240+0000 7fd0331d1b80 1 Processor -- start 2024-05-14T09:35:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.240+0000 7fd0331d1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b152a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.240+0000 7fd0331d1b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b152aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.240+0000 7fd0331d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b152afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.240+0000 7fd0331d1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5644b152d840 con 0x5644b139ee40 2024-05-14T09:35:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.240+0000 7fd0331d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5644b13a4c30 con 0x5644b1525890 2024-05-14T09:35:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.240+0000 7fd0331d1b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5644b152d9c0 con 0x5644b1521070 2024-05-14T09:35:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.240+0000 7fd0331d1b80 10 client.0 _ll_register_callbacks cb 0x5644b13d6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.240+0000 7fd022559700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b152a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd022559700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b152a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49116/0 (socket says 172.21.15.2:49116) 2024-05-14T09:35:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd022559700 1 -- 172.21.15.2:0/1979893661 learned_addr learned my addr 172.21.15.2:0/1979893661 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd021d58700 1 --2- 172.21.15.2:0/1979893661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b152aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd022d5a700 1 --2- 172.21.15.2:0/1979893661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b152afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd022559700 1 -- 172.21.15.2:0/1979893661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 msgr2=0x5644b152aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd022559700 1 --2- 172.21.15.2:0/1979893661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b152aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd022559700 1 -- 172.21.15.2:0/1979893661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 msgr2=0x5644b152afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd022559700 1 --2- 172.21.15.2:0/1979893661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b152afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd022559700 1 -- 172.21.15.2:0/1979893661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b13da690 con 0x5644b139ee40 2024-05-14T09:35:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd022559700 1 --2- 172.21.15.2:0/1979893661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b152a540 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd01801cd60 tx=0x7fd01801c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357ecded5a08c953 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd005ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd005ffb700 1 -- 172.21.15.2:0/1979893661 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01801ce60 con 0x5644b139ee40 2024-05-14T09:35:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd005ffb700 1 -- 172.21.15.2:0/1979893661 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd018010a50 con 0x5644b139ee40 2024-05-14T09:35:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd0331d1b80 1 -- 172.21.15.2:0/1979893661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd01801c420 con 0x5644b139ee40 2024-05-14T09:35:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd005ffb700 1 -- 172.21.15.2:0/1979893661 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd01803b050 con 0x5644b139ee40 2024-05-14T09:35:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.241+0000 7fd0331d1b80 10 client.15444 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.242+0000 7fd0331d1b80 1 -- 172.21.15.2:0/1979893661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644b15383e0 con 0x5644b139ee40 2024-05-14T09:35:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.242+0000 7fd005ffb700 1 -- 172.21.15.2:0/1979893661 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd01801c420 con 0x5644b139ee40 2024-05-14T09:35:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.242+0000 7fd005ffb700 1 -- 172.21.15.2:0/1979893661 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd018004750 con 0x5644b139ee40 2024-05-14T09:35:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.242+0000 7fd0331d1b80 10 client.15444 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.242+0000 7fd0331d1b80 10 client.15444 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T09:35:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.242+0000 7fd0331d1b80 20 client.15444 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.242+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/1979893661 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x5644b153a9c0 0x5644b153cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.242+0000 7fd0331d1b80 4 client.15444 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:35:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.242+0000 7fd0331d1b80 1 -- 172.21.15.2:0/1979893661 --> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644b144c240 con 0x5644b153a9c0 2024-05-14T09:35:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.242+0000 7fd021d58700 1 --2- 172.21.15.2:0/1979893661 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x5644b153a9c0 0x5644b153cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.243+0000 7fd005ffb700 10 client.15444 ms_handle_connect on v2:172.21.15.29:6838/3625063120 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.243+0000 7fd021d58700 1 --2- 172.21.15.2:0/1979893661 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x5644b153a9c0 0x5644b153cde0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd0171b95f9f9b2f server_cookie=8409ed6a3f5e319f in_seq=0 out_seq=0 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.243+0000 7fd005ffb700 1 -- 172.21.15.2:0/1979893661 <== mds.2 v2:172.21.15.29:6838/3625063120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5644b144c240 con 0x5644b153a9c0 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.243+0000 7fd005ffb700 10 client.15444 handle_command_reply: tid=0 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.243+0000 7fd0331d1b80 1 client.15444 shutdown 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.244+0000 7fd0331d1b80 1 -- 172.21.15.2:0/1979893661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 msgr2=0x5644b152a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.244+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/1979893661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b152a540 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd01801cd60 tx=0x7fd01801c800 comp rx=0 tx=0).stop 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.244+0000 7fd0331d1b80 1 -- 172.21.15.2:0/1979893661 shutdown_connections 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.244+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/1979893661 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x5644b153a9c0 0x5644b153cde0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.244+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/1979893661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b152afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.244+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/1979893661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b152a540 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.244+0000 7fd005ffb700 0 client.15444 ms_handle_reset on v2:172.21.15.29:6838/3625063120 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.244+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/1979893661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b152aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.244+0000 7fd0331d1b80 1 -- 172.21.15.2:0/1979893661 >> 172.21.15.2:0/1979893661 conn(0x5644b13e6600 msgr2=0x5644b151b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.244+0000 7fd0331d1b80 1 -- 172.21.15.2:0/1979893661 shutdown_connections 2024-05-14T09:35:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.244+0000 7fd0331d1b80 1 -- 172.21.15.2:0/1979893661 wait complete. 2024-05-14T09:35:45.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.244+0000 7fd0331d1b80 20 client.15444 trim_cache size 0 max 16384 2024-05-14T09:35:45.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.259+0000 7fd0331d1b80 1 Processor -- start 2024-05-14T09:35:45.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.259+0000 7fd0331d1b80 1 -- start start 2024-05-14T09:35:45.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.259+0000 7fd0331d1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b1529860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.259+0000 7fd0331d1b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b1520b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.259+0000 7fd0331d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b1536c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.259+0000 7fd0331d1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1b0ad0 con 0x5644b139ee40 2024-05-14T09:35:45.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.259+0000 7fd0331d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1ae360 con 0x5644b1525890 2024-05-14T09:35:45.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.259+0000 7fd0331d1b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1ac1b0 con 0x5644b1521070 2024-05-14T09:35:45.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.259+0000 7fd022559700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b1529860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.259+0000 7fd022559700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b1529860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49132/0 (socket says 172.21.15.2:49132) 2024-05-14T09:35:45.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.259+0000 7fd022559700 1 -- 172.21.15.2:0/783816354 learned_addr learned my addr 172.21.15.2:0/783816354 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:45.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.260+0000 7fd022d5a700 1 --2- 172.21.15.2:0/783816354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b1536c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.260+0000 7fd021d58700 1 --2- 172.21.15.2:0/783816354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b1520b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.260+0000 7fd022559700 1 -- 172.21.15.2:0/783816354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 msgr2=0x5644b1520b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:45.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.260+0000 7fd022559700 1 --2- 172.21.15.2:0/783816354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b1520b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.260+0000 7fd022559700 1 -- 172.21.15.2:0/783816354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 msgr2=0x5644b1536c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:45.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.260+0000 7fd022559700 1 --2- 172.21.15.2:0/783816354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b1536c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.260+0000 7fd022559700 1 -- 172.21.15.2:0/783816354 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b1513ae0 con 0x5644b139ee40 2024-05-14T09:35:45.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.260+0000 7fd022559700 1 --2- 172.21.15.2:0/783816354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b1529860 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd018005ec0 tx=0x7fd01800af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbb95b9c1db4a1e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:45.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.260+0000 7fd007fff700 1 -- 172.21.15.2:0/783816354 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018043020 con 0x5644b139ee40 2024-05-14T09:35:45.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.260+0000 7fd007fff700 1 -- 172.21.15.2:0/783816354 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd018032070 con 0x5644b139ee40 2024-05-14T09:35:45.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.260+0000 7fd007fff700 1 -- 172.21.15.2:0/783816354 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018016050 con 0x5644b139ee40 2024-05-14T09:35:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.261+0000 7fd0331d1b80 1 -- 172.21.15.2:0/783816354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 msgr2=0x5644b1529860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.261+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/783816354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b1529860 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd018005ec0 tx=0x7fd01800af60 comp rx=0 tx=0).stop 2024-05-14T09:35:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.261+0000 7fd0331d1b80 1 -- 172.21.15.2:0/783816354 shutdown_connections 2024-05-14T09:35:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.261+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/783816354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b1536c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.261+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/783816354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b1529860 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.261+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/783816354 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b1520b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.261+0000 7fd0331d1b80 1 -- 172.21.15.2:0/783816354 >> 172.21.15.2:0/783816354 conn(0x5644b13e6600 msgr2=0x5644b1516be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.261+0000 7fd0331d1b80 1 -- 172.21.15.2:0/783816354 shutdown_connections 2024-05-14T09:35:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.261+0000 7fd0331d1b80 1 -- 172.21.15.2:0/783816354 wait complete. 2024-05-14T09:35:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.261+0000 7fd0331d1b80 1 -- start start 2024-05-14T09:35:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.261+0000 7fd0331d1b80 1 Processor -- start 2024-05-14T09:35:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd0331d1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b153cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd0331d1b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b153d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd0331d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b153d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd0331d1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1ac820 con 0x5644b139ee40 2024-05-14T09:35:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd0331d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1adc90 con 0x5644b1525890 2024-05-14T09:35:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd0331d1b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1acef0 con 0x5644b1521070 2024-05-14T09:35:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd0331d1b80 10 client.0 _ll_register_callbacks cb 0x5644b13d6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd022559700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b153cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd022559700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b153cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49144/0 (socket says 172.21.15.2:49144) 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd022559700 1 -- 172.21.15.2:0/2945974746 learned_addr learned my addr 172.21.15.2:0/2945974746 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd022d5a700 1 --2- 172.21.15.2:0/2945974746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b153d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd021d58700 1 --2- 172.21.15.2:0/2945974746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b153d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd022559700 1 -- 172.21.15.2:0/2945974746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 msgr2=0x5644b153d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd022559700 1 --2- 172.21.15.2:0/2945974746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b153d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd022559700 1 -- 172.21.15.2:0/2945974746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 msgr2=0x5644b153d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd022559700 1 --2- 172.21.15.2:0/2945974746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b153d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd022559700 1 -- 172.21.15.2:0/2945974746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b15a3030 con 0x5644b139ee40 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd022559700 1 --2- 172.21.15.2:0/2945974746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b153cde0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd018004c10 tx=0x7fd01801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09449592c69252a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.262+0000 7fd005ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.263+0000 7fd005ffb700 1 -- 172.21.15.2:0/2945974746 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018016070 con 0x5644b139ee40 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.263+0000 7fd005ffb700 1 -- 172.21.15.2:0/2945974746 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd018010a50 con 0x5644b139ee40 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.263+0000 7fd0331d1b80 1 -- 172.21.15.2:0/2945974746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd018032070 con 0x5644b139ee40 2024-05-14T09:35:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.263+0000 7fd005ffb700 1 -- 172.21.15.2:0/2945974746 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd018056050 con 0x5644b139ee40 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.263+0000 7fd0331d1b80 10 client.15450 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.263+0000 7fd0331d1b80 1 -- 172.21.15.2:0/2945974746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644b15a4270 con 0x5644b139ee40 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.263+0000 7fd005ffb700 1 -- 172.21.15.2:0/2945974746 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd018032070 con 0x5644b139ee40 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.263+0000 7fd005ffb700 1 -- 172.21.15.2:0/2945974746 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd018048450 con 0x5644b139ee40 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.263+0000 7fd0331d1b80 10 client.15450 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.263+0000 7fd0331d1b80 10 client.15450 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.263+0000 7fd0331d1b80 20 client.15450 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.263+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/2945974746 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x5644b15a4890 0x5644b15a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.263+0000 7fd0331d1b80 4 client.15450 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.263+0000 7fd0331d1b80 1 -- 172.21.15.2:0/2945974746 --> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644b14462e0 con 0x5644b15a4890 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.264+0000 7fd021d58700 1 --2- 172.21.15.2:0/2945974746 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x5644b15a4890 0x5644b15a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.264+0000 7fd021d58700 1 --2- 172.21.15.2:0/2945974746 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x5644b15a4890 0x5644b15a6cb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1beb883a98cbf059 server_cookie=c4fb3e5c818cb6c8 in_seq=0 out_seq=0 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.264+0000 7fd005ffb700 10 client.15450 ms_handle_connect on v2:172.21.15.29:6838/3625063120 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.264+0000 7fd005ffb700 1 -- 172.21.15.2:0/2945974746 <== mds.2 v2:172.21.15.29:6838/3625063120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644b14462e0 con 0x5644b15a4890 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.264+0000 7fd005ffb700 10 client.15450 handle_command_reply: tid=0 2024-05-14T09:35:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.264+0000 7fd0331d1b80 1 client.15450 shutdown 2024-05-14T09:35:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.264+0000 7fd0331d1b80 1 -- 172.21.15.2:0/2945974746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 msgr2=0x5644b153cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.264+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/2945974746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b153cde0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd018004c10 tx=0x7fd01801c890 comp rx=0 tx=0).stop 2024-05-14T09:35:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.264+0000 7fd0331d1b80 1 -- 172.21.15.2:0/2945974746 shutdown_connections 2024-05-14T09:35:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.264+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/2945974746 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x5644b15a4890 0x5644b15a6cb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.264+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/2945974746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644b1525890 0x5644b153d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.264+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/2945974746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644b139ee40 0x5644b153cde0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.264+0000 7fd0331d1b80 1 --2- 172.21.15.2:0/2945974746 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5644b1521070 0x5644b153d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0331d1b80 1 -- 172.21.15.2:0/2945974746 >> 172.21.15.2:0/2945974746 conn(0x5644b13e6600 msgr2=0x5644b15295c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd005ffb700 0 client.15450 ms_handle_reset on v2:172.21.15.29:6838/3625063120 2024-05-14T09:35:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0331d1b80 1 -- 172.21.15.2:0/2945974746 shutdown_connections 2024-05-14T09:35:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0331d1b80 1 -- 172.21.15.2:0/2945974746 wait complete. 2024-05-14T09:35:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0331d1b80 20 client.15450 trim_cache size 0 max 16384 2024-05-14T09:35:45.271 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-05-14T09:35:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0247bd700 1 -- 172.21.15.2:0/3031313092 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd0080b2880 msgr2=0x7fd0080b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0247bd700 1 --2- 172.21.15.2:0/3031313092 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd0080b2880 0x7fd0080b4d90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd01c06c2f0 tx=0x7fd01800a3b0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0247bd700 1 -- 172.21.15.2:0/3031313092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c108200 msgr2=0x7fd01c19f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0247bd700 1 --2- 172.21.15.2:0/3031313092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c108200 0x7fd01c19f5d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd01400ada0 tx=0x7fd01400c330 comp rx=0 tx=0).stop 2024-05-14T09:35:45.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0247bd700 1 -- 172.21.15.2:0/3031313092 shutdown_connections 2024-05-14T09:35:45.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0247bd700 1 --2- 172.21.15.2:0/3031313092 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd0080b2880 0x7fd0080b4d90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0247bd700 1 --2- 172.21.15.2:0/3031313092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c108200 0x7fd01c19f5d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0247bd700 1 --2- 172.21.15.2:0/3031313092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd01c103960 0x7fd01c19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0247bd700 1 --2- 172.21.15.2:0/3031313092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd01c1042b0 0x7fd01c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:45.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0247bd700 1 -- 172.21.15.2:0/3031313092 >> 172.21.15.2:0/3031313092 conn(0x7fd01c0fe520 msgr2=0x7fd01c102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:45.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0247bd700 1 -- 172.21.15.2:0/3031313092 shutdown_connections 2024-05-14T09:35:45.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:45.265+0000 7fd0247bd700 1 -- 172.21.15.2:0/3031313092 wait complete. 2024-05-14T09:35:46.071 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T09:35:46.326 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:35:46.738 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:46 smithi002 ceph-mon[31557]: pgmap v448: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-14T09:35:46.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:46 smithi029 ceph-mon[36249]: pgmap v448: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-14T09:35:46.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:46 smithi026 ceph-mon[36056]: pgmap v448: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-14T09:35:48.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.018+0000 7fc06ae98700 1 -- 172.21.15.2:0/668658350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0641034c0 msgr2=0x7fc0641038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:48.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.018+0000 7fc06ae98700 1 --2- 172.21.15.2:0/668658350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0641034c0 0x7fc0641038d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc0580099c0 tx=0x7fc058009cd0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.018+0000 7fc06ae98700 1 -- 172.21.15.2:0/668658350 shutdown_connections 2024-05-14T09:35:48.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.018+0000 7fc06ae98700 1 --2- 172.21.15.2:0/668658350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc064108740 0x7fc0641100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.018+0000 7fc06ae98700 1 --2- 172.21.15.2:0/668658350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0641034c0 0x7fc0641038d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.018+0000 7fc06ae98700 1 --2- 172.21.15.2:0/668658350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc064107df0 0x7fc064108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.018+0000 7fc06ae98700 1 -- 172.21.15.2:0/668658350 >> 172.21.15.2:0/668658350 conn(0x7fc0640fedc0 msgr2=0x7fc0641011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:48.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.018+0000 7fc06ae98700 1 -- 172.21.15.2:0/668658350 shutdown_connections 2024-05-14T09:35:48.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.018+0000 7fc06ae98700 1 -- 172.21.15.2:0/668658350 wait complete. 2024-05-14T09:35:48.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.019+0000 7fc06ae98700 1 Processor -- start 2024-05-14T09:35:48.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.019+0000 7fc06ae98700 1 -- start start 2024-05-14T09:35:48.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.019+0000 7fc06ae98700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0641034c0 0x7fc06419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.019+0000 7fc06ae98700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc064107df0 0x7fc06419c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc06ae98700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc064108740 0x7fc0641a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc063fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc064107df0 0x7fc06419c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc06ae98700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc064113370 con 0x7fc064107df0 2024-05-14T09:35:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc06ae98700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641131f0 con 0x7fc0641034c0 2024-05-14T09:35:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc063fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc064107df0 0x7fc06419c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53126/0 (socket says 172.21.15.2:53126) 2024-05-14T09:35:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc06ae98700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641134f0 con 0x7fc064108740 2024-05-14T09:35:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc068c34700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0641034c0 0x7fc06419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc063fff700 1 -- 172.21.15.2:0/3652680803 learned_addr learned my addr 172.21.15.2:0/3652680803 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc069435700 1 --2- 172.21.15.2:0/3652680803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc064108740 0x7fc0641a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc068c34700 1 -- 172.21.15.2:0/3652680803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc064108740 msgr2=0x7fc0641a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc068c34700 1 --2- 172.21.15.2:0/3652680803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc064108740 0x7fc0641a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc068c34700 1 -- 172.21.15.2:0/3652680803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc064107df0 msgr2=0x7fc06419c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc068c34700 1 --2- 172.21.15.2:0/3652680803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc064107df0 0x7fc06419c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc068c34700 1 -- 172.21.15.2:0/3652680803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06419c9c0 con 0x7fc0641034c0 2024-05-14T09:35:48.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc069435700 1 --2- 172.21.15.2:0/3652680803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc064108740 0x7fc0641a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:35:48.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc068c34700 1 --2- 172.21.15.2:0/3652680803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0641034c0 0x7fc06419bbd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc0580098e0 tx=0x7fc0580045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:48.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.020+0000 7fc061ffb700 1 -- 172.21.15.2:0/3652680803 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058026070 con 0x7fc0641034c0 2024-05-14T09:35:48.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.021+0000 7fc061ffb700 1 -- 172.21.15.2:0/3652680803 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc05800f460 con 0x7fc0641034c0 2024-05-14T09:35:48.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.021+0000 7fc061ffb700 1 -- 172.21.15.2:0/3652680803 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058009e60 con 0x7fc0641034c0 2024-05-14T09:35:48.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.021+0000 7fc06ae98700 1 -- 172.21.15.2:0/3652680803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0641a7250 con 0x7fc0641034c0 2024-05-14T09:35:48.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.021+0000 7fc06ae98700 1 -- 172.21.15.2:0/3652680803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0641a7650 con 0x7fc0641034c0 2024-05-14T09:35:48.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.024+0000 7fc061ffb700 1 -- 172.21.15.2:0/3652680803 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc058029070 con 0x7fc0641034c0 2024-05-14T09:35:48.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.025+0000 7fc061ffb700 1 --2- 172.21.15.2:0/3652680803 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0540b2880 0x7fc0540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.025+0000 7fc063fff700 1 --2- 172.21.15.2:0/3652680803 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0540b2880 0x7fc0540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.025+0000 7fc061ffb700 1 -- 172.21.15.2:0/3652680803 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fc0580e2d50 con 0x7fc0641034c0 2024-05-14T09:35:48.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.026+0000 7fc063fff700 1 --2- 172.21.15.2:0/3652680803 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0540b2880 0x7fc0540b4d90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc064106550 tx=0x7fc050007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:48.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.026+0000 7fc0798acb80 1 Processor -- start 2024-05-14T09:35:48.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.026+0000 7fc0798acb80 1 -- start start 2024-05-14T09:35:48.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc0798acb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e7aae40 0x56526e92e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc0798acb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e92ed80 0x56526e9288d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc0798acb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e928e10 0x56526e929220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc0798acb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56526e84a940 con 0x56526e7aae40 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc0798acb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56526e7afda0 con 0x56526e92ed80 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc068c34700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e7aae40 0x56526e92e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc068c34700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e7aae40 0x56526e92e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53142/0 (socket says 172.21.15.2:53142) 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc068c34700 1 -- 172.21.15.2:0/879769938 learned_addr learned my addr 172.21.15.2:0/879769938 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc063fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e92ed80 0x56526e9288d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc0798acb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56526e846900 con 0x56526e928e10 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc069435700 1 --2- 172.21.15.2:0/879769938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e928e10 0x56526e929220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc068c34700 1 -- 172.21.15.2:0/879769938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e928e10 msgr2=0x56526e929220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc068c34700 1 --2- 172.21.15.2:0/879769938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e928e10 0x56526e929220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc068c34700 1 -- 172.21.15.2:0/879769938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e92ed80 msgr2=0x56526e9288d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc068c34700 1 --2- 172.21.15.2:0/879769938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e92ed80 0x56526e9288d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.027+0000 7fc068c34700 1 -- 172.21.15.2:0/879769938 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56526e7978d0 con 0x56526e7aae40 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.028+0000 7fc068c34700 1 --2- 172.21.15.2:0/879769938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e7aae40 0x56526e92e840 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc0580a9f70 tx=0x7fc0580a99b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76967bbbc8d2ba3b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.028+0000 7fc04effd700 1 -- 172.21.15.2:0/879769938 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0580a9af0 con 0x56526e7aae40 2024-05-14T09:35:48.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.028+0000 7fc04effd700 1 -- 172.21.15.2:0/879769938 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0580aa6e0 con 0x56526e7aae40 2024-05-14T09:35:48.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.028+0000 7fc04effd700 1 -- 172.21.15.2:0/879769938 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0580b2920 con 0x56526e7aae40 2024-05-14T09:35:48.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.029+0000 7fc0798acb80 1 -- 172.21.15.2:0/879769938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e7aae40 msgr2=0x56526e92e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:48.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.029+0000 7fc0798acb80 1 --2- 172.21.15.2:0/879769938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e7aae40 0x56526e92e840 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc0580a9f70 tx=0x7fc0580a99b0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.029+0000 7fc0798acb80 1 -- 172.21.15.2:0/879769938 shutdown_connections 2024-05-14T09:35:48.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.029+0000 7fc0798acb80 1 --2- 172.21.15.2:0/879769938 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e928e10 0x56526e929220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.029+0000 7fc0798acb80 1 --2- 172.21.15.2:0/879769938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e7aae40 0x56526e92e840 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.029+0000 7fc0798acb80 1 --2- 172.21.15.2:0/879769938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e92ed80 0x56526e9288d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.029+0000 7fc0798acb80 1 -- 172.21.15.2:0/879769938 >> 172.21.15.2:0/879769938 conn(0x56526e7f2630 msgr2=0x56526e923a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:48.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.029+0000 7fc0798acb80 1 -- 172.21.15.2:0/879769938 shutdown_connections 2024-05-14T09:35:48.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.029+0000 7fc0798acb80 1 -- 172.21.15.2:0/879769938 wait complete. 2024-05-14T09:35:48.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.030+0000 7fc0798acb80 1 -- start start 2024-05-14T09:35:48.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.030+0000 7fc0798acb80 1 Processor -- start 2024-05-14T09:35:48.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.030+0000 7fc0798acb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e7aae40 0x56526e932440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.030+0000 7fc0798acb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e928e10 0x56526e932980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.030+0000 7fc0798acb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e932ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.030+0000 7fc0798acb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56526e9355e0 con 0x56526e92ed80 2024-05-14T09:35:48.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.030+0000 7fc0798acb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56526e803ed0 con 0x56526e7aae40 2024-05-14T09:35:48.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.030+0000 7fc0798acb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56526e935760 con 0x56526e928e10 2024-05-14T09:35:48.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.030+0000 7fc0798acb80 10 client.0 _ll_register_callbacks cb 0x56526e7e2950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:48.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.031+0000 7fc069435700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e932ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.031+0000 7fc069435700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e932ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53152/0 (socket says 172.21.15.2:53152) 2024-05-14T09:35:48.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.031+0000 7fc069435700 1 -- 172.21.15.2:0/1074947228 learned_addr learned my addr 172.21.15.2:0/1074947228 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:48.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.031+0000 7fc063fff700 1 --2- 172.21.15.2:0/1074947228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e928e10 0x56526e932980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.031+0000 7fc068c34700 1 --2- 172.21.15.2:0/1074947228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e7aae40 0x56526e932440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.031+0000 7fc069435700 1 -- 172.21.15.2:0/1074947228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e928e10 msgr2=0x56526e932980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:48.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.031+0000 7fc069435700 1 --2- 172.21.15.2:0/1074947228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e928e10 0x56526e932980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.031+0000 7fc069435700 1 -- 172.21.15.2:0/1074947228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e7aae40 msgr2=0x56526e932440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:48.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.031+0000 7fc069435700 1 --2- 172.21.15.2:0/1074947228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e7aae40 0x56526e932440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.031+0000 7fc069435700 1 -- 172.21.15.2:0/1074947228 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56526e801760 con 0x56526e92ed80 2024-05-14T09:35:48.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.031+0000 7fc069435700 1 --2- 172.21.15.2:0/1074947228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e932ec0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc05c009550 tx=0x7fc05c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d9ec922d0f795a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:48.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.031+0000 7fc043fff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:48.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.031+0000 7fc043fff700 1 -- 172.21.15.2:0/1074947228 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c012aa0 con 0x56526e92ed80 2024-05-14T09:35:48.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.031+0000 7fc0798acb80 1 -- 172.21.15.2:0/1074947228 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0580aa6e0 con 0x56526e92ed80 2024-05-14T09:35:48.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.031+0000 7fc043fff700 1 -- 172.21.15.2:0/1074947228 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fc05c012c40 con 0x56526e92ed80 2024-05-14T09:35:48.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.032+0000 7fc043fff700 1 -- 172.21.15.2:0/1074947228 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc05c025050 con 0x56526e92ed80 2024-05-14T09:35:48.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.032+0000 7fc0798acb80 10 client.15462 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.032+0000 7fc0798acb80 1 -- 172.21.15.2:0/1074947228 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56526e940150 con 0x56526e92ed80 2024-05-14T09:35:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.032+0000 7fc043fff700 1 -- 172.21.15.2:0/1074947228 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0580aa6e0 con 0x56526e92ed80 2024-05-14T09:35:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.032+0000 7fc043fff700 1 -- 172.21.15.2:0/1074947228 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc05c020410 con 0x56526e92ed80 2024-05-14T09:35:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.032+0000 7fc0798acb80 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.032+0000 7fc0798acb80 10 client.15462 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T09:35:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.032+0000 7fc0798acb80 20 client.15462 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.032+0000 7fc0798acb80 1 --2- 172.21.15.2:0/1074947228 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x56526e942730 0x56526e944bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.032+0000 7fc0798acb80 4 client.15462 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:35:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.032+0000 7fc0798acb80 1 -- 172.21.15.2:0/1074947228 --> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56526e858320 con 0x56526e942730 2024-05-14T09:35:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.033+0000 7fc068c34700 1 --2- 172.21.15.2:0/1074947228 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x56526e942730 0x56526e944bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.033+0000 7fc043fff700 10 client.15462 ms_handle_connect on v2:172.21.15.26:6836/3301773578 2024-05-14T09:35:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.033+0000 7fc068c34700 1 --2- 172.21.15.2:0/1074947228 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x56526e942730 0x56526e944bb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=615f2aa25854e4f3 server_cookie=1a5d68b9eb97d076 in_seq=0 out_seq=0 2024-05-14T09:35:48.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.034+0000 7fc043fff700 1 -- 172.21.15.2:0/1074947228 <== mds.1 v2:172.21.15.26:6836/3301773578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56526e858320 con 0x56526e942730 2024-05-14T09:35:48.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.034+0000 7fc043fff700 10 client.15462 handle_command_reply: tid=0 2024-05-14T09:35:48.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.034+0000 7fc0798acb80 1 client.15462 shutdown 2024-05-14T09:35:48.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.034+0000 7fc0798acb80 1 -- 172.21.15.2:0/1074947228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 msgr2=0x56526e932ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:48.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.034+0000 7fc0798acb80 1 --2- 172.21.15.2:0/1074947228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e932ec0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc05c009550 tx=0x7fc05c007ea0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.034+0000 7fc0798acb80 1 -- 172.21.15.2:0/1074947228 shutdown_connections 2024-05-14T09:35:48.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.034+0000 7fc0798acb80 1 --2- 172.21.15.2:0/1074947228 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x56526e942730 0x56526e944bb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.034+0000 7fc0798acb80 1 --2- 172.21.15.2:0/1074947228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e7aae40 0x56526e932440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.034+0000 7fc0798acb80 1 --2- 172.21.15.2:0/1074947228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e932ec0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.034+0000 7fc0798acb80 1 --2- 172.21.15.2:0/1074947228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e928e10 0x56526e932980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.034+0000 7fc0798acb80 1 -- 172.21.15.2:0/1074947228 >> 172.21.15.2:0/1074947228 conn(0x56526e7f2630 msgr2=0x56526e9278c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:48.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.034+0000 7fc043fff700 0 client.15462 ms_handle_reset on v2:172.21.15.26:6836/3301773578 2024-05-14T09:35:48.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.034+0000 7fc0798acb80 1 -- 172.21.15.2:0/1074947228 shutdown_connections 2024-05-14T09:35:48.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.034+0000 7fc0798acb80 1 -- 172.21.15.2:0/1074947228 wait complete. 2024-05-14T09:35:48.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.034+0000 7fc0798acb80 20 client.15462 trim_cache size 0 max 16384 2024-05-14T09:35:48.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.052+0000 7fc0798acb80 1 Processor -- start 2024-05-14T09:35:48.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.052+0000 7fc0798acb80 1 -- start start 2024-05-14T09:35:48.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc0798acb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e7aae40 0x56526e9288d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc0798acb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e928e10 0x56526e92dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc0798acb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e931460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc0798acb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641b2cd0 con 0x56526e92ed80 2024-05-14T09:35:48.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc0798acb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641b0560 con 0x56526e7aae40 2024-05-14T09:35:48.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc0798acb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641ae3b0 con 0x56526e928e10 2024-05-14T09:35:48.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc069435700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e931460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc069435700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e931460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53164/0 (socket says 172.21.15.2:53164) 2024-05-14T09:35:48.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc069435700 1 -- 172.21.15.2:0/2242804441 learned_addr learned my addr 172.21.15.2:0/2242804441 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:48.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc068c34700 1 --2- 172.21.15.2:0/2242804441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e7aae40 0x56526e9288d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc063fff700 1 --2- 172.21.15.2:0/2242804441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e928e10 0x56526e92dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc069435700 1 -- 172.21.15.2:0/2242804441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e928e10 msgr2=0x56526e92dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:48.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc069435700 1 --2- 172.21.15.2:0/2242804441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e928e10 0x56526e92dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc069435700 1 -- 172.21.15.2:0/2242804441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e7aae40 msgr2=0x56526e9288d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:48.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc069435700 1 --2- 172.21.15.2:0/2242804441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e7aae40 0x56526e9288d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc069435700 1 -- 172.21.15.2:0/2242804441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56526e9244d0 con 0x56526e92ed80 2024-05-14T09:35:48.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.053+0000 7fc068c34700 1 --2- 172.21.15.2:0/2242804441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e7aae40 0x56526e9288d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:35:48.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.054+0000 7fc069435700 1 --2- 172.21.15.2:0/2242804441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e931460 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc05c000f80 tx=0x7fc05c006180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b289f0a8322b2d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:48.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.054+0000 7fc04e7fc700 1 -- 172.21.15.2:0/2242804441 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c0203f0 con 0x56526e92ed80 2024-05-14T09:35:48.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.054+0000 7fc04e7fc700 1 -- 172.21.15.2:0/2242804441 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc05c020590 con 0x56526e92ed80 2024-05-14T09:35:48.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.054+0000 7fc04e7fc700 1 -- 172.21.15.2:0/2242804441 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c012ca0 con 0x56526e92ed80 2024-05-14T09:35:48.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.055+0000 7fc0798acb80 1 -- 172.21.15.2:0/2242804441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 msgr2=0x56526e931460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:48.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.055+0000 7fc0798acb80 1 --2- 172.21.15.2:0/2242804441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e931460 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc05c000f80 tx=0x7fc05c006180 comp rx=0 tx=0).stop 2024-05-14T09:35:48.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.055+0000 7fc0798acb80 1 -- 172.21.15.2:0/2242804441 shutdown_connections 2024-05-14T09:35:48.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.055+0000 7fc0798acb80 1 --2- 172.21.15.2:0/2242804441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e7aae40 0x56526e9288d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.055+0000 7fc0798acb80 1 --2- 172.21.15.2:0/2242804441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e931460 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.055+0000 7fc0798acb80 1 --2- 172.21.15.2:0/2242804441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e928e10 0x56526e92dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.055+0000 7fc0798acb80 1 -- 172.21.15.2:0/2242804441 >> 172.21.15.2:0/2242804441 conn(0x56526e7f2630 msgr2=0x56526e930d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:48.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.055+0000 7fc0798acb80 1 -- 172.21.15.2:0/2242804441 shutdown_connections 2024-05-14T09:35:48.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.055+0000 7fc0798acb80 1 -- 172.21.15.2:0/2242804441 wait complete. 2024-05-14T09:35:48.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.055+0000 7fc0798acb80 1 -- start start 2024-05-14T09:35:48.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.056+0000 7fc0798acb80 1 Processor -- start 2024-05-14T09:35:48.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.056+0000 7fc0798acb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e7aae40 0x56526e93abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.056+0000 7fc0798acb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e928e10 0x56526e949460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.056+0000 7fc0798acb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e9499a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.056+0000 7fc0798acb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641aea20 con 0x56526e92ed80 2024-05-14T09:35:48.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.056+0000 7fc0798acb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641afe90 con 0x56526e928e10 2024-05-14T09:35:48.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.056+0000 7fc0798acb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641af0f0 con 0x56526e7aae40 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.056+0000 7fc0798acb80 10 client.0 _ll_register_callbacks cb 0x56526e7e2950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.056+0000 7fc069435700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e9499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.056+0000 7fc069435700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e9499a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53172/0 (socket says 172.21.15.2:53172) 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.056+0000 7fc069435700 1 -- 172.21.15.2:0/4188876988 learned_addr learned my addr 172.21.15.2:0/4188876988 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.056+0000 7fc068c34700 1 --2- 172.21.15.2:0/4188876988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e7aae40 0x56526e93abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.057+0000 7fc069435700 1 -- 172.21.15.2:0/4188876988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e7aae40 msgr2=0x56526e93abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.057+0000 7fc069435700 1 --2- 172.21.15.2:0/4188876988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e7aae40 0x56526e93abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.057+0000 7fc069435700 1 -- 172.21.15.2:0/4188876988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e928e10 msgr2=0x56526e949460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.057+0000 7fc069435700 1 --2- 172.21.15.2:0/4188876988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e928e10 0x56526e949460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.057+0000 7fc069435700 1 -- 172.21.15.2:0/4188876988 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56526e93b3d0 con 0x56526e92ed80 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.057+0000 7fc069435700 1 --2- 172.21.15.2:0/4188876988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e9499a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00b010 tx=0x7fc05c005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effa857610b9be67 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.057+0000 7fc043fff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.057+0000 7fc043fff700 1 -- 172.21.15.2:0/4188876988 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c004c70 con 0x56526e92ed80 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.057+0000 7fc0798acb80 1 -- 172.21.15.2:0/4188876988 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc05c020590 con 0x56526e92ed80 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.058+0000 7fc043fff700 1 -- 172.21.15.2:0/4188876988 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fc05c0128a0 con 0x56526e92ed80 2024-05-14T09:35:48.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.058+0000 7fc043fff700 1 -- 172.21.15.2:0/4188876988 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc05c043050 con 0x56526e92ed80 2024-05-14T09:35:48.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.058+0000 7fc0798acb80 10 client.15468 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:48.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.058+0000 7fc0798acb80 1 -- 172.21.15.2:0/4188876988 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56526e9b0dd0 con 0x56526e92ed80 2024-05-14T09:35:48.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.058+0000 7fc043fff700 1 -- 172.21.15.2:0/4188876988 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc05c020590 con 0x56526e92ed80 2024-05-14T09:35:48.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.058+0000 7fc043fff700 1 -- 172.21.15.2:0/4188876988 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc05c017c40 con 0x56526e92ed80 2024-05-14T09:35:48.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.058+0000 7fc0798acb80 10 client.15468 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:48.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.058+0000 7fc0798acb80 10 client.15468 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T09:35:48.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.058+0000 7fc0798acb80 20 client.15468 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:48.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.058+0000 7fc0798acb80 1 --2- 172.21.15.2:0/4188876988 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x56526e9b1360 0x56526e9b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:48.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.058+0000 7fc0798acb80 4 client.15468 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T09:35:48.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.058+0000 7fc0798acb80 1 -- 172.21.15.2:0/4188876988 --> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56526e8523c0 con 0x56526e9b1360 2024-05-14T09:35:48.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.059+0000 7fc068c34700 1 --2- 172.21.15.2:0/4188876988 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x56526e9b1360 0x56526e9b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:48.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.059+0000 7fc043fff700 10 client.15468 ms_handle_connect on v2:172.21.15.26:6836/3301773578 2024-05-14T09:35:48.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.059+0000 7fc068c34700 1 --2- 172.21.15.2:0/4188876988 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x56526e9b1360 0x56526e9b3780 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ddcd9c68547c23b6 server_cookie=fcc877c0a73e3638 in_seq=0 out_seq=0 2024-05-14T09:35:48.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.059+0000 7fc043fff700 1 -- 172.21.15.2:0/4188876988 <== mds.1 v2:172.21.15.26:6836/3301773578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56526e8523c0 con 0x56526e9b1360 2024-05-14T09:35:48.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.059+0000 7fc043fff700 10 client.15468 handle_command_reply: tid=0 2024-05-14T09:35:48.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.059+0000 7fc0798acb80 1 client.15468 shutdown 2024-05-14T09:35:48.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.059+0000 7fc0798acb80 1 -- 172.21.15.2:0/4188876988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 msgr2=0x56526e9499a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:48.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.060+0000 7fc0798acb80 1 --2- 172.21.15.2:0/4188876988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e9499a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00b010 tx=0x7fc05c005ce0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.060+0000 7fc0798acb80 1 -- 172.21.15.2:0/4188876988 shutdown_connections 2024-05-14T09:35:48.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.060+0000 7fc0798acb80 1 --2- 172.21.15.2:0/4188876988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56526e92ed80 0x56526e9499a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.060+0000 7fc0798acb80 1 --2- 172.21.15.2:0/4188876988 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56526e7aae40 0x56526e93abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.060+0000 7fc0798acb80 1 --2- 172.21.15.2:0/4188876988 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x56526e9b1360 0x56526e9b3780 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.060+0000 7fc0798acb80 1 --2- 172.21.15.2:0/4188876988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56526e928e10 0x56526e949460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.060+0000 7fc0798acb80 1 -- 172.21.15.2:0/4188876988 >> 172.21.15.2:0/4188876988 conn(0x56526e7f2630 msgr2=0x56526e923a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:48.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.060+0000 7fc043fff700 0 client.15468 ms_handle_reset on v2:172.21.15.26:6836/3301773578 2024-05-14T09:35:48.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.060+0000 7fc0798acb80 1 -- 172.21.15.2:0/4188876988 shutdown_connections 2024-05-14T09:35:48.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.060+0000 7fc0798acb80 1 -- 172.21.15.2:0/4188876988 wait complete. 2024-05-14T09:35:48.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.060+0000 7fc0798acb80 20 client.15468 trim_cache size 0 max 16384 2024-05-14T09:35:48.063 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-05-14T09:35:48.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.061+0000 7fc06ae98700 1 -- 172.21.15.2:0/3652680803 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0540b2880 msgr2=0x7fc0540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:48.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.061+0000 7fc06ae98700 1 --2- 172.21.15.2:0/3652680803 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0540b2880 0x7fc0540b4d90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc064106550 tx=0x7fc050007480 comp rx=0 tx=0).stop 2024-05-14T09:35:48.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.061+0000 7fc06ae98700 1 -- 172.21.15.2:0/3652680803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0641034c0 msgr2=0x7fc06419bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:48.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.061+0000 7fc06ae98700 1 --2- 172.21.15.2:0/3652680803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0641034c0 0x7fc06419bbd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc0580098e0 tx=0x7fc0580045e0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.061+0000 7fc06ae98700 1 -- 172.21.15.2:0/3652680803 shutdown_connections 2024-05-14T09:35:48.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.061+0000 7fc06ae98700 1 --2- 172.21.15.2:0/3652680803 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0540b2880 0x7fc0540b4d90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.061+0000 7fc06ae98700 1 --2- 172.21.15.2:0/3652680803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0641034c0 0x7fc06419bbd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.061+0000 7fc06ae98700 1 --2- 172.21.15.2:0/3652680803 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc064108740 0x7fc0641a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.061+0000 7fc06ae98700 1 --2- 172.21.15.2:0/3652680803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc064107df0 0x7fc06419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:48.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.061+0000 7fc06ae98700 1 -- 172.21.15.2:0/3652680803 >> 172.21.15.2:0/3652680803 conn(0x7fc0640fedc0 msgr2=0x7fc064101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:48.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.061+0000 7fc06ae98700 1 -- 172.21.15.2:0/3652680803 shutdown_connections 2024-05-14T09:35:48.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:48.061+0000 7fc06ae98700 1 -- 172.21.15.2:0/3652680803 wait complete. 2024-05-14T09:35:48.811 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:48 smithi002 ceph-mon[31557]: pgmap v449: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-14T09:35:48.840 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T09:35:48.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:48 smithi029 ceph-mon[36249]: pgmap v449: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-14T09:35:48.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:48 smithi026 ceph-mon[36056]: pgmap v449: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-14T09:35:49.103 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:35:50.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.727+0000 7fe24db8a700 1 -- 172.21.15.2:0/367584876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2481034c0 msgr2=0x7fe2481038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:50.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.727+0000 7fe24db8a700 1 --2- 172.21.15.2:0/367584876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2481034c0 0x7fe2481038d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe2380099c0 tx=0x7fe238009cd0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:50 smithi002 ceph-mon[31557]: pgmap v450: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-14T09:35:50.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.727+0000 7fe24db8a700 1 -- 172.21.15.2:0/367584876 shutdown_connections 2024-05-14T09:35:50.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.727+0000 7fe24db8a700 1 --2- 172.21.15.2:0/367584876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe248108740 0x7fe2481100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.727+0000 7fe24db8a700 1 --2- 172.21.15.2:0/367584876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2481034c0 0x7fe2481038d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.727+0000 7fe24db8a700 1 --2- 172.21.15.2:0/367584876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe248107df0 0x7fe248108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.727+0000 7fe24db8a700 1 -- 172.21.15.2:0/367584876 >> 172.21.15.2:0/367584876 conn(0x7fe2480fedc0 msgr2=0x7fe2481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:50.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.728+0000 7fe24db8a700 1 -- 172.21.15.2:0/367584876 shutdown_connections 2024-05-14T09:35:50.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.728+0000 7fe24db8a700 1 -- 172.21.15.2:0/367584876 wait complete. 2024-05-14T09:35:50.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.729+0000 7fe24db8a700 1 Processor -- start 2024-05-14T09:35:50.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.729+0000 7fe24db8a700 1 -- start start 2024-05-14T09:35:50.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.729+0000 7fe24db8a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2481034c0 0x7fe2480ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.729+0000 7fe24db8a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe248107df0 0x7fe2480ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.729+0000 7fe24db8a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe248108740 0x7fe2480ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.729+0000 7fe24db8a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe248113370 con 0x7fe2481034c0 2024-05-14T09:35:50.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.729+0000 7fe24db8a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481131f0 con 0x7fe248107df0 2024-05-14T09:35:50.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.729+0000 7fe24db8a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481134f0 con 0x7fe248108740 2024-05-14T09:35:50.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.730+0000 7fe2477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2481034c0 0x7fe2480ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.730+0000 7fe2477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2481034c0 0x7fe2480ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53190/0 (socket says 172.21.15.2:53190) 2024-05-14T09:35:50.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.730+0000 7fe2477fe700 1 -- 172.21.15.2:0/2475599170 learned_addr learned my addr 172.21.15.2:0/2475599170 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:50.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.730+0000 7fe247fff700 1 --2- 172.21.15.2:0/2475599170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe248108740 0x7fe2480ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.730+0000 7fe246ffd700 1 --2- 172.21.15.2:0/2475599170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe248107df0 0x7fe2480ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.730+0000 7fe2477fe700 1 -- 172.21.15.2:0/2475599170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe248108740 msgr2=0x7fe2480ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:50.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.730+0000 7fe2477fe700 1 --2- 172.21.15.2:0/2475599170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe248108740 0x7fe2480ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.730+0000 7fe2477fe700 1 -- 172.21.15.2:0/2475599170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe248107df0 msgr2=0x7fe2480ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:50.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.730+0000 7fe2477fe700 1 --2- 172.21.15.2:0/2475599170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe248107df0 0x7fe2480ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.730+0000 7fe2477fe700 1 -- 172.21.15.2:0/2475599170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2481a91b0 con 0x7fe2481034c0 2024-05-14T09:35:50.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.730+0000 7fe2477fe700 1 --2- 172.21.15.2:0/2475599170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2481034c0 0x7fe2480ff2a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe238006010 tx=0x7fe2380045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:50.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.730+0000 7fe246ffd700 1 --2- 172.21.15.2:0/2475599170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe248107df0 0x7fe2480ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:35:50.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.730+0000 7fe244ff9700 1 -- 172.21.15.2:0/2475599170 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238026070 con 0x7fe2481034c0 2024-05-14T09:35:50.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.731+0000 7fe244ff9700 1 -- 172.21.15.2:0/2475599170 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe23800f460 con 0x7fe2481034c0 2024-05-14T09:35:50.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.731+0000 7fe244ff9700 1 -- 172.21.15.2:0/2475599170 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238009e60 con 0x7fe2481034c0 2024-05-14T09:35:50.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.731+0000 7fe24db8a700 1 -- 172.21.15.2:0/2475599170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2481a9440 con 0x7fe2481034c0 2024-05-14T09:35:50.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.731+0000 7fe24db8a700 1 -- 172.21.15.2:0/2475599170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2481a9840 con 0x7fe2481034c0 2024-05-14T09:35:50.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.734+0000 7fe244ff9700 1 -- 172.21.15.2:0/2475599170 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe23800f610 con 0x7fe2481034c0 2024-05-14T09:35:50.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.735+0000 7fe244ff9700 1 --2- 172.21.15.2:0/2475599170 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe2300b2880 0x7fe2300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.735+0000 7fe246ffd700 1 --2- 172.21.15.2:0/2475599170 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe2300b2880 0x7fe2300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.735+0000 7fe244ff9700 1 -- 172.21.15.2:0/2475599170 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fe2380e2ca0 con 0x7fe2481034c0 2024-05-14T09:35:50.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.736+0000 7fe25c59eb80 1 Processor -- start 2024-05-14T09:35:50.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.736+0000 7fe25c59eb80 1 -- start start 2024-05-14T09:35:50.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.736+0000 7fe246ffd700 1 --2- 172.21.15.2:0/2475599170 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe2300b2880 0x7fe2300b4d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe2481023f0 tx=0x7fe23c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:50.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.736+0000 7fe25c59eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016aece40 0x55f016c6a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.736+0000 7fe25c59eb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016c6ac60 0x55f016c6ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.736+0000 7fe25c59eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6f470 0x55f016c6f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.736+0000 7fe25c59eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f016b8c860 con 0x55f016aece40 2024-05-14T09:35:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.736+0000 7fe25c59eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f016b2b7c0 con 0x55f016c6f470 2024-05-14T09:35:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.736+0000 7fe25c59eb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f016b88820 con 0x55f016c6ac60 2024-05-14T09:35:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.736+0000 7fe2477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016aece40 0x55f016c6a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.736+0000 7fe2477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016aece40 0x55f016c6a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53194/0 (socket says 172.21.15.2:53194) 2024-05-14T09:35:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.736+0000 7fe2477fe700 1 -- 172.21.15.2:0/793879494 learned_addr learned my addr 172.21.15.2:0/793879494 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.736+0000 7fe246ffd700 1 --2- 172.21.15.2:0/793879494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016c6ac60 0x55f016c6ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.736+0000 7fe247fff700 1 --2- 172.21.15.2:0/793879494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6f470 0x55f016c6f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.737+0000 7fe2477fe700 1 -- 172.21.15.2:0/793879494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016c6ac60 msgr2=0x55f016c6ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.737+0000 7fe2477fe700 1 --2- 172.21.15.2:0/793879494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016c6ac60 0x55f016c6ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.737+0000 7fe2477fe700 1 -- 172.21.15.2:0/793879494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6f470 msgr2=0x55f016c6f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.737+0000 7fe2477fe700 1 --2- 172.21.15.2:0/793879494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6f470 0x55f016c6f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.737+0000 7fe2477fe700 1 -- 172.21.15.2:0/793879494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f016ad98d0 con 0x55f016aece40 2024-05-14T09:35:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.737+0000 7fe247fff700 1 --2- 172.21.15.2:0/793879494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6f470 0x55f016c6f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.737+0000 7fe2477fe700 1 --2- 172.21.15.2:0/793879494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016aece40 0x55f016c6a720 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe2380a71a0 tx=0x7fe2380a5f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62eb60d30aca288 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.737+0000 7fe22dffb700 1 -- 172.21.15.2:0/793879494 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2380aa630 con 0x55f016aece40 2024-05-14T09:35:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.737+0000 7fe22dffb700 1 -- 172.21.15.2:0/793879494 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2380a9f30 con 0x55f016aece40 2024-05-14T09:35:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.737+0000 7fe22dffb700 1 -- 172.21.15.2:0/793879494 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2380b2ad0 con 0x55f016aece40 2024-05-14T09:35:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.738+0000 7fe25c59eb80 1 -- 172.21.15.2:0/793879494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016aece40 msgr2=0x55f016c6a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.738+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/793879494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016aece40 0x55f016c6a720 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe2380a71a0 tx=0x7fe2380a5f00 comp rx=0 tx=0).stop 2024-05-14T09:35:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.738+0000 7fe25c59eb80 1 -- 172.21.15.2:0/793879494 shutdown_connections 2024-05-14T09:35:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.738+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/793879494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6f470 0x55f016c6f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.738+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/793879494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016aece40 0x55f016c6a720 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.738+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/793879494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016c6ac60 0x55f016c6ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.738+0000 7fe25c59eb80 1 -- 172.21.15.2:0/793879494 >> 172.21.15.2:0/793879494 conn(0x55f016b34600 msgr2=0x55f016c654b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.738+0000 7fe25c59eb80 1 -- 172.21.15.2:0/793879494 shutdown_connections 2024-05-14T09:35:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.738+0000 7fe25c59eb80 1 -- 172.21.15.2:0/793879494 wait complete. 2024-05-14T09:35:50.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.739+0000 7fe25c59eb80 1 -- start start 2024-05-14T09:35:50.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.739+0000 7fe25c59eb80 1 Processor -- start 2024-05-14T09:35:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.739+0000 7fe25c59eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016aece40 0x55f016c74170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.739+0000 7fe25c59eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6ac60 0x55f016c746b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.739+0000 7fe25c59eb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016c6f470 0x55f016c74bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.739+0000 7fe25c59eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f016b8c860 con 0x55f016aece40 2024-05-14T09:35:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.739+0000 7fe25c59eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f016af2c30 con 0x55f016c6ac60 2024-05-14T09:35:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.739+0000 7fe25c59eb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f016c771e0 con 0x55f016c6f470 2024-05-14T09:35:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.739+0000 7fe25c59eb80 10 client.0 _ll_register_callbacks cb 0x55f016b24920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.739+0000 7fe2477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016aece40 0x55f016c74170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.740+0000 7fe2477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016aece40 0x55f016c74170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53202/0 (socket says 172.21.15.2:53202) 2024-05-14T09:35:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.740+0000 7fe2477fe700 1 -- 172.21.15.2:0/2870518431 learned_addr learned my addr 172.21.15.2:0/2870518431 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.740+0000 7fe246ffd700 1 --2- 172.21.15.2:0/2870518431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6ac60 0x55f016c746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.740+0000 7fe247fff700 1 --2- 172.21.15.2:0/2870518431 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016c6f470 0x55f016c74bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.740+0000 7fe2477fe700 1 -- 172.21.15.2:0/2870518431 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016c6f470 msgr2=0x55f016c74bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.740+0000 7fe2477fe700 1 --2- 172.21.15.2:0/2870518431 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016c6f470 0x55f016c74bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.740+0000 7fe2477fe700 1 -- 172.21.15.2:0/2870518431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6ac60 msgr2=0x55f016c746b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.740+0000 7fe2477fe700 1 --2- 172.21.15.2:0/2870518431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6ac60 0x55f016c746b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.740+0000 7fe2477fe700 1 -- 172.21.15.2:0/2870518431 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f016b28690 con 0x55f016aece40 2024-05-14T09:35:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.740+0000 7fe2477fe700 1 --2- 172.21.15.2:0/2870518431 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016aece40 0x55f016c74170 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe238005f50 tx=0x7fe2380b2c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435a66cfbabf322a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.740+0000 7fe226ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.740+0000 7fe226ffd700 1 -- 172.21.15.2:0/2870518431 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2380b2d00 con 0x55f016aece40 2024-05-14T09:35:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.740+0000 7fe226ffd700 1 -- 172.21.15.2:0/2870518431 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fe2380a93a0 con 0x55f016aece40 2024-05-14T09:35:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.740+0000 7fe25c59eb80 1 -- 172.21.15.2:0/2870518431 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2380a9f30 con 0x55f016aece40 2024-05-14T09:35:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.741+0000 7fe226ffd700 1 -- 172.21.15.2:0/2870518431 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe2380aa630 con 0x55f016aece40 2024-05-14T09:35:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.741+0000 7fe25c59eb80 10 client.15480 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.741+0000 7fe25c59eb80 1 -- 172.21.15.2:0/2870518431 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f016c81ba0 con 0x55f016aece40 2024-05-14T09:35:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.741+0000 7fe226ffd700 1 -- 172.21.15.2:0/2870518431 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2380a9f30 con 0x55f016aece40 2024-05-14T09:35:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.741+0000 7fe226ffd700 1 -- 172.21.15.2:0/2870518431 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe2380c0020 con 0x55f016aece40 2024-05-14T09:35:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.741+0000 7fe25c59eb80 10 client.15480 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.741+0000 7fe25c59eb80 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.741+0000 7fe25c59eb80 20 client.15480 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.741+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/2870518431 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f016c84070 0x55f016c864e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.741+0000 7fe25c59eb80 4 client.15480 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:35:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.741+0000 7fe25c59eb80 1 -- 172.21.15.2:0/2870518431 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f016b9a240 con 0x55f016c84070 2024-05-14T09:35:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.741+0000 7fe246ffd700 1 --2- 172.21.15.2:0/2870518431 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f016c84070 0x55f016c864e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.742+0000 7fe246ffd700 1 --2- 172.21.15.2:0/2870518431 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f016c84070 0x55f016c864e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe4811b5e21c615 server_cookie=6bbc3c98776d4456 in_seq=0 out_seq=0 2024-05-14T09:35:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.742+0000 7fe226ffd700 10 client.15480 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.743+0000 7fe226ffd700 1 -- 172.21.15.2:0/2870518431 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f016b9a240 con 0x55f016c84070 2024-05-14T09:35:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.743+0000 7fe226ffd700 10 client.15480 handle_command_reply: tid=0 2024-05-14T09:35:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.743+0000 7fe25c59eb80 1 client.15480 shutdown 2024-05-14T09:35:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.743+0000 7fe25c59eb80 1 -- 172.21.15.2:0/2870518431 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016aece40 msgr2=0x55f016c74170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.743+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/2870518431 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016aece40 0x55f016c74170 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe238005f50 tx=0x7fe2380b2c20 comp rx=0 tx=0).stop 2024-05-14T09:35:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.743+0000 7fe25c59eb80 1 -- 172.21.15.2:0/2870518431 shutdown_connections 2024-05-14T09:35:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.743+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/2870518431 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f016c84070 0x55f016c864e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.743+0000 7fe226ffd700 0 client.15480 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:50.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.743+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/2870518431 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016c6f470 0x55f016c74bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.743+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/2870518431 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016aece40 0x55f016c74170 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.743+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/2870518431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6ac60 0x55f016c746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.743+0000 7fe25c59eb80 1 -- 172.21.15.2:0/2870518431 >> 172.21.15.2:0/2870518431 conn(0x55f016b34600 msgr2=0x55f016c69450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:50.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.743+0000 7fe25c59eb80 1 -- 172.21.15.2:0/2870518431 shutdown_connections 2024-05-14T09:35:50.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.744+0000 7fe25c59eb80 1 -- 172.21.15.2:0/2870518431 wait complete. 2024-05-14T09:35:50.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.744+0000 7fe25c59eb80 20 client.15480 trim_cache size 0 max 16384 2024-05-14T09:35:50.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.759+0000 7fe25c59eb80 1 Processor -- start 2024-05-14T09:35:50.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.759+0000 7fe25c59eb80 1 -- start start 2024-05-14T09:35:50.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.759+0000 7fe25c59eb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016aece40 0x55f016c74a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.759+0000 7fe25c59eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016c6ac60 0x55f016c73170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.759+0000 7fe25c59eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6f470 0x55f016c800b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.760+0000 7fe25c59eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481b4f80 con 0x55f016c6ac60 2024-05-14T09:35:50.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.760+0000 7fe25c59eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481b2810 con 0x55f016c6f470 2024-05-14T09:35:50.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.760+0000 7fe25c59eb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481b0660 con 0x55f016aece40 2024-05-14T09:35:50.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.760+0000 7fe246ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016c6ac60 0x55f016c73170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.760+0000 7fe246ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016c6ac60 0x55f016c73170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53208/0 (socket says 172.21.15.2:53208) 2024-05-14T09:35:50.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.760+0000 7fe246ffd700 1 -- 172.21.15.2:0/3459333554 learned_addr learned my addr 172.21.15.2:0/3459333554 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:50.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.760+0000 7fe247fff700 1 --2- 172.21.15.2:0/3459333554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6f470 0x55f016c800b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.760+0000 7fe2477fe700 1 --2- 172.21.15.2:0/3459333554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016aece40 0x55f016c74a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.760+0000 7fe246ffd700 1 -- 172.21.15.2:0/3459333554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016aece40 msgr2=0x55f016c74a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:50.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.760+0000 7fe246ffd700 1 --2- 172.21.15.2:0/3459333554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016aece40 0x55f016c74a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.760+0000 7fe246ffd700 1 -- 172.21.15.2:0/3459333554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6f470 msgr2=0x55f016c800b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:50.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.760+0000 7fe246ffd700 1 --2- 172.21.15.2:0/3459333554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6f470 0x55f016c800b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.760+0000 7fe246ffd700 1 -- 172.21.15.2:0/3459333554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f016c74f70 con 0x55f016c6ac60 2024-05-14T09:35:50.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.760+0000 7fe246ffd700 1 --2- 172.21.15.2:0/3459333554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016c6ac60 0x55f016c73170 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe23c005950 tx=0x7fe23c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e275d22dc43e8a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:50.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.760+0000 7fe22d7fa700 1 -- 172.21.15.2:0/3459333554 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c004020 con 0x55f016c6ac60 2024-05-14T09:35:50.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.761+0000 7fe22d7fa700 1 -- 172.21.15.2:0/3459333554 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe23c003680 con 0x55f016c6ac60 2024-05-14T09:35:50.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.761+0000 7fe22d7fa700 1 -- 172.21.15.2:0/3459333554 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c028530 con 0x55f016c6ac60 2024-05-14T09:35:50.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.761+0000 7fe25c59eb80 1 -- 172.21.15.2:0/3459333554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016c6ac60 msgr2=0x55f016c73170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:50.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.761+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/3459333554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016c6ac60 0x55f016c73170 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe23c005950 tx=0x7fe23c013000 comp rx=0 tx=0).stop 2024-05-14T09:35:50.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.762+0000 7fe25c59eb80 1 -- 172.21.15.2:0/3459333554 shutdown_connections 2024-05-14T09:35:50.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.762+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/3459333554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6f470 0x55f016c800b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.762+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/3459333554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016aece40 0x55f016c74a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.762+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/3459333554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016c6ac60 0x55f016c73170 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.762+0000 7fe25c59eb80 1 -- 172.21.15.2:0/3459333554 >> 172.21.15.2:0/3459333554 conn(0x55f016b34600 msgr2=0x55f016c72a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:50.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.762+0000 7fe25c59eb80 1 -- 172.21.15.2:0/3459333554 shutdown_connections 2024-05-14T09:35:50.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.762+0000 7fe25c59eb80 1 -- 172.21.15.2:0/3459333554 wait complete. 2024-05-14T09:35:50.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.762+0000 7fe25c59eb80 1 -- start start 2024-05-14T09:35:50.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.762+0000 7fe25c59eb80 1 Processor -- start 2024-05-14T09:35:50.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe25c59eb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016aece40 0x55f016c7c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe25c59eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6ac60 0x55f016c7c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe25c59eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016c6f470 0x55f016c8ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe25c59eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481b0cd0 con 0x55f016c6f470 2024-05-14T09:35:50.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe25c59eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481b2140 con 0x55f016c6ac60 2024-05-14T09:35:50.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe25c59eb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481b13a0 con 0x55f016aece40 2024-05-14T09:35:50.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe247fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016c6f470 0x55f016c8ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe2477fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016aece40 0x55f016c7c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe25c59eb80 10 client.0 _ll_register_callbacks cb 0x55f016b24920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:50.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe247fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016c6f470 0x55f016c8ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53212/0 (socket says 172.21.15.2:53212) 2024-05-14T09:35:50.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe247fff700 1 -- 172.21.15.2:0/3132307345 learned_addr learned my addr 172.21.15.2:0/3132307345 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:50.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe2477fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016aece40 0x55f016c7c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:56676/0 (socket says 172.21.15.2:56676) 2024-05-14T09:35:50.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe246ffd700 1 --2- 172.21.15.2:0/3132307345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6ac60 0x55f016c7c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe247fff700 1 -- 172.21.15.2:0/3132307345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016aece40 msgr2=0x55f016c7c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:50.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe247fff700 1 --2- 172.21.15.2:0/3132307345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016aece40 0x55f016c7c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe247fff700 1 -- 172.21.15.2:0/3132307345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6ac60 msgr2=0x55f016c7c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:50.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe247fff700 1 --2- 172.21.15.2:0/3132307345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6ac60 0x55f016c7c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe247fff700 1 -- 172.21.15.2:0/3132307345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f016c8b220 con 0x55f016c6f470 2024-05-14T09:35:50.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.763+0000 7fe2477fe700 1 --2- 172.21.15.2:0/3132307345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016aece40 0x55f016c7c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:50.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.764+0000 7fe247fff700 1 --2- 172.21.15.2:0/3132307345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016c6f470 0x55f016c8ace0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe240009360 tx=0x7fe240009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4729726947881cf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:50.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.764+0000 7fe226ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:50.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.764+0000 7fe226ffd700 1 -- 172.21.15.2:0/3132307345 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240004db0 con 0x55f016c6f470 2024-05-14T09:35:50.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.764+0000 7fe226ffd700 1 -- 172.21.15.2:0/3132307345 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fe2400178e0 con 0x55f016c6f470 2024-05-14T09:35:50.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.764+0000 7fe25c59eb80 1 -- 172.21.15.2:0/3132307345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe23c003680 con 0x55f016c6f470 2024-05-14T09:35:50.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.764+0000 7fe226ffd700 1 -- 172.21.15.2:0/3132307345 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe24002a050 con 0x55f016c6f470 2024-05-14T09:35:50.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.764+0000 7fe25c59eb80 10 client.15486 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:50.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.764+0000 7fe25c59eb80 1 -- 172.21.15.2:0/3132307345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f016cedfc0 con 0x55f016c6f470 2024-05-14T09:35:50.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.765+0000 7fe226ffd700 1 -- 172.21.15.2:0/3132307345 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe23c003680 con 0x55f016c6f470 2024-05-14T09:35:50.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.765+0000 7fe226ffd700 1 -- 172.21.15.2:0/3132307345 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe240018420 con 0x55f016c6f470 2024-05-14T09:35:50.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.765+0000 7fe25c59eb80 10 client.15486 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:50.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.765+0000 7fe25c59eb80 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:50.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.765+0000 7fe25c59eb80 20 client.15486 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:50.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.765+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/3132307345 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f016cee700 0x55f016cf0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:50.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.765+0000 7fe25c59eb80 4 client.15486 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T09:35:50.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.765+0000 7fe25c59eb80 1 -- 172.21.15.2:0/3132307345 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f016b942e0 con 0x55f016cee700 2024-05-14T09:35:50.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.765+0000 7fe2477fe700 1 --2- 172.21.15.2:0/3132307345 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f016cee700 0x55f016cf0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:50.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe226ffd700 10 client.15486 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:50.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe2477fe700 1 --2- 172.21.15.2:0/3132307345 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f016cee700 0x55f016cf0b20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c97023699c8c981c server_cookie=c3fa6c664bdee0ff in_seq=0 out_seq=0 2024-05-14T09:35:50.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe226ffd700 1 -- 172.21.15.2:0/3132307345 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f016b942e0 con 0x55f016cee700 2024-05-14T09:35:50.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe226ffd700 10 client.15486 handle_command_reply: tid=0 2024-05-14T09:35:50.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe25c59eb80 1 client.15486 shutdown 2024-05-14T09:35:50.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe25c59eb80 1 -- 172.21.15.2:0/3132307345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016c6f470 msgr2=0x55f016c8ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:50.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/3132307345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016c6f470 0x55f016c8ace0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe240009360 tx=0x7fe240009390 comp rx=0 tx=0).stop 2024-05-14T09:35:50.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe25c59eb80 1 -- 172.21.15.2:0/3132307345 shutdown_connections 2024-05-14T09:35:50.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/3132307345 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f016cee700 0x55f016cf0b20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/3132307345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f016c6f470 0x55f016c8ace0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/3132307345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f016aece40 0x55f016c7c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe25c59eb80 1 --2- 172.21.15.2:0/3132307345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f016c6ac60 0x55f016c7c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe25c59eb80 1 -- 172.21.15.2:0/3132307345 >> 172.21.15.2:0/3132307345 conn(0x55f016b34600 msgr2=0x55f016c64be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:50.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe226ffd700 0 client.15486 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:50.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe25c59eb80 1 -- 172.21.15.2:0/3132307345 shutdown_connections 2024-05-14T09:35:50.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.766+0000 7fe25c59eb80 1 -- 172.21.15.2:0/3132307345 wait complete. 2024-05-14T09:35:50.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.767+0000 7fe25c59eb80 20 client.15486 trim_cache size 0 max 16384 2024-05-14T09:35:50.771 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-05-14T09:35:50.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.767+0000 7fe24db8a700 1 -- 172.21.15.2:0/2475599170 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe2300b2880 msgr2=0x7fe2300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:50.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.767+0000 7fe24db8a700 1 --2- 172.21.15.2:0/2475599170 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe2300b2880 0x7fe2300b4d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe2481023f0 tx=0x7fe23c00b410 comp rx=0 tx=0).stop 2024-05-14T09:35:50.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.767+0000 7fe24db8a700 1 -- 172.21.15.2:0/2475599170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2481034c0 msgr2=0x7fe2480ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:50.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.767+0000 7fe24db8a700 1 --2- 172.21.15.2:0/2475599170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2481034c0 0x7fe2480ff2a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe238006010 tx=0x7fe2380045e0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.767+0000 7fe24db8a700 1 -- 172.21.15.2:0/2475599170 shutdown_connections 2024-05-14T09:35:50.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.767+0000 7fe24db8a700 1 --2- 172.21.15.2:0/2475599170 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe2300b2880 0x7fe2300b4d90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.767+0000 7fe24db8a700 1 --2- 172.21.15.2:0/2475599170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe248108740 0x7fe2480ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.767+0000 7fe24db8a700 1 --2- 172.21.15.2:0/2475599170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2481034c0 0x7fe2480ff2a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.767+0000 7fe24db8a700 1 --2- 172.21.15.2:0/2475599170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe248107df0 0x7fe2480ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:50.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.767+0000 7fe24db8a700 1 -- 172.21.15.2:0/2475599170 >> 172.21.15.2:0/2475599170 conn(0x7fe2480fedc0 msgr2=0x7fe2481096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:50.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.767+0000 7fe24db8a700 1 -- 172.21.15.2:0/2475599170 shutdown_connections 2024-05-14T09:35:50.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:50.767+0000 7fe24db8a700 1 -- 172.21.15.2:0/2475599170 wait complete. 2024-05-14T09:35:50.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:50 smithi029 ceph-mon[36249]: pgmap v450: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-14T09:35:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:50 smithi026 ceph-mon[36056]: pgmap v450: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-14T09:35:52.571 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T09:35:52.571 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T09:35:52.829 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:35:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:52 smithi002 ceph-mon[31557]: pgmap v451: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-14T09:35:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:52 smithi029 ceph-mon[36249]: pgmap v451: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-14T09:35:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:52 smithi026 ceph-mon[36056]: pgmap v451: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-14T09:35:54.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.599+0000 7f7e56f23700 1 -- 172.21.15.2:0/1393153518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e501034e0 msgr2=0x7f7e501038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.599+0000 7f7e56f23700 1 --2- 172.21.15.2:0/1393153518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e501034e0 0x7f7e501038f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00b0d0 tx=0x7f7e4c00b3e0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.599+0000 7f7e56f23700 1 -- 172.21.15.2:0/1393153518 shutdown_connections 2024-05-14T09:35:54.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.599+0000 7f7e56f23700 1 --2- 172.21.15.2:0/1393153518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e50108760 0x7f7e501100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.599+0000 7f7e56f23700 1 --2- 172.21.15.2:0/1393153518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e501034e0 0x7f7e501038f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.599+0000 7f7e56f23700 1 --2- 172.21.15.2:0/1393153518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e50107e10 0x7f7e50108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.599+0000 7f7e56f23700 1 -- 172.21.15.2:0/1393153518 >> 172.21.15.2:0/1393153518 conn(0x7f7e500fef30 msgr2=0x7f7e50101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:54.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.599+0000 7f7e56f23700 1 -- 172.21.15.2:0/1393153518 shutdown_connections 2024-05-14T09:35:54.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.600+0000 7f7e56f23700 1 -- 172.21.15.2:0/1393153518 wait complete. 2024-05-14T09:35:54.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.600+0000 7f7e56f23700 1 Processor -- start 2024-05-14T09:35:54.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.600+0000 7f7e56f23700 1 -- start start 2024-05-14T09:35:54.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e56f23700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e501034e0 0x7f7e5019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e56f23700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e50107e10 0x7f7e5019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e56f23700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e50108760 0x7f7e5019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e56f23700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50113390 con 0x7f7e501034e0 2024-05-14T09:35:54.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e56f23700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50113210 con 0x7f7e50108760 2024-05-14T09:35:54.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e56f23700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50113510 con 0x7f7e50107e10 2024-05-14T09:35:54.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e54cbf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e501034e0 0x7f7e5019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e54cbf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e501034e0 0x7f7e5019bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53236/0 (socket says 172.21.15.2:53236) 2024-05-14T09:35:54.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e54cbf700 1 -- 172.21.15.2:0/2174681174 learned_addr learned my addr 172.21.15.2:0/2174681174 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:54.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e47fff700 1 --2- 172.21.15.2:0/2174681174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e50107e10 0x7f7e5019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e554c0700 1 --2- 172.21.15.2:0/2174681174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e50108760 0x7f7e5019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e47fff700 1 -- 172.21.15.2:0/2174681174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e50108760 msgr2=0x7f7e5019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e47fff700 1 --2- 172.21.15.2:0/2174681174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e50108760 0x7f7e5019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e47fff700 1 -- 172.21.15.2:0/2174681174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e501034e0 msgr2=0x7f7e5019bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e47fff700 1 --2- 172.21.15.2:0/2174681174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e501034e0 0x7f7e5019bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e47fff700 1 -- 172.21.15.2:0/2174681174 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e501a7000 con 0x7f7e50107e10 2024-05-14T09:35:54.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.601+0000 7f7e54cbf700 1 --2- 172.21.15.2:0/2174681174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e501034e0 0x7f7e5019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:35:54.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.602+0000 7f7e47fff700 1 --2- 172.21.15.2:0/2174681174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e50107e10 0x7f7e5019c1d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000e9c0 tx=0x7f7e4000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:54.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.602+0000 7f7e45ffb700 1 -- 172.21.15.2:0/2174681174 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4000c9c0 con 0x7f7e50107e10 2024-05-14T09:35:54.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.602+0000 7f7e56f23700 1 -- 172.21.15.2:0/2174681174 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e501a72f0 con 0x7f7e50107e10 2024-05-14T09:35:54.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.602+0000 7f7e45ffb700 1 -- 172.21.15.2:0/2174681174 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e4000cb60 con 0x7f7e50107e10 2024-05-14T09:35:54.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.602+0000 7f7e45ffb700 1 -- 172.21.15.2:0/2174681174 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40018680 con 0x7f7e50107e10 2024-05-14T09:35:54.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.602+0000 7f7e56f23700 1 -- 172.21.15.2:0/2174681174 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e501a7880 con 0x7f7e50107e10 2024-05-14T09:35:54.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.606+0000 7f7e65937b80 1 Processor -- start 2024-05-14T09:35:54.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.606+0000 7f7e65937b80 1 -- start start 2024-05-14T09:35:54.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.606+0000 7f7e45ffb700 1 -- 172.21.15.2:0/2174681174 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f7e40018820 con 0x7f7e50107e10 2024-05-14T09:35:54.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.606+0000 7f7e65937b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571561cfec0 0x55715634d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.606+0000 7f7e65937b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55715634dce0 0x557156351fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.606+0000 7f7e65937b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571563524f0 0x557156352900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.606+0000 7f7e65937b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55715626f8e0 con 0x5571563524f0 2024-05-14T09:35:54.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.606+0000 7f7e65937b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55715620e840 con 0x5571561cfec0 2024-05-14T09:35:54.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.606+0000 7f7e65937b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55715626b8a0 con 0x55715634dce0 2024-05-14T09:35:54.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.607+0000 7f7e554c0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571561cfec0 0x55715634d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.607+0000 7f7e554c0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571561cfec0 0x55715634d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:54392/0 (socket says 172.21.15.2:54392) 2024-05-14T09:35:54.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.607+0000 7f7e554c0700 1 -- 172.21.15.2:0/3336093631 learned_addr learned my addr 172.21.15.2:0/3336093631 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:54.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.607+0000 7f7e47fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571563524f0 0x557156352900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.607+0000 7f7e54cbf700 1 --2- 172.21.15.2:0/3336093631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55715634dce0 0x557156351fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.607+0000 7f7e554c0700 1 -- 172.21.15.2:0/3336093631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55715634dce0 msgr2=0x557156351fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.607+0000 7f7e554c0700 1 --2- 172.21.15.2:0/3336093631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55715634dce0 0x557156351fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.607+0000 7f7e554c0700 1 -- 172.21.15.2:0/3336093631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571563524f0 msgr2=0x557156352900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.607+0000 7f7e554c0700 1 --2- 172.21.15.2:0/3336093631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571563524f0 0x557156352900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.607+0000 7f7e554c0700 1 -- 172.21.15.2:0/3336093631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571561bc950 con 0x5571561cfec0 2024-05-14T09:35:54.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.607+0000 7f7e47fff700 1 --2- 172.21.15.2:0/3336093631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571563524f0 0x557156352900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:54.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.607+0000 7f7e554c0700 1 --2- 172.21.15.2:0/3336093631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571561cfec0 0x55715634d7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e48004660 tx=0x7f7e4800b970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22af990ce21449fd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:54.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.608+0000 7f7e45ffb700 1 --2- 172.21.15.2:0/2174681174 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7e380b2880 0x7f7e380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.608+0000 7f7e36ffd700 1 -- 172.21.15.2:0/3336093631 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48004a30 con 0x5571561cfec0 2024-05-14T09:35:54.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.608+0000 7f7e36ffd700 1 -- 172.21.15.2:0/3336093631 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e48015070 con 0x5571561cfec0 2024-05-14T09:35:54.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.608+0000 7f7e36ffd700 1 -- 172.21.15.2:0/3336093631 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e480039b0 con 0x5571561cfec0 2024-05-14T09:35:54.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.608+0000 7f7e54cbf700 1 --2- 172.21.15.2:0/2174681174 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7e380b2880 0x7f7e380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.608+0000 7f7e45ffb700 1 -- 172.21.15.2:0/2174681174 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f7e400e2810 con 0x7f7e50107e10 2024-05-14T09:35:54.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.609+0000 7f7e54cbf700 1 --2- 172.21.15.2:0/2174681174 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7e380b2880 0x7f7e380b4d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00bfd0 tx=0x7f7e4c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:54.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.609+0000 7f7e65937b80 1 -- 172.21.15.2:0/3336093631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571561cfec0 msgr2=0x55715634d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.609+0000 7f7e65937b80 1 --2- 172.21.15.2:0/3336093631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571561cfec0 0x55715634d7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e48004660 tx=0x7f7e4800b970 comp rx=0 tx=0).stop 2024-05-14T09:35:54.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.609+0000 7f7e65937b80 1 -- 172.21.15.2:0/3336093631 shutdown_connections 2024-05-14T09:35:54.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.609+0000 7f7e65937b80 1 --2- 172.21.15.2:0/3336093631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571561cfec0 0x55715634d7a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.609+0000 7f7e65937b80 1 --2- 172.21.15.2:0/3336093631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571563524f0 0x557156352900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.609+0000 7f7e65937b80 1 --2- 172.21.15.2:0/3336093631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55715634dce0 0x557156351fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.609+0000 7f7e65937b80 1 -- 172.21.15.2:0/3336093631 >> 172.21.15.2:0/3336093631 conn(0x557156217680 msgr2=0x557156348530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:54.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.609+0000 7f7e65937b80 1 -- 172.21.15.2:0/3336093631 shutdown_connections 2024-05-14T09:35:54.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.609+0000 7f7e65937b80 1 -- 172.21.15.2:0/3336093631 wait complete. 2024-05-14T09:35:54.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.610+0000 7f7e65937b80 1 -- start start 2024-05-14T09:35:54.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.610+0000 7f7e65937b80 1 Processor -- start 2024-05-14T09:35:54.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.610+0000 7f7e65937b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571561cfec0 0x557156357230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.610+0000 7f7e65937b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55715634dce0 0x557156357770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.610+0000 7f7e65937b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571563524f0 0x557156357cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e65937b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55715635a4a0 con 0x5571561cfec0 2024-05-14T09:35:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e65937b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5571561d5cb0 con 0x5571563524f0 2024-05-14T09:35:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e65937b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55715635a620 con 0x55715634dce0 2024-05-14T09:35:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e47fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55715634dce0 0x557156357770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e47fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55715634dce0 0x557156357770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:56712/0 (socket says 172.21.15.2:56712) 2024-05-14T09:35:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e47fff700 1 -- 172.21.15.2:0/1268032101 learned_addr learned my addr 172.21.15.2:0/1268032101 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e65937b80 10 client.0 _ll_register_callbacks cb 0x5571562079a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e554c0700 1 --2- 172.21.15.2:0/1268032101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571563524f0 0x557156357cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e54cbf700 1 --2- 172.21.15.2:0/1268032101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571561cfec0 0x557156357230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e47fff700 1 -- 172.21.15.2:0/1268032101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571563524f0 msgr2=0x557156357cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e47fff700 1 --2- 172.21.15.2:0/1268032101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571563524f0 0x557156357cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e47fff700 1 -- 172.21.15.2:0/1268032101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571561cfec0 msgr2=0x557156357230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e47fff700 1 --2- 172.21.15.2:0/1268032101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571561cfec0 0x557156357230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e47fff700 1 -- 172.21.15.2:0/1268032101 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55715620b710 con 0x55715634dce0 2024-05-14T09:35:54.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e54cbf700 1 --2- 172.21.15.2:0/1268032101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571561cfec0 0x557156357230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:54.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e47fff700 1 --2- 172.21.15.2:0/1268032101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55715634dce0 0x557156357770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e400a99a0 tx=0x7f7e400a93e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baa491782a0582f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:54.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.611+0000 7f7e2ffff700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:35:54.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.612+0000 7f7e2ffff700 1 -- 172.21.15.2:0/1268032101 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e400b21b0 con 0x55715634dce0 2024-05-14T09:35:54.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.612+0000 7f7e2ffff700 1 -- 172.21.15.2:0/1268032101 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f7e400a6000 con 0x55715634dce0 2024-05-14T09:35:54.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.612+0000 7f7e65937b80 1 -- 172.21.15.2:0/1268032101 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e48015070 con 0x55715634dce0 2024-05-14T09:35:54.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.612+0000 7f7e2ffff700 1 -- 172.21.15.2:0/1268032101 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e400a9520 con 0x55715634dce0 2024-05-14T09:35:54.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.612+0000 7f7e65937b80 10 client.25222 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:54.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.612+0000 7f7e65937b80 1 -- 172.21.15.2:0/1268032101 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5571563650e0 con 0x55715634dce0 2024-05-14T09:35:54.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.612+0000 7f7e2ffff700 1 -- 172.21.15.2:0/1268032101 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e48015070 con 0x55715634dce0 2024-05-14T09:35:54.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.612+0000 7f7e2ffff700 1 -- 172.21.15.2:0/1268032101 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7e4006f2a0 con 0x55715634dce0 2024-05-14T09:35:54.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.612+0000 7f7e65937b80 10 client.25222 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:54.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.612+0000 7f7e65937b80 10 client.25222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:54.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.612+0000 7f7e65937b80 20 client.25222 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:54.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.613+0000 7f7e65937b80 1 --2- 172.21.15.2:0/1268032101 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557156367720 0x557156369b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.613+0000 7f7e65937b80 4 client.25222 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:35:54.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.613+0000 7f7e65937b80 1 -- 172.21.15.2:0/1268032101 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55715627d2c0 con 0x557156367720 2024-05-14T09:35:54.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.613+0000 7f7e54cbf700 1 --2- 172.21.15.2:0/1268032101 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557156367720 0x557156369b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.613+0000 7f7e2ffff700 10 client.25222 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:54.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.613+0000 7f7e54cbf700 1 --2- 172.21.15.2:0/1268032101 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557156367720 0x557156369b40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0e1f67542722118 server_cookie=d4afd4a974a9a026 in_seq=0 out_seq=0 2024-05-14T09:35:54.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.614+0000 7f7e2ffff700 1 -- 172.21.15.2:0/1268032101 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55715627d2c0 con 0x557156367720 2024-05-14T09:35:54.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.614+0000 7f7e2ffff700 10 client.25222 handle_command_reply: tid=0 2024-05-14T09:35:54.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.614+0000 7f7e65937b80 1 client.25222 shutdown 2024-05-14T09:35:54.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.614+0000 7f7e65937b80 1 -- 172.21.15.2:0/1268032101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55715634dce0 msgr2=0x557156357770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.614+0000 7f7e65937b80 1 --2- 172.21.15.2:0/1268032101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55715634dce0 0x557156357770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e400a99a0 tx=0x7f7e400a93e0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.614+0000 7f7e65937b80 1 -- 172.21.15.2:0/1268032101 shutdown_connections 2024-05-14T09:35:54.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.614+0000 7f7e65937b80 1 --2- 172.21.15.2:0/1268032101 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557156367720 0x557156369b40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.614+0000 7f7e2ffff700 0 client.25222 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:54.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.614+0000 7f7e65937b80 1 --2- 172.21.15.2:0/1268032101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571563524f0 0x557156357cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.614+0000 7f7e65937b80 1 --2- 172.21.15.2:0/1268032101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571561cfec0 0x557156357230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.614+0000 7f7e65937b80 1 --2- 172.21.15.2:0/1268032101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55715634dce0 0x557156357770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.615+0000 7f7e65937b80 1 -- 172.21.15.2:0/1268032101 >> 172.21.15.2:0/1268032101 conn(0x557156217680 msgr2=0x55715634c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:54.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.615+0000 7f7e65937b80 1 -- 172.21.15.2:0/1268032101 shutdown_connections 2024-05-14T09:35:54.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.615+0000 7f7e65937b80 1 -- 172.21.15.2:0/1268032101 wait complete. 2024-05-14T09:35:54.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.615+0000 7f7e65937b80 20 client.25222 trim_cache size 0 max 16384 2024-05-14T09:35:54.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.637+0000 7f7e65937b80 1 Processor -- start 2024-05-14T09:35:54.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.637+0000 7f7e65937b80 1 -- start start 2024-05-14T09:35:54.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.637+0000 7f7e65937b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571561cfec0 0x557156348690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.637+0000 7f7e65937b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55715634dce0 0x557156356190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.637+0000 7f7e65937b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571563524f0 0x55715634cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.637+0000 7f7e65937b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501b0c10 con 0x55715634dce0 2024-05-14T09:35:54.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.637+0000 7f7e65937b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501ae4a0 con 0x5571563524f0 2024-05-14T09:35:54.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.637+0000 7f7e65937b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501ac2f0 con 0x5571561cfec0 2024-05-14T09:35:54.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.637+0000 7f7e47fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55715634dce0 0x557156356190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.638+0000 7f7e47fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55715634dce0 0x557156356190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53260/0 (socket says 172.21.15.2:53260) 2024-05-14T09:35:54.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.638+0000 7f7e47fff700 1 -- 172.21.15.2:0/2302202042 learned_addr learned my addr 172.21.15.2:0/2302202042 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:54.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.638+0000 7f7e54cbf700 1 --2- 172.21.15.2:0/2302202042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571561cfec0 0x557156348690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.638+0000 7f7e554c0700 1 --2- 172.21.15.2:0/2302202042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571563524f0 0x55715634cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.638+0000 7f7e47fff700 1 -- 172.21.15.2:0/2302202042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571561cfec0 msgr2=0x557156348690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.638+0000 7f7e47fff700 1 --2- 172.21.15.2:0/2302202042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571561cfec0 0x557156348690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.638+0000 7f7e47fff700 1 -- 172.21.15.2:0/2302202042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571563524f0 msgr2=0x55715634cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.638+0000 7f7e47fff700 1 --2- 172.21.15.2:0/2302202042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571563524f0 0x55715634cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.638+0000 7f7e47fff700 1 -- 172.21.15.2:0/2302202042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571563493a0 con 0x55715634dce0 2024-05-14T09:35:54.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.638+0000 7f7e47fff700 1 --2- 172.21.15.2:0/2302202042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55715634dce0 0x557156356190 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c5b0 tx=0x7f7e4001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfd7ebbd923a564 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:54.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.638+0000 7f7e367fc700 1 -- 172.21.15.2:0/2302202042 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e400aa110 con 0x55715634dce0 2024-05-14T09:35:54.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.639+0000 7f7e367fc700 1 -- 172.21.15.2:0/2302202042 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e400a9a10 con 0x55715634dce0 2024-05-14T09:35:54.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.639+0000 7f7e367fc700 1 -- 172.21.15.2:0/2302202042 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e400b25c0 con 0x55715634dce0 2024-05-14T09:35:54.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.640+0000 7f7e65937b80 1 -- 172.21.15.2:0/2302202042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55715634dce0 msgr2=0x557156356190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.640+0000 7f7e65937b80 1 --2- 172.21.15.2:0/2302202042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55715634dce0 0x557156356190 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c5b0 tx=0x7f7e4001f000 comp rx=0 tx=0).stop 2024-05-14T09:35:54.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.640+0000 7f7e65937b80 1 -- 172.21.15.2:0/2302202042 shutdown_connections 2024-05-14T09:35:54.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.640+0000 7f7e65937b80 1 --2- 172.21.15.2:0/2302202042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571563524f0 0x55715634cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.640+0000 7f7e65937b80 1 --2- 172.21.15.2:0/2302202042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571561cfec0 0x557156348690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.640+0000 7f7e65937b80 1 --2- 172.21.15.2:0/2302202042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55715634dce0 0x557156356190 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.640+0000 7f7e65937b80 1 -- 172.21.15.2:0/2302202042 >> 172.21.15.2:0/2302202042 conn(0x557156217680 msgr2=0x55715634c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:54.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.640+0000 7f7e65937b80 1 -- 172.21.15.2:0/2302202042 shutdown_connections 2024-05-14T09:35:54.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.640+0000 7f7e65937b80 1 -- 172.21.15.2:0/2302202042 wait complete. 2024-05-14T09:35:54.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.640+0000 7f7e65937b80 1 -- start start 2024-05-14T09:35:54.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.641+0000 7f7e65937b80 1 Processor -- start 2024-05-14T09:35:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.641+0000 7f7e65937b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571561cfec0 0x55715635fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.641+0000 7f7e65937b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55715634dce0 0x55715635c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.641+0000 7f7e65937b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571563524f0 0x55715635cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.641+0000 7f7e65937b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501ac960 con 0x5571561cfec0 2024-05-14T09:35:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.641+0000 7f7e65937b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501addd0 con 0x55715634dce0 2024-05-14T09:35:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.641+0000 7f7e65937b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501ad030 con 0x5571563524f0 2024-05-14T09:35:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.641+0000 7f7e54cbf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571561cfec0 0x55715635fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.641+0000 7f7e54cbf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571561cfec0 0x55715635fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53264/0 (socket says 172.21.15.2:53264) 2024-05-14T09:35:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.641+0000 7f7e54cbf700 1 -- 172.21.15.2:0/3085198393 learned_addr learned my addr 172.21.15.2:0/3085198393 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.641+0000 7f7e65937b80 10 client.0 _ll_register_callbacks cb 0x5571562079a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.641+0000 7f7e554c0700 1 --2- 172.21.15.2:0/3085198393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571563524f0 0x55715635cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.642+0000 7f7e47fff700 1 --2- 172.21.15.2:0/3085198393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55715634dce0 0x55715635c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.642+0000 7f7e54cbf700 1 -- 172.21.15.2:0/3085198393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571563524f0 msgr2=0x55715635cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.642+0000 7f7e54cbf700 1 --2- 172.21.15.2:0/3085198393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571563524f0 0x55715635cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.642+0000 7f7e54cbf700 1 -- 172.21.15.2:0/3085198393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55715634dce0 msgr2=0x55715635c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.642+0000 7f7e54cbf700 1 --2- 172.21.15.2:0/3085198393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55715634dce0 0x55715635c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.642+0000 7f7e54cbf700 1 -- 172.21.15.2:0/3085198393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55715635d400 con 0x5571561cfec0 2024-05-14T09:35:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.642+0000 7f7e54cbf700 1 --2- 172.21.15.2:0/3085198393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571561cfec0 0x55715635fda0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c003b30 tx=0x7f7e4c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2449363273e62a9e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.642+0000 7f7e2ffff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.642+0000 7f7e2ffff700 1 -- 172.21.15.2:0/3085198393 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c037060 con 0x5571561cfec0 2024-05-14T09:35:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.642+0000 7f7e2ffff700 1 -- 172.21.15.2:0/3085198393 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f7e4c022030 con 0x5571561cfec0 2024-05-14T09:35:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.642+0000 7f7e65937b80 1 -- 172.21.15.2:0/3085198393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e400a9a10 con 0x5571561cfec0 2024-05-14T09:35:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.643+0000 7f7e2ffff700 1 -- 172.21.15.2:0/3085198393 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e4c03e050 con 0x5571561cfec0 2024-05-14T09:35:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.643+0000 7f7e65937b80 10 client.15504 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.643+0000 7f7e65937b80 1 -- 172.21.15.2:0/3085198393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5571563d2d00 con 0x5571561cfec0 2024-05-14T09:35:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.643+0000 7f7e2ffff700 1 -- 172.21.15.2:0/3085198393 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e400a9a10 con 0x5571561cfec0 2024-05-14T09:35:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.643+0000 7f7e2ffff700 1 -- 172.21.15.2:0/3085198393 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7e4c032ce0 con 0x5571561cfec0 2024-05-14T09:35:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.643+0000 7f7e65937b80 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.643+0000 7f7e65937b80 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.643+0000 7f7e65937b80 20 client.15504 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.643+0000 7f7e65937b80 1 --2- 172.21.15.2:0/3085198393 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5571563d33f0 0x5571563d5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.643+0000 7f7e65937b80 4 client.15504 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T09:35:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.643+0000 7f7e65937b80 1 -- 172.21.15.2:0/3085198393 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557156277360 con 0x5571563d33f0 2024-05-14T09:35:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.644+0000 7f7e47fff700 1 --2- 172.21.15.2:0/3085198393 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5571563d33f0 0x5571563d5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.644+0000 7f7e2ffff700 10 client.15504 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.644+0000 7f7e47fff700 1 --2- 172.21.15.2:0/3085198393 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5571563d33f0 0x5571563d5810 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65dc18a0123aaefa server_cookie=8563180143efc7a7 in_seq=0 out_seq=0 2024-05-14T09:35:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.645+0000 7f7e2ffff700 1 -- 172.21.15.2:0/3085198393 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557156277360 con 0x5571563d33f0 2024-05-14T09:35:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.645+0000 7f7e2ffff700 10 client.15504 handle_command_reply: tid=0 2024-05-14T09:35:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.645+0000 7f7e65937b80 1 client.15504 shutdown 2024-05-14T09:35:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.645+0000 7f7e65937b80 1 -- 172.21.15.2:0/3085198393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571561cfec0 msgr2=0x55715635fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.645+0000 7f7e65937b80 1 --2- 172.21.15.2:0/3085198393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571561cfec0 0x55715635fda0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c003b30 tx=0x7f7e4c021000 comp rx=0 tx=0).stop 2024-05-14T09:35:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.645+0000 7f7e65937b80 1 -- 172.21.15.2:0/3085198393 shutdown_connections 2024-05-14T09:35:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.645+0000 7f7e65937b80 1 --2- 172.21.15.2:0/3085198393 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5571563d33f0 0x5571563d5810 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.646+0000 7f7e2ffff700 0 client.15504 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.646+0000 7f7e65937b80 1 --2- 172.21.15.2:0/3085198393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5571563524f0 0x55715635cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.646+0000 7f7e65937b80 1 --2- 172.21.15.2:0/3085198393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571561cfec0 0x55715635fda0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.646+0000 7f7e65937b80 1 --2- 172.21.15.2:0/3085198393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55715634dce0 0x55715635c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.646+0000 7f7e65937b80 1 -- 172.21.15.2:0/3085198393 >> 172.21.15.2:0/3085198393 conn(0x557156217680 msgr2=0x5571563483f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.646+0000 7f7e65937b80 1 -- 172.21.15.2:0/3085198393 shutdown_connections 2024-05-14T09:35:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.646+0000 7f7e65937b80 1 -- 172.21.15.2:0/3085198393 wait complete. 2024-05-14T09:35:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.646+0000 7f7e65937b80 20 client.15504 trim_cache size 0 max 16384 2024-05-14T09:35:54.648 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:35:54.648 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-05-14T09:35:54.648 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "739ce1c0-71d3-4aa6-a69b-ff7ca400e738", 2024-05-14T09:35:54.648 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-05-14T09:35:54.648 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:35:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.647+0000 7f7e56f23700 1 -- 172.21.15.2:0/2174681174 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7e380b2880 msgr2=0x7f7e380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.647+0000 7f7e56f23700 1 --2- 172.21.15.2:0/2174681174 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7e380b2880 0x7f7e380b4d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00bfd0 tx=0x7f7e4c00b040 comp rx=0 tx=0).stop 2024-05-14T09:35:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.647+0000 7f7e56f23700 1 -- 172.21.15.2:0/2174681174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e50107e10 msgr2=0x7f7e5019c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.647+0000 7f7e56f23700 1 --2- 172.21.15.2:0/2174681174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e50107e10 0x7f7e5019c1d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000e9c0 tx=0x7f7e4000ecd0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.647+0000 7f7e56f23700 1 -- 172.21.15.2:0/2174681174 shutdown_connections 2024-05-14T09:35:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.647+0000 7f7e56f23700 1 --2- 172.21.15.2:0/2174681174 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7e380b2880 0x7f7e380b4d90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.647+0000 7f7e56f23700 1 --2- 172.21.15.2:0/2174681174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e50108760 0x7f7e5019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.647+0000 7f7e56f23700 1 --2- 172.21.15.2:0/2174681174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e501034e0 0x7f7e5019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.647+0000 7f7e56f23700 1 --2- 172.21.15.2:0/2174681174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e50107e10 0x7f7e5019c1d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.647+0000 7f7e56f23700 1 -- 172.21.15.2:0/2174681174 >> 172.21.15.2:0/2174681174 conn(0x7f7e500fef30 msgr2=0x7f7e501063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.647+0000 7f7e56f23700 1 -- 172.21.15.2:0/2174681174 shutdown_connections 2024-05-14T09:35:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:54.647+0000 7f7e56f23700 1 -- 172.21.15.2:0/2174681174 wait complete. 2024-05-14T09:35:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.814+0000 7f12bdc84640 1 -- 172.21.15.2:0/1966680080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12b8104eb0 msgr2=0x7f12b81052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.814+0000 7f12bdc84640 1 --2- 172.21.15.2:0/1966680080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12b8104eb0 0x7f12b81052c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009900 tx=0x7f12a002a3f0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.814+0000 7f12bdc84640 1 -- 172.21.15.2:0/1966680080 shutdown_connections 2024-05-14T09:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.814+0000 7f12bdc84640 1 --2- 172.21.15.2:0/1966680080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b8109bb0 0x7f12b810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.814+0000 7f12bdc84640 1 --2- 172.21.15.2:0/1966680080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12b8104eb0 0x7f12b81052c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.814+0000 7f12bdc84640 1 --2- 172.21.15.2:0/1966680080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b8103c70 0x7f12b8104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.814+0000 7f12bdc84640 1 -- 172.21.15.2:0/1966680080 >> 172.21.15.2:0/1966680080 conn(0x7f12b80fee60 msgr2=0x7f12b8101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.814+0000 7f12bdc84640 1 -- 172.21.15.2:0/1966680080 shutdown_connections 2024-05-14T09:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.814+0000 7f12bdc84640 1 -- 172.21.15.2:0/1966680080 wait complete. 2024-05-14T09:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.815+0000 7f12bdc84640 1 Processor -- start 2024-05-14T09:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.815+0000 7f12bdc84640 1 -- start start 2024-05-14T09:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.815+0000 7f12bdc84640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12b8103c70 0x7f12b819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.815+0000 7f12bdc84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b8104eb0 0x7f12b819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.815+0000 7f12bdc84640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b8109bb0 0x7f12b81a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.815+0000 7f12bdc84640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f12b810f770 con 0x7f12b8103c70 2024-05-14T09:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.815+0000 7f12bdc84640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f12b810f5f0 con 0x7f12b8104eb0 2024-05-14T09:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.815+0000 7f12bdc84640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f12b810f8f0 con 0x7f12b8109bb0 2024-05-14T09:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.815+0000 7f12b77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12b8103c70 0x7f12b819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.816+0000 7f12b77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12b8103c70 0x7f12b819a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53274/0 (socket says 172.21.15.2:53274) 2024-05-14T09:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.816+0000 7f12b77fe640 1 -- 172.21.15.2:0/2033453352 learned_addr learned my addr 172.21.15.2:0/2033453352 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.816+0000 7f12affff640 1 --2- 172.21.15.2:0/2033453352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b8104eb0 0x7f12b819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.816+0000 7f12b7fff640 1 --2- 172.21.15.2:0/2033453352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b8109bb0 0x7f12b81a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.816+0000 7f12b77fe640 1 -- 172.21.15.2:0/2033453352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b8109bb0 msgr2=0x7f12b81a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.816+0000 7f12b77fe640 1 --2- 172.21.15.2:0/2033453352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b8109bb0 0x7f12b81a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.816+0000 7f12b77fe640 1 -- 172.21.15.2:0/2033453352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b8104eb0 msgr2=0x7f12b819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.816+0000 7f12b77fe640 1 --2- 172.21.15.2:0/2033453352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b8104eb0 0x7f12b819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.816+0000 7f12b77fe640 1 -- 172.21.15.2:0/2033453352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b81a1c80 con 0x7f12b8103c70 2024-05-14T09:35:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.816+0000 7f12b77fe640 1 --2- 172.21.15.2:0/2033453352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12b8103c70 0x7f12b819a1f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f12a800c970 tx=0x7f12a800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.816+0000 7f12b57fa640 1 -- 172.21.15.2:0/2033453352 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8007c70 con 0x7f12b8103c70 2024-05-14T09:35:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.816+0000 7f12b57fa640 1 -- 172.21.15.2:0/2033453352 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a8007e10 con 0x7f12b8103c70 2024-05-14T09:35:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.816+0000 7f12bdc84640 1 -- 172.21.15.2:0/2033453352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b81a1f70 con 0x7f12b8103c70 2024-05-14T09:35:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.816+0000 7f12b57fa640 1 -- 172.21.15.2:0/2033453352 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a80187b0 con 0x7f12b8103c70 2024-05-14T09:35:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.816+0000 7f12bdc84640 1 -- 172.21.15.2:0/2033453352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b81a2430 con 0x7f12b8103c70 2024-05-14T09:35:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.819+0000 7f12b57fa640 1 -- 172.21.15.2:0/2033453352 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f12a800fab0 con 0x7f12b8103c70 2024-05-14T09:35:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.820+0000 7f12c4c06740 1 Processor -- start 2024-05-14T09:35:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.820+0000 7f12b57fa640 1 --2- 172.21.15.2:0/2033453352 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f12a40b2920 0x7f12a40b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.820+0000 7f12c4c06740 1 -- start start 2024-05-14T09:35:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.820+0000 7f12b7fff640 1 --2- 172.21.15.2:0/2033453352 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f12a40b2920 0x7f12a40b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.820+0000 7f12b57fa640 1 -- 172.21.15.2:0/2033453352 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f12a80e2710 con 0x7f12b8103c70 2024-05-14T09:35:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.820+0000 7f12c4c06740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af7c91b0 0x5590afb6aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.820+0000 7f12c4c06740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a02f0 0x5590afb6afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.820+0000 7f12c4c06740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a84d0 0x5590afb71ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.820+0000 7f12c4c06740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5590afacf5d0 con 0x5590af7a02f0 2024-05-14T09:35:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.820+0000 7f12c4c06740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5590afa88890 con 0x5590af7a84d0 2024-05-14T09:35:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.820+0000 7f12c4c06740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5590afacb590 con 0x5590af7c91b0 2024-05-14T09:35:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.821+0000 7f12affff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a02f0 0x5590afb6afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.821+0000 7f12affff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a02f0 0x5590afb6afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53280/0 (socket says 172.21.15.2:53280) 2024-05-14T09:35:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.821+0000 7f12affff640 1 -- 172.21.15.2:0/1429626638 learned_addr learned my addr 172.21.15.2:0/1429626638 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.821+0000 7f12b77fe640 1 --2- 172.21.15.2:0/1429626638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af7c91b0 0x5590afb6aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.821+0000 7f12b7fff640 1 --2- 172.21.15.2:0/2033453352 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f12a40b2920 0x7f12a40b4e30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f12b0008ca0 tx=0x7f12b0008230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.821+0000 7f12b7fff640 1 --2- 172.21.15.2:0/1429626638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a84d0 0x5590afb71ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.821+0000 7f12affff640 1 -- 172.21.15.2:0/1429626638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af7c91b0 msgr2=0x5590afb6aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.821+0000 7f12affff640 1 --2- 172.21.15.2:0/1429626638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af7c91b0 0x5590afb6aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.821+0000 7f12affff640 1 -- 172.21.15.2:0/1429626638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a84d0 msgr2=0x5590afb71ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.821+0000 7f12affff640 1 --2- 172.21.15.2:0/1429626638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a84d0 0x5590afb71ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.821+0000 7f12affff640 1 -- 172.21.15.2:0/1429626638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590af7e1210 con 0x5590af7a02f0 2024-05-14T09:35:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.821+0000 7f12affff640 1 --2- 172.21.15.2:0/1429626638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a02f0 0x5590afb6afe0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f12a00096c0 tx=0x7f12a0031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7c5802fcfdf9b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.821+0000 7f12adffb640 1 -- 172.21.15.2:0/1429626638 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0035030 con 0x5590af7a02f0 2024-05-14T09:35:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.821+0000 7f12adffb640 1 -- 172.21.15.2:0/1429626638 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a00317a0 con 0x5590af7a02f0 2024-05-14T09:35:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.821+0000 7f12adffb640 1 -- 172.21.15.2:0/1429626638 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0003d00 con 0x5590af7a02f0 2024-05-14T09:35:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.822+0000 7f12c4c06740 1 -- 172.21.15.2:0/1429626638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a02f0 msgr2=0x5590afb6afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.822+0000 7f12c4c06740 1 --2- 172.21.15.2:0/1429626638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a02f0 0x5590afb6afe0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f12a00096c0 tx=0x7f12a0031cd0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.822+0000 7f12c4c06740 1 -- 172.21.15.2:0/1429626638 shutdown_connections 2024-05-14T09:35:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.822+0000 7f12c4c06740 1 --2- 172.21.15.2:0/1429626638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a84d0 0x5590afb71ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.822+0000 7f12c4c06740 1 --2- 172.21.15.2:0/1429626638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a02f0 0x5590afb6afe0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.822+0000 7f12c4c06740 1 --2- 172.21.15.2:0/1429626638 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af7c91b0 0x5590afb6aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.822+0000 7f12c4c06740 1 -- 172.21.15.2:0/1429626638 >> 172.21.15.2:0/1429626638 conn(0x5590af7bd300 msgr2=0x5590afb663a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.823+0000 7f12c4c06740 1 -- 172.21.15.2:0/1429626638 shutdown_connections 2024-05-14T09:35:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.823+0000 7f12c4c06740 1 -- 172.21.15.2:0/1429626638 wait complete. 2024-05-14T09:35:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.823+0000 7f12c4c06740 1 -- start start 2024-05-14T09:35:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.823+0000 7f12c4c06740 1 Processor -- start 2024-05-14T09:35:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12c4c06740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a02f0 0x5590afb78a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12c4c06740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a84d0 0x5590afb78f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12c4c06740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af7c91b0 0x5590afb7fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12c4c06740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5590afb728d0 con 0x5590af7a84d0 2024-05-14T09:35:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12c4c06740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5590afa8c370 con 0x5590af7a02f0 2024-05-14T09:35:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12c4c06740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5590afb72a50 con 0x5590af7c91b0 2024-05-14T09:35:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12c4c06740 10 client.0 _ll_register_callbacks cb 0x5590af7e0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12affff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a84d0 0x5590afb78f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12affff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a84d0 0x5590afb78f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53296/0 (socket says 172.21.15.2:53296) 2024-05-14T09:35:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12affff640 1 -- 172.21.15.2:0/1436638247 learned_addr learned my addr 172.21.15.2:0/1436638247 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12b77fe640 1 --2- 172.21.15.2:0/1436638247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a02f0 0x5590afb78a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12b7fff640 1 --2- 172.21.15.2:0/1436638247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af7c91b0 0x5590afb7fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12affff640 1 -- 172.21.15.2:0/1436638247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af7c91b0 msgr2=0x5590afb7fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12affff640 1 --2- 172.21.15.2:0/1436638247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af7c91b0 0x5590afb7fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12affff640 1 -- 172.21.15.2:0/1436638247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a02f0 msgr2=0x5590afb78a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12affff640 1 --2- 172.21.15.2:0/1436638247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a02f0 0x5590afb78a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12affff640 1 -- 172.21.15.2:0/1436638247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590af8ad620 con 0x5590af7a84d0 2024-05-14T09:35:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f12affff640 1 --2- 172.21.15.2:0/1436638247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a84d0 0x5590afb78f60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f12a00092a0 tx=0x7f12a00316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac0c98562dd01f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.824+0000 7f1296ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.825+0000 7f1296ffd640 1 -- 172.21.15.2:0/1436638247 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0035070 con 0x5590af7a84d0 2024-05-14T09:35:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.825+0000 7f12c4c06740 1 -- 172.21.15.2:0/1436638247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12a00317a0 con 0x5590af7a84d0 2024-05-14T09:35:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.825+0000 7f1296ffd640 1 -- 172.21.15.2:0/1436638247 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f12a0003c90 con 0x5590af7a84d0 2024-05-14T09:35:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.825+0000 7f1296ffd640 1 -- 172.21.15.2:0/1436638247 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f12a0057050 con 0x5590af7a84d0 2024-05-14T09:35:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.825+0000 7f12c4c06740 10 client.15516 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.825+0000 7f12c4c06740 1 -- 172.21.15.2:0/1436638247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5590afb82df0 con 0x5590af7a84d0 2024-05-14T09:35:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.825+0000 7f1296ffd640 1 -- 172.21.15.2:0/1436638247 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a00317a0 con 0x5590af7a84d0 2024-05-14T09:35:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.825+0000 7f1296ffd640 1 -- 172.21.15.2:0/1436638247 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f12a003ca10 con 0x5590af7a84d0 2024-05-14T09:35:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.825+0000 7f12c4c06740 10 client.15516 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.825+0000 7f12c4c06740 10 client.15516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.825+0000 7f12c4c06740 20 client.15516 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.825+0000 7f12c4c06740 1 --2- 172.21.15.2:0/1436638247 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590af787d80 0x5590afb86150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.825+0000 7f12c4c06740 4 client.15516 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:35:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.825+0000 7f12c4c06740 1 -- 172.21.15.2:0/1436638247 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590afadd1e0 con 0x5590af787d80 2024-05-14T09:35:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.826+0000 7f12b77fe640 1 --2- 172.21.15.2:0/1436638247 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590af787d80 0x5590afb86150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.826+0000 7f1296ffd640 10 client.15516 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.826+0000 7f12b77fe640 1 --2- 172.21.15.2:0/1436638247 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590af787d80 0x5590afb86150 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30774e4c0f8e3afb server_cookie=e5239b98f4ee0267 in_seq=0 out_seq=0 2024-05-14T09:35:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.827+0000 7f1296ffd640 1 -- 172.21.15.2:0/1436638247 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5590afadd1e0 con 0x5590af787d80 2024-05-14T09:35:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.827+0000 7f1296ffd640 10 client.15516 handle_command_reply: tid=0 2024-05-14T09:35:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.827+0000 7f12c4c06740 1 client.15516 shutdown 2024-05-14T09:35:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.827+0000 7f12c4c06740 1 -- 172.21.15.2:0/1436638247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a84d0 msgr2=0x5590afb78f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.827+0000 7f12c4c06740 1 --2- 172.21.15.2:0/1436638247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a84d0 0x5590afb78f60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f12a00092a0 tx=0x7f12a00316e0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.827+0000 7f12c4c06740 1 -- 172.21.15.2:0/1436638247 shutdown_connections 2024-05-14T09:35:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.827+0000 7f12c4c06740 1 --2- 172.21.15.2:0/1436638247 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590af787d80 0x5590afb86150 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.827+0000 7f12c4c06740 1 --2- 172.21.15.2:0/1436638247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af7c91b0 0x5590afb7fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.827+0000 7f1296ffd640 0 client.15516 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.827+0000 7f12c4c06740 1 --2- 172.21.15.2:0/1436638247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a84d0 0x5590afb78f60 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.827+0000 7f12c4c06740 1 --2- 172.21.15.2:0/1436638247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a02f0 0x5590afb78a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.827+0000 7f12c4c06740 1 -- 172.21.15.2:0/1436638247 >> 172.21.15.2:0/1436638247 conn(0x5590af7bd300 msgr2=0x5590afb69300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.827+0000 7f12c4c06740 1 -- 172.21.15.2:0/1436638247 shutdown_connections 2024-05-14T09:35:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.827+0000 7f12c4c06740 1 -- 172.21.15.2:0/1436638247 wait complete. 2024-05-14T09:35:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.828+0000 7f12c4c06740 20 client.15516 trim_cache size 0 max 16384 2024-05-14T09:35:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.844+0000 7f12c4c06740 1 Processor -- start 2024-05-14T09:35:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.844+0000 7f12c4c06740 1 -- start start 2024-05-14T09:35:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12c4c06740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af787d80 0x5590afb65d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12c4c06740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a02f0 0x5590afb662a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12c4c06740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a84d0 0x5590afb72e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12c4c06740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81ad0a0 con 0x5590af7a84d0 2024-05-14T09:35:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12c4c06740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81aa970 con 0x5590af7a02f0 2024-05-14T09:35:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12c4c06740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81a87c0 con 0x5590af787d80 2024-05-14T09:35:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12b7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a84d0 0x5590afb72e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12b7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a84d0 0x5590afb72e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53302/0 (socket says 172.21.15.2:53302) 2024-05-14T09:35:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12b7fff640 1 -- 172.21.15.2:0/331597237 learned_addr learned my addr 172.21.15.2:0/331597237 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12b77fe640 1 --2- 172.21.15.2:0/331597237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af787d80 0x5590afb65d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12affff640 1 --2- 172.21.15.2:0/331597237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a02f0 0x5590afb662a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12b7fff640 1 -- 172.21.15.2:0/331597237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af787d80 msgr2=0x5590afb65d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12b7fff640 1 --2- 172.21.15.2:0/331597237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af787d80 0x5590afb65d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12b7fff640 1 -- 172.21.15.2:0/331597237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a02f0 msgr2=0x5590afb662a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12b7fff640 1 --2- 172.21.15.2:0/331597237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a02f0 0x5590afb662a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12b7fff640 1 -- 172.21.15.2:0/331597237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590afb65b60 con 0x5590af7a84d0 2024-05-14T09:35:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.845+0000 7f12b7fff640 1 --2- 172.21.15.2:0/331597237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a84d0 0x5590afb72e20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f12b000f500 tx=0x7f12b0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36c793abf66af3b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.846+0000 7f12ad7fa640 1 -- 172.21.15.2:0/331597237 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b001e850 con 0x5590af7a84d0 2024-05-14T09:35:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.846+0000 7f12ad7fa640 1 -- 172.21.15.2:0/331597237 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12b0022070 con 0x5590af7a84d0 2024-05-14T09:35:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.846+0000 7f12ad7fa640 1 -- 172.21.15.2:0/331597237 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b001d440 con 0x5590af7a84d0 2024-05-14T09:35:54.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.847+0000 7f12c4c06740 1 -- 172.21.15.2:0/331597237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a84d0 msgr2=0x5590afb72e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.847+0000 7f12c4c06740 1 --2- 172.21.15.2:0/331597237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a84d0 0x5590afb72e20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f12b000f500 tx=0x7f12b0010000 comp rx=0 tx=0).stop 2024-05-14T09:35:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.847+0000 7f12c4c06740 1 -- 172.21.15.2:0/331597237 shutdown_connections 2024-05-14T09:35:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.847+0000 7f12c4c06740 1 --2- 172.21.15.2:0/331597237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a84d0 0x5590afb72e20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.847+0000 7f12c4c06740 1 --2- 172.21.15.2:0/331597237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a02f0 0x5590afb662a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.847+0000 7f12c4c06740 1 --2- 172.21.15.2:0/331597237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af787d80 0x5590afb65d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.847+0000 7f12c4c06740 1 -- 172.21.15.2:0/331597237 >> 172.21.15.2:0/331597237 conn(0x5590af7bd300 msgr2=0x5590afb6a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.847+0000 7f12c4c06740 1 -- 172.21.15.2:0/331597237 shutdown_connections 2024-05-14T09:35:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.847+0000 7f12c4c06740 1 -- 172.21.15.2:0/331597237 wait complete. 2024-05-14T09:35:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.847+0000 7f12c4c06740 1 -- start start 2024-05-14T09:35:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.847+0000 7f12c4c06740 1 Processor -- start 2024-05-14T09:35:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12c4c06740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af787d80 0x5590afb7cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12c4c06740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a02f0 0x5590afb7d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12c4c06740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a84d0 0x5590afb7e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12c4c06740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81a8e30 con 0x5590af7a02f0 2024-05-14T09:35:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12c4c06740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81aa2a0 con 0x5590af7a84d0 2024-05-14T09:35:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12c4c06740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81a9500 con 0x5590af787d80 2024-05-14T09:35:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12c4c06740 10 client.0 _ll_register_callbacks cb 0x7f12a0031ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12affff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a02f0 0x5590afb7d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12affff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a02f0 0x5590afb7d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53310/0 (socket says 172.21.15.2:53310) 2024-05-14T09:35:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12affff640 1 -- 172.21.15.2:0/2844001043 learned_addr learned my addr 172.21.15.2:0/2844001043 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12b7fff640 1 --2- 172.21.15.2:0/2844001043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a84d0 0x5590afb7e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12b77fe640 1 --2- 172.21.15.2:0/2844001043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af787d80 0x5590afb7cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12affff640 1 -- 172.21.15.2:0/2844001043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af787d80 msgr2=0x5590afb7cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12affff640 1 --2- 172.21.15.2:0/2844001043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af787d80 0x5590afb7cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12affff640 1 -- 172.21.15.2:0/2844001043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a84d0 msgr2=0x5590afb7e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12affff640 1 --2- 172.21.15.2:0/2844001043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a84d0 0x5590afb7e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12affff640 1 -- 172.21.15.2:0/2844001043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590afb7e6c0 con 0x5590af7a02f0 2024-05-14T09:35:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12b77fe640 1 --2- 172.21.15.2:0/2844001043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af787d80 0x5590afb7cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f12affff640 1 --2- 172.21.15.2:0/2844001043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a02f0 0x5590afb7d1d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f12a003cf50 tx=0x7f12a002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b12b672e6e4363 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f1296ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.848+0000 7f1296ffd640 1 -- 172.21.15.2:0/2844001043 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a005f060 con 0x5590af7a02f0 2024-05-14T09:35:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.849+0000 7f1296ffd640 1 -- 172.21.15.2:0/2844001043 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f12a004a040 con 0x5590af7a02f0 2024-05-14T09:35:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.849+0000 7f12c4c06740 1 -- 172.21.15.2:0/2844001043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12b0022070 con 0x5590af7a02f0 2024-05-14T09:35:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.849+0000 7f1296ffd640 1 -- 172.21.15.2:0/2844001043 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f12a0066050 con 0x5590af7a02f0 2024-05-14T09:35:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.849+0000 7f12c4c06740 10 client.15522 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.849+0000 7f12c4c06740 1 -- 172.21.15.2:0/2844001043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5590afbee420 con 0x5590af7a02f0 2024-05-14T09:35:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.849+0000 7f1296ffd640 1 -- 172.21.15.2:0/2844001043 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12b0022070 con 0x5590af7a02f0 2024-05-14T09:35:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.849+0000 7f1296ffd640 1 -- 172.21.15.2:0/2844001043 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f12a0003f60 con 0x5590af7a02f0 2024-05-14T09:35:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.849+0000 7f12c4c06740 10 client.15522 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.849+0000 7f12c4c06740 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.849+0000 7f12c4c06740 20 client.15522 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.849+0000 7f12c4c06740 1 --2- 172.21.15.2:0/2844001043 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590af7c91b0 0x5590afbf0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.849+0000 7f12c4c06740 4 client.15522 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:35:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.849+0000 7f12c4c06740 1 -- 172.21.15.2:0/2844001043 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590afad7050 con 0x5590af7c91b0 2024-05-14T09:35:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.849+0000 7f12b77fe640 1 --2- 172.21.15.2:0/2844001043 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590af7c91b0 0x5590afbf0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.850+0000 7f1296ffd640 10 client.15522 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.850+0000 7f12b77fe640 1 --2- 172.21.15.2:0/2844001043 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590af7c91b0 0x5590afbf0cc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c24d6fb14a8d33f5 server_cookie=832e6a632bddbbb8 in_seq=0 out_seq=0 2024-05-14T09:35:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.854+0000 7f1296ffd640 1 -- 172.21.15.2:0/2844001043 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590afad7050 con 0x5590af7c91b0 2024-05-14T09:35:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.854+0000 7f1296ffd640 10 client.15522 handle_command_reply: tid=0 2024-05-14T09:35:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.854+0000 7f12c4c06740 1 client.15522 shutdown 2024-05-14T09:35:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.854+0000 7f12c4c06740 1 -- 172.21.15.2:0/2844001043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a02f0 msgr2=0x5590afb7d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.854+0000 7f12c4c06740 1 --2- 172.21.15.2:0/2844001043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a02f0 0x5590afb7d1d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f12a003cf50 tx=0x7f12a002a3f0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.854+0000 7f12c4c06740 1 -- 172.21.15.2:0/2844001043 shutdown_connections 2024-05-14T09:35:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.854+0000 7f12c4c06740 1 --2- 172.21.15.2:0/2844001043 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590af7c91b0 0x5590afbf0cc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.854+0000 7f12c4c06740 1 --2- 172.21.15.2:0/2844001043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590af7a84d0 0x5590afb7e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.854+0000 7f12c4c06740 1 --2- 172.21.15.2:0/2844001043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590af7a02f0 0x5590afb7d1d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.854+0000 7f1296ffd640 0 client.15522 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.854+0000 7f12c4c06740 1 --2- 172.21.15.2:0/2844001043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590af787d80 0x5590afb7cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.854+0000 7f12c4c06740 1 -- 172.21.15.2:0/2844001043 >> 172.21.15.2:0/2844001043 conn(0x5590af7bd300 msgr2=0x5590afb65d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.854+0000 7f12c4c06740 1 -- 172.21.15.2:0/2844001043 shutdown_connections 2024-05-14T09:35:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.854+0000 7f12c4c06740 1 -- 172.21.15.2:0/2844001043 wait complete. 2024-05-14T09:35:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.854+0000 7f12c4c06740 20 client.15522 trim_cache size 0 max 16384 2024-05-14T09:35:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:35:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:35:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:35:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:35:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.855+0000 7f12bdc84640 1 -- 172.21.15.2:0/2033453352 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f12a40b2920 msgr2=0x7f12a40b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.855+0000 7f12bdc84640 1 --2- 172.21.15.2:0/2033453352 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f12a40b2920 0x7f12a40b4e30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f12b0008ca0 tx=0x7f12b0008230 comp rx=0 tx=0).stop 2024-05-14T09:35:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.855+0000 7f12bdc84640 1 -- 172.21.15.2:0/2033453352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12b8103c70 msgr2=0x7f12b819a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.855+0000 7f12bdc84640 1 --2- 172.21.15.2:0/2033453352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12b8103c70 0x7f12b819a1f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f12a800c970 tx=0x7f12a800ce30 comp rx=0 tx=0).stop 2024-05-14T09:35:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.855+0000 7f12bdc84640 1 -- 172.21.15.2:0/2033453352 shutdown_connections 2024-05-14T09:35:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.855+0000 7f12bdc84640 1 --2- 172.21.15.2:0/2033453352 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f12a40b2920 0x7f12a40b4e30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.855+0000 7f12bdc84640 1 --2- 172.21.15.2:0/2033453352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f12b8109bb0 0x7f12b81a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.855+0000 7f12bdc84640 1 --2- 172.21.15.2:0/2033453352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b8104eb0 0x7f12b819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.855+0000 7f12bdc84640 1 --2- 172.21.15.2:0/2033453352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12b8103c70 0x7f12b819a1f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.855+0000 7f12bdc84640 1 -- 172.21.15.2:0/2033453352 >> 172.21.15.2:0/2033453352 conn(0x7f12b80fee60 msgr2=0x7f12b81012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.855+0000 7f12bdc84640 1 -- 172.21.15.2:0/2033453352 shutdown_connections 2024-05-14T09:35:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:35:54.855+0000 7f12bdc84640 1 -- 172.21.15.2:0/2033453352 wait complete. 2024-05-14T09:35:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:54 smithi002 ceph-mon[31557]: pgmap v452: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-14T09:35:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:54 smithi029 ceph-mon[36249]: pgmap v452: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-14T09:35:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:54 smithi026 ceph-mon[36056]: pgmap v452: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-14T09:35:55.478 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T09:35:55.743 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:35:55.869 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:55 smithi002 ceph-mon[31557]: scrub queued for path: / 2024-05-14T09:35:55.869 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:55 smithi002 ceph-mon[31557]: scrub summary: idle+waiting paths [/] 2024-05-14T09:35:55.869 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:55 smithi002 ceph-mon[31557]: scrub summary: active paths [/] 2024-05-14T09:35:55.869 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:55 smithi002 ceph-mon[31557]: scrub summary: idle+waiting paths [/] 2024-05-14T09:35:55.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:55 smithi029 ceph-mon[36249]: scrub queued for path: / 2024-05-14T09:35:55.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:55 smithi029 ceph-mon[36249]: scrub summary: idle+waiting paths [/] 2024-05-14T09:35:55.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:55 smithi029 ceph-mon[36249]: scrub summary: active paths [/] 2024-05-14T09:35:55.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:55 smithi029 ceph-mon[36249]: scrub summary: idle+waiting paths [/] 2024-05-14T09:35:55.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:55 smithi026 ceph-mon[36056]: scrub queued for path: / 2024-05-14T09:35:55.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:55 smithi026 ceph-mon[36056]: scrub summary: idle+waiting paths [/] 2024-05-14T09:35:55.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:55 smithi026 ceph-mon[36056]: scrub summary: active paths [/] 2024-05-14T09:35:55.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:55 smithi026 ceph-mon[36056]: scrub summary: idle+waiting paths [/] 2024-05-14T09:35:56.908 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:56 smithi002 ceph-mon[31557]: pgmap v453: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-14T09:35:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:56 smithi029 ceph-mon[36249]: pgmap v453: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-14T09:35:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:56 smithi026 ceph-mon[36056]: pgmap v453: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-14T09:35:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.450+0000 7f4a701e7700 1 -- 172.21.15.2:0/405967439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a681034e0 msgr2=0x7f4a681038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.450+0000 7f4a701e7700 1 --2- 172.21.15.2:0/405967439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a681034e0 0x7f4a681038f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4a640099c0 tx=0x7f4a64009cd0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.450+0000 7f4a701e7700 1 -- 172.21.15.2:0/405967439 shutdown_connections 2024-05-14T09:35:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.450+0000 7f4a701e7700 1 --2- 172.21.15.2:0/405967439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a68108760 0x7f4a681100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.450+0000 7f4a701e7700 1 --2- 172.21.15.2:0/405967439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a681034e0 0x7f4a681038f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.450+0000 7f4a701e7700 1 --2- 172.21.15.2:0/405967439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a68107e10 0x7f4a68108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.450+0000 7f4a701e7700 1 -- 172.21.15.2:0/405967439 >> 172.21.15.2:0/405967439 conn(0x7f4a680fef30 msgr2=0x7f4a68101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.450+0000 7f4a701e7700 1 -- 172.21.15.2:0/405967439 shutdown_connections 2024-05-14T09:35:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.450+0000 7f4a701e7700 1 -- 172.21.15.2:0/405967439 wait complete. 2024-05-14T09:35:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.451+0000 7f4a701e7700 1 Processor -- start 2024-05-14T09:35:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.451+0000 7f4a701e7700 1 -- start start 2024-05-14T09:35:57.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.451+0000 7f4a701e7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a681034e0 0x7f4a6819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a701e7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a68107e10 0x7f4a6819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a701e7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a68108760 0x7f4a6819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a701e7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68113390 con 0x7f4a68108760 2024-05-14T09:35:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a701e7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68113210 con 0x7f4a68107e10 2024-05-14T09:35:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a701e7700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68113510 con 0x7f4a681034e0 2024-05-14T09:35:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a6e784700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a68108760 0x7f4a6819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a6e784700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a68108760 0x7f4a6819f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53340/0 (socket says 172.21.15.2:53340) 2024-05-14T09:35:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a6e784700 1 -- 172.21.15.2:0/2222810619 learned_addr learned my addr 172.21.15.2:0/2222810619 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a6d782700 1 --2- 172.21.15.2:0/2222810619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a68107e10 0x7f4a6819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a6df83700 1 --2- 172.21.15.2:0/2222810619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a681034e0 0x7f4a6819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a6e784700 1 -- 172.21.15.2:0/2222810619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a681034e0 msgr2=0x7f4a6819aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a6e784700 1 --2- 172.21.15.2:0/2222810619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a681034e0 0x7f4a6819aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a6e784700 1 -- 172.21.15.2:0/2222810619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a68107e10 msgr2=0x7f4a6819b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a6e784700 1 --2- 172.21.15.2:0/2222810619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a68107e10 0x7f4a6819b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a6e784700 1 -- 172.21.15.2:0/2222810619 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6819fd20 con 0x7f4a68108760 2024-05-14T09:35:57.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.452+0000 7f4a6e784700 1 --2- 172.21.15.2:0/2222810619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a68108760 0x7f4a6819f680 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4a60007ee0 tx=0x7f4a6000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:57.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.453+0000 7f4a5effd700 1 -- 172.21.15.2:0/2222810619 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60004bb0 con 0x7f4a68108760 2024-05-14T09:35:57.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.453+0000 7f4a5effd700 1 -- 172.21.15.2:0/2222810619 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a60004d50 con 0x7f4a68108760 2024-05-14T09:35:57.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.453+0000 7f4a701e7700 1 -- 172.21.15.2:0/2222810619 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a681a0010 con 0x7f4a68108760 2024-05-14T09:35:57.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.453+0000 7f4a5effd700 1 -- 172.21.15.2:0/2222810619 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60005920 con 0x7f4a68108760 2024-05-14T09:35:57.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.453+0000 7f4a701e7700 1 -- 172.21.15.2:0/2222810619 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a680ffd70 con 0x7f4a68108760 2024-05-14T09:35:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.456+0000 7f4a5effd700 1 -- 172.21.15.2:0/2222810619 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4a6001a020 con 0x7f4a68108760 2024-05-14T09:35:57.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.457+0000 7f4a7ebfbb80 1 Processor -- start 2024-05-14T09:35:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.457+0000 7f4a7ebfbb80 1 -- start start 2024-05-14T09:35:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.457+0000 7f4a5effd700 1 --2- 172.21.15.2:0/2222810619 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4a540b6c90 0x7f4a540b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.457+0000 7f4a7ebfbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f285e40 0x56465f403720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.457+0000 7f4a7ebfbb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f407f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.457+0000 7f4a7ebfbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f408470 0x56465f408880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.457+0000 7f4a6d782700 1 --2- 172.21.15.2:0/2222810619 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4a540b6c90 0x7f4a540b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.457+0000 7f4a5effd700 1 -- 172.21.15.2:0/2222810619 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f4a600e1cb0 con 0x7f4a68108760 2024-05-14T09:35:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.457+0000 7f4a7ebfbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56465f325860 con 0x56465f408470 2024-05-14T09:35:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.457+0000 7f4a7ebfbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56465f2c47c0 con 0x56465f285e40 2024-05-14T09:35:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.457+0000 7f4a7ebfbb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56465f321820 con 0x56465f403c60 2024-05-14T09:35:57.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.458+0000 7f4a6e784700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f408470 0x56465f408880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.458+0000 7f4a6df83700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f285e40 0x56465f403720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.458+0000 7f4a6e784700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f408470 0x56465f408880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53344/0 (socket says 172.21.15.2:53344) 2024-05-14T09:35:57.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.458+0000 7f4a6e784700 1 -- 172.21.15.2:0/3260555461 learned_addr learned my addr 172.21.15.2:0/3260555461 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:57.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.458+0000 7f4a6df83700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f285e40 0x56465f403720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:54512/0 (socket says 172.21.15.2:54512) 2024-05-14T09:35:57.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.458+0000 7f4a6d782700 1 --2- 172.21.15.2:0/3260555461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f407f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.458+0000 7f4a6df83700 1 -- 172.21.15.2:0/3260555461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 msgr2=0x56465f407f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:57.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.458+0000 7f4a6df83700 1 --2- 172.21.15.2:0/3260555461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f407f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.458+0000 7f4a6df83700 1 -- 172.21.15.2:0/3260555461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f408470 msgr2=0x56465f408880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:57.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.458+0000 7f4a6df83700 1 --2- 172.21.15.2:0/3260555461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f408470 0x56465f408880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.458+0000 7f4a6df83700 1 -- 172.21.15.2:0/3260555461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56465f2728d0 con 0x56465f285e40 2024-05-14T09:35:57.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.458+0000 7f4a6d782700 1 --2- 172.21.15.2:0/3260555461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f407f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:57.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.458+0000 7f4a6d782700 1 --2- 172.21.15.2:0/2222810619 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4a540b6c90 0x7f4a540b91a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4a68102640 tx=0x7f4a58013740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:57.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.458+0000 7f4a6df83700 1 --2- 172.21.15.2:0/3260555461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f285e40 0x56465f403720 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009f50 tx=0x7f4a64009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2c05d593b62defb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:57.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.458+0000 7f4a53fff700 1 -- 172.21.15.2:0/3260555461 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64003ae0 con 0x56465f285e40 2024-05-14T09:35:57.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.459+0000 7f4a53fff700 1 -- 172.21.15.2:0/3260555461 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a64005580 con 0x56465f285e40 2024-05-14T09:35:57.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.459+0000 7f4a53fff700 1 -- 172.21.15.2:0/3260555461 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400bc90 con 0x56465f285e40 2024-05-14T09:35:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.459+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/3260555461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f285e40 msgr2=0x56465f403720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.459+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/3260555461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f285e40 0x56465f403720 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009f50 tx=0x7f4a64009710 comp rx=0 tx=0).stop 2024-05-14T09:35:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.460+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/3260555461 shutdown_connections 2024-05-14T09:35:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.460+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/3260555461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f285e40 0x56465f403720 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.460+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/3260555461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f408470 0x56465f408880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.460+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/3260555461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f407f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.460+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/3260555461 >> 172.21.15.2:0/3260555461 conn(0x56465f2cd600 msgr2=0x56465f3fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.460+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/3260555461 shutdown_connections 2024-05-14T09:35:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.460+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/3260555461 wait complete. 2024-05-14T09:35:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.460+0000 7f4a7ebfbb80 1 -- start start 2024-05-14T09:35:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.460+0000 7f4a7ebfbb80 1 Processor -- start 2024-05-14T09:35:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a7ebfbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f285e40 0x56465f40d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a7ebfbb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f40d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a7ebfbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f408470 0x56465f40db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a7ebfbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56465f410300 con 0x56465f285e40 2024-05-14T09:35:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a7ebfbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56465f28bc30 con 0x56465f408470 2024-05-14T09:35:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a7ebfbb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56465f410480 con 0x56465f403c60 2024-05-14T09:35:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a7ebfbb80 10 client.0 _ll_register_callbacks cb 0x56465f2bd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a6df83700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f285e40 0x56465f40d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a6df83700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f285e40 0x56465f40d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53356/0 (socket says 172.21.15.2:53356) 2024-05-14T09:35:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a6df83700 1 -- 172.21.15.2:0/713010866 learned_addr learned my addr 172.21.15.2:0/713010866 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a6e784700 1 --2- 172.21.15.2:0/713010866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f408470 0x56465f40db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a6d782700 1 --2- 172.21.15.2:0/713010866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f40d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a6df83700 1 -- 172.21.15.2:0/713010866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 msgr2=0x56465f40d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a6df83700 1 --2- 172.21.15.2:0/713010866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f40d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a6df83700 1 -- 172.21.15.2:0/713010866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f408470 msgr2=0x56465f40db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a6df83700 1 --2- 172.21.15.2:0/713010866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f408470 0x56465f40db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.461+0000 7f4a6df83700 1 -- 172.21.15.2:0/713010866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56465f2c1690 con 0x56465f285e40 2024-05-14T09:35:57.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.462+0000 7f4a6df83700 1 --2- 172.21.15.2:0/713010866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f285e40 0x56465f40d100 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009d10 tx=0x7f4a64005380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a37fba49a2ca1e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:57.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.462+0000 7f4a517fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:57.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.462+0000 7f4a517fa700 1 -- 172.21.15.2:0/713010866 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6402f020 con 0x56465f285e40 2024-05-14T09:35:57.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.462+0000 7f4a517fa700 1 -- 172.21.15.2:0/713010866 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f4a64003af0 con 0x56465f285e40 2024-05-14T09:35:57.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.462+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/713010866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a64005580 con 0x56465f285e40 2024-05-14T09:35:57.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.462+0000 7f4a517fa700 1 -- 172.21.15.2:0/713010866 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4a64003cf0 con 0x56465f285e40 2024-05-14T09:35:57.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.462+0000 7f4a7ebfbb80 10 client.15534 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.462+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/713010866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56465f41aee0 con 0x56465f285e40 2024-05-14T09:35:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.463+0000 7f4a517fa700 1 -- 172.21.15.2:0/713010866 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a64005580 con 0x56465f285e40 2024-05-14T09:35:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.463+0000 7f4a517fa700 1 -- 172.21.15.2:0/713010866 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4a6400f960 con 0x56465f285e40 2024-05-14T09:35:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.463+0000 7f4a7ebfbb80 10 client.15534 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.463+0000 7f4a7ebfbb80 10 client.15534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.463+0000 7f4a7ebfbb80 20 client.15534 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.463+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/713010866 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56465f41d3b0 0x56465f41f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.463+0000 7f4a7ebfbb80 4 client.15534 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:35:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.463+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/713010866 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56465f333240 con 0x56465f41d3b0 2024-05-14T09:35:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.463+0000 7f4a6d782700 1 --2- 172.21.15.2:0/713010866 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56465f41d3b0 0x56465f41f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.464+0000 7f4a517fa700 10 client.15534 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.464+0000 7f4a6d782700 1 --2- 172.21.15.2:0/713010866 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56465f41d3b0 0x56465f41f7d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83b69c8104e192f0 server_cookie=c4399c2ab3943d8d in_seq=0 out_seq=0 2024-05-14T09:35:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.465+0000 7f4a517fa700 1 -- 172.21.15.2:0/713010866 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56465f333240 con 0x56465f41d3b0 2024-05-14T09:35:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.465+0000 7f4a517fa700 10 client.15534 handle_command_reply: tid=0 2024-05-14T09:35:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.465+0000 7f4a7ebfbb80 1 client.15534 shutdown 2024-05-14T09:35:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.465+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/713010866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f285e40 msgr2=0x56465f40d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.465+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/713010866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f285e40 0x56465f40d100 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009d10 tx=0x7f4a64005380 comp rx=0 tx=0).stop 2024-05-14T09:35:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.465+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/713010866 shutdown_connections 2024-05-14T09:35:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.465+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/713010866 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56465f41d3b0 0x56465f41f7d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.465+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/713010866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f408470 0x56465f40db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.465+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/713010866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f285e40 0x56465f40d100 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.465+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/713010866 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f40d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.465+0000 7f4a517fa700 0 client.15534 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.465+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/713010866 >> 172.21.15.2:0/713010866 conn(0x56465f2cd600 msgr2=0x56465f402450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.465+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/713010866 shutdown_connections 2024-05-14T09:35:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.465+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/713010866 wait complete. 2024-05-14T09:35:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.465+0000 7f4a7ebfbb80 20 client.15534 trim_cache size 0 max 16384 2024-05-14T09:35:57.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.486+0000 7f4a7ebfbb80 1 Processor -- start 2024-05-14T09:35:57.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.486+0000 7f4a7ebfbb80 1 -- start start 2024-05-14T09:35:57.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.487+0000 7f4a7ebfbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f285e40 0x56465f3fe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.487+0000 7f4a7ebfbb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f40c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.487+0000 7f4a7ebfbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f408470 0x56465f40d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.487+0000 7f4a7ebfbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681b0cb0 con 0x56465f285e40 2024-05-14T09:35:57.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.487+0000 7f4a7ebfbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681ae540 con 0x56465f408470 2024-05-14T09:35:57.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.487+0000 7f4a7ebfbb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681ac390 con 0x56465f403c60 2024-05-14T09:35:57.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.487+0000 7f4a6df83700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f285e40 0x56465f3fe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.487+0000 7f4a6df83700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f285e40 0x56465f3fe610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53362/0 (socket says 172.21.15.2:53362) 2024-05-14T09:35:57.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.487+0000 7f4a6df83700 1 -- 172.21.15.2:0/2812388554 learned_addr learned my addr 172.21.15.2:0/2812388554 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:57.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.487+0000 7f4a6d782700 1 --2- 172.21.15.2:0/2812388554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f40c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.487+0000 7f4a6e784700 1 --2- 172.21.15.2:0/2812388554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f408470 0x56465f40d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.488+0000 7f4a6df83700 1 -- 172.21.15.2:0/2812388554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 msgr2=0x56465f40c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.488+0000 7f4a6df83700 1 --2- 172.21.15.2:0/2812388554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f40c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.488+0000 7f4a6df83700 1 -- 172.21.15.2:0/2812388554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f408470 msgr2=0x56465f40d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.488+0000 7f4a6df83700 1 --2- 172.21.15.2:0/2812388554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f408470 0x56465f40d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.488+0000 7f4a6df83700 1 -- 172.21.15.2:0/2812388554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56465f404070 con 0x56465f285e40 2024-05-14T09:35:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.488+0000 7f4a6df83700 1 --2- 172.21.15.2:0/2812388554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f285e40 0x56465f3fe610 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400b580 tx=0x7f4a64009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da713e7a17b82f78 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.488+0000 7f4a537fe700 1 -- 172.21.15.2:0/2812388554 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6402f040 con 0x56465f285e40 2024-05-14T09:35:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.488+0000 7f4a537fe700 1 -- 172.21.15.2:0/2812388554 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a64022d00 con 0x56465f285e40 2024-05-14T09:35:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.488+0000 7f4a537fe700 1 -- 172.21.15.2:0/2812388554 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400f7c0 con 0x56465f285e40 2024-05-14T09:35:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.489+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/2812388554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f285e40 msgr2=0x56465f3fe610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.489+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/2812388554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f285e40 0x56465f3fe610 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400b580 tx=0x7f4a64009700 comp rx=0 tx=0).stop 2024-05-14T09:35:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.489+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/2812388554 shutdown_connections 2024-05-14T09:35:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.489+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/2812388554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f408470 0x56465f40d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.489+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/2812388554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f285e40 0x56465f3fe610 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.489+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/2812388554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f40c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.489+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/2812388554 >> 172.21.15.2:0/2812388554 conn(0x56465f2cd600 msgr2=0x56465f40e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.490+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/2812388554 shutdown_connections 2024-05-14T09:35:57.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.490+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/2812388554 wait complete. 2024-05-14T09:35:57.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.490+0000 7f4a7ebfbb80 1 -- start start 2024-05-14T09:35:57.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.490+0000 7f4a7ebfbb80 1 Processor -- start 2024-05-14T09:35:57.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a7ebfbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f285e40 0x56465f47b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a7ebfbb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f47bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a7ebfbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f408470 0x56465f47c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a7ebfbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681aca00 con 0x56465f408470 2024-05-14T09:35:57.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a7ebfbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681ade70 con 0x56465f285e40 2024-05-14T09:35:57.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a7ebfbb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681ad0d0 con 0x56465f403c60 2024-05-14T09:35:57.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a7ebfbb80 10 client.0 _ll_register_callbacks cb 0x56465f2bd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:35:57.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a6e784700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f408470 0x56465f47c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a6e784700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f408470 0x56465f47c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53374/0 (socket says 172.21.15.2:53374) 2024-05-14T09:35:57.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a6e784700 1 -- 172.21.15.2:0/2611292102 learned_addr learned my addr 172.21.15.2:0/2611292102 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:35:57.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a6d782700 1 --2- 172.21.15.2:0/2611292102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f47bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a6e784700 1 -- 172.21.15.2:0/2611292102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 msgr2=0x56465f47bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:57.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a6e784700 1 --2- 172.21.15.2:0/2611292102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f47bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a6e784700 1 -- 172.21.15.2:0/2611292102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f285e40 msgr2=0x56465f47b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:35:57.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a6df83700 1 --2- 172.21.15.2:0/2611292102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f285e40 0x56465f47b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a6e784700 1 --2- 172.21.15.2:0/2611292102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f285e40 0x56465f47b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a6e784700 1 -- 172.21.15.2:0/2611292102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56465f4869c0 con 0x56465f408470 2024-05-14T09:35:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a6d782700 1 --2- 172.21.15.2:0/2611292102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f47bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:35:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.491+0000 7f4a6df83700 1 --2- 172.21.15.2:0/2611292102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f285e40 0x56465f47b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:35:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.492+0000 7f4a6e784700 1 --2- 172.21.15.2:0/2611292102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f408470 0x56465f47c310 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4a600a8de0 tx=0x7f4a60007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2327ac3af69aa16a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:35:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.492+0000 7f4a517fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:35:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.492+0000 7f4a517fa700 1 -- 172.21.15.2:0/2611292102 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a600a9550 con 0x56465f408470 2024-05-14T09:35:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.492+0000 7f4a517fa700 1 -- 172.21.15.2:0/2611292102 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f4a600b1820 con 0x56465f408470 2024-05-14T09:35:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.492+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/2611292102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a64022d00 con 0x56465f408470 2024-05-14T09:35:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.492+0000 7f4a517fa700 1 -- 172.21.15.2:0/2611292102 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4a600a8e50 con 0x56465f408470 2024-05-14T09:35:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.492+0000 7f4a7ebfbb80 10 client.15540 fetch_fsmap learned FSMap version 27 2024-05-14T09:35:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.492+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/2611292102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56465f487c90 con 0x56465f408470 2024-05-14T09:35:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.492+0000 7f4a517fa700 1 -- 172.21.15.2:0/2611292102 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a64022d00 con 0x56465f408470 2024-05-14T09:35:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.493+0000 7f4a517fa700 1 -- 172.21.15.2:0/2611292102 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4a600a81d0 con 0x56465f408470 2024-05-14T09:35:57.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.493+0000 7f4a7ebfbb80 10 client.15540 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:35:57.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.493+0000 7f4a7ebfbb80 10 client.15540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:35:57.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.493+0000 7f4a7ebfbb80 20 client.15540 populate_metadata read hostname 'smithi002' 2024-05-14T09:35:57.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.493+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/2611292102 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56465f4883e0 0x56465f48a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:35:57.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.493+0000 7f4a7ebfbb80 4 client.15540 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T09:35:57.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.493+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/2611292102 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56465f32d2e0 con 0x56465f4883e0 2024-05-14T09:35:57.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.493+0000 7f4a6df83700 1 --2- 172.21.15.2:0/2611292102 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56465f4883e0 0x56465f48a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:35:57.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.494+0000 7f4a517fa700 10 client.15540 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:57.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.494+0000 7f4a6df83700 1 --2- 172.21.15.2:0/2611292102 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56465f4883e0 0x56465f48a800 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a22e87d130f5e8f6 server_cookie=d3fe96b90ebba66b in_seq=0 out_seq=0 2024-05-14T09:35:57.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.494+0000 7f4a517fa700 1 -- 172.21.15.2:0/2611292102 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56465f32d2e0 con 0x56465f4883e0 2024-05-14T09:35:57.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.494+0000 7f4a517fa700 10 client.15540 handle_command_reply: tid=0 2024-05-14T09:35:57.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.494+0000 7f4a7ebfbb80 1 client.15540 shutdown 2024-05-14T09:35:57.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.494+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/2611292102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f408470 msgr2=0x56465f47c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:57.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.494+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/2611292102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f408470 0x56465f47c310 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4a600a8de0 tx=0x7f4a60007f90 comp rx=0 tx=0).stop 2024-05-14T09:35:57.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.494+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/2611292102 shutdown_connections 2024-05-14T09:35:57.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.494+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/2611292102 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56465f4883e0 0x56465f48a800 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.494+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/2611292102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56465f285e40 0x56465f47b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.494+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/2611292102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56465f408470 0x56465f47c310 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.494+0000 7f4a517fa700 0 client.15540 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:35:57.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.494+0000 7f4a7ebfbb80 1 --2- 172.21.15.2:0/2611292102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56465f403c60 0x56465f47bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.494+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/2611292102 >> 172.21.15.2:0/2611292102 conn(0x56465f2cd600 msgr2=0x56465f3fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:57.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.495+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/2611292102 shutdown_connections 2024-05-14T09:35:57.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.495+0000 7f4a7ebfbb80 1 -- 172.21.15.2:0/2611292102 wait complete. 2024-05-14T09:35:57.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.495+0000 7f4a7ebfbb80 20 client.15540 trim_cache size 0 max 16384 2024-05-14T09:35:57.498 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:35:57.498 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T09:35:57.498 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-05-14T09:35:57.499 INFO:teuthology.orchestra.run.smithi002.stdout: "739ce1c0-71d3-4aa6-a69b-ff7ca400e738": { 2024-05-14T09:35:57.499 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-05-14T09:35:57.499 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "739ce1c0-71d3-4aa6-a69b-ff7ca400e738", 2024-05-14T09:35:57.499 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-05-14T09:35:57.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T09:35:57.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T09:35:57.499 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:35:57.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.495+0000 7f4a701e7700 1 -- 172.21.15.2:0/2222810619 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4a540b6c90 msgr2=0x7f4a540b91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:57.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.495+0000 7f4a701e7700 1 --2- 172.21.15.2:0/2222810619 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4a540b6c90 0x7f4a540b91a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4a68102640 tx=0x7f4a58013740 comp rx=0 tx=0).stop 2024-05-14T09:35:57.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.495+0000 7f4a701e7700 1 -- 172.21.15.2:0/2222810619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a68108760 msgr2=0x7f4a6819f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:35:57.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.495+0000 7f4a701e7700 1 --2- 172.21.15.2:0/2222810619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a68108760 0x7f4a6819f680 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4a60007ee0 tx=0x7f4a6000d3b0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.496+0000 7f4a701e7700 1 -- 172.21.15.2:0/2222810619 shutdown_connections 2024-05-14T09:35:57.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.496+0000 7f4a701e7700 1 --2- 172.21.15.2:0/2222810619 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4a540b6c90 0x7f4a540b91a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.496+0000 7f4a701e7700 1 --2- 172.21.15.2:0/2222810619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a68108760 0x7f4a6819f680 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.496+0000 7f4a701e7700 1 --2- 172.21.15.2:0/2222810619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a681034e0 0x7f4a6819aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.496+0000 7f4a701e7700 1 --2- 172.21.15.2:0/2222810619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a68107e10 0x7f4a6819b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:35:57.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.496+0000 7f4a701e7700 1 -- 172.21.15.2:0/2222810619 >> 172.21.15.2:0/2222810619 conn(0x7f4a680fef30 msgr2=0x7f4a681063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:35:57.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.496+0000 7f4a701e7700 1 -- 172.21.15.2:0/2222810619 shutdown_connections 2024-05-14T09:35:57.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:35:57.496+0000 7f4a701e7700 1 -- 172.21.15.2:0/2222810619 wait complete. 2024-05-14T09:35:58.372 INFO:tasks.cephfs.filesystem:scrub status for tag:739ce1c0-71d3-4aa6-a69b-ff7ca400e738 - {'path': '/', 'tag': '739ce1c0-71d3-4aa6-a69b-ff7ca400e738', 'options': 'recursive,force'} 2024-05-14T09:35:58.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:58 smithi002 ceph-mon[31557]: pgmap v454: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-14T09:35:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:58 smithi029 ceph-mon[36249]: pgmap v454: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-14T09:35:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:58 smithi026 ceph-mon[36056]: pgmap v454: 129 pgs: 129 active+clean; 666 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-14T09:35:59.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:35:59 smithi002 ceph-mon[31557]: scrub summary: idle 2024-05-14T09:35:59.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:35:59 smithi029 ceph-mon[36249]: scrub summary: idle 2024-05-14T09:35:59.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:35:59 smithi026 ceph-mon[36056]: scrub summary: idle 2024-05-14T09:36:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:00 smithi002 ceph-mon[31557]: pgmap v455: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-14T09:36:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:00 smithi029 ceph-mon[36249]: pgmap v455: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-14T09:36:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:00 smithi026 ceph-mon[36056]: pgmap v455: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-14T09:36:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:02 smithi002 ceph-mon[31557]: pgmap v456: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-14T09:36:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:02 smithi029 ceph-mon[36249]: pgmap v456: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-14T09:36:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:02 smithi026 ceph-mon[36056]: pgmap v456: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-14T09:36:04.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:04 smithi002 ceph-mon[31557]: pgmap v457: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-14T09:36:04.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:04 smithi029 ceph-mon[36249]: pgmap v457: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-14T09:36:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:04 smithi026 ceph-mon[36056]: pgmap v457: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-14T09:36:06.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:06 smithi002 ceph-mon[31557]: pgmap v458: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-14T09:36:06.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:06 smithi029 ceph-mon[36249]: pgmap v458: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-14T09:36:06.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:06 smithi026 ceph-mon[36056]: pgmap v458: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-14T09:36:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.008+0000 7fa40fb76640 1 -- 172.21.15.2:0/4241362306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa408102b50 msgr2=0x7fa408103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.008+0000 7fa40fb76640 1 --2- 172.21.15.2:0/4241362306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa408102b50 0x7fa408103850 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa404009900 tx=0x7fa40402a410 comp rx=0 tx=0).stop 2024-05-14T09:36:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.008+0000 7fa40fb76640 1 -- 172.21.15.2:0/4241362306 shutdown_connections 2024-05-14T09:36:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.008+0000 7fa40fb76640 1 --2- 172.21.15.2:0/4241362306 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa408108a90 0x7fa40810aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.008+0000 7fa40fb76640 1 --2- 172.21.15.2:0/4241362306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa408103d90 0x7fa4081041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.008+0000 7fa40fb76640 1 --2- 172.21.15.2:0/4241362306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa408102b50 0x7fa408103850 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.008+0000 7fa40fb76640 1 -- 172.21.15.2:0/4241362306 >> 172.21.15.2:0/4241362306 conn(0x7fa4080fdd40 msgr2=0x7fa4081001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.008+0000 7fa40fb76640 1 -- 172.21.15.2:0/4241362306 shutdown_connections 2024-05-14T09:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.008+0000 7fa40fb76640 1 -- 172.21.15.2:0/4241362306 wait complete. 2024-05-14T09:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.009+0000 7fa40fb76640 1 Processor -- start 2024-05-14T09:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.009+0000 7fa40fb76640 1 -- start start 2024-05-14T09:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.009+0000 7fa40fb76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa408102b50 0x7fa40819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.009+0000 7fa40fb76640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa408103d90 0x7fa40819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.009+0000 7fa40fb76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa408108a90 0x7fa4081a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.009+0000 7fa40fb76640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa40810f740 con 0x7fa408108a90 2024-05-14T09:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.009+0000 7fa40fb76640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa40810f5c0 con 0x7fa408102b50 2024-05-14T09:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.009+0000 7fa40fb76640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa40810f8c0 con 0x7fa408103d90 2024-05-14T09:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.009+0000 7fa40e113640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa408108a90 0x7fa4081a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.009+0000 7fa40e113640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa408108a90 0x7fa4081a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44208/0 (socket says 172.21.15.2:44208) 2024-05-14T09:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.009+0000 7fa40e113640 1 -- 172.21.15.2:0/230750823 learned_addr learned my addr 172.21.15.2:0/230750823 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.010+0000 7fa40d912640 1 --2- 172.21.15.2:0/230750823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa408102b50 0x7fa40819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.010+0000 7fa40d111640 1 --2- 172.21.15.2:0/230750823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa408103d90 0x7fa40819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.010+0000 7fa40e113640 1 -- 172.21.15.2:0/230750823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa408103d90 msgr2=0x7fa40819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.010+0000 7fa40e113640 1 --2- 172.21.15.2:0/230750823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa408103d90 0x7fa40819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.010+0000 7fa40e113640 1 -- 172.21.15.2:0/230750823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa408102b50 msgr2=0x7fa40819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.010+0000 7fa40e113640 1 --2- 172.21.15.2:0/230750823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa408102b50 0x7fa40819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.010+0000 7fa40e113640 1 -- 172.21.15.2:0/230750823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4081a1e30 con 0x7fa408108a90 2024-05-14T09:36:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.010+0000 7fa40e113640 1 --2- 172.21.15.2:0/230750823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa408108a90 0x7fa4081a17c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa40000eb10 tx=0x7fa40000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.010+0000 7fa3feffd640 1 -- 172.21.15.2:0/230750823 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400004b20 con 0x7fa408108a90 2024-05-14T09:36:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.010+0000 7fa3feffd640 1 -- 172.21.15.2:0/230750823 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa400004cc0 con 0x7fa408108a90 2024-05-14T09:36:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.010+0000 7fa40fb76640 1 -- 172.21.15.2:0/230750823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4081a2120 con 0x7fa408108a90 2024-05-14T09:36:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.010+0000 7fa3feffd640 1 -- 172.21.15.2:0/230750823 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400010670 con 0x7fa408108a90 2024-05-14T09:36:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.010+0000 7fa40fb76640 1 -- 172.21.15.2:0/230750823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4081a2670 con 0x7fa408108a90 2024-05-14T09:36:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.013+0000 7fa416a2f740 1 Processor -- start 2024-05-14T09:36:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.014+0000 7fa3feffd640 1 -- 172.21.15.2:0/230750823 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa400010810 con 0x7fa408108a90 2024-05-14T09:36:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.014+0000 7fa416a2f740 1 -- start start 2024-05-14T09:36:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.014+0000 7fa416a2f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b38511b0 0x5566b3bf6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.014+0000 7fa416a2f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b38282f0 0x5566b3bf73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.014+0000 7fa416a2f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38304d0 0x5566b3bfe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.014+0000 7fa416a2f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5566b3b575d0 con 0x5566b38511b0 2024-05-14T09:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.014+0000 7fa416a2f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5566b3b10890 con 0x5566b38304d0 2024-05-14T09:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.014+0000 7fa416a2f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5566b3b53590 con 0x5566b38282f0 2024-05-14T09:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.014+0000 7fa40d111640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b38511b0 0x5566b3bf6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.015+0000 7fa40d111640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b38511b0 0x5566b3bf6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44224/0 (socket says 172.21.15.2:44224) 2024-05-14T09:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.015+0000 7fa40d111640 1 -- 172.21.15.2:0/2091028758 learned_addr learned my addr 172.21.15.2:0/2091028758 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.015+0000 7fa40d912640 1 --2- 172.21.15.2:0/2091028758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b38282f0 0x5566b3bf73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.015+0000 7fa40d111640 1 --2- 172.21.15.2:0/2091028758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38304d0 0x5566b3bfe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.015+0000 7fa40d111640 1 -- 172.21.15.2:0/2091028758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b38282f0 msgr2=0x5566b3bf73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.015+0000 7fa40d111640 1 --2- 172.21.15.2:0/2091028758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b38282f0 0x5566b3bf73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.015+0000 7fa40d111640 1 -- 172.21.15.2:0/2091028758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38304d0 msgr2=0x5566b3bfe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.015+0000 7fa40d111640 1 --2- 172.21.15.2:0/2091028758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38304d0 0x5566b3bfe2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.015+0000 7fa40d111640 1 -- 172.21.15.2:0/2091028758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566b3869210 con 0x5566b38511b0 2024-05-14T09:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.015+0000 7fa3feffd640 1 --2- 172.21.15.2:0/230750823 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa3f40b28a0 0x7fa3f40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.015+0000 7fa40d912640 1 --2- 172.21.15.2:0/230750823 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa3f40b28a0 0x7fa3f40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.015+0000 7fa3feffd640 1 -- 172.21.15.2:0/230750823 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fa4000e2ba0 con 0x7fa408108a90 2024-05-14T09:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.015+0000 7fa40d111640 1 --2- 172.21.15.2:0/2091028758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b38511b0 0x5566b3bf6e70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa3f80125e0 tx=0x7fa3f800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3807c1922b9a4a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.016+0000 7fa3f3fff640 1 -- 172.21.15.2:0/2091028758 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80106b0 con 0x5566b38511b0 2024-05-14T09:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.016+0000 7fa3f3fff640 1 -- 172.21.15.2:0/2091028758 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3f8010d00 con 0x5566b38511b0 2024-05-14T09:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.016+0000 7fa3f3fff640 1 -- 172.21.15.2:0/2091028758 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f801dd70 con 0x5566b38511b0 2024-05-14T09:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.017+0000 7fa416a2f740 1 -- 172.21.15.2:0/2091028758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b38511b0 msgr2=0x5566b3bf6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.017+0000 7fa40d912640 1 --2- 172.21.15.2:0/230750823 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa3f40b28a0 0x7fa3f40b4db0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa404005ab0 tx=0x7fa404031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.017+0000 7fa416a2f740 1 --2- 172.21.15.2:0/2091028758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b38511b0 0x5566b3bf6e70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa3f80125e0 tx=0x7fa3f800b860 comp rx=0 tx=0).stop 2024-05-14T09:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.017+0000 7fa416a2f740 1 -- 172.21.15.2:0/2091028758 shutdown_connections 2024-05-14T09:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.017+0000 7fa416a2f740 1 --2- 172.21.15.2:0/2091028758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38304d0 0x5566b3bfe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.017+0000 7fa416a2f740 1 --2- 172.21.15.2:0/2091028758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b38282f0 0x5566b3bf73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.017+0000 7fa416a2f740 1 --2- 172.21.15.2:0/2091028758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b38511b0 0x5566b3bf6e70 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.017+0000 7fa416a2f740 1 -- 172.21.15.2:0/2091028758 >> 172.21.15.2:0/2091028758 conn(0x5566b3845300 msgr2=0x5566b3bee3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.017+0000 7fa416a2f740 1 -- 172.21.15.2:0/2091028758 shutdown_connections 2024-05-14T09:36:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.017+0000 7fa416a2f740 1 -- 172.21.15.2:0/2091028758 wait complete. 2024-05-14T09:36:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.017+0000 7fa416a2f740 1 -- start start 2024-05-14T09:36:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.017+0000 7fa416a2f740 1 Processor -- start 2024-05-14T09:36:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.018+0000 7fa416a2f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38282f0 0x5566b3c00c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.018+0000 7fa416a2f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b38304d0 0x5566b3c01170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.018+0000 7fa416a2f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b380fd80 0x5566b3c080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.018+0000 7fa416a2f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5566b3bfed40 con 0x5566b380fd80 2024-05-14T09:36:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.018+0000 7fa416a2f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5566b3b14370 con 0x5566b38282f0 2024-05-14T09:36:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.018+0000 7fa416a2f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5566b3bfeec0 con 0x5566b38304d0 2024-05-14T09:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.018+0000 7fa416a2f740 10 client.0 _ll_register_callbacks cb 0x5566b3868a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.018+0000 7fa40e113640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b380fd80 0x5566b3c080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.018+0000 7fa40e113640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b380fd80 0x5566b3c080d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44228/0 (socket says 172.21.15.2:44228) 2024-05-14T09:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.018+0000 7fa40e113640 1 -- 172.21.15.2:0/2401816385 learned_addr learned my addr 172.21.15.2:0/2401816385 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.018+0000 7fa40d912640 1 --2- 172.21.15.2:0/2401816385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38282f0 0x5566b3c00c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.018+0000 7fa40d111640 1 --2- 172.21.15.2:0/2401816385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b38304d0 0x5566b3c01170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.019+0000 7fa40e113640 1 -- 172.21.15.2:0/2401816385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b38304d0 msgr2=0x5566b3c01170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.019+0000 7fa40e113640 1 --2- 172.21.15.2:0/2401816385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b38304d0 0x5566b3c01170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.019+0000 7fa40e113640 1 -- 172.21.15.2:0/2401816385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38282f0 msgr2=0x5566b3c00c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.019+0000 7fa40e113640 1 --2- 172.21.15.2:0/2401816385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38282f0 0x5566b3c00c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.019+0000 7fa40e113640 1 -- 172.21.15.2:0/2401816385 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566b3935620 con 0x5566b380fd80 2024-05-14T09:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.019+0000 7fa40e113640 1 --2- 172.21.15.2:0/2401816385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b380fd80 0x5566b3c080d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa4000a7010 tx=0x7fa4000a97b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145704c56898c81b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.019+0000 7fa3f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.019+0000 7fa3f17fa640 1 -- 172.21.15.2:0/2401816385 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4000a9da0 con 0x5566b380fd80 2024-05-14T09:36:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.019+0000 7fa416a2f740 1 -- 172.21.15.2:0/2401816385 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3f8010d00 con 0x5566b380fd80 2024-05-14T09:36:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.019+0000 7fa3f17fa640 1 -- 172.21.15.2:0/2401816385 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fa4000b2980 con 0x5566b380fd80 2024-05-14T09:36:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.019+0000 7fa3f17fa640 1 -- 172.21.15.2:0/2401816385 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4000ae150 con 0x5566b380fd80 2024-05-14T09:36:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.019+0000 7fa416a2f740 10 client.15552 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.019+0000 7fa416a2f740 1 -- 172.21.15.2:0/2401816385 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5566b3c0b0d0 con 0x5566b380fd80 2024-05-14T09:36:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.019+0000 7fa3f17fa640 1 -- 172.21.15.2:0/2401816385 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3f8010d00 con 0x5566b380fd80 2024-05-14T09:36:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.020+0000 7fa3f17fa640 1 -- 172.21.15.2:0/2401816385 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa4000a5810 con 0x5566b380fd80 2024-05-14T09:36:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.020+0000 7fa416a2f740 10 client.15552 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.020+0000 7fa416a2f740 10 client.15552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.020+0000 7fa416a2f740 20 client.15552 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.020+0000 7fa416a2f740 1 --2- 172.21.15.2:0/2401816385 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5566b37e51c0 0x5566b3c0ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.020+0000 7fa416a2f740 4 client.15552 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:36:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.020+0000 7fa416a2f740 1 -- 172.21.15.2:0/2401816385 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566b3b651e0 con 0x5566b37e51c0 2024-05-14T09:36:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.020+0000 7fa40d912640 1 --2- 172.21.15.2:0/2401816385 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5566b37e51c0 0x5566b3c0ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.021+0000 7fa3f17fa640 10 client.15552 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.021+0000 7fa40d912640 1 --2- 172.21.15.2:0/2401816385 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5566b37e51c0 0x5566b3c0ecc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4367d1f39cd85f server_cookie=c61bb7e0731e6418 in_seq=0 out_seq=0 2024-05-14T09:36:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.021+0000 7fa3f17fa640 1 -- 172.21.15.2:0/2401816385 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5566b3b651e0 con 0x5566b37e51c0 2024-05-14T09:36:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.021+0000 7fa3f17fa640 10 client.15552 handle_command_reply: tid=0 2024-05-14T09:36:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.021+0000 7fa416a2f740 1 client.15552 shutdown 2024-05-14T09:36:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.022+0000 7fa416a2f740 1 -- 172.21.15.2:0/2401816385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b380fd80 msgr2=0x5566b3c080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.022+0000 7fa416a2f740 1 --2- 172.21.15.2:0/2401816385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b380fd80 0x5566b3c080d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa4000a7010 tx=0x7fa4000a97b0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.022+0000 7fa416a2f740 1 -- 172.21.15.2:0/2401816385 shutdown_connections 2024-05-14T09:36:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.022+0000 7fa416a2f740 1 --2- 172.21.15.2:0/2401816385 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5566b37e51c0 0x5566b3c0ecc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.022+0000 7fa416a2f740 1 --2- 172.21.15.2:0/2401816385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b380fd80 0x5566b3c080d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.022+0000 7fa3f17fa640 0 client.15552 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.022+0000 7fa416a2f740 1 --2- 172.21.15.2:0/2401816385 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b38304d0 0x5566b3c01170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.022+0000 7fa416a2f740 1 --2- 172.21.15.2:0/2401816385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38282f0 0x5566b3c00c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.022+0000 7fa416a2f740 1 -- 172.21.15.2:0/2401816385 >> 172.21.15.2:0/2401816385 conn(0x5566b3845300 msgr2=0x5566b3bf1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.022+0000 7fa416a2f740 1 -- 172.21.15.2:0/2401816385 shutdown_connections 2024-05-14T09:36:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.022+0000 7fa416a2f740 1 -- 172.21.15.2:0/2401816385 wait complete. 2024-05-14T09:36:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.022+0000 7fa416a2f740 20 client.15552 trim_cache size 0 max 16384 2024-05-14T09:36:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.037+0000 7fa416a2f740 1 Processor -- start 2024-05-14T09:36:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.037+0000 7fa416a2f740 1 -- start start 2024-05-14T09:36:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa416a2f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b37e51c0 0x5566b3bedd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa416a2f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b380fd80 0x5566b3bee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa416a2f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38282f0 0x5566b3c08090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa416a2f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081ad300 con 0x5566b380fd80 2024-05-14T09:36:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa416a2f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081aabd0 con 0x5566b38282f0 2024-05-14T09:36:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa416a2f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081a8a20 con 0x5566b37e51c0 2024-05-14T09:36:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa40d111640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b380fd80 0x5566b3bee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa40d111640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b380fd80 0x5566b3bee2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44236/0 (socket says 172.21.15.2:44236) 2024-05-14T09:36:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa40d912640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b37e51c0 0x5566b3bedd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa40d111640 1 -- 172.21.15.2:0/4096546335 learned_addr learned my addr 172.21.15.2:0/4096546335 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa40e113640 1 --2- 172.21.15.2:0/4096546335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38282f0 0x5566b3c08090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa40d912640 1 -- 172.21.15.2:0/4096546335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38282f0 msgr2=0x5566b3c08090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa40d912640 1 --2- 172.21.15.2:0/4096546335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38282f0 0x5566b3c08090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa40d912640 1 -- 172.21.15.2:0/4096546335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b380fd80 msgr2=0x5566b3bee2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa40d912640 1 --2- 172.21.15.2:0/4096546335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b380fd80 0x5566b3bee2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa40d912640 1 -- 172.21.15.2:0/4096546335 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566b3be84d0 con 0x5566b37e51c0 2024-05-14T09:36:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.038+0000 7fa40d111640 1 --2- 172.21.15.2:0/4096546335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b380fd80 0x5566b3bee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.039+0000 7fa40d912640 1 --2- 172.21.15.2:0/4096546335 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b37e51c0 0x5566b3bedd60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa40402a930 tx=0x7fa404009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be69ad519e3fe036 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.039+0000 7fa3f37fe640 1 -- 172.21.15.2:0/4096546335 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404035070 con 0x5566b37e51c0 2024-05-14T09:36:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.039+0000 7fa3f37fe640 1 -- 172.21.15.2:0/4096546335 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa404004440 con 0x5566b37e51c0 2024-05-14T09:36:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.039+0000 7fa3f37fe640 1 -- 172.21.15.2:0/4096546335 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404003b20 con 0x5566b37e51c0 2024-05-14T09:36:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.040+0000 7fa416a2f740 1 -- 172.21.15.2:0/4096546335 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b37e51c0 msgr2=0x5566b3bedd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.040+0000 7fa416a2f740 1 --2- 172.21.15.2:0/4096546335 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b37e51c0 0x5566b3bedd60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa40402a930 tx=0x7fa404009260 comp rx=0 tx=0).stop 2024-05-14T09:36:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.040+0000 7fa416a2f740 1 -- 172.21.15.2:0/4096546335 shutdown_connections 2024-05-14T09:36:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.040+0000 7fa416a2f740 1 --2- 172.21.15.2:0/4096546335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38282f0 0x5566b3c08090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.040+0000 7fa416a2f740 1 --2- 172.21.15.2:0/4096546335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b380fd80 0x5566b3bee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.040+0000 7fa416a2f740 1 --2- 172.21.15.2:0/4096546335 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b37e51c0 0x5566b3bedd60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.040+0000 7fa416a2f740 1 -- 172.21.15.2:0/4096546335 >> 172.21.15.2:0/4096546335 conn(0x5566b3845300 msgr2=0x5566b3bfb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.040+0000 7fa416a2f740 1 -- 172.21.15.2:0/4096546335 shutdown_connections 2024-05-14T09:36:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.040+0000 7fa416a2f740 1 -- 172.21.15.2:0/4096546335 wait complete. 2024-05-14T09:36:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.040+0000 7fa416a2f740 1 -- start start 2024-05-14T09:36:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.040+0000 7fa416a2f740 1 Processor -- start 2024-05-14T09:36:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa416a2f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b37e51c0 0x5566b3c05860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa416a2f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b380fd80 0x5566b3c05da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa40d912640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b37e51c0 0x5566b3c05860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa40d912640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b37e51c0 0x5566b3c05860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44250/0 (socket says 172.21.15.2:44250) 2024-05-14T09:36:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa416a2f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38282f0 0x5566b3c75060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa40d912640 1 -- 172.21.15.2:0/118799382 learned_addr learned my addr 172.21.15.2:0/118799382 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa416a2f740 1 -- 172.21.15.2:0/118799382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081a9090 con 0x5566b37e51c0 2024-05-14T09:36:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa416a2f740 1 -- 172.21.15.2:0/118799382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081aa500 con 0x5566b38282f0 2024-05-14T09:36:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa416a2f740 1 -- 172.21.15.2:0/118799382 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081a9760 con 0x5566b380fd80 2024-05-14T09:36:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa416a2f740 10 client.0 _ll_register_callbacks cb 0x7fa4000a9b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa40d111640 1 --2- 172.21.15.2:0/118799382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b380fd80 0x5566b3c05da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa40d912640 1 -- 172.21.15.2:0/118799382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b380fd80 msgr2=0x5566b3c05da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa40e113640 1 --2- 172.21.15.2:0/118799382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38282f0 0x5566b3c75060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa40d912640 1 --2- 172.21.15.2:0/118799382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b380fd80 0x5566b3c05da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa40d912640 1 -- 172.21.15.2:0/118799382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38282f0 msgr2=0x5566b3c75060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa40d912640 1 --2- 172.21.15.2:0/118799382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38282f0 0x5566b3c75060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa40d912640 1 -- 172.21.15.2:0/118799382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566b3c755a0 con 0x5566b37e51c0 2024-05-14T09:36:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.041+0000 7fa40d111640 1 --2- 172.21.15.2:0/118799382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b380fd80 0x5566b3c05da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:36:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.042+0000 7fa40d912640 1 --2- 172.21.15.2:0/118799382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b37e51c0 0x5566b3c05860 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa404003b90 tx=0x7fa404003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1dff77de6667c9a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.042+0000 7fa3f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:36:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.042+0000 7fa3f17fa640 1 -- 172.21.15.2:0/118799382 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40405f060 con 0x5566b37e51c0 2024-05-14T09:36:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.042+0000 7fa3f17fa640 1 -- 172.21.15.2:0/118799382 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fa404052420 con 0x5566b37e51c0 2024-05-14T09:36:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.042+0000 7fa416a2f740 1 -- 172.21.15.2:0/118799382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa404004440 con 0x5566b37e51c0 2024-05-14T09:36:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.042+0000 7fa3f17fa640 1 -- 172.21.15.2:0/118799382 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa404066050 con 0x5566b37e51c0 2024-05-14T09:36:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.042+0000 7fa416a2f740 10 client.15558 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.042+0000 7fa416a2f740 1 -- 172.21.15.2:0/118799382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5566b3c76770 con 0x5566b37e51c0 2024-05-14T09:36:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.043+0000 7fa3f17fa640 1 -- 172.21.15.2:0/118799382 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa404004440 con 0x5566b37e51c0 2024-05-14T09:36:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.043+0000 7fa3f17fa640 1 -- 172.21.15.2:0/118799382 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa40404f070 con 0x5566b37e51c0 2024-05-14T09:36:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.043+0000 7fa416a2f740 10 client.15558 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.043+0000 7fa416a2f740 10 client.15558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.043+0000 7fa416a2f740 20 client.15558 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.043+0000 7fa416a2f740 1 --2- 172.21.15.2:0/118799382 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5566b38304d0 0x5566b3c79080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.043+0000 7fa416a2f740 4 client.15558 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:36:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.043+0000 7fa416a2f740 1 -- 172.21.15.2:0/118799382 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566b3b5f050 con 0x5566b38304d0 2024-05-14T09:36:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.043+0000 7fa40d111640 1 --2- 172.21.15.2:0/118799382 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5566b38304d0 0x5566b3c79080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.044+0000 7fa3f17fa640 10 client.15558 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.044+0000 7fa40d111640 1 --2- 172.21.15.2:0/118799382 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5566b38304d0 0x5566b3c79080 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c679937a959facfc server_cookie=fd019d5775ff1773 in_seq=0 out_seq=0 2024-05-14T09:36:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.048+0000 7fa3f17fa640 1 -- 172.21.15.2:0/118799382 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566b3b5f050 con 0x5566b38304d0 2024-05-14T09:36:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.048+0000 7fa3f17fa640 10 client.15558 handle_command_reply: tid=0 2024-05-14T09:36:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.048+0000 7fa416a2f740 1 client.15558 shutdown 2024-05-14T09:36:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.048+0000 7fa416a2f740 1 -- 172.21.15.2:0/118799382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b37e51c0 msgr2=0x5566b3c05860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.048+0000 7fa416a2f740 1 --2- 172.21.15.2:0/118799382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b37e51c0 0x5566b3c05860 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa404003b90 tx=0x7fa404003bc0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.048+0000 7fa416a2f740 1 -- 172.21.15.2:0/118799382 shutdown_connections 2024-05-14T09:36:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.048+0000 7fa416a2f740 1 --2- 172.21.15.2:0/118799382 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5566b38304d0 0x5566b3c79080 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.048+0000 7fa416a2f740 1 --2- 172.21.15.2:0/118799382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5566b38282f0 0x5566b3c75060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.048+0000 7fa416a2f740 1 --2- 172.21.15.2:0/118799382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5566b380fd80 0x5566b3c05da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.048+0000 7fa416a2f740 1 --2- 172.21.15.2:0/118799382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566b37e51c0 0x5566b3c05860 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.048+0000 7fa416a2f740 1 -- 172.21.15.2:0/118799382 >> 172.21.15.2:0/118799382 conn(0x5566b3845300 msgr2=0x5566b3bedd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.048+0000 7fa3f17fa640 0 client.15558 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.048+0000 7fa416a2f740 1 -- 172.21.15.2:0/118799382 shutdown_connections 2024-05-14T09:36:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.049+0000 7fa416a2f740 1 -- 172.21.15.2:0/118799382 wait complete. 2024-05-14T09:36:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.049+0000 7fa416a2f740 20 client.15558 trim_cache size 0 max 16384 2024-05-14T09:36:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:36:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:36:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:36:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:36:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.049+0000 7fa40fb76640 1 -- 172.21.15.2:0/230750823 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa3f40b28a0 msgr2=0x7fa3f40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.049+0000 7fa40fb76640 1 --2- 172.21.15.2:0/230750823 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa3f40b28a0 0x7fa3f40b4db0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa404005ab0 tx=0x7fa404031400 comp rx=0 tx=0).stop 2024-05-14T09:36:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.049+0000 7fa40fb76640 1 -- 172.21.15.2:0/230750823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa408108a90 msgr2=0x7fa4081a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.049+0000 7fa40fb76640 1 --2- 172.21.15.2:0/230750823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa408108a90 0x7fa4081a17c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa40000eb10 tx=0x7fa40000c2d0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.049+0000 7fa40fb76640 1 -- 172.21.15.2:0/230750823 shutdown_connections 2024-05-14T09:36:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.049+0000 7fa40fb76640 1 --2- 172.21.15.2:0/230750823 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa3f40b28a0 0x7fa3f40b4db0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.049+0000 7fa40fb76640 1 --2- 172.21.15.2:0/230750823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa408108a90 0x7fa4081a17c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.049+0000 7fa40fb76640 1 --2- 172.21.15.2:0/230750823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa408103d90 0x7fa40819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.049+0000 7fa40fb76640 1 --2- 172.21.15.2:0/230750823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa408102b50 0x7fa40819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.049+0000 7fa40fb76640 1 -- 172.21.15.2:0/230750823 >> 172.21.15.2:0/230750823 conn(0x7fa4080fdd40 msgr2=0x7fa408104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.049+0000 7fa40fb76640 1 -- 172.21.15.2:0/230750823 shutdown_connections 2024-05-14T09:36:08.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:08.050+0000 7fa40fb76640 1 -- 172.21.15.2:0/230750823 wait complete. 2024-05-14T09:36:08.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:08 smithi002 ceph-mon[31557]: pgmap v459: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-14T09:36:08.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:08 smithi029 ceph-mon[36249]: pgmap v459: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-14T09:36:08.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:08 smithi026 ceph-mon[36056]: pgmap v459: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-14T09:36:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:10 smithi002 ceph-mon[31557]: pgmap v460: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-14T09:36:10.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:10 smithi029 ceph-mon[36249]: pgmap v460: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-14T09:36:10.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:10 smithi026 ceph-mon[36056]: pgmap v460: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-14T09:36:12.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:12 smithi002 ceph-mon[31557]: pgmap v461: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:36:12.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:12 smithi029 ceph-mon[36249]: pgmap v461: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:36:12.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:12 smithi026 ceph-mon[36056]: pgmap v461: 129 pgs: 129 active+clean; 667 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:36:14.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:14 smithi002 ceph-mon[31557]: pgmap v462: 129 pgs: 129 active+clean; 668 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T09:36:14.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:14 smithi029 ceph-mon[36249]: pgmap v462: 129 pgs: 129 active+clean; 668 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T09:36:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:14 smithi026 ceph-mon[36056]: pgmap v462: 129 pgs: 129 active+clean; 668 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T09:36:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:16 smithi002 ceph-mon[31557]: pgmap v463: 129 pgs: 129 active+clean; 668 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:16.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:16 smithi029 ceph-mon[36249]: pgmap v463: 129 pgs: 129 active+clean; 668 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:16 smithi026 ceph-mon[36056]: pgmap v463: 129 pgs: 129 active+clean; 668 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:18 smithi002 ceph-mon[31557]: pgmap v464: 129 pgs: 129 active+clean; 668 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:18.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:18 smithi029 ceph-mon[36249]: pgmap v464: 129 pgs: 129 active+clean; 668 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:18 smithi026 ceph-mon[36056]: pgmap v464: 129 pgs: 129 active+clean; 668 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:20 smithi002 ceph-mon[31557]: pgmap v465: 129 pgs: 129 active+clean; 668 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T09:36:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:20 smithi029 ceph-mon[36249]: pgmap v465: 129 pgs: 129 active+clean; 668 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T09:36:20.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:20 smithi026 ceph-mon[36056]: pgmap v465: 129 pgs: 129 active+clean; 668 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T09:36:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.204+0000 7f3a210d9640 1 -- 172.21.15.2:0/2934877192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a1c1022b0 msgr2=0x7f3a1c102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.204+0000 7f3a210d9640 1 --2- 172.21.15.2:0/2934877192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a1c1022b0 0x7f3a1c102fb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00aa10 tx=0x7f3a0c02b720 comp rx=0 tx=0).stop 2024-05-14T09:36:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.204+0000 7f3a210d9640 1 -- 172.21.15.2:0/2934877192 shutdown_connections 2024-05-14T09:36:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.204+0000 7f3a210d9640 1 --2- 172.21.15.2:0/2934877192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a1c1081f0 0x7f3a1c10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.204+0000 7f3a210d9640 1 --2- 172.21.15.2:0/2934877192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a1c1034f0 0x7f3a1c103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.204+0000 7f3a210d9640 1 --2- 172.21.15.2:0/2934877192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a1c1022b0 0x7f3a1c102fb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.204+0000 7f3a210d9640 1 -- 172.21.15.2:0/2934877192 >> 172.21.15.2:0/2934877192 conn(0x7f3a1c0fd480 msgr2=0x7f3a1c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.204+0000 7f3a210d9640 1 -- 172.21.15.2:0/2934877192 shutdown_connections 2024-05-14T09:36:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.204+0000 7f3a210d9640 1 -- 172.21.15.2:0/2934877192 wait complete. 2024-05-14T09:36:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.205+0000 7f3a210d9640 1 Processor -- start 2024-05-14T09:36:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.205+0000 7f3a210d9640 1 -- start start 2024-05-14T09:36:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.206+0000 7f3a210d9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a1c1022b0 0x7f3a1c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.206+0000 7f3a210d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a1c1034f0 0x7f3a1c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.206+0000 7f3a210d9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a1c1081f0 0x7f3a1c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.206+0000 7f3a210d9640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c10d540 con 0x7f3a1c1022b0 2024-05-14T09:36:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.206+0000 7f3a210d9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c10d3c0 con 0x7f3a1c1034f0 2024-05-14T09:36:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.206+0000 7f3a210d9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c10d6c0 con 0x7f3a1c1081f0 2024-05-14T09:36:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.206+0000 7f3a1ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a1c1022b0 0x7f3a1c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.206+0000 7f3a1ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a1c1022b0 0x7f3a1c198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38260/0 (socket says 172.21.15.2:38260) 2024-05-14T09:36:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.206+0000 7f3a1ad9d640 1 -- 172.21.15.2:0/3715968044 learned_addr learned my addr 172.21.15.2:0/3715968044 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.206+0000 7f3a1259c640 1 --2- 172.21.15.2:0/3715968044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a1c1034f0 0x7f3a1c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.206+0000 7f3a1b59e640 1 --2- 172.21.15.2:0/3715968044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a1c1081f0 0x7f3a1c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.206+0000 7f3a1ad9d640 1 -- 172.21.15.2:0/3715968044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a1c1081f0 msgr2=0x7f3a1c19f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.206+0000 7f3a1ad9d640 1 --2- 172.21.15.2:0/3715968044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a1c1081f0 0x7f3a1c19f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.206+0000 7f3a1ad9d640 1 -- 172.21.15.2:0/3715968044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a1c1034f0 msgr2=0x7f3a1c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.206+0000 7f3a1ad9d640 1 --2- 172.21.15.2:0/3715968044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a1c1034f0 0x7f3a1c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.207+0000 7f3a1ad9d640 1 -- 172.21.15.2:0/3715968044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1c1a1dd0 con 0x7f3a1c1022b0 2024-05-14T09:36:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.207+0000 7f3a1ad9d640 1 --2- 172.21.15.2:0/3715968044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a1c1022b0 0x7f3a1c198080 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c006010 tx=0x7f3a0c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.207+0000 7f3a18d99640 1 -- 172.21.15.2:0/3715968044 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c03f070 con 0x7f3a1c1022b0 2024-05-14T09:36:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.207+0000 7f3a18d99640 1 -- 172.21.15.2:0/3715968044 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0c032400 con 0x7f3a1c1022b0 2024-05-14T09:36:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.207+0000 7f3a210d9640 1 -- 172.21.15.2:0/3715968044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1c1a2060 con 0x7f3a1c1022b0 2024-05-14T09:36:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.207+0000 7f3a18d99640 1 -- 172.21.15.2:0/3715968044 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c036040 con 0x7f3a1c1022b0 2024-05-14T09:36:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.207+0000 7f3a210d9640 1 -- 172.21.15.2:0/3715968044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a1c1a2550 con 0x7f3a1c1022b0 2024-05-14T09:36:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.210+0000 7f3a27f8c740 1 Processor -- start 2024-05-14T09:36:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.211+0000 7f3a27f8c740 1 -- start start 2024-05-14T09:36:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.211+0000 7f3a27f8c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd871b0 0x5574b0128a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.211+0000 7f3a27f8c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd5e2f0 0x5574b0128fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.211+0000 7f3a27f8c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd664d0 0x5574b012fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.211+0000 7f3a27f8c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5574b008d5d0 con 0x5574afd5e2f0 2024-05-14T09:36:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.211+0000 7f3a27f8c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5574b0046890 con 0x5574afd664d0 2024-05-14T09:36:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.211+0000 7f3a27f8c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5574b0089590 con 0x5574afd871b0 2024-05-14T09:36:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.211+0000 7f3a1b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd871b0 0x5574b0128a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.211+0000 7f3a1b59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd871b0 0x5574b0128a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:47682/0 (socket says 172.21.15.2:47682) 2024-05-14T09:36:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.211+0000 7f3a1b59e640 1 -- 172.21.15.2:0/1522775750 learned_addr learned my addr 172.21.15.2:0/1522775750 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.212+0000 7f3a1259c640 1 --2- 172.21.15.2:0/1522775750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd664d0 0x5574b012fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.212+0000 7f3a18d99640 1 -- 172.21.15.2:0/3715968044 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3a0c042070 con 0x7f3a1c1022b0 2024-05-14T09:36:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.212+0000 7f3a1b59e640 1 -- 172.21.15.2:0/1522775750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd664d0 msgr2=0x5574b012fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.212+0000 7f3a1b59e640 1 --2- 172.21.15.2:0/1522775750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd664d0 0x5574b012fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.212+0000 7f3a1b59e640 1 -- 172.21.15.2:0/1522775750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd5e2f0 msgr2=0x5574b0128fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:36:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.212+0000 7f3a1ad9d640 1 --2- 172.21.15.2:0/1522775750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd5e2f0 0x5574b0128fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.213+0000 7f3a1b59e640 1 --2- 172.21.15.2:0/1522775750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd5e2f0 0x5574b0128fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.213+0000 7f3a1b59e640 1 -- 172.21.15.2:0/1522775750 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574afd9f210 con 0x5574afd871b0 2024-05-14T09:36:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.213+0000 7f3a1ad9d640 1 --2- 172.21.15.2:0/1522775750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd5e2f0 0x5574b0128fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:36:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.213+0000 7f3a1b59e640 1 --2- 172.21.15.2:0/1522775750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd871b0 0x5574b0128a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3a140039f0 tx=0x7f3a14003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae3846c4a87dc232 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.213+0000 7f3a18d99640 1 --2- 172.21.15.2:0/3715968044 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3a080b28a0 0x7f3a080b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.213+0000 7f3a1ad9d640 1 --2- 172.21.15.2:0/3715968044 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3a080b28a0 0x7f3a080b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.213+0000 7f3a1159a640 1 -- 172.21.15.2:0/1522775750 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1401a470 con 0x5574afd871b0 2024-05-14T09:36:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.213+0000 7f3a1159a640 1 -- 172.21.15.2:0/1522775750 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a14015070 con 0x5574afd871b0 2024-05-14T09:36:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.213+0000 7f3a1159a640 1 -- 172.21.15.2:0/1522775750 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a140107e0 con 0x5574afd871b0 2024-05-14T09:36:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.213+0000 7f3a18d99640 1 -- 172.21.15.2:0/3715968044 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3a0c102de0 con 0x7f3a1c1022b0 2024-05-14T09:36:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.214+0000 7f3a1ad9d640 1 --2- 172.21.15.2:0/3715968044 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3a080b28a0 0x7f3a080b4db0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5574b0129e40 tx=0x7f3a0c0c5f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.214+0000 7f3a27f8c740 1 -- 172.21.15.2:0/1522775750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd871b0 msgr2=0x5574b0128a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.214+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/1522775750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd871b0 0x5574b0128a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3a140039f0 tx=0x7f3a14003eb0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.215+0000 7f3a27f8c740 1 -- 172.21.15.2:0/1522775750 shutdown_connections 2024-05-14T09:36:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.215+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/1522775750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd664d0 0x5574b012fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.215+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/1522775750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd5e2f0 0x5574b0128fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.215+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/1522775750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd871b0 0x5574b0128a60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.215+0000 7f3a27f8c740 1 -- 172.21.15.2:0/1522775750 >> 172.21.15.2:0/1522775750 conn(0x5574afd7b300 msgr2=0x5574b01243a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.215+0000 7f3a27f8c740 1 -- 172.21.15.2:0/1522775750 shutdown_connections 2024-05-14T09:36:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.215+0000 7f3a27f8c740 1 -- 172.21.15.2:0/1522775750 wait complete. 2024-05-14T09:36:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.216+0000 7f3a27f8c740 1 -- start start 2024-05-14T09:36:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.216+0000 7f3a27f8c740 1 Processor -- start 2024-05-14T09:36:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.216+0000 7f3a27f8c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 0x5574b01326b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.216+0000 7f3a27f8c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd664d0 0x5574b0132bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.217+0000 7f3a1259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd664d0 0x5574b0132bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.217+0000 7f3a1259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd664d0 0x5574b0132bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38282/0 (socket says 172.21.15.2:38282) 2024-05-14T09:36:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.217+0000 7f3a1259c640 1 -- 172.21.15.2:0/706824104 learned_addr learned my addr 172.21.15.2:0/706824104 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.217+0000 7f3a1b59e640 1 --2- 172.21.15.2:0/706824104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 0x5574b01326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.217+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/706824104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd871b0 0x5574b0139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.217+0000 7f3a27f8c740 1 -- 172.21.15.2:0/706824104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5574b0130930 con 0x5574afd664d0 2024-05-14T09:36:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.217+0000 7f3a27f8c740 1 -- 172.21.15.2:0/706824104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5574b004a370 con 0x5574afd5e2f0 2024-05-14T09:36:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.217+0000 7f3a27f8c740 1 -- 172.21.15.2:0/706824104 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5574b0130ab0 con 0x5574afd871b0 2024-05-14T09:36:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.217+0000 7f3a1259c640 1 -- 172.21.15.2:0/706824104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd871b0 msgr2=0x5574b0139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.217+0000 7f3a1259c640 1 --2- 172.21.15.2:0/706824104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd871b0 0x5574b0139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.217+0000 7f3a1259c640 1 -- 172.21.15.2:0/706824104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 msgr2=0x5574b01326b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.217+0000 7f3a1259c640 1 --2- 172.21.15.2:0/706824104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 0x5574b01326b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.217+0000 7f3a1259c640 1 -- 172.21.15.2:0/706824104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574afe6b620 con 0x5574afd664d0 2024-05-14T09:36:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.218+0000 7f3a1259c640 1 --2- 172.21.15.2:0/706824104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd664d0 0x5574b0132bf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400bad0 tx=0x7f3a0400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fae595c2021f574 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.218+0000 7f3a027fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:36:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.218+0000 7f3a027fc640 1 -- 172.21.15.2:0/706824104 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04003e20 con 0x5574afd664d0 2024-05-14T09:36:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.218+0000 7f3a27f8c740 10 client.0 _ll_register_callbacks cb 0x5574afd9ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.218+0000 7f3a027fc640 1 -- 172.21.15.2:0/706824104 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3a04005020 con 0x5574afd664d0 2024-05-14T09:36:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.218+0000 7f3a27f8c740 1 -- 172.21.15.2:0/706824104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a14015070 con 0x5574afd664d0 2024-05-14T09:36:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.218+0000 7f3a027fc640 1 -- 172.21.15.2:0/706824104 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3a04005220 con 0x5574afd664d0 2024-05-14T09:36:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.218+0000 7f3a27f8c740 10 client.15567 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.218+0000 7f3a27f8c740 1 -- 172.21.15.2:0/706824104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5574b013c8a0 con 0x5574afd664d0 2024-05-14T09:36:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.219+0000 7f3a027fc640 1 -- 172.21.15.2:0/706824104 <== mon.0 v2:172.21.15.2:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3a040053c0 con 0x5574afd664d0 2024-05-14T09:36:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.219+0000 7f3a27f8c740 10 client.15567 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.219+0000 7f3a27f8c740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.219+0000 7f3a27f8c740 20 client.15567 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.219+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/706824104 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5574afd45d80 0x5574b013ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.219+0000 7f3a27f8c740 4 client.15567 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:36:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.219+0000 7f3a27f8c740 1 -- 172.21.15.2:0/706824104 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574b009b1e0 con 0x5574afd45d80 2024-05-14T09:36:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.219+0000 7f3a1b59e640 1 --2- 172.21.15.2:0/706824104 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5574afd45d80 0x5574b013ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.220+0000 7f3a027fc640 10 client.15567 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.220+0000 7f3a1b59e640 1 --2- 172.21.15.2:0/706824104 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5574afd45d80 0x5574b013ffd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8516ad87135c9215 server_cookie=28c023d119defa83 in_seq=0 out_seq=0 2024-05-14T09:36:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.220+0000 7f3a027fc640 1 -- 172.21.15.2:0/706824104 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574b009b1e0 con 0x5574afd45d80 2024-05-14T09:36:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.220+0000 7f3a027fc640 10 client.15567 handle_command_reply: tid=0 2024-05-14T09:36:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.220+0000 7f3a27f8c740 1 client.15567 shutdown 2024-05-14T09:36:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.221+0000 7f3a27f8c740 1 -- 172.21.15.2:0/706824104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd664d0 msgr2=0x5574b0132bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.221+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/706824104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd664d0 0x5574b0132bf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400bad0 tx=0x7f3a0400bf90 comp rx=0 tx=0).stop 2024-05-14T09:36:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.221+0000 7f3a27f8c740 1 -- 172.21.15.2:0/706824104 shutdown_connections 2024-05-14T09:36:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.221+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/706824104 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5574afd45d80 0x5574b013ffd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.221+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/706824104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd871b0 0x5574b0139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.221+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/706824104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd664d0 0x5574b0132bf0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.221+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/706824104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 0x5574b01326b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.221+0000 7f3a27f8c740 1 -- 172.21.15.2:0/706824104 >> 172.21.15.2:0/706824104 conn(0x5574afd7b300 msgr2=0x5574b01233a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.221+0000 7f3a027fc640 0 client.15567 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.221+0000 7f3a27f8c740 1 -- 172.21.15.2:0/706824104 shutdown_connections 2024-05-14T09:36:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.221+0000 7f3a27f8c740 1 -- 172.21.15.2:0/706824104 wait complete. 2024-05-14T09:36:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.221+0000 7f3a27f8c740 20 client.15567 trim_cache size 0 max 16384 2024-05-14T09:36:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.252+0000 7f3a27f8c740 1 Processor -- start 2024-05-14T09:36:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.252+0000 7f3a27f8c740 1 -- start start 2024-05-14T09:36:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.252+0000 7f3a27f8c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 0x5574b0123cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.252+0000 7f3a27f8c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd664d0 0x5574b0124230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.253+0000 7f3a27f8c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd871b0 0x5574b0139a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.253+0000 7f3a1259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd664d0 0x5574b0124230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.253+0000 7f3a1259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd664d0 0x5574b0124230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38288/0 (socket says 172.21.15.2:38288) 2024-05-14T09:36:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.253+0000 7f3a1259c640 1 -- 172.21.15.2:0/653916813 learned_addr learned my addr 172.21.15.2:0/653916813 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.253+0000 7f3a1b59e640 1 --2- 172.21.15.2:0/653916813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 0x5574b0123cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.253+0000 7f3a27f8c740 1 -- 172.21.15.2:0/653916813 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c1af4c0 con 0x5574afd664d0 2024-05-14T09:36:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.253+0000 7f3a27f8c740 1 -- 172.21.15.2:0/653916813 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c1acd90 con 0x5574afd5e2f0 2024-05-14T09:36:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.253+0000 7f3a27f8c740 1 -- 172.21.15.2:0/653916813 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c1aabe0 con 0x5574afd871b0 2024-05-14T09:36:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.253+0000 7f3a1259c640 1 -- 172.21.15.2:0/653916813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd871b0 msgr2=0x5574b0139a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.253+0000 7f3a1259c640 1 --2- 172.21.15.2:0/653916813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd871b0 0x5574b0139a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.253+0000 7f3a1259c640 1 -- 172.21.15.2:0/653916813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 msgr2=0x5574b0123cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.253+0000 7f3a1259c640 1 --2- 172.21.15.2:0/653916813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 0x5574b0123cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.253+0000 7f3a1259c640 1 -- 172.21.15.2:0/653916813 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574b0124f80 con 0x5574afd664d0 2024-05-14T09:36:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.253+0000 7f3a1b59e640 1 --2- 172.21.15.2:0/653916813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 0x5574b0123cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.253+0000 7f3a1259c640 1 --2- 172.21.15.2:0/653916813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd664d0 0x5574b0124230 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3a040045e0 tx=0x7f3a04009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507fb22b1d16da99 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.254+0000 7f3a10d99640 1 -- 172.21.15.2:0/653916813 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04021040 con 0x5574afd664d0 2024-05-14T09:36:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.254+0000 7f3a10d99640 1 -- 172.21.15.2:0/653916813 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a14015070 con 0x5574afd664d0 2024-05-14T09:36:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.254+0000 7f3a10d99640 1 -- 172.21.15.2:0/653916813 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0401ba40 con 0x5574afd664d0 2024-05-14T09:36:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.255+0000 7f3a27f8c740 1 -- 172.21.15.2:0/653916813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd664d0 msgr2=0x5574b0124230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.255+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/653916813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd664d0 0x5574b0124230 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3a040045e0 tx=0x7f3a04009000 comp rx=0 tx=0).stop 2024-05-14T09:36:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.255+0000 7f3a27f8c740 1 -- 172.21.15.2:0/653916813 shutdown_connections 2024-05-14T09:36:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.255+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/653916813 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd871b0 0x5574b0139a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.255+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/653916813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd664d0 0x5574b0124230 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.255+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/653916813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 0x5574b0123cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.255+0000 7f3a27f8c740 1 -- 172.21.15.2:0/653916813 >> 172.21.15.2:0/653916813 conn(0x5574afd7b300 msgr2=0x5574b012cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.255+0000 7f3a27f8c740 1 -- 172.21.15.2:0/653916813 shutdown_connections 2024-05-14T09:36:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.255+0000 7f3a27f8c740 1 -- 172.21.15.2:0/653916813 wait complete. 2024-05-14T09:36:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.255+0000 7f3a27f8c740 1 -- start start 2024-05-14T09:36:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.255+0000 7f3a27f8c740 1 Processor -- start 2024-05-14T09:36:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a27f8c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 0x5574b0135490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a27f8c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd664d0 0x5574b01359d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a27f8c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd871b0 0x5574b0138910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a27f8c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c1ab250 con 0x5574afd871b0 2024-05-14T09:36:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a27f8c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c1ac6c0 con 0x5574afd5e2f0 2024-05-14T09:36:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a27f8c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c1ab920 con 0x5574afd664d0 2024-05-14T09:36:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a27f8c740 10 client.0 _ll_register_callbacks cb 0x7f3a04004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a1b59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 0x5574b0135490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a1b59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 0x5574b0135490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:33368/0 (socket says 172.21.15.2:33368) 2024-05-14T09:36:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a1b59e640 1 -- 172.21.15.2:0/2953147352 learned_addr learned my addr 172.21.15.2:0/2953147352 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a1b59e640 1 --2- 172.21.15.2:0/2953147352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd871b0 0x5574b0138910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a1259c640 1 --2- 172.21.15.2:0/2953147352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd664d0 0x5574b01359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a1b59e640 1 -- 172.21.15.2:0/2953147352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd664d0 msgr2=0x5574b01359d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a1b59e640 1 --2- 172.21.15.2:0/2953147352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd664d0 0x5574b01359d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a1b59e640 1 -- 172.21.15.2:0/2953147352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd871b0 msgr2=0x5574b0138910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a1b59e640 1 --2- 172.21.15.2:0/2953147352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd871b0 0x5574b0138910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.256+0000 7f3a1b59e640 1 -- 172.21.15.2:0/2953147352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574b0138ed0 con 0x5574afd5e2f0 2024-05-14T09:36:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.257+0000 7f3a1b59e640 1 --2- 172.21.15.2:0/2953147352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 0x5574b0135490 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3a14006010 tx=0x7f3a14003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=867aa187a01d5d27 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.257+0000 7f3a027fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:36:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.257+0000 7f3a027fc640 1 -- 172.21.15.2:0/2953147352 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a14011b60 con 0x5574afd5e2f0 2024-05-14T09:36:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.257+0000 7f3a27f8c740 1 -- 172.21.15.2:0/2953147352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a14015070 con 0x5574afd5e2f0 2024-05-14T09:36:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.257+0000 7f3a027fc640 1 -- 172.21.15.2:0/2953147352 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3a14005020 con 0x5574afd5e2f0 2024-05-14T09:36:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.257+0000 7f3a027fc640 1 -- 172.21.15.2:0/2953147352 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3a14005220 con 0x5574afd5e2f0 2024-05-14T09:36:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.257+0000 7f3a27f8c740 10 client.25274 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.257+0000 7f3a27f8c740 1 -- 172.21.15.2:0/2953147352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5574b013a1b0 con 0x5574afd5e2f0 2024-05-14T09:36:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.258+0000 7f3a027fc640 1 -- 172.21.15.2:0/2953147352 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a14015070 con 0x5574afd5e2f0 2024-05-14T09:36:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.258+0000 7f3a027fc640 1 -- 172.21.15.2:0/2953147352 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3a1401a470 con 0x5574afd5e2f0 2024-05-14T09:36:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.258+0000 7f3a27f8c740 10 client.25274 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.258+0000 7f3a27f8c740 10 client.25274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.258+0000 7f3a27f8c740 20 client.25274 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.258+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/2953147352 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5574afd1b1c0 0x5574b01aa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.258+0000 7f3a27f8c740 4 client.25274 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:36:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.258+0000 7f3a27f8c740 1 -- 172.21.15.2:0/2953147352 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574b0095050 con 0x5574afd1b1c0 2024-05-14T09:36:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.258+0000 7f3a1259c640 1 --2- 172.21.15.2:0/2953147352 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5574afd1b1c0 0x5574b01aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.259+0000 7f3a027fc640 10 client.25274 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.259+0000 7f3a1259c640 1 --2- 172.21.15.2:0/2953147352 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5574afd1b1c0 0x5574b01aa390 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c2bc3b0e9cc7f7d server_cookie=c415359f08953695 in_seq=0 out_seq=0 2024-05-14T09:36:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.263+0000 7f3a027fc640 1 -- 172.21.15.2:0/2953147352 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5574b0095050 con 0x5574afd1b1c0 2024-05-14T09:36:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.263+0000 7f3a027fc640 10 client.25274 handle_command_reply: tid=0 2024-05-14T09:36:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.263+0000 7f3a27f8c740 1 client.25274 shutdown 2024-05-14T09:36:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.263+0000 7f3a27f8c740 1 -- 172.21.15.2:0/2953147352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 msgr2=0x5574b0135490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.263+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/2953147352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 0x5574b0135490 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3a14006010 tx=0x7f3a14003eb0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.263+0000 7f3a27f8c740 1 -- 172.21.15.2:0/2953147352 shutdown_connections 2024-05-14T09:36:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.263+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/2953147352 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5574afd1b1c0 0x5574b01aa390 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.263+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/2953147352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574afd871b0 0x5574b0138910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.263+0000 7f3a027fc640 0 client.25274 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.263+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/2953147352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5574afd664d0 0x5574b01359d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.263+0000 7f3a27f8c740 1 --2- 172.21.15.2:0/2953147352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574afd5e2f0 0x5574b0135490 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.263+0000 7f3a27f8c740 1 -- 172.21.15.2:0/2953147352 >> 172.21.15.2:0/2953147352 conn(0x5574afd7b300 msgr2=0x5574b0123cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.263+0000 7f3a27f8c740 1 -- 172.21.15.2:0/2953147352 shutdown_connections 2024-05-14T09:36:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.263+0000 7f3a27f8c740 1 -- 172.21.15.2:0/2953147352 wait complete. 2024-05-14T09:36:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.263+0000 7f3a27f8c740 20 client.25274 trim_cache size 0 max 16384 2024-05-14T09:36:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:36:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:36:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:36:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:36:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.264+0000 7f3a210d9640 1 -- 172.21.15.2:0/3715968044 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3a080b28a0 msgr2=0x7f3a080b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.264+0000 7f3a210d9640 1 --2- 172.21.15.2:0/3715968044 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3a080b28a0 0x7f3a080b4db0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5574b0129e40 tx=0x7f3a0c0c5f50 comp rx=0 tx=0).stop 2024-05-14T09:36:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.264+0000 7f3a210d9640 1 -- 172.21.15.2:0/3715968044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a1c1022b0 msgr2=0x7f3a1c198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.264+0000 7f3a210d9640 1 --2- 172.21.15.2:0/3715968044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a1c1022b0 0x7f3a1c198080 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c006010 tx=0x7f3a0c0043b0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.264+0000 7f3a210d9640 1 -- 172.21.15.2:0/3715968044 shutdown_connections 2024-05-14T09:36:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.264+0000 7f3a210d9640 1 --2- 172.21.15.2:0/3715968044 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3a080b28a0 0x7f3a080b4db0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.264+0000 7f3a210d9640 1 --2- 172.21.15.2:0/3715968044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a1c1081f0 0x7f3a1c19f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.264+0000 7f3a210d9640 1 --2- 172.21.15.2:0/3715968044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a1c1034f0 0x7f3a1c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.264+0000 7f3a210d9640 1 --2- 172.21.15.2:0/3715968044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a1c1022b0 0x7f3a1c198080 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.264+0000 7f3a210d9640 1 -- 172.21.15.2:0/3715968044 >> 172.21.15.2:0/3715968044 conn(0x7f3a1c0fd480 msgr2=0x7f3a1c1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.265+0000 7f3a210d9640 1 -- 172.21.15.2:0/3715968044 shutdown_connections 2024-05-14T09:36:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:21.265+0000 7f3a210d9640 1 -- 172.21.15.2:0/3715968044 wait complete. 2024-05-14T09:36:21.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:21 smithi002 ceph-mon[31557]: pgmap v466: 129 pgs: 129 active+clean; 668 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:21.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:21 smithi029 ceph-mon[36249]: pgmap v466: 129 pgs: 129 active+clean; 668 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:21.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:21 smithi026 ceph-mon[36056]: pgmap v466: 129 pgs: 129 active+clean; 668 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:24.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:24 smithi026 ceph-mon[36056]: pgmap v467: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-14T09:36:24.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:24 smithi002 ceph-mon[31557]: pgmap v467: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-14T09:36:24.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:24 smithi029 ceph-mon[36249]: pgmap v467: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-14T09:36:26.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:26 smithi026 ceph-mon[36056]: pgmap v468: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:26.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:26 smithi002 ceph-mon[31557]: pgmap v468: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:26 smithi029 ceph-mon[36249]: pgmap v468: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:28.374 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T09:36:28.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:28 smithi026 ceph-mon[36056]: pgmap v469: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:28.472 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:28 smithi002 ceph-mon[31557]: pgmap v469: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:28.638 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:36:28.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:28 smithi029 ceph-mon[36249]: pgmap v469: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:30.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:30 smithi002 ceph-mon[31557]: pgmap v470: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T09:36:30.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:30 smithi026 ceph-mon[36056]: pgmap v470: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T09:36:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:30 smithi029 ceph-mon[36249]: pgmap v470: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T09:36:30.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.856+0000 7f5d1b33f700 1 -- 172.21.15.2:0/365021346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d14103080 msgr2=0x7f5d14103490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:30.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.856+0000 7f5d1b33f700 1 --2- 172.21.15.2:0/365021346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d14103080 0x7f5d14103490 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009990 tx=0x7f5d10009ca0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.856+0000 7f5d1b33f700 1 -- 172.21.15.2:0/365021346 shutdown_connections 2024-05-14T09:36:30.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.856+0000 7f5d1b33f700 1 --2- 172.21.15.2:0/365021346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d14110c30 0x7f5d14113050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.856+0000 7f5d1b33f700 1 --2- 172.21.15.2:0/365021346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d14103080 0x7f5d14103490 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.856+0000 7f5d1b33f700 1 --2- 172.21.15.2:0/365021346 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d14103a60 0x7f5d141106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.856+0000 7f5d1b33f700 1 -- 172.21.15.2:0/365021346 >> 172.21.15.2:0/365021346 conn(0x7f5d140fedc0 msgr2=0x7f5d141011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:30.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.856+0000 7f5d1b33f700 1 -- 172.21.15.2:0/365021346 shutdown_connections 2024-05-14T09:36:30.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.856+0000 7f5d1b33f700 1 -- 172.21.15.2:0/365021346 wait complete. 2024-05-14T09:36:30.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.857+0000 7f5d1b33f700 1 Processor -- start 2024-05-14T09:36:30.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.857+0000 7f5d1b33f700 1 -- start start 2024-05-14T09:36:30.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d1b33f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d14103080 0x7f5d1419d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d1b33f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d14103a60 0x7f5d1419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d1b33f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d14110c30 0x7f5d141a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d1b33f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14116680 con 0x7f5d14103080 2024-05-14T09:36:30.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d1b33f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14116500 con 0x7f5d14110c30 2024-05-14T09:36:30.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d1b33f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14116800 con 0x7f5d14103a60 2024-05-14T09:36:30.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d190db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d14103080 0x7f5d1419d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d190db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d14103080 0x7f5d1419d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55308/0 (socket says 172.21.15.2:55308) 2024-05-14T09:36:30.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d190db700 1 -- 172.21.15.2:0/994623047 learned_addr learned my addr 172.21.15.2:0/994623047 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:30.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d188da700 1 --2- 172.21.15.2:0/994623047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d14103a60 0x7f5d1419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d198dc700 1 --2- 172.21.15.2:0/994623047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d14110c30 0x7f5d141a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d190db700 1 -- 172.21.15.2:0/994623047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d14103a60 msgr2=0x7f5d1419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:30.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d190db700 1 --2- 172.21.15.2:0/994623047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d14103a60 0x7f5d1419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d190db700 1 -- 172.21.15.2:0/994623047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d14110c30 msgr2=0x7f5d141a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:30.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d190db700 1 --2- 172.21.15.2:0/994623047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d14110c30 0x7f5d141a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d190db700 1 -- 172.21.15.2:0/994623047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d141a1f60 con 0x7f5d14103080 2024-05-14T09:36:30.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d198dc700 1 --2- 172.21.15.2:0/994623047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d14110c30 0x7f5d141a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:36:30.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.858+0000 7f5d190db700 1 --2- 172.21.15.2:0/994623047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d14103080 0x7f5d1419d090 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5d100057c0 tx=0x7f5d100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:30.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.859+0000 7f5d0a7fc700 1 -- 172.21.15.2:0/994623047 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10027070 con 0x7f5d14103080 2024-05-14T09:36:30.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.859+0000 7f5d0a7fc700 1 -- 172.21.15.2:0/994623047 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d10003f20 con 0x7f5d14103080 2024-05-14T09:36:30.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.859+0000 7f5d1b33f700 1 -- 172.21.15.2:0/994623047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d141a21f0 con 0x7f5d14103080 2024-05-14T09:36:30.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.859+0000 7f5d0a7fc700 1 -- 172.21.15.2:0/994623047 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10022bc0 con 0x7f5d14103080 2024-05-14T09:36:30.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.859+0000 7f5d1b33f700 1 -- 172.21.15.2:0/994623047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d141a26d0 con 0x7f5d14103080 2024-05-14T09:36:30.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.861+0000 7f5d0a7fc700 1 -- 172.21.15.2:0/994623047 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5d1002a070 con 0x7f5d14103080 2024-05-14T09:36:30.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.862+0000 7f5d0a7fc700 1 --2- 172.21.15.2:0/994623047 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5d000b2880 0x7f5d000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.863+0000 7f5d188da700 1 --2- 172.21.15.2:0/994623047 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5d000b2880 0x7f5d000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.863+0000 7f5d29d53b80 1 Processor -- start 2024-05-14T09:36:30.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.863+0000 7f5d0a7fc700 1 -- 172.21.15.2:0/994623047 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f5d100e39a0 con 0x7f5d14103080 2024-05-14T09:36:30.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.863+0000 7f5d29d53b80 1 -- start start 2024-05-14T09:36:30.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.863+0000 7f5d29d53b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f736e40 0x56469f8bd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.863+0000 7f5d29d53b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f8bd8e0 0x56469f8c1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.863+0000 7f5d29d53b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8c2070 0x56469f8c2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.863+0000 7f5d29d53b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56469f7d6940 con 0x56469f8bd8e0 2024-05-14T09:36:30.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.863+0000 7f5d29d53b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56469f73bda0 con 0x56469f736e40 2024-05-14T09:36:30.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.863+0000 7f5d29d53b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56469f7d2900 con 0x56469f8c2070 2024-05-14T09:36:30.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.863+0000 7f5d188da700 1 --2- 172.21.15.2:0/994623047 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5d000b2880 0x7f5d000b4d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5d14112300 tx=0x7f5d0400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:30.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.863+0000 7f5d188da700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f8bd8e0 0x56469f8c1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.863+0000 7f5d188da700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f8bd8e0 0x56469f8c1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55310/0 (socket says 172.21.15.2:55310) 2024-05-14T09:36:30.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.863+0000 7f5d188da700 1 -- 172.21.15.2:0/3730301274 learned_addr learned my addr 172.21.15.2:0/3730301274 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:30.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.864+0000 7f5d198dc700 1 --2- 172.21.15.2:0/3730301274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8c2070 0x56469f8c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.864+0000 7f5d190db700 1 --2- 172.21.15.2:0/3730301274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f736e40 0x56469f8bd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.864+0000 7f5d198dc700 1 -- 172.21.15.2:0/3730301274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f736e40 msgr2=0x56469f8bd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:30.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.864+0000 7f5d198dc700 1 --2- 172.21.15.2:0/3730301274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f736e40 0x56469f8bd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.864+0000 7f5d198dc700 1 -- 172.21.15.2:0/3730301274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f8bd8e0 msgr2=0x56469f8c1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:30.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.864+0000 7f5d198dc700 1 --2- 172.21.15.2:0/3730301274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f8bd8e0 0x56469f8c1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.864+0000 7f5d198dc700 1 -- 172.21.15.2:0/3730301274 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56469f7238d0 con 0x56469f8c2070 2024-05-14T09:36:30.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.864+0000 7f5d188da700 1 --2- 172.21.15.2:0/3730301274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f8bd8e0 0x56469f8c1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:30.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.864+0000 7f5d198dc700 1 --2- 172.21.15.2:0/3730301274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8c2070 0x56469f8c2480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009d10 tx=0x7f5d0c0077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2d97101b7b21959 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:30.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.864+0000 7f5cff7fe700 1 -- 172.21.15.2:0/3730301274 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c004bb0 con 0x56469f8c2070 2024-05-14T09:36:30.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.864+0000 7f5cff7fe700 1 -- 172.21.15.2:0/3730301274 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d0c004d50 con 0x56469f8c2070 2024-05-14T09:36:30.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.864+0000 7f5cff7fe700 1 -- 172.21.15.2:0/3730301274 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c014410 con 0x56469f8c2070 2024-05-14T09:36:30.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.865+0000 7f5d29d53b80 1 -- 172.21.15.2:0/3730301274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8c2070 msgr2=0x56469f8c2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:30.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.865+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/3730301274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8c2070 0x56469f8c2480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009d10 tx=0x7f5d0c0077e0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.865+0000 7f5d29d53b80 1 -- 172.21.15.2:0/3730301274 shutdown_connections 2024-05-14T09:36:30.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.865+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/3730301274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f736e40 0x56469f8bd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.865+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/3730301274 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8c2070 0x56469f8c2480 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.865+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/3730301274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f8bd8e0 0x56469f8c1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.865+0000 7f5d29d53b80 1 -- 172.21.15.2:0/3730301274 >> 172.21.15.2:0/3730301274 conn(0x56469f77e630 msgr2=0x56469f8afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:30.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.866+0000 7f5d29d53b80 1 -- 172.21.15.2:0/3730301274 shutdown_connections 2024-05-14T09:36:30.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.866+0000 7f5d29d53b80 1 -- 172.21.15.2:0/3730301274 wait complete. 2024-05-14T09:36:30.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.866+0000 7f5d29d53b80 1 -- start start 2024-05-14T09:36:30.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.866+0000 7f5d29d53b80 1 Processor -- start 2024-05-14T09:36:30.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d29d53b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f736e40 0x56469f8c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d29d53b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8bd8e0 0x56469f8c7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d29d53b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f8c2070 0x56469f8c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d29d53b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56469f8ca020 con 0x56469f8c2070 2024-05-14T09:36:30.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d29d53b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56469f78fed0 con 0x56469f8bd8e0 2024-05-14T09:36:30.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d29d53b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56469f8ca1a0 con 0x56469f736e40 2024-05-14T09:36:30.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d29d53b80 10 client.0 _ll_register_callbacks cb 0x56469f76e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:30.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d198dc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f8c2070 0x56469f8c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d198dc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f8c2070 0x56469f8c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55326/0 (socket says 172.21.15.2:55326) 2024-05-14T09:36:30.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d198dc700 1 -- 172.21.15.2:0/2299672503 learned_addr learned my addr 172.21.15.2:0/2299672503 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:30.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d188da700 1 --2- 172.21.15.2:0/2299672503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8bd8e0 0x56469f8c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d190db700 1 --2- 172.21.15.2:0/2299672503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f736e40 0x56469f8c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d198dc700 1 -- 172.21.15.2:0/2299672503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f736e40 msgr2=0x56469f8c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:30.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d198dc700 1 --2- 172.21.15.2:0/2299672503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f736e40 0x56469f8c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d198dc700 1 -- 172.21.15.2:0/2299672503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8bd8e0 msgr2=0x56469f8c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:30.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d198dc700 1 --2- 172.21.15.2:0/2299672503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8bd8e0 0x56469f8c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d198dc700 1 -- 172.21.15.2:0/2299672503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56469f78d760 con 0x56469f8c2070 2024-05-14T09:36:30.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d188da700 1 --2- 172.21.15.2:0/2299672503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8bd8e0 0x56469f8c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:30.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5d198dc700 1 --2- 172.21.15.2:0/2299672503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f8c2070 0x56469f8c7780 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c005840 tx=0x7f5d0c006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc751ebf689ab6f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:30.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.867+0000 7f5cfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:36:30.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.868+0000 7f5cfcff9700 1 -- 172.21.15.2:0/2299672503 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c022070 con 0x56469f8c2070 2024-05-14T09:36:30.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.868+0000 7f5cfcff9700 1 -- 172.21.15.2:0/2299672503 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f5d0c0052c0 con 0x56469f8c2070 2024-05-14T09:36:30.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.868+0000 7f5d29d53b80 1 -- 172.21.15.2:0/2299672503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d0c004d50 con 0x56469f8c2070 2024-05-14T09:36:30.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.868+0000 7f5cfcff9700 1 -- 172.21.15.2:0/2299672503 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5d0c02e050 con 0x56469f8c2070 2024-05-14T09:36:30.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.868+0000 7f5d29d53b80 10 client.15585 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:30.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.868+0000 7f5d29d53b80 1 -- 172.21.15.2:0/2299672503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56469f8d4ba0 con 0x56469f8c2070 2024-05-14T09:36:30.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.868+0000 7f5cfcff9700 1 -- 172.21.15.2:0/2299672503 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d0c004d50 con 0x56469f8c2070 2024-05-14T09:36:30.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.868+0000 7f5cfcff9700 1 -- 172.21.15.2:0/2299672503 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5d0c0144e0 con 0x56469f8c2070 2024-05-14T09:36:30.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.868+0000 7f5d29d53b80 10 client.15585 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:30.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.868+0000 7f5d29d53b80 10 client.15585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:30.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.868+0000 7f5d29d53b80 20 client.15585 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:30.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.868+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/2299672503 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56469f8d7070 0x56469f8d9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.869+0000 7f5d29d53b80 4 client.15585 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:36:30.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.869+0000 7f5d29d53b80 1 -- 172.21.15.2:0/2299672503 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56469f7e4320 con 0x56469f8d7070 2024-05-14T09:36:30.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.869+0000 7f5d190db700 1 --2- 172.21.15.2:0/2299672503 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56469f8d7070 0x56469f8d9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.869+0000 7f5cfcff9700 10 client.15585 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:30.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.869+0000 7f5d190db700 1 --2- 172.21.15.2:0/2299672503 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56469f8d7070 0x56469f8d9490 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=238a4f1e8c782212 server_cookie=6269ef58a81b2646 in_seq=0 out_seq=0 2024-05-14T09:36:30.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.870+0000 7f5cfcff9700 1 -- 172.21.15.2:0/2299672503 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56469f7e4320 con 0x56469f8d7070 2024-05-14T09:36:30.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.870+0000 7f5cfcff9700 10 client.15585 handle_command_reply: tid=0 2024-05-14T09:36:30.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.870+0000 7f5d29d53b80 1 client.15585 shutdown 2024-05-14T09:36:30.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.870+0000 7f5d29d53b80 1 -- 172.21.15.2:0/2299672503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f8c2070 msgr2=0x56469f8c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:30.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.870+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/2299672503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f8c2070 0x56469f8c7780 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c005840 tx=0x7f5d0c006000 comp rx=0 tx=0).stop 2024-05-14T09:36:30.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.870+0000 7f5d29d53b80 1 -- 172.21.15.2:0/2299672503 shutdown_connections 2024-05-14T09:36:30.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.870+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/2299672503 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56469f8d7070 0x56469f8d9490 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.870+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/2299672503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f8c2070 0x56469f8c7780 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.870+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/2299672503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f736e40 0x56469f8c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.870+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/2299672503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8bd8e0 0x56469f8c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.870+0000 7f5cfcff9700 0 client.15585 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:30.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.870+0000 7f5d29d53b80 1 -- 172.21.15.2:0/2299672503 >> 172.21.15.2:0/2299672503 conn(0x56469f77e630 msgr2=0x56469f8b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:30.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.871+0000 7f5d29d53b80 1 -- 172.21.15.2:0/2299672503 shutdown_connections 2024-05-14T09:36:30.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.871+0000 7f5d29d53b80 1 -- 172.21.15.2:0/2299672503 wait complete. 2024-05-14T09:36:30.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.871+0000 7f5d29d53b80 20 client.15585 trim_cache size 0 max 16384 2024-05-14T09:36:30.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.891+0000 7f5d29d53b80 1 Processor -- start 2024-05-14T09:36:30.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.891+0000 7f5d29d53b80 1 -- start start 2024-05-14T09:36:30.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d29d53b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f736e40 0x56469f8d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d29d53b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8bd8e0 0x56469f8c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d29d53b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8c2070 0x56469f8c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d29d53b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141b0c20 con 0x56469f736e40 2024-05-14T09:36:30.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d29d53b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141ae4b0 con 0x56469f8c2070 2024-05-14T09:36:30.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d29d53b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141ac320 con 0x56469f8bd8e0 2024-05-14T09:36:30.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d190db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f736e40 0x56469f8d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d190db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f736e40 0x56469f8d3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55332/0 (socket says 172.21.15.2:55332) 2024-05-14T09:36:30.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d190db700 1 -- 172.21.15.2:0/131346280 learned_addr learned my addr 172.21.15.2:0/131346280 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:30.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d198dc700 1 --2- 172.21.15.2:0/131346280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8c2070 0x56469f8c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d188da700 1 --2- 172.21.15.2:0/131346280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8bd8e0 0x56469f8c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d190db700 1 -- 172.21.15.2:0/131346280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8bd8e0 msgr2=0x56469f8c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:30.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d190db700 1 --2- 172.21.15.2:0/131346280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8bd8e0 0x56469f8c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d190db700 1 -- 172.21.15.2:0/131346280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8c2070 msgr2=0x56469f8c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:30.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d190db700 1 --2- 172.21.15.2:0/131346280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8c2070 0x56469f8c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d190db700 1 -- 172.21.15.2:0/131346280 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56469f8b11e0 con 0x56469f736e40 2024-05-14T09:36:30.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d198dc700 1 --2- 172.21.15.2:0/131346280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8c2070 0x56469f8c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:36:30.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.892+0000 7f5d190db700 1 --2- 172.21.15.2:0/131346280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f736e40 0x56469f8d3440 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5d100aabc0 tx=0x7f5d100cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff3cc24fdf0541f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:30.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.893+0000 7f5cfeffd700 1 -- 172.21.15.2:0/131346280 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d100b1040 con 0x56469f736e40 2024-05-14T09:36:30.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.893+0000 7f5cfeffd700 1 -- 172.21.15.2:0/131346280 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d100cbe50 con 0x56469f736e40 2024-05-14T09:36:30.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.893+0000 7f5cfeffd700 1 -- 172.21.15.2:0/131346280 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10022470 con 0x56469f736e40 2024-05-14T09:36:30.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.894+0000 7f5d29d53b80 1 -- 172.21.15.2:0/131346280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f736e40 msgr2=0x56469f8d3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:30.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.894+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/131346280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f736e40 0x56469f8d3440 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5d100aabc0 tx=0x7f5d100cd040 comp rx=0 tx=0).stop 2024-05-14T09:36:30.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.894+0000 7f5d29d53b80 1 -- 172.21.15.2:0/131346280 shutdown_connections 2024-05-14T09:36:30.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.894+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/131346280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8c2070 0x56469f8c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.894+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/131346280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f736e40 0x56469f8d3440 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.894+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/131346280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8bd8e0 0x56469f8c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.894+0000 7f5d29d53b80 1 -- 172.21.15.2:0/131346280 >> 172.21.15.2:0/131346280 conn(0x56469f77e630 msgr2=0x56469f8c54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:30.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.894+0000 7f5d29d53b80 1 -- 172.21.15.2:0/131346280 shutdown_connections 2024-05-14T09:36:30.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.894+0000 7f5d29d53b80 1 -- 172.21.15.2:0/131346280 wait complete. 2024-05-14T09:36:30.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.894+0000 7f5d29d53b80 1 -- start start 2024-05-14T09:36:30.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.894+0000 7f5d29d53b80 1 Processor -- start 2024-05-14T09:36:30.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d29d53b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f736e40 0x56469f938bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d29d53b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8bd8e0 0x56469f9390f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d29d53b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8c2070 0x56469f8db840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d29d53b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141ac970 con 0x56469f736e40 2024-05-14T09:36:30.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d29d53b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141adde0 con 0x56469f8c2070 2024-05-14T09:36:30.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d29d53b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141ad040 con 0x56469f8bd8e0 2024-05-14T09:36:30.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d29d53b80 10 client.0 _ll_register_callbacks cb 0x56469f76e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:30.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d190db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f736e40 0x56469f938bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d190db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f736e40 0x56469f938bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55342/0 (socket says 172.21.15.2:55342) 2024-05-14T09:36:30.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d190db700 1 -- 172.21.15.2:0/1374485189 learned_addr learned my addr 172.21.15.2:0/1374485189 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:30.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d198dc700 1 --2- 172.21.15.2:0/1374485189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8c2070 0x56469f8db840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d188da700 1 --2- 172.21.15.2:0/1374485189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8bd8e0 0x56469f9390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d190db700 1 -- 172.21.15.2:0/1374485189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8bd8e0 msgr2=0x56469f9390f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:30.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d190db700 1 --2- 172.21.15.2:0/1374485189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8bd8e0 0x56469f9390f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d190db700 1 -- 172.21.15.2:0/1374485189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8c2070 msgr2=0x56469f8db840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:30.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d190db700 1 --2- 172.21.15.2:0/1374485189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8c2070 0x56469f8db840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d190db700 1 -- 172.21.15.2:0/1374485189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56469f8dbe50 con 0x56469f736e40 2024-05-14T09:36:30.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.895+0000 7f5d198dc700 1 --2- 172.21.15.2:0/1374485189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8c2070 0x56469f8db840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:36:30.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.896+0000 7f5d188da700 1 --2- 172.21.15.2:0/1374485189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8bd8e0 0x56469f9390f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:36:30.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.896+0000 7f5d190db700 1 --2- 172.21.15.2:0/1374485189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f736e40 0x56469f938bb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5d100e13f0 tx=0x7f5d1000b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2f76f92f9cded8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:30.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.896+0000 7f5cfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:36:30.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.896+0000 7f5cfcff9700 1 -- 172.21.15.2:0/1374485189 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1006f060 con 0x56469f736e40 2024-05-14T09:36:30.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.896+0000 7f5cfcff9700 1 -- 172.21.15.2:0/1374485189 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f5d10022470 con 0x56469f736e40 2024-05-14T09:36:30.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.896+0000 7f5d29d53b80 1 -- 172.21.15.2:0/1374485189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d100cbe50 con 0x56469f736e40 2024-05-14T09:36:30.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.896+0000 7f5cfcff9700 1 -- 172.21.15.2:0/1374485189 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5d100b1040 con 0x56469f736e40 2024-05-14T09:36:30.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.896+0000 7f5d29d53b80 10 client.15591 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:30.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.896+0000 7f5d29d53b80 1 -- 172.21.15.2:0/1374485189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56469f941380 con 0x56469f736e40 2024-05-14T09:36:30.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.896+0000 7f5cfcff9700 1 -- 172.21.15.2:0/1374485189 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d100cbe50 con 0x56469f736e40 2024-05-14T09:36:30.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.896+0000 7f5cfcff9700 1 -- 172.21.15.2:0/1374485189 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5d100b3570 con 0x56469f736e40 2024-05-14T09:36:30.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.897+0000 7f5d29d53b80 10 client.15591 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:30.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.897+0000 7f5d29d53b80 10 client.15591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:30.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.897+0000 7f5d29d53b80 20 client.15591 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:30.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.897+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/1374485189 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56469f941990 0x56469f943db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:30.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.897+0000 7f5d29d53b80 4 client.15591 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T09:36:30.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.897+0000 7f5d29d53b80 1 -- 172.21.15.2:0/1374485189 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56469f7de3c0 con 0x56469f941990 2024-05-14T09:36:30.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.897+0000 7f5d188da700 1 --2- 172.21.15.2:0/1374485189 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56469f941990 0x56469f943db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.897+0000 7f5d188da700 1 --2- 172.21.15.2:0/1374485189 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56469f941990 0x56469f943db0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=458eb245a07a6689 server_cookie=a4bf19fee98d753e in_seq=0 out_seq=0 2024-05-14T09:36:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.897+0000 7f5cfcff9700 10 client.15591 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.898+0000 7f5cfcff9700 1 -- 172.21.15.2:0/1374485189 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56469f7de3c0 con 0x56469f941990 2024-05-14T09:36:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.898+0000 7f5cfcff9700 10 client.15591 handle_command_reply: tid=0 2024-05-14T09:36:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.898+0000 7f5d29d53b80 1 client.15591 shutdown 2024-05-14T09:36:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.898+0000 7f5d29d53b80 1 -- 172.21.15.2:0/1374485189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f736e40 msgr2=0x56469f938bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.898+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/1374485189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f736e40 0x56469f938bb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5d100e13f0 tx=0x7f5d1000b540 comp rx=0 tx=0).stop 2024-05-14T09:36:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.898+0000 7f5d29d53b80 1 -- 172.21.15.2:0/1374485189 shutdown_connections 2024-05-14T09:36:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.898+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/1374485189 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56469f941990 0x56469f943db0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.898+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/1374485189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56469f8c2070 0x56469f8db840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.898+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/1374485189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56469f736e40 0x56469f938bb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.898+0000 7f5d29d53b80 1 --2- 172.21.15.2:0/1374485189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56469f8bd8e0 0x56469f9390f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.898+0000 7f5d29d53b80 1 -- 172.21.15.2:0/1374485189 >> 172.21.15.2:0/1374485189 conn(0x56469f77e630 msgr2=0x56469f8a9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:30.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.898+0000 7f5cfcff9700 0 client.15591 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:30.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.898+0000 7f5d29d53b80 1 -- 172.21.15.2:0/1374485189 shutdown_connections 2024-05-14T09:36:30.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.898+0000 7f5d29d53b80 1 -- 172.21.15.2:0/1374485189 wait complete. 2024-05-14T09:36:30.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.898+0000 7f5d29d53b80 20 client.15591 trim_cache size 0 max 16384 2024-05-14T09:36:30.903 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:36:30.903 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-05-14T09:36:30.903 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-05-14T09:36:30.903 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:36:30.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.899+0000 7f5d1b33f700 1 -- 172.21.15.2:0/994623047 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5d000b2880 msgr2=0x7f5d000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:30.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.899+0000 7f5d1b33f700 1 --2- 172.21.15.2:0/994623047 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5d000b2880 0x7f5d000b4d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5d14112300 tx=0x7f5d0400a380 comp rx=0 tx=0).stop 2024-05-14T09:36:30.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.899+0000 7f5d1b33f700 1 -- 172.21.15.2:0/994623047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d14103080 msgr2=0x7f5d1419d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:30.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.899+0000 7f5d1b33f700 1 --2- 172.21.15.2:0/994623047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d14103080 0x7f5d1419d090 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5d100057c0 tx=0x7f5d100057f0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.899+0000 7f5d1b33f700 1 -- 172.21.15.2:0/994623047 shutdown_connections 2024-05-14T09:36:30.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.899+0000 7f5d1b33f700 1 --2- 172.21.15.2:0/994623047 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5d000b2880 0x7f5d000b4d90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.899+0000 7f5d1b33f700 1 --2- 172.21.15.2:0/994623047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d14110c30 0x7f5d141a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.899+0000 7f5d1b33f700 1 --2- 172.21.15.2:0/994623047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d14103080 0x7f5d1419d090 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.899+0000 7f5d1b33f700 1 --2- 172.21.15.2:0/994623047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5d14103a60 0x7f5d1419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.899+0000 7f5d1b33f700 1 -- 172.21.15.2:0/994623047 >> 172.21.15.2:0/994623047 conn(0x7f5d140fedc0 msgr2=0x7f5d14112900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.899+0000 7f5d1b33f700 1 -- 172.21.15.2:0/994623047 shutdown_connections 2024-05-14T09:36:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:30.899+0000 7f5d1b33f700 1 -- 172.21.15.2:0/994623047 wait complete. 2024-05-14T09:36:30.968 INFO:tasks.workunit.client.0.smithi002.stderr:Updating files: 89% (11031/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-14T09:36:31.771 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T09:36:31.772 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:36:31.884 INFO:tasks.workunit.client.0.smithi002.stderr:Note: switching to 'e2b755a64f0d72176b57299d02e880cf0620d780'. 2024-05-14T09:36:31.884 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-05-14T09:36:31.884 INFO:tasks.workunit.client.0.smithi002.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-14T09:36:31.884 INFO:tasks.workunit.client.0.smithi002.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-14T09:36:31.884 INFO:tasks.workunit.client.0.smithi002.stderr:state without impacting any branches by switching back to a branch. 2024-05-14T09:36:31.884 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-05-14T09:36:31.884 INFO:tasks.workunit.client.0.smithi002.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-14T09:36:31.884 INFO:tasks.workunit.client.0.smithi002.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-14T09:36:31.884 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-05-14T09:36:31.884 INFO:tasks.workunit.client.0.smithi002.stderr: git switch -c 2024-05-14T09:36:31.884 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-05-14T09:36:31.884 INFO:tasks.workunit.client.0.smithi002.stderr:Or undo this operation with: 2024-05-14T09:36:31.885 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-05-14T09:36:31.885 INFO:tasks.workunit.client.0.smithi002.stderr: git switch - 2024-05-14T09:36:31.885 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-05-14T09:36:31.885 INFO:tasks.workunit.client.0.smithi002.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-14T09:36:31.885 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-05-14T09:36:31.885 INFO:tasks.workunit.client.0.smithi002.stderr:HEAD is now at e2b755a64f0 Merge PR #53755 into wip-vshankar-testing-20240509.053109-debug 2024-05-14T09:36:31.890 DEBUG:teuthology.orchestra.run.smithi002:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-14T09:36:31.916 INFO:tasks.workunit.client.0.smithi002.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-14T09:36:31.919 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-14T09:36:31.919 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-14T09:36:32.028 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:36:32.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:32 smithi002 ceph-mon[31557]: pgmap v471: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:32.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:32 smithi026 ceph-mon[36056]: pgmap v471: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:32.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:32 smithi029 ceph-mon[36249]: pgmap v471: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T09:36:33.834 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-14T09:36:33.893 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-14T09:36:33.935 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-14T09:36:33.937 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-14T09:36:33.937 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-14T09:36:33.992 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-14T09:36:33.995 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T09:36:33.995 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-14T09:36:34.052 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-05-14T09:36:34.054 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-05-14T09:36:34.054 DEBUG:teuthology.orchestra.run.smithi002:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e2b755a64f0d72176b57299d02e880cf0620d780 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-05-14T09:36:34.117 INFO:tasks.workunit.client.0.smithi002.stderr:+ dbench 1 2024-05-14T09:36:34.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.126+0000 7ffada87f700 1 -- 172.21.15.2:0/2662112626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffad4109870 msgr2=0x7ffad41111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:34.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.126+0000 7ffada87f700 1 --2- 172.21.15.2:0/2662112626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffad4109870 0x7ffad41111d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00b0d0 tx=0x7ffacc00b3e0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.126+0000 7ffada87f700 1 -- 172.21.15.2:0/2662112626 shutdown_connections 2024-05-14T09:36:34.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.126+0000 7ffada87f700 1 --2- 172.21.15.2:0/2662112626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffad4109870 0x7ffad41111d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.126+0000 7ffada87f700 1 --2- 172.21.15.2:0/2662112626 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffad41045f0 0x7ffad4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.126+0000 7ffada87f700 1 --2- 172.21.15.2:0/2662112626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffad4108f20 0x7ffad4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.126+0000 7ffada87f700 1 -- 172.21.15.2:0/2662112626 >> 172.21.15.2:0/2662112626 conn(0x7ffad4100040 msgr2=0x7ffad4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:34.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.126+0000 7ffada87f700 1 -- 172.21.15.2:0/2662112626 shutdown_connections 2024-05-14T09:36:34.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.126+0000 7ffada87f700 1 -- 172.21.15.2:0/2662112626 wait complete. 2024-05-14T09:36:34.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.126+0000 7ffada87f700 1 Processor -- start 2024-05-14T09:36:34.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.126+0000 7ffada87f700 1 -- start start 2024-05-14T09:36:34.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffada87f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffad41045f0 0x7ffad4075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffada87f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffad4108f20 0x7ffad4076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffada87f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffad4109870 0x7ffad4076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffada87f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffad4114520 con 0x7ffad41045f0 2024-05-14T09:36:34.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffada87f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffad41143a0 con 0x7ffad4108f20 2024-05-14T09:36:34.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffada87f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffad41146a0 con 0x7ffad4109870 2024-05-14T09:36:34.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffad3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffad41045f0 0x7ffad4075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffad3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffad41045f0 0x7ffad4075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55352/0 (socket says 172.21.15.2:55352) 2024-05-14T09:36:34.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffad3fff700 1 -- 172.21.15.2:0/2001761358 learned_addr learned my addr 172.21.15.2:0/2001761358 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:34.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffad8e1c700 1 --2- 172.21.15.2:0/2001761358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffad4109870 0x7ffad4076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffad37fe700 1 --2- 172.21.15.2:0/2001761358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffad4108f20 0x7ffad4076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffad3fff700 1 -- 172.21.15.2:0/2001761358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffad4109870 msgr2=0x7ffad4076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:34.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffad3fff700 1 --2- 172.21.15.2:0/2001761358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffad4109870 0x7ffad4076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffad3fff700 1 -- 172.21.15.2:0/2001761358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffad4108f20 msgr2=0x7ffad4076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:34.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffad3fff700 1 --2- 172.21.15.2:0/2001761358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffad4108f20 0x7ffad4076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.127+0000 7ffad3fff700 1 -- 172.21.15.2:0/2001761358 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad41a91b0 con 0x7ffad41045f0 2024-05-14T09:36:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.128+0000 7ffad37fe700 1 --2- 172.21.15.2:0/2001761358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffad4108f20 0x7ffad4076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:36:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.128+0000 7ffad3fff700 1 --2- 172.21.15.2:0/2001761358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffad41045f0 0x7ffad4075ac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ffac400b530 tx=0x7ffac400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.128+0000 7ffad17fa700 1 -- 172.21.15.2:0/2001761358 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4015020 con 0x7ffad41045f0 2024-05-14T09:36:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.128+0000 7ffada87f700 1 -- 172.21.15.2:0/2001761358 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad41a94a0 con 0x7ffad41045f0 2024-05-14T09:36:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.128+0000 7ffad17fa700 1 -- 172.21.15.2:0/2001761358 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac4011ce0 con 0x7ffad41045f0 2024-05-14T09:36:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.128+0000 7ffad17fa700 1 -- 172.21.15.2:0/2001761358 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4010e40 con 0x7ffad41045f0 2024-05-14T09:36:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.128+0000 7ffada87f700 1 -- 172.21.15.2:0/2001761358 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad41a99e0 con 0x7ffad41045f0 2024-05-14T09:36:34.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.129+0000 7ffada87f700 1 -- 172.21.15.2:0/2001761358 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad40512c0 con 0x7ffad41045f0 2024-05-14T09:36:34.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.131+0000 7ffad17fa700 1 -- 172.21.15.2:0/2001761358 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ffac4018020 con 0x7ffad41045f0 2024-05-14T09:36:34.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.131+0000 7ffad17fa700 1 --2- 172.21.15.2:0/2001761358 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffabc0b2880 0x7ffabc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.131+0000 7ffad37fe700 1 --2- 172.21.15.2:0/2001761358 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffabc0b2880 0x7ffabc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.132+0000 7ffad17fa700 1 -- 172.21.15.2:0/2001761358 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7ffac40e23f0 con 0x7ffad41045f0 2024-05-14T09:36:34.132 INFO:tasks.workunit.client.0.smithi002.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-14T09:36:34.132 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-05-14T09:36:34.132 INFO:tasks.workunit.client.0.smithi002.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-14T09:36:34.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.132+0000 7ffad37fe700 1 --2- 172.21.15.2:0/2001761358 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffabc0b2880 0x7ffabc0b4d90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffac8005950 tx=0x7ffac80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:34.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.135+0000 7ffad17fa700 1 -- 172.21.15.2:0/2001761358 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffac401f020 con 0x7ffad41045f0 2024-05-14T09:36:34.140 INFO:tasks.workunit.client.0.smithi002.stdout:0 of 1 processes prepared for launch 0 sec 2024-05-14T09:36:34.241 INFO:tasks.workunit.client.0.smithi002.stdout:1 of 1 processes prepared for launch 0 sec 2024-05-14T09:36:34.241 INFO:tasks.workunit.client.0.smithi002.stdout:releasing clients 2024-05-14T09:36:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.370+0000 7f31982c6640 1 -- 172.21.15.2:0/2987153281 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3190103d90 msgr2=0x7f31901041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.370+0000 7f31982c6640 1 --2- 172.21.15.2:0/2987153281 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3190103d90 0x7f31901041a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3180009900 tx=0x7f318002a410 comp rx=0 tx=0).stop 2024-05-14T09:36:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.370+0000 7f31982c6640 1 -- 172.21.15.2:0/2987153281 shutdown_connections 2024-05-14T09:36:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.370+0000 7f31982c6640 1 --2- 172.21.15.2:0/2987153281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3190108a90 0x7f319010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.370+0000 7f31982c6640 1 --2- 172.21.15.2:0/2987153281 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3190103d90 0x7f31901041a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.370+0000 7f31982c6640 1 --2- 172.21.15.2:0/2987153281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3190102b50 0x7f3190103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.370+0000 7f31982c6640 1 -- 172.21.15.2:0/2987153281 >> 172.21.15.2:0/2987153281 conn(0x7f31900fdd40 msgr2=0x7f31901001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.370+0000 7f31982c6640 1 -- 172.21.15.2:0/2987153281 shutdown_connections 2024-05-14T09:36:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.370+0000 7f31982c6640 1 -- 172.21.15.2:0/2987153281 wait complete. 2024-05-14T09:36:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.370+0000 7f31982c6640 1 Processor -- start 2024-05-14T09:36:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.370+0000 7f31982c6640 1 -- start start 2024-05-14T09:36:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f31982c6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3190102b50 0x7f319019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f31982c6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3190103d90 0x7f319019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f31982c6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3190108a90 0x7f31901a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f31982c6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f319010f780 con 0x7f3190102b50 2024-05-14T09:36:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f31982c6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f319010f600 con 0x7f3190103d90 2024-05-14T09:36:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f31982c6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f319010f900 con 0x7f3190108a90 2024-05-14T09:36:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f3196062640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3190102b50 0x7f319019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f3196062640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3190102b50 0x7f319019a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55370/0 (socket says 172.21.15.2:55370) 2024-05-14T09:36:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f3196062640 1 -- 172.21.15.2:0/2724494822 learned_addr learned my addr 172.21.15.2:0/2724494822 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f3195861640 1 --2- 172.21.15.2:0/2724494822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3190103d90 0x7f319019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f3196062640 1 -- 172.21.15.2:0/2724494822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3190108a90 msgr2=0x7f31901a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f3196062640 1 --2- 172.21.15.2:0/2724494822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3190108a90 0x7f31901a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f3196062640 1 -- 172.21.15.2:0/2724494822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3190103d90 msgr2=0x7f319019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f3196062640 1 --2- 172.21.15.2:0/2724494822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3190103d90 0x7f319019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f3196062640 1 -- 172.21.15.2:0/2724494822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31901a1df0 con 0x7f3190102b50 2024-05-14T09:36:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f3196062640 1 --2- 172.21.15.2:0/2724494822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3190102b50 0x7f319019a320 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f318c00e9f0 tx=0x7f318c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.371+0000 7f31877fe640 1 -- 172.21.15.2:0/2724494822 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c00ccc0 con 0x7f3190102b50 2024-05-14T09:36:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.372+0000 7f31982c6640 1 -- 172.21.15.2:0/2724494822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31901a20e0 con 0x7f3190102b50 2024-05-14T09:36:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.372+0000 7f31877fe640 1 -- 172.21.15.2:0/2724494822 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f318c0035f0 con 0x7f3190102b50 2024-05-14T09:36:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.372+0000 7f31877fe640 1 -- 172.21.15.2:0/2724494822 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c00ce60 con 0x7f3190102b50 2024-05-14T09:36:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.372+0000 7f31982c6640 1 -- 172.21.15.2:0/2724494822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31901a2630 con 0x7f3190102b50 2024-05-14T09:36:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.375+0000 7f319f179740 1 Processor -- start 2024-05-14T09:36:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.375+0000 7f319f179740 1 -- start start 2024-05-14T09:36:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.375+0000 7f31877fe640 1 -- 172.21.15.2:0/2724494822 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f318c003aa0 con 0x7f3190102b50 2024-05-14T09:36:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.375+0000 7f319f179740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58991b0 0x55dbd5c3ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.375+0000 7f319f179740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58702f0 0x55dbd5c3f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.375+0000 7f319f179740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58784d0 0x55dbd5c462f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.375+0000 7f319f179740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55dbd5b9f5d0 con 0x55dbd58702f0 2024-05-14T09:36:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.375+0000 7f319f179740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dbd5b58890 con 0x55dbd58991b0 2024-05-14T09:36:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.375+0000 7f319f179740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55dbd5b9b590 con 0x55dbd58784d0 2024-05-14T09:36:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.375+0000 7f3196062640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58702f0 0x55dbd5c3f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.375+0000 7f3196863640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58991b0 0x55dbd5c3ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.375+0000 7f3196062640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58702f0 0x55dbd5c3f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55378/0 (socket says 172.21.15.2:55378) 2024-05-14T09:36:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.375+0000 7f3196062640 1 -- 172.21.15.2:0/2263436136 learned_addr learned my addr 172.21.15.2:0/2263436136 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.375+0000 7f3195861640 1 --2- 172.21.15.2:0/2263436136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58784d0 0x55dbd5c462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.376+0000 7f3196062640 1 -- 172.21.15.2:0/2263436136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58784d0 msgr2=0x55dbd5c462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.376+0000 7f3196062640 1 --2- 172.21.15.2:0/2263436136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58784d0 0x55dbd5c462f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.376+0000 7f3196062640 1 -- 172.21.15.2:0/2263436136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58991b0 msgr2=0x55dbd5c3ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.376+0000 7f3196062640 1 --2- 172.21.15.2:0/2263436136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58991b0 0x55dbd5c3ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.376+0000 7f3196062640 1 -- 172.21.15.2:0/2263436136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbd58b1210 con 0x55dbd58702f0 2024-05-14T09:36:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.376+0000 7f3196863640 1 --2- 172.21.15.2:0/2263436136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58991b0 0x55dbd5c3ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.376+0000 7f3196062640 1 --2- 172.21.15.2:0/2263436136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58702f0 0x55dbd5c3f3f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f318c0e6a30 tx=0x7f318c0e6b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3857e2fb465822 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.376+0000 7f31877fe640 1 --2- 172.21.15.2:0/2724494822 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f317c0b28a0 0x7f317c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.376+0000 7f3196863640 1 --2- 172.21.15.2:0/2724494822 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f317c0b28a0 0x7f317c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.376+0000 7f3184bfe640 1 -- 172.21.15.2:0/2263436136 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c0f6070 con 0x55dbd58702f0 2024-05-14T09:36:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.376+0000 7f31877fe640 1 -- 172.21.15.2:0/2724494822 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f318c0e2a20 con 0x7f3190102b50 2024-05-14T09:36:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.376+0000 7f3184bfe640 1 -- 172.21.15.2:0/2263436136 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f318c0a9730 con 0x55dbd58702f0 2024-05-14T09:36:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.376+0000 7f3184bfe640 1 -- 172.21.15.2:0/2263436136 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c0a5ae0 con 0x55dbd58702f0 2024-05-14T09:36:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.377+0000 7f319f179740 1 -- 172.21.15.2:0/2263436136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58702f0 msgr2=0x55dbd5c3f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.377+0000 7f319f179740 1 --2- 172.21.15.2:0/2263436136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58702f0 0x55dbd5c3f3f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f318c0e6a30 tx=0x7f318c0e6b20 comp rx=0 tx=0).stop 2024-05-14T09:36:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.377+0000 7f3196863640 1 --2- 172.21.15.2:0/2724494822 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f317c0b28a0 0x7f317c0b4db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55dbd5c40040 tx=0x7f3188009200 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.377+0000 7f319f179740 1 -- 172.21.15.2:0/2263436136 shutdown_connections 2024-05-14T09:36:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.377+0000 7f319f179740 1 --2- 172.21.15.2:0/2263436136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58784d0 0x55dbd5c462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.377+0000 7f319f179740 1 --2- 172.21.15.2:0/2263436136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58702f0 0x55dbd5c3f3f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.377+0000 7f319f179740 1 --2- 172.21.15.2:0/2263436136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58991b0 0x55dbd5c3ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.377+0000 7f319f179740 1 -- 172.21.15.2:0/2263436136 >> 172.21.15.2:0/2263436136 conn(0x55dbd588d300 msgr2=0x55dbd5c363a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.377+0000 7f319f179740 1 -- 172.21.15.2:0/2263436136 shutdown_connections 2024-05-14T09:36:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.377+0000 7f319f179740 1 -- 172.21.15.2:0/2263436136 wait complete. 2024-05-14T09:36:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.378+0000 7f319f179740 1 -- start start 2024-05-14T09:36:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.378+0000 7f319f179740 1 Processor -- start 2024-05-14T09:36:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.378+0000 7f319f179740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58702f0 0x55dbd5c48a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.378+0000 7f319f179740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58784d0 0x55dbd5c48f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.378+0000 7f319f179740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c4fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.378+0000 7f319f179740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55dbd5c46d40 con 0x55dbd58991b0 2024-05-14T09:36:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.378+0000 7f319f179740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dbd5b5c370 con 0x55dbd58784d0 2024-05-14T09:36:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.378+0000 7f319f179740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55dbd5c46ec0 con 0x55dbd58702f0 2024-05-14T09:36:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.378+0000 7f319f179740 10 client.0 _ll_register_callbacks cb 0x55dbd58b0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.378+0000 7f3195861640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58784d0 0x55dbd5c48f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.378+0000 7f3196863640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c4fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.378+0000 7f3196062640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58702f0 0x55dbd5c48a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.378+0000 7f3195861640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58784d0 0x55dbd5c48f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:44534/0 (socket says 172.21.15.2:44534) 2024-05-14T09:36:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.378+0000 7f3195861640 1 -- 172.21.15.2:0/3218009629 learned_addr learned my addr 172.21.15.2:0/3218009629 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.379+0000 7f3195861640 1 -- 172.21.15.2:0/3218009629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58702f0 msgr2=0x55dbd5c48a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.379+0000 7f3195861640 1 --2- 172.21.15.2:0/3218009629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58702f0 0x55dbd5c48a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.379+0000 7f3195861640 1 -- 172.21.15.2:0/3218009629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 msgr2=0x55dbd5c4fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.379+0000 7f3195861640 1 --2- 172.21.15.2:0/3218009629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c4fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.379+0000 7f3195861640 1 -- 172.21.15.2:0/3218009629 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbd597d620 con 0x55dbd58784d0 2024-05-14T09:36:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.379+0000 7f3196863640 1 --2- 172.21.15.2:0/3218009629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c4fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.379+0000 7f3195861640 1 --2- 172.21.15.2:0/3218009629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58784d0 0x55dbd5c48f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f318002a930 tx=0x7f3180031e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=884e67ad05b9d65e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.379+0000 7f3179ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:36:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.379+0000 7f3179ffb640 1 -- 172.21.15.2:0/3218009629 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180035040 con 0x55dbd58784d0 2024-05-14T09:36:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.379+0000 7f319f179740 1 -- 172.21.15.2:0/3218009629 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f318c0a9730 con 0x55dbd58784d0 2024-05-14T09:36:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.379+0000 7f3179ffb640 1 -- 172.21.15.2:0/3218009629 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3180004040 con 0x55dbd58784d0 2024-05-14T09:36:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.379+0000 7f3179ffb640 1 -- 172.21.15.2:0/3218009629 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f318004f050 con 0x55dbd58784d0 2024-05-14T09:36:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.379+0000 7f319f179740 10 client.25307 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.379+0000 7f319f179740 1 -- 172.21.15.2:0/3218009629 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dbd5c52eb0 con 0x55dbd58784d0 2024-05-14T09:36:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.380+0000 7f3179ffb640 1 -- 172.21.15.2:0/3218009629 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f318c0a9730 con 0x55dbd58784d0 2024-05-14T09:36:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.380+0000 7f3179ffb640 1 -- 172.21.15.2:0/3218009629 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3180044020 con 0x55dbd58784d0 2024-05-14T09:36:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.380+0000 7f319f179740 10 client.25307 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.380+0000 7f319f179740 10 client.25307 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.380+0000 7f319f179740 20 client.25307 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.380+0000 7f319f179740 1 --2- 172.21.15.2:0/3218009629 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbd5857d80 0x55dbd5c565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.380+0000 7f319f179740 4 client.25307 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:36:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.380+0000 7f319f179740 1 -- 172.21.15.2:0/3218009629 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbd5bad1e0 con 0x55dbd5857d80 2024-05-14T09:36:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.380+0000 7f3196062640 1 --2- 172.21.15.2:0/3218009629 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbd5857d80 0x55dbd5c565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.380+0000 7f3179ffb640 10 client.25307 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.380+0000 7f3196062640 1 --2- 172.21.15.2:0/3218009629 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbd5857d80 0x55dbd5c565c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b4cd62595e1f5b server_cookie=8fbbf54c3add2af2 in_seq=0 out_seq=0 2024-05-14T09:36:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.382+0000 7f3179ffb640 1 -- 172.21.15.2:0/3218009629 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dbd5bad1e0 con 0x55dbd5857d80 2024-05-14T09:36:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.382+0000 7f3179ffb640 10 client.25307 handle_command_reply: tid=0 2024-05-14T09:36:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.382+0000 7f319f179740 1 client.25307 shutdown 2024-05-14T09:36:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.382+0000 7f319f179740 1 -- 172.21.15.2:0/3218009629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58784d0 msgr2=0x55dbd5c48f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.382+0000 7f319f179740 1 --2- 172.21.15.2:0/3218009629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58784d0 0x55dbd5c48f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f318002a930 tx=0x7f3180031e80 comp rx=0 tx=0).stop 2024-05-14T09:36:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.382+0000 7f319f179740 1 -- 172.21.15.2:0/3218009629 shutdown_connections 2024-05-14T09:36:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.382+0000 7f319f179740 1 --2- 172.21.15.2:0/3218009629 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbd5857d80 0x55dbd5c565c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.382+0000 7f319f179740 1 --2- 172.21.15.2:0/3218009629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c4fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.382+0000 7f319f179740 1 --2- 172.21.15.2:0/3218009629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58784d0 0x55dbd5c48f60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.382+0000 7f319f179740 1 --2- 172.21.15.2:0/3218009629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58702f0 0x55dbd5c48a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.382+0000 7f3179ffb640 0 client.25307 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.382+0000 7f319f179740 1 -- 172.21.15.2:0/3218009629 >> 172.21.15.2:0/3218009629 conn(0x55dbd588d300 msgr2=0x55dbd5c39300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.382+0000 7f319f179740 1 -- 172.21.15.2:0/3218009629 shutdown_connections 2024-05-14T09:36:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.382+0000 7f319f179740 1 -- 172.21.15.2:0/3218009629 wait complete. 2024-05-14T09:36:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.382+0000 7f319f179740 20 client.25307 trim_cache size 0 max 16384 2024-05-14T09:36:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.398+0000 7f319f179740 1 Processor -- start 2024-05-14T09:36:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f319f179740 1 -- start start 2024-05-14T09:36:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f319f179740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58702f0 0x55dbd5c36230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f319f179740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58784d0 0x55dbd5c4fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f319f179740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c50320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f319f179740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f31901ad2a0 con 0x55dbd58991b0 2024-05-14T09:36:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f319f179740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f31901aab70 con 0x55dbd58702f0 2024-05-14T09:36:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f319f179740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f31901a89c0 con 0x55dbd58784d0 2024-05-14T09:36:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f3196863640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c50320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f3196062640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58702f0 0x55dbd5c36230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f3196863640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c50320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55402/0 (socket says 172.21.15.2:55402) 2024-05-14T09:36:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f3196863640 1 -- 172.21.15.2:0/2653443253 learned_addr learned my addr 172.21.15.2:0/2653443253 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f3195861640 1 --2- 172.21.15.2:0/2653443253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58784d0 0x55dbd5c4fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f3196863640 1 -- 172.21.15.2:0/2653443253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58784d0 msgr2=0x55dbd5c4fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f3196863640 1 --2- 172.21.15.2:0/2653443253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58784d0 0x55dbd5c4fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f3196863640 1 -- 172.21.15.2:0/2653443253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58702f0 msgr2=0x55dbd5c36230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f3196863640 1 --2- 172.21.15.2:0/2653443253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58702f0 0x55dbd5c36230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.399+0000 7f3196863640 1 -- 172.21.15.2:0/2653443253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbd5c304d0 con 0x55dbd58991b0 2024-05-14T09:36:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.400+0000 7f3196863640 1 --2- 172.21.15.2:0/2653443253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c50320 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3188009c50 tx=0x7f3188018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62af080c97b86695 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.400+0000 7f317bfff640 1 -- 172.21.15.2:0/2653443253 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188004d20 con 0x55dbd58991b0 2024-05-14T09:36:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.400+0000 7f317bfff640 1 -- 172.21.15.2:0/2653443253 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f318801be30 con 0x55dbd58991b0 2024-05-14T09:36:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.400+0000 7f317bfff640 1 -- 172.21.15.2:0/2653443253 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188005950 con 0x55dbd58991b0 2024-05-14T09:36:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 -- 172.21.15.2:0/2653443253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 msgr2=0x55dbd5c50320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 --2- 172.21.15.2:0/2653443253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c50320 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3188009c50 tx=0x7f3188018000 comp rx=0 tx=0).stop 2024-05-14T09:36:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 -- 172.21.15.2:0/2653443253 shutdown_connections 2024-05-14T09:36:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 --2- 172.21.15.2:0/2653443253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c50320 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 --2- 172.21.15.2:0/2653443253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58784d0 0x55dbd5c4fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 --2- 172.21.15.2:0/2653443253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58702f0 0x55dbd5c36230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 -- 172.21.15.2:0/2653443253 >> 172.21.15.2:0/2653443253 conn(0x55dbd588d300 msgr2=0x55dbd5c35b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 -- 172.21.15.2:0/2653443253 shutdown_connections 2024-05-14T09:36:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 -- 172.21.15.2:0/2653443253 wait complete. 2024-05-14T09:36:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 -- start start 2024-05-14T09:36:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 Processor -- start 2024-05-14T09:36:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58702f0 0x55dbd5c4d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58784d0 0x55dbd5c4de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c4ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f31901a9030 con 0x55dbd58991b0 2024-05-14T09:36:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f31901aa4a0 con 0x55dbd58784d0 2024-05-14T09:36:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f31901a9700 con 0x55dbd58702f0 2024-05-14T09:36:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.401+0000 7f319f179740 10 client.0 _ll_register_callbacks cb 0x7f3180009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f3196863640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c4ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f3196863640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c4ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55418/0 (socket says 172.21.15.2:55418) 2024-05-14T09:36:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f3196863640 1 -- 172.21.15.2:0/1188092849 learned_addr learned my addr 172.21.15.2:0/1188092849 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f3195861640 1 --2- 172.21.15.2:0/1188092849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58784d0 0x55dbd5c4de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f3196062640 1 --2- 172.21.15.2:0/1188092849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58702f0 0x55dbd5c4d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f3196863640 1 -- 172.21.15.2:0/1188092849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58702f0 msgr2=0x55dbd5c4d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f3196863640 1 --2- 172.21.15.2:0/1188092849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58702f0 0x55dbd5c4d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f3196863640 1 -- 172.21.15.2:0/1188092849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58784d0 msgr2=0x55dbd5c4de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f3196863640 1 --2- 172.21.15.2:0/1188092849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58784d0 0x55dbd5c4de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f3196863640 1 -- 172.21.15.2:0/1188092849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbd5c4f2f0 con 0x55dbd58991b0 2024-05-14T09:36:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f3196863640 1 --2- 172.21.15.2:0/1188092849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c4ed30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f318801bdd0 tx=0x7f3188009270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935bb42085b11597 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f3179ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:36:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f3179ffb640 1 -- 172.21.15.2:0/1188092849 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188034020 con 0x55dbd58991b0 2024-05-14T09:36:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f3179ffb640 1 -- 172.21.15.2:0/1188092849 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3188004b20 con 0x55dbd58991b0 2024-05-14T09:36:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f319f179740 1 -- 172.21.15.2:0/1188092849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f318801be30 con 0x55dbd58991b0 2024-05-14T09:36:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.402+0000 7f3179ffb640 1 -- 172.21.15.2:0/1188092849 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f318803f050 con 0x55dbd58991b0 2024-05-14T09:36:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.403+0000 7f319f179740 10 client.15615 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.403+0000 7f319f179740 1 -- 172.21.15.2:0/1188092849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dbd5c50580 con 0x55dbd58991b0 2024-05-14T09:36:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.403+0000 7f3179ffb640 1 -- 172.21.15.2:0/1188092849 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f318801be30 con 0x55dbd58991b0 2024-05-14T09:36:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.403+0000 7f3179ffb640 1 -- 172.21.15.2:0/1188092849 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3188026070 con 0x55dbd58991b0 2024-05-14T09:36:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.403+0000 7f319f179740 10 client.15615 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.403+0000 7f319f179740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.403+0000 7f319f179740 20 client.15615 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.403+0000 7f319f179740 1 --2- 172.21.15.2:0/1188092849 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbd582d1c0 0x55dbd5cc0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.403+0000 7f319f179740 4 client.15615 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:36:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.403+0000 7f319f179740 1 -- 172.21.15.2:0/1188092849 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dbd5ba7050 con 0x55dbd582d1c0 2024-05-14T09:36:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.403+0000 7f3196062640 1 --2- 172.21.15.2:0/1188092849 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbd582d1c0 0x55dbd5cc0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.404+0000 7f3179ffb640 10 client.15615 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.404+0000 7f3196062640 1 --2- 172.21.15.2:0/1188092849 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbd582d1c0 0x55dbd5cc0d70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c764159059fa48a3 server_cookie=3794132f569f968a in_seq=0 out_seq=0 2024-05-14T09:36:34.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.412+0000 7ffada87f700 1 -- 172.21.15.2:0/2001761358 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffad4076ff0 con 0x7ffad41045f0 2024-05-14T09:36:34.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:34 smithi002 ceph-mon[31557]: pgmap v472: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T09:36:34.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.413+0000 7ffad17fa700 1 -- 172.21.15.2:0/2001761358 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11843 (secure 0 0 0) 0x7ffac40a9cf0 con 0x7ffad41045f0 2024-05-14T09:36:34.413 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:36:34.413 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T09:34:49.632659+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24487,24497,24524]},"id":1}]} 2024-05-14T09:36:34.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.416+0000 7ffada87f700 1 -- 172.21.15.2:0/2001761358 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffabc0b2880 msgr2=0x7ffabc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:34.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.416+0000 7ffada87f700 1 --2- 172.21.15.2:0/2001761358 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffabc0b2880 0x7ffabc0b4d90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffac8005950 tx=0x7ffac80058e0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.416+0000 7ffada87f700 1 -- 172.21.15.2:0/2001761358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffad41045f0 msgr2=0x7ffad4075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:34.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.416+0000 7ffada87f700 1 --2- 172.21.15.2:0/2001761358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffad41045f0 0x7ffad4075ac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ffac400b530 tx=0x7ffac400b840 comp rx=0 tx=0).stop 2024-05-14T09:36:34.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.416+0000 7ffada87f700 1 -- 172.21.15.2:0/2001761358 shutdown_connections 2024-05-14T09:36:34.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.416+0000 7ffada87f700 1 --2- 172.21.15.2:0/2001761358 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffabc0b2880 0x7ffabc0b4d90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.416+0000 7ffada87f700 1 --2- 172.21.15.2:0/2001761358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffad4109870 0x7ffad4076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.416+0000 7ffada87f700 1 --2- 172.21.15.2:0/2001761358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffad41045f0 0x7ffad4075ac0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.416+0000 7ffada87f700 1 --2- 172.21.15.2:0/2001761358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffad4108f20 0x7ffad4076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.416+0000 7ffada87f700 1 -- 172.21.15.2:0/2001761358 >> 172.21.15.2:0/2001761358 conn(0x7ffad4100040 msgr2=0x7ffad4108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:34.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.416+0000 7ffada87f700 1 -- 172.21.15.2:0/2001761358 shutdown_connections 2024-05-14T09:36:34.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:34.416+0000 7ffada87f700 1 -- 172.21.15.2:0/2001761358 wait complete. 2024-05-14T09:36:34.417 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 27 2024-05-14T09:36:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.446+0000 7f3179ffb640 1 -- 172.21.15.2:0/1188092849 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dbd5ba7050 con 0x55dbd582d1c0 2024-05-14T09:36:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.446+0000 7f3179ffb640 10 client.15615 handle_command_reply: tid=0 2024-05-14T09:36:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.446+0000 7f319f179740 1 client.15615 shutdown 2024-05-14T09:36:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.446+0000 7f319f179740 1 -- 172.21.15.2:0/1188092849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 msgr2=0x55dbd5c4ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.446+0000 7f319f179740 1 --2- 172.21.15.2:0/1188092849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c4ed30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f318801bdd0 tx=0x7f3188009270 comp rx=0 tx=0).stop 2024-05-14T09:36:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.446+0000 7f319f179740 1 -- 172.21.15.2:0/1188092849 shutdown_connections 2024-05-14T09:36:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.446+0000 7f319f179740 1 --2- 172.21.15.2:0/1188092849 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbd582d1c0 0x55dbd5cc0d70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.446+0000 7f319f179740 1 --2- 172.21.15.2:0/1188092849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbd58991b0 0x55dbd5c4ed30 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.446+0000 7f3179ffb640 0 client.15615 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.446+0000 7f319f179740 1 --2- 172.21.15.2:0/1188092849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbd58784d0 0x55dbd5c4de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.446+0000 7f319f179740 1 --2- 172.21.15.2:0/1188092849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbd58702f0 0x55dbd5c4d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.446+0000 7f319f179740 1 -- 172.21.15.2:0/1188092849 >> 172.21.15.2:0/1188092849 conn(0x55dbd588d300 msgr2=0x55dbd5c35a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.446+0000 7f319f179740 1 -- 172.21.15.2:0/1188092849 shutdown_connections 2024-05-14T09:36:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.446+0000 7f319f179740 1 -- 172.21.15.2:0/1188092849 wait complete. 2024-05-14T09:36:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.446+0000 7f319f179740 20 client.15615 trim_cache size 0 max 16384 2024-05-14T09:36:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:36:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:36:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:36:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:36:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.447+0000 7f31982c6640 1 -- 172.21.15.2:0/2724494822 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f317c0b28a0 msgr2=0x7f317c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.447+0000 7f31982c6640 1 --2- 172.21.15.2:0/2724494822 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f317c0b28a0 0x7f317c0b4db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55dbd5c40040 tx=0x7f3188009200 comp rx=0 tx=0).stop 2024-05-14T09:36:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.447+0000 7f31982c6640 1 -- 172.21.15.2:0/2724494822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3190102b50 msgr2=0x7f319019a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.447+0000 7f31982c6640 1 --2- 172.21.15.2:0/2724494822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3190102b50 0x7f319019a320 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f318c00e9f0 tx=0x7f318c00eeb0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.447+0000 7f31982c6640 1 -- 172.21.15.2:0/2724494822 shutdown_connections 2024-05-14T09:36:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.447+0000 7f31982c6640 1 --2- 172.21.15.2:0/2724494822 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f317c0b28a0 0x7f317c0b4db0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.447+0000 7f31982c6640 1 --2- 172.21.15.2:0/2724494822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3190108a90 0x7f31901a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.447+0000 7f31982c6640 1 --2- 172.21.15.2:0/2724494822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3190103d90 0x7f319019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.447+0000 7f31982c6640 1 --2- 172.21.15.2:0/2724494822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3190102b50 0x7f319019a320 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.447+0000 7f31982c6640 1 -- 172.21.15.2:0/2724494822 >> 172.21.15.2:0/2724494822 conn(0x7f31900fdd40 msgr2=0x7f3190104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.447+0000 7f31982c6640 1 -- 172.21.15.2:0/2724494822 shutdown_connections 2024-05-14T09:36:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:34.447+0000 7f31982c6640 1 -- 172.21.15.2:0/2724494822 wait complete. 2024-05-14T09:36:34.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:34 smithi029 ceph-mon[36249]: pgmap v472: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T09:36:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:34 smithi026 ceph-mon[36056]: pgmap v472: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T09:36:35.203 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T09:36:35.241 INFO:tasks.workunit.client.0.smithi002.stdout: 1 3224 34.33 MB/sec warmup 1 sec latency 20.470 ms 2024-05-14T09:36:35.495 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:36:35.618 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:35 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2001761358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:36:35.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:35 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2001761358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:36:35.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:35 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2001761358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:36:36.241 INFO:tasks.workunit.client.0.smithi002.stdout: 1 6154 24.06 MB/sec warmup 2 sec latency 15.270 ms 2024-05-14T09:36:36.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:36 smithi002 ceph-mon[31557]: pgmap v473: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:36:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:36 smithi029 ceph-mon[36249]: pgmap v473: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:36:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:36 smithi026 ceph-mon[36056]: pgmap v473: 129 pgs: 129 active+clean; 669 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T09:36:37.252 INFO:tasks.workunit.client.0.smithi002.stdout: 1 8910 21.44 MB/sec warmup 3 sec latency 17.707 ms 2024-05-14T09:36:37.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.426+0000 7ff4fc7f8700 1 -- 172.21.15.2:0/283908957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f4106ed0 msgr2=0x7ff4f4113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:37.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.426+0000 7ff4fc7f8700 1 --2- 172.21.15.2:0/283908957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f4106ed0 0x7ff4f4113360 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009990 tx=0x7ff4ec009ca0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.427+0000 7ff4fc7f8700 1 -- 172.21.15.2:0/283908957 shutdown_connections 2024-05-14T09:36:37.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.427+0000 7ff4fc7f8700 1 --2- 172.21.15.2:0/283908957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f410c5c0 0x7ff4f410c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.427+0000 7ff4fc7f8700 1 --2- 172.21.15.2:0/283908957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f4106ed0 0x7ff4f4113360 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.427+0000 7ff4fc7f8700 1 --2- 172.21.15.2:0/283908957 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f4106580 0x7ff4f4106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.427+0000 7ff4fc7f8700 1 -- 172.21.15.2:0/283908957 >> 172.21.15.2:0/283908957 conn(0x7ff4f4102080 msgr2=0x7ff4f41044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:37.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.427+0000 7ff4fc7f8700 1 -- 172.21.15.2:0/283908957 shutdown_connections 2024-05-14T09:36:37.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.427+0000 7ff4fc7f8700 1 -- 172.21.15.2:0/283908957 wait complete. 2024-05-14T09:36:37.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.427+0000 7ff4fc7f8700 1 Processor -- start 2024-05-14T09:36:37.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.427+0000 7ff4fc7f8700 1 -- start start 2024-05-14T09:36:37.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4fc7f8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f4106580 0x7ff4f419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4fc7f8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f4106ed0 0x7ff4f419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4fc7f8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f410c5c0 0x7ff4f41a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4fc7f8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41166f0 con 0x7ff4f4106ed0 2024-05-14T09:36:37.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4fc7f8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4116570 con 0x7ff4f4106580 2024-05-14T09:36:37.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4fc7f8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4116870 con 0x7ff4f410c5c0 2024-05-14T09:36:37.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4f9d93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f4106ed0 0x7ff4f419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4f9d93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f4106ed0 0x7ff4f419d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55432/0 (socket says 172.21.15.2:55432) 2024-05-14T09:36:37.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4f9d93700 1 -- 172.21.15.2:0/370302448 learned_addr learned my addr 172.21.15.2:0/370302448 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:37.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4fa594700 1 --2- 172.21.15.2:0/370302448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f4106580 0x7ff4f419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4fad95700 1 --2- 172.21.15.2:0/370302448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f410c5c0 0x7ff4f41a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4f9d93700 1 -- 172.21.15.2:0/370302448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f410c5c0 msgr2=0x7ff4f41a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:37.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4f9d93700 1 --2- 172.21.15.2:0/370302448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f410c5c0 0x7ff4f41a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4f9d93700 1 -- 172.21.15.2:0/370302448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f4106580 msgr2=0x7ff4f419cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:37.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4f9d93700 1 --2- 172.21.15.2:0/370302448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f4106580 0x7ff4f419cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4f9d93700 1 -- 172.21.15.2:0/370302448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f41a1e60 con 0x7ff4f4106ed0 2024-05-14T09:36:37.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4fa594700 1 --2- 172.21.15.2:0/370302448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f4106580 0x7ff4f419cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:37.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.428+0000 7ff4f9d93700 1 --2- 172.21.15.2:0/370302448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f4106ed0 0x7ff4f419d580 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800e940 tx=0x7ff4e800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:37.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.429+0000 7ff4f37fe700 1 -- 172.21.15.2:0/370302448 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e800c9e0 con 0x7ff4f4106ed0 2024-05-14T09:36:37.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.429+0000 7ff4f37fe700 1 -- 172.21.15.2:0/370302448 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e800cb80 con 0x7ff4f4106ed0 2024-05-14T09:36:37.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.429+0000 7ff4fc7f8700 1 -- 172.21.15.2:0/370302448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f41a2150 con 0x7ff4f4106ed0 2024-05-14T09:36:37.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.429+0000 7ff4f37fe700 1 -- 172.21.15.2:0/370302448 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8010e60 con 0x7ff4f4106ed0 2024-05-14T09:36:37.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.429+0000 7ff4fc7f8700 1 -- 172.21.15.2:0/370302448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f41a9a70 con 0x7ff4f4106ed0 2024-05-14T09:36:37.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.432+0000 7ff4f37fe700 1 -- 172.21.15.2:0/370302448 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff4e8018680 con 0x7ff4f4106ed0 2024-05-14T09:36:37.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.433+0000 7ff50b20cb80 1 Processor -- start 2024-05-14T09:36:37.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.433+0000 7ff4f37fe700 1 --2- 172.21.15.2:0/370302448 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff4dc0b2880 0x7ff4dc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.433+0000 7ff4fad95700 1 --2- 172.21.15.2:0/370302448 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff4dc0b2880 0x7ff4dc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.433+0000 7ff50b20cb80 1 -- start start 2024-05-14T09:36:37.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff4f37fe700 1 -- 172.21.15.2:0/370302448 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7ff4e80e28d0 con 0x7ff4f4106ed0 2024-05-14T09:36:37.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff50b20cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59c92e40 0x55fd59e10720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff50b20cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59e10c60 0x55fd59e14f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff50b20cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e15880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff50b20cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fd59d32860 con 0x55fd59e10c60 2024-05-14T09:36:37.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff50b20cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fd59cd17c0 con 0x55fd59e15470 2024-05-14T09:36:37.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff50b20cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fd59d2e820 con 0x55fd59c92e40 2024-05-14T09:36:37.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff4fad95700 1 --2- 172.21.15.2:0/370302448 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff4dc0b2880 0x7ff4dc0b4d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00b2e0 tx=0x7ff4ec005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:37.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff4f9d93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59e10c60 0x55fd59e14f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff4f9d93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59e10c60 0x55fd59e14f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55438/0 (socket says 172.21.15.2:55438) 2024-05-14T09:36:37.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff4f9d93700 1 -- 172.21.15.2:0/778410855 learned_addr learned my addr 172.21.15.2:0/778410855 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:37.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff4fad95700 1 --2- 172.21.15.2:0/778410855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e15880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff4fa594700 1 --2- 172.21.15.2:0/778410855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59c92e40 0x55fd59e10720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff4f9d93700 1 -- 172.21.15.2:0/778410855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59c92e40 msgr2=0x55fd59e10720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:37.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff4f9d93700 1 --2- 172.21.15.2:0/778410855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59c92e40 0x55fd59e10720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff4f9d93700 1 -- 172.21.15.2:0/778410855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 msgr2=0x55fd59e15880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:37.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff4f9d93700 1 --2- 172.21.15.2:0/778410855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e15880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.434+0000 7ff4f9d93700 1 -- 172.21.15.2:0/778410855 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd59c7f8d0 con 0x55fd59e10c60 2024-05-14T09:36:37.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.435+0000 7ff4f9d93700 1 --2- 172.21.15.2:0/778410855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59e10c60 0x55fd59e14f30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80a9a00 tx=0x7ff4e80b2290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8962ff4162e186 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:37.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.435+0000 7ff4f0ff9700 1 -- 172.21.15.2:0/778410855 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e80aa170 con 0x55fd59e10c60 2024-05-14T09:36:37.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.435+0000 7ff4f0ff9700 1 -- 172.21.15.2:0/778410855 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e80a9a70 con 0x55fd59e10c60 2024-05-14T09:36:37.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.435+0000 7ff4f0ff9700 1 -- 172.21.15.2:0/778410855 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e80cc0d0 con 0x55fd59e10c60 2024-05-14T09:36:37.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.436+0000 7ff50b20cb80 1 -- 172.21.15.2:0/778410855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59e10c60 msgr2=0x55fd59e14f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:37.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.436+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/778410855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59e10c60 0x55fd59e14f30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80a9a00 tx=0x7ff4e80b2290 comp rx=0 tx=0).stop 2024-05-14T09:36:37.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.436+0000 7ff50b20cb80 1 -- 172.21.15.2:0/778410855 shutdown_connections 2024-05-14T09:36:37.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.436+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/778410855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e15880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.436+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/778410855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59c92e40 0x55fd59e10720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.436+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/778410855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59e10c60 0x55fd59e14f30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.436+0000 7ff50b20cb80 1 -- 172.21.15.2:0/778410855 >> 172.21.15.2:0/778410855 conn(0x55fd59cda600 msgr2=0x55fd59e0b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:37.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.436+0000 7ff50b20cb80 1 -- 172.21.15.2:0/778410855 shutdown_connections 2024-05-14T09:36:37.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.436+0000 7ff50b20cb80 1 -- 172.21.15.2:0/778410855 wait complete. 2024-05-14T09:36:37.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.436+0000 7ff50b20cb80 1 -- start start 2024-05-14T09:36:37.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff50b20cb80 1 Processor -- start 2024-05-14T09:36:37.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff50b20cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e1a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff50b20cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 0x55fd59e1a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff50b20cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e1ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff50b20cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fd59e1d420 con 0x55fd59c92e40 2024-05-14T09:36:37.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff50b20cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fd59c98c30 con 0x55fd59e15470 2024-05-14T09:36:37.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff50b20cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fd59e1d5a0 con 0x55fd59e10c60 2024-05-14T09:36:37.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff50b20cb80 10 client.0 _ll_register_callbacks cb 0x55fd59cca920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:37.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff4fa594700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e1a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff4fa594700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e1a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55450/0 (socket says 172.21.15.2:55450) 2024-05-14T09:36:37.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff4fa594700 1 -- 172.21.15.2:0/770419119 learned_addr learned my addr 172.21.15.2:0/770419119 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:37.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff4fad95700 1 --2- 172.21.15.2:0/770419119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e1ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff4f9d93700 1 --2- 172.21.15.2:0/770419119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 0x55fd59e1a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff4fad95700 1 -- 172.21.15.2:0/770419119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 msgr2=0x55fd59e1a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:37.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff4fad95700 1 --2- 172.21.15.2:0/770419119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 0x55fd59e1a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff4fad95700 1 -- 172.21.15.2:0/770419119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 msgr2=0x55fd59e1a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:37.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff4fad95700 1 --2- 172.21.15.2:0/770419119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e1a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff4fad95700 1 -- 172.21.15.2:0/770419119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd59cce690 con 0x55fd59e15470 2024-05-14T09:36:37.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff4f9d93700 1 --2- 172.21.15.2:0/770419119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 0x55fd59e1a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:36:37.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.437+0000 7ff4fa594700 1 --2- 172.21.15.2:0/770419119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e1a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:37.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.438+0000 7ff4fad95700 1 --2- 172.21.15.2:0/770419119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e1ac00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec006010 tx=0x7ff4ec009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb796f9bd539e496 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:37.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.438+0000 7ff4d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:36:37.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.438+0000 7ff4d9ffb700 1 -- 172.21.15.2:0/770419119 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec00f660 con 0x55fd59e15470 2024-05-14T09:36:37.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.438+0000 7ff50b20cb80 1 -- 172.21.15.2:0/770419119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4e80a9a70 con 0x55fd59e15470 2024-05-14T09:36:37.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.438+0000 7ff4d9ffb700 1 -- 172.21.15.2:0/770419119 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7ff4ec00f800 con 0x55fd59e15470 2024-05-14T09:36:37.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.438+0000 7ff4d9ffb700 1 -- 172.21.15.2:0/770419119 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4ec038050 con 0x55fd59e15470 2024-05-14T09:36:37.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.438+0000 7ff50b20cb80 10 client.25325 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:37.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.438+0000 7ff50b20cb80 1 -- 172.21.15.2:0/770419119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fd59e280a0 con 0x55fd59e15470 2024-05-14T09:36:37.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.439+0000 7ff4d9ffb700 1 -- 172.21.15.2:0/770419119 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e80a9a70 con 0x55fd59e15470 2024-05-14T09:36:37.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.439+0000 7ff4d9ffb700 1 -- 172.21.15.2:0/770419119 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff4ec022070 con 0x55fd59e15470 2024-05-14T09:36:37.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.439+0000 7ff50b20cb80 10 client.25325 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:37.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.439+0000 7ff50b20cb80 10 client.25325 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T09:36:37.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.439+0000 7ff50b20cb80 20 client.25325 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:37.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.439+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/770419119 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55fd59e2a630 0x55fd59e2ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.439+0000 7ff50b20cb80 4 client.25325 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:36:37.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.439+0000 7ff50b20cb80 1 -- 172.21.15.2:0/770419119 --> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd59d40240 con 0x55fd59e2a630 2024-05-14T09:36:37.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.439+0000 7ff4fa594700 1 --2- 172.21.15.2:0/770419119 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55fd59e2a630 0x55fd59e2ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.440+0000 7ff4d9ffb700 10 client.25325 ms_handle_connect on v2:172.21.15.29:6838/3625063120 2024-05-14T09:36:37.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.440+0000 7ff4fa594700 1 --2- 172.21.15.2:0/770419119 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55fd59e2a630 0x55fd59e2ca50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cdf2ffeee14b229 server_cookie=a0c8dbb7c2811ba8 in_seq=0 out_seq=0 2024-05-14T09:36:37.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.440+0000 7ff4d9ffb700 1 -- 172.21.15.2:0/770419119 <== mds.2 v2:172.21.15.29:6838/3625063120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fd59d40240 con 0x55fd59e2a630 2024-05-14T09:36:37.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.440+0000 7ff4d9ffb700 10 client.25325 handle_command_reply: tid=0 2024-05-14T09:36:37.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.440+0000 7ff50b20cb80 1 client.25325 shutdown 2024-05-14T09:36:37.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.441+0000 7ff50b20cb80 1 -- 172.21.15.2:0/770419119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 msgr2=0x55fd59e1ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:37.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.441+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/770419119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e1ac00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec006010 tx=0x7ff4ec009ca0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.441+0000 7ff50b20cb80 1 -- 172.21.15.2:0/770419119 shutdown_connections 2024-05-14T09:36:37.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.441+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/770419119 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55fd59e2a630 0x55fd59e2ca50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.441+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/770419119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e1ac00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.441+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/770419119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e1a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.441+0000 7ff4d9ffb700 0 client.25325 ms_handle_reset on v2:172.21.15.29:6838/3625063120 2024-05-14T09:36:37.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.441+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/770419119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 0x55fd59e1a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.441+0000 7ff50b20cb80 1 -- 172.21.15.2:0/770419119 >> 172.21.15.2:0/770419119 conn(0x55fd59cda600 msgr2=0x55fd59e0f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:37.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.441+0000 7ff50b20cb80 1 -- 172.21.15.2:0/770419119 shutdown_connections 2024-05-14T09:36:37.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.441+0000 7ff50b20cb80 1 -- 172.21.15.2:0/770419119 wait complete. 2024-05-14T09:36:37.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.441+0000 7ff50b20cb80 20 client.25325 trim_cache size 0 max 16384 2024-05-14T09:36:37.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff50b20cb80 1 Processor -- start 2024-05-14T09:36:37.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff50b20cb80 1 -- start start 2024-05-14T09:36:37.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff50b20cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e0b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff50b20cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 0x55fd59e264e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff50b20cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e32210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff50b20cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41b5070 con 0x55fd59c92e40 2024-05-14T09:36:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff50b20cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41b2900 con 0x55fd59e15470 2024-05-14T09:36:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff50b20cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41b0750 con 0x55fd59e10c60 2024-05-14T09:36:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff4fa594700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e0b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff4fa594700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e0b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55462/0 (socket says 172.21.15.2:55462) 2024-05-14T09:36:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff4fa594700 1 -- 172.21.15.2:0/3465207065 learned_addr learned my addr 172.21.15.2:0/3465207065 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff4fad95700 1 --2- 172.21.15.2:0/3465207065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e32210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff4f9d93700 1 --2- 172.21.15.2:0/3465207065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 0x55fd59e264e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff4fa594700 1 -- 172.21.15.2:0/3465207065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 msgr2=0x55fd59e264e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:37.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff4fa594700 1 --2- 172.21.15.2:0/3465207065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 0x55fd59e264e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff4fa594700 1 -- 172.21.15.2:0/3465207065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 msgr2=0x55fd59e32210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:37.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.456+0000 7ff4fa594700 1 --2- 172.21.15.2:0/3465207065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e32210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.457+0000 7ff4fa594700 1 -- 172.21.15.2:0/3465207065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd59e197f0 con 0x55fd59c92e40 2024-05-14T09:36:37.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.457+0000 7ff4fa594700 1 --2- 172.21.15.2:0/3465207065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e0b610 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x55fd59e2d150 tx=0x7ff4e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea194b1b9cc28ab6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:37.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.457+0000 7ff4dbfff700 1 -- 172.21.15.2:0/3465207065 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4024070 con 0x55fd59c92e40 2024-05-14T09:36:37.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.457+0000 7ff4dbfff700 1 -- 172.21.15.2:0/3465207065 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e401ea00 con 0x55fd59c92e40 2024-05-14T09:36:37.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.457+0000 7ff4dbfff700 1 -- 172.21.15.2:0/3465207065 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e400e4f0 con 0x55fd59c92e40 2024-05-14T09:36:37.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.458+0000 7ff50b20cb80 1 -- 172.21.15.2:0/3465207065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 msgr2=0x55fd59e0b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:37.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.458+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/3465207065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e0b610 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x55fd59e2d150 tx=0x7ff4e4004ad0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.458+0000 7ff50b20cb80 1 -- 172.21.15.2:0/3465207065 shutdown_connections 2024-05-14T09:36:37.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.458+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/3465207065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e32210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.458+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/3465207065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e0b610 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.458+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/3465207065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 0x55fd59e264e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.458+0000 7ff50b20cb80 1 -- 172.21.15.2:0/3465207065 >> 172.21.15.2:0/3465207065 conn(0x55fd59cda600 msgr2=0x55fd59e190e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:37.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.458+0000 7ff50b20cb80 1 -- 172.21.15.2:0/3465207065 shutdown_connections 2024-05-14T09:36:37.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.458+0000 7ff50b20cb80 1 -- 172.21.15.2:0/3465207065 wait complete. 2024-05-14T09:36:37.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.458+0000 7ff50b20cb80 1 -- start start 2024-05-14T09:36:37.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.458+0000 7ff50b20cb80 1 Processor -- start 2024-05-14T09:36:37.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff50b20cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e8d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff50b20cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 0x55fd59e8d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff50b20cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e1de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff50b20cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41b0dc0 con 0x55fd59c92e40 2024-05-14T09:36:37.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff50b20cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41b2230 con 0x55fd59e15470 2024-05-14T09:36:37.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff50b20cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41b1490 con 0x55fd59e10c60 2024-05-14T09:36:37.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff50b20cb80 10 client.0 _ll_register_callbacks cb 0x55fd59cca920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:37.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff4fa594700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e8d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff4fa594700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e8d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55474/0 (socket says 172.21.15.2:55474) 2024-05-14T09:36:37.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff4fa594700 1 -- 172.21.15.2:0/3343428686 learned_addr learned my addr 172.21.15.2:0/3343428686 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:37.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff4fad95700 1 --2- 172.21.15.2:0/3343428686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e1de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff4f9d93700 1 --2- 172.21.15.2:0/3343428686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 0x55fd59e8d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff4fad95700 1 -- 172.21.15.2:0/3343428686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 msgr2=0x55fd59e8d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:37.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff4fad95700 1 --2- 172.21.15.2:0/3343428686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 0x55fd59e8d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff4fad95700 1 -- 172.21.15.2:0/3343428686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 msgr2=0x55fd59e8d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:37.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff4fad95700 1 --2- 172.21.15.2:0/3343428686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e8d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff4fad95700 1 -- 172.21.15.2:0/3343428686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd59e1e430 con 0x55fd59e15470 2024-05-14T09:36:37.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff4fa594700 1 --2- 172.21.15.2:0/3343428686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e8d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff4fad95700 1 --2- 172.21.15.2:0/3343428686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e1de20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0098e0 tx=0x7ff4ec023da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26ba0f8ae7bd4a34 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.459+0000 7ff4d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:36:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.460+0000 7ff4d9ffb700 1 -- 172.21.15.2:0/3343428686 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec027040 con 0x55fd59e15470 2024-05-14T09:36:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.460+0000 7ff4d9ffb700 1 -- 172.21.15.2:0/3343428686 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7ff4ec023600 con 0x55fd59e15470 2024-05-14T09:36:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.460+0000 7ff50b20cb80 1 -- 172.21.15.2:0/3343428686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4e401ea00 con 0x55fd59e15470 2024-05-14T09:36:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.460+0000 7ff4d9ffb700 1 -- 172.21.15.2:0/3343428686 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4ec0056b0 con 0x55fd59e15470 2024-05-14T09:36:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.460+0000 7ff50b20cb80 10 client.25331 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.460+0000 7ff50b20cb80 1 -- 172.21.15.2:0/3343428686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fd59e947f0 con 0x55fd59e15470 2024-05-14T09:36:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.460+0000 7ff4d9ffb700 1 -- 172.21.15.2:0/3343428686 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e401ea00 con 0x55fd59e15470 2024-05-14T09:36:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.460+0000 7ff4d9ffb700 1 -- 172.21.15.2:0/3343428686 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff4ec03b070 con 0x55fd59e15470 2024-05-14T09:36:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.460+0000 7ff50b20cb80 10 client.25331 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.460+0000 7ff50b20cb80 10 client.25331 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T09:36:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.460+0000 7ff50b20cb80 20 client.25331 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:37.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.460+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/3343428686 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55fd59e94d10 0x55fd59e97130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:37.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.460+0000 7ff50b20cb80 4 client.25331 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T09:36:37.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.460+0000 7ff50b20cb80 1 -- 172.21.15.2:0/3343428686 --> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd59d3a2e0 con 0x55fd59e94d10 2024-05-14T09:36:37.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.461+0000 7ff4fa594700 1 --2- 172.21.15.2:0/3343428686 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55fd59e94d10 0x55fd59e97130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:37.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.461+0000 7ff4fa594700 1 --2- 172.21.15.2:0/3343428686 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55fd59e94d10 0x55fd59e97130 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7fc4dd842c44a11b server_cookie=9a6a1900ed792739 in_seq=0 out_seq=0 2024-05-14T09:36:37.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.461+0000 7ff4d9ffb700 10 client.25331 ms_handle_connect on v2:172.21.15.29:6838/3625063120 2024-05-14T09:36:37.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.461+0000 7ff4d9ffb700 1 -- 172.21.15.2:0/3343428686 <== mds.2 v2:172.21.15.29:6838/3625063120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd59d3a2e0 con 0x55fd59e94d10 2024-05-14T09:36:37.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.461+0000 7ff4d9ffb700 10 client.25331 handle_command_reply: tid=0 2024-05-14T09:36:37.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.461+0000 7ff50b20cb80 1 client.25331 shutdown 2024-05-14T09:36:37.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff50b20cb80 1 -- 172.21.15.2:0/3343428686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 msgr2=0x55fd59e1de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:37.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/3343428686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e1de20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0098e0 tx=0x7ff4ec023da0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff50b20cb80 1 -- 172.21.15.2:0/3343428686 shutdown_connections 2024-05-14T09:36:37.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/3343428686 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55fd59e94d10 0x55fd59e97130 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/3343428686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd59e15470 0x55fd59e1de20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/3343428686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd59c92e40 0x55fd59e8d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff50b20cb80 1 --2- 172.21.15.2:0/3343428686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fd59e10c60 0x55fd59e8d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff4d9ffb700 0 client.25331 ms_handle_reset on v2:172.21.15.29:6838/3625063120 2024-05-14T09:36:37.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff50b20cb80 1 -- 172.21.15.2:0/3343428686 >> 172.21.15.2:0/3343428686 conn(0x55fd59cda600 msgr2=0x55fd59e190e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:37.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff50b20cb80 1 -- 172.21.15.2:0/3343428686 shutdown_connections 2024-05-14T09:36:37.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff50b20cb80 1 -- 172.21.15.2:0/3343428686 wait complete. 2024-05-14T09:36:37.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff50b20cb80 20 client.25331 trim_cache size 0 max 16384 2024-05-14T09:36:37.467 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-05-14T09:36:37.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff4fc7f8700 1 -- 172.21.15.2:0/370302448 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff4dc0b2880 msgr2=0x7ff4dc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:37.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff4fc7f8700 1 --2- 172.21.15.2:0/370302448 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff4dc0b2880 0x7ff4dc0b4d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00b2e0 tx=0x7ff4ec005960 comp rx=0 tx=0).stop 2024-05-14T09:36:37.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff4fc7f8700 1 -- 172.21.15.2:0/370302448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f4106ed0 msgr2=0x7ff4f419d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:37.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff4fc7f8700 1 --2- 172.21.15.2:0/370302448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f4106ed0 0x7ff4f419d580 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800e940 tx=0x7ff4e800ec50 comp rx=0 tx=0).stop 2024-05-14T09:36:37.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff4fc7f8700 1 -- 172.21.15.2:0/370302448 shutdown_connections 2024-05-14T09:36:37.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff4fc7f8700 1 --2- 172.21.15.2:0/370302448 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff4dc0b2880 0x7ff4dc0b4d90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff4fc7f8700 1 --2- 172.21.15.2:0/370302448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4f4106580 0x7ff4f419cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff4fc7f8700 1 --2- 172.21.15.2:0/370302448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f410c5c0 0x7ff4f41a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff4fc7f8700 1 --2- 172.21.15.2:0/370302448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f4106ed0 0x7ff4f419d580 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:37.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.462+0000 7ff4fc7f8700 1 -- 172.21.15.2:0/370302448 >> 172.21.15.2:0/370302448 conn(0x7ff4f4102080 msgr2=0x7ff4f41046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:37.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.463+0000 7ff4fc7f8700 1 -- 172.21.15.2:0/370302448 shutdown_connections 2024-05-14T09:36:37.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:37.463+0000 7ff4fc7f8700 1 -- 172.21.15.2:0/370302448 wait complete. 2024-05-14T09:36:38.232 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T09:36:38.252 INFO:tasks.workunit.client.0.smithi002.stdout: 1 11454 19.65 MB/sec warmup 4 sec latency 20.231 ms 2024-05-14T09:36:38.489 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:38 smithi029 ceph-mon[36249]: pgmap v474: 129 pgs: 129 active+clean; 2.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 138 KiB/s wr, 20 op/s 2024-05-14T09:36:38.491 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:36:38.502 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:38 smithi026 ceph-mon[36056]: pgmap v474: 129 pgs: 129 active+clean; 2.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 138 KiB/s wr, 20 op/s 2024-05-14T09:36:38.614 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:38 smithi002 ceph-mon[31557]: pgmap v474: 129 pgs: 129 active+clean; 2.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 138 KiB/s wr, 20 op/s 2024-05-14T09:36:39.252 INFO:tasks.workunit.client.0.smithi002.stdout: 1 14629 19.44 MB/sec warmup 5 sec latency 14.350 ms 2024-05-14T09:36:40.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 17847 19.16 MB/sec warmup 6 sec latency 28.352 ms 2024-05-14T09:36:40.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.292+0000 7f46ebb4f700 1 -- 172.21.15.2:0/333424168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e4103080 msgr2=0x7f46e4103490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:40.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.292+0000 7f46ebb4f700 1 --2- 172.21.15.2:0/333424168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e4103080 0x7f46e4103490 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009990 tx=0x7f46e0009ca0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.292+0000 7f46ebb4f700 1 -- 172.21.15.2:0/333424168 shutdown_connections 2024-05-14T09:36:40.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.292+0000 7f46ebb4f700 1 --2- 172.21.15.2:0/333424168 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46e4110c30 0x7f46e4113050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.292+0000 7f46ebb4f700 1 --2- 172.21.15.2:0/333424168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e4103080 0x7f46e4103490 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.292+0000 7f46ebb4f700 1 --2- 172.21.15.2:0/333424168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46e4103a60 0x7f46e41106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.292+0000 7f46ebb4f700 1 -- 172.21.15.2:0/333424168 >> 172.21.15.2:0/333424168 conn(0x7f46e40fedc0 msgr2=0x7f46e41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:40.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.292+0000 7f46ebb4f700 1 -- 172.21.15.2:0/333424168 shutdown_connections 2024-05-14T09:36:40.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.292+0000 7f46ebb4f700 1 -- 172.21.15.2:0/333424168 wait complete. 2024-05-14T09:36:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.293+0000 7f46ebb4f700 1 Processor -- start 2024-05-14T09:36:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.293+0000 7f46ebb4f700 1 -- start start 2024-05-14T09:36:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46ebb4f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46e4103080 0x7f46e419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46ebb4f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46e4103a60 0x7f46e419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46ebb4f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e4110c30 0x7f46e41a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46ebb4f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f46e4116680 con 0x7f46e4110c30 2024-05-14T09:36:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46ebb4f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f46e4116500 con 0x7f46e4103080 2024-05-14T09:36:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46ebb4f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f46e4116800 con 0x7f46e4103a60 2024-05-14T09:36:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46e98eb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46e4103080 0x7f46e419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46ea0ec700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e4110c30 0x7f46e41a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46e98eb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46e4103080 0x7f46e419d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:40848/0 (socket says 172.21.15.2:40848) 2024-05-14T09:36:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46e98eb700 1 -- 172.21.15.2:0/2031169310 learned_addr learned my addr 172.21.15.2:0/2031169310 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46e90ea700 1 --2- 172.21.15.2:0/2031169310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46e4103a60 0x7f46e419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46e98eb700 1 -- 172.21.15.2:0/2031169310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46e4103a60 msgr2=0x7f46e419d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46e98eb700 1 --2- 172.21.15.2:0/2031169310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46e4103a60 0x7f46e419d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46e98eb700 1 -- 172.21.15.2:0/2031169310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e4110c30 msgr2=0x7f46e41a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46e98eb700 1 --2- 172.21.15.2:0/2031169310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e4110c30 0x7f46e41a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46e98eb700 1 -- 172.21.15.2:0/2031169310 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e41a1ee0 con 0x7f46e4103080 2024-05-14T09:36:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.294+0000 7f46ea0ec700 1 --2- 172.21.15.2:0/2031169310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e4110c30 0x7f46e41a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.295+0000 7f46e98eb700 1 --2- 172.21.15.2:0/2031169310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46e4103080 0x7f46e419d010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f46e0000c00 tx=0x7f46e000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.295+0000 7f46daffd700 1 -- 172.21.15.2:0/2031169310 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0027070 con 0x7f46e4103080 2024-05-14T09:36:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.295+0000 7f46ebb4f700 1 -- 172.21.15.2:0/2031169310 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e41a2170 con 0x7f46e4103080 2024-05-14T09:36:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.295+0000 7f46daffd700 1 -- 172.21.15.2:0/2031169310 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46e0022470 con 0x7f46e4103080 2024-05-14T09:36:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.295+0000 7f46daffd700 1 -- 172.21.15.2:0/2031169310 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0017650 con 0x7f46e4103080 2024-05-14T09:36:40.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.295+0000 7f46ebb4f700 1 -- 172.21.15.2:0/2031169310 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e41a2650 con 0x7f46e4103080 2024-05-14T09:36:40.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.299+0000 7f46daffd700 1 -- 172.21.15.2:0/2031169310 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f46e002a070 con 0x7f46e4103080 2024-05-14T09:36:40.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.299+0000 7f46fa563b80 1 Processor -- start 2024-05-14T09:36:40.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46fa563b80 1 -- start start 2024-05-14T09:36:40.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46fa563b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7948e40 0x5609c7ac6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46fa563b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7ac6c60 0x5609c7acaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46fa563b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7acb470 0x5609c7acb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46fa563b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5609c79e8860 con 0x5609c7948e40 2024-05-14T09:36:40.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46fa563b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5609c79877c0 con 0x5609c7acb470 2024-05-14T09:36:40.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46daffd700 1 --2- 172.21.15.2:0/2031169310 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f46d00b2880 0x7f46d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46fa563b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5609c79e4820 con 0x5609c7ac6c60 2024-05-14T09:36:40.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46ea0ec700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7948e40 0x5609c7ac6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46ea0ec700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7948e40 0x5609c7ac6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41692/0 (socket says 172.21.15.2:41692) 2024-05-14T09:36:40.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46ea0ec700 1 -- 172.21.15.2:0/258733079 learned_addr learned my addr 172.21.15.2:0/258733079 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:40.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46e90ea700 1 --2- 172.21.15.2:0/258733079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7acb470 0x5609c7acb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46ea0ec700 1 --2- 172.21.15.2:0/2031169310 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f46d00b2880 0x7f46d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46daffd700 1 -- 172.21.15.2:0/2031169310 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f46e00e33f0 con 0x7f46e4103080 2024-05-14T09:36:40.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46e98eb700 1 --2- 172.21.15.2:0/258733079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7ac6c60 0x5609c7acaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46ea0ec700 1 -- 172.21.15.2:0/258733079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7ac6c60 msgr2=0x5609c7acaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:40.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46ea0ec700 1 --2- 172.21.15.2:0/258733079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7ac6c60 0x5609c7acaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46ea0ec700 1 -- 172.21.15.2:0/258733079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7acb470 msgr2=0x5609c7acb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:40.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46ea0ec700 1 --2- 172.21.15.2:0/258733079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7acb470 0x5609c7acb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.300+0000 7f46ea0ec700 1 -- 172.21.15.2:0/258733079 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c79358d0 con 0x5609c7948e40 2024-05-14T09:36:40.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.301+0000 7f46e98eb700 1 --2- 172.21.15.2:0/258733079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7ac6c60 0x5609c7acaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:36:40.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.301+0000 7f46ea0ec700 1 --2- 172.21.15.2:0/258733079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7948e40 0x5609c7ac6720 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f46dc008f90 tx=0x7f46dc00e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891471fe16404593 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:40.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.301+0000 7f46ea0ec700 1 --2- 172.21.15.2:0/2031169310 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f46d00b2880 0x7f46d00b4d90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009fd0 tx=0x7f46dc01e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:40.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.301+0000 7f46cffff700 1 -- 172.21.15.2:0/258733079 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc008620 con 0x5609c7948e40 2024-05-14T09:36:40.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.301+0000 7f46cffff700 1 -- 172.21.15.2:0/258733079 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46dc028070 con 0x5609c7948e40 2024-05-14T09:36:40.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.301+0000 7f46cffff700 1 -- 172.21.15.2:0/258733079 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc023a40 con 0x5609c7948e40 2024-05-14T09:36:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.302+0000 7f46fa563b80 1 -- 172.21.15.2:0/258733079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7948e40 msgr2=0x5609c7ac6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.302+0000 7f46fa563b80 1 --2- 172.21.15.2:0/258733079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7948e40 0x5609c7ac6720 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f46dc008f90 tx=0x7f46dc00e7d0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.302+0000 7f46fa563b80 1 -- 172.21.15.2:0/258733079 shutdown_connections 2024-05-14T09:36:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.302+0000 7f46fa563b80 1 --2- 172.21.15.2:0/258733079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7acb470 0x5609c7acb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.302+0000 7f46fa563b80 1 --2- 172.21.15.2:0/258733079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7948e40 0x5609c7ac6720 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.302+0000 7f46fa563b80 1 --2- 172.21.15.2:0/258733079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7ac6c60 0x5609c7acaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.302+0000 7f46fa563b80 1 -- 172.21.15.2:0/258733079 >> 172.21.15.2:0/258733079 conn(0x5609c7990600 msgr2=0x5609c7ac14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.302+0000 7f46fa563b80 1 -- 172.21.15.2:0/258733079 shutdown_connections 2024-05-14T09:36:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.302+0000 7f46fa563b80 1 -- 172.21.15.2:0/258733079 wait complete. 2024-05-14T09:36:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.302+0000 7f46fa563b80 1 -- start start 2024-05-14T09:36:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.303+0000 7f46fa563b80 1 Processor -- start 2024-05-14T09:36:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.303+0000 7f46fa563b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7948e40 0x5609c7ad01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.303+0000 7f46fa563b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7ac6c60 0x5609c7ad06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.303+0000 7f46fa563b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7acb470 0x5609c7ad0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.303+0000 7f46fa563b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5609c7ad33c0 con 0x5609c7948e40 2024-05-14T09:36:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.303+0000 7f46fa563b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5609c794ec30 con 0x5609c7ac6c60 2024-05-14T09:36:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.303+0000 7f46fa563b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5609c7ad3540 con 0x5609c7acb470 2024-05-14T09:36:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.303+0000 7f46fa563b80 10 client.0 _ll_register_callbacks cb 0x5609c7980920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.303+0000 7f46e98eb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7948e40 0x5609c7ad01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.303+0000 7f46e98eb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7948e40 0x5609c7ad01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41702/0 (socket says 172.21.15.2:41702) 2024-05-14T09:36:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.303+0000 7f46e98eb700 1 -- 172.21.15.2:0/1497738421 learned_addr learned my addr 172.21.15.2:0/1497738421 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.303+0000 7f46e90ea700 1 --2- 172.21.15.2:0/1497738421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7ac6c60 0x5609c7ad06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.303+0000 7f46ea0ec700 1 --2- 172.21.15.2:0/1497738421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7acb470 0x5609c7ad0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.304+0000 7f46e98eb700 1 -- 172.21.15.2:0/1497738421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7acb470 msgr2=0x5609c7ad0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.304+0000 7f46e98eb700 1 --2- 172.21.15.2:0/1497738421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7acb470 0x5609c7ad0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.304+0000 7f46e98eb700 1 -- 172.21.15.2:0/1497738421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7ac6c60 msgr2=0x5609c7ad06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:40.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.304+0000 7f46e98eb700 1 --2- 172.21.15.2:0/1497738421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7ac6c60 0x5609c7ad06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.304+0000 7f46e98eb700 1 -- 172.21.15.2:0/1497738421 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c7984690 con 0x5609c7948e40 2024-05-14T09:36:40.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.304+0000 7f46e98eb700 1 --2- 172.21.15.2:0/1497738421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7948e40 0x5609c7ad01b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f46e00a7890 tx=0x7f46e00a9d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f3adf04c72be86 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:40.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.304+0000 7f46cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:36:40.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.304+0000 7f46cd7fa700 1 -- 172.21.15.2:0/1497738421 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e00aad20 con 0x5609c7948e40 2024-05-14T09:36:40.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.304+0000 7f46cd7fa700 1 -- 172.21.15.2:0/1497738421 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f46e00e3170 con 0x5609c7948e40 2024-05-14T09:36:40.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.304+0000 7f46fa563b80 1 -- 172.21.15.2:0/1497738421 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46dc028070 con 0x5609c7948e40 2024-05-14T09:36:40.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.304+0000 7f46cd7fa700 1 -- 172.21.15.2:0/1497738421 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f46e00aa620 con 0x5609c7948e40 2024-05-14T09:36:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.304+0000 7f46fa563b80 10 client.15645 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.304+0000 7f46fa563b80 1 -- 172.21.15.2:0/1497738421 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5609c7addfe0 con 0x5609c7948e40 2024-05-14T09:36:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.304+0000 7f46cd7fa700 1 -- 172.21.15.2:0/1497738421 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46dc028070 con 0x5609c7948e40 2024-05-14T09:36:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.304+0000 7f46cd7fa700 1 -- 172.21.15.2:0/1497738421 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f46e00b2f60 con 0x5609c7948e40 2024-05-14T09:36:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.305+0000 7f46fa563b80 10 client.15645 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.305+0000 7f46fa563b80 10 client.15645 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T09:36:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.305+0000 7f46fa563b80 20 client.15645 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.305+0000 7f46fa563b80 1 --2- 172.21.15.2:0/1497738421 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5609c7ae0500 0x5609c7ae29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.305+0000 7f46fa563b80 4 client.15645 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:36:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.305+0000 7f46fa563b80 1 -- 172.21.15.2:0/1497738421 --> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609c79f6240 con 0x5609c7ae0500 2024-05-14T09:36:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.305+0000 7f46e90ea700 1 --2- 172.21.15.2:0/1497738421 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5609c7ae0500 0x5609c7ae29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.305+0000 7f46cd7fa700 10 client.15645 ms_handle_connect on v2:172.21.15.26:6836/3301773578 2024-05-14T09:36:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.305+0000 7f46e90ea700 1 --2- 172.21.15.2:0/1497738421 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5609c7ae0500 0x5609c7ae29d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=149c321de2cc2d77 server_cookie=444d7b0be45dc07b in_seq=0 out_seq=0 2024-05-14T09:36:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.306+0000 7f46cd7fa700 1 -- 172.21.15.2:0/1497738421 <== mds.1 v2:172.21.15.26:6836/3301773578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5609c79f6240 con 0x5609c7ae0500 2024-05-14T09:36:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.306+0000 7f46cd7fa700 10 client.15645 handle_command_reply: tid=0 2024-05-14T09:36:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.306+0000 7f46fa563b80 1 client.15645 shutdown 2024-05-14T09:36:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.306+0000 7f46fa563b80 1 -- 172.21.15.2:0/1497738421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7948e40 msgr2=0x5609c7ad01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.306+0000 7f46fa563b80 1 --2- 172.21.15.2:0/1497738421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7948e40 0x5609c7ad01b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f46e00a7890 tx=0x7f46e00a9d90 comp rx=0 tx=0).stop 2024-05-14T09:36:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.306+0000 7f46fa563b80 1 -- 172.21.15.2:0/1497738421 shutdown_connections 2024-05-14T09:36:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.306+0000 7f46fa563b80 1 --2- 172.21.15.2:0/1497738421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7acb470 0x5609c7ad0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.306+0000 7f46fa563b80 1 --2- 172.21.15.2:0/1497738421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7948e40 0x5609c7ad01b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.306+0000 7f46fa563b80 1 --2- 172.21.15.2:0/1497738421 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5609c7ae0500 0x5609c7ae29d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.306+0000 7f46fa563b80 1 --2- 172.21.15.2:0/1497738421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7ac6c60 0x5609c7ad06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.306+0000 7f46fa563b80 1 -- 172.21.15.2:0/1497738421 >> 172.21.15.2:0/1497738421 conn(0x5609c7990600 msgr2=0x5609c7ac5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:40.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.306+0000 7f46cd7fa700 0 client.15645 ms_handle_reset on v2:172.21.15.26:6836/3301773578 2024-05-14T09:36:40.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.307+0000 7f46fa563b80 1 -- 172.21.15.2:0/1497738421 shutdown_connections 2024-05-14T09:36:40.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.307+0000 7f46fa563b80 1 -- 172.21.15.2:0/1497738421 wait complete. 2024-05-14T09:36:40.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.307+0000 7f46fa563b80 20 client.15645 trim_cache size 0 max 16384 2024-05-14T09:36:40.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.322+0000 7f46fa563b80 1 Processor -- start 2024-05-14T09:36:40.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.322+0000 7f46fa563b80 1 -- start start 2024-05-14T09:36:40.322 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:40 smithi002 ceph-mon[31557]: pgmap v475: 129 pgs: 129 active+clean; 5.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 488 KiB/s wr, 80 op/s 2024-05-14T09:36:40.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.322+0000 7f46fa563b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7948e40 0x5609c7ac1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.322+0000 7f46fa563b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7ac6c60 0x5609c7acade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.322+0000 7f46fa563b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7acb470 0x5609c7acc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.322+0000 7f46fa563b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f46e41b4ea0 con 0x5609c7acb470 2024-05-14T09:36:40.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.322+0000 7f46fa563b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f46e41b2730 con 0x5609c7ac6c60 2024-05-14T09:36:40.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.322+0000 7f46fa563b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f46e41b0580 con 0x5609c7948e40 2024-05-14T09:36:40.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.322+0000 7f46e90ea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7ac6c60 0x5609c7acade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.322+0000 7f46e98eb700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7948e40 0x5609c7ac1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.322+0000 7f46e90ea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7ac6c60 0x5609c7acade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:40874/0 (socket says 172.21.15.2:40874) 2024-05-14T09:36:40.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.322+0000 7f46e90ea700 1 -- 172.21.15.2:0/3452148896 learned_addr learned my addr 172.21.15.2:0/3452148896 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:40.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.322+0000 7f46ea0ec700 1 --2- 172.21.15.2:0/3452148896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7acb470 0x5609c7acc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.323+0000 7f46e90ea700 1 -- 172.21.15.2:0/3452148896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7948e40 msgr2=0x5609c7ac1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:40.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.323+0000 7f46e90ea700 1 --2- 172.21.15.2:0/3452148896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7948e40 0x5609c7ac1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.323+0000 7f46e90ea700 1 -- 172.21.15.2:0/3452148896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7acb470 msgr2=0x5609c7acc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:40.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.323+0000 7f46e90ea700 1 --2- 172.21.15.2:0/3452148896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7acb470 0x5609c7acc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.323+0000 7f46e90ea700 1 -- 172.21.15.2:0/3452148896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c7ace720 con 0x5609c7ac6c60 2024-05-14T09:36:40.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.323+0000 7f46e98eb700 1 --2- 172.21.15.2:0/3452148896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7948e40 0x5609c7ac1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:40.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.323+0000 7f46e90ea700 1 --2- 172.21.15.2:0/3452148896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7ac6c60 0x5609c7acade0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f46d4000c00 tx=0x7f46d4009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3ef665264923f0c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:40.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.323+0000 7f46cf7fe700 1 -- 172.21.15.2:0/3452148896 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4003c60 con 0x5609c7ac6c60 2024-05-14T09:36:40.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.323+0000 7f46cf7fe700 1 -- 172.21.15.2:0/3452148896 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46d4009e50 con 0x5609c7ac6c60 2024-05-14T09:36:40.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.323+0000 7f46cf7fe700 1 -- 172.21.15.2:0/3452148896 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4012a60 con 0x5609c7ac6c60 2024-05-14T09:36:40.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.324+0000 7f46fa563b80 1 -- 172.21.15.2:0/3452148896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7ac6c60 msgr2=0x5609c7acade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:40.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.324+0000 7f46fa563b80 1 --2- 172.21.15.2:0/3452148896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7ac6c60 0x5609c7acade0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f46d4000c00 tx=0x7f46d4009de0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.324+0000 7f46fa563b80 1 -- 172.21.15.2:0/3452148896 shutdown_connections 2024-05-14T09:36:40.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.324+0000 7f46fa563b80 1 --2- 172.21.15.2:0/3452148896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7acb470 0x5609c7acc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.324+0000 7f46fa563b80 1 --2- 172.21.15.2:0/3452148896 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7948e40 0x5609c7ac1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.324+0000 7f46fa563b80 1 --2- 172.21.15.2:0/3452148896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7ac6c60 0x5609c7acade0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.324+0000 7f46fa563b80 1 -- 172.21.15.2:0/3452148896 >> 172.21.15.2:0/3452148896 conn(0x5609c7990600 msgr2=0x5609c7ac5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.324+0000 7f46fa563b80 1 -- 172.21.15.2:0/3452148896 shutdown_connections 2024-05-14T09:36:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.324+0000 7f46fa563b80 1 -- 172.21.15.2:0/3452148896 wait complete. 2024-05-14T09:36:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.325+0000 7f46fa563b80 1 -- start start 2024-05-14T09:36:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.325+0000 7f46fa563b80 1 Processor -- start 2024-05-14T09:36:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.325+0000 7f46fa563b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7948e40 0x5609c7ae7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.325+0000 7f46fa563b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7ac6c60 0x5609c7ae81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.325+0000 7f46fa563b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7acb470 0x5609c7ae8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.325+0000 7f46fa563b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f46e41b0bf0 con 0x5609c7ac6c60 2024-05-14T09:36:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.325+0000 7f46fa563b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f46e41b2060 con 0x5609c7948e40 2024-05-14T09:36:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.325+0000 7f46fa563b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f46e41b12c0 con 0x5609c7acb470 2024-05-14T09:36:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.325+0000 7f46fa563b80 10 client.0 _ll_register_callbacks cb 0x5609c7980920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46e98eb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7948e40 0x5609c7ae7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46e90ea700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7ac6c60 0x5609c7ae81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46e98eb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7948e40 0x5609c7ae7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:40876/0 (socket says 172.21.15.2:40876) 2024-05-14T09:36:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46e90ea700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7ac6c60 0x5609c7ae81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41712/0 (socket says 172.21.15.2:41712) 2024-05-14T09:36:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46e98eb700 1 -- 172.21.15.2:0/3875965253 learned_addr learned my addr 172.21.15.2:0/3875965253 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46ea0ec700 1 --2- 172.21.15.2:0/3875965253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7acb470 0x5609c7ae8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46e98eb700 1 -- 172.21.15.2:0/3875965253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7acb470 msgr2=0x5609c7ae8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46e98eb700 1 --2- 172.21.15.2:0/3875965253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7acb470 0x5609c7ae8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46e98eb700 1 -- 172.21.15.2:0/3875965253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7ac6c60 msgr2=0x5609c7ae81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46e98eb700 1 --2- 172.21.15.2:0/3875965253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7ac6c60 0x5609c7ae81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46e98eb700 1 -- 172.21.15.2:0/3875965253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c7b49fc0 con 0x5609c7948e40 2024-05-14T09:36:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46e98eb700 1 --2- 172.21.15.2:0/3875965253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7948e40 0x5609c7ae7ca0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f46e00aa7e0 tx=0x7f46e0022c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c36e5709f493f63 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:36:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46cd7fa700 1 -- 172.21.15.2:0/3875965253 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e00ae9d0 con 0x5609c7948e40 2024-05-14T09:36:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46cd7fa700 1 -- 172.21.15.2:0/3875965253 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f46e0090b90 con 0x5609c7948e40 2024-05-14T09:36:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.326+0000 7f46fa563b80 1 -- 172.21.15.2:0/3875965253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46d4009e50 con 0x5609c7948e40 2024-05-14T09:36:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.327+0000 7f46cd7fa700 1 -- 172.21.15.2:0/3875965253 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f46e0090d90 con 0x5609c7948e40 2024-05-14T09:36:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.327+0000 7f46fa563b80 10 client.25349 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.327+0000 7f46fa563b80 1 -- 172.21.15.2:0/3875965253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5609c7b49fc0 con 0x5609c7948e40 2024-05-14T09:36:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.327+0000 7f46cd7fa700 1 -- 172.21.15.2:0/3875965253 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46d4009e50 con 0x5609c7948e40 2024-05-14T09:36:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.327+0000 7f46cd7fa700 1 -- 172.21.15.2:0/3875965253 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f46e00a5f90 con 0x5609c7948e40 2024-05-14T09:36:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.327+0000 7f46fa563b80 10 client.25349 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.327+0000 7f46fa563b80 10 client.25349 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T09:36:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.327+0000 7f46fa563b80 20 client.25349 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.327+0000 7f46fa563b80 1 --2- 172.21.15.2:0/3875965253 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5609c7b4b5e0 0x5609c7b4da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.327+0000 7f46fa563b80 4 client.25349 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T09:36:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.327+0000 7f46fa563b80 1 -- 172.21.15.2:0/3875965253 --> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609c79f02e0 con 0x5609c7b4b5e0 2024-05-14T09:36:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.327+0000 7f46e90ea700 1 --2- 172.21.15.2:0/3875965253 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5609c7b4b5e0 0x5609c7b4da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.328+0000 7f46cd7fa700 10 client.25349 ms_handle_connect on v2:172.21.15.26:6836/3301773578 2024-05-14T09:36:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.328+0000 7f46e90ea700 1 --2- 172.21.15.2:0/3875965253 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5609c7b4b5e0 0x5609c7b4da00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72085147b8011407 server_cookie=6e9cffba5040151e in_seq=0 out_seq=0 2024-05-14T09:36:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.328+0000 7f46cd7fa700 1 -- 172.21.15.2:0/3875965253 <== mds.1 v2:172.21.15.26:6836/3301773578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609c79f02e0 con 0x5609c7b4b5e0 2024-05-14T09:36:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.328+0000 7f46cd7fa700 10 client.25349 handle_command_reply: tid=0 2024-05-14T09:36:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.328+0000 7f46fa563b80 1 client.25349 shutdown 2024-05-14T09:36:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.328+0000 7f46fa563b80 1 -- 172.21.15.2:0/3875965253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7948e40 msgr2=0x5609c7ae7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.328+0000 7f46fa563b80 1 --2- 172.21.15.2:0/3875965253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7948e40 0x5609c7ae7ca0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f46e00aa7e0 tx=0x7f46e0022c20 comp rx=0 tx=0).stop 2024-05-14T09:36:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.328+0000 7f46fa563b80 1 -- 172.21.15.2:0/3875965253 shutdown_connections 2024-05-14T09:36:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.328+0000 7f46fa563b80 1 --2- 172.21.15.2:0/3875965253 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5609c7b4b5e0 0x5609c7b4da00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.328+0000 7f46fa563b80 1 --2- 172.21.15.2:0/3875965253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609c7948e40 0x5609c7ae7ca0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.328+0000 7f46fa563b80 1 --2- 172.21.15.2:0/3875965253 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609c7acb470 0x5609c7ae8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.328+0000 7f46cd7fa700 0 client.25349 ms_handle_reset on v2:172.21.15.26:6836/3301773578 2024-05-14T09:36:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.328+0000 7f46fa563b80 1 --2- 172.21.15.2:0/3875965253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609c7ac6c60 0x5609c7ae81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.328+0000 7f46fa563b80 1 -- 172.21.15.2:0/3875965253 >> 172.21.15.2:0/3875965253 conn(0x5609c7990600 msgr2=0x5609c7ac1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.329+0000 7f46fa563b80 1 -- 172.21.15.2:0/3875965253 shutdown_connections 2024-05-14T09:36:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.329+0000 7f46fa563b80 1 -- 172.21.15.2:0/3875965253 wait complete. 2024-05-14T09:36:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.329+0000 7f46fa563b80 20 client.25349 trim_cache size 0 max 16384 2024-05-14T09:36:40.333 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-05-14T09:36:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.329+0000 7f46ebb4f700 1 -- 172.21.15.2:0/2031169310 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f46d00b2880 msgr2=0x7f46d00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.329+0000 7f46ebb4f700 1 --2- 172.21.15.2:0/2031169310 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f46d00b2880 0x7f46d00b4d90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009fd0 tx=0x7f46dc01e000 comp rx=0 tx=0).stop 2024-05-14T09:36:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.329+0000 7f46ebb4f700 1 -- 172.21.15.2:0/2031169310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46e4103080 msgr2=0x7f46e419d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.329+0000 7f46ebb4f700 1 --2- 172.21.15.2:0/2031169310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46e4103080 0x7f46e419d010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f46e0000c00 tx=0x7f46e000bfd0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.329+0000 7f46ebb4f700 1 -- 172.21.15.2:0/2031169310 shutdown_connections 2024-05-14T09:36:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.329+0000 7f46ebb4f700 1 --2- 172.21.15.2:0/2031169310 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f46d00b2880 0x7f46d00b4d90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.329+0000 7f46ebb4f700 1 --2- 172.21.15.2:0/2031169310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46e4103080 0x7f46e419d010 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.329+0000 7f46ebb4f700 1 --2- 172.21.15.2:0/2031169310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e4110c30 0x7f46e41a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.329+0000 7f46ebb4f700 1 --2- 172.21.15.2:0/2031169310 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f46e4103a60 0x7f46e419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:40.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.329+0000 7f46ebb4f700 1 -- 172.21.15.2:0/2031169310 >> 172.21.15.2:0/2031169310 conn(0x7f46e40fedc0 msgr2=0x7f46e4112800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:40.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.329+0000 7f46ebb4f700 1 -- 172.21.15.2:0/2031169310 shutdown_connections 2024-05-14T09:36:40.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:40.330+0000 7f46ebb4f700 1 -- 172.21.15.2:0/2031169310 wait complete. 2024-05-14T09:36:40.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:40 smithi029 ceph-mon[36249]: pgmap v475: 129 pgs: 129 active+clean; 5.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 488 KiB/s wr, 80 op/s 2024-05-14T09:36:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:40 smithi026 ceph-mon[36056]: pgmap v475: 129 pgs: 129 active+clean; 5.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 488 KiB/s wr, 80 op/s 2024-05-14T09:36:41.101 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T09:36:41.252 INFO:tasks.workunit.client.0.smithi002.stdout: 1 20564 18.20 MB/sec warmup 7 sec latency 15.436 ms 2024-05-14T09:36:41.408 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:36:41.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:41 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:36:41.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:41 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:36:41.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:41 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:36:42.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 22973 17.60 MB/sec warmup 8 sec latency 27.960 ms 2024-05-14T09:36:42.663 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:42 smithi002 ceph-mon[31557]: pgmap v476: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 531 KiB/s wr, 80 op/s 2024-05-14T09:36:42.663 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:42 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:36:42.663 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:42 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:36:42.663 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:42 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:36:42.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:42 smithi029 ceph-mon[36249]: pgmap v476: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 531 KiB/s wr, 80 op/s 2024-05-14T09:36:42.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:42 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:36:42.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:42 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:36:42.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:42 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:36:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:42 smithi026 ceph-mon[36056]: pgmap v476: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 531 KiB/s wr, 80 op/s 2024-05-14T09:36:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:42 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:36:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:42 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:36:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:42 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:36:43.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.225+0000 7f142f65a700 1 -- 172.21.15.2:0/3228980861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14281034c0 msgr2=0x7f14281038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:43.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.225+0000 7f142f65a700 1 --2- 172.21.15.2:0/3228980861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14281034c0 0x7f14281038d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f142400b0d0 tx=0x7f142400b3e0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.225+0000 7f142f65a700 1 -- 172.21.15.2:0/3228980861 shutdown_connections 2024-05-14T09:36:43.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.225+0000 7f142f65a700 1 --2- 172.21.15.2:0/3228980861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1428108740 0x7f14281100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.225+0000 7f142f65a700 1 --2- 172.21.15.2:0/3228980861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14281034c0 0x7f14281038d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.225+0000 7f142f65a700 1 --2- 172.21.15.2:0/3228980861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1428107df0 0x7f1428108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.225+0000 7f142f65a700 1 -- 172.21.15.2:0/3228980861 >> 172.21.15.2:0/3228980861 conn(0x7f14280fedc0 msgr2=0x7f14281011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.225+0000 7f142f65a700 1 -- 172.21.15.2:0/3228980861 shutdown_connections 2024-05-14T09:36:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.225+0000 7f142f65a700 1 -- 172.21.15.2:0/3228980861 wait complete. 2024-05-14T09:36:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.226+0000 7f142f65a700 1 Processor -- start 2024-05-14T09:36:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.226+0000 7f142f65a700 1 -- start start 2024-05-14T09:36:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.227+0000 7f142f65a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14281034c0 0x7f142819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.227+0000 7f142f65a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1428107df0 0x7f142819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.227+0000 7f142f65a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1428108740 0x7f14281989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.227+0000 7f142f65a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14281122d0 con 0x7f1428108740 2024-05-14T09:36:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.227+0000 7f142dbf7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1428108740 0x7f14281989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.227+0000 7f142dbf7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1428108740 0x7f14281989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41740/0 (socket says 172.21.15.2:41740) 2024-05-14T09:36:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.227+0000 7f142dbf7700 1 -- 172.21.15.2:0/2386494229 learned_addr learned my addr 172.21.15.2:0/2386494229 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.227+0000 7f142d3f6700 1 --2- 172.21.15.2:0/2386494229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14281034c0 0x7f142819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.227+0000 7f142f65a700 1 -- 172.21.15.2:0/2386494229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1428112150 con 0x7f14281034c0 2024-05-14T09:36:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.227+0000 7f142cbf5700 1 --2- 172.21.15.2:0/2386494229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1428107df0 0x7f142819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.227+0000 7f142f65a700 1 -- 172.21.15.2:0/2386494229 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1428112450 con 0x7f1428107df0 2024-05-14T09:36:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.228+0000 7f142dbf7700 1 -- 172.21.15.2:0/2386494229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1428107df0 msgr2=0x7f142819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:43.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.228+0000 7f142dbf7700 1 --2- 172.21.15.2:0/2386494229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1428107df0 0x7f142819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.228+0000 7f142dbf7700 1 -- 172.21.15.2:0/2386494229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14281034c0 msgr2=0x7f142819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:43.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.228+0000 7f142dbf7700 1 --2- 172.21.15.2:0/2386494229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14281034c0 0x7f142819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.228+0000 7f142dbf7700 1 -- 172.21.15.2:0/2386494229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1428199080 con 0x7f1428108740 2024-05-14T09:36:43.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.228+0000 7f142d3f6700 1 --2- 172.21.15.2:0/2386494229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14281034c0 0x7f142819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:43.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.228+0000 7f142dbf7700 1 --2- 172.21.15.2:0/2386494229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1428108740 0x7f14281989d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f142000e940 tx=0x7f142000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:43.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.231+0000 7f141e7fc700 1 -- 172.21.15.2:0/2386494229 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420004a30 con 0x7f1428108740 2024-05-14T09:36:43.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.231+0000 7f142f65a700 1 -- 172.21.15.2:0/2386494229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1428199370 con 0x7f1428108740 2024-05-14T09:36:43.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.231+0000 7f141e7fc700 1 -- 172.21.15.2:0/2386494229 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14200054d0 con 0x7f1428108740 2024-05-14T09:36:43.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.231+0000 7f141e7fc700 1 -- 172.21.15.2:0/2386494229 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142000ccf0 con 0x7f1428108740 2024-05-14T09:36:43.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.231+0000 7f142f65a700 1 -- 172.21.15.2:0/2386494229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14281a5640 con 0x7f1428108740 2024-05-14T09:36:43.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.235+0000 7f141e7fc700 1 -- 172.21.15.2:0/2386494229 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1420005020 con 0x7f1428108740 2024-05-14T09:36:43.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.236+0000 7f141e7fc700 1 --2- 172.21.15.2:0/2386494229 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14140b2880 0x7f14140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.236+0000 7f142d3f6700 1 --2- 172.21.15.2:0/2386494229 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14140b2880 0x7f14140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.236+0000 7f141e7fc700 1 -- 172.21.15.2:0/2386494229 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f14200e2720 con 0x7f1428108740 2024-05-14T09:36:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.236+0000 7f143e06eb80 1 Processor -- start 2024-05-14T09:36:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.236+0000 7f143e06eb80 1 -- start start 2024-05-14T09:36:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.236+0000 7f142d3f6700 1 --2- 172.21.15.2:0/2386494229 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14140b2880 0x7f14140b4d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f142400bfd0 tx=0x7f142400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f143e06eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b802840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f143e06eb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 0x56063b7fc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f143e06eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b7fce10 0x56063b7fd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f143e06eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56063b71e940 con 0x56063b7fce10 2024-05-14T09:36:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f143e06eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56063b683da0 con 0x56063b67ee40 2024-05-14T09:36:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f143e06eb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56063b71a900 con 0x56063b802d80 2024-05-14T09:36:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f142d3f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b802840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f142d3f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b802840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:40908/0 (socket says 172.21.15.2:40908) 2024-05-14T09:36:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f142d3f6700 1 -- 172.21.15.2:0/1538259663 learned_addr learned my addr 172.21.15.2:0/1538259663 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f142dbf7700 1 --2- 172.21.15.2:0/1538259663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b7fce10 0x56063b7fd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f142cbf5700 1 --2- 172.21.15.2:0/1538259663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 0x56063b7fc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f142d3f6700 1 -- 172.21.15.2:0/1538259663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 msgr2=0x56063b7fc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f142d3f6700 1 --2- 172.21.15.2:0/1538259663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 0x56063b7fc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f142d3f6700 1 -- 172.21.15.2:0/1538259663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b7fce10 msgr2=0x56063b7fd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f142d3f6700 1 --2- 172.21.15.2:0/1538259663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b7fce10 0x56063b7fd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.237+0000 7f142d3f6700 1 -- 172.21.15.2:0/1538259663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56063b66b8d0 con 0x56063b67ee40 2024-05-14T09:36:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.238+0000 7f142d3f6700 1 --2- 172.21.15.2:0/1538259663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b802840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1424015000 tx=0x7f142401e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af86c85bb57f7e2f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:43.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.238+0000 7f14137fe700 1 -- 172.21.15.2:0/1538259663 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14240077c0 con 0x56063b67ee40 2024-05-14T09:36:43.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.238+0000 7f14137fe700 1 -- 172.21.15.2:0/1538259663 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1424029450 con 0x56063b67ee40 2024-05-14T09:36:43.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.238+0000 7f14137fe700 1 -- 172.21.15.2:0/1538259663 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14240052a0 con 0x56063b67ee40 2024-05-14T09:36:43.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.239+0000 7f143e06eb80 1 -- 172.21.15.2:0/1538259663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 msgr2=0x56063b802840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:43.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.239+0000 7f143e06eb80 1 --2- 172.21.15.2:0/1538259663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b802840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1424015000 tx=0x7f142401e000 comp rx=0 tx=0).stop 2024-05-14T09:36:43.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.239+0000 7f143e06eb80 1 -- 172.21.15.2:0/1538259663 shutdown_connections 2024-05-14T09:36:43.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.239+0000 7f143e06eb80 1 --2- 172.21.15.2:0/1538259663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b802840 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.239+0000 7f143e06eb80 1 --2- 172.21.15.2:0/1538259663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b7fce10 0x56063b7fd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.239+0000 7f143e06eb80 1 --2- 172.21.15.2:0/1538259663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 0x56063b7fc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.239+0000 7f143e06eb80 1 -- 172.21.15.2:0/1538259663 >> 172.21.15.2:0/1538259663 conn(0x56063b6c6630 msgr2=0x56063b7f7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.239+0000 7f143e06eb80 1 -- 172.21.15.2:0/1538259663 shutdown_connections 2024-05-14T09:36:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.239+0000 7f143e06eb80 1 -- 172.21.15.2:0/1538259663 wait complete. 2024-05-14T09:36:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.239+0000 7f143e06eb80 1 -- start start 2024-05-14T09:36:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.240+0000 7f143e06eb80 1 Processor -- start 2024-05-14T09:36:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.240+0000 7f143e06eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b8064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.240+0000 7f143e06eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b7fce10 0x56063b806a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.240+0000 7f143e06eb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 0x56063b806f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.240+0000 7f143e06eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56063b80dbd0 con 0x56063b7fce10 2024-05-14T09:36:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.240+0000 7f143e06eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56063b6d7ed0 con 0x56063b67ee40 2024-05-14T09:36:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.240+0000 7f143e06eb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56063b80dd50 con 0x56063b802d80 2024-05-14T09:36:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.240+0000 7f143e06eb80 10 client.0 _ll_register_callbacks cb 0x56063b6b6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.240+0000 7f142d3f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b8064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.240+0000 7f142cbf5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b7fce10 0x56063b806a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.240+0000 7f142dbf7700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 0x56063b806f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.240+0000 7f142d3f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b8064e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:40920/0 (socket says 172.21.15.2:40920) 2024-05-14T09:36:43.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.240+0000 7f142d3f6700 1 -- 172.21.15.2:0/3703732859 learned_addr learned my addr 172.21.15.2:0/3703732859 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:43.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f142dbf7700 1 -- 172.21.15.2:0/3703732859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 msgr2=0x56063b8064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:43.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f142dbf7700 1 --2- 172.21.15.2:0/3703732859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b8064e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f142dbf7700 1 -- 172.21.15.2:0/3703732859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b7fce10 msgr2=0x56063b806a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:43.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f142dbf7700 1 --2- 172.21.15.2:0/3703732859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b7fce10 0x56063b806a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f142dbf7700 1 -- 172.21.15.2:0/3703732859 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56063b6d5760 con 0x56063b802d80 2024-05-14T09:36:43.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f142cbf5700 1 --2- 172.21.15.2:0/3703732859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b7fce10 0x56063b806a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:43.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f142d3f6700 1 --2- 172.21.15.2:0/3703732859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b8064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:43.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f142dbf7700 1 --2- 172.21.15.2:0/3703732859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 0x56063b806f60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f14200dfe00 tx=0x7f14200a50b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0fb5f282a5eb279 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:43.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f1410ff9700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:36:43.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f1410ff9700 1 -- 172.21.15.2:0/3703732859 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14200c3060 con 0x56063b802d80 2024-05-14T09:36:43.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f1410ff9700 1 -- 172.21.15.2:0/3703732859 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f14200e23e0 con 0x56063b802d80 2024-05-14T09:36:43.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f143e06eb80 1 -- 172.21.15.2:0/3703732859 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1424029450 con 0x56063b802d80 2024-05-14T09:36:43.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f1410ff9700 1 -- 172.21.15.2:0/3703732859 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1420005310 con 0x56063b802d80 2024-05-14T09:36:43.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f143e06eb80 10 client.25360 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:43.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f143e06eb80 1 -- 172.21.15.2:0/3703732859 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56063b818800 con 0x56063b802d80 2024-05-14T09:36:43.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.241+0000 7f1410ff9700 1 -- 172.21.15.2:0/3703732859 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1424029450 con 0x56063b802d80 2024-05-14T09:36:43.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.242+0000 7f1410ff9700 1 -- 172.21.15.2:0/3703732859 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f14200cc0e0 con 0x56063b802d80 2024-05-14T09:36:43.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.242+0000 7f143e06eb80 10 client.25360 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:43.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.242+0000 7f143e06eb80 10 client.25360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:43.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.242+0000 7f143e06eb80 20 client.25360 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:43.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.242+0000 7f143e06eb80 1 --2- 172.21.15.2:0/3703732859 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56063b81ae40 0x56063b81d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.242+0000 7f143e06eb80 4 client.25360 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:36:43.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.242+0000 7f143e06eb80 1 -- 172.21.15.2:0/3703732859 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56063b72c320 con 0x56063b81ae40 2024-05-14T09:36:43.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.242+0000 7f142d3f6700 1 --2- 172.21.15.2:0/3703732859 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56063b81ae40 0x56063b81d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.242+0000 7f1410ff9700 10 client.25360 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:43.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.242+0000 7f142d3f6700 1 --2- 172.21.15.2:0/3703732859 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56063b81ae40 0x56063b81d260 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7299bfc2cd93f52a server_cookie=4d24f29eb4d6930b in_seq=0 out_seq=0 2024-05-14T09:36:43.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.243+0000 7f1410ff9700 1 -- 172.21.15.2:0/3703732859 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56063b72c320 con 0x56063b81ae40 2024-05-14T09:36:43.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.243+0000 7f1410ff9700 10 client.25360 handle_command_reply: tid=0 2024-05-14T09:36:43.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.243+0000 7f143e06eb80 1 client.25360 shutdown 2024-05-14T09:36:43.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.243+0000 7f143e06eb80 1 -- 172.21.15.2:0/3703732859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 msgr2=0x56063b806f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:43.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.243+0000 7f143e06eb80 1 --2- 172.21.15.2:0/3703732859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 0x56063b806f60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f14200dfe00 tx=0x7f14200a50b0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.243+0000 7f143e06eb80 1 -- 172.21.15.2:0/3703732859 shutdown_connections 2024-05-14T09:36:43.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.243+0000 7f143e06eb80 1 --2- 172.21.15.2:0/3703732859 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56063b81ae40 0x56063b81d260 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.243+0000 7f143e06eb80 1 --2- 172.21.15.2:0/3703732859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b8064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.243+0000 7f143e06eb80 1 --2- 172.21.15.2:0/3703732859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 0x56063b806f60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.243+0000 7f143e06eb80 1 --2- 172.21.15.2:0/3703732859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b7fce10 0x56063b806a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.243+0000 7f143e06eb80 1 -- 172.21.15.2:0/3703732859 >> 172.21.15.2:0/3703732859 conn(0x56063b6c6630 msgr2=0x56063b7fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:43.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.244+0000 7f1410ff9700 0 client.25360 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:43.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.244+0000 7f143e06eb80 1 -- 172.21.15.2:0/3703732859 shutdown_connections 2024-05-14T09:36:43.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.244+0000 7f143e06eb80 1 -- 172.21.15.2:0/3703732859 wait complete. 2024-05-14T09:36:43.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.244+0000 7f143e06eb80 20 client.25360 trim_cache size 0 max 16384 2024-05-14T09:36:43.252 INFO:tasks.workunit.client.0.smithi002.stdout: 1 25664 17.23 MB/sec warmup 9 sec latency 19.052 ms 2024-05-14T09:36:43.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.259+0000 7f143e06eb80 1 Processor -- start 2024-05-14T09:36:43.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.259+0000 7f143e06eb80 1 -- start start 2024-05-14T09:36:43.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.259+0000 7f143e06eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b7fc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.259+0000 7f143e06eb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b7fce10 0x56063b806da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.259+0000 7f143e06eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b802d80 0x56063b7f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.259+0000 7f143e06eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b0b30 con 0x56063b802d80 2024-05-14T09:36:43.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.259+0000 7f143e06eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f14281ae3c0 con 0x56063b67ee40 2024-05-14T09:36:43.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.259+0000 7f143e06eb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f14281ac210 con 0x56063b7fce10 2024-05-14T09:36:43.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.260+0000 7f142d3f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b7fc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.260+0000 7f142d3f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b7fc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:40934/0 (socket says 172.21.15.2:40934) 2024-05-14T09:36:43.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.260+0000 7f142cbf5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b7fce10 0x56063b806da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.260+0000 7f142d3f6700 1 -- 172.21.15.2:0/3873638647 learned_addr learned my addr 172.21.15.2:0/3873638647 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:43.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.260+0000 7f142dbf7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b802d80 0x56063b7f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.260+0000 7f142d3f6700 1 -- 172.21.15.2:0/3873638647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b7fce10 msgr2=0x56063b806da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:43.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.260+0000 7f142d3f6700 1 --2- 172.21.15.2:0/3873638647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b7fce10 0x56063b806da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.260+0000 7f142d3f6700 1 -- 172.21.15.2:0/3873638647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b802d80 msgr2=0x56063b7f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:43.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.260+0000 7f142d3f6700 1 --2- 172.21.15.2:0/3873638647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b802d80 0x56063b7f1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.260+0000 7f142d3f6700 1 -- 172.21.15.2:0/3873638647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56063b804a50 con 0x56063b67ee40 2024-05-14T09:36:43.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.260+0000 7f142cbf5700 1 --2- 172.21.15.2:0/3873638647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b7fce10 0x56063b806da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:43.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.260+0000 7f142d3f6700 1 --2- 172.21.15.2:0/3873638647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b7fc8d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1424029d80 tx=0x7f142400b3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2724c654221efbbb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:43.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.260+0000 7f1412ffd700 1 -- 172.21.15.2:0/3873638647 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424042070 con 0x56063b67ee40 2024-05-14T09:36:43.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.260+0000 7f1412ffd700 1 -- 172.21.15.2:0/3873638647 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1424029e50 con 0x56063b67ee40 2024-05-14T09:36:43.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.260+0000 7f1412ffd700 1 -- 172.21.15.2:0/3873638647 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424004040 con 0x56063b67ee40 2024-05-14T09:36:43.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.261+0000 7f143e06eb80 1 -- 172.21.15.2:0/3873638647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 msgr2=0x56063b7fc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:43.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.261+0000 7f143e06eb80 1 --2- 172.21.15.2:0/3873638647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b7fc8d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1424029d80 tx=0x7f142400b3e0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.261+0000 7f143e06eb80 1 -- 172.21.15.2:0/3873638647 shutdown_connections 2024-05-14T09:36:43.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.261+0000 7f143e06eb80 1 --2- 172.21.15.2:0/3873638647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b67ee40 0x56063b7fc8d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.261+0000 7f143e06eb80 1 --2- 172.21.15.2:0/3873638647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b802d80 0x56063b7f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.261+0000 7f143e06eb80 1 --2- 172.21.15.2:0/3873638647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b7fce10 0x56063b806da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.261+0000 7f143e06eb80 1 -- 172.21.15.2:0/3873638647 >> 172.21.15.2:0/3873638647 conn(0x56063b6c6630 msgr2=0x56063b7f1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:43.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.261+0000 7f143e06eb80 1 -- 172.21.15.2:0/3873638647 shutdown_connections 2024-05-14T09:36:43.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.261+0000 7f143e06eb80 1 -- 172.21.15.2:0/3873638647 wait complete. 2024-05-14T09:36:43.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.262+0000 7f143e06eb80 1 -- start start 2024-05-14T09:36:43.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.262+0000 7f143e06eb80 1 Processor -- start 2024-05-14T09:36:43.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f143e06eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b67ee40 0x56063b8131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f143e06eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b7fce10 0x56063b821a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f143e06eb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 0x56063b821f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f143e06eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14281ac880 con 0x56063b67ee40 2024-05-14T09:36:43.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f143e06eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f14281adcf0 con 0x56063b7fce10 2024-05-14T09:36:43.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f143e06eb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f14281acf50 con 0x56063b802d80 2024-05-14T09:36:43.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f143e06eb80 10 client.0 _ll_register_callbacks cb 0x56063b6b6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:43.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f142cbf5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b7fce10 0x56063b821a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f142cbf5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b7fce10 0x56063b821a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:40936/0 (socket says 172.21.15.2:40936) 2024-05-14T09:36:43.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f142cbf5700 1 -- 172.21.15.2:0/4221592731 learned_addr learned my addr 172.21.15.2:0/4221592731 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:43.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f142d3f6700 1 --2- 172.21.15.2:0/4221592731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b67ee40 0x56063b8131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f142dbf7700 1 --2- 172.21.15.2:0/4221592731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 0x56063b821f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f142cbf5700 1 -- 172.21.15.2:0/4221592731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 msgr2=0x56063b821f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:43.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f142cbf5700 1 --2- 172.21.15.2:0/4221592731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 0x56063b821f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f142cbf5700 1 -- 172.21.15.2:0/4221592731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b67ee40 msgr2=0x56063b8131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:43.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f142cbf5700 1 --2- 172.21.15.2:0/4221592731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b67ee40 0x56063b8131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f142cbf5700 1 -- 172.21.15.2:0/4221592731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56063b822490 con 0x56063b7fce10 2024-05-14T09:36:43.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f142dbf7700 1 --2- 172.21.15.2:0/4221592731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 0x56063b821f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:43.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f142cbf5700 1 --2- 172.21.15.2:0/4221592731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b7fce10 0x56063b821a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f14281063e0 tx=0x7f1418018f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb2b9449077d98fe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:43.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.263+0000 7f1410ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:36:43.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.264+0000 7f1410ff9700 1 -- 172.21.15.2:0/4221592731 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1418015b70 con 0x56063b7fce10 2024-05-14T09:36:43.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.264+0000 7f143e06eb80 1 -- 172.21.15.2:0/4221592731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1424029e50 con 0x56063b7fce10 2024-05-14T09:36:43.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.264+0000 7f1410ff9700 1 -- 172.21.15.2:0/4221592731 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1418015d10 con 0x56063b7fce10 2024-05-14T09:36:43.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.264+0000 7f1410ff9700 1 -- 172.21.15.2:0/4221592731 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f141802b050 con 0x56063b7fce10 2024-05-14T09:36:43.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.264+0000 7f143e06eb80 10 client.25367 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:43.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.264+0000 7f143e06eb80 1 -- 172.21.15.2:0/4221592731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56063b8095e0 con 0x56063b7fce10 2024-05-14T09:36:43.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.264+0000 7f1410ff9700 1 -- 172.21.15.2:0/4221592731 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1424029e50 con 0x56063b7fce10 2024-05-14T09:36:43.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.264+0000 7f1410ff9700 1 -- 172.21.15.2:0/4221592731 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1418022460 con 0x56063b7fce10 2024-05-14T09:36:43.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.264+0000 7f143e06eb80 10 client.25367 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:43.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.264+0000 7f143e06eb80 10 client.25367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:43.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.265+0000 7f143e06eb80 20 client.25367 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:43.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.265+0000 7f143e06eb80 1 --2- 172.21.15.2:0/4221592731 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56063b809cd0 0x56063b80c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:43.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.265+0000 7f143e06eb80 4 client.25367 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T09:36:43.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.265+0000 7f143e06eb80 1 -- 172.21.15.2:0/4221592731 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56063b7263c0 con 0x56063b809cd0 2024-05-14T09:36:43.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.265+0000 7f142d3f6700 1 --2- 172.21.15.2:0/4221592731 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56063b809cd0 0x56063b80c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:43.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.266+0000 7f142d3f6700 1 --2- 172.21.15.2:0/4221592731 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56063b809cd0 0x56063b80c0f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=202f786a9857f299 server_cookie=925b69001529bbab in_seq=0 out_seq=0 2024-05-14T09:36:43.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.266+0000 7f1410ff9700 10 client.25367 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:43.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.266+0000 7f1410ff9700 1 -- 172.21.15.2:0/4221592731 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56063b7263c0 con 0x56063b809cd0 2024-05-14T09:36:43.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.266+0000 7f1410ff9700 10 client.25367 handle_command_reply: tid=0 2024-05-14T09:36:43.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.267+0000 7f143e06eb80 1 client.25367 shutdown 2024-05-14T09:36:43.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.267+0000 7f143e06eb80 1 -- 172.21.15.2:0/4221592731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b7fce10 msgr2=0x56063b821a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:43.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.267+0000 7f143e06eb80 1 --2- 172.21.15.2:0/4221592731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b7fce10 0x56063b821a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f14281063e0 tx=0x7f1418018f50 comp rx=0 tx=0).stop 2024-05-14T09:36:43.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.267+0000 7f143e06eb80 1 -- 172.21.15.2:0/4221592731 shutdown_connections 2024-05-14T09:36:43.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.267+0000 7f143e06eb80 1 --2- 172.21.15.2:0/4221592731 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56063b809cd0 0x56063b80c0f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.267+0000 7f143e06eb80 1 --2- 172.21.15.2:0/4221592731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56063b802d80 0x56063b821f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.267+0000 7f143e06eb80 1 --2- 172.21.15.2:0/4221592731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56063b67ee40 0x56063b8131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.267+0000 7f143e06eb80 1 --2- 172.21.15.2:0/4221592731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063b7fce10 0x56063b821a10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.267+0000 7f143e06eb80 1 -- 172.21.15.2:0/4221592731 >> 172.21.15.2:0/4221592731 conn(0x56063b6c6630 msgr2=0x56063b804dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:43.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.267+0000 7f1410ff9700 0 client.25367 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:43.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.267+0000 7f143e06eb80 1 -- 172.21.15.2:0/4221592731 shutdown_connections 2024-05-14T09:36:43.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.267+0000 7f143e06eb80 1 -- 172.21.15.2:0/4221592731 wait complete. 2024-05-14T09:36:43.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.267+0000 7f143e06eb80 20 client.25367 trim_cache size 0 max 16384 2024-05-14T09:36:43.271 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-05-14T09:36:43.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.268+0000 7f142f65a700 1 -- 172.21.15.2:0/2386494229 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14140b2880 msgr2=0x7f14140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:43.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.268+0000 7f142f65a700 1 --2- 172.21.15.2:0/2386494229 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14140b2880 0x7f14140b4d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f142400bfd0 tx=0x7f142400b040 comp rx=0 tx=0).stop 2024-05-14T09:36:43.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.268+0000 7f142f65a700 1 -- 172.21.15.2:0/2386494229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1428108740 msgr2=0x7f14281989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:43.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.268+0000 7f142f65a700 1 --2- 172.21.15.2:0/2386494229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1428108740 0x7f14281989d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f142000e940 tx=0x7f142000ec50 comp rx=0 tx=0).stop 2024-05-14T09:36:43.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.268+0000 7f142f65a700 1 -- 172.21.15.2:0/2386494229 shutdown_connections 2024-05-14T09:36:43.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.268+0000 7f142f65a700 1 --2- 172.21.15.2:0/2386494229 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14140b2880 0x7f14140b4d90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.268+0000 7f142f65a700 1 --2- 172.21.15.2:0/2386494229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14281034c0 0x7f142819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.268+0000 7f142f65a700 1 --2- 172.21.15.2:0/2386494229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1428108740 0x7f14281989d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.268+0000 7f142f65a700 1 --2- 172.21.15.2:0/2386494229 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1428107df0 0x7f142819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:43.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.268+0000 7f142f65a700 1 -- 172.21.15.2:0/2386494229 >> 172.21.15.2:0/2386494229 conn(0x7f14280fedc0 msgr2=0x7f1428101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:43.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.269+0000 7f142f65a700 1 -- 172.21.15.2:0/2386494229 shutdown_connections 2024-05-14T09:36:43.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:43.269+0000 7f142f65a700 1 -- 172.21.15.2:0/2386494229 wait complete. 2024-05-14T09:36:44.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 28730 17.35 MB/sec warmup 10 sec latency 14.776 ms 2024-05-14T09:36:44.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:44 smithi002 ceph-mon[31557]: pgmap v477: 129 pgs: 129 active+clean; 11 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-05-14T09:36:44.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:44 smithi029 ceph-mon[36249]: pgmap v477: 129 pgs: 129 active+clean; 11 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-05-14T09:36:44.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:44 smithi026 ceph-mon[36056]: pgmap v477: 129 pgs: 129 active+clean; 11 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-05-14T09:36:45.127 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T09:36:45.127 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T09:36:45.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 31439 16.94 MB/sec warmup 11 sec latency 21.206 ms 2024-05-14T09:36:45.387 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:36:46.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 33487 16.50 MB/sec warmup 12 sec latency 100.889 ms 2024-05-14T09:36:46.526 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:46 smithi002 ceph-mon[31557]: pgmap v478: 129 pgs: 129 active+clean; 11 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-05-14T09:36:46.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:46 smithi029 ceph-mon[36249]: pgmap v478: 129 pgs: 129 active+clean; 11 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-05-14T09:36:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:46 smithi026 ceph-mon[36056]: pgmap v478: 129 pgs: 129 active+clean; 11 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-05-14T09:36:47.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 36229 16.33 MB/sec warmup 13 sec latency 31.123 ms 2024-05-14T09:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.555+0000 7f0313c27640 1 -- 172.21.15.2:0/601974691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f030c102b50 msgr2=0x7f030c103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.555+0000 7f0313c27640 1 --2- 172.21.15.2:0/601974691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f030c102b50 0x7f030c103850 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0308009ca0 tx=0x7f030802a8c0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.556+0000 7f0313c27640 1 -- 172.21.15.2:0/601974691 shutdown_connections 2024-05-14T09:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.556+0000 7f0313c27640 1 --2- 172.21.15.2:0/601974691 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f030c108a90 0x7f030c10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.556+0000 7f0313c27640 1 --2- 172.21.15.2:0/601974691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030c103d90 0x7f030c1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.556+0000 7f0313c27640 1 --2- 172.21.15.2:0/601974691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f030c102b50 0x7f030c103850 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.556+0000 7f0313c27640 1 -- 172.21.15.2:0/601974691 >> 172.21.15.2:0/601974691 conn(0x7f030c0fdd40 msgr2=0x7f030c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.556+0000 7f0313c27640 1 -- 172.21.15.2:0/601974691 shutdown_connections 2024-05-14T09:36:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.556+0000 7f0313c27640 1 -- 172.21.15.2:0/601974691 wait complete. 2024-05-14T09:36:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.556+0000 7f0313c27640 1 Processor -- start 2024-05-14T09:36:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.556+0000 7f0313c27640 1 -- start start 2024-05-14T09:36:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f0313c27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030c102b50 0x7f030c10d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f0313c27640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f030c103d90 0x7f030c10bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f0313c27640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f030c108a90 0x7f030c10c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f0313c27640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f030c10f780 con 0x7f030c108a90 2024-05-14T09:36:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f0313c27640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f030c10f600 con 0x7f030c102b50 2024-05-14T09:36:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f0313c27640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f030c10f900 con 0x7f030c103d90 2024-05-14T09:36:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f03121c4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f030c108a90 0x7f030c10c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f03121c4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f030c108a90 0x7f030c10c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41798/0 (socket says 172.21.15.2:41798) 2024-05-14T09:36:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f03121c4640 1 -- 172.21.15.2:0/2270047665 learned_addr learned my addr 172.21.15.2:0/2270047665 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f03119c3640 1 --2- 172.21.15.2:0/2270047665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030c102b50 0x7f030c10d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f03111c2640 1 --2- 172.21.15.2:0/2270047665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f030c103d90 0x7f030c10bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f03121c4640 1 -- 172.21.15.2:0/2270047665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f030c103d90 msgr2=0x7f030c10bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f03121c4640 1 --2- 172.21.15.2:0/2270047665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f030c103d90 0x7f030c10bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f03121c4640 1 -- 172.21.15.2:0/2270047665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030c102b50 msgr2=0x7f030c10d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f03121c4640 1 --2- 172.21.15.2:0/2270047665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030c102b50 0x7f030c10d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f03121c4640 1 -- 172.21.15.2:0/2270047665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c10c730 con 0x7f030c108a90 2024-05-14T09:36:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f03121c4640 1 --2- 172.21.15.2:0/2270047665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f030c108a90 0x7f030c10c000 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f030400e9b0 tx=0x7f030400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f0302ffd640 1 -- 172.21.15.2:0/2270047665 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030400cd80 con 0x7f030c108a90 2024-05-14T09:36:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f0302ffd640 1 -- 172.21.15.2:0/2270047665 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03040035f0 con 0x7f030c108a90 2024-05-14T09:36:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.557+0000 7f0302ffd640 1 -- 172.21.15.2:0/2270047665 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304010690 con 0x7f030c108a90 2024-05-14T09:36:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.558+0000 7f0313c27640 1 -- 172.21.15.2:0/2270047665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030c1a43b0 con 0x7f030c108a90 2024-05-14T09:36:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.558+0000 7f0313c27640 1 -- 172.21.15.2:0/2270047665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030c1a4820 con 0x7f030c108a90 2024-05-14T09:36:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.560+0000 7f0302ffd640 1 -- 172.21.15.2:0/2270047665 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0304004020 con 0x7f030c108a90 2024-05-14T09:36:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.561+0000 7f031abb8740 1 Processor -- start 2024-05-14T09:36:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.561+0000 7f031abb8740 1 -- start start 2024-05-14T09:36:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.561+0000 7f031abb8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a44851b0 0x5577a4826a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.561+0000 7f031abb8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a445c2f0 0x5577a4826fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.561+0000 7f031abb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a44644d0 0x5577a482dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.561+0000 7f031abb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5577a478b5d0 con 0x5577a44644d0 2024-05-14T09:36:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.561+0000 7f031abb8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5577a4744890 con 0x5577a445c2f0 2024-05-14T09:36:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.561+0000 7f031abb8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5577a4787590 con 0x5577a44851b0 2024-05-14T09:36:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.561+0000 7f03111c2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a44851b0 0x5577a4826a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.561+0000 7f03119c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a445c2f0 0x5577a4826fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.561+0000 7f03119c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a445c2f0 0x5577a4826fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:40962/0 (socket says 172.21.15.2:40962) 2024-05-14T09:36:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.561+0000 7f03111c2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a44851b0 0x5577a4826a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:39592/0 (socket says 172.21.15.2:39592) 2024-05-14T09:36:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.561+0000 7f03119c3640 1 -- 172.21.15.2:0/566518021 learned_addr learned my addr 172.21.15.2:0/566518021 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.561+0000 7f03111c2640 1 --2- 172.21.15.2:0/566518021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a44644d0 0x5577a482dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.561+0000 7f0302ffd640 1 --2- 172.21.15.2:0/2270047665 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f02f80b28a0 0x7f02f80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.562+0000 7f03119c3640 1 -- 172.21.15.2:0/566518021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a44851b0 msgr2=0x5577a4826a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.562+0000 7f03119c3640 1 --2- 172.21.15.2:0/566518021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a44851b0 0x5577a4826a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.562+0000 7f03119c3640 1 -- 172.21.15.2:0/566518021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a44644d0 msgr2=0x5577a482dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.562+0000 7f03119c3640 1 --2- 172.21.15.2:0/566518021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a44644d0 0x5577a482dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.562+0000 7f03119c3640 1 -- 172.21.15.2:0/566518021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577a449d210 con 0x5577a445c2f0 2024-05-14T09:36:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.562+0000 7f03111c2640 1 --2- 172.21.15.2:0/566518021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a44851b0 0x5577a4826a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.562+0000 7f03121c4640 1 --2- 172.21.15.2:0/2270047665 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f02f80b28a0 0x7f02f80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.562+0000 7f0302ffd640 1 -- 172.21.15.2:0/2270047665 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f03040e2960 con 0x7f030c108a90 2024-05-14T09:36:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.562+0000 7f03119c3640 1 --2- 172.21.15.2:0/566518021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a445c2f0 0x5577a4826fe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f030c102420 tx=0x7f0308004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84e971ff60d2c496 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.562+0000 7f03121c4640 1 --2- 172.21.15.2:0/2270047665 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f02f80b28a0 0x7f02f80b4db0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f030406c3c0 tx=0x7f0304069d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.562+0000 7f02f7fff640 1 -- 172.21.15.2:0/566518021 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308031650 con 0x5577a445c2f0 2024-05-14T09:36:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.562+0000 7f02f7fff640 1 -- 172.21.15.2:0/566518021 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0308036030 con 0x5577a445c2f0 2024-05-14T09:36:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.562+0000 7f02f7fff640 1 -- 172.21.15.2:0/566518021 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308004570 con 0x5577a445c2f0 2024-05-14T09:36:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.563+0000 7f031abb8740 1 -- 172.21.15.2:0/566518021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a445c2f0 msgr2=0x5577a4826fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.563+0000 7f031abb8740 1 --2- 172.21.15.2:0/566518021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a445c2f0 0x5577a4826fe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f030c102420 tx=0x7f0308004ab0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.563+0000 7f031abb8740 1 -- 172.21.15.2:0/566518021 shutdown_connections 2024-05-14T09:36:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.563+0000 7f031abb8740 1 --2- 172.21.15.2:0/566518021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a44644d0 0x5577a482dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.563+0000 7f031abb8740 1 --2- 172.21.15.2:0/566518021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a445c2f0 0x5577a4826fe0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.563+0000 7f031abb8740 1 --2- 172.21.15.2:0/566518021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a44851b0 0x5577a4826a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.563+0000 7f031abb8740 1 -- 172.21.15.2:0/566518021 >> 172.21.15.2:0/566518021 conn(0x5577a4479300 msgr2=0x5577a48223a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.563+0000 7f031abb8740 1 -- 172.21.15.2:0/566518021 shutdown_connections 2024-05-14T09:36:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.563+0000 7f031abb8740 1 -- 172.21.15.2:0/566518021 wait complete. 2024-05-14T09:36:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.564+0000 7f031abb8740 1 -- start start 2024-05-14T09:36:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.564+0000 7f031abb8740 1 Processor -- start 2024-05-14T09:36:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.564+0000 7f031abb8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a445c2f0 0x5577a48304c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.564+0000 7f031abb8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a44644d0 0x5577a4830a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.564+0000 7f031abb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a44851b0 0x5577a4837960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.564+0000 7f031abb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5577a482e750 con 0x5577a44851b0 2024-05-14T09:36:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.564+0000 7f031abb8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5577a4748370 con 0x5577a44644d0 2024-05-14T09:36:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.564+0000 7f031abb8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5577a482e8d0 con 0x5577a445c2f0 2024-05-14T09:36:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.564+0000 7f031abb8740 10 client.0 _ll_register_callbacks cb 0x5577a449ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f03111c2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a44851b0 0x5577a4837960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f03119c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a44644d0 0x5577a4830a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f03111c2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a445c2f0 0x5577a48304c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f03111c2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a445c2f0 0x5577a48304c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:39600/0 (socket says 172.21.15.2:39600) 2024-05-14T09:36:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f03111c2640 1 -- 172.21.15.2:0/495949289 learned_addr learned my addr 172.21.15.2:0/495949289 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f03119c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a44644d0 0x5577a4830a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:40968/0 (socket says 172.21.15.2:40968) 2024-05-14T09:36:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f03111c2640 1 -- 172.21.15.2:0/495949289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a44644d0 msgr2=0x5577a4830a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f03111c2640 1 --2- 172.21.15.2:0/495949289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a44644d0 0x5577a4830a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f03111c2640 1 -- 172.21.15.2:0/495949289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a44851b0 msgr2=0x5577a4837960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f03111c2640 1 --2- 172.21.15.2:0/495949289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a44851b0 0x5577a4837960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f03111c2640 1 -- 172.21.15.2:0/495949289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577a4569620 con 0x5577a445c2f0 2024-05-14T09:36:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f03111c2640 1 --2- 172.21.15.2:0/495949289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a445c2f0 0x5577a48304c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f02fc011c50 tx=0x7f02fc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4beb4a0b59214aa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f02f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:36:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f02f57fa640 1 -- 172.21.15.2:0/495949289 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc00fe60 con 0x5577a445c2f0 2024-05-14T09:36:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f02f57fa640 1 -- 172.21.15.2:0/495949289 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f02fc005020 con 0x5577a445c2f0 2024-05-14T09:36:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.565+0000 7f031abb8740 1 -- 172.21.15.2:0/495949289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0308036030 con 0x5577a445c2f0 2024-05-14T09:36:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.566+0000 7f02f57fa640 1 -- 172.21.15.2:0/495949289 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f02fc005220 con 0x5577a445c2f0 2024-05-14T09:36:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.566+0000 7f031abb8740 10 client.25378 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.566+0000 7f031abb8740 1 -- 172.21.15.2:0/495949289 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5577a483a960 con 0x5577a445c2f0 2024-05-14T09:36:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.566+0000 7f02f57fa640 1 -- 172.21.15.2:0/495949289 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0308036030 con 0x5577a445c2f0 2024-05-14T09:36:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.566+0000 7f02f57fa640 1 -- 172.21.15.2:0/495949289 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f02fc022430 con 0x5577a445c2f0 2024-05-14T09:36:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.566+0000 7f031abb8740 10 client.25378 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.566+0000 7f031abb8740 10 client.25378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.566+0000 7f031abb8740 20 client.25378 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.566+0000 7f031abb8740 1 --2- 172.21.15.2:0/495949289 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5577a4443d80 0x5577a483e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.566+0000 7f031abb8740 4 client.25378 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:36:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.566+0000 7f031abb8740 1 -- 172.21.15.2:0/495949289 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577a47991e0 con 0x5577a4443d80 2024-05-14T09:36:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.566+0000 7f03119c3640 1 --2- 172.21.15.2:0/495949289 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5577a4443d80 0x5577a483e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.567+0000 7f03119c3640 1 --2- 172.21.15.2:0/495949289 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5577a4443d80 0x5577a483e060 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0a061fe20840fc9 server_cookie=4a21710b8a84e276 in_seq=0 out_seq=0 2024-05-14T09:36:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.567+0000 7f02f57fa640 10 client.25378 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.568+0000 7f02f57fa640 1 -- 172.21.15.2:0/495949289 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5577a47991e0 con 0x5577a4443d80 2024-05-14T09:36:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.568+0000 7f02f57fa640 10 client.25378 handle_command_reply: tid=0 2024-05-14T09:36:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.568+0000 7f031abb8740 1 client.25378 shutdown 2024-05-14T09:36:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.568+0000 7f031abb8740 1 -- 172.21.15.2:0/495949289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a445c2f0 msgr2=0x5577a48304c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.568+0000 7f031abb8740 1 --2- 172.21.15.2:0/495949289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a445c2f0 0x5577a48304c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f02fc011c50 tx=0x7f02fc004730 comp rx=0 tx=0).stop 2024-05-14T09:36:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.569+0000 7f031abb8740 1 -- 172.21.15.2:0/495949289 shutdown_connections 2024-05-14T09:36:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.569+0000 7f031abb8740 1 --2- 172.21.15.2:0/495949289 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5577a4443d80 0x5577a483e060 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.569+0000 7f031abb8740 1 --2- 172.21.15.2:0/495949289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a44851b0 0x5577a4837960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.569+0000 7f031abb8740 1 --2- 172.21.15.2:0/495949289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a44644d0 0x5577a4830a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.569+0000 7f02f57fa640 0 client.25378 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.569+0000 7f031abb8740 1 --2- 172.21.15.2:0/495949289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a445c2f0 0x5577a48304c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.569+0000 7f031abb8740 1 -- 172.21.15.2:0/495949289 >> 172.21.15.2:0/495949289 conn(0x5577a4479300 msgr2=0x5577a4825300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.569+0000 7f031abb8740 1 -- 172.21.15.2:0/495949289 shutdown_connections 2024-05-14T09:36:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.569+0000 7f031abb8740 1 -- 172.21.15.2:0/495949289 wait complete. 2024-05-14T09:36:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.569+0000 7f031abb8740 20 client.25378 trim_cache size 0 max 16384 2024-05-14T09:36:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.585+0000 7f031abb8740 1 Processor -- start 2024-05-14T09:36:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.585+0000 7f031abb8740 1 -- start start 2024-05-14T09:36:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.585+0000 7f031abb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a4443d80 0x5577a48307a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.585+0000 7f031abb8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a445c2f0 0x5577a48378f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.585+0000 7f031abb8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a44644d0 0x5577a4837e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.585+0000 7f031abb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f030c1b1730 con 0x5577a4443d80 2024-05-14T09:36:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.585+0000 7f031abb8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f030c1af000 con 0x5577a445c2f0 2024-05-14T09:36:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.585+0000 7f031abb8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f030c1ace50 con 0x5577a44644d0 2024-05-14T09:36:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.585+0000 7f03111c2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a4443d80 0x5577a48307a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.585+0000 7f03119c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a445c2f0 0x5577a48378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.585+0000 7f03111c2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a4443d80 0x5577a48307a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41838/0 (socket says 172.21.15.2:41838) 2024-05-14T09:36:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.585+0000 7f03111c2640 1 -- 172.21.15.2:0/2444519724 learned_addr learned my addr 172.21.15.2:0/2444519724 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.586+0000 7f03111c2640 1 --2- 172.21.15.2:0/2444519724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a44644d0 0x5577a4837e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.586+0000 7f03119c3640 1 -- 172.21.15.2:0/2444519724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a44644d0 msgr2=0x5577a4837e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.586+0000 7f03119c3640 1 --2- 172.21.15.2:0/2444519724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a44644d0 0x5577a4837e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.586+0000 7f03119c3640 1 -- 172.21.15.2:0/2444519724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a4443d80 msgr2=0x5577a48307a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.586+0000 7f03119c3640 1 --2- 172.21.15.2:0/2444519724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a4443d80 0x5577a48307a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.586+0000 7f03119c3640 1 -- 172.21.15.2:0/2444519724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577a4826570 con 0x5577a445c2f0 2024-05-14T09:36:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.586+0000 7f03111c2640 1 --2- 172.21.15.2:0/2444519724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a44644d0 0x5577a4837e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:36:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.586+0000 7f03119c3640 1 --2- 172.21.15.2:0/2444519724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a445c2f0 0x5577a48378f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0308000f80 tx=0x7f0308031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e24f1225e14251c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.586+0000 7f02f77fe640 1 -- 172.21.15.2:0/2444519724 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030804a040 con 0x5577a445c2f0 2024-05-14T09:36:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.586+0000 7f02f77fe640 1 -- 172.21.15.2:0/2444519724 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0308036030 con 0x5577a445c2f0 2024-05-14T09:36:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.586+0000 7f02f77fe640 1 -- 172.21.15.2:0/2444519724 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308004420 con 0x5577a445c2f0 2024-05-14T09:36:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.587+0000 7f031abb8740 1 -- 172.21.15.2:0/2444519724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a445c2f0 msgr2=0x5577a48378f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.587+0000 7f031abb8740 1 --2- 172.21.15.2:0/2444519724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a445c2f0 0x5577a48378f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0308000f80 tx=0x7f0308031a20 comp rx=0 tx=0).stop 2024-05-14T09:36:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.587+0000 7f031abb8740 1 -- 172.21.15.2:0/2444519724 shutdown_connections 2024-05-14T09:36:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.587+0000 7f031abb8740 1 --2- 172.21.15.2:0/2444519724 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a44644d0 0x5577a4837e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.587+0000 7f031abb8740 1 --2- 172.21.15.2:0/2444519724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a445c2f0 0x5577a48378f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.587+0000 7f031abb8740 1 --2- 172.21.15.2:0/2444519724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a4443d80 0x5577a48307a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.587+0000 7f031abb8740 1 -- 172.21.15.2:0/2444519724 >> 172.21.15.2:0/2444519724 conn(0x5577a4479300 msgr2=0x5577a4830090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.587+0000 7f031abb8740 1 -- 172.21.15.2:0/2444519724 shutdown_connections 2024-05-14T09:36:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.588+0000 7f031abb8740 1 -- 172.21.15.2:0/2444519724 wait complete. 2024-05-14T09:36:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.588+0000 7f031abb8740 1 -- start start 2024-05-14T09:36:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.588+0000 7f031abb8740 1 Processor -- start 2024-05-14T09:36:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.588+0000 7f031abb8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a4443d80 0x5577a4834f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.588+0000 7f031abb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a445c2f0 0x5577a4835440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.588+0000 7f031abb8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a44644d0 0x5577a48a4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.588+0000 7f031abb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f030c1ad4c0 con 0x5577a445c2f0 2024-05-14T09:36:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.588+0000 7f031abb8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f030c1ae930 con 0x5577a44644d0 2024-05-14T09:36:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.588+0000 7f031abb8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f030c1adb90 con 0x5577a4443d80 2024-05-14T09:36:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f031abb8740 10 client.0 _ll_register_callbacks cb 0x7f02fc011a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f03111c2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a4443d80 0x5577a4834f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f03119c3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a445c2f0 0x5577a4835440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f03111c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a44644d0 0x5577a48a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f03111c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a44644d0 0x5577a48a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:40990/0 (socket says 172.21.15.2:40990) 2024-05-14T09:36:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f03119c3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a445c2f0 0x5577a4835440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41854/0 (socket says 172.21.15.2:41854) 2024-05-14T09:36:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f03111c2640 1 -- 172.21.15.2:0/330547256 learned_addr learned my addr 172.21.15.2:0/330547256 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f03111c2640 1 -- 172.21.15.2:0/330547256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a4443d80 msgr2=0x5577a4834f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f03111c2640 1 --2- 172.21.15.2:0/330547256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a4443d80 0x5577a4834f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f03111c2640 1 -- 172.21.15.2:0/330547256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a445c2f0 msgr2=0x5577a4835440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f03111c2640 1 --2- 172.21.15.2:0/330547256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a445c2f0 0x5577a4835440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f03111c2640 1 -- 172.21.15.2:0/330547256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577a482e960 con 0x5577a44644d0 2024-05-14T09:36:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f03111c2640 1 --2- 172.21.15.2:0/330547256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a44644d0 0x5577a48a4400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f02fc011c50 tx=0x7f02fc005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8e13ccd933c7cea server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f02f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:36:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f02f57fa640 1 -- 172.21.15.2:0/330547256 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0047a0 con 0x5577a44644d0 2024-05-14T09:36:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f031abb8740 1 -- 172.21.15.2:0/330547256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0308036030 con 0x5577a44644d0 2024-05-14T09:36:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.589+0000 7f02f57fa640 1 -- 172.21.15.2:0/330547256 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f02fc004940 con 0x5577a44644d0 2024-05-14T09:36:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.590+0000 7f02f57fa640 1 -- 172.21.15.2:0/330547256 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f02fc047050 con 0x5577a44644d0 2024-05-14T09:36:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.590+0000 7f031abb8740 10 client.25385 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.590+0000 7f031abb8740 1 -- 172.21.15.2:0/330547256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5577a48a5940 con 0x5577a44644d0 2024-05-14T09:36:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.590+0000 7f02f57fa640 1 -- 172.21.15.2:0/330547256 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0308036030 con 0x5577a44644d0 2024-05-14T09:36:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.590+0000 7f02f57fa640 1 -- 172.21.15.2:0/330547256 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f02fc030650 con 0x5577a44644d0 2024-05-14T09:36:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.590+0000 7f031abb8740 10 client.25385 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.590+0000 7f031abb8740 10 client.25385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.590+0000 7f031abb8740 20 client.25385 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.590+0000 7f031abb8740 1 --2- 172.21.15.2:0/330547256 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5577a44851b0 0x5577a48a8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.590+0000 7f031abb8740 4 client.25385 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:36:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.590+0000 7f031abb8740 1 -- 172.21.15.2:0/330547256 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577a48273e0 con 0x5577a44851b0 2024-05-14T09:36:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.590+0000 7f03119c3640 1 --2- 172.21.15.2:0/330547256 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5577a44851b0 0x5577a48a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.591+0000 7f02f57fa640 10 client.25385 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.591+0000 7f03119c3640 1 --2- 172.21.15.2:0/330547256 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5577a44851b0 0x5577a48a8310 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2edd19aab00c5e27 server_cookie=26f7db2518c9f2f7 in_seq=0 out_seq=0 2024-05-14T09:36:47.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.682+0000 7f1c29ee2700 1 -- 172.21.15.2:0/3198767767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c241039d0 msgr2=0x7f1c24107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.682+0000 7f1c29ee2700 1 --2- 172.21.15.2:0/3198767767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c241039d0 0x7f1c24107f00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1c180098f0 tx=0x7f1c18009c00 comp rx=0 tx=0).stop 2024-05-14T09:36:47.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.684+0000 7f1c29ee2700 1 -- 172.21.15.2:0/3198767767 shutdown_connections 2024-05-14T09:36:47.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.684+0000 7f1c29ee2700 1 --2- 172.21.15.2:0/3198767767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c241084d0 0x7f1c241089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.684+0000 7f1c29ee2700 1 --2- 172.21.15.2:0/3198767767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c24103080 0x7f1c24103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.684+0000 7f1c29ee2700 1 --2- 172.21.15.2:0/3198767767 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c241039d0 0x7f1c24107f00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.684+0000 7f1c29ee2700 1 -- 172.21.15.2:0/3198767767 >> 172.21.15.2:0/3198767767 conn(0x7f1c24078b00 msgr2=0x7f1c24078f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:47.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.688+0000 7f1c29ee2700 1 -- 172.21.15.2:0/3198767767 shutdown_connections 2024-05-14T09:36:47.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.688+0000 7f1c29ee2700 1 -- 172.21.15.2:0/3198767767 wait complete. 2024-05-14T09:36:47.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.689+0000 7f1c29ee2700 1 Processor -- start 2024-05-14T09:36:47.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.690+0000 7f1c29ee2700 1 -- start start 2024-05-14T09:36:47.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.690+0000 7f1c29ee2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c24103080 0x7f1c2410fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.690+0000 7f1c29ee2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c241039d0 0x7f1c24110290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.690+0000 7f1c29ee2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c241084d0 0x7f1c24074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.690+0000 7f1c29ee2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2410bc50 con 0x7f1c241084d0 2024-05-14T09:36:47.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.690+0000 7f1c29ee2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2410bad0 con 0x7f1c24103080 2024-05-14T09:36:47.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.690+0000 7f1c29ee2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2410bdd0 con 0x7f1c241039d0 2024-05-14T09:36:47.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.692+0000 7f1c23fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c241039d0 0x7f1c24110290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.692+0000 7f1c28ee0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c24103080 0x7f1c2410fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.692+0000 7f1c23fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c241039d0 0x7f1c24110290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:39644/0 (socket says 172.21.15.2:39644) 2024-05-14T09:36:47.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.692+0000 7f1c23fff700 1 -- 172.21.15.2:0/1931583014 learned_addr learned my addr 172.21.15.2:0/1931583014 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:47.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.693+0000 7f1c296e1700 1 --2- 172.21.15.2:0/1931583014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c241084d0 0x7f1c24074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.694+0000 7f1c296e1700 1 -- 172.21.15.2:0/1931583014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c241039d0 msgr2=0x7f1c24110290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.694+0000 7f1c296e1700 1 --2- 172.21.15.2:0/1931583014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c241039d0 0x7f1c24110290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.694+0000 7f1c296e1700 1 -- 172.21.15.2:0/1931583014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c24103080 msgr2=0x7f1c2410fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.694+0000 7f1c296e1700 1 --2- 172.21.15.2:0/1931583014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c24103080 0x7f1c2410fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.694+0000 7f1c296e1700 1 -- 172.21.15.2:0/1931583014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c24074af0 con 0x7f1c241084d0 2024-05-14T09:36:47.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.694+0000 7f1c296e1700 1 --2- 172.21.15.2:0/1931583014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c241084d0 0x7f1c24074530 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c01ae90 tx=0x7f1c1c01e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:47.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.694+0000 7f1c28ee0700 1 --2- 172.21.15.2:0/1931583014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c24103080 0x7f1c2410fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:47.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.695+0000 7f1c21ffb700 1 -- 172.21.15.2:0/1931583014 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c004bb0 con 0x7f1c241084d0 2024-05-14T09:36:47.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.695+0000 7f1c21ffb700 1 -- 172.21.15.2:0/1931583014 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c1c004d50 con 0x7f1c241084d0 2024-05-14T09:36:47.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.695+0000 7f1c21ffb700 1 -- 172.21.15.2:0/1931583014 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0044c0 con 0x7f1c241084d0 2024-05-14T09:36:47.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.697+0000 7f1c29ee2700 1 -- 172.21.15.2:0/1931583014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c24074de0 con 0x7f1c241084d0 2024-05-14T09:36:47.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.698+0000 7f1c29ee2700 1 -- 172.21.15.2:0/1931583014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c240752a0 con 0x7f1c241084d0 2024-05-14T09:36:47.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.705+0000 7f1c21ffb700 1 -- 172.21.15.2:0/1931583014 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1c1c003b20 con 0x7f1c241084d0 2024-05-14T09:36:47.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.709+0000 7f1c39b58b80 1 Processor -- start 2024-05-14T09:36:47.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.711+0000 7f1c39b58b80 1 -- start start 2024-05-14T09:36:47.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.711+0000 7f1c39b58b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 0x555ceda86b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.711+0000 7f1c39b58b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87040 0x555ceda90690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.712+0000 7f1c39b58b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87450 0x555ceda92be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.712+0000 7f1c39b58b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555ced9a88e0 con 0x555ceda87040 2024-05-14T09:36:47.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.712+0000 7f1c39b58b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555ced947840 con 0x555ceda87450 2024-05-14T09:36:47.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.712+0000 7f1c39b58b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555ced9a48a0 con 0x555ced908ec0 2024-05-14T09:36:47.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.712+0000 7f1c28ee0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87040 0x555ceda90690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.712+0000 7f1c28ee0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87040 0x555ceda90690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59110/0 (socket says 172.21.15.2:59110) 2024-05-14T09:36:47.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.712+0000 7f1c28ee0700 1 -- 172.21.15.2:0/708947809 learned_addr learned my addr 172.21.15.2:0/708947809 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:47.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.712+0000 7f1c21ffb700 1 --2- 172.21.15.2:0/1931583014 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c0c0b2a50 0x7f1c0c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.712+0000 7f1c296e1700 1 --2- 172.21.15.2:0/1931583014 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c0c0b2a50 0x7f1c0c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.712+0000 7f1c21ffb700 1 -- 172.21.15.2:0/1931583014 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1c1c02b070 con 0x7f1c241084d0 2024-05-14T09:36:47.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.713+0000 7f1c28ee0700 1 -- 172.21.15.2:0/708947809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 msgr2=0x555ceda86b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:36:47.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.713+0000 7f1c28ee0700 1 --2- 172.21.15.2:0/708947809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 0x555ceda86b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.713+0000 7f1c28ee0700 1 -- 172.21.15.2:0/708947809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87450 msgr2=0x555ceda92be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:36:47.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.713+0000 7f1c28ee0700 1 --2- 172.21.15.2:0/708947809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87450 0x555ceda92be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.713+0000 7f1c28ee0700 1 -- 172.21.15.2:0/708947809 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ced8f5950 con 0x555ceda87040 2024-05-14T09:36:47.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.713+0000 7f1c296e1700 1 --2- 172.21.15.2:0/1931583014 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c0c0b2a50 0x7f1c0c0b4f60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0b98b0 tx=0x7f1c1c003e10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:47.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.713+0000 7f1c28ee0700 1 --2- 172.21.15.2:0/708947809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87040 0x555ceda90690 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1c140104a0 tx=0x7f1c140107b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548f30d1b760f00a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:47.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.714+0000 7f1c0affd700 1 -- 172.21.15.2:0/708947809 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14011640 con 0x555ceda87040 2024-05-14T09:36:47.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.714+0000 7f1c0affd700 1 -- 172.21.15.2:0/708947809 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c14011c90 con 0x555ceda87040 2024-05-14T09:36:47.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.715+0000 7f1c0affd700 1 -- 172.21.15.2:0/708947809 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14019c50 con 0x555ceda87040 2024-05-14T09:36:47.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.715+0000 7f1c39b58b80 1 -- 172.21.15.2:0/708947809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87040 msgr2=0x555ceda90690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.715+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/708947809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87040 0x555ceda90690 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1c140104a0 tx=0x7f1c140107b0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.715+0000 7f1c39b58b80 1 -- 172.21.15.2:0/708947809 shutdown_connections 2024-05-14T09:36:47.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.715+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/708947809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87450 0x555ceda92be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.715+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/708947809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 0x555ceda86b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.715+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/708947809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87040 0x555ceda90690 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.715+0000 7f1c39b58b80 1 -- 172.21.15.2:0/708947809 >> 172.21.15.2:0/708947809 conn(0x555ced950680 msgr2=0x555ceda818f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:47.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.718+0000 7f1c39b58b80 1 -- 172.21.15.2:0/708947809 shutdown_connections 2024-05-14T09:36:47.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.718+0000 7f1c39b58b80 1 -- 172.21.15.2:0/708947809 wait complete. 2024-05-14T09:36:47.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.719+0000 7f1c39b58b80 1 -- start start 2024-05-14T09:36:47.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.719+0000 7f1c39b58b80 1 Processor -- start 2024-05-14T09:36:47.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.720+0000 7f1c39b58b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ced908ec0 0x555ceda900f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.720+0000 7f1c39b58b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ceda87040 0x555ceda90630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.720+0000 7f1c39b58b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87450 0x555ceda90b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.720+0000 7f1c39b58b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555ceda93610 con 0x555ced908ec0 2024-05-14T09:36:47.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.720+0000 7f1c39b58b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555ced90ecb0 con 0x555ceda87450 2024-05-14T09:36:47.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.720+0000 7f1c39b58b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555ceda93790 con 0x555ceda87040 2024-05-14T09:36:47.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.720+0000 7f1c39b58b80 10 client.0 _ll_register_callbacks cb 0x555ced9409a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:47.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.720+0000 7f1c28ee0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ceda87040 0x555ceda90630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.720+0000 7f1c28ee0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ceda87040 0x555ceda90630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:45330/0 (socket says 172.21.15.2:45330) 2024-05-14T09:36:47.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.720+0000 7f1c28ee0700 1 -- 172.21.15.2:0/4044361450 learned_addr learned my addr 172.21.15.2:0/4044361450 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:47.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.720+0000 7f1c23fff700 1 --2- 172.21.15.2:0/4044361450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87450 0x555ceda90b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.721+0000 7f1c28ee0700 1 -- 172.21.15.2:0/4044361450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87450 msgr2=0x555ceda90b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:47.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.721+0000 7f1c28ee0700 1 --2- 172.21.15.2:0/4044361450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87450 0x555ceda90b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.721+0000 7f1c28ee0700 1 -- 172.21.15.2:0/4044361450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ced908ec0 msgr2=0x555ceda900f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:47.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.721+0000 7f1c28ee0700 1 --2- 172.21.15.2:0/4044361450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ced908ec0 0x555ceda900f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.721+0000 7f1c28ee0700 1 -- 172.21.15.2:0/4044361450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ced944710 con 0x555ceda87040 2024-05-14T09:36:47.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.722+0000 7f1c28ee0700 1 --2- 172.21.15.2:0/4044361450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ceda87040 0x555ceda90630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1c14019690 tx=0x7f1c14019850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6ee8787d0503c16 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:47.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.724+0000 7f1c03fff700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:36:47.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.724+0000 7f1c03fff700 1 -- 172.21.15.2:0/4044361450 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1401f040 con 0x555ceda87040 2024-05-14T09:36:47.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.724+0000 7f1c39b58b80 1 -- 172.21.15.2:0/4044361450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c14011c90 con 0x555ceda87040 2024-05-14T09:36:47.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.724+0000 7f1c03fff700 1 -- 172.21.15.2:0/4044361450 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1c14019ce0 con 0x555ceda87040 2024-05-14T09:36:47.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.727+0000 7f1c03fff700 1 -- 172.21.15.2:0/4044361450 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c14030050 con 0x555ceda87040 2024-05-14T09:36:47.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.729+0000 7f1c39b58b80 10 client.25390 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:47.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.729+0000 7f1c39b58b80 1 -- 172.21.15.2:0/4044361450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555cedac0270 con 0x555ceda87040 2024-05-14T09:36:47.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.732+0000 7f1c03fff700 1 -- 172.21.15.2:0/4044361450 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c14011c90 con 0x555ceda87040 2024-05-14T09:36:47.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.732+0000 7f1c03fff700 1 -- 172.21.15.2:0/4044361450 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1c1402b030 con 0x555ceda87040 2024-05-14T09:36:47.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.732+0000 7f1c39b58b80 10 client.25390 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:47.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.732+0000 7f1c39b58b80 10 client.25390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:47.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.732+0000 7f1c39b58b80 20 client.25390 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:47.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.732+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/4044361450 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cedac2b50 0x555cedac5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.732+0000 7f1c39b58b80 4 client.25390 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:36:47.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.732+0000 7f1c39b58b80 1 -- 172.21.15.2:0/4044361450 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ced9b62c0 con 0x555cedac2b50 2024-05-14T09:36:47.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.733+0000 7f1c23fff700 1 --2- 172.21.15.2:0/4044361450 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cedac2b50 0x555cedac5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.734+0000 7f1c23fff700 1 --2- 172.21.15.2:0/4044361450 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cedac2b50 0x555cedac5000 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e1efb276f7b26a6 server_cookie=a8f66c8770c7702c in_seq=0 out_seq=0 2024-05-14T09:36:47.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.734+0000 7f1c03fff700 10 client.25390 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f02f57fa640 1 -- 172.21.15.2:0/330547256 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5577a48273e0 con 0x5577a44851b0 2024-05-14T09:36:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f02f57fa640 10 client.25385 handle_command_reply: tid=0 2024-05-14T09:36:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f031abb8740 1 client.25385 shutdown 2024-05-14T09:36:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f031abb8740 1 -- 172.21.15.2:0/330547256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a44644d0 msgr2=0x5577a48a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f031abb8740 1 --2- 172.21.15.2:0/330547256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a44644d0 0x5577a48a4400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f02fc011c50 tx=0x7f02fc005d30 comp rx=0 tx=0).stop 2024-05-14T09:36:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f031abb8740 1 -- 172.21.15.2:0/330547256 shutdown_connections 2024-05-14T09:36:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f031abb8740 1 --2- 172.21.15.2:0/330547256 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5577a44851b0 0x5577a48a8310 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f031abb8740 1 --2- 172.21.15.2:0/330547256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577a44644d0 0x5577a48a4400 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f031abb8740 1 --2- 172.21.15.2:0/330547256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5577a445c2f0 0x5577a4835440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f02f57fa640 0 client.25385 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f031abb8740 1 --2- 172.21.15.2:0/330547256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5577a4443d80 0x5577a4834f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f031abb8740 1 -- 172.21.15.2:0/330547256 >> 172.21.15.2:0/330547256 conn(0x5577a4479300 msgr2=0x5577a4830820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f031abb8740 1 -- 172.21.15.2:0/330547256 shutdown_connections 2024-05-14T09:36:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f031abb8740 1 -- 172.21.15.2:0/330547256 wait complete. 2024-05-14T09:36:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f031abb8740 20 client.25385 trim_cache size 0 max 16384 2024-05-14T09:36:47.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f1c03fff700 1 -- 172.21.15.2:0/4044361450 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ced9b62c0 con 0x555cedac2b50 2024-05-14T09:36:47.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f1c03fff700 10 client.25390 handle_command_reply: tid=0 2024-05-14T09:36:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:36:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:36:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:36:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:36:47.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.849+0000 7f1c39b58b80 1 client.25390 shutdown 2024-05-14T09:36:47.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f1c39b58b80 1 -- 172.21.15.2:0/4044361450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ceda87040 msgr2=0x555ceda90630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/4044361450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ceda87040 0x555ceda90630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1c14019690 tx=0x7f1c14019850 comp rx=0 tx=0).stop 2024-05-14T09:36:47.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f1c39b58b80 1 -- 172.21.15.2:0/4044361450 shutdown_connections 2024-05-14T09:36:47.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/4044361450 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cedac2b50 0x555cedac5000 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/4044361450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87450 0x555ceda90b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/4044361450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ced908ec0 0x555ceda900f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/4044361450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ceda87040 0x555ceda90630 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f1c39b58b80 1 -- 172.21.15.2:0/4044361450 >> 172.21.15.2:0/4044361450 conn(0x555ced950680 msgr2=0x555ceda854d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:47.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f1c03fff700 0 client.25390 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:47.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f0313c27640 1 -- 172.21.15.2:0/2270047665 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f02f80b28a0 msgr2=0x7f02f80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f0313c27640 1 --2- 172.21.15.2:0/2270047665 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f02f80b28a0 0x7f02f80b4db0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f030406c3c0 tx=0x7f0304069d90 comp rx=0 tx=0).stop 2024-05-14T09:36:47.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f1c39b58b80 1 -- 172.21.15.2:0/4044361450 shutdown_connections 2024-05-14T09:36:47.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f0313c27640 1 -- 172.21.15.2:0/2270047665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f030c108a90 msgr2=0x7f030c10c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f0313c27640 1 --2- 172.21.15.2:0/2270047665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f030c108a90 0x7f030c10c000 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f030400e9b0 tx=0x7f030400ee70 comp rx=0 tx=0).stop 2024-05-14T09:36:47.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f1c39b58b80 1 -- 172.21.15.2:0/4044361450 wait complete. 2024-05-14T09:36:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f0313c27640 1 -- 172.21.15.2:0/2270047665 shutdown_connections 2024-05-14T09:36:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f0313c27640 1 --2- 172.21.15.2:0/2270047665 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f02f80b28a0 0x7f02f80b4db0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f0313c27640 1 --2- 172.21.15.2:0/2270047665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f030c108a90 0x7f030c10c000 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f0313c27640 1 --2- 172.21.15.2:0/2270047665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f030c103d90 0x7f030c10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f0313c27640 1 --2- 172.21.15.2:0/2270047665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030c102b50 0x7f030c10d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f0313c27640 1 -- 172.21.15.2:0/2270047665 >> 172.21.15.2:0/2270047665 conn(0x7f030c0fdd40 msgr2=0x7f030c104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f1c39b58b80 20 client.25390 trim_cache size 0 max 16384 2024-05-14T09:36:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f0313c27640 1 -- 172.21.15.2:0/2270047665 shutdown_connections 2024-05-14T09:36:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:36:47.850+0000 7f0313c27640 1 -- 172.21.15.2:0/2270047665 wait complete. 2024-05-14T09:36:47.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.873+0000 7f1c39b58b80 1 Processor -- start 2024-05-14T09:36:47.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.873+0000 7f1c39b58b80 1 -- start start 2024-05-14T09:36:47.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c39b58b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 0x555ceda85be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c39b58b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87040 0x555ceda8b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c39b58b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87450 0x555ceda9c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c39b58b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241a8360 con 0x555ceda87450 2024-05-14T09:36:47.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c39b58b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241a5bf0 con 0x555ceda87040 2024-05-14T09:36:47.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c39b58b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241a3a40 con 0x555ced908ec0 2024-05-14T09:36:47.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c23fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 0x555ceda85be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c28ee0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87040 0x555ceda8b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c23fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 0x555ceda85be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:45342/0 (socket says 172.21.15.2:45342) 2024-05-14T09:36:47.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c23fff700 1 -- 172.21.15.2:0/2552849585 learned_addr learned my addr 172.21.15.2:0/2552849585 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:47.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c23fff700 1 --2- 172.21.15.2:0/2552849585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87450 0x555ceda9c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c23fff700 1 -- 172.21.15.2:0/2552849585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87040 msgr2=0x555ceda8b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:47.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c23fff700 1 --2- 172.21.15.2:0/2552849585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87040 0x555ceda8b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c23fff700 1 -- 172.21.15.2:0/2552849585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87450 msgr2=0x555ceda9c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:47.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c23fff700 1 --2- 172.21.15.2:0/2552849585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87450 0x555ceda9c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c23fff700 1 -- 172.21.15.2:0/2552849585 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ceda8fec0 con 0x555ced908ec0 2024-05-14T09:36:47.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c28ee0700 1 --2- 172.21.15.2:0/2552849585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87040 0x555ceda8b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:47.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.874+0000 7f1c23fff700 1 --2- 172.21.15.2:0/2552849585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 0x555ceda85be0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009460 tx=0x7f1c140197b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0ae16bea7f9d794 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:47.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.875+0000 7f1c0a7fc700 1 -- 172.21.15.2:0/2552849585 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c180056b0 con 0x555ced908ec0 2024-05-14T09:36:47.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.875+0000 7f1c0a7fc700 1 -- 172.21.15.2:0/2552849585 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c18003b30 con 0x555ced908ec0 2024-05-14T09:36:47.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.875+0000 7f1c0a7fc700 1 -- 172.21.15.2:0/2552849585 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800bbc0 con 0x555ced908ec0 2024-05-14T09:36:47.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.876+0000 7f1c39b58b80 1 -- 172.21.15.2:0/2552849585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 msgr2=0x555ceda85be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.876+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/2552849585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 0x555ceda85be0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009460 tx=0x7f1c140197b0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.876+0000 7f1c39b58b80 1 -- 172.21.15.2:0/2552849585 shutdown_connections 2024-05-14T09:36:47.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.876+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/2552849585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87450 0x555ceda9c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.876+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/2552849585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 0x555ceda85be0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.876+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/2552849585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87040 0x555ceda8b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.876+0000 7f1c39b58b80 1 -- 172.21.15.2:0/2552849585 >> 172.21.15.2:0/2552849585 conn(0x555ced950680 msgr2=0x555ceda854d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:47.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.876+0000 7f1c39b58b80 1 -- 172.21.15.2:0/2552849585 shutdown_connections 2024-05-14T09:36:47.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.876+0000 7f1c39b58b80 1 -- 172.21.15.2:0/2552849585 wait complete. 2024-05-14T09:36:47.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.876+0000 7f1c39b58b80 1 -- start start 2024-05-14T09:36:47.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.876+0000 7f1c39b58b80 1 Processor -- start 2024-05-14T09:36:47.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c39b58b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 0x555ceda989f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c39b58b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87040 0x555ceda98f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c39b58b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87450 0x555ceda94030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c39b58b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241a40b0 con 0x555ceda87040 2024-05-14T09:36:47.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c39b58b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241a5520 con 0x555ceda87450 2024-05-14T09:36:47.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c39b58b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241a4780 con 0x555ced908ec0 2024-05-14T09:36:47.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c39b58b80 10 client.0 _ll_register_callbacks cb 0x555ced9409a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:47.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c28ee0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87040 0x555ceda98f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c28ee0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87040 0x555ceda98f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59138/0 (socket says 172.21.15.2:59138) 2024-05-14T09:36:47.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c28ee0700 1 -- 172.21.15.2:0/1933045836 learned_addr learned my addr 172.21.15.2:0/1933045836 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:47.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c23fff700 1 --2- 172.21.15.2:0/1933045836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87450 0x555ceda94030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c23fff700 1 --2- 172.21.15.2:0/1933045836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 0x555ceda989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c28ee0700 1 -- 172.21.15.2:0/1933045836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 msgr2=0x555ceda989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:47.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c28ee0700 1 --2- 172.21.15.2:0/1933045836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 0x555ceda989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c28ee0700 1 -- 172.21.15.2:0/1933045836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87450 msgr2=0x555ceda94030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:47.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c28ee0700 1 --2- 172.21.15.2:0/1933045836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87450 0x555ceda94030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c28ee0700 1 -- 172.21.15.2:0/1933045836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ceda94640 con 0x555ceda87040 2024-05-14T09:36:47.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c23fff700 1 --2- 172.21.15.2:0/1933045836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87450 0x555ceda94030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:47.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c28ee0700 1 --2- 172.21.15.2:0/1933045836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87040 0x555ceda98f30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1c14010bb0 tx=0x7f1c14011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731fff8a27053a39 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:47.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.877+0000 7f1c03fff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:36:47.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.878+0000 7f1c03fff700 1 -- 172.21.15.2:0/1933045836 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14010be0 con 0x555ceda87040 2024-05-14T09:36:47.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.878+0000 7f1c39b58b80 1 -- 172.21.15.2:0/1933045836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c18003b30 con 0x555ceda87040 2024-05-14T09:36:47.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.878+0000 7f1c03fff700 1 -- 172.21.15.2:0/1933045836 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1c14011a00 con 0x555ceda87040 2024-05-14T09:36:47.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.878+0000 7f1c03fff700 1 -- 172.21.15.2:0/1933045836 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c14042050 con 0x555ceda87040 2024-05-14T09:36:47.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.878+0000 7f1c39b58b80 10 client.15702 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:47.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.878+0000 7f1c39b58b80 1 -- 172.21.15.2:0/1933045836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555cedb2ce40 con 0x555ceda87040 2024-05-14T09:36:47.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.878+0000 7f1c03fff700 1 -- 172.21.15.2:0/1933045836 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c14012e50 con 0x555ceda87040 2024-05-14T09:36:47.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.878+0000 7f1c03fff700 1 -- 172.21.15.2:0/1933045836 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1c1402b040 con 0x555ceda87040 2024-05-14T09:36:47.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.878+0000 7f1c39b58b80 10 client.15702 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:47.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.878+0000 7f1c39b58b80 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:47.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.878+0000 7f1c39b58b80 20 client.15702 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:47.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.878+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/1933045836 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cedb2d540 0x555cedb2f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:47.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.878+0000 7f1c39b58b80 4 client.15702 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T09:36:47.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.878+0000 7f1c39b58b80 1 -- 172.21.15.2:0/1933045836 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f1c18003b30 con 0x555cedb2d540 2024-05-14T09:36:47.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.879+0000 7f1c23fff700 1 --2- 172.21.15.2:0/1933045836 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cedb2d540 0x555cedb2f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:47.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.879+0000 7f1c03fff700 10 client.15702 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:47.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.879+0000 7f1c23fff700 1 --2- 172.21.15.2:0/1933045836 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cedb2d540 0x555cedb2f960 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b482bf5b15ba2b0 server_cookie=d4f21a867191e94c in_seq=0 out_seq=0 2024-05-14T09:36:47.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.881+0000 7f1c03fff700 1 -- 172.21.15.2:0/1933045836 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f1c18003b30 con 0x555cedb2d540 2024-05-14T09:36:47.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.881+0000 7f1c03fff700 10 client.15702 handle_command_reply: tid=0 2024-05-14T09:36:47.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.881+0000 7f1c39b58b80 1 client.15702 shutdown 2024-05-14T09:36:47.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.881+0000 7f1c39b58b80 1 -- 172.21.15.2:0/1933045836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87040 msgr2=0x555ceda98f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.881+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/1933045836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87040 0x555ceda98f30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1c14010bb0 tx=0x7f1c14011910 comp rx=0 tx=0).stop 2024-05-14T09:36:47.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.881+0000 7f1c39b58b80 1 -- 172.21.15.2:0/1933045836 shutdown_connections 2024-05-14T09:36:47.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.881+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/1933045836 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cedb2d540 0x555cedb2f960 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.881+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/1933045836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ceda87450 0x555ceda94030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.881+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/1933045836 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ced908ec0 0x555ceda989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.881+0000 7f1c39b58b80 1 --2- 172.21.15.2:0/1933045836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ceda87040 0x555ceda98f30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.881+0000 7f1c03fff700 0 client.15702 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:47.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.881+0000 7f1c39b58b80 1 -- 172.21.15.2:0/1933045836 >> 172.21.15.2:0/1933045836 conn(0x555ced950680 msgr2=0x555ceda900b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:47.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.881+0000 7f1c39b58b80 1 -- 172.21.15.2:0/1933045836 shutdown_connections 2024-05-14T09:36:47.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.881+0000 7f1c39b58b80 1 -- 172.21.15.2:0/1933045836 wait complete. 2024-05-14T09:36:47.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.881+0000 7f1c39b58b80 20 client.15702 trim_cache size 0 max 16384 2024-05-14T09:36:47.884 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:36:47.884 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-05-14T09:36:47.884 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "6cdf564e-9fc2-49e3-8b92-09d2b222bfe4", 2024-05-14T09:36:47.884 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-05-14T09:36:47.884 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:36:47.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.882+0000 7f1c29ee2700 1 -- 172.21.15.2:0/1931583014 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c0c0b2a50 msgr2=0x7f1c0c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.882+0000 7f1c29ee2700 1 --2- 172.21.15.2:0/1931583014 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c0c0b2a50 0x7f1c0c0b4f60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0b98b0 tx=0x7f1c1c003e10 comp rx=0 tx=0).stop 2024-05-14T09:36:47.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.882+0000 7f1c29ee2700 1 -- 172.21.15.2:0/1931583014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c241084d0 msgr2=0x7f1c24074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:47.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.882+0000 7f1c29ee2700 1 --2- 172.21.15.2:0/1931583014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c241084d0 0x7f1c24074530 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c01ae90 tx=0x7f1c1c01e3b0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.882+0000 7f1c29ee2700 1 -- 172.21.15.2:0/1931583014 shutdown_connections 2024-05-14T09:36:47.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.882+0000 7f1c29ee2700 1 --2- 172.21.15.2:0/1931583014 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1c0c0b2a50 0x7f1c0c0b4f60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.882+0000 7f1c29ee2700 1 --2- 172.21.15.2:0/1931583014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c24103080 0x7f1c2410fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.882+0000 7f1c29ee2700 1 --2- 172.21.15.2:0/1931583014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c241084d0 0x7f1c24074530 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.882+0000 7f1c29ee2700 1 --2- 172.21.15.2:0/1931583014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1c241039d0 0x7f1c24110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:47.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.882+0000 7f1c29ee2700 1 -- 172.21.15.2:0/1931583014 >> 172.21.15.2:0/1931583014 conn(0x7f1c24078b00 msgr2=0x7f1c24105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:47.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.882+0000 7f1c29ee2700 1 -- 172.21.15.2:0/1931583014 shutdown_connections 2024-05-14T09:36:47.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:47.882+0000 7f1c29ee2700 1 -- 172.21.15.2:0/1931583014 wait complete. 2024-05-14T09:36:48.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 39004 16.39 MB/sec warmup 14 sec latency 88.409 ms 2024-05-14T09:36:48.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:48 smithi002 ceph-mon[31557]: pgmap v479: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-05-14T09:36:48.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:48 smithi029 ceph-mon[36249]: pgmap v479: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-05-14T09:36:48.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:48 smithi026 ceph-mon[36056]: pgmap v479: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-05-14T09:36:48.746 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T09:36:49.005 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:36:49.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 41499 15.89 MB/sec warmup 15 sec latency 28.789 ms 2024-05-14T09:36:49.397 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:49 smithi002 ceph-mon[31557]: scrub queued for path: / 2024-05-14T09:36:49.397 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:49 smithi002 ceph-mon[31557]: scrub summary: idle+waiting paths [/] 2024-05-14T09:36:49.397 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:49 smithi002 ceph-mon[31557]: scrub summary: active paths [/] 2024-05-14T09:36:49.397 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:49 smithi002 ceph-mon[31557]: scrub summary: idle+waiting paths [/] 2024-05-14T09:36:49.397 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:49 smithi002 ceph-mon[31557]: scrub summary: idle 2024-05-14T09:36:49.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:49 smithi029 ceph-mon[36249]: scrub queued for path: / 2024-05-14T09:36:49.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:49 smithi029 ceph-mon[36249]: scrub summary: idle+waiting paths [/] 2024-05-14T09:36:49.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:49 smithi029 ceph-mon[36249]: scrub summary: active paths [/] 2024-05-14T09:36:49.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:49 smithi029 ceph-mon[36249]: scrub summary: idle+waiting paths [/] 2024-05-14T09:36:49.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:49 smithi029 ceph-mon[36249]: scrub summary: idle 2024-05-14T09:36:49.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:49 smithi026 ceph-mon[36056]: scrub queued for path: / 2024-05-14T09:36:49.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:49 smithi026 ceph-mon[36056]: scrub summary: idle+waiting paths [/] 2024-05-14T09:36:49.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:49 smithi026 ceph-mon[36056]: scrub summary: active paths [/] 2024-05-14T09:36:49.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:49 smithi026 ceph-mon[36056]: scrub summary: idle+waiting paths [/] 2024-05-14T09:36:49.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:49 smithi026 ceph-mon[36056]: scrub summary: idle 2024-05-14T09:36:50.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 43829 15.77 MB/sec warmup 16 sec latency 19.979 ms 2024-05-14T09:36:50.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:50 smithi002 ceph-mon[31557]: pgmap v480: 129 pgs: 129 active+clean; 12 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 1.5 MiB/s wr, 224 op/s 2024-05-14T09:36:50.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:50 smithi029 ceph-mon[36249]: pgmap v480: 129 pgs: 129 active+clean; 12 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 1.5 MiB/s wr, 224 op/s 2024-05-14T09:36:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:50 smithi026 ceph-mon[36056]: pgmap v480: 129 pgs: 129 active+clean; 12 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 1.5 MiB/s wr, 224 op/s 2024-05-14T09:36:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.942+0000 7f1d65ef4700 1 -- 172.21.15.2:0/2042872131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d60107df0 msgr2=0x7f1d60108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.942+0000 7f1d65ef4700 1 --2- 172.21.15.2:0/2042872131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d60107df0 0x7f1d60108200 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1d540099c0 tx=0x7f1d54009cd0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.942+0000 7f1d65ef4700 1 -- 172.21.15.2:0/2042872131 shutdown_connections 2024-05-14T09:36:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.942+0000 7f1d65ef4700 1 --2- 172.21.15.2:0/2042872131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d60108740 0x7f1d601100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.942+0000 7f1d65ef4700 1 --2- 172.21.15.2:0/2042872131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d601034c0 0x7f1d601038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.942+0000 7f1d65ef4700 1 --2- 172.21.15.2:0/2042872131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d60107df0 0x7f1d60108200 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.942+0000 7f1d65ef4700 1 -- 172.21.15.2:0/2042872131 >> 172.21.15.2:0/2042872131 conn(0x7f1d600fedf0 msgr2=0x7f1d60101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:50.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.942+0000 7f1d65ef4700 1 -- 172.21.15.2:0/2042872131 shutdown_connections 2024-05-14T09:36:50.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.942+0000 7f1d65ef4700 1 -- 172.21.15.2:0/2042872131 wait complete. 2024-05-14T09:36:50.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.942+0000 7f1d65ef4700 1 Processor -- start 2024-05-14T09:36:50.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.943+0000 7f1d65ef4700 1 -- start start 2024-05-14T09:36:50.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.943+0000 7f1d65ef4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d601034c0 0x7f1d6019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.943+0000 7f1d65ef4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d60107df0 0x7f1d6019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.943+0000 7f1d65ef4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d60108740 0x7f1d601a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.943+0000 7f1d65ef4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1d60113350 con 0x7f1d601034c0 2024-05-14T09:36:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.943+0000 7f1d65ef4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601131d0 con 0x7f1d60107df0 2024-05-14T09:36:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.943+0000 7f1d65ef4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601134d0 con 0x7f1d60108740 2024-05-14T09:36:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.943+0000 7f1d5f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d601034c0 0x7f1d6019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.943+0000 7f1d5f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d601034c0 0x7f1d6019d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59146/0 (socket says 172.21.15.2:59146) 2024-05-14T09:36:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.943+0000 7f1d5effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d60107df0 0x7f1d6019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.943+0000 7f1d5f7fe700 1 -- 172.21.15.2:0/459297781 learned_addr learned my addr 172.21.15.2:0/459297781 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.943+0000 7f1d5ffff700 1 --2- 172.21.15.2:0/459297781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d60108740 0x7f1d601a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.944+0000 7f1d5effd700 1 -- 172.21.15.2:0/459297781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d60108740 msgr2=0x7f1d601a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.944+0000 7f1d5effd700 1 --2- 172.21.15.2:0/459297781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d60108740 0x7f1d601a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.944+0000 7f1d5effd700 1 -- 172.21.15.2:0/459297781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d601034c0 msgr2=0x7f1d6019d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.944+0000 7f1d5effd700 1 --2- 172.21.15.2:0/459297781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d601034c0 0x7f1d6019d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.944+0000 7f1d5effd700 1 -- 172.21.15.2:0/459297781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d601a1ee0 con 0x7f1d60107df0 2024-05-14T09:36:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.944+0000 7f1d5effd700 1 --2- 172.21.15.2:0/459297781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d60107df0 0x7f1d6019d5b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1d540098e0 tx=0x7f1d54004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.944+0000 7f1d5cff9700 1 -- 172.21.15.2:0/459297781 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54026070 con 0x7f1d60107df0 2024-05-14T09:36:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.944+0000 7f1d5cff9700 1 -- 172.21.15.2:0/459297781 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d54003c70 con 0x7f1d60107df0 2024-05-14T09:36:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.944+0000 7f1d5cff9700 1 -- 172.21.15.2:0/459297781 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5400f9c0 con 0x7f1d60107df0 2024-05-14T09:36:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.944+0000 7f1d65ef4700 1 -- 172.21.15.2:0/459297781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d601a2170 con 0x7f1d60107df0 2024-05-14T09:36:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.944+0000 7f1d65ef4700 1 -- 172.21.15.2:0/459297781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d601a2540 con 0x7f1d60107df0 2024-05-14T09:36:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.947+0000 7f1d5cff9700 1 -- 172.21.15.2:0/459297781 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1d54029070 con 0x7f1d60107df0 2024-05-14T09:36:50.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.948+0000 7f1d5cff9700 1 --2- 172.21.15.2:0/459297781 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1d480b2900 0x7f1d480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.948+0000 7f1d5f7fe700 1 --2- 172.21.15.2:0/459297781 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1d480b2900 0x7f1d480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.948+0000 7f1d5cff9700 1 -- 172.21.15.2:0/459297781 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1d540e2dd0 con 0x7f1d60107df0 2024-05-14T09:36:50.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.948+0000 7f1d74908b80 1 Processor -- start 2024-05-14T09:36:50.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d74908b80 1 -- start start 2024-05-14T09:36:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d5f7fe700 1 --2- 172.21.15.2:0/459297781 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1d480b2900 0x7f1d480b4e10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009d20 tx=0x7f1d50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d74908b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 0x56243f9d8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d74908b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9dcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d74908b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 0x56243f9dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d74908b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56243f8fa860 con 0x56243f9d8c60 2024-05-14T09:36:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d74908b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56243f8997c0 con 0x56243f9dd470 2024-05-14T09:36:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d74908b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56243f8f6820 con 0x56243f85ae40 2024-05-14T09:36:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d5effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9dcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d5effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9dcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59150/0 (socket says 172.21.15.2:59150) 2024-05-14T09:36:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d5effd700 1 -- 172.21.15.2:0/3610483258 learned_addr learned my addr 172.21.15.2:0/3610483258 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d5f7fe700 1 --2- 172.21.15.2:0/3610483258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 0x56243f9d8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d5ffff700 1 --2- 172.21.15.2:0/3610483258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 0x56243f9dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d5effd700 1 -- 172.21.15.2:0/3610483258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 msgr2=0x56243f9d8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d5effd700 1 --2- 172.21.15.2:0/3610483258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 0x56243f9d8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d5effd700 1 -- 172.21.15.2:0/3610483258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 msgr2=0x56243f9dd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.949+0000 7f1d5effd700 1 --2- 172.21.15.2:0/3610483258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 0x56243f9dd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.950+0000 7f1d5effd700 1 -- 172.21.15.2:0/3610483258 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56243f8478d0 con 0x56243f9d8c60 2024-05-14T09:36:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.950+0000 7f1d5f7fe700 1 --2- 172.21.15.2:0/3610483258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 0x56243f9d8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.950+0000 7f1d5effd700 1 --2- 172.21.15.2:0/3610483258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9dcf30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1d540a9f90 tx=0x7f1d54003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a5025fe5ec88e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.950+0000 7f1d45ffb700 1 -- 172.21.15.2:0/3610483258 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d540aa700 con 0x56243f9d8c60 2024-05-14T09:36:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.950+0000 7f1d45ffb700 1 -- 172.21.15.2:0/3610483258 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d54003e20 con 0x56243f9d8c60 2024-05-14T09:36:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.950+0000 7f1d45ffb700 1 -- 172.21.15.2:0/3610483258 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5400fb60 con 0x56243f9d8c60 2024-05-14T09:36:50.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.951+0000 7f1d74908b80 1 -- 172.21.15.2:0/3610483258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 msgr2=0x56243f9dcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:50.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.951+0000 7f1d74908b80 1 --2- 172.21.15.2:0/3610483258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9dcf30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1d540a9f90 tx=0x7f1d54003680 comp rx=0 tx=0).stop 2024-05-14T09:36:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.951+0000 7f1d74908b80 1 -- 172.21.15.2:0/3610483258 shutdown_connections 2024-05-14T09:36:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.951+0000 7f1d74908b80 1 --2- 172.21.15.2:0/3610483258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 0x56243f9dd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.951+0000 7f1d74908b80 1 --2- 172.21.15.2:0/3610483258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 0x56243f9d8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.951+0000 7f1d74908b80 1 --2- 172.21.15.2:0/3610483258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9dcf30 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.951+0000 7f1d74908b80 1 -- 172.21.15.2:0/3610483258 >> 172.21.15.2:0/3610483258 conn(0x56243f8a2600 msgr2=0x56243f9d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.951+0000 7f1d74908b80 1 -- 172.21.15.2:0/3610483258 shutdown_connections 2024-05-14T09:36:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.951+0000 7f1d74908b80 1 -- 172.21.15.2:0/3610483258 wait complete. 2024-05-14T09:36:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.952+0000 7f1d74908b80 1 -- start start 2024-05-14T09:36:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.952+0000 7f1d74908b80 1 Processor -- start 2024-05-14T09:36:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.952+0000 7f1d74908b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 0x56243f9e2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.952+0000 7f1d74908b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9e2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.952+0000 7f1d74908b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 0x56243f9e2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.952+0000 7f1d74908b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56243f9e51e0 con 0x56243f9d8c60 2024-05-14T09:36:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.952+0000 7f1d74908b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56243f860c30 con 0x56243f9dd470 2024-05-14T09:36:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.952+0000 7f1d74908b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56243f9e5360 con 0x56243f85ae40 2024-05-14T09:36:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d74908b80 10 client.0 _ll_register_callbacks cb 0x56243f892920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d5effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9e2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d5ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 0x56243f9e2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d5effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9e2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59166/0 (socket says 172.21.15.2:59166) 2024-05-14T09:36:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d5f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 0x56243f9e2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d5ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 0x56243f9e2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:57420/0 (socket says 172.21.15.2:57420) 2024-05-14T09:36:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d5effd700 1 -- 172.21.15.2:0/2353727242 learned_addr learned my addr 172.21.15.2:0/2353727242 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d5effd700 1 -- 172.21.15.2:0/2353727242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 msgr2=0x56243f9e2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d5effd700 1 --2- 172.21.15.2:0/2353727242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 0x56243f9e2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d5effd700 1 -- 172.21.15.2:0/2353727242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 msgr2=0x56243f9e2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d5effd700 1 --2- 172.21.15.2:0/2353727242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 0x56243f9e2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d5effd700 1 -- 172.21.15.2:0/2353727242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56243f896690 con 0x56243f9d8c60 2024-05-14T09:36:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d5effd700 1 --2- 172.21.15.2:0/2353727242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9e2660 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009fd0 tx=0x7f1d540aa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9010bb1c3497c969 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d3effd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:36:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d3effd700 1 -- 172.21.15.2:0/2353727242 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d540c9060 con 0x56243f9d8c60 2024-05-14T09:36:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d3effd700 1 -- 172.21.15.2:0/2353727242 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1d5400fd70 con 0x56243f9d8c60 2024-05-14T09:36:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.953+0000 7f1d74908b80 1 -- 172.21.15.2:0/2353727242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d54003e20 con 0x56243f9d8c60 2024-05-14T09:36:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.954+0000 7f1d3effd700 1 -- 172.21.15.2:0/2353727242 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1d540c4070 con 0x56243f9d8c60 2024-05-14T09:36:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.954+0000 7f1d74908b80 10 client.15714 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.954+0000 7f1d74908b80 1 -- 172.21.15.2:0/2353727242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56243f9e51e0 con 0x56243f9d8c60 2024-05-14T09:36:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.954+0000 7f1d3effd700 1 -- 172.21.15.2:0/2353727242 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d54003e20 con 0x56243f9d8c60 2024-05-14T09:36:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.954+0000 7f1d3effd700 1 -- 172.21.15.2:0/2353727242 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1d540a5970 con 0x56243f9d8c60 2024-05-14T09:36:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.954+0000 7f1d74908b80 10 client.15714 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.954+0000 7f1d74908b80 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.954+0000 7f1d74908b80 20 client.15714 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:50.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.954+0000 7f1d74908b80 1 --2- 172.21.15.2:0/2353727242 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56243f9f20b0 0x56243f9f4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.954+0000 7f1d74908b80 4 client.15714 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:36:50.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.954+0000 7f1d74908b80 1 -- 172.21.15.2:0/2353727242 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56243f908240 con 0x56243f9f20b0 2024-05-14T09:36:50.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.954+0000 7f1d5f7fe700 1 --2- 172.21.15.2:0/2353727242 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56243f9f20b0 0x56243f9f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.955+0000 7f1d3effd700 10 client.15714 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:50.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.955+0000 7f1d5f7fe700 1 --2- 172.21.15.2:0/2353727242 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56243f9f20b0 0x56243f9f4500 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=257e5024f986c374 server_cookie=55addfd6bdcf17a2 in_seq=0 out_seq=0 2024-05-14T09:36:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.957+0000 7f1d3effd700 1 -- 172.21.15.2:0/2353727242 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56243f908240 con 0x56243f9f20b0 2024-05-14T09:36:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.957+0000 7f1d3effd700 10 client.15714 handle_command_reply: tid=0 2024-05-14T09:36:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.957+0000 7f1d74908b80 1 client.15714 shutdown 2024-05-14T09:36:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.957+0000 7f1d74908b80 1 -- 172.21.15.2:0/2353727242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 msgr2=0x56243f9e2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.957+0000 7f1d74908b80 1 --2- 172.21.15.2:0/2353727242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9e2660 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009fd0 tx=0x7f1d540aa000 comp rx=0 tx=0).stop 2024-05-14T09:36:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.957+0000 7f1d74908b80 1 -- 172.21.15.2:0/2353727242 shutdown_connections 2024-05-14T09:36:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.957+0000 7f1d74908b80 1 --2- 172.21.15.2:0/2353727242 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56243f9f20b0 0x56243f9f4500 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.957+0000 7f1d74908b80 1 --2- 172.21.15.2:0/2353727242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 0x56243f9e2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.957+0000 7f1d74908b80 1 --2- 172.21.15.2:0/2353727242 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 0x56243f9e2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.957+0000 7f1d74908b80 1 --2- 172.21.15.2:0/2353727242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9e2660 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.957+0000 7f1d74908b80 1 -- 172.21.15.2:0/2353727242 >> 172.21.15.2:0/2353727242 conn(0x56243f8a2600 msgr2=0x56243f9d7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:50.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.958+0000 7f1d3effd700 0 client.15714 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:50.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.958+0000 7f1d74908b80 1 -- 172.21.15.2:0/2353727242 shutdown_connections 2024-05-14T09:36:50.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.958+0000 7f1d74908b80 1 -- 172.21.15.2:0/2353727242 wait complete. 2024-05-14T09:36:50.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.958+0000 7f1d74908b80 20 client.15714 trim_cache size 0 max 16384 2024-05-14T09:36:50.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.980+0000 7f1d74908b80 1 Processor -- start 2024-05-14T09:36:50.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.980+0000 7f1d74908b80 1 -- start start 2024-05-14T09:36:50.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.980+0000 7f1d74908b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 0x56243f9e1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.980+0000 7f1d74908b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9e29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.980+0000 7f1d74908b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 0x56243f9ee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.980+0000 7f1d74908b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601b4f40 con 0x56243f9d8c60 2024-05-14T09:36:50.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.980+0000 7f1d74908b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601b27d0 con 0x56243f9dd470 2024-05-14T09:36:50.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.980+0000 7f1d74908b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601b0620 con 0x56243f85ae40 2024-05-14T09:36:50.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.980+0000 7f1d5effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9e29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.980+0000 7f1d5ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 0x56243f9ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.981+0000 7f1d5effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9e29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59178/0 (socket says 172.21.15.2:59178) 2024-05-14T09:36:50.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.981+0000 7f1d5effd700 1 -- 172.21.15.2:0/2459944358 learned_addr learned my addr 172.21.15.2:0/2459944358 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:50.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.981+0000 7f1d5f7fe700 1 --2- 172.21.15.2:0/2459944358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 0x56243f9e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.981+0000 7f1d5effd700 1 -- 172.21.15.2:0/2459944358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 msgr2=0x56243f9e1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:50.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.981+0000 7f1d5effd700 1 --2- 172.21.15.2:0/2459944358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 0x56243f9e1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.981+0000 7f1d5effd700 1 -- 172.21.15.2:0/2459944358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 msgr2=0x56243f9ee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:50.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.981+0000 7f1d5effd700 1 --2- 172.21.15.2:0/2459944358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 0x56243f9ee230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.981+0000 7f1d5effd700 1 -- 172.21.15.2:0/2459944358 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56243f9e2f20 con 0x56243f9d8c60 2024-05-14T09:36:50.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.981+0000 7f1d5f7fe700 1 --2- 172.21.15.2:0/2459944358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 0x56243f9e1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:50.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.981+0000 7f1d5effd700 1 --2- 172.21.15.2:0/2459944358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9e29e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1d54005f50 tx=0x7f1d54003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b3638cffce3d51 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:50.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.981+0000 7f1d457fa700 1 -- 172.21.15.2:0/2459944358 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d540c9070 con 0x56243f9d8c60 2024-05-14T09:36:50.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.981+0000 7f1d457fa700 1 -- 172.21.15.2:0/2459944358 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d54003e20 con 0x56243f9d8c60 2024-05-14T09:36:50.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.982+0000 7f1d457fa700 1 -- 172.21.15.2:0/2459944358 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d540ae3b0 con 0x56243f9d8c60 2024-05-14T09:36:50.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.982+0000 7f1d74908b80 1 -- 172.21.15.2:0/2459944358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 msgr2=0x56243f9e29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:50.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.982+0000 7f1d74908b80 1 --2- 172.21.15.2:0/2459944358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9e29e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1d54005f50 tx=0x7f1d54003680 comp rx=0 tx=0).stop 2024-05-14T09:36:50.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.983+0000 7f1d74908b80 1 -- 172.21.15.2:0/2459944358 shutdown_connections 2024-05-14T09:36:50.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.983+0000 7f1d74908b80 1 --2- 172.21.15.2:0/2459944358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9dd470 0x56243f9ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.983+0000 7f1d74908b80 1 --2- 172.21.15.2:0/2459944358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f85ae40 0x56243f9e1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.983+0000 7f1d74908b80 1 --2- 172.21.15.2:0/2459944358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f9d8c60 0x56243f9e29e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.983+0000 7f1d74908b80 1 -- 172.21.15.2:0/2459944358 >> 172.21.15.2:0/2459944358 conn(0x56243f8a2600 msgr2=0x56243f9d2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:50.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.983+0000 7f1d74908b80 1 -- 172.21.15.2:0/2459944358 shutdown_connections 2024-05-14T09:36:50.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.983+0000 7f1d74908b80 1 -- 172.21.15.2:0/2459944358 wait complete. 2024-05-14T09:36:50.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.983+0000 7f1d74908b80 1 -- start start 2024-05-14T09:36:50.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.983+0000 7f1d74908b80 1 Processor -- start 2024-05-14T09:36:50.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d74908b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f85ae40 0x56243f9ea420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d74908b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9d8c60 0x56243f9ea960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d74908b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f9dd470 0x56243f9e7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d74908b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601b0c90 con 0x56243f85ae40 2024-05-14T09:36:50.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d74908b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601b2100 con 0x56243f9d8c60 2024-05-14T09:36:50.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d74908b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601b1360 con 0x56243f9dd470 2024-05-14T09:36:50.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d74908b80 10 client.0 _ll_register_callbacks cb 0x56243f892920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:36:50.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d5f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f85ae40 0x56243f9ea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d5ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f9dd470 0x56243f9e7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d5f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f85ae40 0x56243f9ea420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59184/0 (socket says 172.21.15.2:59184) 2024-05-14T09:36:50.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d5effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9d8c60 0x56243f9ea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d5f7fe700 1 -- 172.21.15.2:0/1012838808 learned_addr learned my addr 172.21.15.2:0/1012838808 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d5f7fe700 1 -- 172.21.15.2:0/1012838808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f9dd470 msgr2=0x56243f9e7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d5f7fe700 1 --2- 172.21.15.2:0/1012838808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f9dd470 0x56243f9e7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d5f7fe700 1 -- 172.21.15.2:0/1012838808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9d8c60 msgr2=0x56243f9ea960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:36:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d5f7fe700 1 --2- 172.21.15.2:0/1012838808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9d8c60 0x56243f9ea960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d5f7fe700 1 -- 172.21.15.2:0/1012838808 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56243f9e7f90 con 0x56243f85ae40 2024-05-14T09:36:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d5ffff700 1 --2- 172.21.15.2:0/1012838808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f9dd470 0x56243f9e7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d5f7fe700 1 --2- 172.21.15.2:0/1012838808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f85ae40 0x56243f9ea420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x56243f9d9620 tx=0x7f1d500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3b1fa196499566 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.984+0000 7f1d3effd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:36:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.985+0000 7f1d3effd700 1 -- 172.21.15.2:0/1012838808 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50033070 con 0x56243f85ae40 2024-05-14T09:36:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.985+0000 7f1d3effd700 1 -- 172.21.15.2:0/1012838808 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1d5000e950 con 0x56243f85ae40 2024-05-14T09:36:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.985+0000 7f1d74908b80 1 -- 172.21.15.2:0/1012838808 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d54003e20 con 0x56243f85ae40 2024-05-14T09:36:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.985+0000 7f1d3effd700 1 -- 172.21.15.2:0/1012838808 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1d5003f050 con 0x56243f85ae40 2024-05-14T09:36:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.985+0000 7f1d74908b80 10 client.15720 fetch_fsmap learned FSMap version 27 2024-05-14T09:36:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.985+0000 7f1d74908b80 1 -- 172.21.15.2:0/1012838808 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56243fa5c270 con 0x56243f85ae40 2024-05-14T09:36:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.985+0000 7f1d3effd700 1 -- 172.21.15.2:0/1012838808 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d54003e20 con 0x56243f85ae40 2024-05-14T09:36:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.985+0000 7f1d3effd700 1 -- 172.21.15.2:0/1012838808 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1d5002c830 con 0x56243f85ae40 2024-05-14T09:36:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.985+0000 7f1d74908b80 10 client.15720 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:36:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.985+0000 7f1d74908b80 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:36:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.985+0000 7f1d74908b80 20 client.15720 populate_metadata read hostname 'smithi002' 2024-05-14T09:36:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.985+0000 7f1d74908b80 1 --2- 172.21.15.2:0/1012838808 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56243fa5c920 0x56243fa5ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.985+0000 7f1d74908b80 4 client.15720 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T09:36:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.985+0000 7f1d74908b80 1 -- 172.21.15.2:0/1012838808 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56243f9022e0 con 0x56243fa5c920 2024-05-14T09:36:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.986+0000 7f1d5effd700 1 --2- 172.21.15.2:0/1012838808 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56243fa5c920 0x56243fa5ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.986+0000 7f1d3effd700 10 client.15720 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.986+0000 7f1d5effd700 1 --2- 172.21.15.2:0/1012838808 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56243fa5c920 0x56243fa5ed40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ff80eaced7037 server_cookie=17a4e51c36e829a5 in_seq=0 out_seq=0 2024-05-14T09:36:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.987+0000 7f1d3effd700 1 -- 172.21.15.2:0/1012838808 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56243f9022e0 con 0x56243fa5c920 2024-05-14T09:36:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.987+0000 7f1d3effd700 10 client.15720 handle_command_reply: tid=0 2024-05-14T09:36:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.987+0000 7f1d74908b80 1 client.15720 shutdown 2024-05-14T09:36:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.987+0000 7f1d74908b80 1 -- 172.21.15.2:0/1012838808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f85ae40 msgr2=0x56243f9ea420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.987+0000 7f1d74908b80 1 --2- 172.21.15.2:0/1012838808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f85ae40 0x56243f9ea420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x56243f9d9620 tx=0x7f1d500047c0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.987+0000 7f1d74908b80 1 -- 172.21.15.2:0/1012838808 shutdown_connections 2024-05-14T09:36:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.987+0000 7f1d74908b80 1 --2- 172.21.15.2:0/1012838808 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56243fa5c920 0x56243fa5ed40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.987+0000 7f1d74908b80 1 --2- 172.21.15.2:0/1012838808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56243f9dd470 0x56243f9e7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.987+0000 7f1d3effd700 0 client.15720 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:36:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.987+0000 7f1d74908b80 1 --2- 172.21.15.2:0/1012838808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56243f85ae40 0x56243f9ea420 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.987+0000 7f1d74908b80 1 --2- 172.21.15.2:0/1012838808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56243f9d8c60 0x56243f9ea960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.987+0000 7f1d74908b80 1 -- 172.21.15.2:0/1012838808 >> 172.21.15.2:0/1012838808 conn(0x56243f8a2600 msgr2=0x56243f9d7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.987+0000 7f1d74908b80 1 -- 172.21.15.2:0/1012838808 shutdown_connections 2024-05-14T09:36:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.987+0000 7f1d74908b80 1 -- 172.21.15.2:0/1012838808 wait complete. 2024-05-14T09:36:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.987+0000 7f1d74908b80 20 client.15720 trim_cache size 0 max 16384 2024-05-14T09:36:50.992 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:36:50.992 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-05-14T09:36:50.992 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-05-14T09:36:50.992 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:36:50.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.988+0000 7f1d65ef4700 1 -- 172.21.15.2:0/459297781 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1d480b2900 msgr2=0x7f1d480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:50.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.988+0000 7f1d65ef4700 1 --2- 172.21.15.2:0/459297781 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1d480b2900 0x7f1d480b4e10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009d20 tx=0x7f1d50009260 comp rx=0 tx=0).stop 2024-05-14T09:36:50.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.988+0000 7f1d65ef4700 1 -- 172.21.15.2:0/459297781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d60107df0 msgr2=0x7f1d6019d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:50.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.988+0000 7f1d65ef4700 1 --2- 172.21.15.2:0/459297781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d60107df0 0x7f1d6019d5b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1d540098e0 tx=0x7f1d54004270 comp rx=0 tx=0).stop 2024-05-14T09:36:50.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.988+0000 7f1d65ef4700 1 -- 172.21.15.2:0/459297781 shutdown_connections 2024-05-14T09:36:50.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.988+0000 7f1d65ef4700 1 --2- 172.21.15.2:0/459297781 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1d480b2900 0x7f1d480b4e10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.988+0000 7f1d65ef4700 1 --2- 172.21.15.2:0/459297781 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d60108740 0x7f1d601a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.988+0000 7f1d65ef4700 1 --2- 172.21.15.2:0/459297781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d601034c0 0x7f1d6019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.988+0000 7f1d65ef4700 1 --2- 172.21.15.2:0/459297781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d60107df0 0x7f1d6019d5b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:50.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.988+0000 7f1d65ef4700 1 -- 172.21.15.2:0/459297781 >> 172.21.15.2:0/459297781 conn(0x7f1d600fedf0 msgr2=0x7f1d601076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:50.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.988+0000 7f1d65ef4700 1 -- 172.21.15.2:0/459297781 shutdown_connections 2024-05-14T09:36:50.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:50.988+0000 7f1d65ef4700 1 -- 172.21.15.2:0/459297781 wait complete. 2024-05-14T09:36:51.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 46869 15.86 MB/sec warmup 17 sec latency 15.826 ms 2024-05-14T09:36:51.883 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T09:36:51.884 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:36:52.152 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:36:52.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 49960 16.01 MB/sec warmup 18 sec latency 8.810 ms 2024-05-14T09:36:52.497 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:52 smithi002 ceph-mon[31557]: pgmap v481: 129 pgs: 129 active+clean; 15 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-05-14T09:36:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:52 smithi029 ceph-mon[36249]: pgmap v481: 129 pgs: 129 active+clean; 15 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-05-14T09:36:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:52 smithi026 ceph-mon[36056]: pgmap v481: 129 pgs: 129 active+clean; 15 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-05-14T09:36:53.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 52699 15.85 MB/sec warmup 19 sec latency 19.588 ms 2024-05-14T09:36:54.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 54876 15.64 MB/sec warmup 20 sec latency 24.190 ms 2024-05-14T09:36:54.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:54 smithi002 ceph-mon[31557]: pgmap v482: 129 pgs: 129 active+clean; 15 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 1.9 MiB/s wr, 402 op/s 2024-05-14T09:36:54.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:54 smithi029 ceph-mon[36249]: pgmap v482: 129 pgs: 129 active+clean; 15 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 1.9 MiB/s wr, 402 op/s 2024-05-14T09:36:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:54 smithi026 ceph-mon[36056]: pgmap v482: 129 pgs: 129 active+clean; 15 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 1.9 MiB/s wr, 402 op/s 2024-05-14T09:36:55.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 56782 15.48 MB/sec warmup 21 sec latency 257.392 ms 2024-05-14T09:36:56.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 57830 14.94 MB/sec warmup 22 sec latency 344.257 ms 2024-05-14T09:36:56.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:56 smithi002 ceph-mon[31557]: pgmap v483: 129 pgs: 129 active+clean; 15 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.3 MiB/s wr, 324 op/s 2024-05-14T09:36:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:56 smithi029 ceph-mon[36249]: pgmap v483: 129 pgs: 129 active+clean; 15 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.3 MiB/s wr, 324 op/s 2024-05-14T09:36:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:56 smithi026 ceph-mon[36056]: pgmap v483: 129 pgs: 129 active+clean; 15 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.3 MiB/s wr, 324 op/s 2024-05-14T09:36:57.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.083+0000 7f1e2e810700 1 -- 172.21.15.2:0/2390881964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e28109870 msgr2=0x7f1e281111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:57.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.083+0000 7f1e2e810700 1 --2- 172.21.15.2:0/2390881964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e28109870 0x7f1e281111d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000b0d0 tx=0x7f1e2000b3e0 comp rx=0 tx=0).stop 2024-05-14T09:36:57.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.084+0000 7f1e2e810700 1 -- 172.21.15.2:0/2390881964 shutdown_connections 2024-05-14T09:36:57.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.084+0000 7f1e2e810700 1 --2- 172.21.15.2:0/2390881964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e281045f0 0x7f1e28104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:57.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.084+0000 7f1e2e810700 1 --2- 172.21.15.2:0/2390881964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e28109870 0x7f1e281111d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:57.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.084+0000 7f1e2e810700 1 --2- 172.21.15.2:0/2390881964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e28108f20 0x7f1e28109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:57.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.084+0000 7f1e2e810700 1 -- 172.21.15.2:0/2390881964 >> 172.21.15.2:0/2390881964 conn(0x7f1e28100040 msgr2=0x7f1e28102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:57.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.084+0000 7f1e2e810700 1 -- 172.21.15.2:0/2390881964 shutdown_connections 2024-05-14T09:36:57.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.084+0000 7f1e2e810700 1 -- 172.21.15.2:0/2390881964 wait complete. 2024-05-14T09:36:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.085+0000 7f1e2e810700 1 Processor -- start 2024-05-14T09:36:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.085+0000 7f1e2e810700 1 -- start start 2024-05-14T09:36:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.085+0000 7f1e2e810700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e281045f0 0x7f1e2819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.085+0000 7f1e2e810700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e28108f20 0x7f1e2819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.085+0000 7f1e2e810700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e28109870 0x7f1e2819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.085+0000 7f1e2e810700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1e281144a0 con 0x7f1e281045f0 2024-05-14T09:36:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.085+0000 7f1e2e810700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28114320 con 0x7f1e28108f20 2024-05-14T09:36:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.085+0000 7f1e2e810700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28114620 con 0x7f1e28109870 2024-05-14T09:36:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.085+0000 7f1e27fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e281045f0 0x7f1e2819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.085+0000 7f1e27fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e281045f0 0x7f1e2819adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59190/0 (socket says 172.21.15.2:59190) 2024-05-14T09:36:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.085+0000 7f1e27fff700 1 -- 172.21.15.2:0/1012963461 learned_addr learned my addr 172.21.15.2:0/1012963461 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:36:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.085+0000 7f1e1ffff700 1 --2- 172.21.15.2:0/1012963461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e28108f20 0x7f1e2819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.085+0000 7f1e2cdad700 1 --2- 172.21.15.2:0/1012963461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e28109870 0x7f1e2819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.086+0000 7f1e27fff700 1 -- 172.21.15.2:0/1012963461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e28109870 msgr2=0x7f1e2819f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.086+0000 7f1e27fff700 1 --2- 172.21.15.2:0/1012963461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e28109870 0x7f1e2819f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.086+0000 7f1e27fff700 1 -- 172.21.15.2:0/1012963461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e28108f20 msgr2=0x7f1e2819b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.086+0000 7f1e27fff700 1 --2- 172.21.15.2:0/1012963461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e28108f20 0x7f1e2819b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.086+0000 7f1e27fff700 1 -- 172.21.15.2:0/1012963461 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2819fcc0 con 0x7f1e281045f0 2024-05-14T09:36:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.086+0000 7f1e2cdad700 1 --2- 172.21.15.2:0/1012963461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e28109870 0x7f1e2819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:36:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.086+0000 7f1e27fff700 1 --2- 172.21.15.2:0/1012963461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e281045f0 0x7f1e2819adf0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800e9a0 tx=0x7f1e1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.086+0000 7f1e25ffb700 1 -- 172.21.15.2:0/1012963461 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1800ca10 con 0x7f1e281045f0 2024-05-14T09:36:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.086+0000 7f1e25ffb700 1 -- 172.21.15.2:0/1012963461 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e1800cbb0 con 0x7f1e281045f0 2024-05-14T09:36:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.086+0000 7f1e2e810700 1 -- 172.21.15.2:0/1012963461 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2819ffb0 con 0x7f1e281045f0 2024-05-14T09:36:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.087+0000 7f1e25ffb700 1 -- 172.21.15.2:0/1012963461 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18018680 con 0x7f1e281045f0 2024-05-14T09:36:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.087+0000 7f1e2e810700 1 -- 172.21.15.2:0/1012963461 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e280759f0 con 0x7f1e281045f0 2024-05-14T09:36:57.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.088+0000 7f1e2e810700 1 -- 172.21.15.2:0/1012963461 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e280512c0 con 0x7f1e281045f0 2024-05-14T09:36:57.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.089+0000 7f1e25ffb700 1 -- 172.21.15.2:0/1012963461 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1e18010430 con 0x7f1e281045f0 2024-05-14T09:36:57.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.090+0000 7f1e25ffb700 1 --2- 172.21.15.2:0/1012963461 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1e140b2900 0x7f1e140b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:36:57.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.090+0000 7f1e1ffff700 1 --2- 172.21.15.2:0/1012963461 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1e140b2900 0x7f1e140b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:36:57.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.090+0000 7f1e25ffb700 1 -- 172.21.15.2:0/1012963461 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1e180e2720 con 0x7f1e281045f0 2024-05-14T09:36:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.091+0000 7f1e1ffff700 1 --2- 172.21.15.2:0/1012963461 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1e140b2900 0x7f1e140b4e10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1e28103a70 tx=0x7f1e1000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:36:57.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.095+0000 7f1e25ffb700 1 -- 172.21.15.2:0/1012963461 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e180aa020 con 0x7f1e281045f0 2024-05-14T09:36:57.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 60441 14.98 MB/sec warmup 23 sec latency 24.051 ms 2024-05-14T09:36:57.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.377+0000 7f1e2e810700 1 -- 172.21.15.2:0/1012963461 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e2819c150 con 0x7f1e281045f0 2024-05-14T09:36:57.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.377+0000 7f1e25ffb700 1 -- 172.21.15.2:0/1012963461 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11843 (secure 0 0 0) 0x7f1e180b20a0 con 0x7f1e281045f0 2024-05-14T09:36:57.379 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:36:57.379 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T09:34:49.632659+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24487,24497,24524]},"id":1}]} 2024-05-14T09:36:57.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.380+0000 7f1e2e810700 1 -- 172.21.15.2:0/1012963461 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1e140b2900 msgr2=0x7f1e140b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:57.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.380+0000 7f1e2e810700 1 --2- 172.21.15.2:0/1012963461 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1e140b2900 0x7f1e140b4e10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1e28103a70 tx=0x7f1e1000b410 comp rx=0 tx=0).stop 2024-05-14T09:36:57.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.380+0000 7f1e2e810700 1 -- 172.21.15.2:0/1012963461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e281045f0 msgr2=0x7f1e2819adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:36:57.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.380+0000 7f1e2e810700 1 --2- 172.21.15.2:0/1012963461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e281045f0 0x7f1e2819adf0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800e9a0 tx=0x7f1e1800ecb0 comp rx=0 tx=0).stop 2024-05-14T09:36:57.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.380+0000 7f1e2e810700 1 -- 172.21.15.2:0/1012963461 shutdown_connections 2024-05-14T09:36:57.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.380+0000 7f1e2e810700 1 --2- 172.21.15.2:0/1012963461 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1e140b2900 0x7f1e140b4e10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:57.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.380+0000 7f1e2e810700 1 --2- 172.21.15.2:0/1012963461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e28109870 0x7f1e2819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:57.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.380+0000 7f1e2e810700 1 --2- 172.21.15.2:0/1012963461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e281045f0 0x7f1e2819adf0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:57.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.380+0000 7f1e2e810700 1 --2- 172.21.15.2:0/1012963461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e28108f20 0x7f1e2819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:36:57.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.380+0000 7f1e2e810700 1 -- 172.21.15.2:0/1012963461 >> 172.21.15.2:0/1012963461 conn(0x7f1e28100040 msgr2=0x7f1e28108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:36:57.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.380+0000 7f1e2e810700 1 -- 172.21.15.2:0/1012963461 shutdown_connections 2024-05-14T09:36:57.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:36:57.380+0000 7f1e2e810700 1 -- 172.21.15.2:0/1012963461 wait complete. 2024-05-14T09:36:57.382 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 27 2024-05-14T09:36:58.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 62759 14.71 MB/sec warmup 24 sec latency 19.044 ms 2024-05-14T09:36:58.257 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T09:36:58.523 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:36:58.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:58 smithi002 ceph-mon[31557]: pgmap v484: 129 pgs: 129 active+clean; 33 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 4.6 MiB/s wr, 424 op/s 2024-05-14T09:36:58.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:36:58 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1012963461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:36:58.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:58 smithi029 ceph-mon[36249]: pgmap v484: 129 pgs: 129 active+clean; 33 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 4.6 MiB/s wr, 424 op/s 2024-05-14T09:36:58.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:36:58 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1012963461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:36:58.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:58 smithi026 ceph-mon[36056]: pgmap v484: 129 pgs: 129 active+clean; 33 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 4.6 MiB/s wr, 424 op/s 2024-05-14T09:36:58.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:36:58 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1012963461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:36:59.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 64962 14.68 MB/sec warmup 25 sec latency 21.515 ms 2024-05-14T09:37:00.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 67840 14.77 MB/sec warmup 26 sec latency 12.937 ms 2024-05-14T09:37:00.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.384+0000 7fdd1554d700 1 -- 172.21.15.2:0/1000954049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd10108760 msgr2=0x7fdd101100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:00.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.384+0000 7fdd1554d700 1 --2- 172.21.15.2:0/1000954049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd10108760 0x7fdd101100c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800b0d0 tx=0x7fdd0800b3e0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.384+0000 7fdd1554d700 1 -- 172.21.15.2:0/1000954049 shutdown_connections 2024-05-14T09:37:00.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.384+0000 7fdd1554d700 1 --2- 172.21.15.2:0/1000954049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd101034e0 0x7fdd101038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.384+0000 7fdd1554d700 1 --2- 172.21.15.2:0/1000954049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd10108760 0x7fdd101100c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.384+0000 7fdd1554d700 1 --2- 172.21.15.2:0/1000954049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd10107e10 0x7fdd10108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.384+0000 7fdd1554d700 1 -- 172.21.15.2:0/1000954049 >> 172.21.15.2:0/1000954049 conn(0x7fdd100fef30 msgr2=0x7fdd10101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:00.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.384+0000 7fdd1554d700 1 -- 172.21.15.2:0/1000954049 shutdown_connections 2024-05-14T09:37:00.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.384+0000 7fdd1554d700 1 -- 172.21.15.2:0/1000954049 wait complete. 2024-05-14T09:37:00.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.385+0000 7fdd1554d700 1 Processor -- start 2024-05-14T09:37:00.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd1554d700 1 -- start start 2024-05-14T09:37:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd1554d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd101034e0 0x7fdd1010a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd1554d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd10107e10 0x7fdd1010a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd1554d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd10108760 0x7fdd1010abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd1554d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101133e0 con 0x7fdd10108760 2024-05-14T09:37:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd1554d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdd10113260 con 0x7fdd10107e10 2024-05-14T09:37:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd1554d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdd10113560 con 0x7fdd101034e0 2024-05-14T09:37:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd0f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd10108760 0x7fdd1010abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd0e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd10107e10 0x7fdd1010a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd0e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd10107e10 0x7fdd1010a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:53704/0 (socket says 172.21.15.2:53704) 2024-05-14T09:37:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd0f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd10108760 0x7fdd1010abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57916/0 (socket says 172.21.15.2:57916) 2024-05-14T09:37:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd0e7fc700 1 -- 172.21.15.2:0/2450310744 learned_addr learned my addr 172.21.15.2:0/2450310744 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd0effd700 1 --2- 172.21.15.2:0/2450310744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd101034e0 0x7fdd1010a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd0e7fc700 1 -- 172.21.15.2:0/2450310744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd101034e0 msgr2=0x7fdd1010a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:00.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd0e7fc700 1 --2- 172.21.15.2:0/2450310744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd101034e0 0x7fdd1010a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd0e7fc700 1 -- 172.21.15.2:0/2450310744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd10108760 msgr2=0x7fdd1010abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:00.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd0e7fc700 1 --2- 172.21.15.2:0/2450310744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd10108760 0x7fdd1010abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.386+0000 7fdd0e7fc700 1 -- 172.21.15.2:0/2450310744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd101a9180 con 0x7fdd10107e10 2024-05-14T09:37:00.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.387+0000 7fdd0e7fc700 1 --2- 172.21.15.2:0/2450310744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd10107e10 0x7fdd1010a6a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400d980 tx=0x7fdd0400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:00.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.387+0000 7fdcf7fff700 1 -- 172.21.15.2:0/2450310744 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04017070 con 0x7fdd10107e10 2024-05-14T09:37:00.390 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:00 smithi002 ceph-mon[31557]: pgmap v485: 129 pgs: 129 active+clean; 45 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.0 MiB/s wr, 534 op/s 2024-05-14T09:37:00.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.387+0000 7fdcf7fff700 1 -- 172.21.15.2:0/2450310744 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd0400fd40 con 0x7fdd10107e10 2024-05-14T09:37:00.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.387+0000 7fdcf7fff700 1 -- 172.21.15.2:0/2450310744 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0400e3b0 con 0x7fdd10107e10 2024-05-14T09:37:00.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.387+0000 7fdd1554d700 1 -- 172.21.15.2:0/2450310744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd101a9470 con 0x7fdd10107e10 2024-05-14T09:37:00.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.387+0000 7fdd1554d700 1 -- 172.21.15.2:0/2450310744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd101a9960 con 0x7fdd10107e10 2024-05-14T09:37:00.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.390+0000 7fdcf7fff700 1 -- 172.21.15.2:0/2450310744 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fdd04010610 con 0x7fdd10107e10 2024-05-14T09:37:00.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.391+0000 7fdcf7fff700 1 --2- 172.21.15.2:0/2450310744 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdcf80b2880 0x7fdcf80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.391+0000 7fdcf7fff700 1 -- 172.21.15.2:0/2450310744 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fdd040e2050 con 0x7fdd10107e10 2024-05-14T09:37:00.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.391+0000 7fdd0effd700 1 --2- 172.21.15.2:0/2450310744 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdcf80b2880 0x7fdcf80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.392+0000 7fdd0effd700 1 --2- 172.21.15.2:0/2450310744 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdcf80b2880 0x7fdcf80b4d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009680 tx=0x7fdd00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:00.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.392+0000 7fdd23f61b80 1 Processor -- start 2024-05-14T09:37:00.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd23f61b80 1 -- start start 2024-05-14T09:37:00.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd23f61b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a7466b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd23f61b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a7466bd80 0x564a746658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd23f61b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a74665e10 0x564a74666220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd23f61b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564a74587940 con 0x564a74665e10 2024-05-14T09:37:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd23f61b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564a744ecda0 con 0x564a744e7e40 2024-05-14T09:37:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd23f61b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564a74583900 con 0x564a7466bd80 2024-05-14T09:37:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd0effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a7466b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd0effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a7466b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:53714/0 (socket says 172.21.15.2:53714) 2024-05-14T09:37:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd0effd700 1 -- 172.21.15.2:0/1205437969 learned_addr learned my addr 172.21.15.2:0/1205437969 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd0f7fe700 1 --2- 172.21.15.2:0/1205437969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a74665e10 0x564a74666220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd0e7fc700 1 --2- 172.21.15.2:0/1205437969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a7466bd80 0x564a746658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd0f7fe700 1 -- 172.21.15.2:0/1205437969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a7466bd80 msgr2=0x564a746658d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd0f7fe700 1 --2- 172.21.15.2:0/1205437969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a7466bd80 0x564a746658d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd0f7fe700 1 -- 172.21.15.2:0/1205437969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 msgr2=0x564a7466b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd0f7fe700 1 --2- 172.21.15.2:0/1205437969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a7466b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd0f7fe700 1 -- 172.21.15.2:0/1205437969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a744d48d0 con 0x564a74665e10 2024-05-14T09:37:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.393+0000 7fdd0effd700 1 --2- 172.21.15.2:0/1205437969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a7466b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:00.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.394+0000 7fdd0f7fe700 1 --2- 172.21.15.2:0/1205437969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a74665e10 0x564a74666220 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800b770 tx=0x7fdd080098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bf4a62445da4e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:00.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.394+0000 7fdcf57fa700 1 -- 172.21.15.2:0/1205437969 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0800baa0 con 0x564a74665e10 2024-05-14T09:37:00.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.394+0000 7fdcf57fa700 1 -- 172.21.15.2:0/1205437969 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd08020070 con 0x564a74665e10 2024-05-14T09:37:00.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.394+0000 7fdcf57fa700 1 -- 172.21.15.2:0/1205437969 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd080052f0 con 0x564a74665e10 2024-05-14T09:37:00.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.395+0000 7fdd23f61b80 1 -- 172.21.15.2:0/1205437969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a74665e10 msgr2=0x564a74666220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:00.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.395+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/1205437969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a74665e10 0x564a74666220 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800b770 tx=0x7fdd080098c0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.395+0000 7fdd23f61b80 1 -- 172.21.15.2:0/1205437969 shutdown_connections 2024-05-14T09:37:00.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.395+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/1205437969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a7466b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.395+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/1205437969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a74665e10 0x564a74666220 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.395+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/1205437969 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a7466bd80 0x564a746658d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.396+0000 7fdd23f61b80 1 -- 172.21.15.2:0/1205437969 >> 172.21.15.2:0/1205437969 conn(0x564a7452f630 msgr2=0x564a74660a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:00.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.396+0000 7fdd23f61b80 1 -- 172.21.15.2:0/1205437969 shutdown_connections 2024-05-14T09:37:00.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.396+0000 7fdd23f61b80 1 -- 172.21.15.2:0/1205437969 wait complete. 2024-05-14T09:37:00.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.396+0000 7fdd23f61b80 1 -- start start 2024-05-14T09:37:00.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.396+0000 7fdd23f61b80 1 Processor -- start 2024-05-14T09:37:00.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd23f61b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a7466f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd23f61b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a74665e10 0x564a7466fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd23f61b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7466bd80 0x564a7466ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd0effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a7466f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd0f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7466bd80 0x564a7466ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd23f61b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564a746727c0 con 0x564a7466bd80 2024-05-14T09:37:00.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd23f61b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564a74540ed0 con 0x564a744e7e40 2024-05-14T09:37:00.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd23f61b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564a74672940 con 0x564a74665e10 2024-05-14T09:37:00.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd0effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a7466f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:53728/0 (socket says 172.21.15.2:53728) 2024-05-14T09:37:00.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd0effd700 1 -- 172.21.15.2:0/1472219250 learned_addr learned my addr 172.21.15.2:0/1472219250 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:00.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd0e7fc700 1 --2- 172.21.15.2:0/1472219250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a74665e10 0x564a7466fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd23f61b80 10 client.0 _ll_register_callbacks cb 0x564a7451f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:00.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd0effd700 1 -- 172.21.15.2:0/1472219250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a74665e10 msgr2=0x564a7466fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:00.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd0effd700 1 --2- 172.21.15.2:0/1472219250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a74665e10 0x564a7466fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd0effd700 1 -- 172.21.15.2:0/1472219250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7466bd80 msgr2=0x564a7466ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:00.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.397+0000 7fdd0effd700 1 --2- 172.21.15.2:0/1472219250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7466bd80 0x564a7466ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.398+0000 7fdd0effd700 1 -- 172.21.15.2:0/1472219250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a7453e760 con 0x564a744e7e40 2024-05-14T09:37:00.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.398+0000 7fdd0effd700 1 --2- 172.21.15.2:0/1472219250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a7466f4c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdd000045a0 tx=0x7fdd000047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=108fafcad07e0452 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:00.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.398+0000 7fdcf2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:37:00.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.398+0000 7fdcf2ff5700 1 -- 172.21.15.2:0/1472219250 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00027420 con 0x564a744e7e40 2024-05-14T09:37:00.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.398+0000 7fdcf2ff5700 1 -- 172.21.15.2:0/1472219250 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fdd0001d410 con 0x564a744e7e40 2024-05-14T09:37:00.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.398+0000 7fdd23f61b80 1 -- 172.21.15.2:0/1472219250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd08020070 con 0x564a744e7e40 2024-05-14T09:37:00.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.399+0000 7fdcf2ff5700 1 -- 172.21.15.2:0/1472219250 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd0001d650 con 0x564a744e7e40 2024-05-14T09:37:00.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.399+0000 7fdd23f61b80 10 client.25436 fetch_fsmap learned FSMap version 27 2024-05-14T09:37:00.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.399+0000 7fdd23f61b80 1 -- 172.21.15.2:0/1472219250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564a7467d3e0 con 0x564a744e7e40 2024-05-14T09:37:00.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.399+0000 7fdcf2ff5700 1 -- 172.21.15.2:0/1472219250 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd000275c0 con 0x564a744e7e40 2024-05-14T09:37:00.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.399+0000 7fdcf2ff5700 1 -- 172.21.15.2:0/1472219250 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdd0001c410 con 0x564a744e7e40 2024-05-14T09:37:00.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.399+0000 7fdd23f61b80 10 client.25436 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:37:00.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.399+0000 7fdd23f61b80 10 client.25436 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T09:37:00.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.399+0000 7fdd23f61b80 20 client.25436 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:00.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.399+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/1472219250 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x564a7467fa20 0x564a74681e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.400+0000 7fdd23f61b80 4 client.25436 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:37:00.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.400+0000 7fdd23f61b80 1 -- 172.21.15.2:0/1472219250 --> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdd08020070 con 0x564a7467fa20 2024-05-14T09:37:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.400+0000 7fdd0e7fc700 1 --2- 172.21.15.2:0/1472219250 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x564a7467fa20 0x564a74681e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.400+0000 7fdcf2ff5700 10 client.25436 ms_handle_connect on v2:172.21.15.29:6838/3625063120 2024-05-14T09:37:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.400+0000 7fdd0e7fc700 1 --2- 172.21.15.2:0/1472219250 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x564a7467fa20 0x564a74681e80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11985f7f8eea2d0e server_cookie=f1dae4486c1ae0ad in_seq=0 out_seq=0 2024-05-14T09:37:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.401+0000 7fdcf2ff5700 1 -- 172.21.15.2:0/1472219250 <== mds.2 v2:172.21.15.29:6838/3625063120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fdd08020070 con 0x564a7467fa20 2024-05-14T09:37:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.401+0000 7fdcf2ff5700 10 client.25436 handle_command_reply: tid=0 2024-05-14T09:37:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.401+0000 7fdd23f61b80 1 client.25436 shutdown 2024-05-14T09:37:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.401+0000 7fdd23f61b80 1 -- 172.21.15.2:0/1472219250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 msgr2=0x564a7466f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.401+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/1472219250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a7466f4c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdd000045a0 tx=0x7fdd000047c0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.401+0000 7fdd23f61b80 1 -- 172.21.15.2:0/1472219250 shutdown_connections 2024-05-14T09:37:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.401+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/1472219250 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x564a7467fa20 0x564a74681e80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.401+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/1472219250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a7466f4c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.401+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/1472219250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7466bd80 0x564a7466ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.401+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/1472219250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a74665e10 0x564a7466fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.401+0000 7fdcf2ff5700 0 client.25436 ms_handle_reset on v2:172.21.15.29:6838/3625063120 2024-05-14T09:37:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.401+0000 7fdd23f61b80 1 -- 172.21.15.2:0/1472219250 >> 172.21.15.2:0/1472219250 conn(0x564a7452f630 msgr2=0x564a746648c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:00.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.402+0000 7fdd23f61b80 1 -- 172.21.15.2:0/1472219250 shutdown_connections 2024-05-14T09:37:00.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.402+0000 7fdd23f61b80 1 -- 172.21.15.2:0/1472219250 wait complete. 2024-05-14T09:37:00.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.402+0000 7fdd23f61b80 20 client.25436 trim_cache size 0 max 16384 2024-05-14T09:37:00.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.418+0000 7fdd23f61b80 1 Processor -- start 2024-05-14T09:37:00.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.418+0000 7fdd23f61b80 1 -- start start 2024-05-14T09:37:00.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.418+0000 7fdd23f61b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a744e7e40 0x564a746658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.418+0000 7fdd23f61b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a74665e10 0x564a7466fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.418+0000 7fdd23f61b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a7466bd80 0x564a7467b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.418+0000 7fdd23f61b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101b0c10 con 0x564a744e7e40 2024-05-14T09:37:00.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.418+0000 7fdd23f61b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101ae4a0 con 0x564a74665e10 2024-05-14T09:37:00.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.418+0000 7fdd23f61b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101ac310 con 0x564a7466bd80 2024-05-14T09:37:00.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.418+0000 7fdd0e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a74665e10 0x564a7466fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.418+0000 7fdd0e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a74665e10 0x564a7466fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:53730/0 (socket says 172.21.15.2:53730) 2024-05-14T09:37:00.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.418+0000 7fdd0e7fc700 1 -- 172.21.15.2:0/438834511 learned_addr learned my addr 172.21.15.2:0/438834511 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:00.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.418+0000 7fdd0f7fe700 1 --2- 172.21.15.2:0/438834511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a7466bd80 0x564a7467b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.418+0000 7fdd0effd700 1 --2- 172.21.15.2:0/438834511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a744e7e40 0x564a746658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.419+0000 7fdd0e7fc700 1 -- 172.21.15.2:0/438834511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a7466bd80 msgr2=0x564a7467b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.419+0000 7fdd0e7fc700 1 --2- 172.21.15.2:0/438834511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a7466bd80 0x564a7467b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.419+0000 7fdd0e7fc700 1 -- 172.21.15.2:0/438834511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a744e7e40 msgr2=0x564a746658d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.419+0000 7fdd0e7fc700 1 --2- 172.21.15.2:0/438834511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a744e7e40 0x564a746658d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.419+0000 7fdd0e7fc700 1 -- 172.21.15.2:0/438834511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a746702c0 con 0x564a74665e10 2024-05-14T09:37:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.419+0000 7fdd0f7fe700 1 --2- 172.21.15.2:0/438834511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a7466bd80 0x564a7467b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:37:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.419+0000 7fdd0e7fc700 1 --2- 172.21.15.2:0/438834511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a74665e10 0x564a7466fd80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdd040a52d0 tx=0x7fdd040109b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c137e21572db6fc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.419+0000 7fdcf4ff9700 1 -- 172.21.15.2:0/438834511 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd040c2020 con 0x564a74665e10 2024-05-14T09:37:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.419+0000 7fdcf4ff9700 1 -- 172.21.15.2:0/438834511 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd040a9250 con 0x564a74665e10 2024-05-14T09:37:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.419+0000 7fdcf4ff9700 1 -- 172.21.15.2:0/438834511 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04020040 con 0x564a74665e10 2024-05-14T09:37:00.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.420+0000 7fdd23f61b80 1 -- 172.21.15.2:0/438834511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a74665e10 msgr2=0x564a7466fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:00.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.420+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/438834511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a74665e10 0x564a7466fd80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdd040a52d0 tx=0x7fdd040109b0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.420+0000 7fdd23f61b80 1 -- 172.21.15.2:0/438834511 shutdown_connections 2024-05-14T09:37:00.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.420+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/438834511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a7466bd80 0x564a7467b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.420+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/438834511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a744e7e40 0x564a746658d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.420+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/438834511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a74665e10 0x564a7466fd80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.420+0000 7fdd23f61b80 1 -- 172.21.15.2:0/438834511 >> 172.21.15.2:0/438834511 conn(0x564a7452f630 msgr2=0x564a7466ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:00.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.420+0000 7fdd23f61b80 1 -- 172.21.15.2:0/438834511 shutdown_connections 2024-05-14T09:37:00.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.420+0000 7fdd23f61b80 1 -- 172.21.15.2:0/438834511 wait complete. 2024-05-14T09:37:00.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.421+0000 7fdd23f61b80 1 -- start start 2024-05-14T09:37:00.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.421+0000 7fdd23f61b80 1 Processor -- start 2024-05-14T09:37:00.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.421+0000 7fdd23f61b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a74686cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.421+0000 7fdd23f61b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a74665e10 0x564a74687230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.421+0000 7fdd23f61b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7466bd80 0x564a74687770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.421+0000 7fdd23f61b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101ac960 con 0x564a7466bd80 2024-05-14T09:37:00.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.421+0000 7fdd23f61b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101addd0 con 0x564a744e7e40 2024-05-14T09:37:00.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.421+0000 7fdd23f61b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101ad030 con 0x564a74665e10 2024-05-14T09:37:00.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.421+0000 7fdd23f61b80 10 client.0 _ll_register_callbacks cb 0x564a7451f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:00.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.421+0000 7fdd0effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a74686cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.421+0000 7fdd0effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a74686cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:53742/0 (socket says 172.21.15.2:53742) 2024-05-14T09:37:00.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.421+0000 7fdd0effd700 1 -- 172.21.15.2:0/3785576262 learned_addr learned my addr 172.21.15.2:0/3785576262 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:00.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.422+0000 7fdd0f7fe700 1 --2- 172.21.15.2:0/3785576262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7466bd80 0x564a74687770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.422+0000 7fdd0e7fc700 1 --2- 172.21.15.2:0/3785576262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a74665e10 0x564a74687230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.422+0000 7fdd0effd700 1 -- 172.21.15.2:0/3785576262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a74665e10 msgr2=0x564a74687230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:00.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.422+0000 7fdd0effd700 1 --2- 172.21.15.2:0/3785576262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a74665e10 0x564a74687230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.422+0000 7fdd0effd700 1 -- 172.21.15.2:0/3785576262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7466bd80 msgr2=0x564a74687770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:00.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.422+0000 7fdd0effd700 1 --2- 172.21.15.2:0/3785576262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7466bd80 0x564a74687770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.422+0000 7fdd0effd700 1 -- 172.21.15.2:0/3785576262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a74678650 con 0x564a744e7e40 2024-05-14T09:37:00.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.422+0000 7fdd0f7fe700 1 --2- 172.21.15.2:0/3785576262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7466bd80 0x564a74687770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:00.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.422+0000 7fdd0effd700 1 --2- 172.21.15.2:0/3785576262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a74686cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdd0001c880 tx=0x7fdd00003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd4bdfbe8b703993 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:00.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.422+0000 7fdcf2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:37:00.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.422+0000 7fdcf2ff5700 1 -- 172.21.15.2:0/3785576262 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0001caf0 con 0x564a744e7e40 2024-05-14T09:37:00.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.422+0000 7fdcf2ff5700 1 -- 172.21.15.2:0/3785576262 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fdd00023040 con 0x564a744e7e40 2024-05-14T09:37:00.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.422+0000 7fdd23f61b80 1 -- 172.21.15.2:0/3785576262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd040a9250 con 0x564a744e7e40 2024-05-14T09:37:00.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.423+0000 7fdcf2ff5700 1 -- 172.21.15.2:0/3785576262 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd0001cc90 con 0x564a744e7e40 2024-05-14T09:37:00.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.423+0000 7fdd23f61b80 10 client.25442 fetch_fsmap learned FSMap version 27 2024-05-14T09:37:00.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.423+0000 7fdd23f61b80 1 -- 172.21.15.2:0/3785576262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564a746ea2e0 con 0x564a744e7e40 2024-05-14T09:37:00.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.423+0000 7fdcf2ff5700 1 -- 172.21.15.2:0/3785576262 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd040a9250 con 0x564a744e7e40 2024-05-14T09:37:00.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.423+0000 7fdcf2ff5700 1 -- 172.21.15.2:0/3785576262 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdd00035900 con 0x564a744e7e40 2024-05-14T09:37:00.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.423+0000 7fdd23f61b80 10 client.25442 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:37:00.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.423+0000 7fdd23f61b80 10 client.25442 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T09:37:00.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.423+0000 7fdd23f61b80 20 client.25442 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:00.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.423+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/3785576262 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x564a746eaa70 0x564a746ece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.423+0000 7fdd23f61b80 4 client.25442 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T09:37:00.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.423+0000 7fdd23f61b80 1 -- 172.21.15.2:0/3785576262 --> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a74595320 con 0x564a746eaa70 2024-05-14T09:37:00.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.423+0000 7fdd0e7fc700 1 --2- 172.21.15.2:0/3785576262 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x564a746eaa70 0x564a746ece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.424+0000 7fdcf2ff5700 10 client.25442 ms_handle_connect on v2:172.21.15.29:6838/3625063120 2024-05-14T09:37:00.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.424+0000 7fdd0e7fc700 1 --2- 172.21.15.2:0/3785576262 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x564a746eaa70 0x564a746ece90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d91d5a9ec7defe28 server_cookie=be7126def39c29ff in_seq=0 out_seq=0 2024-05-14T09:37:00.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.424+0000 7fdcf2ff5700 1 -- 172.21.15.2:0/3785576262 <== mds.2 v2:172.21.15.29:6838/3625063120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a74595320 con 0x564a746eaa70 2024-05-14T09:37:00.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.424+0000 7fdcf2ff5700 10 client.25442 handle_command_reply: tid=0 2024-05-14T09:37:00.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.424+0000 7fdd23f61b80 1 client.25442 shutdown 2024-05-14T09:37:00.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.424+0000 7fdd23f61b80 1 -- 172.21.15.2:0/3785576262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 msgr2=0x564a74686cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:00.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.424+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/3785576262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a74686cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdd0001c880 tx=0x7fdd00003cf0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.424+0000 7fdd23f61b80 1 -- 172.21.15.2:0/3785576262 shutdown_connections 2024-05-14T09:37:00.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.424+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/3785576262 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x564a746eaa70 0x564a746ece90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.424+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/3785576262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a744e7e40 0x564a74686cf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.424+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/3785576262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7466bd80 0x564a74687770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.424+0000 7fdd23f61b80 1 --2- 172.21.15.2:0/3785576262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564a74665e10 0x564a74687230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.424+0000 7fdcf2ff5700 0 client.25442 ms_handle_reset on v2:172.21.15.29:6838/3625063120 2024-05-14T09:37:00.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.424+0000 7fdd23f61b80 1 -- 172.21.15.2:0/3785576262 >> 172.21.15.2:0/3785576262 conn(0x564a7452f630 msgr2=0x564a74670b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:00.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.425+0000 7fdd23f61b80 1 -- 172.21.15.2:0/3785576262 shutdown_connections 2024-05-14T09:37:00.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.425+0000 7fdd23f61b80 1 -- 172.21.15.2:0/3785576262 wait complete. 2024-05-14T09:37:00.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.425+0000 7fdd23f61b80 20 client.25442 trim_cache size 0 max 16384 2024-05-14T09:37:00.431 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-05-14T09:37:00.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.426+0000 7fdd1554d700 1 -- 172.21.15.2:0/2450310744 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdcf80b2880 msgr2=0x7fdcf80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:00.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.426+0000 7fdd1554d700 1 --2- 172.21.15.2:0/2450310744 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdcf80b2880 0x7fdcf80b4d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009680 tx=0x7fdd00009260 comp rx=0 tx=0).stop 2024-05-14T09:37:00.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.426+0000 7fdd1554d700 1 -- 172.21.15.2:0/2450310744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd10107e10 msgr2=0x7fdd1010a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:00.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.426+0000 7fdd1554d700 1 --2- 172.21.15.2:0/2450310744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd10107e10 0x7fdd1010a6a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400d980 tx=0x7fdd0400dc90 comp rx=0 tx=0).stop 2024-05-14T09:37:00.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.427+0000 7fdd1554d700 1 -- 172.21.15.2:0/2450310744 shutdown_connections 2024-05-14T09:37:00.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.427+0000 7fdd1554d700 1 --2- 172.21.15.2:0/2450310744 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdcf80b2880 0x7fdcf80b4d90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.427+0000 7fdd1554d700 1 --2- 172.21.15.2:0/2450310744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd10108760 0x7fdd1010abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.427+0000 7fdd1554d700 1 --2- 172.21.15.2:0/2450310744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdd101034e0 0x7fdd1010a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.427+0000 7fdd1554d700 1 --2- 172.21.15.2:0/2450310744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd10107e10 0x7fdd1010a6a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.427+0000 7fdd1554d700 1 -- 172.21.15.2:0/2450310744 >> 172.21.15.2:0/2450310744 conn(0x7fdd100fef30 msgr2=0x7fdd10101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:00.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.427+0000 7fdd1554d700 1 -- 172.21.15.2:0/2450310744 shutdown_connections 2024-05-14T09:37:00.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:00.427+0000 7fdd1554d700 1 -- 172.21.15.2:0/2450310744 wait complete. 2024-05-14T09:37:00.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:00 smithi029 ceph-mon[36249]: pgmap v485: 129 pgs: 129 active+clean; 45 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.0 MiB/s wr, 534 op/s 2024-05-14T09:37:00.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:00 smithi026 ceph-mon[36056]: pgmap v485: 129 pgs: 129 active+clean; 45 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.0 MiB/s wr, 534 op/s 2024-05-14T09:37:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.965+0000 7fc420a22640 1 -- 172.21.15.2:0/4204396718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc41c1022b0 msgr2=0x7fc41c102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.965+0000 7fc420a22640 1 --2- 172.21.15.2:0/4204396718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc41c1022b0 0x7fc41c102fb0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00aa10 tx=0x7fc40c02b720 comp rx=0 tx=0).stop 2024-05-14T09:37:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.965+0000 7fc420a22640 1 -- 172.21.15.2:0/4204396718 shutdown_connections 2024-05-14T09:37:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.965+0000 7fc420a22640 1 --2- 172.21.15.2:0/4204396718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc41c1081f0 0x7fc41c10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.965+0000 7fc420a22640 1 --2- 172.21.15.2:0/4204396718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc41c1034f0 0x7fc41c103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.965+0000 7fc420a22640 1 --2- 172.21.15.2:0/4204396718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc41c1022b0 0x7fc41c102fb0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.965+0000 7fc420a22640 1 -- 172.21.15.2:0/4204396718 >> 172.21.15.2:0/4204396718 conn(0x7fc41c0fd480 msgr2=0x7fc41c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.965+0000 7fc420a22640 1 -- 172.21.15.2:0/4204396718 shutdown_connections 2024-05-14T09:37:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.965+0000 7fc420a22640 1 -- 172.21.15.2:0/4204396718 wait complete. 2024-05-14T09:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.966+0000 7fc420a22640 1 Processor -- start 2024-05-14T09:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.966+0000 7fc420a22640 1 -- start start 2024-05-14T09:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.966+0000 7fc420a22640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc41c1022b0 0x7fc41c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.966+0000 7fc420a22640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc41c1034f0 0x7fc41c19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.966+0000 7fc420a22640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc41c1081f0 0x7fc41c1a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.966+0000 7fc420a22640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c10d590 con 0x7fc41c1022b0 2024-05-14T09:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.967+0000 7fc420a22640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c10d410 con 0x7fc41c1034f0 2024-05-14T09:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.967+0000 7fc420a22640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c10d710 con 0x7fc41c1081f0 2024-05-14T09:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.967+0000 7fc41a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc41c1022b0 0x7fc41c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.967+0000 7fc413fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc41c1034f0 0x7fc41c19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.967+0000 7fc41a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc41c1022b0 0x7fc41c19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57974/0 (socket says 172.21.15.2:57974) 2024-05-14T09:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.967+0000 7fc41a59c640 1 -- 172.21.15.2:0/920136004 learned_addr learned my addr 172.21.15.2:0/920136004 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.967+0000 7fc41ad9d640 1 --2- 172.21.15.2:0/920136004 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc41c1081f0 0x7fc41c1a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.967+0000 7fc413fff640 1 -- 172.21.15.2:0/920136004 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc41c1081f0 msgr2=0x7fc41c1a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.967+0000 7fc413fff640 1 --2- 172.21.15.2:0/920136004 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc41c1081f0 0x7fc41c1a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.967+0000 7fc413fff640 1 -- 172.21.15.2:0/920136004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc41c1022b0 msgr2=0x7fc41c19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.967+0000 7fc413fff640 1 --2- 172.21.15.2:0/920136004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc41c1022b0 0x7fc41c19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.967+0000 7fc413fff640 1 -- 172.21.15.2:0/920136004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41c1a1e40 con 0x7fc41c1034f0 2024-05-14T09:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.967+0000 7fc413fff640 1 --2- 172.21.15.2:0/920136004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc41c1034f0 0x7fc41c19a840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc40400e9f0 tx=0x7fc40400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.967+0000 7fc4137fe640 1 -- 172.21.15.2:0/920136004 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40400ccc0 con 0x7fc41c1034f0 2024-05-14T09:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.968+0000 7fc420a22640 1 -- 172.21.15.2:0/920136004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41c1a2130 con 0x7fc41c1034f0 2024-05-14T09:37:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.968+0000 7fc4137fe640 1 -- 172.21.15.2:0/920136004 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc404005020 con 0x7fc41c1034f0 2024-05-14T09:37:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.968+0000 7fc4137fe640 1 -- 172.21.15.2:0/920136004 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404004a00 con 0x7fc41c1034f0 2024-05-14T09:37:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.968+0000 7fc420a22640 1 -- 172.21.15.2:0/920136004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc41c1a2680 con 0x7fc41c1034f0 2024-05-14T09:37:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.971+0000 7fc4279ad740 1 Processor -- start 2024-05-14T09:37:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.971+0000 7fc4279ad740 1 -- start start 2024-05-14T09:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.971+0000 7fc4279ad740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4e71b0 0x55e62c888a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.971+0000 7fc4279ad740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c888fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.971+0000 7fc4279ad740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 0x55e62c88fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.971+0000 7fc4279ad740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e62c7ed5d0 con 0x55e62c4be2f0 2024-05-14T09:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.971+0000 7fc4279ad740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e62c7a6890 con 0x55e62c4c64d0 2024-05-14T09:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.971+0000 7fc4279ad740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e62c7e9590 con 0x55e62c4e71b0 2024-05-14T09:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.971+0000 7fc41a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c888fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.971+0000 7fc41ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4e71b0 0x55e62c888a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.972+0000 7fc41a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c888fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57976/0 (socket says 172.21.15.2:57976) 2024-05-14T09:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.972+0000 7fc41a59c640 1 -- 172.21.15.2:0/4156536823 learned_addr learned my addr 172.21.15.2:0/4156536823 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.972+0000 7fc4137fe640 1 -- 172.21.15.2:0/920136004 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc404004ba0 con 0x7fc41c1034f0 2024-05-14T09:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.972+0000 7fc41ad9d640 1 -- 172.21.15.2:0/4156536823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 msgr2=0x55e62c88fee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:37:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.972+0000 7fc41ad9d640 1 --2- 172.21.15.2:0/4156536823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 0x55e62c88fee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.972+0000 7fc41ad9d640 1 -- 172.21.15.2:0/4156536823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 msgr2=0x55e62c888fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.972+0000 7fc41ad9d640 1 --2- 172.21.15.2:0/4156536823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c888fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.972+0000 7fc41ad9d640 1 -- 172.21.15.2:0/4156536823 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e62c4ff210 con 0x55e62c4e71b0 2024-05-14T09:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.972+0000 7fc41ad9d640 1 --2- 172.21.15.2:0/4156536823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4e71b0 0x55e62c888a60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc414009f00 tx=0x7fc414011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=644891339f49027f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.972+0000 7fc410ff9640 1 -- 172.21.15.2:0/4156536823 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414004d20 con 0x55e62c4e71b0 2024-05-14T09:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.972+0000 7fc410ff9640 1 -- 172.21.15.2:0/4156536823 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc414004030 con 0x55e62c4e71b0 2024-05-14T09:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.972+0000 7fc410ff9640 1 -- 172.21.15.2:0/4156536823 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414005280 con 0x55e62c4e71b0 2024-05-14T09:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.973+0000 7fc4137fe640 1 --2- 172.21.15.2:0/920136004 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc4080b28a0 0x7fc4080b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.973+0000 7fc41a59c640 1 --2- 172.21.15.2:0/920136004 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc4080b28a0 0x7fc4080b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.973+0000 7fc4137fe640 1 -- 172.21.15.2:0/920136004 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fc4040e25d0 con 0x7fc41c1034f0 2024-05-14T09:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.973+0000 7fc4279ad740 1 -- 172.21.15.2:0/4156536823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4e71b0 msgr2=0x55e62c888a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.973+0000 7fc4279ad740 1 --2- 172.21.15.2:0/4156536823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4e71b0 0x55e62c888a60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc414009f00 tx=0x7fc414011810 comp rx=0 tx=0).stop 2024-05-14T09:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.973+0000 7fc41a59c640 1 --2- 172.21.15.2:0/920136004 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc4080b28a0 0x7fc4080b4db0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00a7d0 tx=0x7fc40c0305b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 1 -- 172.21.15.2:0/4156536823 shutdown_connections 2024-05-14T09:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 1 --2- 172.21.15.2:0/4156536823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 0x55e62c88fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 1 --2- 172.21.15.2:0/4156536823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c888fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 1 --2- 172.21.15.2:0/4156536823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4e71b0 0x55e62c888a60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 1 -- 172.21.15.2:0/4156536823 >> 172.21.15.2:0/4156536823 conn(0x55e62c4db300 msgr2=0x55e62c8843a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 1 -- 172.21.15.2:0/4156536823 shutdown_connections 2024-05-14T09:37:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 1 -- 172.21.15.2:0/4156536823 wait complete. 2024-05-14T09:37:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 1 -- start start 2024-05-14T09:37:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 1 Processor -- start 2024-05-14T09:37:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c8927d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 0x55e62c892d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4e71b0 0x55e62c899c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e62c890930 con 0x55e62c4be2f0 2024-05-14T09:37:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e62c7aa370 con 0x55e62c4c64d0 2024-05-14T09:37:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e62c890ab0 con 0x55e62c4e71b0 2024-05-14T09:37:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.974+0000 7fc4279ad740 10 client.0 _ll_register_callbacks cb 0x55e62c4fea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc413fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 0x55e62c892d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc41a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c8927d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc41ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4e71b0 0x55e62c899c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc413fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 0x55e62c892d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:53782/0 (socket says 172.21.15.2:53782) 2024-05-14T09:37:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc413fff640 1 -- 172.21.15.2:0/2247934895 learned_addr learned my addr 172.21.15.2:0/2247934895 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc41ad9d640 1 -- 172.21.15.2:0/2247934895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 msgr2=0x55e62c892d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc41ad9d640 1 --2- 172.21.15.2:0/2247934895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 0x55e62c892d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc41ad9d640 1 -- 172.21.15.2:0/2247934895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 msgr2=0x55e62c8927d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc41ad9d640 1 --2- 172.21.15.2:0/2247934895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c8927d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc41ad9d640 1 -- 172.21.15.2:0/2247934895 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e62c5cb620 con 0x55e62c4e71b0 2024-05-14T09:37:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc41a59c640 1 --2- 172.21.15.2:0/2247934895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c8927d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc413fff640 1 --2- 172.21.15.2:0/2247934895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 0x55e62c892d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc41ad9d640 1 --2- 172.21.15.2:0/2247934895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4e71b0 0x55e62c899c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc414009270 tx=0x7fc414004280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8992acbed2cdc339 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc401ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc401ffb640 1 -- 172.21.15.2:0/2247934895 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414027020 con 0x55e62c4e71b0 2024-05-14T09:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc4279ad740 1 -- 172.21.15.2:0/2247934895 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc414004030 con 0x55e62c4e71b0 2024-05-14T09:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.975+0000 7fc401ffb640 1 -- 172.21.15.2:0/2247934895 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fc414003a30 con 0x55e62c4e71b0 2024-05-14T09:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.976+0000 7fc401ffb640 1 -- 172.21.15.2:0/2247934895 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc414032050 con 0x55e62c4e71b0 2024-05-14T09:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.976+0000 7fc4279ad740 10 client.25450 fetch_fsmap learned FSMap version 27 2024-05-14T09:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.976+0000 7fc4279ad740 1 -- 172.21.15.2:0/2247934895 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e62c89cba0 con 0x55e62c4e71b0 2024-05-14T09:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.976+0000 7fc401ffb640 1 -- 172.21.15.2:0/2247934895 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc414004030 con 0x55e62c4e71b0 2024-05-14T09:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.976+0000 7fc401ffb640 1 -- 172.21.15.2:0/2247934895 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc414018070 con 0x55e62c4e71b0 2024-05-14T09:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.976+0000 7fc4279ad740 10 client.25450 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.976+0000 7fc4279ad740 10 client.25450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.976+0000 7fc4279ad740 20 client.25450 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.976+0000 7fc4279ad740 1 --2- 172.21.15.2:0/2247934895 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c4a5d80 0x55e62c8a0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.976+0000 7fc4279ad740 4 client.25450 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.976+0000 7fc4279ad740 1 -- 172.21.15.2:0/2247934895 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e62c7fb1e0 con 0x55e62c4a5d80 2024-05-14T09:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.976+0000 7fc41a59c640 1 --2- 172.21.15.2:0/2247934895 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c4a5d80 0x55e62c8a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.977+0000 7fc401ffb640 10 client.25450 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.977+0000 7fc41a59c640 1 --2- 172.21.15.2:0/2247934895 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c4a5d80 0x55e62c8a0060 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f260e863a70d10c server_cookie=3eceff4a9479f068 in_seq=0 out_seq=0 2024-05-14T09:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.978+0000 7fc401ffb640 1 -- 172.21.15.2:0/2247934895 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e62c7fb1e0 con 0x55e62c4a5d80 2024-05-14T09:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.978+0000 7fc401ffb640 10 client.25450 handle_command_reply: tid=0 2024-05-14T09:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.978+0000 7fc4279ad740 1 client.25450 shutdown 2024-05-14T09:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.978+0000 7fc4279ad740 1 -- 172.21.15.2:0/2247934895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4e71b0 msgr2=0x55e62c899c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.978+0000 7fc4279ad740 1 --2- 172.21.15.2:0/2247934895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4e71b0 0x55e62c899c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc414009270 tx=0x7fc414004280 comp rx=0 tx=0).stop 2024-05-14T09:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.978+0000 7fc4279ad740 1 -- 172.21.15.2:0/2247934895 shutdown_connections 2024-05-14T09:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.978+0000 7fc4279ad740 1 --2- 172.21.15.2:0/2247934895 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c4a5d80 0x55e62c8a0060 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.978+0000 7fc4279ad740 1 --2- 172.21.15.2:0/2247934895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4e71b0 0x55e62c899c70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.978+0000 7fc4279ad740 1 --2- 172.21.15.2:0/2247934895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 0x55e62c892d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.978+0000 7fc401ffb640 0 client.25450 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.978+0000 7fc4279ad740 1 --2- 172.21.15.2:0/2247934895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c8927d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.978+0000 7fc4279ad740 1 -- 172.21.15.2:0/2247934895 >> 172.21.15.2:0/2247934895 conn(0x55e62c4db300 msgr2=0x55e62c8833a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.978+0000 7fc4279ad740 1 -- 172.21.15.2:0/2247934895 shutdown_connections 2024-05-14T09:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.978+0000 7fc4279ad740 1 -- 172.21.15.2:0/2247934895 wait complete. 2024-05-14T09:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.978+0000 7fc4279ad740 20 client.25450 trim_cache size 0 max 16384 2024-05-14T09:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.994+0000 7fc4279ad740 1 Processor -- start 2024-05-14T09:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.994+0000 7fc4279ad740 1 -- start start 2024-05-14T09:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc4279ad740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4db300 0x55e62c884470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc4279ad740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4be2f0 0x55e62c899b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc4279ad740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4c64d0 0x55e62c89a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc4279ad740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c1af470 con 0x55e62c4c64d0 2024-05-14T09:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc41a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4db300 0x55e62c884470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc41ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4c64d0 0x55e62c89a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc413fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4be2f0 0x55e62c899b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc41a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4db300 0x55e62c884470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:49138/0 (socket says 172.21.15.2:49138) 2024-05-14T09:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc41a59c640 1 -- 172.21.15.2:0/2788023580 learned_addr learned my addr 172.21.15.2:0/2788023580 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc4279ad740 1 -- 172.21.15.2:0/2788023580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c1acd40 con 0x55e62c4be2f0 2024-05-14T09:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc4279ad740 1 -- 172.21.15.2:0/2788023580 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c1aab90 con 0x55e62c4db300 2024-05-14T09:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc41a59c640 1 -- 172.21.15.2:0/2788023580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4be2f0 msgr2=0x55e62c899b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc41a59c640 1 --2- 172.21.15.2:0/2788023580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4be2f0 0x55e62c899b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc41a59c640 1 -- 172.21.15.2:0/2788023580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4c64d0 msgr2=0x55e62c89a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc41a59c640 1 --2- 172.21.15.2:0/2788023580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4c64d0 0x55e62c89a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.995+0000 7fc41a59c640 1 -- 172.21.15.2:0/2788023580 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e62c884f80 con 0x55e62c4db300 2024-05-14T09:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.996+0000 7fc41a59c640 1 --2- 172.21.15.2:0/2788023580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4db300 0x55e62c884470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc40c030550 tx=0x7fc40c040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2352aeeb88e4db server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.996+0000 7fc403fff640 1 -- 172.21.15.2:0/2788023580 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c036040 con 0x55e62c4db300 2024-05-14T09:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.996+0000 7fc403fff640 1 -- 172.21.15.2:0/2788023580 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc40c05b070 con 0x55e62c4db300 2024-05-14T09:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.996+0000 7fc403fff640 1 -- 172.21.15.2:0/2788023580 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c003d90 con 0x55e62c4db300 2024-05-14T09:37:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.997+0000 7fc4279ad740 1 -- 172.21.15.2:0/2788023580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4db300 msgr2=0x55e62c884470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.997+0000 7fc4279ad740 1 --2- 172.21.15.2:0/2788023580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4db300 0x55e62c884470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc40c030550 tx=0x7fc40c040000 comp rx=0 tx=0).stop 2024-05-14T09:37:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.997+0000 7fc4279ad740 1 -- 172.21.15.2:0/2788023580 shutdown_connections 2024-05-14T09:37:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.997+0000 7fc4279ad740 1 --2- 172.21.15.2:0/2788023580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4c64d0 0x55e62c89a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.997+0000 7fc4279ad740 1 --2- 172.21.15.2:0/2788023580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4be2f0 0x55e62c899b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.997+0000 7fc4279ad740 1 --2- 172.21.15.2:0/2788023580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4db300 0x55e62c884470 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.997+0000 7fc4279ad740 1 -- 172.21.15.2:0/2788023580 >> 172.21.15.2:0/2788023580 conn(0x55e62c8891f0 msgr2=0x55e62c883d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.997+0000 7fc4279ad740 1 -- 172.21.15.2:0/2788023580 shutdown_connections 2024-05-14T09:37:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.997+0000 7fc4279ad740 1 -- 172.21.15.2:0/2788023580 wait complete. 2024-05-14T09:37:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.997+0000 7fc4279ad740 1 -- start start 2024-05-14T09:37:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.997+0000 7fc4279ad740 1 Processor -- start 2024-05-14T09:37:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc4279ad740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c8955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc4279ad740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 0x55e62c895ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc4279ad740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4db300 0x55e62c906390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc4279ad740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c1ab200 con 0x55e62c4be2f0 2024-05-14T09:37:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc4279ad740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c1ac670 con 0x55e62c4c64d0 2024-05-14T09:37:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc4279ad740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c1ab8d0 con 0x55e62c4db300 2024-05-14T09:37:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc41a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c8955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc4279ad740 10 client.0 _ll_register_callbacks cb 0x7fc404005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc41a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c8955a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58010/0 (socket says 172.21.15.2:58010) 2024-05-14T09:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc41a59c640 1 -- 172.21.15.2:0/3360978148 learned_addr learned my addr 172.21.15.2:0/3360978148 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc413fff640 1 --2- 172.21.15.2:0/3360978148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 0x55e62c895ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc41ad9d640 1 --2- 172.21.15.2:0/3360978148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4db300 0x55e62c906390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc41a59c640 1 -- 172.21.15.2:0/3360978148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4db300 msgr2=0x55e62c906390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc41a59c640 1 --2- 172.21.15.2:0/3360978148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4db300 0x55e62c906390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc41a59c640 1 -- 172.21.15.2:0/3360978148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 msgr2=0x55e62c895ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc41a59c640 1 --2- 172.21.15.2:0/3360978148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 0x55e62c895ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc41a59c640 1 -- 172.21.15.2:0/3360978148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e62c906950 con 0x55e62c4be2f0 2024-05-14T09:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.998+0000 7fc413fff640 1 --2- 172.21.15.2:0/3360978148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 0x55e62c895ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.999+0000 7fc41a59c640 1 --2- 172.21.15.2:0/3360978148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c8955a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc40c058040 tx=0x7fc40c0044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98395ae95a1c2baf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.999+0000 7fc401ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.999+0000 7fc401ffb640 1 -- 172.21.15.2:0/3360978148 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c064070 con 0x55e62c4be2f0 2024-05-14T09:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.999+0000 7fc401ffb640 1 -- 172.21.15.2:0/3360978148 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fc40c003ec0 con 0x55e62c4be2f0 2024-05-14T09:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.999+0000 7fc4279ad740 1 -- 172.21.15.2:0/3360978148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc40c05b070 con 0x55e62c4be2f0 2024-05-14T09:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.999+0000 7fc401ffb640 1 -- 172.21.15.2:0/3360978148 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc40c036070 con 0x55e62c4be2f0 2024-05-14T09:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.999+0000 7fc4279ad740 10 client.15762 fetch_fsmap learned FSMap version 27 2024-05-14T09:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.999+0000 7fc4279ad740 1 -- 172.21.15.2:0/3360978148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e62c907c50 con 0x55e62c4be2f0 2024-05-14T09:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:00.999+0000 7fc401ffb640 1 -- 172.21.15.2:0/3360978148 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc40c05b070 con 0x55e62c4be2f0 2024-05-14T09:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.000+0000 7fc401ffb640 1 -- 172.21.15.2:0/3360978148 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc40c00b040 con 0x55e62c4be2f0 2024-05-14T09:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.000+0000 7fc4279ad740 10 client.15762 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.000+0000 7fc4279ad740 10 client.15762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.000+0000 7fc4279ad740 20 client.15762 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.000+0000 7fc4279ad740 1 --2- 172.21.15.2:0/3360978148 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c4e71b0 0x55e62c90a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.000+0000 7fc4279ad740 4 client.15762 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:37:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.000+0000 7fc4279ad740 1 -- 172.21.15.2:0/3360978148 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e62c7f5050 con 0x55e62c4e71b0 2024-05-14T09:37:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.000+0000 7fc413fff640 1 --2- 172.21.15.2:0/3360978148 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c4e71b0 0x55e62c90a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.000+0000 7fc401ffb640 10 client.15762 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.000+0000 7fc413fff640 1 --2- 172.21.15.2:0/3360978148 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c4e71b0 0x55e62c90a6a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f07f0099820881b2 server_cookie=4c9f557faed04a8c in_seq=0 out_seq=0 2024-05-14T09:37:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.168+0000 7fc401ffb640 1 -- 172.21.15.2:0/3360978148 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e62c7f5050 con 0x55e62c4e71b0 2024-05-14T09:37:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.168+0000 7fc401ffb640 10 client.15762 handle_command_reply: tid=0 2024-05-14T09:37:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.168+0000 7fc4279ad740 1 client.15762 shutdown 2024-05-14T09:37:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.169+0000 7fc4279ad740 1 -- 172.21.15.2:0/3360978148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 msgr2=0x55e62c8955a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.169+0000 7fc4279ad740 1 --2- 172.21.15.2:0/3360978148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c8955a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc40c058040 tx=0x7fc40c0044d0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.169+0000 7fc4279ad740 1 -- 172.21.15.2:0/3360978148 shutdown_connections 2024-05-14T09:37:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.169+0000 7fc4279ad740 1 --2- 172.21.15.2:0/3360978148 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c4e71b0 0x55e62c90a6a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.169+0000 7fc4279ad740 1 --2- 172.21.15.2:0/3360978148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c4db300 0x55e62c906390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.169+0000 7fc401ffb640 0 client.15762 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.169+0000 7fc4279ad740 1 --2- 172.21.15.2:0/3360978148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c4c64d0 0x55e62c895ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.169+0000 7fc4279ad740 1 --2- 172.21.15.2:0/3360978148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c4be2f0 0x55e62c8955a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.169+0000 7fc4279ad740 1 -- 172.21.15.2:0/3360978148 >> 172.21.15.2:0/3360978148 conn(0x55e62c8891f0 msgr2=0x55e62c882970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.170+0000 7fc4279ad740 1 -- 172.21.15.2:0/3360978148 shutdown_connections 2024-05-14T09:37:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.170+0000 7fc4279ad740 1 -- 172.21.15.2:0/3360978148 wait complete. 2024-05-14T09:37:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.170+0000 7fc4279ad740 20 client.15762 trim_cache size 0 max 16384 2024-05-14T09:37:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:37:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:37:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:37:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:37:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.171+0000 7fc420a22640 1 -- 172.21.15.2:0/920136004 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc4080b28a0 msgr2=0x7fc4080b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.171+0000 7fc420a22640 1 --2- 172.21.15.2:0/920136004 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc4080b28a0 0x7fc4080b4db0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00a7d0 tx=0x7fc40c0305b0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.171+0000 7fc420a22640 1 -- 172.21.15.2:0/920136004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc41c1034f0 msgr2=0x7fc41c19a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.171+0000 7fc420a22640 1 --2- 172.21.15.2:0/920136004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc41c1034f0 0x7fc41c19a840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc40400e9f0 tx=0x7fc40400eeb0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.171+0000 7fc420a22640 1 -- 172.21.15.2:0/920136004 shutdown_connections 2024-05-14T09:37:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.171+0000 7fc420a22640 1 --2- 172.21.15.2:0/920136004 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc4080b28a0 0x7fc4080b4db0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.171+0000 7fc420a22640 1 --2- 172.21.15.2:0/920136004 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc41c1081f0 0x7fc41c1a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.171+0000 7fc420a22640 1 --2- 172.21.15.2:0/920136004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc41c1034f0 0x7fc41c19a840 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.171+0000 7fc420a22640 1 --2- 172.21.15.2:0/920136004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc41c1022b0 0x7fc41c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.171+0000 7fc420a22640 1 -- 172.21.15.2:0/920136004 >> 172.21.15.2:0/920136004 conn(0x7fc41c0fd480 msgr2=0x7fc41c1061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.171+0000 7fc420a22640 1 -- 172.21.15.2:0/920136004 shutdown_connections 2024-05-14T09:37:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:01.171+0000 7fc420a22640 1 -- 172.21.15.2:0/920136004 wait complete. 2024-05-14T09:37:01.176 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T09:37:01.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 70334 14.69 MB/sec warmup 27 sec latency 76.965 ms 2024-05-14T09:37:01.457 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:37:02.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 72451 14.58 MB/sec warmup 28 sec latency 27.111 ms 2024-05-14T09:37:02.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:02 smithi002 ceph-mon[31557]: pgmap v486: 129 pgs: 129 active+clean; 39 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.9 MiB/s wr, 500 op/s 2024-05-14T09:37:02.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:02 smithi029 ceph-mon[36249]: pgmap v486: 129 pgs: 129 active+clean; 39 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.9 MiB/s wr, 500 op/s 2024-05-14T09:37:02.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:02 smithi026 ceph-mon[36056]: pgmap v486: 129 pgs: 129 active+clean; 39 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.9 MiB/s wr, 500 op/s 2024-05-14T09:37:03.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 75225 14.57 MB/sec warmup 29 sec latency 23.491 ms 2024-05-14T09:37:03.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.429+0000 7f5cf636d700 1 -- 172.21.15.2:0/4098962177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cf01034c0 msgr2=0x7f5cf01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:03.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.429+0000 7f5cf636d700 1 --2- 172.21.15.2:0/4098962177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cf01034c0 0x7f5cf01038d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5ce00099c0 tx=0x7f5ce0009cd0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.429+0000 7f5cf636d700 1 -- 172.21.15.2:0/4098962177 shutdown_connections 2024-05-14T09:37:03.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.429+0000 7f5cf636d700 1 --2- 172.21.15.2:0/4098962177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cf0108740 0x7f5cf01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.429+0000 7f5cf636d700 1 --2- 172.21.15.2:0/4098962177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cf01034c0 0x7f5cf01038d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.429+0000 7f5cf636d700 1 --2- 172.21.15.2:0/4098962177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cf0107df0 0x7f5cf0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.429+0000 7f5cf636d700 1 -- 172.21.15.2:0/4098962177 >> 172.21.15.2:0/4098962177 conn(0x7f5cf00fedc0 msgr2=0x7f5cf01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:03.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.429+0000 7f5cf636d700 1 -- 172.21.15.2:0/4098962177 shutdown_connections 2024-05-14T09:37:03.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.429+0000 7f5cf636d700 1 -- 172.21.15.2:0/4098962177 wait complete. 2024-05-14T09:37:03.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.430+0000 7f5cf636d700 1 Processor -- start 2024-05-14T09:37:03.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.430+0000 7f5cf636d700 1 -- start start 2024-05-14T09:37:03.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.430+0000 7f5cf636d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cf01034c0 0x7f5cf00ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.430+0000 7f5cf636d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cf0107df0 0x7f5cf00ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.430+0000 7f5cf636d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cf0108740 0x7f5cf00ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.430+0000 7f5cf636d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf0113390 con 0x7f5cf0107df0 2024-05-14T09:37:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.430+0000 7f5cf636d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf0113210 con 0x7f5cf0108740 2024-05-14T09:37:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.430+0000 7f5cf636d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf0113510 con 0x7f5cf01034c0 2024-05-14T09:37:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.431+0000 7f5cf490a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cf0108740 0x7f5cf00ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.431+0000 7f5ceffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cf01034c0 0x7f5cf00ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.431+0000 7f5ce7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cf0107df0 0x7f5cf00ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.431+0000 7f5cf490a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cf0108740 0x7f5cf00ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:53824/0 (socket says 172.21.15.2:53824) 2024-05-14T09:37:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.431+0000 7f5cf490a700 1 -- 172.21.15.2:0/178799456 learned_addr learned my addr 172.21.15.2:0/178799456 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.431+0000 7f5cf490a700 1 -- 172.21.15.2:0/178799456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cf01034c0 msgr2=0x7f5cf00ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.431+0000 7f5cf490a700 1 --2- 172.21.15.2:0/178799456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cf01034c0 0x7f5cf00ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.431+0000 7f5cf490a700 1 -- 172.21.15.2:0/178799456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cf0107df0 msgr2=0x7f5cf00ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.431+0000 7f5cf490a700 1 --2- 172.21.15.2:0/178799456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cf0107df0 0x7f5cf00ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.431+0000 7f5cf490a700 1 -- 172.21.15.2:0/178799456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf01a91b0 con 0x7f5cf0108740 2024-05-14T09:37:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.431+0000 7f5ceffff700 1 --2- 172.21.15.2:0/178799456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cf01034c0 0x7f5cf00ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.431+0000 7f5ce7fff700 1 --2- 172.21.15.2:0/178799456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cf0107df0 0x7f5cf00ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.431+0000 7f5cf490a700 1 --2- 172.21.15.2:0/178799456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cf0108740 0x7f5cf00ffd20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800ef30 tx=0x7f5ce800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.432+0000 7f5cedffb700 1 -- 172.21.15.2:0/178799456 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80092e0 con 0x7f5cf0108740 2024-05-14T09:37:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.432+0000 7f5cedffb700 1 -- 172.21.15.2:0/178799456 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce8004bb0 con 0x7f5cf0108740 2024-05-14T09:37:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.432+0000 7f5cf636d700 1 -- 172.21.15.2:0/178799456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf01a9440 con 0x7f5cf0108740 2024-05-14T09:37:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.432+0000 7f5cedffb700 1 -- 172.21.15.2:0/178799456 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8004020 con 0x7f5cf0108740 2024-05-14T09:37:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.432+0000 7f5cf636d700 1 -- 172.21.15.2:0/178799456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf01a9930 con 0x7f5cf0108740 2024-05-14T09:37:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.434+0000 7f5cedffb700 1 -- 172.21.15.2:0/178799456 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5ce801b030 con 0x7f5cf0108740 2024-05-14T09:37:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.435+0000 7f5cedffb700 1 --2- 172.21.15.2:0/178799456 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5cdc0b2880 0x7f5cdc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.435+0000 7f5ceffff700 1 --2- 172.21.15.2:0/178799456 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5cdc0b2880 0x7f5cdc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.435+0000 7f5cedffb700 1 -- 172.21.15.2:0/178799456 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f5ce80e2070 con 0x7f5cf0108740 2024-05-14T09:37:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.436+0000 7f5ceffff700 1 --2- 172.21.15.2:0/178799456 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5cdc0b2880 0x7f5cdc0b4d90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0000c00 tx=0x7f5ce001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.436+0000 7f5d04d81b80 1 Processor -- start 2024-05-14T09:37:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.436+0000 7f5d04d81b80 1 -- start start 2024-05-14T09:37:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5d04d81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c7773e40 0x5641c78f7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5d04d81b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f7d80 0x5641c78f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5d04d81b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c78f1e10 0x5641c78f2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5d04d81b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5641c7813940 con 0x5641c78f1e10 2024-05-14T09:37:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5d04d81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5641c7778da0 con 0x5641c7773e40 2024-05-14T09:37:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5d04d81b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5641c780f900 con 0x5641c78f7d80 2024-05-14T09:37:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5ceffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c7773e40 0x5641c78f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5cf490a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c78f1e10 0x5641c78f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5ce7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f7d80 0x5641c78f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5cf490a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c78f1e10 0x5641c78f2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58040/0 (socket says 172.21.15.2:58040) 2024-05-14T09:37:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5cf490a700 1 -- 172.21.15.2:0/1777862436 learned_addr learned my addr 172.21.15.2:0/1777862436 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5ceffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c7773e40 0x5641c78f7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:53840/0 (socket says 172.21.15.2:53840) 2024-05-14T09:37:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5cf490a700 1 -- 172.21.15.2:0/1777862436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f7d80 msgr2=0x5641c78f18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5cf490a700 1 --2- 172.21.15.2:0/1777862436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f7d80 0x5641c78f18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5cf490a700 1 -- 172.21.15.2:0/1777862436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c7773e40 msgr2=0x5641c78f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5cf490a700 1 --2- 172.21.15.2:0/1777862436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c7773e40 0x5641c78f7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5cf490a700 1 -- 172.21.15.2:0/1777862436 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641c77608d0 con 0x5641c78f1e10 2024-05-14T09:37:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.437+0000 7f5cf490a700 1 --2- 172.21.15.2:0/1777862436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c78f1e10 0x5641c78f2220 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80a9290 tx=0x7f5ce800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7504d991cebe5a7b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.438+0000 7f5ce67fc700 1 -- 172.21.15.2:0/1777862436 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80a9300 con 0x5641c78f1e10 2024-05-14T09:37:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.438+0000 7f5ce67fc700 1 -- 172.21.15.2:0/1777862436 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce80ad6b0 con 0x5641c78f1e10 2024-05-14T09:37:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.438+0000 7f5ce67fc700 1 -- 172.21.15.2:0/1777862436 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80ad360 con 0x5641c78f1e10 2024-05-14T09:37:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.439+0000 7f5d04d81b80 1 -- 172.21.15.2:0/1777862436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c78f1e10 msgr2=0x5641c78f2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.439+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/1777862436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c78f1e10 0x5641c78f2220 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80a9290 tx=0x7f5ce800ca10 comp rx=0 tx=0).stop 2024-05-14T09:37:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.439+0000 7f5d04d81b80 1 -- 172.21.15.2:0/1777862436 shutdown_connections 2024-05-14T09:37:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.439+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/1777862436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c7773e40 0x5641c78f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.439+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/1777862436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c78f1e10 0x5641c78f2220 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.439+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/1777862436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f7d80 0x5641c78f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.439+0000 7f5d04d81b80 1 -- 172.21.15.2:0/1777862436 >> 172.21.15.2:0/1777862436 conn(0x5641c77bb630 msgr2=0x5641c78eca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.439+0000 7f5d04d81b80 1 -- 172.21.15.2:0/1777862436 shutdown_connections 2024-05-14T09:37:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.439+0000 7f5d04d81b80 1 -- 172.21.15.2:0/1777862436 wait complete. 2024-05-14T09:37:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.439+0000 7f5d04d81b80 1 -- start start 2024-05-14T09:37:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.439+0000 7f5d04d81b80 1 Processor -- start 2024-05-14T09:37:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5d04d81b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c7773e40 0x5641c78fb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5d04d81b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c78f1e10 0x5641c78fba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5d04d81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f7d80 0x5641c78fbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5d04d81b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5641c78fe820 con 0x5641c78f1e10 2024-05-14T09:37:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5d04d81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5641c77cced0 con 0x5641c78f7d80 2024-05-14T09:37:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5d04d81b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5641c78fe9a0 con 0x5641c7773e40 2024-05-14T09:37:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5d04d81b80 10 client.0 _ll_register_callbacks cb 0x5641c77ab950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5ce7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c78f1e10 0x5641c78fba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5ce7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c78f1e10 0x5641c78fba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58044/0 (socket says 172.21.15.2:58044) 2024-05-14T09:37:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5ce7fff700 1 -- 172.21.15.2:0/560527585 learned_addr learned my addr 172.21.15.2:0/560527585 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5cf490a700 1 --2- 172.21.15.2:0/560527585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f7d80 0x5641c78fbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5ceffff700 1 --2- 172.21.15.2:0/560527585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c7773e40 0x5641c78fb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5ce7fff700 1 -- 172.21.15.2:0/560527585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c7773e40 msgr2=0x5641c78fb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5ce7fff700 1 --2- 172.21.15.2:0/560527585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c7773e40 0x5641c78fb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5ce7fff700 1 -- 172.21.15.2:0/560527585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f7d80 msgr2=0x5641c78fbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5ce7fff700 1 --2- 172.21.15.2:0/560527585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f7d80 0x5641c78fbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5ce7fff700 1 -- 172.21.15.2:0/560527585 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641c77ca760 con 0x5641c78f1e10 2024-05-14T09:37:03.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5cf490a700 1 --2- 172.21.15.2:0/560527585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f7d80 0x5641c78fbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:03.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5ce7fff700 1 --2- 172.21.15.2:0/560527585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c78f1e10 0x5641c78fba00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8012d20 tx=0x7f5cd800f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403fc3a6b38e0a21 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:03.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.440+0000 7f5ccf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:37:03.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.441+0000 7f5ccf7fe700 1 -- 172.21.15.2:0/560527585 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd800fb80 con 0x5641c78f1e10 2024-05-14T09:37:03.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.441+0000 7f5ccf7fe700 1 -- 172.21.15.2:0/560527585 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f5cd800fd20 con 0x5641c78f1e10 2024-05-14T09:37:03.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.441+0000 7f5d04d81b80 1 -- 172.21.15.2:0/560527585 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce80ad6b0 con 0x5641c78f1e10 2024-05-14T09:37:03.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.441+0000 7f5ccf7fe700 1 -- 172.21.15.2:0/560527585 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5cd8004050 con 0x5641c78f1e10 2024-05-14T09:37:03.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.441+0000 7f5d04d81b80 10 client.15774 fetch_fsmap learned FSMap version 27 2024-05-14T09:37:03.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.441+0000 7f5d04d81b80 1 -- 172.21.15.2:0/560527585 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5641c79094b0 con 0x5641c78f1e10 2024-05-14T09:37:03.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.441+0000 7f5ccf7fe700 1 -- 172.21.15.2:0/560527585 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce80ad6b0 con 0x5641c78f1e10 2024-05-14T09:37:03.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.441+0000 7f5ccf7fe700 1 -- 172.21.15.2:0/560527585 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5cd8010430 con 0x5641c78f1e10 2024-05-14T09:37:03.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.441+0000 7f5d04d81b80 10 client.15774 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:37:03.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.441+0000 7f5d04d81b80 10 client.15774 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T09:37:03.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.441+0000 7f5d04d81b80 20 client.15774 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:03.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.441+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/560527585 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5641c790baf0 0x5641c790df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.441+0000 7f5d04d81b80 4 client.15774 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:37:03.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.441+0000 7f5d04d81b80 1 -- 172.21.15.2:0/560527585 --> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641c7821320 con 0x5641c790baf0 2024-05-14T09:37:03.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.442+0000 7f5ceffff700 1 --2- 172.21.15.2:0/560527585 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5641c790baf0 0x5641c790df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.442+0000 7f5ccf7fe700 10 client.15774 ms_handle_connect on v2:172.21.15.26:6836/3301773578 2024-05-14T09:37:03.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.442+0000 7f5ceffff700 1 --2- 172.21.15.2:0/560527585 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5641c790baf0 0x5641c790df10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=715fad34bb4becdc server_cookie=82973cd323d9df7d in_seq=0 out_seq=0 2024-05-14T09:37:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.443+0000 7f5ccf7fe700 1 -- 172.21.15.2:0/560527585 <== mds.1 v2:172.21.15.26:6836/3301773578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5641c7821320 con 0x5641c790baf0 2024-05-14T09:37:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.443+0000 7f5ccf7fe700 10 client.15774 handle_command_reply: tid=0 2024-05-14T09:37:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.443+0000 7f5d04d81b80 1 client.15774 shutdown 2024-05-14T09:37:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.443+0000 7f5d04d81b80 1 -- 172.21.15.2:0/560527585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c78f1e10 msgr2=0x5641c78fba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.443+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/560527585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c78f1e10 0x5641c78fba00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8012d20 tx=0x7f5cd800f6a0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.444+0000 7f5d04d81b80 1 -- 172.21.15.2:0/560527585 shutdown_connections 2024-05-14T09:37:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.444+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/560527585 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5641c790baf0 0x5641c790df10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.444+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/560527585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f7d80 0x5641c78fbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.444+0000 7f5ccf7fe700 0 client.15774 ms_handle_reset on v2:172.21.15.26:6836/3301773578 2024-05-14T09:37:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.444+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/560527585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c7773e40 0x5641c78fb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.444+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/560527585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c78f1e10 0x5641c78fba00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.444+0000 7f5d04d81b80 1 -- 172.21.15.2:0/560527585 >> 172.21.15.2:0/560527585 conn(0x5641c77bb630 msgr2=0x5641c78f08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.444+0000 7f5d04d81b80 1 -- 172.21.15.2:0/560527585 shutdown_connections 2024-05-14T09:37:03.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.444+0000 7f5d04d81b80 1 -- 172.21.15.2:0/560527585 wait complete. 2024-05-14T09:37:03.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.444+0000 7f5d04d81b80 20 client.15774 trim_cache size 0 max 16384 2024-05-14T09:37:03.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.462+0000 7f5d04d81b80 1 Processor -- start 2024-05-14T09:37:03.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.462+0000 7f5d04d81b80 1 -- start start 2024-05-14T09:37:03.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.462+0000 7f5d04d81b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c7773e40 0x5641c78f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.462+0000 7f5d04d81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f1e10 0x5641c79078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.462+0000 7f5d04d81b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f7d80 0x5641c78fba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.462+0000 7f5d04d81b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01b5080 con 0x5641c7773e40 2024-05-14T09:37:03.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.462+0000 7f5d04d81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01b2910 con 0x5641c78f1e10 2024-05-14T09:37:03.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.462+0000 7f5d04d81b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01b0760 con 0x5641c78f7d80 2024-05-14T09:37:03.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.462+0000 7f5ce7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f1e10 0x5641c79078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.462+0000 7f5cf490a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f7d80 0x5641c78fba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.462+0000 7f5ceffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c7773e40 0x5641c78f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.462+0000 7f5ce7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f1e10 0x5641c79078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:53864/0 (socket says 172.21.15.2:53864) 2024-05-14T09:37:03.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.462+0000 7f5cf490a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f7d80 0x5641c78fba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:49180/0 (socket says 172.21.15.2:49180) 2024-05-14T09:37:03.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.462+0000 7f5ce7fff700 1 -- 172.21.15.2:0/4191712481 learned_addr learned my addr 172.21.15.2:0/4191712481 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:03.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.463+0000 7f5ce7fff700 1 -- 172.21.15.2:0/4191712481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f7d80 msgr2=0x5641c78fba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:03.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.463+0000 7f5ce7fff700 1 --2- 172.21.15.2:0/4191712481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f7d80 0x5641c78fba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.463+0000 7f5ce7fff700 1 -- 172.21.15.2:0/4191712481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c7773e40 msgr2=0x5641c78f18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:03.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.463+0000 7f5ce7fff700 1 --2- 172.21.15.2:0/4191712481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c7773e40 0x5641c78f18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.463+0000 7f5ce7fff700 1 -- 172.21.15.2:0/4191712481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641c78e7260 con 0x5641c78f1e10 2024-05-14T09:37:03.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.463+0000 7f5cf490a700 1 --2- 172.21.15.2:0/4191712481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f7d80 0x5641c78fba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.463+0000 7f5ce7fff700 1 --2- 172.21.15.2:0/4191712481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f1e10 0x5641c79078e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8010d00 tx=0x7f5cd8010d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1072bc3e777eb817 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.463+0000 7f5ce5ffb700 1 -- 172.21.15.2:0/4191712481 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd802d020 con 0x5641c78f1e10 2024-05-14T09:37:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.463+0000 7f5ce5ffb700 1 -- 172.21.15.2:0/4191712481 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cd8010640 con 0x5641c78f1e10 2024-05-14T09:37:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.463+0000 7f5ce5ffb700 1 -- 172.21.15.2:0/4191712481 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80038e0 con 0x5641c78f1e10 2024-05-14T09:37:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.464+0000 7f5d04d81b80 1 -- 172.21.15.2:0/4191712481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f1e10 msgr2=0x5641c79078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.464+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/4191712481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f1e10 0x5641c79078e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8010d00 tx=0x7f5cd8010d30 comp rx=0 tx=0).stop 2024-05-14T09:37:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.464+0000 7f5d04d81b80 1 -- 172.21.15.2:0/4191712481 shutdown_connections 2024-05-14T09:37:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.464+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/4191712481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f7d80 0x5641c78fba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.464+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/4191712481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c7773e40 0x5641c78f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.464+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/4191712481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f1e10 0x5641c79078e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.464+0000 7f5d04d81b80 1 -- 172.21.15.2:0/4191712481 >> 172.21.15.2:0/4191712481 conn(0x5641c77bb630 msgr2=0x5641c78f9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.464+0000 7f5d04d81b80 1 -- 172.21.15.2:0/4191712481 shutdown_connections 2024-05-14T09:37:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.464+0000 7f5d04d81b80 1 -- 172.21.15.2:0/4191712481 wait complete. 2024-05-14T09:37:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.464+0000 7f5d04d81b80 1 -- start start 2024-05-14T09:37:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5d04d81b80 1 Processor -- start 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5d04d81b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c7773e40 0x5641c7913ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5d04d81b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f1e10 0x5641c7903c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5d04d81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f7d80 0x5641c79041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5d04d81b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01b0dd0 con 0x5641c7773e40 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5d04d81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01b2240 con 0x5641c78f7d80 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5d04d81b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01b14a0 con 0x5641c78f1e10 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5ceffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c7773e40 0x5641c7913ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5d04d81b80 10 client.0 _ll_register_callbacks cb 0x5641c77ab950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5ceffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c7773e40 0x5641c7913ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58050/0 (socket says 172.21.15.2:58050) 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5ceffff700 1 -- 172.21.15.2:0/4145026031 learned_addr learned my addr 172.21.15.2:0/4145026031 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5cf490a700 1 --2- 172.21.15.2:0/4145026031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f7d80 0x5641c79041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5ce7fff700 1 --2- 172.21.15.2:0/4145026031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f1e10 0x5641c7903c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5ceffff700 1 -- 172.21.15.2:0/4145026031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f1e10 msgr2=0x5641c7903c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5ceffff700 1 --2- 172.21.15.2:0/4145026031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f1e10 0x5641c7903c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5ceffff700 1 -- 172.21.15.2:0/4145026031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f7d80 msgr2=0x5641c79041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:03.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5ceffff700 1 --2- 172.21.15.2:0/4145026031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f7d80 0x5641c79041a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.465+0000 7f5ceffff700 1 -- 172.21.15.2:0/4145026031 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641c79126b0 con 0x5641c7773e40 2024-05-14T09:37:03.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.466+0000 7f5ceffff700 1 --2- 172.21.15.2:0/4145026031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c7773e40 0x5641c7913ac0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5ce00098e0 tx=0x7f5ce0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6770dccea80649 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:03.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.466+0000 7f5ccf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:37:03.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.466+0000 7f5ccf7fe700 1 -- 172.21.15.2:0/4145026031 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0037020 con 0x5641c7773e40 2024-05-14T09:37:03.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.466+0000 7f5d04d81b80 1 -- 172.21.15.2:0/4145026031 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cd8010640 con 0x5641c7773e40 2024-05-14T09:37:03.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.466+0000 7f5ccf7fe700 1 -- 172.21.15.2:0/4145026031 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f5ce00039e0 con 0x5641c7773e40 2024-05-14T09:37:03.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.466+0000 7f5ccf7fe700 1 -- 172.21.15.2:0/4145026031 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ce0042050 con 0x5641c7773e40 2024-05-14T09:37:03.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.466+0000 7f5d04d81b80 10 client.15780 fetch_fsmap learned FSMap version 27 2024-05-14T09:37:03.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.466+0000 7f5d04d81b80 1 -- 172.21.15.2:0/4145026031 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5641c79753f0 con 0x5641c7773e40 2024-05-14T09:37:03.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.466+0000 7f5ccf7fe700 1 -- 172.21.15.2:0/4145026031 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cd8010640 con 0x5641c7773e40 2024-05-14T09:37:03.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.467+0000 7f5ccf7fe700 1 -- 172.21.15.2:0/4145026031 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5ce0033410 con 0x5641c7773e40 2024-05-14T09:37:03.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.467+0000 7f5d04d81b80 10 client.15780 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T09:37:03.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.467+0000 7f5d04d81b80 10 client.15780 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T09:37:03.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.467+0000 7f5d04d81b80 20 client.15780 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:03.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.467+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/4145026031 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5641c790baf0 0x5641c7977be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.467+0000 7f5d04d81b80 4 client.15780 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T09:37:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.467+0000 7f5d04d81b80 1 -- 172.21.15.2:0/4145026031 --> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641c781b3c0 con 0x5641c790baf0 2024-05-14T09:37:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.467+0000 7f5ce7fff700 1 --2- 172.21.15.2:0/4145026031 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5641c790baf0 0x5641c7977be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.467+0000 7f5ccf7fe700 10 client.15780 ms_handle_connect on v2:172.21.15.26:6836/3301773578 2024-05-14T09:37:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.467+0000 7f5ce7fff700 1 --2- 172.21.15.2:0/4145026031 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5641c790baf0 0x5641c7977be0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5328cd55eadfb8d server_cookie=1f150204bc1b85fb in_seq=0 out_seq=0 2024-05-14T09:37:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.468+0000 7f5ccf7fe700 1 -- 172.21.15.2:0/4145026031 <== mds.1 v2:172.21.15.26:6836/3301773578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641c781b3c0 con 0x5641c790baf0 2024-05-14T09:37:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.468+0000 7f5ccf7fe700 10 client.15780 handle_command_reply: tid=0 2024-05-14T09:37:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.468+0000 7f5d04d81b80 1 client.15780 shutdown 2024-05-14T09:37:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.468+0000 7f5d04d81b80 1 -- 172.21.15.2:0/4145026031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c7773e40 msgr2=0x5641c7913ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.468+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/4145026031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c7773e40 0x5641c7913ac0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5ce00098e0 tx=0x7f5ce0009310 comp rx=0 tx=0).stop 2024-05-14T09:37:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.468+0000 7f5d04d81b80 1 -- 172.21.15.2:0/4145026031 shutdown_connections 2024-05-14T09:37:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.468+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/4145026031 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x5641c790baf0 0x5641c7977be0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.468+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/4145026031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5641c78f7d80 0x5641c79041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.468+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/4145026031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641c7773e40 0x5641c7913ac0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.468+0000 7f5d04d81b80 1 --2- 172.21.15.2:0/4145026031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5641c78f1e10 0x5641c7903c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.468+0000 7f5ccf7fe700 0 client.15780 ms_handle_reset on v2:172.21.15.26:6836/3301773578 2024-05-14T09:37:03.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.468+0000 7f5d04d81b80 1 -- 172.21.15.2:0/4145026031 >> 172.21.15.2:0/4145026031 conn(0x5641c77bb630 msgr2=0x5641c78ec270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:03.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.468+0000 7f5d04d81b80 1 -- 172.21.15.2:0/4145026031 shutdown_connections 2024-05-14T09:37:03.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.468+0000 7f5d04d81b80 1 -- 172.21.15.2:0/4145026031 wait complete. 2024-05-14T09:37:03.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.468+0000 7f5d04d81b80 20 client.15780 trim_cache size 0 max 16384 2024-05-14T09:37:03.483 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-05-14T09:37:03.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.469+0000 7f5cf636d700 1 -- 172.21.15.2:0/178799456 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5cdc0b2880 msgr2=0x7f5cdc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:03.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.469+0000 7f5cf636d700 1 --2- 172.21.15.2:0/178799456 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5cdc0b2880 0x7f5cdc0b4d90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0000c00 tx=0x7f5ce001a040 comp rx=0 tx=0).stop 2024-05-14T09:37:03.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.469+0000 7f5cf636d700 1 -- 172.21.15.2:0/178799456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cf0108740 msgr2=0x7f5cf00ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:03.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.469+0000 7f5cf636d700 1 --2- 172.21.15.2:0/178799456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cf0108740 0x7f5cf00ffd20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800ef30 tx=0x7f5ce800c3a0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.469+0000 7f5cf636d700 1 -- 172.21.15.2:0/178799456 shutdown_connections 2024-05-14T09:37:03.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.469+0000 7f5cf636d700 1 --2- 172.21.15.2:0/178799456 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5cdc0b2880 0x7f5cdc0b4d90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.469+0000 7f5cf636d700 1 --2- 172.21.15.2:0/178799456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cf0108740 0x7f5cf00ffd20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.469+0000 7f5cf636d700 1 --2- 172.21.15.2:0/178799456 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cf01034c0 0x7f5cf00ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.469+0000 7f5cf636d700 1 --2- 172.21.15.2:0/178799456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cf0107df0 0x7f5cf00ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:03.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.469+0000 7f5cf636d700 1 -- 172.21.15.2:0/178799456 >> 172.21.15.2:0/178799456 conn(0x7f5cf00fedc0 msgr2=0x7f5cf01063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:03.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.469+0000 7f5cf636d700 1 -- 172.21.15.2:0/178799456 shutdown_connections 2024-05-14T09:37:03.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:03.469+0000 7f5cf636d700 1 -- 172.21.15.2:0/178799456 wait complete. 2024-05-14T09:37:04.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 78198 14.67 MB/sec warmup 30 sec latency 23.155 ms 2024-05-14T09:37:04.632 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:04 smithi002 ceph-mon[31557]: pgmap v487: 129 pgs: 129 active+clean; 28 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 544 op/s 2024-05-14T09:37:04.662 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T09:37:04.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:04 smithi029 ceph-mon[36249]: pgmap v487: 129 pgs: 129 active+clean; 28 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 544 op/s 2024-05-14T09:37:04.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:04 smithi026 ceph-mon[36056]: pgmap v487: 129 pgs: 129 active+clean; 28 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 544 op/s 2024-05-14T09:37:04.917 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:37:05.253 INFO:tasks.workunit.client.0.smithi002.stdout: 1 78246 14.20 MB/sec warmup 31 sec latency 919.470 ms 2024-05-14T09:37:05.344 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:05 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:37:05.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:05 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:37:05.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:05 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:37:06.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 78246 13.76 MB/sec warmup 32 sec latency 1919.568 ms 2024-05-14T09:37:06.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:06 smithi002 ceph-mon[31557]: pgmap v488: 129 pgs: 129 active+clean; 28 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.5 MiB/s wr, 387 op/s 2024-05-14T09:37:06.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:06 smithi029 ceph-mon[36249]: pgmap v488: 129 pgs: 129 active+clean; 28 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.5 MiB/s wr, 387 op/s 2024-05-14T09:37:06.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:06 smithi026 ceph-mon[36056]: pgmap v488: 129 pgs: 129 active+clean; 28 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.5 MiB/s wr, 387 op/s 2024-05-14T09:37:07.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 78246 13.34 MB/sec warmup 33 sec latency 2919.622 ms 2024-05-14T09:37:07.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.333+0000 7f197163f700 1 -- 172.21.15.2:0/3253439984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f196c1045e0 msgr2=0x7f196c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:07.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.333+0000 7f197163f700 1 --2- 172.21.15.2:0/3253439984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f196c1045e0 0x7f196c1049f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f195c0099c0 tx=0x7f195c009cd0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.333+0000 7f197163f700 1 -- 172.21.15.2:0/3253439984 shutdown_connections 2024-05-14T09:37:07.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.333+0000 7f197163f700 1 --2- 172.21.15.2:0/3253439984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f196c109860 0x7f196c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.333+0000 7f197163f700 1 --2- 172.21.15.2:0/3253439984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f196c1045e0 0x7f196c1049f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.333+0000 7f197163f700 1 --2- 172.21.15.2:0/3253439984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c108f10 0x7f196c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.333+0000 7f197163f700 1 -- 172.21.15.2:0/3253439984 >> 172.21.15.2:0/3253439984 conn(0x7f196c0ffe70 msgr2=0x7f196c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:07.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.333+0000 7f197163f700 1 -- 172.21.15.2:0/3253439984 shutdown_connections 2024-05-14T09:37:07.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.333+0000 7f197163f700 1 -- 172.21.15.2:0/3253439984 wait complete. 2024-05-14T09:37:07.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.334+0000 7f197163f700 1 Processor -- start 2024-05-14T09:37:07.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.334+0000 7f197163f700 1 -- start start 2024-05-14T09:37:07.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f197163f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f196c1045e0 0x7f196c075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f197163f700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c108f10 0x7f196c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f197163f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f196c109860 0x7f196c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f197163f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f196c114490 con 0x7f196c109860 2024-05-14T09:37:07.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f197163f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f196c114310 con 0x7f196c1045e0 2024-05-14T09:37:07.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f197163f700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f196c114610 con 0x7f196c108f10 2024-05-14T09:37:07.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f196b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f196c109860 0x7f196c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f196b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f196c109860 0x7f196c076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58060/0 (socket says 172.21.15.2:58060) 2024-05-14T09:37:07.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f196affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f196c1045e0 0x7f196c075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f196b7fe700 1 -- 172.21.15.2:0/2361875826 learned_addr learned my addr 172.21.15.2:0/2361875826 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:07.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f196a7fc700 1 --2- 172.21.15.2:0/2361875826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c108f10 0x7f196c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f196b7fe700 1 -- 172.21.15.2:0/2361875826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c108f10 msgr2=0x7f196c076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:07.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f196b7fe700 1 --2- 172.21.15.2:0/2361875826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c108f10 0x7f196c076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f196b7fe700 1 -- 172.21.15.2:0/2361875826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f196c1045e0 msgr2=0x7f196c075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:07.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f196b7fe700 1 --2- 172.21.15.2:0/2361875826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f196c1045e0 0x7f196c075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f196b7fe700 1 -- 172.21.15.2:0/2361875826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c1a91b0 con 0x7f196c109860 2024-05-14T09:37:07.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.335+0000 7f196affd700 1 --2- 172.21.15.2:0/2361875826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f196c1045e0 0x7f196c075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:07.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.336+0000 7f196b7fe700 1 --2- 172.21.15.2:0/2361875826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f196c109860 0x7f196c076540 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1964007ee0 tx=0x7f196400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:07.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.336+0000 7f1953fff700 1 -- 172.21.15.2:0/2361875826 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964004bb0 con 0x7f196c109860 2024-05-14T09:37:07.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.336+0000 7f197163f700 1 -- 172.21.15.2:0/2361875826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196c1a94a0 con 0x7f196c109860 2024-05-14T09:37:07.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.336+0000 7f1953fff700 1 -- 172.21.15.2:0/2361875826 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1964004d50 con 0x7f196c109860 2024-05-14T09:37:07.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.336+0000 7f1953fff700 1 -- 172.21.15.2:0/2361875826 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964005920 con 0x7f196c109860 2024-05-14T09:37:07.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.336+0000 7f197163f700 1 -- 172.21.15.2:0/2361875826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196c1a9990 con 0x7f196c109860 2024-05-14T09:37:07.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.339+0000 7f1953fff700 1 -- 172.21.15.2:0/2361875826 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f196401a020 con 0x7f196c109860 2024-05-14T09:37:07.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.340+0000 7f1953fff700 1 --2- 172.21.15.2:0/2361875826 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f19540b2880 0x7f19540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.340+0000 7f1980053b80 1 Processor -- start 2024-05-14T09:37:07.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.340+0000 7f196affd700 1 --2- 172.21.15.2:0/2361875826 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f19540b2880 0x7f19540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.341+0000 7f1980053b80 1 -- start start 2024-05-14T09:37:07.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.341+0000 7f1953fff700 1 -- 172.21.15.2:0/2361875826 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f19640e1cb0 con 0x7f196c109860 2024-05-14T09:37:07.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.341+0000 7f1980053b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731997e40 0x559731b1b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.341+0000 7f1980053b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731b1bd80 0x559731b158d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.341+0000 7f1980053b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731b15e10 0x559731b16220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.341+0000 7f1980053b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559731a37940 con 0x559731997e40 2024-05-14T09:37:07.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.341+0000 7f1980053b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55973199cda0 con 0x559731b15e10 2024-05-14T09:37:07.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.341+0000 7f1980053b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559731a33900 con 0x559731b1bd80 2024-05-14T09:37:07.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.341+0000 7f196affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731997e40 0x559731b1b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.341+0000 7f196affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731997e40 0x559731b1b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58076/0 (socket says 172.21.15.2:58076) 2024-05-14T09:37:07.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.341+0000 7f196affd700 1 -- 172.21.15.2:0/92820243 learned_addr learned my addr 172.21.15.2:0/92820243 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:07.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.341+0000 7f196affd700 1 --2- 172.21.15.2:0/2361875826 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f19540b2880 0x7f19540b4d90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f195c009990 tx=0x7f195c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:07.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.341+0000 7f196b7fe700 1 --2- 172.21.15.2:0/92820243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731b15e10 0x559731b16220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.341+0000 7f196affd700 1 -- 172.21.15.2:0/92820243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731b1bd80 msgr2=0x559731b158d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:37:07.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.342+0000 7f196affd700 1 --2- 172.21.15.2:0/92820243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731b1bd80 0x559731b158d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.342+0000 7f196affd700 1 -- 172.21.15.2:0/92820243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731b15e10 msgr2=0x559731b16220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:07.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.342+0000 7f196affd700 1 --2- 172.21.15.2:0/92820243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731b15e10 0x559731b16220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.342+0000 7f196affd700 1 -- 172.21.15.2:0/92820243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597319848d0 con 0x559731997e40 2024-05-14T09:37:07.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.342+0000 7f196b7fe700 1 --2- 172.21.15.2:0/92820243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731b15e10 0x559731b16220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:37:07.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.342+0000 7f196affd700 1 --2- 172.21.15.2:0/92820243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731997e40 0x559731b1b840 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f195c0098e0 tx=0x7f195c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce54542acb8b441 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:07.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.342+0000 7f19517fa700 1 -- 172.21.15.2:0/92820243 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c026080 con 0x559731997e40 2024-05-14T09:37:07.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.342+0000 7f19517fa700 1 -- 172.21.15.2:0/92820243 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195c026220 con 0x559731997e40 2024-05-14T09:37:07.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.342+0000 7f19517fa700 1 -- 172.21.15.2:0/92820243 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c005290 con 0x559731997e40 2024-05-14T09:37:07.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.343+0000 7f1980053b80 1 -- 172.21.15.2:0/92820243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731997e40 msgr2=0x559731b1b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:07.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.343+0000 7f1980053b80 1 --2- 172.21.15.2:0/92820243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731997e40 0x559731b1b840 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f195c0098e0 tx=0x7f195c01f000 comp rx=0 tx=0).stop 2024-05-14T09:37:07.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.343+0000 7f1980053b80 1 -- 172.21.15.2:0/92820243 shutdown_connections 2024-05-14T09:37:07.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.343+0000 7f1980053b80 1 --2- 172.21.15.2:0/92820243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731b15e10 0x559731b16220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.343+0000 7f1980053b80 1 --2- 172.21.15.2:0/92820243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731997e40 0x559731b1b840 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.343+0000 7f1980053b80 1 --2- 172.21.15.2:0/92820243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731b1bd80 0x559731b158d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.343+0000 7f1980053b80 1 -- 172.21.15.2:0/92820243 >> 172.21.15.2:0/92820243 conn(0x5597319df630 msgr2=0x559731b10a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:07.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.343+0000 7f1980053b80 1 -- 172.21.15.2:0/92820243 shutdown_connections 2024-05-14T09:37:07.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.343+0000 7f1980053b80 1 -- 172.21.15.2:0/92820243 wait complete. 2024-05-14T09:37:07.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.344+0000 7f1980053b80 1 -- start start 2024-05-14T09:37:07.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.344+0000 7f1980053b80 1 Processor -- start 2024-05-14T09:37:07.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.344+0000 7f1980053b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731997e40 0x559731b1f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.344+0000 7f1980053b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731b15e10 0x559731b1fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.344+0000 7f1980053b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b1bd80 0x559731b1ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.344+0000 7f1980053b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559731b227c0 con 0x559731b1bd80 2024-05-14T09:37:07.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.344+0000 7f1980053b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5597319f0ed0 con 0x559731997e40 2024-05-14T09:37:07.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.344+0000 7f1980053b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559731b22940 con 0x559731b15e10 2024-05-14T09:37:07.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.344+0000 7f1980053b80 10 client.0 _ll_register_callbacks cb 0x5597319cf950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:07.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.344+0000 7f196b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b1bd80 0x559731b1ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.344+0000 7f196b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b1bd80 0x559731b1ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58078/0 (socket says 172.21.15.2:58078) 2024-05-14T09:37:07.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.344+0000 7f196b7fe700 1 -- 172.21.15.2:0/2105167313 learned_addr learned my addr 172.21.15.2:0/2105167313 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:07.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.344+0000 7f196a7fc700 1 --2- 172.21.15.2:0/2105167313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731b15e10 0x559731b1fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.344+0000 7f196affd700 1 --2- 172.21.15.2:0/2105167313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731997e40 0x559731b1f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.345+0000 7f196b7fe700 1 -- 172.21.15.2:0/2105167313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731b15e10 msgr2=0x559731b1fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:07.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.345+0000 7f196b7fe700 1 --2- 172.21.15.2:0/2105167313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731b15e10 0x559731b1fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.345+0000 7f196b7fe700 1 -- 172.21.15.2:0/2105167313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731997e40 msgr2=0x559731b1f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:07.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.345+0000 7f196b7fe700 1 --2- 172.21.15.2:0/2105167313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731997e40 0x559731b1f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.345+0000 7f196b7fe700 1 -- 172.21.15.2:0/2105167313 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597319ee760 con 0x559731b1bd80 2024-05-14T09:37:07.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.345+0000 7f196b7fe700 1 --2- 172.21.15.2:0/2105167313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b1bd80 0x559731b1ff60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f196400d700 tx=0x7f19640a9360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af31ba5816bda655 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:07.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.345+0000 7f194eff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:37:07.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.345+0000 7f194eff5700 1 -- 172.21.15.2:0/2105167313 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19640a9550 con 0x559731b1bd80 2024-05-14T09:37:07.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.345+0000 7f194eff5700 1 -- 172.21.15.2:0/2105167313 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1964005110 con 0x559731b1bd80 2024-05-14T09:37:07.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.345+0000 7f1980053b80 1 -- 172.21.15.2:0/2105167313 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f195c026220 con 0x559731b1bd80 2024-05-14T09:37:07.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.345+0000 7f194eff5700 1 -- 172.21.15.2:0/2105167313 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f19640a8e50 con 0x559731b1bd80 2024-05-14T09:37:07.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.345+0000 7f1980053b80 10 client.15792 fetch_fsmap learned FSMap version 28 2024-05-14T09:37:07.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.345+0000 7f1980053b80 1 -- 172.21.15.2:0/2105167313 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559731b2d360 con 0x559731b1bd80 2024-05-14T09:37:07.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.346+0000 7f194eff5700 1 -- 172.21.15.2:0/2105167313 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195c026220 con 0x559731b1bd80 2024-05-14T09:37:07.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.346+0000 7f194eff5700 1 -- 172.21.15.2:0/2105167313 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f19640c0030 con 0x559731b1bd80 2024-05-14T09:37:07.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.346+0000 7f1980053b80 10 client.15792 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T09:37:07.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.346+0000 7f1980053b80 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:07.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.346+0000 7f1980053b80 20 client.15792 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:07.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.346+0000 7f1980053b80 1 --2- 172.21.15.2:0/2105167313 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559731b2f940 0x559731b31dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.346+0000 7f1980053b80 4 client.15792 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:37:07.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.346+0000 7f1980053b80 1 -- 172.21.15.2:0/2105167313 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559731a45320 con 0x559731b2f940 2024-05-14T09:37:07.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.346+0000 7f196affd700 1 --2- 172.21.15.2:0/2105167313 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559731b2f940 0x559731b31dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.347+0000 7f196affd700 1 --2- 172.21.15.2:0/2105167313 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559731b2f940 0x559731b31dd0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5010009354ecfddc server_cookie=69fa702849ca9883 in_seq=0 out_seq=0 2024-05-14T09:37:07.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.347+0000 7f194eff5700 10 client.15792 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:07.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.347+0000 7f194eff5700 1 -- 172.21.15.2:0/2105167313 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559731a45320 con 0x559731b2f940 2024-05-14T09:37:07.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.347+0000 7f194eff5700 10 client.15792 handle_command_reply: tid=0 2024-05-14T09:37:07.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.348+0000 7f1980053b80 1 client.15792 shutdown 2024-05-14T09:37:07.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.348+0000 7f1980053b80 1 -- 172.21.15.2:0/2105167313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b1bd80 msgr2=0x559731b1ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:07.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.348+0000 7f1980053b80 1 --2- 172.21.15.2:0/2105167313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b1bd80 0x559731b1ff60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f196400d700 tx=0x7f19640a9360 comp rx=0 tx=0).stop 2024-05-14T09:37:07.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.348+0000 7f1980053b80 1 -- 172.21.15.2:0/2105167313 shutdown_connections 2024-05-14T09:37:07.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.348+0000 7f1980053b80 1 --2- 172.21.15.2:0/2105167313 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559731b2f940 0x559731b31dd0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.348+0000 7f194eff5700 0 client.15792 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:07.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.348+0000 7f1980053b80 1 --2- 172.21.15.2:0/2105167313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731997e40 0x559731b1f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.348+0000 7f1980053b80 1 --2- 172.21.15.2:0/2105167313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b1bd80 0x559731b1ff60 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.348+0000 7f1980053b80 1 --2- 172.21.15.2:0/2105167313 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731b15e10 0x559731b1fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.348+0000 7f1980053b80 1 -- 172.21.15.2:0/2105167313 >> 172.21.15.2:0/2105167313 conn(0x5597319df630 msgr2=0x559731b148c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:07.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.348+0000 7f1980053b80 1 -- 172.21.15.2:0/2105167313 shutdown_connections 2024-05-14T09:37:07.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.348+0000 7f1980053b80 1 -- 172.21.15.2:0/2105167313 wait complete. 2024-05-14T09:37:07.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.348+0000 7f1980053b80 20 client.15792 trim_cache size 0 max 16384 2024-05-14T09:37:07.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.363+0000 7f1980053b80 1 Processor -- start 2024-05-14T09:37:07.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.363+0000 7f1980053b80 1 -- start start 2024-05-14T09:37:07.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.363+0000 7f1980053b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731997e40 0x559731b158d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.363+0000 7f1980053b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b15e10 0x559731b1ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.363+0000 7f1980053b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731b1bd80 0x559731b1ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.363+0000 7f1980053b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1b4ed0 con 0x559731b15e10 2024-05-14T09:37:07.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.363+0000 7f1980053b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1b2760 con 0x559731b1bd80 2024-05-14T09:37:07.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.363+0000 7f1980053b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1b05b0 con 0x559731997e40 2024-05-14T09:37:07.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.364+0000 7f196a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b15e10 0x559731b1ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.364+0000 7f196a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b15e10 0x559731b1ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58090/0 (socket says 172.21.15.2:58090) 2024-05-14T09:37:07.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.364+0000 7f196a7fc700 1 -- 172.21.15.2:0/3177712151 learned_addr learned my addr 172.21.15.2:0/3177712151 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:07.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.364+0000 7f196affd700 1 --2- 172.21.15.2:0/3177712151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731997e40 0x559731b158d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.364+0000 7f196b7fe700 1 --2- 172.21.15.2:0/3177712151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731b1bd80 0x559731b1ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.364+0000 7f196a7fc700 1 -- 172.21.15.2:0/3177712151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731997e40 msgr2=0x559731b158d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:07.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.364+0000 7f196a7fc700 1 --2- 172.21.15.2:0/3177712151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731997e40 0x559731b158d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.364+0000 7f196a7fc700 1 -- 172.21.15.2:0/3177712151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731b1bd80 msgr2=0x559731b1ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:07.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.364+0000 7f196a7fc700 1 --2- 172.21.15.2:0/3177712151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731b1bd80 0x559731b1ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.364+0000 7f196a7fc700 1 -- 172.21.15.2:0/3177712151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559731b11730 con 0x559731b15e10 2024-05-14T09:37:07.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.364+0000 7f196b7fe700 1 --2- 172.21.15.2:0/3177712151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731b1bd80 0x559731b1ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:37:07.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.364+0000 7f196a7fc700 1 --2- 172.21.15.2:0/3177712151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b15e10 0x559731b1ad40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1960005950 tx=0x7f196000b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91419dfc910ec61 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:07.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.364+0000 7f1950ff9700 1 -- 172.21.15.2:0/3177712151 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960003880 con 0x559731b15e10 2024-05-14T09:37:07.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.364+0000 7f1950ff9700 1 -- 172.21.15.2:0/3177712151 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1960011440 con 0x559731b15e10 2024-05-14T09:37:07.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.364+0000 7f1950ff9700 1 -- 172.21.15.2:0/3177712151 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19600106a0 con 0x559731b15e10 2024-05-14T09:37:07.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.365+0000 7f1980053b80 1 -- 172.21.15.2:0/3177712151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b15e10 msgr2=0x559731b1ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:07.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.365+0000 7f1980053b80 1 --2- 172.21.15.2:0/3177712151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b15e10 0x559731b1ad40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1960005950 tx=0x7f196000b9f0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.365+0000 7f1980053b80 1 -- 172.21.15.2:0/3177712151 shutdown_connections 2024-05-14T09:37:07.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.365+0000 7f1980053b80 1 --2- 172.21.15.2:0/3177712151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731b1bd80 0x559731b1ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.365+0000 7f1980053b80 1 --2- 172.21.15.2:0/3177712151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731997e40 0x559731b158d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.365+0000 7f1980053b80 1 --2- 172.21.15.2:0/3177712151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b15e10 0x559731b1ad40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.365+0000 7f1980053b80 1 -- 172.21.15.2:0/3177712151 >> 172.21.15.2:0/3177712151 conn(0x5597319df630 msgr2=0x559731b1f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:07.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.365+0000 7f1980053b80 1 -- 172.21.15.2:0/3177712151 shutdown_connections 2024-05-14T09:37:07.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.365+0000 7f1980053b80 1 -- 172.21.15.2:0/3177712151 wait complete. 2024-05-14T09:37:07.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.366+0000 7f1980053b80 1 -- start start 2024-05-14T09:37:07.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.366+0000 7f1980053b80 1 Processor -- start 2024-05-14T09:37:07.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.366+0000 7f1980053b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731997e40 0x559731b37b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.366+0000 7f1980053b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731b15e10 0x559731b27bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.366+0000 7f1980053b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b1bd80 0x559731b28110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.366+0000 7f1980053b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1b0c20 con 0x559731b1bd80 2024-05-14T09:37:07.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.366+0000 7f1980053b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1b2090 con 0x559731997e40 2024-05-14T09:37:07.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.366+0000 7f1980053b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1b12f0 con 0x559731b15e10 2024-05-14T09:37:07.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.366+0000 7f1980053b80 10 client.0 _ll_register_callbacks cb 0x5597319cf950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:07.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.366+0000 7f196b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b1bd80 0x559731b28110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.366+0000 7f196a7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731b15e10 0x559731b27bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.366+0000 7f196b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b1bd80 0x559731b28110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58106/0 (socket says 172.21.15.2:58106) 2024-05-14T09:37:07.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.366+0000 7f196b7fe700 1 -- 172.21.15.2:0/1405709513 learned_addr learned my addr 172.21.15.2:0/1405709513 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:07.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.367+0000 7f196affd700 1 --2- 172.21.15.2:0/1405709513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731997e40 0x559731b37b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.367+0000 7f196b7fe700 1 -- 172.21.15.2:0/1405709513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731b15e10 msgr2=0x559731b27bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:07.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.367+0000 7f196b7fe700 1 --2- 172.21.15.2:0/1405709513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731b15e10 0x559731b27bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.367+0000 7f196b7fe700 1 -- 172.21.15.2:0/1405709513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731997e40 msgr2=0x559731b37b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:07.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.367+0000 7f196b7fe700 1 --2- 172.21.15.2:0/1405709513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731997e40 0x559731b37b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.367+0000 7f196b7fe700 1 -- 172.21.15.2:0/1405709513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559731b36630 con 0x559731b1bd80 2024-05-14T09:37:07.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.367+0000 7f196a7fc700 1 --2- 172.21.15.2:0/1405709513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731b15e10 0x559731b27bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:07.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.367+0000 7f196b7fe700 1 --2- 172.21.15.2:0/1405709513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b1bd80 0x559731b28110 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f19640a9010 tx=0x7f1964005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac01f5d2b996ad0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:07.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.367+0000 7f194eff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:37:07.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.367+0000 7f194eff5700 1 -- 172.21.15.2:0/1405709513 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19640a8c20 con 0x559731b1bd80 2024-05-14T09:37:07.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.367+0000 7f194eff5700 1 -- 172.21.15.2:0/1405709513 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f19640a5440 con 0x559731b1bd80 2024-05-14T09:37:07.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.367+0000 7f1980053b80 1 -- 172.21.15.2:0/1405709513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1960011440 con 0x559731b1bd80 2024-05-14T09:37:07.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.367+0000 7f194eff5700 1 -- 172.21.15.2:0/1405709513 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f19640a4b30 con 0x559731b1bd80 2024-05-14T09:37:07.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.367+0000 7f1980053b80 10 client.15798 fetch_fsmap learned FSMap version 28 2024-05-14T09:37:07.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.367+0000 7f1980053b80 1 -- 172.21.15.2:0/1405709513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559731b993f0 con 0x559731b1bd80 2024-05-14T09:37:07.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.368+0000 7f194eff5700 1 -- 172.21.15.2:0/1405709513 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1960011440 con 0x559731b1bd80 2024-05-14T09:37:07.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.368+0000 7f194eff5700 1 -- 172.21.15.2:0/1405709513 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f19640c0040 con 0x559731b1bd80 2024-05-14T09:37:07.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.368+0000 7f1980053b80 10 client.15798 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T09:37:07.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.368+0000 7f1980053b80 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:07.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.368+0000 7f1980053b80 20 client.15798 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:07.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.368+0000 7f1980053b80 1 --2- 172.21.15.2:0/1405709513 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559731b99af0 0x559731b9bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:07.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.368+0000 7f1980053b80 4 client.15798 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T09:37:07.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.368+0000 7f1980053b80 1 -- 172.21.15.2:0/1405709513 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559731a3f3c0 con 0x559731b99af0 2024-05-14T09:37:07.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.368+0000 7f196affd700 1 --2- 172.21.15.2:0/1405709513 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559731b99af0 0x559731b9bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:07.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f194eff5700 10 client.15798 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:07.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f196affd700 1 --2- 172.21.15.2:0/1405709513 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559731b99af0 0x559731b9bf10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a1cc764a66cd35 server_cookie=11f40c7a12ee5d55 in_seq=0 out_seq=0 2024-05-14T09:37:07.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f194eff5700 1 -- 172.21.15.2:0/1405709513 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559731a3f3c0 con 0x559731b99af0 2024-05-14T09:37:07.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f194eff5700 10 client.15798 handle_command_reply: tid=0 2024-05-14T09:37:07.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f1980053b80 1 client.15798 shutdown 2024-05-14T09:37:07.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f1980053b80 1 -- 172.21.15.2:0/1405709513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b1bd80 msgr2=0x559731b28110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:07.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f1980053b80 1 --2- 172.21.15.2:0/1405709513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b1bd80 0x559731b28110 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f19640a9010 tx=0x7f1964005320 comp rx=0 tx=0).stop 2024-05-14T09:37:07.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f1980053b80 1 -- 172.21.15.2:0/1405709513 shutdown_connections 2024-05-14T09:37:07.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f1980053b80 1 --2- 172.21.15.2:0/1405709513 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559731b99af0 0x559731b9bf10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f1980053b80 1 --2- 172.21.15.2:0/1405709513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559731997e40 0x559731b37b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f1980053b80 1 --2- 172.21.15.2:0/1405709513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559731b1bd80 0x559731b28110 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f194eff5700 0 client.15798 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:07.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f1980053b80 1 --2- 172.21.15.2:0/1405709513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559731b15e10 0x559731b27bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f1980053b80 1 -- 172.21.15.2:0/1405709513 >> 172.21.15.2:0/1405709513 conn(0x5597319df630 msgr2=0x559731b0ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:07.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f1980053b80 1 -- 172.21.15.2:0/1405709513 shutdown_connections 2024-05-14T09:37:07.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f1980053b80 1 -- 172.21.15.2:0/1405709513 wait complete. 2024-05-14T09:37:07.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.369+0000 7f1980053b80 20 client.15798 trim_cache size 0 max 16384 2024-05-14T09:37:07.374 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-05-14T09:37:07.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.370+0000 7f197163f700 1 -- 172.21.15.2:0/2361875826 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f19540b2880 msgr2=0x7f19540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:07.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.370+0000 7f197163f700 1 --2- 172.21.15.2:0/2361875826 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f19540b2880 0x7f19540b4d90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f195c009990 tx=0x7f195c01a040 comp rx=0 tx=0).stop 2024-05-14T09:37:07.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.370+0000 7f197163f700 1 -- 172.21.15.2:0/2361875826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f196c109860 msgr2=0x7f196c076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:07.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.370+0000 7f197163f700 1 --2- 172.21.15.2:0/2361875826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f196c109860 0x7f196c076540 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1964007ee0 tx=0x7f196400d3b0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.370+0000 7f197163f700 1 -- 172.21.15.2:0/2361875826 shutdown_connections 2024-05-14T09:37:07.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.370+0000 7f197163f700 1 --2- 172.21.15.2:0/2361875826 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f19540b2880 0x7f19540b4d90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.370+0000 7f197163f700 1 --2- 172.21.15.2:0/2361875826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f196c1045e0 0x7f196c075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.370+0000 7f197163f700 1 --2- 172.21.15.2:0/2361875826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f196c109860 0x7f196c076540 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.370+0000 7f197163f700 1 --2- 172.21.15.2:0/2361875826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f196c108f10 0x7f196c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:07.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.370+0000 7f197163f700 1 -- 172.21.15.2:0/2361875826 >> 172.21.15.2:0/2361875826 conn(0x7f196c0ffe70 msgr2=0x7f196c100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:07.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.370+0000 7f197163f700 1 -- 172.21.15.2:0/2361875826 shutdown_connections 2024-05-14T09:37:07.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:07.370+0000 7f197163f700 1 -- 172.21.15.2:0/2361875826 wait complete. 2024-05-14T09:37:08.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 78246 12.95 MB/sec warmup 34 sec latency 3919.714 ms 2024-05-14T09:37:08.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:08 smithi002 ceph-mon[31557]: pgmap v489: 129 pgs: 129 active+clean; 28 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 468 op/s 2024-05-14T09:37:08.690 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:08 smithi029 ceph-mon[36249]: pgmap v489: 129 pgs: 129 active+clean; 28 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 468 op/s 2024-05-14T09:37:08.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:08 smithi026 ceph-mon[36056]: pgmap v489: 129 pgs: 129 active+clean; 28 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 468 op/s 2024-05-14T09:37:09.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 78647 12.62 MB/sec warmup 35 sec latency 4393.946 ms 2024-05-14T09:37:09.526 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T09:37:09.527 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T09:37:09.796 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:37:10.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 81292 12.64 MB/sec warmup 36 sec latency 12.339 ms 2024-05-14T09:37:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:10 smithi002 ceph-mon[31557]: pgmap v490: 129 pgs: 129 active+clean; 28 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 374 op/s 2024-05-14T09:37:10.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:10 smithi029 ceph-mon[36249]: pgmap v490: 129 pgs: 129 active+clean; 28 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 374 op/s 2024-05-14T09:37:10.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:10 smithi026 ceph-mon[36056]: pgmap v490: 129 pgs: 129 active+clean; 28 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 374 op/s 2024-05-14T09:37:11.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 82704 12.50 MB/sec warmup 37 sec latency 37.826 ms 2024-05-14T09:37:11.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.624+0000 7f1682c57700 1 -- 172.21.15.2:0/2700865434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f167c1034c0 msgr2=0x7f167c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:11.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.624+0000 7f1682c57700 1 --2- 172.21.15.2:0/2700865434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f167c1034c0 0x7f167c1038d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f16780099c0 tx=0x7f1678009cd0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.624+0000 7f1682c57700 1 -- 172.21.15.2:0/2700865434 shutdown_connections 2024-05-14T09:37:11.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.624+0000 7f1682c57700 1 --2- 172.21.15.2:0/2700865434 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f167c108740 0x7f167c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.624+0000 7f1682c57700 1 --2- 172.21.15.2:0/2700865434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f167c1034c0 0x7f167c1038d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.624+0000 7f1682c57700 1 --2- 172.21.15.2:0/2700865434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c107df0 0x7f167c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.624+0000 7f1682c57700 1 -- 172.21.15.2:0/2700865434 >> 172.21.15.2:0/2700865434 conn(0x7f167c0fedc0 msgr2=0x7f167c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:11.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.624+0000 7f1682c57700 1 -- 172.21.15.2:0/2700865434 shutdown_connections 2024-05-14T09:37:11.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.624+0000 7f1682c57700 1 -- 172.21.15.2:0/2700865434 wait complete. 2024-05-14T09:37:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.625+0000 7f1682c57700 1 Processor -- start 2024-05-14T09:37:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.625+0000 7f1682c57700 1 -- start start 2024-05-14T09:37:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.625+0000 7f1682c57700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f167c1034c0 0x7f167c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.625+0000 7f1682c57700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f167c107df0 0x7f167c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.625+0000 7f1682c57700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c108740 0x7f167c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.625+0000 7f1682c57700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f167c113370 con 0x7f167c107df0 2024-05-14T09:37:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.625+0000 7f1682c57700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1131f0 con 0x7f167c108740 2024-05-14T09:37:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.625+0000 7f1682c57700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1134f0 con 0x7f167c1034c0 2024-05-14T09:37:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.625+0000 7f1673fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f167c107df0 0x7f167c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.625+0000 7f16809f3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f167c1034c0 0x7f167c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.625+0000 7f16809f3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f167c1034c0 0x7f167c19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:37742/0 (socket says 172.21.15.2:37742) 2024-05-14T09:37:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.625+0000 7f16809f3700 1 -- 172.21.15.2:0/1899500275 learned_addr learned my addr 172.21.15.2:0/1899500275 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.625+0000 7f16811f4700 1 --2- 172.21.15.2:0/1899500275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c108740 0x7f167c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.625+0000 7f1673fff700 1 --2- 172.21.15.2:0/1899500275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f167c107df0 0x7f167c19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41694/0 (socket says 172.21.15.2:41694) 2024-05-14T09:37:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.626+0000 7f16809f3700 1 -- 172.21.15.2:0/1899500275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c108740 msgr2=0x7f167c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.626+0000 7f16809f3700 1 --2- 172.21.15.2:0/1899500275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c108740 0x7f167c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.626+0000 7f16809f3700 1 -- 172.21.15.2:0/1899500275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f167c107df0 msgr2=0x7f167c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.626+0000 7f16809f3700 1 --2- 172.21.15.2:0/1899500275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f167c107df0 0x7f167c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.626+0000 7f16809f3700 1 -- 172.21.15.2:0/1899500275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167c19fce0 con 0x7f167c1034c0 2024-05-14T09:37:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.626+0000 7f16809f3700 1 --2- 172.21.15.2:0/1899500275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f167c1034c0 0x7f167c19ae10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1678006010 tx=0x7f1678004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.626+0000 7f1671ffb700 1 -- 172.21.15.2:0/1899500275 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678026070 con 0x7f167c1034c0 2024-05-14T09:37:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.626+0000 7f1682c57700 1 -- 172.21.15.2:0/1899500275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167c19ff70 con 0x7f167c1034c0 2024-05-14T09:37:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.626+0000 7f1671ffb700 1 -- 172.21.15.2:0/1899500275 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1678004630 con 0x7f167c1034c0 2024-05-14T09:37:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.626+0000 7f1671ffb700 1 -- 172.21.15.2:0/1899500275 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167800fb20 con 0x7f167c1034c0 2024-05-14T09:37:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.626+0000 7f1682c57700 1 -- 172.21.15.2:0/1899500275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f167c1a78f0 con 0x7f167c1034c0 2024-05-14T09:37:11.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.630+0000 7f1671ffb700 1 -- 172.21.15.2:0/1899500275 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1678029070 con 0x7f167c1034c0 2024-05-14T09:37:11.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.631+0000 7f1671ffb700 1 --2- 172.21.15.2:0/1899500275 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f16640b2880 0x7f16640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.631+0000 7f1673fff700 1 --2- 172.21.15.2:0/1899500275 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f16640b2880 0x7f16640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.631+0000 7f1671ffb700 1 -- 172.21.15.2:0/1899500275 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f16780e2d70 con 0x7f167c1034c0 2024-05-14T09:37:11.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.631+0000 7f169166bb80 1 Processor -- start 2024-05-14T09:37:11.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.631+0000 7f169166bb80 1 -- start start 2024-05-14T09:37:11.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.631+0000 7f1673fff700 1 --2- 172.21.15.2:0/1899500275 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f16640b2880 0x7f16640b4d90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f167c106c00 tx=0x7f166c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:11.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.631+0000 7f169166bb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97b75ec0 0x55ba97cf37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.631+0000 7f169166bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 0x55ba97cf7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.631+0000 7f169166bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 0x55ba97cf8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.631+0000 7f169166bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ba97c158e0 con 0x55ba97cf84f0 2024-05-14T09:37:11.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.631+0000 7f169166bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ba97bb4840 con 0x55ba97cf3ce0 2024-05-14T09:37:11.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.631+0000 7f169166bb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ba97c118a0 con 0x55ba97b75ec0 2024-05-14T09:37:11.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.632+0000 7f16809f3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97b75ec0 0x55ba97cf37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.632+0000 7f1673fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 0x55ba97cf7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.632+0000 7f16811f4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 0x55ba97cf8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.632+0000 7f16809f3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97b75ec0 0x55ba97cf37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:37750/0 (socket says 172.21.15.2:37750) 2024-05-14T09:37:11.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.632+0000 7f16809f3700 1 -- 172.21.15.2:0/2098305150 learned_addr learned my addr 172.21.15.2:0/2098305150 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:11.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.632+0000 7f1673fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 0x55ba97cf7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:49126/0 (socket says 172.21.15.2:49126) 2024-05-14T09:37:11.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.632+0000 7f16809f3700 1 -- 172.21.15.2:0/2098305150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 msgr2=0x55ba97cf7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:11.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.632+0000 7f16809f3700 1 --2- 172.21.15.2:0/2098305150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 0x55ba97cf7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.632+0000 7f16809f3700 1 -- 172.21.15.2:0/2098305150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 msgr2=0x55ba97cf8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:11.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.632+0000 7f16809f3700 1 --2- 172.21.15.2:0/2098305150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 0x55ba97cf8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.632+0000 7f16809f3700 1 -- 172.21.15.2:0/2098305150 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba97b62950 con 0x55ba97b75ec0 2024-05-14T09:37:11.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.632+0000 7f16809f3700 1 --2- 172.21.15.2:0/2098305150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97b75ec0 0x55ba97cf37a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f16780a6080 tx=0x7f1678009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a737be0cdc600dd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:11.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.632+0000 7f1662ffd700 1 -- 172.21.15.2:0/2098305150 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16780aa700 con 0x55ba97b75ec0 2024-05-14T09:37:11.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.632+0000 7f1662ffd700 1 -- 172.21.15.2:0/2098305150 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16780aa000 con 0x55ba97b75ec0 2024-05-14T09:37:11.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.632+0000 7f1662ffd700 1 -- 172.21.15.2:0/2098305150 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16780a5970 con 0x55ba97b75ec0 2024-05-14T09:37:11.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.633+0000 7f169166bb80 1 -- 172.21.15.2:0/2098305150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97b75ec0 msgr2=0x55ba97cf37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:11.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.633+0000 7f169166bb80 1 --2- 172.21.15.2:0/2098305150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97b75ec0 0x55ba97cf37a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f16780a6080 tx=0x7f1678009f60 comp rx=0 tx=0).stop 2024-05-14T09:37:11.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.633+0000 7f169166bb80 1 -- 172.21.15.2:0/2098305150 shutdown_connections 2024-05-14T09:37:11.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.633+0000 7f169166bb80 1 --2- 172.21.15.2:0/2098305150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 0x55ba97cf8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.633+0000 7f169166bb80 1 --2- 172.21.15.2:0/2098305150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97b75ec0 0x55ba97cf37a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.633+0000 7f169166bb80 1 --2- 172.21.15.2:0/2098305150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 0x55ba97cf7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.633+0000 7f169166bb80 1 -- 172.21.15.2:0/2098305150 >> 172.21.15.2:0/2098305150 conn(0x55ba97bbd680 msgr2=0x55ba97cee530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:11.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.633+0000 7f169166bb80 1 -- 172.21.15.2:0/2098305150 shutdown_connections 2024-05-14T09:37:11.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.633+0000 7f169166bb80 1 -- 172.21.15.2:0/2098305150 wait complete. 2024-05-14T09:37:11.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.634+0000 7f169166bb80 1 -- start start 2024-05-14T09:37:11.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.634+0000 7f169166bb80 1 Processor -- start 2024-05-14T09:37:11.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.634+0000 7f169166bb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97b75ec0 0x55ba97cfd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.634+0000 7f169166bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 0x55ba97cfd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.634+0000 7f169166bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 0x55ba97cfdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.634+0000 7f169166bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ba97d00260 con 0x55ba97cf84f0 2024-05-14T09:37:11.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.634+0000 7f169166bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ba97b7bcb0 con 0x55ba97cf3ce0 2024-05-14T09:37:11.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.634+0000 7f169166bb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ba97d003e0 con 0x55ba97b75ec0 2024-05-14T09:37:11.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.634+0000 7f169166bb80 10 client.0 _ll_register_callbacks cb 0x55ba97bad9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:11.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.634+0000 7f16809f3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97b75ec0 0x55ba97cfd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.634+0000 7f1673fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 0x55ba97cfd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.634+0000 7f16811f4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 0x55ba97cfdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.634+0000 7f16809f3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97b75ec0 0x55ba97cfd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:37758/0 (socket says 172.21.15.2:37758) 2024-05-14T09:37:11.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.634+0000 7f16809f3700 1 -- 172.21.15.2:0/1374411590 learned_addr learned my addr 172.21.15.2:0/1374411590 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:11.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.635+0000 7f16811f4700 1 -- 172.21.15.2:0/1374411590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97b75ec0 msgr2=0x55ba97cfd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:11.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.635+0000 7f16811f4700 1 --2- 172.21.15.2:0/1374411590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97b75ec0 0x55ba97cfd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.635+0000 7f16811f4700 1 -- 172.21.15.2:0/1374411590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 msgr2=0x55ba97cfd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:11.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.635+0000 7f16811f4700 1 --2- 172.21.15.2:0/1374411590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 0x55ba97cfd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.635+0000 7f16811f4700 1 -- 172.21.15.2:0/1374411590 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba97bb1710 con 0x55ba97cf84f0 2024-05-14T09:37:11.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.635+0000 7f16809f3700 1 --2- 172.21.15.2:0/1374411590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97b75ec0 0x55ba97cfd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:11.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.635+0000 7f1673fff700 1 --2- 172.21.15.2:0/1374411590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 0x55ba97cfd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:11.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.635+0000 7f16811f4700 1 --2- 172.21.15.2:0/1374411590 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 0x55ba97cfdd40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1674011ec0 tx=0x7f1674014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300f786a4f1802e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:11.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.635+0000 7f165bfff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:37:11.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.635+0000 7f165bfff700 1 -- 172.21.15.2:0/1374411590 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1674004bb0 con 0x55ba97cf84f0 2024-05-14T09:37:11.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.635+0000 7f165bfff700 1 -- 172.21.15.2:0/1374411590 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1674004d50 con 0x55ba97cf84f0 2024-05-14T09:37:11.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.635+0000 7f169166bb80 1 -- 172.21.15.2:0/1374411590 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16780aa000 con 0x55ba97cf84f0 2024-05-14T09:37:11.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.635+0000 7f165bfff700 1 -- 172.21.15.2:0/1374411590 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1674026050 con 0x55ba97cf84f0 2024-05-14T09:37:11.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.635+0000 7f169166bb80 10 client.15810 fetch_fsmap learned FSMap version 28 2024-05-14T09:37:11.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.635+0000 7f169166bb80 1 -- 172.21.15.2:0/1374411590 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ba97d0aed0 con 0x55ba97cf84f0 2024-05-14T09:37:11.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.636+0000 7f165bfff700 1 -- 172.21.15.2:0/1374411590 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16780aa000 con 0x55ba97cf84f0 2024-05-14T09:37:11.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.636+0000 7f165bfff700 1 -- 172.21.15.2:0/1374411590 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1674004170 con 0x55ba97cf84f0 2024-05-14T09:37:11.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.636+0000 7f169166bb80 10 client.15810 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T09:37:11.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.636+0000 7f169166bb80 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:11.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.636+0000 7f169166bb80 20 client.15810 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:11.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.636+0000 7f169166bb80 1 --2- 172.21.15.2:0/1374411590 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ba97d0d5b0 0x55ba97d0f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.636+0000 7f169166bb80 4 client.15810 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:37:11.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.636+0000 7f169166bb80 1 -- 172.21.15.2:0/1374411590 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba97c232c0 con 0x55ba97d0d5b0 2024-05-14T09:37:11.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.636+0000 7f16809f3700 1 --2- 172.21.15.2:0/1374411590 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ba97d0d5b0 0x55ba97d0f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.636+0000 7f165bfff700 10 client.15810 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:11.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.637+0000 7f16809f3700 1 --2- 172.21.15.2:0/1374411590 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ba97d0d5b0 0x55ba97d0f9d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8266afe0bdb109b2 server_cookie=19632e5a1763cfb5 in_seq=0 out_seq=0 2024-05-14T09:37:11.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.637+0000 7f165bfff700 1 -- 172.21.15.2:0/1374411590 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ba97c232c0 con 0x55ba97d0d5b0 2024-05-14T09:37:11.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.637+0000 7f165bfff700 10 client.15810 handle_command_reply: tid=0 2024-05-14T09:37:11.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.637+0000 7f169166bb80 1 client.15810 shutdown 2024-05-14T09:37:11.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.637+0000 7f169166bb80 1 -- 172.21.15.2:0/1374411590 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 msgr2=0x55ba97cfdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:11.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.637+0000 7f169166bb80 1 --2- 172.21.15.2:0/1374411590 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 0x55ba97cfdd40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1674011ec0 tx=0x7f1674014600 comp rx=0 tx=0).stop 2024-05-14T09:37:11.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.638+0000 7f169166bb80 1 -- 172.21.15.2:0/1374411590 shutdown_connections 2024-05-14T09:37:11.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.638+0000 7f169166bb80 1 --2- 172.21.15.2:0/1374411590 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ba97d0d5b0 0x55ba97d0f9d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.638+0000 7f169166bb80 1 --2- 172.21.15.2:0/1374411590 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 0x55ba97cfdd40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.638+0000 7f165bfff700 0 client.15810 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:11.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.638+0000 7f169166bb80 1 --2- 172.21.15.2:0/1374411590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97b75ec0 0x55ba97cfd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.638+0000 7f169166bb80 1 --2- 172.21.15.2:0/1374411590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 0x55ba97cfd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.638+0000 7f169166bb80 1 -- 172.21.15.2:0/1374411590 >> 172.21.15.2:0/1374411590 conn(0x55ba97bbd680 msgr2=0x55ba97cf24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:11.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.638+0000 7f169166bb80 1 -- 172.21.15.2:0/1374411590 shutdown_connections 2024-05-14T09:37:11.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.638+0000 7f169166bb80 1 -- 172.21.15.2:0/1374411590 wait complete. 2024-05-14T09:37:11.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.638+0000 7f169166bb80 20 client.15810 trim_cache size 0 max 16384 2024-05-14T09:37:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.659+0000 7f169166bb80 1 Processor -- start 2024-05-14T09:37:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.659+0000 7f169166bb80 1 -- start start 2024-05-14T09:37:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f169166bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97b75ec0 0x55ba97cf2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f169166bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 0x55ba97cf9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f169166bb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97cf84f0 0x55ba97cff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f169166bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1b2da0 con 0x55ba97b75ec0 2024-05-14T09:37:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f169166bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1b0630 con 0x55ba97cf3ce0 2024-05-14T09:37:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f169166bb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1ae480 con 0x55ba97cf84f0 2024-05-14T09:37:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f16809f3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97b75ec0 0x55ba97cf2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f16809f3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97b75ec0 0x55ba97cf2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41728/0 (socket says 172.21.15.2:41728) 2024-05-14T09:37:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f16809f3700 1 -- 172.21.15.2:0/3821076212 learned_addr learned my addr 172.21.15.2:0/3821076212 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f16811f4700 1 --2- 172.21.15.2:0/3821076212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97cf84f0 0x55ba97cff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f1673fff700 1 --2- 172.21.15.2:0/3821076212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 0x55ba97cf9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f16809f3700 1 -- 172.21.15.2:0/3821076212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97cf84f0 msgr2=0x55ba97cff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f16809f3700 1 --2- 172.21.15.2:0/3821076212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97cf84f0 0x55ba97cff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f16809f3700 1 -- 172.21.15.2:0/3821076212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 msgr2=0x55ba97cf9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f16809f3700 1 --2- 172.21.15.2:0/3821076212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 0x55ba97cf9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f16809f3700 1 -- 172.21.15.2:0/3821076212 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba97cef3a0 con 0x55ba97b75ec0 2024-05-14T09:37:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f16809f3700 1 --2- 172.21.15.2:0/3821076212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97b75ec0 0x55ba97cf2f00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1678009990 tx=0x7f16780a5ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde704471bfaa327 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:11.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.660+0000 7f16627fc700 1 -- 172.21.15.2:0/3821076212 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16780aa000 con 0x55ba97b75ec0 2024-05-14T09:37:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.661+0000 7f16627fc700 1 -- 172.21.15.2:0/3821076212 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16780b2dd0 con 0x55ba97b75ec0 2024-05-14T09:37:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.661+0000 7f16627fc700 1 -- 172.21.15.2:0/3821076212 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16780a5cd0 con 0x55ba97b75ec0 2024-05-14T09:37:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.661+0000 7f169166bb80 1 -- 172.21.15.2:0/3821076212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97b75ec0 msgr2=0x55ba97cf2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.661+0000 7f169166bb80 1 --2- 172.21.15.2:0/3821076212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97b75ec0 0x55ba97cf2f00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1678009990 tx=0x7f16780a5ff0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.661+0000 7f169166bb80 1 -- 172.21.15.2:0/3821076212 shutdown_connections 2024-05-14T09:37:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.661+0000 7f169166bb80 1 --2- 172.21.15.2:0/3821076212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97cf84f0 0x55ba97cff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.661+0000 7f169166bb80 1 --2- 172.21.15.2:0/3821076212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97b75ec0 0x55ba97cf2f00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.661+0000 7f169166bb80 1 --2- 172.21.15.2:0/3821076212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97cf3ce0 0x55ba97cf9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.661+0000 7f169166bb80 1 -- 172.21.15.2:0/3821076212 >> 172.21.15.2:0/3821076212 conn(0x55ba97bbd680 msgr2=0x55ba97cfe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.662+0000 7f169166bb80 1 -- 172.21.15.2:0/3821076212 shutdown_connections 2024-05-14T09:37:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.662+0000 7f169166bb80 1 -- 172.21.15.2:0/3821076212 wait complete. 2024-05-14T09:37:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.662+0000 7f169166bb80 1 -- start start 2024-05-14T09:37:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.662+0000 7f169166bb80 1 Processor -- start 2024-05-14T09:37:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f169166bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97b75ec0 0x55ba97d03970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f169166bb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97cf3ce0 0x55ba97d03eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f169166bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 0x55ba97d76d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f169166bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1aeaf0 con 0x55ba97cf84f0 2024-05-14T09:37:11.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f169166bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1aff60 con 0x55ba97b75ec0 2024-05-14T09:37:11.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f169166bb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1af1c0 con 0x55ba97cf3ce0 2024-05-14T09:37:11.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f169166bb80 10 client.0 _ll_register_callbacks cb 0x55ba97bad9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:11.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f16809f3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97b75ec0 0x55ba97d03970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f1673fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97cf3ce0 0x55ba97d03eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f16811f4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 0x55ba97d76d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f1673fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97cf3ce0 0x55ba97d03eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:37776/0 (socket says 172.21.15.2:37776) 2024-05-14T09:37:11.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f1673fff700 1 -- 172.21.15.2:0/2122526773 learned_addr learned my addr 172.21.15.2:0/2122526773 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:11.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f16809f3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97b75ec0 0x55ba97d03970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:49142/0 (socket says 172.21.15.2:49142) 2024-05-14T09:37:11.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f1673fff700 1 -- 172.21.15.2:0/2122526773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97b75ec0 msgr2=0x55ba97d03970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:11.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f1673fff700 1 --2- 172.21.15.2:0/2122526773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97b75ec0 0x55ba97d03970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f1673fff700 1 -- 172.21.15.2:0/2122526773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 msgr2=0x55ba97d76d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:11.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f1673fff700 1 --2- 172.21.15.2:0/2122526773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 0x55ba97d76d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f1673fff700 1 -- 172.21.15.2:0/2122526773 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba97d77290 con 0x55ba97cf3ce0 2024-05-14T09:37:11.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f16809f3700 1 --2- 172.21.15.2:0/2122526773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97b75ec0 0x55ba97d03970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:11.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f16811f4700 1 --2- 172.21.15.2:0/2122526773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 0x55ba97d76d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:11.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f1673fff700 1 --2- 172.21.15.2:0/2122526773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97cf3ce0 0x55ba97d03eb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f167c076ab0 tx=0x7f166c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34d2c00b1d31dda9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:11.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.663+0000 7f165bfff700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:37:11.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.664+0000 7f165bfff700 1 -- 172.21.15.2:0/2122526773 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c01ea40 con 0x55ba97cf3ce0 2024-05-14T09:37:11.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.664+0000 7f169166bb80 1 -- 172.21.15.2:0/2122526773 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16780b2dd0 con 0x55ba97cf3ce0 2024-05-14T09:37:11.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.664+0000 7f165bfff700 1 -- 172.21.15.2:0/2122526773 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f166c01ebe0 con 0x55ba97cf3ce0 2024-05-14T09:37:11.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.664+0000 7f165bfff700 1 -- 172.21.15.2:0/2122526773 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f166c037050 con 0x55ba97cf3ce0 2024-05-14T09:37:11.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.664+0000 7f169166bb80 10 client.25507 fetch_fsmap learned FSMap version 28 2024-05-14T09:37:11.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.664+0000 7f169166bb80 1 -- 172.21.15.2:0/2122526773 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ba97d785b0 con 0x55ba97cf3ce0 2024-05-14T09:37:11.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.664+0000 7f165bfff700 1 -- 172.21.15.2:0/2122526773 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16780b2dd0 con 0x55ba97cf3ce0 2024-05-14T09:37:11.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.664+0000 7f165bfff700 1 -- 172.21.15.2:0/2122526773 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f166c0283e0 con 0x55ba97cf3ce0 2024-05-14T09:37:11.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.664+0000 7f169166bb80 10 client.25507 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T09:37:11.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.664+0000 7f169166bb80 10 client.25507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:11.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.664+0000 7f169166bb80 20 client.25507 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:11.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.664+0000 7f169166bb80 1 --2- 172.21.15.2:0/2122526773 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ba97d0d5b0 0x55ba97d7ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:11.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.664+0000 7f169166bb80 4 client.25507 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T09:37:11.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.664+0000 7f169166bb80 1 -- 172.21.15.2:0/2122526773 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ba97c1d360 con 0x55ba97d0d5b0 2024-05-14T09:37:11.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.664+0000 7f16809f3700 1 --2- 172.21.15.2:0/2122526773 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ba97d0d5b0 0x55ba97d7ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:11.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.665+0000 7f165bfff700 10 client.25507 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:11.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.665+0000 7f16809f3700 1 --2- 172.21.15.2:0/2122526773 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ba97d0d5b0 0x55ba97d7ad00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b7d5a541776be5f server_cookie=e17698e8dc8becc in_seq=0 out_seq=0 2024-05-14T09:37:11.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.666+0000 7f165bfff700 1 -- 172.21.15.2:0/2122526773 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ba97c1d360 con 0x55ba97d0d5b0 2024-05-14T09:37:11.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.666+0000 7f165bfff700 10 client.25507 handle_command_reply: tid=0 2024-05-14T09:37:11.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.666+0000 7f169166bb80 1 client.25507 shutdown 2024-05-14T09:37:11.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.666+0000 7f169166bb80 1 -- 172.21.15.2:0/2122526773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97cf3ce0 msgr2=0x55ba97d03eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:11.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.666+0000 7f169166bb80 1 --2- 172.21.15.2:0/2122526773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97cf3ce0 0x55ba97d03eb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f167c076ab0 tx=0x7f166c0047c0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.666+0000 7f169166bb80 1 -- 172.21.15.2:0/2122526773 shutdown_connections 2024-05-14T09:37:11.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.666+0000 7f169166bb80 1 --2- 172.21.15.2:0/2122526773 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ba97d0d5b0 0x55ba97d7ad00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.666+0000 7f169166bb80 1 --2- 172.21.15.2:0/2122526773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ba97b75ec0 0x55ba97d03970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.666+0000 7f169166bb80 1 --2- 172.21.15.2:0/2122526773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba97cf84f0 0x55ba97d76d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.666+0000 7f169166bb80 1 --2- 172.21.15.2:0/2122526773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ba97cf3ce0 0x55ba97d03eb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.666+0000 7f165bfff700 0 client.25507 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:11.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.666+0000 7f169166bb80 1 -- 172.21.15.2:0/2122526773 >> 172.21.15.2:0/2122526773 conn(0x55ba97bbd680 msgr2=0x55ba97cff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:11.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.666+0000 7f169166bb80 1 -- 172.21.15.2:0/2122526773 shutdown_connections 2024-05-14T09:37:11.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.666+0000 7f169166bb80 1 -- 172.21.15.2:0/2122526773 wait complete. 2024-05-14T09:37:11.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.667+0000 7f169166bb80 20 client.25507 trim_cache size 0 max 16384 2024-05-14T09:37:11.671 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:37:11.671 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-05-14T09:37:11.671 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "2fb7ad7a-d3c3-4509-8fe3-03072ae1d9ad", 2024-05-14T09:37:11.671 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-05-14T09:37:11.671 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:37:11.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.667+0000 7f1682c57700 1 -- 172.21.15.2:0/1899500275 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f16640b2880 msgr2=0x7f16640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:11.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.667+0000 7f1682c57700 1 --2- 172.21.15.2:0/1899500275 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f16640b2880 0x7f16640b4d90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f167c106c00 tx=0x7f166c009260 comp rx=0 tx=0).stop 2024-05-14T09:37:11.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.667+0000 7f1682c57700 1 -- 172.21.15.2:0/1899500275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f167c1034c0 msgr2=0x7f167c19ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:11.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.667+0000 7f1682c57700 1 --2- 172.21.15.2:0/1899500275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f167c1034c0 0x7f167c19ae10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1678006010 tx=0x7f1678004270 comp rx=0 tx=0).stop 2024-05-14T09:37:11.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.667+0000 7f1682c57700 1 -- 172.21.15.2:0/1899500275 shutdown_connections 2024-05-14T09:37:11.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.667+0000 7f1682c57700 1 --2- 172.21.15.2:0/1899500275 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f16640b2880 0x7f16640b4d90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.667+0000 7f1682c57700 1 --2- 172.21.15.2:0/1899500275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c108740 0x7f167c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.667+0000 7f1682c57700 1 --2- 172.21.15.2:0/1899500275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f167c1034c0 0x7f167c19ae10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.667+0000 7f1682c57700 1 --2- 172.21.15.2:0/1899500275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f167c107df0 0x7f167c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:11.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.667+0000 7f1682c57700 1 -- 172.21.15.2:0/1899500275 >> 172.21.15.2:0/1899500275 conn(0x7f167c0fedc0 msgr2=0x7f167c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:11.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.668+0000 7f1682c57700 1 -- 172.21.15.2:0/1899500275 shutdown_connections 2024-05-14T09:37:11.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:11.668+0000 7f1682c57700 1 -- 172.21.15.2:0/1899500275 wait complete. 2024-05-14T09:37:11.952 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.677+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022c [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04 auth v1664 ap=1 s=260096 n(v0 rc2024-05-14T09:36:54.337056+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@62},l=25001 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ecea94c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022c:[<0x10000000229/COURSES.X04 v6>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x1000000022c:[<0x10000000229/COURSES.X04 v1664>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.953 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.678+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000231 [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0 auth v2789 ap=1 s=260096 n(v0 rc2024-05-14T09:36:34.463498+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=25001 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ecea94c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000231:[<0x10000000229/STUDENTS.YG0 v16>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x10000000231:[<0x10000000229/STUDENTS.YG0 v2789>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.953 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.678+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000237 [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX auth v2801 ap=1 s=260096 n(v0 rc2024-05-14T09:36:34.491498+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=25001 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ecea94cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000237:[<0x10000000229/ENROLL.PX v28>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x10000000237:[<0x10000000229/ENROLL.PX v2801>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.953 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.680+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022b [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX auth v1632 ap=1 s=260096 n(v0 rc2024-05-14T09:36:54.337056+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@62},l=25001 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ecea8da680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022b:[<0x10000000229/COURSES.PX v4>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x1000000022b:[<0x10000000229/COURSES.PX v1632>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.953 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.680+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022f [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX auth v1652 ap=1 s=260096 n(v0 rc2024-05-14T09:36:54.336056+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@62},l=25001 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ecea94d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022f:[<0x10000000229/STUDENTS.PX v12>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x1000000022f:[<0x10000000229/STUDENTS.PX v1652>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.953 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.681+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023b [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV auth v1630 ap=1 s=260096 n(v0 rc2024-05-14T09:36:34.507497+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@62},l=25001 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ecea94db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023b:[<0x10000000229/STUDENTS.TV v36>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x1000000023b:[<0x10000000229/STUDENTS.TV v1630>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.953 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.681+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023c [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL auth v2875 ap=1 s=260096 n(v0 rc2024-05-14T09:36:34.512497+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=25001 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ecea94e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023c:[<0x10000000229/STUDENTS.FSL v38>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x1000000023c:[<0x10000000229/STUDENTS.FSL v2875>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.953 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.682+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023d [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL auth v1644 ap=1 s=260096 n(v0 rc2024-05-14T09:36:34.516497+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@62},l=25001 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ecea94f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023d:[<0x10000000229/COURSES.FSL v40>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x1000000023d:[<0x10000000229/COURSES.FSL v1644>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.953 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.682+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000239 [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03 auth v2805 ap=1 s=260096 n(v0 rc2024-05-14T09:36:34.499497+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=25001 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ecea94ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000239:[<0x10000000229/ENROLL.Y03 v32>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x10000000239:[<0x10000000229/ENROLL.Y03 v2805>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.953 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.682+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000233 [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1 auth v2793 ap=1 s=260096 n(v0 rc2024-05-14T09:36:34.471498+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=25001 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ecea94e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000233:[<0x10000000229/STUDENTS.YG1 v20>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x10000000233:[<0x10000000229/STUDENTS.YG1 v2793>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.953 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.682+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000234 [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL auth v2795 ap=1 s=260096 n(v0 rc2024-05-14T09:36:34.476498+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=25001 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ecea94f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000234:[<0x10000000229/STUDENTS.VAL v22>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x10000000234:[<0x10000000229/STUDENTS.VAL v2795>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.953 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.683+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022d [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04 auth v1640 ap=1 s=260096 n(v0 rc2024-05-14T09:36:54.337056+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@62},l=25001 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ecea95a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022d:[<0x10000000229/COURSES.Y04 v8>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x1000000022d:[<0x10000000229/COURSES.Y04 v1640>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.954 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.687+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000235 [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB auth v2797 ap=1 s=260096 n(v0 rc2024-05-14T09:36:34.482498+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=25001 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ecea95b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000235:[<0x10000000229/STUDENTS.MB v24>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x10000000235:[<0x10000000229/STUDENTS.MB v2797>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.954 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.687+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023a [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL auth v2807 ap=1 s=260096 n(v0 rc2024-05-14T09:36:34.503497+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=25001 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ecea95b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023a:[<0x10000000229/ENROLL.VAL v34>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x1000000023a:[<0x10000000229/ENROLL.VAL v2807>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.954 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.687+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022a [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB auth v1656 ap=1 s=260096 n(v0 rc2024-05-14T09:36:54.337056+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@62},l=25001 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ecea95a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022a:[<0x10000000229/COURSES.DB v2>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x1000000022a:[<0x10000000229/COURSES.DB v1656>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.954 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.687+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000236 [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB auth v2799 ap=1 s=260096 n(v0 rc2024-05-14T09:36:34.487498+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=25001 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ecea95bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000236:[<0x10000000229/ENROLL.DB v26>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x10000000236:[<0x10000000229/ENROLL.DB v2799>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.954 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.688+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000238 [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03 auth v2803 ap=1 s=260096 n(v0 rc2024-05-14T09:36:34.495498+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=25001 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ecea95ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000238:[<0x10000000229/ENROLL.X03 v30>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x10000000238:[<0x10000000229/ENROLL.X03 v2803>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.954 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.688+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000232 [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1 auth v2791 ap=1 s=260096 n(v0 rc2024-05-14T09:36:34.467498+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=25001 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ecea95c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000232:[<0x10000000229/STUDENTS.XG1 v18>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x10000000232:[<0x10000000229/STUDENTS.XG1 v2791>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.954 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.688+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022e [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB auth v1654 ap=1 s=260096 n(v0 rc2024-05-14T09:36:54.336056+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@62},l=25001 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ecea95d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022e:[<0x10000000229/STUDENTS.DB v10>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x1000000022e:[<0x10000000229/STUDENTS.DB v1654>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:11.954 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 09:37:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T09:37:11.688+0000 7f65977b6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000230 [2,head] /volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0 auth v2787 ap=1 s=260096 n(v0 rc2024-05-14T09:36:34.459498+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25001=0-4194304@1} caps={25001=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=25001 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ecea95cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000230:[<0x10000000229/STUDENTS.XG0 v14>,<0x10000000009/PARADOX v7670>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v47>,<0x10000000006/clients v38>,<0x10000000005/tmp v38>,<0x100000001fb/client.0 v36>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v53>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v59>,<0x1/volumes v56>]//[3]","memoryvalue":"(4)0x10000000230:[<0x10000000229/STUDENTS.XG0 v2787>,<0x10000000009/PARADOX v17312>,<0x10000000008/~dmtmp v108>,<0x10000000007/client0 v85>,<0x10000000006/clients v76>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08 v91>,<0x10000000001/sv_1 v99>,<0x10000000000/qa v27>,<0x1/volumes v90>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T09:37:12.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 85406 12.62 MB/sec warmup 38 sec latency 23.679 ms 2024-05-14T09:37:12.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:12 smithi002 ceph-mon[31557]: pgmap v491: 129 pgs: 129 active+clean; 29 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 969 KiB/s wr, 261 op/s 2024-05-14T09:37:12.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:12 smithi002 ceph-mon[31557]: scrub queued for path: / 2024-05-14T09:37:12.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:12 smithi002 ceph-mon[31557]: scrub summary: idle+waiting paths [/] 2024-05-14T09:37:12.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:12 smithi002 ceph-mon[31557]: scrub summary: active paths [/] 2024-05-14T09:37:12.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:12 smithi002 ceph-mon[31557]: scrub summary: active paths [/] 2024-05-14T09:37:12.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:12 smithi002 ceph-mon[31557]: scrub summary: idle+waiting paths [/] 2024-05-14T09:37:12.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:12 smithi002 ceph-mon[31557]: scrub summary: idle+waiting paths [/] 2024-05-14T09:37:12.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:12 smithi029 ceph-mon[36249]: pgmap v491: 129 pgs: 129 active+clean; 29 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 969 KiB/s wr, 261 op/s 2024-05-14T09:37:12.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:12 smithi029 ceph-mon[36249]: scrub queued for path: / 2024-05-14T09:37:12.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:12 smithi029 ceph-mon[36249]: scrub summary: idle+waiting paths [/] 2024-05-14T09:37:12.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:12 smithi029 ceph-mon[36249]: scrub summary: active paths [/] 2024-05-14T09:37:12.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:12 smithi029 ceph-mon[36249]: scrub summary: active paths [/] 2024-05-14T09:37:12.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:12 smithi029 ceph-mon[36249]: scrub summary: idle+waiting paths [/] 2024-05-14T09:37:12.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:12 smithi029 ceph-mon[36249]: scrub summary: idle+waiting paths [/] 2024-05-14T09:37:12.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:12 smithi026 ceph-mon[36056]: pgmap v491: 129 pgs: 129 active+clean; 29 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 969 KiB/s wr, 261 op/s 2024-05-14T09:37:12.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:12 smithi026 ceph-mon[36056]: scrub queued for path: / 2024-05-14T09:37:12.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:12 smithi026 ceph-mon[36056]: scrub summary: idle+waiting paths [/] 2024-05-14T09:37:12.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:12 smithi026 ceph-mon[36056]: scrub summary: active paths [/] 2024-05-14T09:37:12.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:12 smithi026 ceph-mon[36056]: scrub summary: active paths [/] 2024-05-14T09:37:12.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:12 smithi026 ceph-mon[36056]: scrub summary: idle+waiting paths [/] 2024-05-14T09:37:12.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:12 smithi026 ceph-mon[36056]: scrub summary: idle+waiting paths [/] 2024-05-14T09:37:12.800 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T09:37:13.072 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:37:13.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 86880 12.44 MB/sec warmup 39 sec latency 17.890 ms 2024-05-14T09:37:13.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: scrub summary: active paths [/] 2024-05-14T09:37:13.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x1000000022c (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x10000000231 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x10000000237 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x1000000022b (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x1000000022f (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x1000000023b (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x1000000023c (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x1000000023d (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x10000000239 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x10000000233 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x10000000234 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x1000000022d (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x10000000235 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x1000000023a (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x1000000022a (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x10000000236 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x10000000238 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x10000000232 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x1000000022e (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: Scrub error on inode 0x10000000230 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:13 smithi002 ceph-mon[31557]: scrub summary: idle+waiting paths [/] 2024-05-14T09:37:13.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: scrub summary: active paths [/] 2024-05-14T09:37:13.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x1000000022c (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x10000000231 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x10000000237 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x1000000022b (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x1000000022f (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x1000000023b (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x1000000023c (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x1000000023d (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x10000000239 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x10000000233 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x10000000234 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x1000000022d (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x10000000235 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x1000000023a (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x1000000022a (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x10000000236 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x10000000238 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x10000000232 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x1000000022e (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.689 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: Scrub error on inode 0x10000000230 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.689 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:13 smithi029 ceph-mon[36249]: scrub summary: idle+waiting paths [/] 2024-05-14T09:37:13.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: scrub summary: active paths [/] 2024-05-14T09:37:13.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x1000000022c (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x10000000231 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x10000000237 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x1000000022b (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x1000000022f (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x1000000023b (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x1000000023c (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x1000000023d (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x10000000239 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x10000000233 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x10000000234 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x1000000022d (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x10000000235 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.703 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x1000000023a (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.703 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x1000000022a (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.703 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x10000000236 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.703 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x10000000238 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.703 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x10000000232 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.703 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x1000000022e (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.703 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: Scrub error on inode 0x10000000230 (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-05-14T09:37:13.703 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:13 smithi026 ceph-mon[36056]: scrub summary: idle+waiting paths [/] 2024-05-14T09:37:14.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 89145 12.47 MB/sec warmup 40 sec latency 10.048 ms 2024-05-14T09:37:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.278+0000 7fb27f83b640 1 -- 172.21.15.2:0/1690555486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb278108a90 msgr2=0x7fb27810aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.278+0000 7fb27f83b640 1 --2- 172.21.15.2:0/1690555486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb278108a90 0x7fb27810aeb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb270009ca0 tx=0x7fb27002a8c0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.278+0000 7fb27f83b640 1 -- 172.21.15.2:0/1690555486 shutdown_connections 2024-05-14T09:37:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.278+0000 7fb27f83b640 1 --2- 172.21.15.2:0/1690555486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb278108a90 0x7fb27810aeb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.278+0000 7fb27f83b640 1 --2- 172.21.15.2:0/1690555486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb278103d90 0x7fb2781041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.278+0000 7fb27f83b640 1 --2- 172.21.15.2:0/1690555486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb278102b50 0x7fb278103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.278+0000 7fb27f83b640 1 -- 172.21.15.2:0/1690555486 >> 172.21.15.2:0/1690555486 conn(0x7fb2780fdd40 msgr2=0x7fb2781001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.278+0000 7fb27f83b640 1 -- 172.21.15.2:0/1690555486 shutdown_connections 2024-05-14T09:37:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.278+0000 7fb27f83b640 1 -- 172.21.15.2:0/1690555486 wait complete. 2024-05-14T09:37:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.279+0000 7fb27f83b640 1 Processor -- start 2024-05-14T09:37:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.279+0000 7fb27f83b640 1 -- start start 2024-05-14T09:37:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.279+0000 7fb27f83b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb278102b50 0x7fb2780ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.279+0000 7fb27f83b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb278103d90 0x7fb2780fe270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.279+0000 7fb27f83b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb278108a90 0x7fb2780fe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.279+0000 7fb27f83b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb27810e680 con 0x7fb278102b50 2024-05-14T09:37:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.279+0000 7fb27f83b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb27810e500 con 0x7fb278108a90 2024-05-14T09:37:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.279+0000 7fb27f83b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb27810e800 con 0x7fb278103d90 2024-05-14T09:37:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.279+0000 7fb27d5d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb278102b50 0x7fb2780ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.279+0000 7fb27d5d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb278102b50 0x7fb2780ffbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41762/0 (socket says 172.21.15.2:41762) 2024-05-14T09:37:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.279+0000 7fb27d5d7640 1 -- 172.21.15.2:0/3002984721 learned_addr learned my addr 172.21.15.2:0/3002984721 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.279+0000 7fb27cdd6640 1 --2- 172.21.15.2:0/3002984721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb278103d90 0x7fb2780fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.279+0000 7fb27ddd8640 1 --2- 172.21.15.2:0/3002984721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb278108a90 0x7fb2780fe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.280+0000 7fb27d5d7640 1 -- 172.21.15.2:0/3002984721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb278103d90 msgr2=0x7fb2780fe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.280+0000 7fb27d5d7640 1 --2- 172.21.15.2:0/3002984721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb278103d90 0x7fb2780fe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.280+0000 7fb27d5d7640 1 -- 172.21.15.2:0/3002984721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb278108a90 msgr2=0x7fb2780fe7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.280+0000 7fb27d5d7640 1 --2- 172.21.15.2:0/3002984721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb278108a90 0x7fb2780fe7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.280+0000 7fb27d5d7640 1 -- 172.21.15.2:0/3002984721 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2780feee0 con 0x7fb278102b50 2024-05-14T09:37:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.280+0000 7fb27d5d7640 1 --2- 172.21.15.2:0/3002984721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb278102b50 0x7fb2780ffbd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb27400e9f0 tx=0x7fb27400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.282+0000 7fb26e7fc640 1 -- 172.21.15.2:0/3002984721 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27400ccc0 con 0x7fb278102b50 2024-05-14T09:37:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.282+0000 7fb26e7fc640 1 -- 172.21.15.2:0/3002984721 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2740035f0 con 0x7fb278102b50 2024-05-14T09:37:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.282+0000 7fb27f83b640 1 -- 172.21.15.2:0/3002984721 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2781a43b0 con 0x7fb278102b50 2024-05-14T09:37:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.282+0000 7fb26e7fc640 1 -- 172.21.15.2:0/3002984721 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27400ce60 con 0x7fb278102b50 2024-05-14T09:37:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.282+0000 7fb27f83b640 1 -- 172.21.15.2:0/3002984721 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2781a48a0 con 0x7fb278102b50 2024-05-14T09:37:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb2867ca740 1 Processor -- start 2024-05-14T09:37:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb26e7fc640 1 -- 172.21.15.2:0/3002984721 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fb274010690 con 0x7fb278102b50 2024-05-14T09:37:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb2867ca740 1 -- start start 2024-05-14T09:37:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb2867ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c261b0 0x563088fc7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb2867ca740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088bfd2f0 0x563088fc7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb2867ca740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088c054d0 0x563088fceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb2867ca740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563088f2c5d0 con 0x563088bfd2f0 2024-05-14T09:37:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb2867ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563088ee5890 con 0x563088c261b0 2024-05-14T09:37:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb2867ca740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563088f28590 con 0x563088c054d0 2024-05-14T09:37:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb27d5d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088bfd2f0 0x563088fc7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb27ddd8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c261b0 0x563088fc7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb27d5d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088bfd2f0 0x563088fc7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41770/0 (socket says 172.21.15.2:41770) 2024-05-14T09:37:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb27d5d7640 1 -- 172.21.15.2:0/755067047 learned_addr learned my addr 172.21.15.2:0/755067047 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb27cdd6640 1 --2- 172.21.15.2:0/755067047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088c054d0 0x563088fceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb27ddd8640 1 -- 172.21.15.2:0/755067047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088c054d0 msgr2=0x563088fceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb27ddd8640 1 --2- 172.21.15.2:0/755067047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088c054d0 0x563088fceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb27ddd8640 1 -- 172.21.15.2:0/755067047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088bfd2f0 msgr2=0x563088fc7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb27ddd8640 1 --2- 172.21.15.2:0/755067047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088bfd2f0 0x563088fc7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.285+0000 7fb27ddd8640 1 -- 172.21.15.2:0/755067047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563088c3e210 con 0x563088c261b0 2024-05-14T09:37:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.286+0000 7fb26e7fc640 1 --2- 172.21.15.2:0/3002984721 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2640b2920 0x7fb2640b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.286+0000 7fb27ddd8640 1 --2- 172.21.15.2:0/755067047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c261b0 0x563088fc7a60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb270031b50 tx=0x7fb27002f410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cd30de83522190e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.286+0000 7fb27d5d7640 1 --2- 172.21.15.2:0/3002984721 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2640b2920 0x7fb2640b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.286+0000 7fb2637fe640 1 -- 172.21.15.2:0/755067047 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270031650 con 0x563088c261b0 2024-05-14T09:37:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.286+0000 7fb2637fe640 1 -- 172.21.15.2:0/755067047 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb270036030 con 0x563088c261b0 2024-05-14T09:37:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.286+0000 7fb2637fe640 1 -- 172.21.15.2:0/755067047 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2700046d0 con 0x563088c261b0 2024-05-14T09:37:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.286+0000 7fb26e7fc640 1 -- 172.21.15.2:0/3002984721 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fb2740e28d0 con 0x7fb278102b50 2024-05-14T09:37:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.287+0000 7fb2867ca740 1 -- 172.21.15.2:0/755067047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c261b0 msgr2=0x563088fc7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.287+0000 7fb2867ca740 1 --2- 172.21.15.2:0/755067047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c261b0 0x563088fc7a60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb270031b50 tx=0x7fb27002f410 comp rx=0 tx=0).stop 2024-05-14T09:37:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.287+0000 7fb27d5d7640 1 --2- 172.21.15.2:0/3002984721 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2640b2920 0x7fb2640b4e30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb27400c6a0 tx=0x7fb2740e8550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.287+0000 7fb2867ca740 1 -- 172.21.15.2:0/755067047 shutdown_connections 2024-05-14T09:37:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.287+0000 7fb2867ca740 1 --2- 172.21.15.2:0/755067047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088c054d0 0x563088fceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.287+0000 7fb2867ca740 1 --2- 172.21.15.2:0/755067047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088bfd2f0 0x563088fc7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.287+0000 7fb2867ca740 1 --2- 172.21.15.2:0/755067047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c261b0 0x563088fc7a60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.287+0000 7fb2867ca740 1 -- 172.21.15.2:0/755067047 >> 172.21.15.2:0/755067047 conn(0x563088c1a300 msgr2=0x563088fc33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.287+0000 7fb2867ca740 1 -- 172.21.15.2:0/755067047 shutdown_connections 2024-05-14T09:37:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.287+0000 7fb2867ca740 1 -- 172.21.15.2:0/755067047 wait complete. 2024-05-14T09:37:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.287+0000 7fb2867ca740 1 -- start start 2024-05-14T09:37:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb2867ca740 1 Processor -- start 2024-05-14T09:37:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb2867ca740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088bfd2f0 0x563088fd14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb2867ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c054d0 0x563088fd1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb2867ca740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088c261b0 0x563088fd8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb2867ca740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563088fcf750 con 0x563088c261b0 2024-05-14T09:37:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb2867ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563088ee9370 con 0x563088c054d0 2024-05-14T09:37:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb2867ca740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563088fcf8d0 con 0x563088bfd2f0 2024-05-14T09:37:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb2867ca740 10 client.0 _ll_register_callbacks cb 0x563088c3da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb27cdd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c054d0 0x563088fd1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb27ddd8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088bfd2f0 0x563088fd14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb27cdd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c054d0 0x563088fd1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:49178/0 (socket says 172.21.15.2:49178) 2024-05-14T09:37:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb27cdd6640 1 -- 172.21.15.2:0/3327011992 learned_addr learned my addr 172.21.15.2:0/3327011992 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb27ddd8640 1 --2- 172.21.15.2:0/3327011992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088c261b0 0x563088fd8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb27cdd6640 1 -- 172.21.15.2:0/3327011992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088bfd2f0 msgr2=0x563088fd14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb27cdd6640 1 --2- 172.21.15.2:0/3327011992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088bfd2f0 0x563088fd14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb27cdd6640 1 -- 172.21.15.2:0/3327011992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088c261b0 msgr2=0x563088fd8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb27cdd6640 1 --2- 172.21.15.2:0/3327011992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088c261b0 0x563088fd8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.288+0000 7fb27cdd6640 1 -- 172.21.15.2:0/3327011992 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563088d0a620 con 0x563088c054d0 2024-05-14T09:37:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.289+0000 7fb27cdd6640 1 --2- 172.21.15.2:0/3327011992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c054d0 0x563088fd1a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb268012700 tx=0x7fb268012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26b67f3c93dd67ff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.289+0000 7fb260ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:37:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.289+0000 7fb260ff9640 1 -- 172.21.15.2:0/3327011992 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26800f8c0 con 0x563088c054d0 2024-05-14T09:37:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.289+0000 7fb2867ca740 1 -- 172.21.15.2:0/3327011992 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb270036030 con 0x563088c054d0 2024-05-14T09:37:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.289+0000 7fb260ff9640 1 -- 172.21.15.2:0/3327011992 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fb26800fa60 con 0x563088c054d0 2024-05-14T09:37:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.289+0000 7fb260ff9640 1 -- 172.21.15.2:0/3327011992 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb26800fca0 con 0x563088c054d0 2024-05-14T09:37:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.289+0000 7fb2867ca740 10 client.25517 fetch_fsmap learned FSMap version 28 2024-05-14T09:37:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.289+0000 7fb2867ca740 1 -- 172.21.15.2:0/3327011992 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563088fdb9c0 con 0x563088c054d0 2024-05-14T09:37:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.289+0000 7fb260ff9640 1 -- 172.21.15.2:0/3327011992 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb270036030 con 0x563088c054d0 2024-05-14T09:37:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.289+0000 7fb260ff9640 1 -- 172.21.15.2:0/3327011992 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb268010430 con 0x563088c054d0 2024-05-14T09:37:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.290+0000 7fb2867ca740 10 client.25517 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T09:37:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.290+0000 7fb2867ca740 10 client.25517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.290+0000 7fb2867ca740 20 client.25517 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.290+0000 7fb2867ca740 1 --2- 172.21.15.2:0/3327011992 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563088be4d80 0x563088fdf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.290+0000 7fb2867ca740 4 client.25517 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:37:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.290+0000 7fb2867ca740 1 -- 172.21.15.2:0/3327011992 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563088f3a1e0 con 0x563088be4d80 2024-05-14T09:37:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.290+0000 7fb27ddd8640 1 --2- 172.21.15.2:0/3327011992 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563088be4d80 0x563088fdf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.290+0000 7fb260ff9640 10 client.25517 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.290+0000 7fb27ddd8640 1 --2- 172.21.15.2:0/3327011992 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563088be4d80 0x563088fdf190 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dfebb266568233b server_cookie=ca355e23c29c7f1 in_seq=0 out_seq=0 2024-05-14T09:37:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.291+0000 7fb260ff9640 1 -- 172.21.15.2:0/3327011992 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563088f3a1e0 con 0x563088be4d80 2024-05-14T09:37:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.291+0000 7fb260ff9640 10 client.25517 handle_command_reply: tid=0 2024-05-14T09:37:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.291+0000 7fb2867ca740 1 client.25517 shutdown 2024-05-14T09:37:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.291+0000 7fb2867ca740 1 -- 172.21.15.2:0/3327011992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c054d0 msgr2=0x563088fd1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.291+0000 7fb2867ca740 1 --2- 172.21.15.2:0/3327011992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c054d0 0x563088fd1a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb268012700 tx=0x7fb268012bc0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.291+0000 7fb2867ca740 1 -- 172.21.15.2:0/3327011992 shutdown_connections 2024-05-14T09:37:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.291+0000 7fb2867ca740 1 --2- 172.21.15.2:0/3327011992 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563088be4d80 0x563088fdf190 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.291+0000 7fb2867ca740 1 --2- 172.21.15.2:0/3327011992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088c261b0 0x563088fd8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.291+0000 7fb2867ca740 1 --2- 172.21.15.2:0/3327011992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c054d0 0x563088fd1a10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.291+0000 7fb2867ca740 1 --2- 172.21.15.2:0/3327011992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088bfd2f0 0x563088fd14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.291+0000 7fb2867ca740 1 -- 172.21.15.2:0/3327011992 >> 172.21.15.2:0/3327011992 conn(0x563088c1a300 msgr2=0x563088fc23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.291+0000 7fb260ff9640 0 client.25517 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.291+0000 7fb2867ca740 1 -- 172.21.15.2:0/3327011992 shutdown_connections 2024-05-14T09:37:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.291+0000 7fb2867ca740 1 -- 172.21.15.2:0/3327011992 wait complete. 2024-05-14T09:37:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.291+0000 7fb2867ca740 20 client.25517 trim_cache size 0 max 16384 2024-05-14T09:37:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.308+0000 7fb2867ca740 1 Processor -- start 2024-05-14T09:37:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.308+0000 7fb2867ca740 1 -- start start 2024-05-14T09:37:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.308+0000 7fb2867ca740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088bfd2f0 0x563088fc3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.308+0000 7fb2867ca740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088c054d0 0x563088fd8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.308+0000 7fb2867ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c261b0 0x563088fc61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.308+0000 7fb2867ca740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781b16d0 con 0x563088c054d0 2024-05-14T09:37:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.308+0000 7fb2867ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781aefa0 con 0x563088c261b0 2024-05-14T09:37:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.308+0000 7fb2867ca740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781acdf0 con 0x563088bfd2f0 2024-05-14T09:37:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.308+0000 7fb27cdd6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088c054d0 0x563088fd8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.308+0000 7fb27ddd8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088bfd2f0 0x563088fc3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.308+0000 7fb27ddd8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088bfd2f0 0x563088fc3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:37822/0 (socket says 172.21.15.2:37822) 2024-05-14T09:37:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.308+0000 7fb27cdd6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088c054d0 0x563088fd8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41780/0 (socket says 172.21.15.2:41780) 2024-05-14T09:37:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.308+0000 7fb27ddd8640 1 -- 172.21.15.2:0/2447010418 learned_addr learned my addr 172.21.15.2:0/2447010418 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.308+0000 7fb27ddd8640 1 --2- 172.21.15.2:0/2447010418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c261b0 0x563088fc61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.309+0000 7fb27cdd6640 1 -- 172.21.15.2:0/2447010418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088bfd2f0 msgr2=0x563088fc3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.309+0000 7fb27cdd6640 1 --2- 172.21.15.2:0/2447010418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088bfd2f0 0x563088fc3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.309+0000 7fb27cdd6640 1 -- 172.21.15.2:0/2447010418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c261b0 msgr2=0x563088fc61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.309+0000 7fb27cdd6640 1 --2- 172.21.15.2:0/2447010418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c261b0 0x563088fc61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.309+0000 7fb27cdd6640 1 -- 172.21.15.2:0/2447010418 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563088fc3940 con 0x563088c054d0 2024-05-14T09:37:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.309+0000 7fb27ddd8640 1 --2- 172.21.15.2:0/2447010418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088bfd2f0 0x563088fc3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.309+0000 7fb27cdd6640 1 --2- 172.21.15.2:0/2447010418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088c054d0 0x563088fd8890 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb2680108d0 tx=0x7fb268009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d8c59593c753ff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.309+0000 7fb262ffd640 1 -- 172.21.15.2:0/2447010418 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2680104f0 con 0x563088c054d0 2024-05-14T09:37:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.309+0000 7fb262ffd640 1 -- 172.21.15.2:0/2447010418 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb268010ac0 con 0x563088c054d0 2024-05-14T09:37:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.309+0000 7fb262ffd640 1 -- 172.21.15.2:0/2447010418 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268005520 con 0x563088c054d0 2024-05-14T09:37:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.310+0000 7fb2867ca740 1 -- 172.21.15.2:0/2447010418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088c054d0 msgr2=0x563088fd8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.310+0000 7fb2867ca740 1 --2- 172.21.15.2:0/2447010418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088c054d0 0x563088fd8890 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb2680108d0 tx=0x7fb268009ec0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.310+0000 7fb2867ca740 1 -- 172.21.15.2:0/2447010418 shutdown_connections 2024-05-14T09:37:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.310+0000 7fb2867ca740 1 --2- 172.21.15.2:0/2447010418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c261b0 0x563088fc61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.310+0000 7fb2867ca740 1 --2- 172.21.15.2:0/2447010418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088c054d0 0x563088fd8890 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.310+0000 7fb2867ca740 1 --2- 172.21.15.2:0/2447010418 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088bfd2f0 0x563088fc3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.310+0000 7fb2867ca740 1 -- 172.21.15.2:0/2447010418 >> 172.21.15.2:0/2447010418 conn(0x563088c1a300 msgr2=0x563088fc2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.310+0000 7fb2867ca740 1 -- 172.21.15.2:0/2447010418 shutdown_connections 2024-05-14T09:37:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.310+0000 7fb2867ca740 1 -- 172.21.15.2:0/2447010418 wait complete. 2024-05-14T09:37:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.311+0000 7fb2867ca740 1 -- start start 2024-05-14T09:37:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.311+0000 7fb2867ca740 1 Processor -- start 2024-05-14T09:37:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.311+0000 7fb2867ca740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088bfd2f0 0x563088fd5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.311+0000 7fb2867ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c054d0 0x563088fd6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.311+0000 7fb2867ca740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088c261b0 0x563088fd7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.311+0000 7fb2867ca740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781ad460 con 0x563088bfd2f0 2024-05-14T09:37:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.311+0000 7fb2867ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781ae8d0 con 0x563088c054d0 2024-05-14T09:37:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.311+0000 7fb2867ca740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781adb30 con 0x563088c261b0 2024-05-14T09:37:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.311+0000 7fb2867ca740 10 client.0 _ll_register_callbacks cb 0x7fb268009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.311+0000 7fb27ddd8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088bfd2f0 0x563088fd5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.311+0000 7fb27cdd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c054d0 0x563088fd6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.311+0000 7fb27ddd8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088bfd2f0 0x563088fd5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41784/0 (socket says 172.21.15.2:41784) 2024-05-14T09:37:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.311+0000 7fb27ddd8640 1 -- 172.21.15.2:0/539282476 learned_addr learned my addr 172.21.15.2:0/539282476 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.312+0000 7fb27ddd8640 1 --2- 172.21.15.2:0/539282476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088c261b0 0x563088fd7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.312+0000 7fb27cdd6640 1 -- 172.21.15.2:0/539282476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088c261b0 msgr2=0x563088fd7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.312+0000 7fb27cdd6640 1 --2- 172.21.15.2:0/539282476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088c261b0 0x563088fd7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.312+0000 7fb27cdd6640 1 -- 172.21.15.2:0/539282476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088bfd2f0 msgr2=0x563088fd5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.312+0000 7fb27cdd6640 1 --2- 172.21.15.2:0/539282476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088bfd2f0 0x563088fd5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.312+0000 7fb27cdd6640 1 -- 172.21.15.2:0/539282476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563088fd7650 con 0x563088c054d0 2024-05-14T09:37:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.312+0000 7fb27ddd8640 1 --2- 172.21.15.2:0/539282476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088c261b0 0x563088fd7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:37:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.312+0000 7fb27cdd6640 1 --2- 172.21.15.2:0/539282476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c054d0 0x563088fd6160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb2680126d0 tx=0x7fb268011eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b150560c5d44082f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.312+0000 7fb260ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:37:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.312+0000 7fb260ff9640 1 -- 172.21.15.2:0/539282476 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26800f9a0 con 0x563088c054d0 2024-05-14T09:37:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.312+0000 7fb260ff9640 1 -- 172.21.15.2:0/539282476 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fb268005260 con 0x563088c054d0 2024-05-14T09:37:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.312+0000 7fb2867ca740 1 -- 172.21.15.2:0/539282476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb268010ac0 con 0x563088c054d0 2024-05-14T09:37:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.312+0000 7fb260ff9640 1 -- 172.21.15.2:0/539282476 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb268005480 con 0x563088c054d0 2024-05-14T09:37:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.312+0000 7fb2867ca740 10 client.25523 fetch_fsmap learned FSMap version 28 2024-05-14T09:37:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.312+0000 7fb2867ca740 1 -- 172.21.15.2:0/539282476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563088fd88e0 con 0x563088c054d0 2024-05-14T09:37:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.313+0000 7fb260ff9640 1 -- 172.21.15.2:0/539282476 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb268010ac0 con 0x563088c054d0 2024-05-14T09:37:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.313+0000 7fb260ff9640 1 -- 172.21.15.2:0/539282476 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb268031060 con 0x563088c054d0 2024-05-14T09:37:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.313+0000 7fb2867ca740 10 client.25523 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T09:37:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.313+0000 7fb2867ca740 10 client.25523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.313+0000 7fb2867ca740 20 client.25523 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.313+0000 7fb2867ca740 1 --2- 172.21.15.2:0/539282476 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563088bba1c0 0x563089049970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.313+0000 7fb2867ca740 4 client.25523 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:37:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.313+0000 7fb2867ca740 1 -- 172.21.15.2:0/539282476 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563088f34050 con 0x563088bba1c0 2024-05-14T09:37:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.313+0000 7fb27ddd8640 1 --2- 172.21.15.2:0/539282476 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563088bba1c0 0x563089049970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.313+0000 7fb260ff9640 10 client.25523 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.313+0000 7fb27ddd8640 1 --2- 172.21.15.2:0/539282476 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563088bba1c0 0x563089049970 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a92ae7b3dafff1c server_cookie=929e1df94b0db88f in_seq=0 out_seq=0 2024-05-14T09:37:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.419+0000 7fb260ff9640 1 -- 172.21.15.2:0/539282476 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563088f34050 con 0x563088bba1c0 2024-05-14T09:37:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.419+0000 7fb260ff9640 10 client.25523 handle_command_reply: tid=0 2024-05-14T09:37:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.419+0000 7fb2867ca740 1 client.25523 shutdown 2024-05-14T09:37:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.419+0000 7fb2867ca740 1 -- 172.21.15.2:0/539282476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c054d0 msgr2=0x563088fd6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.419+0000 7fb2867ca740 1 --2- 172.21.15.2:0/539282476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c054d0 0x563088fd6160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb2680126d0 tx=0x7fb268011eb0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.419+0000 7fb2867ca740 1 -- 172.21.15.2:0/539282476 shutdown_connections 2024-05-14T09:37:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.419+0000 7fb2867ca740 1 --2- 172.21.15.2:0/539282476 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563088bba1c0 0x563089049970 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.419+0000 7fb2867ca740 1 --2- 172.21.15.2:0/539282476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563088c261b0 0x563088fd7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.419+0000 7fb2867ca740 1 --2- 172.21.15.2:0/539282476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563088c054d0 0x563088fd6160 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.419+0000 7fb2867ca740 1 --2- 172.21.15.2:0/539282476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563088bfd2f0 0x563088fd5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.419+0000 7fb260ff9640 0 client.25523 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.419+0000 7fb2867ca740 1 -- 172.21.15.2:0/539282476 >> 172.21.15.2:0/539282476 conn(0x563088c1a300 msgr2=0x563088fc1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.419+0000 7fb2867ca740 1 -- 172.21.15.2:0/539282476 shutdown_connections 2024-05-14T09:37:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.419+0000 7fb2867ca740 1 -- 172.21.15.2:0/539282476 wait complete. 2024-05-14T09:37:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.419+0000 7fb2867ca740 20 client.25523 trim_cache size 0 max 16384 2024-05-14T09:37:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:37:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:37:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:37:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:37:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.421+0000 7fb27f83b640 1 -- 172.21.15.2:0/3002984721 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2640b2920 msgr2=0x7fb2640b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.421+0000 7fb27f83b640 1 --2- 172.21.15.2:0/3002984721 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2640b2920 0x7fb2640b4e30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb27400c6a0 tx=0x7fb2740e8550 comp rx=0 tx=0).stop 2024-05-14T09:37:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.421+0000 7fb27f83b640 1 -- 172.21.15.2:0/3002984721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb278102b50 msgr2=0x7fb2780ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.421+0000 7fb27f83b640 1 --2- 172.21.15.2:0/3002984721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb278102b50 0x7fb2780ffbd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb27400e9f0 tx=0x7fb27400eeb0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.421+0000 7fb27f83b640 1 -- 172.21.15.2:0/3002984721 shutdown_connections 2024-05-14T09:37:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.421+0000 7fb27f83b640 1 --2- 172.21.15.2:0/3002984721 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2640b2920 0x7fb2640b4e30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.421+0000 7fb27f83b640 1 --2- 172.21.15.2:0/3002984721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb278108a90 0x7fb2780fe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.421+0000 7fb27f83b640 1 --2- 172.21.15.2:0/3002984721 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb278103d90 0x7fb2780fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.421+0000 7fb27f83b640 1 --2- 172.21.15.2:0/3002984721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb278102b50 0x7fb2780ffbd0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.421+0000 7fb27f83b640 1 -- 172.21.15.2:0/3002984721 >> 172.21.15.2:0/3002984721 conn(0x7fb2780fdd40 msgr2=0x7fb278109ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.422+0000 7fb27f83b640 1 -- 172.21.15.2:0/3002984721 shutdown_connections 2024-05-14T09:37:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:14.422+0000 7fb27f83b640 1 -- 172.21.15.2:0/3002984721 wait complete. 2024-05-14T09:37:14.512 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:14 smithi002 ceph-mon[31557]: pgmap v492: 129 pgs: 129 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 271 op/s 2024-05-14T09:37:14.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:14 smithi029 ceph-mon[36249]: pgmap v492: 129 pgs: 129 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 271 op/s 2024-05-14T09:37:14.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:14 smithi026 ceph-mon[36056]: pgmap v492: 129 pgs: 129 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 271 op/s 2024-05-14T09:37:14.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.888+0000 7f62661fc700 1 -- 172.21.15.2:0/986941581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6260103ba0 msgr2=0x7f62601080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.888+0000 7f62661fc700 1 --2- 172.21.15.2:0/986941581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6260103ba0 0x7f62601080d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6254009990 tx=0x7f6254009ca0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.889+0000 7f62661fc700 1 -- 172.21.15.2:0/986941581 shutdown_connections 2024-05-14T09:37:14.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.889+0000 7f62661fc700 1 --2- 172.21.15.2:0/986941581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6260103250 0x7f6260103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.889+0000 7f62661fc700 1 --2- 172.21.15.2:0/986941581 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f62601086a0 0x7f6260108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.889+0000 7f62661fc700 1 --2- 172.21.15.2:0/986941581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6260103ba0 0x7f62601080d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.889+0000 7f62661fc700 1 -- 172.21.15.2:0/986941581 >> 172.21.15.2:0/986941581 conn(0x7f62600fedc0 msgr2=0x7f62601011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:14.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.890+0000 7f62661fc700 1 -- 172.21.15.2:0/986941581 shutdown_connections 2024-05-14T09:37:14.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.890+0000 7f62661fc700 1 -- 172.21.15.2:0/986941581 wait complete. 2024-05-14T09:37:14.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.891+0000 7f62661fc700 1 Processor -- start 2024-05-14T09:37:14.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.892+0000 7f62661fc700 1 -- start start 2024-05-14T09:37:14.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.892+0000 7f62661fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6260103250 0x7f62601a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.892+0000 7f62661fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6260103ba0 0x7f62601a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.892+0000 7f62661fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62601086a0 0x7f62601a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.892+0000 7f62661fc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f626011aa90 con 0x7f62601086a0 2024-05-14T09:37:14.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.892+0000 7f62661fc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f626011a910 con 0x7f6260103250 2024-05-14T09:37:14.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.892+0000 7f62661fc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f626011ac10 con 0x7f6260103ba0 2024-05-14T09:37:14.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.892+0000 7f625ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62601086a0 0x7f62601a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.892+0000 7f625ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62601086a0 0x7f62601a5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41800/0 (socket says 172.21.15.2:41800) 2024-05-14T09:37:14.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.892+0000 7f625ffff700 1 -- 172.21.15.2:0/1644002994 learned_addr learned my addr 172.21.15.2:0/1644002994 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:14.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.892+0000 7f625effd700 1 --2- 172.21.15.2:0/1644002994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6260103ba0 0x7f62601a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.893+0000 7f625f7fe700 1 --2- 172.21.15.2:0/1644002994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6260103250 0x7f62601a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.893+0000 7f625ffff700 1 -- 172.21.15.2:0/1644002994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6260103ba0 msgr2=0x7f62601a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.893+0000 7f625ffff700 1 --2- 172.21.15.2:0/1644002994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6260103ba0 0x7f62601a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.893+0000 7f625ffff700 1 -- 172.21.15.2:0/1644002994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6260103250 msgr2=0x7f62601a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.893+0000 7f625ffff700 1 --2- 172.21.15.2:0/1644002994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6260103250 0x7f62601a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.893+0000 7f625ffff700 1 -- 172.21.15.2:0/1644002994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62601a62b0 con 0x7f62601086a0 2024-05-14T09:37:14.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.893+0000 7f625effd700 1 --2- 172.21.15.2:0/1644002994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6260103ba0 0x7f62601a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:14.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.893+0000 7f625ffff700 1 --2- 172.21.15.2:0/1644002994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62601086a0 0x7f62601a5c10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f625800ef30 tx=0x7f625800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:14.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.894+0000 7f625cff9700 1 -- 172.21.15.2:0/1644002994 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62580092e0 con 0x7f62601086a0 2024-05-14T09:37:14.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.894+0000 7f62661fc700 1 -- 172.21.15.2:0/1644002994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62601a6540 con 0x7f62601086a0 2024-05-14T09:37:14.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.894+0000 7f62661fc700 1 -- 172.21.15.2:0/1644002994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62601adec0 con 0x7f62601086a0 2024-05-14T09:37:14.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.894+0000 7f625cff9700 1 -- 172.21.15.2:0/1644002994 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6258004bb0 con 0x7f62601086a0 2024-05-14T09:37:14.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.894+0000 7f625cff9700 1 -- 172.21.15.2:0/1644002994 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258004020 con 0x7f62601086a0 2024-05-14T09:37:14.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.899+0000 7f625cff9700 1 -- 172.21.15.2:0/1644002994 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f625801b030 con 0x7f62601086a0 2024-05-14T09:37:14.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.901+0000 7f625cff9700 1 --2- 172.21.15.2:0/1644002994 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f62480b2980 0x7f62480b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.901+0000 7f625cff9700 1 -- 172.21.15.2:0/1644002994 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f62580e1ec0 con 0x7f62601086a0 2024-05-14T09:37:14.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.901+0000 7f625f7fe700 1 --2- 172.21.15.2:0/1644002994 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f62480b2980 0x7f62480b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.902+0000 7f6274c10b80 1 Processor -- start 2024-05-14T09:37:14.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.902+0000 7f6274c10b80 1 -- start start 2024-05-14T09:37:14.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.902+0000 7f625f7fe700 1 --2- 172.21.15.2:0/1644002994 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f62480b2980 0x7f62480b4e90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6250009c60 tx=0x7f6250009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:14.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.902+0000 7f6274c10b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b6616bde40 0x55b661841840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.902+0000 7f6274c10b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 0x55b66183b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.902+0000 7f6274c10b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b66183be10 0x55b66183c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.902+0000 7f6274c10b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b66175d940 con 0x55b6616bde40 2024-05-14T09:37:14.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.902+0000 7f6274c10b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b6616c2da0 con 0x55b66183be10 2024-05-14T09:37:14.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.902+0000 7f6274c10b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b661759900 con 0x55b661841d80 2024-05-14T09:37:14.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.902+0000 7f625f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b6616bde40 0x55b661841840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.902+0000 7f625ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b66183be10 0x55b66183c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.903+0000 7f625effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 0x55b66183b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.903+0000 7f625f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b6616bde40 0x55b661841840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41812/0 (socket says 172.21.15.2:41812) 2024-05-14T09:37:14.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.903+0000 7f625ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b66183be10 0x55b66183c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:49218/0 (socket says 172.21.15.2:49218) 2024-05-14T09:37:14.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.903+0000 7f625f7fe700 1 -- 172.21.15.2:0/821615140 learned_addr learned my addr 172.21.15.2:0/821615140 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:14.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.903+0000 7f625ffff700 1 -- 172.21.15.2:0/821615140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 msgr2=0x55b66183b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.903+0000 7f625ffff700 1 --2- 172.21.15.2:0/821615140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 0x55b66183b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.903+0000 7f625ffff700 1 -- 172.21.15.2:0/821615140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b6616bde40 msgr2=0x55b661841840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.903+0000 7f625ffff700 1 --2- 172.21.15.2:0/821615140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b6616bde40 0x55b661841840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.903+0000 7f625ffff700 1 -- 172.21.15.2:0/821615140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6616aa8d0 con 0x55b66183be10 2024-05-14T09:37:14.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.903+0000 7f625ffff700 1 --2- 172.21.15.2:0/821615140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b66183be10 0x55b66183c220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f625800ef00 tx=0x7f62580a8c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b1b2323b8d5ff0d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:14.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.903+0000 7f6245ffb700 1 -- 172.21.15.2:0/821615140 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625800c940 con 0x55b66183be10 2024-05-14T09:37:14.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.904+0000 7f6245ffb700 1 -- 172.21.15.2:0/821615140 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62580a90f0 con 0x55b66183be10 2024-05-14T09:37:14.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.904+0000 7f6245ffb700 1 -- 172.21.15.2:0/821615140 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62580b1c90 con 0x55b66183be10 2024-05-14T09:37:14.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.905+0000 7f6274c10b80 1 -- 172.21.15.2:0/821615140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b66183be10 msgr2=0x55b66183c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.905+0000 7f6274c10b80 1 --2- 172.21.15.2:0/821615140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b66183be10 0x55b66183c220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f625800ef00 tx=0x7f62580a8c00 comp rx=0 tx=0).stop 2024-05-14T09:37:14.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.905+0000 7f6274c10b80 1 -- 172.21.15.2:0/821615140 shutdown_connections 2024-05-14T09:37:14.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.905+0000 7f6274c10b80 1 --2- 172.21.15.2:0/821615140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b66183be10 0x55b66183c220 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.905+0000 7f6274c10b80 1 --2- 172.21.15.2:0/821615140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b6616bde40 0x55b661841840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.905+0000 7f6274c10b80 1 --2- 172.21.15.2:0/821615140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 0x55b66183b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.905+0000 7f6274c10b80 1 -- 172.21.15.2:0/821615140 >> 172.21.15.2:0/821615140 conn(0x55b661705630 msgr2=0x55b661836a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:14.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.905+0000 7f6274c10b80 1 -- 172.21.15.2:0/821615140 shutdown_connections 2024-05-14T09:37:14.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.905+0000 7f6274c10b80 1 -- 172.21.15.2:0/821615140 wait complete. 2024-05-14T09:37:14.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.905+0000 7f6274c10b80 1 -- start start 2024-05-14T09:37:14.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.905+0000 7f6274c10b80 1 Processor -- start 2024-05-14T09:37:14.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.906+0000 7f6274c10b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6616bde40 0x55b6618454e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.906+0000 7f6274c10b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b66183be10 0x55b661845a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.906+0000 7f6274c10b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 0x55b661845f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.906+0000 7f6274c10b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b6618487c0 con 0x55b66183be10 2024-05-14T09:37:14.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.906+0000 7f6274c10b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b661716ed0 con 0x55b6616bde40 2024-05-14T09:37:14.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.906+0000 7f6274c10b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b661848940 con 0x55b661841d80 2024-05-14T09:37:14.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.906+0000 7f6274c10b80 10 client.0 _ll_register_callbacks cb 0x55b6616f5950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:14.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.906+0000 7f625ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 0x55b661845f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.906+0000 7f625effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b66183be10 0x55b661845a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.906+0000 7f625ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 0x55b661845f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:37856/0 (socket says 172.21.15.2:37856) 2024-05-14T09:37:14.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.906+0000 7f625effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b66183be10 0x55b661845a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41816/0 (socket says 172.21.15.2:41816) 2024-05-14T09:37:14.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.906+0000 7f625ffff700 1 -- 172.21.15.2:0/51462540 learned_addr learned my addr 172.21.15.2:0/51462540 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:14.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.906+0000 7f625f7fe700 1 --2- 172.21.15.2:0/51462540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6616bde40 0x55b6618454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.906+0000 7f625f7fe700 1 -- 172.21.15.2:0/51462540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 msgr2=0x55b661845f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.906+0000 7f625f7fe700 1 --2- 172.21.15.2:0/51462540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 0x55b661845f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.907+0000 7f625f7fe700 1 -- 172.21.15.2:0/51462540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b66183be10 msgr2=0x55b661845a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.907+0000 7f625f7fe700 1 --2- 172.21.15.2:0/51462540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b66183be10 0x55b661845a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.907+0000 7f625f7fe700 1 -- 172.21.15.2:0/51462540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b661714760 con 0x55b6616bde40 2024-05-14T09:37:14.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.907+0000 7f625effd700 1 --2- 172.21.15.2:0/51462540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b66183be10 0x55b661845a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:14.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.907+0000 7f625f7fe700 1 --2- 172.21.15.2:0/51462540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6616bde40 0x55b6618454e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6250003900 tx=0x7f62500047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19b586b7526933a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:14.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.907+0000 7f623effd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:37:14.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.907+0000 7f623effd700 1 -- 172.21.15.2:0/51462540 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625001d620 con 0x55b6616bde40 2024-05-14T09:37:14.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.907+0000 7f6274c10b80 1 -- 172.21.15.2:0/51462540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62580a90f0 con 0x55b6616bde40 2024-05-14T09:37:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.907+0000 7f623effd700 1 -- 172.21.15.2:0/51462540 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f625001d7c0 con 0x55b6616bde40 2024-05-14T09:37:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.907+0000 7f623effd700 1 -- 172.21.15.2:0/51462540 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f625001d9c0 con 0x55b6616bde40 2024-05-14T09:37:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.907+0000 7f6274c10b80 10 client.25535 fetch_fsmap learned FSMap version 28 2024-05-14T09:37:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.907+0000 7f6274c10b80 1 -- 172.21.15.2:0/51462540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6618532b0 con 0x55b6616bde40 2024-05-14T09:37:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.907+0000 7f623effd700 1 -- 172.21.15.2:0/51462540 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62580a90f0 con 0x55b6616bde40 2024-05-14T09:37:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.908+0000 7f623effd700 1 -- 172.21.15.2:0/51462540 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6250027cb0 con 0x55b6616bde40 2024-05-14T09:37:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.908+0000 7f6274c10b80 10 client.25535 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T09:37:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.908+0000 7f6274c10b80 10 client.25535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.908+0000 7f6274c10b80 20 client.25535 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.908+0000 7f6274c10b80 1 --2- 172.21.15.2:0/51462540 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b6618558e0 0x55b661857d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.908+0000 7f6274c10b80 4 client.25535 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:37:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.908+0000 7f6274c10b80 1 -- 172.21.15.2:0/51462540 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b66176b320 con 0x55b6618558e0 2024-05-14T09:37:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.908+0000 7f625effd700 1 --2- 172.21.15.2:0/51462540 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b6618558e0 0x55b661857d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.908+0000 7f623effd700 10 client.25535 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:14.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.909+0000 7f625effd700 1 --2- 172.21.15.2:0/51462540 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b6618558e0 0x55b661857d00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4773cb3368756747 server_cookie=42d161fcf5ac1545 in_seq=0 out_seq=0 2024-05-14T09:37:14.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.910+0000 7f623effd700 1 -- 172.21.15.2:0/51462540 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b66176b320 con 0x55b6618558e0 2024-05-14T09:37:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.910+0000 7f623effd700 10 client.25535 handle_command_reply: tid=0 2024-05-14T09:37:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.910+0000 7f6274c10b80 1 client.25535 shutdown 2024-05-14T09:37:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.910+0000 7f6274c10b80 1 -- 172.21.15.2:0/51462540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6616bde40 msgr2=0x55b6618454e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.910+0000 7f6274c10b80 1 --2- 172.21.15.2:0/51462540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6616bde40 0x55b6618454e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6250003900 tx=0x7f62500047c0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.910+0000 7f6274c10b80 1 -- 172.21.15.2:0/51462540 shutdown_connections 2024-05-14T09:37:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.910+0000 7f6274c10b80 1 --2- 172.21.15.2:0/51462540 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b6618558e0 0x55b661857d00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.910+0000 7f6274c10b80 1 --2- 172.21.15.2:0/51462540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6616bde40 0x55b6618454e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.910+0000 7f623effd700 0 client.25535 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.910+0000 7f6274c10b80 1 --2- 172.21.15.2:0/51462540 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 0x55b661845f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.910+0000 7f6274c10b80 1 --2- 172.21.15.2:0/51462540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b66183be10 0x55b661845a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.910+0000 7f6274c10b80 1 -- 172.21.15.2:0/51462540 >> 172.21.15.2:0/51462540 conn(0x55b661705630 msgr2=0x55b66183a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.910+0000 7f6274c10b80 1 -- 172.21.15.2:0/51462540 shutdown_connections 2024-05-14T09:37:14.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.910+0000 7f6274c10b80 1 -- 172.21.15.2:0/51462540 wait complete. 2024-05-14T09:37:14.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.910+0000 7f6274c10b80 20 client.25535 trim_cache size 0 max 16384 2024-05-14T09:37:14.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.931+0000 7f6274c10b80 1 Processor -- start 2024-05-14T09:37:14.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.931+0000 7f6274c10b80 1 -- start start 2024-05-14T09:37:14.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.931+0000 7f6274c10b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6616bde40 0x55b66183b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.931+0000 7f6274c10b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b66183be10 0x55b66183fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.931+0000 7f6274c10b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 0x55b661851880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.931+0000 7f6274c10b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f624c011580 con 0x55b66183be10 2024-05-14T09:37:14.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.931+0000 7f6274c10b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f624c00ee10 con 0x55b6616bde40 2024-05-14T09:37:14.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.931+0000 7f6274c10b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f624c00cc60 con 0x55b661841d80 2024-05-14T09:37:14.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.931+0000 7f625effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b66183be10 0x55b66183fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.931+0000 7f625effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b66183be10 0x55b66183fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41832/0 (socket says 172.21.15.2:41832) 2024-05-14T09:37:14.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.931+0000 7f625effd700 1 -- 172.21.15.2:0/2876872557 learned_addr learned my addr 172.21.15.2:0/2876872557 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:14.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.931+0000 7f625ffff700 1 --2- 172.21.15.2:0/2876872557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 0x55b661851880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.931+0000 7f625f7fe700 1 --2- 172.21.15.2:0/2876872557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6616bde40 0x55b66183b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.932+0000 7f625effd700 1 -- 172.21.15.2:0/2876872557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 msgr2=0x55b661851880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.932+0000 7f625effd700 1 --2- 172.21.15.2:0/2876872557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 0x55b661851880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.932+0000 7f625effd700 1 -- 172.21.15.2:0/2876872557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6616bde40 msgr2=0x55b66183b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.932+0000 7f625effd700 1 --2- 172.21.15.2:0/2876872557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6616bde40 0x55b66183b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.932+0000 7f625effd700 1 -- 172.21.15.2:0/2876872557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b661843a50 con 0x55b66183be10 2024-05-14T09:37:14.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.932+0000 7f625effd700 1 --2- 172.21.15.2:0/2876872557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b66183be10 0x55b66183fb80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f62540098e0 tx=0x7f6254005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6878cb180470a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:14.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.932+0000 7f62457fa700 1 -- 172.21.15.2:0/2876872557 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254003880 con 0x55b66183be10 2024-05-14T09:37:14.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.932+0000 7f62457fa700 1 -- 172.21.15.2:0/2876872557 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6254029030 con 0x55b66183be10 2024-05-14T09:37:14.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.932+0000 7f62457fa700 1 -- 172.21.15.2:0/2876872557 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254029850 con 0x55b66183be10 2024-05-14T09:37:14.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.933+0000 7f6274c10b80 1 -- 172.21.15.2:0/2876872557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b66183be10 msgr2=0x55b66183fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.933+0000 7f6274c10b80 1 --2- 172.21.15.2:0/2876872557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b66183be10 0x55b66183fb80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f62540098e0 tx=0x7f6254005fd0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.933+0000 7f6274c10b80 1 -- 172.21.15.2:0/2876872557 shutdown_connections 2024-05-14T09:37:14.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.933+0000 7f6274c10b80 1 --2- 172.21.15.2:0/2876872557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6616bde40 0x55b66183b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.933+0000 7f6274c10b80 1 --2- 172.21.15.2:0/2876872557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b661841d80 0x55b661851880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.933+0000 7f6274c10b80 1 --2- 172.21.15.2:0/2876872557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b66183be10 0x55b66183fb80 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.933+0000 7f6274c10b80 1 -- 172.21.15.2:0/2876872557 >> 172.21.15.2:0/2876872557 conn(0x55b661705630 msgr2=0x55b6618368e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:14.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.933+0000 7f6274c10b80 1 -- 172.21.15.2:0/2876872557 shutdown_connections 2024-05-14T09:37:14.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.933+0000 7f6274c10b80 1 -- 172.21.15.2:0/2876872557 wait complete. 2024-05-14T09:37:14.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.933+0000 7f6274c10b80 1 -- start start 2024-05-14T09:37:14.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.934+0000 7f6274c10b80 1 Processor -- start 2024-05-14T09:37:14.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.934+0000 7f6274c10b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b6616bde40 0x55b66184bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.934+0000 7f6274c10b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b66183be10 0x55b66184c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.934+0000 7f6274c10b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b661841d80 0x55b6618bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.934+0000 7f6274c10b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f624c00d2d0 con 0x55b6616bde40 2024-05-14T09:37:14.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.934+0000 7f6274c10b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f624c00e740 con 0x55b661841d80 2024-05-14T09:37:14.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.934+0000 7f6274c10b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f624c00d9a0 con 0x55b66183be10 2024-05-14T09:37:14.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.934+0000 7f6274c10b80 10 client.0 _ll_register_callbacks cb 0x55b6616f5950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:14.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.934+0000 7f625effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b66183be10 0x55b66184c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.934+0000 7f625effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b66183be10 0x55b66184c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:37882/0 (socket says 172.21.15.2:37882) 2024-05-14T09:37:14.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.934+0000 7f625ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b661841d80 0x55b6618bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.934+0000 7f625f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b6616bde40 0x55b66184bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.934+0000 7f625effd700 1 -- 172.21.15.2:0/3843954656 learned_addr learned my addr 172.21.15.2:0/3843954656 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:14.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.935+0000 7f625effd700 1 -- 172.21.15.2:0/3843954656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b661841d80 msgr2=0x55b6618bf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.935+0000 7f625effd700 1 --2- 172.21.15.2:0/3843954656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b661841d80 0x55b6618bf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.935+0000 7f625effd700 1 -- 172.21.15.2:0/3843954656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b6616bde40 msgr2=0x55b66184bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:14.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.935+0000 7f625effd700 1 --2- 172.21.15.2:0/3843954656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b6616bde40 0x55b66184bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.935+0000 7f625effd700 1 -- 172.21.15.2:0/3843954656 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b66184c800 con 0x55b66183be10 2024-05-14T09:37:14.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.935+0000 7f625f7fe700 1 --2- 172.21.15.2:0/3843954656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b6616bde40 0x55b66184bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:14.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.935+0000 7f625ffff700 1 --2- 172.21.15.2:0/3843954656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b661841d80 0x55b6618bf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:14.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.935+0000 7f625effd700 1 --2- 172.21.15.2:0/3843954656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b66183be10 0x55b66184c000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6254009ce0 tx=0x7f62540056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f61a8c7eb995684a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:14.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.935+0000 7f623effd700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:37:14.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.935+0000 7f623effd700 1 -- 172.21.15.2:0/3843954656 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62540298e0 con 0x55b66183be10 2024-05-14T09:37:14.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.935+0000 7f6274c10b80 1 -- 172.21.15.2:0/3843954656 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6254029030 con 0x55b66183be10 2024-05-14T09:37:14.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.935+0000 7f623effd700 1 -- 172.21.15.2:0/3843954656 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f6254003680 con 0x55b66183be10 2024-05-14T09:37:14.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.936+0000 7f623effd700 1 -- 172.21.15.2:0/3843954656 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6254042050 con 0x55b66183be10 2024-05-14T09:37:14.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.936+0000 7f6274c10b80 10 client.25543 fetch_fsmap learned FSMap version 28 2024-05-14T09:37:14.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.936+0000 7f6274c10b80 1 -- 172.21.15.2:0/3843954656 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6618c0580 con 0x55b66183be10 2024-05-14T09:37:14.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.936+0000 7f623effd700 1 -- 172.21.15.2:0/3843954656 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6254029030 con 0x55b66183be10 2024-05-14T09:37:14.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.936+0000 7f623effd700 1 -- 172.21.15.2:0/3843954656 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f625400f720 con 0x55b66183be10 2024-05-14T09:37:14.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.936+0000 7f6274c10b80 10 client.25543 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T09:37:14.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.936+0000 7f6274c10b80 10 client.25543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:14.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.936+0000 7f6274c10b80 20 client.25543 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:14.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.936+0000 7f6274c10b80 1 --2- 172.21.15.2:0/3843954656 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b6618558e0 0x55b6618c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:14.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.936+0000 7f6274c10b80 4 client.25543 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T09:37:14.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.936+0000 7f6274c10b80 1 -- 172.21.15.2:0/3843954656 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6617653c0 con 0x55b6618558e0 2024-05-14T09:37:14.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.936+0000 7f625f7fe700 1 --2- 172.21.15.2:0/3843954656 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b6618558e0 0x55b6618c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:14.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.937+0000 7f623effd700 10 client.25543 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:14.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.937+0000 7f625f7fe700 1 --2- 172.21.15.2:0/3843954656 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b6618558e0 0x55b6618c2c80 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d05cd8a9bdc4d13 server_cookie=ca3b16ba31b91fb1 in_seq=0 out_seq=0 2024-05-14T09:37:14.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.938+0000 7f623effd700 1 -- 172.21.15.2:0/3843954656 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b6617653c0 con 0x55b6618558e0 2024-05-14T09:37:14.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.938+0000 7f623effd700 10 client.25543 handle_command_reply: tid=0 2024-05-14T09:37:14.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.938+0000 7f6274c10b80 1 client.25543 shutdown 2024-05-14T09:37:14.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.938+0000 7f6274c10b80 1 -- 172.21.15.2:0/3843954656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b66183be10 msgr2=0x55b66184c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.938+0000 7f6274c10b80 1 --2- 172.21.15.2:0/3843954656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b66183be10 0x55b66184c000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6254009ce0 tx=0x7f62540056b0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.938+0000 7f6274c10b80 1 -- 172.21.15.2:0/3843954656 shutdown_connections 2024-05-14T09:37:14.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.938+0000 7f6274c10b80 1 --2- 172.21.15.2:0/3843954656 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b6618558e0 0x55b6618c2c80 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.938+0000 7f6274c10b80 1 --2- 172.21.15.2:0/3843954656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b661841d80 0x55b6618bf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.938+0000 7f6274c10b80 1 --2- 172.21.15.2:0/3843954656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b6616bde40 0x55b66184bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.938+0000 7f6274c10b80 1 --2- 172.21.15.2:0/3843954656 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b66183be10 0x55b66184c000 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.938+0000 7f623effd700 0 client.25543 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:14.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.938+0000 7f6274c10b80 1 -- 172.21.15.2:0/3843954656 >> 172.21.15.2:0/3843954656 conn(0x55b661705630 msgr2=0x55b6618368e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:14.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.938+0000 7f6274c10b80 1 -- 172.21.15.2:0/3843954656 shutdown_connections 2024-05-14T09:37:14.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.938+0000 7f6274c10b80 1 -- 172.21.15.2:0/3843954656 wait complete. 2024-05-14T09:37:14.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.938+0000 7f6274c10b80 20 client.25543 trim_cache size 0 max 16384 2024-05-14T09:37:14.943 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:37:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T09:37:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-05-14T09:37:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: "2fb7ad7a-d3c3-4509-8fe3-03072ae1d9ad": { 2024-05-14T09:37:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-05-14T09:37:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "2fb7ad7a-d3c3-4509-8fe3-03072ae1d9ad", 2024-05-14T09:37:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-05-14T09:37:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T09:37:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T09:37:14.944 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:37:14.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.939+0000 7f62661fc700 1 -- 172.21.15.2:0/1644002994 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f62480b2980 msgr2=0x7f62480b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.939+0000 7f62661fc700 1 --2- 172.21.15.2:0/1644002994 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f62480b2980 0x7f62480b4e90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6250009c60 tx=0x7f6250009260 comp rx=0 tx=0).stop 2024-05-14T09:37:14.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.939+0000 7f62661fc700 1 -- 172.21.15.2:0/1644002994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62601086a0 msgr2=0x7f62601a5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:14.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.939+0000 7f62661fc700 1 --2- 172.21.15.2:0/1644002994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62601086a0 0x7f62601a5c10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f625800ef30 tx=0x7f625800c3a0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.939+0000 7f62661fc700 1 -- 172.21.15.2:0/1644002994 shutdown_connections 2024-05-14T09:37:14.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.939+0000 7f62661fc700 1 --2- 172.21.15.2:0/1644002994 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f62480b2980 0x7f62480b4e90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.939+0000 7f62661fc700 1 --2- 172.21.15.2:0/1644002994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6260103250 0x7f62601a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.939+0000 7f62661fc700 1 --2- 172.21.15.2:0/1644002994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62601086a0 0x7f62601a5c10 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.939+0000 7f62661fc700 1 --2- 172.21.15.2:0/1644002994 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6260103ba0 0x7f62601a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:14.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.939+0000 7f62661fc700 1 -- 172.21.15.2:0/1644002994 >> 172.21.15.2:0/1644002994 conn(0x7f62600fedc0 msgr2=0x7f62601060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:14.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.939+0000 7f62661fc700 1 -- 172.21.15.2:0/1644002994 shutdown_connections 2024-05-14T09:37:14.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:14.939+0000 7f62661fc700 1 -- 172.21.15.2:0/1644002994 wait complete. 2024-05-14T09:37:15.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 91111 12.34 MB/sec warmup 41 sec latency 76.798 ms 2024-05-14T09:37:15.758 INFO:tasks.cephfs.filesystem:scrub status for tag:2fb7ad7a-d3c3-4509-8fe3-03072ae1d9ad - {'path': '/', 'tag': '2fb7ad7a-d3c3-4509-8fe3-03072ae1d9ad', 'options': 'recursive,force'} 2024-05-14T09:37:16.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 92775 12.33 MB/sec warmup 42 sec latency 22.354 ms 2024-05-14T09:37:16.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:16 smithi002 ceph-mon[31557]: pgmap v493: 129 pgs: 129 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 685 KiB/s wr, 145 op/s 2024-05-14T09:37:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:16 smithi029 ceph-mon[36249]: pgmap v493: 129 pgs: 129 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 685 KiB/s wr, 145 op/s 2024-05-14T09:37:16.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:16 smithi026 ceph-mon[36056]: pgmap v493: 129 pgs: 129 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 685 KiB/s wr, 145 op/s 2024-05-14T09:37:17.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 95189 12.34 MB/sec warmup 43 sec latency 12.761 ms 2024-05-14T09:37:18.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 96760 12.23 MB/sec warmup 44 sec latency 18.927 ms 2024-05-14T09:37:18.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:18 smithi002 ceph-mon[31557]: pgmap v494: 129 pgs: 129 active+clean; 31 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 265 op/s 2024-05-14T09:37:18.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:18 smithi029 ceph-mon[36249]: pgmap v494: 129 pgs: 129 active+clean; 31 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 265 op/s 2024-05-14T09:37:18.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:18 smithi026 ceph-mon[36056]: pgmap v494: 129 pgs: 129 active+clean; 31 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 265 op/s 2024-05-14T09:37:19.254 INFO:tasks.workunit.client.0.smithi002.stdout: 1 99502 12.33 MB/sec warmup 45 sec latency 16.313 ms 2024-05-14T09:37:19.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:19 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a[31553]: 2024-05-14T09:37:19.283+0000 7f82afa15700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-14T09:37:19.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:19 smithi002 ceph-mon[31557]: scrub summary: idle 2024-05-14T09:37:19.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:19 smithi002 ceph-mon[31557]: scrub summary: idle 2024-05-14T09:37:19.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:19 smithi029 ceph-mon[36249]: scrub summary: idle 2024-05-14T09:37:19.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:19 smithi029 ceph-mon[36249]: scrub summary: idle 2024-05-14T09:37:19.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:19 smithi026 ceph-mon[36056]: scrub summary: idle 2024-05-14T09:37:19.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:19 smithi026 ceph-mon[36056]: scrub summary: idle 2024-05-14T09:37:20.255 INFO:tasks.workunit.client.0.smithi002.stdout: 1 101020 12.21 MB/sec warmup 46 sec latency 12.279 ms 2024-05-14T09:37:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:20 smithi002 ceph-mon[31557]: scrub summary: idle 2024-05-14T09:37:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:20 smithi002 ceph-mon[31557]: pgmap v495: 129 pgs: 129 active+clean; 32 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 881 KiB/s wr, 226 op/s 2024-05-14T09:37:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:20 smithi002 ceph-mon[31557]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-14T09:37:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:20 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] up:active 2024-05-14T09:37:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:20 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:37:20.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:20 smithi029 ceph-mon[36249]: scrub summary: idle 2024-05-14T09:37:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:20 smithi029 ceph-mon[36249]: pgmap v495: 129 pgs: 129 active+clean; 32 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 881 KiB/s wr, 226 op/s 2024-05-14T09:37:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:20 smithi029 ceph-mon[36249]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-14T09:37:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:20 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] up:active 2024-05-14T09:37:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:20 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:37:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:20 smithi026 ceph-mon[36056]: scrub summary: idle 2024-05-14T09:37:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:20 smithi026 ceph-mon[36056]: pgmap v495: 129 pgs: 129 active+clean; 32 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 881 KiB/s wr, 226 op/s 2024-05-14T09:37:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:20 smithi026 ceph-mon[36056]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-14T09:37:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:20 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] up:active 2024-05-14T09:37:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:20 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:37:21.255 INFO:tasks.workunit.client.0.smithi002.stdout: 1 103263 12.24 MB/sec warmup 47 sec latency 24.591 ms 2024-05-14T09:37:22.255 INFO:tasks.workunit.client.0.smithi002.stdout: 1 105249 12.13 MB/sec warmup 48 sec latency 19.557 ms 2024-05-14T09:37:22.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:22 smithi002 ceph-mon[31557]: pgmap v496: 129 pgs: 129 active+clean; 34 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1023 KiB/s wr, 269 op/s 2024-05-14T09:37:22.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:22 smithi029 ceph-mon[36249]: pgmap v496: 129 pgs: 129 active+clean; 34 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1023 KiB/s wr, 269 op/s 2024-05-14T09:37:22.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:22 smithi026 ceph-mon[36056]: pgmap v496: 129 pgs: 129 active+clean; 34 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1023 KiB/s wr, 269 op/s 2024-05-14T09:37:23.255 INFO:tasks.workunit.client.0.smithi002.stdout: 1 106918 12.13 MB/sec warmup 49 sec latency 15.701 ms 2024-05-14T09:37:24.255 INFO:tasks.workunit.client.0.smithi002.stdout: 1 109058 12.05 MB/sec warmup 50 sec latency 16.233 ms 2024-05-14T09:37:24.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:24 smithi002 ceph-mon[31557]: pgmap v497: 129 pgs: 129 active+clean; 36 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.2 MiB/s wr, 299 op/s 2024-05-14T09:37:24.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:24 smithi029 ceph-mon[36249]: pgmap v497: 129 pgs: 129 active+clean; 36 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.2 MiB/s wr, 299 op/s 2024-05-14T09:37:24.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:24 smithi026 ceph-mon[36056]: pgmap v497: 129 pgs: 129 active+clean; 36 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.2 MiB/s wr, 299 op/s 2024-05-14T09:37:25.255 INFO:tasks.workunit.client.0.smithi002.stdout: 1 110738 12.04 MB/sec warmup 51 sec latency 20.644 ms 2024-05-14T09:37:26.255 INFO:tasks.workunit.client.0.smithi002.stdout: 1 113524 12.14 MB/sec warmup 52 sec latency 12.086 ms 2024-05-14T09:37:26.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:26 smithi002 ceph-mon[31557]: pgmap v498: 129 pgs: 129 active+clean; 36 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.0 MiB/s wr, 260 op/s 2024-05-14T09:37:26.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:26 smithi029 ceph-mon[36249]: pgmap v498: 129 pgs: 129 active+clean; 36 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.0 MiB/s wr, 260 op/s 2024-05-14T09:37:26.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:26 smithi026 ceph-mon[36056]: pgmap v498: 129 pgs: 129 active+clean; 36 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.0 MiB/s wr, 260 op/s 2024-05-14T09:37:27.255 INFO:tasks.workunit.client.0.smithi002.stdout: 1 114802 11.98 MB/sec warmup 53 sec latency 21.067 ms 2024-05-14T09:37:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.526+0000 7f2714d7e640 1 -- 172.21.15.2:0/1748501222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2710108ab0 msgr2=0x7f271010aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.526+0000 7f2714d7e640 1 --2- 172.21.15.2:0/1748501222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2710108ab0 0x7f271010aed0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2708009ca0 tx=0x7f270802a8c0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.526+0000 7f2714d7e640 1 -- 172.21.15.2:0/1748501222 shutdown_connections 2024-05-14T09:37:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.526+0000 7f2714d7e640 1 --2- 172.21.15.2:0/1748501222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2710108ab0 0x7f271010aed0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.526+0000 7f2714d7e640 1 --2- 172.21.15.2:0/1748501222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2710103db0 0x7f27101041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.526+0000 7f2714d7e640 1 --2- 172.21.15.2:0/1748501222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2710102b70 0x7f2710103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.526+0000 7f2714d7e640 1 -- 172.21.15.2:0/1748501222 >> 172.21.15.2:0/1748501222 conn(0x7f27100fdd60 msgr2=0x7f2710100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.526+0000 7f2714d7e640 1 -- 172.21.15.2:0/1748501222 shutdown_connections 2024-05-14T09:37:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.526+0000 7f2714d7e640 1 -- 172.21.15.2:0/1748501222 wait complete. 2024-05-14T09:37:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f2714d7e640 1 Processor -- start 2024-05-14T09:37:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f2714d7e640 1 -- start start 2024-05-14T09:37:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f2714d7e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2710102b70 0x7f271019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f2714d7e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2710103db0 0x7f271019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f2714d7e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2710108ab0 0x7f27101a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f2714d7e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f271010f7a0 con 0x7f2710102b70 2024-05-14T09:37:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f2714d7e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f271010f620 con 0x7f2710108ab0 2024-05-14T09:37:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f2714d7e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f271010f920 con 0x7f2710103db0 2024-05-14T09:37:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f270e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2710102b70 0x7f271019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f270e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2710102b70 0x7f271019a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47992/0 (socket says 172.21.15.2:47992) 2024-05-14T09:37:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f270e59c640 1 -- 172.21.15.2:0/3643224645 learned_addr learned my addr 172.21.15.2:0/3643224645 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f270ed9d640 1 --2- 172.21.15.2:0/3643224645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2710108ab0 0x7f27101a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f270dd9b640 1 --2- 172.21.15.2:0/3643224645 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2710103db0 0x7f271019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f270e59c640 1 -- 172.21.15.2:0/3643224645 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2710103db0 msgr2=0x7f271019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f270e59c640 1 --2- 172.21.15.2:0/3643224645 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2710103db0 0x7f271019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f270e59c640 1 -- 172.21.15.2:0/3643224645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2710108ab0 msgr2=0x7f27101a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f270e59c640 1 --2- 172.21.15.2:0/3643224645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2710108ab0 0x7f27101a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f270e59c640 1 -- 172.21.15.2:0/3643224645 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27101a1db0 con 0x7f2710102b70 2024-05-14T09:37:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.527+0000 7f270dd9b640 1 --2- 172.21.15.2:0/3643224645 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2710103db0 0x7f271019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:37:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.528+0000 7f270e59c640 1 --2- 172.21.15.2:0/3643224645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2710102b70 0x7f271019a270 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f270000e9f0 tx=0x7f270000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.528+0000 7f26ff7fe640 1 -- 172.21.15.2:0/3643224645 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270000ccc0 con 0x7f2710102b70 2024-05-14T09:37:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.528+0000 7f2714d7e640 1 -- 172.21.15.2:0/3643224645 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27101a20a0 con 0x7f2710102b70 2024-05-14T09:37:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.528+0000 7f26ff7fe640 1 -- 172.21.15.2:0/3643224645 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27000035f0 con 0x7f2710102b70 2024-05-14T09:37:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.528+0000 7f2714d7e640 1 -- 172.21.15.2:0/3643224645 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27101a25f0 con 0x7f2710102b70 2024-05-14T09:37:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.528+0000 7f26ff7fe640 1 -- 172.21.15.2:0/3643224645 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270000ce60 con 0x7f2710102b70 2024-05-14T09:37:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.531+0000 7f271bc30740 1 Processor -- start 2024-05-14T09:37:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.531+0000 7f271bc30740 1 -- start start 2024-05-14T09:37:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.531+0000 7f271bc30740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94e91b0 0x556cd988aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.531+0000 7f271bc30740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c02f0 0x556cd988afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.531+0000 7f271bc30740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94c84d0 0x556cd9891ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.531+0000 7f271bc30740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556cd97ef5d0 con 0x556cd94c02f0 2024-05-14T09:37:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.531+0000 7f271bc30740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556cd97a8890 con 0x556cd94e91b0 2024-05-14T09:37:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.531+0000 7f271bc30740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556cd97eb590 con 0x556cd94c84d0 2024-05-14T09:37:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.532+0000 7f26ff7fe640 1 -- 172.21.15.2:0/3643224645 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2700010690 con 0x7f2710102b70 2024-05-14T09:37:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.532+0000 7f270ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94e91b0 0x556cd988aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.532+0000 7f270ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94e91b0 0x556cd988aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:37106/0 (socket says 172.21.15.2:37106) 2024-05-14T09:37:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.532+0000 7f270ed9d640 1 -- 172.21.15.2:0/4196720586 learned_addr learned my addr 172.21.15.2:0/4196720586 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.532+0000 7f270dd9b640 1 --2- 172.21.15.2:0/4196720586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94c84d0 0x556cd9891ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.532+0000 7f270e59c640 1 --2- 172.21.15.2:0/4196720586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c02f0 0x556cd988afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.532+0000 7f270ed9d640 1 -- 172.21.15.2:0/4196720586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94c84d0 msgr2=0x556cd9891ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.532+0000 7f270ed9d640 1 --2- 172.21.15.2:0/4196720586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94c84d0 0x556cd9891ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.532+0000 7f270ed9d640 1 -- 172.21.15.2:0/4196720586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c02f0 msgr2=0x556cd988afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.532+0000 7f270ed9d640 1 --2- 172.21.15.2:0/4196720586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c02f0 0x556cd988afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.532+0000 7f270ed9d640 1 -- 172.21.15.2:0/4196720586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cd9501210 con 0x556cd94e91b0 2024-05-14T09:37:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.532+0000 7f270e59c640 1 --2- 172.21.15.2:0/4196720586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c02f0 0x556cd988afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:37:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.532+0000 7f270ed9d640 1 --2- 172.21.15.2:0/4196720586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94e91b0 0x556cd988aa60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2708035000 tx=0x7f270802f410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5e65b9e957bf2cd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.532+0000 7f26fcff9640 1 -- 172.21.15.2:0/4196720586 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2708031650 con 0x556cd94e91b0 2024-05-14T09:37:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.533+0000 7f26fcff9640 1 -- 172.21.15.2:0/4196720586 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2708036030 con 0x556cd94e91b0 2024-05-14T09:37:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.533+0000 7f26ff7fe640 1 --2- 172.21.15.2:0/3643224645 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f26f80b28a0 0x7f26f80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.533+0000 7f26fcff9640 1 -- 172.21.15.2:0/4196720586 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2708004760 con 0x556cd94e91b0 2024-05-14T09:37:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.533+0000 7f270e59c640 1 --2- 172.21.15.2:0/3643224645 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f26f80b28a0 0x7f26f80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.533+0000 7f26ff7fe640 1 -- 172.21.15.2:0/3643224645 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f27000e29e0 con 0x7f2710102b70 2024-05-14T09:37:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.533+0000 7f271bc30740 1 -- 172.21.15.2:0/4196720586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94e91b0 msgr2=0x556cd988aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.533+0000 7f270e59c640 1 --2- 172.21.15.2:0/3643224645 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f26f80b28a0 0x7f26f80b4db0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f27000e0430 tx=0x7f2700003a10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.533+0000 7f271bc30740 1 --2- 172.21.15.2:0/4196720586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94e91b0 0x556cd988aa60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2708035000 tx=0x7f270802f410 comp rx=0 tx=0).stop 2024-05-14T09:37:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.533+0000 7f271bc30740 1 -- 172.21.15.2:0/4196720586 shutdown_connections 2024-05-14T09:37:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.533+0000 7f271bc30740 1 --2- 172.21.15.2:0/4196720586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94c84d0 0x556cd9891ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.533+0000 7f271bc30740 1 --2- 172.21.15.2:0/4196720586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c02f0 0x556cd988afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.533+0000 7f271bc30740 1 --2- 172.21.15.2:0/4196720586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94e91b0 0x556cd988aa60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.533+0000 7f271bc30740 1 -- 172.21.15.2:0/4196720586 >> 172.21.15.2:0/4196720586 conn(0x556cd94dd300 msgr2=0x556cd98863a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.534+0000 7f271bc30740 1 -- 172.21.15.2:0/4196720586 shutdown_connections 2024-05-14T09:37:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.534+0000 7f271bc30740 1 -- 172.21.15.2:0/4196720586 wait complete. 2024-05-14T09:37:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.534+0000 7f271bc30740 1 -- start start 2024-05-14T09:37:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.534+0000 7f271bc30740 1 Processor -- start 2024-05-14T09:37:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.534+0000 7f271bc30740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94c02f0 0x556cd9894490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.534+0000 7f271bc30740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c84d0 0x556cd98949d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.534+0000 7f271bc30740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94e91b0 0x556cd989b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.534+0000 7f271bc30740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556cd9892750 con 0x556cd94c84d0 2024-05-14T09:37:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.534+0000 7f271bc30740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556cd97ac370 con 0x556cd94e91b0 2024-05-14T09:37:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.534+0000 7f271bc30740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556cd98928d0 con 0x556cd94c02f0 2024-05-14T09:37:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.534+0000 7f271bc30740 10 client.0 _ll_register_callbacks cb 0x556cd9500a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.534+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c84d0 0x556cd98949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.534+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c84d0 0x556cd98949d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48008/0 (socket says 172.21.15.2:48008) 2024-05-14T09:37:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.534+0000 7f270dd9b640 1 -- 172.21.15.2:0/2604852188 learned_addr learned my addr 172.21.15.2:0/2604852188 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.534+0000 7f270ed9d640 1 --2- 172.21.15.2:0/2604852188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94c02f0 0x556cd9894490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.535+0000 7f270ed9d640 1 --2- 172.21.15.2:0/2604852188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94e91b0 0x556cd989b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.535+0000 7f270dd9b640 1 -- 172.21.15.2:0/2604852188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94c02f0 msgr2=0x556cd9894490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.535+0000 7f270dd9b640 1 --2- 172.21.15.2:0/2604852188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94c02f0 0x556cd9894490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.535+0000 7f270dd9b640 1 -- 172.21.15.2:0/2604852188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94e91b0 msgr2=0x556cd989b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.535+0000 7f270dd9b640 1 --2- 172.21.15.2:0/2604852188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94e91b0 0x556cd989b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.535+0000 7f270dd9b640 1 -- 172.21.15.2:0/2604852188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cd95cd620 con 0x556cd94c84d0 2024-05-14T09:37:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.535+0000 7f270dd9b640 1 --2- 172.21.15.2:0/2604852188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c84d0 0x556cd98949d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2704013710 tx=0x7f2704013b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723a536fdd902bca server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.535+0000 7f26f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:37:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.535+0000 7f26f5ffb640 1 -- 172.21.15.2:0/2604852188 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704011dd0 con 0x556cd94c84d0 2024-05-14T09:37:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.535+0000 7f26f5ffb640 1 -- 172.21.15.2:0/2604852188 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f2704004020 con 0x556cd94c84d0 2024-05-14T09:37:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.535+0000 7f271bc30740 1 -- 172.21.15.2:0/2604852188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2708036030 con 0x556cd94c84d0 2024-05-14T09:37:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.535+0000 7f26f5ffb640 1 -- 172.21.15.2:0/2604852188 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2704004260 con 0x556cd94c84d0 2024-05-14T09:37:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.535+0000 7f271bc30740 10 client.15864 fetch_fsmap learned FSMap version 29 2024-05-14T09:37:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.535+0000 7f271bc30740 1 -- 172.21.15.2:0/2604852188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556cd989e8c0 con 0x556cd94c84d0 2024-05-14T09:37:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.536+0000 7f26f5ffb640 1 -- 172.21.15.2:0/2604852188 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2708036030 con 0x556cd94c84d0 2024-05-14T09:37:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.536+0000 7f26f5ffb640 1 -- 172.21.15.2:0/2604852188 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f270401a070 con 0x556cd94c84d0 2024-05-14T09:37:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.536+0000 7f271bc30740 10 client.15864 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:37:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.536+0000 7f271bc30740 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.536+0000 7f271bc30740 20 client.15864 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.536+0000 7f271bc30740 1 --2- 172.21.15.2:0/2604852188 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cd94a7d80 0x556cd98a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.536+0000 7f271bc30740 4 client.15864 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:37:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.536+0000 7f271bc30740 1 -- 172.21.15.2:0/2604852188 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cd97fd1e0 con 0x556cd94a7d80 2024-05-14T09:37:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.536+0000 7f270ed9d640 1 --2- 172.21.15.2:0/2604852188 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cd94a7d80 0x556cd98a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.536+0000 7f26f5ffb640 10 client.15864 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.536+0000 7f270ed9d640 1 --2- 172.21.15.2:0/2604852188 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cd94a7d80 0x556cd98a20c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85be423d04e6b15d server_cookie=5ab61e856a4486bc in_seq=0 out_seq=0 2024-05-14T09:37:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.537+0000 7f26f5ffb640 1 -- 172.21.15.2:0/2604852188 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556cd97fd1e0 con 0x556cd94a7d80 2024-05-14T09:37:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.537+0000 7f26f5ffb640 10 client.15864 handle_command_reply: tid=0 2024-05-14T09:37:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.537+0000 7f271bc30740 1 client.15864 shutdown 2024-05-14T09:37:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.537+0000 7f271bc30740 1 -- 172.21.15.2:0/2604852188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c84d0 msgr2=0x556cd98949d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.537+0000 7f271bc30740 1 --2- 172.21.15.2:0/2604852188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c84d0 0x556cd98949d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2704013710 tx=0x7f2704013b10 comp rx=0 tx=0).stop 2024-05-14T09:37:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.537+0000 7f271bc30740 1 -- 172.21.15.2:0/2604852188 shutdown_connections 2024-05-14T09:37:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.537+0000 7f271bc30740 1 --2- 172.21.15.2:0/2604852188 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cd94a7d80 0x556cd98a20c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.537+0000 7f271bc30740 1 --2- 172.21.15.2:0/2604852188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94e91b0 0x556cd989b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.537+0000 7f271bc30740 1 --2- 172.21.15.2:0/2604852188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c84d0 0x556cd98949d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.537+0000 7f26f5ffb640 0 client.15864 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.537+0000 7f271bc30740 1 --2- 172.21.15.2:0/2604852188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94c02f0 0x556cd9894490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.537+0000 7f271bc30740 1 -- 172.21.15.2:0/2604852188 >> 172.21.15.2:0/2604852188 conn(0x556cd94dd300 msgr2=0x556cd98853a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.537+0000 7f271bc30740 1 -- 172.21.15.2:0/2604852188 shutdown_connections 2024-05-14T09:37:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.537+0000 7f271bc30740 1 -- 172.21.15.2:0/2604852188 wait complete. 2024-05-14T09:37:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.537+0000 7f271bc30740 20 client.15864 trim_cache size 0 max 16384 2024-05-14T09:37:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f271bc30740 1 Processor -- start 2024-05-14T09:37:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f271bc30740 1 -- start start 2024-05-14T09:37:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f271bc30740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94c02f0 0x556cd988eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f271bc30740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c84d0 0x556cd988efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f271bc30740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94e91b0 0x556cd9885cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f271bc30740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f27101af4e0 con 0x556cd94c84d0 2024-05-14T09:37:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f271bc30740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f27101acdb0 con 0x556cd94c02f0 2024-05-14T09:37:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f271bc30740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f27101aac00 con 0x556cd94e91b0 2024-05-14T09:37:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c84d0 0x556cd988efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c84d0 0x556cd988efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48016/0 (socket says 172.21.15.2:48016) 2024-05-14T09:37:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f270dd9b640 1 -- 172.21.15.2:0/3162209876 learned_addr learned my addr 172.21.15.2:0/3162209876 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f270ed9d640 1 --2- 172.21.15.2:0/3162209876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94c02f0 0x556cd988eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f270ed9d640 1 --2- 172.21.15.2:0/3162209876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94e91b0 0x556cd9885cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f270dd9b640 1 -- 172.21.15.2:0/3162209876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94e91b0 msgr2=0x556cd9885cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f270dd9b640 1 --2- 172.21.15.2:0/3162209876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94e91b0 0x556cd9885cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f270dd9b640 1 -- 172.21.15.2:0/3162209876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94c02f0 msgr2=0x556cd988eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f270dd9b640 1 --2- 172.21.15.2:0/3162209876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94c02f0 0x556cd988eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f270dd9b640 1 -- 172.21.15.2:0/3162209876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cd9886d00 con 0x556cd94c84d0 2024-05-14T09:37:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f270ed9d640 1 --2- 172.21.15.2:0/3162209876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94e91b0 0x556cd9885cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:37:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.553+0000 7f270dd9b640 1 --2- 172.21.15.2:0/3162209876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c84d0 0x556cd988efe0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2704009d40 tx=0x7f270401b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc5e76df6a38525 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.554+0000 7f26f7fff640 1 -- 172.21.15.2:0/3162209876 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270402c020 con 0x556cd94c84d0 2024-05-14T09:37:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.554+0000 7f26f7fff640 1 -- 172.21.15.2:0/3162209876 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27040054e0 con 0x556cd94c84d0 2024-05-14T09:37:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.554+0000 7f26f7fff640 1 -- 172.21.15.2:0/3162209876 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704028480 con 0x556cd94c84d0 2024-05-14T09:37:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.554+0000 7f271bc30740 1 -- 172.21.15.2:0/3162209876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c84d0 msgr2=0x556cd988efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.554+0000 7f271bc30740 1 --2- 172.21.15.2:0/3162209876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c84d0 0x556cd988efe0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2704009d40 tx=0x7f270401b3f0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.554+0000 7f271bc30740 1 -- 172.21.15.2:0/3162209876 shutdown_connections 2024-05-14T09:37:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.554+0000 7f271bc30740 1 --2- 172.21.15.2:0/3162209876 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94e91b0 0x556cd9885cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.554+0000 7f271bc30740 1 --2- 172.21.15.2:0/3162209876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94c84d0 0x556cd988efe0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.554+0000 7f271bc30740 1 --2- 172.21.15.2:0/3162209876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94c02f0 0x556cd988eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.554+0000 7f271bc30740 1 -- 172.21.15.2:0/3162209876 >> 172.21.15.2:0/3162209876 conn(0x556cd94dd300 msgr2=0x556cd989bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f271bc30740 1 -- 172.21.15.2:0/3162209876 shutdown_connections 2024-05-14T09:37:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f271bc30740 1 -- 172.21.15.2:0/3162209876 wait complete. 2024-05-14T09:37:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f271bc30740 1 -- start start 2024-05-14T09:37:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f271bc30740 1 Processor -- start 2024-05-14T09:37:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f271bc30740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94c02f0 0x556cd99058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f271bc30740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94c84d0 0x556cd9905df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f271bc30740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94e91b0 0x556cd990cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f271bc30740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f27101ab270 con 0x556cd94e91b0 2024-05-14T09:37:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f271bc30740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f27101ac6e0 con 0x556cd94c84d0 2024-05-14T09:37:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f271bc30740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f27101ab940 con 0x556cd94c02f0 2024-05-14T09:37:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f271bc30740 10 client.0 _ll_register_callbacks cb 0x7f27040097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f270ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94c02f0 0x556cd99058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94c84d0 0x556cd9905df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f270ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94e91b0 0x556cd990cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94c84d0 0x556cd9905df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:37146/0 (socket says 172.21.15.2:37146) 2024-05-14T09:37:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.555+0000 7f270dd9b640 1 -- 172.21.15.2:0/2535909282 learned_addr learned my addr 172.21.15.2:0/2535909282 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.556+0000 7f270dd9b640 1 -- 172.21.15.2:0/2535909282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94c02f0 msgr2=0x556cd99058b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.556+0000 7f270dd9b640 1 --2- 172.21.15.2:0/2535909282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94c02f0 0x556cd99058b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.556+0000 7f270dd9b640 1 -- 172.21.15.2:0/2535909282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94e91b0 msgr2=0x556cd990cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.556+0000 7f270dd9b640 1 --2- 172.21.15.2:0/2535909282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94e91b0 0x556cd990cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.556+0000 7f270dd9b640 1 -- 172.21.15.2:0/2535909282 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cd990d310 con 0x556cd94c84d0 2024-05-14T09:37:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.556+0000 7f270ed9d640 1 --2- 172.21.15.2:0/2535909282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94e91b0 0x556cd990cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.556+0000 7f270dd9b640 1 --2- 172.21.15.2:0/2535909282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94c84d0 0x556cd9905df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2704009350 tx=0x7f2704005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff219698efc9ab10 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.556+0000 7f26f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:37:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.556+0000 7f26f5ffb640 1 -- 172.21.15.2:0/2535909282 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704005240 con 0x556cd94c84d0 2024-05-14T09:37:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.556+0000 7f271bc30740 1 -- 172.21.15.2:0/2535909282 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27040054e0 con 0x556cd94c84d0 2024-05-14T09:37:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.556+0000 7f26f5ffb640 1 -- 172.21.15.2:0/2535909282 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f2704028480 con 0x556cd94c84d0 2024-05-14T09:37:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.556+0000 7f26f5ffb640 1 -- 172.21.15.2:0/2535909282 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2704042050 con 0x556cd94c84d0 2024-05-14T09:37:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.556+0000 7f271bc30740 10 client.25559 fetch_fsmap learned FSMap version 29 2024-05-14T09:37:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.556+0000 7f271bc30740 1 -- 172.21.15.2:0/2535909282 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556cd990e5a0 con 0x556cd94c84d0 2024-05-14T09:37:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.557+0000 7f26f5ffb640 1 -- 172.21.15.2:0/2535909282 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27040054e0 con 0x556cd94c84d0 2024-05-14T09:37:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.557+0000 7f26f5ffb640 1 -- 172.21.15.2:0/2535909282 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f270401a040 con 0x556cd94c84d0 2024-05-14T09:37:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.557+0000 7f271bc30740 10 client.25559 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:37:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.557+0000 7f271bc30740 10 client.25559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.557+0000 7f271bc30740 20 client.25559 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.557+0000 7f271bc30740 1 --2- 172.21.15.2:0/2535909282 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cd947d1c0 0x556cd9910ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.557+0000 7f271bc30740 4 client.25559 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:37:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.557+0000 7f271bc30740 1 -- 172.21.15.2:0/2535909282 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556cd97f7050 con 0x556cd947d1c0 2024-05-14T09:37:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.557+0000 7f270ed9d640 1 --2- 172.21.15.2:0/2535909282 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cd947d1c0 0x556cd9910ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.557+0000 7f26f5ffb640 10 client.25559 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.557+0000 7f270ed9d640 1 --2- 172.21.15.2:0/2535909282 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cd947d1c0 0x556cd9910ee0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8fa594ff415823d server_cookie=e84a2c989a46adfd in_seq=0 out_seq=0 2024-05-14T09:37:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.644+0000 7f26f5ffb640 1 -- 172.21.15.2:0/2535909282 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556cd97f7050 con 0x556cd947d1c0 2024-05-14T09:37:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.644+0000 7f26f5ffb640 10 client.25559 handle_command_reply: tid=0 2024-05-14T09:37:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.644+0000 7f271bc30740 1 client.25559 shutdown 2024-05-14T09:37:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.644+0000 7f271bc30740 1 -- 172.21.15.2:0/2535909282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94c84d0 msgr2=0x556cd9905df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.644+0000 7f271bc30740 1 --2- 172.21.15.2:0/2535909282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94c84d0 0x556cd9905df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2704009350 tx=0x7f2704005700 comp rx=0 tx=0).stop 2024-05-14T09:37:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.644+0000 7f271bc30740 1 -- 172.21.15.2:0/2535909282 shutdown_connections 2024-05-14T09:37:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.644+0000 7f271bc30740 1 --2- 172.21.15.2:0/2535909282 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cd947d1c0 0x556cd9910ee0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.644+0000 7f271bc30740 1 --2- 172.21.15.2:0/2535909282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cd94e91b0 0x556cd990cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.644+0000 7f271bc30740 1 --2- 172.21.15.2:0/2535909282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cd94c84d0 0x556cd9905df0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.644+0000 7f271bc30740 1 --2- 172.21.15.2:0/2535909282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cd94c02f0 0x556cd99058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.644+0000 7f26f5ffb640 0 client.25559 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.644+0000 7f271bc30740 1 -- 172.21.15.2:0/2535909282 >> 172.21.15.2:0/2535909282 conn(0x556cd94dd300 msgr2=0x556cd9885cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.644+0000 7f271bc30740 1 -- 172.21.15.2:0/2535909282 shutdown_connections 2024-05-14T09:37:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.644+0000 7f271bc30740 1 -- 172.21.15.2:0/2535909282 wait complete. 2024-05-14T09:37:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.644+0000 7f271bc30740 20 client.25559 trim_cache size 0 max 16384 2024-05-14T09:37:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:37:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:37:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:37:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:37:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.645+0000 7f2714d7e640 1 -- 172.21.15.2:0/3643224645 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f26f80b28a0 msgr2=0x7f26f80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.645+0000 7f2714d7e640 1 --2- 172.21.15.2:0/3643224645 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f26f80b28a0 0x7f26f80b4db0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f27000e0430 tx=0x7f2700003a10 comp rx=0 tx=0).stop 2024-05-14T09:37:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.645+0000 7f2714d7e640 1 -- 172.21.15.2:0/3643224645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2710102b70 msgr2=0x7f271019a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.645+0000 7f2714d7e640 1 --2- 172.21.15.2:0/3643224645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2710102b70 0x7f271019a270 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f270000e9f0 tx=0x7f270000eeb0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.645+0000 7f2714d7e640 1 -- 172.21.15.2:0/3643224645 shutdown_connections 2024-05-14T09:37:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.645+0000 7f2714d7e640 1 --2- 172.21.15.2:0/3643224645 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f26f80b28a0 0x7f26f80b4db0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.645+0000 7f2714d7e640 1 --2- 172.21.15.2:0/3643224645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2710108ab0 0x7f27101a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.645+0000 7f2714d7e640 1 --2- 172.21.15.2:0/3643224645 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2710103db0 0x7f271019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.645+0000 7f2714d7e640 1 --2- 172.21.15.2:0/3643224645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2710102b70 0x7f271019a270 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.645+0000 7f2714d7e640 1 -- 172.21.15.2:0/3643224645 >> 172.21.15.2:0/3643224645 conn(0x7f27100fdd60 msgr2=0x7f2710104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.645+0000 7f2714d7e640 1 -- 172.21.15.2:0/3643224645 shutdown_connections 2024-05-14T09:37:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:27.645+0000 7f2714d7e640 1 -- 172.21.15.2:0/3643224645 wait complete. 2024-05-14T09:37:28.255 INFO:tasks.workunit.client.0.smithi002.stdout: 1 117246 12.05 MB/sec warmup 54 sec latency 43.813 ms 2024-05-14T09:37:28.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:28 smithi002 ceph-mon[31557]: pgmap v499: 129 pgs: 129 active+clean; 40 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 340 op/s 2024-05-14T09:37:28.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:28 smithi029 ceph-mon[36249]: pgmap v499: 129 pgs: 129 active+clean; 40 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 340 op/s 2024-05-14T09:37:28.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:28 smithi026 ceph-mon[36056]: pgmap v499: 129 pgs: 129 active+clean; 40 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 340 op/s 2024-05-14T09:37:29.255 INFO:tasks.workunit.client.0.smithi002.stdout: 1 118782 11.95 MB/sec warmup 55 sec latency 14.078 ms 2024-05-14T09:37:30.255 INFO:tasks.workunit.client.0.smithi002.stdout: 1 121009 11.98 MB/sec warmup 56 sec latency 10.990 ms 2024-05-14T09:37:30.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:30 smithi002 ceph-mon[31557]: pgmap v500: 129 pgs: 129 active+clean; 41 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 254 op/s 2024-05-14T09:37:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:30 smithi029 ceph-mon[36249]: pgmap v500: 129 pgs: 129 active+clean; 41 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 254 op/s 2024-05-14T09:37:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:30 smithi026 ceph-mon[36056]: pgmap v500: 129 pgs: 129 active+clean; 41 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 254 op/s 2024-05-14T09:37:31.255 INFO:tasks.workunit.client.0.smithi002.stdout: 1 122449 11.87 MB/sec warmup 57 sec latency 153.049 ms 2024-05-14T09:37:32.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 124389 11.89 MB/sec warmup 58 sec latency 76.977 ms 2024-05-14T09:37:32.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:32 smithi002 ceph-mon[31557]: pgmap v501: 129 pgs: 129 active+clean; 39 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 272 op/s 2024-05-14T09:37:32.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:32 smithi029 ceph-mon[36249]: pgmap v501: 129 pgs: 129 active+clean; 39 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 272 op/s 2024-05-14T09:37:32.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:32 smithi026 ceph-mon[36056]: pgmap v501: 129 pgs: 129 active+clean; 39 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 272 op/s 2024-05-14T09:37:33.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 125835 11.79 MB/sec warmup 59 sec latency 14.906 ms 2024-05-14T09:37:34.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 128110 11.82 MB/sec warmup 60 sec latency 11.430 ms 2024-05-14T09:37:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:34 smithi002 ceph-mon[31557]: pgmap v502: 129 pgs: 129 active+clean; 44 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 318 op/s 2024-05-14T09:37:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:34 smithi029 ceph-mon[36249]: pgmap v502: 129 pgs: 129 active+clean; 44 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 318 op/s 2024-05-14T09:37:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:34 smithi026 ceph-mon[36056]: pgmap v502: 129 pgs: 129 active+clean; 44 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 318 op/s 2024-05-14T09:37:35.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 130181 11.74 MB/sec warmup 61 sec latency 13.199 ms 2024-05-14T09:37:36.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 131852 11.75 MB/sec warmup 62 sec latency 13.596 ms 2024-05-14T09:37:36.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:36 smithi002 ceph-mon[31557]: pgmap v503: 129 pgs: 129 active+clean; 44 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 268 op/s 2024-05-14T09:37:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:36 smithi029 ceph-mon[36249]: pgmap v503: 129 pgs: 129 active+clean; 44 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 268 op/s 2024-05-14T09:37:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:36 smithi026 ceph-mon[36056]: pgmap v503: 129 pgs: 129 active+clean; 44 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 268 op/s 2024-05-14T09:37:37.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 133881 11.69 MB/sec warmup 63 sec latency 40.294 ms 2024-05-14T09:37:38.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 135729 11.69 MB/sec warmup 64 sec latency 13.591 ms 2024-05-14T09:37:38.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:38 smithi002 ceph-mon[31557]: pgmap v504: 129 pgs: 129 active+clean; 45 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 336 op/s 2024-05-14T09:37:38.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:38 smithi029 ceph-mon[36249]: pgmap v504: 129 pgs: 129 active+clean; 45 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 336 op/s 2024-05-14T09:37:38.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:38 smithi026 ceph-mon[36056]: pgmap v504: 129 pgs: 129 active+clean; 45 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 336 op/s 2024-05-14T09:37:39.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 138473 11.77 MB/sec warmup 65 sec latency 13.508 ms 2024-05-14T09:37:40.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 139777 11.69 MB/sec warmup 66 sec latency 22.288 ms 2024-05-14T09:37:40.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:40 smithi002 ceph-mon[31557]: pgmap v505: 129 pgs: 129 active+clean; 47 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 329 op/s 2024-05-14T09:37:40.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:40 smithi029 ceph-mon[36249]: pgmap v505: 129 pgs: 129 active+clean; 47 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 329 op/s 2024-05-14T09:37:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:40 smithi026 ceph-mon[36056]: pgmap v505: 129 pgs: 129 active+clean; 47 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 329 op/s 2024-05-14T09:37:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.747+0000 7f580eac6640 1 -- 172.21.15.2:0/748980436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5808109ba0 msgr2=0x7f580810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.747+0000 7f580eac6640 1 --2- 172.21.15.2:0/748980436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5808109ba0 0x7f580810bfc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5800009ca0 tx=0x7f580002a8c0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.747+0000 7f580eac6640 1 -- 172.21.15.2:0/748980436 shutdown_connections 2024-05-14T09:37:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.747+0000 7f580eac6640 1 --2- 172.21.15.2:0/748980436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5808109ba0 0x7f580810bfc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.747+0000 7f580eac6640 1 --2- 172.21.15.2:0/748980436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5808104ea0 0x7f58081052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.747+0000 7f580eac6640 1 --2- 172.21.15.2:0/748980436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5808103c60 0x7f5808104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.747+0000 7f580eac6640 1 -- 172.21.15.2:0/748980436 >> 172.21.15.2:0/748980436 conn(0x7f58080fee70 msgr2=0x7f5808101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.747+0000 7f580eac6640 1 -- 172.21.15.2:0/748980436 shutdown_connections 2024-05-14T09:37:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.747+0000 7f580eac6640 1 -- 172.21.15.2:0/748980436 wait complete. 2024-05-14T09:37:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.748+0000 7f580eac6640 1 Processor -- start 2024-05-14T09:37:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.748+0000 7f580eac6640 1 -- start start 2024-05-14T09:37:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.748+0000 7f580eac6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5808103c60 0x7f5808077a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.748+0000 7f580eac6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5808104ea0 0x7f5808076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.748+0000 7f580eac6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5808109ba0 0x7f5808076670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.748+0000 7f580eac6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f580810f790 con 0x7f5808103c60 2024-05-14T09:37:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.748+0000 7f580eac6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f580810f610 con 0x7f5808109ba0 2024-05-14T09:37:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.748+0000 7f580eac6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f580810f910 con 0x7f5808104ea0 2024-05-14T09:37:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.748+0000 7f580c862640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5808103c60 0x7f5808077a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.748+0000 7f580c862640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5808103c60 0x7f5808077a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57936/0 (socket says 172.21.15.2:57936) 2024-05-14T09:37:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.748+0000 7f580c862640 1 -- 172.21.15.2:0/1798534583 learned_addr learned my addr 172.21.15.2:0/1798534583 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.749+0000 7f5807fff640 1 --2- 172.21.15.2:0/1798534583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5808104ea0 0x7f5808076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.749+0000 7f580d063640 1 --2- 172.21.15.2:0/1798534583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5808109ba0 0x7f5808076670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.749+0000 7f580c862640 1 -- 172.21.15.2:0/1798534583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5808104ea0 msgr2=0x7f5808076130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.749+0000 7f580c862640 1 --2- 172.21.15.2:0/1798534583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5808104ea0 0x7f5808076130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.749+0000 7f580c862640 1 -- 172.21.15.2:0/1798534583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5808109ba0 msgr2=0x7f5808076670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.749+0000 7f580c862640 1 --2- 172.21.15.2:0/1798534583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5808109ba0 0x7f5808076670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.749+0000 7f580c862640 1 -- 172.21.15.2:0/1798534583 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5808076da0 con 0x7f5808103c60 2024-05-14T09:37:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.749+0000 7f580c862640 1 --2- 172.21.15.2:0/1798534583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5808103c60 0x7f5808077a90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00e9f0 tx=0x7f57fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.749+0000 7f5805ffb640 1 -- 172.21.15.2:0/1798534583 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc00ccc0 con 0x7f5808103c60 2024-05-14T09:37:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.749+0000 7f580eac6640 1 -- 172.21.15.2:0/1798534583 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58081a4300 con 0x7f5808103c60 2024-05-14T09:37:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.749+0000 7f5805ffb640 1 -- 172.21.15.2:0/1798534583 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57fc0035f0 con 0x7f5808103c60 2024-05-14T09:37:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.749+0000 7f5805ffb640 1 -- 172.21.15.2:0/1798534583 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc00ce60 con 0x7f5808103c60 2024-05-14T09:37:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.749+0000 7f580eac6640 1 -- 172.21.15.2:0/1798534583 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58081a47c0 con 0x7f5808103c60 2024-05-14T09:37:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.752+0000 7f5805ffb640 1 -- 172.21.15.2:0/1798534583 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f57fc003aa0 con 0x7f5808103c60 2024-05-14T09:37:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.752+0000 7f5815976740 1 Processor -- start 2024-05-14T09:37:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.752+0000 7f5815976740 1 -- start start 2024-05-14T09:37:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.752+0000 7f5815976740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3c81b0 0x55e62c769a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.752+0000 7f5815976740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c39f2f0 0x55e62c769fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.752+0000 7f5815976740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c3a74d0 0x55e62c770ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.752+0000 7f5815976740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e62c6ce5d0 con 0x55e62c39f2f0 2024-05-14T09:37:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.752+0000 7f5815976740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e62c687890 con 0x55e62c3c81b0 2024-05-14T09:37:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.752+0000 7f5815976740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e62c6ca590 con 0x55e62c3a74d0 2024-05-14T09:37:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.752+0000 7f580d063640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3c81b0 0x55e62c769a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.752+0000 7f580c862640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c39f2f0 0x55e62c769fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.752+0000 7f580d063640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3c81b0 0x55e62c769a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:58032/0 (socket says 172.21.15.2:58032) 2024-05-14T09:37:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.753+0000 7f580d063640 1 -- 172.21.15.2:0/613631012 learned_addr learned my addr 172.21.15.2:0/613631012 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.753+0000 7f5807fff640 1 --2- 172.21.15.2:0/613631012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c3a74d0 0x55e62c770ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.753+0000 7f580d063640 1 -- 172.21.15.2:0/613631012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c3a74d0 msgr2=0x55e62c770ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.753+0000 7f580d063640 1 --2- 172.21.15.2:0/613631012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c3a74d0 0x55e62c770ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.753+0000 7f580d063640 1 -- 172.21.15.2:0/613631012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c39f2f0 msgr2=0x55e62c769fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.753+0000 7f580d063640 1 --2- 172.21.15.2:0/613631012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c39f2f0 0x55e62c769fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.753+0000 7f580d063640 1 -- 172.21.15.2:0/613631012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e62c3e0210 con 0x55e62c3c81b0 2024-05-14T09:37:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.753+0000 7f5805ffb640 1 --2- 172.21.15.2:0/1798534583 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f57f80b28a0 0x7f57f80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.753+0000 7f580d063640 1 --2- 172.21.15.2:0/613631012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3c81b0 0x55e62c769a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f580002f390 tx=0x7f580002f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ac6afd0096a887 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.753+0000 7f580c862640 1 --2- 172.21.15.2:0/1798534583 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f57f80b28a0 0x7f57f80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.753+0000 7f57f2ffd640 1 -- 172.21.15.2:0/613631012 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800031650 con 0x55e62c3c81b0 2024-05-14T09:37:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.753+0000 7f5805ffb640 1 -- 172.21.15.2:0/1798534583 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f57fc0e29c0 con 0x7f5808103c60 2024-05-14T09:37:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.753+0000 7f57f2ffd640 1 -- 172.21.15.2:0/613631012 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5800036040 con 0x55e62c3c81b0 2024-05-14T09:37:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.753+0000 7f57f2ffd640 1 -- 172.21.15.2:0/613631012 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58000046e0 con 0x55e62c3c81b0 2024-05-14T09:37:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.754+0000 7f5815976740 1 -- 172.21.15.2:0/613631012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3c81b0 msgr2=0x55e62c769a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.754+0000 7f5815976740 1 --2- 172.21.15.2:0/613631012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3c81b0 0x55e62c769a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f580002f390 tx=0x7f580002f3c0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.754+0000 7f580c862640 1 --2- 172.21.15.2:0/1798534583 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f57f80b28a0 0x7f57f80b4db0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55e62c76ae40 tx=0x7f57fc0e8550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.754+0000 7f5815976740 1 -- 172.21.15.2:0/613631012 shutdown_connections 2024-05-14T09:37:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.754+0000 7f5815976740 1 --2- 172.21.15.2:0/613631012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c3a74d0 0x55e62c770ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.754+0000 7f5815976740 1 --2- 172.21.15.2:0/613631012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c39f2f0 0x55e62c769fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.754+0000 7f5815976740 1 --2- 172.21.15.2:0/613631012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3c81b0 0x55e62c769a60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.754+0000 7f5815976740 1 -- 172.21.15.2:0/613631012 >> 172.21.15.2:0/613631012 conn(0x55e62c3bc300 msgr2=0x55e62c7653a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.754+0000 7f5815976740 1 -- 172.21.15.2:0/613631012 shutdown_connections 2024-05-14T09:37:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.754+0000 7f5815976740 1 -- 172.21.15.2:0/613631012 wait complete. 2024-05-14T09:37:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.755+0000 7f5815976740 1 -- start start 2024-05-14T09:37:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.755+0000 7f5815976740 1 Processor -- start 2024-05-14T09:37:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.755+0000 7f5815976740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c39f2f0 0x55e62c773610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.755+0000 7f5815976740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c3a74d0 0x55e62c773b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.755+0000 7f5815976740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c3c81b0 0x55e62c77aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.755+0000 7f5815976740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e62c7718d0 con 0x55e62c3a74d0 2024-05-14T09:37:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.755+0000 7f5815976740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e62c68b370 con 0x55e62c39f2f0 2024-05-14T09:37:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.755+0000 7f5815976740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e62c771a50 con 0x55e62c3c81b0 2024-05-14T09:37:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.755+0000 7f5815976740 10 client.0 _ll_register_callbacks cb 0x55e62c3dfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f5807fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c3a74d0 0x55e62c773b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f580d063640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c39f2f0 0x55e62c773610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f5807fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c3a74d0 0x55e62c773b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57950/0 (socket says 172.21.15.2:57950) 2024-05-14T09:37:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f5807fff640 1 -- 172.21.15.2:0/2093949210 learned_addr learned my addr 172.21.15.2:0/2093949210 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f580d063640 1 --2- 172.21.15.2:0/2093949210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c3c81b0 0x55e62c77aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f5807fff640 1 -- 172.21.15.2:0/2093949210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c3c81b0 msgr2=0x55e62c77aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f5807fff640 1 --2- 172.21.15.2:0/2093949210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c3c81b0 0x55e62c77aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f5807fff640 1 -- 172.21.15.2:0/2093949210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c39f2f0 msgr2=0x55e62c773610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f5807fff640 1 --2- 172.21.15.2:0/2093949210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c39f2f0 0x55e62c773610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f5807fff640 1 -- 172.21.15.2:0/2093949210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e62c4ac620 con 0x55e62c3a74d0 2024-05-14T09:37:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f5807fff640 1 --2- 172.21.15.2:0/2093949210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c3a74d0 0x55e62c773b50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f57f4012720 tx=0x7f57f4012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7165f293f2c356e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f57e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:37:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f57e7fff640 1 -- 172.21.15.2:0/2093949210 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4004030 con 0x55e62c3a74d0 2024-05-14T09:37:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f57e7fff640 1 -- 172.21.15.2:0/2093949210 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f57f40041d0 con 0x55e62c3a74d0 2024-05-14T09:37:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f5815976740 1 -- 172.21.15.2:0/2093949210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5800036040 con 0x55e62c3a74d0 2024-05-14T09:37:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.756+0000 7f57e7fff640 1 -- 172.21.15.2:0/2093949210 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f57f4025050 con 0x55e62c3a74d0 2024-05-14T09:37:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.757+0000 7f5815976740 10 client.15882 fetch_fsmap learned FSMap version 29 2024-05-14T09:37:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.757+0000 7f5815976740 1 -- 172.21.15.2:0/2093949210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e62c77dab0 con 0x55e62c3a74d0 2024-05-14T09:37:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.757+0000 7f57e7fff640 1 -- 172.21.15.2:0/2093949210 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5800036040 con 0x55e62c3a74d0 2024-05-14T09:37:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.757+0000 7f57e7fff640 1 -- 172.21.15.2:0/2093949210 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f57f4003990 con 0x55e62c3a74d0 2024-05-14T09:37:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.757+0000 7f5815976740 10 client.15882 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:37:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.757+0000 7f5815976740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.757+0000 7f5815976740 20 client.15882 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.757+0000 7f5815976740 1 --2- 172.21.15.2:0/2093949210 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c386d80 0x55e62c7812c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.757+0000 7f5815976740 4 client.15882 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:37:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.757+0000 7f5815976740 1 -- 172.21.15.2:0/2093949210 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e62c6dc1e0 con 0x55e62c386d80 2024-05-14T09:37:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.757+0000 7f580d063640 1 --2- 172.21.15.2:0/2093949210 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c386d80 0x55e62c7812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.758+0000 7f57e7fff640 10 client.15882 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.758+0000 7f580d063640 1 --2- 172.21.15.2:0/2093949210 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c386d80 0x55e62c7812c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5435f707f503dd4a server_cookie=1150ef1e6bc20dbb in_seq=0 out_seq=0 2024-05-14T09:37:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.758+0000 7f57e7fff640 1 -- 172.21.15.2:0/2093949210 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e62c6dc1e0 con 0x55e62c386d80 2024-05-14T09:37:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.758+0000 7f57e7fff640 10 client.15882 handle_command_reply: tid=0 2024-05-14T09:37:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.758+0000 7f5815976740 1 client.15882 shutdown 2024-05-14T09:37:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.759+0000 7f5815976740 1 -- 172.21.15.2:0/2093949210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c3a74d0 msgr2=0x55e62c773b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.759+0000 7f5815976740 1 --2- 172.21.15.2:0/2093949210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c3a74d0 0x55e62c773b50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f57f4012720 tx=0x7f57f4012be0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.759+0000 7f5815976740 1 -- 172.21.15.2:0/2093949210 shutdown_connections 2024-05-14T09:37:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.759+0000 7f5815976740 1 --2- 172.21.15.2:0/2093949210 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c386d80 0x55e62c7812c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.759+0000 7f5815976740 1 --2- 172.21.15.2:0/2093949210 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c3c81b0 0x55e62c77aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.759+0000 7f5815976740 1 --2- 172.21.15.2:0/2093949210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c3a74d0 0x55e62c773b50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.759+0000 7f57e7fff640 0 client.15882 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.759+0000 7f5815976740 1 --2- 172.21.15.2:0/2093949210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c39f2f0 0x55e62c773610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.759+0000 7f5815976740 1 -- 172.21.15.2:0/2093949210 >> 172.21.15.2:0/2093949210 conn(0x55e62c3bc300 msgr2=0x55e62c7643a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.759+0000 7f5815976740 1 -- 172.21.15.2:0/2093949210 shutdown_connections 2024-05-14T09:37:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.759+0000 7f5815976740 1 -- 172.21.15.2:0/2093949210 wait complete. 2024-05-14T09:37:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.759+0000 7f5815976740 20 client.15882 trim_cache size 0 max 16384 2024-05-14T09:37:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.776+0000 7f5815976740 1 Processor -- start 2024-05-14T09:37:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.776+0000 7f5815976740 1 -- start start 2024-05-14T09:37:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.776+0000 7f5815976740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c39f2f0 0x55e62c764cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.776+0000 7f5815976740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c3a74d0 0x55e62c765230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.776+0000 7f5815976740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3c81b0 0x55e62c77a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.776+0000 7f5815976740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f58081b1740 con 0x55e62c39f2f0 2024-05-14T09:37:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.776+0000 7f5815976740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f58081af010 con 0x55e62c3c81b0 2024-05-14T09:37:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.776+0000 7f5815976740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f58081ace60 con 0x55e62c3a74d0 2024-05-14T09:37:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.776+0000 7f580d063640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c39f2f0 0x55e62c764cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.776+0000 7f580d063640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c39f2f0 0x55e62c764cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57966/0 (socket says 172.21.15.2:57966) 2024-05-14T09:37:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.777+0000 7f580d063640 1 -- 172.21.15.2:0/4087973025 learned_addr learned my addr 172.21.15.2:0/4087973025 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.777+0000 7f580d063640 1 --2- 172.21.15.2:0/4087973025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3c81b0 0x55e62c77a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.777+0000 7f5807fff640 1 --2- 172.21.15.2:0/4087973025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c3a74d0 0x55e62c765230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.777+0000 7f580d063640 1 -- 172.21.15.2:0/4087973025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c3a74d0 msgr2=0x55e62c765230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.777+0000 7f580d063640 1 --2- 172.21.15.2:0/4087973025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c3a74d0 0x55e62c765230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.777+0000 7f580d063640 1 -- 172.21.15.2:0/4087973025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3c81b0 msgr2=0x55e62c77a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.777+0000 7f580d063640 1 --2- 172.21.15.2:0/4087973025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3c81b0 0x55e62c77a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.777+0000 7f580d063640 1 -- 172.21.15.2:0/4087973025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e62c765d00 con 0x55e62c39f2f0 2024-05-14T09:37:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.777+0000 7f580d063640 1 --2- 172.21.15.2:0/4087973025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c39f2f0 0x55e62c764cf0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5800009a60 tx=0x7f5800031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe02427c94d4ab0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.777+0000 7f57f27fc640 1 -- 172.21.15.2:0/4087973025 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800004600 con 0x55e62c39f2f0 2024-05-14T09:37:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.777+0000 7f57f27fc640 1 -- 172.21.15.2:0/4087973025 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5800036070 con 0x55e62c39f2f0 2024-05-14T09:37:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.777+0000 7f57f27fc640 1 -- 172.21.15.2:0/4087973025 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580002f660 con 0x55e62c39f2f0 2024-05-14T09:37:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.778+0000 7f5815976740 1 -- 172.21.15.2:0/4087973025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c39f2f0 msgr2=0x55e62c764cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.778+0000 7f5815976740 1 --2- 172.21.15.2:0/4087973025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c39f2f0 0x55e62c764cf0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5800009a60 tx=0x7f5800031af0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.778+0000 7f5815976740 1 -- 172.21.15.2:0/4087973025 shutdown_connections 2024-05-14T09:37:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.778+0000 7f5815976740 1 --2- 172.21.15.2:0/4087973025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3c81b0 0x55e62c77a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.778+0000 7f5815976740 1 --2- 172.21.15.2:0/4087973025 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c3a74d0 0x55e62c765230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.778+0000 7f5815976740 1 --2- 172.21.15.2:0/4087973025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c39f2f0 0x55e62c764cf0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.778+0000 7f5815976740 1 -- 172.21.15.2:0/4087973025 >> 172.21.15.2:0/4087973025 conn(0x55e62c3bc300 msgr2=0x55e62c76da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.778+0000 7f5815976740 1 -- 172.21.15.2:0/4087973025 shutdown_connections 2024-05-14T09:37:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.778+0000 7f5815976740 1 -- 172.21.15.2:0/4087973025 wait complete. 2024-05-14T09:37:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.779+0000 7f5815976740 1 -- start start 2024-05-14T09:37:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.779+0000 7f5815976740 1 Processor -- start 2024-05-14T09:37:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.779+0000 7f5815976740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c39f2f0 0x55e62c778670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.779+0000 7f5815976740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3a74d0 0x55e62c778bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.779+0000 7f5815976740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c3c81b0 0x55e62c779ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.779+0000 7f5815976740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f58081ad4d0 con 0x55e62c3c81b0 2024-05-14T09:37:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.779+0000 7f5815976740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f58081ae940 con 0x55e62c3a74d0 2024-05-14T09:37:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.779+0000 7f5815976740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f58081adba0 con 0x55e62c39f2f0 2024-05-14T09:37:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.780+0000 7f5815976740 10 client.0 _ll_register_callbacks cb 0x7f57f4009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.780+0000 7f580d063640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c39f2f0 0x55e62c778670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.780+0000 7f5807fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3a74d0 0x55e62c778bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.780+0000 7f580d063640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c3c81b0 0x55e62c779ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.780+0000 7f5807fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3a74d0 0x55e62c778bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:58054/0 (socket says 172.21.15.2:58054) 2024-05-14T09:37:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.780+0000 7f5807fff640 1 -- 172.21.15.2:0/170046577 learned_addr learned my addr 172.21.15.2:0/170046577 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.780+0000 7f5807fff640 1 -- 172.21.15.2:0/170046577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c39f2f0 msgr2=0x55e62c778670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.780+0000 7f5807fff640 1 --2- 172.21.15.2:0/170046577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c39f2f0 0x55e62c778670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.780+0000 7f5807fff640 1 -- 172.21.15.2:0/170046577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c3c81b0 msgr2=0x55e62c779ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.780+0000 7f5807fff640 1 --2- 172.21.15.2:0/170046577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c3c81b0 0x55e62c779ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.780+0000 7f5807fff640 1 -- 172.21.15.2:0/170046577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e62c77a0a0 con 0x55e62c3a74d0 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.780+0000 7f5807fff640 1 --2- 172.21.15.2:0/170046577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3a74d0 0x55e62c778bb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55e62c765ff0 tx=0x7f57f400fbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d8ba5e2c041df0c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.780+0000 7f57e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.781+0000 7f57e7fff640 1 -- 172.21.15.2:0/170046577 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4022040 con 0x55e62c3a74d0 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.781+0000 7f57e7fff640 1 -- 172.21.15.2:0/170046577 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f57f40035f0 con 0x55e62c3a74d0 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.781+0000 7f5815976740 1 -- 172.21.15.2:0/170046577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5800036070 con 0x55e62c3a74d0 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.781+0000 7f57e7fff640 1 -- 172.21.15.2:0/170046577 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f57f4037050 con 0x55e62c3a74d0 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.781+0000 7f5815976740 10 client.25577 fetch_fsmap learned FSMap version 29 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.781+0000 7f5815976740 1 -- 172.21.15.2:0/170046577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e62c77b360 con 0x55e62c3a74d0 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.781+0000 7f57e7fff640 1 -- 172.21.15.2:0/170046577 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5800036070 con 0x55e62c3a74d0 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.781+0000 7f57e7fff640 1 -- 172.21.15.2:0/170046577 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f57f4010430 con 0x55e62c3a74d0 2024-05-14T09:37:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.781+0000 7f5815976740 10 client.25577 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:37:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.781+0000 7f5815976740 10 client.25577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.781+0000 7f5815976740 20 client.25577 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.781+0000 7f5815976740 1 --2- 172.21.15.2:0/170046577 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c35c1c0 0x55e62c7ebc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.781+0000 7f5815976740 4 client.25577 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:37:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.781+0000 7f5815976740 1 -- 172.21.15.2:0/170046577 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e62c6d6050 con 0x55e62c35c1c0 2024-05-14T09:37:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.781+0000 7f580d063640 1 --2- 172.21.15.2:0/170046577 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c35c1c0 0x55e62c7ebc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.782+0000 7f57e7fff640 10 client.25577 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.782+0000 7f580d063640 1 --2- 172.21.15.2:0/170046577 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c35c1c0 0x55e62c7ebc70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8f981b10580d146 server_cookie=30314f9429fa7183 in_seq=0 out_seq=0 2024-05-14T09:37:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.864+0000 7f57e7fff640 1 -- 172.21.15.2:0/170046577 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e62c6d6050 con 0x55e62c35c1c0 2024-05-14T09:37:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.864+0000 7f57e7fff640 10 client.25577 handle_command_reply: tid=0 2024-05-14T09:37:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.864+0000 7f5815976740 1 client.25577 shutdown 2024-05-14T09:37:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.865+0000 7f5815976740 1 -- 172.21.15.2:0/170046577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3a74d0 msgr2=0x55e62c778bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.865+0000 7f5815976740 1 --2- 172.21.15.2:0/170046577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3a74d0 0x55e62c778bb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55e62c765ff0 tx=0x7f57f400fbc0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.865+0000 7f5815976740 1 -- 172.21.15.2:0/170046577 shutdown_connections 2024-05-14T09:37:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.865+0000 7f5815976740 1 --2- 172.21.15.2:0/170046577 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e62c35c1c0 0x55e62c7ebc70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.865+0000 7f5815976740 1 --2- 172.21.15.2:0/170046577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e62c3c81b0 0x55e62c779ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.865+0000 7f5815976740 1 --2- 172.21.15.2:0/170046577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e62c3a74d0 0x55e62c778bb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.865+0000 7f5815976740 1 --2- 172.21.15.2:0/170046577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e62c39f2f0 0x55e62c778670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.865+0000 7f5815976740 1 -- 172.21.15.2:0/170046577 >> 172.21.15.2:0/170046577 conn(0x55e62c3bc300 msgr2=0x55e62c764cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.865+0000 7f57e7fff640 0 client.25577 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.865+0000 7f5815976740 1 -- 172.21.15.2:0/170046577 shutdown_connections 2024-05-14T09:37:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.865+0000 7f5815976740 1 -- 172.21.15.2:0/170046577 wait complete. 2024-05-14T09:37:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.865+0000 7f5815976740 20 client.25577 trim_cache size 0 max 16384 2024-05-14T09:37:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:37:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:37:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:37:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:37:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.866+0000 7f580eac6640 1 -- 172.21.15.2:0/1798534583 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f57f80b28a0 msgr2=0x7f57f80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.866+0000 7f580eac6640 1 --2- 172.21.15.2:0/1798534583 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f57f80b28a0 0x7f57f80b4db0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55e62c76ae40 tx=0x7f57fc0e8550 comp rx=0 tx=0).stop 2024-05-14T09:37:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.866+0000 7f580eac6640 1 -- 172.21.15.2:0/1798534583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5808103c60 msgr2=0x7f5808077a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.866+0000 7f580eac6640 1 --2- 172.21.15.2:0/1798534583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5808103c60 0x7f5808077a90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00e9f0 tx=0x7f57fc00eeb0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.866+0000 7f580eac6640 1 -- 172.21.15.2:0/1798534583 shutdown_connections 2024-05-14T09:37:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.866+0000 7f580eac6640 1 --2- 172.21.15.2:0/1798534583 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f57f80b28a0 0x7f57f80b4db0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.866+0000 7f580eac6640 1 --2- 172.21.15.2:0/1798534583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5808109ba0 0x7f5808076670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.866+0000 7f580eac6640 1 --2- 172.21.15.2:0/1798534583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5808104ea0 0x7f5808076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.866+0000 7f580eac6640 1 --2- 172.21.15.2:0/1798534583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5808103c60 0x7f5808077a90 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.866+0000 7f580eac6640 1 -- 172.21.15.2:0/1798534583 >> 172.21.15.2:0/1798534583 conn(0x7f58080fee70 msgr2=0x7f5808105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.866+0000 7f580eac6640 1 -- 172.21.15.2:0/1798534583 shutdown_connections 2024-05-14T09:37:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:40.866+0000 7f580eac6640 1 -- 172.21.15.2:0/1798534583 wait complete. 2024-05-14T09:37:41.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 141726 11.66 MB/sec warmup 67 sec latency 113.969 ms 2024-05-14T09:37:42.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 142795 11.57 MB/sec warmup 68 sec latency 53.494 ms 2024-05-14T09:37:42.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:42 smithi002 ceph-mon[31557]: pgmap v506: 129 pgs: 129 active+clean; 49 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 322 op/s 2024-05-14T09:37:42.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:42 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:37:42.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:42 smithi029 ceph-mon[36249]: pgmap v506: 129 pgs: 129 active+clean; 49 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 322 op/s 2024-05-14T09:37:42.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:42 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:37:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:42 smithi026 ceph-mon[36056]: pgmap v506: 129 pgs: 129 active+clean; 49 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 322 op/s 2024-05-14T09:37:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:42 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:37:43.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 145452 11.64 MB/sec warmup 69 sec latency 22.626 ms 2024-05-14T09:37:43.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:43 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:37:43.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:43 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:37:43.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:43 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:37:43.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:43 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:37:43.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:43 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:37:43.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:43 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:37:43.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:43 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:37:43.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:43 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:37:43.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:43 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:37:44.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 146143 11.50 MB/sec warmup 70 sec latency 303.643 ms 2024-05-14T09:37:44.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:44 smithi002 ceph-mon[31557]: pgmap v507: 129 pgs: 129 active+clean; 50 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 382 op/s 2024-05-14T09:37:44.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:44 smithi029 ceph-mon[36249]: pgmap v507: 129 pgs: 129 active+clean; 50 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 382 op/s 2024-05-14T09:37:44.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:44 smithi026 ceph-mon[36056]: pgmap v507: 129 pgs: 129 active+clean; 50 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 382 op/s 2024-05-14T09:37:45.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 147017 11.41 MB/sec warmup 71 sec latency 930.830 ms 2024-05-14T09:37:45.759 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T09:37:46.038 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:37:46.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 149281 11.44 MB/sec warmup 72 sec latency 14.141 ms 2024-05-14T09:37:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:46 smithi002 ceph-mon[31557]: pgmap v508: 129 pgs: 129 active+clean; 50 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 288 op/s 2024-05-14T09:37:46.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:46 smithi029 ceph-mon[36249]: pgmap v508: 129 pgs: 129 active+clean; 50 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 288 op/s 2024-05-14T09:37:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:46 smithi026 ceph-mon[36056]: pgmap v508: 129 pgs: 129 active+clean; 50 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 288 op/s 2024-05-14T09:37:47.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 150692 11.36 MB/sec warmup 73 sec latency 80.168 ms 2024-05-14T09:37:47.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.913+0000 7f44accf1700 1 -- 172.21.15.2:0/2797797457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a81089f0 msgr2=0x7f44a8108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:47.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.913+0000 7f44accf1700 1 --2- 172.21.15.2:0/2797797457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a81089f0 0x7f44a8108e00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4498009990 tx=0x7f4498009ca0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.914+0000 7f44accf1700 1 -- 172.21.15.2:0/2797797457 shutdown_connections 2024-05-14T09:37:47.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.914+0000 7f44accf1700 1 --2- 172.21.15.2:0/2797797457 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44a8103310 0x7f44a810f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.914+0000 7f44accf1700 1 --2- 172.21.15.2:0/2797797457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a81089f0 0x7f44a8108e00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.914+0000 7f44accf1700 1 --2- 172.21.15.2:0/2797797457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a81029c0 0x7f44a8102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.914+0000 7f44accf1700 1 -- 172.21.15.2:0/2797797457 >> 172.21.15.2:0/2797797457 conn(0x7f44a80fe6f0 msgr2=0x7f44a8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:47.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.914+0000 7f44accf1700 1 -- 172.21.15.2:0/2797797457 shutdown_connections 2024-05-14T09:37:47.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.914+0000 7f44accf1700 1 -- 172.21.15.2:0/2797797457 wait complete. 2024-05-14T09:37:47.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.915+0000 7f44accf1700 1 Processor -- start 2024-05-14T09:37:47.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.915+0000 7f44accf1700 1 -- start start 2024-05-14T09:37:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.915+0000 7f44accf1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a81029c0 0x7f44a819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.915+0000 7f44accf1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a8103310 0x7f44a819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.915+0000 7f44accf1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44a81089f0 0x7f44a819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.915+0000 7f44accf1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81122f0 con 0x7f44a8103310 2024-05-14T09:37:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.915+0000 7f44accf1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8112170 con 0x7f44a81029c0 2024-05-14T09:37:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.915+0000 7f44a5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a8103310 0x7f44a819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.915+0000 7f44a5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a8103310 0x7f44a819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44208/0 (socket says 172.21.15.2:44208) 2024-05-14T09:37:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.915+0000 7f44a5d9b700 1 -- 172.21.15.2:0/872695690 learned_addr learned my addr 172.21.15.2:0/872695690 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.915+0000 7f44accf1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8112470 con 0x7f44a81089f0 2024-05-14T09:37:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.915+0000 7f44a659c700 1 --2- 172.21.15.2:0/872695690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a81029c0 0x7f44a819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.915+0000 7f44a6d9d700 1 --2- 172.21.15.2:0/872695690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44a81089f0 0x7f44a819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.916+0000 7f44a659c700 1 -- 172.21.15.2:0/872695690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44a81089f0 msgr2=0x7f44a819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.916+0000 7f44a659c700 1 --2- 172.21.15.2:0/872695690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44a81089f0 0x7f44a819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.916+0000 7f44a659c700 1 -- 172.21.15.2:0/872695690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a8103310 msgr2=0x7f44a819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.916+0000 7f44a659c700 1 --2- 172.21.15.2:0/872695690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a8103310 0x7f44a819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.916+0000 7f44a659c700 1 -- 172.21.15.2:0/872695690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a819fd30 con 0x7f44a81029c0 2024-05-14T09:37:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.916+0000 7f44a659c700 1 --2- 172.21.15.2:0/872695690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a81029c0 0x7f44a819ae60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f44980098e0 tx=0x7f449800b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.916+0000 7f44977fe700 1 -- 172.21.15.2:0/872695690 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498027070 con 0x7f44a81029c0 2024-05-14T09:37:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.916+0000 7f44accf1700 1 -- 172.21.15.2:0/872695690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a819ffc0 con 0x7f44a81029c0 2024-05-14T09:37:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.916+0000 7f44977fe700 1 -- 172.21.15.2:0/872695690 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4498022470 con 0x7f44a81029c0 2024-05-14T09:37:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.916+0000 7f44977fe700 1 -- 172.21.15.2:0/872695690 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498017650 con 0x7f44a81029c0 2024-05-14T09:37:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.916+0000 7f44accf1700 1 -- 172.21.15.2:0/872695690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a81a04c0 con 0x7f44a81029c0 2024-05-14T09:37:47.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.919+0000 7f44977fe700 1 -- 172.21.15.2:0/872695690 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f449802a070 con 0x7f44a81029c0 2024-05-14T09:37:47.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.920+0000 7f44bb705b80 1 Processor -- start 2024-05-14T09:37:47.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.922+0000 7f44bb705b80 1 -- start start 2024-05-14T09:37:47.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.922+0000 7f44977fe700 1 --2- 172.21.15.2:0/872695690 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f44900b2900 0x7f44900b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.922+0000 7f44a6d9d700 1 --2- 172.21.15.2:0/872695690 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f44900b2900 0x7f44900b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.922+0000 7f44977fe700 1 -- 172.21.15.2:0/872695690 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f44980e3240 con 0x7f44a81029c0 2024-05-14T09:37:47.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.923+0000 7f44bb705b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe95ece40 0x561fe9770840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.923+0000 7f44bb705b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe9770d80 0x561fe976a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.923+0000 7f44bb705b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe976ae10 0x561fe976b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.923+0000 7f44bb705b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561fe968c940 con 0x561fe95ece40 2024-05-14T09:37:47.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.923+0000 7f44bb705b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561fe95f1da0 con 0x561fe9770d80 2024-05-14T09:37:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.923+0000 7f44bb705b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561fe9688900 con 0x561fe976ae10 2024-05-14T09:37:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.923+0000 7f44a5d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe9770d80 0x561fe976a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.923+0000 7f44a5d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe9770d80 0x561fe976a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:47986/0 (socket says 172.21.15.2:47986) 2024-05-14T09:37:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.923+0000 7f44a5d9b700 1 -- 172.21.15.2:0/2056418976 learned_addr learned my addr 172.21.15.2:0/2056418976 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.924+0000 7f44a659c700 1 --2- 172.21.15.2:0/2056418976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe95ece40 0x561fe9770840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.924+0000 7f44a5d9b700 1 -- 172.21.15.2:0/2056418976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe976ae10 msgr2=0x561fe976b220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:37:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.924+0000 7f44a5d9b700 1 --2- 172.21.15.2:0/2056418976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe976ae10 0x561fe976b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.924+0000 7f44a5d9b700 1 -- 172.21.15.2:0/2056418976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe95ece40 msgr2=0x561fe9770840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.924+0000 7f44a5d9b700 1 --2- 172.21.15.2:0/2056418976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe95ece40 0x561fe9770840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.924+0000 7f44a5d9b700 1 -- 172.21.15.2:0/2056418976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe95d98d0 con 0x561fe9770d80 2024-05-14T09:37:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.924+0000 7f44a5d9b700 1 --2- 172.21.15.2:0/2056418976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe9770d80 0x561fe976a8d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f449c00ae20 tx=0x7f449c00a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39d8eca6b9ce9be9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.925+0000 7f4494ff9700 1 -- 172.21.15.2:0/2056418976 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c0043d0 con 0x561fe9770d80 2024-05-14T09:37:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.925+0000 7f4494ff9700 1 -- 172.21.15.2:0/2056418976 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f449c003b30 con 0x561fe9770d80 2024-05-14T09:37:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.927+0000 7f44bb705b80 1 -- 172.21.15.2:0/2056418976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe9770d80 msgr2=0x561fe976a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.927+0000 7f44a6d9d700 1 --2- 172.21.15.2:0/872695690 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f44900b2900 0x7f44900b4e10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f44a000f540 tx=0x7f44a000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.927+0000 7f4494ff9700 1 -- 172.21.15.2:0/2056418976 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c01caa0 con 0x561fe9770d80 2024-05-14T09:37:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.927+0000 7f44bb705b80 1 --2- 172.21.15.2:0/2056418976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe9770d80 0x561fe976a8d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f449c00ae20 tx=0x7f449c00a370 comp rx=0 tx=0).stop 2024-05-14T09:37:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.927+0000 7f44bb705b80 1 -- 172.21.15.2:0/2056418976 shutdown_connections 2024-05-14T09:37:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.927+0000 7f44bb705b80 1 --2- 172.21.15.2:0/2056418976 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe976ae10 0x561fe976b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.927+0000 7f44bb705b80 1 --2- 172.21.15.2:0/2056418976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe95ece40 0x561fe9770840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.927+0000 7f44bb705b80 1 --2- 172.21.15.2:0/2056418976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe9770d80 0x561fe976a8d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.927+0000 7f44bb705b80 1 -- 172.21.15.2:0/2056418976 >> 172.21.15.2:0/2056418976 conn(0x561fe9634630 msgr2=0x561fe9765a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:47.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.927+0000 7f44bb705b80 1 -- 172.21.15.2:0/2056418976 shutdown_connections 2024-05-14T09:37:47.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.927+0000 7f44bb705b80 1 -- 172.21.15.2:0/2056418976 wait complete. 2024-05-14T09:37:47.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.928+0000 7f44bb705b80 1 -- start start 2024-05-14T09:37:47.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.928+0000 7f44bb705b80 1 Processor -- start 2024-05-14T09:37:47.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.929+0000 7f44bb705b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 0x561fe9774510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.929+0000 7f44bb705b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe976ae10 0x561fe9774a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.929+0000 7f44bb705b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe9770d80 0x561fe9774f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.929+0000 7f44bb705b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561fe9777870 con 0x561fe976ae10 2024-05-14T09:37:47.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.929+0000 7f44bb705b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561fe9645ed0 con 0x561fe9770d80 2024-05-14T09:37:47.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.929+0000 7f44bb705b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561fe97779f0 con 0x561fe95ece40 2024-05-14T09:37:47.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.929+0000 7f44bb705b80 10 client.0 _ll_register_callbacks cb 0x561fe9624950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:47.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.930+0000 7f44a5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe976ae10 0x561fe9774a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.930+0000 7f44a5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe976ae10 0x561fe9774a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44226/0 (socket says 172.21.15.2:44226) 2024-05-14T09:37:47.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.930+0000 7f44a5d9b700 1 -- 172.21.15.2:0/1276422347 learned_addr learned my addr 172.21.15.2:0/1276422347 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:47.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.930+0000 7f44a6d9d700 1 --2- 172.21.15.2:0/1276422347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe9770d80 0x561fe9774f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.930+0000 7f44a659c700 1 --2- 172.21.15.2:0/1276422347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 0x561fe9774510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.930+0000 7f44a6d9d700 1 -- 172.21.15.2:0/1276422347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 msgr2=0x561fe9774510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:47.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.930+0000 7f44a6d9d700 1 --2- 172.21.15.2:0/1276422347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 0x561fe9774510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.931+0000 7f44a6d9d700 1 -- 172.21.15.2:0/1276422347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe976ae10 msgr2=0x561fe9774a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:47.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.931+0000 7f44a6d9d700 1 --2- 172.21.15.2:0/1276422347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe976ae10 0x561fe9774a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.931+0000 7f44a6d9d700 1 -- 172.21.15.2:0/1276422347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe9643760 con 0x561fe9770d80 2024-05-14T09:37:47.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.931+0000 7f44a6d9d700 1 --2- 172.21.15.2:0/1276422347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe9770d80 0x561fe9774f90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f44a000fc90 tx=0x7f44a0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3c514fa90b7a99f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:47.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.931+0000 7f448dffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:37:47.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.932+0000 7f448dffb700 1 -- 172.21.15.2:0/1276422347 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0025070 con 0x561fe9770d80 2024-05-14T09:37:47.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.932+0000 7f448dffb700 1 -- 172.21.15.2:0/1276422347 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f44a0004db0 con 0x561fe9770d80 2024-05-14T09:37:47.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.932+0000 7f44bb705b80 1 -- 172.21.15.2:0/1276422347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f449c003b30 con 0x561fe9770d80 2024-05-14T09:37:47.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.932+0000 7f448dffb700 1 -- 172.21.15.2:0/1276422347 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44a0015b80 con 0x561fe9770d80 2024-05-14T09:37:47.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.932+0000 7f44bb705b80 10 client.25589 fetch_fsmap learned FSMap version 29 2024-05-14T09:37:47.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.932+0000 7f44bb705b80 1 -- 172.21.15.2:0/1276422347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561fe9793570 con 0x561fe9770d80 2024-05-14T09:37:47.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.933+0000 7f448dffb700 1 -- 172.21.15.2:0/1276422347 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f449c003b30 con 0x561fe9770d80 2024-05-14T09:37:47.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.933+0000 7f448dffb700 1 -- 172.21.15.2:0/1276422347 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f44a0020070 con 0x561fe9770d80 2024-05-14T09:37:47.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.933+0000 7f44bb705b80 10 client.25589 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:37:47.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.933+0000 7f44bb705b80 10 client.25589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:47.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.933+0000 7f44bb705b80 20 client.25589 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:47.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.933+0000 7f44bb705b80 1 --2- 172.21.15.2:0/1276422347 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561fe9795a40 0x561fe9797e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.933+0000 7f44bb705b80 4 client.25589 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:37:47.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.933+0000 7f44bb705b80 1 -- 172.21.15.2:0/1276422347 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fe969a320 con 0x561fe9795a40 2024-05-14T09:37:47.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.934+0000 7f44a659c700 1 --2- 172.21.15.2:0/1276422347 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561fe9795a40 0x561fe9797e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.935+0000 7f448dffb700 10 client.25589 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:47.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.935+0000 7f44a659c700 1 --2- 172.21.15.2:0/1276422347 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561fe9795a40 0x561fe9797e60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79b6879a96290740 server_cookie=e331b4899bc4476d in_seq=0 out_seq=0 2024-05-14T09:37:47.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.936+0000 7f448dffb700 1 -- 172.21.15.2:0/1276422347 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561fe969a320 con 0x561fe9795a40 2024-05-14T09:37:47.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.936+0000 7f448dffb700 10 client.25589 handle_command_reply: tid=0 2024-05-14T09:37:47.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.936+0000 7f44bb705b80 1 client.25589 shutdown 2024-05-14T09:37:47.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.937+0000 7f44bb705b80 1 -- 172.21.15.2:0/1276422347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe9770d80 msgr2=0x561fe9774f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:47.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.937+0000 7f44bb705b80 1 --2- 172.21.15.2:0/1276422347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe9770d80 0x561fe9774f90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f44a000fc90 tx=0x7f44a0004ad0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.937+0000 7f44bb705b80 1 -- 172.21.15.2:0/1276422347 shutdown_connections 2024-05-14T09:37:47.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.937+0000 7f44bb705b80 1 --2- 172.21.15.2:0/1276422347 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561fe9795a40 0x561fe9797e60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.937+0000 7f44bb705b80 1 --2- 172.21.15.2:0/1276422347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe9770d80 0x561fe9774f90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.937+0000 7f44bb705b80 1 --2- 172.21.15.2:0/1276422347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 0x561fe9774510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.937+0000 7f44bb705b80 1 --2- 172.21.15.2:0/1276422347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe976ae10 0x561fe9774a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.937+0000 7f44bb705b80 1 -- 172.21.15.2:0/1276422347 >> 172.21.15.2:0/1276422347 conn(0x561fe9634630 msgr2=0x561fe97698c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:47.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.937+0000 7f448dffb700 0 client.25589 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:47.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.937+0000 7f44bb705b80 1 -- 172.21.15.2:0/1276422347 shutdown_connections 2024-05-14T09:37:47.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.938+0000 7f44bb705b80 1 -- 172.21.15.2:0/1276422347 wait complete. 2024-05-14T09:37:47.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.938+0000 7f44bb705b80 20 client.25589 trim_cache size 0 max 16384 2024-05-14T09:37:47.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.978+0000 7f44bb705b80 1 Processor -- start 2024-05-14T09:37:47.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.978+0000 7f44bb705b80 1 -- start start 2024-05-14T09:37:47.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.978+0000 7f44bb705b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 0x561fe976a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.978+0000 7f44bb705b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe976ae10 0x561fe977ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.978+0000 7f44bb705b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe9770d80 0x561fe97765f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.979+0000 7f44bb705b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81b0aa0 con 0x561fe9770d80 2024-05-14T09:37:47.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.979+0000 7f44bb705b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81ae330 con 0x561fe976ae10 2024-05-14T09:37:47.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.979+0000 7f44bb705b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81ac180 con 0x561fe95ece40 2024-05-14T09:37:47.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.979+0000 7f44a5d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe976ae10 0x561fe977ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.979+0000 7f44a659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 0x561fe976a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.979+0000 7f44a5d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe976ae10 0x561fe977ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:48010/0 (socket says 172.21.15.2:48010) 2024-05-14T09:37:47.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.979+0000 7f44a5d9b700 1 -- 172.21.15.2:0/3114317209 learned_addr learned my addr 172.21.15.2:0/3114317209 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:47.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.979+0000 7f44a6d9d700 1 --2- 172.21.15.2:0/3114317209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe9770d80 0x561fe97765f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.979+0000 7f44a5d9b700 1 -- 172.21.15.2:0/3114317209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 msgr2=0x561fe976a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:47.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.979+0000 7f44a5d9b700 1 --2- 172.21.15.2:0/3114317209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 0x561fe976a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.979+0000 7f44a5d9b700 1 -- 172.21.15.2:0/3114317209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe9770d80 msgr2=0x561fe97765f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:47.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.979+0000 7f44a5d9b700 1 --2- 172.21.15.2:0/3114317209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe9770d80 0x561fe97765f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.979+0000 7f44a5d9b700 1 -- 172.21.15.2:0/3114317209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe9776db0 con 0x561fe976ae10 2024-05-14T09:37:47.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.979+0000 7f44a5d9b700 1 --2- 172.21.15.2:0/3114317209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe976ae10 0x561fe977ed00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f449c01d000 tx=0x7f449c011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c5eefd4482d7cde server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:47.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.980+0000 7f448ffff700 1 -- 172.21.15.2:0/3114317209 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c011120 con 0x561fe976ae10 2024-05-14T09:37:47.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.980+0000 7f448ffff700 1 -- 172.21.15.2:0/3114317209 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f449c01c6b0 con 0x561fe976ae10 2024-05-14T09:37:47.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.980+0000 7f448ffff700 1 -- 172.21.15.2:0/3114317209 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c0038e0 con 0x561fe976ae10 2024-05-14T09:37:47.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.980+0000 7f44bb705b80 1 -- 172.21.15.2:0/3114317209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe976ae10 msgr2=0x561fe977ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:47.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.980+0000 7f44bb705b80 1 --2- 172.21.15.2:0/3114317209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe976ae10 0x561fe977ed00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f449c01d000 tx=0x7f449c011f90 comp rx=0 tx=0).stop 2024-05-14T09:37:47.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.981+0000 7f44bb705b80 1 -- 172.21.15.2:0/3114317209 shutdown_connections 2024-05-14T09:37:47.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.981+0000 7f44bb705b80 1 --2- 172.21.15.2:0/3114317209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe9770d80 0x561fe97765f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.981+0000 7f44bb705b80 1 --2- 172.21.15.2:0/3114317209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 0x561fe976a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.981+0000 7f44bb705b80 1 --2- 172.21.15.2:0/3114317209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe976ae10 0x561fe977ed00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.981+0000 7f44bb705b80 1 -- 172.21.15.2:0/3114317209 >> 172.21.15.2:0/3114317209 conn(0x561fe9634630 msgr2=0x561fe975fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:47.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.981+0000 7f44bb705b80 1 -- 172.21.15.2:0/3114317209 shutdown_connections 2024-05-14T09:37:47.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.981+0000 7f44bb705b80 1 -- 172.21.15.2:0/3114317209 wait complete. 2024-05-14T09:37:47.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.981+0000 7f44bb705b80 1 -- start start 2024-05-14T09:37:47.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.981+0000 7f44bb705b80 1 Processor -- start 2024-05-14T09:37:47.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.981+0000 7f44bb705b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 0x561fe977e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.981+0000 7f44bb705b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe976ae10 0x561fe977ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.981+0000 7f44bb705b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe9770d80 0x561fe977b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.981+0000 7f44bb705b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81ac7f0 con 0x561fe9770d80 2024-05-14T09:37:47.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.981+0000 7f44bb705b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81adc60 con 0x561fe976ae10 2024-05-14T09:37:47.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.981+0000 7f44bb705b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81acec0 con 0x561fe95ece40 2024-05-14T09:37:47.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44bb705b80 10 client.0 _ll_register_callbacks cb 0x561fe9624950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:47.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44a659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 0x561fe977e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44a5d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe976ae10 0x561fe977ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44a6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe9770d80 0x561fe977b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44a659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 0x561fe977e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:40852/0 (socket says 172.21.15.2:40852) 2024-05-14T09:37:47.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44a5d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe976ae10 0x561fe977ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:48016/0 (socket says 172.21.15.2:48016) 2024-05-14T09:37:47.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44a659c700 1 -- 172.21.15.2:0/2365582833 learned_addr learned my addr 172.21.15.2:0/2365582833 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:47.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44a659c700 1 -- 172.21.15.2:0/2365582833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe976ae10 msgr2=0x561fe977ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:47.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44a659c700 1 --2- 172.21.15.2:0/2365582833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe976ae10 0x561fe977ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44a659c700 1 -- 172.21.15.2:0/2365582833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe9770d80 msgr2=0x561fe977b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:47.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44a659c700 1 --2- 172.21.15.2:0/2365582833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe9770d80 0x561fe977b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44a659c700 1 -- 172.21.15.2:0/2365582833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe977b6f0 con 0x561fe95ece40 2024-05-14T09:37:47.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44a5d9b700 1 --2- 172.21.15.2:0/2365582833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe976ae10 0x561fe977ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:47.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44a6d9d700 1 --2- 172.21.15.2:0/2365582833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe9770d80 0x561fe977b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:47.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44a659c700 1 --2- 172.21.15.2:0/2365582833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 0x561fe977e8b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44980e2010 tx=0x7f4498022ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=811199a950bc016 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:47.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f448dffb700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:37:47.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f448dffb700 1 -- 172.21.15.2:0/2365582833 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44980aab70 con 0x561fe95ece40 2024-05-14T09:37:47.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f44bb705b80 1 -- 172.21.15.2:0/2365582833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f449c01c6b0 con 0x561fe95ece40 2024-05-14T09:37:47.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.982+0000 7f448dffb700 1 -- 172.21.15.2:0/2365582833 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f4498022940 con 0x561fe95ece40 2024-05-14T09:37:47.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.983+0000 7f448dffb700 1 -- 172.21.15.2:0/2365582833 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44980aa470 con 0x561fe95ece40 2024-05-14T09:37:47.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.983+0000 7f44bb705b80 10 client.25591 fetch_fsmap learned FSMap version 29 2024-05-14T09:37:47.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.983+0000 7f44bb705b80 1 -- 172.21.15.2:0/2365582833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561fe9800e90 con 0x561fe95ece40 2024-05-14T09:37:47.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.983+0000 7f448dffb700 1 -- 172.21.15.2:0/2365582833 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f449c01c6b0 con 0x561fe95ece40 2024-05-14T09:37:47.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.983+0000 7f448dffb700 1 -- 172.21.15.2:0/2365582833 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f44980aa6d0 con 0x561fe95ece40 2024-05-14T09:37:47.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.983+0000 7f44bb705b80 10 client.25591 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:37:47.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.983+0000 7f44bb705b80 10 client.25591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:47.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.983+0000 7f44bb705b80 20 client.25591 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:47.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.983+0000 7f44bb705b80 1 --2- 172.21.15.2:0/2365582833 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561fe98015a0 0x561fe98039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:47.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.983+0000 7f44bb705b80 4 client.25591 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T09:37:47.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.983+0000 7f44bb705b80 1 -- 172.21.15.2:0/2365582833 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561fe96943c0 con 0x561fe98015a0 2024-05-14T09:37:47.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.983+0000 7f44a5d9b700 1 --2- 172.21.15.2:0/2365582833 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561fe98015a0 0x561fe98039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:47.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f44a5d9b700 1 --2- 172.21.15.2:0/2365582833 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561fe98015a0 0x561fe98039c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3b674ebb09503b1 server_cookie=98566c7ac280b45c in_seq=0 out_seq=0 2024-05-14T09:37:47.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f448dffb700 10 client.25591 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:47.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f448dffb700 1 -- 172.21.15.2:0/2365582833 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561fe96943c0 con 0x561fe98015a0 2024-05-14T09:37:47.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f448dffb700 10 client.25591 handle_command_reply: tid=0 2024-05-14T09:37:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f44bb705b80 1 client.25591 shutdown 2024-05-14T09:37:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f44bb705b80 1 -- 172.21.15.2:0/2365582833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 msgr2=0x561fe977e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f44bb705b80 1 --2- 172.21.15.2:0/2365582833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 0x561fe977e8b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44980e2010 tx=0x7f4498022ce0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f44bb705b80 1 -- 172.21.15.2:0/2365582833 shutdown_connections 2024-05-14T09:37:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f44bb705b80 1 --2- 172.21.15.2:0/2365582833 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561fe98015a0 0x561fe98039c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f44bb705b80 1 --2- 172.21.15.2:0/2365582833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561fe9770d80 0x561fe977b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f44bb705b80 1 --2- 172.21.15.2:0/2365582833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561fe95ece40 0x561fe977e8b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f448dffb700 0 client.25591 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f44bb705b80 1 --2- 172.21.15.2:0/2365582833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fe976ae10 0x561fe977ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f44bb705b80 1 -- 172.21.15.2:0/2365582833 >> 172.21.15.2:0/2365582833 conn(0x561fe9634630 msgr2=0x561fe975fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f44bb705b80 1 -- 172.21.15.2:0/2365582833 shutdown_connections 2024-05-14T09:37:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.984+0000 7f44bb705b80 1 -- 172.21.15.2:0/2365582833 wait complete. 2024-05-14T09:37:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.985+0000 7f44bb705b80 20 client.25591 trim_cache size 0 max 16384 2024-05-14T09:37:47.991 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T09:37:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-05-14T09:37:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-05-14T09:37:47.991 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T09:37:47.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.985+0000 7f44accf1700 1 -- 172.21.15.2:0/872695690 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f44900b2900 msgr2=0x7f44900b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:47.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.985+0000 7f44accf1700 1 --2- 172.21.15.2:0/872695690 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f44900b2900 0x7f44900b4e10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f44a000f540 tx=0x7f44a000b040 comp rx=0 tx=0).stop 2024-05-14T09:37:47.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.985+0000 7f44accf1700 1 -- 172.21.15.2:0/872695690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a81029c0 msgr2=0x7f44a819ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:47.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.985+0000 7f44accf1700 1 --2- 172.21.15.2:0/872695690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a81029c0 0x7f44a819ae60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f44980098e0 tx=0x7f449800b670 comp rx=0 tx=0).stop 2024-05-14T09:37:47.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.985+0000 7f44accf1700 1 -- 172.21.15.2:0/872695690 shutdown_connections 2024-05-14T09:37:47.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.985+0000 7f44accf1700 1 --2- 172.21.15.2:0/872695690 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f44900b2900 0x7f44900b4e10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.985+0000 7f44accf1700 1 --2- 172.21.15.2:0/872695690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a81029c0 0x7f44a819ae60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.985+0000 7f44accf1700 1 --2- 172.21.15.2:0/872695690 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f44a81089f0 0x7f44a819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.985+0000 7f44accf1700 1 --2- 172.21.15.2:0/872695690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a8103310 0x7f44a819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:47.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.985+0000 7f44accf1700 1 -- 172.21.15.2:0/872695690 >> 172.21.15.2:0/872695690 conn(0x7f44a80fe6f0 msgr2=0x7f44a8106d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:47.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.986+0000 7f44accf1700 1 -- 172.21.15.2:0/872695690 shutdown_connections 2024-05-14T09:37:47.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:47.986+0000 7f44accf1700 1 -- 172.21.15.2:0/872695690 wait complete. 2024-05-14T09:37:48.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 152875 11.39 MB/sec warmup 74 sec latency 31.507 ms 2024-05-14T09:37:48.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:48 smithi002 ceph-mon[31557]: pgmap v509: 129 pgs: 129 active+clean; 49 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 361 op/s 2024-05-14T09:37:48.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:48 smithi029 ceph-mon[36249]: pgmap v509: 129 pgs: 129 active+clean; 49 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 361 op/s 2024-05-14T09:37:48.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:48 smithi026 ceph-mon[36056]: pgmap v509: 129 pgs: 129 active+clean; 49 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 361 op/s 2024-05-14T09:37:48.788 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T09:37:48.789 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T09:37:49.065 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:37:49.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 154951 11.34 MB/sec warmup 75 sec latency 14.835 ms 2024-05-14T09:37:50.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 156378 11.34 MB/sec warmup 76 sec latency 298.529 ms 2024-05-14T09:37:50.661 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:50 smithi002 ceph-mon[31557]: pgmap v510: 129 pgs: 129 active+clean; 49 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 322 op/s 2024-05-14T09:37:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:50 smithi029 ceph-mon[36249]: pgmap v510: 129 pgs: 129 active+clean; 49 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 322 op/s 2024-05-14T09:37:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:50 smithi026 ceph-mon[36056]: pgmap v510: 129 pgs: 129 active+clean; 49 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 322 op/s 2024-05-14T09:37:50.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.943+0000 7f33a7c56700 1 -- 172.21.15.2:0/1478641050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33a011d930 msgr2=0x7f33a011fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:50.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.943+0000 7f33a7c56700 1 --2- 172.21.15.2:0/1478641050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33a011d930 0x7f33a011fd50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f339c009990 tx=0x7f339c009ca0 comp rx=0 tx=0).stop 2024-05-14T09:37:50.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.943+0000 7f33a7c56700 1 -- 172.21.15.2:0/1478641050 shutdown_connections 2024-05-14T09:37:50.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.943+0000 7f33a7c56700 1 --2- 172.21.15.2:0/1478641050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a010a930 0x7f33a010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:50.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.943+0000 7f33a7c56700 1 --2- 172.21.15.2:0/1478641050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33a011d930 0x7f33a011fd50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:50.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.943+0000 7f33a7c56700 1 --2- 172.21.15.2:0/1478641050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33a010b280 0x7f33a011d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:50.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.943+0000 7f33a7c56700 1 -- 172.21.15.2:0/1478641050 >> 172.21.15.2:0/1478641050 conn(0x7f33a006f4d0 msgr2=0x7f33a006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:50.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.944+0000 7f33a7c56700 1 -- 172.21.15.2:0/1478641050 shutdown_connections 2024-05-14T09:37:50.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.944+0000 7f33a7c56700 1 -- 172.21.15.2:0/1478641050 wait complete. 2024-05-14T09:37:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.945+0000 7f33a7c56700 1 Processor -- start 2024-05-14T09:37:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.946+0000 7f33a7c56700 1 -- start start 2024-05-14T09:37:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.946+0000 7f33a7c56700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33a010a930 0x7f33a01af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.946+0000 7f33a7c56700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a010b280 0x7f33a01afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.946+0000 7f33a7c56700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33a011d930 0x7f33a01a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.946+0000 7f33a7c56700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01232c0 con 0x7f33a010a930 2024-05-14T09:37:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.946+0000 7f33a7c56700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33a0123140 con 0x7f33a010b280 2024-05-14T09:37:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.946+0000 7f33a7c56700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f33a0123440 con 0x7f33a011d930 2024-05-14T09:37:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.946+0000 7f33a51f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a010b280 0x7f33a01afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.946+0000 7f33a51f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a010b280 0x7f33a01afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:48034/0 (socket says 172.21.15.2:48034) 2024-05-14T09:37:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.946+0000 7f33a51f1700 1 -- 172.21.15.2:0/3092374336 learned_addr learned my addr 172.21.15.2:0/3092374336 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.946+0000 7f33a59f2700 1 --2- 172.21.15.2:0/3092374336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33a010a930 0x7f33a01af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:50.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.946+0000 7f33a61f3700 1 --2- 172.21.15.2:0/3092374336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33a011d930 0x7f33a01a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:50.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.947+0000 7f33a51f1700 1 -- 172.21.15.2:0/3092374336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33a011d930 msgr2=0x7f33a01a9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:50.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.947+0000 7f33a51f1700 1 --2- 172.21.15.2:0/3092374336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33a011d930 0x7f33a01a9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:50.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.947+0000 7f33a51f1700 1 -- 172.21.15.2:0/3092374336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33a010a930 msgr2=0x7f33a01af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:50.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.947+0000 7f33a51f1700 1 --2- 172.21.15.2:0/3092374336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33a010a930 0x7f33a01af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:50.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.947+0000 7f33a51f1700 1 -- 172.21.15.2:0/3092374336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a01aa0b0 con 0x7f33a010b280 2024-05-14T09:37:50.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.947+0000 7f33a59f2700 1 --2- 172.21.15.2:0/3092374336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33a010a930 0x7f33a01af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:50.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.948+0000 7f33a51f1700 1 --2- 172.21.15.2:0/3092374336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a010b280 0x7f33a01afd20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f339000d8d0 tx=0x7f339000dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:50.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.949+0000 7f3396ffd700 1 -- 172.21.15.2:0/3092374336 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390016070 con 0x7f33a010b280 2024-05-14T09:37:50.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.949+0000 7f3396ffd700 1 -- 172.21.15.2:0/3092374336 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f339000fd40 con 0x7f33a010b280 2024-05-14T09:37:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.950+0000 7f3396ffd700 1 -- 172.21.15.2:0/3092374336 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339000e3b0 con 0x7f33a010b280 2024-05-14T09:37:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.951+0000 7f33a7c56700 1 -- 172.21.15.2:0/3092374336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a01aa3a0 con 0x7f33a010b280 2024-05-14T09:37:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.951+0000 7f33a7c56700 1 -- 172.21.15.2:0/3092374336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a01b6670 con 0x7f33a010b280 2024-05-14T09:37:50.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.952+0000 7f33a7c56700 1 -- 172.21.15.2:0/3092374336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a010c070 con 0x7f33a010b280 2024-05-14T09:37:50.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.955+0000 7f3396ffd700 1 -- 172.21.15.2:0/3092374336 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3390019070 con 0x7f33a010b280 2024-05-14T09:37:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.956+0000 7f3396ffd700 1 --2- 172.21.15.2:0/3092374336 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f338c0b28e0 0x7f338c0b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.956+0000 7f33a59f2700 1 --2- 172.21.15.2:0/3092374336 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f338c0b28e0 0x7f338c0b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.956+0000 7f3396ffd700 1 -- 172.21.15.2:0/3092374336 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f33900e2410 con 0x7f33a010b280 2024-05-14T09:37:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.956+0000 7f33a59f2700 1 --2- 172.21.15.2:0/3092374336 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f338c0b28e0 0x7f338c0b4df0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f33a01aab40 tx=0x7f339800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:50.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:50.959+0000 7f3396ffd700 1 -- 172.21.15.2:0/3092374336 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3390020070 con 0x7f33a010b280 2024-05-14T09:37:51.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:51.250+0000 7f33a7c56700 1 -- 172.21.15.2:0/3092374336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33a00512f0 con 0x7f33a010b280 2024-05-14T09:37:51.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:51.251+0000 7f3396ffd700 1 -- 172.21.15.2:0/3092374336 <== mon.2 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11847 (secure 0 0 0) 0x7f33a00512f0 con 0x7f33a010b280 2024-05-14T09:37:51.252 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T09:37:51.252 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T09:37:18.737412+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/2529273890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2529273890},{"type":"v1","addr":"172.21.15.26:6835","nonce":2529273890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":103,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24497,24524,24487]},"id":1}]} 2024-05-14T09:37:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:51.254+0000 7f33a7c56700 1 -- 172.21.15.2:0/3092374336 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f338c0b28e0 msgr2=0x7f338c0b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:51.254+0000 7f33a7c56700 1 --2- 172.21.15.2:0/3092374336 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f338c0b28e0 0x7f338c0b4df0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f33a01aab40 tx=0x7f339800b040 comp rx=0 tx=0).stop 2024-05-14T09:37:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:51.254+0000 7f33a7c56700 1 -- 172.21.15.2:0/3092374336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a010b280 msgr2=0x7f33a01afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:51.254+0000 7f33a7c56700 1 --2- 172.21.15.2:0/3092374336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a010b280 0x7f33a01afd20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f339000d8d0 tx=0x7f339000dbe0 comp rx=0 tx=0).stop 2024-05-14T09:37:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:51.254+0000 7f33a7c56700 1 -- 172.21.15.2:0/3092374336 shutdown_connections 2024-05-14T09:37:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:51.254+0000 7f33a7c56700 1 --2- 172.21.15.2:0/3092374336 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f338c0b28e0 0x7f338c0b4df0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:51.254+0000 7f33a7c56700 1 --2- 172.21.15.2:0/3092374336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33a011d930 0x7f33a01a9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:51.254+0000 7f33a7c56700 1 --2- 172.21.15.2:0/3092374336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33a010a930 0x7f33a01af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:51.254+0000 7f33a7c56700 1 --2- 172.21.15.2:0/3092374336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a010b280 0x7f33a01afd20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:51.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:51.254+0000 7f33a7c56700 1 -- 172.21.15.2:0/3092374336 >> 172.21.15.2:0/3092374336 conn(0x7f33a006f4d0 msgr2=0x7f33a011b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:51.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:51.254+0000 7f33a7c56700 1 -- 172.21.15.2:0/3092374336 shutdown_connections 2024-05-14T09:37:51.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:51.254+0000 7f33a7c56700 1 -- 172.21.15.2:0/3092374336 wait complete. 2024-05-14T09:37:51.255 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 29 2024-05-14T09:37:51.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 158278 11.27 MB/sec warmup 77 sec latency 36.391 ms 2024-05-14T09:37:51.560 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:51 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3092374336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:37:51.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:51 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3092374336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:37:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:51 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3092374336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T09:37:51.907 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T09:37:52.213 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:37:52.256 INFO:tasks.workunit.client.0.smithi002.stdout: 1 160029 11.30 MB/sec warmup 78 sec latency 53.411 ms 2024-05-14T09:37:52.597 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:52 smithi002 ceph-mon[31557]: pgmap v511: 129 pgs: 129 active+clean; 50 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 275 op/s 2024-05-14T09:37:52.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:52 smithi029 ceph-mon[36249]: pgmap v511: 129 pgs: 129 active+clean; 50 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 275 op/s 2024-05-14T09:37:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:52 smithi026 ceph-mon[36056]: pgmap v511: 129 pgs: 129 active+clean; 50 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 275 op/s 2024-05-14T09:37:53.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 161931 11.24 MB/sec warmup 79 sec latency 24.166 ms 2024-05-14T09:37:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.973+0000 7fa8bf59d640 1 -- 172.21.15.2:0/2985107261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b8101710 msgr2=0x7fa8b8101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.973+0000 7fa8bf59d640 1 --2- 172.21.15.2:0/2985107261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b8101710 0x7fa8b8101b80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009860 tx=0x7fa8ac02a340 comp rx=0 tx=0).stop 2024-05-14T09:37:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.973+0000 7fa8bf59d640 1 -- 172.21.15.2:0/2985107261 shutdown_connections 2024-05-14T09:37:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.973+0000 7fa8bf59d640 1 --2- 172.21.15.2:0/2985107261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8b8102210 0x7fa8b810bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.973+0000 7fa8bf59d640 1 --2- 172.21.15.2:0/2985107261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b8101710 0x7fa8b8101b80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.973+0000 7fa8bf59d640 1 --2- 172.21.15.2:0/2985107261 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8b8105060 0x7fa8b8105470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.973+0000 7fa8bf59d640 1 -- 172.21.15.2:0/2985107261 >> 172.21.15.2:0/2985107261 conn(0x7fa8b80fd350 msgr2=0x7fa8b80ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.973+0000 7fa8bf59d640 1 -- 172.21.15.2:0/2985107261 shutdown_connections 2024-05-14T09:37:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.973+0000 7fa8bf59d640 1 -- 172.21.15.2:0/2985107261 wait complete. 2024-05-14T09:37:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.973+0000 7fa8bf59d640 1 Processor -- start 2024-05-14T09:37:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bf59d640 1 -- start start 2024-05-14T09:37:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bf59d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8b8101710 0x7fa8b8104210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bf59d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b8102210 0x7fa8b81028b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bf59d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8b8105060 0x7fa8b8102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bf59d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b810f690 con 0x7fa8b8105060 2024-05-14T09:37:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bf59d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b810f510 con 0x7fa8b8102210 2024-05-14T09:37:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bf59d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b810f810 con 0x7fa8b8101710 2024-05-14T09:37:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bed9c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8b8105060 0x7fa8b8102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bdd9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b8102210 0x7fa8b81028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8be59b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8b8101710 0x7fa8b8104210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bed9c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8b8105060 0x7fa8b8102df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44286/0 (socket says 172.21.15.2:44286) 2024-05-14T09:37:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bed9c640 1 -- 172.21.15.2:0/780043088 learned_addr learned my addr 172.21.15.2:0/780043088 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bed9c640 1 -- 172.21.15.2:0/780043088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8b8101710 msgr2=0x7fa8b8104210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bed9c640 1 --2- 172.21.15.2:0/780043088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8b8101710 0x7fa8b8104210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bed9c640 1 -- 172.21.15.2:0/780043088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b8102210 msgr2=0x7fa8b81028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bed9c640 1 --2- 172.21.15.2:0/780043088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b8102210 0x7fa8b81028b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bed9c640 1 -- 172.21.15.2:0/780043088 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b8103520 con 0x7fa8b8105060 2024-05-14T09:37:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8bdd9a640 1 --2- 172.21.15.2:0/780043088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b8102210 0x7fa8b81028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.974+0000 7fa8be59b640 1 --2- 172.21.15.2:0/780043088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8b8101710 0x7fa8b8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.975+0000 7fa8bed9c640 1 --2- 172.21.15.2:0/780043088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8b8105060 0x7fa8b8102df0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009fd0 tx=0x7fa8b000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.975+0000 7fa8ab7fe640 1 -- 172.21.15.2:0/780043088 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0004b20 con 0x7fa8b8105060 2024-05-14T09:37:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.975+0000 7fa8ab7fe640 1 -- 172.21.15.2:0/780043088 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8b0003e20 con 0x7fa8b8105060 2024-05-14T09:37:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.975+0000 7fa8bf59d640 1 -- 172.21.15.2:0/780043088 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b81a4260 con 0x7fa8b8105060 2024-05-14T09:37:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.975+0000 7fa8ab7fe640 1 -- 172.21.15.2:0/780043088 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0004820 con 0x7fa8b8105060 2024-05-14T09:37:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.975+0000 7fa8bf59d640 1 -- 172.21.15.2:0/780043088 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b81a4720 con 0x7fa8b8105060 2024-05-14T09:37:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.977+0000 7fa8ab7fe640 1 -- 172.21.15.2:0/780043088 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa8b001d020 con 0x7fa8b8105060 2024-05-14T09:37:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.977+0000 7fa8c778e740 1 Processor -- start 2024-05-14T09:37:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.978+0000 7fa8c778e740 1 -- start start 2024-05-14T09:37:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.978+0000 7fa8c778e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7d51b0 0x563e2cb76c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.978+0000 7fa8c778e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7ac2f0 0x563e2cb771b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.978+0000 7fa8c778e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7b44d0 0x563e2cb7e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.978+0000 7fa8c778e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563e2cadb5d0 con 0x563e2c7ac2f0 2024-05-14T09:37:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.978+0000 7fa8c778e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563e2ca94890 con 0x563e2c7b44d0 2024-05-14T09:37:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.978+0000 7fa8c778e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563e2cad7590 con 0x563e2c7d51b0 2024-05-14T09:37:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.978+0000 7fa8be59b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7ac2f0 0x563e2cb771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.978+0000 7fa8bdd9a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7d51b0 0x563e2cb76c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.978+0000 7fa8be59b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7ac2f0 0x563e2cb771b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44290/0 (socket says 172.21.15.2:44290) 2024-05-14T09:37:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.978+0000 7fa8be59b640 1 -- 172.21.15.2:0/3750525192 learned_addr learned my addr 172.21.15.2:0/3750525192 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.978+0000 7fa8bdd9a640 1 --2- 172.21.15.2:0/3750525192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7b44d0 0x563e2cb7e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.979+0000 7fa8be59b640 1 -- 172.21.15.2:0/3750525192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7d51b0 msgr2=0x563e2cb76c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.979+0000 7fa8be59b640 1 --2- 172.21.15.2:0/3750525192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7d51b0 0x563e2cb76c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.979+0000 7fa8be59b640 1 -- 172.21.15.2:0/3750525192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7b44d0 msgr2=0x563e2cb7e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.979+0000 7fa8be59b640 1 --2- 172.21.15.2:0/3750525192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7b44d0 0x563e2cb7e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.979+0000 7fa8ab7fe640 1 --2- 172.21.15.2:0/780043088 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa8a40b28a0 0x7fa8a40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.979+0000 7fa8be59b640 1 -- 172.21.15.2:0/3750525192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e2c7ed210 con 0x563e2c7ac2f0 2024-05-14T09:37:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.979+0000 7fa8ab7fe640 1 -- 172.21.15.2:0/780043088 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fa8b00e25c0 con 0x7fa8b8105060 2024-05-14T09:37:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.979+0000 7fa8bdd9a640 1 --2- 172.21.15.2:0/780043088 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa8a40b28a0 0x7fa8a40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.979+0000 7fa8be59b640 1 --2- 172.21.15.2:0/3750525192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7ac2f0 0x563e2cb771b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40114f0 tx=0x7fa8b40119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc57036ef7ee2a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.979+0000 7fa8a8ff9640 1 -- 172.21.15.2:0/3750525192 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b401e070 con 0x563e2c7ac2f0 2024-05-14T09:37:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.979+0000 7fa8a8ff9640 1 -- 172.21.15.2:0/3750525192 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8b40035f0 con 0x563e2c7ac2f0 2024-05-14T09:37:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.979+0000 7fa8a8ff9640 1 -- 172.21.15.2:0/3750525192 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b400f7a0 con 0x563e2c7ac2f0 2024-05-14T09:37:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.980+0000 7fa8c778e740 1 -- 172.21.15.2:0/3750525192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7ac2f0 msgr2=0x563e2cb771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.980+0000 7fa8bdd9a640 1 --2- 172.21.15.2:0/780043088 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa8a40b28a0 0x7fa8a40b4db0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac02a930 tx=0x7fa8ac042040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.980+0000 7fa8c778e740 1 --2- 172.21.15.2:0/3750525192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7ac2f0 0x563e2cb771b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40114f0 tx=0x7fa8b40119b0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.980+0000 7fa8c778e740 1 -- 172.21.15.2:0/3750525192 shutdown_connections 2024-05-14T09:37:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.980+0000 7fa8c778e740 1 --2- 172.21.15.2:0/3750525192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7b44d0 0x563e2cb7e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.980+0000 7fa8c778e740 1 --2- 172.21.15.2:0/3750525192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7ac2f0 0x563e2cb771b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.980+0000 7fa8c778e740 1 --2- 172.21.15.2:0/3750525192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7d51b0 0x563e2cb76c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.980+0000 7fa8c778e740 1 -- 172.21.15.2:0/3750525192 >> 172.21.15.2:0/3750525192 conn(0x563e2c7c9300 msgr2=0x563e2cb725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.980+0000 7fa8c778e740 1 -- 172.21.15.2:0/3750525192 shutdown_connections 2024-05-14T09:37:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.980+0000 7fa8c778e740 1 -- 172.21.15.2:0/3750525192 wait complete. 2024-05-14T09:37:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.981+0000 7fa8c778e740 1 -- start start 2024-05-14T09:37:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.981+0000 7fa8c778e740 1 Processor -- start 2024-05-14T09:37:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.981+0000 7fa8c778e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 0x563e2cb80810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.981+0000 7fa8c778e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 0x563e2cb80d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.981+0000 7fa8c778e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb87cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.981+0000 7fa8c778e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563e2cb7eb00 con 0x563e2c7d51b0 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.981+0000 7fa8c778e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563e2ca98370 con 0x563e2c7ac2f0 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.981+0000 7fa8c778e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563e2cb7ec80 con 0x563e2c7b44d0 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.981+0000 7fa8c778e740 10 client.0 _ll_register_callbacks cb 0x563e2c7eca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.981+0000 7fa8bed9c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb87cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.981+0000 7fa8bdd9a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 0x563e2cb80d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.981+0000 7fa8bed9c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb87cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44298/0 (socket says 172.21.15.2:44298) 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.981+0000 7fa8bed9c640 1 -- 172.21.15.2:0/4045955751 learned_addr learned my addr 172.21.15.2:0/4045955751 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.981+0000 7fa8bdd9a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 0x563e2cb80d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:40910/0 (socket says 172.21.15.2:40910) 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.981+0000 7fa8be59b640 1 --2- 172.21.15.2:0/4045955751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 0x563e2cb80810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8bed9c640 1 -- 172.21.15.2:0/4045955751 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 msgr2=0x563e2cb80d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8bed9c640 1 --2- 172.21.15.2:0/4045955751 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 0x563e2cb80d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8bed9c640 1 -- 172.21.15.2:0/4045955751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 msgr2=0x563e2cb80810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8bed9c640 1 --2- 172.21.15.2:0/4045955751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 0x563e2cb80810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8bed9c640 1 -- 172.21.15.2:0/4045955751 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e2c8b9620 con 0x563e2c7d51b0 2024-05-14T09:37:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8be59b640 1 --2- 172.21.15.2:0/4045955751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 0x563e2cb80810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:37:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8bed9c640 1 --2- 172.21.15.2:0/4045955751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb87cb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00a9750 tx=0x7fa8b00a8ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d088c15242787fbe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:37:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8a1ffb640 1 -- 172.21.15.2:0/4045955751 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b001a070 con 0x563e2c7d51b0 2024-05-14T09:37:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8a1ffb640 1 -- 172.21.15.2:0/4045955751 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fa8b00cc0c0 con 0x563e2c7d51b0 2024-05-14T09:37:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8c778e740 1 -- 172.21.15.2:0/4045955751 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8b40035f0 con 0x563e2c7d51b0 2024-05-14T09:37:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8a1ffb640 1 -- 172.21.15.2:0/4045955751 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa8b00a9ec0 con 0x563e2c7d51b0 2024-05-14T09:37:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8c778e740 10 client.15924 fetch_fsmap learned FSMap version 29 2024-05-14T09:37:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8c778e740 1 -- 172.21.15.2:0/4045955751 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563e2cb8acb0 con 0x563e2c7d51b0 2024-05-14T09:37:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8a1ffb640 1 -- 172.21.15.2:0/4045955751 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8b00c7020 con 0x563e2c7d51b0 2024-05-14T09:37:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.982+0000 7fa8a1ffb640 1 -- 172.21.15.2:0/4045955751 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa8b000b040 con 0x563e2c7d51b0 2024-05-14T09:37:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.983+0000 7fa8c778e740 10 client.15924 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:37:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.983+0000 7fa8c778e740 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.983+0000 7fa8c778e740 20 client.15924 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.983+0000 7fa8c778e740 1 --2- 172.21.15.2:0/4045955751 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e2c793d80 0x563e2cb8e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.983+0000 7fa8c778e740 4 client.15924 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:37:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.983+0000 7fa8c778e740 1 -- 172.21.15.2:0/4045955751 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e2cae91e0 con 0x563e2c793d80 2024-05-14T09:37:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.983+0000 7fa8be59b640 1 --2- 172.21.15.2:0/4045955751 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e2c793d80 0x563e2cb8e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.983+0000 7fa8a1ffb640 10 client.15924 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.983+0000 7fa8be59b640 1 --2- 172.21.15.2:0/4045955751 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e2c793d80 0x563e2cb8e3d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7252e15d7549bb0a server_cookie=e3afdfd8d06639c4 in_seq=0 out_seq=0 2024-05-14T09:37:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.984+0000 7fa8a1ffb640 1 -- 172.21.15.2:0/4045955751 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563e2cae91e0 con 0x563e2c793d80 2024-05-14T09:37:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.984+0000 7fa8a1ffb640 10 client.15924 handle_command_reply: tid=0 2024-05-14T09:37:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.984+0000 7fa8c778e740 1 client.15924 shutdown 2024-05-14T09:37:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.984+0000 7fa8c778e740 1 -- 172.21.15.2:0/4045955751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 msgr2=0x563e2cb87cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.984+0000 7fa8c778e740 1 --2- 172.21.15.2:0/4045955751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb87cb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00a9750 tx=0x7fa8b00a8ff0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.984+0000 7fa8c778e740 1 -- 172.21.15.2:0/4045955751 shutdown_connections 2024-05-14T09:37:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.984+0000 7fa8c778e740 1 --2- 172.21.15.2:0/4045955751 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e2c793d80 0x563e2cb8e3d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.984+0000 7fa8c778e740 1 --2- 172.21.15.2:0/4045955751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb87cb0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.984+0000 7fa8c778e740 1 --2- 172.21.15.2:0/4045955751 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 0x563e2cb80d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.984+0000 7fa8a1ffb640 0 client.15924 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.984+0000 7fa8c778e740 1 --2- 172.21.15.2:0/4045955751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 0x563e2cb80810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.984+0000 7fa8c778e740 1 -- 172.21.15.2:0/4045955751 >> 172.21.15.2:0/4045955751 conn(0x563e2c7c9300 msgr2=0x563e2cb754d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.984+0000 7fa8c778e740 1 -- 172.21.15.2:0/4045955751 shutdown_connections 2024-05-14T09:37:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.984+0000 7fa8c778e740 1 -- 172.21.15.2:0/4045955751 wait complete. 2024-05-14T09:37:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:53.984+0000 7fa8c778e740 20 client.15924 trim_cache size 0 max 16384 2024-05-14T09:37:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.001+0000 7fa8c778e740 1 Processor -- start 2024-05-14T09:37:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.001+0000 7fa8c778e740 1 -- start start 2024-05-14T09:37:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.001+0000 7fa8c778e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 0x563e2cb7f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.001+0000 7fa8c778e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 0x563e2cb87bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.001+0000 7fa8c778e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb71f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.001+0000 7fa8c778e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b81b1600 con 0x563e2c7d51b0 2024-05-14T09:37:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.001+0000 7fa8c778e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b81aeed0 con 0x563e2c7ac2f0 2024-05-14T09:37:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.001+0000 7fa8c778e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b81acd20 con 0x563e2c7b44d0 2024-05-14T09:37:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.001+0000 7fa8bed9c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb71f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.001+0000 7fa8bdd9a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 0x563e2cb87bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.001+0000 7fa8be59b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 0x563e2cb7f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.001+0000 7fa8bdd9a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 0x563e2cb87bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:40924/0 (socket says 172.21.15.2:40924) 2024-05-14T09:37:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.001+0000 7fa8bdd9a640 1 -- 172.21.15.2:0/1142386951 learned_addr learned my addr 172.21.15.2:0/1142386951 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.002+0000 7fa8bdd9a640 1 -- 172.21.15.2:0/1142386951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 msgr2=0x563e2cb7f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.002+0000 7fa8bdd9a640 1 --2- 172.21.15.2:0/1142386951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 0x563e2cb7f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.002+0000 7fa8bdd9a640 1 -- 172.21.15.2:0/1142386951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 msgr2=0x563e2cb71f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.002+0000 7fa8bdd9a640 1 --2- 172.21.15.2:0/1142386951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb71f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.002+0000 7fa8bdd9a640 1 -- 172.21.15.2:0/1142386951 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e2cb73150 con 0x563e2c7b44d0 2024-05-14T09:37:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.002+0000 7fa8bed9c640 1 --2- 172.21.15.2:0/1142386951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb71f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.002+0000 7fa8bdd9a640 1 --2- 172.21.15.2:0/1142386951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 0x563e2cb87bd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac034000 tx=0x7fa8ac005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62676415da1b9be5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.002+0000 7fa8a3fff640 1 -- 172.21.15.2:0/1142386951 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac037040 con 0x563e2c7b44d0 2024-05-14T09:37:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.002+0000 7fa8a3fff640 1 -- 172.21.15.2:0/1142386951 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8ac032ca0 con 0x563e2c7b44d0 2024-05-14T09:37:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.002+0000 7fa8a3fff640 1 -- 172.21.15.2:0/1142386951 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac04c480 con 0x563e2c7b44d0 2024-05-14T09:37:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.003+0000 7fa8c778e740 1 -- 172.21.15.2:0/1142386951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 msgr2=0x563e2cb87bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.004+0000 7fa8c778e740 1 --2- 172.21.15.2:0/1142386951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 0x563e2cb87bd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac034000 tx=0x7fa8ac005760 comp rx=0 tx=0).stop 2024-05-14T09:37:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.004+0000 7fa8c778e740 1 -- 172.21.15.2:0/1142386951 shutdown_connections 2024-05-14T09:37:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.004+0000 7fa8c778e740 1 --2- 172.21.15.2:0/1142386951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb71f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.004+0000 7fa8c778e740 1 --2- 172.21.15.2:0/1142386951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 0x563e2cb87bd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.004+0000 7fa8c778e740 1 --2- 172.21.15.2:0/1142386951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 0x563e2cb7f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.004+0000 7fa8c778e740 1 -- 172.21.15.2:0/1142386951 >> 172.21.15.2:0/1142386951 conn(0x563e2c7c9300 msgr2=0x563e2cb7f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.004+0000 7fa8c778e740 1 -- 172.21.15.2:0/1142386951 shutdown_connections 2024-05-14T09:37:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.004+0000 7fa8c778e740 1 -- 172.21.15.2:0/1142386951 wait complete. 2024-05-14T09:37:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.004+0000 7fa8c778e740 1 -- start start 2024-05-14T09:37:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.004+0000 7fa8c778e740 1 Processor -- start 2024-05-14T09:37:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8c778e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 0x563e2cb85000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8c778e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 0x563e2cb85540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8c778e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb86470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8c778e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b81ad390 con 0x563e2c7d51b0 2024-05-14T09:37:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8c778e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b81ae800 con 0x563e2c7ac2f0 2024-05-14T09:37:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8c778e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b81ada60 con 0x563e2c7b44d0 2024-05-14T09:37:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8c778e740 10 client.0 _ll_register_callbacks cb 0x7fa8b0004cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8bed9c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb86470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8bdd9a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 0x563e2cb85540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8be59b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 0x563e2cb85000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8bed9c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb86470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44320/0 (socket says 172.21.15.2:44320) 2024-05-14T09:37:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8bed9c640 1 -- 172.21.15.2:0/3491171504 learned_addr learned my addr 172.21.15.2:0/3491171504 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8bed9c640 1 -- 172.21.15.2:0/3491171504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 msgr2=0x563e2cb85540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8bed9c640 1 --2- 172.21.15.2:0/3491171504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 0x563e2cb85540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8bed9c640 1 -- 172.21.15.2:0/3491171504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 msgr2=0x563e2cb85000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8bed9c640 1 --2- 172.21.15.2:0/3491171504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 0x563e2cb85000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8bed9c640 1 -- 172.21.15.2:0/3491171504 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e2cb86a30 con 0x563e2c7d51b0 2024-05-14T09:37:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.005+0000 7fa8be59b640 1 --2- 172.21.15.2:0/3491171504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 0x563e2cb85000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8bed9c640 1 --2- 172.21.15.2:0/3491171504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb86470 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00a5d60 tx=0x7fa8b00a9420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0fcea0c14e394cc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:37:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8a1ffb640 1 -- 172.21.15.2:0/3491171504 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b001a070 con 0x563e2c7d51b0 2024-05-14T09:37:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8a1ffb640 1 -- 172.21.15.2:0/3491171504 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fa8b00a5db0 con 0x563e2c7d51b0 2024-05-14T09:37:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8c778e740 1 -- 172.21.15.2:0/3491171504 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8ac032ca0 con 0x563e2c7d51b0 2024-05-14T09:37:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8a1ffb640 1 -- 172.21.15.2:0/3491171504 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa8b000ce30 con 0x563e2c7d51b0 2024-05-14T09:37:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8c778e740 10 client.15930 fetch_fsmap learned FSMap version 29 2024-05-14T09:37:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8c778e740 1 -- 172.21.15.2:0/3491171504 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563e2cb87d30 con 0x563e2c7d51b0 2024-05-14T09:37:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8a1ffb640 1 -- 172.21.15.2:0/3491171504 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8ac032ca0 con 0x563e2c7d51b0 2024-05-14T09:37:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8a1ffb640 1 -- 172.21.15.2:0/3491171504 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa8b00a8a20 con 0x563e2c7d51b0 2024-05-14T09:37:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8c778e740 10 client.15930 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:37:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8c778e740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:37:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8c778e740 20 client.15930 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8c778e740 1 --2- 172.21.15.2:0/3491171504 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e2c7691c0 0x563e2cbf8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8c778e740 4 client.15930 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:37:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.006+0000 7fa8c778e740 1 -- 172.21.15.2:0/3491171504 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e2cb775b0 con 0x563e2c7691c0 2024-05-14T09:37:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.007+0000 7fa8be59b640 1 --2- 172.21.15.2:0/3491171504 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e2c7691c0 0x563e2cbf8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.007+0000 7fa8a1ffb640 10 client.15930 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.007+0000 7fa8be59b640 1 --2- 172.21.15.2:0/3491171504 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e2c7691c0 0x563e2cbf8dc0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eda7f1a3a33ae76 server_cookie=5e3a8bfb328f9074 in_seq=0 out_seq=0 2024-05-14T09:37:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.096+0000 7fa8a1ffb640 1 -- 172.21.15.2:0/3491171504 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e2cb775b0 con 0x563e2c7691c0 2024-05-14T09:37:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.096+0000 7fa8a1ffb640 10 client.15930 handle_command_reply: tid=0 2024-05-14T09:37:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.096+0000 7fa8c778e740 1 client.15930 shutdown 2024-05-14T09:37:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.096+0000 7fa8c778e740 1 -- 172.21.15.2:0/3491171504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 msgr2=0x563e2cb86470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.096+0000 7fa8c778e740 1 --2- 172.21.15.2:0/3491171504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb86470 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00a5d60 tx=0x7fa8b00a9420 comp rx=0 tx=0).stop 2024-05-14T09:37:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.096+0000 7fa8c778e740 1 -- 172.21.15.2:0/3491171504 shutdown_connections 2024-05-14T09:37:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.096+0000 7fa8c778e740 1 --2- 172.21.15.2:0/3491171504 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e2c7691c0 0x563e2cbf8dc0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.096+0000 7fa8c778e740 1 --2- 172.21.15.2:0/3491171504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e2c7d51b0 0x563e2cb86470 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.096+0000 7fa8c778e740 1 --2- 172.21.15.2:0/3491171504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e2c7b44d0 0x563e2cb85540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.096+0000 7fa8a1ffb640 0 client.15930 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:37:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.096+0000 7fa8c778e740 1 --2- 172.21.15.2:0/3491171504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e2c7ac2f0 0x563e2cb85000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.096+0000 7fa8c778e740 1 -- 172.21.15.2:0/3491171504 >> 172.21.15.2:0/3491171504 conn(0x563e2c7c9300 msgr2=0x563e2cb71f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.096+0000 7fa8c778e740 1 -- 172.21.15.2:0/3491171504 shutdown_connections 2024-05-14T09:37:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.096+0000 7fa8c778e740 1 -- 172.21.15.2:0/3491171504 wait complete. 2024-05-14T09:37:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.096+0000 7fa8c778e740 20 client.15930 trim_cache size 0 max 16384 2024-05-14T09:37:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:37:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:37:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:37:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:37:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.097+0000 7fa8bf59d640 1 -- 172.21.15.2:0/780043088 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa8a40b28a0 msgr2=0x7fa8a40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.097+0000 7fa8bf59d640 1 --2- 172.21.15.2:0/780043088 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa8a40b28a0 0x7fa8a40b4db0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac02a930 tx=0x7fa8ac042040 comp rx=0 tx=0).stop 2024-05-14T09:37:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.097+0000 7fa8bf59d640 1 -- 172.21.15.2:0/780043088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8b8105060 msgr2=0x7fa8b8102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.097+0000 7fa8bf59d640 1 --2- 172.21.15.2:0/780043088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8b8105060 0x7fa8b8102df0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009fd0 tx=0x7fa8b000c800 comp rx=0 tx=0).stop 2024-05-14T09:37:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.097+0000 7fa8bf59d640 1 -- 172.21.15.2:0/780043088 shutdown_connections 2024-05-14T09:37:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.097+0000 7fa8bf59d640 1 --2- 172.21.15.2:0/780043088 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa8a40b28a0 0x7fa8a40b4db0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.097+0000 7fa8bf59d640 1 --2- 172.21.15.2:0/780043088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8b8105060 0x7fa8b8102df0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.097+0000 7fa8bf59d640 1 --2- 172.21.15.2:0/780043088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b8102210 0x7fa8b81028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.097+0000 7fa8bf59d640 1 --2- 172.21.15.2:0/780043088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa8b8101710 0x7fa8b8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.097+0000 7fa8bf59d640 1 -- 172.21.15.2:0/780043088 >> 172.21.15.2:0/780043088 conn(0x7fa8b80fd350 msgr2=0x7fa8b8105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.097+0000 7fa8bf59d640 1 -- 172.21.15.2:0/780043088 shutdown_connections 2024-05-14T09:37:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:37:54.097+0000 7fa8bf59d640 1 -- 172.21.15.2:0/780043088 wait complete. 2024-05-14T09:37:54.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.159+0000 7facf4e1d700 1 -- 172.21.15.2:0/1498203492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facf0106ec0 msgr2=0x7facf0113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:54.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.159+0000 7facf4e1d700 1 --2- 172.21.15.2:0/1498203492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facf0106ec0 0x7facf0113350 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7face800b0a0 tx=0x7face800b3b0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.160+0000 7facf4e1d700 1 -- 172.21.15.2:0/1498203492 shutdown_connections 2024-05-14T09:37:54.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.160+0000 7facf4e1d700 1 --2- 172.21.15.2:0/1498203492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facf0106ec0 0x7facf0113350 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.160+0000 7facf4e1d700 1 --2- 172.21.15.2:0/1498203492 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facf010c5b0 0x7facf010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.160+0000 7facf4e1d700 1 --2- 172.21.15.2:0/1498203492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facf0106570 0x7facf0106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.160+0000 7facf4e1d700 1 -- 172.21.15.2:0/1498203492 >> 172.21.15.2:0/1498203492 conn(0x7facf0102050 msgr2=0x7facf01044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:54.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.160+0000 7facf4e1d700 1 -- 172.21.15.2:0/1498203492 shutdown_connections 2024-05-14T09:37:54.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.160+0000 7facf4e1d700 1 -- 172.21.15.2:0/1498203492 wait complete. 2024-05-14T09:37:54.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.160+0000 7facf4e1d700 1 Processor -- start 2024-05-14T09:37:54.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7facf4e1d700 1 -- start start 2024-05-14T09:37:54.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7facf4e1d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facf0106570 0x7facf019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7facf4e1d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facf0106ec0 0x7facf019d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7facf4e1d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facf010c5b0 0x7facf01a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7facf4e1d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7facf01166e0 con 0x7facf0106ec0 2024-05-14T09:37:54.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7facf4e1d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7facf0116560 con 0x7facf0106570 2024-05-14T09:37:54.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7facf4e1d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7facf0116860 con 0x7facf010c5b0 2024-05-14T09:37:54.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7face7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facf0106ec0 0x7facf019d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7faceed9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facf010c5b0 0x7facf01a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7face7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facf0106ec0 0x7facf019d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44332/0 (socket says 172.21.15.2:44332) 2024-05-14T09:37:54.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7face7fff700 1 -- 172.21.15.2:0/1403735676 learned_addr learned my addr 172.21.15.2:0/1403735676 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:54.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7facee59c700 1 --2- 172.21.15.2:0/1403735676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facf0106570 0x7facf019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7faceed9d700 1 -- 172.21.15.2:0/1403735676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facf0106570 msgr2=0x7facf019cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:54.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7faceed9d700 1 --2- 172.21.15.2:0/1403735676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facf0106570 0x7facf019cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7faceed9d700 1 -- 172.21.15.2:0/1403735676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facf0106ec0 msgr2=0x7facf019d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:54.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7faceed9d700 1 --2- 172.21.15.2:0/1403735676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facf0106ec0 0x7facf019d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.161+0000 7faceed9d700 1 -- 172.21.15.2:0/1403735676 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf01a1dd0 con 0x7facf010c5b0 2024-05-14T09:37:54.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.162+0000 7face7fff700 1 --2- 172.21.15.2:0/1403735676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facf0106ec0 0x7facf019d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:54.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.162+0000 7faceed9d700 1 --2- 172.21.15.2:0/1403735676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facf010c5b0 0x7facf01a17e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7face800d010 tx=0x7face8004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:54.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.162+0000 7face77fe700 1 -- 172.21.15.2:0/1403735676 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8021070 con 0x7facf010c5b0 2024-05-14T09:37:54.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.162+0000 7face77fe700 1 -- 172.21.15.2:0/1403735676 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face8007e30 con 0x7facf010c5b0 2024-05-14T09:37:54.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.162+0000 7face77fe700 1 -- 172.21.15.2:0/1403735676 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80045a0 con 0x7facf010c5b0 2024-05-14T09:37:54.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.162+0000 7facf4e1d700 1 -- 172.21.15.2:0/1403735676 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf01a2060 con 0x7facf010c5b0 2024-05-14T09:37:54.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.162+0000 7facf4e1d700 1 -- 172.21.15.2:0/1403735676 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf01a2550 con 0x7facf010c5b0 2024-05-14T09:37:54.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.165+0000 7face77fe700 1 -- 172.21.15.2:0/1403735676 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7face8004740 con 0x7facf010c5b0 2024-05-14T09:37:54.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.166+0000 7face77fe700 1 --2- 172.21.15.2:0/1403735676 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7facdc0b2880 0x7facdc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.166+0000 7facee59c700 1 --2- 172.21.15.2:0/1403735676 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7facdc0b2880 0x7facdc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.166+0000 7face77fe700 1 -- 172.21.15.2:0/1403735676 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7face800f020 con 0x7facf010c5b0 2024-05-14T09:37:54.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.166+0000 7fad03831b80 1 Processor -- start 2024-05-14T09:37:54.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.166+0000 7facee59c700 1 --2- 172.21.15.2:0/1403735676 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7facdc0b2880 0x7facdc0b4d90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7facf01079e0 tx=0x7face000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:54.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7fad03831b80 1 -- start start 2024-05-14T09:37:54.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7fad03831b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867b08e40 0x55b867c8c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7fad03831b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c8cd80 0x55b867c868d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7fad03831b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c86e10 0x55b867c87220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7fad03831b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b867ba8940 con 0x55b867c86e10 2024-05-14T09:37:54.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7fad03831b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b867b0dda0 con 0x55b867c8cd80 2024-05-14T09:37:54.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7fad03831b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b867ba4900 con 0x55b867b08e40 2024-05-14T09:37:54.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7faceed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c86e10 0x55b867c87220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7facee59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867b08e40 0x55b867c8c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7faceed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c86e10 0x55b867c87220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44346/0 (socket says 172.21.15.2:44346) 2024-05-14T09:37:54.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7faceed9d700 1 -- 172.21.15.2:0/1662239028 learned_addr learned my addr 172.21.15.2:0/1662239028 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:54.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7face7fff700 1 --2- 172.21.15.2:0/1662239028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c8cd80 0x55b867c868d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7faceed9d700 1 -- 172.21.15.2:0/1662239028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867b08e40 msgr2=0x55b867c8c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:54.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7faceed9d700 1 --2- 172.21.15.2:0/1662239028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867b08e40 0x55b867c8c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7faceed9d700 1 -- 172.21.15.2:0/1662239028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c8cd80 msgr2=0x55b867c868d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:54.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7faceed9d700 1 --2- 172.21.15.2:0/1662239028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c8cd80 0x55b867c868d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.167+0000 7faceed9d700 1 -- 172.21.15.2:0/1662239028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b867af58d0 con 0x55b867c86e10 2024-05-14T09:37:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.168+0000 7faceed9d700 1 --2- 172.21.15.2:0/1662239028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c86e10 0x55b867c87220 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7face80a6d30 tx=0x7face80aa730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2265fdad224b8197 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.168+0000 7face4ff9700 1 -- 172.21.15.2:0/1662239028 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80b33d0 con 0x55b867c86e10 2024-05-14T09:37:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.168+0000 7face4ff9700 1 -- 172.21.15.2:0/1662239028 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face80aacb0 con 0x55b867c86e10 2024-05-14T09:37:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.168+0000 7face4ff9700 1 -- 172.21.15.2:0/1662239028 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80aaf10 con 0x55b867c86e10 2024-05-14T09:37:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.169+0000 7fad03831b80 1 -- 172.21.15.2:0/1662239028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c86e10 msgr2=0x55b867c87220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.169+0000 7fad03831b80 1 --2- 172.21.15.2:0/1662239028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c86e10 0x55b867c87220 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7face80a6d30 tx=0x7face80aa730 comp rx=0 tx=0).stop 2024-05-14T09:37:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.169+0000 7fad03831b80 1 -- 172.21.15.2:0/1662239028 shutdown_connections 2024-05-14T09:37:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.169+0000 7fad03831b80 1 --2- 172.21.15.2:0/1662239028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c86e10 0x55b867c87220 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.169+0000 7fad03831b80 1 --2- 172.21.15.2:0/1662239028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867b08e40 0x55b867c8c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.169+0000 7fad03831b80 1 --2- 172.21.15.2:0/1662239028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c8cd80 0x55b867c868d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.169+0000 7fad03831b80 1 -- 172.21.15.2:0/1662239028 >> 172.21.15.2:0/1662239028 conn(0x55b867b50630 msgr2=0x55b867c81a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.169+0000 7fad03831b80 1 -- 172.21.15.2:0/1662239028 shutdown_connections 2024-05-14T09:37:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.169+0000 7fad03831b80 1 -- 172.21.15.2:0/1662239028 wait complete. 2024-05-14T09:37:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.169+0000 7fad03831b80 1 -- start start 2024-05-14T09:37:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.170+0000 7fad03831b80 1 Processor -- start 2024-05-14T09:37:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.170+0000 7fad03831b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867b08e40 0x55b867c90540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.170+0000 7fad03831b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867c86e10 0x55b867c90a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.170+0000 7fad03831b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c8cd80 0x55b867c90fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.170+0000 7fad03831b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b867c935e0 con 0x55b867b08e40 2024-05-14T09:37:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.170+0000 7fad03831b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b867b61ed0 con 0x55b867c8cd80 2024-05-14T09:37:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.170+0000 7fad03831b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b867c93760 con 0x55b867c86e10 2024-05-14T09:37:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.170+0000 7fad03831b80 10 client.0 _ll_register_callbacks cb 0x55b867b40950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.170+0000 7faceed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c8cd80 0x55b867c90fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.170+0000 7faceed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c8cd80 0x55b867c90fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:48108/0 (socket says 172.21.15.2:48108) 2024-05-14T09:37:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.170+0000 7face7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867c86e10 0x55b867c90a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.170+0000 7faceed9d700 1 -- 172.21.15.2:0/2159166647 learned_addr learned my addr 172.21.15.2:0/2159166647 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.170+0000 7facee59c700 1 --2- 172.21.15.2:0/2159166647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867b08e40 0x55b867c90540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.171+0000 7faceed9d700 1 -- 172.21.15.2:0/2159166647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867c86e10 msgr2=0x55b867c90a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.171+0000 7faceed9d700 1 --2- 172.21.15.2:0/2159166647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867c86e10 0x55b867c90a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.171+0000 7faceed9d700 1 -- 172.21.15.2:0/2159166647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867b08e40 msgr2=0x55b867c90540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.171+0000 7faceed9d700 1 --2- 172.21.15.2:0/2159166647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867b08e40 0x55b867c90540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.171+0000 7faceed9d700 1 -- 172.21.15.2:0/2159166647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b867b5f760 con 0x55b867c8cd80 2024-05-14T09:37:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.171+0000 7face7fff700 1 --2- 172.21.15.2:0/2159166647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867c86e10 0x55b867c90a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.171+0000 7faceed9d700 1 --2- 172.21.15.2:0/2159166647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c8cd80 0x55b867c90fc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7face80aa670 tx=0x7face80b34c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af1ce436e724ddb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.171+0000 7facd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:37:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.171+0000 7facd5ffb700 1 -- 172.21.15.2:0/2159166647 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80c3070 con 0x55b867c8cd80 2024-05-14T09:37:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.171+0000 7fad03831b80 1 -- 172.21.15.2:0/2159166647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7face80aacb0 con 0x55b867c8cd80 2024-05-14T09:37:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.171+0000 7facd5ffb700 1 -- 172.21.15.2:0/2159166647 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7face80ab120 con 0x55b867c8cd80 2024-05-14T09:37:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.171+0000 7facd5ffb700 1 -- 172.21.15.2:0/2159166647 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7face80c52c0 con 0x55b867c8cd80 2024-05-14T09:37:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.171+0000 7fad03831b80 10 client.25631 fetch_fsmap learned FSMap version 29 2024-05-14T09:37:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.171+0000 7fad03831b80 1 -- 172.21.15.2:0/2159166647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b867c935e0 con 0x55b867c8cd80 2024-05-14T09:37:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.171+0000 7facd5ffb700 1 -- 172.21.15.2:0/2159166647 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face80aacb0 con 0x55b867c8cd80 2024-05-14T09:37:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.172+0000 7facd5ffb700 1 -- 172.21.15.2:0/2159166647 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7face8011070 con 0x55b867c8cd80 2024-05-14T09:37:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.172+0000 7fad03831b80 10 client.25631 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:37:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.172+0000 7fad03831b80 10 client.25631 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T09:37:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.172+0000 7fad03831b80 20 client.25631 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.172+0000 7fad03831b80 1 --2- 172.21.15.2:0/2159166647 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55b867ca0650 0x55b867ca2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.172+0000 7fad03831b80 4 client.25631 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:37:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.172+0000 7fad03831b80 1 -- 172.21.15.2:0/2159166647 --> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b867bb6320 con 0x55b867ca0650 2024-05-14T09:37:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.172+0000 7facee59c700 1 --2- 172.21.15.2:0/2159166647 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55b867ca0650 0x55b867ca2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.172+0000 7facd5ffb700 10 client.25631 ms_handle_connect on v2:172.21.15.29:6838/3625063120 2024-05-14T09:37:54.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.173+0000 7facee59c700 1 --2- 172.21.15.2:0/2159166647 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55b867ca0650 0x55b867ca2aa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=262f28c12b7d3541 server_cookie=f3a2e1c1469f127b in_seq=0 out_seq=0 2024-05-14T09:37:54.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.174+0000 7facd5ffb700 1 -- 172.21.15.2:0/2159166647 <== mds.2 v2:172.21.15.29:6838/3625063120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b867bb6320 con 0x55b867ca0650 2024-05-14T09:37:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.174+0000 7facd5ffb700 10 client.25631 handle_command_reply: tid=0 2024-05-14T09:37:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.174+0000 7fad03831b80 1 client.25631 shutdown 2024-05-14T09:37:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.174+0000 7fad03831b80 1 -- 172.21.15.2:0/2159166647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c8cd80 msgr2=0x55b867c90fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.174+0000 7fad03831b80 1 --2- 172.21.15.2:0/2159166647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c8cd80 0x55b867c90fc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7face80aa670 tx=0x7face80b34c0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.174+0000 7fad03831b80 1 -- 172.21.15.2:0/2159166647 shutdown_connections 2024-05-14T09:37:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.174+0000 7fad03831b80 1 --2- 172.21.15.2:0/2159166647 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55b867ca0650 0x55b867ca2aa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.174+0000 7fad03831b80 1 --2- 172.21.15.2:0/2159166647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c8cd80 0x55b867c90fc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.174+0000 7fad03831b80 1 --2- 172.21.15.2:0/2159166647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867b08e40 0x55b867c90540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.174+0000 7facd5ffb700 0 client.25631 ms_handle_reset on v2:172.21.15.29:6838/3625063120 2024-05-14T09:37:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.174+0000 7fad03831b80 1 --2- 172.21.15.2:0/2159166647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867c86e10 0x55b867c90a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.174+0000 7fad03831b80 1 -- 172.21.15.2:0/2159166647 >> 172.21.15.2:0/2159166647 conn(0x55b867b50630 msgr2=0x55b867c858c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.174+0000 7fad03831b80 1 -- 172.21.15.2:0/2159166647 shutdown_connections 2024-05-14T09:37:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.174+0000 7fad03831b80 1 -- 172.21.15.2:0/2159166647 wait complete. 2024-05-14T09:37:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.174+0000 7fad03831b80 20 client.25631 trim_cache size 0 max 16384 2024-05-14T09:37:54.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.191+0000 7fad03831b80 1 Processor -- start 2024-05-14T09:37:54.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.191+0000 7fad03831b80 1 -- start start 2024-05-14T09:37:54.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7fad03831b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867b08e40 0x55b867c868d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7fad03831b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867c86e10 0x55b867c9c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7fad03831b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c8cd80 0x55b867c8a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7fad03831b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7facf01b4e50 con 0x55b867c8cd80 2024-05-14T09:37:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7fad03831b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7facf01b26e0 con 0x55b867b08e40 2024-05-14T09:37:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7fad03831b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7facf01b0530 con 0x55b867c86e10 2024-05-14T09:37:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7faceed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c8cd80 0x55b867c8a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7faceed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c8cd80 0x55b867c8a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44360/0 (socket says 172.21.15.2:44360) 2024-05-14T09:37:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7faceed9d700 1 -- 172.21.15.2:0/1385394837 learned_addr learned my addr 172.21.15.2:0/1385394837 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7face7fff700 1 --2- 172.21.15.2:0/1385394837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867c86e10 0x55b867c9c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7facee59c700 1 --2- 172.21.15.2:0/1385394837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867b08e40 0x55b867c868d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7faceed9d700 1 -- 172.21.15.2:0/1385394837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867c86e10 msgr2=0x55b867c9c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7faceed9d700 1 --2- 172.21.15.2:0/1385394837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867c86e10 0x55b867c9c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7faceed9d700 1 -- 172.21.15.2:0/1385394837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867b08e40 msgr2=0x55b867c868d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7faceed9d700 1 --2- 172.21.15.2:0/1385394837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867b08e40 0x55b867c868d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7faceed9d700 1 -- 172.21.15.2:0/1385394837 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b867c8c3a0 con 0x55b867c8cd80 2024-05-14T09:37:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7face7fff700 1 --2- 172.21.15.2:0/1385394837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867c86e10 0x55b867c9c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.192+0000 7faceed9d700 1 --2- 172.21.15.2:0/1385394837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c8cd80 0x55b867c8a860 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7face80a6d30 tx=0x7face80aa730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4584638cd763da8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.193+0000 7facd7fff700 1 -- 172.21.15.2:0/1385394837 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80c3070 con 0x55b867c8cd80 2024-05-14T09:37:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.193+0000 7facd7fff700 1 -- 172.21.15.2:0/1385394837 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face80aa7c0 con 0x55b867c8cd80 2024-05-14T09:37:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.193+0000 7facd7fff700 1 -- 172.21.15.2:0/1385394837 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80b35f0 con 0x55b867c8cd80 2024-05-14T09:37:54.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.194+0000 7fad03831b80 1 -- 172.21.15.2:0/1385394837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c8cd80 msgr2=0x55b867c8a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:54.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.194+0000 7fad03831b80 1 --2- 172.21.15.2:0/1385394837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c8cd80 0x55b867c8a860 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7face80a6d30 tx=0x7face80aa730 comp rx=0 tx=0).stop 2024-05-14T09:37:54.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.194+0000 7fad03831b80 1 -- 172.21.15.2:0/1385394837 shutdown_connections 2024-05-14T09:37:54.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.194+0000 7fad03831b80 1 --2- 172.21.15.2:0/1385394837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867b08e40 0x55b867c868d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.194+0000 7fad03831b80 1 --2- 172.21.15.2:0/1385394837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c8cd80 0x55b867c8a860 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.194+0000 7fad03831b80 1 --2- 172.21.15.2:0/1385394837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867c86e10 0x55b867c9c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.194+0000 7fad03831b80 1 -- 172.21.15.2:0/1385394837 >> 172.21.15.2:0/1385394837 conn(0x55b867b50630 msgr2=0x55b867c8a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:54.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.194+0000 7fad03831b80 1 -- 172.21.15.2:0/1385394837 shutdown_connections 2024-05-14T09:37:54.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.194+0000 7fad03831b80 1 -- 172.21.15.2:0/1385394837 wait complete. 2024-05-14T09:37:54.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.194+0000 7fad03831b80 1 -- start start 2024-05-14T09:37:54.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.194+0000 7fad03831b80 1 Processor -- start 2024-05-14T09:37:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.194+0000 7fad03831b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867b08e40 0x55b867c95880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7fad03831b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c86e10 0x55b867c95dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7fad03831b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c8cd80 0x55b867c96300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7fad03831b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7facf01b0ba0 con 0x55b867c8cd80 2024-05-14T09:37:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7fad03831b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7facf01b2010 con 0x55b867c86e10 2024-05-14T09:37:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7fad03831b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7facf01b1270 con 0x55b867b08e40 2024-05-14T09:37:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7fad03831b80 10 client.0 _ll_register_callbacks cb 0x55b867b40950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7face7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c86e10 0x55b867c95dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7facee59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867b08e40 0x55b867c95880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7face7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c86e10 0x55b867c95dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:48124/0 (socket says 172.21.15.2:48124) 2024-05-14T09:37:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7facee59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867b08e40 0x55b867c95880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:40978/0 (socket says 172.21.15.2:40978) 2024-05-14T09:37:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7face7fff700 1 -- 172.21.15.2:0/2800752078 learned_addr learned my addr 172.21.15.2:0/2800752078 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7faceed9d700 1 --2- 172.21.15.2:0/2800752078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c8cd80 0x55b867c96300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7face7fff700 1 -- 172.21.15.2:0/2800752078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867b08e40 msgr2=0x55b867c95880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7face7fff700 1 --2- 172.21.15.2:0/2800752078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867b08e40 0x55b867c95880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7face7fff700 1 -- 172.21.15.2:0/2800752078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c8cd80 msgr2=0x55b867c96300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7face7fff700 1 --2- 172.21.15.2:0/2800752078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c8cd80 0x55b867c96300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7face7fff700 1 -- 172.21.15.2:0/2800752078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b867d08f90 con 0x55b867c86e10 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7facee59c700 1 --2- 172.21.15.2:0/2800752078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867b08e40 0x55b867c95880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7face7fff700 1 --2- 172.21.15.2:0/2800752078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c86e10 0x55b867c95dc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55b867c831f0 tx=0x7facd800f990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab3a52af572e720f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7facd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.195+0000 7facd5ffb700 1 -- 172.21.15.2:0/2800752078 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8004230 con 0x55b867c86e10 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.196+0000 7facd5ffb700 1 -- 172.21.15.2:0/2800752078 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7facd80043d0 con 0x55b867c86e10 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.196+0000 7fad03831b80 1 -- 172.21.15.2:0/2800752078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7face80aa7c0 con 0x55b867c86e10 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.196+0000 7facd5ffb700 1 -- 172.21.15.2:0/2800752078 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7facd802b050 con 0x55b867c86e10 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.196+0000 7fad03831b80 10 client.25637 fetch_fsmap learned FSMap version 29 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.196+0000 7fad03831b80 1 -- 172.21.15.2:0/2800752078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b867d0a230 con 0x55b867c86e10 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.196+0000 7facd5ffb700 1 -- 172.21.15.2:0/2800752078 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face80aa7c0 con 0x55b867c86e10 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.196+0000 7facd5ffb700 1 -- 172.21.15.2:0/2800752078 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7facd801d6b0 con 0x55b867c86e10 2024-05-14T09:37:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.196+0000 7fad03831b80 10 client.25637 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:37:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.196+0000 7fad03831b80 10 client.25637 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T09:37:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.196+0000 7fad03831b80 20 client.25637 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.196+0000 7fad03831b80 1 --2- 172.21.15.2:0/2800752078 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55b867d0a9f0 0x55b867d0ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.196+0000 7fad03831b80 4 client.25637 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T09:37:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.196+0000 7fad03831b80 1 -- 172.21.15.2:0/2800752078 --> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b867bb03c0 con 0x55b867d0a9f0 2024-05-14T09:37:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.197+0000 7facee59c700 1 --2- 172.21.15.2:0/2800752078 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55b867d0a9f0 0x55b867d0ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.197+0000 7facee59c700 1 --2- 172.21.15.2:0/2800752078 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55b867d0a9f0 0x55b867d0ce10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ce983c859f010c4 server_cookie=721bdae0bd102555 in_seq=0 out_seq=0 2024-05-14T09:37:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.197+0000 7facd5ffb700 10 client.25637 ms_handle_connect on v2:172.21.15.29:6838/3625063120 2024-05-14T09:37:54.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.198+0000 7facd5ffb700 1 -- 172.21.15.2:0/2800752078 <== mds.2 v2:172.21.15.29:6838/3625063120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b867bb03c0 con 0x55b867d0a9f0 2024-05-14T09:37:54.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.198+0000 7facd5ffb700 10 client.25637 handle_command_reply: tid=0 2024-05-14T09:37:54.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.198+0000 7fad03831b80 1 client.25637 shutdown 2024-05-14T09:37:54.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.198+0000 7fad03831b80 1 -- 172.21.15.2:0/2800752078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c86e10 msgr2=0x55b867c95dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:54.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.198+0000 7fad03831b80 1 --2- 172.21.15.2:0/2800752078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c86e10 0x55b867c95dc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55b867c831f0 tx=0x7facd800f990 comp rx=0 tx=0).stop 2024-05-14T09:37:54.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.198+0000 7fad03831b80 1 -- 172.21.15.2:0/2800752078 shutdown_connections 2024-05-14T09:37:54.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.198+0000 7fad03831b80 1 --2- 172.21.15.2:0/2800752078 >> [v2:172.21.15.29:6838/3625063120,v1:172.21.15.29:6839/3625063120] conn(0x55b867d0a9f0 0x55b867d0ce10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.198+0000 7fad03831b80 1 --2- 172.21.15.2:0/2800752078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b867c8cd80 0x55b867c96300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.198+0000 7fad03831b80 1 --2- 172.21.15.2:0/2800752078 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b867b08e40 0x55b867c95880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.198+0000 7facd5ffb700 0 client.25637 ms_handle_reset on v2:172.21.15.29:6838/3625063120 2024-05-14T09:37:54.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.198+0000 7fad03831b80 1 --2- 172.21.15.2:0/2800752078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b867c86e10 0x55b867c95dc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.198+0000 7fad03831b80 1 -- 172.21.15.2:0/2800752078 >> 172.21.15.2:0/2800752078 conn(0x55b867b50630 msgr2=0x55b867c8ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:54.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.198+0000 7fad03831b80 1 -- 172.21.15.2:0/2800752078 shutdown_connections 2024-05-14T09:37:54.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.198+0000 7fad03831b80 1 -- 172.21.15.2:0/2800752078 wait complete. 2024-05-14T09:37:54.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.198+0000 7fad03831b80 20 client.25637 trim_cache size 0 max 16384 2024-05-14T09:37:54.202 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-05-14T09:37:54.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.199+0000 7facf4e1d700 1 -- 172.21.15.2:0/1403735676 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7facdc0b2880 msgr2=0x7facdc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:54.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.199+0000 7facf4e1d700 1 --2- 172.21.15.2:0/1403735676 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7facdc0b2880 0x7facdc0b4d90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7facf01079e0 tx=0x7face000a380 comp rx=0 tx=0).stop 2024-05-14T09:37:54.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.199+0000 7facf4e1d700 1 -- 172.21.15.2:0/1403735676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facf010c5b0 msgr2=0x7facf01a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:54.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.199+0000 7facf4e1d700 1 --2- 172.21.15.2:0/1403735676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facf010c5b0 0x7facf01a17e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7face800d010 tx=0x7face8004d00 comp rx=0 tx=0).stop 2024-05-14T09:37:54.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.199+0000 7facf4e1d700 1 -- 172.21.15.2:0/1403735676 shutdown_connections 2024-05-14T09:37:54.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.199+0000 7facf4e1d700 1 --2- 172.21.15.2:0/1403735676 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7facdc0b2880 0x7facdc0b4d90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.199+0000 7facf4e1d700 1 --2- 172.21.15.2:0/1403735676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facf0106570 0x7facf019cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.199+0000 7facf4e1d700 1 --2- 172.21.15.2:0/1403735676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7facf010c5b0 0x7facf01a17e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.199+0000 7facf4e1d700 1 --2- 172.21.15.2:0/1403735676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facf0106ec0 0x7facf019d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:54.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.199+0000 7facf4e1d700 1 -- 172.21.15.2:0/1403735676 >> 172.21.15.2:0/1403735676 conn(0x7facf0102050 msgr2=0x7facf0103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:54.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.199+0000 7facf4e1d700 1 -- 172.21.15.2:0/1403735676 shutdown_connections 2024-05-14T09:37:54.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:54.199+0000 7facf4e1d700 1 -- 172.21.15.2:0/1403735676 wait complete. 2024-05-14T09:37:54.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 163602 11.26 MB/sec warmup 80 sec latency 45.553 ms 2024-05-14T09:37:54.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:54 smithi029 ceph-mon[36249]: pgmap v512: 129 pgs: 129 active+clean; 52 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 338 op/s 2024-05-14T09:37:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:54 smithi026 ceph-mon[36056]: pgmap v512: 129 pgs: 129 active+clean; 52 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 338 op/s 2024-05-14T09:37:54.715 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:54 smithi002 ceph-mon[31557]: pgmap v512: 129 pgs: 129 active+clean; 52 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 338 op/s 2024-05-14T09:37:55.034 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T09:37:55.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 165736 11.22 MB/sec warmup 81 sec latency 19.020 ms 2024-05-14T09:37:55.295 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:37:56.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 167456 11.22 MB/sec warmup 82 sec latency 17.561 ms 2024-05-14T09:37:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:56 smithi029 ceph-mon[36249]: pgmap v513: 129 pgs: 129 active+clean; 52 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-05-14T09:37:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:56 smithi026 ceph-mon[36056]: pgmap v513: 129 pgs: 129 active+clean; 52 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-05-14T09:37:56.705 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:56 smithi002 ceph-mon[31557]: pgmap v513: 129 pgs: 129 active+clean; 52 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-05-14T09:37:57.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.129+0000 7fd5f00c9700 1 -- 172.21.15.2:0/3148258971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e8109300 msgr2=0x7fd5e8109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:57.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.129+0000 7fd5f00c9700 1 --2- 172.21.15.2:0/3148258971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e8109300 0x7fd5e8109710 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009990 tx=0x7fd5e4009ca0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.129+0000 7fd5f00c9700 1 -- 172.21.15.2:0/3148258971 shutdown_connections 2024-05-14T09:37:57.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.129+0000 7fd5f00c9700 1 --2- 172.21.15.2:0/3148258971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5e81030e0 0x7fd5e8111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.129+0000 7fd5f00c9700 1 --2- 172.21.15.2:0/3148258971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e8109300 0x7fd5e8109710 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.129+0000 7fd5f00c9700 1 --2- 172.21.15.2:0/3148258971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5e8102790 0x7fd5e8102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.129+0000 7fd5f00c9700 1 -- 172.21.15.2:0/3148258971 >> 172.21.15.2:0/3148258971 conn(0x7fd5e80fe520 msgr2=0x7fd5e8100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:57.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.129+0000 7fd5f00c9700 1 -- 172.21.15.2:0/3148258971 shutdown_connections 2024-05-14T09:37:57.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.129+0000 7fd5f00c9700 1 -- 172.21.15.2:0/3148258971 wait complete. 2024-05-14T09:37:57.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.130+0000 7fd5f00c9700 1 Processor -- start 2024-05-14T09:37:57.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.130+0000 7fd5f00c9700 1 -- start start 2024-05-14T09:37:57.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.130+0000 7fd5f00c9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e8102790 0x7fd5e819af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.130+0000 7fd5f00c9700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5e81030e0 0x7fd5e819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.130+0000 7fd5f00c9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5e8109300 0x7fd5e819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.130+0000 7fd5f00c9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e81145e0 con 0x7fd5e8102790 2024-05-14T09:37:57.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.130+0000 7fd5f00c9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e8114460 con 0x7fd5e8109300 2024-05-14T09:37:57.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.130+0000 7fd5f00c9700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e8114760 con 0x7fd5e81030e0 2024-05-14T09:37:57.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.130+0000 7fd5ede65700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e8102790 0x7fd5e819af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5ede65700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e8102790 0x7fd5e819af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44392/0 (socket says 172.21.15.2:44392) 2024-05-14T09:37:57.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5ede65700 1 -- 172.21.15.2:0/3917440160 learned_addr learned my addr 172.21.15.2:0/3917440160 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:57.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5ee666700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5e8109300 0x7fd5e819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5ed664700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5e81030e0 0x7fd5e819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5ede65700 1 -- 172.21.15.2:0/3917440160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5e81030e0 msgr2=0x7fd5e819b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:57.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5ede65700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5e81030e0 0x7fd5e819b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5ede65700 1 -- 172.21.15.2:0/3917440160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5e8109300 msgr2=0x7fd5e819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:57.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5ede65700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5e8109300 0x7fd5e819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5ede65700 1 -- 172.21.15.2:0/3917440160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e819fe20 con 0x7fd5e8102790 2024-05-14T09:37:57.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5ed664700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5e81030e0 0x7fd5e819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:37:57.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5ee666700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5e8109300 0x7fd5e819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:57.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5ede65700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e8102790 0x7fd5e819af50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4000c00 tx=0x7fd5e40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:57.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5deffd700 1 -- 172.21.15.2:0/3917440160 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4026070 con 0x7fd5e8102790 2024-05-14T09:37:57.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5f00c9700 1 -- 172.21.15.2:0/3917440160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e81a00b0 con 0x7fd5e8102790 2024-05-14T09:37:57.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5deffd700 1 -- 172.21.15.2:0/3917440160 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e4021410 con 0x7fd5e8102790 2024-05-14T09:37:57.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5deffd700 1 -- 172.21.15.2:0/3917440160 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4017650 con 0x7fd5e8102790 2024-05-14T09:37:57.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.131+0000 7fd5f00c9700 1 -- 172.21.15.2:0/3917440160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e81a7a30 con 0x7fd5e8102790 2024-05-14T09:37:57.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.134+0000 7fd5deffd700 1 -- 172.21.15.2:0/3917440160 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd5e4029070 con 0x7fd5e8102790 2024-05-14T09:37:57.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.135+0000 7fd5feaddb80 1 Processor -- start 2024-05-14T09:37:57.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.135+0000 7fd5feaddb80 1 -- start start 2024-05-14T09:37:57.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5deffd700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd5d40b2900 0x7fd5d40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5feaddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec76bfe40 0x562ec783d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5feaddb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec783dc60 0x562ec7841f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5feaddb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec7842470 0x562ec7842880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5feaddb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562ec775f860 con 0x562ec783dc60 2024-05-14T09:37:57.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5feaddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562ec76fe7c0 con 0x562ec76bfe40 2024-05-14T09:37:57.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5feaddb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562ec775b820 con 0x562ec7842470 2024-05-14T09:37:57.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5ed664700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec783dc60 0x562ec7841f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5ede65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec76bfe40 0x562ec783d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5ed664700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec783dc60 0x562ec7841f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44406/0 (socket says 172.21.15.2:44406) 2024-05-14T09:37:57.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5ed664700 1 -- 172.21.15.2:0/3113214459 learned_addr learned my addr 172.21.15.2:0/3113214459 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:57.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5ee666700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd5d40b2900 0x7fd5d40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5deffd700 1 -- 172.21.15.2:0/3917440160 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd5e40e3230 con 0x7fd5e8102790 2024-05-14T09:37:57.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5ede65700 1 -- 172.21.15.2:0/3113214459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec7842470 msgr2=0x562ec7842880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:57.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5ede65700 1 --2- 172.21.15.2:0/3113214459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec7842470 0x562ec7842880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5ede65700 1 -- 172.21.15.2:0/3113214459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec783dc60 msgr2=0x562ec7841f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:57.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5ede65700 1 --2- 172.21.15.2:0/3113214459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec783dc60 0x562ec7841f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5ede65700 1 -- 172.21.15.2:0/3113214459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ec76ac8d0 con 0x562ec76bfe40 2024-05-14T09:37:57.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5ed664700 1 --2- 172.21.15.2:0/3113214459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec783dc60 0x562ec7841f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:57.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5ee666700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd5d40b2900 0x7fd5d40b4e10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0009ec0 tx=0x7fd5e0009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:57.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.136+0000 7fd5ede65700 1 --2- 172.21.15.2:0/3113214459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec76bfe40 0x562ec783d720 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40a6540 tx=0x7fd5e4027f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8db2d9aafb17c952 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:57.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.137+0000 7fd5d3fff700 1 -- 172.21.15.2:0/3113214459 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40afa00 con 0x562ec76bfe40 2024-05-14T09:37:57.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.137+0000 7fd5d3fff700 1 -- 172.21.15.2:0/3113214459 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e40920d0 con 0x562ec76bfe40 2024-05-14T09:37:57.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.137+0000 7fd5d3fff700 1 -- 172.21.15.2:0/3113214459 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40aabc0 con 0x562ec76bfe40 2024-05-14T09:37:57.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.138+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3113214459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec76bfe40 msgr2=0x562ec783d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:57.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.138+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/3113214459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec76bfe40 0x562ec783d720 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40a6540 tx=0x7fd5e4027f90 comp rx=0 tx=0).stop 2024-05-14T09:37:57.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.138+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3113214459 shutdown_connections 2024-05-14T09:37:57.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.138+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/3113214459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec76bfe40 0x562ec783d720 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.138+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/3113214459 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec7842470 0x562ec7842880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.138+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/3113214459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec783dc60 0x562ec7841f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.138+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3113214459 >> 172.21.15.2:0/3113214459 conn(0x562ec7707600 msgr2=0x562ec78384b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:57.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.138+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3113214459 shutdown_connections 2024-05-14T09:37:57.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.138+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3113214459 wait complete. 2024-05-14T09:37:57.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.138+0000 7fd5feaddb80 1 -- start start 2024-05-14T09:37:57.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.138+0000 7fd5feaddb80 1 Processor -- start 2024-05-14T09:37:57.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5feaddb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec76bfe40 0x562ec7847150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5feaddb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec783dc60 0x562ec7847690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5feaddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec7842470 0x562ec7847bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5feaddb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562ec76fe7c0 con 0x562ec783dc60 2024-05-14T09:37:57.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5feaddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562ec76c5c30 con 0x562ec7842470 2024-05-14T09:37:57.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5feaddb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562ec784a1e0 con 0x562ec76bfe40 2024-05-14T09:37:57.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5feaddb80 10 client.0 _ll_register_callbacks cb 0x562ec76f7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:57.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5ed664700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec783dc60 0x562ec7847690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5ed664700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec783dc60 0x562ec7847690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44416/0 (socket says 172.21.15.2:44416) 2024-05-14T09:37:57.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5ed664700 1 -- 172.21.15.2:0/44777684 learned_addr learned my addr 172.21.15.2:0/44777684 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:57.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5ee666700 1 --2- 172.21.15.2:0/44777684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec7842470 0x562ec7847bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5ede65700 1 --2- 172.21.15.2:0/44777684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec76bfe40 0x562ec7847150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5ede65700 1 -- 172.21.15.2:0/44777684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec7842470 msgr2=0x562ec7847bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5ede65700 1 --2- 172.21.15.2:0/44777684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec7842470 0x562ec7847bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5ede65700 1 -- 172.21.15.2:0/44777684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec783dc60 msgr2=0x562ec7847690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5ede65700 1 --2- 172.21.15.2:0/44777684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec783dc60 0x562ec7847690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5ede65700 1 -- 172.21.15.2:0/44777684 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ec76fb690 con 0x562ec76bfe40 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5ee666700 1 --2- 172.21.15.2:0/44777684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec7842470 0x562ec7847bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.139+0000 7fd5ed664700 1 --2- 172.21.15.2:0/44777684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec783dc60 0x562ec7847690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.140+0000 7fd5ede65700 1 --2- 172.21.15.2:0/44777684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec76bfe40 0x562ec7847150 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40aa1c0 tx=0x7fd5e40aa590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90365f820299810c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.140+0000 7fd5d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.140+0000 7fd5d17fa700 1 -- 172.21.15.2:0/44777684 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40ab5f0 con 0x562ec76bfe40 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.140+0000 7fd5feaddb80 1 -- 172.21.15.2:0/44777684 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5e40920d0 con 0x562ec76bfe40 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.140+0000 7fd5d17fa700 1 -- 172.21.15.2:0/44777684 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd5e40afa00 con 0x562ec76bfe40 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.140+0000 7fd5d17fa700 1 -- 172.21.15.2:0/44777684 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5e40a9fd0 con 0x562ec76bfe40 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.140+0000 7fd5feaddb80 10 client.25636 fetch_fsmap learned FSMap version 29 2024-05-14T09:37:57.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.140+0000 7fd5feaddb80 1 -- 172.21.15.2:0/44777684 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ec7854c90 con 0x562ec76bfe40 2024-05-14T09:37:57.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.140+0000 7fd5d17fa700 1 -- 172.21.15.2:0/44777684 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e40920d0 con 0x562ec76bfe40 2024-05-14T09:37:57.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.141+0000 7fd5d17fa700 1 -- 172.21.15.2:0/44777684 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd5e40afce0 con 0x562ec76bfe40 2024-05-14T09:37:57.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.141+0000 7fd5feaddb80 10 client.25636 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:37:57.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.141+0000 7fd5feaddb80 10 client.25636 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T09:37:57.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.141+0000 7fd5feaddb80 20 client.25636 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:57.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.141+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/44777684 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x562ec7857160 0x562ec7859580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.141+0000 7fd5feaddb80 4 client.25636 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:37:57.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.141+0000 7fd5feaddb80 1 -- 172.21.15.2:0/44777684 --> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ec776d240 con 0x562ec7857160 2024-05-14T09:37:57.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.141+0000 7fd5ed664700 1 --2- 172.21.15.2:0/44777684 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x562ec7857160 0x562ec7859580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.141+0000 7fd5d17fa700 10 client.25636 ms_handle_connect on v2:172.21.15.26:6836/3301773578 2024-05-14T09:37:57.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.141+0000 7fd5ed664700 1 --2- 172.21.15.2:0/44777684 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x562ec7857160 0x562ec7859580 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9a68891c209b05c server_cookie=4f95442ce30a143d in_seq=0 out_seq=0 2024-05-14T09:37:57.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.142+0000 7fd5d17fa700 1 -- 172.21.15.2:0/44777684 <== mds.1 v2:172.21.15.26:6836/3301773578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562ec776d240 con 0x562ec7857160 2024-05-14T09:37:57.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.142+0000 7fd5d17fa700 10 client.25636 handle_command_reply: tid=0 2024-05-14T09:37:57.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.142+0000 7fd5feaddb80 1 client.25636 shutdown 2024-05-14T09:37:57.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.142+0000 7fd5feaddb80 1 -- 172.21.15.2:0/44777684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec76bfe40 msgr2=0x562ec7847150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:57.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.142+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/44777684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec76bfe40 0x562ec7847150 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40aa1c0 tx=0x7fd5e40aa590 comp rx=0 tx=0).stop 2024-05-14T09:37:57.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.142+0000 7fd5feaddb80 1 -- 172.21.15.2:0/44777684 shutdown_connections 2024-05-14T09:37:57.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.142+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/44777684 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x562ec7857160 0x562ec7859580 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.142+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/44777684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec7842470 0x562ec7847bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.142+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/44777684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec76bfe40 0x562ec7847150 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.142+0000 7fd5d17fa700 0 client.25636 ms_handle_reset on v2:172.21.15.26:6836/3301773578 2024-05-14T09:37:57.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.142+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/44777684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec783dc60 0x562ec7847690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.142+0000 7fd5feaddb80 1 -- 172.21.15.2:0/44777684 >> 172.21.15.2:0/44777684 conn(0x562ec7707600 msgr2=0x562ec7842880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:57.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.143+0000 7fd5feaddb80 1 -- 172.21.15.2:0/44777684 shutdown_connections 2024-05-14T09:37:57.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.143+0000 7fd5feaddb80 1 -- 172.21.15.2:0/44777684 wait complete. 2024-05-14T09:37:57.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.143+0000 7fd5feaddb80 20 client.25636 trim_cache size 0 max 16384 2024-05-14T09:37:57.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.162+0000 7fd5feaddb80 1 Processor -- start 2024-05-14T09:37:57.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.162+0000 7fd5feaddb80 1 -- start start 2024-05-14T09:37:57.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.162+0000 7fd5feaddb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec76bfe40 0x562ec7838610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.162+0000 7fd5feaddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec783dc60 0x562ec7843890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.162+0000 7fd5feaddb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec7842470 0x562ec7853150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5feaddb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e81b0ac0 con 0x562ec7842470 2024-05-14T09:37:57.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5feaddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e81ae350 con 0x562ec783dc60 2024-05-14T09:37:57.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5feaddb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e81ac1a0 con 0x562ec76bfe40 2024-05-14T09:37:57.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5ede65700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec76bfe40 0x562ec7838610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5ee666700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec7842470 0x562ec7853150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5ede65700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec76bfe40 0x562ec7838610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:41034/0 (socket says 172.21.15.2:41034) 2024-05-14T09:37:57.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5ede65700 1 -- 172.21.15.2:0/3270828472 learned_addr learned my addr 172.21.15.2:0/3270828472 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:57.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5ee666700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec7842470 0x562ec7853150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44428/0 (socket says 172.21.15.2:44428) 2024-05-14T09:37:57.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5ed664700 1 --2- 172.21.15.2:0/3270828472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec783dc60 0x562ec7843890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5ede65700 1 -- 172.21.15.2:0/3270828472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec783dc60 msgr2=0x562ec7843890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:57.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5ede65700 1 --2- 172.21.15.2:0/3270828472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec783dc60 0x562ec7843890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5ede65700 1 -- 172.21.15.2:0/3270828472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec7842470 msgr2=0x562ec7853150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:57.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5ede65700 1 --2- 172.21.15.2:0/3270828472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec7842470 0x562ec7853150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5ede65700 1 -- 172.21.15.2:0/3270828472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ec78456c0 con 0x562ec76bfe40 2024-05-14T09:37:57.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5ede65700 1 --2- 172.21.15.2:0/3270828472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec76bfe40 0x562ec7838610 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40a7730 tx=0x7fd5e4021380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae86ed367e663577 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:57.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5d37fe700 1 -- 172.21.15.2:0/3270828472 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40920b0 con 0x562ec76bfe40 2024-05-14T09:37:57.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5d37fe700 1 -- 172.21.15.2:0/3270828472 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e40ab5b0 con 0x562ec76bfe40 2024-05-14T09:37:57.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.163+0000 7fd5d37fe700 1 -- 172.21.15.2:0/3270828472 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40b2e00 con 0x562ec76bfe40 2024-05-14T09:37:57.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.164+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3270828472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec76bfe40 msgr2=0x562ec7838610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:57.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.164+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/3270828472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec76bfe40 0x562ec7838610 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40a7730 tx=0x7fd5e4021380 comp rx=0 tx=0).stop 2024-05-14T09:37:57.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.165+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3270828472 shutdown_connections 2024-05-14T09:37:57.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.165+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/3270828472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec7842470 0x562ec7853150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.165+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/3270828472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec76bfe40 0x562ec7838610 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.165+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/3270828472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec783dc60 0x562ec7843890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.165+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3270828472 >> 172.21.15.2:0/3270828472 conn(0x562ec7707600 msgr2=0x562ec7841a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:57.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.165+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3270828472 shutdown_connections 2024-05-14T09:37:57.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.165+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3270828472 wait complete. 2024-05-14T09:37:57.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.165+0000 7fd5feaddb80 1 -- start start 2024-05-14T09:37:57.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.165+0000 7fd5feaddb80 1 Processor -- start 2024-05-14T09:37:57.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5feaddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec76bfe40 0x562ec784c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5feaddb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec783dc60 0x562ec784c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5feaddb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec7842470 0x562ec78bf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5feaddb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e81ac810 con 0x562ec7842470 2024-05-14T09:37:57.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5feaddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e81adc80 con 0x562ec76bfe40 2024-05-14T09:37:57.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5feaddb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e81acee0 con 0x562ec783dc60 2024-05-14T09:37:57.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5feaddb80 10 client.0 _ll_register_callbacks cb 0x562ec76f7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:37:57.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5ed664700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec783dc60 0x562ec784c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5ee666700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec7842470 0x562ec78bf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5ede65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec76bfe40 0x562ec784c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5ed664700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec783dc60 0x562ec784c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:41036/0 (socket says 172.21.15.2:41036) 2024-05-14T09:37:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5ee666700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec7842470 0x562ec78bf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44436/0 (socket says 172.21.15.2:44436) 2024-05-14T09:37:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5ee666700 1 -- 172.21.15.2:0/3023285307 learned_addr learned my addr 172.21.15.2:0/3023285307 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:37:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5ede65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec76bfe40 0x562ec784c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:48190/0 (socket says 172.21.15.2:48190) 2024-05-14T09:37:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5ee666700 1 -- 172.21.15.2:0/3023285307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec783dc60 msgr2=0x562ec784c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5ee666700 1 --2- 172.21.15.2:0/3023285307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec783dc60 0x562ec784c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5ee666700 1 -- 172.21.15.2:0/3023285307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec76bfe40 msgr2=0x562ec784c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:37:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5ee666700 1 --2- 172.21.15.2:0/3023285307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec76bfe40 0x562ec784c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5ee666700 1 -- 172.21.15.2:0/3023285307 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ec784d2c0 con 0x562ec7842470 2024-05-14T09:37:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5ed664700 1 --2- 172.21.15.2:0/3023285307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec783dc60 0x562ec784c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.166+0000 7fd5ede65700 1 --2- 172.21.15.2:0/3023285307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec76bfe40 0x562ec784c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:37:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.167+0000 7fd5ee666700 1 --2- 172.21.15.2:0/3023285307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec7842470 0x562ec78bf730 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000ed30 tx=0x7fd5e0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70323eda24e47c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:37:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.167+0000 7fd5d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:37:57.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.167+0000 7fd5d17fa700 1 -- 172.21.15.2:0/3023285307 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e002a030 con 0x562ec7842470 2024-05-14T09:37:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.167+0000 7fd5d17fa700 1 -- 172.21.15.2:0/3023285307 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd5e0017df0 con 0x562ec7842470 2024-05-14T09:37:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.167+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3023285307 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5e40ab5b0 con 0x562ec7842470 2024-05-14T09:37:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.167+0000 7fd5d17fa700 1 -- 172.21.15.2:0/3023285307 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5e0037050 con 0x562ec7842470 2024-05-14T09:37:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.167+0000 7fd5feaddb80 10 client.15966 fetch_fsmap learned FSMap version 29 2024-05-14T09:37:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.167+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3023285307 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ec78c0d40 con 0x562ec7842470 2024-05-14T09:37:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.168+0000 7fd5d17fa700 1 -- 172.21.15.2:0/3023285307 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e40ab5b0 con 0x562ec7842470 2024-05-14T09:37:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.168+0000 7fd5d17fa700 1 -- 172.21.15.2:0/3023285307 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd5e0019460 con 0x562ec7842470 2024-05-14T09:37:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.168+0000 7fd5feaddb80 10 client.15966 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:37:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.168+0000 7fd5feaddb80 10 client.15966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T09:37:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.168+0000 7fd5feaddb80 20 client.15966 populate_metadata read hostname 'smithi002' 2024-05-14T09:37:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.168+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/3023285307 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x562ec78c14d0 0x562ec78c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:37:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.168+0000 7fd5feaddb80 4 client.15966 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T09:37:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.168+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3023285307 --> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ec77672e0 con 0x562ec78c14d0 2024-05-14T09:37:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.168+0000 7fd5ede65700 1 --2- 172.21.15.2:0/3023285307 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x562ec78c14d0 0x562ec78c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:37:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.169+0000 7fd5d17fa700 10 client.15966 ms_handle_connect on v2:172.21.15.26:6836/3301773578 2024-05-14T09:37:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.169+0000 7fd5ede65700 1 --2- 172.21.15.2:0/3023285307 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x562ec78c14d0 0x562ec78c38f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b030335d985fa28 server_cookie=50ef6e50eaaab008 in_seq=0 out_seq=0 2024-05-14T09:37:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.169+0000 7fd5d17fa700 1 -- 172.21.15.2:0/3023285307 <== mds.1 v2:172.21.15.26:6836/3301773578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4653 (crc 0 0 0) 0x562ec77672e0 con 0x562ec78c14d0 2024-05-14T09:37:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.169+0000 7fd5d17fa700 10 client.15966 handle_command_reply: tid=0 2024-05-14T09:37:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.169+0000 7fd5feaddb80 1 client.15966 shutdown 2024-05-14T09:37:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.169+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3023285307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec7842470 msgr2=0x562ec78bf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.169+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/3023285307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec7842470 0x562ec78bf730 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000ed30 tx=0x7fd5e0004b40 comp rx=0 tx=0).stop 2024-05-14T09:37:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.169+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3023285307 shutdown_connections 2024-05-14T09:37:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.169+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/3023285307 >> [v2:172.21.15.26:6836/3301773578,v1:172.21.15.26:6837/3301773578] conn(0x562ec78c14d0 0x562ec78c38f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.169+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/3023285307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562ec76bfe40 0x562ec784c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.169+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/3023285307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562ec7842470 0x562ec78bf730 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.169+0000 7fd5d17fa700 0 client.15966 ms_handle_reset on v2:172.21.15.26:6836/3301773578 2024-05-14T09:37:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.169+0000 7fd5feaddb80 1 --2- 172.21.15.2:0/3023285307 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562ec783dc60 0x562ec784c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.169+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3023285307 >> 172.21.15.2:0/3023285307 conn(0x562ec7707600 msgr2=0x562ec783c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.170+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3023285307 shutdown_connections 2024-05-14T09:37:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.170+0000 7fd5feaddb80 1 -- 172.21.15.2:0/3023285307 wait complete. 2024-05-14T09:37:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.170+0000 7fd5feaddb80 20 client.15966 trim_cache size 0 max 16384 2024-05-14T09:37:57.174 INFO:teuthology.orchestra.run.smithi002.stdout:[ 2024-05-14T09:37:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 52762283, 2024-05-14T09:37:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628332, 2024-05-14T09:37:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04" 2024-05-14T09:37:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 130992758, 2024-05-14T09:37:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628344, 2024-05-14T09:37:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03" 2024-05-14T09:37:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 427194348, 2024-05-14T09:37:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628330, 2024-05-14T09:37:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB" 2024-05-14T09:37:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 537590187, 2024-05-14T09:37:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628336, 2024-05-14T09:37:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0" 2024-05-14T09:37:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 606052340, 2024-05-14T09:37:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628339, 2024-05-14T09:37:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1" 2024-05-14T09:37:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 932282660, 2024-05-14T09:37:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628340, 2024-05-14T09:37:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL" 2024-05-14T09:37:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1113424645, 2024-05-14T09:37:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628346, 2024-05-14T09:37:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL" 2024-05-14T09:37:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1660974705, 2024-05-14T09:37:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628343, 2024-05-14T09:37:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX" 2024-05-14T09:37:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 1678539431, 2024-05-14T09:37:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628338, 2024-05-14T09:37:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1" 2024-05-14T09:37:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2403355850, 2024-05-14T09:37:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628349, 2024-05-14T09:37:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL" 2024-05-14T09:37:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2442331187, 2024-05-14T09:37:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628335, 2024-05-14T09:37:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX" 2024-05-14T09:37:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 2474256081, 2024-05-14T09:37:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628333, 2024-05-14T09:37:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04" 2024-05-14T09:37:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 3130160028, 2024-05-14T09:37:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628341, 2024-05-14T09:37:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB" 2024-05-14T09:37:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 3251567932, 2024-05-14T09:37:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628347, 2024-05-14T09:37:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV" 2024-05-14T09:37:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 3258124601, 2024-05-14T09:37:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628337, 2024-05-14T09:37:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0" 2024-05-14T09:37:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 3811181793, 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628342, 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB" 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 4052639750, 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628348, 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL" 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 4065218329, 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628345, 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03" 2024-05-14T09:37:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 4135004744, 2024-05-14T09:37:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628334, 2024-05-14T09:37:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB" 2024-05-14T09:37:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T09:37:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T09:37:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "damage_type": "backtrace", 2024-05-14T09:37:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 4271314553, 2024-05-14T09:37:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 1099511628331, 2024-05-14T09:37:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX" 2024-05-14T09:37:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T09:37:57.182 INFO:teuthology.orchestra.run.smithi002.stdout:] 2024-05-14T09:37:57.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.170+0000 7fd5f00c9700 1 -- 172.21.15.2:0/3917440160 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd5d40b2900 msgr2=0x7fd5d40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:57.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.170+0000 7fd5f00c9700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd5d40b2900 0x7fd5d40b4e10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0009ec0 tx=0x7fd5e0009920 comp rx=0 tx=0).stop 2024-05-14T09:37:57.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.170+0000 7fd5f00c9700 1 -- 172.21.15.2:0/3917440160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e8102790 msgr2=0x7fd5e819af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:37:57.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.170+0000 7fd5f00c9700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e8102790 0x7fd5e819af50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4000c00 tx=0x7fd5e40047b0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.170+0000 7fd5f00c9700 1 -- 172.21.15.2:0/3917440160 shutdown_connections 2024-05-14T09:37:57.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.170+0000 7fd5f00c9700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd5d40b2900 0x7fd5d40b4e10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.170+0000 7fd5f00c9700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5e8109300 0x7fd5e819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.170+0000 7fd5f00c9700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e8102790 0x7fd5e819af50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.170+0000 7fd5f00c9700 1 --2- 172.21.15.2:0/3917440160 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd5e81030e0 0x7fd5e819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:37:57.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.170+0000 7fd5f00c9700 1 -- 172.21.15.2:0/3917440160 >> 172.21.15.2:0/3917440160 conn(0x7fd5e80fe520 msgr2=0x7fd5e8107180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:37:57.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.170+0000 7fd5f00c9700 1 -- 172.21.15.2:0/3917440160 shutdown_connections 2024-05-14T09:37:57.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:37:57.170+0000 7fd5f00c9700 1 -- 172.21.15.2:0/3917440160 wait complete. 2024-05-14T09:37:57.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 170102 11.25 MB/sec warmup 83 sec latency 37.334 ms 2024-05-14T09:37:58.213 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T09:37:58.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 171311 11.20 MB/sec warmup 84 sec latency 26.965 ms 2024-05-14T09:37:58.475 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T09:37:58.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:37:58 smithi029 ceph-mon[36249]: pgmap v514: 129 pgs: 129 active+clean; 54 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 313 op/s 2024-05-14T09:37:58.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:37:58 smithi026 ceph-mon[36056]: pgmap v514: 129 pgs: 129 active+clean; 54 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 313 op/s 2024-05-14T09:37:58.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:37:58 smithi002 ceph-mon[31557]: pgmap v514: 129 pgs: 129 active+clean; 54 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 313 op/s 2024-05-14T09:37:59.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 173983 11.25 MB/sec warmup 85 sec latency 22.465 ms 2024-05-14T09:38:00.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 175275 11.20 MB/sec warmup 86 sec latency 21.411 ms 2024-05-14T09:38:00.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.289+0000 7fd2a02d1700 1 -- 172.21.15.2:0/4193730279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2981045f0 msgr2=0x7fd298104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:00.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.289+0000 7fd2a02d1700 1 --2- 172.21.15.2:0/4193730279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2981045f0 0x7fd298104a00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd2940099c0 tx=0x7fd294009cd0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.290+0000 7fd2a02d1700 1 -- 172.21.15.2:0/4193730279 shutdown_connections 2024-05-14T09:38:00.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.290+0000 7fd2a02d1700 1 --2- 172.21.15.2:0/4193730279 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd298109870 0x7fd2981111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.290+0000 7fd2a02d1700 1 --2- 172.21.15.2:0/4193730279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2981045f0 0x7fd298104a00 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.290+0000 7fd2a02d1700 1 --2- 172.21.15.2:0/4193730279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd298108f20 0x7fd298109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.290+0000 7fd2a02d1700 1 -- 172.21.15.2:0/4193730279 >> 172.21.15.2:0/4193730279 conn(0x7fd298100040 msgr2=0x7fd298102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:00.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.290+0000 7fd2a02d1700 1 -- 172.21.15.2:0/4193730279 shutdown_connections 2024-05-14T09:38:00.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.290+0000 7fd2a02d1700 1 -- 172.21.15.2:0/4193730279 wait complete. 2024-05-14T09:38:00.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.290+0000 7fd2a02d1700 1 Processor -- start 2024-05-14T09:38:00.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd2a02d1700 1 -- start start 2024-05-14T09:38:00.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd2a02d1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2981045f0 0x7fd29819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd2a02d1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd298108f20 0x7fd29819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd2a02d1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd298109870 0x7fd29819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd2a02d1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981144a0 con 0x7fd2981045f0 2024-05-14T09:38:00.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd2a02d1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd298114320 con 0x7fd298109870 2024-05-14T09:38:00.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd2a02d1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd298114620 con 0x7fd298108f20 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd29d86c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd298108f20 0x7fd29819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd29d86c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd298108f20 0x7fd29819b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:36180/0 (socket says 172.21.15.2:36180) 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd29d86c700 1 -- 172.21.15.2:0/3474669376 learned_addr learned my addr 172.21.15.2:0/3474669376 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd29e06d700 1 --2- 172.21.15.2:0/3474669376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2981045f0 0x7fd29819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd29e86e700 1 --2- 172.21.15.2:0/3474669376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd298109870 0x7fd29819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd29d86c700 1 -- 172.21.15.2:0/3474669376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd298109870 msgr2=0x7fd29819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd29d86c700 1 --2- 172.21.15.2:0/3474669376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd298109870 0x7fd29819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd29d86c700 1 -- 172.21.15.2:0/3474669376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2981045f0 msgr2=0x7fd29819ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd29d86c700 1 --2- 172.21.15.2:0/3474669376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2981045f0 0x7fd29819ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd29d86c700 1 -- 172.21.15.2:0/3474669376 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29819fd40 con 0x7fd298108f20 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.291+0000 7fd29e86e700 1 --2- 172.21.15.2:0/3474669376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd298109870 0x7fd29819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.292+0000 7fd29d86c700 1 --2- 172.21.15.2:0/3474669376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd298108f20 0x7fd29819b410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd28800e970 tx=0x7fd28800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.292+0000 7fd28f7fe700 1 -- 172.21.15.2:0/3474669376 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288004a30 con 0x7fd298108f20 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.292+0000 7fd2a02d1700 1 -- 172.21.15.2:0/3474669376 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2981a0030 con 0x7fd298108f20 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.292+0000 7fd28f7fe700 1 -- 172.21.15.2:0/3474669376 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2880054d0 con 0x7fd298108f20 2024-05-14T09:38:00.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.292+0000 7fd2a02d1700 1 -- 172.21.15.2:0/3474669376 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2980759f0 con 0x7fd298108f20 2024-05-14T09:38:00.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.292+0000 7fd28f7fe700 1 -- 172.21.15.2:0/3474669376 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28800ccc0 con 0x7fd298108f20 2024-05-14T09:38:00.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.296+0000 7fd28f7fe700 1 -- 172.21.15.2:0/3474669376 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd288021440 con 0x7fd298108f20 2024-05-14T09:38:00.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.297+0000 7fd28f7fe700 1 --2- 172.21.15.2:0/3474669376 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd2840b2880 0x7fd2840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.297+0000 7fd2aece5b80 1 Processor -- start 2024-05-14T09:38:00.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.297+0000 7fd29e06d700 1 --2- 172.21.15.2:0/3474669376 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd2840b2880 0x7fd2840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.297+0000 7fd28f7fe700 1 -- 172.21.15.2:0/3474669376 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd2880e27c0 con 0x7fd298108f20 2024-05-14T09:38:00.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.297+0000 7fd2aece5b80 1 -- start start 2024-05-14T09:38:00.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.297+0000 7fd2aece5b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c6558e40 0x5625c66dc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.297+0000 7fd2aece5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66dcd80 0x5625c66d68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd2aece5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66d6e10 0x5625c66d7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd2aece5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5625c65f8940 con 0x5625c66d6e10 2024-05-14T09:38:00.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd2aece5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5625c655dda0 con 0x5625c66dcd80 2024-05-14T09:38:00.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd2aece5b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5625c65f4900 con 0x5625c6558e40 2024-05-14T09:38:00.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd29e06d700 1 --2- 172.21.15.2:0/3474669376 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd2840b2880 0x7fd2840b4d90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd294000c00 tx=0x7fd29401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:00.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd29d86c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66dcd80 0x5625c66d68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd29e06d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c6558e40 0x5625c66dc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd29d86c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66dcd80 0x5625c66d68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:59494/0 (socket says 172.21.15.2:59494) 2024-05-14T09:38:00.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd29e06d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c6558e40 0x5625c66dc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:36194/0 (socket says 172.21.15.2:36194) 2024-05-14T09:38:00.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd29d86c700 1 -- 172.21.15.2:0/1584168806 learned_addr learned my addr 172.21.15.2:0/1584168806 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:00.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd29e86e700 1 --2- 172.21.15.2:0/1584168806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66d6e10 0x5625c66d7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd29e06d700 1 -- 172.21.15.2:0/1584168806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66dcd80 msgr2=0x5625c66d68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:00.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd29e06d700 1 --2- 172.21.15.2:0/1584168806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66dcd80 0x5625c66d68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd29e06d700 1 -- 172.21.15.2:0/1584168806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66d6e10 msgr2=0x5625c66d7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:00.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd29e06d700 1 --2- 172.21.15.2:0/1584168806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66d6e10 0x5625c66d7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd29e06d700 1 -- 172.21.15.2:0/1584168806 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625c65458d0 con 0x5625c6558e40 2024-05-14T09:38:00.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd29e86e700 1 --2- 172.21.15.2:0/1584168806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66d6e10 0x5625c66d7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:38:00.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.298+0000 7fd29e06d700 1 --2- 172.21.15.2:0/1584168806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c6558e40 0x5625c66dc840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd294009990 tx=0x7fd294009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59574379fac25181 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:00.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.299+0000 7fd28cd47700 1 -- 172.21.15.2:0/1584168806 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294003e20 con 0x5625c6558e40 2024-05-14T09:38:00.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.299+0000 7fd28cd47700 1 -- 172.21.15.2:0/1584168806 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd29402b070 con 0x5625c6558e40 2024-05-14T09:38:00.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.299+0000 7fd28cd47700 1 -- 172.21.15.2:0/1584168806 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29400f900 con 0x5625c6558e40 2024-05-14T09:38:00.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.300+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1584168806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c6558e40 msgr2=0x5625c66dc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:00.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.300+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/1584168806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c6558e40 0x5625c66dc840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd294009990 tx=0x7fd294009cd0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.300+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1584168806 shutdown_connections 2024-05-14T09:38:00.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.300+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/1584168806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66d6e10 0x5625c66d7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.300+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/1584168806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c6558e40 0x5625c66dc840 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.300+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/1584168806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66dcd80 0x5625c66d68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.300+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1584168806 >> 172.21.15.2:0/1584168806 conn(0x5625c65a0630 msgr2=0x5625c66d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:00.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.300+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1584168806 shutdown_connections 2024-05-14T09:38:00.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.300+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1584168806 wait complete. 2024-05-14T09:38:00.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.300+0000 7fd2aece5b80 1 -- start start 2024-05-14T09:38:00.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.300+0000 7fd2aece5b80 1 Processor -- start 2024-05-14T09:38:00.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd2aece5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c6558e40 0x5625c66e04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd2aece5b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c66d6e10 0x5625c66e0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd2aece5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66dcd80 0x5625c66e0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd2aece5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5625c66e37c0 con 0x5625c66dcd80 2024-05-14T09:38:00.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd2aece5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5625c65b1ed0 con 0x5625c6558e40 2024-05-14T09:38:00.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd2aece5b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5625c66e3940 con 0x5625c66d6e10 2024-05-14T09:38:00.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd2aece5b80 10 client.0 _ll_register_callbacks cb 0x5625c6590950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:38:00.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd29e06d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c6558e40 0x5625c66e04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd29e06d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c6558e40 0x5625c66e04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:59508/0 (socket says 172.21.15.2:59508) 2024-05-14T09:38:00.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd29e06d700 1 -- 172.21.15.2:0/3200774076 learned_addr learned my addr 172.21.15.2:0/3200774076 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:00.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd29e86e700 1 --2- 172.21.15.2:0/3200774076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66dcd80 0x5625c66e0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd29d86c700 1 --2- 172.21.15.2:0/3200774076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c66d6e10 0x5625c66e0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd29e06d700 1 -- 172.21.15.2:0/3200774076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c66d6e10 msgr2=0x5625c66e0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:00.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd29e06d700 1 --2- 172.21.15.2:0/3200774076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c66d6e10 0x5625c66e0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd29e06d700 1 -- 172.21.15.2:0/3200774076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66dcd80 msgr2=0x5625c66e0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:00.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd29e06d700 1 --2- 172.21.15.2:0/3200774076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66dcd80 0x5625c66e0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.301+0000 7fd29e06d700 1 -- 172.21.15.2:0/3200774076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625c65af760 con 0x5625c6558e40 2024-05-14T09:38:00.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.302+0000 7fd29e06d700 1 --2- 172.21.15.2:0/3200774076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c6558e40 0x5625c66e04c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd29400f540 tx=0x7fd29400f7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=141c306dbceea014 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:00.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.302+0000 7fd281ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:38:00.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.302+0000 7fd281ffb700 1 -- 172.21.15.2:0/3200774076 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29400bba0 con 0x5625c6558e40 2024-05-14T09:38:00.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.302+0000 7fd2aece5b80 1 -- 172.21.15.2:0/3200774076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd29402b070 con 0x5625c6558e40 2024-05-14T09:38:00.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.302+0000 7fd281ffb700 1 -- 172.21.15.2:0/3200774076 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd29400bd40 con 0x5625c6558e40 2024-05-14T09:38:00.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.302+0000 7fd281ffb700 1 -- 172.21.15.2:0/3200774076 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd2940056a0 con 0x5625c6558e40 2024-05-14T09:38:00.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.302+0000 7fd2aece5b80 10 client.25667 fetch_fsmap learned FSMap version 29 2024-05-14T09:38:00.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.302+0000 7fd2aece5b80 1 -- 172.21.15.2:0/3200774076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5625c66ee3c0 con 0x5625c6558e40 2024-05-14T09:38:00.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.302+0000 7fd281ffb700 1 -- 172.21.15.2:0/3200774076 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd29402b070 con 0x5625c6558e40 2024-05-14T09:38:00.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.303+0000 7fd281ffb700 1 -- 172.21.15.2:0/3200774076 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd294021070 con 0x5625c6558e40 2024-05-14T09:38:00.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.303+0000 7fd2aece5b80 10 client.25667 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:38:00.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.303+0000 7fd2aece5b80 10 client.25667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:38:00.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.303+0000 7fd2aece5b80 20 client.25667 populate_metadata read hostname 'smithi002' 2024-05-14T09:38:00.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.303+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/3200774076 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5625c66f0890 0x5625c66f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.303+0000 7fd2aece5b80 4 client.25667 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:38:00.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.303+0000 7fd2aece5b80 1 -- 172.21.15.2:0/3200774076 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625c6606320 con 0x5625c66f0890 2024-05-14T09:38:00.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.303+0000 7fd29d86c700 1 --2- 172.21.15.2:0/3200774076 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5625c66f0890 0x5625c66f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.303+0000 7fd29d86c700 1 --2- 172.21.15.2:0/3200774076 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5625c66f0890 0x5625c66f2cb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8871e1e13783c58a server_cookie=9f189bcd94d9e249 in_seq=0 out_seq=0 2024-05-14T09:38:00.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.303+0000 7fd281ffb700 10 client.25667 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:00.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.304+0000 7fd281ffb700 1 -- 172.21.15.2:0/3200774076 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5625c6606320 con 0x5625c66f0890 2024-05-14T09:38:00.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.304+0000 7fd281ffb700 10 client.25667 handle_command_reply: tid=0 2024-05-14T09:38:00.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.304+0000 7fd2aece5b80 1 client.25667 shutdown 2024-05-14T09:38:00.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.304+0000 7fd2aece5b80 1 -- 172.21.15.2:0/3200774076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c6558e40 msgr2=0x5625c66e04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:00.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.304+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/3200774076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c6558e40 0x5625c66e04c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd29400f540 tx=0x7fd29400f7d0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.304+0000 7fd2aece5b80 1 -- 172.21.15.2:0/3200774076 shutdown_connections 2024-05-14T09:38:00.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.304+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/3200774076 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5625c66f0890 0x5625c66f2cb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.304+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/3200774076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c6558e40 0x5625c66e04c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.304+0000 7fd281ffb700 0 client.25667 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:00.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.304+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/3200774076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66dcd80 0x5625c66e0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.304+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/3200774076 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c66d6e10 0x5625c66e0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.304+0000 7fd2aece5b80 1 -- 172.21.15.2:0/3200774076 >> 172.21.15.2:0/3200774076 conn(0x5625c65a0630 msgr2=0x5625c66d58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:00.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.304+0000 7fd2aece5b80 1 -- 172.21.15.2:0/3200774076 shutdown_connections 2024-05-14T09:38:00.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.304+0000 7fd2aece5b80 1 -- 172.21.15.2:0/3200774076 wait complete. 2024-05-14T09:38:00.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.305+0000 7fd2aece5b80 20 client.25667 trim_cache size 0 max 16384 2024-05-14T09:38:00.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.336+0000 7fd2aece5b80 1 Processor -- start 2024-05-14T09:38:00.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.336+0000 7fd2aece5b80 1 -- start start 2024-05-14T09:38:00.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.336+0000 7fd2aece5b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c6558e40 0x5625c66d68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.336+0000 7fd2aece5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66d6e10 0x5625c66ecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.336+0000 7fd2aece5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66dcd80 0x5625c66e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.337+0000 7fd2aece5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981b0c90 con 0x5625c66d6e10 2024-05-14T09:38:00.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.337+0000 7fd2aece5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981ae520 con 0x5625c66dcd80 2024-05-14T09:38:00.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.337+0000 7fd2aece5b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981ac370 con 0x5625c6558e40 2024-05-14T09:38:00.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.337+0000 7fd29d86c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66d6e10 0x5625c66ecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.337+0000 7fd29e06d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c6558e40 0x5625c66d68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.337+0000 7fd29d86c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66d6e10 0x5625c66ecbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34286/0 (socket says 172.21.15.2:34286) 2024-05-14T09:38:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.337+0000 7fd29d86c700 1 -- 172.21.15.2:0/1508980512 learned_addr learned my addr 172.21.15.2:0/1508980512 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.337+0000 7fd29e86e700 1 --2- 172.21.15.2:0/1508980512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66dcd80 0x5625c66e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.337+0000 7fd29d86c700 1 -- 172.21.15.2:0/1508980512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c6558e40 msgr2=0x5625c66d68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.337+0000 7fd29d86c700 1 --2- 172.21.15.2:0/1508980512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c6558e40 0x5625c66d68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.337+0000 7fd29d86c700 1 -- 172.21.15.2:0/1508980512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66dcd80 msgr2=0x5625c66e0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.337+0000 7fd29d86c700 1 --2- 172.21.15.2:0/1508980512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66dcd80 0x5625c66e0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.337+0000 7fd29d86c700 1 -- 172.21.15.2:0/1508980512 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625c66d33b0 con 0x5625c66d6e10 2024-05-14T09:38:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.337+0000 7fd29d86c700 1 --2- 172.21.15.2:0/1508980512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66d6e10 0x5625c66ecbe0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd28800ceb0 tx=0x7fd28800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa90477442491854 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.337+0000 7fd283fff700 1 -- 172.21.15.2:0/1508980512 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28808f070 con 0x5625c66d6e10 2024-05-14T09:38:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.338+0000 7fd283fff700 1 -- 172.21.15.2:0/1508980512 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2880b2100 con 0x5625c66d6e10 2024-05-14T09:38:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.338+0000 7fd283fff700 1 -- 172.21.15.2:0/1508980512 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2880dff50 con 0x5625c66d6e10 2024-05-14T09:38:00.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.339+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1508980512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66d6e10 msgr2=0x5625c66ecbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:00.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.339+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/1508980512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66d6e10 0x5625c66ecbe0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd28800ceb0 tx=0x7fd28800c760 comp rx=0 tx=0).stop 2024-05-14T09:38:00.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.339+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1508980512 shutdown_connections 2024-05-14T09:38:00.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.339+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/1508980512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66dcd80 0x5625c66e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.339+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/1508980512 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c6558e40 0x5625c66d68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.339+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/1508980512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c66d6e10 0x5625c66ecbe0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.339+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1508980512 >> 172.21.15.2:0/1508980512 conn(0x5625c65a0630 msgr2=0x5625c66dbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:00.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.339+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1508980512 shutdown_connections 2024-05-14T09:38:00.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.339+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1508980512 wait complete. 2024-05-14T09:38:00.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.339+0000 7fd2aece5b80 1 -- start start 2024-05-14T09:38:00.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.340+0000 7fd2aece5b80 1 Processor -- start 2024-05-14T09:38:00.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.340+0000 7fd2aece5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c6558e40 0x5625c66f6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.340+0000 7fd2aece5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66d6e10 0x5625c66f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.340+0000 7fd2aece5b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c66dcd80 0x5625c66f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.340+0000 7fd2aece5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981ac9e0 con 0x5625c6558e40 2024-05-14T09:38:00.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.340+0000 7fd2aece5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981ade50 con 0x5625c66d6e10 2024-05-14T09:38:00.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.340+0000 7fd2aece5b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981ad0b0 con 0x5625c66dcd80 2024-05-14T09:38:00.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.340+0000 7fd2aece5b80 10 client.0 _ll_register_callbacks cb 0x5625c6590950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:38:00.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.340+0000 7fd29e06d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c6558e40 0x5625c66f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.340+0000 7fd29e86e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c66dcd80 0x5625c66f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.340+0000 7fd29e06d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c6558e40 0x5625c66f6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34288/0 (socket says 172.21.15.2:34288) 2024-05-14T09:38:00.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.340+0000 7fd29e06d700 1 -- 172.21.15.2:0/1383253616 learned_addr learned my addr 172.21.15.2:0/1383253616 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:00.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.340+0000 7fd29d86c700 1 --2- 172.21.15.2:0/1383253616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66d6e10 0x5625c66f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.341+0000 7fd29e86e700 1 -- 172.21.15.2:0/1383253616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66d6e10 msgr2=0x5625c66f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:00.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.341+0000 7fd29e86e700 1 --2- 172.21.15.2:0/1383253616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66d6e10 0x5625c66f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.341+0000 7fd29e86e700 1 -- 172.21.15.2:0/1383253616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c6558e40 msgr2=0x5625c66f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:00.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.341+0000 7fd29e86e700 1 --2- 172.21.15.2:0/1383253616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c6558e40 0x5625c66f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.341+0000 7fd29e86e700 1 -- 172.21.15.2:0/1383253616 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625c66f4020 con 0x5625c66dcd80 2024-05-14T09:38:00.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.341+0000 7fd29e86e700 1 --2- 172.21.15.2:0/1383253616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c66dcd80 0x5625c66f39e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd290007780 tx=0x7fd2900077b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=488437e21ed21e4f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:00.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.341+0000 7fd281ffb700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:38:00.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.341+0000 7fd281ffb700 1 -- 172.21.15.2:0/1383253616 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290004db0 con 0x5625c66dcd80 2024-05-14T09:38:00.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.341+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1383253616 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5625c66003c0 con 0x5625c66dcd80 2024-05-14T09:38:00.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.341+0000 7fd281ffb700 1 -- 172.21.15.2:0/1383253616 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fd29001e620 con 0x5625c66dcd80 2024-05-14T09:38:00.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.341+0000 7fd281ffb700 1 -- 172.21.15.2:0/1383253616 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd29002d050 con 0x5625c66dcd80 2024-05-14T09:38:00.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.341+0000 7fd2aece5b80 10 client.25660 fetch_fsmap learned FSMap version 29 2024-05-14T09:38:00.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.341+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1383253616 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5625c675ac60 con 0x5625c66dcd80 2024-05-14T09:38:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.342+0000 7fd281ffb700 1 -- 172.21.15.2:0/1383253616 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5625c66003c0 con 0x5625c66dcd80 2024-05-14T09:38:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.342+0000 7fd281ffb700 1 -- 172.21.15.2:0/1383253616 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd2900054a0 con 0x5625c66dcd80 2024-05-14T09:38:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.342+0000 7fd2aece5b80 10 client.25660 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:38:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.342+0000 7fd2aece5b80 10 client.25660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T09:38:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.342+0000 7fd2aece5b80 20 client.25660 populate_metadata read hostname 'smithi002' 2024-05-14T09:38:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.342+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/1383253616 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5625c675b380 0x5625c675d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.342+0000 7fd2aece5b80 4 client.25660 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T09:38:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.342+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1383253616 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625c6478250 con 0x5625c675b380 2024-05-14T09:38:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.342+0000 7fd29e06d700 1 --2- 172.21.15.2:0/1383253616 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5625c675b380 0x5625c675d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd281ffb700 10 client.25660 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd29e06d700 1 --2- 172.21.15.2:0/1383253616 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5625c675b380 0x5625c675d7a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24acb8ed7c8fa612 server_cookie=f8617789ac5c3e8 in_seq=0 out_seq=0 2024-05-14T09:38:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd281ffb700 1 -- 172.21.15.2:0/1383253616 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625c6478250 con 0x5625c675b380 2024-05-14T09:38:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd281ffb700 10 client.25660 handle_command_reply: tid=0 2024-05-14T09:38:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd2aece5b80 1 client.25660 shutdown 2024-05-14T09:38:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1383253616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c66dcd80 msgr2=0x5625c66f39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/1383253616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c66dcd80 0x5625c66f39e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd290007780 tx=0x7fd2900077b0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1383253616 shutdown_connections 2024-05-14T09:38:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/1383253616 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5625c675b380 0x5625c675d7a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/1383253616 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5625c66dcd80 0x5625c66f39e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/1383253616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5625c6558e40 0x5625c66f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd281ffb700 0 client.25660 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:00.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd2aece5b80 1 --2- 172.21.15.2:0/1383253616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625c66d6e10 0x5625c66f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1383253616 >> 172.21.15.2:0/1383253616 conn(0x5625c65a0630 msgr2=0x5625c66dbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:00.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1383253616 shutdown_connections 2024-05-14T09:38:00.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd2aece5b80 1 -- 172.21.15.2:0/1383253616 wait complete. 2024-05-14T09:38:00.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.343+0000 7fd2aece5b80 20 client.25660 trim_cache size 0 max 16384 2024-05-14T09:38:00.347 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-05-14T09:38:00.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.344+0000 7fd2a02d1700 1 -- 172.21.15.2:0/3474669376 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd2840b2880 msgr2=0x7fd2840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:00.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.344+0000 7fd2a02d1700 1 --2- 172.21.15.2:0/3474669376 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd2840b2880 0x7fd2840b4d90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd294000c00 tx=0x7fd29401a040 comp rx=0 tx=0).stop 2024-05-14T09:38:00.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.344+0000 7fd2a02d1700 1 -- 172.21.15.2:0/3474669376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd298108f20 msgr2=0x7fd29819b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:00.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.344+0000 7fd2a02d1700 1 --2- 172.21.15.2:0/3474669376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd298108f20 0x7fd29819b410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd28800e970 tx=0x7fd28800ec80 comp rx=0 tx=0).stop 2024-05-14T09:38:00.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.344+0000 7fd2a02d1700 1 -- 172.21.15.2:0/3474669376 shutdown_connections 2024-05-14T09:38:00.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.344+0000 7fd2a02d1700 1 --2- 172.21.15.2:0/3474669376 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd2840b2880 0x7fd2840b4d90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.344+0000 7fd2a02d1700 1 --2- 172.21.15.2:0/3474669376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd298109870 0x7fd29819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.344+0000 7fd2a02d1700 1 --2- 172.21.15.2:0/3474669376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2981045f0 0x7fd29819ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.344+0000 7fd2a02d1700 1 --2- 172.21.15.2:0/3474669376 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd298108f20 0x7fd29819b410 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:00.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.344+0000 7fd2a02d1700 1 -- 172.21.15.2:0/3474669376 >> 172.21.15.2:0/3474669376 conn(0x7fd298100040 msgr2=0x7fd298108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:00.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.344+0000 7fd2a02d1700 1 -- 172.21.15.2:0/3474669376 shutdown_connections 2024-05-14T09:38:00.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T09:38:00.344+0000 7fd2a02d1700 1 -- 172.21.15.2:0/3474669376 wait complete. 2024-05-14T09:38:00.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:00 smithi002 ceph-mon[31557]: pgmap v515: 129 pgs: 129 active+clean; 53 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 330 op/s 2024-05-14T09:38:00.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:00 smithi029 ceph-mon[36249]: pgmap v515: 129 pgs: 129 active+clean; 53 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 330 op/s 2024-05-14T09:38:00.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:00 smithi026 ceph-mon[36056]: pgmap v515: 129 pgs: 129 active+clean; 53 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 330 op/s 2024-05-14T09:38:01.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 177614 11.23 MB/sec warmup 87 sec latency 29.675 ms 2024-05-14T09:38:01.656 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 52762283, 'ino': 1099511628332, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04'}, {'damage_type': 'backtrace', 'id': 130992758, 'ino': 1099511628344, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03'}, {'damage_type': 'backtrace', 'id': 427194348, 'ino': 1099511628330, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB'}, {'damage_type': 'backtrace', 'id': 537590187, 'ino': 1099511628336, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0'}, {'damage_type': 'backtrace', 'id': 606052340, 'ino': 1099511628339, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1'}, {'damage_type': 'backtrace', 'id': 932282660, 'ino': 1099511628340, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL'}, {'damage_type': 'backtrace', 'id': 1113424645, 'ino': 1099511628346, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL'}, {'damage_type': 'backtrace', 'id': 1660974705, 'ino': 1099511628343, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX'}, {'damage_type': 'backtrace', 'id': 1678539431, 'ino': 1099511628338, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1'}, {'damage_type': 'backtrace', 'id': 2403355850, 'ino': 1099511628349, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL'}, {'damage_type': 'backtrace', 'id': 2442331187, 'ino': 1099511628335, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX'}, {'damage_type': 'backtrace', 'id': 2474256081, 'ino': 1099511628333, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04'}, {'damage_type': 'backtrace', 'id': 3130160028, 'ino': 1099511628341, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB'}, {'damage_type': 'backtrace', 'id': 3251567932, 'ino': 1099511628347, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV'}, {'damage_type': 'backtrace', 'id': 3258124601, 'ino': 1099511628337, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0'}, {'damage_type': 'backtrace', 'id': 3811181793, 'ino': 1099511628342, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB'}, {'damage_type': 'backtrace', 'id': 4052639750, 'ino': 1099511628348, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL'}, {'damage_type': 'backtrace', 'id': 4065218329, 'ino': 1099511628345, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03'}, {'damage_type': 'backtrace', 'id': 4135004744, 'ino': 1099511628334, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB'}, {'damage_type': 'backtrace', 'id': 4271314553, 'ino': 1099511628331, 'path': '/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX'}] 2024-05-14T09:38:01.657 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/fwd_scrub.py", line 48, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/fwd_scrub.py", line 70, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/fwd_scrub.py", line 81, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-05-14T09:38:02.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 179507 11.17 MB/sec warmup 88 sec latency 12.997 ms 2024-05-14T09:38:02.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:02 smithi029 ceph-mon[36249]: pgmap v516: 129 pgs: 129 active+clean; 52 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 307 op/s 2024-05-14T09:38:02.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:02 smithi026 ceph-mon[36056]: pgmap v516: 129 pgs: 129 active+clean; 52 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 307 op/s 2024-05-14T09:38:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:02 smithi002 ceph-mon[31557]: pgmap v516: 129 pgs: 129 active+clean; 52 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 307 op/s 2024-05-14T09:38:03.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 181298 11.19 MB/sec warmup 89 sec latency 58.769 ms 2024-05-14T09:38:04.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 183337 11.15 MB/sec warmup 90 sec latency 18.131 ms 2024-05-14T09:38:04.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:04 smithi026 ceph-mon[36056]: pgmap v517: 129 pgs: 129 active+clean; 56 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 400 op/s 2024-05-14T09:38:04.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:04 smithi002 ceph-mon[31557]: pgmap v517: 129 pgs: 129 active+clean; 56 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 400 op/s 2024-05-14T09:38:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:04 smithi029 ceph-mon[36249]: pgmap v517: 129 pgs: 129 active+clean; 56 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 400 op/s 2024-05-14T09:38:05.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 184999 11.16 MB/sec warmup 91 sec latency 22.574 ms 2024-05-14T09:38:06.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 187355 11.16 MB/sec warmup 92 sec latency 27.893 ms 2024-05-14T09:38:06.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:06 smithi026 ceph-mon[36056]: pgmap v518: 129 pgs: 129 active+clean; 56 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 310 op/s 2024-05-14T09:38:06.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:06 smithi002 ceph-mon[31557]: pgmap v518: 129 pgs: 129 active+clean; 56 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 310 op/s 2024-05-14T09:38:06.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:06 smithi029 ceph-mon[36249]: pgmap v518: 129 pgs: 129 active+clean; 56 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 310 op/s 2024-05-14T09:38:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.208+0000 7fb7d1dd5640 1 -- 172.21.15.2:0/2453502226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7cc103d90 msgr2=0x7fb7cc1041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.208+0000 7fb7d1dd5640 1 --2- 172.21.15.2:0/2453502226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7cc103d90 0x7fb7cc1041a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009900 tx=0x7fb7c002a410 comp rx=0 tx=0).stop 2024-05-14T09:38:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.208+0000 7fb7d1dd5640 1 -- 172.21.15.2:0/2453502226 shutdown_connections 2024-05-14T09:38:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.208+0000 7fb7d1dd5640 1 --2- 172.21.15.2:0/2453502226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7cc108a90 0x7fb7cc10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.208+0000 7fb7d1dd5640 1 --2- 172.21.15.2:0/2453502226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7cc103d90 0x7fb7cc1041a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.208+0000 7fb7d1dd5640 1 --2- 172.21.15.2:0/2453502226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7cc102b50 0x7fb7cc103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.208+0000 7fb7d1dd5640 1 -- 172.21.15.2:0/2453502226 >> 172.21.15.2:0/2453502226 conn(0x7fb7cc0fdd40 msgr2=0x7fb7cc1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.208+0000 7fb7d1dd5640 1 -- 172.21.15.2:0/2453502226 shutdown_connections 2024-05-14T09:38:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.208+0000 7fb7d1dd5640 1 -- 172.21.15.2:0/2453502226 wait complete. 2024-05-14T09:38:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.209+0000 7fb7d1dd5640 1 Processor -- start 2024-05-14T09:38:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.209+0000 7fb7d1dd5640 1 -- start start 2024-05-14T09:38:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7d1dd5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7cc102b50 0x7fb7cc19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7d1dd5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7cc103d90 0x7fb7cc19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7d1dd5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7cc108a90 0x7fb7cc1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7d1dd5640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10e670 con 0x7fb7cc103d90 2024-05-14T09:38:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7d1dd5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10e4f0 con 0x7fb7cc102b50 2024-05-14T09:38:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7d1dd5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10e7f0 con 0x7fb7cc108a90 2024-05-14T09:38:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7cc102b50 0x7fb7cc19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7caffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7cc103d90 0x7fb7cc19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7cc108a90 0x7fb7cc1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7cc102b50 0x7fb7cc19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:59548/0 (socket says 172.21.15.2:59548) 2024-05-14T09:38:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7cc108a90 0x7fb7cc1a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:36264/0 (socket says 172.21.15.2:36264) 2024-05-14T09:38:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7cb7fe640 1 -- 172.21.15.2:0/916777966 learned_addr learned my addr 172.21.15.2:0/916777966 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7cb7fe640 1 -- 172.21.15.2:0/916777966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7cc108a90 msgr2=0x7fb7cc1a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7cb7fe640 1 --2- 172.21.15.2:0/916777966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7cc108a90 0x7fb7cc1a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7cb7fe640 1 -- 172.21.15.2:0/916777966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7cc103d90 msgr2=0x7fb7cc19a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7cb7fe640 1 --2- 172.21.15.2:0/916777966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7cc103d90 0x7fb7cc19a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7cb7fe640 1 -- 172.21.15.2:0/916777966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc1a1df0 con 0x7fb7cc102b50 2024-05-14T09:38:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7caffd640 1 --2- 172.21.15.2:0/916777966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7cc103d90 0x7fb7cc19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:38:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.210+0000 7fb7cb7fe640 1 --2- 172.21.15.2:0/916777966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7cc102b50 0x7fb7cc19a300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00e9f0 tx=0x7fb7bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.211+0000 7fb7c8ff9640 1 -- 172.21.15.2:0/916777966 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc00ccc0 con 0x7fb7cc102b50 2024-05-14T09:38:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.211+0000 7fb7d1dd5640 1 -- 172.21.15.2:0/916777966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7cc1a20e0 con 0x7fb7cc102b50 2024-05-14T09:38:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.211+0000 7fb7c8ff9640 1 -- 172.21.15.2:0/916777966 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7bc0035f0 con 0x7fb7cc102b50 2024-05-14T09:38:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.211+0000 7fb7c8ff9640 1 -- 172.21.15.2:0/916777966 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc00ce60 con 0x7fb7cc102b50 2024-05-14T09:38:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.211+0000 7fb7d1dd5640 1 -- 172.21.15.2:0/916777966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7cc1a2630 con 0x7fb7cc102b50 2024-05-14T09:38:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.213+0000 7fb7d8d64740 1 Processor -- start 2024-05-14T09:38:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7d8d64740 1 -- start start 2024-05-14T09:38:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7d8d64740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 0x55bb7878ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7d8d64740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c12f0 0x55bb7878bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7d8d64740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c94d0 0x55bb78792ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7d8d64740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55bb786f05d0 con 0x55bb783c12f0 2024-05-14T09:38:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7d8d64740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bb786a9890 con 0x55bb783ea1b0 2024-05-14T09:38:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7d8d64740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bb786ec590 con 0x55bb783c94d0 2024-05-14T09:38:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 0x55bb7878ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 0x55bb7878ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:59562/0 (socket says 172.21.15.2:59562) 2024-05-14T09:38:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7cbfff640 1 -- 172.21.15.2:0/3105258583 learned_addr learned my addr 172.21.15.2:0/3105258583 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7caffd640 1 --2- 172.21.15.2:0/3105258583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c94d0 0x55bb78792ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7cbfff640 1 -- 172.21.15.2:0/3105258583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c94d0 msgr2=0x55bb78792ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7cbfff640 1 --2- 172.21.15.2:0/3105258583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c94d0 0x55bb78792ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7cbfff640 1 -- 172.21.15.2:0/3105258583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c12f0 msgr2=0x55bb7878bfe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:38:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7cbfff640 1 --2- 172.21.15.2:0/3105258583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c12f0 0x55bb7878bfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.214+0000 7fb7cbfff640 1 -- 172.21.15.2:0/3105258583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb78402210 con 0x55bb783ea1b0 2024-05-14T09:38:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.215+0000 7fb7c8ff9640 1 -- 172.21.15.2:0/916777966 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fb7bc004030 con 0x7fb7cc102b50 2024-05-14T09:38:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.215+0000 7fb7cbfff640 1 --2- 172.21.15.2:0/3105258583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 0x55bb7878ba60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009fd0 tx=0x7fb7c40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1ae2ea9fe3e452e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.215+0000 7fb7b1ffb640 1 -- 172.21.15.2:0/3105258583 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4004d20 con 0x55bb783ea1b0 2024-05-14T09:38:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.215+0000 7fb7b1ffb640 1 -- 172.21.15.2:0/3105258583 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7c4004050 con 0x55bb783ea1b0 2024-05-14T09:38:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.215+0000 7fb7b1ffb640 1 -- 172.21.15.2:0/3105258583 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4005280 con 0x55bb783ea1b0 2024-05-14T09:38:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.216+0000 7fb7c8ff9640 1 --2- 172.21.15.2:0/916777966 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb7b40b28a0 0x7fb7b40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.216+0000 7fb7cb7fe640 1 --2- 172.21.15.2:0/916777966 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb7b40b28a0 0x7fb7b40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.216+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3105258583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 msgr2=0x55bb7878ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.216+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/3105258583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 0x55bb7878ba60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009fd0 tx=0x7fb7c40118d0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.216+0000 7fb7c8ff9640 1 -- 172.21.15.2:0/916777966 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fb7bc0e2a10 con 0x7fb7cc102b50 2024-05-14T09:38:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.216+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3105258583 shutdown_connections 2024-05-14T09:38:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.216+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/3105258583 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c94d0 0x55bb78792ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.216+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/3105258583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c12f0 0x55bb7878bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.216+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/3105258583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 0x55bb7878ba60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.216+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3105258583 >> 172.21.15.2:0/3105258583 conn(0x55bb783de300 msgr2=0x55bb787873a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.217+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3105258583 shutdown_connections 2024-05-14T09:38:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.217+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3105258583 wait complete. 2024-05-14T09:38:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.217+0000 7fb7cb7fe640 1 --2- 172.21.15.2:0/916777966 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb7b40b28a0 0x7fb7b40b4db0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0a9b40 tx=0x7fb7bc01d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.217+0000 7fb7d8d64740 1 -- start start 2024-05-14T09:38:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.218+0000 7fb7d8d64740 1 Processor -- start 2024-05-14T09:38:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.218+0000 7fb7d8d64740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c12f0 0x55bb78795700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.218+0000 7fb7d8d64740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c94d0 0x55bb78795c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.218+0000 7fb7d8d64740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 0x55bb7879cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.218+0000 7fb7d8d64740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55bb78793930 con 0x55bb783c94d0 2024-05-14T09:38:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.218+0000 7fb7d8d64740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bb786ad370 con 0x55bb783ea1b0 2024-05-14T09:38:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.218+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c12f0 0x55bb78795700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.218+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c12f0 0x55bb78795700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:36278/0 (socket says 172.21.15.2:36278) 2024-05-14T09:38:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.218+0000 7fb7caffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c94d0 0x55bb78795c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.218+0000 7fb7cbfff640 1 -- 172.21.15.2:0/3281640251 learned_addr learned my addr 172.21.15.2:0/3281640251 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.218+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3281640251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bb78793ab0 con 0x55bb783c12f0 2024-05-14T09:38:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.218+0000 7fb7d8d64740 10 client.0 _ll_register_callbacks cb 0x55bb78401a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:38:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.218+0000 7fb7cbfff640 1 --2- 172.21.15.2:0/3281640251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 0x55bb7879cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.219+0000 7fb7cbfff640 1 -- 172.21.15.2:0/3281640251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 msgr2=0x55bb7879cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.219+0000 7fb7cbfff640 1 --2- 172.21.15.2:0/3281640251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 0x55bb7879cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.219+0000 7fb7cbfff640 1 -- 172.21.15.2:0/3281640251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c94d0 msgr2=0x55bb78795c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.219+0000 7fb7cbfff640 1 --2- 172.21.15.2:0/3281640251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c94d0 0x55bb78795c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.219+0000 7fb7cbfff640 1 -- 172.21.15.2:0/3281640251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb784ce620 con 0x55bb783c12f0 2024-05-14T09:38:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.219+0000 7fb7caffd640 1 --2- 172.21.15.2:0/3281640251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c94d0 0x55bb78795c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:38:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.219+0000 7fb7cbfff640 1 --2- 172.21.15.2:0/3281640251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c12f0 0x55bb78795700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4003fd0 tx=0x7fb7c4004840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24eb21b57e384822 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.219+0000 7fb7aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:38:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.219+0000 7fb7aaffd640 1 -- 172.21.15.2:0/3281640251 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c400f6f0 con 0x55bb783c12f0 2024-05-14T09:38:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.220+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3281640251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7c4004050 con 0x55bb783c12f0 2024-05-14T09:38:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.220+0000 7fb7aaffd640 1 -- 172.21.15.2:0/3281640251 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fb7c400f890 con 0x55bb783c12f0 2024-05-14T09:38:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.220+0000 7fb7aaffd640 1 -- 172.21.15.2:0/3281640251 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7c400fa90 con 0x55bb783c12f0 2024-05-14T09:38:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.220+0000 7fb7d8d64740 10 client.25672 fetch_fsmap learned FSMap version 29 2024-05-14T09:38:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.220+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3281640251 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb7879fad0 con 0x55bb783c12f0 2024-05-14T09:38:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.220+0000 7fb7aaffd640 1 -- 172.21.15.2:0/3281640251 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7c400fc30 con 0x55bb783c12f0 2024-05-14T09:38:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.221+0000 7fb7aaffd640 1 -- 172.21.15.2:0/3281640251 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb7c4025070 con 0x55bb783c12f0 2024-05-14T09:38:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.221+0000 7fb7d8d64740 10 client.25672 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:38:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.221+0000 7fb7d8d64740 10 client.25672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:38:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.221+0000 7fb7d8d64740 20 client.25672 populate_metadata read hostname 'smithi002' 2024-05-14T09:38:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.221+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/3281640251 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb783a8d80 0x55bb787a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.221+0000 7fb7d8d64740 4 client.25672 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:38:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.221+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3281640251 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb7c4004050 con 0x55bb783a8d80 2024-05-14T09:38:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.221+0000 7fb7caffd640 1 --2- 172.21.15.2:0/3281640251 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb783a8d80 0x55bb787a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.222+0000 7fb7aaffd640 10 client.25672 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.222+0000 7fb7caffd640 1 --2- 172.21.15.2:0/3281640251 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb783a8d80 0x55bb787a34f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e12c61b8ccf61a9 server_cookie=f3a909d06ee653d7 in_seq=0 out_seq=0 2024-05-14T09:38:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.223+0000 7fb7aaffd640 1 -- 172.21.15.2:0/3281640251 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fb7c4004050 con 0x55bb783a8d80 2024-05-14T09:38:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.223+0000 7fb7aaffd640 10 client.25672 handle_command_reply: tid=0 2024-05-14T09:38:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.223+0000 7fb7d8d64740 1 client.25672 shutdown 2024-05-14T09:38:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.223+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3281640251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c12f0 msgr2=0x55bb78795700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.223+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/3281640251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c12f0 0x55bb78795700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4003fd0 tx=0x7fb7c4004840 comp rx=0 tx=0).stop 2024-05-14T09:38:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.223+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3281640251 shutdown_connections 2024-05-14T09:38:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.223+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/3281640251 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb783a8d80 0x55bb787a34f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.223+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/3281640251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 0x55bb7879cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.223+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/3281640251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c94d0 0x55bb78795c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.223+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/3281640251 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c12f0 0x55bb78795700 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.223+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3281640251 >> 172.21.15.2:0/3281640251 conn(0x55bb783de300 msgr2=0x55bb787863a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.223+0000 7fb7aaffd640 0 client.25672 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.224+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3281640251 shutdown_connections 2024-05-14T09:38:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.224+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3281640251 wait complete. 2024-05-14T09:38:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.224+0000 7fb7d8d64740 20 client.25672 trim_cache size 0 max 16384 2024-05-14T09:38:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.239+0000 7fb7d8d64740 1 Processor -- start 2024-05-14T09:38:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.239+0000 7fb7d8d64740 1 -- start start 2024-05-14T09:38:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.239+0000 7fb7d8d64740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c12f0 0x55bb78786cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.239+0000 7fb7d8d64740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c94d0 0x55bb78787230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.239+0000 7fb7d8d64740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 0x55bb7879cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.239+0000 7fb7d8d64740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1af630 con 0x55bb783c94d0 2024-05-14T09:38:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.239+0000 7fb7d8d64740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1acf00 con 0x55bb783ea1b0 2024-05-14T09:38:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.239+0000 7fb7d8d64740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1aad50 con 0x55bb783c12f0 2024-05-14T09:38:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.240+0000 7fb7caffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c94d0 0x55bb78787230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.240+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c12f0 0x55bb78786cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.240+0000 7fb7caffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c94d0 0x55bb78787230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34312/0 (socket says 172.21.15.2:34312) 2024-05-14T09:38:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.240+0000 7fb7caffd640 1 -- 172.21.15.2:0/3450083238 learned_addr learned my addr 172.21.15.2:0/3450083238 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.240+0000 7fb7cbfff640 1 --2- 172.21.15.2:0/3450083238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 0x55bb7879cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.240+0000 7fb7caffd640 1 -- 172.21.15.2:0/3450083238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c12f0 msgr2=0x55bb78786cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.240+0000 7fb7caffd640 1 --2- 172.21.15.2:0/3450083238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c12f0 0x55bb78786cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.240+0000 7fb7caffd640 1 -- 172.21.15.2:0/3450083238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 msgr2=0x55bb7879cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.240+0000 7fb7caffd640 1 --2- 172.21.15.2:0/3450083238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 0x55bb7879cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.240+0000 7fb7caffd640 1 -- 172.21.15.2:0/3450083238 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb78787d00 con 0x55bb783c94d0 2024-05-14T09:38:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.240+0000 7fb7caffd640 1 --2- 172.21.15.2:0/3450083238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c94d0 0x55bb78787230 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0004730 tx=0x7fb7c00042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d76594973e8c865 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.240+0000 7fb7b17fa640 1 -- 172.21.15.2:0/3450083238 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0035040 con 0x55bb783c94d0 2024-05-14T09:38:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.240+0000 7fb7b17fa640 1 -- 172.21.15.2:0/3450083238 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7c0003aa0 con 0x55bb783c94d0 2024-05-14T09:38:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.240+0000 7fb7b17fa640 1 -- 172.21.15.2:0/3450083238 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0031bb0 con 0x55bb783c94d0 2024-05-14T09:38:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.241+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3450083238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c94d0 msgr2=0x55bb78787230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.241+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/3450083238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c94d0 0x55bb78787230 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0004730 tx=0x7fb7c00042b0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.241+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3450083238 shutdown_connections 2024-05-14T09:38:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.241+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/3450083238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783ea1b0 0x55bb7879cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.241+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/3450083238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c94d0 0x55bb78787230 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.241+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/3450083238 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783c12f0 0x55bb78786cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.241+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3450083238 >> 172.21.15.2:0/3450083238 conn(0x55bb783de300 msgr2=0x55bb7878fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.241+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3450083238 shutdown_connections 2024-05-14T09:38:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.241+0000 7fb7d8d64740 1 -- 172.21.15.2:0/3450083238 wait complete. 2024-05-14T09:38:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.242+0000 7fb7d8d64740 1 -- start start 2024-05-14T09:38:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.242+0000 7fb7d8d64740 1 Processor -- start 2024-05-14T09:38:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.242+0000 7fb7d8d64740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c12f0 0x55bb787990e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.242+0000 7fb7d8d64740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783c94d0 0x55bb78799620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.242+0000 7fb7d8d64740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783ea1b0 0x55bb7879c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.242+0000 7fb7d8d64740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1ab3c0 con 0x55bb783c12f0 2024-05-14T09:38:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.242+0000 7fb7d8d64740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1ac830 con 0x55bb783c94d0 2024-05-14T09:38:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.242+0000 7fb7d8d64740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1aba90 con 0x55bb783ea1b0 2024-05-14T09:38:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.242+0000 7fb7d8d64740 10 client.0 _ll_register_callbacks cb 0x7fb7bc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:38:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.242+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c12f0 0x55bb787990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.242+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c12f0 0x55bb787990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34326/0 (socket says 172.21.15.2:34326) 2024-05-14T09:38:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.242+0000 7fb7cbfff640 1 -- 172.21.15.2:0/247049893 learned_addr learned my addr 172.21.15.2:0/247049893 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.242+0000 7fb7caffd640 1 --2- 172.21.15.2:0/247049893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783c94d0 0x55bb78799620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.242+0000 7fb7cbfff640 1 --2- 172.21.15.2:0/247049893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783ea1b0 0x55bb7879c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7cbfff640 1 -- 172.21.15.2:0/247049893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783ea1b0 msgr2=0x55bb7879c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7cbfff640 1 --2- 172.21.15.2:0/247049893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783ea1b0 0x55bb7879c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7cbfff640 1 -- 172.21.15.2:0/247049893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783c94d0 msgr2=0x55bb78799620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7cbfff640 1 --2- 172.21.15.2:0/247049893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783c94d0 0x55bb78799620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7cbfff640 1 -- 172.21.15.2:0/247049893 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7879cb20 con 0x55bb783c12f0 2024-05-14T09:38:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7cbfff640 1 --2- 172.21.15.2:0/247049893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c12f0 0x55bb787990e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4011f50 tx=0x7fb7c4009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1afa173fc666f1b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:38:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7aaffd640 1 -- 172.21.15.2:0/247049893 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4019430 con 0x55bb783c12f0 2024-05-14T09:38:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7aaffd640 1 -- 172.21.15.2:0/247049893 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fb7c40195d0 con 0x55bb783c12f0 2024-05-14T09:38:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7d8d64740 1 -- 172.21.15.2:0/247049893 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7c0003aa0 con 0x55bb783c12f0 2024-05-14T09:38:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7aaffd640 1 -- 172.21.15.2:0/247049893 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7c40197f0 con 0x55bb783c12f0 2024-05-14T09:38:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7d8d64740 10 client.15999 fetch_fsmap learned FSMap version 29 2024-05-14T09:38:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7d8d64740 1 -- 172.21.15.2:0/247049893 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb78809d90 con 0x55bb783c12f0 2024-05-14T09:38:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7aaffd640 1 -- 172.21.15.2:0/247049893 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7c0003aa0 con 0x55bb783c12f0 2024-05-14T09:38:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7aaffd640 1 -- 172.21.15.2:0/247049893 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb7c4025040 con 0x55bb783c12f0 2024-05-14T09:38:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7d8d64740 10 client.15999 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:38:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7d8d64740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:38:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7d8d64740 20 client.15999 populate_metadata read hostname 'smithi002' 2024-05-14T09:38:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.243+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/247049893 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb7837e1c0 0x55bb7880c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.244+0000 7fb7d8d64740 4 client.15999 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:38:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.244+0000 7fb7d8d64740 1 -- 172.21.15.2:0/247049893 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb786fe1e0 con 0x55bb7837e1c0 2024-05-14T09:38:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.244+0000 7fb7caffd640 1 --2- 172.21.15.2:0/247049893 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb7837e1c0 0x55bb7880c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.244+0000 7fb7aaffd640 10 client.15999 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.244+0000 7fb7caffd640 1 --2- 172.21.15.2:0/247049893 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb7837e1c0 0x55bb7880c5c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f48cd066bf64b2e server_cookie=dcfb8a75189399a1 in_seq=0 out_seq=0 2024-05-14T09:38:07.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 188813 11.13 MB/sec warmup 93 sec latency 17.205 ms 2024-05-14T09:38:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.339+0000 7fb7aaffd640 1 -- 172.21.15.2:0/247049893 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb786fe1e0 con 0x55bb7837e1c0 2024-05-14T09:38:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.339+0000 7fb7aaffd640 10 client.15999 handle_command_reply: tid=0 2024-05-14T09:38:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.339+0000 7fb7d8d64740 1 client.15999 shutdown 2024-05-14T09:38:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.339+0000 7fb7d8d64740 1 -- 172.21.15.2:0/247049893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c12f0 msgr2=0x55bb787990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.339+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/247049893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c12f0 0x55bb787990e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4011f50 tx=0x7fb7c4009fd0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.339+0000 7fb7d8d64740 1 -- 172.21.15.2:0/247049893 shutdown_connections 2024-05-14T09:38:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.339+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/247049893 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb7837e1c0 0x55bb7880c5c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.339+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/247049893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb783ea1b0 0x55bb7879c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.339+0000 7fb7aaffd640 0 client.15999 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.339+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/247049893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb783c94d0 0x55bb78799620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.339+0000 7fb7d8d64740 1 --2- 172.21.15.2:0/247049893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb783c12f0 0x55bb787990e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.339+0000 7fb7d8d64740 1 -- 172.21.15.2:0/247049893 >> 172.21.15.2:0/247049893 conn(0x55bb783de300 msgr2=0x55bb78786cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.340+0000 7fb7d8d64740 1 -- 172.21.15.2:0/247049893 shutdown_connections 2024-05-14T09:38:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.340+0000 7fb7d8d64740 1 -- 172.21.15.2:0/247049893 wait complete. 2024-05-14T09:38:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.340+0000 7fb7d8d64740 20 client.15999 trim_cache size 0 max 16384 2024-05-14T09:38:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:38:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:38:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:38:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:38:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.341+0000 7fb7d1dd5640 1 -- 172.21.15.2:0/916777966 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb7b40b28a0 msgr2=0x7fb7b40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.341+0000 7fb7d1dd5640 1 --2- 172.21.15.2:0/916777966 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb7b40b28a0 0x7fb7b40b4db0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0a9b40 tx=0x7fb7bc01d000 comp rx=0 tx=0).stop 2024-05-14T09:38:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.341+0000 7fb7d1dd5640 1 -- 172.21.15.2:0/916777966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7cc102b50 msgr2=0x7fb7cc19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.341+0000 7fb7d1dd5640 1 --2- 172.21.15.2:0/916777966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7cc102b50 0x7fb7cc19a300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00e9f0 tx=0x7fb7bc00eeb0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.341+0000 7fb7d1dd5640 1 -- 172.21.15.2:0/916777966 shutdown_connections 2024-05-14T09:38:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.341+0000 7fb7d1dd5640 1 --2- 172.21.15.2:0/916777966 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb7b40b28a0 0x7fb7b40b4db0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.341+0000 7fb7d1dd5640 1 --2- 172.21.15.2:0/916777966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb7cc108a90 0x7fb7cc1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.341+0000 7fb7d1dd5640 1 --2- 172.21.15.2:0/916777966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7cc103d90 0x7fb7cc19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.341+0000 7fb7d1dd5640 1 --2- 172.21.15.2:0/916777966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7cc102b50 0x7fb7cc19a300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.341+0000 7fb7d1dd5640 1 -- 172.21.15.2:0/916777966 >> 172.21.15.2:0/916777966 conn(0x7fb7cc0fdd40 msgr2=0x7fb7cc109e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.341+0000 7fb7d1dd5640 1 -- 172.21.15.2:0/916777966 shutdown_connections 2024-05-14T09:38:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:07.341+0000 7fb7d1dd5640 1 -- 172.21.15.2:0/916777966 wait complete. 2024-05-14T09:38:08.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 190883 11.12 MB/sec warmup 94 sec latency 38.827 ms 2024-05-14T09:38:08.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:08 smithi026 ceph-mon[36056]: pgmap v519: 129 pgs: 129 active+clean; 56 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 340 op/s 2024-05-14T09:38:08.738 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:08 smithi029 ceph-mon[36249]: pgmap v519: 129 pgs: 129 active+clean; 56 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 340 op/s 2024-05-14T09:38:08.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:08 smithi002 ceph-mon[31557]: pgmap v519: 129 pgs: 129 active+clean; 56 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 340 op/s 2024-05-14T09:38:09.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 192352 11.09 MB/sec warmup 95 sec latency 18.023 ms 2024-05-14T09:38:10.257 INFO:tasks.workunit.client.0.smithi002.stdout: 1 195156 11.16 MB/sec warmup 96 sec latency 12.173 ms 2024-05-14T09:38:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:10 smithi002 ceph-mon[31557]: pgmap v520: 129 pgs: 129 active+clean; 55 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 351 op/s 2024-05-14T09:38:10.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:10 smithi026 ceph-mon[36056]: pgmap v520: 129 pgs: 129 active+clean; 55 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 351 op/s 2024-05-14T09:38:10.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:10 smithi029 ceph-mon[36249]: pgmap v520: 129 pgs: 129 active+clean; 55 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 351 op/s 2024-05-14T09:38:11.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 196531 11.11 MB/sec warmup 97 sec latency 14.000 ms 2024-05-14T09:38:12.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 198878 11.14 MB/sec warmup 98 sec latency 32.976 ms 2024-05-14T09:38:12.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:12 smithi002 ceph-mon[31557]: pgmap v521: 129 pgs: 129 active+clean; 55 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-05-14T09:38:12.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:12 smithi029 ceph-mon[36249]: pgmap v521: 129 pgs: 129 active+clean; 55 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-05-14T09:38:12.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:12 smithi026 ceph-mon[36056]: pgmap v521: 129 pgs: 129 active+clean; 55 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-05-14T09:38:13.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 200197 11.08 MB/sec warmup 99 sec latency 36.229 ms 2024-05-14T09:38:14.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 202479 11.10 MB/sec warmup 100 sec latency 11.648 ms 2024-05-14T09:38:14.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:14 smithi002 ceph-mon[31557]: pgmap v522: 129 pgs: 129 active+clean; 54 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 403 op/s 2024-05-14T09:38:14.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:14 smithi029 ceph-mon[36249]: pgmap v522: 129 pgs: 129 active+clean; 54 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 403 op/s 2024-05-14T09:38:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:14 smithi026 ceph-mon[36056]: pgmap v522: 129 pgs: 129 active+clean; 54 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 403 op/s 2024-05-14T09:38:15.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 203674 11.05 MB/sec warmup 101 sec latency 113.282 ms 2024-05-14T09:38:16.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 205933 11.07 MB/sec warmup 102 sec latency 156.437 ms 2024-05-14T09:38:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:16 smithi002 ceph-mon[31557]: pgmap v523: 129 pgs: 129 active+clean; 54 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 285 op/s 2024-05-14T09:38:16.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:16 smithi029 ceph-mon[36249]: pgmap v523: 129 pgs: 129 active+clean; 54 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 285 op/s 2024-05-14T09:38:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:16 smithi026 ceph-mon[36056]: pgmap v523: 129 pgs: 129 active+clean; 54 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 285 op/s 2024-05-14T09:38:17.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 207217 11.02 MB/sec warmup 103 sec latency 19.960 ms 2024-05-14T09:38:18.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 209438 11.04 MB/sec warmup 104 sec latency 29.232 ms 2024-05-14T09:38:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:18 smithi002 ceph-mon[31557]: pgmap v524: 129 pgs: 129 active+clean; 55 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 327 op/s 2024-05-14T09:38:18.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:18 smithi029 ceph-mon[36249]: pgmap v524: 129 pgs: 129 active+clean; 55 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 327 op/s 2024-05-14T09:38:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:18 smithi026 ceph-mon[36056]: pgmap v524: 129 pgs: 129 active+clean; 55 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 327 op/s 2024-05-14T09:38:19.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 210849 10.99 MB/sec warmup 105 sec latency 18.069 ms 2024-05-14T09:38:20.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 213103 11.02 MB/sec warmup 106 sec latency 44.517 ms 2024-05-14T09:38:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.446+0000 7f1a71ee0640 1 -- 172.21.15.2:0/389431551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c103d90 msgr2=0x7f1a6c1041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.446+0000 7f1a71ee0640 1 --2- 172.21.15.2:0/389431551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c103d90 0x7f1a6c1041a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009900 tx=0x7f1a5402a410 comp rx=0 tx=0).stop 2024-05-14T09:38:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.446+0000 7f1a71ee0640 1 -- 172.21.15.2:0/389431551 shutdown_connections 2024-05-14T09:38:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.446+0000 7f1a71ee0640 1 --2- 172.21.15.2:0/389431551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a6c108a90 0x7f1a6c10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.446+0000 7f1a71ee0640 1 --2- 172.21.15.2:0/389431551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c103d90 0x7f1a6c1041a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.446+0000 7f1a71ee0640 1 --2- 172.21.15.2:0/389431551 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c102b50 0x7f1a6c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.446+0000 7f1a71ee0640 1 -- 172.21.15.2:0/389431551 >> 172.21.15.2:0/389431551 conn(0x7f1a6c0fdd40 msgr2=0x7f1a6c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.446+0000 7f1a71ee0640 1 -- 172.21.15.2:0/389431551 shutdown_connections 2024-05-14T09:38:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.446+0000 7f1a71ee0640 1 -- 172.21.15.2:0/389431551 wait complete. 2024-05-14T09:38:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.447+0000 7f1a71ee0640 1 Processor -- start 2024-05-14T09:38:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.447+0000 7f1a71ee0640 1 -- start start 2024-05-14T09:38:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.447+0000 7f1a71ee0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c102b50 0x7f1a6c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.447+0000 7f1a71ee0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a6c103d90 0x7f1a6c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.447+0000 7f1a71ee0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c108a90 0x7f1a6c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.447+0000 7f1a71ee0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c10e670 con 0x7f1a6c102b50 2024-05-14T09:38:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.447+0000 7f1a71ee0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c10e4f0 con 0x7f1a6c103d90 2024-05-14T09:38:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.447+0000 7f1a71ee0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c10e7f0 con 0x7f1a6c108a90 2024-05-14T09:38:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.447+0000 7f1a6b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c102b50 0x7f1a6c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.447+0000 7f1a6b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c102b50 0x7f1a6c199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50014/0 (socket says 172.21.15.2:50014) 2024-05-14T09:38:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.447+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a6c103d90 0x7f1a6c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.447+0000 7f1a6b7fe640 1 -- 172.21.15.2:0/70082230 learned_addr learned my addr 172.21.15.2:0/70082230 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.447+0000 7f1a6bfff640 1 --2- 172.21.15.2:0/70082230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c108a90 0x7f1a6c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.447+0000 7f1a6b7fe640 1 -- 172.21.15.2:0/70082230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c108a90 msgr2=0x7f1a6c1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.447+0000 7f1a6b7fe640 1 --2- 172.21.15.2:0/70082230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c108a90 0x7f1a6c1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.448+0000 7f1a6b7fe640 1 -- 172.21.15.2:0/70082230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a6c103d90 msgr2=0x7f1a6c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.448+0000 7f1a6b7fe640 1 --2- 172.21.15.2:0/70082230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a6c103d90 0x7f1a6c199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.448+0000 7f1a6b7fe640 1 -- 172.21.15.2:0/70082230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6c1a0ce0 con 0x7f1a6c102b50 2024-05-14T09:38:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.448+0000 7f1a62ffd640 1 --2- 172.21.15.2:0/70082230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a6c103d90 0x7f1a6c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:38:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.448+0000 7f1a6b7fe640 1 --2- 172.21.15.2:0/70082230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c102b50 0x7f1a6c199210 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00eb00 tx=0x7f1a5c009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.448+0000 7f1a697fa640 1 -- 172.21.15.2:0/70082230 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c01c070 con 0x7f1a6c102b50 2024-05-14T09:38:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.448+0000 7f1a697fa640 1 -- 172.21.15.2:0/70082230 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a5c00bba0 con 0x7f1a6c102b50 2024-05-14T09:38:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.448+0000 7f1a71ee0640 1 -- 172.21.15.2:0/70082230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6c06a6c0 con 0x7f1a6c102b50 2024-05-14T09:38:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.448+0000 7f1a697fa640 1 -- 172.21.15.2:0/70082230 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c003d30 con 0x7f1a6c102b50 2024-05-14T09:38:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.448+0000 7f1a71ee0640 1 -- 172.21.15.2:0/70082230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6c06abc0 con 0x7f1a6c102b50 2024-05-14T09:38:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a78d97740 1 Processor -- start 2024-05-14T09:38:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a78d97740 1 -- start start 2024-05-14T09:38:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a78d97740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89571b0 0x5598d8cf8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a78d97740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 0x5598d8cf8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a697fa640 1 -- 172.21.15.2:0/70082230 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1a5c01f070 con 0x7f1a6c102b50 2024-05-14T09:38:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a78d97740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89364d0 0x5598d8cffee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a78d97740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5598d8c5d5d0 con 0x5598d892e2f0 2024-05-14T09:38:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a78d97740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5598d8c16890 con 0x5598d89571b0 2024-05-14T09:38:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a78d97740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5598d8c59590 con 0x5598d89364d0 2024-05-14T09:38:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a6bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89571b0 0x5598d8cf8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a6b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 0x5598d8cf8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89364d0 0x5598d8cffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a6bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89571b0 0x5598d8cf8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:51858/0 (socket says 172.21.15.2:51858) 2024-05-14T09:38:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a6bfff640 1 -- 172.21.15.2:0/987182356 learned_addr learned my addr 172.21.15.2:0/987182356 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a6bfff640 1 -- 172.21.15.2:0/987182356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89364d0 msgr2=0x5598d8cffee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a6bfff640 1 --2- 172.21.15.2:0/987182356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89364d0 0x5598d8cffee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a6bfff640 1 -- 172.21.15.2:0/987182356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 msgr2=0x5598d8cf8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.451+0000 7f1a6bfff640 1 --2- 172.21.15.2:0/987182356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 0x5598d8cf8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.452+0000 7f1a6bfff640 1 -- 172.21.15.2:0/987182356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598d896f210 con 0x5598d89571b0 2024-05-14T09:38:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.452+0000 7f1a6bfff640 1 --2- 172.21.15.2:0/987182356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89571b0 0x5598d8cf8a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009fd0 tx=0x7f1a640118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c41a2477103ac90a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.452+0000 7f1a697fa640 1 --2- 172.21.15.2:0/70082230 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1a580b28a0 0x7f1a580b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.452+0000 7f1a61ffb640 1 -- 172.21.15.2:0/987182356 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64004d20 con 0x5598d89571b0 2024-05-14T09:38:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.452+0000 7f1a61ffb640 1 -- 172.21.15.2:0/987182356 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a64004050 con 0x5598d89571b0 2024-05-14T09:38:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.452+0000 7f1a61ffb640 1 -- 172.21.15.2:0/987182356 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64005260 con 0x5598d89571b0 2024-05-14T09:38:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.452+0000 7f1a6b7fe640 1 --2- 172.21.15.2:0/70082230 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1a580b28a0 0x7f1a580b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.452+0000 7f1a697fa640 1 -- 172.21.15.2:0/70082230 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1a5c0e3d00 con 0x7f1a6c102b50 2024-05-14T09:38:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.453+0000 7f1a78d97740 1 -- 172.21.15.2:0/987182356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89571b0 msgr2=0x5598d8cf8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.453+0000 7f1a78d97740 1 --2- 172.21.15.2:0/987182356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89571b0 0x5598d8cf8a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009fd0 tx=0x7f1a640118d0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.453+0000 7f1a6b7fe640 1 --2- 172.21.15.2:0/70082230 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1a580b28a0 0x7f1a580b4db0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5598d8cf9e40 tx=0x7f1a5c0e77e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.453+0000 7f1a78d97740 1 -- 172.21.15.2:0/987182356 shutdown_connections 2024-05-14T09:38:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.453+0000 7f1a78d97740 1 --2- 172.21.15.2:0/987182356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89364d0 0x5598d8cffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.453+0000 7f1a78d97740 1 --2- 172.21.15.2:0/987182356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 0x5598d8cf8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.453+0000 7f1a78d97740 1 --2- 172.21.15.2:0/987182356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89571b0 0x5598d8cf8a60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.453+0000 7f1a78d97740 1 -- 172.21.15.2:0/987182356 >> 172.21.15.2:0/987182356 conn(0x5598d894b300 msgr2=0x5598d8cf43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.453+0000 7f1a78d97740 1 -- 172.21.15.2:0/987182356 shutdown_connections 2024-05-14T09:38:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.453+0000 7f1a78d97740 1 -- 172.21.15.2:0/987182356 wait complete. 2024-05-14T09:38:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.453+0000 7f1a78d97740 1 -- start start 2024-05-14T09:38:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a78d97740 1 Processor -- start 2024-05-14T09:38:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a78d97740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 0x5598d8d025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a78d97740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89364d0 0x5598d8d02b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a78d97740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89571b0 0x5598d8d09a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a78d97740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5598d8d00930 con 0x5598d892e2f0 2024-05-14T09:38:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a78d97740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5598d8c1a370 con 0x5598d89364d0 2024-05-14T09:38:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a78d97740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5598d8d00ab0 con 0x5598d89571b0 2024-05-14T09:38:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a78d97740 10 client.0 _ll_register_callbacks cb 0x5598d896ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:38:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a6bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 0x5598d8d025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89364d0 0x5598d8d02b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89364d0 0x5598d8d02b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:51870/0 (socket says 172.21.15.2:51870) 2024-05-14T09:38:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a62ffd640 1 -- 172.21.15.2:0/3992115084 learned_addr learned my addr 172.21.15.2:0/3992115084 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a6bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 0x5598d8d025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50030/0 (socket says 172.21.15.2:50030) 2024-05-14T09:38:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a6bfff640 1 --2- 172.21.15.2:0/3992115084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89571b0 0x5598d8d09a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a62ffd640 1 -- 172.21.15.2:0/3992115084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89571b0 msgr2=0x5598d8d09a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a62ffd640 1 --2- 172.21.15.2:0/3992115084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89571b0 0x5598d8d09a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a62ffd640 1 -- 172.21.15.2:0/3992115084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 msgr2=0x5598d8d025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a62ffd640 1 --2- 172.21.15.2:0/3992115084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 0x5598d8d025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.454+0000 7f1a62ffd640 1 -- 172.21.15.2:0/3992115084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598d8a3b620 con 0x5598d89364d0 2024-05-14T09:38:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.455+0000 7f1a62ffd640 1 --2- 172.21.15.2:0/3992115084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89364d0 0x5598d8d02b30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1a54031440 tx=0x7f1a540317b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8229082f9f0efd49 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.455+0000 7f1a4affd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:38:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.455+0000 7f1a4affd640 1 -- 172.21.15.2:0/3992115084 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54035040 con 0x5598d89364d0 2024-05-14T09:38:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.455+0000 7f1a78d97740 1 -- 172.21.15.2:0/3992115084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a64004050 con 0x5598d89364d0 2024-05-14T09:38:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.455+0000 7f1a4affd640 1 -- 172.21.15.2:0/3992115084 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1a54004450 con 0x5598d89364d0 2024-05-14T09:38:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.455+0000 7f1a4affd640 1 -- 172.21.15.2:0/3992115084 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1a540035f0 con 0x5598d89364d0 2024-05-14T09:38:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.455+0000 7f1a78d97740 10 client.25703 fetch_fsmap learned FSMap version 29 2024-05-14T09:38:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.455+0000 7f1a78d97740 1 -- 172.21.15.2:0/3992115084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5598d8d0ca80 con 0x5598d89364d0 2024-05-14T09:38:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.455+0000 7f1a4affd640 1 -- 172.21.15.2:0/3992115084 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a64004050 con 0x5598d89364d0 2024-05-14T09:38:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.455+0000 7f1a4affd640 1 -- 172.21.15.2:0/3992115084 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1a54044070 con 0x5598d89364d0 2024-05-14T09:38:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.455+0000 7f1a78d97740 10 client.25703 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:38:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.455+0000 7f1a78d97740 10 client.25703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:38:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.455+0000 7f1a78d97740 20 client.25703 populate_metadata read hostname 'smithi002' 2024-05-14T09:38:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.455+0000 7f1a78d97740 1 --2- 172.21.15.2:0/3992115084 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5598d8915d80 0x5598d8d10190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.456+0000 7f1a78d97740 4 client.25703 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:38:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.456+0000 7f1a78d97740 1 -- 172.21.15.2:0/3992115084 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598d8c6b1e0 con 0x5598d8915d80 2024-05-14T09:38:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.456+0000 7f1a6bfff640 1 --2- 172.21.15.2:0/3992115084 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5598d8915d80 0x5598d8d10190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.456+0000 7f1a4affd640 10 client.25703 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.456+0000 7f1a6bfff640 1 --2- 172.21.15.2:0/3992115084 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5598d8915d80 0x5598d8d10190 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e7472fe8483329c server_cookie=66cdf1b91a3b3c2c in_seq=0 out_seq=0 2024-05-14T09:38:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.457+0000 7f1a4affd640 1 -- 172.21.15.2:0/3992115084 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5598d8c6b1e0 con 0x5598d8915d80 2024-05-14T09:38:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.457+0000 7f1a4affd640 10 client.25703 handle_command_reply: tid=0 2024-05-14T09:38:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.457+0000 7f1a78d97740 1 client.25703 shutdown 2024-05-14T09:38:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.457+0000 7f1a78d97740 1 -- 172.21.15.2:0/3992115084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89364d0 msgr2=0x5598d8d02b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.457+0000 7f1a78d97740 1 --2- 172.21.15.2:0/3992115084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89364d0 0x5598d8d02b30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1a54031440 tx=0x7f1a540317b0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.457+0000 7f1a78d97740 1 -- 172.21.15.2:0/3992115084 shutdown_connections 2024-05-14T09:38:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.457+0000 7f1a78d97740 1 --2- 172.21.15.2:0/3992115084 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5598d8915d80 0x5598d8d10190 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.457+0000 7f1a78d97740 1 --2- 172.21.15.2:0/3992115084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89571b0 0x5598d8d09a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.457+0000 7f1a78d97740 1 --2- 172.21.15.2:0/3992115084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89364d0 0x5598d8d02b30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.457+0000 7f1a78d97740 1 --2- 172.21.15.2:0/3992115084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 0x5598d8d025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.457+0000 7f1a78d97740 1 -- 172.21.15.2:0/3992115084 >> 172.21.15.2:0/3992115084 conn(0x5598d894b300 msgr2=0x5598d8cf7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.457+0000 7f1a4affd640 0 client.25703 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.457+0000 7f1a78d97740 1 -- 172.21.15.2:0/3992115084 shutdown_connections 2024-05-14T09:38:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.457+0000 7f1a78d97740 1 -- 172.21.15.2:0/3992115084 wait complete. 2024-05-14T09:38:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.457+0000 7f1a78d97740 20 client.25703 trim_cache size 0 max 16384 2024-05-14T09:38:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a78d97740 1 Processor -- start 2024-05-14T09:38:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a78d97740 1 -- start start 2024-05-14T09:38:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a78d97740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d894b300 0x5598d8cfca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a78d97740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 0x5598d8cfcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a78d97740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89364d0 0x5598d8cf71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a78d97740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c1ad310 con 0x5598d892e2f0 2024-05-14T09:38:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a78d97740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c1aabe0 con 0x5598d89364d0 2024-05-14T09:38:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a78d97740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c1a8a30 con 0x5598d894b300 2024-05-14T09:38:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 0x5598d8cfcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 0x5598d8cfcf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50032/0 (socket says 172.21.15.2:50032) 2024-05-14T09:38:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a62ffd640 1 -- 172.21.15.2:0/2187547480 learned_addr learned my addr 172.21.15.2:0/2187547480 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a6bfff640 1 --2- 172.21.15.2:0/2187547480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d894b300 0x5598d8cfca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a6bfff640 1 --2- 172.21.15.2:0/2187547480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89364d0 0x5598d8cf71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a62ffd640 1 -- 172.21.15.2:0/2187547480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d894b300 msgr2=0x5598d8cfca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a62ffd640 1 --2- 172.21.15.2:0/2187547480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d894b300 0x5598d8cfca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a62ffd640 1 -- 172.21.15.2:0/2187547480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89364d0 msgr2=0x5598d8cf71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a62ffd640 1 --2- 172.21.15.2:0/2187547480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89364d0 0x5598d8cf71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a62ffd640 1 -- 172.21.15.2:0/2187547480 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598d8cf4d00 con 0x5598d892e2f0 2024-05-14T09:38:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.474+0000 7f1a6bfff640 1 --2- 172.21.15.2:0/2187547480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d894b300 0x5598d8cfca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:38:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.475+0000 7f1a62ffd640 1 --2- 172.21.15.2:0/2187547480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 0x5598d8cfcf60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1a54046990 tx=0x7f1a540314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bbc806e1a4706d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.475+0000 7f1a617fa640 1 -- 172.21.15.2:0/2187547480 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54035070 con 0x5598d892e2f0 2024-05-14T09:38:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.475+0000 7f1a617fa640 1 -- 172.21.15.2:0/2187547480 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a54003c40 con 0x5598d892e2f0 2024-05-14T09:38:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.475+0000 7f1a617fa640 1 -- 172.21.15.2:0/2187547480 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54046c70 con 0x5598d892e2f0 2024-05-14T09:38:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.476+0000 7f1a78d97740 1 -- 172.21.15.2:0/2187547480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 msgr2=0x5598d8cfcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.476+0000 7f1a78d97740 1 --2- 172.21.15.2:0/2187547480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 0x5598d8cfcf60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1a54046990 tx=0x7f1a540314c0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.476+0000 7f1a78d97740 1 -- 172.21.15.2:0/2187547480 shutdown_connections 2024-05-14T09:38:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.476+0000 7f1a78d97740 1 --2- 172.21.15.2:0/2187547480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d89364d0 0x5598d8cf71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.476+0000 7f1a78d97740 1 --2- 172.21.15.2:0/2187547480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d892e2f0 0x5598d8cfcf60 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.476+0000 7f1a78d97740 1 --2- 172.21.15.2:0/2187547480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d894b300 0x5598d8cfca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.476+0000 7f1a78d97740 1 -- 172.21.15.2:0/2187547480 >> 172.21.15.2:0/2187547480 conn(0x5598d8d01450 msgr2=0x5598d8cf8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.476+0000 7f1a78d97740 1 -- 172.21.15.2:0/2187547480 shutdown_connections 2024-05-14T09:38:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.476+0000 7f1a78d97740 1 -- 172.21.15.2:0/2187547480 wait complete. 2024-05-14T09:38:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.476+0000 7f1a78d97740 1 -- start start 2024-05-14T09:38:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.476+0000 7f1a78d97740 1 Processor -- start 2024-05-14T09:38:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a78d97740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d892e2f0 0x5598d8d06880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a78d97740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89364d0 0x5598d8d06dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a78d97740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d894b300 0x5598d8d07c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a78d97740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c1a90a0 con 0x5598d894b300 2024-05-14T09:38:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a78d97740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c1aa510 con 0x5598d892e2f0 2024-05-14T09:38:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a78d97740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c1a9770 con 0x5598d89364d0 2024-05-14T09:38:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a78d97740 10 client.0 _ll_register_callbacks cb 0x7f1a54009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:38:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a6bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d892e2f0 0x5598d8d06880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89364d0 0x5598d8d06dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89364d0 0x5598d8d06dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:35080/0 (socket says 172.21.15.2:35080) 2024-05-14T09:38:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a62ffd640 1 -- 172.21.15.2:0/4002605639 learned_addr learned my addr 172.21.15.2:0/4002605639 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a6bfff640 1 --2- 172.21.15.2:0/4002605639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d894b300 0x5598d8d07c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a62ffd640 1 -- 172.21.15.2:0/4002605639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d892e2f0 msgr2=0x5598d8d06880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a62ffd640 1 --2- 172.21.15.2:0/4002605639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d892e2f0 0x5598d8d06880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a62ffd640 1 -- 172.21.15.2:0/4002605639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d894b300 msgr2=0x5598d8d07c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a62ffd640 1 --2- 172.21.15.2:0/4002605639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d894b300 0x5598d8d07c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a62ffd640 1 -- 172.21.15.2:0/4002605639 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598d8d08240 con 0x5598d89364d0 2024-05-14T09:38:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a62ffd640 1 --2- 172.21.15.2:0/4002605639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89364d0 0x5598d8d06dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1a54046770 tx=0x7f1a54005e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77db2a3fcbfab9b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.477+0000 7f1a4affd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:38:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.478+0000 7f1a4affd640 1 -- 172.21.15.2:0/4002605639 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54062060 con 0x5598d89364d0 2024-05-14T09:38:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.478+0000 7f1a4affd640 1 -- 172.21.15.2:0/4002605639 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1a54003790 con 0x5598d89364d0 2024-05-14T09:38:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.478+0000 7f1a78d97740 1 -- 172.21.15.2:0/4002605639 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a54003c40 con 0x5598d89364d0 2024-05-14T09:38:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.478+0000 7f1a4affd640 1 -- 172.21.15.2:0/4002605639 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1a54069050 con 0x5598d89364d0 2024-05-14T09:38:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.478+0000 7f1a78d97740 10 client.25696 fetch_fsmap learned FSMap version 29 2024-05-14T09:38:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.478+0000 7f1a78d97740 1 -- 172.21.15.2:0/4002605639 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5598d8d78540 con 0x5598d89364d0 2024-05-14T09:38:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.478+0000 7f1a4affd640 1 -- 172.21.15.2:0/4002605639 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a54003c40 con 0x5598d89364d0 2024-05-14T09:38:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.478+0000 7f1a4affd640 1 -- 172.21.15.2:0/4002605639 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1a5405c070 con 0x5598d89364d0 2024-05-14T09:38:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.478+0000 7f1a78d97740 10 client.25696 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:38:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.478+0000 7f1a78d97740 10 client.25696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:38:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.478+0000 7f1a78d97740 20 client.25696 populate_metadata read hostname 'smithi002' 2024-05-14T09:38:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.478+0000 7f1a78d97740 1 --2- 172.21.15.2:0/4002605639 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5598d89571b0 0x5598d8d7ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.478+0000 7f1a78d97740 4 client.25696 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:38:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.478+0000 7f1a78d97740 1 -- 172.21.15.2:0/4002605639 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5598d8c65050 con 0x5598d89571b0 2024-05-14T09:38:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.478+0000 7f1a6bfff640 1 --2- 172.21.15.2:0/4002605639 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5598d89571b0 0x5598d8d7ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.479+0000 7f1a4affd640 10 client.25696 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.479+0000 7f1a6bfff640 1 --2- 172.21.15.2:0/4002605639 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5598d89571b0 0x5598d8d7ae40 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4925f5c591f50c6 server_cookie=2db25c7dfae3e898 in_seq=0 out_seq=0 2024-05-14T09:38:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.566+0000 7f1a4affd640 1 -- 172.21.15.2:0/4002605639 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5598d8c65050 con 0x5598d89571b0 2024-05-14T09:38:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.566+0000 7f1a4affd640 10 client.25696 handle_command_reply: tid=0 2024-05-14T09:38:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.566+0000 7f1a78d97740 1 client.25696 shutdown 2024-05-14T09:38:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.566+0000 7f1a78d97740 1 -- 172.21.15.2:0/4002605639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89364d0 msgr2=0x5598d8d06dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.566+0000 7f1a78d97740 1 --2- 172.21.15.2:0/4002605639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89364d0 0x5598d8d06dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1a54046770 tx=0x7f1a54005e90 comp rx=0 tx=0).stop 2024-05-14T09:38:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.566+0000 7f1a78d97740 1 -- 172.21.15.2:0/4002605639 shutdown_connections 2024-05-14T09:38:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.566+0000 7f1a78d97740 1 --2- 172.21.15.2:0/4002605639 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5598d89571b0 0x5598d8d7ae40 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.566+0000 7f1a78d97740 1 --2- 172.21.15.2:0/4002605639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5598d894b300 0x5598d8d07c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.566+0000 7f1a78d97740 1 --2- 172.21.15.2:0/4002605639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5598d89364d0 0x5598d8d06dc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.566+0000 7f1a4affd640 0 client.25696 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.566+0000 7f1a78d97740 1 --2- 172.21.15.2:0/4002605639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5598d892e2f0 0x5598d8d06880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.566+0000 7f1a78d97740 1 -- 172.21.15.2:0/4002605639 >> 172.21.15.2:0/4002605639 conn(0x5598d8d01450 msgr2=0x5598d8d025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.567+0000 7f1a78d97740 1 -- 172.21.15.2:0/4002605639 shutdown_connections 2024-05-14T09:38:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.567+0000 7f1a78d97740 1 -- 172.21.15.2:0/4002605639 wait complete. 2024-05-14T09:38:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.567+0000 7f1a78d97740 20 client.25696 trim_cache size 0 max 16384 2024-05-14T09:38:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:38:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:38:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:38:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:38:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.567+0000 7f1a71ee0640 1 -- 172.21.15.2:0/70082230 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1a580b28a0 msgr2=0x7f1a580b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.567+0000 7f1a71ee0640 1 --2- 172.21.15.2:0/70082230 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1a580b28a0 0x7f1a580b4db0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5598d8cf9e40 tx=0x7f1a5c0e77e0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.567+0000 7f1a71ee0640 1 -- 172.21.15.2:0/70082230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c102b50 msgr2=0x7f1a6c199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.567+0000 7f1a71ee0640 1 --2- 172.21.15.2:0/70082230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c102b50 0x7f1a6c199210 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00eb00 tx=0x7f1a5c009ea0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.567+0000 7f1a71ee0640 1 -- 172.21.15.2:0/70082230 shutdown_connections 2024-05-14T09:38:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.567+0000 7f1a71ee0640 1 --2- 172.21.15.2:0/70082230 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1a580b28a0 0x7f1a580b4db0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.567+0000 7f1a71ee0640 1 --2- 172.21.15.2:0/70082230 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a6c108a90 0x7f1a6c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.567+0000 7f1a71ee0640 1 --2- 172.21.15.2:0/70082230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a6c103d90 0x7f1a6c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.567+0000 7f1a71ee0640 1 --2- 172.21.15.2:0/70082230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c102b50 0x7f1a6c199210 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.567+0000 7f1a71ee0640 1 -- 172.21.15.2:0/70082230 >> 172.21.15.2:0/70082230 conn(0x7f1a6c0fdd40 msgr2=0x7f1a6c104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.568+0000 7f1a71ee0640 1 -- 172.21.15.2:0/70082230 shutdown_connections 2024-05-14T09:38:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:20.568+0000 7f1a71ee0640 1 -- 172.21.15.2:0/70082230 wait complete. 2024-05-14T09:38:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:20 smithi002 ceph-mon[31557]: pgmap v525: 129 pgs: 129 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 362 op/s 2024-05-14T09:38:20.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:20 smithi029 ceph-mon[36249]: pgmap v525: 129 pgs: 129 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 362 op/s 2024-05-14T09:38:20.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:20 smithi026 ceph-mon[36056]: pgmap v525: 129 pgs: 129 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 362 op/s 2024-05-14T09:38:21.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 214458 10.97 MB/sec warmup 107 sec latency 46.561 ms 2024-05-14T09:38:22.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 216727 10.99 MB/sec warmup 108 sec latency 27.319 ms 2024-05-14T09:38:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:22 smithi002 ceph-mon[31557]: pgmap v526: 129 pgs: 129 active+clean; 57 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 288 op/s 2024-05-14T09:38:22.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:22 smithi029 ceph-mon[36249]: pgmap v526: 129 pgs: 129 active+clean; 57 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 288 op/s 2024-05-14T09:38:22.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:22 smithi026 ceph-mon[36056]: pgmap v526: 129 pgs: 129 active+clean; 57 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 288 op/s 2024-05-14T09:38:23.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 218114 10.94 MB/sec warmup 109 sec latency 43.862 ms 2024-05-14T09:38:24.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 220328 10.97 MB/sec warmup 110 sec latency 21.123 ms 2024-05-14T09:38:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:24 smithi002 ceph-mon[31557]: pgmap v527: 129 pgs: 129 active+clean; 55 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 399 op/s 2024-05-14T09:38:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:24 smithi029 ceph-mon[36249]: pgmap v527: 129 pgs: 129 active+clean; 55 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 399 op/s 2024-05-14T09:38:24.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:24 smithi026 ceph-mon[36056]: pgmap v527: 129 pgs: 129 active+clean; 55 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 399 op/s 2024-05-14T09:38:25.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 222334 10.93 MB/sec warmup 111 sec latency 14.923 ms 2024-05-14T09:38:26.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 224048 10.94 MB/sec warmup 112 sec latency 16.231 ms 2024-05-14T09:38:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:26 smithi002 ceph-mon[31557]: pgmap v528: 129 pgs: 129 active+clean; 55 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2024-05-14T09:38:26.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:26 smithi029 ceph-mon[36249]: pgmap v528: 129 pgs: 129 active+clean; 55 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2024-05-14T09:38:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:26 smithi026 ceph-mon[36056]: pgmap v528: 129 pgs: 129 active+clean; 55 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2024-05-14T09:38:27.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 226109 10.92 MB/sec warmup 113 sec latency 14.989 ms 2024-05-14T09:38:28.258 INFO:tasks.workunit.client.0.smithi002.stdout: 1 227610 10.92 MB/sec warmup 114 sec latency 79.836 ms 2024-05-14T09:38:28.723 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:28 smithi002 ceph-mon[31557]: pgmap v529: 129 pgs: 129 active+clean; 58 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 356 op/s 2024-05-14T09:38:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:28 smithi029 ceph-mon[36249]: pgmap v529: 129 pgs: 129 active+clean; 58 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 356 op/s 2024-05-14T09:38:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:28 smithi026 ceph-mon[36056]: pgmap v529: 129 pgs: 129 active+clean; 58 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 356 op/s 2024-05-14T09:38:29.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 229833 10.92 MB/sec warmup 115 sec latency 22.033 ms 2024-05-14T09:38:30.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 231645 10.91 MB/sec warmup 116 sec latency 14.552 ms 2024-05-14T09:38:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:30 smithi002 ceph-mon[31557]: pgmap v530: 129 pgs: 129 active+clean; 57 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 337 op/s 2024-05-14T09:38:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:30 smithi029 ceph-mon[36249]: pgmap v530: 129 pgs: 129 active+clean; 57 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 337 op/s 2024-05-14T09:38:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:30 smithi026 ceph-mon[36056]: pgmap v530: 129 pgs: 129 active+clean; 57 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 337 op/s 2024-05-14T09:38:31.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 233980 10.93 MB/sec warmup 117 sec latency 101.748 ms 2024-05-14T09:38:32.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 235326 10.89 MB/sec warmup 118 sec latency 15.037 ms 2024-05-14T09:38:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:32 smithi002 ceph-mon[31557]: pgmap v531: 129 pgs: 129 active+clean; 58 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 294 op/s 2024-05-14T09:38:32.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:32 smithi029 ceph-mon[36249]: pgmap v531: 129 pgs: 129 active+clean; 58 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 294 op/s 2024-05-14T09:38:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:32 smithi026 ceph-mon[36056]: pgmap v531: 129 pgs: 129 active+clean; 58 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 294 op/s 2024-05-14T09:38:33.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 237700 10.93 MB/sec warmup 119 sec latency 44.033 ms 2024-05-14T09:38:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.672+0000 7f945e422640 1 -- 172.21.15.2:0/2211746340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9458103c90 msgr2=0x7f9458104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.672+0000 7f945e422640 1 --2- 172.21.15.2:0/2211746340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9458103c90 0x7f9458104990 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9448009900 tx=0x7f944802a410 comp rx=0 tx=0).stop 2024-05-14T09:38:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.673+0000 7f945e422640 1 -- 172.21.15.2:0/2211746340 shutdown_connections 2024-05-14T09:38:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.673+0000 7f945e422640 1 --2- 172.21.15.2:0/2211746340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9458109bd0 0x7f945810bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.673+0000 7f945e422640 1 --2- 172.21.15.2:0/2211746340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9458104ed0 0x7f94581052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.673+0000 7f945e422640 1 --2- 172.21.15.2:0/2211746340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9458103c90 0x7f9458104990 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.673+0000 7f945e422640 1 -- 172.21.15.2:0/2211746340 >> 172.21.15.2:0/2211746340 conn(0x7f94580feea0 msgr2=0x7f9458101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.673+0000 7f945e422640 1 -- 172.21.15.2:0/2211746340 shutdown_connections 2024-05-14T09:38:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.673+0000 7f945e422640 1 -- 172.21.15.2:0/2211746340 wait complete. 2024-05-14T09:38:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.673+0000 7f945e422640 1 Processor -- start 2024-05-14T09:38:33.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.673+0000 7f945e422640 1 -- start start 2024-05-14T09:38:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f945e422640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9458103c90 0x7f945819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f945e422640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9458104ed0 0x7f945819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f945e422640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9458109bd0 0x7f94581a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f945e422640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f945810f790 con 0x7f9458109bd0 2024-05-14T09:38:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f945e422640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f945810f610 con 0x7f9458103c90 2024-05-14T09:38:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f945e422640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f945810f910 con 0x7f9458104ed0 2024-05-14T09:38:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f9457fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9458103c90 0x7f945819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f944ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9458104ed0 0x7f945819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f945c9bf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9458109bd0 0x7f94581a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f9457fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9458103c90 0x7f945819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:58324/0 (socket says 172.21.15.2:58324) 2024-05-14T09:38:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f9457fff640 1 -- 172.21.15.2:0/2873708643 learned_addr learned my addr 172.21.15.2:0/2873708643 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f944ffff640 1 -- 172.21.15.2:0/2873708643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9458103c90 msgr2=0x7f945819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f944ffff640 1 --2- 172.21.15.2:0/2873708643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9458103c90 0x7f945819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f944ffff640 1 -- 172.21.15.2:0/2873708643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9458109bd0 msgr2=0x7f94581a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f944ffff640 1 --2- 172.21.15.2:0/2873708643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9458109bd0 0x7f94581a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f944ffff640 1 -- 172.21.15.2:0/2873708643 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94581a1e40 con 0x7f9458104ed0 2024-05-14T09:38:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f944ffff640 1 --2- 172.21.15.2:0/2873708643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9458104ed0 0x7f945819a840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f944000e9f0 tx=0x7f944000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.674+0000 7f9455ffb640 1 -- 172.21.15.2:0/2873708643 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944000ccc0 con 0x7f9458104ed0 2024-05-14T09:38:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.675+0000 7f945e422640 1 -- 172.21.15.2:0/2873708643 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94581a2130 con 0x7f9458104ed0 2024-05-14T09:38:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.675+0000 7f9455ffb640 1 -- 172.21.15.2:0/2873708643 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94400108e0 con 0x7f9458104ed0 2024-05-14T09:38:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.675+0000 7f9455ffb640 1 -- 172.21.15.2:0/2873708643 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440018880 con 0x7f9458104ed0 2024-05-14T09:38:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.675+0000 7f945e422640 1 -- 172.21.15.2:0/2873708643 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94581a26d0 con 0x7f9458104ed0 2024-05-14T09:38:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.677+0000 7f94653ad740 1 Processor -- start 2024-05-14T09:38:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.677+0000 7f94653ad740 1 -- start start 2024-05-14T09:38:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.677+0000 7f94653ad740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 0x562292970a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.677+0000 7f94653ad740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 0x562292970fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.677+0000 7f94653ad740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x562292977ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.677+0000 7f94653ad740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5622928d55d0 con 0x5622925ae4d0 2024-05-14T09:38:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.677+0000 7f94653ad740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56229288e890 con 0x5622925a62f0 2024-05-14T09:38:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.677+0000 7f94653ad740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5622928d1590 con 0x5622925cf1b0 2024-05-14T09:38:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.677+0000 7f945c9bf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 0x562292970a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.677+0000 7f945c9bf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 0x562292970a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:59566/0 (socket says 172.21.15.2:59566) 2024-05-14T09:38:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.677+0000 7f945c9bf640 1 -- 172.21.15.2:0/3591878630 learned_addr learned my addr 172.21.15.2:0/3591878630 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.677+0000 7f9455ffb640 1 -- 172.21.15.2:0/2873708643 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9440010430 con 0x7f9458104ed0 2024-05-14T09:38:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.678+0000 7f9457fff640 1 --2- 172.21.15.2:0/3591878630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 0x562292970fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.678+0000 7f944ffff640 1 --2- 172.21.15.2:0/3591878630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x562292977ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.678+0000 7f945c9bf640 1 -- 172.21.15.2:0/3591878630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 msgr2=0x562292970fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.678+0000 7f945c9bf640 1 --2- 172.21.15.2:0/3591878630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 0x562292970fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.678+0000 7f945c9bf640 1 -- 172.21.15.2:0/3591878630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 msgr2=0x562292977ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.678+0000 7f945c9bf640 1 --2- 172.21.15.2:0/3591878630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x562292977ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.678+0000 7f945c9bf640 1 -- 172.21.15.2:0/3591878630 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622925e7210 con 0x5622925cf1b0 2024-05-14T09:38:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.678+0000 7f944ffff640 1 --2- 172.21.15.2:0/3591878630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x562292977ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:38:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.678+0000 7f945c9bf640 1 --2- 172.21.15.2:0/3591878630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 0x562292970a60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9450009fd0 tx=0x7f94500118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff264a24d906358d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.678+0000 7f944e7fc640 1 -- 172.21.15.2:0/3591878630 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450004d20 con 0x5622925cf1b0 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.678+0000 7f944e7fc640 1 -- 172.21.15.2:0/3591878630 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9450004050 con 0x5622925cf1b0 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.678+0000 7f944e7fc640 1 -- 172.21.15.2:0/3591878630 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450005280 con 0x5622925cf1b0 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.678+0000 7f9455ffb640 1 --2- 172.21.15.2:0/2873708643 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f94440b28a0 0x7f94440b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.679+0000 7f9457fff640 1 --2- 172.21.15.2:0/2873708643 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f94440b28a0 0x7f94440b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.679+0000 7f9455ffb640 1 -- 172.21.15.2:0/2873708643 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f94400e2970 con 0x7f9458104ed0 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.679+0000 7f94653ad740 1 -- 172.21.15.2:0/3591878630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 msgr2=0x562292970a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.679+0000 7f94653ad740 1 --2- 172.21.15.2:0/3591878630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 0x562292970a60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9450009fd0 tx=0x7f94500118d0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.679+0000 7f94653ad740 1 -- 172.21.15.2:0/3591878630 shutdown_connections 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.679+0000 7f94653ad740 1 --2- 172.21.15.2:0/3591878630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x562292977ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.679+0000 7f94653ad740 1 --2- 172.21.15.2:0/3591878630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 0x562292970fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.679+0000 7f94653ad740 1 --2- 172.21.15.2:0/3591878630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 0x562292970a60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.679+0000 7f94653ad740 1 -- 172.21.15.2:0/3591878630 >> 172.21.15.2:0/3591878630 conn(0x5622925c3300 msgr2=0x56229296c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.679+0000 7f9457fff640 1 --2- 172.21.15.2:0/2873708643 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f94440b28a0 0x7f94440b4db0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9448031500 tx=0x7f9448009660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.679+0000 7f94653ad740 1 -- 172.21.15.2:0/3591878630 shutdown_connections 2024-05-14T09:38:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.679+0000 7f94653ad740 1 -- 172.21.15.2:0/3591878630 wait complete. 2024-05-14T09:38:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.680+0000 7f94653ad740 1 -- start start 2024-05-14T09:38:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.680+0000 7f94653ad740 1 Processor -- start 2024-05-14T09:38:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.680+0000 7f94653ad740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 0x56229297a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.680+0000 7f94653ad740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x56229297abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.680+0000 7f94653ad740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 0x562292981b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.680+0000 7f94653ad740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562292978930 con 0x5622925ae4d0 2024-05-14T09:38:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.680+0000 7f94653ad740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562292892370 con 0x5622925a62f0 2024-05-14T09:38:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.680+0000 7f94653ad740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562292978ab0 con 0x5622925cf1b0 2024-05-14T09:38:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.680+0000 7f94653ad740 10 client.0 _ll_register_callbacks cb 0x5622925e6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:38:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.680+0000 7f944ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x56229297abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.680+0000 7f944ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x56229297abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33814/0 (socket says 172.21.15.2:33814) 2024-05-14T09:38:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.680+0000 7f944ffff640 1 -- 172.21.15.2:0/2089442770 learned_addr learned my addr 172.21.15.2:0/2089442770 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.680+0000 7f9457fff640 1 --2- 172.21.15.2:0/2089442770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 0x56229297a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.680+0000 7f945c9bf640 1 --2- 172.21.15.2:0/2089442770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 0x562292981b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.681+0000 7f944ffff640 1 -- 172.21.15.2:0/2089442770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 msgr2=0x562292981b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.681+0000 7f944ffff640 1 --2- 172.21.15.2:0/2089442770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 0x562292981b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.681+0000 7f944ffff640 1 -- 172.21.15.2:0/2089442770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 msgr2=0x56229297a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.681+0000 7f944ffff640 1 --2- 172.21.15.2:0/2089442770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 0x56229297a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.681+0000 7f944ffff640 1 -- 172.21.15.2:0/2089442770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622926b3620 con 0x5622925ae4d0 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.681+0000 7f9457fff640 1 --2- 172.21.15.2:0/2089442770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 0x56229297a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.681+0000 7f944ffff640 1 --2- 172.21.15.2:0/2089442770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x56229297abb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f94400a5ca0 tx=0x7f94400a5470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644a0640de9a7f42 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.681+0000 7f94377fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.681+0000 7f94377fe640 1 -- 172.21.15.2:0/2089442770 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94400a9c20 con 0x5622925ae4d0 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.681+0000 7f94377fe640 1 -- 172.21.15.2:0/2089442770 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f9440010a90 con 0x5622925ae4d0 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.681+0000 7f94653ad740 1 -- 172.21.15.2:0/2089442770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9450004050 con 0x5622925ae4d0 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.681+0000 7f94377fe640 1 -- 172.21.15.2:0/2089442770 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f94400adf20 con 0x5622925ae4d0 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.681+0000 7f94653ad740 10 client.16026 fetch_fsmap learned FSMap version 29 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.681+0000 7f94653ad740 1 -- 172.21.15.2:0/2089442770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562292984a20 con 0x5622925ae4d0 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.681+0000 7f94377fe640 1 -- 172.21.15.2:0/2089442770 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9450004050 con 0x5622925ae4d0 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.682+0000 7f94377fe640 1 -- 172.21.15.2:0/2089442770 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f94400a88f0 con 0x5622925ae4d0 2024-05-14T09:38:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.682+0000 7f94653ad740 10 client.16026 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:38:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.682+0000 7f94653ad740 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:38:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.682+0000 7f94653ad740 20 client.16026 populate_metadata read hostname 'smithi002' 2024-05-14T09:38:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.682+0000 7f94653ad740 1 --2- 172.21.15.2:0/2089442770 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56229258dd80 0x562292988140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.682+0000 7f94653ad740 4 client.16026 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:38:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.682+0000 7f94653ad740 1 -- 172.21.15.2:0/2089442770 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622928e31e0 con 0x56229258dd80 2024-05-14T09:38:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.682+0000 7f9457fff640 1 --2- 172.21.15.2:0/2089442770 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56229258dd80 0x562292988140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.682+0000 7f9457fff640 1 --2- 172.21.15.2:0/2089442770 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56229258dd80 0x562292988140 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa813d936be10339 server_cookie=bd594266e8cab630 in_seq=0 out_seq=0 2024-05-14T09:38:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.682+0000 7f94377fe640 10 client.16026 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.683+0000 7f94377fe640 1 -- 172.21.15.2:0/2089442770 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5622928e31e0 con 0x56229258dd80 2024-05-14T09:38:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.683+0000 7f94377fe640 10 client.16026 handle_command_reply: tid=0 2024-05-14T09:38:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.683+0000 7f94653ad740 1 client.16026 shutdown 2024-05-14T09:38:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.683+0000 7f94653ad740 1 -- 172.21.15.2:0/2089442770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 msgr2=0x56229297abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.683+0000 7f94653ad740 1 --2- 172.21.15.2:0/2089442770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x56229297abb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f94400a5ca0 tx=0x7f94400a5470 comp rx=0 tx=0).stop 2024-05-14T09:38:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.683+0000 7f94653ad740 1 -- 172.21.15.2:0/2089442770 shutdown_connections 2024-05-14T09:38:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.683+0000 7f94653ad740 1 --2- 172.21.15.2:0/2089442770 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56229258dd80 0x562292988140 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.683+0000 7f94653ad740 1 --2- 172.21.15.2:0/2089442770 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 0x562292981b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.683+0000 7f94653ad740 1 --2- 172.21.15.2:0/2089442770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x56229297abb0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.683+0000 7f94653ad740 1 --2- 172.21.15.2:0/2089442770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 0x56229297a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.683+0000 7f94653ad740 1 -- 172.21.15.2:0/2089442770 >> 172.21.15.2:0/2089442770 conn(0x5622925c3300 msgr2=0x56229296f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.683+0000 7f94377fe640 0 client.16026 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.684+0000 7f94653ad740 1 -- 172.21.15.2:0/2089442770 shutdown_connections 2024-05-14T09:38:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.684+0000 7f94653ad740 1 -- 172.21.15.2:0/2089442770 wait complete. 2024-05-14T09:38:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.684+0000 7f94653ad740 20 client.16026 trim_cache size 0 max 16384 2024-05-14T09:38:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f94653ad740 1 Processor -- start 2024-05-14T09:38:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f94653ad740 1 -- start start 2024-05-14T09:38:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f94653ad740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 0x562292974b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f94653ad740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x562292981a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f94653ad740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 0x562292975040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f94653ad740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f94581af600 con 0x5622925ae4d0 2024-05-14T09:38:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f94653ad740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94581aced0 con 0x5622925a62f0 2024-05-14T09:38:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f94653ad740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f94581aad20 con 0x5622925cf1b0 2024-05-14T09:38:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f944ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x562292981a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f944ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x562292981a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33818/0 (socket says 172.21.15.2:33818) 2024-05-14T09:38:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f944ffff640 1 -- 172.21.15.2:0/1933381198 learned_addr learned my addr 172.21.15.2:0/1933381198 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f9457fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 0x562292974b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f945c9bf640 1 --2- 172.21.15.2:0/1933381198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 0x562292975040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f944ffff640 1 -- 172.21.15.2:0/1933381198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 msgr2=0x562292975040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f944ffff640 1 --2- 172.21.15.2:0/1933381198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 0x562292975040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f944ffff640 1 -- 172.21.15.2:0/1933381198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 msgr2=0x562292974b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f944ffff640 1 --2- 172.21.15.2:0/1933381198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 0x562292974b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f944ffff640 1 -- 172.21.15.2:0/1933381198 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56229296cd00 con 0x5622925ae4d0 2024-05-14T09:38:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f944ffff640 1 --2- 172.21.15.2:0/1933381198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x562292981a30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f94400e2df0 tx=0x7f94400e6020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b182987d2480e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f944dffb640 1 -- 172.21.15.2:0/1933381198 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94400b29d0 con 0x5622925ae4d0 2024-05-14T09:38:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f944dffb640 1 -- 172.21.15.2:0/1933381198 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94400c4140 con 0x5622925ae4d0 2024-05-14T09:38:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.700+0000 7f944dffb640 1 -- 172.21.15.2:0/1933381198 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94400af3c0 con 0x5622925ae4d0 2024-05-14T09:38:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.701+0000 7f94653ad740 1 -- 172.21.15.2:0/1933381198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 msgr2=0x562292981a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.701+0000 7f94653ad740 1 --2- 172.21.15.2:0/1933381198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x562292981a30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f94400e2df0 tx=0x7f94400e6020 comp rx=0 tx=0).stop 2024-05-14T09:38:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.701+0000 7f94653ad740 1 -- 172.21.15.2:0/1933381198 shutdown_connections 2024-05-14T09:38:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.701+0000 7f94653ad740 1 --2- 172.21.15.2:0/1933381198 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925cf1b0 0x562292975040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.701+0000 7f94653ad740 1 --2- 172.21.15.2:0/1933381198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925ae4d0 0x562292981a30 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.701+0000 7f94653ad740 1 --2- 172.21.15.2:0/1933381198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925a62f0 0x562292974b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.701+0000 7f94653ad740 1 -- 172.21.15.2:0/1933381198 >> 172.21.15.2:0/1933381198 conn(0x5622925c3300 msgr2=0x56229297a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.701+0000 7f94653ad740 1 -- 172.21.15.2:0/1933381198 shutdown_connections 2024-05-14T09:38:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.701+0000 7f94653ad740 1 -- 172.21.15.2:0/1933381198 wait complete. 2024-05-14T09:38:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.702+0000 7f94653ad740 1 -- start start 2024-05-14T09:38:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.702+0000 7f94653ad740 1 Processor -- start 2024-05-14T09:38:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.702+0000 7f94653ad740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925a62f0 0x56229297ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.702+0000 7f94653ad740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925ae4d0 0x56229297ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.702+0000 7f94653ad740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925cf1b0 0x5622929ee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.702+0000 7f94653ad740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f94581ab390 con 0x5622925a62f0 2024-05-14T09:38:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.702+0000 7f94653ad740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94581ac800 con 0x5622925cf1b0 2024-05-14T09:38:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.702+0000 7f94653ad740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f94581aba60 con 0x5622925ae4d0 2024-05-14T09:38:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.702+0000 7f94653ad740 10 client.0 _ll_register_callbacks cb 0x7f944000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:38:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.702+0000 7f9457fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925a62f0 0x56229297ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.702+0000 7f944ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925ae4d0 0x56229297ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.702+0000 7f9457fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925a62f0 0x56229297ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33834/0 (socket says 172.21.15.2:33834) 2024-05-14T09:38:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.702+0000 7f9457fff640 1 -- 172.21.15.2:0/2996636860 learned_addr learned my addr 172.21.15.2:0/2996636860 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.702+0000 7f945c9bf640 1 --2- 172.21.15.2:0/2996636860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925cf1b0 0x5622929ee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.703+0000 7f9457fff640 1 -- 172.21.15.2:0/2996636860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925ae4d0 msgr2=0x56229297ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.703+0000 7f9457fff640 1 --2- 172.21.15.2:0/2996636860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925ae4d0 0x56229297ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.703+0000 7f9457fff640 1 -- 172.21.15.2:0/2996636860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925cf1b0 msgr2=0x5622929ee4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.703+0000 7f9457fff640 1 --2- 172.21.15.2:0/2996636860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925cf1b0 0x5622929ee4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.703+0000 7f9457fff640 1 -- 172.21.15.2:0/2996636860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622929eea20 con 0x5622925a62f0 2024-05-14T09:38:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.703+0000 7f944ffff640 1 --2- 172.21.15.2:0/2996636860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925ae4d0 0x56229297ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:38:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.703+0000 7f9457fff640 1 --2- 172.21.15.2:0/2996636860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925a62f0 0x56229297ea30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9448005ab0 tx=0x7f9448009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c334d176783afa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.703+0000 7f94377fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:38:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.703+0000 7f94377fe640 1 -- 172.21.15.2:0/2996636860 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448035040 con 0x5622925a62f0 2024-05-14T09:38:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.703+0000 7f94377fe640 1 -- 172.21.15.2:0/2996636860 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f9448005040 con 0x5622925a62f0 2024-05-14T09:38:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.703+0000 7f94653ad740 1 -- 172.21.15.2:0/2996636860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94400c4140 con 0x5622925a62f0 2024-05-14T09:38:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.703+0000 7f94377fe640 1 -- 172.21.15.2:0/2996636860 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9448005240 con 0x5622925a62f0 2024-05-14T09:38:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.704+0000 7f94653ad740 10 client.16032 fetch_fsmap learned FSMap version 29 2024-05-14T09:38:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.704+0000 7f94653ad740 1 -- 172.21.15.2:0/2996636860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5622929efd60 con 0x5622925a62f0 2024-05-14T09:38:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.704+0000 7f94377fe640 1 -- 172.21.15.2:0/2996636860 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94400c4140 con 0x5622925a62f0 2024-05-14T09:38:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.704+0000 7f94377fe640 1 -- 172.21.15.2:0/2996636860 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9448053300 con 0x5622925a62f0 2024-05-14T09:38:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.704+0000 7f94653ad740 10 client.16032 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:38:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.704+0000 7f94653ad740 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:38:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.704+0000 7f94653ad740 20 client.16032 populate_metadata read hostname 'smithi002' 2024-05-14T09:38:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.704+0000 7f94653ad740 1 --2- 172.21.15.2:0/2996636860 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5622925631c0 0x5622929f2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.704+0000 7f94653ad740 4 client.16032 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:38:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.704+0000 7f94653ad740 1 -- 172.21.15.2:0/2996636860 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622928dd050 con 0x5622925631c0 2024-05-14T09:38:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.704+0000 7f944ffff640 1 --2- 172.21.15.2:0/2996636860 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5622925631c0 0x5622929f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.705+0000 7f94377fe640 10 client.16032 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.705+0000 7f944ffff640 1 --2- 172.21.15.2:0/2996636860 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5622925631c0 0x5622929f2790 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=907073104b6513f9 server_cookie=22d8d7d1091efc75 in_seq=0 out_seq=0 2024-05-14T09:38:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f94377fe640 1 -- 172.21.15.2:0/2996636860 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5622928dd050 con 0x5622925631c0 2024-05-14T09:38:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f94377fe640 10 client.16032 handle_command_reply: tid=0 2024-05-14T09:38:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f94653ad740 1 client.16032 shutdown 2024-05-14T09:38:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f94653ad740 1 -- 172.21.15.2:0/2996636860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925a62f0 msgr2=0x56229297ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f94653ad740 1 --2- 172.21.15.2:0/2996636860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925a62f0 0x56229297ea30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9448005ab0 tx=0x7f9448009260 comp rx=0 tx=0).stop 2024-05-14T09:38:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f94653ad740 1 -- 172.21.15.2:0/2996636860 shutdown_connections 2024-05-14T09:38:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f94653ad740 1 --2- 172.21.15.2:0/2996636860 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5622925631c0 0x5622929f2790 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f94653ad740 1 --2- 172.21.15.2:0/2996636860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622925cf1b0 0x5622929ee4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f94653ad740 1 --2- 172.21.15.2:0/2996636860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5622925ae4d0 0x56229297ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f94377fe640 0 client.16032 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f94653ad740 1 --2- 172.21.15.2:0/2996636860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622925a62f0 0x56229297ea30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f94653ad740 1 -- 172.21.15.2:0/2996636860 >> 172.21.15.2:0/2996636860 conn(0x5622925c3300 msgr2=0x56229296f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f94653ad740 1 -- 172.21.15.2:0/2996636860 shutdown_connections 2024-05-14T09:38:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f94653ad740 1 -- 172.21.15.2:0/2996636860 wait complete. 2024-05-14T09:38:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f94653ad740 20 client.16032 trim_cache size 0 max 16384 2024-05-14T09:38:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:38:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:38:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:38:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:38:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f945e422640 1 -- 172.21.15.2:0/2873708643 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f94440b28a0 msgr2=0x7f94440b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f945e422640 1 --2- 172.21.15.2:0/2873708643 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f94440b28a0 0x7f94440b4db0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9448031500 tx=0x7f9448009660 comp rx=0 tx=0).stop 2024-05-14T09:38:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f945e422640 1 -- 172.21.15.2:0/2873708643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9458104ed0 msgr2=0x7f945819a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f945e422640 1 --2- 172.21.15.2:0/2873708643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9458104ed0 0x7f945819a840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f944000e9f0 tx=0x7f944000eeb0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f945e422640 1 -- 172.21.15.2:0/2873708643 shutdown_connections 2024-05-14T09:38:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f945e422640 1 --2- 172.21.15.2:0/2873708643 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f94440b28a0 0x7f94440b4db0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f945e422640 1 --2- 172.21.15.2:0/2873708643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9458109bd0 0x7f94581a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f945e422640 1 --2- 172.21.15.2:0/2873708643 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9458104ed0 0x7f945819a840 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f945e422640 1 --2- 172.21.15.2:0/2873708643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9458103c90 0x7f945819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f945e422640 1 -- 172.21.15.2:0/2873708643 >> 172.21.15.2:0/2873708643 conn(0x7f94580feea0 msgr2=0x7f9458105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f945e422640 1 -- 172.21.15.2:0/2873708643 shutdown_connections 2024-05-14T09:38:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:33.793+0000 7f945e422640 1 -- 172.21.15.2:0/2873708643 wait complete. 2024-05-14T09:38:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:34 smithi002 ceph-mon[31557]: pgmap v532: 129 pgs: 129 active+clean; 60 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 362 op/s 2024-05-14T09:38:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:34 smithi029 ceph-mon[36249]: pgmap v532: 129 pgs: 129 active+clean; 60 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 362 op/s 2024-05-14T09:38:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:34 smithi026 ceph-mon[36056]: pgmap v532: 129 pgs: 129 active+clean; 60 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 362 op/s 2024-05-14T09:38:35.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 241354 16.90 MB/sec execute 1 sec latency 104.984 ms 2024-05-14T09:38:36.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 242848 11.73 MB/sec execute 2 sec latency 16.833 ms 2024-05-14T09:38:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:36 smithi002 ceph-mon[31557]: pgmap v533: 129 pgs: 129 active+clean; 60 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 237 op/s 2024-05-14T09:38:36.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:36 smithi029 ceph-mon[36249]: pgmap v533: 129 pgs: 129 active+clean; 60 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 237 op/s 2024-05-14T09:38:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:36 smithi026 ceph-mon[36056]: pgmap v533: 129 pgs: 129 active+clean; 60 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 237 op/s 2024-05-14T09:38:37.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 245154 12.46 MB/sec execute 3 sec latency 15.216 ms 2024-05-14T09:38:38.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 246680 10.70 MB/sec execute 4 sec latency 62.739 ms 2024-05-14T09:38:38.738 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:38 smithi029 ceph-mon[36249]: pgmap v534: 129 pgs: 129 active+clean; 56 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 302 op/s 2024-05-14T09:38:38.752 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:38 smithi026 ceph-mon[36056]: pgmap v534: 129 pgs: 129 active+clean; 56 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 302 op/s 2024-05-14T09:38:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:38 smithi002 ceph-mon[31557]: pgmap v534: 129 pgs: 129 active+clean; 56 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 302 op/s 2024-05-14T09:38:39.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 248700 11.30 MB/sec execute 5 sec latency 28.816 ms 2024-05-14T09:38:40.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 250730 10.59 MB/sec execute 6 sec latency 14.636 ms 2024-05-14T09:38:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:40 smithi002 ceph-mon[31557]: pgmap v535: 129 pgs: 129 active+clean; 59 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 312 op/s 2024-05-14T09:38:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:40 smithi029 ceph-mon[36249]: pgmap v535: 129 pgs: 129 active+clean; 59 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 312 op/s 2024-05-14T09:38:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:40 smithi026 ceph-mon[36056]: pgmap v535: 129 pgs: 129 active+clean; 59 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 312 op/s 2024-05-14T09:38:41.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 252432 10.81 MB/sec execute 7 sec latency 15.491 ms 2024-05-14T09:38:42.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 254752 10.86 MB/sec execute 8 sec latency 12.637 ms 2024-05-14T09:38:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:42 smithi002 ceph-mon[31557]: pgmap v536: 129 pgs: 129 active+clean; 60 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-05-14T09:38:42.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:42 smithi029 ceph-mon[36249]: pgmap v536: 129 pgs: 129 active+clean; 60 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-05-14T09:38:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:42 smithi026 ceph-mon[36056]: pgmap v536: 129 pgs: 129 active+clean; 60 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-05-14T09:38:43.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 256304 10.61 MB/sec execute 9 sec latency 15.294 ms 2024-05-14T09:38:43.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:43 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:38:43.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:43 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:38:43.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:43 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:38:43.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:43 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:38:43.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:43 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:38:43.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:43 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:38:43.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:43 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:38:43.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:43 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:38:43.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:43 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:38:43.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:43 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:38:43.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:43 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:38:43.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:43 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:38:44.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 258771 10.92 MB/sec execute 10 sec latency 46.904 ms 2024-05-14T09:38:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:44 smithi002 ceph-mon[31557]: pgmap v537: 129 pgs: 129 active+clean; 63 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 380 op/s 2024-05-14T09:38:44.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:44 smithi029 ceph-mon[36249]: pgmap v537: 129 pgs: 129 active+clean; 63 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 380 op/s 2024-05-14T09:38:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:44 smithi026 ceph-mon[36056]: pgmap v537: 129 pgs: 129 active+clean; 63 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 380 op/s 2024-05-14T09:38:45.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 260095 10.53 MB/sec execute 11 sec latency 46.242 ms 2024-05-14T09:38:46.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 262584 10.96 MB/sec execute 12 sec latency 12.649 ms 2024-05-14T09:38:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.897+0000 7f3150514640 1 -- 172.21.15.2:0/4272475519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31481070a0 msgr2=0x7f31481094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.897+0000 7f3150514640 1 --2- 172.21.15.2:0/4272475519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31481070a0 0x7f31481094c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f313c009900 tx=0x7f313c02a410 comp rx=0 tx=0).stop 2024-05-14T09:38:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.897+0000 7f3150514640 1 -- 172.21.15.2:0/4272475519 shutdown_connections 2024-05-14T09:38:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.897+0000 7f3150514640 1 --2- 172.21.15.2:0/4272475519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3148109a00 0x7f314810be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.897+0000 7f3150514640 1 --2- 172.21.15.2:0/4272475519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31481070a0 0x7f31481094c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.897+0000 7f3150514640 1 --2- 172.21.15.2:0/4272475519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3148104740 0x7f3148106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.897+0000 7f3150514640 1 -- 172.21.15.2:0/4272475519 >> 172.21.15.2:0/4272475519 conn(0x7f31480fdd40 msgr2=0x7f31481001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.897+0000 7f3150514640 1 -- 172.21.15.2:0/4272475519 shutdown_connections 2024-05-14T09:38:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.897+0000 7f3150514640 1 -- 172.21.15.2:0/4272475519 wait complete. 2024-05-14T09:38:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.898+0000 7f3150514640 1 Processor -- start 2024-05-14T09:38:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.898+0000 7f3150514640 1 -- start start 2024-05-14T09:38:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.898+0000 7f3150514640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3148104740 0x7f314819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.898+0000 7f3150514640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31481070a0 0x7f314819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.898+0000 7f3150514640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3148109a00 0x7f31481a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.898+0000 7f3150514640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f314810f780 con 0x7f31481070a0 2024-05-14T09:38:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.898+0000 7f3150514640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f314810f600 con 0x7f3148109a00 2024-05-14T09:38:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.898+0000 7f3150514640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f314810f900 con 0x7f3148104740 2024-05-14T09:38:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.898+0000 7f314daaf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31481070a0 0x7f314819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.898+0000 7f314daaf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31481070a0 0x7f314819a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53962/0 (socket says 172.21.15.2:53962) 2024-05-14T09:38:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.899+0000 7f314daaf640 1 -- 172.21.15.2:0/1384006214 learned_addr learned my addr 172.21.15.2:0/1384006214 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.899+0000 7f314eab1640 1 --2- 172.21.15.2:0/1384006214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3148109a00 0x7f31481a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.899+0000 7f314e2b0640 1 --2- 172.21.15.2:0/1384006214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3148104740 0x7f314819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.899+0000 7f314daaf640 1 -- 172.21.15.2:0/1384006214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3148104740 msgr2=0x7f314819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.899+0000 7f314daaf640 1 --2- 172.21.15.2:0/1384006214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3148104740 0x7f314819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.899+0000 7f314daaf640 1 -- 172.21.15.2:0/1384006214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3148109a00 msgr2=0x7f31481a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.899+0000 7f314daaf640 1 --2- 172.21.15.2:0/1384006214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3148109a00 0x7f31481a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.899+0000 7f314daaf640 1 -- 172.21.15.2:0/1384006214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31481a1e30 con 0x7f31481070a0 2024-05-14T09:38:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.899+0000 7f314eab1640 1 --2- 172.21.15.2:0/1384006214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3148109a00 0x7f31481a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:38:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.899+0000 7f314daaf640 1 --2- 172.21.15.2:0/1384006214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31481070a0 0x7f314819a830 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f313c006010 tx=0x7f313c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.899+0000 7f313b7fe640 1 -- 172.21.15.2:0/1384006214 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c03f030 con 0x7f31481070a0 2024-05-14T09:38:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.899+0000 7f3150514640 1 -- 172.21.15.2:0/1384006214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31481a20c0 con 0x7f31481070a0 2024-05-14T09:38:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.899+0000 7f313b7fe640 1 -- 172.21.15.2:0/1384006214 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f313c031910 con 0x7f31481070a0 2024-05-14T09:38:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.899+0000 7f313b7fe640 1 -- 172.21.15.2:0/1384006214 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c035070 con 0x7f31481070a0 2024-05-14T09:38:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.899+0000 7f3150514640 1 -- 172.21.15.2:0/1384006214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31481a2600 con 0x7f31481070a0 2024-05-14T09:38:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.902+0000 7f313b7fe640 1 -- 172.21.15.2:0/1384006214 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f313c042030 con 0x7f31481070a0 2024-05-14T09:38:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.902+0000 7f31573ca740 1 Processor -- start 2024-05-14T09:38:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.902+0000 7f31573ca740 1 -- start start 2024-05-14T09:38:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f31573ca740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645da81b0 0x55c64614de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f31573ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d7f2f0 0x55c64614e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f31573ca740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d874d0 0x55c6461552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f31573ca740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c6460ae5d0 con 0x55c645da81b0 2024-05-14T09:38:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f31573ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c646067890 con 0x55c645d7f2f0 2024-05-14T09:38:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f31573ca740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c6460aa590 con 0x55c645d874d0 2024-05-14T09:38:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f314eab1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645da81b0 0x55c64614de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f314eab1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645da81b0 0x55c64614de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53970/0 (socket says 172.21.15.2:53970) 2024-05-14T09:38:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f314eab1640 1 -- 172.21.15.2:0/1010340519 learned_addr learned my addr 172.21.15.2:0/1010340519 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f314e2b0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d7f2f0 0x55c64614e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f314daaf640 1 --2- 172.21.15.2:0/1010340519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d874d0 0x55c6461552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f314e2b0640 1 -- 172.21.15.2:0/1010340519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d874d0 msgr2=0x55c6461552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f314e2b0640 1 --2- 172.21.15.2:0/1010340519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d874d0 0x55c6461552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f314e2b0640 1 -- 172.21.15.2:0/1010340519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645da81b0 msgr2=0x55c64614de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f314e2b0640 1 --2- 172.21.15.2:0/1010340519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645da81b0 0x55c64614de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f314e2b0640 1 -- 172.21.15.2:0/1010340519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c645dc0210 con 0x55c645d7f2f0 2024-05-14T09:38:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f314eab1640 1 --2- 172.21.15.2:0/1010340519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645da81b0 0x55c64614de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:38:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f313b7fe640 1 --2- 172.21.15.2:0/1384006214 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f31340b2920 0x7f31340b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.903+0000 7f314eab1640 1 --2- 172.21.15.2:0/1384006214 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f31340b2920 0x7f31340b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.904+0000 7f313b7fe640 1 -- 172.21.15.2:0/1384006214 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f313c102b00 con 0x7f31481070a0 2024-05-14T09:38:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.904+0000 7f314e2b0640 1 --2- 172.21.15.2:0/1010340519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d7f2f0 0x55c64614e3f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3144009b40 tx=0x7f31440113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a451accf4b0c9693 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.904+0000 7f3138bfe640 1 -- 172.21.15.2:0/1010340519 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144011e10 con 0x55c645d7f2f0 2024-05-14T09:38:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.904+0000 7f3138bfe640 1 -- 172.21.15.2:0/1010340519 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3144012900 con 0x55c645d7f2f0 2024-05-14T09:38:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.904+0000 7f3138bfe640 1 -- 172.21.15.2:0/1010340519 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314400f920 con 0x55c645d7f2f0 2024-05-14T09:38:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.905+0000 7f31573ca740 1 -- 172.21.15.2:0/1010340519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d7f2f0 msgr2=0x55c64614e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.905+0000 7f31573ca740 1 --2- 172.21.15.2:0/1010340519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d7f2f0 0x55c64614e3f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3144009b40 tx=0x7f31440113e0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.905+0000 7f314eab1640 1 --2- 172.21.15.2:0/1384006214 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f31340b2920 0x7f31340b4e30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55c64614f040 tx=0x7f3140009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.905+0000 7f31573ca740 1 -- 172.21.15.2:0/1010340519 shutdown_connections 2024-05-14T09:38:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.905+0000 7f31573ca740 1 --2- 172.21.15.2:0/1010340519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d874d0 0x55c6461552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.905+0000 7f31573ca740 1 --2- 172.21.15.2:0/1010340519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d7f2f0 0x55c64614e3f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.905+0000 7f31573ca740 1 --2- 172.21.15.2:0/1010340519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645da81b0 0x55c64614de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.905+0000 7f31573ca740 1 -- 172.21.15.2:0/1010340519 >> 172.21.15.2:0/1010340519 conn(0x55c645d9c300 msgr2=0x55c6461453a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.905+0000 7f31573ca740 1 -- 172.21.15.2:0/1010340519 shutdown_connections 2024-05-14T09:38:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.905+0000 7f31573ca740 1 -- 172.21.15.2:0/1010340519 wait complete. 2024-05-14T09:38:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.905+0000 7f31573ca740 1 -- start start 2024-05-14T09:38:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.905+0000 7f31573ca740 1 Processor -- start 2024-05-14T09:38:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f31573ca740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645d7f2f0 0x55c646157a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f31573ca740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d874d0 0x55c646157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f31573ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645da81b0 0x55c64615eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f31573ca740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c646155d40 con 0x55c645d7f2f0 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f31573ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c64606b370 con 0x55c645da81b0 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f31573ca740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c646155ec0 con 0x55c645d874d0 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f31573ca740 10 client.0 _ll_register_callbacks cb 0x55c645dbfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f314e2b0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645d7f2f0 0x55c646157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f314e2b0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645d7f2f0 0x55c646157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53972/0 (socket says 172.21.15.2:53972) 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f314e2b0640 1 -- 172.21.15.2:0/1353101044 learned_addr learned my addr 172.21.15.2:0/1353101044 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f314daaf640 1 --2- 172.21.15.2:0/1353101044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d874d0 0x55c646157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f314eab1640 1 --2- 172.21.15.2:0/1353101044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645da81b0 0x55c64615eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f314e2b0640 1 -- 172.21.15.2:0/1353101044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d874d0 msgr2=0x55c646157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f314e2b0640 1 --2- 172.21.15.2:0/1353101044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d874d0 0x55c646157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f314e2b0640 1 -- 172.21.15.2:0/1353101044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645da81b0 msgr2=0x55c64615eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f314e2b0640 1 --2- 172.21.15.2:0/1353101044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645da81b0 0x55c64615eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f314e2b0640 1 -- 172.21.15.2:0/1353101044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c645e8c620 con 0x55c645d7f2f0 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f314e2b0640 1 --2- 172.21.15.2:0/1353101044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645d7f2f0 0x55c646157a30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3144009740 tx=0x7f31440126e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e75d54d1a4300f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.906+0000 7f3131ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:38:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.907+0000 7f3131ffb640 1 -- 172.21.15.2:0/1353101044 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144028070 con 0x55c645d7f2f0 2024-05-14T09:38:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.907+0000 7f31573ca740 1 -- 172.21.15.2:0/1353101044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3144012900 con 0x55c645d7f2f0 2024-05-14T09:38:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.907+0000 7f3131ffb640 1 -- 172.21.15.2:0/1353101044 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3144005040 con 0x55c645d7f2f0 2024-05-14T09:38:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.907+0000 7f3131ffb640 1 -- 172.21.15.2:0/1353101044 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3144034050 con 0x55c645d7f2f0 2024-05-14T09:38:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.907+0000 7f31573ca740 10 client.16044 fetch_fsmap learned FSMap version 29 2024-05-14T09:38:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.907+0000 7f31573ca740 1 -- 172.21.15.2:0/1353101044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c646161e50 con 0x55c645d7f2f0 2024-05-14T09:38:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.907+0000 7f3131ffb640 1 -- 172.21.15.2:0/1353101044 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3144012900 con 0x55c645d7f2f0 2024-05-14T09:38:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.907+0000 7f3131ffb640 1 -- 172.21.15.2:0/1353101044 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f314400f660 con 0x55c645d7f2f0 2024-05-14T09:38:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.907+0000 7f31573ca740 10 client.16044 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:38:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.907+0000 7f31573ca740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:38:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.907+0000 7f31573ca740 20 client.16044 populate_metadata read hostname 'smithi002' 2024-05-14T09:38:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.907+0000 7f31573ca740 1 --2- 172.21.15.2:0/1353101044 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c645d66d80 0x55c646165330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.907+0000 7f31573ca740 4 client.16044 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:38:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.907+0000 7f31573ca740 1 -- 172.21.15.2:0/1353101044 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6460bc1e0 con 0x55c645d66d80 2024-05-14T09:38:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.907+0000 7f314daaf640 1 --2- 172.21.15.2:0/1353101044 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c645d66d80 0x55c646165330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.908+0000 7f314daaf640 1 --2- 172.21.15.2:0/1353101044 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c645d66d80 0x55c646165330 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1caab1a82240f344 server_cookie=9e27723ee6dfb0fb in_seq=0 out_seq=0 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.908+0000 7f3131ffb640 10 client.16044 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.909+0000 7f3131ffb640 1 -- 172.21.15.2:0/1353101044 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c6460bc1e0 con 0x55c645d66d80 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.909+0000 7f3131ffb640 10 client.16044 handle_command_reply: tid=0 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.909+0000 7f31573ca740 1 client.16044 shutdown 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.910+0000 7f31573ca740 1 -- 172.21.15.2:0/1353101044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645d7f2f0 msgr2=0x55c646157a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.910+0000 7f31573ca740 1 --2- 172.21.15.2:0/1353101044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645d7f2f0 0x55c646157a30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3144009740 tx=0x7f31440126e0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.910+0000 7f31573ca740 1 -- 172.21.15.2:0/1353101044 shutdown_connections 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.910+0000 7f31573ca740 1 --2- 172.21.15.2:0/1353101044 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c645d66d80 0x55c646165330 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.910+0000 7f31573ca740 1 --2- 172.21.15.2:0/1353101044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645da81b0 0x55c64615eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.910+0000 7f31573ca740 1 --2- 172.21.15.2:0/1353101044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d874d0 0x55c646157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.910+0000 7f3131ffb640 0 client.16044 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.910+0000 7f31573ca740 1 --2- 172.21.15.2:0/1353101044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645d7f2f0 0x55c646157a30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.910+0000 7f31573ca740 1 -- 172.21.15.2:0/1353101044 >> 172.21.15.2:0/1353101044 conn(0x55c645d9c300 msgr2=0x55c646148300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.910+0000 7f31573ca740 1 -- 172.21.15.2:0/1353101044 shutdown_connections 2024-05-14T09:38:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.910+0000 7f31573ca740 1 -- 172.21.15.2:0/1353101044 wait complete. 2024-05-14T09:38:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.910+0000 7f31573ca740 20 client.16044 trim_cache size 0 max 16384 2024-05-14T09:38:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:46 smithi002 ceph-mon[31557]: pgmap v538: 129 pgs: 129 active+clean; 63 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 280 op/s 2024-05-14T09:38:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.928+0000 7f31573ca740 1 Processor -- start 2024-05-14T09:38:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.928+0000 7f31573ca740 1 -- start start 2024-05-14T09:38:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.928+0000 7f31573ca740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645d7f2f0 0x55c646157d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.928+0000 7f31573ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d874d0 0x55c64615edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.928+0000 7f31573ca740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645da81b0 0x55c64615f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.928+0000 7f31573ca740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f31481ad290 con 0x55c645d7f2f0 2024-05-14T09:38:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.928+0000 7f31573ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f31481aab60 con 0x55c645d874d0 2024-05-14T09:38:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.928+0000 7f31573ca740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f31481a89b0 con 0x55c645da81b0 2024-05-14T09:38:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.929+0000 7f314e2b0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645d7f2f0 0x55c646157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.929+0000 7f314daaf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d874d0 0x55c64615edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.929+0000 7f314daaf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d874d0 0x55c64615edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:41738/0 (socket says 172.21.15.2:41738) 2024-05-14T09:38:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.929+0000 7f314e2b0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645d7f2f0 0x55c646157d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53980/0 (socket says 172.21.15.2:53980) 2024-05-14T09:38:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.929+0000 7f314daaf640 1 -- 172.21.15.2:0/3727132515 learned_addr learned my addr 172.21.15.2:0/3727132515 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.929+0000 7f314eab1640 1 --2- 172.21.15.2:0/3727132515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645da81b0 0x55c64615f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.929+0000 7f314daaf640 1 -- 172.21.15.2:0/3727132515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645da81b0 msgr2=0x55c64615f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.929+0000 7f314daaf640 1 --2- 172.21.15.2:0/3727132515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645da81b0 0x55c64615f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.929+0000 7f314daaf640 1 -- 172.21.15.2:0/3727132515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645d7f2f0 msgr2=0x55c646157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.929+0000 7f314daaf640 1 --2- 172.21.15.2:0/3727132515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645d7f2f0 0x55c646157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.929+0000 7f314daaf640 1 -- 172.21.15.2:0/3727132515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c646158ab0 con 0x55c645d874d0 2024-05-14T09:38:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.929+0000 7f314e2b0640 1 --2- 172.21.15.2:0/3727132515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645d7f2f0 0x55c646157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:38:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.929+0000 7f314daaf640 1 --2- 172.21.15.2:0/3727132515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d874d0 0x55c64615edf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f313c0c5d80 tx=0x7f313c1065e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=773e31a6cb6491d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.929+0000 7f3133fff640 1 -- 172.21.15.2:0/3727132515 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c049040 con 0x55c645d874d0 2024-05-14T09:38:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.930+0000 7f3133fff640 1 -- 172.21.15.2:0/3727132515 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f313c10a040 con 0x55c645d874d0 2024-05-14T09:38:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.930+0000 7f3133fff640 1 -- 172.21.15.2:0/3727132515 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c0ca400 con 0x55c645d874d0 2024-05-14T09:38:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.931+0000 7f31573ca740 1 -- 172.21.15.2:0/3727132515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d874d0 msgr2=0x55c64615edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.931+0000 7f31573ca740 1 --2- 172.21.15.2:0/3727132515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d874d0 0x55c64615edf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f313c0c5d80 tx=0x7f313c1065e0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.931+0000 7f31573ca740 1 -- 172.21.15.2:0/3727132515 shutdown_connections 2024-05-14T09:38:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.931+0000 7f31573ca740 1 --2- 172.21.15.2:0/3727132515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645da81b0 0x55c64615f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.931+0000 7f31573ca740 1 --2- 172.21.15.2:0/3727132515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d874d0 0x55c64615edf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.931+0000 7f31573ca740 1 --2- 172.21.15.2:0/3727132515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645d7f2f0 0x55c646157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.931+0000 7f31573ca740 1 -- 172.21.15.2:0/3727132515 >> 172.21.15.2:0/3727132515 conn(0x55c645d9c300 msgr2=0x55c646157600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.931+0000 7f31573ca740 1 -- 172.21.15.2:0/3727132515 shutdown_connections 2024-05-14T09:38:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.931+0000 7f31573ca740 1 -- 172.21.15.2:0/3727132515 wait complete. 2024-05-14T09:38:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.931+0000 7f31573ca740 1 -- start start 2024-05-14T09:38:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.931+0000 7f31573ca740 1 Processor -- start 2024-05-14T09:38:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.931+0000 7f31573ca740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d7f2f0 0x55c64615c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f31573ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d874d0 0x55c64615cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f31573ca740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645da81b0 0x55c6461cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f31573ca740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f31481a9020 con 0x55c645da81b0 2024-05-14T09:38:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f31573ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f31481aa490 con 0x55c645d874d0 2024-05-14T09:38:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f31573ca740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f31481a96f0 con 0x55c645d7f2f0 2024-05-14T09:38:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f314e2b0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d7f2f0 0x55c64615c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f314daaf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d874d0 0x55c64615cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f314e2b0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d7f2f0 0x55c64615c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:59872/0 (socket says 172.21.15.2:59872) 2024-05-14T09:38:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f314e2b0640 1 -- 172.21.15.2:0/3325380519 learned_addr learned my addr 172.21.15.2:0/3325380519 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:38:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f314daaf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d874d0 0x55c64615cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:41750/0 (socket says 172.21.15.2:41750) 2024-05-14T09:38:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f31573ca740 10 client.0 _ll_register_callbacks cb 0x7f3140009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:38:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f314eab1640 1 --2- 172.21.15.2:0/3325380519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645da81b0 0x55c6461cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f314e2b0640 1 -- 172.21.15.2:0/3325380519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d874d0 msgr2=0x55c64615cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f314e2b0640 1 --2- 172.21.15.2:0/3325380519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d874d0 0x55c64615cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f314e2b0640 1 -- 172.21.15.2:0/3325380519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645da81b0 msgr2=0x55c6461cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:38:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f314e2b0640 1 --2- 172.21.15.2:0/3325380519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645da81b0 0x55c6461cfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f314e2b0640 1 -- 172.21.15.2:0/3325380519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c64614d670 con 0x55c645d7f2f0 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f314e2b0640 1 --2- 172.21.15.2:0/3325380519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d7f2f0 0x55c64615c600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3144009b40 tx=0x7f3144012a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e83879d906b10410 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.932+0000 7f3131ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.933+0000 7f3131ffb640 1 -- 172.21.15.2:0/3325380519 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144028040 con 0x55c645d7f2f0 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.933+0000 7f3131ffb640 1 -- 172.21.15.2:0/3325380519 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f314400f640 con 0x55c645d7f2f0 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.933+0000 7f31573ca740 1 -- 172.21.15.2:0/3325380519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f313c10a040 con 0x55c645d7f2f0 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.933+0000 7f3131ffb640 1 -- 172.21.15.2:0/3325380519 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f314400f840 con 0x55c645d7f2f0 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.933+0000 7f31573ca740 10 client.25732 fetch_fsmap learned FSMap version 29 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.933+0000 7f31573ca740 1 -- 172.21.15.2:0/3325380519 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c6461d1140 con 0x55c645d7f2f0 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.933+0000 7f3131ffb640 1 -- 172.21.15.2:0/3325380519 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f313c10a040 con 0x55c645d7f2f0 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.933+0000 7f3131ffb640 1 -- 172.21.15.2:0/3325380519 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3144003770 con 0x55c645d7f2f0 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.933+0000 7f31573ca740 10 client.25732 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.933+0000 7f31573ca740 10 client.25732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.933+0000 7f31573ca740 20 client.25732 populate_metadata read hostname 'smithi002' 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.933+0000 7f31573ca740 1 --2- 172.21.15.2:0/3325380519 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c645d3c1c0 0x55c6461d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:38:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.933+0000 7f31573ca740 4 client.25732 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:38:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.933+0000 7f31573ca740 1 -- 172.21.15.2:0/3325380519 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6460b6050 con 0x55c645d3c1c0 2024-05-14T09:38:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.934+0000 7f314daaf640 1 --2- 172.21.15.2:0/3325380519 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c645d3c1c0 0x55c6461d3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:38:46.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:46 smithi029 ceph-mon[36249]: pgmap v538: 129 pgs: 129 active+clean; 63 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 280 op/s 2024-05-14T09:38:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.934+0000 7f3131ffb640 10 client.25732 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:46.934+0000 7f314daaf640 1 --2- 172.21.15.2:0/3325380519 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c645d3c1c0 0x55c6461d3b90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cf62102720a83a4 server_cookie=653e9004c7c99ee5 in_seq=0 out_seq=0 2024-05-14T09:38:46.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:46 smithi026 ceph-mon[36056]: pgmap v538: 129 pgs: 129 active+clean; 63 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 280 op/s 2024-05-14T09:38:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.024+0000 7f3131ffb640 1 -- 172.21.15.2:0/3325380519 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6460b6050 con 0x55c645d3c1c0 2024-05-14T09:38:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.024+0000 7f3131ffb640 10 client.25732 handle_command_reply: tid=0 2024-05-14T09:38:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.024+0000 7f31573ca740 1 client.25732 shutdown 2024-05-14T09:38:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f31573ca740 1 -- 172.21.15.2:0/3325380519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d7f2f0 msgr2=0x55c64615c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f31573ca740 1 --2- 172.21.15.2:0/3325380519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d7f2f0 0x55c64615c600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3144009b40 tx=0x7f3144012a00 comp rx=0 tx=0).stop 2024-05-14T09:38:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f31573ca740 1 -- 172.21.15.2:0/3325380519 shutdown_connections 2024-05-14T09:38:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f31573ca740 1 --2- 172.21.15.2:0/3325380519 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c645d3c1c0 0x55c6461d3b90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f31573ca740 1 --2- 172.21.15.2:0/3325380519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c645da81b0 0x55c6461cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f3131ffb640 0 client.25732 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:38:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f31573ca740 1 --2- 172.21.15.2:0/3325380519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c645d874d0 0x55c64615cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f31573ca740 1 --2- 172.21.15.2:0/3325380519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c645d7f2f0 0x55c64615c600 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f31573ca740 1 -- 172.21.15.2:0/3325380519 >> 172.21.15.2:0/3325380519 conn(0x55c645d9c300 msgr2=0x55c6461481f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f31573ca740 1 -- 172.21.15.2:0/3325380519 shutdown_connections 2024-05-14T09:38:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f31573ca740 1 -- 172.21.15.2:0/3325380519 wait complete. 2024-05-14T09:38:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f31573ca740 20 client.25732 trim_cache size 0 max 16384 2024-05-14T09:38:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:38:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:38:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:38:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:38:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f3150514640 1 -- 172.21.15.2:0/1384006214 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f31340b2920 msgr2=0x7f31340b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f3150514640 1 --2- 172.21.15.2:0/1384006214 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f31340b2920 0x7f31340b4e30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55c64614f040 tx=0x7f3140009500 comp rx=0 tx=0).stop 2024-05-14T09:38:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f3150514640 1 -- 172.21.15.2:0/1384006214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31481070a0 msgr2=0x7f314819a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:38:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.025+0000 7f3150514640 1 --2- 172.21.15.2:0/1384006214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31481070a0 0x7f314819a830 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f313c006010 tx=0x7f313c0042f0 comp rx=0 tx=0).stop 2024-05-14T09:38:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.026+0000 7f3150514640 1 -- 172.21.15.2:0/1384006214 shutdown_connections 2024-05-14T09:38:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.026+0000 7f3150514640 1 --2- 172.21.15.2:0/1384006214 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f31340b2920 0x7f31340b4e30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.026+0000 7f3150514640 1 --2- 172.21.15.2:0/1384006214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3148109a00 0x7f31481a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.026+0000 7f3150514640 1 --2- 172.21.15.2:0/1384006214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31481070a0 0x7f314819a830 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.026+0000 7f3150514640 1 --2- 172.21.15.2:0/1384006214 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3148104740 0x7f314819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:38:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.026+0000 7f3150514640 1 -- 172.21.15.2:0/1384006214 >> 172.21.15.2:0/1384006214 conn(0x7f31480fdd40 msgr2=0x7f3148105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:38:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.026+0000 7f3150514640 1 -- 172.21.15.2:0/1384006214 shutdown_connections 2024-05-14T09:38:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:38:47.026+0000 7f3150514640 1 -- 172.21.15.2:0/1384006214 wait complete. 2024-05-14T09:38:47.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 263774 10.39 MB/sec execute 13 sec latency 47.969 ms 2024-05-14T09:38:48.259 INFO:tasks.workunit.client.0.smithi002.stdout: 1 266352 10.84 MB/sec execute 14 sec latency 14.007 ms 2024-05-14T09:38:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:48 smithi002 ceph-mon[31557]: pgmap v539: 129 pgs: 129 active+clean; 64 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 364 op/s 2024-05-14T09:38:48.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:48 smithi029 ceph-mon[36249]: pgmap v539: 129 pgs: 129 active+clean; 64 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 364 op/s 2024-05-14T09:38:48.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:48 smithi026 ceph-mon[36056]: pgmap v539: 129 pgs: 129 active+clean; 64 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 364 op/s 2024-05-14T09:38:49.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 267836 10.49 MB/sec execute 15 sec latency 46.452 ms 2024-05-14T09:38:50.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 269994 10.70 MB/sec execute 16 sec latency 61.619 ms 2024-05-14T09:38:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:50 smithi002 ceph-mon[31557]: pgmap v540: 129 pgs: 129 active+clean; 62 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 364 op/s 2024-05-14T09:38:50.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:50 smithi029 ceph-mon[36249]: pgmap v540: 129 pgs: 129 active+clean; 62 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 364 op/s 2024-05-14T09:38:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:50 smithi026 ceph-mon[36056]: pgmap v540: 129 pgs: 129 active+clean; 62 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 364 op/s 2024-05-14T09:38:51.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 272032 10.49 MB/sec execute 17 sec latency 16.026 ms 2024-05-14T09:38:52.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 273745 10.58 MB/sec execute 18 sec latency 9.662 ms 2024-05-14T09:38:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:52 smithi002 ceph-mon[31557]: pgmap v541: 129 pgs: 129 active+clean; 62 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 274 op/s 2024-05-14T09:38:52.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:52 smithi029 ceph-mon[36249]: pgmap v541: 129 pgs: 129 active+clean; 62 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 274 op/s 2024-05-14T09:38:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:52 smithi026 ceph-mon[36056]: pgmap v541: 129 pgs: 129 active+clean; 62 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 274 op/s 2024-05-14T09:38:53.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 276171 10.68 MB/sec execute 19 sec latency 15.717 ms 2024-05-14T09:38:54.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 277494 10.50 MB/sec execute 20 sec latency 56.555 ms 2024-05-14T09:38:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:54 smithi002 ceph-mon[31557]: pgmap v542: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 380 op/s 2024-05-14T09:38:54.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:54 smithi029 ceph-mon[36249]: pgmap v542: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 380 op/s 2024-05-14T09:38:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:54 smithi026 ceph-mon[36056]: pgmap v542: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 380 op/s 2024-05-14T09:38:55.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 280043 10.66 MB/sec execute 21 sec latency 109.377 ms 2024-05-14T09:38:56.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 281463 10.49 MB/sec execute 22 sec latency 12.082 ms 2024-05-14T09:38:56.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:56 smithi002 ceph-mon[31557]: pgmap v543: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 275 op/s 2024-05-14T09:38:56.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:56 smithi029 ceph-mon[36249]: pgmap v543: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 275 op/s 2024-05-14T09:38:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:56 smithi026 ceph-mon[36056]: pgmap v543: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 275 op/s 2024-05-14T09:38:57.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 283987 10.76 MB/sec execute 23 sec latency 14.079 ms 2024-05-14T09:38:58.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 285433 10.54 MB/sec execute 24 sec latency 12.657 ms 2024-05-14T09:38:58.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:38:58 smithi002 ceph-mon[31557]: pgmap v544: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 363 op/s 2024-05-14T09:38:58.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:38:58 smithi029 ceph-mon[36249]: pgmap v544: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 363 op/s 2024-05-14T09:38:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:38:58 smithi026 ceph-mon[36056]: pgmap v544: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 363 op/s 2024-05-14T09:38:59.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 287578 10.66 MB/sec execute 25 sec latency 51.786 ms 2024-05-14T09:39:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.138+0000 7f24832c6640 1 -- 172.21.15.2:0/2464199829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f247c108a90 msgr2=0x7f247c10aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.138+0000 7f24832c6640 1 --2- 172.21.15.2:0/2464199829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f247c108a90 0x7f247c10aeb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2474009ca0 tx=0x7f247402a8c0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.138+0000 7f24832c6640 1 -- 172.21.15.2:0/2464199829 shutdown_connections 2024-05-14T09:39:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.138+0000 7f24832c6640 1 --2- 172.21.15.2:0/2464199829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f247c108a90 0x7f247c10aeb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.138+0000 7f24832c6640 1 --2- 172.21.15.2:0/2464199829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c103d90 0x7f247c1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.138+0000 7f24832c6640 1 --2- 172.21.15.2:0/2464199829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f247c102b50 0x7f247c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.138+0000 7f24832c6640 1 -- 172.21.15.2:0/2464199829 >> 172.21.15.2:0/2464199829 conn(0x7f247c0fdd40 msgr2=0x7f247c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.138+0000 7f24832c6640 1 -- 172.21.15.2:0/2464199829 shutdown_connections 2024-05-14T09:39:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.138+0000 7f24832c6640 1 -- 172.21.15.2:0/2464199829 wait complete. 2024-05-14T09:39:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f24832c6640 1 Processor -- start 2024-05-14T09:39:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f24832c6640 1 -- start start 2024-05-14T09:39:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f24832c6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f247c102b50 0x7f247c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f24832c6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c103d90 0x7f247c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f24832c6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f247c108a90 0x7f247c1a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f24832c6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f247c10e680 con 0x7f247c108a90 2024-05-14T09:39:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f24832c6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f247c10e500 con 0x7f247c102b50 2024-05-14T09:39:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f24832c6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f247c10e800 con 0x7f247c103d90 2024-05-14T09:39:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f2481863640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f247c108a90 0x7f247c1a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f2480861640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c103d90 0x7f247c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f2481062640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f247c102b50 0x7f247c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f2481863640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f247c108a90 0x7f247c1a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40752/0 (socket says 172.21.15.2:40752) 2024-05-14T09:39:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f2481863640 1 -- 172.21.15.2:0/3028834454 learned_addr learned my addr 172.21.15.2:0/3028834454 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f2480861640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c103d90 0x7f247c19a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:47222/0 (socket says 172.21.15.2:47222) 2024-05-14T09:39:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f2481863640 1 -- 172.21.15.2:0/3028834454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c103d90 msgr2=0x7f247c19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f2481863640 1 --2- 172.21.15.2:0/3028834454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c103d90 0x7f247c19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f2481863640 1 -- 172.21.15.2:0/3028834454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f247c102b50 msgr2=0x7f247c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f2481863640 1 --2- 172.21.15.2:0/3028834454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f247c102b50 0x7f247c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.139+0000 7f2481863640 1 -- 172.21.15.2:0/3028834454 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c1a1e30 con 0x7f247c108a90 2024-05-14T09:39:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.140+0000 7f2481863640 1 --2- 172.21.15.2:0/3028834454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f247c108a90 0x7f247c1a17c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f24740097b0 tx=0x7f2474031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.140+0000 7f24727fc640 1 -- 172.21.15.2:0/3028834454 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474003bf0 con 0x7f247c108a90 2024-05-14T09:39:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.140+0000 7f24727fc640 1 -- 172.21.15.2:0/3028834454 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2474003d90 con 0x7f247c108a90 2024-05-14T09:39:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.140+0000 7f24832c6640 1 -- 172.21.15.2:0/3028834454 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247c1a20c0 con 0x7f247c108a90 2024-05-14T09:39:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.140+0000 7f24727fc640 1 -- 172.21.15.2:0/3028834454 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247402fc30 con 0x7f247c108a90 2024-05-14T09:39:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.140+0000 7f24832c6640 1 -- 172.21.15.2:0/3028834454 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247c1a2660 con 0x7f247c108a90 2024-05-14T09:39:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.143+0000 7f248a179740 1 Processor -- start 2024-05-14T09:39:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.143+0000 7f24727fc640 1 -- 172.21.15.2:0/3028834454 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2474004040 con 0x7f247c108a90 2024-05-14T09:39:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.143+0000 7f248a179740 1 -- start start 2024-05-14T09:39:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.143+0000 7f248a179740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac3201b0 0x557fac6c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.143+0000 7f248a179740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac2f72f0 0x557fac6c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.143+0000 7f248a179740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.143+0000 7f248a179740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557fac6265d0 con 0x557fac2f72f0 2024-05-14T09:39:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.143+0000 7f248a179740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557fac5df890 con 0x557fac3201b0 2024-05-14T09:39:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.143+0000 7f248a179740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557fac622590 con 0x557fac2ff4d0 2024-05-14T09:39:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f2481062640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac2f72f0 0x557fac6c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f2480861640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac3201b0 0x557fac6c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f2481062640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac2f72f0 0x557fac6c63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40766/0 (socket says 172.21.15.2:40766) 2024-05-14T09:39:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f2481062640 1 -- 172.21.15.2:0/1540770181 learned_addr learned my addr 172.21.15.2:0/1540770181 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f2480861640 1 --2- 172.21.15.2:0/1540770181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f2481062640 1 -- 172.21.15.2:0/1540770181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 msgr2=0x557fac6cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f2481062640 1 --2- 172.21.15.2:0/1540770181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f2481062640 1 -- 172.21.15.2:0/1540770181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac3201b0 msgr2=0x557fac6c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f2481062640 1 --2- 172.21.15.2:0/1540770181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac3201b0 0x557fac6c5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f2481062640 1 -- 172.21.15.2:0/1540770181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fac338210 con 0x557fac2f72f0 2024-05-14T09:39:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f2480861640 1 --2- 172.21.15.2:0/1540770181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac3201b0 0x557fac6c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f24727fc640 1 --2- 172.21.15.2:0/3028834454 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f24680b2920 0x7f24680b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f2481062640 1 --2- 172.21.15.2:0/1540770181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac2f72f0 0x557fac6c63f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2478009c60 tx=0x7f2478011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540966b91a4c49d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f2480861640 1 --2- 172.21.15.2:0/3028834454 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f24680b2920 0x7f24680b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f24727fc640 1 -- 172.21.15.2:0/3028834454 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f2474103c70 con 0x7f247c108a90 2024-05-14T09:39:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f24677fe640 1 -- 172.21.15.2:0/1540770181 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478004220 con 0x557fac2f72f0 2024-05-14T09:39:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f24677fe640 1 -- 172.21.15.2:0/1540770181 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24780043c0 con 0x557fac2f72f0 2024-05-14T09:39:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.144+0000 7f24677fe640 1 -- 172.21.15.2:0/1540770181 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478004cb0 con 0x557fac2f72f0 2024-05-14T09:39:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.145+0000 7f248a179740 1 -- 172.21.15.2:0/1540770181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac2f72f0 msgr2=0x557fac6c63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.145+0000 7f2480861640 1 --2- 172.21.15.2:0/3028834454 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f24680b2920 0x7f24680b4e30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x557fac6c7040 tx=0x7f246c00b6e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.145+0000 7f248a179740 1 --2- 172.21.15.2:0/1540770181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac2f72f0 0x557fac6c63f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2478009c60 tx=0x7f2478011810 comp rx=0 tx=0).stop 2024-05-14T09:39:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.145+0000 7f248a179740 1 -- 172.21.15.2:0/1540770181 shutdown_connections 2024-05-14T09:39:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.145+0000 7f248a179740 1 --2- 172.21.15.2:0/1540770181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.145+0000 7f248a179740 1 --2- 172.21.15.2:0/1540770181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac2f72f0 0x557fac6c63f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.145+0000 7f248a179740 1 --2- 172.21.15.2:0/1540770181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac3201b0 0x557fac6c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.145+0000 7f248a179740 1 -- 172.21.15.2:0/1540770181 >> 172.21.15.2:0/1540770181 conn(0x557fac314300 msgr2=0x557fac6bd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.145+0000 7f248a179740 1 -- 172.21.15.2:0/1540770181 shutdown_connections 2024-05-14T09:39:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.145+0000 7f248a179740 1 -- 172.21.15.2:0/1540770181 wait complete. 2024-05-14T09:39:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.146+0000 7f248a179740 1 -- start start 2024-05-14T09:39:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.146+0000 7f248a179740 1 Processor -- start 2024-05-14T09:39:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.146+0000 7f248a179740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac2f72f0 0x557fac6cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.146+0000 7f248a179740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.146+0000 7f248a179740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac3201b0 0x557fac6d6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.146+0000 7f248a179740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557fac6cdd40 con 0x557fac3201b0 2024-05-14T09:39:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.146+0000 7f248a179740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557fac5e3370 con 0x557fac2f72f0 2024-05-14T09:39:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.146+0000 7f248a179740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557fac6cdec0 con 0x557fac2ff4d0 2024-05-14T09:39:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.146+0000 7f248a179740 10 client.0 _ll_register_callbacks cb 0x557fac337a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:39:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.146+0000 7f2481062640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac2f72f0 0x557fac6cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.146+0000 7f2481863640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac3201b0 0x557fac6d6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.146+0000 7f2481062640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac2f72f0 0x557fac6cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:34642/0 (socket says 172.21.15.2:34642) 2024-05-14T09:39:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.146+0000 7f2481062640 1 -- 172.21.15.2:0/1045720266 learned_addr learned my addr 172.21.15.2:0/1045720266 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f2480861640 1 --2- 172.21.15.2:0/1045720266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f2481863640 1 -- 172.21.15.2:0/1045720266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 msgr2=0x557fac6cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f2481863640 1 --2- 172.21.15.2:0/1045720266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f2481863640 1 -- 172.21.15.2:0/1045720266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac2f72f0 msgr2=0x557fac6cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f2481863640 1 --2- 172.21.15.2:0/1045720266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac2f72f0 0x557fac6cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f2481863640 1 -- 172.21.15.2:0/1045720266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fac404620 con 0x557fac3201b0 2024-05-14T09:39:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f2481062640 1 --2- 172.21.15.2:0/1045720266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac2f72f0 0x557fac6cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f2480861640 1 --2- 172.21.15.2:0/1045720266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f2481863640 1 --2- 172.21.15.2:0/1045720266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac3201b0 0x557fac6d6ec0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f24741013a0 tx=0x7f2474009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809c63c43444721f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f2464ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:39:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f2464ff9640 1 -- 172.21.15.2:0/1045720266 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24740c6d30 con 0x557fac3201b0 2024-05-14T09:39:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f2464ff9640 1 -- 172.21.15.2:0/1045720266 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f24740046a0 con 0x557fac3201b0 2024-05-14T09:39:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f248a179740 1 -- 172.21.15.2:0/1045720266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24780043c0 con 0x557fac3201b0 2024-05-14T09:39:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f2464ff9640 1 -- 172.21.15.2:0/1045720266 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f24740cf220 con 0x557fac3201b0 2024-05-14T09:39:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f248a179740 10 client.16062 fetch_fsmap learned FSMap version 29 2024-05-14T09:39:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f248a179740 1 -- 172.21.15.2:0/1045720266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557fac6cdd40 con 0x557fac3201b0 2024-05-14T09:39:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.147+0000 7f2464ff9640 1 -- 172.21.15.2:0/1045720266 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24780043c0 con 0x557fac3201b0 2024-05-14T09:39:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.148+0000 7f2464ff9640 1 -- 172.21.15.2:0/1045720266 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f24740cb160 con 0x557fac3201b0 2024-05-14T09:39:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.148+0000 7f248a179740 10 client.16062 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:39:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.148+0000 7f248a179740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:39:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.148+0000 7f248a179740 20 client.16062 populate_metadata read hostname 'smithi002' 2024-05-14T09:39:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.148+0000 7f248a179740 1 --2- 172.21.15.2:0/1045720266 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557fac2ded80 0x557fac6dd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.148+0000 7f248a179740 4 client.16062 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:39:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.148+0000 7f248a179740 1 -- 172.21.15.2:0/1045720266 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fac6341e0 con 0x557fac2ded80 2024-05-14T09:39:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.148+0000 7f2481062640 1 --2- 172.21.15.2:0/1045720266 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557fac2ded80 0x557fac6dd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.148+0000 7f2464ff9640 10 client.16062 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.148+0000 7f2481062640 1 --2- 172.21.15.2:0/1045720266 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557fac2ded80 0x557fac6dd650 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=723be545aba49402 server_cookie=bc4dfe18d7950c92 in_seq=0 out_seq=0 2024-05-14T09:39:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.150+0000 7f2464ff9640 1 -- 172.21.15.2:0/1045720266 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557fac6341e0 con 0x557fac2ded80 2024-05-14T09:39:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.150+0000 7f2464ff9640 10 client.16062 handle_command_reply: tid=0 2024-05-14T09:39:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.150+0000 7f248a179740 1 client.16062 shutdown 2024-05-14T09:39:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.150+0000 7f248a179740 1 -- 172.21.15.2:0/1045720266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac3201b0 msgr2=0x557fac6d6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.150+0000 7f248a179740 1 --2- 172.21.15.2:0/1045720266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac3201b0 0x557fac6d6ec0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f24741013a0 tx=0x7f2474009990 comp rx=0 tx=0).stop 2024-05-14T09:39:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.150+0000 7f248a179740 1 -- 172.21.15.2:0/1045720266 shutdown_connections 2024-05-14T09:39:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.150+0000 7f248a179740 1 --2- 172.21.15.2:0/1045720266 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557fac2ded80 0x557fac6dd650 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.150+0000 7f248a179740 1 --2- 172.21.15.2:0/1045720266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac3201b0 0x557fac6d6ec0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.150+0000 7f248a179740 1 --2- 172.21.15.2:0/1045720266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.150+0000 7f2464ff9640 0 client.16062 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.150+0000 7f248a179740 1 --2- 172.21.15.2:0/1045720266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac2f72f0 0x557fac6cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.150+0000 7f248a179740 1 -- 172.21.15.2:0/1045720266 >> 172.21.15.2:0/1045720266 conn(0x557fac314300 msgr2=0x557fac6c0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.151+0000 7f248a179740 1 -- 172.21.15.2:0/1045720266 shutdown_connections 2024-05-14T09:39:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.151+0000 7f248a179740 1 -- 172.21.15.2:0/1045720266 wait complete. 2024-05-14T09:39:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.151+0000 7f248a179740 20 client.16062 trim_cache size 0 max 16384 2024-05-14T09:39:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.166+0000 7f248a179740 1 Processor -- start 2024-05-14T09:39:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.166+0000 7f248a179740 1 -- start start 2024-05-14T09:39:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.166+0000 7f248a179740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac2f72f0 0x557fac6ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.166+0000 7f248a179740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6d6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.166+0000 7f248a179740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac3201b0 0x557fac6ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.166+0000 7f248a179740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f247c1ad2d0 con 0x557fac2f72f0 2024-05-14T09:39:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.166+0000 7f248a179740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f247c1aaba0 con 0x557fac3201b0 2024-05-14T09:39:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.166+0000 7f248a179740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f247c1a89f0 con 0x557fac2ff4d0 2024-05-14T09:39:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.167+0000 7f2481062640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac2f72f0 0x557fac6ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.167+0000 7f2481062640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac2f72f0 0x557fac6ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40782/0 (socket says 172.21.15.2:40782) 2024-05-14T09:39:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.167+0000 7f2481062640 1 -- 172.21.15.2:0/4069624066 learned_addr learned my addr 172.21.15.2:0/4069624066 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.167+0000 7f2481863640 1 --2- 172.21.15.2:0/4069624066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac3201b0 0x557fac6ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.167+0000 7f2480861640 1 --2- 172.21.15.2:0/4069624066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6d6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.167+0000 7f2481062640 1 -- 172.21.15.2:0/4069624066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 msgr2=0x557fac6d6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.167+0000 7f2481062640 1 --2- 172.21.15.2:0/4069624066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6d6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.167+0000 7f2481062640 1 -- 172.21.15.2:0/4069624066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac3201b0 msgr2=0x557fac6ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.167+0000 7f2481062640 1 --2- 172.21.15.2:0/4069624066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac3201b0 0x557fac6ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.167+0000 7f2481062640 1 -- 172.21.15.2:0/4069624066 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fac6ced40 con 0x557fac2f72f0 2024-05-14T09:39:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.167+0000 7f2480861640 1 --2- 172.21.15.2:0/4069624066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6d6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:39:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.167+0000 7f2481062640 1 --2- 172.21.15.2:0/4069624066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac2f72f0 0x557fac6ce2c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2478011d10 tx=0x7f24780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58bda8743407b0cd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.167+0000 7f2466ffd640 1 -- 172.21.15.2:0/4069624066 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247801d040 con 0x557fac2f72f0 2024-05-14T09:39:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.167+0000 7f2466ffd640 1 -- 172.21.15.2:0/4069624066 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24780044f0 con 0x557fac2f72f0 2024-05-14T09:39:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.167+0000 7f2466ffd640 1 -- 172.21.15.2:0/4069624066 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247802c530 con 0x557fac2f72f0 2024-05-14T09:39:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.168+0000 7f248a179740 1 -- 172.21.15.2:0/4069624066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac2f72f0 msgr2=0x557fac6ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.168+0000 7f248a179740 1 --2- 172.21.15.2:0/4069624066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac2f72f0 0x557fac6ce2c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2478011d10 tx=0x7f24780046c0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.168+0000 7f248a179740 1 -- 172.21.15.2:0/4069624066 shutdown_connections 2024-05-14T09:39:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.168+0000 7f248a179740 1 --2- 172.21.15.2:0/4069624066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac3201b0 0x557fac6ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.168+0000 7f248a179740 1 --2- 172.21.15.2:0/4069624066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6d6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.168+0000 7f248a179740 1 --2- 172.21.15.2:0/4069624066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac2f72f0 0x557fac6ce2c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.168+0000 7f248a179740 1 -- 172.21.15.2:0/4069624066 >> 172.21.15.2:0/4069624066 conn(0x557fac314300 msgr2=0x557fac6c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.168+0000 7f248a179740 1 -- 172.21.15.2:0/4069624066 shutdown_connections 2024-05-14T09:39:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.168+0000 7f248a179740 1 -- 172.21.15.2:0/4069624066 wait complete. 2024-05-14T09:39:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.169+0000 7f248a179740 1 -- start start 2024-05-14T09:39:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.169+0000 7f248a179740 1 Processor -- start 2024-05-14T09:39:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.169+0000 7f248a179740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac2f72f0 0x557fac6d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.169+0000 7f248a179740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.169+0000 7f248a179740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac3201b0 0x557fac7439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.169+0000 7f248a179740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f247c1a9060 con 0x557fac3201b0 2024-05-14T09:39:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.169+0000 7f248a179740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f247c1aa4d0 con 0x557fac2f72f0 2024-05-14T09:39:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.169+0000 7f248a179740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f247c1a9730 con 0x557fac2ff4d0 2024-05-14T09:39:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.169+0000 7f248a179740 10 client.0 _ll_register_callbacks cb 0x7f2474009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:39:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.169+0000 7f2481863640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac3201b0 0x557fac7439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.169+0000 7f2481863640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac3201b0 0x557fac7439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40788/0 (socket says 172.21.15.2:40788) 2024-05-14T09:39:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.169+0000 7f2481863640 1 -- 172.21.15.2:0/1101457275 learned_addr learned my addr 172.21.15.2:0/1101457275 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.169+0000 7f2480861640 1 --2- 172.21.15.2:0/1101457275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.169+0000 7f2481062640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac2f72f0 0x557fac6d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.170+0000 7f2480861640 1 -- 172.21.15.2:0/1101457275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac2f72f0 msgr2=0x557fac6d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.170+0000 7f2480861640 1 --2- 172.21.15.2:0/1101457275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac2f72f0 0x557fac6d3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.170+0000 7f2480861640 1 -- 172.21.15.2:0/1101457275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac3201b0 msgr2=0x557fac7439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.170+0000 7f2480861640 1 --2- 172.21.15.2:0/1101457275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac3201b0 0x557fac7439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.170+0000 7f2480861640 1 -- 172.21.15.2:0/1101457275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fac6d7030 con 0x557fac2ff4d0 2024-05-14T09:39:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.170+0000 7f2481863640 1 --2- 172.21.15.2:0/1101457275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac3201b0 0x557fac7439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.170+0000 7f2480861640 1 --2- 172.21.15.2:0/1101457275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6d4150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x557fac6d0a90 tx=0x7f246c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4eecd59601e9d18 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.170+0000 7f2464ff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:39:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.170+0000 7f2464ff9640 1 -- 172.21.15.2:0/1101457275 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c022970 con 0x557fac2ff4d0 2024-05-14T09:39:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.170+0000 7f2464ff9640 1 -- 172.21.15.2:0/1101457275 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f246c01b030 con 0x557fac2ff4d0 2024-05-14T09:39:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.170+0000 7f248a179740 1 -- 172.21.15.2:0/1101457275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24780044f0 con 0x557fac2ff4d0 2024-05-14T09:39:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.170+0000 7f2464ff9640 1 -- 172.21.15.2:0/1101457275 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f246c022b10 con 0x557fac2ff4d0 2024-05-14T09:39:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.170+0000 7f248a179740 10 client.25750 fetch_fsmap learned FSMap version 29 2024-05-14T09:39:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.170+0000 7f248a179740 1 -- 172.21.15.2:0/1101457275 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557fac744ee0 con 0x557fac2ff4d0 2024-05-14T09:39:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.171+0000 7f2464ff9640 1 -- 172.21.15.2:0/1101457275 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24780044f0 con 0x557fac2ff4d0 2024-05-14T09:39:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.171+0000 7f2464ff9640 1 -- 172.21.15.2:0/1101457275 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f246c02cce0 con 0x557fac2ff4d0 2024-05-14T09:39:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.171+0000 7f248a179740 10 client.25750 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:39:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.171+0000 7f248a179740 10 client.25750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:39:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.171+0000 7f248a179740 20 client.25750 populate_metadata read hostname 'smithi002' 2024-05-14T09:39:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.171+0000 7f248a179740 1 --2- 172.21.15.2:0/1101457275 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557fac2b41c0 0x557fac7479b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.171+0000 7f248a179740 4 client.25750 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:39:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.171+0000 7f248a179740 1 -- 172.21.15.2:0/1101457275 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557fac62e050 con 0x557fac2b41c0 2024-05-14T09:39:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.171+0000 7f2481062640 1 --2- 172.21.15.2:0/1101457275 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557fac2b41c0 0x557fac7479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.171+0000 7f2464ff9640 10 client.25750 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.171+0000 7f2481062640 1 --2- 172.21.15.2:0/1101457275 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557fac2b41c0 0x557fac7479b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d5c2ff0a932ea8 server_cookie=19db96f457b91c5a in_seq=0 out_seq=0 2024-05-14T09:39:00.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 288757 10.46 MB/sec execute 26 sec latency 106.219 ms 2024-05-14T09:39:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.275+0000 7f2464ff9640 1 -- 172.21.15.2:0/1101457275 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557fac62e050 con 0x557fac2b41c0 2024-05-14T09:39:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.275+0000 7f2464ff9640 10 client.25750 handle_command_reply: tid=0 2024-05-14T09:39:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.275+0000 7f248a179740 1 client.25750 shutdown 2024-05-14T09:39:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.275+0000 7f248a179740 1 -- 172.21.15.2:0/1101457275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 msgr2=0x557fac6d4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.275+0000 7f248a179740 1 --2- 172.21.15.2:0/1101457275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6d4150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x557fac6d0a90 tx=0x7f246c004730 comp rx=0 tx=0).stop 2024-05-14T09:39:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.275+0000 7f248a179740 1 -- 172.21.15.2:0/1101457275 shutdown_connections 2024-05-14T09:39:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.275+0000 7f248a179740 1 --2- 172.21.15.2:0/1101457275 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557fac2b41c0 0x557fac7479b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.275+0000 7f248a179740 1 --2- 172.21.15.2:0/1101457275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fac3201b0 0x557fac7439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.275+0000 7f248a179740 1 --2- 172.21.15.2:0/1101457275 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557fac2ff4d0 0x557fac6d4150 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.275+0000 7f2464ff9640 0 client.25750 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.275+0000 7f248a179740 1 --2- 172.21.15.2:0/1101457275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557fac2f72f0 0x557fac6d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.275+0000 7f248a179740 1 -- 172.21.15.2:0/1101457275 >> 172.21.15.2:0/1101457275 conn(0x557fac314300 msgr2=0x557fac6c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.275+0000 7f248a179740 1 -- 172.21.15.2:0/1101457275 shutdown_connections 2024-05-14T09:39:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.275+0000 7f248a179740 1 -- 172.21.15.2:0/1101457275 wait complete. 2024-05-14T09:39:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.275+0000 7f248a179740 20 client.25750 trim_cache size 0 max 16384 2024-05-14T09:39:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:39:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:39:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:39:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:39:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.276+0000 7f24832c6640 1 -- 172.21.15.2:0/3028834454 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f24680b2920 msgr2=0x7f24680b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.276+0000 7f24832c6640 1 --2- 172.21.15.2:0/3028834454 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f24680b2920 0x7f24680b4e30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x557fac6c7040 tx=0x7f246c00b6e0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.276+0000 7f24832c6640 1 -- 172.21.15.2:0/3028834454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f247c108a90 msgr2=0x7f247c1a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.276+0000 7f24832c6640 1 --2- 172.21.15.2:0/3028834454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f247c108a90 0x7f247c1a17c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f24740097b0 tx=0x7f2474031840 comp rx=0 tx=0).stop 2024-05-14T09:39:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.276+0000 7f24832c6640 1 -- 172.21.15.2:0/3028834454 shutdown_connections 2024-05-14T09:39:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.276+0000 7f24832c6640 1 --2- 172.21.15.2:0/3028834454 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f24680b2920 0x7f24680b4e30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.276+0000 7f24832c6640 1 --2- 172.21.15.2:0/3028834454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f247c108a90 0x7f247c1a17c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.276+0000 7f24832c6640 1 --2- 172.21.15.2:0/3028834454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f247c103d90 0x7f247c19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.276+0000 7f24832c6640 1 --2- 172.21.15.2:0/3028834454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f247c102b50 0x7f247c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.276+0000 7f24832c6640 1 -- 172.21.15.2:0/3028834454 >> 172.21.15.2:0/3028834454 conn(0x7f247c0fdd40 msgr2=0x7f247c109ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.276+0000 7f24832c6640 1 -- 172.21.15.2:0/3028834454 shutdown_connections 2024-05-14T09:39:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:00.276+0000 7f24832c6640 1 -- 172.21.15.2:0/3028834454 wait complete. 2024-05-14T09:39:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:00 smithi002 ceph-mon[31557]: pgmap v545: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 305 op/s 2024-05-14T09:39:00.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:00 smithi029 ceph-mon[36249]: pgmap v545: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 305 op/s 2024-05-14T09:39:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:00 smithi026 ceph-mon[36056]: pgmap v545: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 305 op/s 2024-05-14T09:39:01.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 291155 10.58 MB/sec execute 27 sec latency 49.882 ms 2024-05-14T09:39:02.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 292686 10.40 MB/sec execute 28 sec latency 22.753 ms 2024-05-14T09:39:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:02 smithi002 ceph-mon[31557]: pgmap v546: 129 pgs: 129 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 274 op/s 2024-05-14T09:39:02.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:02 smithi029 ceph-mon[36249]: pgmap v546: 129 pgs: 129 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 274 op/s 2024-05-14T09:39:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:02 smithi026 ceph-mon[36056]: pgmap v546: 129 pgs: 129 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 274 op/s 2024-05-14T09:39:03.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 294791 10.52 MB/sec execute 29 sec latency 16.974 ms 2024-05-14T09:39:04.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 296768 10.40 MB/sec execute 30 sec latency 28.572 ms 2024-05-14T09:39:04.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:04 smithi002 ceph-mon[31557]: pgmap v547: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 396 op/s 2024-05-14T09:39:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:04 smithi029 ceph-mon[36249]: pgmap v547: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 396 op/s 2024-05-14T09:39:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:04 smithi026 ceph-mon[36056]: pgmap v547: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 396 op/s 2024-05-14T09:39:05.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 298336 10.46 MB/sec execute 31 sec latency 117.924 ms 2024-05-14T09:39:06.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 300310 10.35 MB/sec execute 32 sec latency 15.254 ms 2024-05-14T09:39:06.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:06 smithi002 ceph-mon[31557]: pgmap v548: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 281 op/s 2024-05-14T09:39:06.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:06 smithi029 ceph-mon[36249]: pgmap v548: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 281 op/s 2024-05-14T09:39:06.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:06 smithi026 ceph-mon[36056]: pgmap v548: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 281 op/s 2024-05-14T09:39:07.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 301941 10.40 MB/sec execute 33 sec latency 30.687 ms 2024-05-14T09:39:08.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 303957 10.31 MB/sec execute 34 sec latency 13.283 ms 2024-05-14T09:39:08.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:08 smithi002 ceph-mon[31557]: pgmap v549: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 342 op/s 2024-05-14T09:39:08.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:08 smithi029 ceph-mon[36249]: pgmap v549: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 342 op/s 2024-05-14T09:39:08.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:08 smithi026 ceph-mon[36056]: pgmap v549: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 342 op/s 2024-05-14T09:39:09.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 305460 10.35 MB/sec execute 35 sec latency 83.745 ms 2024-05-14T09:39:10.260 INFO:tasks.workunit.client.0.smithi002.stdout: 1 307455 10.26 MB/sec execute 36 sec latency 21.736 ms 2024-05-14T09:39:10.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:10 smithi002 ceph-mon[31557]: pgmap v550: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 320 op/s 2024-05-14T09:39:10.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:10 smithi029 ceph-mon[36249]: pgmap v550: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 320 op/s 2024-05-14T09:39:10.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:10 smithi026 ceph-mon[36056]: pgmap v550: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 320 op/s 2024-05-14T09:39:11.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 309028 10.31 MB/sec execute 37 sec latency 75.504 ms 2024-05-14T09:39:12.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 310998 10.23 MB/sec execute 38 sec latency 18.612 ms 2024-05-14T09:39:12.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:12 smithi002 ceph-mon[31557]: pgmap v551: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 318 op/s 2024-05-14T09:39:12.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:12 smithi029 ceph-mon[36249]: pgmap v551: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 318 op/s 2024-05-14T09:39:12.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:12 smithi026 ceph-mon[36056]: pgmap v551: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 318 op/s 2024-05-14T09:39:13.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 312578 10.27 MB/sec execute 39 sec latency 18.696 ms 2024-05-14T09:39:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.375+0000 7fc5e50a8640 1 -- 172.21.15.2:0/3906943536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5e0108a90 msgr2=0x7fc5e010aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.375+0000 7fc5e50a8640 1 --2- 172.21.15.2:0/3906943536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5e0108a90 0x7fc5e010aeb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009ca0 tx=0x7fc5d802a8c0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.376+0000 7fc5e50a8640 1 -- 172.21.15.2:0/3906943536 shutdown_connections 2024-05-14T09:39:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.376+0000 7fc5e50a8640 1 --2- 172.21.15.2:0/3906943536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5e0108a90 0x7fc5e010aeb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.376+0000 7fc5e50a8640 1 --2- 172.21.15.2:0/3906943536 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5e0103d90 0x7fc5e01041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.376+0000 7fc5e50a8640 1 --2- 172.21.15.2:0/3906943536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5e0102b50 0x7fc5e0103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.376+0000 7fc5e50a8640 1 -- 172.21.15.2:0/3906943536 >> 172.21.15.2:0/3906943536 conn(0x7fc5e00fdd60 msgr2=0x7fc5e01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.376+0000 7fc5e50a8640 1 -- 172.21.15.2:0/3906943536 shutdown_connections 2024-05-14T09:39:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.376+0000 7fc5e50a8640 1 -- 172.21.15.2:0/3906943536 wait complete. 2024-05-14T09:39:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.376+0000 7fc5e50a8640 1 Processor -- start 2024-05-14T09:39:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5e50a8640 1 -- start start 2024-05-14T09:39:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5e50a8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5e0102b50 0x7fc5e0199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5e50a8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5e0103d90 0x7fc5e01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5e50a8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5e0108a90 0x7fc5e01a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5e50a8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e010e6d0 con 0x7fc5e0103d90 2024-05-14T09:39:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5e50a8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e010e550 con 0x7fc5e0102b50 2024-05-14T09:39:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5e50a8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e010e850 con 0x7fc5e0108a90 2024-05-14T09:39:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5de59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5e0103d90 0x7fc5e01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5ded9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5e0102b50 0x7fc5e0199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5de59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5e0103d90 0x7fc5e01996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39386/0 (socket says 172.21.15.2:39386) 2024-05-14T09:39:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5de59c640 1 -- 172.21.15.2:0/584630565 learned_addr learned my addr 172.21.15.2:0/584630565 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5df59e640 1 --2- 172.21.15.2:0/584630565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5e0108a90 0x7fc5e01a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5de59c640 1 -- 172.21.15.2:0/584630565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5e0108a90 msgr2=0x7fc5e01a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5de59c640 1 --2- 172.21.15.2:0/584630565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5e0108a90 0x7fc5e01a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5de59c640 1 -- 172.21.15.2:0/584630565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5e0102b50 msgr2=0x7fc5e0199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5de59c640 1 --2- 172.21.15.2:0/584630565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5e0102b50 0x7fc5e0199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5de59c640 1 -- 172.21.15.2:0/584630565 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e01a0cb0 con 0x7fc5e0103d90 2024-05-14T09:39:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.377+0000 7fc5ded9d640 1 --2- 172.21.15.2:0/584630565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5e0102b50 0x7fc5e0199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.378+0000 7fc5de59c640 1 --2- 172.21.15.2:0/584630565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5e0103d90 0x7fc5e01996b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400e9f0 tx=0x7fc5d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.378+0000 7fc5c7fff640 1 -- 172.21.15.2:0/584630565 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d400cd80 con 0x7fc5e0103d90 2024-05-14T09:39:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.378+0000 7fc5c7fff640 1 -- 172.21.15.2:0/584630565 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5d40054d0 con 0x7fc5e0103d90 2024-05-14T09:39:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.378+0000 7fc5c7fff640 1 -- 172.21.15.2:0/584630565 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4010690 con 0x7fc5e0103d90 2024-05-14T09:39:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.378+0000 7fc5e50a8640 1 -- 172.21.15.2:0/584630565 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e01a0fa0 con 0x7fc5e0103d90 2024-05-14T09:39:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.378+0000 7fc5e50a8640 1 -- 172.21.15.2:0/584630565 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e01a13b0 con 0x7fc5e0103d90 2024-05-14T09:39:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.380+0000 7fc5ec038740 1 Processor -- start 2024-05-14T09:39:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.380+0000 7fc5ec038740 1 -- start start 2024-05-14T09:39:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5ec038740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8811b0 0x55d23ec22a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5ec038740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec22fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5ec038740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8604d0 0x55d23ec29ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5ec038740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d23eb875d0 con 0x55d23e8582f0 2024-05-14T09:39:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5ec038740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d23eb40890 con 0x55d23e8604d0 2024-05-14T09:39:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5ec038740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d23eb83590 con 0x55d23e8811b0 2024-05-14T09:39:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5ded9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec22fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5ded9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec22fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39400/0 (socket says 172.21.15.2:39400) 2024-05-14T09:39:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5ded9d640 1 -- 172.21.15.2:0/4048171905 learned_addr learned my addr 172.21.15.2:0/4048171905 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5df59e640 1 --2- 172.21.15.2:0/4048171905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8811b0 0x55d23ec22a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5c7fff640 1 -- 172.21.15.2:0/584630565 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc5d4004e90 con 0x7fc5e0103d90 2024-05-14T09:39:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5ded9d640 1 -- 172.21.15.2:0/4048171905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8811b0 msgr2=0x55d23ec22a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5ded9d640 1 --2- 172.21.15.2:0/4048171905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8811b0 0x55d23ec22a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5de59c640 1 --2- 172.21.15.2:0/4048171905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8604d0 0x55d23ec29ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5ded9d640 1 -- 172.21.15.2:0/4048171905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8604d0 msgr2=0x55d23ec29ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5ded9d640 1 --2- 172.21.15.2:0/4048171905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8604d0 0x55d23ec29ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5ded9d640 1 -- 172.21.15.2:0/4048171905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d23e899210 con 0x55d23e8582f0 2024-05-14T09:39:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5df59e640 1 --2- 172.21.15.2:0/4048171905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8811b0 0x55d23ec22a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5de59c640 1 --2- 172.21.15.2:0/4048171905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8604d0 0x55d23ec29ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:39:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.381+0000 7fc5ded9d640 1 --2- 172.21.15.2:0/4048171905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec22fe0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00114f0 tx=0x7fc5d00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd5ac3e42eae57e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.382+0000 7fc5c57fa640 1 -- 172.21.15.2:0/4048171905 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d001e070 con 0x55d23e8582f0 2024-05-14T09:39:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.382+0000 7fc5c57fa640 1 -- 172.21.15.2:0/4048171905 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5d000fad0 con 0x55d23e8582f0 2024-05-14T09:39:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.382+0000 7fc5c57fa640 1 -- 172.21.15.2:0/4048171905 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0019b70 con 0x55d23e8582f0 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.382+0000 7fc5c7fff640 1 --2- 172.21.15.2:0/584630565 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc5c80b28a0 0x7fc5c80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.382+0000 7fc5df59e640 1 --2- 172.21.15.2:0/584630565 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc5c80b28a0 0x7fc5c80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.382+0000 7fc5c7fff640 1 -- 172.21.15.2:0/584630565 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fc5d40e29d0 con 0x7fc5e0103d90 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.383+0000 7fc5ec038740 1 -- 172.21.15.2:0/4048171905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 msgr2=0x55d23ec22fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.383+0000 7fc5ec038740 1 --2- 172.21.15.2:0/4048171905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec22fe0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00114f0 tx=0x7fc5d00119b0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.383+0000 7fc5df59e640 1 --2- 172.21.15.2:0/584630565 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc5c80b28a0 0x7fc5c80b4db0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55d23ec23c30 tx=0x7fc5d8031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.383+0000 7fc5ec038740 1 -- 172.21.15.2:0/4048171905 shutdown_connections 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.383+0000 7fc5ec038740 1 --2- 172.21.15.2:0/4048171905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8604d0 0x55d23ec29ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.383+0000 7fc5ec038740 1 --2- 172.21.15.2:0/4048171905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec22fe0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.383+0000 7fc5ec038740 1 --2- 172.21.15.2:0/4048171905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8811b0 0x55d23ec22a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.383+0000 7fc5ec038740 1 -- 172.21.15.2:0/4048171905 >> 172.21.15.2:0/4048171905 conn(0x55d23e875300 msgr2=0x55d23ec1e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.383+0000 7fc5ec038740 1 -- 172.21.15.2:0/4048171905 shutdown_connections 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.383+0000 7fc5ec038740 1 -- 172.21.15.2:0/4048171905 wait complete. 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.383+0000 7fc5ec038740 1 -- start start 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.383+0000 7fc5ec038740 1 Processor -- start 2024-05-14T09:39:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5ec038740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec2e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5ec038740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8604d0 0x55d23ec2edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5ec038740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8811b0 0x55d23ec35d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5ec038740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d23ec2cb40 con 0x55d23e8582f0 2024-05-14T09:39:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5ec038740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d23eb44370 con 0x55d23e8604d0 2024-05-14T09:39:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5ec038740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d23ec2ccc0 con 0x55d23e8811b0 2024-05-14T09:39:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5ec038740 10 client.0 _ll_register_callbacks cb 0x55d23e898a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:39:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5de59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8604d0 0x55d23ec2edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5ded9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec2e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5de59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8604d0 0x55d23ec2edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:39850/0 (socket says 172.21.15.2:39850) 2024-05-14T09:39:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5de59c640 1 -- 172.21.15.2:0/3197004619 learned_addr learned my addr 172.21.15.2:0/3197004619 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5df59e640 1 --2- 172.21.15.2:0/3197004619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8811b0 0x55d23ec35d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5de59c640 1 -- 172.21.15.2:0/3197004619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8811b0 msgr2=0x55d23ec35d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5de59c640 1 --2- 172.21.15.2:0/3197004619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8811b0 0x55d23ec35d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5de59c640 1 -- 172.21.15.2:0/3197004619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 msgr2=0x55d23ec2e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5de59c640 1 --2- 172.21.15.2:0/3197004619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec2e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5de59c640 1 -- 172.21.15.2:0/3197004619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d23e965620 con 0x55d23e8604d0 2024-05-14T09:39:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5ded9d640 1 --2- 172.21.15.2:0/3197004619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec2e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5de59c640 1 --2- 172.21.15.2:0/3197004619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8604d0 0x55d23ec2edf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400c6a0 tx=0x7fc5d40a95e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bc55772e2cd7b28 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:39:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.384+0000 7fc5c2ff5640 1 -- 172.21.15.2:0/3197004619 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d40c3060 con 0x55d23e8604d0 2024-05-14T09:39:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.385+0000 7fc5ec038740 1 -- 172.21.15.2:0/3197004619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5d000fad0 con 0x55d23e8604d0 2024-05-14T09:39:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.385+0000 7fc5c2ff5640 1 -- 172.21.15.2:0/3197004619 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fc5d40b25a0 con 0x55d23e8604d0 2024-05-14T09:39:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.385+0000 7fc5c2ff5640 1 -- 172.21.15.2:0/3197004619 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc5d40a96e0 con 0x55d23e8604d0 2024-05-14T09:39:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.385+0000 7fc5ec038740 10 client.25772 fetch_fsmap learned FSMap version 29 2024-05-14T09:39:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.385+0000 7fc5ec038740 1 -- 172.21.15.2:0/3197004619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d23ec38bf0 con 0x55d23e8604d0 2024-05-14T09:39:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.385+0000 7fc5c2ff5640 1 -- 172.21.15.2:0/3197004619 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5d000fad0 con 0x55d23e8604d0 2024-05-14T09:39:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.385+0000 7fc5c2ff5640 1 -- 172.21.15.2:0/3197004619 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc5d40a5040 con 0x55d23e8604d0 2024-05-14T09:39:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.385+0000 7fc5ec038740 10 client.25772 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:39:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.385+0000 7fc5ec038740 10 client.25772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:39:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.385+0000 7fc5ec038740 20 client.25772 populate_metadata read hostname 'smithi002' 2024-05-14T09:39:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.385+0000 7fc5ec038740 1 --2- 172.21.15.2:0/3197004619 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d23e83fd80 0x55d23ec3c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.385+0000 7fc5ec038740 4 client.25772 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:39:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.385+0000 7fc5ec038740 1 -- 172.21.15.2:0/3197004619 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d23eb951e0 con 0x55d23e83fd80 2024-05-14T09:39:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.385+0000 7fc5ded9d640 1 --2- 172.21.15.2:0/3197004619 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d23e83fd80 0x55d23ec3c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.386+0000 7fc5c2ff5640 10 client.25772 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.386+0000 7fc5ded9d640 1 --2- 172.21.15.2:0/3197004619 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d23e83fd80 0x55d23ec3c380 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd34a69fe2b9723 server_cookie=913c5dbcdf6d3027 in_seq=0 out_seq=0 2024-05-14T09:39:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.387+0000 7fc5c2ff5640 1 -- 172.21.15.2:0/3197004619 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d23eb951e0 con 0x55d23e83fd80 2024-05-14T09:39:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.387+0000 7fc5c2ff5640 10 client.25772 handle_command_reply: tid=0 2024-05-14T09:39:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.387+0000 7fc5ec038740 1 client.25772 shutdown 2024-05-14T09:39:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.387+0000 7fc5ec038740 1 -- 172.21.15.2:0/3197004619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8604d0 msgr2=0x55d23ec2edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.387+0000 7fc5ec038740 1 --2- 172.21.15.2:0/3197004619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8604d0 0x55d23ec2edf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400c6a0 tx=0x7fc5d40a95e0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.387+0000 7fc5ec038740 1 -- 172.21.15.2:0/3197004619 shutdown_connections 2024-05-14T09:39:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.387+0000 7fc5ec038740 1 --2- 172.21.15.2:0/3197004619 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d23e83fd80 0x55d23ec3c380 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.387+0000 7fc5ec038740 1 --2- 172.21.15.2:0/3197004619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8811b0 0x55d23ec35d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.387+0000 7fc5ec038740 1 --2- 172.21.15.2:0/3197004619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8604d0 0x55d23ec2edf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.387+0000 7fc5ec038740 1 --2- 172.21.15.2:0/3197004619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec2e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.387+0000 7fc5c2ff5640 0 client.25772 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.387+0000 7fc5ec038740 1 -- 172.21.15.2:0/3197004619 >> 172.21.15.2:0/3197004619 conn(0x55d23e875300 msgr2=0x55d23ec21300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.387+0000 7fc5ec038740 1 -- 172.21.15.2:0/3197004619 shutdown_connections 2024-05-14T09:39:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.387+0000 7fc5ec038740 1 -- 172.21.15.2:0/3197004619 wait complete. 2024-05-14T09:39:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.387+0000 7fc5ec038740 20 client.25772 trim_cache size 0 max 16384 2024-05-14T09:39:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ec038740 1 Processor -- start 2024-05-14T09:39:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ec038740 1 -- start start 2024-05-14T09:39:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ec038740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec25e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ec038740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8604d0 0x55d23ec35c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ec038740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8811b0 0x55d23ec361b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ec038740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01ad340 con 0x55d23e8582f0 2024-05-14T09:39:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ec038740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01aac10 con 0x55d23e8811b0 2024-05-14T09:39:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ec038740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01a8a60 con 0x55d23e8604d0 2024-05-14T09:39:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ded9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec25e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ded9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec25e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39418/0 (socket says 172.21.15.2:39418) 2024-05-14T09:39:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ded9d640 1 -- 172.21.15.2:0/4148269429 learned_addr learned my addr 172.21.15.2:0/4148269429 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5de59c640 1 --2- 172.21.15.2:0/4148269429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8604d0 0x55d23ec35c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5df59e640 1 --2- 172.21.15.2:0/4148269429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8811b0 0x55d23ec361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ded9d640 1 -- 172.21.15.2:0/4148269429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8604d0 msgr2=0x55d23ec35c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ded9d640 1 --2- 172.21.15.2:0/4148269429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8604d0 0x55d23ec35c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ded9d640 1 -- 172.21.15.2:0/4148269429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8811b0 msgr2=0x55d23ec361b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ded9d640 1 --2- 172.21.15.2:0/4148269429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8811b0 0x55d23ec361b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.403+0000 7fc5ded9d640 1 -- 172.21.15.2:0/4148269429 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d23ec23c60 con 0x55d23e8582f0 2024-05-14T09:39:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.404+0000 7fc5ded9d640 1 --2- 172.21.15.2:0/4148269429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec25e80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0011eb0 tx=0x7fc5d0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a5ea21eac67f9d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.404+0000 7fc5c4ff9640 1 -- 172.21.15.2:0/4148269429 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d001e040 con 0x55d23e8582f0 2024-05-14T09:39:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.404+0000 7fc5c4ff9640 1 -- 172.21.15.2:0/4148269429 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5d0005550 con 0x55d23e8582f0 2024-05-14T09:39:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.404+0000 7fc5c4ff9640 1 -- 172.21.15.2:0/4148269429 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0003850 con 0x55d23e8582f0 2024-05-14T09:39:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.405+0000 7fc5ec038740 1 -- 172.21.15.2:0/4148269429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 msgr2=0x55d23ec25e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.405+0000 7fc5ec038740 1 --2- 172.21.15.2:0/4148269429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec25e80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0011eb0 tx=0x7fc5d0005dc0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.405+0000 7fc5ec038740 1 -- 172.21.15.2:0/4148269429 shutdown_connections 2024-05-14T09:39:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.405+0000 7fc5ec038740 1 --2- 172.21.15.2:0/4148269429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8811b0 0x55d23ec361b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.405+0000 7fc5ec038740 1 --2- 172.21.15.2:0/4148269429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8604d0 0x55d23ec35c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.405+0000 7fc5ec038740 1 --2- 172.21.15.2:0/4148269429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8582f0 0x55d23ec25e80 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.405+0000 7fc5ec038740 1 -- 172.21.15.2:0/4148269429 >> 172.21.15.2:0/4148269429 conn(0x55d23e875300 msgr2=0x55d23ec26a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.405+0000 7fc5ec038740 1 -- 172.21.15.2:0/4148269429 shutdown_connections 2024-05-14T09:39:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.405+0000 7fc5ec038740 1 -- 172.21.15.2:0/4148269429 wait complete. 2024-05-14T09:39:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.405+0000 7fc5ec038740 1 -- start start 2024-05-14T09:39:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.405+0000 7fc5ec038740 1 Processor -- start 2024-05-14T09:39:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.405+0000 7fc5ec038740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8582f0 0x55d23ec333c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.405+0000 7fc5ec038740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8604d0 0x55d23ec33900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5ec038740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8811b0 0x55d23eca2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5ec038740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01a90d0 con 0x55d23e8604d0 2024-05-14T09:39:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5ec038740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01aa540 con 0x55d23e8811b0 2024-05-14T09:39:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5ec038740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01a97a0 con 0x55d23e8582f0 2024-05-14T09:39:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5ec038740 10 client.0 _ll_register_callbacks cb 0x7fc5d400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5de59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8604d0 0x55d23ec33900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5ded9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8582f0 0x55d23ec333c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5ded9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8582f0 0x55d23ec333c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:43030/0 (socket says 172.21.15.2:43030) 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5ded9d640 1 -- 172.21.15.2:0/1004053311 learned_addr learned my addr 172.21.15.2:0/1004053311 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5de59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8604d0 0x55d23ec33900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39422/0 (socket says 172.21.15.2:39422) 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5df59e640 1 --2- 172.21.15.2:0/1004053311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8811b0 0x55d23eca2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5de59c640 1 -- 172.21.15.2:0/1004053311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8582f0 msgr2=0x55d23ec333c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5de59c640 1 --2- 172.21.15.2:0/1004053311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8582f0 0x55d23ec333c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5de59c640 1 -- 172.21.15.2:0/1004053311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8811b0 msgr2=0x55d23eca2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5de59c640 1 --2- 172.21.15.2:0/1004053311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8811b0 0x55d23eca2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5de59c640 1 -- 172.21.15.2:0/1004053311 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d23eca2c60 con 0x55d23e8604d0 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5de59c640 1 --2- 172.21.15.2:0/1004053311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8604d0 0x55d23ec33900 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40e0100 tx=0x7fc5d4009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3224887730650e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5c2ff5640 1 -- 172.21.15.2:0/1004053311 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d40b7060 con 0x55d23e8604d0 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5c2ff5640 1 -- 172.21.15.2:0/1004053311 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fc5d40a9040 con 0x55d23e8604d0 2024-05-14T09:39:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.406+0000 7fc5ec038740 1 -- 172.21.15.2:0/1004053311 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5d0005550 con 0x55d23e8604d0 2024-05-14T09:39:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.407+0000 7fc5c2ff5640 1 -- 172.21.15.2:0/1004053311 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc5d40be050 con 0x55d23e8604d0 2024-05-14T09:39:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.407+0000 7fc5ec038740 10 client.16086 fetch_fsmap learned FSMap version 29 2024-05-14T09:39:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.407+0000 7fc5ec038740 1 -- 172.21.15.2:0/1004053311 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d23eca3dc0 con 0x55d23e8604d0 2024-05-14T09:39:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.407+0000 7fc5c2ff5640 1 -- 172.21.15.2:0/1004053311 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5d0005550 con 0x55d23e8604d0 2024-05-14T09:39:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.407+0000 7fc5c2ff5640 1 -- 172.21.15.2:0/1004053311 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc5d40a5040 con 0x55d23e8604d0 2024-05-14T09:39:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.407+0000 7fc5ec038740 10 client.16086 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T09:39:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.407+0000 7fc5ec038740 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:39:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.407+0000 7fc5ec038740 20 client.16086 populate_metadata read hostname 'smithi002' 2024-05-14T09:39:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.407+0000 7fc5ec038740 1 --2- 172.21.15.2:0/1004053311 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d23e8151c0 0x55d23eca66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.407+0000 7fc5ec038740 4 client.16086 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:39:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.407+0000 7fc5ec038740 1 -- 172.21.15.2:0/1004053311 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d23eb8f050 con 0x55d23e8151c0 2024-05-14T09:39:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.407+0000 7fc5ded9d640 1 --2- 172.21.15.2:0/1004053311 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d23e8151c0 0x55d23eca66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.407+0000 7fc5ded9d640 1 --2- 172.21.15.2:0/1004053311 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d23e8151c0 0x55d23eca66a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ad4a462119814b0 server_cookie=8d81fce66bff6ad3 in_seq=0 out_seq=0 2024-05-14T09:39:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.408+0000 7fc5c2ff5640 10 client.16086 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.494+0000 7fc5c2ff5640 1 -- 172.21.15.2:0/1004053311 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d23eb8f050 con 0x55d23e8151c0 2024-05-14T09:39:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.494+0000 7fc5c2ff5640 10 client.16086 handle_command_reply: tid=0 2024-05-14T09:39:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.494+0000 7fc5ec038740 1 client.16086 shutdown 2024-05-14T09:39:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.494+0000 7fc5ec038740 1 -- 172.21.15.2:0/1004053311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8604d0 msgr2=0x55d23ec33900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.494+0000 7fc5ec038740 1 --2- 172.21.15.2:0/1004053311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8604d0 0x55d23ec33900 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40e0100 tx=0x7fc5d4009720 comp rx=0 tx=0).stop 2024-05-14T09:39:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.494+0000 7fc5ec038740 1 -- 172.21.15.2:0/1004053311 shutdown_connections 2024-05-14T09:39:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.494+0000 7fc5ec038740 1 --2- 172.21.15.2:0/1004053311 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d23e8151c0 0x55d23eca66a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.494+0000 7fc5ec038740 1 --2- 172.21.15.2:0/1004053311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d23e8811b0 0x55d23eca2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.494+0000 7fc5ec038740 1 --2- 172.21.15.2:0/1004053311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d23e8604d0 0x55d23ec33900 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.494+0000 7fc5c2ff5640 0 client.16086 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.494+0000 7fc5ec038740 1 --2- 172.21.15.2:0/1004053311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d23e8582f0 0x55d23ec333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.494+0000 7fc5ec038740 1 -- 172.21.15.2:0/1004053311 >> 172.21.15.2:0/1004053311 conn(0x55d23e875300 msgr2=0x55d23ec26a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.494+0000 7fc5ec038740 1 -- 172.21.15.2:0/1004053311 shutdown_connections 2024-05-14T09:39:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.494+0000 7fc5ec038740 1 -- 172.21.15.2:0/1004053311 wait complete. 2024-05-14T09:39:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.494+0000 7fc5ec038740 20 client.16086 trim_cache size 0 max 16384 2024-05-14T09:39:13.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:39:13.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:39:13.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:39:13.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:39:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.495+0000 7fc5e50a8640 1 -- 172.21.15.2:0/584630565 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc5c80b28a0 msgr2=0x7fc5c80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.495+0000 7fc5e50a8640 1 --2- 172.21.15.2:0/584630565 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc5c80b28a0 0x7fc5c80b4db0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55d23ec23c30 tx=0x7fc5d8031af0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.495+0000 7fc5e50a8640 1 -- 172.21.15.2:0/584630565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5e0103d90 msgr2=0x7fc5e01996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.495+0000 7fc5e50a8640 1 --2- 172.21.15.2:0/584630565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5e0103d90 0x7fc5e01996b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400e9f0 tx=0x7fc5d400edf0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.495+0000 7fc5e50a8640 1 -- 172.21.15.2:0/584630565 shutdown_connections 2024-05-14T09:39:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.495+0000 7fc5e50a8640 1 --2- 172.21.15.2:0/584630565 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc5c80b28a0 0x7fc5c80b4db0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.495+0000 7fc5e50a8640 1 --2- 172.21.15.2:0/584630565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc5e0108a90 0x7fc5e01a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.495+0000 7fc5e50a8640 1 --2- 172.21.15.2:0/584630565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5e0103d90 0x7fc5e01996b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.495+0000 7fc5e50a8640 1 --2- 172.21.15.2:0/584630565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5e0102b50 0x7fc5e0199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.495+0000 7fc5e50a8640 1 -- 172.21.15.2:0/584630565 >> 172.21.15.2:0/584630565 conn(0x7fc5e00fdd60 msgr2=0x7fc5e0104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.495+0000 7fc5e50a8640 1 -- 172.21.15.2:0/584630565 shutdown_connections 2024-05-14T09:39:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:13.495+0000 7fc5e50a8640 1 -- 172.21.15.2:0/584630565 wait complete. 2024-05-14T09:39:14.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 314296 10.16 MB/sec execute 40 sec latency 69.863 ms 2024-05-14T09:39:14.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:14 smithi002 ceph-mon[31557]: pgmap v552: 129 pgs: 129 active+clean; 74 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 375 op/s 2024-05-14T09:39:14.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:14 smithi029 ceph-mon[36249]: pgmap v552: 129 pgs: 129 active+clean; 74 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 375 op/s 2024-05-14T09:39:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:14 smithi026 ceph-mon[36056]: pgmap v552: 129 pgs: 129 active+clean; 74 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 375 op/s 2024-05-14T09:39:15.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 316115 10.24 MB/sec execute 41 sec latency 29.199 ms 2024-05-14T09:39:16.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 318159 10.17 MB/sec execute 42 sec latency 15.471 ms 2024-05-14T09:39:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:16 smithi002 ceph-mon[31557]: pgmap v553: 129 pgs: 129 active+clean; 74 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 241 op/s 2024-05-14T09:39:16.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:16 smithi029 ceph-mon[36249]: pgmap v553: 129 pgs: 129 active+clean; 74 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 241 op/s 2024-05-14T09:39:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:16 smithi026 ceph-mon[36056]: pgmap v553: 129 pgs: 129 active+clean; 74 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 241 op/s 2024-05-14T09:39:17.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 319807 10.21 MB/sec execute 43 sec latency 15.590 ms 2024-05-14T09:39:18.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 322166 10.26 MB/sec execute 44 sec latency 12.538 ms 2024-05-14T09:39:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:18 smithi002 ceph-mon[31557]: pgmap v554: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-05-14T09:39:18.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:18 smithi029 ceph-mon[36249]: pgmap v554: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-05-14T09:39:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:18 smithi026 ceph-mon[36056]: pgmap v554: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-05-14T09:39:19.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 323387 10.18 MB/sec execute 45 sec latency 44.302 ms 2024-05-14T09:39:20.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 325767 10.23 MB/sec execute 46 sec latency 21.805 ms 2024-05-14T09:39:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:20 smithi026 ceph-mon[36056]: pgmap v555: 129 pgs: 129 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 334 op/s 2024-05-14T09:39:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:20 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:39:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:20 smithi002 ceph-mon[31557]: pgmap v555: 129 pgs: 129 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 334 op/s 2024-05-14T09:39:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:20 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:39:20.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:20 smithi029 ceph-mon[36249]: pgmap v555: 129 pgs: 129 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 334 op/s 2024-05-14T09:39:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:20 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:39:21.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 327209 10.17 MB/sec execute 47 sec latency 15.943 ms 2024-05-14T09:39:22.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 329722 10.24 MB/sec execute 48 sec latency 14.720 ms 2024-05-14T09:39:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:22 smithi002 ceph-mon[31557]: pgmap v556: 129 pgs: 129 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 276 op/s 2024-05-14T09:39:22.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:22 smithi029 ceph-mon[36249]: pgmap v556: 129 pgs: 129 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 276 op/s 2024-05-14T09:39:22.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:22 smithi026 ceph-mon[36056]: pgmap v556: 129 pgs: 129 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 276 op/s 2024-05-14T09:39:23.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 330944 10.17 MB/sec execute 49 sec latency 14.925 ms 2024-05-14T09:39:24.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 333437 10.27 MB/sec execute 50 sec latency 16.626 ms 2024-05-14T09:39:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:24 smithi002 ceph-mon[31557]: pgmap v557: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 358 op/s 2024-05-14T09:39:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:24 smithi029 ceph-mon[36249]: pgmap v557: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 358 op/s 2024-05-14T09:39:24.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:24 smithi026 ceph-mon[36056]: pgmap v557: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 358 op/s 2024-05-14T09:39:25.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 334209 10.11 MB/sec execute 51 sec latency 142.663 ms 2024-05-14T09:39:26.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 336980 10.25 MB/sec execute 52 sec latency 18.202 ms 2024-05-14T09:39:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.600+0000 7ff65a6af640 1 -- 172.21.15.2:0/2016955018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff654103d90 msgr2=0x7ff6541041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.600+0000 7ff65a6af640 1 --2- 172.21.15.2:0/2016955018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff654103d90 0x7ff6541041a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff648009900 tx=0x7ff64802a410 comp rx=0 tx=0).stop 2024-05-14T09:39:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.600+0000 7ff65a6af640 1 -- 172.21.15.2:0/2016955018 shutdown_connections 2024-05-14T09:39:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.600+0000 7ff65a6af640 1 --2- 172.21.15.2:0/2016955018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff654108a90 0x7ff65410aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.600+0000 7ff65a6af640 1 --2- 172.21.15.2:0/2016955018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff654103d90 0x7ff6541041a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.600+0000 7ff65a6af640 1 --2- 172.21.15.2:0/2016955018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff654102b50 0x7ff654103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.600+0000 7ff65a6af640 1 -- 172.21.15.2:0/2016955018 >> 172.21.15.2:0/2016955018 conn(0x7ff6540fdd40 msgr2=0x7ff6541001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.600+0000 7ff65a6af640 1 -- 172.21.15.2:0/2016955018 shutdown_connections 2024-05-14T09:39:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.600+0000 7ff65a6af640 1 -- 172.21.15.2:0/2016955018 wait complete. 2024-05-14T09:39:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.601+0000 7ff65a6af640 1 Processor -- start 2024-05-14T09:39:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.601+0000 7ff65a6af640 1 -- start start 2024-05-14T09:39:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.601+0000 7ff65a6af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff654102b50 0x7ff654199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.601+0000 7ff65a6af640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff654103d90 0x7ff6541996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.601+0000 7ff65a6af640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff654108a90 0x7ff6541a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.601+0000 7ff65a6af640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff65410e650 con 0x7ff654103d90 2024-05-14T09:39:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.601+0000 7ff65a6af640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff65410e4d0 con 0x7ff654102b50 2024-05-14T09:39:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.601+0000 7ff65a6af640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff65410e7d0 con 0x7ff654108a90 2024-05-14T09:39:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.601+0000 7ff6537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff654103d90 0x7ff6541996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.601+0000 7ff6537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff654103d90 0x7ff6541996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43324/0 (socket says 172.21.15.2:43324) 2024-05-14T09:39:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.601+0000 7ff6537fe640 1 -- 172.21.15.2:0/1396518349 learned_addr learned my addr 172.21.15.2:0/1396518349 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.601+0000 7ff658c4c640 1 --2- 172.21.15.2:0/1396518349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff654108a90 0x7ff6541a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.602+0000 7ff653fff640 1 --2- 172.21.15.2:0/1396518349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff654102b50 0x7ff654199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.602+0000 7ff6537fe640 1 -- 172.21.15.2:0/1396518349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff654108a90 msgr2=0x7ff6541a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.602+0000 7ff6537fe640 1 --2- 172.21.15.2:0/1396518349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff654108a90 0x7ff6541a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.602+0000 7ff6537fe640 1 -- 172.21.15.2:0/1396518349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff654102b50 msgr2=0x7ff654199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.602+0000 7ff6537fe640 1 --2- 172.21.15.2:0/1396518349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff654102b50 0x7ff654199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.602+0000 7ff6537fe640 1 -- 172.21.15.2:0/1396518349 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6541a0cd0 con 0x7ff654103d90 2024-05-14T09:39:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.602+0000 7ff6537fe640 1 --2- 172.21.15.2:0/1396518349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff654103d90 0x7ff6541996d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff648006010 tx=0x7ff648031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.602+0000 7ff6517fa640 1 -- 172.21.15.2:0/1396518349 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64803f030 con 0x7ff654103d90 2024-05-14T09:39:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.602+0000 7ff6517fa640 1 -- 172.21.15.2:0/1396518349 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff648003aa0 con 0x7ff654103d90 2024-05-14T09:39:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.602+0000 7ff65a6af640 1 -- 172.21.15.2:0/1396518349 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6541a0f60 con 0x7ff654103d90 2024-05-14T09:39:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.602+0000 7ff6517fa640 1 -- 172.21.15.2:0/1396518349 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648035070 con 0x7ff654103d90 2024-05-14T09:39:26.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.602+0000 7ff65a6af640 1 -- 172.21.15.2:0/1396518349 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6541a1370 con 0x7ff654103d90 2024-05-14T09:39:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.605+0000 7ff661564740 1 Processor -- start 2024-05-14T09:39:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.605+0000 7ff661564740 1 -- start start 2024-05-14T09:39:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff661564740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87cc1b0 0x5557b8b6da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff661564740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87a32f0 0x5557b8b6dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff661564740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87ab4d0 0x5557b8b74ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff661564740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5557b8ad25d0 con 0x5557b87cc1b0 2024-05-14T09:39:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff661564740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5557b8a8b890 con 0x5557b87ab4d0 2024-05-14T09:39:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff661564740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5557b8ace590 con 0x5557b87a32f0 2024-05-14T09:39:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff658c4c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87cc1b0 0x5557b8b6da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff653fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87a32f0 0x5557b8b6dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff658c4c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87cc1b0 0x5557b8b6da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43336/0 (socket says 172.21.15.2:43336) 2024-05-14T09:39:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff658c4c640 1 -- 172.21.15.2:0/539110779 learned_addr learned my addr 172.21.15.2:0/539110779 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff653fff640 1 -- 172.21.15.2:0/539110779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87ab4d0 msgr2=0x5557b8b74ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:39:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff6517fa640 1 -- 172.21.15.2:0/1396518349 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff648042030 con 0x7ff654103d90 2024-05-14T09:39:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff653fff640 1 --2- 172.21.15.2:0/539110779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87ab4d0 0x5557b8b74ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff653fff640 1 -- 172.21.15.2:0/539110779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87cc1b0 msgr2=0x5557b8b6da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff653fff640 1 --2- 172.21.15.2:0/539110779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87cc1b0 0x5557b8b6da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff653fff640 1 -- 172.21.15.2:0/539110779 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557b87e4210 con 0x5557b87a32f0 2024-05-14T09:39:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.606+0000 7ff653fff640 1 --2- 172.21.15.2:0/539110779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87a32f0 0x5557b8b6dfe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff654102420 tx=0x7ff644011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6e913bc847be263 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.607+0000 7ff63a7fc640 1 -- 172.21.15.2:0/539110779 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644004050 con 0x5557b87a32f0 2024-05-14T09:39:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.607+0000 7ff63a7fc640 1 -- 172.21.15.2:0/539110779 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6440054d0 con 0x5557b87a32f0 2024-05-14T09:39:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.607+0000 7ff63a7fc640 1 -- 172.21.15.2:0/539110779 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644004bc0 con 0x5557b87a32f0 2024-05-14T09:39:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.607+0000 7ff6517fa640 1 --2- 172.21.15.2:0/1396518349 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff63c0b28a0 0x7ff63c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.607+0000 7ff658c4c640 1 --2- 172.21.15.2:0/1396518349 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff63c0b28a0 0x7ff63c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.607+0000 7ff6517fa640 1 -- 172.21.15.2:0/1396518349 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7ff648102b70 con 0x7ff654103d90 2024-05-14T09:39:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.608+0000 7ff661564740 1 -- 172.21.15.2:0/539110779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87a32f0 msgr2=0x5557b8b6dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.608+0000 7ff661564740 1 --2- 172.21.15.2:0/539110779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87a32f0 0x5557b8b6dfe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff654102420 tx=0x7ff644011810 comp rx=0 tx=0).stop 2024-05-14T09:39:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.608+0000 7ff661564740 1 -- 172.21.15.2:0/539110779 shutdown_connections 2024-05-14T09:39:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.608+0000 7ff661564740 1 --2- 172.21.15.2:0/539110779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87ab4d0 0x5557b8b74ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.608+0000 7ff661564740 1 --2- 172.21.15.2:0/539110779 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87a32f0 0x5557b8b6dfe0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.608+0000 7ff661564740 1 --2- 172.21.15.2:0/539110779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87cc1b0 0x5557b8b6da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.608+0000 7ff661564740 1 -- 172.21.15.2:0/539110779 >> 172.21.15.2:0/539110779 conn(0x5557b87c0300 msgr2=0x5557b8b693a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.608+0000 7ff658c4c640 1 --2- 172.21.15.2:0/1396518349 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff63c0b28a0 0x7ff63c0b4db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5557b8b6ec30 tx=0x7ff64c009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.608+0000 7ff661564740 1 -- 172.21.15.2:0/539110779 shutdown_connections 2024-05-14T09:39:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.608+0000 7ff661564740 1 -- 172.21.15.2:0/539110779 wait complete. 2024-05-14T09:39:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.608+0000 7ff661564740 1 -- start start 2024-05-14T09:39:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.608+0000 7ff661564740 1 Processor -- start 2024-05-14T09:39:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff661564740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b79860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff661564740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87ab4d0 0x5557b8b79da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff661564740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87cc1b0 0x5557b8b80d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff661564740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5557b8b75930 con 0x5557b87a32f0 2024-05-14T09:39:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff661564740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5557b8a8f370 con 0x5557b87cc1b0 2024-05-14T09:39:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff661564740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5557b8b75ab0 con 0x5557b87ab4d0 2024-05-14T09:39:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff661564740 10 client.0 _ll_register_callbacks cb 0x5557b87e3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:39:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff653fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b79860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff6537fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87ab4d0 0x5557b8b79da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff653fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b79860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43342/0 (socket says 172.21.15.2:43342) 2024-05-14T09:39:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff653fff640 1 -- 172.21.15.2:0/3287863021 learned_addr learned my addr 172.21.15.2:0/3287863021 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff658c4c640 1 --2- 172.21.15.2:0/3287863021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87cc1b0 0x5557b8b80d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff6537fe640 1 -- 172.21.15.2:0/3287863021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87cc1b0 msgr2=0x5557b8b80d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff6537fe640 1 --2- 172.21.15.2:0/3287863021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87cc1b0 0x5557b8b80d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff6537fe640 1 -- 172.21.15.2:0/3287863021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 msgr2=0x5557b8b79860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff6537fe640 1 --2- 172.21.15.2:0/3287863021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b79860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.609+0000 7ff6537fe640 1 -- 172.21.15.2:0/3287863021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557b88b0620 con 0x5557b87ab4d0 2024-05-14T09:39:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff6537fe640 1 --2- 172.21.15.2:0/3287863021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87ab4d0 0x5557b8b79da0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff6480c9b20 tx=0x7ff6480c9590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dadd97e0a8cfa2a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff6337fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:39:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff6337fe640 1 -- 172.21.15.2:0/3287863021 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648049040 con 0x5557b87ab4d0 2024-05-14T09:39:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff661564740 1 -- 172.21.15.2:0/3287863021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6440054d0 con 0x5557b87ab4d0 2024-05-14T09:39:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff6337fe640 1 -- 172.21.15.2:0/3287863021 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7ff6480e43f0 con 0x5557b87ab4d0 2024-05-14T09:39:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff6337fe640 1 -- 172.21.15.2:0/3287863021 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6480c9e20 con 0x5557b87ab4d0 2024-05-14T09:39:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff661564740 10 client.25780 fetch_fsmap learned FSMap version 30 2024-05-14T09:39:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff661564740 1 -- 172.21.15.2:0/3287863021 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5557b8b75a50 con 0x5557b87ab4d0 2024-05-14T09:39:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff6337fe640 1 -- 172.21.15.2:0/3287863021 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6440054d0 con 0x5557b87ab4d0 2024-05-14T09:39:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff6337fe640 1 -- 172.21.15.2:0/3287863021 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7ff6480d2740 con 0x5557b87ab4d0 2024-05-14T09:39:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff661564740 10 client.25780 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T09:39:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff661564740 10 client.25780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:39:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff661564740 20 client.25780 populate_metadata read hostname 'smithi002' 2024-05-14T09:39:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff661564740 1 --2- 172.21.15.2:0/3287863021 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5557b878ad80 0x5557b8b873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff661564740 4 client.25780 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:39:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.610+0000 7ff661564740 1 -- 172.21.15.2:0/3287863021 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557b8ae01e0 con 0x5557b878ad80 2024-05-14T09:39:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.611+0000 7ff653fff640 1 --2- 172.21.15.2:0/3287863021 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5557b878ad80 0x5557b8b873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.611+0000 7ff6337fe640 10 client.25780 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.611+0000 7ff653fff640 1 --2- 172.21.15.2:0/3287863021 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5557b878ad80 0x5557b8b873a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89f847e86e931bc6 server_cookie=3f4f26f6cdd26ca3 in_seq=0 out_seq=0 2024-05-14T09:39:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.612+0000 7ff6337fe640 1 -- 172.21.15.2:0/3287863021 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5557b8ae01e0 con 0x5557b878ad80 2024-05-14T09:39:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.612+0000 7ff6337fe640 10 client.25780 handle_command_reply: tid=0 2024-05-14T09:39:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.612+0000 7ff661564740 1 client.25780 shutdown 2024-05-14T09:39:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.612+0000 7ff661564740 1 -- 172.21.15.2:0/3287863021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87ab4d0 msgr2=0x5557b8b79da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.612+0000 7ff661564740 1 --2- 172.21.15.2:0/3287863021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87ab4d0 0x5557b8b79da0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff6480c9b20 tx=0x7ff6480c9590 comp rx=0 tx=0).stop 2024-05-14T09:39:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.612+0000 7ff661564740 1 -- 172.21.15.2:0/3287863021 shutdown_connections 2024-05-14T09:39:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.612+0000 7ff661564740 1 --2- 172.21.15.2:0/3287863021 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5557b878ad80 0x5557b8b873a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.612+0000 7ff661564740 1 --2- 172.21.15.2:0/3287863021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87cc1b0 0x5557b8b80d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.612+0000 7ff661564740 1 --2- 172.21.15.2:0/3287863021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87ab4d0 0x5557b8b79da0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.612+0000 7ff6337fe640 0 client.25780 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.612+0000 7ff661564740 1 --2- 172.21.15.2:0/3287863021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b79860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.612+0000 7ff661564740 1 -- 172.21.15.2:0/3287863021 >> 172.21.15.2:0/3287863021 conn(0x5557b87c0300 msgr2=0x5557b8b6c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.612+0000 7ff661564740 1 -- 172.21.15.2:0/3287863021 shutdown_connections 2024-05-14T09:39:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.612+0000 7ff661564740 1 -- 172.21.15.2:0/3287863021 wait complete. 2024-05-14T09:39:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.612+0000 7ff661564740 20 client.25780 trim_cache size 0 max 16384 2024-05-14T09:39:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.628+0000 7ff661564740 1 Processor -- start 2024-05-14T09:39:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.628+0000 7ff661564740 1 -- start start 2024-05-14T09:39:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff661564740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b71080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff661564740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87ab4d0 0x5557b8b80c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff661564740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87cc1b0 0x5557b8b71a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff661564740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff6541ad2f0 con 0x5557b87a32f0 2024-05-14T09:39:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff661564740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff6541aabc0 con 0x5557b87cc1b0 2024-05-14T09:39:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff661564740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff6541a8a10 con 0x5557b87ab4d0 2024-05-14T09:39:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff653fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b71080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff653fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b71080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43346/0 (socket says 172.21.15.2:43346) 2024-05-14T09:39:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff653fff640 1 -- 172.21.15.2:0/485664475 learned_addr learned my addr 172.21.15.2:0/485664475 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff6537fe640 1 --2- 172.21.15.2:0/485664475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87ab4d0 0x5557b8b80c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff658c4c640 1 --2- 172.21.15.2:0/485664475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87cc1b0 0x5557b8b71a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff653fff640 1 -- 172.21.15.2:0/485664475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87ab4d0 msgr2=0x5557b8b80c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff653fff640 1 --2- 172.21.15.2:0/485664475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87ab4d0 0x5557b8b80c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff653fff640 1 -- 172.21.15.2:0/485664475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87cc1b0 msgr2=0x5557b8b71a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff653fff640 1 --2- 172.21.15.2:0/485664475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87cc1b0 0x5557b8b71a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff653fff640 1 -- 172.21.15.2:0/485664475 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557b8b74e00 con 0x5557b87a32f0 2024-05-14T09:39:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff6537fe640 1 --2- 172.21.15.2:0/485664475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87ab4d0 0x5557b8b80c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.629+0000 7ff653fff640 1 --2- 172.21.15.2:0/485664475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b71080 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff644009230 tx=0x7ff644004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f4db90db230210 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.630+0000 7ff639ffb640 1 -- 172.21.15.2:0/485664475 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644033070 con 0x5557b87a32f0 2024-05-14T09:39:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.630+0000 7ff639ffb640 1 -- 172.21.15.2:0/485664475 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff644011d10 con 0x5557b87a32f0 2024-05-14T09:39:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.630+0000 7ff639ffb640 1 -- 172.21.15.2:0/485664475 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644003ab0 con 0x5557b87a32f0 2024-05-14T09:39:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.630+0000 7ff661564740 1 -- 172.21.15.2:0/485664475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 msgr2=0x5557b8b71080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.630+0000 7ff661564740 1 --2- 172.21.15.2:0/485664475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b71080 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff644009230 tx=0x7ff644004700 comp rx=0 tx=0).stop 2024-05-14T09:39:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.630+0000 7ff661564740 1 -- 172.21.15.2:0/485664475 shutdown_connections 2024-05-14T09:39:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.630+0000 7ff661564740 1 --2- 172.21.15.2:0/485664475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87cc1b0 0x5557b8b71a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.630+0000 7ff661564740 1 --2- 172.21.15.2:0/485664475 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87ab4d0 0x5557b8b80c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.630+0000 7ff661564740 1 --2- 172.21.15.2:0/485664475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b71080 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.630+0000 7ff661564740 1 -- 172.21.15.2:0/485664475 >> 172.21.15.2:0/485664475 conn(0x5557b87c0300 msgr2=0x5557b8b63f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.630+0000 7ff661564740 1 -- 172.21.15.2:0/485664475 shutdown_connections 2024-05-14T09:39:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.631+0000 7ff661564740 1 -- 172.21.15.2:0/485664475 wait complete. 2024-05-14T09:39:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.631+0000 7ff661564740 1 -- start start 2024-05-14T09:39:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.631+0000 7ff661564740 1 Processor -- start 2024-05-14T09:39:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.631+0000 7ff661564740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b7db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.631+0000 7ff661564740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87ab4d0 0x5557b8b7e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.631+0000 7ff661564740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87cc1b0 0x5557b8bed740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.631+0000 7ff661564740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff6541a9080 con 0x5557b87a32f0 2024-05-14T09:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.631+0000 7ff661564740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff6541aa4f0 con 0x5557b87ab4d0 2024-05-14T09:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.631+0000 7ff661564740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff6541a9750 con 0x5557b87cc1b0 2024-05-14T09:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.631+0000 7ff661564740 10 client.0 _ll_register_callbacks cb 0x7ff648005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.631+0000 7ff653fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b7db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.631+0000 7ff653fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b7db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43348/0 (socket says 172.21.15.2:43348) 2024-05-14T09:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.631+0000 7ff653fff640 1 -- 172.21.15.2:0/715176586 learned_addr learned my addr 172.21.15.2:0/715176586 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.632+0000 7ff658c4c640 1 --2- 172.21.15.2:0/715176586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87cc1b0 0x5557b8bed740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.632+0000 7ff6537fe640 1 --2- 172.21.15.2:0/715176586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87ab4d0 0x5557b8b7e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.632+0000 7ff653fff640 1 -- 172.21.15.2:0/715176586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87cc1b0 msgr2=0x5557b8bed740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.632+0000 7ff653fff640 1 --2- 172.21.15.2:0/715176586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87cc1b0 0x5557b8bed740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.632+0000 7ff653fff640 1 -- 172.21.15.2:0/715176586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87ab4d0 msgr2=0x5557b8b7e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.632+0000 7ff653fff640 1 --2- 172.21.15.2:0/715176586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87ab4d0 0x5557b8b7e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.632+0000 7ff653fff640 1 -- 172.21.15.2:0/715176586 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557b8b7e720 con 0x5557b87a32f0 2024-05-14T09:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.632+0000 7ff653fff640 1 --2- 172.21.15.2:0/715176586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b7db70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff6440099b0 tx=0x7ff644004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6451ffa8f5b3610 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.632+0000 7ff6337fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:39:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.632+0000 7ff6337fe640 1 -- 172.21.15.2:0/715176586 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644033070 con 0x5557b87a32f0 2024-05-14T09:39:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.632+0000 7ff6337fe640 1 -- 172.21.15.2:0/715176586 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7ff644003c40 con 0x5557b87a32f0 2024-05-14T09:39:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.632+0000 7ff661564740 1 -- 172.21.15.2:0/715176586 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff644011d10 con 0x5557b87a32f0 2024-05-14T09:39:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.632+0000 7ff6337fe640 1 -- 172.21.15.2:0/715176586 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff644044050 con 0x5557b87a32f0 2024-05-14T09:39:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.633+0000 7ff661564740 10 client.16104 fetch_fsmap learned FSMap version 30 2024-05-14T09:39:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.633+0000 7ff661564740 1 -- 172.21.15.2:0/715176586 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5557b8beed20 con 0x5557b87a32f0 2024-05-14T09:39:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.633+0000 7ff6337fe640 1 -- 172.21.15.2:0/715176586 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff644011d10 con 0x5557b87a32f0 2024-05-14T09:39:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.633+0000 7ff6337fe640 1 -- 172.21.15.2:0/715176586 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7ff6440195c0 con 0x5557b87a32f0 2024-05-14T09:39:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.633+0000 7ff661564740 10 client.16104 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T09:39:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.633+0000 7ff661564740 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:39:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.633+0000 7ff661564740 20 client.16104 populate_metadata read hostname 'smithi002' 2024-05-14T09:39:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.633+0000 7ff661564740 1 --2- 172.21.15.2:0/715176586 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5557b87601c0 0x5557b8bf1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.633+0000 7ff661564740 4 client.16104 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:39:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.633+0000 7ff661564740 1 -- 172.21.15.2:0/715176586 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557b8ada050 con 0x5557b87601c0 2024-05-14T09:39:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.633+0000 7ff6537fe640 1 --2- 172.21.15.2:0/715176586 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5557b87601c0 0x5557b8bf1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.634+0000 7ff6337fe640 10 client.16104 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.634+0000 7ff6537fe640 1 --2- 172.21.15.2:0/715176586 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5557b87601c0 0x5557b8bf1560 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1246e59b90264da server_cookie=a364de14d8cf6437 in_seq=0 out_seq=0 2024-05-14T09:39:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.723+0000 7ff6337fe640 1 -- 172.21.15.2:0/715176586 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557b8ada050 con 0x5557b87601c0 2024-05-14T09:39:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.723+0000 7ff6337fe640 10 client.16104 handle_command_reply: tid=0 2024-05-14T09:39:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.723+0000 7ff661564740 1 client.16104 shutdown 2024-05-14T09:39:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.723+0000 7ff661564740 1 -- 172.21.15.2:0/715176586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 msgr2=0x5557b8b7db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.723+0000 7ff661564740 1 --2- 172.21.15.2:0/715176586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b7db70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff6440099b0 tx=0x7ff644004220 comp rx=0 tx=0).stop 2024-05-14T09:39:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.723+0000 7ff661564740 1 -- 172.21.15.2:0/715176586 shutdown_connections 2024-05-14T09:39:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.723+0000 7ff661564740 1 --2- 172.21.15.2:0/715176586 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5557b87601c0 0x5557b8bf1560 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.723+0000 7ff661564740 1 --2- 172.21.15.2:0/715176586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557b87cc1b0 0x5557b8bed740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.723+0000 7ff661564740 1 --2- 172.21.15.2:0/715176586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557b87ab4d0 0x5557b8b7e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.723+0000 7ff6337fe640 0 client.16104 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.723+0000 7ff661564740 1 --2- 172.21.15.2:0/715176586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557b87a32f0 0x5557b8b7db70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.723+0000 7ff661564740 1 -- 172.21.15.2:0/715176586 >> 172.21.15.2:0/715176586 conn(0x5557b87c0300 msgr2=0x5557b8b63f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.723+0000 7ff661564740 1 -- 172.21.15.2:0/715176586 shutdown_connections 2024-05-14T09:39:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.723+0000 7ff661564740 1 -- 172.21.15.2:0/715176586 wait complete. 2024-05-14T09:39:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.724+0000 7ff661564740 20 client.16104 trim_cache size 0 max 16384 2024-05-14T09:39:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:39:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:39:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:39:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:39:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.724+0000 7ff65a6af640 1 -- 172.21.15.2:0/1396518349 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff63c0b28a0 msgr2=0x7ff63c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.724+0000 7ff65a6af640 1 --2- 172.21.15.2:0/1396518349 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff63c0b28a0 0x7ff63c0b4db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5557b8b6ec30 tx=0x7ff64c009600 comp rx=0 tx=0).stop 2024-05-14T09:39:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.724+0000 7ff65a6af640 1 -- 172.21.15.2:0/1396518349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff654103d90 msgr2=0x7ff6541996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.724+0000 7ff65a6af640 1 --2- 172.21.15.2:0/1396518349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff654103d90 0x7ff6541996d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff648006010 tx=0x7ff648031a50 comp rx=0 tx=0).stop 2024-05-14T09:39:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.724+0000 7ff65a6af640 1 -- 172.21.15.2:0/1396518349 shutdown_connections 2024-05-14T09:39:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.724+0000 7ff65a6af640 1 --2- 172.21.15.2:0/1396518349 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff63c0b28a0 0x7ff63c0b4db0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.724+0000 7ff65a6af640 1 --2- 172.21.15.2:0/1396518349 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff654108a90 0x7ff6541a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.724+0000 7ff65a6af640 1 --2- 172.21.15.2:0/1396518349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff654103d90 0x7ff6541996d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.724+0000 7ff65a6af640 1 --2- 172.21.15.2:0/1396518349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff654102b50 0x7ff654199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.724+0000 7ff65a6af640 1 -- 172.21.15.2:0/1396518349 >> 172.21.15.2:0/1396518349 conn(0x7ff6540fdd40 msgr2=0x7ff654104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.725+0000 7ff65a6af640 1 -- 172.21.15.2:0/1396518349 shutdown_connections 2024-05-14T09:39:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:26.725+0000 7ff65a6af640 1 -- 172.21.15.2:0/1396518349 wait complete. 2024-05-14T09:39:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:26 smithi002 ceph-mon[31557]: pgmap v558: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-05-14T09:39:26.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:26 smithi029 ceph-mon[36249]: pgmap v558: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-05-14T09:39:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:26 smithi026 ceph-mon[36056]: pgmap v558: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-05-14T09:39:27.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 338362 10.18 MB/sec execute 53 sec latency 62.872 ms 2024-05-14T09:39:28.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 340738 10.25 MB/sec execute 54 sec latency 23.764 ms 2024-05-14T09:39:28.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:28 smithi002 ceph-mon[31557]: pgmap v559: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 332 op/s 2024-05-14T09:39:28.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:28 smithi029 ceph-mon[36249]: pgmap v559: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 332 op/s 2024-05-14T09:39:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:28 smithi026 ceph-mon[36056]: pgmap v559: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 332 op/s 2024-05-14T09:39:29.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 342076 10.16 MB/sec execute 55 sec latency 13.800 ms 2024-05-14T09:39:30.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 344293 10.22 MB/sec execute 56 sec latency 80.713 ms 2024-05-14T09:39:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:30 smithi002 ceph-mon[31557]: pgmap v560: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 345 op/s 2024-05-14T09:39:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:30 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:39:30.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:30 smithi029 ceph-mon[36249]: pgmap v560: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 345 op/s 2024-05-14T09:39:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:30 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:39:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:30 smithi026 ceph-mon[36056]: pgmap v560: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 345 op/s 2024-05-14T09:39:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:30 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:39:31.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 345898 10.14 MB/sec execute 57 sec latency 30.902 ms 2024-05-14T09:39:32.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 348016 10.20 MB/sec execute 58 sec latency 15.799 ms 2024-05-14T09:39:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:32 smithi002 ceph-mon[31557]: pgmap v561: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 280 op/s 2024-05-14T09:39:32.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:32 smithi029 ceph-mon[36249]: pgmap v561: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 280 op/s 2024-05-14T09:39:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:32 smithi026 ceph-mon[36056]: pgmap v561: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 280 op/s 2024-05-14T09:39:33.261 INFO:tasks.workunit.client.0.smithi002.stdout: 1 350012 10.15 MB/sec execute 59 sec latency 15.497 ms 2024-05-14T09:39:34.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 351600 10.18 MB/sec execute 60 sec latency 13.788 ms 2024-05-14T09:39:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:34 smithi002 ceph-mon[31557]: pgmap v562: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 390 op/s 2024-05-14T09:39:34.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:34 smithi029 ceph-mon[36249]: pgmap v562: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 390 op/s 2024-05-14T09:39:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:34 smithi026 ceph-mon[36056]: pgmap v562: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 390 op/s 2024-05-14T09:39:35.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 353649 10.14 MB/sec execute 61 sec latency 46.502 ms 2024-05-14T09:39:36.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 355270 10.16 MB/sec execute 62 sec latency 28.456 ms 2024-05-14T09:39:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:36 smithi002 ceph-mon[31557]: pgmap v563: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 284 op/s 2024-05-14T09:39:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:36 smithi029 ceph-mon[36249]: pgmap v563: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 284 op/s 2024-05-14T09:39:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:36 smithi026 ceph-mon[36056]: pgmap v563: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 284 op/s 2024-05-14T09:39:37.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 357258 10.13 MB/sec execute 63 sec latency 66.231 ms 2024-05-14T09:39:38.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 358843 10.14 MB/sec execute 64 sec latency 24.217 ms 2024-05-14T09:39:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:38 smithi002 ceph-mon[31557]: pgmap v564: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-05-14T09:39:38.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:38 smithi029 ceph-mon[36249]: pgmap v564: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-05-14T09:39:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:38 smithi026 ceph-mon[36056]: pgmap v564: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-05-14T09:39:39.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 361435 10.19 MB/sec execute 65 sec latency 15.950 ms 2024-05-14T09:39:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.834+0000 7f3baa994640 1 -- 172.21.15.2:0/1178723557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba4108a90 msgr2=0x7f3ba410aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.834+0000 7f3baa994640 1 --2- 172.21.15.2:0/1178723557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba4108a90 0x7f3ba410aeb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009ca0 tx=0x7f3b9c02a8c0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.834+0000 7f3baa994640 1 -- 172.21.15.2:0/1178723557 shutdown_connections 2024-05-14T09:39:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.834+0000 7f3baa994640 1 --2- 172.21.15.2:0/1178723557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba4108a90 0x7f3ba410aeb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.834+0000 7f3baa994640 1 --2- 172.21.15.2:0/1178723557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ba4103d90 0x7f3ba41041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.834+0000 7f3baa994640 1 --2- 172.21.15.2:0/1178723557 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4102b50 0x7f3ba4103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.834+0000 7f3baa994640 1 -- 172.21.15.2:0/1178723557 >> 172.21.15.2:0/1178723557 conn(0x7f3ba40fdd60 msgr2=0x7f3ba41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.834+0000 7f3baa994640 1 -- 172.21.15.2:0/1178723557 shutdown_connections 2024-05-14T09:39:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.834+0000 7f3baa994640 1 -- 172.21.15.2:0/1178723557 wait complete. 2024-05-14T09:39:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.834+0000 7f3baa994640 1 Processor -- start 2024-05-14T09:39:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3baa994640 1 -- start start 2024-05-14T09:39:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3baa994640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ba4102b50 0x7f3ba4199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3baa994640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba4103d90 0x7f3ba4199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3baa994640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4108a90 0x7f3ba41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3baa994640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba410e660 con 0x7f3ba4103d90 2024-05-14T09:39:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3baa994640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba410e4e0 con 0x7f3ba4102b50 2024-05-14T09:39:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3baa994640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba410e7e0 con 0x7f3ba4108a90 2024-05-14T09:39:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3ba37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba4103d90 0x7f3ba4199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3ba37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba4103d90 0x7f3ba4199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57286/0 (socket says 172.21.15.2:57286) 2024-05-14T09:39:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3ba37fe640 1 -- 172.21.15.2:0/4163825105 learned_addr learned my addr 172.21.15.2:0/4163825105 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3ba3fff640 1 --2- 172.21.15.2:0/4163825105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ba4102b50 0x7f3ba4199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3ba8f31640 1 --2- 172.21.15.2:0/4163825105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4108a90 0x7f3ba41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3ba37fe640 1 -- 172.21.15.2:0/4163825105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4108a90 msgr2=0x7f3ba41a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3ba37fe640 1 --2- 172.21.15.2:0/4163825105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4108a90 0x7f3ba41a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3ba37fe640 1 -- 172.21.15.2:0/4163825105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ba4102b50 msgr2=0x7f3ba4199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3ba37fe640 1 --2- 172.21.15.2:0/4163825105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ba4102b50 0x7f3ba4199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3ba37fe640 1 -- 172.21.15.2:0/4163825105 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba41a0c80 con 0x7f3ba4103d90 2024-05-14T09:39:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.835+0000 7f3ba3fff640 1 --2- 172.21.15.2:0/4163825105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ba4102b50 0x7f3ba4199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.836+0000 7f3ba37fe640 1 --2- 172.21.15.2:0/4163825105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba4103d90 0x7f3ba4199680 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800e9f0 tx=0x7f3b9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.836+0000 7f3ba17fa640 1 -- 172.21.15.2:0/4163825105 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800cd80 con 0x7f3ba4103d90 2024-05-14T09:39:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.836+0000 7f3ba17fa640 1 -- 172.21.15.2:0/4163825105 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b980108e0 con 0x7f3ba4103d90 2024-05-14T09:39:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.836+0000 7f3baa994640 1 -- 172.21.15.2:0/4163825105 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba41a0f70 con 0x7f3ba4103d90 2024-05-14T09:39:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.836+0000 7f3ba17fa640 1 -- 172.21.15.2:0/4163825105 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98018980 con 0x7f3ba4103d90 2024-05-14T09:39:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.836+0000 7f3baa994640 1 -- 172.21.15.2:0/4163825105 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba41a13e0 con 0x7f3ba4103d90 2024-05-14T09:39:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.838+0000 7f3bb1850740 1 Processor -- start 2024-05-14T09:39:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.838+0000 7f3bb1850740 1 -- start start 2024-05-14T09:39:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3ba17fa640 1 -- 172.21.15.2:0/4163825105 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3b98018b20 con 0x7f3ba4103d90 2024-05-14T09:39:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3bb1850740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e75bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3bb1850740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e38f2f0 0x56169e75c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3bb1850740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e3974d0 0x56169e7630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3bb1850740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56169e6be5d0 con 0x56169e3b81b0 2024-05-14T09:39:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3bb1850740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56169e677890 con 0x56169e3974d0 2024-05-14T09:39:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3bb1850740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56169e6ba590 con 0x56169e38f2f0 2024-05-14T09:39:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3ba8f31640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e75bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3ba8f31640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e75bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57294/0 (socket says 172.21.15.2:57294) 2024-05-14T09:39:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3ba8f31640 1 -- 172.21.15.2:0/1774117205 learned_addr learned my addr 172.21.15.2:0/1774117205 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3ba3fff640 1 --2- 172.21.15.2:0/1774117205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e38f2f0 0x56169e75c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3ba37fe640 1 --2- 172.21.15.2:0/1774117205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e3974d0 0x56169e7630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3ba3fff640 1 -- 172.21.15.2:0/1774117205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e3974d0 msgr2=0x56169e7630f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3ba3fff640 1 --2- 172.21.15.2:0/1774117205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e3974d0 0x56169e7630f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3ba3fff640 1 -- 172.21.15.2:0/1774117205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 msgr2=0x56169e75bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3ba3fff640 1 --2- 172.21.15.2:0/1774117205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e75bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3ba3fff640 1 -- 172.21.15.2:0/1774117205 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56169e3d0210 con 0x56169e38f2f0 2024-05-14T09:39:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3ba8f31640 1 --2- 172.21.15.2:0/1774117205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e75bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.839+0000 7f3ba3fff640 1 --2- 172.21.15.2:0/1774117205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e38f2f0 0x56169e75c1f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4102420 tx=0x7f3b940118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=898a05ace2cf83a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.840+0000 7f3ba17fa640 1 --2- 172.21.15.2:0/4163825105 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3b8c0b28a0 0x7f3b8c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.840+0000 7f3b8a7fc640 1 -- 172.21.15.2:0/1774117205 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b94004310 con 0x56169e38f2f0 2024-05-14T09:39:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.840+0000 7f3b8a7fc640 1 -- 172.21.15.2:0/1774117205 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b940044b0 con 0x56169e38f2f0 2024-05-14T09:39:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.840+0000 7f3b8a7fc640 1 -- 172.21.15.2:0/1774117205 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b940035f0 con 0x56169e38f2f0 2024-05-14T09:39:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.840+0000 7f3ba8f31640 1 --2- 172.21.15.2:0/4163825105 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3b8c0b28a0 0x7f3b8c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.840+0000 7f3ba17fa640 1 -- 172.21.15.2:0/4163825105 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3b980e29b0 con 0x7f3ba4103d90 2024-05-14T09:39:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.841+0000 7f3bb1850740 1 -- 172.21.15.2:0/1774117205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e38f2f0 msgr2=0x56169e75c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.841+0000 7f3bb1850740 1 --2- 172.21.15.2:0/1774117205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e38f2f0 0x56169e75c1f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4102420 tx=0x7f3b940118d0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.841+0000 7f3ba8f31640 1 --2- 172.21.15.2:0/4163825105 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3b8c0b28a0 0x7f3b8c0b4db0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009a60 tx=0x7f3b9c02f310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.841+0000 7f3bb1850740 1 -- 172.21.15.2:0/1774117205 shutdown_connections 2024-05-14T09:39:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.841+0000 7f3bb1850740 1 --2- 172.21.15.2:0/1774117205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e3974d0 0x56169e7630f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.841+0000 7f3bb1850740 1 --2- 172.21.15.2:0/1774117205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e38f2f0 0x56169e75c1f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.841+0000 7f3bb1850740 1 --2- 172.21.15.2:0/1774117205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e75bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.841+0000 7f3bb1850740 1 -- 172.21.15.2:0/1774117205 >> 172.21.15.2:0/1774117205 conn(0x56169e3ac300 msgr2=0x56169e7553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.841+0000 7f3bb1850740 1 -- 172.21.15.2:0/1774117205 shutdown_connections 2024-05-14T09:39:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.841+0000 7f3bb1850740 1 -- 172.21.15.2:0/1774117205 wait complete. 2024-05-14T09:39:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.841+0000 7f3bb1850740 1 -- start start 2024-05-14T09:39:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.842+0000 7f3bb1850740 1 Processor -- start 2024-05-14T09:39:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.842+0000 7f3bb1850740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 0x56169e765830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.842+0000 7f3bb1850740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 0x56169e765d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.842+0000 7f3bb1850740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e76ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.842+0000 7f3bb1850740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56169e763b40 con 0x56169e3b81b0 2024-05-14T09:39:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.842+0000 7f3bb1850740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56169e67b370 con 0x56169e38f2f0 2024-05-14T09:39:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.842+0000 7f3bb1850740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56169e763cc0 con 0x56169e3974d0 2024-05-14T09:39:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.842+0000 7f3bb1850740 10 client.0 _ll_register_callbacks cb 0x56169e3cfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:39:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.842+0000 7f3ba8f31640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e76ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.842+0000 7f3ba3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 0x56169e765830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.842+0000 7f3ba8f31640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e76ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57302/0 (socket says 172.21.15.2:57302) 2024-05-14T09:39:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.842+0000 7f3ba8f31640 1 -- 172.21.15.2:0/3832326681 learned_addr learned my addr 172.21.15.2:0/3832326681 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.842+0000 7f3ba37fe640 1 --2- 172.21.15.2:0/3832326681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 0x56169e765d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3ba8f31640 1 -- 172.21.15.2:0/3832326681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 msgr2=0x56169e765d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3ba8f31640 1 --2- 172.21.15.2:0/3832326681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 0x56169e765d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3ba8f31640 1 -- 172.21.15.2:0/3832326681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 msgr2=0x56169e765830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3ba8f31640 1 --2- 172.21.15.2:0/3832326681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 0x56169e765830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3ba8f31640 1 -- 172.21.15.2:0/3832326681 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56169e49c620 con 0x56169e3b81b0 2024-05-14T09:39:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3ba3fff640 1 --2- 172.21.15.2:0/3832326681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 0x56169e765830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3ba37fe640 1 --2- 172.21.15.2:0/3832326681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 0x56169e765d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3ba8f31640 1 --2- 172.21.15.2:0/3832326681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e76ccd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009ca0 tx=0x7f3b9c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8161110811f2a49 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3b837fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:39:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3b837fe640 1 -- 172.21.15.2:0/3832326681 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c004310 con 0x56169e3b81b0 2024-05-14T09:39:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3b837fe640 1 -- 172.21.15.2:0/3832326681 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3b9c0044b0 con 0x56169e3b81b0 2024-05-14T09:39:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3bb1850740 1 -- 172.21.15.2:0/3832326681 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b940044b0 con 0x56169e3b81b0 2024-05-14T09:39:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3b837fe640 1 -- 172.21.15.2:0/3832326681 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3b9c0046b0 con 0x56169e3b81b0 2024-05-14T09:39:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3bb1850740 10 client.16116 fetch_fsmap learned FSMap version 31 2024-05-14T09:39:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3bb1850740 1 -- 172.21.15.2:0/3832326681 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56169e76fc00 con 0x56169e3b81b0 2024-05-14T09:39:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.843+0000 7f3b837fe640 1 -- 172.21.15.2:0/3832326681 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b940044b0 con 0x56169e3b81b0 2024-05-14T09:39:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.844+0000 7f3b837fe640 1 -- 172.21.15.2:0/3832326681 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f3b9c051070 con 0x56169e3b81b0 2024-05-14T09:39:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.844+0000 7f3bb1850740 10 client.16116 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T09:39:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.844+0000 7f3bb1850740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:39:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.844+0000 7f3bb1850740 20 client.16116 populate_metadata read hostname 'smithi002' 2024-05-14T09:39:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.844+0000 7f3bb1850740 1 --2- 172.21.15.2:0/3832326681 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56169e376d80 0x56169e773390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.844+0000 7f3bb1850740 4 client.16116 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:39:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.844+0000 7f3bb1850740 1 -- 172.21.15.2:0/3832326681 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56169e6cc1e0 con 0x56169e376d80 2024-05-14T09:39:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.844+0000 7f3ba3fff640 1 --2- 172.21.15.2:0/3832326681 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56169e376d80 0x56169e773390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.844+0000 7f3b837fe640 10 client.16116 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.844+0000 7f3ba3fff640 1 --2- 172.21.15.2:0/3832326681 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56169e376d80 0x56169e773390 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed4db0975a1361 server_cookie=f4b2562a7da874eb in_seq=0 out_seq=0 2024-05-14T09:39:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.845+0000 7f3b837fe640 1 -- 172.21.15.2:0/3832326681 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56169e6cc1e0 con 0x56169e376d80 2024-05-14T09:39:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.845+0000 7f3b837fe640 10 client.16116 handle_command_reply: tid=0 2024-05-14T09:39:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.845+0000 7f3bb1850740 1 client.16116 shutdown 2024-05-14T09:39:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.845+0000 7f3bb1850740 1 -- 172.21.15.2:0/3832326681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 msgr2=0x56169e76ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.845+0000 7f3bb1850740 1 --2- 172.21.15.2:0/3832326681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e76ccd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009ca0 tx=0x7f3b9c02a8c0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.845+0000 7f3bb1850740 1 -- 172.21.15.2:0/3832326681 shutdown_connections 2024-05-14T09:39:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.845+0000 7f3bb1850740 1 --2- 172.21.15.2:0/3832326681 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56169e376d80 0x56169e773390 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.845+0000 7f3bb1850740 1 --2- 172.21.15.2:0/3832326681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e76ccd0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.845+0000 7f3bb1850740 1 --2- 172.21.15.2:0/3832326681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 0x56169e765d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.845+0000 7f3b837fe640 0 client.16116 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.845+0000 7f3bb1850740 1 --2- 172.21.15.2:0/3832326681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 0x56169e765830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.845+0000 7f3bb1850740 1 -- 172.21.15.2:0/3832326681 >> 172.21.15.2:0/3832326681 conn(0x56169e3ac300 msgr2=0x56169e758300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.846+0000 7f3bb1850740 1 -- 172.21.15.2:0/3832326681 shutdown_connections 2024-05-14T09:39:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.846+0000 7f3bb1850740 1 -- 172.21.15.2:0/3832326681 wait complete. 2024-05-14T09:39:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.846+0000 7f3bb1850740 20 client.16116 trim_cache size 0 max 16384 2024-05-14T09:39:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.862+0000 7f3bb1850740 1 Processor -- start 2024-05-14T09:39:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3bb1850740 1 -- start start 2024-05-14T09:39:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3bb1850740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 0x56169e75fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3bb1850740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 0x56169e760170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3bb1850740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e76cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3bb1850740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba41ad380 con 0x56169e3b81b0 2024-05-14T09:39:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3bb1850740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba41aac50 con 0x56169e38f2f0 2024-05-14T09:39:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3bb1850740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba41a8aa0 con 0x56169e3974d0 2024-05-14T09:39:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3ba8f31640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e76cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3ba8f31640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e76cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57308/0 (socket says 172.21.15.2:57308) 2024-05-14T09:39:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3ba37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 0x56169e760170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3ba8f31640 1 -- 172.21.15.2:0/1515713157 learned_addr learned my addr 172.21.15.2:0/1515713157 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3ba3fff640 1 --2- 172.21.15.2:0/1515713157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 0x56169e75fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3ba8f31640 1 -- 172.21.15.2:0/1515713157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 msgr2=0x56169e760170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3ba8f31640 1 --2- 172.21.15.2:0/1515713157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 0x56169e760170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3ba8f31640 1 -- 172.21.15.2:0/1515713157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 msgr2=0x56169e75fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3ba8f31640 1 --2- 172.21.15.2:0/1515713157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 0x56169e75fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3ba8f31640 1 -- 172.21.15.2:0/1515713157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56169e750620 con 0x56169e3b81b0 2024-05-14T09:39:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3ba37fe640 1 --2- 172.21.15.2:0/1515713157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 0x56169e760170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.863+0000 7f3ba8f31640 1 --2- 172.21.15.2:0/1515713157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e76cbf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c031b50 tx=0x7f3b9c02b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cd642453a109cc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.864+0000 7f3b89ffb640 1 -- 172.21.15.2:0/1515713157 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c004310 con 0x56169e3b81b0 2024-05-14T09:39:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.864+0000 7f3b89ffb640 1 -- 172.21.15.2:0/1515713157 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b9c036050 con 0x56169e3b81b0 2024-05-14T09:39:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.864+0000 7f3b89ffb640 1 -- 172.21.15.2:0/1515713157 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c02f930 con 0x56169e3b81b0 2024-05-14T09:39:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 -- 172.21.15.2:0/1515713157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 msgr2=0x56169e76cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 --2- 172.21.15.2:0/1515713157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e76cbf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c031b50 tx=0x7f3b9c02b040 comp rx=0 tx=0).stop 2024-05-14T09:39:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 -- 172.21.15.2:0/1515713157 shutdown_connections 2024-05-14T09:39:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 --2- 172.21.15.2:0/1515713157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e76cbf0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 --2- 172.21.15.2:0/1515713157 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 0x56169e760170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 --2- 172.21.15.2:0/1515713157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 0x56169e75fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 -- 172.21.15.2:0/1515713157 >> 172.21.15.2:0/1515713157 conn(0x56169e3ac300 msgr2=0x56169e74ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 -- 172.21.15.2:0/1515713157 shutdown_connections 2024-05-14T09:39:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 -- 172.21.15.2:0/1515713157 wait complete. 2024-05-14T09:39:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 -- start start 2024-05-14T09:39:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 Processor -- start 2024-05-14T09:39:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 0x56169e76a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 0x56169e76a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e7d9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba41a9110 con 0x56169e3b81b0 2024-05-14T09:39:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba41aa580 con 0x56169e38f2f0 2024-05-14T09:39:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.865+0000 7f3bb1850740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba41a97e0 con 0x56169e3974d0 2024-05-14T09:39:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3bb1850740 10 client.0 _ll_register_callbacks cb 0x7f3b9c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:39:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3ba8f31640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e7d9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3ba8f31640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e7d9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57310/0 (socket says 172.21.15.2:57310) 2024-05-14T09:39:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3ba3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 0x56169e76a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3ba8f31640 1 -- 172.21.15.2:0/130993783 learned_addr learned my addr 172.21.15.2:0/130993783 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3ba37fe640 1 --2- 172.21.15.2:0/130993783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 0x56169e76a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3ba8f31640 1 -- 172.21.15.2:0/130993783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 msgr2=0x56169e76a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3ba8f31640 1 --2- 172.21.15.2:0/130993783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 0x56169e76a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3ba8f31640 1 -- 172.21.15.2:0/130993783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 msgr2=0x56169e76a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3ba8f31640 1 --2- 172.21.15.2:0/130993783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 0x56169e76a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3ba8f31640 1 -- 172.21.15.2:0/130993783 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56169e76d5c0 con 0x56169e3b81b0 2024-05-14T09:39:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3ba37fe640 1 --2- 172.21.15.2:0/130993783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 0x56169e76a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3ba8f31640 1 --2- 172.21.15.2:0/130993783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e7d9730 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c030880 tx=0x7f3b9c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311adcf970d8424a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3b837fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:39:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3b837fe640 1 -- 172.21.15.2:0/130993783 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c06a060 con 0x56169e3b81b0 2024-05-14T09:39:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3bb1850740 1 -- 172.21.15.2:0/130993783 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b9c036050 con 0x56169e3b81b0 2024-05-14T09:39:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3b837fe640 1 -- 172.21.15.2:0/130993783 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3b9c004d40 con 0x56169e3b81b0 2024-05-14T09:39:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.866+0000 7f3b837fe640 1 -- 172.21.15.2:0/130993783 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3b9c02f6d0 con 0x56169e3b81b0 2024-05-14T09:39:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.867+0000 7f3bb1850740 10 client.16122 fetch_fsmap learned FSMap version 31 2024-05-14T09:39:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.867+0000 7f3bb1850740 1 -- 172.21.15.2:0/130993783 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56169e7daae0 con 0x56169e3b81b0 2024-05-14T09:39:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.867+0000 7f3b837fe640 1 -- 172.21.15.2:0/130993783 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b9c036050 con 0x56169e3b81b0 2024-05-14T09:39:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.867+0000 7f3b837fe640 1 -- 172.21.15.2:0/130993783 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f3b9c051040 con 0x56169e3b81b0 2024-05-14T09:39:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.867+0000 7f3bb1850740 10 client.16122 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T09:39:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.867+0000 7f3bb1850740 10 client.16122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:39:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.867+0000 7f3bb1850740 20 client.16122 populate_metadata read hostname 'smithi002' 2024-05-14T09:39:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.867+0000 7f3bb1850740 1 --2- 172.21.15.2:0/130993783 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56169e34c1c0 0x56169e7dd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.867+0000 7f3bb1850740 4 client.16122 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:39:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.867+0000 7f3bb1850740 1 -- 172.21.15.2:0/130993783 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56169e6c6050 con 0x56169e34c1c0 2024-05-14T09:39:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.867+0000 7f3ba3fff640 1 --2- 172.21.15.2:0/130993783 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56169e34c1c0 0x56169e7dd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.868+0000 7f3b837fe640 10 client.16122 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.868+0000 7f3ba3fff640 1 --2- 172.21.15.2:0/130993783 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56169e34c1c0 0x56169e7dd240 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ce0709032b6590 server_cookie=d24830b439cdd9dd in_seq=0 out_seq=0 2024-05-14T09:39:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.979+0000 7f3b837fe640 1 -- 172.21.15.2:0/130993783 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56169e6c6050 con 0x56169e34c1c0 2024-05-14T09:39:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.979+0000 7f3b837fe640 10 client.16122 handle_command_reply: tid=0 2024-05-14T09:39:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.979+0000 7f3bb1850740 1 client.16122 shutdown 2024-05-14T09:39:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.979+0000 7f3bb1850740 1 -- 172.21.15.2:0/130993783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 msgr2=0x56169e7d9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.979+0000 7f3bb1850740 1 --2- 172.21.15.2:0/130993783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e7d9730 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c030880 tx=0x7f3b9c009360 comp rx=0 tx=0).stop 2024-05-14T09:39:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.979+0000 7f3bb1850740 1 -- 172.21.15.2:0/130993783 shutdown_connections 2024-05-14T09:39:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.979+0000 7f3bb1850740 1 --2- 172.21.15.2:0/130993783 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56169e34c1c0 0x56169e7dd240 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.979+0000 7f3bb1850740 1 --2- 172.21.15.2:0/130993783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56169e3b81b0 0x56169e7d9730 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.979+0000 7f3bb1850740 1 --2- 172.21.15.2:0/130993783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56169e3974d0 0x56169e76a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.979+0000 7f3bb1850740 1 --2- 172.21.15.2:0/130993783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56169e38f2f0 0x56169e76a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.979+0000 7f3b837fe640 0 client.16122 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.979+0000 7f3bb1850740 1 -- 172.21.15.2:0/130993783 >> 172.21.15.2:0/130993783 conn(0x56169e3ac300 msgr2=0x56169e74ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.980+0000 7f3bb1850740 1 -- 172.21.15.2:0/130993783 shutdown_connections 2024-05-14T09:39:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.980+0000 7f3bb1850740 1 -- 172.21.15.2:0/130993783 wait complete. 2024-05-14T09:39:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.980+0000 7f3bb1850740 20 client.16122 trim_cache size 0 max 16384 2024-05-14T09:39:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:39:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:39:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:39:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:39:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.980+0000 7f3baa994640 1 -- 172.21.15.2:0/4163825105 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3b8c0b28a0 msgr2=0x7f3b8c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.980+0000 7f3baa994640 1 --2- 172.21.15.2:0/4163825105 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3b8c0b28a0 0x7f3b8c0b4db0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009a60 tx=0x7f3b9c02f310 comp rx=0 tx=0).stop 2024-05-14T09:39:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.980+0000 7f3baa994640 1 -- 172.21.15.2:0/4163825105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba4103d90 msgr2=0x7f3ba4199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.980+0000 7f3baa994640 1 --2- 172.21.15.2:0/4163825105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba4103d90 0x7f3ba4199680 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800e9f0 tx=0x7f3b9800eeb0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.981+0000 7f3baa994640 1 -- 172.21.15.2:0/4163825105 shutdown_connections 2024-05-14T09:39:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.981+0000 7f3baa994640 1 --2- 172.21.15.2:0/4163825105 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3b8c0b28a0 0x7f3b8c0b4db0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.981+0000 7f3baa994640 1 --2- 172.21.15.2:0/4163825105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ba4108a90 0x7f3ba41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.981+0000 7f3baa994640 1 --2- 172.21.15.2:0/4163825105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba4103d90 0x7f3ba4199680 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.981+0000 7f3baa994640 1 --2- 172.21.15.2:0/4163825105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ba4102b50 0x7f3ba4199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.981+0000 7f3baa994640 1 -- 172.21.15.2:0/4163825105 >> 172.21.15.2:0/4163825105 conn(0x7f3ba40fdd60 msgr2=0x7f3ba4104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.981+0000 7f3baa994640 1 -- 172.21.15.2:0/4163825105 shutdown_connections 2024-05-14T09:39:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:39.981+0000 7f3baa994640 1 -- 172.21.15.2:0/4163825105 wait complete. 2024-05-14T09:39:40.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 362386 10.13 MB/sec execute 66 sec latency 66.782 ms 2024-05-14T09:39:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:40 smithi002 ceph-mon[31557]: pgmap v565: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 329 op/s 2024-05-14T09:39:40.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:40 smithi029 ceph-mon[36249]: pgmap v565: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 329 op/s 2024-05-14T09:39:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:40 smithi026 ceph-mon[36056]: pgmap v565: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 329 op/s 2024-05-14T09:39:41.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 364804 10.16 MB/sec execute 67 sec latency 18.553 ms 2024-05-14T09:39:42.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 366476 10.14 MB/sec execute 68 sec latency 9.935 ms 2024-05-14T09:39:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:42 smithi002 ceph-mon[31557]: pgmap v566: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 294 op/s 2024-05-14T09:39:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:42 smithi029 ceph-mon[36249]: pgmap v566: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 294 op/s 2024-05-14T09:39:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:42 smithi026 ceph-mon[36056]: pgmap v566: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 294 op/s 2024-05-14T09:39:43.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 369121 10.24 MB/sec execute 69 sec latency 12.915 ms 2024-05-14T09:39:44.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 370406 10.17 MB/sec execute 70 sec latency 20.158 ms 2024-05-14T09:39:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:44 smithi002 ceph-mon[31557]: pgmap v567: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 391 op/s 2024-05-14T09:39:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:39:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:39:44.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:39:44.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:39:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:44 smithi029 ceph-mon[36249]: pgmap v567: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 391 op/s 2024-05-14T09:39:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:39:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:39:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:39:44.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:39:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:44 smithi026 ceph-mon[36056]: pgmap v567: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 391 op/s 2024-05-14T09:39:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:39:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:39:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:39:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:39:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:39:45.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 372587 10.20 MB/sec execute 71 sec latency 70.890 ms 2024-05-14T09:39:45.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:45 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T09:39:45.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:45 smithi002 ceph-mon[31557]: Unable to set osd_memory_target on smithi002 to 185877708: error parsing value: Value '185877708' is below minimum 939524096 2024-05-14T09:39:45.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:45 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-05-14T09:39:45.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:45 smithi002 ceph-mon[31557]: Unable to set osd_memory_target on smithi026 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-14T09:39:45.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:45 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T09:39:45.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:45 smithi029 ceph-mon[36249]: Unable to set osd_memory_target on smithi002 to 185877708: error parsing value: Value '185877708' is below minimum 939524096 2024-05-14T09:39:45.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:45 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-05-14T09:39:45.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:45 smithi029 ceph-mon[36249]: Unable to set osd_memory_target on smithi026 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-14T09:39:45.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:45 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T09:39:45.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:45 smithi026 ceph-mon[36056]: Unable to set osd_memory_target on smithi002 to 185877708: error parsing value: Value '185877708' is below minimum 939524096 2024-05-14T09:39:45.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:45 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-05-14T09:39:45.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:45 smithi026 ceph-mon[36056]: Unable to set osd_memory_target on smithi026 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-14T09:39:46.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 373884 10.15 MB/sec execute 72 sec latency 18.900 ms 2024-05-14T09:39:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:46 smithi002 ceph-mon[31557]: pgmap v568: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 273 op/s 2024-05-14T09:39:46.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:46 smithi029 ceph-mon[36249]: pgmap v568: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 273 op/s 2024-05-14T09:39:46.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:46 smithi026 ceph-mon[36056]: pgmap v568: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 273 op/s 2024-05-14T09:39:47.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 376254 10.20 MB/sec execute 73 sec latency 22.643 ms 2024-05-14T09:39:48.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 378052 10.14 MB/sec execute 74 sec latency 12.773 ms 2024-05-14T09:39:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:48 smithi002 ceph-mon[31557]: pgmap v569: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-05-14T09:39:48.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:48 smithi029 ceph-mon[36249]: pgmap v569: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-05-14T09:39:48.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:48 smithi026 ceph-mon[36056]: pgmap v569: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-05-14T09:39:49.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 379899 10.19 MB/sec execute 75 sec latency 12.370 ms 2024-05-14T09:39:50.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 381542 10.12 MB/sec execute 76 sec latency 34.700 ms 2024-05-14T09:39:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:50 smithi002 ceph-mon[31557]: pgmap v570: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 357 op/s 2024-05-14T09:39:50.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:50 smithi029 ceph-mon[36249]: pgmap v570: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 357 op/s 2024-05-14T09:39:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:50 smithi026 ceph-mon[36056]: pgmap v570: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 357 op/s 2024-05-14T09:39:51.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 383422 10.17 MB/sec execute 77 sec latency 27.660 ms 2024-05-14T09:39:52.262 INFO:tasks.workunit.client.0.smithi002.stdout: 1 385268 10.11 MB/sec execute 78 sec latency 18.158 ms 2024-05-14T09:39:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:52 smithi002 ceph-mon[31557]: pgmap v571: 129 pgs: 129 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 329 op/s 2024-05-14T09:39:52.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:52 smithi029 ceph-mon[36249]: pgmap v571: 129 pgs: 129 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 329 op/s 2024-05-14T09:39:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:52 smithi026 ceph-mon[36056]: pgmap v571: 129 pgs: 129 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 329 op/s 2024-05-14T09:39:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.088+0000 7f17512c6640 1 -- 172.21.15.2:0/2420587596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f174c106d70 msgr2=0x7f174c109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.088+0000 7f17512c6640 1 --2- 172.21.15.2:0/2420587596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f174c106d70 0x7f174c109250 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1740009900 tx=0x7f174002a410 comp rx=0 tx=0).stop 2024-05-14T09:39:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.088+0000 7f17512c6640 1 -- 172.21.15.2:0/2420587596 shutdown_connections 2024-05-14T09:39:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.088+0000 7f17512c6640 1 --2- 172.21.15.2:0/2420587596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f174c1098e0 0x7f174c10bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.088+0000 7f17512c6640 1 --2- 172.21.15.2:0/2420587596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f174c106d70 0x7f174c109250 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.088+0000 7f17512c6640 1 --2- 172.21.15.2:0/2420587596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174c101f90 0x7f174c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.088+0000 7f17512c6640 1 -- 172.21.15.2:0/2420587596 >> 172.21.15.2:0/2420587596 conn(0x7f174c0fd4e0 msgr2=0x7f174c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.088+0000 7f17512c6640 1 -- 172.21.15.2:0/2420587596 shutdown_connections 2024-05-14T09:39:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.088+0000 7f17512c6640 1 -- 172.21.15.2:0/2420587596 wait complete. 2024-05-14T09:39:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.089+0000 7f17512c6640 1 Processor -- start 2024-05-14T09:39:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.089+0000 7f17512c6640 1 -- start start 2024-05-14T09:39:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.089+0000 7f17512c6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f174c101f90 0x7f174c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.089+0000 7f17512c6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f174c106d70 0x7f174c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.089+0000 7f17512c6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174c1098e0 0x7f174c1a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.089+0000 7f17512c6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f174c10f770 con 0x7f174c101f90 2024-05-14T09:39:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.089+0000 7f17512c6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f174c10f5f0 con 0x7f174c1098e0 2024-05-14T09:39:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.089+0000 7f17512c6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f174c10f8f0 con 0x7f174c106d70 2024-05-14T09:39:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.089+0000 7f174affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f174c101f90 0x7f174c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.089+0000 7f174affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f174c101f90 0x7f174c19a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59802/0 (socket says 172.21.15.2:59802) 2024-05-14T09:39:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.089+0000 7f174affd640 1 -- 172.21.15.2:0/236362698 learned_addr learned my addr 172.21.15.2:0/236362698 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.089+0000 7f174a7fc640 1 --2- 172.21.15.2:0/236362698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f174c106d70 0x7f174c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.089+0000 7f174b7fe640 1 --2- 172.21.15.2:0/236362698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174c1098e0 0x7f174c1a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.090+0000 7f174affd640 1 -- 172.21.15.2:0/236362698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f174c106d70 msgr2=0x7f174c19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.090+0000 7f174affd640 1 --2- 172.21.15.2:0/236362698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f174c106d70 0x7f174c19a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.090+0000 7f174affd640 1 -- 172.21.15.2:0/236362698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174c1098e0 msgr2=0x7f174c1a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.090+0000 7f174affd640 1 --2- 172.21.15.2:0/236362698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174c1098e0 0x7f174c1a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.090+0000 7f174affd640 1 -- 172.21.15.2:0/236362698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c1a1d90 con 0x7f174c101f90 2024-05-14T09:39:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.090+0000 7f174b7fe640 1 --2- 172.21.15.2:0/236362698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174c1098e0 0x7f174c1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:39:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.090+0000 7f174affd640 1 --2- 172.21.15.2:0/236362698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f174c101f90 0x7f174c19a250 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f173c00e9c0 tx=0x7f173c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.090+0000 7f1733fff640 1 -- 172.21.15.2:0/236362698 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c00cd80 con 0x7f174c101f90 2024-05-14T09:39:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.090+0000 7f1733fff640 1 -- 172.21.15.2:0/236362698 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f173c0054d0 con 0x7f174c101f90 2024-05-14T09:39:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.090+0000 7f1733fff640 1 -- 172.21.15.2:0/236362698 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c010430 con 0x7f174c101f90 2024-05-14T09:39:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.090+0000 7f17512c6640 1 -- 172.21.15.2:0/236362698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174c1a2080 con 0x7f174c101f90 2024-05-14T09:39:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.090+0000 7f17512c6640 1 -- 172.21.15.2:0/236362698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174c1a25d0 con 0x7f174c101f90 2024-05-14T09:39:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f175817b740 1 Processor -- start 2024-05-14T09:39:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f175817b740 1 -- start start 2024-05-14T09:39:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f1733fff640 1 -- 172.21.15.2:0/236362698 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f173c0105d0 con 0x7f174c101f90 2024-05-14T09:39:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f175817b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29871b0 0x55e1f2d2ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f175817b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f295e2f0 0x55e1f2d2d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f175817b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29664d0 0x55e1f2d342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f175817b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e1f2c8d5d0 con 0x55e1f295e2f0 2024-05-14T09:39:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f175817b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e1f2c46890 con 0x55e1f29664d0 2024-05-14T09:39:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f175817b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e1f2c89590 con 0x55e1f29871b0 2024-05-14T09:39:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f174affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f295e2f0 0x55e1f2d2d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f174a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29664d0 0x55e1f2d342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f174b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29871b0 0x55e1f2d2ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f174a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29664d0 0x55e1f2d342f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:54896/0 (socket says 172.21.15.2:54896) 2024-05-14T09:39:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f174a7fc640 1 -- 172.21.15.2:0/1561009055 learned_addr learned my addr 172.21.15.2:0/1561009055 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f174b7fe640 1 -- 172.21.15.2:0/1561009055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29664d0 msgr2=0x55e1f2d342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f174b7fe640 1 --2- 172.21.15.2:0/1561009055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29664d0 0x55e1f2d342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f174b7fe640 1 -- 172.21.15.2:0/1561009055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f295e2f0 msgr2=0x55e1f2d2d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f174b7fe640 1 --2- 172.21.15.2:0/1561009055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f295e2f0 0x55e1f2d2d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.093+0000 7f174b7fe640 1 -- 172.21.15.2:0/1561009055 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1f299f210 con 0x55e1f29871b0 2024-05-14T09:39:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.094+0000 7f174b7fe640 1 --2- 172.21.15.2:0/1561009055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29871b0 0x55e1f2d2ce70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1744009f90 tx=0x7f1744010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da7c4f24c472404c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.094+0000 7f1733fff640 1 --2- 172.21.15.2:0/236362698 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f17340b28a0 0x7f17340b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.094+0000 7f17317fa640 1 -- 172.21.15.2:0/1561009055 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744004d20 con 0x55e1f29871b0 2024-05-14T09:39:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.094+0000 7f174affd640 1 --2- 172.21.15.2:0/236362698 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f17340b28a0 0x7f17340b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.094+0000 7f17317fa640 1 -- 172.21.15.2:0/1561009055 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17440119b0 con 0x55e1f29871b0 2024-05-14T09:39:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.094+0000 7f1733fff640 1 -- 172.21.15.2:0/236362698 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f173c0e2ae0 con 0x7f174c101f90 2024-05-14T09:39:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.094+0000 7f17317fa640 1 -- 172.21.15.2:0/1561009055 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744005580 con 0x55e1f29871b0 2024-05-14T09:39:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.095+0000 7f175817b740 1 -- 172.21.15.2:0/1561009055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29871b0 msgr2=0x55e1f2d2ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.095+0000 7f174affd640 1 --2- 172.21.15.2:0/236362698 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f17340b28a0 0x7f17340b4db0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55e1f2d2e250 tx=0x7f173c0caf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.095+0000 7f175817b740 1 --2- 172.21.15.2:0/1561009055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29871b0 0x55e1f2d2ce70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1744009f90 tx=0x7f1744010860 comp rx=0 tx=0).stop 2024-05-14T09:39:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.095+0000 7f175817b740 1 -- 172.21.15.2:0/1561009055 shutdown_connections 2024-05-14T09:39:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.095+0000 7f175817b740 1 --2- 172.21.15.2:0/1561009055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29664d0 0x55e1f2d342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.095+0000 7f175817b740 1 --2- 172.21.15.2:0/1561009055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f295e2f0 0x55e1f2d2d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.095+0000 7f175817b740 1 --2- 172.21.15.2:0/1561009055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29871b0 0x55e1f2d2ce70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.095+0000 7f175817b740 1 -- 172.21.15.2:0/1561009055 >> 172.21.15.2:0/1561009055 conn(0x55e1f297b300 msgr2=0x55e1f2d243a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.095+0000 7f175817b740 1 -- 172.21.15.2:0/1561009055 shutdown_connections 2024-05-14T09:39:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.095+0000 7f175817b740 1 -- 172.21.15.2:0/1561009055 wait complete. 2024-05-14T09:39:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.095+0000 7f175817b740 1 -- start start 2024-05-14T09:39:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.095+0000 7f175817b740 1 Processor -- start 2024-05-14T09:39:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f175817b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f295e2f0 0x55e1f2d36ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f175817b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29664d0 0x55e1f2d36ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f175817b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29871b0 0x55e1f2d3df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f175817b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e1f2d34d40 con 0x55e1f295e2f0 2024-05-14T09:39:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f175817b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e1f2c4a370 con 0x55e1f29871b0 2024-05-14T09:39:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f175817b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e1f2d34ec0 con 0x55e1f29664d0 2024-05-14T09:39:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f175817b740 10 client.0 _ll_register_callbacks cb 0x55e1f299ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:39:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f174b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f295e2f0 0x55e1f2d36ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f174a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29664d0 0x55e1f2d36ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f174a7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29664d0 0x55e1f2d36ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:37194/0 (socket says 172.21.15.2:37194) 2024-05-14T09:39:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f174a7fc640 1 -- 172.21.15.2:0/1424645146 learned_addr learned my addr 172.21.15.2:0/1424645146 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f174b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f295e2f0 0x55e1f2d36ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59808/0 (socket says 172.21.15.2:59808) 2024-05-14T09:39:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f174b7fe640 1 --2- 172.21.15.2:0/1424645146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29871b0 0x55e1f2d3df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f174a7fc640 1 -- 172.21.15.2:0/1424645146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29871b0 msgr2=0x55e1f2d3df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f174a7fc640 1 --2- 172.21.15.2:0/1424645146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29871b0 0x55e1f2d3df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f174a7fc640 1 -- 172.21.15.2:0/1424645146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f295e2f0 msgr2=0x55e1f2d36ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f174a7fc640 1 --2- 172.21.15.2:0/1424645146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f295e2f0 0x55e1f2d36ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f174a7fc640 1 -- 172.21.15.2:0/1424645146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1f2a6b620 con 0x55e1f29664d0 2024-05-14T09:39:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f174b7fe640 1 --2- 172.21.15.2:0/1424645146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f295e2f0 0x55e1f2d36ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f174a7fc640 1 --2- 172.21.15.2:0/1424645146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29664d0 0x55e1f2d36ff0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1740009230 tx=0x7f1740031e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=409446690ae55e31 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.096+0000 7f172eff5640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:39:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.097+0000 7f172eff5640 1 -- 172.21.15.2:0/1424645146 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740035040 con 0x55e1f29664d0 2024-05-14T09:39:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.097+0000 7f175817b740 1 -- 172.21.15.2:0/1424645146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17440119b0 con 0x55e1f29664d0 2024-05-14T09:39:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.097+0000 7f172eff5640 1 -- 172.21.15.2:0/1424645146 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f1740004240 con 0x55e1f29664d0 2024-05-14T09:39:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.097+0000 7f172eff5640 1 -- 172.21.15.2:0/1424645146 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f17400318f0 con 0x55e1f29664d0 2024-05-14T09:39:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.097+0000 7f175817b740 10 client.25816 fetch_fsmap learned FSMap version 31 2024-05-14T09:39:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.097+0000 7f175817b740 1 -- 172.21.15.2:0/1424645146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e1f2d40f40 con 0x55e1f29664d0 2024-05-14T09:39:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.097+0000 7f172eff5640 1 -- 172.21.15.2:0/1424645146 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17440119b0 con 0x55e1f29664d0 2024-05-14T09:39:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.097+0000 7f172eff5640 1 -- 172.21.15.2:0/1424645146 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f1740044020 con 0x55e1f29664d0 2024-05-14T09:39:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.097+0000 7f175817b740 10 client.25816 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T09:39:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.097+0000 7f175817b740 10 client.25816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:39:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.097+0000 7f175817b740 20 client.25816 populate_metadata read hostname 'smithi002' 2024-05-14T09:39:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.097+0000 7f175817b740 1 --2- 172.21.15.2:0/1424645146 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e1f2945d80 0x55e1f2d44630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.097+0000 7f175817b740 4 client.25816 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:39:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.097+0000 7f175817b740 1 -- 172.21.15.2:0/1424645146 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1f2c9b1e0 con 0x55e1f2945d80 2024-05-14T09:39:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.098+0000 7f174b7fe640 1 --2- 172.21.15.2:0/1424645146 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e1f2945d80 0x55e1f2d44630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.098+0000 7f174b7fe640 1 --2- 172.21.15.2:0/1424645146 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e1f2945d80 0x55e1f2d44630 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f5a81630060ca15 server_cookie=8e7405450c3e329c in_seq=0 out_seq=0 2024-05-14T09:39:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.098+0000 7f172eff5640 10 client.25816 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.099+0000 7f172eff5640 1 -- 172.21.15.2:0/1424645146 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e1f2c9b1e0 con 0x55e1f2945d80 2024-05-14T09:39:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.099+0000 7f172eff5640 10 client.25816 handle_command_reply: tid=0 2024-05-14T09:39:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.099+0000 7f175817b740 1 client.25816 shutdown 2024-05-14T09:39:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.099+0000 7f175817b740 1 -- 172.21.15.2:0/1424645146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29664d0 msgr2=0x55e1f2d36ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.099+0000 7f175817b740 1 --2- 172.21.15.2:0/1424645146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29664d0 0x55e1f2d36ff0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1740009230 tx=0x7f1740031e80 comp rx=0 tx=0).stop 2024-05-14T09:39:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.099+0000 7f175817b740 1 -- 172.21.15.2:0/1424645146 shutdown_connections 2024-05-14T09:39:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.099+0000 7f175817b740 1 --2- 172.21.15.2:0/1424645146 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e1f2945d80 0x55e1f2d44630 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.099+0000 7f175817b740 1 --2- 172.21.15.2:0/1424645146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29871b0 0x55e1f2d3df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.099+0000 7f175817b740 1 --2- 172.21.15.2:0/1424645146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29664d0 0x55e1f2d36ff0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.099+0000 7f172eff5640 0 client.25816 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.099+0000 7f175817b740 1 --2- 172.21.15.2:0/1424645146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f295e2f0 0x55e1f2d36ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.099+0000 7f175817b740 1 -- 172.21.15.2:0/1424645146 >> 172.21.15.2:0/1424645146 conn(0x55e1f297b300 msgr2=0x55e1f2d233a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.099+0000 7f175817b740 1 -- 172.21.15.2:0/1424645146 shutdown_connections 2024-05-14T09:39:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.099+0000 7f175817b740 1 -- 172.21.15.2:0/1424645146 wait complete. 2024-05-14T09:39:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.099+0000 7f175817b740 20 client.25816 trim_cache size 0 max 16384 2024-05-14T09:39:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.116+0000 7f175817b740 1 Processor -- start 2024-05-14T09:39:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.116+0000 7f175817b740 1 -- start start 2024-05-14T09:39:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.116+0000 7f175817b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f2945d80 0x55e1f2d24400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.116+0000 7f175817b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f295e2f0 0x55e1f2d3de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.116+0000 7f175817b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29664d0 0x55e1f2d3e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.116+0000 7f175817b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1ad290 con 0x55e1f2945d80 2024-05-14T09:39:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.116+0000 7f175817b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1aab60 con 0x55e1f295e2f0 2024-05-14T09:39:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.116+0000 7f175817b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1a89b0 con 0x55e1f29664d0 2024-05-14T09:39:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.116+0000 7f174b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f2945d80 0x55e1f2d24400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.116+0000 7f174a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f295e2f0 0x55e1f2d3de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.116+0000 7f174b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f2945d80 0x55e1f2d24400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59816/0 (socket says 172.21.15.2:59816) 2024-05-14T09:39:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.116+0000 7f174b7fe640 1 -- 172.21.15.2:0/1861732 learned_addr learned my addr 172.21.15.2:0/1861732 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.116+0000 7f174a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f295e2f0 0x55e1f2d3de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:54912/0 (socket says 172.21.15.2:54912) 2024-05-14T09:39:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.116+0000 7f174b7fe640 1 --2- 172.21.15.2:0/1861732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29664d0 0x55e1f2d3e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.117+0000 7f174a7fc640 1 -- 172.21.15.2:0/1861732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29664d0 msgr2=0x55e1f2d3e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.117+0000 7f174a7fc640 1 --2- 172.21.15.2:0/1861732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29664d0 0x55e1f2d3e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.117+0000 7f174a7fc640 1 -- 172.21.15.2:0/1861732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f2945d80 msgr2=0x55e1f2d24400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.117+0000 7f174a7fc640 1 --2- 172.21.15.2:0/1861732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f2945d80 0x55e1f2d24400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.117+0000 7f174a7fc640 1 -- 172.21.15.2:0/1861732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1f2d24940 con 0x55e1f295e2f0 2024-05-14T09:39:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.117+0000 7f174b7fe640 1 --2- 172.21.15.2:0/1861732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f2945d80 0x55e1f2d24400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.117+0000 7f174a7fc640 1 --2- 172.21.15.2:0/1861732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f295e2f0 0x55e1f2d3de70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1740046770 tx=0x7f174002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a525cec595bf8c2e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.117+0000 7f1730ff9640 1 -- 172.21.15.2:0/1861732 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740035040 con 0x55e1f295e2f0 2024-05-14T09:39:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.117+0000 7f1730ff9640 1 -- 172.21.15.2:0/1861732 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17400469f0 con 0x55e1f295e2f0 2024-05-14T09:39:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.117+0000 7f1730ff9640 1 -- 172.21.15.2:0/1861732 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740003b20 con 0x55e1f295e2f0 2024-05-14T09:39:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.118+0000 7f175817b740 1 -- 172.21.15.2:0/1861732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f295e2f0 msgr2=0x55e1f2d3de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.118+0000 7f175817b740 1 --2- 172.21.15.2:0/1861732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f295e2f0 0x55e1f2d3de70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1740046770 tx=0x7f174002a410 comp rx=0 tx=0).stop 2024-05-14T09:39:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.118+0000 7f175817b740 1 -- 172.21.15.2:0/1861732 shutdown_connections 2024-05-14T09:39:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.118+0000 7f175817b740 1 --2- 172.21.15.2:0/1861732 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f29664d0 0x55e1f2d3e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.118+0000 7f175817b740 1 --2- 172.21.15.2:0/1861732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f295e2f0 0x55e1f2d3de70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.118+0000 7f175817b740 1 --2- 172.21.15.2:0/1861732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f2945d80 0x55e1f2d24400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.118+0000 7f175817b740 1 -- 172.21.15.2:0/1861732 >> 172.21.15.2:0/1861732 conn(0x55e1f297b300 msgr2=0x55e1f2d23cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.118+0000 7f175817b740 1 -- 172.21.15.2:0/1861732 shutdown_connections 2024-05-14T09:39:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.118+0000 7f175817b740 1 -- 172.21.15.2:0/1861732 wait complete. 2024-05-14T09:39:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.118+0000 7f175817b740 1 -- start start 2024-05-14T09:39:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.118+0000 7f175817b740 1 Processor -- start 2024-05-14T09:39:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f175817b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f2945d80 0x55e1f2d3b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f175817b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f295e2f0 0x55e1f2d3bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f175817b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29664d0 0x55e1f2daa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f175817b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1a9020 con 0x55e1f2945d80 2024-05-14T09:39:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f175817b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1aa490 con 0x55e1f29664d0 2024-05-14T09:39:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f175817b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1a96f0 con 0x55e1f295e2f0 2024-05-14T09:39:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f175817b740 10 client.0 _ll_register_callbacks cb 0x7f1740009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:39:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f174b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f2945d80 0x55e1f2d3b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f174b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f2945d80 0x55e1f2d3b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59824/0 (socket says 172.21.15.2:59824) 2024-05-14T09:39:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f174b7fe640 1 -- 172.21.15.2:0/1907969962 learned_addr learned my addr 172.21.15.2:0/1907969962 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:39:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f174a7fc640 1 --2- 172.21.15.2:0/1907969962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f295e2f0 0x55e1f2d3bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f174b7fe640 1 --2- 172.21.15.2:0/1907969962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29664d0 0x55e1f2daa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f174b7fe640 1 -- 172.21.15.2:0/1907969962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f295e2f0 msgr2=0x55e1f2d3bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f174b7fe640 1 --2- 172.21.15.2:0/1907969962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f295e2f0 0x55e1f2d3bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f174b7fe640 1 -- 172.21.15.2:0/1907969962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29664d0 msgr2=0x55e1f2daa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f174b7fe640 1 --2- 172.21.15.2:0/1907969962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29664d0 0x55e1f2daa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f174b7fe640 1 -- 172.21.15.2:0/1907969962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1f2daaf40 con 0x55e1f2945d80 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f174a7fc640 1 --2- 172.21.15.2:0/1907969962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f295e2f0 0x55e1f2d3bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f174b7fe640 1 --2- 172.21.15.2:0/1907969962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f2945d80 0x55e1f2d3b790 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1744009f90 tx=0x7f1744009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1077be4208fbe9d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f172eff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.119+0000 7f172eff5640 1 -- 172.21.15.2:0/1907969962 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174402c580 con 0x55e1f2945d80 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.120+0000 7f175817b740 1 -- 172.21.15.2:0/1907969962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17400469f0 con 0x55e1f2945d80 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.120+0000 7f172eff5640 1 -- 172.21.15.2:0/1907969962 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f174402c720 con 0x55e1f2945d80 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.120+0000 7f172eff5640 1 -- 172.21.15.2:0/1907969962 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f174402c920 con 0x55e1f2945d80 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.120+0000 7f175817b740 10 client.16140 fetch_fsmap learned FSMap version 31 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.120+0000 7f175817b740 1 -- 172.21.15.2:0/1907969962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e1f2dac290 con 0x55e1f2945d80 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.120+0000 7f172eff5640 1 -- 172.21.15.2:0/1907969962 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17400469f0 con 0x55e1f2945d80 2024-05-14T09:39:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.120+0000 7f172eff5640 1 -- 172.21.15.2:0/1907969962 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f1744038030 con 0x55e1f2945d80 2024-05-14T09:39:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.120+0000 7f175817b740 10 client.16140 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T09:39:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.120+0000 7f175817b740 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:39:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.120+0000 7f175817b740 20 client.16140 populate_metadata read hostname 'smithi002' 2024-05-14T09:39:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.120+0000 7f175817b740 1 --2- 172.21.15.2:0/1907969962 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e1f29871b0 0x55e1f2dae970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:39:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.120+0000 7f175817b740 4 client.16140 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:39:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.120+0000 7f175817b740 1 -- 172.21.15.2:0/1907969962 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1f2c95050 con 0x55e1f29871b0 2024-05-14T09:39:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.120+0000 7f174a7fc640 1 --2- 172.21.15.2:0/1907969962 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e1f29871b0 0x55e1f2dae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:39:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.121+0000 7f174a7fc640 1 --2- 172.21.15.2:0/1907969962 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e1f29871b0 0x55e1f2dae970 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e8341b0bc905a2e server_cookie=d65c1ea21a512d32 in_seq=0 out_seq=0 2024-05-14T09:39:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.121+0000 7f172eff5640 10 client.16140 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:53.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.226+0000 7f172eff5640 1 -- 172.21.15.2:0/1907969962 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e1f2c95050 con 0x55e1f29871b0 2024-05-14T09:39:53.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.226+0000 7f172eff5640 10 client.16140 handle_command_reply: tid=0 2024-05-14T09:39:53.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.226+0000 7f175817b740 1 client.16140 shutdown 2024-05-14T09:39:53.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.226+0000 7f175817b740 1 -- 172.21.15.2:0/1907969962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f2945d80 msgr2=0x55e1f2d3b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.226+0000 7f175817b740 1 --2- 172.21.15.2:0/1907969962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f2945d80 0x55e1f2d3b790 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1744009f90 tx=0x7f1744009500 comp rx=0 tx=0).stop 2024-05-14T09:39:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.226+0000 7f175817b740 1 -- 172.21.15.2:0/1907969962 shutdown_connections 2024-05-14T09:39:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.226+0000 7f175817b740 1 --2- 172.21.15.2:0/1907969962 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e1f29871b0 0x55e1f2dae970 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.226+0000 7f175817b740 1 --2- 172.21.15.2:0/1907969962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e1f29664d0 0x55e1f2daa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.226+0000 7f172eff5640 0 client.16140 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:39:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.226+0000 7f175817b740 1 --2- 172.21.15.2:0/1907969962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1f295e2f0 0x55e1f2d3bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.226+0000 7f175817b740 1 --2- 172.21.15.2:0/1907969962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1f2945d80 0x55e1f2d3b790 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.226+0000 7f175817b740 1 -- 172.21.15.2:0/1907969962 >> 172.21.15.2:0/1907969962 conn(0x55e1f297b300 msgr2=0x55e1f2d271f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.226+0000 7f175817b740 1 -- 172.21.15.2:0/1907969962 shutdown_connections 2024-05-14T09:39:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.226+0000 7f175817b740 1 -- 172.21.15.2:0/1907969962 wait complete. 2024-05-14T09:39:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.226+0000 7f175817b740 20 client.16140 trim_cache size 0 max 16384 2024-05-14T09:39:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:39:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:39:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:39:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:39:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.227+0000 7f17512c6640 1 -- 172.21.15.2:0/236362698 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f17340b28a0 msgr2=0x7f17340b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.227+0000 7f17512c6640 1 --2- 172.21.15.2:0/236362698 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f17340b28a0 0x7f17340b4db0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55e1f2d2e250 tx=0x7f173c0caf90 comp rx=0 tx=0).stop 2024-05-14T09:39:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.227+0000 7f17512c6640 1 -- 172.21.15.2:0/236362698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f174c101f90 msgr2=0x7f174c19a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:39:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.227+0000 7f17512c6640 1 --2- 172.21.15.2:0/236362698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f174c101f90 0x7f174c19a250 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f173c00e9c0 tx=0x7f173c00ee80 comp rx=0 tx=0).stop 2024-05-14T09:39:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.227+0000 7f17512c6640 1 -- 172.21.15.2:0/236362698 shutdown_connections 2024-05-14T09:39:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.227+0000 7f17512c6640 1 --2- 172.21.15.2:0/236362698 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f17340b28a0 0x7f17340b4db0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.227+0000 7f17512c6640 1 --2- 172.21.15.2:0/236362698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174c1098e0 0x7f174c1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.227+0000 7f17512c6640 1 --2- 172.21.15.2:0/236362698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f174c106d70 0x7f174c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.227+0000 7f17512c6640 1 --2- 172.21.15.2:0/236362698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f174c101f90 0x7f174c19a250 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:39:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.227+0000 7f17512c6640 1 -- 172.21.15.2:0/236362698 >> 172.21.15.2:0/236362698 conn(0x7f174c0fd4e0 msgr2=0x7f174c107cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:39:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.227+0000 7f17512c6640 1 -- 172.21.15.2:0/236362698 shutdown_connections 2024-05-14T09:39:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:39:53.227+0000 7f17512c6640 1 -- 172.21.15.2:0/236362698 wait complete. 2024-05-14T09:39:53.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 387001 10.15 MB/sec execute 79 sec latency 91.372 ms 2024-05-14T09:39:54.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 388876 10.10 MB/sec execute 80 sec latency 14.022 ms 2024-05-14T09:39:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:54 smithi002 ceph-mon[31557]: pgmap v572: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2024-05-14T09:39:54.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:54 smithi029 ceph-mon[36249]: pgmap v572: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2024-05-14T09:39:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:54 smithi026 ceph-mon[36056]: pgmap v572: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2024-05-14T09:39:55.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 390547 10.14 MB/sec execute 81 sec latency 67.540 ms 2024-05-14T09:39:56.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 392361 10.09 MB/sec execute 82 sec latency 28.444 ms 2024-05-14T09:39:56.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:56 smithi002 ceph-mon[31557]: pgmap v573: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-05-14T09:39:56.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:56 smithi029 ceph-mon[36249]: pgmap v573: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-05-14T09:39:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:56 smithi026 ceph-mon[36056]: pgmap v573: 129 pgs: 129 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-05-14T09:39:57.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 394146 10.13 MB/sec execute 83 sec latency 23.546 ms 2024-05-14T09:39:58.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 396103 10.09 MB/sec execute 84 sec latency 79.949 ms 2024-05-14T09:39:58.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:39:58 smithi002 ceph-mon[31557]: pgmap v574: 129 pgs: 129 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 335 op/s 2024-05-14T09:39:58.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:39:58 smithi029 ceph-mon[36249]: pgmap v574: 129 pgs: 129 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 335 op/s 2024-05-14T09:39:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:39:58 smithi026 ceph-mon[36056]: pgmap v574: 129 pgs: 129 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 335 op/s 2024-05-14T09:39:59.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 397569 10.11 MB/sec execute 85 sec latency 186.961 ms 2024-05-14T09:40:00.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 398815 10.05 MB/sec execute 86 sec latency 90.460 ms 2024-05-14T09:40:00.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a[31553]: 2024-05-14T09:39:59.998+0000 7f82afa15700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-14T09:40:00.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a[31553]: 2024-05-14T09:39:59.998+0000 7f82afa15700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-14T09:40:00.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a[31553]: 2024-05-14T09:39:59.998+0000 7f82afa15700 -1 log_channel(cluster) log [ERR] : mds.h(mds.1): Metadata damage detected 2024-05-14T09:40:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:00 smithi002 ceph-mon[31557]: pgmap v575: 129 pgs: 129 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 305 op/s 2024-05-14T09:40:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:00 smithi002 ceph-mon[31557]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-14T09:40:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:00 smithi002 ceph-mon[31557]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-14T09:40:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:00 smithi002 ceph-mon[31557]: mds.h(mds.1): Metadata damage detected 2024-05-14T09:40:00.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:00 smithi029 ceph-mon[36249]: pgmap v575: 129 pgs: 129 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 305 op/s 2024-05-14T09:40:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:00 smithi029 ceph-mon[36249]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-14T09:40:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:00 smithi029 ceph-mon[36249]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-14T09:40:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:00 smithi029 ceph-mon[36249]: mds.h(mds.1): Metadata damage detected 2024-05-14T09:40:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:00 smithi026 ceph-mon[36056]: pgmap v575: 129 pgs: 129 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 305 op/s 2024-05-14T09:40:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:00 smithi026 ceph-mon[36056]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-14T09:40:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:00 smithi026 ceph-mon[36056]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-14T09:40:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:00 smithi026 ceph-mon[36056]: mds.h(mds.1): Metadata damage detected 2024-05-14T09:40:01.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 401099 10.10 MB/sec execute 87 sec latency 31.326 ms 2024-05-14T09:40:02.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 402635 10.05 MB/sec execute 88 sec latency 14.871 ms 2024-05-14T09:40:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:02 smithi002 ceph-mon[31557]: pgmap v576: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 324 op/s 2024-05-14T09:40:02.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:02 smithi029 ceph-mon[36249]: pgmap v576: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 324 op/s 2024-05-14T09:40:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:02 smithi026 ceph-mon[36056]: pgmap v576: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 324 op/s 2024-05-14T09:40:03.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 404772 10.09 MB/sec execute 89 sec latency 19.811 ms 2024-05-14T09:40:04.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 406790 10.05 MB/sec execute 90 sec latency 80.626 ms 2024-05-14T09:40:04.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:04 smithi002 ceph-mon[31557]: pgmap v577: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 324 op/s 2024-05-14T09:40:04.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:04 smithi029 ceph-mon[36249]: pgmap v577: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 324 op/s 2024-05-14T09:40:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:04 smithi026 ceph-mon[36056]: pgmap v577: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 324 op/s 2024-05-14T09:40:05.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 408344 10.08 MB/sec execute 91 sec latency 70.576 ms 2024-05-14T09:40:06.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 410317 10.04 MB/sec execute 92 sec latency 71.748 ms 2024-05-14T09:40:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.333+0000 7f47814e0640 1 -- 172.21.15.2:0/2028836563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c103d90 msgr2=0x7f477c1041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.333+0000 7f47814e0640 1 --2- 172.21.15.2:0/2028836563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c103d90 0x7f477c1041a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4770009900 tx=0x7f477002a410 comp rx=0 tx=0).stop 2024-05-14T09:40:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.333+0000 7f47814e0640 1 -- 172.21.15.2:0/2028836563 shutdown_connections 2024-05-14T09:40:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.333+0000 7f47814e0640 1 --2- 172.21.15.2:0/2028836563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f477c108a90 0x7f477c10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.333+0000 7f47814e0640 1 --2- 172.21.15.2:0/2028836563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c103d90 0x7f477c1041a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.333+0000 7f47814e0640 1 --2- 172.21.15.2:0/2028836563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c102b50 0x7f477c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.333+0000 7f47814e0640 1 -- 172.21.15.2:0/2028836563 >> 172.21.15.2:0/2028836563 conn(0x7f477c0fdd40 msgr2=0x7f477c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.333+0000 7f47814e0640 1 -- 172.21.15.2:0/2028836563 shutdown_connections 2024-05-14T09:40:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.333+0000 7f47814e0640 1 -- 172.21.15.2:0/2028836563 wait complete. 2024-05-14T09:40:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f47814e0640 1 Processor -- start 2024-05-14T09:40:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f47814e0640 1 -- start start 2024-05-14T09:40:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f47814e0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c102b50 0x7f477c19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f47814e0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f477c103d90 0x7f477c19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f47814e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c108a90 0x7f477c1a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f47814e0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f477c10e680 con 0x7f477c102b50 2024-05-14T09:40:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f47814e0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f477c10e500 con 0x7f477c108a90 2024-05-14T09:40:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f47814e0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f477c10e800 con 0x7f477c103d90 2024-05-14T09:40:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f477affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c102b50 0x7f477c19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f477affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c102b50 0x7f477c19c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35824/0 (socket says 172.21.15.2:35824) 2024-05-14T09:40:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f477affd640 1 -- 172.21.15.2:0/2645718546 learned_addr learned my addr 172.21.15.2:0/2645718546 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f477a7fc640 1 --2- 172.21.15.2:0/2645718546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f477c103d90 0x7f477c19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f477b7fe640 1 --2- 172.21.15.2:0/2645718546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c108a90 0x7f477c1a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f477affd640 1 -- 172.21.15.2:0/2645718546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f477c103d90 msgr2=0x7f477c19c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f477affd640 1 --2- 172.21.15.2:0/2645718546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f477c103d90 0x7f477c19c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f477affd640 1 -- 172.21.15.2:0/2645718546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c108a90 msgr2=0x7f477c1a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f477affd640 1 --2- 172.21.15.2:0/2645718546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c108a90 0x7f477c1a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.334+0000 7f477affd640 1 -- 172.21.15.2:0/2645718546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477c1a3fa0 con 0x7f477c102b50 2024-05-14T09:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.335+0000 7f477affd640 1 --2- 172.21.15.2:0/2645718546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c102b50 0x7f477c19c460 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e9f0 tx=0x7f476c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.335+0000 7f4763fff640 1 -- 172.21.15.2:0/2645718546 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c00ccc0 con 0x7f477c102b50 2024-05-14T09:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.335+0000 7f4763fff640 1 -- 172.21.15.2:0/2645718546 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f476c0035f0 con 0x7f477c102b50 2024-05-14T09:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.335+0000 7f47814e0640 1 -- 172.21.15.2:0/2645718546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477c1a4290 con 0x7f477c102b50 2024-05-14T09:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.335+0000 7f4763fff640 1 -- 172.21.15.2:0/2645718546 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c00ce60 con 0x7f477c102b50 2024-05-14T09:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.335+0000 7f47814e0640 1 -- 172.21.15.2:0/2645718546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f477c1a47e0 con 0x7f477c102b50 2024-05-14T09:40:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.337+0000 7f4788396740 1 Processor -- start 2024-05-14T09:40:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f4788396740 1 -- start start 2024-05-14T09:40:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f4763fff640 1 -- 172.21.15.2:0/2645718546 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f476c003aa0 con 0x7f477c102b50 2024-05-14T09:40:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f4788396740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cba81b0 0x56376cf49a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f4788396740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cb7f2f0 0x56376cf49fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f4788396740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb874d0 0x56376cf50ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f4788396740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56376ceae5d0 con 0x56376cba81b0 2024-05-14T09:40:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f4788396740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56376ce67890 con 0x56376cb7f2f0 2024-05-14T09:40:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f4788396740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56376ceaa590 con 0x56376cb874d0 2024-05-14T09:40:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f477b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cba81b0 0x56376cf49a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f477b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cba81b0 0x56376cf49a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35840/0 (socket says 172.21.15.2:35840) 2024-05-14T09:40:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f477affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cb7f2f0 0x56376cf49fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f477b7fe640 1 -- 172.21.15.2:0/4126568137 learned_addr learned my addr 172.21.15.2:0/4126568137 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f477a7fc640 1 --2- 172.21.15.2:0/4126568137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb874d0 0x56376cf50ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f477b7fe640 1 -- 172.21.15.2:0/4126568137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb874d0 msgr2=0x56376cf50ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f477b7fe640 1 --2- 172.21.15.2:0/4126568137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb874d0 0x56376cf50ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f477b7fe640 1 -- 172.21.15.2:0/4126568137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cb7f2f0 msgr2=0x56376cf49fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f477b7fe640 1 --2- 172.21.15.2:0/4126568137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cb7f2f0 0x56376cf49fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f477b7fe640 1 -- 172.21.15.2:0/4126568137 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56376cbc0210 con 0x56376cba81b0 2024-05-14T09:40:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.338+0000 7f477b7fe640 1 --2- 172.21.15.2:0/4126568137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cba81b0 0x56376cf49a60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4774009fd0 tx=0x7f47740118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123be6d9ec729d5f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.339+0000 7f47617fa640 1 -- 172.21.15.2:0/4126568137 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774004d20 con 0x56376cba81b0 2024-05-14T09:40:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.339+0000 7f4763fff640 1 --2- 172.21.15.2:0/2645718546 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f47640b28a0 0x7f47640b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.339+0000 7f47617fa640 1 -- 172.21.15.2:0/4126568137 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4774004050 con 0x56376cba81b0 2024-05-14T09:40:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.339+0000 7f47617fa640 1 -- 172.21.15.2:0/4126568137 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774005240 con 0x56376cba81b0 2024-05-14T09:40:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.339+0000 7f477affd640 1 --2- 172.21.15.2:0/2645718546 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f47640b28a0 0x7f47640b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.339+0000 7f4763fff640 1 -- 172.21.15.2:0/2645718546 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f476c0e2910 con 0x7f477c102b50 2024-05-14T09:40:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.340+0000 7f4788396740 1 -- 172.21.15.2:0/4126568137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cba81b0 msgr2=0x56376cf49a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.340+0000 7f477affd640 1 --2- 172.21.15.2:0/2645718546 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f47640b28a0 0x7f47640b4db0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x56376cf4ae40 tx=0x7f476c0e8550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.340+0000 7f4788396740 1 --2- 172.21.15.2:0/4126568137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cba81b0 0x56376cf49a60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4774009fd0 tx=0x7f47740118d0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.340+0000 7f4788396740 1 -- 172.21.15.2:0/4126568137 shutdown_connections 2024-05-14T09:40:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.340+0000 7f4788396740 1 --2- 172.21.15.2:0/4126568137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb874d0 0x56376cf50ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.340+0000 7f4788396740 1 --2- 172.21.15.2:0/4126568137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cb7f2f0 0x56376cf49fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.340+0000 7f4788396740 1 --2- 172.21.15.2:0/4126568137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cba81b0 0x56376cf49a60 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.340+0000 7f4788396740 1 -- 172.21.15.2:0/4126568137 >> 172.21.15.2:0/4126568137 conn(0x56376cb9c300 msgr2=0x56376cf453a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.340+0000 7f4788396740 1 -- 172.21.15.2:0/4126568137 shutdown_connections 2024-05-14T09:40:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.340+0000 7f4788396740 1 -- 172.21.15.2:0/4126568137 wait complete. 2024-05-14T09:40:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.340+0000 7f4788396740 1 -- start start 2024-05-14T09:40:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.340+0000 7f4788396740 1 Processor -- start 2024-05-14T09:40:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f4788396740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb7f2f0 0x56376cf536a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f4788396740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cb874d0 0x56376cf53be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f4788396740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cba81b0 0x56376cf5aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f4788396740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56376cf51930 con 0x56376cb874d0 2024-05-14T09:40:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f4788396740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56376ce6b370 con 0x56376cba81b0 2024-05-14T09:40:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f4788396740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56376cf51ab0 con 0x56376cb7f2f0 2024-05-14T09:40:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f4788396740 10 client.0 _ll_register_callbacks cb 0x56376cbbfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:40:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f477a7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cb874d0 0x56376cf53be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f477a7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cb874d0 0x56376cf53be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35848/0 (socket says 172.21.15.2:35848) 2024-05-14T09:40:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f477a7fc640 1 -- 172.21.15.2:0/2971179315 learned_addr learned my addr 172.21.15.2:0/2971179315 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f477b7fe640 1 --2- 172.21.15.2:0/2971179315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb7f2f0 0x56376cf536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f477b7fe640 1 --2- 172.21.15.2:0/2971179315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cba81b0 0x56376cf5aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f477a7fc640 1 -- 172.21.15.2:0/2971179315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb7f2f0 msgr2=0x56376cf536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f477a7fc640 1 --2- 172.21.15.2:0/2971179315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb7f2f0 0x56376cf536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f477a7fc640 1 -- 172.21.15.2:0/2971179315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cba81b0 msgr2=0x56376cf5aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f477a7fc640 1 --2- 172.21.15.2:0/2971179315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cba81b0 0x56376cf5aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f477a7fc640 1 -- 172.21.15.2:0/2971179315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56376cc8c620 con 0x56376cb874d0 2024-05-14T09:40:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f477a7fc640 1 --2- 172.21.15.2:0/2971179315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cb874d0 0x56376cf53be0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f477002a930 tx=0x7f4770031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7181656206b8b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f475eff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:40:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f475eff5640 1 -- 172.21.15.2:0/2971179315 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770035040 con 0x56376cb874d0 2024-05-14T09:40:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f475eff5640 1 -- 172.21.15.2:0/2971179315 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f4770004040 con 0x56376cb874d0 2024-05-14T09:40:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.341+0000 7f4788396740 1 -- 172.21.15.2:0/2971179315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4774004050 con 0x56376cb874d0 2024-05-14T09:40:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.342+0000 7f475eff5640 1 -- 172.21.15.2:0/2971179315 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f477004e050 con 0x56376cb874d0 2024-05-14T09:40:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.342+0000 7f4788396740 10 client.16152 fetch_fsmap learned FSMap version 31 2024-05-14T09:40:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.342+0000 7f4788396740 1 -- 172.21.15.2:0/2971179315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56376cf5daa0 con 0x56376cb874d0 2024-05-14T09:40:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.342+0000 7f475eff5640 1 -- 172.21.15.2:0/2971179315 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4774004050 con 0x56376cb874d0 2024-05-14T09:40:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.342+0000 7f475eff5640 1 -- 172.21.15.2:0/2971179315 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f4770044030 con 0x56376cb874d0 2024-05-14T09:40:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.342+0000 7f4788396740 10 client.16152 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T09:40:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.342+0000 7f4788396740 10 client.16152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:40:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.342+0000 7f4788396740 20 client.16152 populate_metadata read hostname 'smithi002' 2024-05-14T09:40:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.342+0000 7f4788396740 1 --2- 172.21.15.2:0/2971179315 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56376cb66d80 0x56376cf611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.342+0000 7f4788396740 4 client.16152 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:40:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.342+0000 7f4788396740 1 -- 172.21.15.2:0/2971179315 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56376cebc1e0 con 0x56376cb66d80 2024-05-14T09:40:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.342+0000 7f477b7fe640 1 --2- 172.21.15.2:0/2971179315 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56376cb66d80 0x56376cf611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.343+0000 7f475eff5640 10 client.16152 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.343+0000 7f477b7fe640 1 --2- 172.21.15.2:0/2971179315 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56376cb66d80 0x56376cf611e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9d9ec377ce21e7e server_cookie=4407dfb631c218fd in_seq=0 out_seq=0 2024-05-14T09:40:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.343+0000 7f475eff5640 1 -- 172.21.15.2:0/2971179315 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56376cebc1e0 con 0x56376cb66d80 2024-05-14T09:40:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.344+0000 7f475eff5640 10 client.16152 handle_command_reply: tid=0 2024-05-14T09:40:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.344+0000 7f4788396740 1 client.16152 shutdown 2024-05-14T09:40:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.344+0000 7f4788396740 1 -- 172.21.15.2:0/2971179315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cb874d0 msgr2=0x56376cf53be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.344+0000 7f4788396740 1 --2- 172.21.15.2:0/2971179315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cb874d0 0x56376cf53be0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f477002a930 tx=0x7f4770031e80 comp rx=0 tx=0).stop 2024-05-14T09:40:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.344+0000 7f4788396740 1 -- 172.21.15.2:0/2971179315 shutdown_connections 2024-05-14T09:40:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.344+0000 7f4788396740 1 --2- 172.21.15.2:0/2971179315 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56376cb66d80 0x56376cf611e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.344+0000 7f4788396740 1 --2- 172.21.15.2:0/2971179315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cba81b0 0x56376cf5aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.344+0000 7f4788396740 1 --2- 172.21.15.2:0/2971179315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cb874d0 0x56376cf53be0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.344+0000 7f475eff5640 0 client.16152 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.344+0000 7f4788396740 1 --2- 172.21.15.2:0/2971179315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb7f2f0 0x56376cf536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.344+0000 7f4788396740 1 -- 172.21.15.2:0/2971179315 >> 172.21.15.2:0/2971179315 conn(0x56376cb9c300 msgr2=0x56376cf443a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.344+0000 7f4788396740 1 -- 172.21.15.2:0/2971179315 shutdown_connections 2024-05-14T09:40:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.344+0000 7f4788396740 1 -- 172.21.15.2:0/2971179315 wait complete. 2024-05-14T09:40:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.344+0000 7f4788396740 20 client.16152 trim_cache size 0 max 16384 2024-05-14T09:40:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f4788396740 1 Processor -- start 2024-05-14T09:40:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f4788396740 1 -- start start 2024-05-14T09:40:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f4788396740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cb7f2f0 0x56376cf43970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f4788396740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb874d0 0x56376cf43eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f4788396740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cba81b0 0x56376cf443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f4788396740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f477c1b16d0 con 0x56376cb7f2f0 2024-05-14T09:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f4788396740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f477c1aefa0 con 0x56376cba81b0 2024-05-14T09:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f4788396740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f477c1acdf0 con 0x56376cb874d0 2024-05-14T09:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f477b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cb7f2f0 0x56376cf43970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f477b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cb7f2f0 0x56376cf43970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35860/0 (socket says 172.21.15.2:35860) 2024-05-14T09:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f477b7fe640 1 -- 172.21.15.2:0/2835872136 learned_addr learned my addr 172.21.15.2:0/2835872136 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f477a7fc640 1 --2- 172.21.15.2:0/2835872136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb874d0 0x56376cf43eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f477b7fe640 1 --2- 172.21.15.2:0/2835872136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cba81b0 0x56376cf443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f477b7fe640 1 -- 172.21.15.2:0/2835872136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb874d0 msgr2=0x56376cf43eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f477b7fe640 1 --2- 172.21.15.2:0/2835872136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb874d0 0x56376cf43eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f477b7fe640 1 -- 172.21.15.2:0/2835872136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cba81b0 msgr2=0x56376cf443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f477b7fe640 1 --2- 172.21.15.2:0/2835872136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cba81b0 0x56376cf443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.360+0000 7f477b7fe640 1 -- 172.21.15.2:0/2835872136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56376cf49a20 con 0x56376cb7f2f0 2024-05-14T09:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.361+0000 7f477b7fe640 1 --2- 172.21.15.2:0/2835872136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cb7f2f0 0x56376cf43970 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4774009f70 tx=0x7f47740118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8a6a7f14847379 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.361+0000 7f4760ff9640 1 -- 172.21.15.2:0/2835872136 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47740042d0 con 0x56376cb7f2f0 2024-05-14T09:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.361+0000 7f4760ff9640 1 -- 172.21.15.2:0/2835872136 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4774004470 con 0x56376cb7f2f0 2024-05-14T09:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.361+0000 7f4760ff9640 1 -- 172.21.15.2:0/2835872136 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774028450 con 0x56376cb7f2f0 2024-05-14T09:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.361+0000 7f4788396740 1 -- 172.21.15.2:0/2835872136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cb7f2f0 msgr2=0x56376cf43970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.361+0000 7f4788396740 1 --2- 172.21.15.2:0/2835872136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cb7f2f0 0x56376cf43970 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4774009f70 tx=0x7f47740118d0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.361+0000 7f4788396740 1 -- 172.21.15.2:0/2835872136 shutdown_connections 2024-05-14T09:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.361+0000 7f4788396740 1 --2- 172.21.15.2:0/2835872136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cba81b0 0x56376cf443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.361+0000 7f4788396740 1 --2- 172.21.15.2:0/2835872136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb874d0 0x56376cf43eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.361+0000 7f4788396740 1 --2- 172.21.15.2:0/2835872136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cb7f2f0 0x56376cf43970 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.361+0000 7f4788396740 1 -- 172.21.15.2:0/2835872136 >> 172.21.15.2:0/2835872136 conn(0x56376cb9c300 msgr2=0x56376cf44cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f4788396740 1 -- 172.21.15.2:0/2835872136 shutdown_connections 2024-05-14T09:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f4788396740 1 -- 172.21.15.2:0/2835872136 wait complete. 2024-05-14T09:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f4788396740 1 -- start start 2024-05-14T09:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f4788396740 1 Processor -- start 2024-05-14T09:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f4788396740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb7f2f0 0x56376cf58b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f4788396740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cb874d0 0x56376cf59040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f4788396740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cba81b0 0x56376cf59f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f4788396740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f477c1ad460 con 0x56376cba81b0 2024-05-14T09:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f4788396740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f477c1ae8d0 con 0x56376cb874d0 2024-05-14T09:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f4788396740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f477c1adb30 con 0x56376cb7f2f0 2024-05-14T09:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f4788396740 10 client.0 _ll_register_callbacks cb 0x7f4770009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f477b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb7f2f0 0x56376cf58b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f477a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cb874d0 0x56376cf59040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f477b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cba81b0 0x56376cf59f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f477a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cb874d0 0x56376cf59040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:50510/0 (socket says 172.21.15.2:50510) 2024-05-14T09:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f477b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cba81b0 0x56376cf59f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35868/0 (socket says 172.21.15.2:35868) 2024-05-14T09:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.362+0000 7f477a7fc640 1 -- 172.21.15.2:0/1534459848 learned_addr learned my addr 172.21.15.2:0/1534459848 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.363+0000 7f477b7fe640 1 -- 172.21.15.2:0/1534459848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb7f2f0 msgr2=0x56376cf58b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.363+0000 7f477b7fe640 1 --2- 172.21.15.2:0/1534459848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb7f2f0 0x56376cf58b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.363+0000 7f477b7fe640 1 -- 172.21.15.2:0/1534459848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cb874d0 msgr2=0x56376cf59040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.363+0000 7f477b7fe640 1 --2- 172.21.15.2:0/1534459848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cb874d0 0x56376cf59040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.363+0000 7f477b7fe640 1 -- 172.21.15.2:0/1534459848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56376cf5a530 con 0x56376cba81b0 2024-05-14T09:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.363+0000 7f477a7fc640 1 --2- 172.21.15.2:0/1534459848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cb874d0 0x56376cf59040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.363+0000 7f477b7fe640 1 --2- 172.21.15.2:0/1534459848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cba81b0 0x56376cf59f70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4774006010 tx=0x7f4774009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8775567c3356069 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.363+0000 7f475eff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.363+0000 7f475eff5640 1 -- 172.21.15.2:0/1534459848 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774033040 con 0x56376cba81b0 2024-05-14T09:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.363+0000 7f475eff5640 1 -- 172.21.15.2:0/1534459848 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f477400f6e0 con 0x56376cba81b0 2024-05-14T09:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.363+0000 7f4788396740 1 -- 172.21.15.2:0/1534459848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4774004470 con 0x56376cba81b0 2024-05-14T09:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.363+0000 7f475eff5640 1 -- 172.21.15.2:0/1534459848 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f477400f8e0 con 0x56376cba81b0 2024-05-14T09:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.363+0000 7f4788396740 10 client.16158 fetch_fsmap learned FSMap version 31 2024-05-14T09:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.363+0000 7f4788396740 1 -- 172.21.15.2:0/1534459848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56376cfc9590 con 0x56376cba81b0 2024-05-14T09:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.363+0000 7f475eff5640 1 -- 172.21.15.2:0/1534459848 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4774004470 con 0x56376cba81b0 2024-05-14T09:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.364+0000 7f475eff5640 1 -- 172.21.15.2:0/1534459848 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f4774039070 con 0x56376cba81b0 2024-05-14T09:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.364+0000 7f4788396740 10 client.16158 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T09:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.364+0000 7f4788396740 10 client.16158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.364+0000 7f4788396740 20 client.16158 populate_metadata read hostname 'smithi002' 2024-05-14T09:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.364+0000 7f4788396740 1 --2- 172.21.15.2:0/1534459848 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56376cb3c1c0 0x56376cfcbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.364+0000 7f4788396740 4 client.16158 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.364+0000 7f4788396740 1 -- 172.21.15.2:0/1534459848 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56376ceb6050 con 0x56376cb3c1c0 2024-05-14T09:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.364+0000 7f477a7fc640 1 --2- 172.21.15.2:0/1534459848 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56376cb3c1c0 0x56376cfcbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.364+0000 7f475eff5640 10 client.16158 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.364+0000 7f477a7fc640 1 --2- 172.21.15.2:0/1534459848 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56376cb3c1c0 0x56376cfcbcf0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3af6b15a48115371 server_cookie=94ebc9dd347716d4 in_seq=0 out_seq=0 2024-05-14T09:40:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.455+0000 7f475eff5640 1 -- 172.21.15.2:0/1534459848 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56376ceb6050 con 0x56376cb3c1c0 2024-05-14T09:40:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.455+0000 7f475eff5640 10 client.16158 handle_command_reply: tid=0 2024-05-14T09:40:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.455+0000 7f4788396740 1 client.16158 shutdown 2024-05-14T09:40:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.455+0000 7f4788396740 1 -- 172.21.15.2:0/1534459848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cba81b0 msgr2=0x56376cf59f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.455+0000 7f4788396740 1 --2- 172.21.15.2:0/1534459848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cba81b0 0x56376cf59f70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4774006010 tx=0x7f4774009520 comp rx=0 tx=0).stop 2024-05-14T09:40:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.455+0000 7f4788396740 1 -- 172.21.15.2:0/1534459848 shutdown_connections 2024-05-14T09:40:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.455+0000 7f4788396740 1 --2- 172.21.15.2:0/1534459848 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56376cb3c1c0 0x56376cfcbcf0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.455+0000 7f4788396740 1 --2- 172.21.15.2:0/1534459848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56376cba81b0 0x56376cf59f70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.455+0000 7f475eff5640 0 client.16158 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.455+0000 7f4788396740 1 --2- 172.21.15.2:0/1534459848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56376cb874d0 0x56376cf59040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.455+0000 7f4788396740 1 --2- 172.21.15.2:0/1534459848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56376cb7f2f0 0x56376cf58b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.455+0000 7f4788396740 1 -- 172.21.15.2:0/1534459848 >> 172.21.15.2:0/1534459848 conn(0x56376cb9c300 msgr2=0x56376cf43970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.455+0000 7f4788396740 1 -- 172.21.15.2:0/1534459848 shutdown_connections 2024-05-14T09:40:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.455+0000 7f4788396740 1 -- 172.21.15.2:0/1534459848 wait complete. 2024-05-14T09:40:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.455+0000 7f4788396740 20 client.16158 trim_cache size 0 max 16384 2024-05-14T09:40:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:40:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:40:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:40:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:40:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.456+0000 7f47814e0640 1 -- 172.21.15.2:0/2645718546 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f47640b28a0 msgr2=0x7f47640b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.456+0000 7f47814e0640 1 --2- 172.21.15.2:0/2645718546 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f47640b28a0 0x7f47640b4db0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x56376cf4ae40 tx=0x7f476c0e8550 comp rx=0 tx=0).stop 2024-05-14T09:40:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.456+0000 7f47814e0640 1 -- 172.21.15.2:0/2645718546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c102b50 msgr2=0x7f477c19c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.456+0000 7f47814e0640 1 --2- 172.21.15.2:0/2645718546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c102b50 0x7f477c19c460 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e9f0 tx=0x7f476c00eeb0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.456+0000 7f47814e0640 1 -- 172.21.15.2:0/2645718546 shutdown_connections 2024-05-14T09:40:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.456+0000 7f47814e0640 1 --2- 172.21.15.2:0/2645718546 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f47640b28a0 0x7f47640b4db0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.456+0000 7f47814e0640 1 --2- 172.21.15.2:0/2645718546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c108a90 0x7f477c1a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.456+0000 7f47814e0640 1 --2- 172.21.15.2:0/2645718546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f477c103d90 0x7f477c19c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.456+0000 7f47814e0640 1 --2- 172.21.15.2:0/2645718546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c102b50 0x7f477c19c460 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.456+0000 7f47814e0640 1 -- 172.21.15.2:0/2645718546 >> 172.21.15.2:0/2645718546 conn(0x7f477c0fdd40 msgr2=0x7f477c100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.456+0000 7f47814e0640 1 -- 172.21.15.2:0/2645718546 shutdown_connections 2024-05-14T09:40:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:06.456+0000 7f47814e0640 1 -- 172.21.15.2:0/2645718546 wait complete. 2024-05-14T09:40:06.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:06 smithi002 ceph-mon[31557]: pgmap v578: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.3 MiB/s wr, 260 op/s 2024-05-14T09:40:06.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:06 smithi029 ceph-mon[36249]: pgmap v578: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.3 MiB/s wr, 260 op/s 2024-05-14T09:40:06.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:06 smithi026 ceph-mon[36056]: pgmap v578: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.3 MiB/s wr, 260 op/s 2024-05-14T09:40:07.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 411928 10.07 MB/sec execute 93 sec latency 85.721 ms 2024-05-14T09:40:08.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 413961 10.04 MB/sec execute 94 sec latency 17.418 ms 2024-05-14T09:40:08.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:08 smithi002 ceph-mon[31557]: pgmap v579: 129 pgs: 129 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 364 op/s 2024-05-14T09:40:08.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:08 smithi029 ceph-mon[36249]: pgmap v579: 129 pgs: 129 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 364 op/s 2024-05-14T09:40:08.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:08 smithi026 ceph-mon[36056]: pgmap v579: 129 pgs: 129 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 364 op/s 2024-05-14T09:40:09.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 415361 10.05 MB/sec execute 95 sec latency 301.270 ms 2024-05-14T09:40:10.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 417387 10.02 MB/sec execute 96 sec latency 38.188 ms 2024-05-14T09:40:10.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:10 smithi002 ceph-mon[31557]: pgmap v580: 129 pgs: 129 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 300 op/s 2024-05-14T09:40:10.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:10 smithi029 ceph-mon[36249]: pgmap v580: 129 pgs: 129 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 300 op/s 2024-05-14T09:40:10.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:10 smithi026 ceph-mon[36056]: pgmap v580: 129 pgs: 129 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 300 op/s 2024-05-14T09:40:11.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 418989 10.04 MB/sec execute 97 sec latency 64.510 ms 2024-05-14T09:40:12.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 421017 10.02 MB/sec execute 98 sec latency 27.739 ms 2024-05-14T09:40:12.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:12 smithi002 ceph-mon[31557]: pgmap v581: 129 pgs: 129 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 326 op/s 2024-05-14T09:40:12.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:12 smithi029 ceph-mon[36249]: pgmap v581: 129 pgs: 129 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 326 op/s 2024-05-14T09:40:12.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:12 smithi026 ceph-mon[36056]: pgmap v581: 129 pgs: 129 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 326 op/s 2024-05-14T09:40:13.263 INFO:tasks.workunit.client.0.smithi002.stdout: 1 422661 10.04 MB/sec execute 99 sec latency 29.344 ms 2024-05-14T09:40:14.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 424734 10.02 MB/sec execute 100 sec latency 16.326 ms 2024-05-14T09:40:14.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:14 smithi002 ceph-mon[31557]: pgmap v582: 129 pgs: 129 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-05-14T09:40:14.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:14 smithi029 ceph-mon[36249]: pgmap v582: 129 pgs: 129 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-05-14T09:40:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:14 smithi026 ceph-mon[36056]: pgmap v582: 129 pgs: 129 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-05-14T09:40:15.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 426417 10.03 MB/sec execute 101 sec latency 14.904 ms 2024-05-14T09:40:16.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 428672 10.05 MB/sec execute 102 sec latency 29.476 ms 2024-05-14T09:40:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:16 smithi002 ceph-mon[31557]: pgmap v583: 129 pgs: 129 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 267 op/s 2024-05-14T09:40:16.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:16 smithi029 ceph-mon[36249]: pgmap v583: 129 pgs: 129 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 267 op/s 2024-05-14T09:40:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:16 smithi026 ceph-mon[36056]: pgmap v583: 129 pgs: 129 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 267 op/s 2024-05-14T09:40:17.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 430212 10.03 MB/sec execute 103 sec latency 17.374 ms 2024-05-14T09:40:18.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 432844 10.09 MB/sec execute 104 sec latency 22.871 ms 2024-05-14T09:40:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:18 smithi002 ceph-mon[31557]: pgmap v584: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 364 op/s 2024-05-14T09:40:18.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:18 smithi029 ceph-mon[36249]: pgmap v584: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 364 op/s 2024-05-14T09:40:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:18 smithi026 ceph-mon[36056]: pgmap v584: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 364 op/s 2024-05-14T09:40:19.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 434168 10.06 MB/sec execute 105 sec latency 18.325 ms 2024-05-14T09:40:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.564+0000 7f0dd18bd640 1 -- 172.21.15.2:0/1578648429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dcc103c50 msgr2=0x7f0dcc104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.564+0000 7f0dd18bd640 1 --2- 172.21.15.2:0/1578648429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dcc103c50 0x7f0dcc104950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009900 tx=0x7f0dbc02a410 comp rx=0 tx=0).stop 2024-05-14T09:40:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.564+0000 7f0dd18bd640 1 -- 172.21.15.2:0/1578648429 shutdown_connections 2024-05-14T09:40:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.564+0000 7f0dd18bd640 1 --2- 172.21.15.2:0/1578648429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dcc109b90 0x7f0dcc10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.564+0000 7f0dd18bd640 1 --2- 172.21.15.2:0/1578648429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0dcc104e90 0x7f0dcc1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.564+0000 7f0dd18bd640 1 --2- 172.21.15.2:0/1578648429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dcc103c50 0x7f0dcc104950 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.564+0000 7f0dd18bd640 1 -- 172.21.15.2:0/1578648429 >> 172.21.15.2:0/1578648429 conn(0x7f0dcc0fee40 msgr2=0x7f0dcc1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.564+0000 7f0dd18bd640 1 -- 172.21.15.2:0/1578648429 shutdown_connections 2024-05-14T09:40:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.564+0000 7f0dd18bd640 1 -- 172.21.15.2:0/1578648429 wait complete. 2024-05-14T09:40:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.565+0000 7f0dd18bd640 1 Processor -- start 2024-05-14T09:40:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.565+0000 7f0dd18bd640 1 -- start start 2024-05-14T09:40:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.565+0000 7f0dd18bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dcc103c50 0x7f0dcc19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.565+0000 7f0dd18bd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dcc104e90 0x7f0dcc19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.565+0000 7f0dd18bd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0dcc109b90 0x7f0dcc1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.565+0000 7f0dd18bd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc10f7a0 con 0x7f0dcc109b90 2024-05-14T09:40:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.565+0000 7f0dd18bd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc10f620 con 0x7f0dcc103c50 2024-05-14T09:40:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.565+0000 7f0dd18bd640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc10f920 con 0x7f0dcc104e90 2024-05-14T09:40:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.565+0000 7f0dcb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0dcc109b90 0x7f0dcc1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.565+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dcc104e90 0x7f0dcc19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.565+0000 7f0dcb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0dcc109b90 0x7f0dcc1a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47162/0 (socket says 172.21.15.2:47162) 2024-05-14T09:40:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.565+0000 7f0dcb7fe640 1 -- 172.21.15.2:0/3350959416 learned_addr learned my addr 172.21.15.2:0/3350959416 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.565+0000 7f0dcaffd640 1 --2- 172.21.15.2:0/3350959416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dcc103c50 0x7f0dcc19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.566+0000 7f0dcb7fe640 1 -- 172.21.15.2:0/3350959416 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dcc104e90 msgr2=0x7f0dcc19a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.566+0000 7f0dcb7fe640 1 --2- 172.21.15.2:0/3350959416 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dcc104e90 0x7f0dcc19a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.566+0000 7f0dcb7fe640 1 -- 172.21.15.2:0/3350959416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dcc103c50 msgr2=0x7f0dcc19a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.566+0000 7f0dcb7fe640 1 --2- 172.21.15.2:0/3350959416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dcc103c50 0x7f0dcc19a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.566+0000 7f0dcb7fe640 1 -- 172.21.15.2:0/3350959416 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc1a1e00 con 0x7f0dcc109b90 2024-05-14T09:40:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.566+0000 7f0dc3fff640 1 --2- 172.21.15.2:0/3350959416 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dcc104e90 0x7f0dcc19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:40:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.566+0000 7f0dcb7fe640 1 --2- 172.21.15.2:0/3350959416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0dcc109b90 0x7f0dcc1a1790 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400eb10 tx=0x7f0dc400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.566+0000 7f0dd08bb640 1 -- 172.21.15.2:0/3350959416 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4004b20 con 0x7f0dcc109b90 2024-05-14T09:40:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.566+0000 7f0dd18bd640 1 -- 172.21.15.2:0/3350959416 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dcc1a20f0 con 0x7f0dcc109b90 2024-05-14T09:40:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.566+0000 7f0dd08bb640 1 -- 172.21.15.2:0/3350959416 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc4004cc0 con 0x7f0dcc109b90 2024-05-14T09:40:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.566+0000 7f0dd08bb640 1 -- 172.21.15.2:0/3350959416 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4010670 con 0x7f0dcc109b90 2024-05-14T09:40:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.566+0000 7f0dd18bd640 1 -- 172.21.15.2:0/3350959416 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dcc1a2640 con 0x7f0dcc109b90 2024-05-14T09:40:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.569+0000 7f0dd8772740 1 Processor -- start 2024-05-14T09:40:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.569+0000 7f0dd08bb640 1 -- 172.21.15.2:0/3350959416 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0dc4005020 con 0x7f0dcc109b90 2024-05-14T09:40:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.569+0000 7f0dd8772740 1 -- start start 2024-05-14T09:40:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dd8772740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b5e1b0 0x55dbf2f03e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dd8772740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f043f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dd8772740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f0b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dd8772740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55dbf2e645d0 con 0x55dbf2b352f0 2024-05-14T09:40:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dd8772740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dbf2e1d890 con 0x55dbf2b3d4d0 2024-05-14T09:40:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dd8772740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55dbf2e60590 con 0x55dbf2b5e1b0 2024-05-14T09:40:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dcaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b5e1b0 0x55dbf2f03e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dcaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f043f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47170/0 (socket says 172.21.15.2:47170) 2024-05-14T09:40:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dcaffd640 1 -- 172.21.15.2:0/440592325 learned_addr learned my addr 172.21.15.2:0/440592325 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dc3fff640 1 --2- 172.21.15.2:0/440592325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f0b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dc3fff640 1 -- 172.21.15.2:0/440592325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b3d4d0 msgr2=0x55dbf2f0b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dc3fff640 1 --2- 172.21.15.2:0/440592325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f0b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dc3fff640 1 -- 172.21.15.2:0/440592325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 msgr2=0x55dbf2f043f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dc3fff640 1 --2- 172.21.15.2:0/440592325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f043f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dc3fff640 1 -- 172.21.15.2:0/440592325 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf2b76210 con 0x55dbf2b5e1b0 2024-05-14T09:40:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dcaffd640 1 --2- 172.21.15.2:0/440592325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:40:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.570+0000 7f0dd08bb640 1 --2- 172.21.15.2:0/3350959416 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0db80b28a0 0x7f0db80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.571+0000 7f0dcaffd640 1 --2- 172.21.15.2:0/3350959416 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0db80b28a0 0x7f0db80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.571+0000 7f0dc3fff640 1 --2- 172.21.15.2:0/440592325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b5e1b0 0x55dbf2f03e70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc19b360 tx=0x7f0db40096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77faae9af5d18d1e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.571+0000 7f0dd08bb640 1 -- 172.21.15.2:0/3350959416 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f0dc40e2b90 con 0x7f0dcc109b90 2024-05-14T09:40:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.571+0000 7f0dc1ffb640 1 -- 172.21.15.2:0/440592325 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4011050 con 0x55dbf2b5e1b0 2024-05-14T09:40:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.571+0000 7f0dc1ffb640 1 -- 172.21.15.2:0/440592325 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0db40054d0 con 0x55dbf2b5e1b0 2024-05-14T09:40:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.571+0000 7f0dc1ffb640 1 -- 172.21.15.2:0/440592325 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db40047a0 con 0x55dbf2b5e1b0 2024-05-14T09:40:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.572+0000 7f0dd8772740 1 -- 172.21.15.2:0/440592325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b5e1b0 msgr2=0x55dbf2f03e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.572+0000 7f0dcaffd640 1 --2- 172.21.15.2:0/3350959416 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0db80b28a0 0x7f0db80b4db0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55dbf2f05250 tx=0x7f0dbc0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.572+0000 7f0dd8772740 1 --2- 172.21.15.2:0/440592325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b5e1b0 0x55dbf2f03e70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc19b360 tx=0x7f0db40096c0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.572+0000 7f0dd8772740 1 -- 172.21.15.2:0/440592325 shutdown_connections 2024-05-14T09:40:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.572+0000 7f0dd8772740 1 --2- 172.21.15.2:0/440592325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f0b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.572+0000 7f0dd8772740 1 --2- 172.21.15.2:0/440592325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.572+0000 7f0dd8772740 1 --2- 172.21.15.2:0/440592325 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b5e1b0 0x55dbf2f03e70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.572+0000 7f0dd8772740 1 -- 172.21.15.2:0/440592325 >> 172.21.15.2:0/440592325 conn(0x55dbf2b52300 msgr2=0x55dbf2eff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.572+0000 7f0dd8772740 1 -- 172.21.15.2:0/440592325 shutdown_connections 2024-05-14T09:40:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.572+0000 7f0dd8772740 1 -- 172.21.15.2:0/440592325 wait complete. 2024-05-14T09:40:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.572+0000 7f0dd8772740 1 -- start start 2024-05-14T09:40:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dd8772740 1 Processor -- start 2024-05-14T09:40:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dd8772740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b352f0 0x55dbf2f0dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dd8772740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f0e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dd8772740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b5e1b0 0x55dbf2f14f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dd8772740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55dbf2f0bce0 con 0x55dbf2b5e1b0 2024-05-14T09:40:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dd8772740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dbf2e21370 con 0x55dbf2b352f0 2024-05-14T09:40:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dd8772740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55dbf2f0be60 con 0x55dbf2b3d4d0 2024-05-14T09:40:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dd8772740 10 client.0 _ll_register_callbacks cb 0x55dbf2b75a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:40:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f0e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dcaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b352f0 0x55dbf2f0dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dcaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b352f0 0x55dbf2f0dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:56260/0 (socket says 172.21.15.2:56260) 2024-05-14T09:40:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f0e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:53504/0 (socket says 172.21.15.2:53504) 2024-05-14T09:40:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dcaffd640 1 -- 172.21.15.2:0/235813639 learned_addr learned my addr 172.21.15.2:0/235813639 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dcb7fe640 1 --2- 172.21.15.2:0/235813639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b5e1b0 0x55dbf2f14f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dcaffd640 1 -- 172.21.15.2:0/235813639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b3d4d0 msgr2=0x55dbf2f0e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dcaffd640 1 --2- 172.21.15.2:0/235813639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f0e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dcaffd640 1 -- 172.21.15.2:0/235813639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b5e1b0 msgr2=0x55dbf2f14f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dcaffd640 1 --2- 172.21.15.2:0/235813639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b5e1b0 0x55dbf2f14f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.573+0000 7f0dcaffd640 1 -- 172.21.15.2:0/235813639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf2c42620 con 0x55dbf2b352f0 2024-05-14T09:40:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.574+0000 7f0dcaffd640 1 --2- 172.21.15.2:0/235813639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b352f0 0x55dbf2f0dac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc02aa00 tx=0x7f0dbc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdcd51b96e2f6298 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.574+0000 7f0daaffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:40:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.574+0000 7f0daaffd640 1 -- 172.21.15.2:0/235813639 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc035040 con 0x55dbf2b352f0 2024-05-14T09:40:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.574+0000 7f0daaffd640 1 -- 172.21.15.2:0/235813639 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f0dbc003df0 con 0x55dbf2b352f0 2024-05-14T09:40:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.574+0000 7f0dd8772740 1 -- 172.21.15.2:0/235813639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0db40054d0 con 0x55dbf2b352f0 2024-05-14T09:40:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.574+0000 7f0daaffd640 1 -- 172.21.15.2:0/235813639 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0dbc05a050 con 0x55dbf2b352f0 2024-05-14T09:40:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.574+0000 7f0dd8772740 10 client.25859 fetch_fsmap learned FSMap version 31 2024-05-14T09:40:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.574+0000 7f0dd8772740 1 -- 172.21.15.2:0/235813639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dbf2f17f00 con 0x55dbf2b352f0 2024-05-14T09:40:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.574+0000 7f0daaffd640 1 -- 172.21.15.2:0/235813639 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0db40054d0 con 0x55dbf2b352f0 2024-05-14T09:40:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.574+0000 7f0daaffd640 1 -- 172.21.15.2:0/235813639 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f0dbc044030 con 0x55dbf2b352f0 2024-05-14T09:40:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.574+0000 7f0dd8772740 10 client.25859 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T09:40:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.574+0000 7f0dd8772740 10 client.25859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:40:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.574+0000 7f0dd8772740 20 client.25859 populate_metadata read hostname 'smithi002' 2024-05-14T09:40:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.575+0000 7f0dd8772740 1 --2- 172.21.15.2:0/235813639 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbf2b1cd80 0x55dbf2f1b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.575+0000 7f0dd8772740 4 client.25859 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:40:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.575+0000 7f0dd8772740 1 -- 172.21.15.2:0/235813639 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbf2e721e0 con 0x55dbf2b1cd80 2024-05-14T09:40:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.575+0000 7f0dc3fff640 1 --2- 172.21.15.2:0/235813639 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbf2b1cd80 0x55dbf2f1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.575+0000 7f0daaffd640 10 client.25859 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.575+0000 7f0dc3fff640 1 --2- 172.21.15.2:0/235813639 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbf2b1cd80 0x55dbf2f1b600 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a080cd0c4e2b1b4b server_cookie=7ded5564d94f695 in_seq=0 out_seq=0 2024-05-14T09:40:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.576+0000 7f0daaffd640 1 -- 172.21.15.2:0/235813639 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dbf2e721e0 con 0x55dbf2b1cd80 2024-05-14T09:40:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.576+0000 7f0daaffd640 10 client.25859 handle_command_reply: tid=0 2024-05-14T09:40:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.576+0000 7f0dd8772740 1 client.25859 shutdown 2024-05-14T09:40:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.576+0000 7f0dd8772740 1 -- 172.21.15.2:0/235813639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b352f0 msgr2=0x55dbf2f0dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.576+0000 7f0dd8772740 1 --2- 172.21.15.2:0/235813639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b352f0 0x55dbf2f0dac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc02aa00 tx=0x7f0dbc009260 comp rx=0 tx=0).stop 2024-05-14T09:40:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.576+0000 7f0dd8772740 1 -- 172.21.15.2:0/235813639 shutdown_connections 2024-05-14T09:40:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.576+0000 7f0dd8772740 1 --2- 172.21.15.2:0/235813639 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbf2b1cd80 0x55dbf2f1b600 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.576+0000 7f0dd8772740 1 --2- 172.21.15.2:0/235813639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b5e1b0 0x55dbf2f14f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.576+0000 7f0daaffd640 0 client.25859 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.576+0000 7f0dd8772740 1 --2- 172.21.15.2:0/235813639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f0e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.576+0000 7f0dd8772740 1 --2- 172.21.15.2:0/235813639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b352f0 0x55dbf2f0dac0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.576+0000 7f0dd8772740 1 -- 172.21.15.2:0/235813639 >> 172.21.15.2:0/235813639 conn(0x55dbf2b52300 msgr2=0x55dbf2f02710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.576+0000 7f0dd8772740 1 -- 172.21.15.2:0/235813639 shutdown_connections 2024-05-14T09:40:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.576+0000 7f0dd8772740 1 -- 172.21.15.2:0/235813639 wait complete. 2024-05-14T09:40:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.576+0000 7f0dd8772740 20 client.25859 trim_cache size 0 max 16384 2024-05-14T09:40:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.593+0000 7f0dd8772740 1 Processor -- start 2024-05-14T09:40:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.593+0000 7f0dd8772740 1 -- start start 2024-05-14T09:40:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.593+0000 7f0dd8772740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b1cd80 0x55dbf2f085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dd8772740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f0dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dd8772740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f14ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dd8772740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1ad260 con 0x55dbf2b352f0 2024-05-14T09:40:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dd8772740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1aab30 con 0x55dbf2b1cd80 2024-05-14T09:40:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dd8772740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1a8980 con 0x55dbf2b3d4d0 2024-05-14T09:40:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f0dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dcaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b1cd80 0x55dbf2f085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dcaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b1cd80 0x55dbf2f085c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:56268/0 (socket says 172.21.15.2:56268) 2024-05-14T09:40:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dcaffd640 1 -- 172.21.15.2:0/4120962992 learned_addr learned my addr 172.21.15.2:0/4120962992 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f0dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47196/0 (socket says 172.21.15.2:47196) 2024-05-14T09:40:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dcb7fe640 1 --2- 172.21.15.2:0/4120962992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f14ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dcaffd640 1 -- 172.21.15.2:0/4120962992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b3d4d0 msgr2=0x55dbf2f14ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dcaffd640 1 --2- 172.21.15.2:0/4120962992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f14ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dcaffd640 1 -- 172.21.15.2:0/4120962992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 msgr2=0x55dbf2f0dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dcaffd640 1 --2- 172.21.15.2:0/4120962992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f0dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dcaffd640 1 -- 172.21.15.2:0/4120962992 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf2f0b6b0 con 0x55dbf2b1cd80 2024-05-14T09:40:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dc3fff640 1 --2- 172.21.15.2:0/4120962992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f0dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:40:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dcaffd640 1 --2- 172.21.15.2:0/4120962992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b1cd80 0x55dbf2f085c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc006010 tx=0x7f0dbc051620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66110485363d0e62 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.594+0000 7f0dc17fa640 1 -- 172.21.15.2:0/4120962992 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc035040 con 0x55dbf2b1cd80 2024-05-14T09:40:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.595+0000 7f0dc17fa640 1 -- 172.21.15.2:0/4120962992 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dbc04c070 con 0x55dbf2b1cd80 2024-05-14T09:40:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.595+0000 7f0dc17fa640 1 -- 172.21.15.2:0/4120962992 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc031cb0 con 0x55dbf2b1cd80 2024-05-14T09:40:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.595+0000 7f0dd8772740 1 -- 172.21.15.2:0/4120962992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b1cd80 msgr2=0x55dbf2f085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.595+0000 7f0dd8772740 1 --2- 172.21.15.2:0/4120962992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b1cd80 0x55dbf2f085c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc006010 tx=0x7f0dbc051620 comp rx=0 tx=0).stop 2024-05-14T09:40:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.595+0000 7f0dd8772740 1 -- 172.21.15.2:0/4120962992 shutdown_connections 2024-05-14T09:40:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.595+0000 7f0dd8772740 1 --2- 172.21.15.2:0/4120962992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f14ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.595+0000 7f0dd8772740 1 --2- 172.21.15.2:0/4120962992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f0dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.595+0000 7f0dd8772740 1 --2- 172.21.15.2:0/4120962992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b1cd80 0x55dbf2f085c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.595+0000 7f0dd8772740 1 -- 172.21.15.2:0/4120962992 >> 172.21.15.2:0/4120962992 conn(0x55dbf2b52300 msgr2=0x55dbf2f0d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.595+0000 7f0dd8772740 1 -- 172.21.15.2:0/4120962992 shutdown_connections 2024-05-14T09:40:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.595+0000 7f0dd8772740 1 -- 172.21.15.2:0/4120962992 wait complete. 2024-05-14T09:40:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.596+0000 7f0dd8772740 1 -- start start 2024-05-14T09:40:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.596+0000 7f0dd8772740 1 Processor -- start 2024-05-14T09:40:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.596+0000 7f0dd8772740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b1cd80 0x55dbf2f12e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.596+0000 7f0dd8772740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f13350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.596+0000 7f0dd8772740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f14280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.596+0000 7f0dd8772740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1a8ff0 con 0x55dbf2b352f0 2024-05-14T09:40:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.596+0000 7f0dd8772740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1aa460 con 0x55dbf2b3d4d0 2024-05-14T09:40:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.596+0000 7f0dd8772740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1a96c0 con 0x55dbf2b1cd80 2024-05-14T09:40:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.596+0000 7f0dd8772740 10 client.0 _ll_register_callbacks cb 0x7f0dbc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:40:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.596+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f13350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.596+0000 7f0dcaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b1cd80 0x55dbf2f12e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.596+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f13350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47198/0 (socket says 172.21.15.2:47198) 2024-05-14T09:40:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.596+0000 7f0dc3fff640 1 -- 172.21.15.2:0/3424287797 learned_addr learned my addr 172.21.15.2:0/3424287797 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.597+0000 7f0dcb7fe640 1 --2- 172.21.15.2:0/3424287797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f14280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.597+0000 7f0dc3fff640 1 -- 172.21.15.2:0/3424287797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b1cd80 msgr2=0x55dbf2f12e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.597+0000 7f0dc3fff640 1 --2- 172.21.15.2:0/3424287797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b1cd80 0x55dbf2f12e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.597+0000 7f0dc3fff640 1 -- 172.21.15.2:0/3424287797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b3d4d0 msgr2=0x55dbf2f14280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.597+0000 7f0dc3fff640 1 --2- 172.21.15.2:0/3424287797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f14280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.597+0000 7f0dc3fff640 1 -- 172.21.15.2:0/3424287797 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf2f0bef0 con 0x55dbf2b352f0 2024-05-14T09:40:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.597+0000 7f0dcb7fe640 1 --2- 172.21.15.2:0/3424287797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f14280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:40:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.597+0000 7f0dc3fff640 1 --2- 172.21.15.2:0/3424287797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f13350 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0db40057c0 tx=0x7f0db4009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9314bf72225ac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.597+0000 7f0daaffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:40:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.597+0000 7f0daaffd640 1 -- 172.21.15.2:0/3424287797 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db400fac0 con 0x55dbf2b352f0 2024-05-14T09:40:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.597+0000 7f0daaffd640 1 -- 172.21.15.2:0/3424287797 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f0db4011040 con 0x55dbf2b352f0 2024-05-14T09:40:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.597+0000 7f0dd8772740 1 -- 172.21.15.2:0/3424287797 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dbc04c070 con 0x55dbf2b352f0 2024-05-14T09:40:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.597+0000 7f0daaffd640 1 -- 172.21.15.2:0/3424287797 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0db4038050 con 0x55dbf2b352f0 2024-05-14T09:40:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.597+0000 7f0dd8772740 10 client.16176 fetch_fsmap learned FSMap version 31 2024-05-14T09:40:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.597+0000 7f0dd8772740 1 -- 172.21.15.2:0/3424287797 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dbf2f15900 con 0x55dbf2b352f0 2024-05-14T09:40:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.598+0000 7f0daaffd640 1 -- 172.21.15.2:0/3424287797 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dbc04c070 con 0x55dbf2b352f0 2024-05-14T09:40:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.598+0000 7f0daaffd640 1 -- 172.21.15.2:0/3424287797 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f0db4033030 con 0x55dbf2b352f0 2024-05-14T09:40:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.598+0000 7f0dd8772740 10 client.16176 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T09:40:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.598+0000 7f0dd8772740 10 client.16176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:40:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.598+0000 7f0dd8772740 20 client.16176 populate_metadata read hostname 'smithi002' 2024-05-14T09:40:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.598+0000 7f0dd8772740 1 --2- 172.21.15.2:0/3424287797 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbf2b5e1b0 0x55dbf2f85ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.598+0000 7f0dd8772740 4 client.16176 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:40:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.598+0000 7f0dd8772740 1 -- 172.21.15.2:0/3424287797 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dbf2e6c050 con 0x55dbf2b5e1b0 2024-05-14T09:40:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.598+0000 7f0dcaffd640 1 --2- 172.21.15.2:0/3424287797 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbf2b5e1b0 0x55dbf2f85ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.599+0000 7f0daaffd640 10 client.16176 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.598+0000 7f0dcaffd640 1 --2- 172.21.15.2:0/3424287797 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbf2b5e1b0 0x55dbf2f85ef0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=758ad363044e846e server_cookie=9a00df8476a3d350 in_seq=0 out_seq=0 2024-05-14T09:40:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.690+0000 7f0daaffd640 1 -- 172.21.15.2:0/3424287797 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dbf2e6c050 con 0x55dbf2b5e1b0 2024-05-14T09:40:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.690+0000 7f0daaffd640 10 client.16176 handle_command_reply: tid=0 2024-05-14T09:40:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.690+0000 7f0dd8772740 1 client.16176 shutdown 2024-05-14T09:40:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.690+0000 7f0dd8772740 1 -- 172.21.15.2:0/3424287797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 msgr2=0x55dbf2f13350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.690+0000 7f0dd8772740 1 --2- 172.21.15.2:0/3424287797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f13350 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0db40057c0 tx=0x7f0db4009e00 comp rx=0 tx=0).stop 2024-05-14T09:40:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.691+0000 7f0dd8772740 1 -- 172.21.15.2:0/3424287797 shutdown_connections 2024-05-14T09:40:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.691+0000 7f0dd8772740 1 --2- 172.21.15.2:0/3424287797 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dbf2b5e1b0 0x55dbf2f85ef0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.691+0000 7f0dd8772740 1 --2- 172.21.15.2:0/3424287797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dbf2b3d4d0 0x55dbf2f14280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.691+0000 7f0dd8772740 1 --2- 172.21.15.2:0/3424287797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbf2b352f0 0x55dbf2f13350 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.691+0000 7f0dd8772740 1 --2- 172.21.15.2:0/3424287797 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dbf2b1cd80 0x55dbf2f12e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.691+0000 7f0dd8772740 1 -- 172.21.15.2:0/3424287797 >> 172.21.15.2:0/3424287797 conn(0x55dbf2b52300 msgr2=0x55dbf2ef9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.691+0000 7f0daaffd640 0 client.16176 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.691+0000 7f0dd8772740 1 -- 172.21.15.2:0/3424287797 shutdown_connections 2024-05-14T09:40:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.691+0000 7f0dd8772740 1 -- 172.21.15.2:0/3424287797 wait complete. 2024-05-14T09:40:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.691+0000 7f0dd8772740 20 client.16176 trim_cache size 0 max 16384 2024-05-14T09:40:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:40:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:40:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:40:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:40:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.691+0000 7f0dd18bd640 1 -- 172.21.15.2:0/3350959416 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0db80b28a0 msgr2=0x7f0db80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.691+0000 7f0dd18bd640 1 --2- 172.21.15.2:0/3350959416 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0db80b28a0 0x7f0db80b4db0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55dbf2f05250 tx=0x7f0dbc0314a0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.691+0000 7f0dd18bd640 1 -- 172.21.15.2:0/3350959416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0dcc109b90 msgr2=0x7f0dcc1a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.691+0000 7f0dd18bd640 1 --2- 172.21.15.2:0/3350959416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0dcc109b90 0x7f0dcc1a1790 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400eb10 tx=0x7f0dc400c2d0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.692+0000 7f0dd18bd640 1 -- 172.21.15.2:0/3350959416 shutdown_connections 2024-05-14T09:40:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.692+0000 7f0dd18bd640 1 --2- 172.21.15.2:0/3350959416 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0db80b28a0 0x7f0db80b4db0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.692+0000 7f0dd18bd640 1 --2- 172.21.15.2:0/3350959416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0dcc109b90 0x7f0dcc1a1790 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.692+0000 7f0dd18bd640 1 --2- 172.21.15.2:0/3350959416 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0dcc104e90 0x7f0dcc19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.692+0000 7f0dd18bd640 1 --2- 172.21.15.2:0/3350959416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dcc103c50 0x7f0dcc19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.692+0000 7f0dd18bd640 1 -- 172.21.15.2:0/3350959416 >> 172.21.15.2:0/3350959416 conn(0x7f0dcc0fee40 msgr2=0x7f0dcc105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.692+0000 7f0dd18bd640 1 -- 172.21.15.2:0/3350959416 shutdown_connections 2024-05-14T09:40:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:19.692+0000 7f0dd18bd640 1 -- 172.21.15.2:0/3350959416 wait complete. 2024-05-14T09:40:20.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 436550 10.09 MB/sec execute 106 sec latency 75.231 ms 2024-05-14T09:40:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:20 smithi002 ceph-mon[31557]: pgmap v585: 129 pgs: 129 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 297 op/s 2024-05-14T09:40:20.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:20 smithi029 ceph-mon[36249]: pgmap v585: 129 pgs: 129 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 297 op/s 2024-05-14T09:40:20.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:20 smithi026 ceph-mon[36056]: pgmap v585: 129 pgs: 129 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 297 op/s 2024-05-14T09:40:21.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 437962 10.05 MB/sec execute 107 sec latency 38.126 ms 2024-05-14T09:40:22.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 440218 10.08 MB/sec execute 108 sec latency 20.135 ms 2024-05-14T09:40:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:22 smithi002 ceph-mon[31557]: pgmap v586: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 269 op/s 2024-05-14T09:40:22.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:22 smithi029 ceph-mon[36249]: pgmap v586: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 269 op/s 2024-05-14T09:40:22.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:22 smithi026 ceph-mon[36056]: pgmap v586: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 269 op/s 2024-05-14T09:40:23.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 442184 10.05 MB/sec execute 109 sec latency 16.116 ms 2024-05-14T09:40:24.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 443636 10.07 MB/sec execute 110 sec latency 200.721 ms 2024-05-14T09:40:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:24 smithi002 ceph-mon[31557]: pgmap v587: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 349 op/s 2024-05-14T09:40:24.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:24 smithi029 ceph-mon[36249]: pgmap v587: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 349 op/s 2024-05-14T09:40:24.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:24 smithi026 ceph-mon[36056]: pgmap v587: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 349 op/s 2024-05-14T09:40:25.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 445099 10.03 MB/sec execute 111 sec latency 17.574 ms 2024-05-14T09:40:26.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 447238 10.06 MB/sec execute 112 sec latency 30.107 ms 2024-05-14T09:40:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:26 smithi002 ceph-mon[31557]: pgmap v588: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 299 op/s 2024-05-14T09:40:26.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:26 smithi029 ceph-mon[36249]: pgmap v588: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 299 op/s 2024-05-14T09:40:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:26 smithi026 ceph-mon[36056]: pgmap v588: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 299 op/s 2024-05-14T09:40:27.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 449037 10.03 MB/sec execute 113 sec latency 23.767 ms 2024-05-14T09:40:28.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 450917 10.06 MB/sec execute 114 sec latency 28.033 ms 2024-05-14T09:40:28.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:28 smithi002 ceph-mon[31557]: pgmap v589: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 331 op/s 2024-05-14T09:40:28.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:28 smithi029 ceph-mon[36249]: pgmap v589: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 331 op/s 2024-05-14T09:40:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:28 smithi026 ceph-mon[36056]: pgmap v589: 129 pgs: 129 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 331 op/s 2024-05-14T09:40:29.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 452930 10.03 MB/sec execute 115 sec latency 15.716 ms 2024-05-14T09:40:29.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:29 smithi002 ceph-mon[31557]: pgmap v590: 129 pgs: 129 active+clean; 94 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-05-14T09:40:29.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:29 smithi029 ceph-mon[36249]: pgmap v590: 129 pgs: 129 active+clean; 94 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-05-14T09:40:29.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:29 smithi026 ceph-mon[36056]: pgmap v590: 129 pgs: 129 active+clean; 94 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-05-14T09:40:30.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 454542 10.05 MB/sec execute 116 sec latency 23.435 ms 2024-05-14T09:40:31.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 456474 10.02 MB/sec execute 117 sec latency 33.328 ms 2024-05-14T09:40:32.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 458052 10.04 MB/sec execute 118 sec latency 32.692 ms 2024-05-14T09:40:32.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:32 smithi002 ceph-mon[31557]: pgmap v591: 129 pgs: 129 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2024-05-14T09:40:32.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:32 smithi029 ceph-mon[36249]: pgmap v591: 129 pgs: 129 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2024-05-14T09:40:32.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:32 smithi026 ceph-mon[36056]: pgmap v591: 129 pgs: 129 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2024-05-14T09:40:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.798+0000 7f79f7f9d640 1 -- 172.21.15.2:0/1710449116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79f0102510 msgr2=0x7f79f0102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.798+0000 7f79f7f9d640 1 --2- 172.21.15.2:0/1710449116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79f0102510 0x7f79f0102920 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009930 tx=0x7f79ec02a420 comp rx=0 tx=0).stop 2024-05-14T09:40:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.798+0000 7f79f7f9d640 1 -- 172.21.15.2:0/1710449116 shutdown_connections 2024-05-14T09:40:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.798+0000 7f79f7f9d640 1 --2- 172.21.15.2:0/1710449116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79f01037b0 0x7f79f0103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.798+0000 7f79f7f9d640 1 --2- 172.21.15.2:0/1710449116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79f0102e60 0x7f79f0103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.798+0000 7f79f7f9d640 1 --2- 172.21.15.2:0/1710449116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79f0102510 0x7f79f0102920 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.798+0000 7f79f7f9d640 1 -- 172.21.15.2:0/1710449116 >> 172.21.15.2:0/1710449116 conn(0x7f79f00fdd20 msgr2=0x7f79f01001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.798+0000 7f79f7f9d640 1 -- 172.21.15.2:0/1710449116 shutdown_connections 2024-05-14T09:40:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.798+0000 7f79f7f9d640 1 -- 172.21.15.2:0/1710449116 wait complete. 2024-05-14T09:40:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.799+0000 7f79f7f9d640 1 Processor -- start 2024-05-14T09:40:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.799+0000 7f79f7f9d640 1 -- start start 2024-05-14T09:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.799+0000 7f79f7f9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79f0102510 0x7f79f019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.799+0000 7f79f7f9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79f0102e60 0x7f79f019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.799+0000 7f79f7f9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79f01037b0 0x7f79f01a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.799+0000 7f79f7f9d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0111940 con 0x7f79f0102e60 2024-05-14T09:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.799+0000 7f79f7f9d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01117c0 con 0x7f79f0102510 2024-05-14T09:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.799+0000 7f79f7f9d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0111ac0 con 0x7f79f01037b0 2024-05-14T09:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.799+0000 7f79f5538640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79f0102e60 0x7f79f019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.799+0000 7f79f5538640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79f0102e60 0x7f79f019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37086/0 (socket says 172.21.15.2:37086) 2024-05-14T09:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.799+0000 7f79f5538640 1 -- 172.21.15.2:0/3993799534 learned_addr learned my addr 172.21.15.2:0/3993799534 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.799+0000 7f79f5d39640 1 --2- 172.21.15.2:0/3993799534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79f0102510 0x7f79f019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.799+0000 7f79f653a640 1 --2- 172.21.15.2:0/3993799534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79f01037b0 0x7f79f01a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.799+0000 7f79f5538640 1 -- 172.21.15.2:0/3993799534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79f01037b0 msgr2=0x7f79f01a39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.799+0000 7f79f5538640 1 --2- 172.21.15.2:0/3993799534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79f01037b0 0x7f79f01a39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.800+0000 7f79f5538640 1 -- 172.21.15.2:0/3993799534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79f0102510 msgr2=0x7f79f019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.800+0000 7f79f5538640 1 --2- 172.21.15.2:0/3993799534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79f0102510 0x7f79f019c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.800+0000 7f79f5538640 1 -- 172.21.15.2:0/3993799534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f01a4030 con 0x7f79f0102e60 2024-05-14T09:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.800+0000 7f79f5538640 1 --2- 172.21.15.2:0/3993799534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79f0102e60 0x7f79f019ca80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f79e000e9f0 tx=0x7f79e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.800+0000 7f79e6ffd640 1 -- 172.21.15.2:0/3993799534 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e000cd80 con 0x7f79f0102e60 2024-05-14T09:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.800+0000 7f79e6ffd640 1 -- 172.21.15.2:0/3993799534 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e00108e0 con 0x7f79f0102e60 2024-05-14T09:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.800+0000 7f79f7f9d640 1 -- 172.21.15.2:0/3993799534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f01a4320 con 0x7f79f0102e60 2024-05-14T09:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.800+0000 7f79e6ffd640 1 -- 172.21.15.2:0/3993799534 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0018980 con 0x7f79f0102e60 2024-05-14T09:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.800+0000 7f79f7f9d640 1 -- 172.21.15.2:0/3993799534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f01a48c0 con 0x7f79f0102e60 2024-05-14T09:40:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.804+0000 7f79fee54740 1 Processor -- start 2024-05-14T09:40:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.804+0000 7f79fee54740 1 -- start start 2024-05-14T09:40:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.804+0000 7f79fee54740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bbc21b0 0x55df6bf63a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.804+0000 7f79fee54740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf63fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.804+0000 7f79fee54740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bba14d0 0x55df6bf6aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.804+0000 7f79fee54740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55df6bec85d0 con 0x55df6bb992f0 2024-05-14T09:40:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.804+0000 7f79fee54740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55df6be81890 con 0x55df6bba14d0 2024-05-14T09:40:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.804+0000 7f79fee54740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55df6bec4590 con 0x55df6bbc21b0 2024-05-14T09:40:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.804+0000 7f79f5d39640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf63fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.804+0000 7f79f653a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bbc21b0 0x55df6bf63a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.804+0000 7f79f5d39640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf63fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37100/0 (socket says 172.21.15.2:37100) 2024-05-14T09:40:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.804+0000 7f79f5d39640 1 -- 172.21.15.2:0/1029027255 learned_addr learned my addr 172.21.15.2:0/1029027255 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.804+0000 7f79f653a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bbc21b0 0x55df6bf63a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:56440/0 (socket says 172.21.15.2:56440) 2024-05-14T09:40:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.804+0000 7f79e6ffd640 1 -- 172.21.15.2:0/3993799534 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f79e0010430 con 0x7f79f0102e60 2024-05-14T09:40:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.805+0000 7f79f653a640 1 -- 172.21.15.2:0/1029027255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bba14d0 msgr2=0x55df6bf6aee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:40:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.805+0000 7f79f5538640 1 --2- 172.21.15.2:0/1029027255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bba14d0 0x55df6bf6aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.805+0000 7f79f653a640 1 --2- 172.21.15.2:0/1029027255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bba14d0 0x55df6bf6aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.805+0000 7f79f653a640 1 -- 172.21.15.2:0/1029027255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 msgr2=0x55df6bf63fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.805+0000 7f79f653a640 1 --2- 172.21.15.2:0/1029027255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf63fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.805+0000 7f79f653a640 1 -- 172.21.15.2:0/1029027255 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6bbda210 con 0x55df6bbc21b0 2024-05-14T09:40:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.805+0000 7f79f5538640 1 --2- 172.21.15.2:0/1029027255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bba14d0 0x55df6bf6aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:40:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.805+0000 7f79f5d39640 1 --2- 172.21.15.2:0/1029027255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf63fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:40:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.805+0000 7f79f653a640 1 --2- 172.21.15.2:0/1029027255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bbc21b0 0x55df6bf63a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f79e8005fd0 tx=0x7f79e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f57e83beb412d54 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.805+0000 7f79e6ffd640 1 --2- 172.21.15.2:0/3993799534 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f79dc0b2920 0x7f79dc0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.805+0000 7f79dbfff640 1 -- 172.21.15.2:0/1029027255 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8004d20 con 0x55df6bbc21b0 2024-05-14T09:40:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.806+0000 7f79f5d39640 1 --2- 172.21.15.2:0/3993799534 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f79dc0b2920 0x7f79dc0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.806+0000 7f79e6ffd640 1 -- 172.21.15.2:0/3993799534 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f79e00e28c0 con 0x7f79f0102e60 2024-05-14T09:40:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.806+0000 7f79dbfff640 1 -- 172.21.15.2:0/1029027255 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e8004030 con 0x55df6bbc21b0 2024-05-14T09:40:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.806+0000 7f79dbfff640 1 -- 172.21.15.2:0/1029027255 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8005280 con 0x55df6bbc21b0 2024-05-14T09:40:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.807+0000 7f79fee54740 1 -- 172.21.15.2:0/1029027255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bbc21b0 msgr2=0x55df6bf63a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.807+0000 7f79f5d39640 1 --2- 172.21.15.2:0/3993799534 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f79dc0b2920 0x7f79dc0b4e30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55df6bf64e40 tx=0x7f79ec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.807+0000 7f79fee54740 1 --2- 172.21.15.2:0/1029027255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bbc21b0 0x55df6bf63a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f79e8005fd0 tx=0x7f79e8011810 comp rx=0 tx=0).stop 2024-05-14T09:40:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.807+0000 7f79fee54740 1 -- 172.21.15.2:0/1029027255 shutdown_connections 2024-05-14T09:40:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.807+0000 7f79fee54740 1 --2- 172.21.15.2:0/1029027255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bba14d0 0x55df6bf6aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.807+0000 7f79fee54740 1 --2- 172.21.15.2:0/1029027255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf63fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.807+0000 7f79fee54740 1 --2- 172.21.15.2:0/1029027255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bbc21b0 0x55df6bf63a60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.807+0000 7f79fee54740 1 -- 172.21.15.2:0/1029027255 >> 172.21.15.2:0/1029027255 conn(0x55df6bbb6300 msgr2=0x55df6bf5f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.807+0000 7f79fee54740 1 -- 172.21.15.2:0/1029027255 shutdown_connections 2024-05-14T09:40:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.807+0000 7f79fee54740 1 -- 172.21.15.2:0/1029027255 wait complete. 2024-05-14T09:40:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.808+0000 7f79fee54740 1 -- start start 2024-05-14T09:40:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.808+0000 7f79fee54740 1 Processor -- start 2024-05-14T09:40:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.808+0000 7f79fee54740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf6d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.808+0000 7f79fee54740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bba14d0 0x55df6bf6dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.808+0000 7f79fee54740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bbc21b0 0x55df6bf74bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.808+0000 7f79fee54740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55df6bf6b930 con 0x55df6bb992f0 2024-05-14T09:40:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.808+0000 7f79fee54740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55df6be85370 con 0x55df6bbc21b0 2024-05-14T09:40:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.808+0000 7f79fee54740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55df6bf6bab0 con 0x55df6bba14d0 2024-05-14T09:40:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.808+0000 7f79f5d39640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf6d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.808+0000 7f79fee54740 10 client.0 _ll_register_callbacks cb 0x55df6bbd9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:40:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.809+0000 7f79f5d39640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf6d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37102/0 (socket says 172.21.15.2:37102) 2024-05-14T09:40:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.809+0000 7f79f5d39640 1 -- 172.21.15.2:0/2207857332 learned_addr learned my addr 172.21.15.2:0/2207857332 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.809+0000 7f79f5538640 1 --2- 172.21.15.2:0/2207857332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bba14d0 0x55df6bf6dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.809+0000 7f79f653a640 1 --2- 172.21.15.2:0/2207857332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bbc21b0 0x55df6bf74bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.809+0000 7f79f5d39640 1 -- 172.21.15.2:0/2207857332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bba14d0 msgr2=0x55df6bf6dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.809+0000 7f79f5d39640 1 --2- 172.21.15.2:0/2207857332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bba14d0 0x55df6bf6dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.809+0000 7f79f5d39640 1 -- 172.21.15.2:0/2207857332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bbc21b0 msgr2=0x55df6bf74bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.809+0000 7f79f5d39640 1 --2- 172.21.15.2:0/2207857332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bbc21b0 0x55df6bf74bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.809+0000 7f79f5d39640 1 -- 172.21.15.2:0/2207857332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6bca6620 con 0x55df6bb992f0 2024-05-14T09:40:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.809+0000 7f79f5d39640 1 --2- 172.21.15.2:0/2207857332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf6d730 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009230 tx=0x7f79ec02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11d47e58e43eb4b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.809+0000 7f79d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:40:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.809+0000 7f79d97fa640 1 -- 172.21.15.2:0/2207857332 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec035040 con 0x55df6bb992f0 2024-05-14T09:40:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.810+0000 7f79d97fa640 1 -- 172.21.15.2:0/2207857332 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f79ec0035f0 con 0x55df6bb992f0 2024-05-14T09:40:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.810+0000 7f79fee54740 1 -- 172.21.15.2:0/2207857332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e8004030 con 0x55df6bb992f0 2024-05-14T09:40:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.810+0000 7f79d97fa640 1 -- 172.21.15.2:0/2207857332 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f79ec0037f0 con 0x55df6bb992f0 2024-05-14T09:40:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.810+0000 7f79fee54740 10 client.16188 fetch_fsmap learned FSMap version 31 2024-05-14T09:40:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.810+0000 7f79fee54740 1 -- 172.21.15.2:0/2207857332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55df6bf77ae0 con 0x55df6bb992f0 2024-05-14T09:40:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.810+0000 7f79d97fa640 1 -- 172.21.15.2:0/2207857332 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e8004030 con 0x55df6bb992f0 2024-05-14T09:40:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.810+0000 7f79d97fa640 1 -- 172.21.15.2:0/2207857332 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f79ec044070 con 0x55df6bb992f0 2024-05-14T09:40:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.810+0000 7f79fee54740 10 client.16188 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T09:40:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.810+0000 7f79fee54740 10 client.16188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:40:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.810+0000 7f79fee54740 20 client.16188 populate_metadata read hostname 'smithi002' 2024-05-14T09:40:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.811+0000 7f79fee54740 1 --2- 172.21.15.2:0/2207857332 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55df6bb80d80 0x55df6bf7b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.811+0000 7f79fee54740 4 client.16188 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:40:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.811+0000 7f79fee54740 1 -- 172.21.15.2:0/2207857332 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df6bed61e0 con 0x55df6bb80d80 2024-05-14T09:40:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.811+0000 7f79f5538640 1 --2- 172.21.15.2:0/2207857332 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55df6bb80d80 0x55df6bf7b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.811+0000 7f79d97fa640 10 client.16188 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.811+0000 7f79f5538640 1 --2- 172.21.15.2:0/2207857332 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55df6bb80d80 0x55df6bf7b1b0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=955118e0a92e7719 server_cookie=9f827f1fe2c88e69 in_seq=0 out_seq=0 2024-05-14T09:40:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.812+0000 7f79d97fa640 1 -- 172.21.15.2:0/2207857332 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55df6bed61e0 con 0x55df6bb80d80 2024-05-14T09:40:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.812+0000 7f79d97fa640 10 client.16188 handle_command_reply: tid=0 2024-05-14T09:40:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.812+0000 7f79fee54740 1 client.16188 shutdown 2024-05-14T09:40:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.812+0000 7f79fee54740 1 -- 172.21.15.2:0/2207857332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 msgr2=0x55df6bf6d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.812+0000 7f79fee54740 1 --2- 172.21.15.2:0/2207857332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf6d730 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009230 tx=0x7f79ec02a420 comp rx=0 tx=0).stop 2024-05-14T09:40:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.812+0000 7f79fee54740 1 -- 172.21.15.2:0/2207857332 shutdown_connections 2024-05-14T09:40:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.812+0000 7f79fee54740 1 --2- 172.21.15.2:0/2207857332 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55df6bb80d80 0x55df6bf7b1b0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.812+0000 7f79fee54740 1 --2- 172.21.15.2:0/2207857332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bbc21b0 0x55df6bf74bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.812+0000 7f79fee54740 1 --2- 172.21.15.2:0/2207857332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bba14d0 0x55df6bf6dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.812+0000 7f79d97fa640 0 client.16188 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.812+0000 7f79fee54740 1 --2- 172.21.15.2:0/2207857332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf6d730 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.813+0000 7f79fee54740 1 -- 172.21.15.2:0/2207857332 >> 172.21.15.2:0/2207857332 conn(0x55df6bbb6300 msgr2=0x55df6bf5e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.813+0000 7f79fee54740 1 -- 172.21.15.2:0/2207857332 shutdown_connections 2024-05-14T09:40:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.813+0000 7f79fee54740 1 -- 172.21.15.2:0/2207857332 wait complete. 2024-05-14T09:40:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.813+0000 7f79fee54740 20 client.16188 trim_cache size 0 max 16384 2024-05-14T09:40:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.830+0000 7f79fee54740 1 Processor -- start 2024-05-14T09:40:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.830+0000 7f79fee54740 1 -- start start 2024-05-14T09:40:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79fee54740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bb80d80 0x55df6bf68170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79fee54740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf6da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79fee54740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bba14d0 0x55df6bf74b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79fee54740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01b1820 con 0x55df6bb992f0 2024-05-14T09:40:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79fee54740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01af0f0 con 0x55df6bb80d80 2024-05-14T09:40:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79fee54740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01acf40 con 0x55df6bba14d0 2024-05-14T09:40:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79f5538640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf6da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79f5d39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bb80d80 0x55df6bf68170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79f5538640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf6da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37108/0 (socket says 172.21.15.2:37108) 2024-05-14T09:40:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79f5538640 1 -- 172.21.15.2:0/323138786 learned_addr learned my addr 172.21.15.2:0/323138786 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79f653a640 1 --2- 172.21.15.2:0/323138786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bba14d0 0x55df6bf74b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79f5538640 1 -- 172.21.15.2:0/323138786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bba14d0 msgr2=0x55df6bf74b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79f5538640 1 --2- 172.21.15.2:0/323138786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bba14d0 0x55df6bf74b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79f5538640 1 -- 172.21.15.2:0/323138786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bb80d80 msgr2=0x55df6bf68170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79f5538640 1 --2- 172.21.15.2:0/323138786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bb80d80 0x55df6bf68170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79f5538640 1 -- 172.21.15.2:0/323138786 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6bf5fd00 con 0x55df6bb992f0 2024-05-14T09:40:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79f5d39640 1 --2- 172.21.15.2:0/323138786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bb80d80 0x55df6bf68170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:40:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79f5538640 1 --2- 172.21.15.2:0/323138786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf6da10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f79e000c6a0 tx=0x7f79e00e4dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc97568fb50e2863 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79db7fe640 1 -- 172.21.15.2:0/323138786 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e00ade70 con 0x55df6bb992f0 2024-05-14T09:40:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79db7fe640 1 -- 172.21.15.2:0/323138786 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e00b2920 con 0x55df6bb992f0 2024-05-14T09:40:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.831+0000 7f79db7fe640 1 -- 172.21.15.2:0/323138786 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0018d80 con 0x55df6bb992f0 2024-05-14T09:40:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.832+0000 7f79fee54740 1 -- 172.21.15.2:0/323138786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 msgr2=0x55df6bf6da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.832+0000 7f79fee54740 1 --2- 172.21.15.2:0/323138786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf6da10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f79e000c6a0 tx=0x7f79e00e4dd0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.832+0000 7f79fee54740 1 -- 172.21.15.2:0/323138786 shutdown_connections 2024-05-14T09:40:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.832+0000 7f79fee54740 1 --2- 172.21.15.2:0/323138786 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bba14d0 0x55df6bf74b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.832+0000 7f79fee54740 1 --2- 172.21.15.2:0/323138786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb992f0 0x55df6bf6da10 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.832+0000 7f79fee54740 1 --2- 172.21.15.2:0/323138786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bb80d80 0x55df6bf68170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.832+0000 7f79fee54740 1 -- 172.21.15.2:0/323138786 >> 172.21.15.2:0/323138786 conn(0x55df6bbb6300 msgr2=0x55df6bf6d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.832+0000 7f79fee54740 1 -- 172.21.15.2:0/323138786 shutdown_connections 2024-05-14T09:40:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.832+0000 7f79fee54740 1 -- 172.21.15.2:0/323138786 wait complete. 2024-05-14T09:40:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.833+0000 7f79fee54740 1 -- start start 2024-05-14T09:40:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.833+0000 7f79fee54740 1 Processor -- start 2024-05-14T09:40:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.833+0000 7f79fee54740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb80d80 0x55df6bf72780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.833+0000 7f79fee54740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bb992f0 0x55df6bf72cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.833+0000 7f79fee54740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bba14d0 0x55df6bf73bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.833+0000 7f79fee54740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01ad5b0 con 0x55df6bb80d80 2024-05-14T09:40:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.833+0000 7f79fee54740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01aea20 con 0x55df6bb992f0 2024-05-14T09:40:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.833+0000 7f79fee54740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01adc80 con 0x55df6bba14d0 2024-05-14T09:40:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.833+0000 7f79fee54740 10 client.0 _ll_register_callbacks cb 0x7f79ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:40:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.833+0000 7f79f5d39640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb80d80 0x55df6bf72780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.833+0000 7f79f5d39640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb80d80 0x55df6bf72780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37122/0 (socket says 172.21.15.2:37122) 2024-05-14T09:40:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.833+0000 7f79f5d39640 1 -- 172.21.15.2:0/3765905865 learned_addr learned my addr 172.21.15.2:0/3765905865 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79f5538640 1 --2- 172.21.15.2:0/3765905865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bb992f0 0x55df6bf72cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79f653a640 1 --2- 172.21.15.2:0/3765905865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bba14d0 0x55df6bf73bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79f5538640 1 -- 172.21.15.2:0/3765905865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bba14d0 msgr2=0x55df6bf73bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79f5538640 1 --2- 172.21.15.2:0/3765905865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bba14d0 0x55df6bf73bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79f5538640 1 -- 172.21.15.2:0/3765905865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb80d80 msgr2=0x55df6bf72780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79f5538640 1 --2- 172.21.15.2:0/3765905865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb80d80 0x55df6bf72780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79f5538640 1 -- 172.21.15.2:0/3765905865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6bf741b0 con 0x55df6bb992f0 2024-05-14T09:40:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79f5d39640 1 --2- 172.21.15.2:0/3765905865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb80d80 0x55df6bf72780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:40:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79f5538640 1 --2- 172.21.15.2:0/3765905865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bb992f0 0x55df6bf72cc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f79e000e9c0 tx=0x7f79e00e2c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79c17177aefcf9d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:40:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79d97fa640 1 -- 172.21.15.2:0/3765905865 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e00ade70 con 0x55df6bb992f0 2024-05-14T09:40:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79fee54740 1 -- 172.21.15.2:0/3765905865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e00b2920 con 0x55df6bb992f0 2024-05-14T09:40:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79d97fa640 1 -- 172.21.15.2:0/3765905865 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f79e00a8dd0 con 0x55df6bb992f0 2024-05-14T09:40:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79d97fa640 1 -- 172.21.15.2:0/3765905865 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f79e00e00b0 con 0x55df6bb992f0 2024-05-14T09:40:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79fee54740 10 client.25880 fetch_fsmap learned FSMap version 31 2024-05-14T09:40:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.834+0000 7f79fee54740 1 -- 172.21.15.2:0/3765905865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55df6bf75440 con 0x55df6bb992f0 2024-05-14T09:40:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.835+0000 7f79d97fa640 1 -- 172.21.15.2:0/3765905865 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e00b2920 con 0x55df6bb992f0 2024-05-14T09:40:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.835+0000 7f79d97fa640 1 -- 172.21.15.2:0/3765905865 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f79e00a54e0 con 0x55df6bb992f0 2024-05-14T09:40:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.835+0000 7f79fee54740 10 client.25880 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T09:40:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.835+0000 7f79fee54740 10 client.25880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:40:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.835+0000 7f79fee54740 20 client.25880 populate_metadata read hostname 'smithi002' 2024-05-14T09:40:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.835+0000 7f79fee54740 1 --2- 172.21.15.2:0/3765905865 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55df6bbc21b0 0x55df6bfe5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.835+0000 7f79fee54740 4 client.25880 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:40:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.835+0000 7f79fee54740 1 -- 172.21.15.2:0/3765905865 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df6bed0050 con 0x55df6bbc21b0 2024-05-14T09:40:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.835+0000 7f79f5d39640 1 --2- 172.21.15.2:0/3765905865 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55df6bbc21b0 0x55df6bfe5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.835+0000 7f79d97fa640 10 client.25880 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.836+0000 7f79f5d39640 1 --2- 172.21.15.2:0/3765905865 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55df6bbc21b0 0x55df6bfe5ad0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bf3f5fe8b680e0b server_cookie=4df0e267cb7e009c in_seq=0 out_seq=0 2024-05-14T09:40:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.935+0000 7f79d97fa640 1 -- 172.21.15.2:0/3765905865 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df6bed0050 con 0x55df6bbc21b0 2024-05-14T09:40:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.935+0000 7f79d97fa640 10 client.25880 handle_command_reply: tid=0 2024-05-14T09:40:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.935+0000 7f79fee54740 1 client.25880 shutdown 2024-05-14T09:40:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.935+0000 7f79fee54740 1 -- 172.21.15.2:0/3765905865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bb992f0 msgr2=0x55df6bf72cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.935+0000 7f79fee54740 1 --2- 172.21.15.2:0/3765905865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bb992f0 0x55df6bf72cc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f79e000e9c0 tx=0x7f79e00e2c50 comp rx=0 tx=0).stop 2024-05-14T09:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.935+0000 7f79fee54740 1 -- 172.21.15.2:0/3765905865 shutdown_connections 2024-05-14T09:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.935+0000 7f79fee54740 1 --2- 172.21.15.2:0/3765905865 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55df6bbc21b0 0x55df6bfe5ad0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.935+0000 7f79fee54740 1 --2- 172.21.15.2:0/3765905865 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55df6bba14d0 0x55df6bf73bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.935+0000 7f79fee54740 1 --2- 172.21.15.2:0/3765905865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df6bb992f0 0x55df6bf72cc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.935+0000 7f79fee54740 1 --2- 172.21.15.2:0/3765905865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df6bb80d80 0x55df6bf72780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.935+0000 7f79fee54740 1 -- 172.21.15.2:0/3765905865 >> 172.21.15.2:0/3765905865 conn(0x55df6bbb6300 msgr2=0x55df6bf5d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.935+0000 7f79d97fa640 0 client.25880 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.935+0000 7f79fee54740 1 -- 172.21.15.2:0/3765905865 shutdown_connections 2024-05-14T09:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.935+0000 7f79fee54740 1 -- 172.21.15.2:0/3765905865 wait complete. 2024-05-14T09:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.935+0000 7f79fee54740 20 client.25880 trim_cache size 0 max 16384 2024-05-14T09:40:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:40:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:40:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:40:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:40:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.936+0000 7f79f7f9d640 1 -- 172.21.15.2:0/3993799534 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f79dc0b2920 msgr2=0x7f79dc0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.936+0000 7f79f7f9d640 1 --2- 172.21.15.2:0/3993799534 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f79dc0b2920 0x7f79dc0b4e30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55df6bf64e40 tx=0x7f79ec000bc0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.936+0000 7f79f7f9d640 1 -- 172.21.15.2:0/3993799534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79f0102e60 msgr2=0x7f79f019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.936+0000 7f79f7f9d640 1 --2- 172.21.15.2:0/3993799534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79f0102e60 0x7f79f019ca80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f79e000e9f0 tx=0x7f79e000eeb0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.936+0000 7f79f7f9d640 1 -- 172.21.15.2:0/3993799534 shutdown_connections 2024-05-14T09:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.936+0000 7f79f7f9d640 1 --2- 172.21.15.2:0/3993799534 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f79dc0b2920 0x7f79dc0b4e30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.936+0000 7f79f7f9d640 1 --2- 172.21.15.2:0/3993799534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79f01037b0 0x7f79f01a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.936+0000 7f79f7f9d640 1 --2- 172.21.15.2:0/3993799534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79f0102e60 0x7f79f019ca80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.936+0000 7f79f7f9d640 1 --2- 172.21.15.2:0/3993799534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79f0102510 0x7f79f019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.936+0000 7f79f7f9d640 1 -- 172.21.15.2:0/3993799534 >> 172.21.15.2:0/3993799534 conn(0x7f79f00fdd20 msgr2=0x7f79f010a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.936+0000 7f79f7f9d640 1 -- 172.21.15.2:0/3993799534 shutdown_connections 2024-05-14T09:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:32.936+0000 7f79f7f9d640 1 -- 172.21.15.2:0/3993799534 wait complete. 2024-05-14T09:40:33.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 459118 10.14 MB/sec execute 119 sec latency 56.217 ms 2024-05-14T09:40:34.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 461838 10.19 MB/sec execute 120 sec latency 13.540 ms 2024-05-14T09:40:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:34 smithi002 ceph-mon[31557]: pgmap v592: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 377 op/s 2024-05-14T09:40:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:34 smithi029 ceph-mon[36249]: pgmap v592: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 377 op/s 2024-05-14T09:40:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:34 smithi026 ceph-mon[36056]: pgmap v592: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 377 op/s 2024-05-14T09:40:35.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 463407 10.17 MB/sec execute 121 sec latency 19.582 ms 2024-05-14T09:40:36.264 INFO:tasks.workunit.client.0.smithi002.stdout: 1 465700 10.19 MB/sec execute 122 sec latency 50.922 ms 2024-05-14T09:40:36.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:36 smithi002 ceph-mon[31557]: pgmap v593: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-05-14T09:40:36.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:36 smithi029 ceph-mon[36249]: pgmap v593: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-05-14T09:40:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:36 smithi026 ceph-mon[36056]: pgmap v593: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-05-14T09:40:37.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 467743 10.17 MB/sec execute 123 sec latency 33.171 ms 2024-05-14T09:40:37.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:37 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:40:37.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:37 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:40:37.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:37 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:40:38.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 469495 10.19 MB/sec execute 124 sec latency 19.589 ms 2024-05-14T09:40:38.587 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:38 smithi029 ceph-mon[36249]: pgmap v594: 129 pgs: 129 active+clean; 88 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 299 op/s 2024-05-14T09:40:38.601 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:38 smithi026 ceph-mon[36056]: pgmap v594: 129 pgs: 129 active+clean; 88 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 299 op/s 2024-05-14T09:40:38.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:38 smithi002 ceph-mon[31557]: pgmap v594: 129 pgs: 129 active+clean; 88 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 299 op/s 2024-05-14T09:40:39.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 472282 10.22 MB/sec execute 125 sec latency 15.921 ms 2024-05-14T09:40:40.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 474027 10.22 MB/sec execute 126 sec latency 16.036 ms 2024-05-14T09:40:40.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:40 smithi002 ceph-mon[31557]: pgmap v595: 129 pgs: 129 active+clean; 84 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 346 op/s 2024-05-14T09:40:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:40 smithi029 ceph-mon[36249]: pgmap v595: 129 pgs: 129 active+clean; 84 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 346 op/s 2024-05-14T09:40:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:40 smithi026 ceph-mon[36056]: pgmap v595: 129 pgs: 129 active+clean; 84 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 346 op/s 2024-05-14T09:40:41.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 476307 10.25 MB/sec execute 127 sec latency 59.454 ms 2024-05-14T09:40:42.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 477778 10.21 MB/sec execute 128 sec latency 31.545 ms 2024-05-14T09:40:42.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:42 smithi002 ceph-mon[31557]: pgmap v596: 129 pgs: 129 active+clean; 84 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.4 MiB/s wr, 319 op/s 2024-05-14T09:40:42.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:42 smithi029 ceph-mon[36249]: pgmap v596: 129 pgs: 129 active+clean; 84 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.4 MiB/s wr, 319 op/s 2024-05-14T09:40:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:42 smithi026 ceph-mon[36056]: pgmap v596: 129 pgs: 129 active+clean; 84 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.4 MiB/s wr, 319 op/s 2024-05-14T09:40:43.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 480072 10.24 MB/sec execute 129 sec latency 15.642 ms 2024-05-14T09:40:44.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 482230 10.22 MB/sec execute 130 sec latency 15.314 ms 2024-05-14T09:40:44.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:44 smithi002 ceph-mon[31557]: pgmap v597: 129 pgs: 129 active+clean; 85 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 371 op/s 2024-05-14T09:40:44.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:44 smithi029 ceph-mon[36249]: pgmap v597: 129 pgs: 129 active+clean; 85 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 371 op/s 2024-05-14T09:40:44.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:44 smithi026 ceph-mon[36056]: pgmap v597: 129 pgs: 129 active+clean; 85 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 371 op/s 2024-05-14T09:40:45.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 483902 10.23 MB/sec execute 131 sec latency 15.590 ms 2024-05-14T09:40:45.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:45 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:40:45.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:45 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:40:45.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:45 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:40:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.039+0000 7f66866e2640 1 -- 172.21.15.2:0/1407583430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6680103c50 msgr2=0x7f6680104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.039+0000 7f66866e2640 1 --2- 172.21.15.2:0/1407583430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6680103c50 0x7f6680104950 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6670009900 tx=0x7f667002a410 comp rx=0 tx=0).stop 2024-05-14T09:40:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.039+0000 7f66866e2640 1 -- 172.21.15.2:0/1407583430 shutdown_connections 2024-05-14T09:40:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.039+0000 7f66866e2640 1 --2- 172.21.15.2:0/1407583430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6680109b90 0x7f668010bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.039+0000 7f66866e2640 1 --2- 172.21.15.2:0/1407583430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6680104e90 0x7f66801052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.039+0000 7f66866e2640 1 --2- 172.21.15.2:0/1407583430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6680103c50 0x7f6680104950 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.039+0000 7f66866e2640 1 -- 172.21.15.2:0/1407583430 >> 172.21.15.2:0/1407583430 conn(0x7f66800fee60 msgr2=0x7f66801012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.039+0000 7f66866e2640 1 -- 172.21.15.2:0/1407583430 shutdown_connections 2024-05-14T09:40:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.039+0000 7f66866e2640 1 -- 172.21.15.2:0/1407583430 wait complete. 2024-05-14T09:40:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.040+0000 7f66866e2640 1 Processor -- start 2024-05-14T09:40:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.040+0000 7f66866e2640 1 -- start start 2024-05-14T09:40:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.040+0000 7f66866e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6680103c50 0x7f668019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.040+0000 7f66866e2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6680104e90 0x7f668019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.040+0000 7f66866e2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6680109b90 0x7f66801a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.040+0000 7f66866e2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f668010f750 con 0x7f6680109b90 2024-05-14T09:40:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.040+0000 7f66866e2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f668010f5d0 con 0x7f6680103c50 2024-05-14T09:40:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.040+0000 7f66866e2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f668010f8d0 con 0x7f6680104e90 2024-05-14T09:40:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f6684c7f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6680109b90 0x7f66801a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f667ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6680103c50 0x7f668019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f6684c7f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6680109b90 0x7f66801a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47868/0 (socket says 172.21.15.2:47868) 2024-05-14T09:40:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f667ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6680103c50 0x7f668019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:43626/0 (socket says 172.21.15.2:43626) 2024-05-14T09:40:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f6684c7f640 1 -- 172.21.15.2:0/3452956285 learned_addr learned my addr 172.21.15.2:0/3452956285 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f667f7fe640 1 --2- 172.21.15.2:0/3452956285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6680104e90 0x7f668019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f6684c7f640 1 -- 172.21.15.2:0/3452956285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6680104e90 msgr2=0x7f668019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f6684c7f640 1 --2- 172.21.15.2:0/3452956285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6680104e90 0x7f668019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f6684c7f640 1 -- 172.21.15.2:0/3452956285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6680103c50 msgr2=0x7f668019a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f6684c7f640 1 --2- 172.21.15.2:0/3452956285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6680103c50 0x7f668019a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f6684c7f640 1 -- 172.21.15.2:0/3452956285 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66801a1ea0 con 0x7f6680109b90 2024-05-14T09:40:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f667f7fe640 1 --2- 172.21.15.2:0/3452956285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6680104e90 0x7f668019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:40:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f667ffff640 1 --2- 172.21.15.2:0/3452956285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6680103c50 0x7f668019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:40:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f6684c7f640 1 --2- 172.21.15.2:0/3452956285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6680109b90 0x7f66801a1830 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f667800eb10 tx=0x7f667800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f667d7fa640 1 -- 172.21.15.2:0/3452956285 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678004b20 con 0x7f6680109b90 2024-05-14T09:40:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f667d7fa640 1 -- 172.21.15.2:0/3452956285 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6678004cc0 con 0x7f6680109b90 2024-05-14T09:40:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f66866e2640 1 -- 172.21.15.2:0/3452956285 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66801a2190 con 0x7f6680109b90 2024-05-14T09:40:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f667d7fa640 1 -- 172.21.15.2:0/3452956285 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678010670 con 0x7f6680109b90 2024-05-14T09:40:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.041+0000 7f66866e2640 1 -- 172.21.15.2:0/3452956285 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66801a2690 con 0x7f6680109b90 2024-05-14T09:40:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.044+0000 7f668d590740 1 Processor -- start 2024-05-14T09:40:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.044+0000 7f667d7fa640 1 -- 172.21.15.2:0/3452956285 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6678005020 con 0x7f6680109b90 2024-05-14T09:40:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.044+0000 7f668d590740 1 -- start start 2024-05-14T09:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.044+0000 7f668d590740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 0x55f5c4053a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.044+0000 7f668d590740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4053fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.044+0000 7f668d590740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3c914d0 0x55f5c405aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.044+0000 7f668d590740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3fb85d0 con 0x55f5c3c892f0 2024-05-14T09:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.044+0000 7f668d590740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3f71890 con 0x55f5c3c914d0 2024-05-14T09:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.044+0000 7f668d590740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3fb4590 con 0x55f5c3cb21b0 2024-05-14T09:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.044+0000 7f667ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4053fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f667f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 0x55f5c4053a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f667ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4053fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47870/0 (socket says 172.21.15.2:47870) 2024-05-14T09:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f667ffff640 1 -- 172.21.15.2:0/1674039547 learned_addr learned my addr 172.21.15.2:0/1674039547 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f667f7fe640 1 --2- 172.21.15.2:0/1674039547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3c914d0 0x55f5c405aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f667ffff640 1 -- 172.21.15.2:0/1674039547 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 msgr2=0x55f5c4053a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f667ffff640 1 --2- 172.21.15.2:0/1674039547 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 0x55f5c4053a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f667ffff640 1 -- 172.21.15.2:0/1674039547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3c914d0 msgr2=0x55f5c405aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f667ffff640 1 --2- 172.21.15.2:0/1674039547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3c914d0 0x55f5c405aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f667ffff640 1 -- 172.21.15.2:0/1674039547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c3cca210 con 0x55f5c3c892f0 2024-05-14T09:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f667d7fa640 1 --2- 172.21.15.2:0/3452956285 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f66680b28a0 0x7f66680b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f667ffff640 1 --2- 172.21.15.2:0/1674039547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4053fe0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6670009230 tx=0x7f6670032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe5f6f8fddd9ce7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f667f7fe640 1 --2- 172.21.15.2:0/3452956285 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f66680b28a0 0x7f66680b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f66667fc640 1 -- 172.21.15.2:0/1674039547 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670040070 con 0x55f5c3c892f0 2024-05-14T09:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f667d7fa640 1 -- 172.21.15.2:0/3452956285 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f66780e2cb0 con 0x7f6680109b90 2024-05-14T09:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f66667fc640 1 -- 172.21.15.2:0/1674039547 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6670031820 con 0x55f5c3c892f0 2024-05-14T09:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.045+0000 7f66667fc640 1 -- 172.21.15.2:0/1674039547 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670035040 con 0x55f5c3c892f0 2024-05-14T09:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.046+0000 7f668d590740 1 -- 172.21.15.2:0/1674039547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 msgr2=0x55f5c4053fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.046+0000 7f667f7fe640 1 --2- 172.21.15.2:0/3452956285 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f66680b28a0 0x7f66680b4db0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f66740058c0 tx=0x7f6674005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.046+0000 7f668d590740 1 --2- 172.21.15.2:0/1674039547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4053fe0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6670009230 tx=0x7f6670032f90 comp rx=0 tx=0).stop 2024-05-14T09:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.046+0000 7f668d590740 1 -- 172.21.15.2:0/1674039547 shutdown_connections 2024-05-14T09:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.046+0000 7f668d590740 1 --2- 172.21.15.2:0/1674039547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3c914d0 0x55f5c405aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.046+0000 7f668d590740 1 --2- 172.21.15.2:0/1674039547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4053fe0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.046+0000 7f668d590740 1 --2- 172.21.15.2:0/1674039547 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 0x55f5c4053a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.046+0000 7f668d590740 1 -- 172.21.15.2:0/1674039547 >> 172.21.15.2:0/1674039547 conn(0x55f5c3ca6300 msgr2=0x55f5c404f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.046+0000 7f668d590740 1 -- 172.21.15.2:0/1674039547 shutdown_connections 2024-05-14T09:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.046+0000 7f668d590740 1 -- 172.21.15.2:0/1674039547 wait complete. 2024-05-14T09:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.047+0000 7f668d590740 1 -- start start 2024-05-14T09:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.047+0000 7f668d590740 1 Processor -- start 2024-05-14T09:40:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.047+0000 7f668d590740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c405d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.047+0000 7f668d590740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3c914d0 0x55f5c405db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.047+0000 7f668d590740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3cb21b0 0x55f5c4064ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.047+0000 7f668d590740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f5c405b8d0 con 0x55f5c3c892f0 2024-05-14T09:40:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.047+0000 7f668d590740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3f75370 con 0x55f5c3cb21b0 2024-05-14T09:40:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.047+0000 7f668d590740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f5c405ba50 con 0x55f5c3c914d0 2024-05-14T09:40:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.047+0000 7f668d590740 10 client.0 _ll_register_callbacks cb 0x55f5c3cc9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:40:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.047+0000 7f667ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c405d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.047+0000 7f667ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c405d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47876/0 (socket says 172.21.15.2:47876) 2024-05-14T09:40:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.047+0000 7f667ffff640 1 -- 172.21.15.2:0/3650052111 learned_addr learned my addr 172.21.15.2:0/3650052111 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.047+0000 7f667f7fe640 1 --2- 172.21.15.2:0/3650052111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3c914d0 0x55f5c405db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.048+0000 7f6684c7f640 1 --2- 172.21.15.2:0/3650052111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3cb21b0 0x55f5c4064ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.048+0000 7f667ffff640 1 -- 172.21.15.2:0/3650052111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3c914d0 msgr2=0x55f5c405db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.048+0000 7f667ffff640 1 --2- 172.21.15.2:0/3650052111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3c914d0 0x55f5c405db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.048+0000 7f667ffff640 1 -- 172.21.15.2:0/3650052111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3cb21b0 msgr2=0x55f5c4064ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.048+0000 7f667ffff640 1 --2- 172.21.15.2:0/3650052111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3cb21b0 0x55f5c4064ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.048+0000 7f667ffff640 1 -- 172.21.15.2:0/3650052111 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c3d96620 con 0x55f5c3c892f0 2024-05-14T09:40:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.048+0000 7f6684c7f640 1 --2- 172.21.15.2:0/3650052111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3cb21b0 0x55f5c4064ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:40:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.048+0000 7f667ffff640 1 --2- 172.21.15.2:0/3650052111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c405d610 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6670006010 tx=0x7f66700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0ec598c96640fd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.048+0000 7f665f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:40:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.048+0000 7f665f7fe640 1 -- 172.21.15.2:0/3650052111 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667004b070 con 0x55f5c3c892f0 2024-05-14T09:40:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.048+0000 7f665f7fe640 1 -- 172.21.15.2:0/3650052111 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f6670003800 con 0x55f5c3c892f0 2024-05-14T09:40:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.048+0000 7f668d590740 1 -- 172.21.15.2:0/3650052111 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6670031820 con 0x55f5c3c892f0 2024-05-14T09:40:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.049+0000 7f665f7fe640 1 -- 172.21.15.2:0/3650052111 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6670057050 con 0x55f5c3c892f0 2024-05-14T09:40:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.049+0000 7f668d590740 10 client.16206 fetch_fsmap learned FSMap version 32 2024-05-14T09:40:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.049+0000 7f668d590740 1 -- 172.21.15.2:0/3650052111 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f5c40679e0 con 0x55f5c3c892f0 2024-05-14T09:40:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.049+0000 7f665f7fe640 1 -- 172.21.15.2:0/3650052111 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6670031820 con 0x55f5c3c892f0 2024-05-14T09:40:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.049+0000 7f665f7fe640 1 -- 172.21.15.2:0/3650052111 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f667003c620 con 0x55f5c3c892f0 2024-05-14T09:40:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.049+0000 7f668d590740 10 client.16206 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T09:40:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.049+0000 7f668d590740 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:40:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.049+0000 7f668d590740 20 client.16206 populate_metadata read hostname 'smithi002' 2024-05-14T09:40:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.049+0000 7f668d590740 1 --2- 172.21.15.2:0/3650052111 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f5c3c70d80 0x55f5c406ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.049+0000 7f668d590740 4 client.16206 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:40:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.049+0000 7f668d590740 1 -- 172.21.15.2:0/3650052111 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5c3fc61e0 con 0x55f5c3c70d80 2024-05-14T09:40:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.049+0000 7f667f7fe640 1 --2- 172.21.15.2:0/3650052111 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f5c3c70d80 0x55f5c406ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.050+0000 7f667f7fe640 1 --2- 172.21.15.2:0/3650052111 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f5c3c70d80 0x55f5c406ad40 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b69c2556b110d0 server_cookie=bdf9a547a9c06edf in_seq=0 out_seq=0 2024-05-14T09:40:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.050+0000 7f665f7fe640 10 client.16206 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.051+0000 7f665f7fe640 1 -- 172.21.15.2:0/3650052111 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f5c3fc61e0 con 0x55f5c3c70d80 2024-05-14T09:40:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.051+0000 7f665f7fe640 10 client.16206 handle_command_reply: tid=0 2024-05-14T09:40:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.051+0000 7f668d590740 1 client.16206 shutdown 2024-05-14T09:40:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.051+0000 7f668d590740 1 -- 172.21.15.2:0/3650052111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 msgr2=0x55f5c405d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.051+0000 7f668d590740 1 --2- 172.21.15.2:0/3650052111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c405d610 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6670006010 tx=0x7f66700042f0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.051+0000 7f668d590740 1 -- 172.21.15.2:0/3650052111 shutdown_connections 2024-05-14T09:40:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.051+0000 7f668d590740 1 --2- 172.21.15.2:0/3650052111 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f5c3c70d80 0x55f5c406ad40 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.051+0000 7f668d590740 1 --2- 172.21.15.2:0/3650052111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3cb21b0 0x55f5c4064ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.051+0000 7f668d590740 1 --2- 172.21.15.2:0/3650052111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3c914d0 0x55f5c405db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.051+0000 7f668d590740 1 --2- 172.21.15.2:0/3650052111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c405d610 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.051+0000 7f665f7fe640 0 client.16206 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.051+0000 7f668d590740 1 -- 172.21.15.2:0/3650052111 >> 172.21.15.2:0/3650052111 conn(0x55f5c3ca6300 msgr2=0x55f5c4052300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.051+0000 7f668d590740 1 -- 172.21.15.2:0/3650052111 shutdown_connections 2024-05-14T09:40:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.051+0000 7f668d590740 1 -- 172.21.15.2:0/3650052111 wait complete. 2024-05-14T09:40:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.051+0000 7f668d590740 20 client.16206 trim_cache size 0 max 16384 2024-05-14T09:40:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.066+0000 7f668d590740 1 Processor -- start 2024-05-14T09:40:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.066+0000 7f668d590740 1 -- start start 2024-05-14T09:40:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.066+0000 7f668d590740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4058110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.066+0000 7f668d590740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3c914d0 0x55f5c404f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.066+0000 7f668d590740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 0x55f5c40649d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.066+0000 7f668d590740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f66801af5b0 con 0x55f5c3c892f0 2024-05-14T09:40:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.066+0000 7f668d590740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f66801ace50 con 0x55f5c3c914d0 2024-05-14T09:40:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.066+0000 7f668d590740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f66801aaca0 con 0x55f5c3cb21b0 2024-05-14T09:40:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.067+0000 7f667ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4058110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.067+0000 7f667ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4058110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47878/0 (socket says 172.21.15.2:47878) 2024-05-14T09:40:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.067+0000 7f667ffff640 1 -- 172.21.15.2:0/57415892 learned_addr learned my addr 172.21.15.2:0/57415892 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.067+0000 7f667f7fe640 1 --2- 172.21.15.2:0/57415892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3c914d0 0x55f5c404f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.067+0000 7f6684c7f640 1 --2- 172.21.15.2:0/57415892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 0x55f5c40649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.067+0000 7f667ffff640 1 -- 172.21.15.2:0/57415892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 msgr2=0x55f5c40649d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.067+0000 7f667ffff640 1 --2- 172.21.15.2:0/57415892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 0x55f5c40649d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.067+0000 7f667ffff640 1 -- 172.21.15.2:0/57415892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3c914d0 msgr2=0x55f5c404f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.067+0000 7f667ffff640 1 --2- 172.21.15.2:0/57415892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3c914d0 0x55f5c404f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.067+0000 7f667ffff640 1 -- 172.21.15.2:0/57415892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c404e980 con 0x55f5c3c892f0 2024-05-14T09:40:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.067+0000 7f6684c7f640 1 --2- 172.21.15.2:0/57415892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 0x55f5c40649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:40:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.067+0000 7f667ffff640 1 --2- 172.21.15.2:0/57415892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4058110 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f667003cb90 tx=0x7f6670005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76340b36cccd79e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.067+0000 7f6665ffb640 1 -- 172.21.15.2:0/57415892 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670060070 con 0x55f5c3c892f0 2024-05-14T09:40:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.067+0000 7f6665ffb640 1 -- 172.21.15.2:0/57415892 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6670054030 con 0x55f5c3c892f0 2024-05-14T09:40:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.067+0000 7f6665ffb640 1 -- 172.21.15.2:0/57415892 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667004b070 con 0x55f5c3c892f0 2024-05-14T09:40:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.068+0000 7f668d590740 1 -- 172.21.15.2:0/57415892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 msgr2=0x55f5c4058110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.068+0000 7f668d590740 1 --2- 172.21.15.2:0/57415892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4058110 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f667003cb90 tx=0x7f6670005a20 comp rx=0 tx=0).stop 2024-05-14T09:40:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.068+0000 7f668d590740 1 -- 172.21.15.2:0/57415892 shutdown_connections 2024-05-14T09:40:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.068+0000 7f668d590740 1 --2- 172.21.15.2:0/57415892 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 0x55f5c40649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.068+0000 7f668d590740 1 --2- 172.21.15.2:0/57415892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3c914d0 0x55f5c404f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.068+0000 7f668d590740 1 --2- 172.21.15.2:0/57415892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4058110 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.068+0000 7f668d590740 1 -- 172.21.15.2:0/57415892 >> 172.21.15.2:0/57415892 conn(0x55f5c3ca6300 msgr2=0x55f5c404ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.068+0000 7f668d590740 1 -- 172.21.15.2:0/57415892 shutdown_connections 2024-05-14T09:40:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.068+0000 7f668d590740 1 -- 172.21.15.2:0/57415892 wait complete. 2024-05-14T09:40:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.068+0000 7f668d590740 1 -- start start 2024-05-14T09:40:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f668d590740 1 Processor -- start 2024-05-14T09:40:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f668d590740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4061e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f668d590740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3c914d0 0x55f5c40623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f668d590740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 0x55f5c40d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f668d590740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f66801ab310 con 0x55f5c3c892f0 2024-05-14T09:40:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f668d590740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f66801ac780 con 0x55f5c3c914d0 2024-05-14T09:40:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f668d590740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f66801ab9e0 con 0x55f5c3cb21b0 2024-05-14T09:40:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f668d590740 10 client.0 _ll_register_callbacks cb 0x7f667400cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f667ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4061e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f667ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4061e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47882/0 (socket says 172.21.15.2:47882) 2024-05-14T09:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f667ffff640 1 -- 172.21.15.2:0/4284875679 learned_addr learned my addr 172.21.15.2:0/4284875679 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f667f7fe640 1 --2- 172.21.15.2:0/4284875679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3c914d0 0x55f5c40623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f6684c7f640 1 --2- 172.21.15.2:0/4284875679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 0x55f5c40d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f667ffff640 1 -- 172.21.15.2:0/4284875679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 msgr2=0x55f5c40d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f667ffff640 1 --2- 172.21.15.2:0/4284875679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 0x55f5c40d1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f667ffff640 1 -- 172.21.15.2:0/4284875679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3c914d0 msgr2=0x55f5c40623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f667ffff640 1 --2- 172.21.15.2:0/4284875679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3c914d0 0x55f5c40623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f667ffff640 1 -- 172.21.15.2:0/4284875679 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c4065380 con 0x55f5c3c892f0 2024-05-14T09:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.069+0000 7f6684c7f640 1 --2- 172.21.15.2:0/4284875679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 0x55f5c40d1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f667ffff640 1 --2- 172.21.15.2:0/4284875679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4061e80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6670031440 tx=0x7f6670009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c079f008891711e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f665f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:40:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f665f7fe640 1 -- 172.21.15.2:0/4284875679 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670069060 con 0x55f5c3c892f0 2024-05-14T09:40:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f665f7fe640 1 -- 172.21.15.2:0/4284875679 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f667003c640 con 0x55f5c3c892f0 2024-05-14T09:40:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f668d590740 1 -- 172.21.15.2:0/4284875679 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6670054030 con 0x55f5c3c892f0 2024-05-14T09:40:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f665f7fe640 1 -- 172.21.15.2:0/4284875679 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6670070050 con 0x55f5c3c892f0 2024-05-14T09:40:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f668d590740 10 client.16212 fetch_fsmap learned FSMap version 32 2024-05-14T09:40:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f668d590740 1 -- 172.21.15.2:0/4284875679 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f5c40d2390 con 0x55f5c3c892f0 2024-05-14T09:40:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f665f7fe640 1 -- 172.21.15.2:0/4284875679 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6670054030 con 0x55f5c3c892f0 2024-05-14T09:40:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f665f7fe640 1 -- 172.21.15.2:0/4284875679 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f6670003820 con 0x55f5c3c892f0 2024-05-14T09:40:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f668d590740 10 client.16212 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T09:40:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f668d590740 10 client.16212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:40:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f668d590740 20 client.16212 populate_metadata read hostname 'smithi002' 2024-05-14T09:40:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f668d590740 1 --2- 172.21.15.2:0/4284875679 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f5c3c461c0 0x55f5c40d4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f668d590740 4 client.16212 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:40:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.070+0000 7f668d590740 1 -- 172.21.15.2:0/4284875679 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5c3fc0050 con 0x55f5c3c461c0 2024-05-14T09:40:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.071+0000 7f667f7fe640 1 --2- 172.21.15.2:0/4284875679 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f5c3c461c0 0x55f5c40d4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.071+0000 7f665f7fe640 10 client.16212 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.071+0000 7f667f7fe640 1 --2- 172.21.15.2:0/4284875679 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f5c3c461c0 0x55f5c40d4cd0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b27da039e22f5a server_cookie=fc8c04d226f4f189 in_seq=0 out_seq=0 2024-05-14T09:40:46.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 486208 10.24 MB/sec execute 132 sec latency 151.821 ms 2024-05-14T09:40:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.358+0000 7f665f7fe640 1 -- 172.21.15.2:0/4284875679 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f5c3fc0050 con 0x55f5c3c461c0 2024-05-14T09:40:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.358+0000 7f665f7fe640 10 client.16212 handle_command_reply: tid=0 2024-05-14T09:40:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.358+0000 7f668d590740 1 client.16212 shutdown 2024-05-14T09:40:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.358+0000 7f668d590740 1 -- 172.21.15.2:0/4284875679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 msgr2=0x55f5c4061e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.358+0000 7f668d590740 1 --2- 172.21.15.2:0/4284875679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4061e80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6670031440 tx=0x7f6670009260 comp rx=0 tx=0).stop 2024-05-14T09:40:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.358+0000 7f668d590740 1 -- 172.21.15.2:0/4284875679 shutdown_connections 2024-05-14T09:40:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.358+0000 7f668d590740 1 --2- 172.21.15.2:0/4284875679 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f5c3c461c0 0x55f5c40d4cd0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.358+0000 7f668d590740 1 --2- 172.21.15.2:0/4284875679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f5c3cb21b0 0x55f5c40d1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.358+0000 7f668d590740 1 --2- 172.21.15.2:0/4284875679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5c3c914d0 0x55f5c40623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.358+0000 7f665f7fe640 0 client.16212 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.358+0000 7f668d590740 1 --2- 172.21.15.2:0/4284875679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f5c3c892f0 0x55f5c4061e80 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.358+0000 7f668d590740 1 -- 172.21.15.2:0/4284875679 >> 172.21.15.2:0/4284875679 conn(0x55f5c3ca6300 msgr2=0x55f5c4052980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.358+0000 7f668d590740 1 -- 172.21.15.2:0/4284875679 shutdown_connections 2024-05-14T09:40:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.358+0000 7f668d590740 1 -- 172.21.15.2:0/4284875679 wait complete. 2024-05-14T09:40:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.358+0000 7f668d590740 20 client.16212 trim_cache size 0 max 16384 2024-05-14T09:40:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:40:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:40:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:40:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:40:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.359+0000 7f66866e2640 1 -- 172.21.15.2:0/3452956285 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f66680b28a0 msgr2=0x7f66680b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.359+0000 7f66866e2640 1 --2- 172.21.15.2:0/3452956285 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f66680b28a0 0x7f66680b4db0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f66740058c0 tx=0x7f6674005850 comp rx=0 tx=0).stop 2024-05-14T09:40:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.359+0000 7f66866e2640 1 -- 172.21.15.2:0/3452956285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6680109b90 msgr2=0x7f66801a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.359+0000 7f66866e2640 1 --2- 172.21.15.2:0/3452956285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6680109b90 0x7f66801a1830 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f667800eb10 tx=0x7f667800c2d0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.359+0000 7f66866e2640 1 -- 172.21.15.2:0/3452956285 shutdown_connections 2024-05-14T09:40:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.359+0000 7f66866e2640 1 --2- 172.21.15.2:0/3452956285 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f66680b28a0 0x7f66680b4db0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.359+0000 7f66866e2640 1 --2- 172.21.15.2:0/3452956285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6680109b90 0x7f66801a1830 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.359+0000 7f66866e2640 1 --2- 172.21.15.2:0/3452956285 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6680104e90 0x7f668019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.359+0000 7f66866e2640 1 --2- 172.21.15.2:0/3452956285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6680103c50 0x7f668019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.359+0000 7f66866e2640 1 -- 172.21.15.2:0/3452956285 >> 172.21.15.2:0/3452956285 conn(0x7f66800fee60 msgr2=0x7f6680105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.359+0000 7f66866e2640 1 -- 172.21.15.2:0/3452956285 shutdown_connections 2024-05-14T09:40:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:46.359+0000 7f66866e2640 1 -- 172.21.15.2:0/3452956285 wait complete. 2024-05-14T09:40:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:46 smithi002 ceph-mon[31557]: pgmap v598: 129 pgs: 129 active+clean; 85 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 2.3 MiB/s wr, 263 op/s 2024-05-14T09:40:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:40:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:40:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:40:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:40:46.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:46 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi029 to 177.2M 2024-05-14T09:40:46.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:46 smithi002 ceph-mon[31557]: Unable to set osd_memory_target on smithi029 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-14T09:40:46.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:40:46.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:40:46.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:40:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:46 smithi029 ceph-mon[36249]: pgmap v598: 129 pgs: 129 active+clean; 85 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 2.3 MiB/s wr, 263 op/s 2024-05-14T09:40:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:40:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:40:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:40:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:40:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:46 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi029 to 177.2M 2024-05-14T09:40:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:46 smithi029 ceph-mon[36249]: Unable to set osd_memory_target on smithi029 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-14T09:40:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:40:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:40:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:40:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:46 smithi026 ceph-mon[36056]: pgmap v598: 129 pgs: 129 active+clean; 85 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 2.3 MiB/s wr, 263 op/s 2024-05-14T09:40:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:40:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:40:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:40:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:40:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:46 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi029 to 177.2M 2024-05-14T09:40:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:46 smithi026 ceph-mon[36056]: Unable to set osd_memory_target on smithi029 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-14T09:40:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:40:46.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:40:46.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:40:47.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 487483 10.22 MB/sec execute 133 sec latency 195.466 ms 2024-05-14T09:40:48.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 490249 10.27 MB/sec execute 134 sec latency 16.422 ms 2024-05-14T09:40:48.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:48 smithi002 ceph-mon[31557]: pgmap v599: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 2.9 MiB/s wr, 360 op/s 2024-05-14T09:40:48.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:48 smithi029 ceph-mon[36249]: pgmap v599: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 2.9 MiB/s wr, 360 op/s 2024-05-14T09:40:48.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:48 smithi026 ceph-mon[36056]: pgmap v599: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 2.9 MiB/s wr, 360 op/s 2024-05-14T09:40:49.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 491121 10.21 MB/sec execute 135 sec latency 251.788 ms 2024-05-14T09:40:50.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 493970 10.26 MB/sec execute 136 sec latency 260.697 ms 2024-05-14T09:40:50.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:50 smithi002 ceph-mon[31557]: pgmap v600: 129 pgs: 129 active+clean; 86 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 2.8 MiB/s wr, 396 op/s 2024-05-14T09:40:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:50 smithi029 ceph-mon[36249]: pgmap v600: 129 pgs: 129 active+clean; 86 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 2.8 MiB/s wr, 396 op/s 2024-05-14T09:40:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:50 smithi026 ceph-mon[36056]: pgmap v600: 129 pgs: 129 active+clean; 86 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 2.8 MiB/s wr, 396 op/s 2024-05-14T09:40:51.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 495698 10.24 MB/sec execute 137 sec latency 15.120 ms 2024-05-14T09:40:52.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 497683 10.26 MB/sec execute 138 sec latency 47.219 ms 2024-05-14T09:40:52.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:52 smithi002 ceph-mon[31557]: pgmap v601: 129 pgs: 129 active+clean; 82 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.4 MiB/s wr, 342 op/s 2024-05-14T09:40:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:52 smithi029 ceph-mon[36249]: pgmap v601: 129 pgs: 129 active+clean; 82 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.4 MiB/s wr, 342 op/s 2024-05-14T09:40:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:52 smithi026 ceph-mon[36056]: pgmap v601: 129 pgs: 129 active+clean; 82 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.4 MiB/s wr, 342 op/s 2024-05-14T09:40:53.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 499786 10.24 MB/sec execute 139 sec latency 41.932 ms 2024-05-14T09:40:54.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 501338 10.25 MB/sec execute 140 sec latency 50.832 ms 2024-05-14T09:40:54.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:54 smithi002 ceph-mon[31557]: pgmap v602: 129 pgs: 129 active+clean; 86 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 3.1 MiB/s wr, 390 op/s 2024-05-14T09:40:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:54 smithi029 ceph-mon[36249]: pgmap v602: 129 pgs: 129 active+clean; 86 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 3.1 MiB/s wr, 390 op/s 2024-05-14T09:40:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:54 smithi026 ceph-mon[36056]: pgmap v602: 129 pgs: 129 active+clean; 86 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 3.1 MiB/s wr, 390 op/s 2024-05-14T09:40:55.265 INFO:tasks.workunit.client.0.smithi002.stdout: 1 503914 10.27 MB/sec execute 141 sec latency 14.580 ms 2024-05-14T09:40:56.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 505698 10.26 MB/sec execute 142 sec latency 15.124 ms 2024-05-14T09:40:56.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:56 smithi002 ceph-mon[31557]: pgmap v603: 129 pgs: 129 active+clean; 86 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.2 MiB/s wr, 333 op/s 2024-05-14T09:40:56.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:56 smithi029 ceph-mon[36249]: pgmap v603: 129 pgs: 129 active+clean; 86 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.2 MiB/s wr, 333 op/s 2024-05-14T09:40:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:56 smithi026 ceph-mon[36056]: pgmap v603: 129 pgs: 129 active+clean; 86 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.2 MiB/s wr, 333 op/s 2024-05-14T09:40:57.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 508295 10.30 MB/sec execute 143 sec latency 76.138 ms 2024-05-14T09:40:58.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 510412 10.28 MB/sec execute 144 sec latency 31.911 ms 2024-05-14T09:40:58.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:40:58 smithi002 ceph-mon[31557]: pgmap v604: 129 pgs: 129 active+clean; 87 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 2.8 MiB/s wr, 361 op/s 2024-05-14T09:40:58.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:40:58 smithi029 ceph-mon[36249]: pgmap v604: 129 pgs: 129 active+clean; 87 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 2.8 MiB/s wr, 361 op/s 2024-05-14T09:40:58.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:40:58 smithi026 ceph-mon[36056]: pgmap v604: 129 pgs: 129 active+clean; 87 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 2.8 MiB/s wr, 361 op/s 2024-05-14T09:40:59.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 512163 10.29 MB/sec execute 145 sec latency 15.385 ms 2024-05-14T09:40:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.477+0000 7faf71284640 1 -- 172.21.15.2:0/538961482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf6c102b50 msgr2=0x7faf6c103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.477+0000 7faf71284640 1 --2- 172.21.15.2:0/538961482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf6c102b50 0x7faf6c103850 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009900 tx=0x7faf5c02a410 comp rx=0 tx=0).stop 2024-05-14T09:40:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.477+0000 7faf71284640 1 -- 172.21.15.2:0/538961482 shutdown_connections 2024-05-14T09:40:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.477+0000 7faf71284640 1 --2- 172.21.15.2:0/538961482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf6c108a90 0x7faf6c10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.477+0000 7faf71284640 1 --2- 172.21.15.2:0/538961482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf6c103d90 0x7faf6c1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.477+0000 7faf71284640 1 --2- 172.21.15.2:0/538961482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf6c102b50 0x7faf6c103850 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.477+0000 7faf71284640 1 -- 172.21.15.2:0/538961482 >> 172.21.15.2:0/538961482 conn(0x7faf6c0fdd40 msgr2=0x7faf6c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.477+0000 7faf71284640 1 -- 172.21.15.2:0/538961482 shutdown_connections 2024-05-14T09:40:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.478+0000 7faf71284640 1 -- 172.21.15.2:0/538961482 wait complete. 2024-05-14T09:40:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.478+0000 7faf71284640 1 Processor -- start 2024-05-14T09:40:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.478+0000 7faf71284640 1 -- start start 2024-05-14T09:40:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf71284640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf6c102b50 0x7faf6c06b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf71284640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf6c103d90 0x7faf6c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf71284640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf6c108a90 0x7faf6c1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf71284640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c10e650 con 0x7faf6c102b50 2024-05-14T09:40:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf71284640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c10e4d0 con 0x7faf6c103d90 2024-05-14T09:40:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf71284640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c10e7d0 con 0x7faf6c108a90 2024-05-14T09:40:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf6affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf6c102b50 0x7faf6c06b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf6affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf6c102b50 0x7faf6c06b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54502/0 (socket says 172.21.15.2:54502) 2024-05-14T09:40:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf6affd640 1 -- 172.21.15.2:0/3653448186 learned_addr learned my addr 172.21.15.2:0/3653448186 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf627fc640 1 --2- 172.21.15.2:0/3653448186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf6c103d90 0x7faf6c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf6b7fe640 1 --2- 172.21.15.2:0/3653448186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf6c108a90 0x7faf6c1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf6affd640 1 -- 172.21.15.2:0/3653448186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf6c108a90 msgr2=0x7faf6c1a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf6affd640 1 --2- 172.21.15.2:0/3653448186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf6c108a90 0x7faf6c1a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf6affd640 1 -- 172.21.15.2:0/3653448186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf6c103d90 msgr2=0x7faf6c19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf6affd640 1 --2- 172.21.15.2:0/3653448186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf6c103d90 0x7faf6c19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf6affd640 1 -- 172.21.15.2:0/3653448186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6c1a1de0 con 0x7faf6c102b50 2024-05-14T09:40:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.479+0000 7faf627fc640 1 --2- 172.21.15.2:0/3653448186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf6c103d90 0x7faf6c19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:40:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.480+0000 7faf6affd640 1 --2- 172.21.15.2:0/3653448186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf6c102b50 0x7faf6c06b0f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7faf5c006010 tx=0x7faf5c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.480+0000 7faf68ff9640 1 -- 172.21.15.2:0/3653448186 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c03f030 con 0x7faf6c102b50 2024-05-14T09:40:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.480+0000 7faf68ff9640 1 -- 172.21.15.2:0/3653448186 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf5c031910 con 0x7faf6c102b50 2024-05-14T09:40:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.480+0000 7faf71284640 1 -- 172.21.15.2:0/3653448186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6c1a2070 con 0x7faf6c102b50 2024-05-14T09:40:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.480+0000 7faf68ff9640 1 -- 172.21.15.2:0/3653448186 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c035070 con 0x7faf6c102b50 2024-05-14T09:40:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.480+0000 7faf71284640 1 -- 172.21.15.2:0/3653448186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf6c1a2560 con 0x7faf6c102b50 2024-05-14T09:40:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.483+0000 7faf68ff9640 1 -- 172.21.15.2:0/3653448186 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7faf5c042030 con 0x7faf6c102b50 2024-05-14T09:40:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.483+0000 7faf78207740 1 Processor -- start 2024-05-14T09:40:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.483+0000 7faf78207740 1 -- start start 2024-05-14T09:40:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.483+0000 7faf78207740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc2121b0 0x557ccc5b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.483+0000 7faf78207740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc1e92f0 0x557ccc5b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.483+0000 7faf78207740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1f14d0 0x557ccc5baee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.483+0000 7faf78207740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557ccc5185d0 con 0x557ccc1e92f0 2024-05-14T09:40:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.483+0000 7faf78207740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557ccc4d1890 con 0x557ccc1f14d0 2024-05-14T09:40:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.483+0000 7faf78207740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557ccc514590 con 0x557ccc2121b0 2024-05-14T09:40:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.483+0000 7faf627fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1f14d0 0x557ccc5baee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.483+0000 7faf6affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc1e92f0 0x557ccc5b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.484+0000 7faf6affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc1e92f0 0x557ccc5b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54504/0 (socket says 172.21.15.2:54504) 2024-05-14T09:40:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.484+0000 7faf6affd640 1 -- 172.21.15.2:0/3414482037 learned_addr learned my addr 172.21.15.2:0/3414482037 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.483+0000 7faf627fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1f14d0 0x557ccc5baee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:39836/0 (socket says 172.21.15.2:39836) 2024-05-14T09:40:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.484+0000 7faf6b7fe640 1 --2- 172.21.15.2:0/3414482037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc2121b0 0x557ccc5b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.484+0000 7faf68ff9640 1 --2- 172.21.15.2:0/3653448186 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7faf580b28a0 0x7faf580b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.484+0000 7faf627fc640 1 -- 172.21.15.2:0/3414482037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc2121b0 msgr2=0x557ccc5b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.484+0000 7faf627fc640 1 --2- 172.21.15.2:0/3414482037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc2121b0 0x557ccc5b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.484+0000 7faf627fc640 1 -- 172.21.15.2:0/3414482037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc1e92f0 msgr2=0x557ccc5b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.484+0000 7faf627fc640 1 --2- 172.21.15.2:0/3414482037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc1e92f0 0x557ccc5b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.484+0000 7faf627fc640 1 -- 172.21.15.2:0/3414482037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ccc22a210 con 0x557ccc1f14d0 2024-05-14T09:40:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.484+0000 7faf6affd640 1 --2- 172.21.15.2:0/3414482037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc1e92f0 0x557ccc5b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:40:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.484+0000 7faf6b7fe640 1 --2- 172.21.15.2:0/3414482037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc2121b0 0x557ccc5b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:40:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.484+0000 7faf68ff9640 1 -- 172.21.15.2:0/3653448186 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7faf5c102990 con 0x7faf6c102b50 2024-05-14T09:40:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.484+0000 7faf6b7fe640 1 --2- 172.21.15.2:0/3653448186 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7faf580b28a0 0x7faf580b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.484+0000 7faf627fc640 1 --2- 172.21.15.2:0/3414482037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1f14d0 0x557ccc5baee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faf54009fa0 tx=0x7faf540118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff4625b2af95152 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.484+0000 7faf617fa640 1 -- 172.21.15.2:0/3414482037 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54004030 con 0x557ccc1f14d0 2024-05-14T09:40:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.485+0000 7faf617fa640 1 -- 172.21.15.2:0/3414482037 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf54005020 con 0x557ccc1f14d0 2024-05-14T09:40:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.485+0000 7faf617fa640 1 -- 172.21.15.2:0/3414482037 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54004bc0 con 0x557ccc1f14d0 2024-05-14T09:40:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.485+0000 7faf6b7fe640 1 --2- 172.21.15.2:0/3653448186 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7faf580b28a0 0x7faf580b4db0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x557ccc5b4c30 tx=0x7faf64004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.486+0000 7faf78207740 1 -- 172.21.15.2:0/3414482037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1f14d0 msgr2=0x557ccc5baee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.486+0000 7faf78207740 1 --2- 172.21.15.2:0/3414482037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1f14d0 0x557ccc5baee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faf54009fa0 tx=0x7faf540118d0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.486+0000 7faf78207740 1 -- 172.21.15.2:0/3414482037 shutdown_connections 2024-05-14T09:40:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.486+0000 7faf78207740 1 --2- 172.21.15.2:0/3414482037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1f14d0 0x557ccc5baee0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.486+0000 7faf78207740 1 --2- 172.21.15.2:0/3414482037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc1e92f0 0x557ccc5b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.486+0000 7faf78207740 1 --2- 172.21.15.2:0/3414482037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc2121b0 0x557ccc5b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.486+0000 7faf78207740 1 -- 172.21.15.2:0/3414482037 >> 172.21.15.2:0/3414482037 conn(0x557ccc206300 msgr2=0x557ccc5af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.486+0000 7faf78207740 1 -- 172.21.15.2:0/3414482037 shutdown_connections 2024-05-14T09:40:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.486+0000 7faf78207740 1 -- 172.21.15.2:0/3414482037 wait complete. 2024-05-14T09:40:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.487+0000 7faf78207740 1 -- start start 2024-05-14T09:40:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.487+0000 7faf78207740 1 Processor -- start 2024-05-14T09:40:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.487+0000 7faf78207740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 0x557ccc5bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.487+0000 7faf78207740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc1f14d0 0x557ccc5bdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.487+0000 7faf78207740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc2121b0 0x557ccc5c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.487+0000 7faf78207740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557ccc5bb980 con 0x557ccc2121b0 2024-05-14T09:40:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.487+0000 7faf78207740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557ccc4d5370 con 0x557ccc1e92f0 2024-05-14T09:40:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.487+0000 7faf78207740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557ccc5bbb00 con 0x557ccc1f14d0 2024-05-14T09:40:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.487+0000 7faf78207740 10 client.0 _ll_register_callbacks cb 0x557ccc229a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:40:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.487+0000 7faf6affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 0x557ccc5bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.487+0000 7faf627fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc1f14d0 0x557ccc5bdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.487+0000 7faf6affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 0x557ccc5bd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:39846/0 (socket says 172.21.15.2:39846) 2024-05-14T09:40:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.487+0000 7faf6affd640 1 -- 172.21.15.2:0/961352110 learned_addr learned my addr 172.21.15.2:0/961352110 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.487+0000 7faf6b7fe640 1 --2- 172.21.15.2:0/961352110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc2121b0 0x557ccc5c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.488+0000 7faf627fc640 1 -- 172.21.15.2:0/961352110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 msgr2=0x557ccc5bd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.488+0000 7faf627fc640 1 --2- 172.21.15.2:0/961352110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 0x557ccc5bd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.488+0000 7faf627fc640 1 -- 172.21.15.2:0/961352110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc2121b0 msgr2=0x557ccc5c4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.488+0000 7faf627fc640 1 --2- 172.21.15.2:0/961352110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc2121b0 0x557ccc5c4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.488+0000 7faf627fc640 1 -- 172.21.15.2:0/961352110 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ccc2f6620 con 0x557ccc1f14d0 2024-05-14T09:40:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.488+0000 7faf6b7fe640 1 --2- 172.21.15.2:0/961352110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc2121b0 0x557ccc5c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:40:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.488+0000 7faf627fc640 1 --2- 172.21.15.2:0/961352110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc1f14d0 0x557ccc5bdca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faf54000c00 tx=0x7faf54004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f9d0cd220f6e56f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.488+0000 7faf527fc640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:40:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.488+0000 7faf527fc640 1 -- 172.21.15.2:0/961352110 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5401d050 con 0x557ccc1f14d0 2024-05-14T09:40:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.488+0000 7faf527fc640 1 -- 172.21.15.2:0/961352110 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7faf54016040 con 0x557ccc1f14d0 2024-05-14T09:40:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.488+0000 7faf78207740 1 -- 172.21.15.2:0/961352110 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf54005020 con 0x557ccc1f14d0 2024-05-14T09:40:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.488+0000 7faf527fc640 1 -- 172.21.15.2:0/961352110 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faf54031050 con 0x557ccc1f14d0 2024-05-14T09:40:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.488+0000 7faf78207740 10 client.25900 fetch_fsmap learned FSMap version 33 2024-05-14T09:40:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.489+0000 7faf78207740 1 -- 172.21.15.2:0/961352110 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557ccc5c7ac0 con 0x557ccc1f14d0 2024-05-14T09:40:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.489+0000 7faf527fc640 1 -- 172.21.15.2:0/961352110 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf5401e040 con 0x557ccc1f14d0 2024-05-14T09:40:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.489+0000 7faf527fc640 1 -- 172.21.15.2:0/961352110 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7faf54003b80 con 0x557ccc1f14d0 2024-05-14T09:40:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.489+0000 7faf78207740 10 client.25900 fetch_fsmap finished waiting for FSMap version 33 2024-05-14T09:40:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.489+0000 7faf78207740 10 client.25900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:40:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.489+0000 7faf78207740 20 client.25900 populate_metadata read hostname 'smithi002' 2024-05-14T09:40:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.489+0000 7faf78207740 1 --2- 172.21.15.2:0/961352110 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557ccc1d0d80 0x557ccc5cb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.489+0000 7faf78207740 4 client.25900 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:40:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.489+0000 7faf78207740 1 -- 172.21.15.2:0/961352110 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7faf54005020 con 0x557ccc1d0d80 2024-05-14T09:40:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.489+0000 7faf6affd640 1 --2- 172.21.15.2:0/961352110 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557ccc1d0d80 0x557ccc5cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.490+0000 7faf527fc640 10 client.25900 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.490+0000 7faf6affd640 1 --2- 172.21.15.2:0/961352110 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557ccc1d0d80 0x557ccc5cb050 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967e7e263b23a46 server_cookie=db2ca3ebb2cc9787 in_seq=0 out_seq=0 2024-05-14T09:40:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.491+0000 7faf527fc640 1 -- 172.21.15.2:0/961352110 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7faf54005020 con 0x557ccc1d0d80 2024-05-14T09:40:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.491+0000 7faf527fc640 10 client.25900 handle_command_reply: tid=0 2024-05-14T09:40:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.491+0000 7faf78207740 1 client.25900 shutdown 2024-05-14T09:40:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.492+0000 7faf78207740 1 -- 172.21.15.2:0/961352110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc1f14d0 msgr2=0x557ccc5bdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.492+0000 7faf78207740 1 --2- 172.21.15.2:0/961352110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc1f14d0 0x557ccc5bdca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faf54000c00 tx=0x7faf54004270 comp rx=0 tx=0).stop 2024-05-14T09:40:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.492+0000 7faf78207740 1 -- 172.21.15.2:0/961352110 shutdown_connections 2024-05-14T09:40:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.492+0000 7faf78207740 1 --2- 172.21.15.2:0/961352110 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557ccc1d0d80 0x557ccc5cb050 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.492+0000 7faf78207740 1 --2- 172.21.15.2:0/961352110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc2121b0 0x557ccc5c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.492+0000 7faf78207740 1 --2- 172.21.15.2:0/961352110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc1f14d0 0x557ccc5bdca0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.492+0000 7faf527fc640 0 client.25900 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.492+0000 7faf78207740 1 --2- 172.21.15.2:0/961352110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 0x557ccc5bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.492+0000 7faf78207740 1 -- 172.21.15.2:0/961352110 >> 172.21.15.2:0/961352110 conn(0x557ccc206300 msgr2=0x557ccc5ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.492+0000 7faf78207740 1 -- 172.21.15.2:0/961352110 shutdown_connections 2024-05-14T09:40:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.492+0000 7faf78207740 1 -- 172.21.15.2:0/961352110 wait complete. 2024-05-14T09:40:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.492+0000 7faf78207740 20 client.25900 trim_cache size 0 max 16384 2024-05-14T09:40:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.510+0000 7faf78207740 1 Processor -- start 2024-05-14T09:40:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.510+0000 7faf78207740 1 -- start start 2024-05-14T09:40:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.510+0000 7faf78207740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 0x557ccc5af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.510+0000 7faf78207740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc1f14d0 0x557ccc5bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.510+0000 7faf78207740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc2121b0 0x557ccc5bc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.510+0000 7faf78207740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c1af590 con 0x557ccc1f14d0 2024-05-14T09:40:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.510+0000 7faf78207740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c1ace30 con 0x557ccc1e92f0 2024-05-14T09:40:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.510+0000 7faf78207740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c1aac80 con 0x557ccc2121b0 2024-05-14T09:40:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.510+0000 7faf627fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc1f14d0 0x557ccc5bbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.510+0000 7faf627fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc1f14d0 0x557ccc5bbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54532/0 (socket says 172.21.15.2:54532) 2024-05-14T09:40:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.510+0000 7faf627fc640 1 -- 172.21.15.2:0/3215615069 learned_addr learned my addr 172.21.15.2:0/3215615069 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.510+0000 7faf6affd640 1 --2- 172.21.15.2:0/3215615069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 0x557ccc5af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.511+0000 7faf6b7fe640 1 --2- 172.21.15.2:0/3215615069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc2121b0 0x557ccc5bc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.511+0000 7faf627fc640 1 -- 172.21.15.2:0/3215615069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc2121b0 msgr2=0x557ccc5bc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.511+0000 7faf627fc640 1 --2- 172.21.15.2:0/3215615069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc2121b0 0x557ccc5bc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.511+0000 7faf627fc640 1 -- 172.21.15.2:0/3215615069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 msgr2=0x557ccc5af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.511+0000 7faf627fc640 1 --2- 172.21.15.2:0/3215615069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 0x557ccc5af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.511+0000 7faf627fc640 1 -- 172.21.15.2:0/3215615069 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ccc5bb2a0 con 0x557ccc1f14d0 2024-05-14T09:40:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.511+0000 7faf6b7fe640 1 --2- 172.21.15.2:0/3215615069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc2121b0 0x557ccc5bc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:40:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.511+0000 7faf6affd640 1 --2- 172.21.15.2:0/3215615069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 0x557ccc5af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:40:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.511+0000 7faf627fc640 1 --2- 172.21.15.2:0/3215615069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc1f14d0 0x557ccc5bbed0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7faf5400a010 tx=0x7faf54005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c21efa49f71239e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.511+0000 7faf60ff9640 1 -- 172.21.15.2:0/3215615069 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5403a070 con 0x557ccc1f14d0 2024-05-14T09:40:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.511+0000 7faf60ff9640 1 -- 172.21.15.2:0/3215615069 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf5400f840 con 0x557ccc1f14d0 2024-05-14T09:40:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.511+0000 7faf60ff9640 1 -- 172.21.15.2:0/3215615069 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5401d070 con 0x557ccc1f14d0 2024-05-14T09:40:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.512+0000 7faf78207740 1 -- 172.21.15.2:0/3215615069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc1f14d0 msgr2=0x557ccc5bbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.512+0000 7faf78207740 1 --2- 172.21.15.2:0/3215615069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc1f14d0 0x557ccc5bbed0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7faf5400a010 tx=0x7faf54005b90 comp rx=0 tx=0).stop 2024-05-14T09:40:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.512+0000 7faf78207740 1 -- 172.21.15.2:0/3215615069 shutdown_connections 2024-05-14T09:40:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.512+0000 7faf78207740 1 --2- 172.21.15.2:0/3215615069 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc2121b0 0x557ccc5bc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.512+0000 7faf78207740 1 --2- 172.21.15.2:0/3215615069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc1f14d0 0x557ccc5bbed0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.512+0000 7faf78207740 1 --2- 172.21.15.2:0/3215615069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 0x557ccc5af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.512+0000 7faf78207740 1 -- 172.21.15.2:0/3215615069 >> 172.21.15.2:0/3215615069 conn(0x557ccc206300 msgr2=0x557ccc5aecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.512+0000 7faf78207740 1 -- 172.21.15.2:0/3215615069 shutdown_connections 2024-05-14T09:40:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.512+0000 7faf78207740 1 -- 172.21.15.2:0/3215615069 wait complete. 2024-05-14T09:40:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.513+0000 7faf78207740 1 -- start start 2024-05-14T09:40:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.513+0000 7faf78207740 1 Processor -- start 2024-05-14T09:40:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.513+0000 7faf78207740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 0x557ccc5c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.513+0000 7faf78207740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc1f14d0 0x557ccc5c2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.513+0000 7faf78207740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc2121b0 0x557ccc5c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.513+0000 7faf78207740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c1ab2f0 con 0x557ccc2121b0 2024-05-14T09:40:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.513+0000 7faf78207740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c1ac760 con 0x557ccc1e92f0 2024-05-14T09:40:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.513+0000 7faf78207740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c1ab9c0 con 0x557ccc1f14d0 2024-05-14T09:40:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.513+0000 7faf78207740 10 client.0 _ll_register_callbacks cb 0x7faf5c1083a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:40:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.513+0000 7faf6b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc2121b0 0x557ccc5c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.513+0000 7faf6b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc2121b0 0x557ccc5c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54546/0 (socket says 172.21.15.2:54546) 2024-05-14T09:40:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.513+0000 7faf6b7fe640 1 -- 172.21.15.2:0/2528154875 learned_addr learned my addr 172.21.15.2:0/2528154875 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:40:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.513+0000 7faf6affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 0x557ccc5c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.513+0000 7faf627fc640 1 --2- 172.21.15.2:0/2528154875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc1f14d0 0x557ccc5c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.514+0000 7faf6affd640 1 -- 172.21.15.2:0/2528154875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc1f14d0 msgr2=0x557ccc5c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.514+0000 7faf6affd640 1 --2- 172.21.15.2:0/2528154875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc1f14d0 0x557ccc5c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.514+0000 7faf6affd640 1 -- 172.21.15.2:0/2528154875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc2121b0 msgr2=0x557ccc5c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.514+0000 7faf6affd640 1 --2- 172.21.15.2:0/2528154875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc2121b0 0x557ccc5c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.514+0000 7faf6affd640 1 -- 172.21.15.2:0/2528154875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ccc5af940 con 0x557ccc1e92f0 2024-05-14T09:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.514+0000 7faf6b7fe640 1 --2- 172.21.15.2:0/2528154875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc2121b0 0x557ccc5c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.514+0000 7faf6affd640 1 --2- 172.21.15.2:0/2528154875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 0x557ccc5c1c40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0c9bd0 tx=0x7faf5c0fff40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eea10be963a9b4e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.514+0000 7faf527fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.514+0000 7faf527fc640 1 -- 172.21.15.2:0/2528154875 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c0e3060 con 0x557ccc1e92f0 2024-05-14T09:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.514+0000 7faf78207740 1 -- 172.21.15.2:0/2528154875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf5400f840 con 0x557ccc1e92f0 2024-05-14T09:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.514+0000 7faf527fc640 1 -- 172.21.15.2:0/2528154875 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7faf5c0c9ea0 con 0x557ccc1e92f0 2024-05-14T09:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.514+0000 7faf527fc640 1 -- 172.21.15.2:0/2528154875 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faf5c049040 con 0x557ccc1e92f0 2024-05-14T09:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.514+0000 7faf78207740 10 client.25916 fetch_fsmap learned FSMap version 33 2024-05-14T09:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.514+0000 7faf78207740 1 -- 172.21.15.2:0/2528154875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557ccc6378e0 con 0x557ccc1e92f0 2024-05-14T09:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.515+0000 7faf527fc640 1 -- 172.21.15.2:0/2528154875 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf5400f840 con 0x557ccc1e92f0 2024-05-14T09:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.515+0000 7faf527fc640 1 -- 172.21.15.2:0/2528154875 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7faf5c0d2560 con 0x557ccc1e92f0 2024-05-14T09:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.515+0000 7faf78207740 10 client.25916 fetch_fsmap finished waiting for FSMap version 33 2024-05-14T09:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.515+0000 7faf78207740 10 client.25916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.515+0000 7faf78207740 20 client.25916 populate_metadata read hostname 'smithi002' 2024-05-14T09:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.515+0000 7faf78207740 1 --2- 172.21.15.2:0/2528154875 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557ccc1a61c0 0x557ccc63a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.515+0000 7faf78207740 4 client.25916 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.515+0000 7faf78207740 1 -- 172.21.15.2:0/2528154875 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ccc5261e0 con 0x557ccc1a61c0 2024-05-14T09:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.515+0000 7faf627fc640 1 --2- 172.21.15.2:0/2528154875 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557ccc1a61c0 0x557ccc63a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.515+0000 7faf527fc640 10 client.25916 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.515+0000 7faf627fc640 1 --2- 172.21.15.2:0/2528154875 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557ccc1a61c0 0x557ccc63a1e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6220f1bb393a3df4 server_cookie=fddfe001725b5b7b in_seq=0 out_seq=0 2024-05-14T09:40:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.676+0000 7faf527fc640 1 -- 172.21.15.2:0/2528154875 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ccc5261e0 con 0x557ccc1a61c0 2024-05-14T09:40:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.676+0000 7faf527fc640 10 client.25916 handle_command_reply: tid=0 2024-05-14T09:40:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.676+0000 7faf78207740 1 client.25916 shutdown 2024-05-14T09:40:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.676+0000 7faf78207740 1 -- 172.21.15.2:0/2528154875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 msgr2=0x557ccc5c1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.676+0000 7faf78207740 1 --2- 172.21.15.2:0/2528154875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 0x557ccc5c1c40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0c9bd0 tx=0x7faf5c0fff40 comp rx=0 tx=0).stop 2024-05-14T09:40:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.676+0000 7faf78207740 1 -- 172.21.15.2:0/2528154875 shutdown_connections 2024-05-14T09:40:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.676+0000 7faf78207740 1 --2- 172.21.15.2:0/2528154875 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557ccc1a61c0 0x557ccc63a1e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.676+0000 7faf78207740 1 --2- 172.21.15.2:0/2528154875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ccc2121b0 0x557ccc5c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.676+0000 7faf78207740 1 --2- 172.21.15.2:0/2528154875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557ccc1f14d0 0x557ccc5c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.676+0000 7faf78207740 1 --2- 172.21.15.2:0/2528154875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ccc1e92f0 0x557ccc5c1c40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.676+0000 7faf527fc640 0 client.25916 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:40:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.676+0000 7faf78207740 1 -- 172.21.15.2:0/2528154875 >> 172.21.15.2:0/2528154875 conn(0x557ccc206300 msgr2=0x557ccc5bbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.676+0000 7faf78207740 1 -- 172.21.15.2:0/2528154875 shutdown_connections 2024-05-14T09:40:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.676+0000 7faf78207740 1 -- 172.21.15.2:0/2528154875 wait complete. 2024-05-14T09:40:59.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.677+0000 7faf78207740 20 client.25916 trim_cache size 0 max 16384 2024-05-14T09:40:59.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:40:59.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:40:59.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:40:59.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:40:59.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.677+0000 7faf71284640 1 -- 172.21.15.2:0/3653448186 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7faf580b28a0 msgr2=0x7faf580b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:59.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.677+0000 7faf71284640 1 --2- 172.21.15.2:0/3653448186 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7faf580b28a0 0x7faf580b4db0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x557ccc5b4c30 tx=0x7faf64004a40 comp rx=0 tx=0).stop 2024-05-14T09:40:59.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.677+0000 7faf71284640 1 -- 172.21.15.2:0/3653448186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf6c102b50 msgr2=0x7faf6c06b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:40:59.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.677+0000 7faf71284640 1 --2- 172.21.15.2:0/3653448186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf6c102b50 0x7faf6c06b0f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7faf5c006010 tx=0x7faf5c004270 comp rx=0 tx=0).stop 2024-05-14T09:40:59.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.677+0000 7faf71284640 1 -- 172.21.15.2:0/3653448186 shutdown_connections 2024-05-14T09:40:59.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.677+0000 7faf71284640 1 --2- 172.21.15.2:0/3653448186 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7faf580b28a0 0x7faf580b4db0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.677+0000 7faf71284640 1 --2- 172.21.15.2:0/3653448186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf6c108a90 0x7faf6c1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.677+0000 7faf71284640 1 --2- 172.21.15.2:0/3653448186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf6c103d90 0x7faf6c19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.677+0000 7faf71284640 1 --2- 172.21.15.2:0/3653448186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf6c102b50 0x7faf6c06b0f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:40:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.677+0000 7faf71284640 1 -- 172.21.15.2:0/3653448186 >> 172.21.15.2:0/3653448186 conn(0x7faf6c0fdd40 msgr2=0x7faf6c104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:40:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.677+0000 7faf71284640 1 -- 172.21.15.2:0/3653448186 shutdown_connections 2024-05-14T09:40:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:40:59.677+0000 7faf71284640 1 -- 172.21.15.2:0/3653448186 wait complete. 2024-05-14T09:41:00.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 514071 10.28 MB/sec execute 146 sec latency 88.731 ms 2024-05-14T09:41:00.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:00 smithi002 ceph-mon[31557]: pgmap v605: 129 pgs: 129 active+clean; 90 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 2.9 MiB/s wr, 338 op/s 2024-05-14T09:41:00.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:00 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:41:00.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:00 smithi029 ceph-mon[36249]: pgmap v605: 129 pgs: 129 active+clean; 90 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 2.9 MiB/s wr, 338 op/s 2024-05-14T09:41:00.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:00 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:41:00.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:00 smithi026 ceph-mon[36056]: pgmap v605: 129 pgs: 129 active+clean; 90 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 2.9 MiB/s wr, 338 op/s 2024-05-14T09:41:00.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:00 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:41:01.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 515996 10.29 MB/sec execute 147 sec latency 13.388 ms 2024-05-14T09:41:02.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 518472 10.31 MB/sec execute 148 sec latency 54.613 ms 2024-05-14T09:41:02.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:02 smithi002 ceph-mon[31557]: pgmap v606: 129 pgs: 129 active+clean; 91 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-05-14T09:41:02.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:02 smithi029 ceph-mon[36249]: pgmap v606: 129 pgs: 129 active+clean; 91 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-05-14T09:41:02.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:02 smithi026 ceph-mon[36056]: pgmap v606: 129 pgs: 129 active+clean; 91 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-05-14T09:41:03.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 519855 10.29 MB/sec execute 149 sec latency 34.025 ms 2024-05-14T09:41:04.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 522478 10.33 MB/sec execute 150 sec latency 22.255 ms 2024-05-14T09:41:04.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:04 smithi002 ceph-mon[31557]: pgmap v607: 129 pgs: 129 active+clean; 87 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 3.5 MiB/s wr, 402 op/s 2024-05-14T09:41:04.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:04 smithi029 ceph-mon[36249]: pgmap v607: 129 pgs: 129 active+clean; 87 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 3.5 MiB/s wr, 402 op/s 2024-05-14T09:41:04.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:04 smithi026 ceph-mon[36056]: pgmap v607: 129 pgs: 129 active+clean; 87 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 3.5 MiB/s wr, 402 op/s 2024-05-14T09:41:05.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 524592 10.31 MB/sec execute 151 sec latency 18.983 ms 2024-05-14T09:41:05.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:05 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:41:05.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:05 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:41:05.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:05 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:41:06.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 526441 10.32 MB/sec execute 152 sec latency 12.355 ms 2024-05-14T09:41:06.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:06 smithi002 ceph-mon[31557]: pgmap v608: 129 pgs: 129 active+clean; 87 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 2.5 MiB/s wr, 299 op/s 2024-05-14T09:41:06.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:06 smithi029 ceph-mon[36249]: pgmap v608: 129 pgs: 129 active+clean; 87 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 2.5 MiB/s wr, 299 op/s 2024-05-14T09:41:06.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:06 smithi026 ceph-mon[36056]: pgmap v608: 129 pgs: 129 active+clean; 87 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 2.5 MiB/s wr, 299 op/s 2024-05-14T09:41:07.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 528745 10.34 MB/sec execute 153 sec latency 92.803 ms 2024-05-14T09:41:08.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 530193 10.32 MB/sec execute 154 sec latency 30.327 ms 2024-05-14T09:41:08.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:08 smithi002 ceph-mon[31557]: pgmap v609: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 3.7 MiB/s wr, 346 op/s 2024-05-14T09:41:08.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:08 smithi029 ceph-mon[36249]: pgmap v609: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 3.7 MiB/s wr, 346 op/s 2024-05-14T09:41:08.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:08 smithi026 ceph-mon[36056]: pgmap v609: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 3.7 MiB/s wr, 346 op/s 2024-05-14T09:41:09.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 533002 10.36 MB/sec execute 155 sec latency 13.102 ms 2024-05-14T09:41:10.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 534323 10.34 MB/sec execute 156 sec latency 68.368 ms 2024-05-14T09:41:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:10 smithi002 ceph-mon[31557]: pgmap v610: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 3.9 MiB/s wr, 392 op/s 2024-05-14T09:41:10.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:10 smithi029 ceph-mon[36249]: pgmap v610: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 3.9 MiB/s wr, 392 op/s 2024-05-14T09:41:10.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:10 smithi026 ceph-mon[36056]: pgmap v610: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 3.9 MiB/s wr, 392 op/s 2024-05-14T09:41:11.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 536686 10.36 MB/sec execute 157 sec latency 57.653 ms 2024-05-14T09:41:12.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 538098 10.33 MB/sec execute 158 sec latency 97.737 ms 2024-05-14T09:41:12.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:12 smithi002 ceph-mon[31557]: pgmap v611: 129 pgs: 129 active+clean; 105 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 3.8 MiB/s wr, 343 op/s 2024-05-14T09:41:12.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:12 smithi029 ceph-mon[36249]: pgmap v611: 129 pgs: 129 active+clean; 105 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 3.8 MiB/s wr, 343 op/s 2024-05-14T09:41:12.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:12 smithi026 ceph-mon[36056]: pgmap v611: 129 pgs: 129 active+clean; 105 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 3.8 MiB/s wr, 343 op/s 2024-05-14T09:41:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.783+0000 7fe9ea7b4640 1 -- 172.21.15.2:0/4227301478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9e4103d90 msgr2=0x7fe9e41041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.783+0000 7fe9ea7b4640 1 --2- 172.21.15.2:0/4227301478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9e4103d90 0x7fe9e41041a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009900 tx=0x7fe9d802a410 comp rx=0 tx=0).stop 2024-05-14T09:41:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.783+0000 7fe9ea7b4640 1 -- 172.21.15.2:0/4227301478 shutdown_connections 2024-05-14T09:41:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.783+0000 7fe9ea7b4640 1 --2- 172.21.15.2:0/4227301478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e4108a90 0x7fe9e410aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.783+0000 7fe9ea7b4640 1 --2- 172.21.15.2:0/4227301478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9e4103d90 0x7fe9e41041a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.783+0000 7fe9ea7b4640 1 --2- 172.21.15.2:0/4227301478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9e4102b50 0x7fe9e4103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.783+0000 7fe9ea7b4640 1 -- 172.21.15.2:0/4227301478 >> 172.21.15.2:0/4227301478 conn(0x7fe9e40fdd40 msgr2=0x7fe9e41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.783+0000 7fe9ea7b4640 1 -- 172.21.15.2:0/4227301478 shutdown_connections 2024-05-14T09:41:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.783+0000 7fe9ea7b4640 1 -- 172.21.15.2:0/4227301478 wait complete. 2024-05-14T09:41:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.784+0000 7fe9ea7b4640 1 Processor -- start 2024-05-14T09:41:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.784+0000 7fe9ea7b4640 1 -- start start 2024-05-14T09:41:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.784+0000 7fe9ea7b4640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9e4102b50 0x7fe9e410d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.784+0000 7fe9ea7b4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e4103d90 0x7fe9e410bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.784+0000 7fe9ea7b4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9e4108a90 0x7fe9e410c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.784+0000 7fe9ea7b4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e410f780 con 0x7fe9e4103d90 2024-05-14T09:41:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.784+0000 7fe9ea7b4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e410f600 con 0x7fe9e4108a90 2024-05-14T09:41:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.784+0000 7fe9ea7b4640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e410f900 con 0x7fe9e4102b50 2024-05-14T09:41:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9e37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e4103d90 0x7fe9e410bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9e37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e4103d90 0x7fe9e410bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37268/0 (socket says 172.21.15.2:37268) 2024-05-14T09:41:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9e37fe640 1 -- 172.21.15.2:0/2694850584 learned_addr learned my addr 172.21.15.2:0/2694850584 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9e8d51640 1 --2- 172.21.15.2:0/2694850584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9e4108a90 0x7fe9e410c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9e3fff640 1 --2- 172.21.15.2:0/2694850584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9e4102b50 0x7fe9e410d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9e37fe640 1 -- 172.21.15.2:0/2694850584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9e4102b50 msgr2=0x7fe9e410d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9e37fe640 1 --2- 172.21.15.2:0/2694850584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9e4102b50 0x7fe9e410d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9e37fe640 1 -- 172.21.15.2:0/2694850584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9e4108a90 msgr2=0x7fe9e410c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9e37fe640 1 --2- 172.21.15.2:0/2694850584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9e4108a90 0x7fe9e410c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9e37fe640 1 -- 172.21.15.2:0/2694850584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e410c730 con 0x7fe9e4103d90 2024-05-14T09:41:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9e3fff640 1 --2- 172.21.15.2:0/2694850584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9e4102b50 0x7fe9e410d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:41:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9e37fe640 1 --2- 172.21.15.2:0/2694850584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e4103d90 0x7fe9e410bac0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8006010 tx=0x7fe9d8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9e17fa640 1 -- 172.21.15.2:0/2694850584 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d803f030 con 0x7fe9e4103d90 2024-05-14T09:41:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9ea7b4640 1 -- 172.21.15.2:0/2694850584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e41a43e0 con 0x7fe9e4103d90 2024-05-14T09:41:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9ea7b4640 1 -- 172.21.15.2:0/2694850584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e41a47e0 con 0x7fe9e4103d90 2024-05-14T09:41:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9e17fa640 1 -- 172.21.15.2:0/2694850584 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9d8003aa0 con 0x7fe9e4103d90 2024-05-14T09:41:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.785+0000 7fe9e17fa640 1 -- 172.21.15.2:0/2694850584 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8049060 con 0x7fe9e4103d90 2024-05-14T09:41:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.788+0000 7fe9f1662740 1 Processor -- start 2024-05-14T09:41:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.788+0000 7fe9f1662740 1 -- start start 2024-05-14T09:41:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.788+0000 7fe9e17fa640 1 -- 172.21.15.2:0/2694850584 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe9d8042030 con 0x7fe9e4103d90 2024-05-14T09:41:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9f1662740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cfb1b0 0x56431709ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9f1662740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cd22f0 0x56431709cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9f1662740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cda4d0 0x5643170a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9f1662740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5643170015d0 con 0x564316cd22f0 2024-05-14T09:41:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9f1662740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564316fba890 con 0x564316cfb1b0 2024-05-14T09:41:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9f1662740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564316ffd590 con 0x564316cda4d0 2024-05-14T09:41:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9e3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cd22f0 0x56431709cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9e3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cd22f0 0x56431709cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37280/0 (socket says 172.21.15.2:37280) 2024-05-14T09:41:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9e3fff640 1 -- 172.21.15.2:0/218502582 learned_addr learned my addr 172.21.15.2:0/218502582 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9e8d51640 1 --2- 172.21.15.2:0/218502582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cfb1b0 0x56431709ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9e37fe640 1 --2- 172.21.15.2:0/218502582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cda4d0 0x5643170a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9e8d51640 1 -- 172.21.15.2:0/218502582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cda4d0 msgr2=0x5643170a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9e8d51640 1 --2- 172.21.15.2:0/218502582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cda4d0 0x5643170a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9e8d51640 1 -- 172.21.15.2:0/218502582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cd22f0 msgr2=0x56431709cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9e8d51640 1 --2- 172.21.15.2:0/218502582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cd22f0 0x56431709cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9e8d51640 1 -- 172.21.15.2:0/218502582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564316d13210 con 0x564316cfb1b0 2024-05-14T09:41:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9e3fff640 1 --2- 172.21.15.2:0/218502582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cd22f0 0x56431709cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:41:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9e37fe640 1 --2- 172.21.15.2:0/218502582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cda4d0 0x5643170a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:41:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9e8d51640 1 --2- 172.21.15.2:0/218502582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cfb1b0 0x56431709ca60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009f10 tx=0x7fe9dc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f467d61fc8bc2d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.789+0000 7fe9e17fa640 1 --2- 172.21.15.2:0/2694850584 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe9cc0b28a0 0x7fe9cc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.790+0000 7fe9ca7fc640 1 -- 172.21.15.2:0/218502582 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc004d20 con 0x564316cfb1b0 2024-05-14T09:41:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.790+0000 7fe9ca7fc640 1 -- 172.21.15.2:0/218502582 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9dc004030 con 0x564316cfb1b0 2024-05-14T09:41:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.790+0000 7fe9ca7fc640 1 -- 172.21.15.2:0/218502582 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc005220 con 0x564316cfb1b0 2024-05-14T09:41:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.790+0000 7fe9e3fff640 1 --2- 172.21.15.2:0/2694850584 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe9cc0b28a0 0x7fe9cc0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.790+0000 7fe9e17fa640 1 -- 172.21.15.2:0/2694850584 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fe9d8102900 con 0x7fe9e4103d90 2024-05-14T09:41:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.790+0000 7fe9f1662740 1 -- 172.21.15.2:0/218502582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cfb1b0 msgr2=0x56431709ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.790+0000 7fe9f1662740 1 --2- 172.21.15.2:0/218502582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cfb1b0 0x56431709ca60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009f10 tx=0x7fe9dc011810 comp rx=0 tx=0).stop 2024-05-14T09:41:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.790+0000 7fe9e3fff640 1 --2- 172.21.15.2:0/2694850584 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe9cc0b28a0 0x7fe9cc0b4db0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x56431709de40 tx=0x7fe9d400b600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.790+0000 7fe9f1662740 1 -- 172.21.15.2:0/218502582 shutdown_connections 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.790+0000 7fe9f1662740 1 --2- 172.21.15.2:0/218502582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cda4d0 0x5643170a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.790+0000 7fe9f1662740 1 --2- 172.21.15.2:0/218502582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cd22f0 0x56431709cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.790+0000 7fe9f1662740 1 --2- 172.21.15.2:0/218502582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cfb1b0 0x56431709ca60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.790+0000 7fe9f1662740 1 -- 172.21.15.2:0/218502582 >> 172.21.15.2:0/218502582 conn(0x564316cef300 msgr2=0x5643170983a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.791+0000 7fe9f1662740 1 -- 172.21.15.2:0/218502582 shutdown_connections 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.791+0000 7fe9f1662740 1 -- 172.21.15.2:0/218502582 wait complete. 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.791+0000 7fe9f1662740 1 -- start start 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.791+0000 7fe9f1662740 1 Processor -- start 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.791+0000 7fe9f1662740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cd22f0 0x5643170a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.791+0000 7fe9f1662740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cda4d0 0x5643170a6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.791+0000 7fe9f1662740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cfb1b0 0x5643170adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.791+0000 7fe9f1662740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5643170a4930 con 0x564316cda4d0 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.791+0000 7fe9f1662740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564316fbe370 con 0x564316cd22f0 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.791+0000 7fe9f1662740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5643170a4ab0 con 0x564316cfb1b0 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9f1662740 10 client.0 _ll_register_callbacks cb 0x564316d12a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:41:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9e3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cd22f0 0x5643170a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9e3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cd22f0 0x5643170a66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:44626/0 (socket says 172.21.15.2:44626) 2024-05-14T09:41:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9e3fff640 1 -- 172.21.15.2:0/2302336062 learned_addr learned my addr 172.21.15.2:0/2302336062 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9e37fe640 1 --2- 172.21.15.2:0/2302336062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cda4d0 0x5643170a6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9e8d51640 1 --2- 172.21.15.2:0/2302336062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cfb1b0 0x5643170adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9e3fff640 1 -- 172.21.15.2:0/2302336062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cfb1b0 msgr2=0x5643170adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9e3fff640 1 --2- 172.21.15.2:0/2302336062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cfb1b0 0x5643170adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9e3fff640 1 -- 172.21.15.2:0/2302336062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cda4d0 msgr2=0x5643170a6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9e3fff640 1 --2- 172.21.15.2:0/2302336062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cda4d0 0x5643170a6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9e3fff640 1 -- 172.21.15.2:0/2302336062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564316ddf620 con 0x564316cd22f0 2024-05-14T09:41:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9e8d51640 1 --2- 172.21.15.2:0/2302336062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cfb1b0 0x5643170adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:41:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9e37fe640 1 --2- 172.21.15.2:0/2302336062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cda4d0 0x5643170a6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:41:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9e3fff640 1 --2- 172.21.15.2:0/2302336062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cd22f0 0x5643170a66e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40042c0 tx=0x7fe9d40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4599471b0db2accb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.792+0000 7fe9c37fe640 1 -- 172.21.15.2:0/2302336062 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4010750 con 0x564316cd22f0 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.793+0000 7fe9c37fe640 1 -- 172.21.15.2:0/2302336062 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fe9d40108f0 con 0x564316cd22f0 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.793+0000 7fe9f1662740 1 -- 172.21.15.2:0/2302336062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9dc004030 con 0x564316cd22f0 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.793+0000 7fe9c37fe640 1 -- 172.21.15.2:0/2302336062 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe9d4030050 con 0x564316cd22f0 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.793+0000 7fe9f1662740 10 client.25928 fetch_fsmap learned FSMap version 34 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.793+0000 7fe9f1662740 1 -- 172.21.15.2:0/2302336062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5643170b0b80 con 0x564316cd22f0 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.793+0000 7fe9c37fe640 1 -- 172.21.15.2:0/2302336062 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9dc004030 con 0x564316cd22f0 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.793+0000 7fe9c37fe640 1 -- 172.21.15.2:0/2302336062 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fe9d401fbb0 con 0x564316cd22f0 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.793+0000 7fe9f1662740 10 client.25928 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.793+0000 7fe9f1662740 10 client.25928 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.793+0000 7fe9f1662740 20 client.25928 populate_metadata read hostname 'smithi002' 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.793+0000 7fe9f1662740 1 --2- 172.21.15.2:0/2302336062 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564316cb9d80 0x5643170b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.793+0000 7fe9f1662740 4 client.25928 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.793+0000 7fe9f1662740 1 -- 172.21.15.2:0/2302336062 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56431700f1e0 con 0x564316cb9d80 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.793+0000 7fe9e37fe640 1 --2- 172.21.15.2:0/2302336062 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564316cb9d80 0x5643170b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.794+0000 7fe9e37fe640 1 --2- 172.21.15.2:0/2302336062 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564316cb9d80 0x5643170b42c0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857ed5fffe2e307a server_cookie=621eced820c1ff54 in_seq=0 out_seq=0 2024-05-14T09:41:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.794+0000 7fe9c37fe640 10 client.25928 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.795+0000 7fe9c37fe640 1 -- 172.21.15.2:0/2302336062 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56431700f1e0 con 0x564316cb9d80 2024-05-14T09:41:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.795+0000 7fe9c37fe640 10 client.25928 handle_command_reply: tid=0 2024-05-14T09:41:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.796+0000 7fe9f1662740 1 client.25928 shutdown 2024-05-14T09:41:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.796+0000 7fe9f1662740 1 -- 172.21.15.2:0/2302336062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cd22f0 msgr2=0x5643170a66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.796+0000 7fe9f1662740 1 --2- 172.21.15.2:0/2302336062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cd22f0 0x5643170a66e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40042c0 tx=0x7fe9d40046c0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.796+0000 7fe9f1662740 1 -- 172.21.15.2:0/2302336062 shutdown_connections 2024-05-14T09:41:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.796+0000 7fe9f1662740 1 --2- 172.21.15.2:0/2302336062 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564316cb9d80 0x5643170b42c0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.796+0000 7fe9f1662740 1 --2- 172.21.15.2:0/2302336062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cfb1b0 0x5643170adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.796+0000 7fe9f1662740 1 --2- 172.21.15.2:0/2302336062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cda4d0 0x5643170a6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.796+0000 7fe9c37fe640 0 client.25928 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.796+0000 7fe9f1662740 1 --2- 172.21.15.2:0/2302336062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cd22f0 0x5643170a66e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.796+0000 7fe9f1662740 1 -- 172.21.15.2:0/2302336062 >> 172.21.15.2:0/2302336062 conn(0x564316cef300 msgr2=0x5643170973a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.796+0000 7fe9f1662740 1 -- 172.21.15.2:0/2302336062 shutdown_connections 2024-05-14T09:41:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.796+0000 7fe9f1662740 1 -- 172.21.15.2:0/2302336062 wait complete. 2024-05-14T09:41:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.796+0000 7fe9f1662740 20 client.25928 trim_cache size 0 max 16384 2024-05-14T09:41:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.814+0000 7fe9f1662740 1 Processor -- start 2024-05-14T09:41:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.814+0000 7fe9f1662740 1 -- start start 2024-05-14T09:41:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.814+0000 7fe9f1662740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cef300 0x5643170a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.814+0000 7fe9f1662740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cd22f0 0x5643170adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.814+0000 7fe9f1662740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cda4d0 0x5643170ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9f1662740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e41b1700 con 0x564316cd22f0 2024-05-14T09:41:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9f1662740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e41aefa0 con 0x564316cda4d0 2024-05-14T09:41:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9f1662740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e41acdf0 con 0x564316cef300 2024-05-14T09:41:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9e37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cd22f0 0x5643170adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9e37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cd22f0 0x5643170adb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37292/0 (socket says 172.21.15.2:37292) 2024-05-14T09:41:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9e37fe640 1 -- 172.21.15.2:0/1277053043 learned_addr learned my addr 172.21.15.2:0/1277053043 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9e8d51640 1 --2- 172.21.15.2:0/1277053043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cda4d0 0x5643170ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9e3fff640 1 --2- 172.21.15.2:0/1277053043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cef300 0x5643170a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9e37fe640 1 -- 172.21.15.2:0/1277053043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cef300 msgr2=0x5643170a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9e37fe640 1 --2- 172.21.15.2:0/1277053043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cef300 0x5643170a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9e37fe640 1 -- 172.21.15.2:0/1277053043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cda4d0 msgr2=0x5643170ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9e37fe640 1 --2- 172.21.15.2:0/1277053043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cda4d0 0x5643170ae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9e37fe640 1 -- 172.21.15.2:0/1277053043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56431709dc60 con 0x564316cd22f0 2024-05-14T09:41:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9e37fe640 1 --2- 172.21.15.2:0/1277053043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cd22f0 0x5643170adb10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80c9a90 tx=0x7fe9d8104e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a612a6cb8f3931 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9c9ffb640 1 -- 172.21.15.2:0/1277053043 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8035040 con 0x564316cd22f0 2024-05-14T09:41:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9c9ffb640 1 -- 172.21.15.2:0/1277053043 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9d80d22a0 con 0x564316cd22f0 2024-05-14T09:41:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.815+0000 7fe9c9ffb640 1 -- 172.21.15.2:0/1277053043 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d80d2960 con 0x564316cd22f0 2024-05-14T09:41:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.816+0000 7fe9f1662740 1 -- 172.21.15.2:0/1277053043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cd22f0 msgr2=0x5643170adb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.816+0000 7fe9f1662740 1 --2- 172.21.15.2:0/1277053043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cd22f0 0x5643170adb10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80c9a90 tx=0x7fe9d8104e10 comp rx=0 tx=0).stop 2024-05-14T09:41:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.816+0000 7fe9f1662740 1 -- 172.21.15.2:0/1277053043 shutdown_connections 2024-05-14T09:41:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.816+0000 7fe9f1662740 1 --2- 172.21.15.2:0/1277053043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cda4d0 0x5643170ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.816+0000 7fe9f1662740 1 --2- 172.21.15.2:0/1277053043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cd22f0 0x5643170adb10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.816+0000 7fe9f1662740 1 --2- 172.21.15.2:0/1277053043 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cef300 0x5643170a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.816+0000 7fe9f1662740 1 -- 172.21.15.2:0/1277053043 >> 172.21.15.2:0/1277053043 conn(0x5643170a12f0 msgr2=0x5643170a62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.816+0000 7fe9f1662740 1 -- 172.21.15.2:0/1277053043 shutdown_connections 2024-05-14T09:41:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.816+0000 7fe9f1662740 1 -- 172.21.15.2:0/1277053043 wait complete. 2024-05-14T09:41:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.817+0000 7fe9f1662740 1 -- start start 2024-05-14T09:41:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.817+0000 7fe9f1662740 1 Processor -- start 2024-05-14T09:41:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.817+0000 7fe9f1662740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cd22f0 0x5643170ab750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.817+0000 7fe9f1662740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cda4d0 0x5643170abc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.817+0000 7fe9f1662740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cef300 0x5643170acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.817+0000 7fe9f1662740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e41ad460 con 0x564316cda4d0 2024-05-14T09:41:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.817+0000 7fe9f1662740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e41ae8d0 con 0x564316cd22f0 2024-05-14T09:41:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.817+0000 7fe9f1662740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e41adb30 con 0x564316cef300 2024-05-14T09:41:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.817+0000 7fe9f1662740 10 client.0 _ll_register_callbacks cb 0x7fe9d400b840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:41:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.817+0000 7fe9e37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cda4d0 0x5643170abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.817+0000 7fe9e37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cda4d0 0x5643170abc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37296/0 (socket says 172.21.15.2:37296) 2024-05-14T09:41:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.817+0000 7fe9e37fe640 1 -- 172.21.15.2:0/3231859765 learned_addr learned my addr 172.21.15.2:0/3231859765 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.817+0000 7fe9e3fff640 1 --2- 172.21.15.2:0/3231859765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cd22f0 0x5643170ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.817+0000 7fe9e8d51640 1 --2- 172.21.15.2:0/3231859765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cef300 0x5643170acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.818+0000 7fe9e37fe640 1 -- 172.21.15.2:0/3231859765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cef300 msgr2=0x5643170acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.818+0000 7fe9e37fe640 1 --2- 172.21.15.2:0/3231859765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cef300 0x5643170acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.818+0000 7fe9e37fe640 1 -- 172.21.15.2:0/3231859765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cd22f0 msgr2=0x5643170ab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.818+0000 7fe9e37fe640 1 --2- 172.21.15.2:0/3231859765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cd22f0 0x5643170ab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.818+0000 7fe9e37fe640 1 -- 172.21.15.2:0/3231859765 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643170ad0c0 con 0x564316cda4d0 2024-05-14T09:41:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.818+0000 7fe9e37fe640 1 --2- 172.21.15.2:0/3231859765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cda4d0 0x5643170abc90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8031440 tx=0x7fe9d80fff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634ed94c0dfe6214 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.818+0000 7fe9c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:41:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.818+0000 7fe9c37fe640 1 -- 172.21.15.2:0/3231859765 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d80e4060 con 0x564316cda4d0 2024-05-14T09:41:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.818+0000 7fe9c37fe640 1 -- 172.21.15.2:0/3231859765 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fe9d80c5d90 con 0x564316cda4d0 2024-05-14T09:41:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.818+0000 7fe9f1662740 1 -- 172.21.15.2:0/3231859765 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564317009050 con 0x564316cda4d0 2024-05-14T09:41:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.818+0000 7fe9c37fe640 1 -- 172.21.15.2:0/3231859765 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe9d8035070 con 0x564316cda4d0 2024-05-14T09:41:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.818+0000 7fe9f1662740 10 client.16248 fetch_fsmap learned FSMap version 34 2024-05-14T09:41:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.818+0000 7fe9f1662740 1 -- 172.21.15.2:0/3231859765 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5643170ae3c0 con 0x564316cda4d0 2024-05-14T09:41:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.818+0000 7fe9c37fe640 1 -- 172.21.15.2:0/3231859765 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564317009050 con 0x564316cda4d0 2024-05-14T09:41:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.818+0000 7fe9c37fe640 1 -- 172.21.15.2:0/3231859765 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fe9d804c060 con 0x564316cda4d0 2024-05-14T09:41:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.819+0000 7fe9f1662740 10 client.16248 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:41:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.819+0000 7fe9f1662740 10 client.16248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:41:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.819+0000 7fe9f1662740 20 client.16248 populate_metadata read hostname 'smithi002' 2024-05-14T09:41:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.819+0000 7fe9f1662740 1 --2- 172.21.15.2:0/3231859765 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564316cfb1b0 0x56431711eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.819+0000 7fe9f1662740 4 client.16248 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:41:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.819+0000 7fe9f1662740 1 -- 172.21.15.2:0/3231859765 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564316f845f0 con 0x564316cfb1b0 2024-05-14T09:41:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.819+0000 7fe9e3fff640 1 --2- 172.21.15.2:0/3231859765 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564316cfb1b0 0x56431711eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.819+0000 7fe9c37fe640 10 client.16248 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.819+0000 7fe9e3fff640 1 --2- 172.21.15.2:0/3231859765 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564316cfb1b0 0x56431711eea0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c398256f6548d87 server_cookie=9918de5d7afc881c in_seq=0 out_seq=0 2024-05-14T09:41:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.940+0000 7fe9c37fe640 1 -- 172.21.15.2:0/3231859765 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564316f845f0 con 0x564316cfb1b0 2024-05-14T09:41:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.940+0000 7fe9c37fe640 10 client.16248 handle_command_reply: tid=0 2024-05-14T09:41:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.940+0000 7fe9f1662740 1 client.16248 shutdown 2024-05-14T09:41:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.940+0000 7fe9f1662740 1 -- 172.21.15.2:0/3231859765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cda4d0 msgr2=0x5643170abc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.940+0000 7fe9f1662740 1 --2- 172.21.15.2:0/3231859765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cda4d0 0x5643170abc90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8031440 tx=0x7fe9d80fff80 comp rx=0 tx=0).stop 2024-05-14T09:41:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.940+0000 7fe9f1662740 1 -- 172.21.15.2:0/3231859765 shutdown_connections 2024-05-14T09:41:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.940+0000 7fe9f1662740 1 --2- 172.21.15.2:0/3231859765 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564316cfb1b0 0x56431711eea0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.940+0000 7fe9f1662740 1 --2- 172.21.15.2:0/3231859765 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564316cef300 0x5643170acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.940+0000 7fe9c37fe640 0 client.16248 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.940+0000 7fe9f1662740 1 --2- 172.21.15.2:0/3231859765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564316cda4d0 0x5643170abc90 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.940+0000 7fe9f1662740 1 --2- 172.21.15.2:0/3231859765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564316cd22f0 0x5643170ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.940+0000 7fe9f1662740 1 -- 172.21.15.2:0/3231859765 >> 172.21.15.2:0/3231859765 conn(0x5643170a12f0 msgr2=0x564317096970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.940+0000 7fe9f1662740 1 -- 172.21.15.2:0/3231859765 shutdown_connections 2024-05-14T09:41:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.940+0000 7fe9f1662740 1 -- 172.21.15.2:0/3231859765 wait complete. 2024-05-14T09:41:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.940+0000 7fe9f1662740 20 client.16248 trim_cache size 0 max 16384 2024-05-14T09:41:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:41:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:41:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:41:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:41:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.941+0000 7fe9ea7b4640 1 -- 172.21.15.2:0/2694850584 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe9cc0b28a0 msgr2=0x7fe9cc0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.941+0000 7fe9ea7b4640 1 --2- 172.21.15.2:0/2694850584 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe9cc0b28a0 0x7fe9cc0b4db0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x56431709de40 tx=0x7fe9d400b600 comp rx=0 tx=0).stop 2024-05-14T09:41:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.941+0000 7fe9ea7b4640 1 -- 172.21.15.2:0/2694850584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e4103d90 msgr2=0x7fe9e410bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.941+0000 7fe9ea7b4640 1 --2- 172.21.15.2:0/2694850584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e4103d90 0x7fe9e410bac0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8006010 tx=0x7fe9d8031a50 comp rx=0 tx=0).stop 2024-05-14T09:41:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.941+0000 7fe9ea7b4640 1 -- 172.21.15.2:0/2694850584 shutdown_connections 2024-05-14T09:41:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.941+0000 7fe9ea7b4640 1 --2- 172.21.15.2:0/2694850584 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe9cc0b28a0 0x7fe9cc0b4db0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.941+0000 7fe9ea7b4640 1 --2- 172.21.15.2:0/2694850584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9e4108a90 0x7fe9e410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.941+0000 7fe9ea7b4640 1 --2- 172.21.15.2:0/2694850584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e4103d90 0x7fe9e410bac0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.941+0000 7fe9ea7b4640 1 --2- 172.21.15.2:0/2694850584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9e4102b50 0x7fe9e410d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.941+0000 7fe9ea7b4640 1 -- 172.21.15.2:0/2694850584 >> 172.21.15.2:0/2694850584 conn(0x7fe9e40fdd40 msgr2=0x7fe9e4104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.941+0000 7fe9ea7b4640 1 -- 172.21.15.2:0/2694850584 shutdown_connections 2024-05-14T09:41:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:12.941+0000 7fe9ea7b4640 1 -- 172.21.15.2:0/2694850584 wait complete. 2024-05-14T09:41:13.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 540215 10.35 MB/sec execute 159 sec latency 48.745 ms 2024-05-14T09:41:14.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 541959 10.32 MB/sec execute 160 sec latency 22.165 ms 2024-05-14T09:41:14.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:14 smithi002 ceph-mon[31557]: pgmap v612: 129 pgs: 129 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 5.1 MiB/s wr, 408 op/s 2024-05-14T09:41:14.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:14 smithi029 ceph-mon[36249]: pgmap v612: 129 pgs: 129 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 5.1 MiB/s wr, 408 op/s 2024-05-14T09:41:14.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:14 smithi026 ceph-mon[36056]: pgmap v612: 129 pgs: 129 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 5.1 MiB/s wr, 408 op/s 2024-05-14T09:41:15.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 543978 10.34 MB/sec execute 161 sec latency 20.499 ms 2024-05-14T09:41:16.266 INFO:tasks.workunit.client.0.smithi002.stdout: 1 546091 10.33 MB/sec execute 162 sec latency 43.378 ms 2024-05-14T09:41:16.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:16 smithi002 ceph-mon[31557]: pgmap v613: 129 pgs: 129 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 3.9 MiB/s wr, 246 op/s 2024-05-14T09:41:16.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:16 smithi029 ceph-mon[36249]: pgmap v613: 129 pgs: 129 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 3.9 MiB/s wr, 246 op/s 2024-05-14T09:41:16.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:16 smithi026 ceph-mon[36056]: pgmap v613: 129 pgs: 129 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 3.9 MiB/s wr, 246 op/s 2024-05-14T09:41:17.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 547654 10.33 MB/sec execute 163 sec latency 53.159 ms 2024-05-14T09:41:18.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 549947 10.34 MB/sec execute 164 sec latency 17.102 ms 2024-05-14T09:41:18.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:18 smithi002 ceph-mon[31557]: pgmap v614: 129 pgs: 129 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 4.5 MiB/s wr, 343 op/s 2024-05-14T09:41:18.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:18 smithi029 ceph-mon[36249]: pgmap v614: 129 pgs: 129 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 4.5 MiB/s wr, 343 op/s 2024-05-14T09:41:18.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:18 smithi026 ceph-mon[36056]: pgmap v614: 129 pgs: 129 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 4.5 MiB/s wr, 343 op/s 2024-05-14T09:41:19.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 551419 10.32 MB/sec execute 165 sec latency 34.640 ms 2024-05-14T09:41:20.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 554141 10.36 MB/sec execute 166 sec latency 24.570 ms 2024-05-14T09:41:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:20 smithi002 ceph-mon[31557]: pgmap v615: 129 pgs: 129 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 3.7 MiB/s wr, 345 op/s 2024-05-14T09:41:20.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:20 smithi029 ceph-mon[36249]: pgmap v615: 129 pgs: 129 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 3.7 MiB/s wr, 345 op/s 2024-05-14T09:41:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:20 smithi026 ceph-mon[36056]: pgmap v615: 129 pgs: 129 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 3.7 MiB/s wr, 345 op/s 2024-05-14T09:41:21.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 555595 10.34 MB/sec execute 167 sec latency 25.630 ms 2024-05-14T09:41:22.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 557846 10.36 MB/sec execute 168 sec latency 69.804 ms 2024-05-14T09:41:22.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:22 smithi002 ceph-mon[31557]: pgmap v616: 129 pgs: 129 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 344 op/s 2024-05-14T09:41:22.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:22 smithi029 ceph-mon[36249]: pgmap v616: 129 pgs: 129 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 344 op/s 2024-05-14T09:41:22.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:22 smithi026 ceph-mon[36056]: pgmap v616: 129 pgs: 129 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 344 op/s 2024-05-14T09:41:23.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 558892 10.32 MB/sec execute 169 sec latency 104.318 ms 2024-05-14T09:41:24.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 561191 10.35 MB/sec execute 170 sec latency 184.107 ms 2024-05-14T09:41:24.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:24 smithi002 ceph-mon[31557]: pgmap v617: 129 pgs: 129 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 371 op/s 2024-05-14T09:41:24.749 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:24 smithi026 ceph-mon[36056]: pgmap v617: 129 pgs: 129 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 371 op/s 2024-05-14T09:41:24.750 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:24 smithi029 ceph-mon[36249]: pgmap v617: 129 pgs: 129 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 371 op/s 2024-05-14T09:41:25.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 562127 10.30 MB/sec execute 171 sec latency 96.651 ms 2024-05-14T09:41:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.047+0000 7fdc7593f640 1 -- 172.21.15.2:0/370162859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc70101880 msgr2=0x7fdc70101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.047+0000 7fdc7593f640 1 --2- 172.21.15.2:0/370162859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc70101880 0x7fdc70101cf0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009900 tx=0x7fdc6402a410 comp rx=0 tx=0).stop 2024-05-14T09:41:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.047+0000 7fdc7593f640 1 -- 172.21.15.2:0/370162859 shutdown_connections 2024-05-14T09:41:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.047+0000 7fdc7593f640 1 --2- 172.21.15.2:0/370162859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc70102380 0x7fdc7010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.047+0000 7fdc7593f640 1 --2- 172.21.15.2:0/370162859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc70101880 0x7fdc70101cf0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.047+0000 7fdc7593f640 1 --2- 172.21.15.2:0/370162859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc701051d0 0x7fdc701055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.047+0000 7fdc7593f640 1 -- 172.21.15.2:0/370162859 >> 172.21.15.2:0/370162859 conn(0x7fdc700fd480 msgr2=0x7fdc700ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.047+0000 7fdc7593f640 1 -- 172.21.15.2:0/370162859 shutdown_connections 2024-05-14T09:41:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.047+0000 7fdc7593f640 1 -- 172.21.15.2:0/370162859 wait complete. 2024-05-14T09:41:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.047+0000 7fdc7593f640 1 Processor -- start 2024-05-14T09:41:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc7593f640 1 -- start start 2024-05-14T09:41:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc7593f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc70101880 0x7fdc7019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc7593f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc70102380 0x7fdc7019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc7593f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc701051d0 0x7fdc701a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc7593f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7010f740 con 0x7fdc70102380 2024-05-14T09:41:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc7593f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7010f5c0 con 0x7fdc70101880 2024-05-14T09:41:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc7593f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7010f8c0 con 0x7fdc701051d0 2024-05-14T09:41:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc6e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc70102380 0x7fdc7019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc6f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc701051d0 0x7fdc701a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc6effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc70101880 0x7fdc7019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc6e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc70102380 0x7fdc7019a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44278/0 (socket says 172.21.15.2:44278) 2024-05-14T09:41:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc6e7fc640 1 -- 172.21.15.2:0/1253538469 learned_addr learned my addr 172.21.15.2:0/1253538469 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc6e7fc640 1 -- 172.21.15.2:0/1253538469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc701051d0 msgr2=0x7fdc701a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc6e7fc640 1 --2- 172.21.15.2:0/1253538469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc701051d0 0x7fdc701a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc6e7fc640 1 -- 172.21.15.2:0/1253538469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc70101880 msgr2=0x7fdc7019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc6e7fc640 1 --2- 172.21.15.2:0/1253538469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc70101880 0x7fdc7019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.048+0000 7fdc6e7fc640 1 -- 172.21.15.2:0/1253538469 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc701a1df0 con 0x7fdc70102380 2024-05-14T09:41:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.049+0000 7fdc6e7fc640 1 --2- 172.21.15.2:0/1253538469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc70102380 0x7fdc7019a7f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdc64006010 tx=0x7fdc64031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.049+0000 7fdc7493d640 1 -- 172.21.15.2:0/1253538469 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6403f030 con 0x7fdc70102380 2024-05-14T09:41:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.049+0000 7fdc7593f640 1 -- 172.21.15.2:0/1253538469 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc701a2080 con 0x7fdc70102380 2024-05-14T09:41:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.049+0000 7fdc7493d640 1 -- 172.21.15.2:0/1253538469 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc64003aa0 con 0x7fdc70102380 2024-05-14T09:41:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.049+0000 7fdc7493d640 1 -- 172.21.15.2:0/1253538469 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64035070 con 0x7fdc70102380 2024-05-14T09:41:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.049+0000 7fdc7593f640 1 -- 172.21.15.2:0/1253538469 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc701a25c0 con 0x7fdc70102380 2024-05-14T09:41:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.051+0000 7fdc7493d640 1 -- 172.21.15.2:0/1253538469 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fdc64042030 con 0x7fdc70102380 2024-05-14T09:41:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.052+0000 7fdc7c7fa740 1 Processor -- start 2024-05-14T09:41:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.052+0000 7fdc7c7fa740 1 -- start start 2024-05-14T09:41:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.052+0000 7fdc7c7fa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb18a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.052+0000 7fdc7c7fa740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e74e2f0 0x56005eb18fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.052+0000 7fdc7493d640 1 --2- 172.21.15.2:0/1253538469 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdc580b28a0 0x7fdc580b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.052+0000 7fdc7c7fa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e7564d0 0x56005eb1fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.052+0000 7fdc7c7fa740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56005ea7d5d0 con 0x56005e74e2f0 2024-05-14T09:41:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.052+0000 7fdc7c7fa740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56005ea36890 con 0x56005e7564d0 2024-05-14T09:41:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.052+0000 7fdc7c7fa740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56005ea79590 con 0x56005e7771b0 2024-05-14T09:41:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc6e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e74e2f0 0x56005eb18fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc6f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb18a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc6e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e74e2f0 0x56005eb18fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44280/0 (socket says 172.21.15.2:44280) 2024-05-14T09:41:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc6e7fc640 1 -- 172.21.15.2:0/431458747 learned_addr learned my addr 172.21.15.2:0/431458747 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc6effd640 1 --2- 172.21.15.2:0/1253538469 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdc580b28a0 0x7fdc580b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc6f7fe640 1 --2- 172.21.15.2:0/431458747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e7564d0 0x56005eb1fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc7493d640 1 -- 172.21.15.2:0/1253538469 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fdc64102bd0 con 0x7fdc70102380 2024-05-14T09:41:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc6f7fe640 1 -- 172.21.15.2:0/431458747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e7564d0 msgr2=0x56005eb1fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc6f7fe640 1 --2- 172.21.15.2:0/431458747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e7564d0 0x56005eb1fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc6f7fe640 1 -- 172.21.15.2:0/431458747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e74e2f0 msgr2=0x56005eb18fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc6f7fe640 1 --2- 172.21.15.2:0/431458747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e74e2f0 0x56005eb18fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc6f7fe640 1 -- 172.21.15.2:0/431458747 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56005e78f210 con 0x56005e7771b0 2024-05-14T09:41:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc6e7fc640 1 --2- 172.21.15.2:0/431458747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e74e2f0 0x56005eb18fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:41:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc6f7fe640 1 --2- 172.21.15.2:0/431458747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb18a60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdc68011a70 tx=0x7fdc68004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a0547703939fe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc6effd640 1 --2- 172.21.15.2:0/1253538469 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdc580b28a0 0x7fdc580b4db0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdc7006a3c0 tx=0x7fdc6000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.053+0000 7fdc55ffb640 1 -- 172.21.15.2:0/431458747 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68004b20 con 0x56005e7771b0 2024-05-14T09:41:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.054+0000 7fdc55ffb640 1 -- 172.21.15.2:0/431458747 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc68004cc0 con 0x56005e7771b0 2024-05-14T09:41:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.054+0000 7fdc55ffb640 1 -- 172.21.15.2:0/431458747 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68005540 con 0x56005e7771b0 2024-05-14T09:41:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.054+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/431458747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 msgr2=0x56005eb18a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.054+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/431458747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb18a60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdc68011a70 tx=0x7fdc68004ab0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.054+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/431458747 shutdown_connections 2024-05-14T09:41:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.054+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/431458747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e7564d0 0x56005eb1fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.054+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/431458747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e74e2f0 0x56005eb18fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.054+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/431458747 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb18a60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.054+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/431458747 >> 172.21.15.2:0/431458747 conn(0x56005e76b300 msgr2=0x56005eb143a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.055+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/431458747 shutdown_connections 2024-05-14T09:41:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.055+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/431458747 wait complete. 2024-05-14T09:41:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.055+0000 7fdc7c7fa740 1 -- start start 2024-05-14T09:41:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.055+0000 7fdc7c7fa740 1 Processor -- start 2024-05-14T09:41:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.055+0000 7fdc7c7fa740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e74e2f0 0x56005eb22790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.055+0000 7fdc7c7fa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e7564d0 0x56005eb22cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.055+0000 7fdc7c7fa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb29c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.055+0000 7fdc7c7fa740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56005eb20930 con 0x56005e74e2f0 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.055+0000 7fdc7c7fa740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56005ea3a370 con 0x56005e7564d0 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.055+0000 7fdc7c7fa740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56005eb20ab0 con 0x56005e7771b0 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc7c7fa740 10 client.0 _ll_register_callbacks cb 0x56005e78ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc6effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e74e2f0 0x56005eb22790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc6effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e74e2f0 0x56005eb22790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44282/0 (socket says 172.21.15.2:44282) 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc6e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e7564d0 0x56005eb22cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc6effd640 1 -- 172.21.15.2:0/1073532728 learned_addr learned my addr 172.21.15.2:0/1073532728 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc6f7fe640 1 --2- 172.21.15.2:0/1073532728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb29c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc6effd640 1 -- 172.21.15.2:0/1073532728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 msgr2=0x56005eb29c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc6effd640 1 --2- 172.21.15.2:0/1073532728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb29c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc6effd640 1 -- 172.21.15.2:0/1073532728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e7564d0 msgr2=0x56005eb22cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc6effd640 1 --2- 172.21.15.2:0/1073532728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e7564d0 0x56005eb22cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc6effd640 1 -- 172.21.15.2:0/1073532728 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005e85b620 con 0x56005e74e2f0 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc6e7fc640 1 --2- 172.21.15.2:0/1073532728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e7564d0 0x56005eb22cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:41:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc6effd640 1 --2- 172.21.15.2:0/1073532728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e74e2f0 0x56005eb22790 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000b6e0 tx=0x7fdc60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5db54946db1cbde server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc4effd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:41:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc4effd640 1 -- 172.21.15.2:0/1073532728 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6001e8f0 con 0x56005e74e2f0 2024-05-14T09:41:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc4effd640 1 -- 172.21.15.2:0/1073532728 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fdc6001ea90 con 0x56005e74e2f0 2024-05-14T09:41:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.056+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/1073532728 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc68004cc0 con 0x56005e74e2f0 2024-05-14T09:41:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.057+0000 7fdc4effd640 1 -- 172.21.15.2:0/1073532728 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdc6002f050 con 0x56005e74e2f0 2024-05-14T09:41:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.057+0000 7fdc7c7fa740 10 client.16260 fetch_fsmap learned FSMap version 34 2024-05-14T09:41:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.057+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/1073532728 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56005eb2cc30 con 0x56005e74e2f0 2024-05-14T09:41:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.057+0000 7fdc4effd640 1 -- 172.21.15.2:0/1073532728 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc68004cc0 con 0x56005e74e2f0 2024-05-14T09:41:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.057+0000 7fdc4effd640 1 -- 172.21.15.2:0/1073532728 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fdc6001c3b0 con 0x56005e74e2f0 2024-05-14T09:41:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.057+0000 7fdc7c7fa740 10 client.16260 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:41:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.057+0000 7fdc7c7fa740 10 client.16260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:41:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.057+0000 7fdc7c7fa740 20 client.16260 populate_metadata read hostname 'smithi002' 2024-05-14T09:41:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.057+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/1073532728 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56005e735d80 0x56005eb30430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.057+0000 7fdc7c7fa740 4 client.16260 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:41:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.057+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/1073532728 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56005ea8b1e0 con 0x56005e735d80 2024-05-14T09:41:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.057+0000 7fdc6e7fc640 1 --2- 172.21.15.2:0/1073532728 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56005e735d80 0x56005eb30430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.058+0000 7fdc4effd640 10 client.16260 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.058+0000 7fdc6e7fc640 1 --2- 172.21.15.2:0/1073532728 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56005e735d80 0x56005eb30430 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=671d47794a0b45fd server_cookie=683034cec8fa397 in_seq=0 out_seq=0 2024-05-14T09:41:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.059+0000 7fdc4effd640 1 -- 172.21.15.2:0/1073532728 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56005ea8b1e0 con 0x56005e735d80 2024-05-14T09:41:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.059+0000 7fdc4effd640 10 client.16260 handle_command_reply: tid=0 2024-05-14T09:41:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.059+0000 7fdc7c7fa740 1 client.16260 shutdown 2024-05-14T09:41:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.059+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/1073532728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e74e2f0 msgr2=0x56005eb22790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.059+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/1073532728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e74e2f0 0x56005eb22790 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000b6e0 tx=0x7fdc60004730 comp rx=0 tx=0).stop 2024-05-14T09:41:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.059+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/1073532728 shutdown_connections 2024-05-14T09:41:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.059+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/1073532728 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56005e735d80 0x56005eb30430 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.059+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/1073532728 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb29c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.059+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/1073532728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e7564d0 0x56005eb22cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.059+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/1073532728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e74e2f0 0x56005eb22790 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.059+0000 7fdc4effd640 0 client.16260 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.059+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/1073532728 >> 172.21.15.2:0/1073532728 conn(0x56005e76b300 msgr2=0x56005eb17300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.059+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/1073532728 shutdown_connections 2024-05-14T09:41:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.059+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/1073532728 wait complete. 2024-05-14T09:41:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.059+0000 7fdc7c7fa740 20 client.16260 trim_cache size 0 max 16384 2024-05-14T09:41:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.075+0000 7fdc7c7fa740 1 Processor -- start 2024-05-14T09:41:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.075+0000 7fdc7c7fa740 1 -- start start 2024-05-14T09:41:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc7c7fa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e74e2f0 0x56005eb17900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc7c7fa740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e7564d0 0x56005eb29fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc7c7fa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb13c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc7c7fa740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701af570 con 0x56005e7564d0 2024-05-14T09:41:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc7c7fa740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701ace10 con 0x56005e74e2f0 2024-05-14T09:41:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc7c7fa740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701aac60 con 0x56005e7771b0 2024-05-14T09:41:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc6e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e7564d0 0x56005eb29fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc6f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb13c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc6effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e74e2f0 0x56005eb17900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc6e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e7564d0 0x56005eb29fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44296/0 (socket says 172.21.15.2:44296) 2024-05-14T09:41:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc6f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb13c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:33196/0 (socket says 172.21.15.2:33196) 2024-05-14T09:41:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc6e7fc640 1 -- 172.21.15.2:0/2253411204 learned_addr learned my addr 172.21.15.2:0/2253411204 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc6e7fc640 1 -- 172.21.15.2:0/2253411204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 msgr2=0x56005eb13c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc6e7fc640 1 --2- 172.21.15.2:0/2253411204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb13c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc6e7fc640 1 -- 172.21.15.2:0/2253411204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e74e2f0 msgr2=0x56005eb17900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc6e7fc640 1 --2- 172.21.15.2:0/2253411204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e74e2f0 0x56005eb17900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc6e7fc640 1 -- 172.21.15.2:0/2253411204 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56005eb14d00 con 0x56005e7564d0 2024-05-14T09:41:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.076+0000 7fdc6e7fc640 1 --2- 172.21.15.2:0/2253411204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e7564d0 0x56005eb29fb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdc64038020 tx=0x7fdc64100180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dadbd4afc19095 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.077+0000 7fdc557fa640 1 -- 172.21.15.2:0/2253411204 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64049040 con 0x56005e7564d0 2024-05-14T09:41:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.077+0000 7fdc557fa640 1 -- 172.21.15.2:0/2253411204 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc64094070 con 0x56005e7564d0 2024-05-14T09:41:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.077+0000 7fdc557fa640 1 -- 172.21.15.2:0/2253411204 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc640cd040 con 0x56005e7564d0 2024-05-14T09:41:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.077+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/2253411204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e7564d0 msgr2=0x56005eb29fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.077+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/2253411204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e7564d0 0x56005eb29fb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdc64038020 tx=0x7fdc64100180 comp rx=0 tx=0).stop 2024-05-14T09:41:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.078+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/2253411204 shutdown_connections 2024-05-14T09:41:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.078+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/2253411204 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb13c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.078+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/2253411204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e7564d0 0x56005eb29fb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.078+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/2253411204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e74e2f0 0x56005eb17900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.078+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/2253411204 >> 172.21.15.2:0/2253411204 conn(0x56005e76b300 msgr2=0x56005eb171f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.078+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/2253411204 shutdown_connections 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.078+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/2253411204 wait complete. 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.078+0000 7fdc7c7fa740 1 -- start start 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.078+0000 7fdc7c7fa740 1 Processor -- start 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.078+0000 7fdc7c7fa740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e74e2f0 0x56005eb93810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.078+0000 7fdc7c7fa740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e7564d0 0x56005eb93d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.078+0000 7fdc7c7fa740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb9acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.078+0000 7fdc7c7fa740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701ab2d0 con 0x56005e7564d0 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.078+0000 7fdc7c7fa740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701ac740 con 0x56005e74e2f0 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.078+0000 7fdc7c7fa740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701ab9a0 con 0x56005e7771b0 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc7c7fa740 10 client.0 _ll_register_callbacks cb 0x7fdc6000b8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc6e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e7564d0 0x56005eb93d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc6e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e7564d0 0x56005eb93d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44306/0 (socket says 172.21.15.2:44306) 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc6e7fc640 1 -- 172.21.15.2:0/3274180713 learned_addr learned my addr 172.21.15.2:0/3274180713 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc6f7fe640 1 --2- 172.21.15.2:0/3274180713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb9acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc6effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e74e2f0 0x56005eb93810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc6e7fc640 1 -- 172.21.15.2:0/3274180713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 msgr2=0x56005eb9acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc6e7fc640 1 --2- 172.21.15.2:0/3274180713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb9acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc6e7fc640 1 -- 172.21.15.2:0/3274180713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e74e2f0 msgr2=0x56005eb93810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc6e7fc640 1 --2- 172.21.15.2:0/3274180713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e74e2f0 0x56005eb93810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc6e7fc640 1 -- 172.21.15.2:0/3274180713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005eb9b270 con 0x56005e7564d0 2024-05-14T09:41:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc6e7fc640 1 --2- 172.21.15.2:0/3274180713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e7564d0 0x56005eb93d50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdc6408b200 tx=0x7fdc6408f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667550bfa63d0732 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc4effd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:41:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc4effd640 1 -- 172.21.15.2:0/3274180713 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc640cd070 con 0x56005e7564d0 2024-05-14T09:41:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc4effd640 1 -- 172.21.15.2:0/3274180713 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7fdc640c6110 con 0x56005e7564d0 2024-05-14T09:41:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/3274180713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc64094070 con 0x56005e7564d0 2024-05-14T09:41:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc4effd640 1 -- 172.21.15.2:0/3274180713 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdc64049040 con 0x56005e7564d0 2024-05-14T09:41:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc7c7fa740 10 client.16266 fetch_fsmap learned FSMap version 34 2024-05-14T09:41:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.079+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/3274180713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56005eb9c5a0 con 0x56005e7564d0 2024-05-14T09:41:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.080+0000 7fdc4effd640 1 -- 172.21.15.2:0/3274180713 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc64094070 con 0x56005e7564d0 2024-05-14T09:41:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.080+0000 7fdc4effd640 1 -- 172.21.15.2:0/3274180713 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fdc640d27a0 con 0x56005e7564d0 2024-05-14T09:41:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.080+0000 7fdc7c7fa740 10 client.16266 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:41:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.080+0000 7fdc7c7fa740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:41:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.080+0000 7fdc7c7fa740 20 client.16266 populate_metadata read hostname 'smithi002' 2024-05-14T09:41:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.080+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/3274180713 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56005e70b1c0 0x56005eb9ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.080+0000 7fdc7c7fa740 4 client.16266 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:41:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.080+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/3274180713 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56005ea85050 con 0x56005e70b1c0 2024-05-14T09:41:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.080+0000 7fdc6effd640 1 --2- 172.21.15.2:0/3274180713 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56005e70b1c0 0x56005eb9ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.080+0000 7fdc4effd640 10 client.16266 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.080+0000 7fdc6effd640 1 --2- 172.21.15.2:0/3274180713 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56005e70b1c0 0x56005eb9ef40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=320b16f423f43529 server_cookie=fc97f93dc87c9b28 in_seq=0 out_seq=0 2024-05-14T09:41:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.197+0000 7fdc4effd640 1 -- 172.21.15.2:0/3274180713 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56005ea85050 con 0x56005e70b1c0 2024-05-14T09:41:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.197+0000 7fdc4effd640 10 client.16266 handle_command_reply: tid=0 2024-05-14T09:41:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.197+0000 7fdc7c7fa740 1 client.16266 shutdown 2024-05-14T09:41:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.197+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/3274180713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e7564d0 msgr2=0x56005eb93d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.198+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/3274180713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e7564d0 0x56005eb93d50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdc6408b200 tx=0x7fdc6408f420 comp rx=0 tx=0).stop 2024-05-14T09:41:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.198+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/3274180713 shutdown_connections 2024-05-14T09:41:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.198+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/3274180713 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56005e70b1c0 0x56005eb9ef40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.198+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/3274180713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56005e7771b0 0x56005eb9acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.198+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/3274180713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56005e7564d0 0x56005eb93d50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.198+0000 7fdc4effd640 0 client.16266 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.198+0000 7fdc7c7fa740 1 --2- 172.21.15.2:0/3274180713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56005e74e2f0 0x56005eb93810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.198+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/3274180713 >> 172.21.15.2:0/3274180713 conn(0x56005e76b300 msgr2=0x56005eb171f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.198+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/3274180713 shutdown_connections 2024-05-14T09:41:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.198+0000 7fdc7c7fa740 1 -- 172.21.15.2:0/3274180713 wait complete. 2024-05-14T09:41:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.198+0000 7fdc7c7fa740 20 client.16266 trim_cache size 0 max 16384 2024-05-14T09:41:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:41:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:41:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:41:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:41:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.199+0000 7fdc7593f640 1 -- 172.21.15.2:0/1253538469 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdc580b28a0 msgr2=0x7fdc580b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.199+0000 7fdc7593f640 1 --2- 172.21.15.2:0/1253538469 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdc580b28a0 0x7fdc580b4db0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdc7006a3c0 tx=0x7fdc6000b390 comp rx=0 tx=0).stop 2024-05-14T09:41:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.199+0000 7fdc7593f640 1 -- 172.21.15.2:0/1253538469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc70102380 msgr2=0x7fdc7019a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.199+0000 7fdc7593f640 1 --2- 172.21.15.2:0/1253538469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc70102380 0x7fdc7019a7f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdc64006010 tx=0x7fdc64031a50 comp rx=0 tx=0).stop 2024-05-14T09:41:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.199+0000 7fdc7593f640 1 -- 172.21.15.2:0/1253538469 shutdown_connections 2024-05-14T09:41:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.199+0000 7fdc7593f640 1 --2- 172.21.15.2:0/1253538469 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdc580b28a0 0x7fdc580b4db0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.199+0000 7fdc7593f640 1 --2- 172.21.15.2:0/1253538469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdc701051d0 0x7fdc701a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.199+0000 7fdc7593f640 1 --2- 172.21.15.2:0/1253538469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc70102380 0x7fdc7019a7f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.199+0000 7fdc7593f640 1 --2- 172.21.15.2:0/1253538469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc70101880 0x7fdc7019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.199+0000 7fdc7593f640 1 -- 172.21.15.2:0/1253538469 >> 172.21.15.2:0/1253538469 conn(0x7fdc700fd480 msgr2=0x7fdc70105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.199+0000 7fdc7593f640 1 -- 172.21.15.2:0/1253538469 shutdown_connections 2024-05-14T09:41:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:26.199+0000 7fdc7593f640 1 -- 172.21.15.2:0/1253538469 wait complete. 2024-05-14T09:41:26.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 564441 10.31 MB/sec execute 172 sec latency 74.883 ms 2024-05-14T09:41:26.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:26 smithi002 ceph-mon[31557]: pgmap v618: 129 pgs: 129 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 2.0 MiB/s wr, 271 op/s 2024-05-14T09:41:26.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:26 smithi029 ceph-mon[36249]: pgmap v618: 129 pgs: 129 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 2.0 MiB/s wr, 271 op/s 2024-05-14T09:41:26.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:26 smithi026 ceph-mon[36056]: pgmap v618: 129 pgs: 129 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 2.0 MiB/s wr, 271 op/s 2024-05-14T09:41:27.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 565534 10.29 MB/sec execute 173 sec latency 54.464 ms 2024-05-14T09:41:28.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 567971 10.30 MB/sec execute 174 sec latency 17.485 ms 2024-05-14T09:41:28.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:28 smithi002 ceph-mon[31557]: pgmap v619: 129 pgs: 129 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 360 op/s 2024-05-14T09:41:28.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:28 smithi029 ceph-mon[36249]: pgmap v619: 129 pgs: 129 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 360 op/s 2024-05-14T09:41:28.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:28 smithi026 ceph-mon[36056]: pgmap v619: 129 pgs: 129 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 360 op/s 2024-05-14T09:41:29.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 569150 10.28 MB/sec execute 175 sec latency 51.783 ms 2024-05-14T09:41:30.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 571829 10.32 MB/sec execute 176 sec latency 36.763 ms 2024-05-14T09:41:30.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:30 smithi002 ceph-mon[31557]: pgmap v620: 129 pgs: 129 active+clean; 110 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 337 op/s 2024-05-14T09:41:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:30 smithi029 ceph-mon[36249]: pgmap v620: 129 pgs: 129 active+clean; 110 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 337 op/s 2024-05-14T09:41:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:30 smithi026 ceph-mon[36056]: pgmap v620: 129 pgs: 129 active+clean; 110 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 337 op/s 2024-05-14T09:41:31.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 573274 10.30 MB/sec execute 177 sec latency 14.546 ms 2024-05-14T09:41:32.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 575648 10.32 MB/sec execute 178 sec latency 23.539 ms 2024-05-14T09:41:32.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:32 smithi002 ceph-mon[31557]: pgmap v621: 129 pgs: 129 active+clean; 107 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 321 op/s 2024-05-14T09:41:32.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:32 smithi029 ceph-mon[36249]: pgmap v621: 129 pgs: 129 active+clean; 107 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 321 op/s 2024-05-14T09:41:32.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:32 smithi026 ceph-mon[36056]: pgmap v621: 129 pgs: 129 active+clean; 107 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 321 op/s 2024-05-14T09:41:33.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 576818 10.29 MB/sec execute 179 sec latency 49.822 ms 2024-05-14T09:41:34.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 579181 10.31 MB/sec execute 180 sec latency 22.769 ms 2024-05-14T09:41:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:34 smithi002 ceph-mon[31557]: pgmap v622: 129 pgs: 129 active+clean; 109 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 341 op/s 2024-05-14T09:41:34.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:34 smithi029 ceph-mon[36249]: pgmap v622: 129 pgs: 129 active+clean; 109 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 341 op/s 2024-05-14T09:41:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:34 smithi026 ceph-mon[36056]: pgmap v622: 129 pgs: 129 active+clean; 109 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 341 op/s 2024-05-14T09:41:35.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 580636 10.28 MB/sec execute 181 sec latency 14.810 ms 2024-05-14T09:41:36.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 582862 10.30 MB/sec execute 182 sec latency 14.217 ms 2024-05-14T09:41:36.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:36 smithi002 ceph-mon[31557]: pgmap v623: 129 pgs: 129 active+clean; 109 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 2.2 MiB/s wr, 289 op/s 2024-05-14T09:41:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:36 smithi029 ceph-mon[36249]: pgmap v623: 129 pgs: 129 active+clean; 109 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 2.2 MiB/s wr, 289 op/s 2024-05-14T09:41:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:36 smithi026 ceph-mon[36056]: pgmap v623: 129 pgs: 129 active+clean; 109 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 2.2 MiB/s wr, 289 op/s 2024-05-14T09:41:37.267 INFO:tasks.workunit.client.0.smithi002.stdout: 1 584948 10.29 MB/sec execute 183 sec latency 14.558 ms 2024-05-14T09:41:38.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 586463 10.30 MB/sec execute 184 sec latency 59.590 ms 2024-05-14T09:41:38.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:38 smithi002 ceph-mon[31557]: pgmap v624: 129 pgs: 129 active+clean; 109 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 328 op/s 2024-05-14T09:41:38.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:38 smithi029 ceph-mon[36249]: pgmap v624: 129 pgs: 129 active+clean; 109 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 328 op/s 2024-05-14T09:41:38.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:38 smithi026 ceph-mon[36056]: pgmap v624: 129 pgs: 129 active+clean; 109 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 328 op/s 2024-05-14T09:41:39.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 588461 10.28 MB/sec execute 185 sec latency 35.792 ms 2024-05-14T09:41:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.304+0000 7f3770e61640 1 -- 172.21.15.2:0/2991863918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376c102b50 msgr2=0x7f376c103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.304+0000 7f3770e61640 1 --2- 172.21.15.2:0/2991863918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376c102b50 0x7f376c103850 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f375c009900 tx=0x7f375c02a410 comp rx=0 tx=0).stop 2024-05-14T09:41:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.304+0000 7f3770e61640 1 -- 172.21.15.2:0/2991863918 shutdown_connections 2024-05-14T09:41:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.304+0000 7f3770e61640 1 --2- 172.21.15.2:0/2991863918 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f376c108a90 0x7f376c10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.304+0000 7f3770e61640 1 --2- 172.21.15.2:0/2991863918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c103d90 0x7f376c1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.304+0000 7f3770e61640 1 --2- 172.21.15.2:0/2991863918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376c102b50 0x7f376c103850 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.304+0000 7f3770e61640 1 -- 172.21.15.2:0/2991863918 >> 172.21.15.2:0/2991863918 conn(0x7f376c0fdd60 msgr2=0x7f376c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.304+0000 7f3770e61640 1 -- 172.21.15.2:0/2991863918 shutdown_connections 2024-05-14T09:41:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.304+0000 7f3770e61640 1 -- 172.21.15.2:0/2991863918 wait complete. 2024-05-14T09:41:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.305+0000 7f3770e61640 1 Processor -- start 2024-05-14T09:41:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.305+0000 7f3770e61640 1 -- start start 2024-05-14T09:41:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f3770e61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c102b50 0x7f376c0ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f3770e61640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376c103d90 0x7f376c0fe290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f3770e61640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f376c108a90 0x7f376c0fe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f3770e61640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f376c10e630 con 0x7f376c103d90 2024-05-14T09:41:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f3770e61640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f376c10e4b0 con 0x7f376c102b50 2024-05-14T09:41:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f3770e61640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f376c10e7b0 con 0x7f376c108a90 2024-05-14T09:41:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f376a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c102b50 0x7f376c0ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f3769d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376c103d90 0x7f376c0fe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f376a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c102b50 0x7f376c0ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:35908/0 (socket says 172.21.15.2:35908) 2024-05-14T09:41:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f376a59c640 1 -- 172.21.15.2:0/2604626711 learned_addr learned my addr 172.21.15.2:0/2604626711 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f376ad9d640 1 --2- 172.21.15.2:0/2604626711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f376c108a90 0x7f376c0fe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f3769d9b640 1 -- 172.21.15.2:0/2604626711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f376c108a90 msgr2=0x7f376c0fe7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f3769d9b640 1 --2- 172.21.15.2:0/2604626711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f376c108a90 0x7f376c0fe7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f3769d9b640 1 -- 172.21.15.2:0/2604626711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c102b50 msgr2=0x7f376c0ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f3769d9b640 1 --2- 172.21.15.2:0/2604626711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c102b50 0x7f376c0ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f3769d9b640 1 -- 172.21.15.2:0/2604626711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c0fef00 con 0x7f376c103d90 2024-05-14T09:41:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f3769d9b640 1 --2- 172.21.15.2:0/2604626711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376c103d90 0x7f376c0fe290 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f376000e9f0 tx=0x7f376000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.306+0000 7f375b7fe640 1 -- 172.21.15.2:0/2604626711 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376000ccc0 con 0x7f376c103d90 2024-05-14T09:41:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.307+0000 7f375b7fe640 1 -- 172.21.15.2:0/2604626711 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37600108e0 con 0x7f376c103d90 2024-05-14T09:41:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.307+0000 7f375b7fe640 1 -- 172.21.15.2:0/2604626711 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760018880 con 0x7f376c103d90 2024-05-14T09:41:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.307+0000 7f3770e61640 1 -- 172.21.15.2:0/2604626711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376c1a4390 con 0x7f376c103d90 2024-05-14T09:41:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.307+0000 7f3770e61640 1 -- 172.21.15.2:0/2604626711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376c1a4880 con 0x7f376c103d90 2024-05-14T09:41:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.309+0000 7f3777dfb740 1 Processor -- start 2024-05-14T09:41:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f3777dfb740 1 -- start start 2024-05-14T09:41:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f375b7fe640 1 -- 172.21.15.2:0/2604626711 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3760010430 con 0x7f376c103d90 2024-05-14T09:41:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f3777dfb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48d51b0 0x556cb4c76a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f3777dfb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48ac2f0 0x556cb4c76fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f3777dfb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48b44d0 0x556cb4c7dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f3777dfb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556cb4bdb5d0 con 0x556cb48b44d0 2024-05-14T09:41:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f3777dfb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556cb4b94890 con 0x556cb48ac2f0 2024-05-14T09:41:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f3777dfb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556cb4bd7590 con 0x556cb48d51b0 2024-05-14T09:41:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f376ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48d51b0 0x556cb4c76a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f376a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48ac2f0 0x556cb4c76fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f3769d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48b44d0 0x556cb4c7dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f376ad9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48d51b0 0x556cb4c76a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:55354/0 (socket says 172.21.15.2:55354) 2024-05-14T09:41:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f376ad9d640 1 -- 172.21.15.2:0/4014129905 learned_addr learned my addr 172.21.15.2:0/4014129905 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f376ad9d640 1 -- 172.21.15.2:0/4014129905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48ac2f0 msgr2=0x556cb4c76fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f376ad9d640 1 --2- 172.21.15.2:0/4014129905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48ac2f0 0x556cb4c76fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f376ad9d640 1 -- 172.21.15.2:0/4014129905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48b44d0 msgr2=0x556cb4c7dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f376ad9d640 1 --2- 172.21.15.2:0/4014129905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48b44d0 0x556cb4c7dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.310+0000 7f376ad9d640 1 -- 172.21.15.2:0/4014129905 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cb48ed210 con 0x556cb48d51b0 2024-05-14T09:41:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.311+0000 7f376ad9d640 1 --2- 172.21.15.2:0/4014129905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48d51b0 0x556cb4c76a60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3764009fd0 tx=0x7f37640118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0c0be3e19147099 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.311+0000 7f375b7fe640 1 --2- 172.21.15.2:0/2604626711 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f37540b2920 0x7f37540b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.311+0000 7f3758ff9640 1 -- 172.21.15.2:0/4014129905 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764004d20 con 0x556cb48d51b0 2024-05-14T09:41:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.311+0000 7f3758ff9640 1 -- 172.21.15.2:0/4014129905 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3764004050 con 0x556cb48d51b0 2024-05-14T09:41:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.311+0000 7f3758ff9640 1 -- 172.21.15.2:0/4014129905 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764005260 con 0x556cb48d51b0 2024-05-14T09:41:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.311+0000 7f376a59c640 1 --2- 172.21.15.2:0/2604626711 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f37540b2920 0x7f37540b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.311+0000 7f375b7fe640 1 -- 172.21.15.2:0/2604626711 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f37600e28f0 con 0x7f376c103d90 2024-05-14T09:41:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.312+0000 7f3777dfb740 1 -- 172.21.15.2:0/4014129905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48d51b0 msgr2=0x556cb4c76a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.312+0000 7f376a59c640 1 --2- 172.21.15.2:0/2604626711 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f37540b2920 0x7f37540b4e30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f375c031500 tx=0x7f375c005df0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.312+0000 7f3777dfb740 1 --2- 172.21.15.2:0/4014129905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48d51b0 0x556cb4c76a60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3764009fd0 tx=0x7f37640118d0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.312+0000 7f3777dfb740 1 -- 172.21.15.2:0/4014129905 shutdown_connections 2024-05-14T09:41:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.312+0000 7f3777dfb740 1 --2- 172.21.15.2:0/4014129905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48b44d0 0x556cb4c7dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.312+0000 7f3777dfb740 1 --2- 172.21.15.2:0/4014129905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48ac2f0 0x556cb4c76fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.312+0000 7f3777dfb740 1 --2- 172.21.15.2:0/4014129905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48d51b0 0x556cb4c76a60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.312+0000 7f3777dfb740 1 -- 172.21.15.2:0/4014129905 >> 172.21.15.2:0/4014129905 conn(0x556cb48c9300 msgr2=0x556cb4c723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.312+0000 7f3777dfb740 1 -- 172.21.15.2:0/4014129905 shutdown_connections 2024-05-14T09:41:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.312+0000 7f3777dfb740 1 -- 172.21.15.2:0/4014129905 wait complete. 2024-05-14T09:41:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.312+0000 7f3777dfb740 1 -- start start 2024-05-14T09:41:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f3777dfb740 1 Processor -- start 2024-05-14T09:41:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f3777dfb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 0x556cb4c806d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f3777dfb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48b44d0 0x556cb4c80c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f3777dfb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48d51b0 0x556cb4c87b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f3777dfb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556cb4c7e930 con 0x556cb48d51b0 2024-05-14T09:41:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f3777dfb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556cb4b98370 con 0x556cb48b44d0 2024-05-14T09:41:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f3777dfb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556cb4c7eab0 con 0x556cb48ac2f0 2024-05-14T09:41:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f3777dfb740 10 client.0 _ll_register_callbacks cb 0x556cb48eca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:41:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f376a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 0x556cb4c806d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f376ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48d51b0 0x556cb4c87b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f376a59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 0x556cb4c806d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:55364/0 (socket says 172.21.15.2:55364) 2024-05-14T09:41:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f376a59c640 1 -- 172.21.15.2:0/1800204156 learned_addr learned my addr 172.21.15.2:0/1800204156 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f3769d9b640 1 --2- 172.21.15.2:0/1800204156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48b44d0 0x556cb4c80c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f376ad9d640 1 -- 172.21.15.2:0/1800204156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 msgr2=0x556cb4c806d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f376ad9d640 1 --2- 172.21.15.2:0/1800204156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 0x556cb4c806d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f376ad9d640 1 -- 172.21.15.2:0/1800204156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48b44d0 msgr2=0x556cb4c80c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f376ad9d640 1 --2- 172.21.15.2:0/1800204156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48b44d0 0x556cb4c80c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f376ad9d640 1 -- 172.21.15.2:0/1800204156 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cb49b9620 con 0x556cb48d51b0 2024-05-14T09:41:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.313+0000 7f3769d9b640 1 --2- 172.21.15.2:0/1800204156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48b44d0 0x556cb4c80c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:41:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f376ad9d640 1 --2- 172.21.15.2:0/1800204156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48d51b0 0x556cb4c87b70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f376401b040 tx=0x7f3764003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde250a9b2b51f1b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f3751ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:41:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f3751ffb640 1 -- 172.21.15.2:0/1800204156 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764028020 con 0x556cb48d51b0 2024-05-14T09:41:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f3751ffb640 1 -- 172.21.15.2:0/1800204156 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3764004410 con 0x556cb48d51b0 2024-05-14T09:41:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f3777dfb740 1 -- 172.21.15.2:0/1800204156 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3764004050 con 0x556cb48d51b0 2024-05-14T09:41:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f3751ffb640 1 -- 172.21.15.2:0/1800204156 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3764033050 con 0x556cb48d51b0 2024-05-14T09:41:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f3777dfb740 10 client.16278 fetch_fsmap learned FSMap version 34 2024-05-14T09:41:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f3777dfb740 1 -- 172.21.15.2:0/1800204156 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556cb4c8aaa0 con 0x556cb48d51b0 2024-05-14T09:41:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f3751ffb640 1 -- 172.21.15.2:0/1800204156 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3764004050 con 0x556cb48d51b0 2024-05-14T09:41:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f3751ffb640 1 -- 172.21.15.2:0/1800204156 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f376400f4e0 con 0x556cb48d51b0 2024-05-14T09:41:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f3777dfb740 10 client.16278 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:41:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f3777dfb740 10 client.16278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:41:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f3777dfb740 20 client.16278 populate_metadata read hostname 'smithi002' 2024-05-14T09:41:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f3777dfb740 1 --2- 172.21.15.2:0/1800204156 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cb4893d80 0x556cb4c8e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f3777dfb740 4 client.16278 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.314+0000 7f3777dfb740 1 -- 172.21.15.2:0/1800204156 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cb4be91e0 con 0x556cb4893d80 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.315+0000 7f376a59c640 1 --2- 172.21.15.2:0/1800204156 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cb4893d80 0x556cb4c8e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.315+0000 7f376a59c640 1 --2- 172.21.15.2:0/1800204156 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cb4893d80 0x556cb4c8e100 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91bd30d370aeb8ac server_cookie=f6689582bec10fc2 in_seq=0 out_seq=0 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.315+0000 7f3751ffb640 10 client.16278 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.316+0000 7f3751ffb640 1 -- 172.21.15.2:0/1800204156 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556cb4be91e0 con 0x556cb4893d80 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.316+0000 7f3751ffb640 10 client.16278 handle_command_reply: tid=0 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.316+0000 7f3777dfb740 1 client.16278 shutdown 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.316+0000 7f3777dfb740 1 -- 172.21.15.2:0/1800204156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48d51b0 msgr2=0x556cb4c87b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.316+0000 7f3777dfb740 1 --2- 172.21.15.2:0/1800204156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48d51b0 0x556cb4c87b70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f376401b040 tx=0x7f3764003ce0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.316+0000 7f3777dfb740 1 -- 172.21.15.2:0/1800204156 shutdown_connections 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.316+0000 7f3777dfb740 1 --2- 172.21.15.2:0/1800204156 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cb4893d80 0x556cb4c8e100 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.316+0000 7f3777dfb740 1 --2- 172.21.15.2:0/1800204156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48d51b0 0x556cb4c87b70 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.316+0000 7f3751ffb640 0 client.16278 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.316+0000 7f3777dfb740 1 --2- 172.21.15.2:0/1800204156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48b44d0 0x556cb4c80c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.316+0000 7f3777dfb740 1 --2- 172.21.15.2:0/1800204156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 0x556cb4c806d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.316+0000 7f3777dfb740 1 -- 172.21.15.2:0/1800204156 >> 172.21.15.2:0/1800204156 conn(0x556cb48c9300 msgr2=0x556cb4c75300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.316+0000 7f3777dfb740 1 -- 172.21.15.2:0/1800204156 shutdown_connections 2024-05-14T09:41:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.316+0000 7f3777dfb740 1 -- 172.21.15.2:0/1800204156 wait complete. 2024-05-14T09:41:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.316+0000 7f3777dfb740 20 client.16278 trim_cache size 0 max 16384 2024-05-14T09:41:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f3777dfb740 1 Processor -- start 2024-05-14T09:41:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f3777dfb740 1 -- start start 2024-05-14T09:41:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f3777dfb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb4893d80 0x556cb4c7b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f3777dfb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 0x556cb4c80070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f3777dfb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48b44d0 0x556cb4c87a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f3777dfb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1b1860 con 0x556cb48b44d0 2024-05-14T09:41:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f3777dfb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1af100 con 0x556cb4893d80 2024-05-14T09:41:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f3777dfb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1acf50 con 0x556cb48ac2f0 2024-05-14T09:41:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f3769d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 0x556cb4c80070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f376a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb4893d80 0x556cb4c7b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f376ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48b44d0 0x556cb4c87a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f376ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48b44d0 0x556cb4c87a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51020/0 (socket says 172.21.15.2:51020) 2024-05-14T09:41:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f376a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb4893d80 0x556cb4c7b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:35934/0 (socket says 172.21.15.2:35934) 2024-05-14T09:41:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f3769d9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 0x556cb4c80070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:55370/0 (socket says 172.21.15.2:55370) 2024-05-14T09:41:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f376ad9d640 1 -- 172.21.15.2:0/4294020932 learned_addr learned my addr 172.21.15.2:0/4294020932 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f3769d9b640 1 -- 172.21.15.2:0/4294020932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb4893d80 msgr2=0x556cb4c7b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f3769d9b640 1 --2- 172.21.15.2:0/4294020932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb4893d80 0x556cb4c7b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f3769d9b640 1 -- 172.21.15.2:0/4294020932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48b44d0 msgr2=0x556cb4c87a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f3769d9b640 1 --2- 172.21.15.2:0/4294020932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48b44d0 0x556cb4c87a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f3769d9b640 1 -- 172.21.15.2:0/4294020932 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cb4c72d00 con 0x556cb48ac2f0 2024-05-14T09:41:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.335+0000 7f376ad9d640 1 --2- 172.21.15.2:0/4294020932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48b44d0 0x556cb4c87a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:41:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.336+0000 7f3769d9b640 1 --2- 172.21.15.2:0/4294020932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 0x556cb4c80070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f37600e0020 tx=0x7f37600e8550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a2d819cfa7c838a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.336+0000 7f3753fff640 1 -- 172.21.15.2:0/4294020932 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37600a5330 con 0x556cb48ac2f0 2024-05-14T09:41:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.336+0000 7f3753fff640 1 -- 172.21.15.2:0/4294020932 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3760010720 con 0x556cb48ac2f0 2024-05-14T09:41:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.336+0000 7f3753fff640 1 -- 172.21.15.2:0/4294020932 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760010a90 con 0x556cb48ac2f0 2024-05-14T09:41:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 -- 172.21.15.2:0/4294020932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 msgr2=0x556cb4c80070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 --2- 172.21.15.2:0/4294020932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 0x556cb4c80070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f37600e0020 tx=0x7f37600e8550 comp rx=0 tx=0).stop 2024-05-14T09:41:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 -- 172.21.15.2:0/4294020932 shutdown_connections 2024-05-14T09:41:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 --2- 172.21.15.2:0/4294020932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb48b44d0 0x556cb4c87a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 --2- 172.21.15.2:0/4294020932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 0x556cb4c80070 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 --2- 172.21.15.2:0/4294020932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb4893d80 0x556cb4c7b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 -- 172.21.15.2:0/4294020932 >> 172.21.15.2:0/4294020932 conn(0x556cb48c9300 msgr2=0x556cb4c7f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 -- 172.21.15.2:0/4294020932 shutdown_connections 2024-05-14T09:41:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 -- 172.21.15.2:0/4294020932 wait complete. 2024-05-14T09:41:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 -- start start 2024-05-14T09:41:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 Processor -- start 2024-05-14T09:41:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb4893d80 0x556cb4c85200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 0x556cb4c85740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48b44d0 0x556cb4cf4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1ad5c0 con 0x556cb4893d80 2024-05-14T09:41:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1aea30 con 0x556cb48b44d0 2024-05-14T09:41:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.337+0000 7f3777dfb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1adc90 con 0x556cb48ac2f0 2024-05-14T09:41:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f3777dfb740 10 client.0 _ll_register_callbacks cb 0x7f37600e8420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:41:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f376a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb4893d80 0x556cb4c85200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f376a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb4893d80 0x556cb4c85200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51036/0 (socket says 172.21.15.2:51036) 2024-05-14T09:41:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f376a59c640 1 -- 172.21.15.2:0/1747312052 learned_addr learned my addr 172.21.15.2:0/1747312052 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f376ad9d640 1 --2- 172.21.15.2:0/1747312052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48b44d0 0x556cb4cf4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f3769d9b640 1 --2- 172.21.15.2:0/1747312052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 0x556cb4c85740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f376a59c640 1 -- 172.21.15.2:0/1747312052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 msgr2=0x556cb4c85740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f376a59c640 1 --2- 172.21.15.2:0/1747312052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 0x556cb4c85740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f376a59c640 1 -- 172.21.15.2:0/1747312052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48b44d0 msgr2=0x556cb4cf4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f376a59c640 1 --2- 172.21.15.2:0/1747312052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48b44d0 0x556cb4cf4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f376a59c640 1 -- 172.21.15.2:0/1747312052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cb4cf49a0 con 0x556cb4893d80 2024-05-14T09:41:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f376a59c640 1 --2- 172.21.15.2:0/1747312052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb4893d80 0x556cb4c85200 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f375c031440 tx=0x7f375c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5990410db4fe5a84 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f3751ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:41:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f3751ffb640 1 -- 172.21.15.2:0/1747312052 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c035040 con 0x556cb4893d80 2024-05-14T09:41:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f3751ffb640 1 -- 172.21.15.2:0/1747312052 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f375c053680 con 0x556cb4893d80 2024-05-14T09:41:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.338+0000 7f3777dfb740 1 -- 172.21.15.2:0/1747312052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556cb4be3050 con 0x556cb4893d80 2024-05-14T09:41:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.339+0000 7f3751ffb640 1 -- 172.21.15.2:0/1747312052 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f375c063050 con 0x556cb4893d80 2024-05-14T09:41:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.339+0000 7f3777dfb740 10 client.16284 fetch_fsmap learned FSMap version 34 2024-05-14T09:41:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.339+0000 7f3777dfb740 1 -- 172.21.15.2:0/1747312052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556cb4cf5b80 con 0x556cb4893d80 2024-05-14T09:41:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.339+0000 7f3751ffb640 1 -- 172.21.15.2:0/1747312052 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556cb4be3050 con 0x556cb4893d80 2024-05-14T09:41:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.339+0000 7f3751ffb640 1 -- 172.21.15.2:0/1747312052 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f375c050070 con 0x556cb4893d80 2024-05-14T09:41:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.339+0000 7f3777dfb740 10 client.16284 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:41:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.339+0000 7f3777dfb740 10 client.16284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:41:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.339+0000 7f3777dfb740 20 client.16284 populate_metadata read hostname 'smithi002' 2024-05-14T09:41:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.339+0000 7f3777dfb740 1 --2- 172.21.15.2:0/1747312052 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cb48d51b0 0x556cb4cf85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.339+0000 7f3777dfb740 4 client.16284 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:41:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.339+0000 7f3777dfb740 1 -- 172.21.15.2:0/1747312052 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556cb4b5e5f0 con 0x556cb48d51b0 2024-05-14T09:41:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.339+0000 7f3769d9b640 1 --2- 172.21.15.2:0/1747312052 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cb48d51b0 0x556cb4cf85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.340+0000 7f3751ffb640 10 client.16284 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.340+0000 7f3769d9b640 1 --2- 172.21.15.2:0/1747312052 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cb48d51b0 0x556cb4cf85b0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ebbb9221213d8c server_cookie=71e1fd1d0fdabe71 in_seq=0 out_seq=0 2024-05-14T09:41:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.454+0000 7f3751ffb640 1 -- 172.21.15.2:0/1747312052 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556cb4b5e5f0 con 0x556cb48d51b0 2024-05-14T09:41:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.454+0000 7f3751ffb640 10 client.16284 handle_command_reply: tid=0 2024-05-14T09:41:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.454+0000 7f3777dfb740 1 client.16284 shutdown 2024-05-14T09:41:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.454+0000 7f3777dfb740 1 -- 172.21.15.2:0/1747312052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb4893d80 msgr2=0x556cb4c85200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.454+0000 7f3777dfb740 1 --2- 172.21.15.2:0/1747312052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb4893d80 0x556cb4c85200 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f375c031440 tx=0x7f375c009260 comp rx=0 tx=0).stop 2024-05-14T09:41:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.454+0000 7f3777dfb740 1 -- 172.21.15.2:0/1747312052 shutdown_connections 2024-05-14T09:41:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.454+0000 7f3777dfb740 1 --2- 172.21.15.2:0/1747312052 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556cb48d51b0 0x556cb4cf85b0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.454+0000 7f3777dfb740 1 --2- 172.21.15.2:0/1747312052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cb48b44d0 0x556cb4cf4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.454+0000 7f3777dfb740 1 --2- 172.21.15.2:0/1747312052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556cb48ac2f0 0x556cb4c85740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.454+0000 7f3777dfb740 1 --2- 172.21.15.2:0/1747312052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556cb4893d80 0x556cb4c85200 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.454+0000 7f3751ffb640 0 client.16284 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.454+0000 7f3777dfb740 1 -- 172.21.15.2:0/1747312052 >> 172.21.15.2:0/1747312052 conn(0x556cb48c9300 msgr2=0x556cb4c80390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.454+0000 7f3777dfb740 1 -- 172.21.15.2:0/1747312052 shutdown_connections 2024-05-14T09:41:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.454+0000 7f3777dfb740 1 -- 172.21.15.2:0/1747312052 wait complete. 2024-05-14T09:41:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.455+0000 7f3777dfb740 20 client.16284 trim_cache size 0 max 16384 2024-05-14T09:41:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:41:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:41:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:41:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:41:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.455+0000 7f3770e61640 1 -- 172.21.15.2:0/2604626711 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f37540b2920 msgr2=0x7f37540b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.455+0000 7f3770e61640 1 --2- 172.21.15.2:0/2604626711 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f37540b2920 0x7f37540b4e30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f375c031500 tx=0x7f375c005df0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.455+0000 7f3770e61640 1 -- 172.21.15.2:0/2604626711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376c103d90 msgr2=0x7f376c0fe290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.455+0000 7f3770e61640 1 --2- 172.21.15.2:0/2604626711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376c103d90 0x7f376c0fe290 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f376000e9f0 tx=0x7f376000eeb0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.455+0000 7f3770e61640 1 -- 172.21.15.2:0/2604626711 shutdown_connections 2024-05-14T09:41:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.455+0000 7f3770e61640 1 --2- 172.21.15.2:0/2604626711 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f37540b2920 0x7f37540b4e30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.455+0000 7f3770e61640 1 --2- 172.21.15.2:0/2604626711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f376c108a90 0x7f376c0fe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.455+0000 7f3770e61640 1 --2- 172.21.15.2:0/2604626711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376c103d90 0x7f376c0fe290 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.455+0000 7f3770e61640 1 --2- 172.21.15.2:0/2604626711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f376c102b50 0x7f376c0ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.455+0000 7f3770e61640 1 -- 172.21.15.2:0/2604626711 >> 172.21.15.2:0/2604626711 conn(0x7f376c0fdd60 msgr2=0x7f376c109f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.455+0000 7f3770e61640 1 -- 172.21.15.2:0/2604626711 shutdown_connections 2024-05-14T09:41:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:39.455+0000 7f3770e61640 1 -- 172.21.15.2:0/2604626711 wait complete. 2024-05-14T09:41:40.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 590005 10.29 MB/sec execute 186 sec latency 105.811 ms 2024-05-14T09:41:40.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:40 smithi002 ceph-mon[31557]: pgmap v625: 129 pgs: 129 active+clean; 112 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 323 op/s 2024-05-14T09:41:40.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:40 smithi029 ceph-mon[36249]: pgmap v625: 129 pgs: 129 active+clean; 112 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 323 op/s 2024-05-14T09:41:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:40 smithi026 ceph-mon[36056]: pgmap v625: 129 pgs: 129 active+clean; 112 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 323 op/s 2024-05-14T09:41:41.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 592063 10.27 MB/sec execute 187 sec latency 15.312 ms 2024-05-14T09:41:42.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 593704 10.28 MB/sec execute 188 sec latency 18.002 ms 2024-05-14T09:41:42.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:42 smithi002 ceph-mon[31557]: pgmap v626: 129 pgs: 129 active+clean; 112 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 2.6 MiB/s wr, 266 op/s 2024-05-14T09:41:42.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:42 smithi029 ceph-mon[36249]: pgmap v626: 129 pgs: 129 active+clean; 112 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 2.6 MiB/s wr, 266 op/s 2024-05-14T09:41:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:42 smithi026 ceph-mon[36056]: pgmap v626: 129 pgs: 129 active+clean; 112 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 2.6 MiB/s wr, 266 op/s 2024-05-14T09:41:43.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 595582 10.26 MB/sec execute 189 sec latency 50.703 ms 2024-05-14T09:41:44.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 597175 10.27 MB/sec execute 190 sec latency 17.857 ms 2024-05-14T09:41:44.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:44 smithi002 ceph-mon[31557]: pgmap v627: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 381 op/s 2024-05-14T09:41:44.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:44 smithi029 ceph-mon[36249]: pgmap v627: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 381 op/s 2024-05-14T09:41:44.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:44 smithi026 ceph-mon[36056]: pgmap v627: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 381 op/s 2024-05-14T09:41:45.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 599302 10.26 MB/sec execute 191 sec latency 16.383 ms 2024-05-14T09:41:46.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 600847 10.26 MB/sec execute 192 sec latency 13.888 ms 2024-05-14T09:41:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:46 smithi002 ceph-mon[31557]: pgmap v628: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 2.4 MiB/s wr, 288 op/s 2024-05-14T09:41:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:41:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:41:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:41:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:41:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:46 smithi029 ceph-mon[36249]: pgmap v628: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 2.4 MiB/s wr, 288 op/s 2024-05-14T09:41:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:41:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:41:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:41:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:41:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:46 smithi026 ceph-mon[36056]: pgmap v628: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 2.4 MiB/s wr, 288 op/s 2024-05-14T09:41:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:41:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:41:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:41:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:41:47.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 603027 10.27 MB/sec execute 193 sec latency 15.560 ms 2024-05-14T09:41:48.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 604514 10.26 MB/sec execute 194 sec latency 52.077 ms 2024-05-14T09:41:48.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:48 smithi002 ceph-mon[31557]: pgmap v629: 129 pgs: 129 active+clean; 111 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 318 op/s 2024-05-14T09:41:48.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:48 smithi029 ceph-mon[36249]: pgmap v629: 129 pgs: 129 active+clean; 111 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 318 op/s 2024-05-14T09:41:48.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:48 smithi026 ceph-mon[36056]: pgmap v629: 129 pgs: 129 active+clean; 111 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 318 op/s 2024-05-14T09:41:49.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 606677 10.26 MB/sec execute 195 sec latency 17.673 ms 2024-05-14T09:41:50.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 608134 10.25 MB/sec execute 196 sec latency 13.409 ms 2024-05-14T09:41:50.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:50 smithi002 ceph-mon[31557]: pgmap v630: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 351 op/s 2024-05-14T09:41:50.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:50 smithi029 ceph-mon[36249]: pgmap v630: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 351 op/s 2024-05-14T09:41:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:50 smithi026 ceph-mon[36056]: pgmap v630: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 351 op/s 2024-05-14T09:41:51.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 610128 10.25 MB/sec execute 197 sec latency 179.853 ms 2024-05-14T09:41:52.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 611695 10.25 MB/sec execute 198 sec latency 13.093 ms 2024-05-14T09:41:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.558+0000 7f0b4df0d640 1 -- 172.21.15.2:0/2614652574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b48103d90 msgr2=0x7f0b481041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.558+0000 7f0b4df0d640 1 --2- 172.21.15.2:0/2614652574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b48103d90 0x7f0b481041a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009860 tx=0x7f0b3c02a340 comp rx=0 tx=0).stop 2024-05-14T09:41:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.558+0000 7f0b4df0d640 1 -- 172.21.15.2:0/2614652574 shutdown_connections 2024-05-14T09:41:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.558+0000 7f0b4df0d640 1 --2- 172.21.15.2:0/2614652574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b48108a90 0x7f0b4810aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.558+0000 7f0b4df0d640 1 --2- 172.21.15.2:0/2614652574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b48103d90 0x7f0b481041a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.558+0000 7f0b4df0d640 1 --2- 172.21.15.2:0/2614652574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b48102b50 0x7f0b48103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.558+0000 7f0b4df0d640 1 -- 172.21.15.2:0/2614652574 >> 172.21.15.2:0/2614652574 conn(0x7f0b480fdd40 msgr2=0x7f0b481001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.558+0000 7f0b4df0d640 1 -- 172.21.15.2:0/2614652574 shutdown_connections 2024-05-14T09:41:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.559+0000 7f0b4df0d640 1 -- 172.21.15.2:0/2614652574 wait complete. 2024-05-14T09:41:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.559+0000 7f0b4df0d640 1 Processor -- start 2024-05-14T09:41:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.559+0000 7f0b4df0d640 1 -- start start 2024-05-14T09:41:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b4df0d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b48102b50 0x7f0b48199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b4df0d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b48103d90 0x7f0b481996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b4df0d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b48108a90 0x7f0b481a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b4df0d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4810e650 con 0x7f0b48103d90 2024-05-14T09:41:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b4df0d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4810e4d0 con 0x7f0b48102b50 2024-05-14T09:41:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b4df0d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4810e7d0 con 0x7f0b48108a90 2024-05-14T09:41:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b46ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b48103d90 0x7f0b481996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b46ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b48103d90 0x7f0b481996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37354/0 (socket says 172.21.15.2:37354) 2024-05-14T09:41:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b46ffd640 1 -- 172.21.15.2:0/957764675 learned_addr learned my addr 172.21.15.2:0/957764675 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b477fe640 1 --2- 172.21.15.2:0/957764675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b48102b50 0x7f0b48199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b47fff640 1 --2- 172.21.15.2:0/957764675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b48108a90 0x7f0b481a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b46ffd640 1 -- 172.21.15.2:0/957764675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b48108a90 msgr2=0x7f0b481a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b46ffd640 1 --2- 172.21.15.2:0/957764675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b48108a90 0x7f0b481a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b46ffd640 1 -- 172.21.15.2:0/957764675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b48102b50 msgr2=0x7f0b48199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b46ffd640 1 --2- 172.21.15.2:0/957764675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b48102b50 0x7f0b48199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b46ffd640 1 -- 172.21.15.2:0/957764675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b481a0cd0 con 0x7f0b48103d90 2024-05-14T09:41:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b46ffd640 1 --2- 172.21.15.2:0/957764675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b48103d90 0x7f0b481996d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009980 tx=0x7f0b3c032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b44ff9640 1 -- 172.21.15.2:0/957764675 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c040020 con 0x7f0b48103d90 2024-05-14T09:41:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b44ff9640 1 -- 172.21.15.2:0/957764675 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b3c030dd0 con 0x7f0b48103d90 2024-05-14T09:41:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.560+0000 7f0b4df0d640 1 -- 172.21.15.2:0/957764675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b481a0f60 con 0x7f0b48103d90 2024-05-14T09:41:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.561+0000 7f0b44ff9640 1 -- 172.21.15.2:0/957764675 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c037040 con 0x7f0b48103d90 2024-05-14T09:41:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.561+0000 7f0b4df0d640 1 -- 172.21.15.2:0/957764675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b481a1370 con 0x7f0b48103d90 2024-05-14T09:41:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.563+0000 7f0b54dc1740 1 Processor -- start 2024-05-14T09:41:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.563+0000 7f0b44ff9640 1 -- 172.21.15.2:0/957764675 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0b3c043020 con 0x7f0b48103d90 2024-05-14T09:41:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.563+0000 7f0b54dc1740 1 -- start start 2024-05-14T09:41:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b54dc1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbff1b0 0x5604cdfa20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b54dc1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbd62f0 0x5604cdfa07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b54dc1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbde4d0 0x5604cdfa0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b54dc1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5604cdf055d0 con 0x5604cdbff1b0 2024-05-14T09:41:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b54dc1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5604cdebe890 con 0x5604cdbd62f0 2024-05-14T09:41:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b54dc1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5604cdf01590 con 0x5604cdbde4d0 2024-05-14T09:41:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b47fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbff1b0 0x5604cdfa20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b47fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbff1b0 0x5604cdfa20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37356/0 (socket says 172.21.15.2:37356) 2024-05-14T09:41:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b47fff640 1 -- 172.21.15.2:0/3028277845 learned_addr learned my addr 172.21.15.2:0/3028277845 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b477fe640 1 --2- 172.21.15.2:0/3028277845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbd62f0 0x5604cdfa07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b46ffd640 1 --2- 172.21.15.2:0/3028277845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbde4d0 0x5604cdfa0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b47fff640 1 -- 172.21.15.2:0/3028277845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbde4d0 msgr2=0x5604cdfa0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b47fff640 1 --2- 172.21.15.2:0/3028277845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbde4d0 0x5604cdfa0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b47fff640 1 -- 172.21.15.2:0/3028277845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbd62f0 msgr2=0x5604cdfa07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b47fff640 1 --2- 172.21.15.2:0/3028277845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbd62f0 0x5604cdfa07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b47fff640 1 -- 172.21.15.2:0/3028277845 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604cdc17210 con 0x5604cdbff1b0 2024-05-14T09:41:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b477fe640 1 --2- 172.21.15.2:0/3028277845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbd62f0 0x5604cdfa07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:41:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b44ff9640 1 --2- 172.21.15.2:0/957764675 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0b300b28a0 0x7f0b300b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.564+0000 7f0b47fff640 1 --2- 172.21.15.2:0/3028277845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbff1b0 0x5604cdfa20e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0b40011440 tx=0x7f0b40011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8088467ae37864f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.565+0000 7f0b477fe640 1 --2- 172.21.15.2:0/957764675 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0b300b28a0 0x7f0b300b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.565+0000 7f0b44ff9640 1 -- 172.21.15.2:0/957764675 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f0b3c102440 con 0x7f0b48103d90 2024-05-14T09:41:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.565+0000 7f0b2dffb640 1 -- 172.21.15.2:0/3028277845 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40003c30 con 0x5604cdbff1b0 2024-05-14T09:41:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.565+0000 7f0b2dffb640 1 -- 172.21.15.2:0/3028277845 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b40003dd0 con 0x5604cdbff1b0 2024-05-14T09:41:52.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.565+0000 7f0b2dffb640 1 -- 172.21.15.2:0/3028277845 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4000fa80 con 0x5604cdbff1b0 2024-05-14T09:41:52.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.566+0000 7f0b54dc1740 1 -- 172.21.15.2:0/3028277845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbff1b0 msgr2=0x5604cdfa20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:52.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.566+0000 7f0b477fe640 1 --2- 172.21.15.2:0/957764675 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0b300b28a0 0x7f0b300b4db0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5604cdfa1e00 tx=0x7f0b38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:52.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.566+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/3028277845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbff1b0 0x5604cdfa20e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0b40011440 tx=0x7f0b40011900 comp rx=0 tx=0).stop 2024-05-14T09:41:52.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.566+0000 7f0b54dc1740 1 -- 172.21.15.2:0/3028277845 shutdown_connections 2024-05-14T09:41:52.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.566+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/3028277845 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbde4d0 0x5604cdfa0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.566+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/3028277845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbd62f0 0x5604cdfa07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.566+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/3028277845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbff1b0 0x5604cdfa20e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.566+0000 7f0b54dc1740 1 -- 172.21.15.2:0/3028277845 >> 172.21.15.2:0/3028277845 conn(0x5604cdbf3300 msgr2=0x5604cdf9c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:52.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.566+0000 7f0b54dc1740 1 -- 172.21.15.2:0/3028277845 shutdown_connections 2024-05-14T09:41:52.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.566+0000 7f0b54dc1740 1 -- 172.21.15.2:0/3028277845 wait complete. 2024-05-14T09:41:52.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.566+0000 7f0b54dc1740 1 -- start start 2024-05-14T09:41:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.566+0000 7f0b54dc1740 1 Processor -- start 2024-05-14T09:41:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b54dc1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbd62f0 0x5604cdfaeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b54dc1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbde4d0 0x5604cdfaf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b54dc1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbff1b0 0x5604cdfb6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b54dc1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5604cdfacdc0 con 0x5604cdbde4d0 2024-05-14T09:41:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b54dc1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5604cdec2370 con 0x5604cdbff1b0 2024-05-14T09:41:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b54dc1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5604cdfacf40 con 0x5604cdbd62f0 2024-05-14T09:41:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b54dc1740 10 client.0 _ll_register_callbacks cb 0x5604cdc16a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b46ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbde4d0 0x5604cdfaf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b46ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbde4d0 0x5604cdfaf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37358/0 (socket says 172.21.15.2:37358) 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b46ffd640 1 -- 172.21.15.2:0/3174128231 learned_addr learned my addr 172.21.15.2:0/3174128231 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b477fe640 1 --2- 172.21.15.2:0/3174128231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbd62f0 0x5604cdfaeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b47fff640 1 --2- 172.21.15.2:0/3174128231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbff1b0 0x5604cdfb6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b46ffd640 1 -- 172.21.15.2:0/3174128231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbd62f0 msgr2=0x5604cdfaeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b46ffd640 1 --2- 172.21.15.2:0/3174128231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbd62f0 0x5604cdfaeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b46ffd640 1 -- 172.21.15.2:0/3174128231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbff1b0 msgr2=0x5604cdfb6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b46ffd640 1 --2- 172.21.15.2:0/3174128231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbff1b0 0x5604cdfb6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b46ffd640 1 -- 172.21.15.2:0/3174128231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604cdce3620 con 0x5604cdbde4d0 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b46ffd640 1 --2- 172.21.15.2:0/3174128231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbde4d0 0x5604cdfaf0a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0ffb20 tx=0x7f0b3c0c55c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1514798f921fe95 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.567+0000 7f0b26ffd640 1 -- 172.21.15.2:0/3174128231 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c0c8f90 con 0x5604cdbde4d0 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.568+0000 7f0b54dc1740 1 -- 172.21.15.2:0/3174128231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b40003dd0 con 0x5604cdbde4d0 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.568+0000 7f0b26ffd640 1 -- 172.21.15.2:0/3174128231 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f0b3c0b1070 con 0x5604cdbde4d0 2024-05-14T09:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.568+0000 7f0b26ffd640 1 -- 172.21.15.2:0/3174128231 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0b3c0c8db0 con 0x5604cdbde4d0 2024-05-14T09:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.568+0000 7f0b54dc1740 10 client.16296 fetch_fsmap learned FSMap version 34 2024-05-14T09:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.568+0000 7f0b54dc1740 1 -- 172.21.15.2:0/3174128231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5604cdfb8f30 con 0x5604cdbde4d0 2024-05-14T09:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.568+0000 7f0b26ffd640 1 -- 172.21.15.2:0/3174128231 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b40003dd0 con 0x5604cdbde4d0 2024-05-14T09:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.568+0000 7f0b26ffd640 1 -- 172.21.15.2:0/3174128231 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f0b3c102110 con 0x5604cdbde4d0 2024-05-14T09:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.568+0000 7f0b54dc1740 10 client.16296 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.568+0000 7f0b54dc1740 10 client.16296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.568+0000 7f0b54dc1740 20 client.16296 populate_metadata read hostname 'smithi002' 2024-05-14T09:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.568+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/3174128231 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5604cdbbdd80 0x5604cdfbc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.568+0000 7f0b54dc1740 4 client.16296 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.568+0000 7f0b54dc1740 1 -- 172.21.15.2:0/3174128231 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604cdf131e0 con 0x5604cdbbdd80 2024-05-14T09:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.568+0000 7f0b477fe640 1 --2- 172.21.15.2:0/3174128231 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5604cdbbdd80 0x5604cdfbc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.569+0000 7f0b26ffd640 10 client.16296 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.569+0000 7f0b477fe640 1 --2- 172.21.15.2:0/3174128231 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5604cdbbdd80 0x5604cdfbc620 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8edf1342f3593876 server_cookie=3632226ad21b6ed2 in_seq=0 out_seq=0 2024-05-14T09:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.569+0000 7f0b26ffd640 1 -- 172.21.15.2:0/3174128231 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5604cdf131e0 con 0x5604cdbbdd80 2024-05-14T09:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.569+0000 7f0b26ffd640 10 client.16296 handle_command_reply: tid=0 2024-05-14T09:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.570+0000 7f0b54dc1740 1 client.16296 shutdown 2024-05-14T09:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.570+0000 7f0b54dc1740 1 -- 172.21.15.2:0/3174128231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbde4d0 msgr2=0x5604cdfaf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.570+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/3174128231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbde4d0 0x5604cdfaf0a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0ffb20 tx=0x7f0b3c0c55c0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.570+0000 7f0b54dc1740 1 -- 172.21.15.2:0/3174128231 shutdown_connections 2024-05-14T09:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.570+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/3174128231 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5604cdbbdd80 0x5604cdfbc620 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.570+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/3174128231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbff1b0 0x5604cdfb6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.570+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/3174128231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbde4d0 0x5604cdfaf0a0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.570+0000 7f0b26ffd640 0 client.16296 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.570+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/3174128231 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbd62f0 0x5604cdfaeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.570+0000 7f0b54dc1740 1 -- 172.21.15.2:0/3174128231 >> 172.21.15.2:0/3174128231 conn(0x5604cdbf3300 msgr2=0x5604cdf9f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.570+0000 7f0b54dc1740 1 -- 172.21.15.2:0/3174128231 shutdown_connections 2024-05-14T09:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.570+0000 7f0b54dc1740 1 -- 172.21.15.2:0/3174128231 wait complete. 2024-05-14T09:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.570+0000 7f0b54dc1740 20 client.16296 trim_cache size 0 max 16384 2024-05-14T09:41:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.585+0000 7f0b54dc1740 1 Processor -- start 2024-05-14T09:41:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.585+0000 7f0b54dc1740 1 -- start start 2024-05-14T09:41:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b54dc1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbd62f0 0x5604cdf9bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b54dc1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbde4d0 0x5604cdfb5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b54dc1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbff1b0 0x5604cdf9c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b54dc1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0b481ad370 con 0x5604cdbd62f0 2024-05-14T09:41:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b54dc1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0b481aac10 con 0x5604cdbff1b0 2024-05-14T09:41:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b54dc1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0b481a8a60 con 0x5604cdbde4d0 2024-05-14T09:41:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b477fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbd62f0 0x5604cdf9bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b477fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbd62f0 0x5604cdf9bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37372/0 (socket says 172.21.15.2:37372) 2024-05-14T09:41:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b46ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbde4d0 0x5604cdfb5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b477fe640 1 -- 172.21.15.2:0/489087340 learned_addr learned my addr 172.21.15.2:0/489087340 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b47fff640 1 --2- 172.21.15.2:0/489087340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbff1b0 0x5604cdf9c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b477fe640 1 -- 172.21.15.2:0/489087340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbde4d0 msgr2=0x5604cdfb5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b477fe640 1 --2- 172.21.15.2:0/489087340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbde4d0 0x5604cdfb5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b477fe640 1 -- 172.21.15.2:0/489087340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbff1b0 msgr2=0x5604cdf9c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b477fe640 1 --2- 172.21.15.2:0/489087340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbff1b0 0x5604cdf9c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b477fe640 1 -- 172.21.15.2:0/489087340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604cdfa1c20 con 0x5604cdbd62f0 2024-05-14T09:41:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.586+0000 7f0b477fe640 1 --2- 172.21.15.2:0/489087340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbd62f0 0x5604cdf9bc00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0b48103750 tx=0x7f0b3801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6710a1a3cd8715b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.587+0000 7f0b2d7fa640 1 -- 172.21.15.2:0/489087340 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38032030 con 0x5604cdbd62f0 2024-05-14T09:41:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.587+0000 7f0b2d7fa640 1 -- 172.21.15.2:0/489087340 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b3801dd50 con 0x5604cdbd62f0 2024-05-14T09:41:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.587+0000 7f0b2d7fa640 1 -- 172.21.15.2:0/489087340 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3802b400 con 0x5604cdbd62f0 2024-05-14T09:41:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.588+0000 7f0b54dc1740 1 -- 172.21.15.2:0/489087340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbd62f0 msgr2=0x5604cdf9bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.588+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/489087340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbd62f0 0x5604cdf9bc00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0b48103750 tx=0x7f0b3801b000 comp rx=0 tx=0).stop 2024-05-14T09:41:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.588+0000 7f0b54dc1740 1 -- 172.21.15.2:0/489087340 shutdown_connections 2024-05-14T09:41:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.588+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/489087340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbff1b0 0x5604cdf9c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.588+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/489087340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbde4d0 0x5604cdfb5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.588+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/489087340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbd62f0 0x5604cdf9bc00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.588+0000 7f0b54dc1740 1 -- 172.21.15.2:0/489087340 >> 172.21.15.2:0/489087340 conn(0x5604cdbf3300 msgr2=0x5604cdfa9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.588+0000 7f0b54dc1740 1 -- 172.21.15.2:0/489087340 shutdown_connections 2024-05-14T09:41:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.588+0000 7f0b54dc1740 1 -- 172.21.15.2:0/489087340 wait complete. 2024-05-14T09:41:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.588+0000 7f0b54dc1740 1 -- start start 2024-05-14T09:41:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.588+0000 7f0b54dc1740 1 Processor -- start 2024-05-14T09:41:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b54dc1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbd62f0 0x5604cdfb3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b54dc1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbde4d0 0x5604cdfb3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b54dc1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbff1b0 0x5604cdfb4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b54dc1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0b481a90d0 con 0x5604cdbff1b0 2024-05-14T09:41:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b54dc1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0b481aa540 con 0x5604cdbd62f0 2024-05-14T09:41:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b54dc1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0b481a97a0 con 0x5604cdbde4d0 2024-05-14T09:41:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b54dc1740 10 client.0 _ll_register_callbacks cb 0x7f0b3c105c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:41:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b47fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbff1b0 0x5604cdfb4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbd62f0 0x5604cdfb3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b47fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbff1b0 0x5604cdfb4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37386/0 (socket says 172.21.15.2:37386) 2024-05-14T09:41:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b47fff640 1 -- 172.21.15.2:0/1693211193 learned_addr learned my addr 172.21.15.2:0/1693211193 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:41:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b46ffd640 1 --2- 172.21.15.2:0/1693211193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbde4d0 0x5604cdfb3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b477fe640 1 -- 172.21.15.2:0/1693211193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbde4d0 msgr2=0x5604cdfb3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b477fe640 1 --2- 172.21.15.2:0/1693211193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbde4d0 0x5604cdfb3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b477fe640 1 -- 172.21.15.2:0/1693211193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbff1b0 msgr2=0x5604cdfb4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:41:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b477fe640 1 --2- 172.21.15.2:0/1693211193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbff1b0 0x5604cdfb4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b477fe640 1 -- 172.21.15.2:0/1693211193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604cdfb5040 con 0x5604cdbd62f0 2024-05-14T09:41:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b47fff640 1 --2- 172.21.15.2:0/1693211193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbff1b0 0x5604cdfb4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:41:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b477fe640 1 --2- 172.21.15.2:0/1693211193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbd62f0 0x5604cdfb3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5604cdfa0d60 tx=0x7f0b3801d6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ba4d11b14a644ee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:41:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:41:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.589+0000 7f0b26ffd640 1 -- 172.21.15.2:0/1693211193 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3803b030 con 0x5604cdbd62f0 2024-05-14T09:41:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.590+0000 7f0b26ffd640 1 -- 172.21.15.2:0/1693211193 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f0b38005280 con 0x5604cdbd62f0 2024-05-14T09:41:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.590+0000 7f0b54dc1740 1 -- 172.21.15.2:0/1693211193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b3801dd50 con 0x5604cdbd62f0 2024-05-14T09:41:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.590+0000 7f0b26ffd640 1 -- 172.21.15.2:0/1693211193 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0b38048050 con 0x5604cdbd62f0 2024-05-14T09:41:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.590+0000 7f0b54dc1740 10 client.25988 fetch_fsmap learned FSMap version 34 2024-05-14T09:41:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.590+0000 7f0b54dc1740 1 -- 172.21.15.2:0/1693211193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5604cdfb62d0 con 0x5604cdbd62f0 2024-05-14T09:41:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.590+0000 7f0b26ffd640 1 -- 172.21.15.2:0/1693211193 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b3801dd50 con 0x5604cdbd62f0 2024-05-14T09:41:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.590+0000 7f0b26ffd640 1 -- 172.21.15.2:0/1693211193 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f0b3802d260 con 0x5604cdbd62f0 2024-05-14T09:41:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.590+0000 7f0b54dc1740 10 client.25988 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:41:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.590+0000 7f0b54dc1740 10 client.25988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:41:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.590+0000 7f0b54dc1740 20 client.25988 populate_metadata read hostname 'smithi002' 2024-05-14T09:41:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.590+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/1693211193 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5604cdb931c0 0x5604ce0271c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:41:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.590+0000 7f0b54dc1740 4 client.25988 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:41:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.590+0000 7f0b54dc1740 1 -- 172.21.15.2:0/1693211193 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604cdf0d050 con 0x5604cdb931c0 2024-05-14T09:41:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.590+0000 7f0b46ffd640 1 --2- 172.21.15.2:0/1693211193 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5604cdb931c0 0x5604ce0271c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:41:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.591+0000 7f0b26ffd640 10 client.25988 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.591+0000 7f0b46ffd640 1 --2- 172.21.15.2:0/1693211193 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5604cdb931c0 0x5604ce0271c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a18af0eb4b7497d2 server_cookie=e3c71c32baffb9e0 in_seq=0 out_seq=0 2024-05-14T09:41:52.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:52 smithi002 ceph-mon[31557]: pgmap v631: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 305 op/s 2024-05-14T09:41:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:52 smithi029 ceph-mon[36249]: pgmap v631: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 305 op/s 2024-05-14T09:41:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.696+0000 7f0b26ffd640 1 -- 172.21.15.2:0/1693211193 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5604cdf0d050 con 0x5604cdb931c0 2024-05-14T09:41:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.696+0000 7f0b26ffd640 10 client.25988 handle_command_reply: tid=0 2024-05-14T09:41:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.697+0000 7f0b54dc1740 1 client.25988 shutdown 2024-05-14T09:41:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.697+0000 7f0b54dc1740 1 -- 172.21.15.2:0/1693211193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbd62f0 msgr2=0x5604cdfb3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.697+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/1693211193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbd62f0 0x5604cdfb3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5604cdfa0d60 tx=0x7f0b3801d6e0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.697+0000 7f0b54dc1740 1 -- 172.21.15.2:0/1693211193 shutdown_connections 2024-05-14T09:41:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.697+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/1693211193 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5604cdb931c0 0x5604ce0271c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.697+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/1693211193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604cdbff1b0 0x5604cdfb4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.697+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/1693211193 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5604cdbde4d0 0x5604cdfb3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.697+0000 7f0b26ffd640 0 client.25988 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:41:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.697+0000 7f0b54dc1740 1 --2- 172.21.15.2:0/1693211193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604cdbd62f0 0x5604cdfb3680 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.697+0000 7f0b54dc1740 1 -- 172.21.15.2:0/1693211193 >> 172.21.15.2:0/1693211193 conn(0x5604cdbf3300 msgr2=0x5604cdfa9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.697+0000 7f0b54dc1740 1 -- 172.21.15.2:0/1693211193 shutdown_connections 2024-05-14T09:41:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.697+0000 7f0b54dc1740 1 -- 172.21.15.2:0/1693211193 wait complete. 2024-05-14T09:41:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.697+0000 7f0b54dc1740 20 client.25988 trim_cache size 0 max 16384 2024-05-14T09:41:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:41:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:41:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:41:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:41:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.698+0000 7f0b4df0d640 1 -- 172.21.15.2:0/957764675 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0b300b28a0 msgr2=0x7f0b300b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.698+0000 7f0b4df0d640 1 --2- 172.21.15.2:0/957764675 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0b300b28a0 0x7f0b300b4db0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5604cdfa1e00 tx=0x7f0b38009260 comp rx=0 tx=0).stop 2024-05-14T09:41:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.698+0000 7f0b4df0d640 1 -- 172.21.15.2:0/957764675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b48103d90 msgr2=0x7f0b481996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:41:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.698+0000 7f0b4df0d640 1 --2- 172.21.15.2:0/957764675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b48103d90 0x7f0b481996d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009980 tx=0x7f0b3c032dc0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.698+0000 7f0b4df0d640 1 -- 172.21.15.2:0/957764675 shutdown_connections 2024-05-14T09:41:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.698+0000 7f0b4df0d640 1 --2- 172.21.15.2:0/957764675 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0b300b28a0 0x7f0b300b4db0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.698+0000 7f0b4df0d640 1 --2- 172.21.15.2:0/957764675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b48108a90 0x7f0b481a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.698+0000 7f0b4df0d640 1 --2- 172.21.15.2:0/957764675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b48103d90 0x7f0b481996d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.698+0000 7f0b4df0d640 1 --2- 172.21.15.2:0/957764675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b48102b50 0x7f0b48199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:41:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.698+0000 7f0b4df0d640 1 -- 172.21.15.2:0/957764675 >> 172.21.15.2:0/957764675 conn(0x7f0b480fdd40 msgr2=0x7f0b48104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:41:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.698+0000 7f0b4df0d640 1 -- 172.21.15.2:0/957764675 shutdown_connections 2024-05-14T09:41:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:41:52.698+0000 7f0b4df0d640 1 -- 172.21.15.2:0/957764675 wait complete. 2024-05-14T09:41:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:52 smithi026 ceph-mon[36056]: pgmap v631: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 305 op/s 2024-05-14T09:41:53.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 613475 10.23 MB/sec execute 199 sec latency 82.808 ms 2024-05-14T09:41:54.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 614942 10.24 MB/sec execute 200 sec latency 30.409 ms 2024-05-14T09:41:54.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:54 smithi002 ceph-mon[31557]: pgmap v632: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 364 op/s 2024-05-14T09:41:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:54 smithi029 ceph-mon[36249]: pgmap v632: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 364 op/s 2024-05-14T09:41:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:54 smithi026 ceph-mon[36056]: pgmap v632: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 364 op/s 2024-05-14T09:41:55.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 617133 10.24 MB/sec execute 201 sec latency 11.917 ms 2024-05-14T09:41:56.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 618453 10.23 MB/sec execute 202 sec latency 152.517 ms 2024-05-14T09:41:56.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:56 smithi002 ceph-mon[31557]: pgmap v633: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-05-14T09:41:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:56 smithi029 ceph-mon[36249]: pgmap v633: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-05-14T09:41:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:56 smithi026 ceph-mon[36056]: pgmap v633: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-05-14T09:41:57.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 620748 10.23 MB/sec execute 203 sec latency 14.076 ms 2024-05-14T09:41:58.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 622206 10.22 MB/sec execute 204 sec latency 38.245 ms 2024-05-14T09:41:58.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:41:58 smithi002 ceph-mon[31557]: pgmap v634: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 326 op/s 2024-05-14T09:41:58.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:41:58 smithi029 ceph-mon[36249]: pgmap v634: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 326 op/s 2024-05-14T09:41:58.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:41:58 smithi026 ceph-mon[36056]: pgmap v634: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 326 op/s 2024-05-14T09:41:59.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 624457 10.23 MB/sec execute 205 sec latency 21.242 ms 2024-05-14T09:42:00.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 626045 10.22 MB/sec execute 206 sec latency 13.957 ms 2024-05-14T09:42:00.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:00 smithi002 ceph-mon[31557]: pgmap v635: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 337 op/s 2024-05-14T09:42:00.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:00 smithi029 ceph-mon[36249]: pgmap v635: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 337 op/s 2024-05-14T09:42:00.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:00 smithi026 ceph-mon[36056]: pgmap v635: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 337 op/s 2024-05-14T09:42:01.268 INFO:tasks.workunit.client.0.smithi002.stdout: 1 628821 10.26 MB/sec execute 207 sec latency 13.096 ms 2024-05-14T09:42:02.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 630070 10.23 MB/sec execute 208 sec latency 94.366 ms 2024-05-14T09:42:02.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:02 smithi002 ceph-mon[31557]: pgmap v636: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.4 MiB/s wr, 302 op/s 2024-05-14T09:42:02.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:02 smithi029 ceph-mon[36249]: pgmap v636: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.4 MiB/s wr, 302 op/s 2024-05-14T09:42:02.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:02 smithi026 ceph-mon[36056]: pgmap v636: 129 pgs: 129 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.4 MiB/s wr, 302 op/s 2024-05-14T09:42:03.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 632383 10.25 MB/sec execute 209 sec latency 77.398 ms 2024-05-14T09:42:04.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 633725 10.23 MB/sec execute 210 sec latency 42.733 ms 2024-05-14T09:42:04.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:04 smithi002 ceph-mon[31557]: pgmap v637: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 359 op/s 2024-05-14T09:42:04.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:04 smithi029 ceph-mon[36249]: pgmap v637: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 359 op/s 2024-05-14T09:42:04.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:04 smithi026 ceph-mon[36056]: pgmap v637: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 359 op/s 2024-05-14T09:42:05.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 636066 10.25 MB/sec execute 211 sec latency 14.957 ms 2024-05-14T09:42:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.807+0000 7f6734fce640 1 -- 172.21.15.2:0/2242336432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6730103c90 msgr2=0x7f6730104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.807+0000 7f6734fce640 1 --2- 172.21.15.2:0/2242336432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6730103c90 0x7f6730104990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6720009900 tx=0x7f672002a410 comp rx=0 tx=0).stop 2024-05-14T09:42:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.807+0000 7f6734fce640 1 -- 172.21.15.2:0/2242336432 shutdown_connections 2024-05-14T09:42:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.807+0000 7f6734fce640 1 --2- 172.21.15.2:0/2242336432 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6730109bd0 0x7f673010bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.807+0000 7f6734fce640 1 --2- 172.21.15.2:0/2242336432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6730104ed0 0x7f67301052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.807+0000 7f6734fce640 1 --2- 172.21.15.2:0/2242336432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6730103c90 0x7f6730104990 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.807+0000 7f6734fce640 1 -- 172.21.15.2:0/2242336432 >> 172.21.15.2:0/2242336432 conn(0x7f67300feea0 msgr2=0x7f6730101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.807+0000 7f6734fce640 1 -- 172.21.15.2:0/2242336432 shutdown_connections 2024-05-14T09:42:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.807+0000 7f6734fce640 1 -- 172.21.15.2:0/2242336432 wait complete. 2024-05-14T09:42:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.808+0000 7f6734fce640 1 Processor -- start 2024-05-14T09:42:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.808+0000 7f6734fce640 1 -- start start 2024-05-14T09:42:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.808+0000 7f6734fce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6730103c90 0x7f673019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.808+0000 7f6734fce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6730104ed0 0x7f673019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.808+0000 7f6734fce640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6730109bd0 0x7f67301a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.808+0000 7f6734fce640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f673010f770 con 0x7f6730109bd0 2024-05-14T09:42:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.808+0000 7f6734fce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f673010f5f0 con 0x7f6730103c90 2024-05-14T09:42:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.808+0000 7f6734fce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f673010f8f0 con 0x7f6730104ed0 2024-05-14T09:42:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.808+0000 7f672ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6730109bd0 0x7f67301a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.808+0000 7f672dd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6730104ed0 0x7f673019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.808+0000 7f672e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6730103c90 0x7f673019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.808+0000 7f672ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6730109bd0 0x7f67301a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46068/0 (socket says 172.21.15.2:46068) 2024-05-14T09:42:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.809+0000 7f672ed9d640 1 -- 172.21.15.2:0/1368783065 learned_addr learned my addr 172.21.15.2:0/1368783065 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.809+0000 7f672ed9d640 1 -- 172.21.15.2:0/1368783065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6730104ed0 msgr2=0x7f673019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.809+0000 7f672ed9d640 1 --2- 172.21.15.2:0/1368783065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6730104ed0 0x7f673019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.809+0000 7f672ed9d640 1 -- 172.21.15.2:0/1368783065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6730103c90 msgr2=0x7f673019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.809+0000 7f672ed9d640 1 --2- 172.21.15.2:0/1368783065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6730103c90 0x7f673019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.809+0000 7f672ed9d640 1 -- 172.21.15.2:0/1368783065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67301a1e90 con 0x7f6730109bd0 2024-05-14T09:42:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.809+0000 7f672ed9d640 1 --2- 172.21.15.2:0/1368783065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6730109bd0 0x7f67301a1820 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f672800eb10 tx=0x7f672800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.809+0000 7f671f7fe640 1 -- 172.21.15.2:0/1368783065 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728004b20 con 0x7f6730109bd0 2024-05-14T09:42:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.809+0000 7f671f7fe640 1 -- 172.21.15.2:0/1368783065 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6728004cc0 con 0x7f6730109bd0 2024-05-14T09:42:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.809+0000 7f6734fce640 1 -- 172.21.15.2:0/1368783065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67301a2180 con 0x7f6730109bd0 2024-05-14T09:42:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.809+0000 7f671f7fe640 1 -- 172.21.15.2:0/1368783065 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728010670 con 0x7f6730109bd0 2024-05-14T09:42:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.809+0000 7f6734fce640 1 -- 172.21.15.2:0/1368783065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67301a2680 con 0x7f6730109bd0 2024-05-14T09:42:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.812+0000 7f673bb22740 1 Processor -- start 2024-05-14T09:42:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.812+0000 7f671f7fe640 1 -- 172.21.15.2:0/1368783065 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6728010830 con 0x7f6730109bd0 2024-05-14T09:42:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.812+0000 7f673bb22740 1 -- start start 2024-05-14T09:42:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f673bb22740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fea1b0 0x56121638fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f673bb22740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 0x5612163903f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f673bb22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fc94d0 0x5612163972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f673bb22740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5612162f05d0 con 0x561215fea1b0 2024-05-14T09:42:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f673bb22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5612162a9890 con 0x561215fc94d0 2024-05-14T09:42:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f673bb22740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5612162ec590 con 0x561215fc12f0 2024-05-14T09:42:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f672dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fea1b0 0x56121638fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f672dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fea1b0 0x56121638fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46074/0 (socket says 172.21.15.2:46074) 2024-05-14T09:42:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f672dd9b640 1 -- 172.21.15.2:0/1018310104 learned_addr learned my addr 172.21.15.2:0/1018310104 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f672dd9b640 1 --2- 172.21.15.2:0/1018310104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fc94d0 0x5612163972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f672e59c640 1 --2- 172.21.15.2:0/1018310104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 0x5612163903f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f672dd9b640 1 -- 172.21.15.2:0/1018310104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 msgr2=0x5612163903f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f672dd9b640 1 --2- 172.21.15.2:0/1018310104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 0x5612163903f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f672dd9b640 1 -- 172.21.15.2:0/1018310104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fc94d0 msgr2=0x5612163972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f672dd9b640 1 --2- 172.21.15.2:0/1018310104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fc94d0 0x5612163972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f672dd9b640 1 -- 172.21.15.2:0/1018310104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561216002210 con 0x561215fea1b0 2024-05-14T09:42:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.813+0000 7f671f7fe640 1 --2- 172.21.15.2:0/1368783065 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67180b28a0 0x7f67180b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.814+0000 7f672dd9b640 1 --2- 172.21.15.2:0/1018310104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fea1b0 0x56121638fe70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6724013630 tx=0x7f67240097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c7d3f1ef3be51f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.814+0000 7f672e59c640 1 --2- 172.21.15.2:0/1368783065 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67180b28a0 0x7f67180b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.814+0000 7f671f7fe640 1 -- 172.21.15.2:0/1368783065 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f67280e2bf0 con 0x7f6730109bd0 2024-05-14T09:42:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.814+0000 7f671cff9640 1 -- 172.21.15.2:0/1018310104 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724004020 con 0x561215fea1b0 2024-05-14T09:42:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.814+0000 7f671cff9640 1 -- 172.21.15.2:0/1018310104 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67240041c0 con 0x561215fea1b0 2024-05-14T09:42:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.814+0000 7f671cff9640 1 -- 172.21.15.2:0/1018310104 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672400f7e0 con 0x561215fea1b0 2024-05-14T09:42:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.815+0000 7f673bb22740 1 -- 172.21.15.2:0/1018310104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fea1b0 msgr2=0x56121638fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.815+0000 7f672e59c640 1 --2- 172.21.15.2:0/1368783065 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67180b28a0 0x7f67180b4db0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6720005ab0 tx=0x7f6720031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.815+0000 7f673bb22740 1 --2- 172.21.15.2:0/1018310104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fea1b0 0x56121638fe70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6724013630 tx=0x7f67240097a0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.815+0000 7f673bb22740 1 -- 172.21.15.2:0/1018310104 shutdown_connections 2024-05-14T09:42:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.815+0000 7f673bb22740 1 --2- 172.21.15.2:0/1018310104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fc94d0 0x5612163972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.815+0000 7f673bb22740 1 --2- 172.21.15.2:0/1018310104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 0x5612163903f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.815+0000 7f673bb22740 1 --2- 172.21.15.2:0/1018310104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fea1b0 0x56121638fe70 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.815+0000 7f673bb22740 1 -- 172.21.15.2:0/1018310104 >> 172.21.15.2:0/1018310104 conn(0x561215fde300 msgr2=0x5612163873a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.815+0000 7f673bb22740 1 -- 172.21.15.2:0/1018310104 shutdown_connections 2024-05-14T09:42:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.815+0000 7f673bb22740 1 -- 172.21.15.2:0/1018310104 wait complete. 2024-05-14T09:42:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.815+0000 7f673bb22740 1 -- start start 2024-05-14T09:42:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.815+0000 7f673bb22740 1 Processor -- start 2024-05-14T09:42:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f673bb22740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 0x561216399b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f673bb22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fc94d0 0x56121639a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f673bb22740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fea1b0 0x5612163a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f673bb22740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561216397d40 con 0x561215fea1b0 2024-05-14T09:42:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f673bb22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5612162ad370 con 0x561215fc94d0 2024-05-14T09:42:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f673bb22740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561216397ec0 con 0x561215fc12f0 2024-05-14T09:42:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f673bb22740 10 client.0 _ll_register_callbacks cb 0x561216001a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:42:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f672ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fea1b0 0x5612163a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f672ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fea1b0 0x5612163a0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46084/0 (socket says 172.21.15.2:46084) 2024-05-14T09:42:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f672ed9d640 1 -- 172.21.15.2:0/1489807894 learned_addr learned my addr 172.21.15.2:0/1489807894 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f672dd9b640 1 --2- 172.21.15.2:0/1489807894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fc94d0 0x56121639a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f672e59c640 1 --2- 172.21.15.2:0/1489807894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 0x561216399b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f672ed9d640 1 -- 172.21.15.2:0/1489807894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 msgr2=0x561216399b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f672ed9d640 1 --2- 172.21.15.2:0/1489807894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 0x561216399b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f672ed9d640 1 -- 172.21.15.2:0/1489807894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fc94d0 msgr2=0x56121639a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f672ed9d640 1 --2- 172.21.15.2:0/1489807894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fc94d0 0x56121639a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f672ed9d640 1 -- 172.21.15.2:0/1489807894 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612160ce620 con 0x561215fea1b0 2024-05-14T09:42:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f672e59c640 1 --2- 172.21.15.2:0/1489807894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 0x561216399b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:42:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f672ed9d640 1 --2- 172.21.15.2:0/1489807894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fea1b0 0x5612163a0fb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f67280a5e70 tx=0x7f67280a9620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e34bd4d78568eb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.816+0000 7f6715ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:42:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.817+0000 7f6715ffb640 1 -- 172.21.15.2:0/1489807894 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67280a9df0 con 0x561215fea1b0 2024-05-14T09:42:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.817+0000 7f6715ffb640 1 -- 172.21.15.2:0/1489807894 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f67280b29d0 con 0x561215fea1b0 2024-05-14T09:42:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.817+0000 7f673bb22740 1 -- 172.21.15.2:0/1489807894 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67240041c0 con 0x561215fea1b0 2024-05-14T09:42:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.817+0000 7f6715ffb640 1 -- 172.21.15.2:0/1489807894 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f67280ae1a0 con 0x561215fea1b0 2024-05-14T09:42:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.817+0000 7f673bb22740 10 client.16314 fetch_fsmap learned FSMap version 34 2024-05-14T09:42:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.817+0000 7f673bb22740 1 -- 172.21.15.2:0/1489807894 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5612163a3fb0 con 0x561215fea1b0 2024-05-14T09:42:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.817+0000 7f6715ffb640 1 -- 172.21.15.2:0/1489807894 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67240041c0 con 0x561215fea1b0 2024-05-14T09:42:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.817+0000 7f6715ffb640 1 -- 172.21.15.2:0/1489807894 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f67280a5960 con 0x561215fea1b0 2024-05-14T09:42:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.817+0000 7f673bb22740 10 client.16314 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:42:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.817+0000 7f673bb22740 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:42:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.817+0000 7f673bb22740 20 client.16314 populate_metadata read hostname 'smithi002' 2024-05-14T09:42:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.817+0000 7f673bb22740 1 --2- 172.21.15.2:0/1489807894 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561215fa8d80 0x5612163a7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.817+0000 7f673bb22740 4 client.16314 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:42:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.817+0000 7f673bb22740 1 -- 172.21.15.2:0/1489807894 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612162fe1e0 con 0x561215fa8d80 2024-05-14T09:42:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.817+0000 7f672e59c640 1 --2- 172.21.15.2:0/1489807894 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561215fa8d80 0x5612163a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.818+0000 7f6715ffb640 10 client.16314 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.818+0000 7f672e59c640 1 --2- 172.21.15.2:0/1489807894 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561215fa8d80 0x5612163a7740 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f711b22b795857e server_cookie=5a9c3578208855 in_seq=0 out_seq=0 2024-05-14T09:42:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.819+0000 7f6715ffb640 1 -- 172.21.15.2:0/1489807894 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5612162fe1e0 con 0x561215fa8d80 2024-05-14T09:42:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.819+0000 7f6715ffb640 10 client.16314 handle_command_reply: tid=0 2024-05-14T09:42:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.819+0000 7f673bb22740 1 client.16314 shutdown 2024-05-14T09:42:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.819+0000 7f673bb22740 1 -- 172.21.15.2:0/1489807894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fea1b0 msgr2=0x5612163a0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.819+0000 7f673bb22740 1 --2- 172.21.15.2:0/1489807894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fea1b0 0x5612163a0fb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f67280a5e70 tx=0x7f67280a9620 comp rx=0 tx=0).stop 2024-05-14T09:42:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.819+0000 7f673bb22740 1 -- 172.21.15.2:0/1489807894 shutdown_connections 2024-05-14T09:42:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.819+0000 7f673bb22740 1 --2- 172.21.15.2:0/1489807894 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561215fa8d80 0x5612163a7740 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.819+0000 7f673bb22740 1 --2- 172.21.15.2:0/1489807894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fea1b0 0x5612163a0fb0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.819+0000 7f6715ffb640 0 client.16314 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.819+0000 7f673bb22740 1 --2- 172.21.15.2:0/1489807894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fc94d0 0x56121639a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.819+0000 7f673bb22740 1 --2- 172.21.15.2:0/1489807894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 0x561216399b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.819+0000 7f673bb22740 1 -- 172.21.15.2:0/1489807894 >> 172.21.15.2:0/1489807894 conn(0x561215fde300 msgr2=0x56121638a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.819+0000 7f673bb22740 1 -- 172.21.15.2:0/1489807894 shutdown_connections 2024-05-14T09:42:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.819+0000 7f673bb22740 1 -- 172.21.15.2:0/1489807894 wait complete. 2024-05-14T09:42:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.819+0000 7f673bb22740 20 client.16314 trim_cache size 0 max 16384 2024-05-14T09:42:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f673bb22740 1 Processor -- start 2024-05-14T09:42:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f673bb22740 1 -- start start 2024-05-14T09:42:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f673bb22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fc12f0 0x561216393f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f673bb22740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fc94d0 0x5612163a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f673bb22740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fea1b0 0x5612163944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f673bb22740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f67301ad370 con 0x561215fc94d0 2024-05-14T09:42:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f673bb22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f67301aac10 con 0x561215fc12f0 2024-05-14T09:42:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f673bb22740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f67301a8a60 con 0x561215fea1b0 2024-05-14T09:42:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f672dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fc94d0 0x5612163a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f672dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fc94d0 0x5612163a0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46090/0 (socket says 172.21.15.2:46090) 2024-05-14T09:42:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f672ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fea1b0 0x5612163944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f672dd9b640 1 -- 172.21.15.2:0/3194935855 learned_addr learned my addr 172.21.15.2:0/3194935855 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f672e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fc12f0 0x561216393f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f672dd9b640 1 -- 172.21.15.2:0/3194935855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fea1b0 msgr2=0x5612163944b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f672dd9b640 1 --2- 172.21.15.2:0/3194935855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fea1b0 0x5612163944b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f672dd9b640 1 -- 172.21.15.2:0/3194935855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fc12f0 msgr2=0x561216393f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f672dd9b640 1 --2- 172.21.15.2:0/3194935855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fc12f0 0x561216393f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f672dd9b640 1 -- 172.21.15.2:0/3194935855 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561216387d00 con 0x561215fc94d0 2024-05-14T09:42:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.836+0000 7f672ed9d640 1 --2- 172.21.15.2:0/3194935855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fea1b0 0x5612163944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.837+0000 7f672dd9b640 1 --2- 172.21.15.2:0/3194935855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fc94d0 0x5612163a0ed0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6724013f80 tx=0x7f67240133f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806129db54837988 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.837+0000 7f6717fff640 1 -- 172.21.15.2:0/3194935855 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724005270 con 0x561215fc94d0 2024-05-14T09:42:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.837+0000 7f6717fff640 1 -- 172.21.15.2:0/3194935855 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6724003aa0 con 0x561215fc94d0 2024-05-14T09:42:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.837+0000 7f6717fff640 1 -- 172.21.15.2:0/3194935855 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672400fbb0 con 0x561215fc94d0 2024-05-14T09:42:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.838+0000 7f673bb22740 1 -- 172.21.15.2:0/3194935855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fc94d0 msgr2=0x5612163a0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.838+0000 7f673bb22740 1 --2- 172.21.15.2:0/3194935855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fc94d0 0x5612163a0ed0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6724013f80 tx=0x7f67240133f0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.838+0000 7f673bb22740 1 -- 172.21.15.2:0/3194935855 shutdown_connections 2024-05-14T09:42:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.838+0000 7f673bb22740 1 --2- 172.21.15.2:0/3194935855 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fea1b0 0x5612163944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.838+0000 7f673bb22740 1 --2- 172.21.15.2:0/3194935855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fc94d0 0x5612163a0ed0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.838+0000 7f673bb22740 1 --2- 172.21.15.2:0/3194935855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fc12f0 0x561216393f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.838+0000 7f673bb22740 1 -- 172.21.15.2:0/3194935855 >> 172.21.15.2:0/3194935855 conn(0x561215fde300 msgr2=0x561216399b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.838+0000 7f673bb22740 1 -- 172.21.15.2:0/3194935855 shutdown_connections 2024-05-14T09:42:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.838+0000 7f673bb22740 1 -- 172.21.15.2:0/3194935855 wait complete. 2024-05-14T09:42:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.838+0000 7f673bb22740 1 -- start start 2024-05-14T09:42:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.838+0000 7f673bb22740 1 Processor -- start 2024-05-14T09:42:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f673bb22740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 0x56121639dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f673bb22740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fc94d0 0x56121639e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f673bb22740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fea1b0 0x56121640dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f673bb22740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f67301a90d0 con 0x561215fc94d0 2024-05-14T09:42:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f673bb22740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f67301aa540 con 0x561215fea1b0 2024-05-14T09:42:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f673bb22740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f67301a97a0 con 0x561215fc12f0 2024-05-14T09:42:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f673bb22740 10 client.0 _ll_register_callbacks cb 0x7f6728004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:42:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f672dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fc94d0 0x56121639e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f672dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fc94d0 0x56121639e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46098/0 (socket says 172.21.15.2:46098) 2024-05-14T09:42:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f672dd9b640 1 -- 172.21.15.2:0/2221114133 learned_addr learned my addr 172.21.15.2:0/2221114133 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f672ed9d640 1 --2- 172.21.15.2:0/2221114133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fea1b0 0x56121640dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f672e59c640 1 --2- 172.21.15.2:0/2221114133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 0x56121639dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f672dd9b640 1 -- 172.21.15.2:0/2221114133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 msgr2=0x56121639dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f672dd9b640 1 --2- 172.21.15.2:0/2221114133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 0x56121639dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f672dd9b640 1 -- 172.21.15.2:0/2221114133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fea1b0 msgr2=0x56121640dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f672dd9b640 1 --2- 172.21.15.2:0/2221114133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fea1b0 0x56121640dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f672dd9b640 1 -- 172.21.15.2:0/2221114133 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56121640e020 con 0x561215fc94d0 2024-05-14T09:42:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f672ed9d640 1 --2- 172.21.15.2:0/2221114133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fea1b0 0x56121640dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f672e59c640 1 --2- 172.21.15.2:0/2221114133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 0x56121639dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f672dd9b640 1 --2- 172.21.15.2:0/2221114133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fc94d0 0x56121639e2b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6724009250 tx=0x7f6724009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e34ff72fa17fa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f6715ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:42:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.839+0000 7f6715ffb640 1 -- 172.21.15.2:0/2221114133 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724005270 con 0x561215fc94d0 2024-05-14T09:42:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.840+0000 7f673bb22740 1 -- 172.21.15.2:0/2221114133 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6724003aa0 con 0x561215fc94d0 2024-05-14T09:42:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.840+0000 7f6715ffb640 1 -- 172.21.15.2:0/2221114133 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f6724005410 con 0x561215fc94d0 2024-05-14T09:42:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.840+0000 7f6715ffb640 1 -- 172.21.15.2:0/2221114133 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f672403b050 con 0x561215fc94d0 2024-05-14T09:42:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.840+0000 7f673bb22740 10 client.16320 fetch_fsmap learned FSMap version 34 2024-05-14T09:42:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.840+0000 7f673bb22740 1 -- 172.21.15.2:0/2221114133 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56121640f190 con 0x561215fc94d0 2024-05-14T09:42:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.840+0000 7f6715ffb640 1 -- 172.21.15.2:0/2221114133 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6724003aa0 con 0x561215fc94d0 2024-05-14T09:42:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.840+0000 7f6715ffb640 1 -- 172.21.15.2:0/2221114133 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f672401f8f0 con 0x561215fc94d0 2024-05-14T09:42:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.840+0000 7f673bb22740 10 client.16320 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:42:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.840+0000 7f673bb22740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:42:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.840+0000 7f673bb22740 20 client.16320 populate_metadata read hostname 'smithi002' 2024-05-14T09:42:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.840+0000 7f673bb22740 1 --2- 172.21.15.2:0/2221114133 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561215f7e1c0 0x561216411980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.840+0000 7f673bb22740 4 client.16320 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:42:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.840+0000 7f673bb22740 1 -- 172.21.15.2:0/2221114133 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612162f8050 con 0x561215f7e1c0 2024-05-14T09:42:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.840+0000 7f672e59c640 1 --2- 172.21.15.2:0/2221114133 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561215f7e1c0 0x561216411980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.841+0000 7f6715ffb640 10 client.16320 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.841+0000 7f672e59c640 1 --2- 172.21.15.2:0/2221114133 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561215f7e1c0 0x561216411980 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e8f2b8418a9cffd server_cookie=284552861d451957 in_seq=0 out_seq=0 2024-05-14T09:42:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.946+0000 7f6715ffb640 1 -- 172.21.15.2:0/2221114133 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612162f8050 con 0x561215f7e1c0 2024-05-14T09:42:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.946+0000 7f6715ffb640 10 client.16320 handle_command_reply: tid=0 2024-05-14T09:42:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.946+0000 7f673bb22740 1 client.16320 shutdown 2024-05-14T09:42:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.946+0000 7f673bb22740 1 -- 172.21.15.2:0/2221114133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fc94d0 msgr2=0x56121639e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.946+0000 7f673bb22740 1 --2- 172.21.15.2:0/2221114133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fc94d0 0x56121639e2b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6724009250 tx=0x7f6724009940 comp rx=0 tx=0).stop 2024-05-14T09:42:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.946+0000 7f673bb22740 1 -- 172.21.15.2:0/2221114133 shutdown_connections 2024-05-14T09:42:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.946+0000 7f673bb22740 1 --2- 172.21.15.2:0/2221114133 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561215f7e1c0 0x561216411980 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.946+0000 7f673bb22740 1 --2- 172.21.15.2:0/2221114133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561215fea1b0 0x56121640dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.946+0000 7f673bb22740 1 --2- 172.21.15.2:0/2221114133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561215fc94d0 0x56121639e2b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.946+0000 7f673bb22740 1 --2- 172.21.15.2:0/2221114133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561215fc12f0 0x56121639dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.946+0000 7f673bb22740 1 -- 172.21.15.2:0/2221114133 >> 172.21.15.2:0/2221114133 conn(0x561215fde300 msgr2=0x561216399ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.946+0000 7f6715ffb640 0 client.16320 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.946+0000 7f673bb22740 1 -- 172.21.15.2:0/2221114133 shutdown_connections 2024-05-14T09:42:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.946+0000 7f673bb22740 1 -- 172.21.15.2:0/2221114133 wait complete. 2024-05-14T09:42:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.946+0000 7f673bb22740 20 client.16320 trim_cache size 0 max 16384 2024-05-14T09:42:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:42:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:42:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:42:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:42:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.947+0000 7f6734fce640 1 -- 172.21.15.2:0/1368783065 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67180b28a0 msgr2=0x7f67180b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.947+0000 7f6734fce640 1 --2- 172.21.15.2:0/1368783065 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67180b28a0 0x7f67180b4db0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6720005ab0 tx=0x7f6720031400 comp rx=0 tx=0).stop 2024-05-14T09:42:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.947+0000 7f6734fce640 1 -- 172.21.15.2:0/1368783065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6730109bd0 msgr2=0x7f67301a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.947+0000 7f6734fce640 1 --2- 172.21.15.2:0/1368783065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6730109bd0 0x7f67301a1820 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f672800eb10 tx=0x7f672800c2d0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.947+0000 7f6734fce640 1 -- 172.21.15.2:0/1368783065 shutdown_connections 2024-05-14T09:42:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.947+0000 7f6734fce640 1 --2- 172.21.15.2:0/1368783065 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67180b28a0 0x7f67180b4db0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.947+0000 7f6734fce640 1 --2- 172.21.15.2:0/1368783065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6730109bd0 0x7f67301a1820 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.947+0000 7f6734fce640 1 --2- 172.21.15.2:0/1368783065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6730104ed0 0x7f673019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.947+0000 7f6734fce640 1 --2- 172.21.15.2:0/1368783065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6730103c90 0x7f673019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.947+0000 7f6734fce640 1 -- 172.21.15.2:0/1368783065 >> 172.21.15.2:0/1368783065 conn(0x7f67300feea0 msgr2=0x7f6730105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.947+0000 7f6734fce640 1 -- 172.21.15.2:0/1368783065 shutdown_connections 2024-05-14T09:42:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:05.947+0000 7f6734fce640 1 -- 172.21.15.2:0/1368783065 wait complete. 2024-05-14T09:42:06.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 637910 10.23 MB/sec execute 212 sec latency 60.120 ms 2024-05-14T09:42:06.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:06 smithi002 ceph-mon[31557]: pgmap v638: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.1 MiB/s wr, 283 op/s 2024-05-14T09:42:06.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:06 smithi029 ceph-mon[36249]: pgmap v638: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.1 MiB/s wr, 283 op/s 2024-05-14T09:42:06.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:06 smithi026 ceph-mon[36056]: pgmap v638: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.1 MiB/s wr, 283 op/s 2024-05-14T09:42:07.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 639681 10.24 MB/sec execute 213 sec latency 20.618 ms 2024-05-14T09:42:08.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 641686 10.23 MB/sec execute 214 sec latency 54.458 ms 2024-05-14T09:42:08.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:08 smithi002 ceph-mon[31557]: pgmap v639: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 344 op/s 2024-05-14T09:42:08.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:08 smithi029 ceph-mon[36249]: pgmap v639: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 344 op/s 2024-05-14T09:42:08.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:08 smithi026 ceph-mon[36056]: pgmap v639: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 344 op/s 2024-05-14T09:42:09.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 643202 10.23 MB/sec execute 215 sec latency 34.483 ms 2024-05-14T09:42:10.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 645357 10.22 MB/sec execute 216 sec latency 15.228 ms 2024-05-14T09:42:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:10 smithi002 ceph-mon[31557]: pgmap v640: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 321 op/s 2024-05-14T09:42:10.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:10 smithi029 ceph-mon[36249]: pgmap v640: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 321 op/s 2024-05-14T09:42:10.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:10 smithi026 ceph-mon[36056]: pgmap v640: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 321 op/s 2024-05-14T09:42:11.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 646930 10.23 MB/sec execute 217 sec latency 11.539 ms 2024-05-14T09:42:12.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 649372 10.24 MB/sec execute 218 sec latency 13.461 ms 2024-05-14T09:42:12.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:12 smithi002 ceph-mon[31557]: pgmap v641: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 300 op/s 2024-05-14T09:42:12.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:12 smithi029 ceph-mon[36249]: pgmap v641: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 300 op/s 2024-05-14T09:42:12.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:12 smithi026 ceph-mon[36056]: pgmap v641: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 300 op/s 2024-05-14T09:42:13.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 650758 10.22 MB/sec execute 219 sec latency 43.778 ms 2024-05-14T09:42:14.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 652913 10.23 MB/sec execute 220 sec latency 41.321 ms 2024-05-14T09:42:14.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:14 smithi002 ceph-mon[31557]: pgmap v642: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-05-14T09:42:14.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:14 smithi029 ceph-mon[36249]: pgmap v642: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-05-14T09:42:14.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:14 smithi026 ceph-mon[36056]: pgmap v642: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-05-14T09:42:15.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 654353 10.22 MB/sec execute 221 sec latency 17.015 ms 2024-05-14T09:42:16.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 656809 10.23 MB/sec execute 222 sec latency 17.371 ms 2024-05-14T09:42:16.673 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:16 smithi002 ceph-mon[31557]: pgmap v643: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 2.4 MiB/s wr, 263 op/s 2024-05-14T09:42:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:16 smithi029 ceph-mon[36249]: pgmap v643: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 2.4 MiB/s wr, 263 op/s 2024-05-14T09:42:16.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:16 smithi026 ceph-mon[36056]: pgmap v643: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 2.4 MiB/s wr, 263 op/s 2024-05-14T09:42:17.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 658107 10.22 MB/sec execute 223 sec latency 19.714 ms 2024-05-14T09:42:18.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 660600 10.24 MB/sec execute 224 sec latency 14.351 ms 2024-05-14T09:42:18.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:18 smithi002 ceph-mon[31557]: pgmap v644: 129 pgs: 129 active+clean; 122 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2024-05-14T09:42:18.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:18 smithi029 ceph-mon[36249]: pgmap v644: 129 pgs: 129 active+clean; 122 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2024-05-14T09:42:18.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:18 smithi026 ceph-mon[36056]: pgmap v644: 129 pgs: 129 active+clean; 122 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2024-05-14T09:42:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 -- 172.21.15.2:0/1877558634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a30106d50 msgr2=0x7f3a30109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 --2- 172.21.15.2:0/1877558634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a30106d50 0x7f3a30109230 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009900 tx=0x7f3a2002a410 comp rx=0 tx=0).stop 2024-05-14T09:42:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 -- 172.21.15.2:0/1877558634 shutdown_connections 2024-05-14T09:42:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 --2- 172.21.15.2:0/1877558634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a301098c0 0x7f3a3010bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 --2- 172.21.15.2:0/1877558634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a30106d50 0x7f3a30109230 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 --2- 172.21.15.2:0/1877558634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a30104360 0x7f3a30106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 -- 172.21.15.2:0/1877558634 >> 172.21.15.2:0/1877558634 conn(0x7f3a300fdd20 msgr2=0x7f3a301001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 -- 172.21.15.2:0/1877558634 shutdown_connections 2024-05-14T09:42:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 -- 172.21.15.2:0/1877558634 wait complete. 2024-05-14T09:42:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 Processor -- start 2024-05-14T09:42:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 -- start start 2024-05-14T09:42:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a30104360 0x7f3a3019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a30106d50 0x7f3a3019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a301098c0 0x7f3a301a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3010f710 con 0x7f3a30106d50 2024-05-14T09:42:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3010f590 con 0x7f3a30104360 2024-05-14T09:42:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3746e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3010f890 con 0x7f3a301098c0 2024-05-14T09:42:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a34a09640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a30106d50 0x7f3a3019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a34a09640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a30106d50 0x7f3a3019a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49784/0 (socket says 172.21.15.2:49784) 2024-05-14T09:42:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a34a09640 1 -- 172.21.15.2:0/2303016921 learned_addr learned my addr 172.21.15.2:0/2303016921 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a35a0b640 1 --2- 172.21.15.2:0/2303016921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a301098c0 0x7f3a301a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a3520a640 1 --2- 172.21.15.2:0/2303016921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a30104360 0x7f3a3019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a34a09640 1 -- 172.21.15.2:0/2303016921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a301098c0 msgr2=0x7f3a301a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a34a09640 1 --2- 172.21.15.2:0/2303016921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a301098c0 0x7f3a301a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a34a09640 1 -- 172.21.15.2:0/2303016921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a30104360 msgr2=0x7f3a3019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a34a09640 1 --2- 172.21.15.2:0/2303016921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a30104360 0x7f3a3019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a34a09640 1 -- 172.21.15.2:0/2303016921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a301a1df0 con 0x7f3a30106d50 2024-05-14T09:42:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.051+0000 7f3a35a0b640 1 --2- 172.21.15.2:0/2303016921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a301098c0 0x7f3a301a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.052+0000 7f3a34a09640 1 --2- 172.21.15.2:0/2303016921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a30106d50 0x7f3a3019a7f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3a20006010 tx=0x7f3a200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.052+0000 7f3a267fc640 1 -- 172.21.15.2:0/2303016921 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2003f030 con 0x7f3a30106d50 2024-05-14T09:42:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.052+0000 7f3a267fc640 1 -- 172.21.15.2:0/2303016921 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a20031910 con 0x7f3a30106d50 2024-05-14T09:42:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.052+0000 7f3a3746e640 1 -- 172.21.15.2:0/2303016921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a301a2080 con 0x7f3a30106d50 2024-05-14T09:42:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.052+0000 7f3a3746e640 1 -- 172.21.15.2:0/2303016921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a301a2570 con 0x7f3a30106d50 2024-05-14T09:42:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.052+0000 7f3a267fc640 1 -- 172.21.15.2:0/2303016921 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a20035070 con 0x7f3a30106d50 2024-05-14T09:42:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.054+0000 7f3a3e3ff740 1 Processor -- start 2024-05-14T09:42:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a3e3ff740 1 -- start start 2024-05-14T09:42:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a267fc640 1 -- 172.21.15.2:0/2303016921 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3a20042030 con 0x7f3a30106d50 2024-05-14T09:42:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a3e3ff740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d4a1b0 0x5610470eba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a3e3ff740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470ebfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a3e3ff740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 0x5610470f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a3e3ff740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5610470505d0 con 0x561046d212f0 2024-05-14T09:42:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a3e3ff740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561047009890 con 0x561046d294d0 2024-05-14T09:42:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a3e3ff740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56104704c590 con 0x561046d4a1b0 2024-05-14T09:42:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a3520a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470ebfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a3520a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470ebfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49790/0 (socket says 172.21.15.2:49790) 2024-05-14T09:42:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a3520a640 1 -- 172.21.15.2:0/3709101992 learned_addr learned my addr 172.21.15.2:0/3709101992 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a34a09640 1 --2- 172.21.15.2:0/3709101992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 0x5610470f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a35a0b640 1 --2- 172.21.15.2:0/3709101992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d4a1b0 0x5610470eba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a3520a640 1 -- 172.21.15.2:0/3709101992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d4a1b0 msgr2=0x5610470eba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a3520a640 1 --2- 172.21.15.2:0/3709101992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d4a1b0 0x5610470eba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a3520a640 1 -- 172.21.15.2:0/3709101992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 msgr2=0x5610470f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a3520a640 1 --2- 172.21.15.2:0/3709101992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 0x5610470f2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a3520a640 1 -- 172.21.15.2:0/3709101992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561046d62210 con 0x561046d212f0 2024-05-14T09:42:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.055+0000 7f3a34a09640 1 --2- 172.21.15.2:0/3709101992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 0x5610470f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.056+0000 7f3a267fc640 1 --2- 172.21.15.2:0/2303016921 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3a1c0b28a0 0x7f3a1c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.056+0000 7f3a3520a640 1 --2- 172.21.15.2:0/3709101992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470ebfe0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c009c80 tx=0x7f3a2c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a11c8b3e2e0906 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.056+0000 7f3a35a0b640 1 --2- 172.21.15.2:0/2303016921 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3a1c0b28a0 0x7f3a1c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.056+0000 7f3a1b7fe640 1 -- 172.21.15.2:0/3709101992 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c01e070 con 0x561046d212f0 2024-05-14T09:42:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.056+0000 7f3a267fc640 1 -- 172.21.15.2:0/2303016921 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3a20102bc0 con 0x7f3a30106d50 2024-05-14T09:42:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.056+0000 7f3a1b7fe640 1 -- 172.21.15.2:0/3709101992 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a2c009e30 con 0x561046d212f0 2024-05-14T09:42:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.056+0000 7f3a1b7fe640 1 -- 172.21.15.2:0/3709101992 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c00ea10 con 0x561046d212f0 2024-05-14T09:42:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.057+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/3709101992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 msgr2=0x5610470ebfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.057+0000 7f3a35a0b640 1 --2- 172.21.15.2:0/2303016921 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3a1c0b28a0 0x7f3a1c0b4db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x5610470ecc30 tx=0x7f3a28009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.057+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/3709101992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470ebfe0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c009c80 tx=0x7f3a2c011810 comp rx=0 tx=0).stop 2024-05-14T09:42:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.057+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/3709101992 shutdown_connections 2024-05-14T09:42:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.057+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/3709101992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 0x5610470f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.057+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/3709101992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470ebfe0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.057+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/3709101992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d4a1b0 0x5610470eba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.057+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/3709101992 >> 172.21.15.2:0/3709101992 conn(0x561046d3e300 msgr2=0x5610470e73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.057+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/3709101992 shutdown_connections 2024-05-14T09:42:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.057+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/3709101992 wait complete. 2024-05-14T09:42:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.057+0000 7f3a3e3ff740 1 -- start start 2024-05-14T09:42:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.057+0000 7f3a3e3ff740 1 Processor -- start 2024-05-14T09:42:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3e3ff740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3e3ff740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d294d0 0x5610470f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3e3ff740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d4a1b0 0x5610470fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3e3ff740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5610470f3930 con 0x561046d212f0 2024-05-14T09:42:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3e3ff740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56104700d370 con 0x561046d4a1b0 2024-05-14T09:42:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3e3ff740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5610470f3ab0 con 0x561046d294d0 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3e3ff740 10 client.0 _ll_register_callbacks cb 0x561046d61a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3520a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3520a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470f56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49806/0 (socket says 172.21.15.2:49806) 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3520a640 1 -- 172.21.15.2:0/923766511 learned_addr learned my addr 172.21.15.2:0/923766511 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a35a0b640 1 --2- 172.21.15.2:0/923766511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d4a1b0 0x5610470fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a34a09640 1 --2- 172.21.15.2:0/923766511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d294d0 0x5610470f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3520a640 1 -- 172.21.15.2:0/923766511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d294d0 msgr2=0x5610470f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3520a640 1 --2- 172.21.15.2:0/923766511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d294d0 0x5610470f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3520a640 1 -- 172.21.15.2:0/923766511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d4a1b0 msgr2=0x5610470fcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3520a640 1 --2- 172.21.15.2:0/923766511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d4a1b0 0x5610470fcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3520a640 1 -- 172.21.15.2:0/923766511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561046e2e620 con 0x561046d212f0 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a34a09640 1 --2- 172.21.15.2:0/923766511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d294d0 0x5610470f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a3520a640 1 --2- 172.21.15.2:0/923766511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470f56a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0058c0 tx=0x7f3a2c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d473f1a0be33d42 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.058+0000 7f3a18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.059+0000 7f3a18ff9640 1 -- 172.21.15.2:0/923766511 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c01e050 con 0x561046d212f0 2024-05-14T09:42:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.059+0000 7f3a18ff9640 1 -- 172.21.15.2:0/923766511 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3a2c004230 con 0x561046d212f0 2024-05-14T09:42:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.059+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/923766511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a2c009e30 con 0x561046d212f0 2024-05-14T09:42:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.059+0000 7f3a18ff9640 1 -- 172.21.15.2:0/923766511 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3a2c032050 con 0x561046d212f0 2024-05-14T09:42:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.059+0000 7f3a3e3ff740 10 client.16332 fetch_fsmap learned FSMap version 34 2024-05-14T09:42:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.059+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/923766511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5610470ffa70 con 0x561046d212f0 2024-05-14T09:42:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.059+0000 7f3a18ff9640 1 -- 172.21.15.2:0/923766511 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a2c009e30 con 0x561046d212f0 2024-05-14T09:42:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.059+0000 7f3a18ff9640 1 -- 172.21.15.2:0/923766511 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f3a2c02d030 con 0x561046d212f0 2024-05-14T09:42:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.059+0000 7f3a3e3ff740 10 client.16332 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:42:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.059+0000 7f3a3e3ff740 10 client.16332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:42:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.059+0000 7f3a3e3ff740 20 client.16332 populate_metadata read hostname 'smithi002' 2024-05-14T09:42:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.059+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/923766511 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561046d08d80 0x561047103080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.059+0000 7f3a3e3ff740 4 client.16332 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:42:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.059+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/923766511 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56104705e1e0 con 0x561046d08d80 2024-05-14T09:42:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.059+0000 7f3a34a09640 1 --2- 172.21.15.2:0/923766511 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561046d08d80 0x561047103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.060+0000 7f3a18ff9640 10 client.16332 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.060+0000 7f3a34a09640 1 --2- 172.21.15.2:0/923766511 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561046d08d80 0x561047103080 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f0a7c5016a6eb54 server_cookie=73246fce6dc0bb8b in_seq=0 out_seq=0 2024-05-14T09:42:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.062+0000 7f3a18ff9640 1 -- 172.21.15.2:0/923766511 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56104705e1e0 con 0x561046d08d80 2024-05-14T09:42:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.062+0000 7f3a18ff9640 10 client.16332 handle_command_reply: tid=0 2024-05-14T09:42:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.062+0000 7f3a3e3ff740 1 client.16332 shutdown 2024-05-14T09:42:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.062+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/923766511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 msgr2=0x5610470f56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.062+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/923766511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470f56a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0058c0 tx=0x7f3a2c005720 comp rx=0 tx=0).stop 2024-05-14T09:42:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.062+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/923766511 shutdown_connections 2024-05-14T09:42:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.062+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/923766511 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561046d08d80 0x561047103080 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.062+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/923766511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d4a1b0 0x5610470fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.062+0000 7f3a18ff9640 0 client.16332 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.062+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/923766511 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d294d0 0x5610470f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.062+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/923766511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470f56a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.062+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/923766511 >> 172.21.15.2:0/923766511 conn(0x561046d3e300 msgr2=0x5610470ea300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.062+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/923766511 shutdown_connections 2024-05-14T09:42:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.062+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/923766511 wait complete. 2024-05-14T09:42:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.062+0000 7f3a3e3ff740 20 client.16332 trim_cache size 0 max 16384 2024-05-14T09:42:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.077+0000 7f3a3e3ff740 1 Processor -- start 2024-05-14T09:42:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.077+0000 7f3a3e3ff740 1 -- start start 2024-05-14T09:42:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.077+0000 7f3a3e3ff740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.077+0000 7f3a3e3ff740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 0x5610470f5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a3e3ff740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d4a1b0 0x5610470fca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a3e3ff740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3a301af580 con 0x561046d212f0 2024-05-14T09:42:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a3e3ff740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a301ace20 con 0x561046d294d0 2024-05-14T09:42:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a3e3ff740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a301aac70 con 0x561046d4a1b0 2024-05-14T09:42:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a3520a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a3520a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470f00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49816/0 (socket says 172.21.15.2:49816) 2024-05-14T09:42:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a3520a640 1 -- 172.21.15.2:0/1381915979 learned_addr learned my addr 172.21.15.2:0/1381915979 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a34a09640 1 --2- 172.21.15.2:0/1381915979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 0x5610470f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a35a0b640 1 --2- 172.21.15.2:0/1381915979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d4a1b0 0x5610470fca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a3520a640 1 -- 172.21.15.2:0/1381915979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d4a1b0 msgr2=0x5610470fca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a3520a640 1 --2- 172.21.15.2:0/1381915979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d4a1b0 0x5610470fca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a3520a640 1 -- 172.21.15.2:0/1381915979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 msgr2=0x5610470f5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a3520a640 1 --2- 172.21.15.2:0/1381915979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 0x5610470f5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a3520a640 1 -- 172.21.15.2:0/1381915979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610470f6710 con 0x561046d212f0 2024-05-14T09:42:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a3520a640 1 --2- 172.21.15.2:0/1381915979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470f00d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c000c00 tx=0x7f3a2c0059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cc732d440aee28 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a1affd640 1 -- 172.21.15.2:0/1381915979 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c01e070 con 0x561046d212f0 2024-05-14T09:42:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a1affd640 1 -- 172.21.15.2:0/1381915979 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a2c01f040 con 0x561046d212f0 2024-05-14T09:42:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.078+0000 7f3a1affd640 1 -- 172.21.15.2:0/1381915979 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c003a30 con 0x561046d212f0 2024-05-14T09:42:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.079+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/1381915979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 msgr2=0x5610470f00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.079+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/1381915979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470f00d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c000c00 tx=0x7f3a2c0059b0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.079+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/1381915979 shutdown_connections 2024-05-14T09:42:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.079+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/1381915979 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d4a1b0 0x5610470fca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.079+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/1381915979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 0x5610470f5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.079+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/1381915979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d212f0 0x5610470f00d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.079+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/1381915979 >> 172.21.15.2:0/1381915979 conn(0x561046d3e300 msgr2=0x5610470f5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.079+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/1381915979 shutdown_connections 2024-05-14T09:42:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.079+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/1381915979 wait complete. 2024-05-14T09:42:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.080+0000 7f3a3e3ff740 1 -- start start 2024-05-14T09:42:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.080+0000 7f3a3e3ff740 1 Processor -- start 2024-05-14T09:42:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.080+0000 7f3a3e3ff740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d212f0 0x5610470fa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.080+0000 7f3a3e3ff740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 0x5610470fa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.080+0000 7f3a3e3ff740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d4a1b0 0x5610471693e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.080+0000 7f3a3e3ff740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3a301ab2e0 con 0x561046d4a1b0 2024-05-14T09:42:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.080+0000 7f3a3e3ff740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a301ac750 con 0x561046d294d0 2024-05-14T09:42:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.080+0000 7f3a3e3ff740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3a301ab9b0 con 0x561046d212f0 2024-05-14T09:42:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.080+0000 7f3a3e3ff740 10 client.0 _ll_register_callbacks cb 0x7f3a2c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:42:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.080+0000 7f3a35a0b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d4a1b0 0x5610471693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.080+0000 7f3a35a0b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d4a1b0 0x5610471693e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49828/0 (socket says 172.21.15.2:49828) 2024-05-14T09:42:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.080+0000 7f3a3520a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d212f0 0x5610470fa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.080+0000 7f3a35a0b640 1 -- 172.21.15.2:0/2774428200 learned_addr learned my addr 172.21.15.2:0/2774428200 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.080+0000 7f3a34a09640 1 --2- 172.21.15.2:0/2774428200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 0x5610470fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.081+0000 7f3a35a0b640 1 -- 172.21.15.2:0/2774428200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d212f0 msgr2=0x5610470fa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.081+0000 7f3a35a0b640 1 --2- 172.21.15.2:0/2774428200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d212f0 0x5610470fa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.081+0000 7f3a35a0b640 1 -- 172.21.15.2:0/2774428200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 msgr2=0x5610470fa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.081+0000 7f3a35a0b640 1 --2- 172.21.15.2:0/2774428200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 0x5610470fa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.081+0000 7f3a35a0b640 1 -- 172.21.15.2:0/2774428200 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610470fd530 con 0x561046d4a1b0 2024-05-14T09:42:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.081+0000 7f3a3520a640 1 --2- 172.21.15.2:0/2774428200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d212f0 0x5610470fa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.081+0000 7f3a35a0b640 1 --2- 172.21.15.2:0/2774428200 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d4a1b0 0x5610471693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800fef0 tx=0x7f3a28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d243ce357a226422 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.081+0000 7f3a18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:42:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.081+0000 7f3a18ff9640 1 -- 172.21.15.2:0/2774428200 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2802e060 con 0x561046d4a1b0 2024-05-14T09:42:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.081+0000 7f3a18ff9640 1 -- 172.21.15.2:0/2774428200 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3a28011b50 con 0x561046d4a1b0 2024-05-14T09:42:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.081+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/2774428200 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a2c01f040 con 0x561046d4a1b0 2024-05-14T09:42:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.081+0000 7f3a18ff9640 1 -- 172.21.15.2:0/2774428200 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3a28035050 con 0x561046d4a1b0 2024-05-14T09:42:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.081+0000 7f3a3e3ff740 10 client.16338 fetch_fsmap learned FSMap version 34 2024-05-14T09:42:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.081+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/2774428200 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56104716a960 con 0x561046d4a1b0 2024-05-14T09:42:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.081+0000 7f3a18ff9640 1 -- 172.21.15.2:0/2774428200 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a2c01f040 con 0x561046d4a1b0 2024-05-14T09:42:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.082+0000 7f3a18ff9640 1 -- 172.21.15.2:0/2774428200 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f3a2801f420 con 0x561046d4a1b0 2024-05-14T09:42:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.082+0000 7f3a3e3ff740 10 client.16338 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:42:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.082+0000 7f3a3e3ff740 10 client.16338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:42:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.082+0000 7f3a3e3ff740 20 client.16338 populate_metadata read hostname 'smithi002' 2024-05-14T09:42:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.082+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/2774428200 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561046cde1c0 0x56104716d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.082+0000 7f3a3e3ff740 4 client.16338 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:42:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.082+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/2774428200 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561047058050 con 0x561046cde1c0 2024-05-14T09:42:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.082+0000 7f3a3520a640 1 --2- 172.21.15.2:0/2774428200 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561046cde1c0 0x56104716d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.082+0000 7f3a18ff9640 10 client.16338 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.082+0000 7f3a3520a640 1 --2- 172.21.15.2:0/2774428200 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561046cde1c0 0x56104716d410 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38d68ab236fe2cb3 server_cookie=7743fb7d4359ad5c in_seq=0 out_seq=0 2024-05-14T09:42:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.179+0000 7f3a18ff9640 1 -- 172.21.15.2:0/2774428200 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561047058050 con 0x561046cde1c0 2024-05-14T09:42:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.179+0000 7f3a18ff9640 10 client.16338 handle_command_reply: tid=0 2024-05-14T09:42:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.179+0000 7f3a3e3ff740 1 client.16338 shutdown 2024-05-14T09:42:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.179+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/2774428200 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d4a1b0 msgr2=0x5610471693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.179+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/2774428200 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d4a1b0 0x5610471693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800fef0 tx=0x7f3a28004a40 comp rx=0 tx=0).stop 2024-05-14T09:42:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.179+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/2774428200 shutdown_connections 2024-05-14T09:42:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.179+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/2774428200 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561046cde1c0 0x56104716d410 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.179+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/2774428200 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561046d4a1b0 0x5610471693e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.179+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/2774428200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561046d294d0 0x5610470fa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.179+0000 7f3a18ff9640 0 client.16338 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.179+0000 7f3a3e3ff740 1 --2- 172.21.15.2:0/2774428200 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561046d212f0 0x5610470fa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.179+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/2774428200 >> 172.21.15.2:0/2774428200 conn(0x561046d3e300 msgr2=0x5610470ea980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.179+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/2774428200 shutdown_connections 2024-05-14T09:42:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.179+0000 7f3a3e3ff740 1 -- 172.21.15.2:0/2774428200 wait complete. 2024-05-14T09:42:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.179+0000 7f3a3e3ff740 20 client.16338 trim_cache size 0 max 16384 2024-05-14T09:42:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:42:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:42:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:42:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:42:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.180+0000 7f3a3746e640 1 -- 172.21.15.2:0/2303016921 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3a1c0b28a0 msgr2=0x7f3a1c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.180+0000 7f3a3746e640 1 --2- 172.21.15.2:0/2303016921 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3a1c0b28a0 0x7f3a1c0b4db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x5610470ecc30 tx=0x7f3a28009500 comp rx=0 tx=0).stop 2024-05-14T09:42:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.180+0000 7f3a3746e640 1 -- 172.21.15.2:0/2303016921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a30106d50 msgr2=0x7f3a3019a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.180+0000 7f3a3746e640 1 --2- 172.21.15.2:0/2303016921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a30106d50 0x7f3a3019a7f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3a20006010 tx=0x7f3a200042f0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.180+0000 7f3a3746e640 1 -- 172.21.15.2:0/2303016921 shutdown_connections 2024-05-14T09:42:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.180+0000 7f3a3746e640 1 --2- 172.21.15.2:0/2303016921 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3a1c0b28a0 0x7f3a1c0b4db0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.180+0000 7f3a3746e640 1 --2- 172.21.15.2:0/2303016921 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a301098c0 0x7f3a301a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.180+0000 7f3a3746e640 1 --2- 172.21.15.2:0/2303016921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a30106d50 0x7f3a3019a7f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.180+0000 7f3a3746e640 1 --2- 172.21.15.2:0/2303016921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a30104360 0x7f3a3019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.180+0000 7f3a3746e640 1 -- 172.21.15.2:0/2303016921 >> 172.21.15.2:0/2303016921 conn(0x7f3a300fdd20 msgr2=0x7f3a30104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.180+0000 7f3a3746e640 1 -- 172.21.15.2:0/2303016921 shutdown_connections 2024-05-14T09:42:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:19.180+0000 7f3a3746e640 1 -- 172.21.15.2:0/2303016921 wait complete. 2024-05-14T09:42:19.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 661320 10.21 MB/sec execute 225 sec latency 52.505 ms 2024-05-14T09:42:20.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 663803 10.22 MB/sec execute 226 sec latency 17.373 ms 2024-05-14T09:42:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:20 smithi002 ceph-mon[31557]: pgmap v645: 129 pgs: 129 active+clean; 123 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 318 op/s 2024-05-14T09:42:20.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:20 smithi029 ceph-mon[36249]: pgmap v645: 129 pgs: 129 active+clean; 123 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 318 op/s 2024-05-14T09:42:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:20 smithi026 ceph-mon[36056]: pgmap v645: 129 pgs: 129 active+clean; 123 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 318 op/s 2024-05-14T09:42:21.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 664864 10.20 MB/sec execute 227 sec latency 79.644 ms 2024-05-14T09:42:22.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 667278 10.21 MB/sec execute 228 sec latency 16.317 ms 2024-05-14T09:42:22.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:22 smithi002 ceph-mon[31557]: pgmap v646: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 289 op/s 2024-05-14T09:42:22.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:22 smithi029 ceph-mon[36249]: pgmap v646: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 289 op/s 2024-05-14T09:42:22.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:22 smithi026 ceph-mon[36056]: pgmap v646: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 289 op/s 2024-05-14T09:42:23.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 668449 10.19 MB/sec execute 229 sec latency 14.932 ms 2024-05-14T09:42:24.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 670603 10.20 MB/sec execute 230 sec latency 49.663 ms 2024-05-14T09:42:24.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:24 smithi002 ceph-mon[31557]: pgmap v647: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 356 op/s 2024-05-14T09:42:24.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:24 smithi029 ceph-mon[36249]: pgmap v647: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 356 op/s 2024-05-14T09:42:24.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:24 smithi026 ceph-mon[36056]: pgmap v647: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 356 op/s 2024-05-14T09:42:25.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 672030 10.19 MB/sec execute 231 sec latency 26.486 ms 2024-05-14T09:42:26.269 INFO:tasks.workunit.client.0.smithi002.stdout: 1 674392 10.20 MB/sec execute 232 sec latency 137.861 ms 2024-05-14T09:42:26.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:26 smithi002 ceph-mon[31557]: pgmap v648: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.2 MiB/s wr, 260 op/s 2024-05-14T09:42:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:26 smithi029 ceph-mon[36249]: pgmap v648: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.2 MiB/s wr, 260 op/s 2024-05-14T09:42:26.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:26 smithi026 ceph-mon[36056]: pgmap v648: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.2 MiB/s wr, 260 op/s 2024-05-14T09:42:27.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 675690 10.19 MB/sec execute 233 sec latency 19.050 ms 2024-05-14T09:42:28.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 678372 10.21 MB/sec execute 234 sec latency 13.902 ms 2024-05-14T09:42:28.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:28 smithi029 ceph-mon[36249]: pgmap v649: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 335 op/s 2024-05-14T09:42:28.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:28 smithi026 ceph-mon[36056]: pgmap v649: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 335 op/s 2024-05-14T09:42:28.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:28 smithi002 ceph-mon[31557]: pgmap v649: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 335 op/s 2024-05-14T09:42:29.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 679589 10.19 MB/sec execute 235 sec latency 70.251 ms 2024-05-14T09:42:30.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 681972 10.21 MB/sec execute 236 sec latency 26.823 ms 2024-05-14T09:42:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:30 smithi029 ceph-mon[36249]: pgmap v650: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 2.6 MiB/s wr, 290 op/s 2024-05-14T09:42:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:30 smithi026 ceph-mon[36056]: pgmap v650: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 2.6 MiB/s wr, 290 op/s 2024-05-14T09:42:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:30 smithi002 ceph-mon[31557]: pgmap v650: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 2.6 MiB/s wr, 290 op/s 2024-05-14T09:42:31.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 683479 10.19 MB/sec execute 237 sec latency 11.496 ms 2024-05-14T09:42:32.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 685695 10.21 MB/sec execute 238 sec latency 13.327 ms 2024-05-14T09:42:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.293+0000 7f0caab9d640 1 -- 172.21.15.2:0/1054268441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca41088b0 msgr2=0x7f0ca410acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.293+0000 7f0caab9d640 1 --2- 172.21.15.2:0/1054268441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca41088b0 0x7f0ca410acd0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009900 tx=0x7f0c9c02a410 comp rx=0 tx=0).stop 2024-05-14T09:42:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.293+0000 7f0caab9d640 1 -- 172.21.15.2:0/1054268441 shutdown_connections 2024-05-14T09:42:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.293+0000 7f0caab9d640 1 --2- 172.21.15.2:0/1054268441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca41088b0 0x7f0ca410acd0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.293+0000 7f0caab9d640 1 --2- 172.21.15.2:0/1054268441 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ca4103bb0 0x7f0ca4103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.293+0000 7f0caab9d640 1 --2- 172.21.15.2:0/1054268441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca4102970 0x7f0ca4103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.293+0000 7f0caab9d640 1 -- 172.21.15.2:0/1054268441 >> 172.21.15.2:0/1054268441 conn(0x7f0ca40fdb60 msgr2=0x7f0ca4100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.293+0000 7f0caab9d640 1 -- 172.21.15.2:0/1054268441 shutdown_connections 2024-05-14T09:42:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.293+0000 7f0caab9d640 1 -- 172.21.15.2:0/1054268441 wait complete. 2024-05-14T09:42:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.294+0000 7f0caab9d640 1 Processor -- start 2024-05-14T09:42:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.294+0000 7f0caab9d640 1 -- start start 2024-05-14T09:42:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.294+0000 7f0caab9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca4102970 0x7f0ca406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.294+0000 7f0caab9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca4103bb0 0x7f0ca419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.294+0000 7f0caab9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ca41088b0 0x7f0ca41a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.294+0000 7f0caab9d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca410e540 con 0x7f0ca4103bb0 2024-05-14T09:42:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.294+0000 7f0caab9d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca410e3c0 con 0x7f0ca4102970 2024-05-14T09:42:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.294+0000 7f0caab9d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca410e6c0 con 0x7f0ca41088b0 2024-05-14T09:42:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.294+0000 7f0ca939a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca4103bb0 0x7f0ca419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.294+0000 7f0ca939a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca4103bb0 0x7f0ca419ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35350/0 (socket says 172.21.15.2:35350) 2024-05-14T09:42:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.294+0000 7f0ca939a640 1 -- 172.21.15.2:0/995113582 learned_addr learned my addr 172.21.15.2:0/995113582 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.294+0000 7f0ca9b9b640 1 --2- 172.21.15.2:0/995113582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca4102970 0x7f0ca406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.294+0000 7f0caa39c640 1 --2- 172.21.15.2:0/995113582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ca41088b0 0x7f0ca41a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.295+0000 7f0ca9b9b640 1 -- 172.21.15.2:0/995113582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ca41088b0 msgr2=0x7f0ca41a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.295+0000 7f0ca9b9b640 1 --2- 172.21.15.2:0/995113582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ca41088b0 0x7f0ca41a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.295+0000 7f0ca9b9b640 1 -- 172.21.15.2:0/995113582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca4103bb0 msgr2=0x7f0ca419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.295+0000 7f0ca9b9b640 1 --2- 172.21.15.2:0/995113582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca4103bb0 0x7f0ca419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.295+0000 7f0ca9b9b640 1 -- 172.21.15.2:0/995113582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca41a1c70 con 0x7f0ca4102970 2024-05-14T09:42:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.295+0000 7f0ca939a640 1 --2- 172.21.15.2:0/995113582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca4103bb0 0x7f0ca419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.295+0000 7f0ca9b9b640 1 --2- 172.21.15.2:0/995113582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca4102970 0x7f0ca406b0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4102240 tx=0x7f0c9400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.295+0000 7f0ca2ffd640 1 -- 172.21.15.2:0/995113582 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c94003970 con 0x7f0ca4102970 2024-05-14T09:42:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.295+0000 7f0ca2ffd640 1 -- 172.21.15.2:0/995113582 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c94004030 con 0x7f0ca4102970 2024-05-14T09:42:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.295+0000 7f0caab9d640 1 -- 172.21.15.2:0/995113582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca41a1f60 con 0x7f0ca4102970 2024-05-14T09:42:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.295+0000 7f0ca2ffd640 1 -- 172.21.15.2:0/995113582 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c94010690 con 0x7f0ca4102970 2024-05-14T09:42:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.295+0000 7f0caab9d640 1 -- 172.21.15.2:0/995113582 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca41a23d0 con 0x7f0ca4102970 2024-05-14T09:42:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.298+0000 7f0ca2ffd640 1 -- 172.21.15.2:0/995113582 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0c94003b10 con 0x7f0ca4102970 2024-05-14T09:42:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.298+0000 7f0cb2d8e740 1 Processor -- start 2024-05-14T09:42:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.299+0000 7f0cb2d8e740 1 -- start start 2024-05-14T09:42:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.299+0000 7f0cb2d8e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d231b0 0x5575e20c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.299+0000 7f0cb2d8e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1cfa2f0 0x5575e20c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.299+0000 7f0cb2d8e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1d024d0 0x5575e20cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.299+0000 7f0cb2d8e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5575e20295d0 con 0x5575e1d024d0 2024-05-14T09:42:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.299+0000 7f0cb2d8e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5575e1fe2890 con 0x5575e1cfa2f0 2024-05-14T09:42:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.299+0000 7f0cb2d8e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5575e2025590 con 0x5575e1d231b0 2024-05-14T09:42:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.299+0000 7f0ca9b9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1cfa2f0 0x5575e20c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.299+0000 7f0ca939a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1d024d0 0x5575e20cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.299+0000 7f0ca9b9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1cfa2f0 0x5575e20c51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:45216/0 (socket says 172.21.15.2:45216) 2024-05-14T09:42:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.299+0000 7f0ca939a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1d024d0 0x5575e20cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35358/0 (socket says 172.21.15.2:35358) 2024-05-14T09:42:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.299+0000 7f0ca9b9b640 1 -- 172.21.15.2:0/3715807111 learned_addr learned my addr 172.21.15.2:0/3715807111 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.299+0000 7f0caa39c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d231b0 0x5575e20c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.300+0000 7f0ca2ffd640 1 --2- 172.21.15.2:0/995113582 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c8c0b2920 0x7f0c8c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.300+0000 7f0ca939a640 1 -- 172.21.15.2:0/3715807111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d231b0 msgr2=0x5575e20c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.300+0000 7f0ca939a640 1 --2- 172.21.15.2:0/3715807111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d231b0 0x5575e20c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.300+0000 7f0ca939a640 1 -- 172.21.15.2:0/3715807111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1cfa2f0 msgr2=0x5575e20c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.300+0000 7f0ca939a640 1 --2- 172.21.15.2:0/3715807111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1cfa2f0 0x5575e20c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.300+0000 7f0ca939a640 1 -- 172.21.15.2:0/3715807111 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e1d3b210 con 0x5575e1d024d0 2024-05-14T09:42:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.300+0000 7f0ca9b9b640 1 --2- 172.21.15.2:0/3715807111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1cfa2f0 0x5575e20c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.300+0000 7f0caa39c640 1 --2- 172.21.15.2:0/995113582 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c8c0b2920 0x7f0c8c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.300+0000 7f0ca939a640 1 --2- 172.21.15.2:0/3715807111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1d024d0 0x5575e20cc0b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0ca406aa70 tx=0x7f0c980119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76a953b8f30659f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.300+0000 7f0ca2ffd640 1 -- 172.21.15.2:0/995113582 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f0c940e2a20 con 0x7f0ca4102970 2024-05-14T09:42:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.300+0000 7f0c8bfff640 1 -- 172.21.15.2:0/3715807111 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c98005020 con 0x5575e1d024d0 2024-05-14T09:42:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.300+0000 7f0c8bfff640 1 -- 172.21.15.2:0/3715807111 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c980047a0 con 0x5575e1d024d0 2024-05-14T09:42:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.300+0000 7f0c8bfff640 1 -- 172.21.15.2:0/3715807111 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800f900 con 0x5575e1d024d0 2024-05-14T09:42:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.301+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/3715807111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1d024d0 msgr2=0x5575e20cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.301+0000 7f0caa39c640 1 --2- 172.21.15.2:0/995113582 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c8c0b2920 0x7f0c8c0b4e30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c031440 tx=0x7f0c9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.301+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/3715807111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1d024d0 0x5575e20cc0b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0ca406aa70 tx=0x7f0c980119a0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.301+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/3715807111 shutdown_connections 2024-05-14T09:42:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.301+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/3715807111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1d024d0 0x5575e20cc0b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.301+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/3715807111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1cfa2f0 0x5575e20c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.301+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/3715807111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d231b0 0x5575e20c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.301+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/3715807111 >> 172.21.15.2:0/3715807111 conn(0x5575e1d17300 msgr2=0x5575e20c05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.301+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/3715807111 shutdown_connections 2024-05-14T09:42:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.301+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/3715807111 wait complete. 2024-05-14T09:42:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.302+0000 7f0cb2d8e740 1 -- start start 2024-05-14T09:42:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.302+0000 7f0cb2d8e740 1 Processor -- start 2024-05-14T09:42:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.302+0000 7f0cb2d8e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1cfa2f0 0x5575e20ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.302+0000 7f0cb2d8e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1d024d0 0x5575e20ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.302+0000 7f0cb2d8e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d231b0 0x5575e20d5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.302+0000 7f0cb2d8e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5575e20ccb00 con 0x5575e1cfa2f0 2024-05-14T09:42:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.302+0000 7f0cb2d8e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5575e1fe6370 con 0x5575e1d024d0 2024-05-14T09:42:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.302+0000 7f0cb2d8e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5575e20ccc80 con 0x5575e1d231b0 2024-05-14T09:42:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.302+0000 7f0cb2d8e740 10 client.0 _ll_register_callbacks cb 0x5575e1d3aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:42:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.302+0000 7f0ca9b9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1cfa2f0 0x5575e20ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.302+0000 7f0ca939a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1d024d0 0x5575e20ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.302+0000 7f0ca9b9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1cfa2f0 0x5575e20ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35360/0 (socket says 172.21.15.2:35360) 2024-05-14T09:42:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.302+0000 7f0ca9b9b640 1 -- 172.21.15.2:0/4268490463 learned_addr learned my addr 172.21.15.2:0/4268490463 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.302+0000 7f0caa39c640 1 --2- 172.21.15.2:0/4268490463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d231b0 0x5575e20d5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.303+0000 7f0ca9b9b640 1 -- 172.21.15.2:0/4268490463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d231b0 msgr2=0x5575e20d5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.303+0000 7f0ca9b9b640 1 --2- 172.21.15.2:0/4268490463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d231b0 0x5575e20d5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.303+0000 7f0ca9b9b640 1 -- 172.21.15.2:0/4268490463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1d024d0 msgr2=0x5575e20ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.303+0000 7f0ca9b9b640 1 --2- 172.21.15.2:0/4268490463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1d024d0 0x5575e20ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.303+0000 7f0ca9b9b640 1 -- 172.21.15.2:0/4268490463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e1e07620 con 0x5575e1cfa2f0 2024-05-14T09:42:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.303+0000 7f0ca939a640 1 --2- 172.21.15.2:0/4268490463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1d024d0 0x5575e20ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.303+0000 7f0ca9b9b640 1 --2- 172.21.15.2:0/4268490463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1cfa2f0 0x5575e20ce840 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0c940a61c0 tx=0x7f0c940046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3ffec5bfdf6540 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.303+0000 7f0c897fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:42:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.303+0000 7f0c897fa640 1 -- 172.21.15.2:0/4268490463 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c940a9390 con 0x5575e1cfa2f0 2024-05-14T09:42:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.303+0000 7f0c897fa640 1 -- 172.21.15.2:0/4268490463 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f0c94004b60 con 0x5575e1cfa2f0 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.303+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/4268490463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c980047a0 con 0x5575e1cfa2f0 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.303+0000 7f0c897fa640 1 -- 172.21.15.2:0/4268490463 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0c94003e00 con 0x5575e1cfa2f0 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.303+0000 7f0cb2d8e740 10 client.16350 fetch_fsmap learned FSMap version 34 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.303+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/4268490463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5575e20d8c80 con 0x5575e1cfa2f0 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.303+0000 7f0c897fa640 1 -- 172.21.15.2:0/4268490463 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c980047a0 con 0x5575e1cfa2f0 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.304+0000 7f0c897fa640 1 -- 172.21.15.2:0/4268490463 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f0c94020020 con 0x5575e1cfa2f0 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.304+0000 7f0cb2d8e740 10 client.16350 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.304+0000 7f0cb2d8e740 10 client.16350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.304+0000 7f0cb2d8e740 20 client.16350 populate_metadata read hostname 'smithi002' 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.304+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/4268490463 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5575e1ce1d80 0x5575e20dc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.304+0000 7f0cb2d8e740 4 client.16350 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.304+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/4268490463 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575e20371e0 con 0x5575e1ce1d80 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.304+0000 7f0ca939a640 1 --2- 172.21.15.2:0/4268490463 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5575e1ce1d80 0x5575e20dc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.304+0000 7f0c897fa640 10 client.16350 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.304+0000 7f0ca939a640 1 --2- 172.21.15.2:0/4268490463 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5575e1ce1d80 0x5575e20dc490 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e010854ab8ff5f01 server_cookie=c58a79e6f66dda2 in_seq=0 out_seq=0 2024-05-14T09:42:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.305+0000 7f0c897fa640 1 -- 172.21.15.2:0/4268490463 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5575e20371e0 con 0x5575e1ce1d80 2024-05-14T09:42:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.305+0000 7f0c897fa640 10 client.16350 handle_command_reply: tid=0 2024-05-14T09:42:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.305+0000 7f0cb2d8e740 1 client.16350 shutdown 2024-05-14T09:42:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.305+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/4268490463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1cfa2f0 msgr2=0x5575e20ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.305+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/4268490463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1cfa2f0 0x5575e20ce840 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0c940a61c0 tx=0x7f0c940046f0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.305+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/4268490463 shutdown_connections 2024-05-14T09:42:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.305+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/4268490463 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5575e1ce1d80 0x5575e20dc490 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.305+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/4268490463 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d231b0 0x5575e20d5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.305+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/4268490463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1d024d0 0x5575e20ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.305+0000 7f0c897fa640 0 client.16350 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.305+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/4268490463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1cfa2f0 0x5575e20ce840 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.305+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/4268490463 >> 172.21.15.2:0/4268490463 conn(0x5575e1d17300 msgr2=0x5575e20c34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.305+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/4268490463 shutdown_connections 2024-05-14T09:42:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.305+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/4268490463 wait complete. 2024-05-14T09:42:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.306+0000 7f0cb2d8e740 20 client.16350 trim_cache size 0 max 16384 2024-05-14T09:42:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.324+0000 7f0cb2d8e740 1 Processor -- start 2024-05-14T09:42:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.324+0000 7f0cb2d8e740 1 -- start start 2024-05-14T09:42:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.324+0000 7f0cb2d8e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1cfa2f0 0x5575e20c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.324+0000 7f0cb2d8e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d024d0 0x5575e20ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.324+0000 7f0cb2d8e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1d231b0 0x5575e20bfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.324+0000 7f0cb2d8e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca41af2c0 con 0x5575e1d231b0 2024-05-14T09:42:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.324+0000 7f0cb2d8e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca41acb60 con 0x5575e1cfa2f0 2024-05-14T09:42:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.324+0000 7f0cb2d8e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca41aa9b0 con 0x5575e1d024d0 2024-05-14T09:42:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.324+0000 7f0caa39c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1d231b0 0x5575e20bfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.324+0000 7f0caa39c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1d231b0 0x5575e20bfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35374/0 (socket says 172.21.15.2:35374) 2024-05-14T09:42:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.324+0000 7f0caa39c640 1 -- 172.21.15.2:0/2775202356 learned_addr learned my addr 172.21.15.2:0/2775202356 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.324+0000 7f0ca939a640 1 --2- 172.21.15.2:0/2775202356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d024d0 0x5575e20ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.324+0000 7f0caa39c640 1 -- 172.21.15.2:0/2775202356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d024d0 msgr2=0x5575e20ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.324+0000 7f0caa39c640 1 --2- 172.21.15.2:0/2775202356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d024d0 0x5575e20ceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.325+0000 7f0caa39c640 1 -- 172.21.15.2:0/2775202356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1cfa2f0 msgr2=0x5575e20c9300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:42:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.325+0000 7f0caa39c640 1 --2- 172.21.15.2:0/2775202356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1cfa2f0 0x5575e20c9300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.325+0000 7f0caa39c640 1 -- 172.21.15.2:0/2775202356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e20c5a30 con 0x5575e1d231b0 2024-05-14T09:42:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.325+0000 7f0caa39c640 1 --2- 172.21.15.2:0/2775202356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1d231b0 0x5575e20bfda0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009900 tx=0x7f0c9c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f882114c64fa43 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.325+0000 7f0c8b7fe640 1 -- 172.21.15.2:0/2775202356 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c03c700 con 0x5575e1d231b0 2024-05-14T09:42:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.325+0000 7f0c8b7fe640 1 -- 172.21.15.2:0/2775202356 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c9c04e070 con 0x5575e1d231b0 2024-05-14T09:42:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.325+0000 7f0c8b7fe640 1 -- 172.21.15.2:0/2775202356 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c053d00 con 0x5575e1d231b0 2024-05-14T09:42:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.326+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/2775202356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1d231b0 msgr2=0x5575e20bfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.326+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/2775202356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1d231b0 0x5575e20bfda0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009900 tx=0x7f0c9c041000 comp rx=0 tx=0).stop 2024-05-14T09:42:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.326+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/2775202356 shutdown_connections 2024-05-14T09:42:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.326+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/2775202356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1d231b0 0x5575e20bfda0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.326+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/2775202356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d024d0 0x5575e20ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.326+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/2775202356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1cfa2f0 0x5575e20c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.326+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/2775202356 >> 172.21.15.2:0/2775202356 conn(0x5575e1d17300 msgr2=0x5575e20ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.326+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/2775202356 shutdown_connections 2024-05-14T09:42:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.326+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/2775202356 wait complete. 2024-05-14T09:42:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.327+0000 7f0cb2d8e740 1 -- start start 2024-05-14T09:42:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.327+0000 7f0cb2d8e740 1 Processor -- start 2024-05-14T09:42:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.327+0000 7f0cb2d8e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1cfa2f0 0x5575e20d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.327+0000 7f0cb2d8e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d024d0 0x5575e20d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.327+0000 7f0cb2d8e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1d231b0 0x5575e21427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.327+0000 7f0cb2d8e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca41ab020 con 0x5575e1cfa2f0 2024-05-14T09:42:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.327+0000 7f0cb2d8e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca41ac490 con 0x5575e1d231b0 2024-05-14T09:42:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.327+0000 7f0cb2d8e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca41ab6f0 con 0x5575e1d024d0 2024-05-14T09:42:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.327+0000 7f0cb2d8e740 10 client.0 _ll_register_callbacks cb 0x7f0c94004790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:42:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.327+0000 7f0ca9b9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1cfa2f0 0x5575e20d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.327+0000 7f0ca9b9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1cfa2f0 0x5575e20d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35380/0 (socket says 172.21.15.2:35380) 2024-05-14T09:42:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.327+0000 7f0ca9b9b640 1 -- 172.21.15.2:0/2351450222 learned_addr learned my addr 172.21.15.2:0/2351450222 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.327+0000 7f0ca939a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d024d0 0x5575e20d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.327+0000 7f0caa39c640 1 --2- 172.21.15.2:0/2351450222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1d231b0 0x5575e21427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.328+0000 7f0ca9b9b640 1 -- 172.21.15.2:0/2351450222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d024d0 msgr2=0x5575e20d27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.328+0000 7f0ca9b9b640 1 --2- 172.21.15.2:0/2351450222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d024d0 0x5575e20d27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.328+0000 7f0ca9b9b640 1 -- 172.21.15.2:0/2351450222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1d231b0 msgr2=0x5575e21427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.328+0000 7f0ca9b9b640 1 --2- 172.21.15.2:0/2351450222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1d231b0 0x5575e21427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.328+0000 7f0ca9b9b640 1 -- 172.21.15.2:0/2351450222 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e2142d30 con 0x5575e1cfa2f0 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.328+0000 7f0caa39c640 1 --2- 172.21.15.2:0/2351450222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1d231b0 0x5575e21427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.328+0000 7f0ca9b9b640 1 --2- 172.21.15.2:0/2351450222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1cfa2f0 0x5575e20d2290 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0c940a9920 tx=0x7f0c940c4350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11364bbbc275374 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.328+0000 7f0c897fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.328+0000 7f0c897fa640 1 -- 172.21.15.2:0/2351450222 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c940e6030 con 0x5575e1cfa2f0 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.328+0000 7f0c897fa640 1 -- 172.21.15.2:0/2351450222 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f0c940a9090 con 0x5575e1cfa2f0 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.328+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/2351450222 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c9c04e070 con 0x5575e1cfa2f0 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.328+0000 7f0c897fa640 1 -- 172.21.15.2:0/2351450222 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0c9408b050 con 0x5575e1cfa2f0 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.328+0000 7f0cb2d8e740 10 client.16356 fetch_fsmap learned FSMap version 34 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.328+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/2351450222 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5575e2143f40 con 0x5575e1cfa2f0 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.328+0000 7f0c897fa640 1 -- 172.21.15.2:0/2351450222 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c9c04e070 con 0x5575e1cfa2f0 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.329+0000 7f0c897fa640 1 -- 172.21.15.2:0/2351450222 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f0c94020070 con 0x5575e1cfa2f0 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.329+0000 7f0cb2d8e740 10 client.16356 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.329+0000 7f0cb2d8e740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.329+0000 7f0cb2d8e740 20 client.16356 populate_metadata read hostname 'smithi002' 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.329+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/2351450222 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5575e1cb71c0 0x5575e2146980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.329+0000 7f0cb2d8e740 4 client.16356 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:42:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.329+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/2351450222 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575e2031050 con 0x5575e1cb71c0 2024-05-14T09:42:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.329+0000 7f0ca939a640 1 --2- 172.21.15.2:0/2351450222 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5575e1cb71c0 0x5575e2146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.329+0000 7f0c897fa640 10 client.16356 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.329+0000 7f0ca939a640 1 --2- 172.21.15.2:0/2351450222 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5575e1cb71c0 0x5575e2146980 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faa92298819bdcd5 server_cookie=3c21ee9ebeeb180 in_seq=0 out_seq=0 2024-05-14T09:42:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.418+0000 7f0c897fa640 1 -- 172.21.15.2:0/2351450222 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5575e2031050 con 0x5575e1cb71c0 2024-05-14T09:42:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.418+0000 7f0c897fa640 10 client.16356 handle_command_reply: tid=0 2024-05-14T09:42:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.418+0000 7f0cb2d8e740 1 client.16356 shutdown 2024-05-14T09:42:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.419+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/2351450222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1cfa2f0 msgr2=0x5575e20d2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.419+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/2351450222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1cfa2f0 0x5575e20d2290 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0c940a9920 tx=0x7f0c940c4350 comp rx=0 tx=0).stop 2024-05-14T09:42:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.419+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/2351450222 shutdown_connections 2024-05-14T09:42:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.419+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/2351450222 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5575e1cb71c0 0x5575e2146980 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.419+0000 7f0c897fa640 0 client.16356 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.419+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/2351450222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575e1d231b0 0x5575e21427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.419+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/2351450222 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5575e1d024d0 0x5575e20d27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.419+0000 7f0cb2d8e740 1 --2- 172.21.15.2:0/2351450222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575e1cfa2f0 0x5575e20d2290 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.419+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/2351450222 >> 172.21.15.2:0/2351450222 conn(0x5575e1d17300 msgr2=0x5575e20c33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.419+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/2351450222 shutdown_connections 2024-05-14T09:42:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.419+0000 7f0cb2d8e740 1 -- 172.21.15.2:0/2351450222 wait complete. 2024-05-14T09:42:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.419+0000 7f0cb2d8e740 20 client.16356 trim_cache size 0 max 16384 2024-05-14T09:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.420+0000 7f0caab9d640 1 -- 172.21.15.2:0/995113582 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c8c0b2920 msgr2=0x7f0c8c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.420+0000 7f0caab9d640 1 --2- 172.21.15.2:0/995113582 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c8c0b2920 0x7f0c8c0b4e30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c031440 tx=0x7f0c9c009260 comp rx=0 tx=0).stop 2024-05-14T09:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.420+0000 7f0caab9d640 1 -- 172.21.15.2:0/995113582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca4102970 msgr2=0x7f0ca406b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.420+0000 7f0caab9d640 1 --2- 172.21.15.2:0/995113582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca4102970 0x7f0ca406b0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4102240 tx=0x7f0c9400ef90 comp rx=0 tx=0).stop 2024-05-14T09:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.420+0000 7f0caab9d640 1 -- 172.21.15.2:0/995113582 shutdown_connections 2024-05-14T09:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.420+0000 7f0caab9d640 1 --2- 172.21.15.2:0/995113582 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c8c0b2920 0x7f0c8c0b4e30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.420+0000 7f0caab9d640 1 --2- 172.21.15.2:0/995113582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ca41088b0 0x7f0ca41a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.420+0000 7f0caab9d640 1 --2- 172.21.15.2:0/995113582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca4103bb0 0x7f0ca419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.420+0000 7f0caab9d640 1 --2- 172.21.15.2:0/995113582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca4102970 0x7f0ca406b0d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.420+0000 7f0caab9d640 1 -- 172.21.15.2:0/995113582 >> 172.21.15.2:0/995113582 conn(0x7f0ca40fdb60 msgr2=0x7f0ca4104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.420+0000 7f0caab9d640 1 -- 172.21.15.2:0/995113582 shutdown_connections 2024-05-14T09:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:32.420+0000 7f0caab9d640 1 -- 172.21.15.2:0/995113582 wait complete. 2024-05-14T09:42:32.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:32 smithi029 ceph-mon[36249]: pgmap v651: 129 pgs: 129 active+clean; 126 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 302 op/s 2024-05-14T09:42:32.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:32 smithi026 ceph-mon[36056]: pgmap v651: 129 pgs: 129 active+clean; 126 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 302 op/s 2024-05-14T09:42:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:32 smithi002 ceph-mon[31557]: pgmap v651: 129 pgs: 129 active+clean; 126 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 302 op/s 2024-05-14T09:42:33.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 687226 10.19 MB/sec execute 239 sec latency 81.360 ms 2024-05-14T09:42:34.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 689159 10.20 MB/sec execute 240 sec latency 88.727 ms 2024-05-14T09:42:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:34 smithi029 ceph-mon[36249]: pgmap v652: 129 pgs: 129 active+clean; 128 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-05-14T09:42:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:34 smithi026 ceph-mon[36056]: pgmap v652: 129 pgs: 129 active+clean; 128 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-05-14T09:42:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:34 smithi002 ceph-mon[31557]: pgmap v652: 129 pgs: 129 active+clean; 128 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-05-14T09:42:35.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 690607 10.18 MB/sec execute 241 sec latency 40.428 ms 2024-05-14T09:42:36.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 692799 10.20 MB/sec execute 242 sec latency 12.259 ms 2024-05-14T09:42:36.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:36 smithi002 ceph-mon[31557]: pgmap v653: 129 pgs: 129 active+clean; 128 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-05-14T09:42:36.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:36 smithi029 ceph-mon[36249]: pgmap v653: 129 pgs: 129 active+clean; 128 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-05-14T09:42:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:36 smithi026 ceph-mon[36056]: pgmap v653: 129 pgs: 129 active+clean; 128 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-05-14T09:42:37.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 694860 10.18 MB/sec execute 243 sec latency 13.758 ms 2024-05-14T09:42:38.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 696515 10.19 MB/sec execute 244 sec latency 14.744 ms 2024-05-14T09:42:38.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:38 smithi002 ceph-mon[31557]: pgmap v654: 129 pgs: 129 active+clean; 126 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 344 op/s 2024-05-14T09:42:38.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:38 smithi029 ceph-mon[36249]: pgmap v654: 129 pgs: 129 active+clean; 126 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 344 op/s 2024-05-14T09:42:38.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:38 smithi026 ceph-mon[36056]: pgmap v654: 129 pgs: 129 active+clean; 126 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 344 op/s 2024-05-14T09:42:39.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 698490 10.18 MB/sec execute 245 sec latency 64.924 ms 2024-05-14T09:42:40.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 700157 10.19 MB/sec execute 246 sec latency 28.185 ms 2024-05-14T09:42:40.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:40 smithi002 ceph-mon[31557]: pgmap v655: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 309 op/s 2024-05-14T09:42:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:40 smithi029 ceph-mon[36249]: pgmap v655: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 309 op/s 2024-05-14T09:42:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:40 smithi026 ceph-mon[36056]: pgmap v655: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 309 op/s 2024-05-14T09:42:41.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 702755 10.20 MB/sec execute 247 sec latency 11.569 ms 2024-05-14T09:42:42.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 704266 10.19 MB/sec execute 248 sec latency 10.555 ms 2024-05-14T09:42:42.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:42 smithi002 ceph-mon[31557]: pgmap v656: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 291 op/s 2024-05-14T09:42:42.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:42 smithi029 ceph-mon[36249]: pgmap v656: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 291 op/s 2024-05-14T09:42:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:42 smithi026 ceph-mon[36056]: pgmap v656: 129 pgs: 129 active+clean; 128 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 291 op/s 2024-05-14T09:42:43.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 706820 10.21 MB/sec execute 249 sec latency 23.671 ms 2024-05-14T09:42:44.270 INFO:tasks.workunit.client.0.smithi002.stdout: 1 708145 10.20 MB/sec execute 250 sec latency 37.138 ms 2024-05-14T09:42:44.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:44 smithi002 ceph-mon[31557]: pgmap v657: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 347 op/s 2024-05-14T09:42:44.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:44 smithi029 ceph-mon[36249]: pgmap v657: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 347 op/s 2024-05-14T09:42:44.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:44 smithi026 ceph-mon[36056]: pgmap v657: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 347 op/s 2024-05-14T09:42:45.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 710435 10.21 MB/sec execute 251 sec latency 29.461 ms 2024-05-14T09:42:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.525+0000 7f3f99215640 1 -- 172.21.15.2:0/825092966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f941018c0 msgr2=0x7f3f94101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.525+0000 7f3f99215640 1 --2- 172.21.15.2:0/825092966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f941018c0 0x7f3f94101d30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009900 tx=0x7f3f7c02a410 comp rx=0 tx=0).stop 2024-05-14T09:42:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.526+0000 7f3f99215640 1 -- 172.21.15.2:0/825092966 shutdown_connections 2024-05-14T09:42:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.526+0000 7f3f99215640 1 --2- 172.21.15.2:0/825092966 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f941023c0 0x7f3f9410be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.526+0000 7f3f99215640 1 --2- 172.21.15.2:0/825092966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f941018c0 0x7f3f94101d30 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.526+0000 7f3f99215640 1 --2- 172.21.15.2:0/825092966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f94105210 0x7f3f94105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.526+0000 7f3f99215640 1 -- 172.21.15.2:0/825092966 >> 172.21.15.2:0/825092966 conn(0x7f3f940fd4e0 msgr2=0x7f3f940ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.526+0000 7f3f99215640 1 -- 172.21.15.2:0/825092966 shutdown_connections 2024-05-14T09:42:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.526+0000 7f3f99215640 1 -- 172.21.15.2:0/825092966 wait complete. 2024-05-14T09:42:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.526+0000 7f3f99215640 1 Processor -- start 2024-05-14T09:42:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f99215640 1 -- start start 2024-05-14T09:42:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f99215640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f941018c0 0x7f3f9419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f99215640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f941023c0 0x7f3f9419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f99215640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f94105210 0x7f3f941a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f99215640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9410f7d0 con 0x7f3f94105210 2024-05-14T09:42:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f99215640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9410f650 con 0x7f3f941018c0 2024-05-14T09:42:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f99215640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9410f950 con 0x7f3f941023c0 2024-05-14T09:42:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f92d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f941018c0 0x7f3f9419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f9359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f94105210 0x7f3f941a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f92d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f941018c0 0x7f3f9419a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:38082/0 (socket says 172.21.15.2:38082) 2024-05-14T09:42:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f8bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f941023c0 0x7f3f9419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f92d9d640 1 -- 172.21.15.2:0/3233533042 learned_addr learned my addr 172.21.15.2:0/3233533042 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f9359e640 1 -- 172.21.15.2:0/3233533042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f941023c0 msgr2=0x7f3f9419a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f9359e640 1 --2- 172.21.15.2:0/3233533042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f941023c0 0x7f3f9419a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f9359e640 1 -- 172.21.15.2:0/3233533042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f941018c0 msgr2=0x7f3f9419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f9359e640 1 --2- 172.21.15.2:0/3233533042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f941018c0 0x7f3f9419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f9359e640 1 -- 172.21.15.2:0/3233533042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f941a1de0 con 0x7f3f94105210 2024-05-14T09:42:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.527+0000 7f3f92d9d640 1 --2- 172.21.15.2:0/3233533042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f941018c0 0x7f3f9419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.528+0000 7f3f9359e640 1 --2- 172.21.15.2:0/3233533042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f94105210 0x7f3f941a1770 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009fd0 tx=0x7f3f8c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.528+0000 7f3f90d99640 1 -- 172.21.15.2:0/3233533042 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c00ce20 con 0x7f3f94105210 2024-05-14T09:42:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.528+0000 7f3f90d99640 1 -- 172.21.15.2:0/3233533042 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f8c005020 con 0x7f3f94105210 2024-05-14T09:42:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.528+0000 7f3f99215640 1 -- 172.21.15.2:0/3233533042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f941a20d0 con 0x7f3f94105210 2024-05-14T09:42:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.528+0000 7f3f90d99640 1 -- 172.21.15.2:0/3233533042 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c010690 con 0x7f3f94105210 2024-05-14T09:42:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.528+0000 7f3f99215640 1 -- 172.21.15.2:0/3233533042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f941a25a0 con 0x7f3f94105210 2024-05-14T09:42:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.530+0000 7f3fa01a3740 1 Processor -- start 2024-05-14T09:42:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.530+0000 7f3f90d99640 1 -- 172.21.15.2:0/3233533042 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3f8c010830 con 0x7f3f94105210 2024-05-14T09:42:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3fa01a3740 1 -- start start 2024-05-14T09:42:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3fa01a3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd0dfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3fa01a3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd152f0 0x55dcdd0dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3fa01a3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0e6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3fa01a3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd0445d0 con 0x55dcdcd3e1b0 2024-05-14T09:42:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3fa01a3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dcdcffd890 con 0x55dcdcd152f0 2024-05-14T09:42:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3fa01a3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd040590 con 0x55dcdcd1d4d0 2024-05-14T09:42:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3f8bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd0dfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3f92d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd152f0 0x55dcdd0dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3f8bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd0dfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52256/0 (socket says 172.21.15.2:52256) 2024-05-14T09:42:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3f8bfff640 1 -- 172.21.15.2:0/204025029 learned_addr learned my addr 172.21.15.2:0/204025029 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3f8bfff640 1 --2- 172.21.15.2:0/204025029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0e6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3f92d9d640 1 -- 172.21.15.2:0/204025029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd1d4d0 msgr2=0x55dcdd0e6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3f92d9d640 1 --2- 172.21.15.2:0/204025029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0e6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3f92d9d640 1 -- 172.21.15.2:0/204025029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd3e1b0 msgr2=0x55dcdd0dfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3f92d9d640 1 --2- 172.21.15.2:0/204025029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd0dfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3f92d9d640 1 -- 172.21.15.2:0/204025029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcdcd56210 con 0x55dcdcd152f0 2024-05-14T09:42:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.531+0000 7f3f8bfff640 1 --2- 172.21.15.2:0/204025029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0e6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:42:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.532+0000 7f3f90d99640 1 --2- 172.21.15.2:0/3233533042 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3f800b28a0 0x7f3f800b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.532+0000 7f3f92d9d640 1 --2- 172.21.15.2:0/204025029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd152f0 0x55dcdd0dffe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3f840114c0 tx=0x7f3f84011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e2da3af13f91d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.532+0000 7f3f8bfff640 1 --2- 172.21.15.2:0/3233533042 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3f800b28a0 0x7f3f800b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.532+0000 7f3f90d99640 1 -- 172.21.15.2:0/3233533042 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3f8c0e2d40 con 0x7f3f94105210 2024-05-14T09:42:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.532+0000 7f3f897fa640 1 -- 172.21.15.2:0/204025029 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84004250 con 0x55dcdcd152f0 2024-05-14T09:42:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.532+0000 7f3f897fa640 1 -- 172.21.15.2:0/204025029 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f840043f0 con 0x55dcdcd152f0 2024-05-14T09:42:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.532+0000 7f3f897fa640 1 -- 172.21.15.2:0/204025029 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f840035f0 con 0x55dcdcd152f0 2024-05-14T09:42:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.533+0000 7f3fa01a3740 1 -- 172.21.15.2:0/204025029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd152f0 msgr2=0x55dcdd0dffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.533+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/204025029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd152f0 0x55dcdd0dffe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3f840114c0 tx=0x7f3f84011980 comp rx=0 tx=0).stop 2024-05-14T09:42:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.533+0000 7f3f8bfff640 1 --2- 172.21.15.2:0/3233533042 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3f800b28a0 0x7f3f800b4db0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55dcdd0e0c30 tx=0x7f3f7c03f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.533+0000 7f3fa01a3740 1 -- 172.21.15.2:0/204025029 shutdown_connections 2024-05-14T09:42:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.533+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/204025029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0e6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.533+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/204025029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd152f0 0x55dcdd0dffe0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.533+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/204025029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd0dfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.533+0000 7f3fa01a3740 1 -- 172.21.15.2:0/204025029 >> 172.21.15.2:0/204025029 conn(0x55dcdcd32300 msgr2=0x55dcdd0db3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.533+0000 7f3fa01a3740 1 -- 172.21.15.2:0/204025029 shutdown_connections 2024-05-14T09:42:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.533+0000 7f3fa01a3740 1 -- 172.21.15.2:0/204025029 wait complete. 2024-05-14T09:42:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.533+0000 7f3fa01a3740 1 -- start start 2024-05-14T09:42:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3fa01a3740 1 Processor -- start 2024-05-14T09:42:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3fa01a3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd152f0 0x55dcdd0e9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3fa01a3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0e9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3fa01a3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd0f0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3fa01a3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd0e7930 con 0x55dcdcd3e1b0 2024-05-14T09:42:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3fa01a3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd001370 con 0x55dcdcd1d4d0 2024-05-14T09:42:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3fa01a3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd0e7ab0 con 0x55dcdcd152f0 2024-05-14T09:42:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3fa01a3740 10 client.0 _ll_register_callbacks cb 0x55dcdcd55a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:42:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3f9359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd0f0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3f92d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd152f0 0x55dcdd0e9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3f9359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd0f0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52258/0 (socket says 172.21.15.2:52258) 2024-05-14T09:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3f9359e640 1 -- 172.21.15.2:0/2089390136 learned_addr learned my addr 172.21.15.2:0/2089390136 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3f8bfff640 1 --2- 172.21.15.2:0/2089390136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0e9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3f92d9d640 1 -- 172.21.15.2:0/2089390136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd1d4d0 msgr2=0x55dcdd0e9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3f92d9d640 1 --2- 172.21.15.2:0/2089390136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0e9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3f92d9d640 1 -- 172.21.15.2:0/2089390136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd3e1b0 msgr2=0x55dcdd0f0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3f92d9d640 1 --2- 172.21.15.2:0/2089390136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd0f0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3f92d9d640 1 -- 172.21.15.2:0/2089390136 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcdce22620 con 0x55dcdcd152f0 2024-05-14T09:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.534+0000 7f3f9359e640 1 --2- 172.21.15.2:0/2089390136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd0f0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.535+0000 7f3f92d9d640 1 --2- 172.21.15.2:0/2089390136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd152f0 0x55dcdd0e9670 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3f8401b040 tx=0x7f3f840040d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a050c087aa23a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.535+0000 7f3f7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.535+0000 7f3f7a7fc640 1 -- 172.21.15.2:0/2089390136 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84028070 con 0x55dcdcd152f0 2024-05-14T09:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.535+0000 7f3fa01a3740 1 -- 172.21.15.2:0/2089390136 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f840043f0 con 0x55dcdcd152f0 2024-05-14T09:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.535+0000 7f3f7a7fc640 1 -- 172.21.15.2:0/2089390136 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3f84003a00 con 0x55dcdcd152f0 2024-05-14T09:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.535+0000 7f3f7a7fc640 1 -- 172.21.15.2:0/2089390136 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3f84034050 con 0x55dcdcd152f0 2024-05-14T09:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.535+0000 7f3fa01a3740 10 client.26038 fetch_fsmap learned FSMap version 34 2024-05-14T09:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.535+0000 7f3fa01a3740 1 -- 172.21.15.2:0/2089390136 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55dcdd0f3a90 con 0x55dcdcd152f0 2024-05-14T09:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.535+0000 7f3f7a7fc640 1 -- 172.21.15.2:0/2089390136 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f840043f0 con 0x55dcdcd152f0 2024-05-14T09:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.535+0000 7f3f7a7fc640 1 -- 172.21.15.2:0/2089390136 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f3f84004c90 con 0x55dcdcd152f0 2024-05-14T09:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.535+0000 7f3fa01a3740 10 client.26038 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.536+0000 7f3fa01a3740 10 client.26038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.536+0000 7f3fa01a3740 20 client.26038 populate_metadata read hostname 'smithi002' 2024-05-14T09:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.536+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/2089390136 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dcdccfcd80 0x55dcdd0f7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.536+0000 7f3fa01a3740 4 client.26038 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.536+0000 7f3fa01a3740 1 -- 172.21.15.2:0/2089390136 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcdd0521e0 con 0x55dcdccfcd80 2024-05-14T09:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.536+0000 7f3f8bfff640 1 --2- 172.21.15.2:0/2089390136 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dcdccfcd80 0x55dcdd0f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.536+0000 7f3f8bfff640 1 --2- 172.21.15.2:0/2089390136 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dcdccfcd80 0x55dcdd0f7010 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fbe0adc8c7ead82 server_cookie=41778ef11ab5ee6a in_seq=0 out_seq=0 2024-05-14T09:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.536+0000 7f3f7a7fc640 10 client.26038 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.537+0000 7f3f7a7fc640 1 -- 172.21.15.2:0/2089390136 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dcdd0521e0 con 0x55dcdccfcd80 2024-05-14T09:42:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.537+0000 7f3f7a7fc640 10 client.26038 handle_command_reply: tid=0 2024-05-14T09:42:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.537+0000 7f3fa01a3740 1 client.26038 shutdown 2024-05-14T09:42:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.537+0000 7f3fa01a3740 1 -- 172.21.15.2:0/2089390136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd152f0 msgr2=0x55dcdd0e9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.537+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/2089390136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd152f0 0x55dcdd0e9670 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3f8401b040 tx=0x7f3f840040d0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.537+0000 7f3fa01a3740 1 -- 172.21.15.2:0/2089390136 shutdown_connections 2024-05-14T09:42:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.537+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/2089390136 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dcdccfcd80 0x55dcdd0f7010 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.537+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/2089390136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd0f0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.537+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/2089390136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0e9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.537+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/2089390136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd152f0 0x55dcdd0e9670 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.537+0000 7f3f7a7fc640 0 client.26038 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.537+0000 7f3fa01a3740 1 -- 172.21.15.2:0/2089390136 >> 172.21.15.2:0/2089390136 conn(0x55dcdcd32300 msgr2=0x55dcdd0de300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.537+0000 7f3fa01a3740 1 -- 172.21.15.2:0/2089390136 shutdown_connections 2024-05-14T09:42:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.537+0000 7f3fa01a3740 1 -- 172.21.15.2:0/2089390136 wait complete. 2024-05-14T09:42:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.537+0000 7f3fa01a3740 20 client.26038 trim_cache size 0 max 16384 2024-05-14T09:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.552+0000 7f3fa01a3740 1 Processor -- start 2024-05-14T09:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.552+0000 7f3fa01a3740 1 -- start start 2024-05-14T09:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.552+0000 7f3fa01a3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd152f0 0x55dcdd0e9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.552+0000 7f3fa01a3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0f0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.552+0000 7f3fa01a3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd0dab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.552+0000 7f3fa01a3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941af560 con 0x55dcdcd152f0 2024-05-14T09:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.552+0000 7f3fa01a3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941ace00 con 0x55dcdcd3e1b0 2024-05-14T09:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.552+0000 7f3fa01a3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941aac50 con 0x55dcdcd1d4d0 2024-05-14T09:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.553+0000 7f3f92d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd152f0 0x55dcdd0e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.553+0000 7f3f92d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd152f0 0x55dcdd0e9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52268/0 (socket says 172.21.15.2:52268) 2024-05-14T09:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.553+0000 7f3f8bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0f0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.553+0000 7f3f9359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd0dab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.553+0000 7f3f92d9d640 1 -- 172.21.15.2:0/3084544829 learned_addr learned my addr 172.21.15.2:0/3084544829 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.553+0000 7f3f8bfff640 1 -- 172.21.15.2:0/3084544829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd3e1b0 msgr2=0x55dcdd0dab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.553+0000 7f3f8bfff640 1 --2- 172.21.15.2:0/3084544829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd0dab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.553+0000 7f3f8bfff640 1 -- 172.21.15.2:0/3084544829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd152f0 msgr2=0x55dcdd0e9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.553+0000 7f3f8bfff640 1 --2- 172.21.15.2:0/3084544829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd152f0 0x55dcdd0e9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.553+0000 7f3f8bfff640 1 -- 172.21.15.2:0/3084544829 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcdd0ea6e0 con 0x55dcdcd1d4d0 2024-05-14T09:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.553+0000 7f3f92d9d640 1 --2- 172.21.15.2:0/3084544829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd152f0 0x55dcdd0e9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.553+0000 7f3f8bfff640 1 --2- 172.21.15.2:0/3084544829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0f0a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0096c0 tx=0x7f3f7c02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e7d2e473ee01339 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.553+0000 7f3f88ff9640 1 -- 172.21.15.2:0/3084544829 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c035050 con 0x55dcdcd1d4d0 2024-05-14T09:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.553+0000 7f3f88ff9640 1 -- 172.21.15.2:0/3084544829 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7c054260 con 0x55dcdcd1d4d0 2024-05-14T09:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.553+0000 7f3f88ff9640 1 -- 172.21.15.2:0/3084544829 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c031800 con 0x55dcdcd1d4d0 2024-05-14T09:42:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.554+0000 7f3fa01a3740 1 -- 172.21.15.2:0/3084544829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd1d4d0 msgr2=0x55dcdd0f0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.554+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/3084544829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0f0a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0096c0 tx=0x7f3f7c02a410 comp rx=0 tx=0).stop 2024-05-14T09:42:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.554+0000 7f3fa01a3740 1 -- 172.21.15.2:0/3084544829 shutdown_connections 2024-05-14T09:42:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.554+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/3084544829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd0dab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.554+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/3084544829 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0f0a30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.554+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/3084544829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd152f0 0x55dcdd0e9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.554+0000 7f3fa01a3740 1 -- 172.21.15.2:0/3084544829 >> 172.21.15.2:0/3084544829 conn(0x55dcdcd32300 msgr2=0x55dcdd0e9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.554+0000 7f3fa01a3740 1 -- 172.21.15.2:0/3084544829 shutdown_connections 2024-05-14T09:42:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.554+0000 7f3fa01a3740 1 -- 172.21.15.2:0/3084544829 wait complete. 2024-05-14T09:42:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.555+0000 7f3fa01a3740 1 -- start start 2024-05-14T09:42:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.555+0000 7f3fa01a3740 1 Processor -- start 2024-05-14T09:42:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.555+0000 7f3fa01a3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd152f0 0x55dcdd0ed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.555+0000 7f3fa01a3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0edc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.555+0000 7f3fa01a3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd15d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.555+0000 7f3fa01a3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941ab2c0 con 0x55dcdcd152f0 2024-05-14T09:42:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.555+0000 7f3fa01a3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941ac730 con 0x55dcdcd1d4d0 2024-05-14T09:42:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.555+0000 7f3fa01a3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941ab990 con 0x55dcdcd3e1b0 2024-05-14T09:42:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.555+0000 7f3fa01a3740 10 client.0 _ll_register_callbacks cb 0x7f3f8c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:42:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.555+0000 7f3f92d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd152f0 0x55dcdd0ed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.555+0000 7f3f92d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd152f0 0x55dcdd0ed6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52278/0 (socket says 172.21.15.2:52278) 2024-05-14T09:42:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.555+0000 7f3f92d9d640 1 -- 172.21.15.2:0/64231508 learned_addr learned my addr 172.21.15.2:0/64231508 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.555+0000 7f3f9359e640 1 --2- 172.21.15.2:0/64231508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd15d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.555+0000 7f3f8bfff640 1 --2- 172.21.15.2:0/64231508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.556+0000 7f3f92d9d640 1 -- 172.21.15.2:0/64231508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd3e1b0 msgr2=0x55dcdd15d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.556+0000 7f3f92d9d640 1 --2- 172.21.15.2:0/64231508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd15d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.556+0000 7f3f92d9d640 1 -- 172.21.15.2:0/64231508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd1d4d0 msgr2=0x55dcdd0edc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.556+0000 7f3f92d9d640 1 --2- 172.21.15.2:0/64231508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0edc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.556+0000 7f3f92d9d640 1 -- 172.21.15.2:0/64231508 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcdd0f0bf0 con 0x55dcdcd152f0 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.556+0000 7f3f92d9d640 1 --2- 172.21.15.2:0/64231508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd152f0 0x55dcdd0ed6f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3f840095a0 tx=0x7f3f84005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74076f54d8206cd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.556+0000 7f3f7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.556+0000 7f3f7a7fc640 1 -- 172.21.15.2:0/64231508 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8403d060 con 0x55dcdcd152f0 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.556+0000 7f3f7a7fc640 1 -- 172.21.15.2:0/64231508 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f3f84011040 con 0x55dcdcd152f0 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.556+0000 7f3fa01a3740 1 -- 172.21.15.2:0/64231508 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f7c054260 con 0x55dcdcd152f0 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.556+0000 7f3f7a7fc640 1 -- 172.21.15.2:0/64231508 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3f84044050 con 0x55dcdcd152f0 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.556+0000 7f3fa01a3740 10 client.16374 fetch_fsmap learned FSMap version 34 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.556+0000 7f3fa01a3740 1 -- 172.21.15.2:0/64231508 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55dcdd15e7b0 con 0x55dcdcd152f0 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.556+0000 7f3f7a7fc640 1 -- 172.21.15.2:0/64231508 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7c054260 con 0x55dcdcd152f0 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.557+0000 7f3f7a7fc640 1 -- 172.21.15.2:0/64231508 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f3f84003940 con 0x55dcdcd152f0 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.557+0000 7f3fa01a3740 10 client.16374 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.557+0000 7f3fa01a3740 10 client.16374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.557+0000 7f3fa01a3740 20 client.16374 populate_metadata read hostname 'smithi002' 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.557+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/64231508 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dcdccd21c0 0x55dcdd1611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.557+0000 7f3fa01a3740 4 client.16374 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:42:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.557+0000 7f3fa01a3740 1 -- 172.21.15.2:0/64231508 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dcdd04c050 con 0x55dcdccd21c0 2024-05-14T09:42:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.557+0000 7f3f8bfff640 1 --2- 172.21.15.2:0/64231508 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dcdccd21c0 0x55dcdd1611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.557+0000 7f3f7a7fc640 10 client.16374 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.557+0000 7f3f8bfff640 1 --2- 172.21.15.2:0/64231508 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dcdccd21c0 0x55dcdd1611d0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6299ebe144364e5 server_cookie=d3900c80de0d09ca in_seq=0 out_seq=0 2024-05-14T09:42:45.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.646+0000 7f3f7a7fc640 1 -- 172.21.15.2:0/64231508 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dcdd04c050 con 0x55dcdccd21c0 2024-05-14T09:42:45.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.646+0000 7f3f7a7fc640 10 client.16374 handle_command_reply: tid=0 2024-05-14T09:42:45.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.646+0000 7f3fa01a3740 1 client.16374 shutdown 2024-05-14T09:42:45.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.647+0000 7f3fa01a3740 1 -- 172.21.15.2:0/64231508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd152f0 msgr2=0x55dcdd0ed6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:45.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.647+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/64231508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd152f0 0x55dcdd0ed6f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3f840095a0 tx=0x7f3f84005d30 comp rx=0 tx=0).stop 2024-05-14T09:42:45.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.647+0000 7f3fa01a3740 1 -- 172.21.15.2:0/64231508 shutdown_connections 2024-05-14T09:42:45.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.647+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/64231508 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55dcdccd21c0 0x55dcdd1611d0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.647+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/64231508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55dcdcd3e1b0 0x55dcdd15d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.647+0000 7f3f7a7fc640 0 client.16374 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.647+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/64231508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcdcd1d4d0 0x55dcdd0edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.647+0000 7f3fa01a3740 1 --2- 172.21.15.2:0/64231508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dcdcd152f0 0x55dcdd0ed6f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.647+0000 7f3fa01a3740 1 -- 172.21.15.2:0/64231508 >> 172.21.15.2:0/64231508 conn(0x55dcdcd32300 msgr2=0x55dcdd0d9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.647+0000 7f3fa01a3740 1 -- 172.21.15.2:0/64231508 shutdown_connections 2024-05-14T09:42:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.647+0000 7f3fa01a3740 1 -- 172.21.15.2:0/64231508 wait complete. 2024-05-14T09:42:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.647+0000 7f3fa01a3740 20 client.16374 trim_cache size 0 max 16384 2024-05-14T09:42:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:42:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:42:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:42:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:42:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.648+0000 7f3f99215640 1 -- 172.21.15.2:0/3233533042 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3f800b28a0 msgr2=0x7f3f800b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.648+0000 7f3f99215640 1 --2- 172.21.15.2:0/3233533042 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3f800b28a0 0x7f3f800b4db0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55dcdd0e0c30 tx=0x7f3f7c03f040 comp rx=0 tx=0).stop 2024-05-14T09:42:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.648+0000 7f3f99215640 1 -- 172.21.15.2:0/3233533042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f94105210 msgr2=0x7f3f941a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.648+0000 7f3f99215640 1 --2- 172.21.15.2:0/3233533042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f94105210 0x7f3f941a1770 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009fd0 tx=0x7f3f8c00ef10 comp rx=0 tx=0).stop 2024-05-14T09:42:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.648+0000 7f3f99215640 1 -- 172.21.15.2:0/3233533042 shutdown_connections 2024-05-14T09:42:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.648+0000 7f3f99215640 1 --2- 172.21.15.2:0/3233533042 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f3f800b28a0 0x7f3f800b4db0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.648+0000 7f3f99215640 1 --2- 172.21.15.2:0/3233533042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f94105210 0x7f3f941a1770 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.648+0000 7f3f99215640 1 --2- 172.21.15.2:0/3233533042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3f941023c0 0x7f3f9419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.648+0000 7f3f99215640 1 --2- 172.21.15.2:0/3233533042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f941018c0 0x7f3f9419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.648+0000 7f3f99215640 1 -- 172.21.15.2:0/3233533042 >> 172.21.15.2:0/3233533042 conn(0x7f3f940fd4e0 msgr2=0x7f3f94105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.648+0000 7f3f99215640 1 -- 172.21.15.2:0/3233533042 shutdown_connections 2024-05-14T09:42:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:45.648+0000 7f3f99215640 1 -- 172.21.15.2:0/3233533042 wait complete. 2024-05-14T09:42:46.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 711753 10.19 MB/sec execute 252 sec latency 79.383 ms 2024-05-14T09:42:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:46 smithi002 ceph-mon[31557]: pgmap v658: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-05-14T09:42:46.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:46 smithi029 ceph-mon[36249]: pgmap v658: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-05-14T09:42:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:46 smithi026 ceph-mon[36056]: pgmap v658: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-05-14T09:42:47.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 714009 10.21 MB/sec execute 253 sec latency 11.990 ms 2024-05-14T09:42:47.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:42:47.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:42:47.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:42:47.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:47 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:42:47.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:42:47.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:42:47.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:42:47.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:47 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:42:47.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:42:47.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:42:47.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:42:47.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:47 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:42:48.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 715541 10.19 MB/sec execute 254 sec latency 14.359 ms 2024-05-14T09:42:48.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:48 smithi002 ceph-mon[31557]: pgmap v659: 129 pgs: 129 active+clean; 133 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 377 op/s 2024-05-14T09:42:48.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:48 smithi029 ceph-mon[36249]: pgmap v659: 129 pgs: 129 active+clean; 133 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 377 op/s 2024-05-14T09:42:48.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:48 smithi026 ceph-mon[36056]: pgmap v659: 129 pgs: 129 active+clean; 133 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 377 op/s 2024-05-14T09:42:49.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 717527 10.20 MB/sec execute 255 sec latency 67.303 ms 2024-05-14T09:42:50.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 718806 10.18 MB/sec execute 256 sec latency 29.668 ms 2024-05-14T09:42:50.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:50 smithi002 ceph-mon[31557]: pgmap v660: 129 pgs: 129 active+clean; 134 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 317 op/s 2024-05-14T09:42:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:50 smithi029 ceph-mon[36249]: pgmap v660: 129 pgs: 129 active+clean; 134 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 317 op/s 2024-05-14T09:42:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:50 smithi026 ceph-mon[36056]: pgmap v660: 129 pgs: 129 active+clean; 134 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 317 op/s 2024-05-14T09:42:51.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 721087 10.20 MB/sec execute 257 sec latency 13.335 ms 2024-05-14T09:42:52.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 722620 10.18 MB/sec execute 258 sec latency 14.349 ms 2024-05-14T09:42:52.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:52 smithi002 ceph-mon[31557]: pgmap v661: 129 pgs: 129 active+clean; 131 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 291 op/s 2024-05-14T09:42:52.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:52 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:42:52.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:52 smithi029 ceph-mon[36249]: pgmap v661: 129 pgs: 129 active+clean; 131 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 291 op/s 2024-05-14T09:42:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:52 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:42:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:52 smithi026 ceph-mon[36056]: pgmap v661: 129 pgs: 129 active+clean; 131 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 291 op/s 2024-05-14T09:42:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:52 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:42:53.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 724680 10.19 MB/sec execute 259 sec latency 17.514 ms 2024-05-14T09:42:54.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 726096 10.17 MB/sec execute 260 sec latency 61.576 ms 2024-05-14T09:42:54.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:54 smithi002 ceph-mon[31557]: pgmap v662: 129 pgs: 129 active+clean; 133 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 381 op/s 2024-05-14T09:42:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:54 smithi029 ceph-mon[36249]: pgmap v662: 129 pgs: 129 active+clean; 133 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 381 op/s 2024-05-14T09:42:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:54 smithi026 ceph-mon[36056]: pgmap v662: 129 pgs: 129 active+clean; 133 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 381 op/s 2024-05-14T09:42:55.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 728093 10.18 MB/sec execute 261 sec latency 46.715 ms 2024-05-14T09:42:55.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:55 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:42:55.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:55 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:42:55.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:55 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:42:56.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 728949 10.15 MB/sec execute 262 sec latency 205.417 ms 2024-05-14T09:42:56.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:56 smithi002 ceph-mon[31557]: pgmap v663: 129 pgs: 129 active+clean; 133 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 2.2 MiB/s wr, 273 op/s 2024-05-14T09:42:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:56 smithi029 ceph-mon[36249]: pgmap v663: 129 pgs: 129 active+clean; 133 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 2.2 MiB/s wr, 273 op/s 2024-05-14T09:42:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:56 smithi026 ceph-mon[36056]: pgmap v663: 129 pgs: 129 active+clean; 133 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 2.2 MiB/s wr, 273 op/s 2024-05-14T09:42:57.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 731310 10.17 MB/sec execute 263 sec latency 32.164 ms 2024-05-14T09:42:58.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 732408 10.15 MB/sec execute 264 sec latency 75.455 ms 2024-05-14T09:42:58.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:42:58 smithi002 ceph-mon[31557]: pgmap v664: 129 pgs: 129 active+clean; 134 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 2.6 MiB/s wr, 310 op/s 2024-05-14T09:42:58.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:42:58 smithi029 ceph-mon[36249]: pgmap v664: 129 pgs: 129 active+clean; 134 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 2.6 MiB/s wr, 310 op/s 2024-05-14T09:42:58.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:42:58 smithi026 ceph-mon[36056]: pgmap v664: 129 pgs: 129 active+clean; 134 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 2.6 MiB/s wr, 310 op/s 2024-05-14T09:42:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.750+0000 7f34f8234640 1 -- 172.21.15.2:0/2713515150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f0104740 msgr2=0x7f34f0106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.750+0000 7f34f8234640 1 --2- 172.21.15.2:0/2713515150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f0104740 0x7f34f0106b60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009860 tx=0x7f34ec02a340 comp rx=0 tx=0).stop 2024-05-14T09:42:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.751+0000 7f34f8234640 1 -- 172.21.15.2:0/2713515150 shutdown_connections 2024-05-14T09:42:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.751+0000 7f34f8234640 1 --2- 172.21.15.2:0/2713515150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f0109a00 0x7f34f010be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.751+0000 7f34f8234640 1 --2- 172.21.15.2:0/2713515150 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34f01070a0 0x7f34f01094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.751+0000 7f34f8234640 1 --2- 172.21.15.2:0/2713515150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f0104740 0x7f34f0106b60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.751+0000 7f34f8234640 1 -- 172.21.15.2:0/2713515150 >> 172.21.15.2:0/2713515150 conn(0x7f34f00fdd40 msgr2=0x7f34f01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.751+0000 7f34f8234640 1 -- 172.21.15.2:0/2713515150 shutdown_connections 2024-05-14T09:42:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.751+0000 7f34f8234640 1 -- 172.21.15.2:0/2713515150 wait complete. 2024-05-14T09:42:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.751+0000 7f34f8234640 1 Processor -- start 2024-05-14T09:42:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.751+0000 7f34f8234640 1 -- start start 2024-05-14T09:42:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.751+0000 7f34f8234640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34f0104740 0x7f34f019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f8234640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f01070a0 0x7f34f019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f8234640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f0109a00 0x7f34f01a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f8234640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f34f010f730 con 0x7f34f01070a0 2024-05-14T09:42:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f8234640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34f010f5b0 con 0x7f34f0109a00 2024-05-14T09:42:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f8234640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f34f010f8b0 con 0x7f34f0104740 2024-05-14T09:42:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f57cf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f01070a0 0x7f34f019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f5fd0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34f0104740 0x7f34f019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f5fd0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34f0104740 0x7f34f019a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:42746/0 (socket says 172.21.15.2:42746) 2024-05-14T09:42:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f5fd0640 1 -- 172.21.15.2:0/1997241295 learned_addr learned my addr 172.21.15.2:0/1997241295 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f57cf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f01070a0 0x7f34f019a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44726/0 (socket says 172.21.15.2:44726) 2024-05-14T09:42:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f67d1640 1 --2- 172.21.15.2:0/1997241295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f0109a00 0x7f34f01a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f5fd0640 1 -- 172.21.15.2:0/1997241295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f0109a00 msgr2=0x7f34f01a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f5fd0640 1 --2- 172.21.15.2:0/1997241295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f0109a00 0x7f34f01a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f5fd0640 1 -- 172.21.15.2:0/1997241295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f01070a0 msgr2=0x7f34f019a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f5fd0640 1 --2- 172.21.15.2:0/1997241295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f01070a0 0x7f34f019a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f5fd0640 1 -- 172.21.15.2:0/1997241295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f01a1d60 con 0x7f34f0104740 2024-05-14T09:42:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f67d1640 1 --2- 172.21.15.2:0/1997241295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f0109a00 0x7f34f01a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f57cf640 1 --2- 172.21.15.2:0/1997241295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f01070a0 0x7f34f019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f5fd0640 1 --2- 172.21.15.2:0/1997241295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34f0104740 0x7f34f019a2d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f34ec000c00 tx=0x7f34ec032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34e6ffd640 1 -- 172.21.15.2:0/1997241295 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec040020 con 0x7f34f0104740 2024-05-14T09:42:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34f8234640 1 -- 172.21.15.2:0/1997241295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f01a1ff0 con 0x7f34f0104740 2024-05-14T09:42:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.752+0000 7f34e6ffd640 1 -- 172.21.15.2:0/1997241295 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34ec030dd0 con 0x7f34f0104740 2024-05-14T09:42:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.753+0000 7f34e6ffd640 1 -- 172.21.15.2:0/1997241295 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec037040 con 0x7f34f0104740 2024-05-14T09:42:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.753+0000 7f34f8234640 1 -- 172.21.15.2:0/1997241295 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f01a2400 con 0x7f34f0104740 2024-05-14T09:42:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.755+0000 7f34ff1c5740 1 Processor -- start 2024-05-14T09:42:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.755+0000 7f34ff1c5740 1 -- start start 2024-05-14T09:42:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.755+0000 7f34ff1c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d536c1b0 0x5639d570da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.755+0000 7f34ff1c5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d53432f0 0x5639d570dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.755+0000 7f34e6ffd640 1 -- 172.21.15.2:0/1997241295 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f34ec043020 con 0x7f34f0104740 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.755+0000 7f34ff1c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d534b4d0 0x5639d5714ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.755+0000 7f34ff1c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5639d56725d0 con 0x5639d536c1b0 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.755+0000 7f34ff1c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5639d562b890 con 0x5639d534b4d0 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.755+0000 7f34ff1c5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5639d566e590 con 0x5639d53432f0 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34f67d1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d536c1b0 0x5639d570da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34f67d1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d536c1b0 0x5639d570da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44742/0 (socket says 172.21.15.2:44742) 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34f67d1640 1 -- 172.21.15.2:0/1329083723 learned_addr learned my addr 172.21.15.2:0/1329083723 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34f5fd0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d53432f0 0x5639d570dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34f57cf640 1 --2- 172.21.15.2:0/1329083723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d534b4d0 0x5639d5714ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34f57cf640 1 -- 172.21.15.2:0/1329083723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d53432f0 msgr2=0x5639d570dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34f57cf640 1 --2- 172.21.15.2:0/1329083723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d53432f0 0x5639d570dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34f57cf640 1 -- 172.21.15.2:0/1329083723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d536c1b0 msgr2=0x5639d570da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34f57cf640 1 --2- 172.21.15.2:0/1329083723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d536c1b0 0x5639d570da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34f57cf640 1 -- 172.21.15.2:0/1329083723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639d5384210 con 0x5639d534b4d0 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34f67d1640 1 --2- 172.21.15.2:0/1329083723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d536c1b0 0x5639d570da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34f57cf640 1 --2- 172.21.15.2:0/1329083723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d534b4d0 0x5639d5714ee0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f34e000af90 tx=0x7f34e000f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=306c7a5c81f96935 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34dbfff640 1 -- 172.21.15.2:0/1329083723 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0005220 con 0x5639d534b4d0 2024-05-14T09:42:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34dbfff640 1 -- 172.21.15.2:0/1329083723 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34e0004c50 con 0x5639d534b4d0 2024-05-14T09:42:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34e6ffd640 1 --2- 172.21.15.2:0/1997241295 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f34dc0b28a0 0x7f34dc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.756+0000 7f34dbfff640 1 -- 172.21.15.2:0/1329083723 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0010b70 con 0x5639d534b4d0 2024-05-14T09:42:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.757+0000 7f34f67d1640 1 --2- 172.21.15.2:0/1997241295 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f34dc0b28a0 0x7f34dc0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.757+0000 7f34e6ffd640 1 -- 172.21.15.2:0/1997241295 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f34ec102420 con 0x7f34f0104740 2024-05-14T09:42:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.757+0000 7f34ff1c5740 1 -- 172.21.15.2:0/1329083723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d534b4d0 msgr2=0x5639d5714ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.757+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/1329083723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d534b4d0 0x5639d5714ee0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f34e000af90 tx=0x7f34e000f850 comp rx=0 tx=0).stop 2024-05-14T09:42:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.757+0000 7f34f67d1640 1 --2- 172.21.15.2:0/1997241295 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f34dc0b28a0 0x7f34dc0b4db0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5639d570ec30 tx=0x7f34e80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.757+0000 7f34ff1c5740 1 -- 172.21.15.2:0/1329083723 shutdown_connections 2024-05-14T09:42:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.757+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/1329083723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d534b4d0 0x5639d5714ee0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.757+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/1329083723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d53432f0 0x5639d570dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.757+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/1329083723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d536c1b0 0x5639d570da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.757+0000 7f34ff1c5740 1 -- 172.21.15.2:0/1329083723 >> 172.21.15.2:0/1329083723 conn(0x5639d5360300 msgr2=0x5639d57093a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.757+0000 7f34ff1c5740 1 -- 172.21.15.2:0/1329083723 shutdown_connections 2024-05-14T09:42:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.758+0000 7f34ff1c5740 1 -- 172.21.15.2:0/1329083723 wait complete. 2024-05-14T09:42:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.758+0000 7f34ff1c5740 1 -- start start 2024-05-14T09:42:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.758+0000 7f34ff1c5740 1 Processor -- start 2024-05-14T09:42:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.758+0000 7f34ff1c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d53432f0 0x5639d57176a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.758+0000 7f34ff1c5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 0x5639d5717be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.758+0000 7f34ff1c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d536c1b0 0x5639d571ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.758+0000 7f34ff1c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5639d5715930 con 0x5639d536c1b0 2024-05-14T09:42:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.758+0000 7f34ff1c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5639d562f370 con 0x5639d53432f0 2024-05-14T09:42:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.758+0000 7f34ff1c5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5639d5715ab0 con 0x5639d534b4d0 2024-05-14T09:42:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.758+0000 7f34ff1c5740 10 client.0 _ll_register_callbacks cb 0x5639d5383a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:42:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.758+0000 7f34f67d1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d536c1b0 0x5639d571ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.758+0000 7f34f57cf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 0x5639d5717be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.759+0000 7f34f5fd0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d53432f0 0x5639d57176a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.759+0000 7f34f57cf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 0x5639d5717be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:42758/0 (socket says 172.21.15.2:42758) 2024-05-14T09:42:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.759+0000 7f34f67d1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d536c1b0 0x5639d571ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44746/0 (socket says 172.21.15.2:44746) 2024-05-14T09:42:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.759+0000 7f34f57cf640 1 -- 172.21.15.2:0/446585045 learned_addr learned my addr 172.21.15.2:0/446585045 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.759+0000 7f34f57cf640 1 -- 172.21.15.2:0/446585045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d53432f0 msgr2=0x5639d57176a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.759+0000 7f34f57cf640 1 --2- 172.21.15.2:0/446585045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d53432f0 0x5639d57176a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.759+0000 7f34f57cf640 1 -- 172.21.15.2:0/446585045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d536c1b0 msgr2=0x5639d571ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.759+0000 7f34f57cf640 1 --2- 172.21.15.2:0/446585045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d536c1b0 0x5639d571ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.759+0000 7f34f57cf640 1 -- 172.21.15.2:0/446585045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639d5450620 con 0x5639d534b4d0 2024-05-14T09:42:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.759+0000 7f34f67d1640 1 --2- 172.21.15.2:0/446585045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d536c1b0 0x5639d571ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.759+0000 7f34f57cf640 1 --2- 172.21.15.2:0/446585045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 0x5639d5717be0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f34e0005030 tx=0x7f34e0005060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b71ba51a13a3cb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.759+0000 7f34d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.759+0000 7f34d97fa640 1 -- 172.21.15.2:0/446585045 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0027070 con 0x5639d534b4d0 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.759+0000 7f34ff1c5740 1 -- 172.21.15.2:0/446585045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34e0004c50 con 0x5639d534b4d0 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.759+0000 7f34d97fa640 1 -- 172.21.15.2:0/446585045 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f34e0010870 con 0x5639d534b4d0 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.760+0000 7f34d97fa640 1 -- 172.21.15.2:0/446585045 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f34e0033050 con 0x5639d534b4d0 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.760+0000 7f34ff1c5740 10 client.26053 fetch_fsmap learned FSMap version 36 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.760+0000 7f34ff1c5740 1 -- 172.21.15.2:0/446585045 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5639d5721a50 con 0x5639d534b4d0 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.760+0000 7f34d97fa640 1 -- 172.21.15.2:0/446585045 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34e0004c50 con 0x5639d534b4d0 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.760+0000 7f34d97fa640 1 -- 172.21.15.2:0/446585045 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f34e0018810 con 0x5639d534b4d0 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.760+0000 7f34ff1c5740 10 client.26053 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.760+0000 7f34ff1c5740 10 client.26053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.760+0000 7f34ff1c5740 20 client.26053 populate_metadata read hostname 'smithi002' 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.760+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/446585045 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5639d532ad80 0x5639d5724ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.760+0000 7f34ff1c5740 4 client.26053 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.760+0000 7f34ff1c5740 1 -- 172.21.15.2:0/446585045 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639d56801e0 con 0x5639d532ad80 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.760+0000 7f34f5fd0640 1 --2- 172.21.15.2:0/446585045 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5639d532ad80 0x5639d5724ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.761+0000 7f34f5fd0640 1 --2- 172.21.15.2:0/446585045 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5639d532ad80 0x5639d5724ed0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df721c427b75a869 server_cookie=312a449c5754a078 in_seq=0 out_seq=0 2024-05-14T09:42:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.761+0000 7f34d97fa640 10 client.26053 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.761+0000 7f34d97fa640 1 -- 172.21.15.2:0/446585045 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5639d56801e0 con 0x5639d532ad80 2024-05-14T09:42:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.761+0000 7f34d97fa640 10 client.26053 handle_command_reply: tid=0 2024-05-14T09:42:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.761+0000 7f34ff1c5740 1 client.26053 shutdown 2024-05-14T09:42:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.761+0000 7f34ff1c5740 1 -- 172.21.15.2:0/446585045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 msgr2=0x5639d5717be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.761+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/446585045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 0x5639d5717be0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f34e0005030 tx=0x7f34e0005060 comp rx=0 tx=0).stop 2024-05-14T09:42:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.761+0000 7f34ff1c5740 1 -- 172.21.15.2:0/446585045 shutdown_connections 2024-05-14T09:42:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.761+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/446585045 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5639d532ad80 0x5639d5724ed0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.761+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/446585045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d536c1b0 0x5639d571ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.761+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/446585045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 0x5639d5717be0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.761+0000 7f34d97fa640 0 client.26053 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.761+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/446585045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d53432f0 0x5639d57176a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.761+0000 7f34ff1c5740 1 -- 172.21.15.2:0/446585045 >> 172.21.15.2:0/446585045 conn(0x5639d5360300 msgr2=0x5639d570c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.762+0000 7f34ff1c5740 1 -- 172.21.15.2:0/446585045 shutdown_connections 2024-05-14T09:42:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.762+0000 7f34ff1c5740 1 -- 172.21.15.2:0/446585045 wait complete. 2024-05-14T09:42:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.762+0000 7f34ff1c5740 20 client.26053 trim_cache size 0 max 16384 2024-05-14T09:42:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.777+0000 7f34ff1c5740 1 Processor -- start 2024-05-14T09:42:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.777+0000 7f34ff1c5740 1 -- start start 2024-05-14T09:42:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.777+0000 7f34ff1c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d53432f0 0x5639d5712130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.777+0000 7f34ff1c5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 0x5639d5716590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.777+0000 7f34ff1c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d536c1b0 0x5639d571e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.777+0000 7f34ff1c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f34f01ad1d0 con 0x5639d53432f0 2024-05-14T09:42:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.777+0000 7f34ff1c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34f01aaaa0 con 0x5639d536c1b0 2024-05-14T09:42:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.777+0000 7f34ff1c5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f34f01a88f0 con 0x5639d534b4d0 2024-05-14T09:42:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.777+0000 7f34f5fd0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d53432f0 0x5639d5712130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.777+0000 7f34f5fd0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d53432f0 0x5639d5712130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44748/0 (socket says 172.21.15.2:44748) 2024-05-14T09:42:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.777+0000 7f34f5fd0640 1 -- 172.21.15.2:0/2654792103 learned_addr learned my addr 172.21.15.2:0/2654792103 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.777+0000 7f34f67d1640 1 --2- 172.21.15.2:0/2654792103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d536c1b0 0x5639d571e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.777+0000 7f34f57cf640 1 --2- 172.21.15.2:0/2654792103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 0x5639d5716590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.778+0000 7f34f5fd0640 1 -- 172.21.15.2:0/2654792103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 msgr2=0x5639d5716590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.778+0000 7f34f5fd0640 1 --2- 172.21.15.2:0/2654792103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 0x5639d5716590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.778+0000 7f34f5fd0640 1 -- 172.21.15.2:0/2654792103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d536c1b0 msgr2=0x5639d571e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.778+0000 7f34f5fd0640 1 --2- 172.21.15.2:0/2654792103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d536c1b0 0x5639d571e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.778+0000 7f34f5fd0640 1 -- 172.21.15.2:0/2654792103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639d5709d00 con 0x5639d53432f0 2024-05-14T09:42:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.778+0000 7f34f5fd0640 1 --2- 172.21.15.2:0/2654792103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d53432f0 0x5639d5712130 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0d0010 tx=0x7f34ec0ff9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d4e8498a6c83bc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.778+0000 7f34db7fe640 1 -- 172.21.15.2:0/2654792103 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec0c5570 con 0x5639d53432f0 2024-05-14T09:42:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.778+0000 7f34db7fe640 1 -- 172.21.15.2:0/2654792103 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34ec105d50 con 0x5639d53432f0 2024-05-14T09:42:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.778+0000 7f34db7fe640 1 -- 172.21.15.2:0/2654792103 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec0c8e20 con 0x5639d53432f0 2024-05-14T09:42:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.779+0000 7f34ff1c5740 1 -- 172.21.15.2:0/2654792103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d53432f0 msgr2=0x5639d5712130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.779+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/2654792103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d53432f0 0x5639d5712130 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0d0010 tx=0x7f34ec0ff9d0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.779+0000 7f34ff1c5740 1 -- 172.21.15.2:0/2654792103 shutdown_connections 2024-05-14T09:42:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.779+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/2654792103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d536c1b0 0x5639d571e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.779+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/2654792103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 0x5639d5716590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.779+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/2654792103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d53432f0 0x5639d5712130 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.779+0000 7f34ff1c5740 1 -- 172.21.15.2:0/2654792103 >> 172.21.15.2:0/2654792103 conn(0x5639d5360300 msgr2=0x5639d5715e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.779+0000 7f34ff1c5740 1 -- 172.21.15.2:0/2654792103 shutdown_connections 2024-05-14T09:42:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.779+0000 7f34ff1c5740 1 -- 172.21.15.2:0/2654792103 wait complete. 2024-05-14T09:42:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.780+0000 7f34ff1c5740 1 -- start start 2024-05-14T09:42:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.780+0000 7f34ff1c5740 1 Processor -- start 2024-05-14T09:42:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.780+0000 7f34ff1c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d53432f0 0x5639d571c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.780+0000 7f34ff1c5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 0x5639d571c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.780+0000 7f34ff1c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d536c1b0 0x5639d578b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.780+0000 7f34ff1c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f34f01a8f60 con 0x5639d53432f0 2024-05-14T09:42:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.780+0000 7f34ff1c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34f01aa3d0 con 0x5639d536c1b0 2024-05-14T09:42:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.780+0000 7f34ff1c5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f34f01a9630 con 0x5639d534b4d0 2024-05-14T09:42:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.780+0000 7f34ff1c5740 10 client.0 _ll_register_callbacks cb 0x7f34e8009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:42:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.780+0000 7f34f5fd0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d53432f0 0x5639d571c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.780+0000 7f34f57cf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 0x5639d571c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.780+0000 7f34f5fd0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d53432f0 0x5639d571c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44756/0 (socket says 172.21.15.2:44756) 2024-05-14T09:42:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.780+0000 7f34f5fd0640 1 -- 172.21.15.2:0/30204381 learned_addr learned my addr 172.21.15.2:0/30204381 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:42:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.781+0000 7f34f67d1640 1 --2- 172.21.15.2:0/30204381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d536c1b0 0x5639d578b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.781+0000 7f34f5fd0640 1 -- 172.21.15.2:0/30204381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 msgr2=0x5639d571c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.781+0000 7f34f5fd0640 1 --2- 172.21.15.2:0/30204381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 0x5639d571c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.781+0000 7f34f5fd0640 1 -- 172.21.15.2:0/30204381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d536c1b0 msgr2=0x5639d578b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:42:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.781+0000 7f34f5fd0640 1 --2- 172.21.15.2:0/30204381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d536c1b0 0x5639d578b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.781+0000 7f34f5fd0640 1 -- 172.21.15.2:0/30204381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639d571cc00 con 0x5639d53432f0 2024-05-14T09:42:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.781+0000 7f34f57cf640 1 --2- 172.21.15.2:0/30204381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 0x5639d571c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:42:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.781+0000 7f34f5fd0640 1 --2- 172.21.15.2:0/30204381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d53432f0 0x5639d571c0b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0c5730 tx=0x7f34ec004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8535dc2c113e7ce7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:42:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.781+0000 7f34d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:42:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.781+0000 7f34d97fa640 1 -- 172.21.15.2:0/30204381 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec0c9000 con 0x5639d53432f0 2024-05-14T09:42:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.781+0000 7f34d97fa640 1 -- 172.21.15.2:0/30204381 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f34ec0c5940 con 0x5639d53432f0 2024-05-14T09:42:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.781+0000 7f34ff1c5740 1 -- 172.21.15.2:0/30204381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34ec105d50 con 0x5639d53432f0 2024-05-14T09:42:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.782+0000 7f34d97fa640 1 -- 172.21.15.2:0/30204381 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f34ec0c9480 con 0x5639d53432f0 2024-05-14T09:42:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.782+0000 7f34ff1c5740 10 client.16392 fetch_fsmap learned FSMap version 36 2024-05-14T09:42:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.782+0000 7f34ff1c5740 1 -- 172.21.15.2:0/30204381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5639d578c7d0 con 0x5639d53432f0 2024-05-14T09:42:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.782+0000 7f34d97fa640 1 -- 172.21.15.2:0/30204381 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34ec105d50 con 0x5639d53432f0 2024-05-14T09:42:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.782+0000 7f34d97fa640 1 -- 172.21.15.2:0/30204381 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f34ec0041d0 con 0x5639d53432f0 2024-05-14T09:42:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.782+0000 7f34ff1c5740 10 client.16392 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T09:42:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.782+0000 7f34ff1c5740 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:42:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.782+0000 7f34ff1c5740 20 client.16392 populate_metadata read hostname 'smithi002' 2024-05-14T09:42:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.782+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/30204381 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5639d53001c0 0x5639d578f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:42:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.782+0000 7f34ff1c5740 4 client.16392 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:42:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.782+0000 7f34ff1c5740 1 -- 172.21.15.2:0/30204381 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639d567a050 con 0x5639d53001c0 2024-05-14T09:42:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.783+0000 7f34f57cf640 1 --2- 172.21.15.2:0/30204381 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5639d53001c0 0x5639d578f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:42:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.783+0000 7f34d97fa640 10 client.16392 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.783+0000 7f34f57cf640 1 --2- 172.21.15.2:0/30204381 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5639d53001c0 0x5639d578f1a0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b9b35f2677ca4b server_cookie=54aabbc75772bb9d in_seq=0 out_seq=0 2024-05-14T09:42:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.880+0000 7f34d97fa640 1 -- 172.21.15.2:0/30204381 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639d567a050 con 0x5639d53001c0 2024-05-14T09:42:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.880+0000 7f34d97fa640 10 client.16392 handle_command_reply: tid=0 2024-05-14T09:42:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.880+0000 7f34ff1c5740 1 client.16392 shutdown 2024-05-14T09:42:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.880+0000 7f34ff1c5740 1 -- 172.21.15.2:0/30204381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d53432f0 msgr2=0x5639d571c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.880+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/30204381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d53432f0 0x5639d571c0b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0c5730 tx=0x7f34ec004570 comp rx=0 tx=0).stop 2024-05-14T09:42:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34ff1c5740 1 -- 172.21.15.2:0/30204381 shutdown_connections 2024-05-14T09:42:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/30204381 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5639d53001c0 0x5639d578f1a0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/30204381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639d536c1b0 0x5639d578b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34d97fa640 0 client.16392 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:42:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/30204381 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639d534b4d0 0x5639d571c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34ff1c5740 1 --2- 172.21.15.2:0/30204381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639d53432f0 0x5639d571c0b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34ff1c5740 1 -- 172.21.15.2:0/30204381 >> 172.21.15.2:0/30204381 conn(0x5639d5360300 msgr2=0x5639d5729680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34ff1c5740 1 -- 172.21.15.2:0/30204381 shutdown_connections 2024-05-14T09:42:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34ff1c5740 1 -- 172.21.15.2:0/30204381 wait complete. 2024-05-14T09:42:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34ff1c5740 20 client.16392 trim_cache size 0 max 16384 2024-05-14T09:42:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:42:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:42:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:42:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:42:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34f8234640 1 -- 172.21.15.2:0/1997241295 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f34dc0b28a0 msgr2=0x7f34dc0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34f8234640 1 --2- 172.21.15.2:0/1997241295 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f34dc0b28a0 0x7f34dc0b4db0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5639d570ec30 tx=0x7f34e80095c0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34f8234640 1 -- 172.21.15.2:0/1997241295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34f0104740 msgr2=0x7f34f019a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:42:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34f8234640 1 --2- 172.21.15.2:0/1997241295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34f0104740 0x7f34f019a2d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f34ec000c00 tx=0x7f34ec032d50 comp rx=0 tx=0).stop 2024-05-14T09:42:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34f8234640 1 -- 172.21.15.2:0/1997241295 shutdown_connections 2024-05-14T09:42:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34f8234640 1 --2- 172.21.15.2:0/1997241295 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f34dc0b28a0 0x7f34dc0b4db0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34f8234640 1 --2- 172.21.15.2:0/1997241295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f0109a00 0x7f34f01a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34f8234640 1 --2- 172.21.15.2:0/1997241295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f01070a0 0x7f34f019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34f8234640 1 --2- 172.21.15.2:0/1997241295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f34f0104740 0x7f34f019a2d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:42:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.881+0000 7f34f8234640 1 -- 172.21.15.2:0/1997241295 >> 172.21.15.2:0/1997241295 conn(0x7f34f00fdd40 msgr2=0x7f34f0105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:42:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.882+0000 7f34f8234640 1 -- 172.21.15.2:0/1997241295 shutdown_connections 2024-05-14T09:42:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:42:58.882+0000 7f34f8234640 1 -- 172.21.15.2:0/1997241295 wait complete. 2024-05-14T09:42:59.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 734794 10.16 MB/sec execute 265 sec latency 45.058 ms 2024-05-14T09:43:00.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 735519 10.14 MB/sec execute 266 sec latency 144.558 ms 2024-05-14T09:43:00.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:00 smithi002 ceph-mon[31557]: pgmap v665: 129 pgs: 129 active+clean; 136 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 2.4 MiB/s wr, 275 op/s 2024-05-14T09:43:00.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:00 smithi029 ceph-mon[36249]: pgmap v665: 129 pgs: 129 active+clean; 136 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 2.4 MiB/s wr, 275 op/s 2024-05-14T09:43:00.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:00 smithi026 ceph-mon[36056]: pgmap v665: 129 pgs: 129 active+clean; 136 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 2.4 MiB/s wr, 275 op/s 2024-05-14T09:43:01.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 737633 10.13 MB/sec execute 267 sec latency 15.068 ms 2024-05-14T09:43:02.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 739185 10.14 MB/sec execute 268 sec latency 12.336 ms 2024-05-14T09:43:02.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:02 smithi002 ceph-mon[31557]: pgmap v666: 129 pgs: 129 active+clean; 136 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.3 MiB/s wr, 240 op/s 2024-05-14T09:43:02.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:02 smithi029 ceph-mon[36249]: pgmap v666: 129 pgs: 129 active+clean; 136 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.3 MiB/s wr, 240 op/s 2024-05-14T09:43:02.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:02 smithi026 ceph-mon[36056]: pgmap v666: 129 pgs: 129 active+clean; 136 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.3 MiB/s wr, 240 op/s 2024-05-14T09:43:03.271 INFO:tasks.workunit.client.0.smithi002.stdout: 1 741595 10.15 MB/sec execute 269 sec latency 21.790 ms 2024-05-14T09:43:04.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 742996 10.13 MB/sec execute 270 sec latency 15.464 ms 2024-05-14T09:43:04.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:04 smithi002 ceph-mon[31557]: pgmap v667: 129 pgs: 129 active+clean; 135 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-05-14T09:43:04.690 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:04 smithi029 ceph-mon[36249]: pgmap v667: 129 pgs: 129 active+clean; 135 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-05-14T09:43:04.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:04 smithi026 ceph-mon[36056]: pgmap v667: 129 pgs: 129 active+clean; 135 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-05-14T09:43:05.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 745027 10.14 MB/sec execute 271 sec latency 56.142 ms 2024-05-14T09:43:06.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 746457 10.13 MB/sec execute 272 sec latency 25.868 ms 2024-05-14T09:43:06.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:06 smithi002 ceph-mon[31557]: pgmap v668: 129 pgs: 129 active+clean; 135 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-05-14T09:43:06.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:06 smithi029 ceph-mon[36249]: pgmap v668: 129 pgs: 129 active+clean; 135 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-05-14T09:43:06.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:06 smithi026 ceph-mon[36056]: pgmap v668: 129 pgs: 129 active+clean; 135 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-05-14T09:43:07.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 749039 10.14 MB/sec execute 273 sec latency 13.663 ms 2024-05-14T09:43:08.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 750403 10.13 MB/sec execute 274 sec latency 11.640 ms 2024-05-14T09:43:08.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:08 smithi002 ceph-mon[31557]: pgmap v669: 129 pgs: 129 active+clean; 135 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 2.6 MiB/s wr, 271 op/s 2024-05-14T09:43:08.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:08 smithi029 ceph-mon[36249]: pgmap v669: 129 pgs: 129 active+clean; 135 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 2.6 MiB/s wr, 271 op/s 2024-05-14T09:43:08.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:08 smithi026 ceph-mon[36056]: pgmap v669: 129 pgs: 129 active+clean; 135 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 2.6 MiB/s wr, 271 op/s 2024-05-14T09:43:09.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 752879 10.15 MB/sec execute 275 sec latency 20.021 ms 2024-05-14T09:43:10.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 753837 10.13 MB/sec execute 276 sec latency 53.095 ms 2024-05-14T09:43:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:10 smithi002 ceph-mon[31557]: pgmap v670: 129 pgs: 129 active+clean; 138 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-05-14T09:43:10.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:10 smithi029 ceph-mon[36249]: pgmap v670: 129 pgs: 129 active+clean; 138 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-05-14T09:43:10.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:10 smithi026 ceph-mon[36056]: pgmap v670: 129 pgs: 129 active+clean; 138 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-05-14T09:43:11.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 756131 10.14 MB/sec execute 277 sec latency 43.115 ms 2024-05-14T09:43:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.986+0000 7ffb6e089640 1 -- 172.21.15.2:0/1714844046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb68103c60 msgr2=0x7ffb68104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.986+0000 7ffb6e089640 1 --2- 172.21.15.2:0/1714844046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb68103c60 0x7ffb68104960 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009900 tx=0x7ffb5802a410 comp rx=0 tx=0).stop 2024-05-14T09:43:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.986+0000 7ffb6e089640 1 -- 172.21.15.2:0/1714844046 shutdown_connections 2024-05-14T09:43:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.986+0000 7ffb6e089640 1 --2- 172.21.15.2:0/1714844046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb68109ba0 0x7ffb6810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.986+0000 7ffb6e089640 1 --2- 172.21.15.2:0/1714844046 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb68104ea0 0x7ffb681052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.986+0000 7ffb6e089640 1 --2- 172.21.15.2:0/1714844046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb68103c60 0x7ffb68104960 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.986+0000 7ffb6e089640 1 -- 172.21.15.2:0/1714844046 >> 172.21.15.2:0/1714844046 conn(0x7ffb680fee30 msgr2=0x7ffb68101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.986+0000 7ffb6e089640 1 -- 172.21.15.2:0/1714844046 shutdown_connections 2024-05-14T09:43:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.987+0000 7ffb6e089640 1 -- 172.21.15.2:0/1714844046 wait complete. 2024-05-14T09:43:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.987+0000 7ffb6e089640 1 Processor -- start 2024-05-14T09:43:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.987+0000 7ffb6e089640 1 -- start start 2024-05-14T09:43:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.987+0000 7ffb6e089640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb68103c60 0x7ffb6819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.987+0000 7ffb6e089640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb68104ea0 0x7ffb6819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.987+0000 7ffb6e089640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb68109ba0 0x7ffb681a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.987+0000 7ffb6e089640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6810f740 con 0x7ffb68104ea0 2024-05-14T09:43:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.987+0000 7ffb6e089640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6810f5c0 con 0x7ffb68109ba0 2024-05-14T09:43:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.987+0000 7ffb6e089640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6810f8c0 con 0x7ffb68103c60 2024-05-14T09:43:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb5ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb68104ea0 0x7ffb6819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb677fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb68103c60 0x7ffb6819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb5ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb68104ea0 0x7ffb6819a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38772/0 (socket says 172.21.15.2:38772) 2024-05-14T09:43:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb5ffff640 1 -- 172.21.15.2:0/1342710750 learned_addr learned my addr 172.21.15.2:0/1342710750 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb67fff640 1 --2- 172.21.15.2:0/1342710750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb68109ba0 0x7ffb681a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb5ffff640 1 -- 172.21.15.2:0/1342710750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb68103c60 msgr2=0x7ffb6819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb5ffff640 1 --2- 172.21.15.2:0/1342710750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb68103c60 0x7ffb6819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb5ffff640 1 -- 172.21.15.2:0/1342710750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb68109ba0 msgr2=0x7ffb681a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb5ffff640 1 --2- 172.21.15.2:0/1342710750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb68109ba0 0x7ffb681a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb5ffff640 1 -- 172.21.15.2:0/1342710750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb681a1df0 con 0x7ffb68104ea0 2024-05-14T09:43:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb5ffff640 1 --2- 172.21.15.2:0/1342710750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb68104ea0 0x7ffb6819a7f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000d950 tx=0x7ffb5000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb657fa640 1 -- 172.21.15.2:0/1342710750 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50017070 con 0x7ffb68104ea0 2024-05-14T09:43:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb6e089640 1 -- 172.21.15.2:0/1342710750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb681a20e0 con 0x7ffb68104ea0 2024-05-14T09:43:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb657fa640 1 -- 172.21.15.2:0/1342710750 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb5000de80 con 0x7ffb68104ea0 2024-05-14T09:43:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb657fa640 1 -- 172.21.15.2:0/1342710750 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5000f620 con 0x7ffb68104ea0 2024-05-14T09:43:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.988+0000 7ffb6e089640 1 -- 172.21.15.2:0/1342710750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb681a2680 con 0x7ffb68104ea0 2024-05-14T09:43:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.991+0000 7ffb75019740 1 Processor -- start 2024-05-14T09:43:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.991+0000 7ffb75019740 1 -- start start 2024-05-14T09:43:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb75019740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c17a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb75019740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 0x560fe7c17fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb75019740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe78554d0 0x560fe7c1eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb75019740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560fe7b7c5d0 con 0x560fe78761b0 2024-05-14T09:43:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb75019740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560fe7b35890 con 0x560fe784d2f0 2024-05-14T09:43:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb75019740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560fe7b78590 con 0x560fe78554d0 2024-05-14T09:43:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb657fa640 1 -- 172.21.15.2:0/1342710750 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ffb5000f7c0 con 0x7ffb68104ea0 2024-05-14T09:43:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb67fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c17a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb67fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c17a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38774/0 (socket says 172.21.15.2:38774) 2024-05-14T09:43:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb67fff640 1 -- 172.21.15.2:0/2729440415 learned_addr learned my addr 172.21.15.2:0/2729440415 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb677fe640 1 --2- 172.21.15.2:0/2729440415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 0x560fe7c17fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb5ffff640 1 --2- 172.21.15.2:0/2729440415 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe78554d0 0x560fe7c1eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb67fff640 1 -- 172.21.15.2:0/2729440415 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe78554d0 msgr2=0x560fe7c1eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb67fff640 1 --2- 172.21.15.2:0/2729440415 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe78554d0 0x560fe7c1eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb67fff640 1 -- 172.21.15.2:0/2729440415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 msgr2=0x560fe7c17fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb67fff640 1 --2- 172.21.15.2:0/2729440415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 0x560fe7c17fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb67fff640 1 -- 172.21.15.2:0/2729440415 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fe788e210 con 0x560fe78761b0 2024-05-14T09:43:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb677fe640 1 --2- 172.21.15.2:0/2729440415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 0x560fe7c17fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:43:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.992+0000 7ffb67fff640 1 --2- 172.21.15.2:0/2729440415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c17a60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ffb60011440 tx=0x7ffb60011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f183d2624622be server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.993+0000 7ffb5dffb640 1 -- 172.21.15.2:0/2729440415 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60003c30 con 0x560fe78761b0 2024-05-14T09:43:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.993+0000 7ffb5dffb640 1 -- 172.21.15.2:0/2729440415 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb60003dd0 con 0x560fe78761b0 2024-05-14T09:43:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.993+0000 7ffb5dffb640 1 -- 172.21.15.2:0/2729440415 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000fa80 con 0x560fe78761b0 2024-05-14T09:43:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.993+0000 7ffb657fa640 1 --2- 172.21.15.2:0/1342710750 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffb540b28a0 0x7ffb540b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.993+0000 7ffb677fe640 1 --2- 172.21.15.2:0/1342710750 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffb540b28a0 0x7ffb540b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.993+0000 7ffb657fa640 1 -- 172.21.15.2:0/1342710750 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7ffb500e2220 con 0x7ffb68104ea0 2024-05-14T09:43:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.994+0000 7ffb75019740 1 -- 172.21.15.2:0/2729440415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 msgr2=0x560fe7c17a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.994+0000 7ffb677fe640 1 --2- 172.21.15.2:0/1342710750 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffb540b28a0 0x7ffb540b4db0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffb58005ab0 tx=0x7ffb58031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.994+0000 7ffb75019740 1 --2- 172.21.15.2:0/2729440415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c17a60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ffb60011440 tx=0x7ffb60011900 comp rx=0 tx=0).stop 2024-05-14T09:43:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.994+0000 7ffb75019740 1 -- 172.21.15.2:0/2729440415 shutdown_connections 2024-05-14T09:43:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.994+0000 7ffb75019740 1 --2- 172.21.15.2:0/2729440415 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe78554d0 0x560fe7c1eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.994+0000 7ffb75019740 1 --2- 172.21.15.2:0/2729440415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 0x560fe7c17fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.994+0000 7ffb75019740 1 --2- 172.21.15.2:0/2729440415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c17a60 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.994+0000 7ffb75019740 1 -- 172.21.15.2:0/2729440415 >> 172.21.15.2:0/2729440415 conn(0x560fe786a300 msgr2=0x560fe7c133a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.994+0000 7ffb75019740 1 -- 172.21.15.2:0/2729440415 shutdown_connections 2024-05-14T09:43:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.994+0000 7ffb75019740 1 -- 172.21.15.2:0/2729440415 wait complete. 2024-05-14T09:43:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.994+0000 7ffb75019740 1 -- start start 2024-05-14T09:43:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.994+0000 7ffb75019740 1 Processor -- start 2024-05-14T09:43:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb75019740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 0x560fe7c21730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb75019740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe78554d0 0x560fe7c21c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb75019740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c28bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb75019740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560fe7c1f930 con 0x560fe78761b0 2024-05-14T09:43:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb75019740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560fe7b39370 con 0x560fe784d2f0 2024-05-14T09:43:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb75019740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560fe7c1fab0 con 0x560fe78554d0 2024-05-14T09:43:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb75019740 10 client.0 _ll_register_callbacks cb 0x560fe788da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:43:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb67fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c28bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb67fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c28bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38790/0 (socket says 172.21.15.2:38790) 2024-05-14T09:43:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb67fff640 1 -- 172.21.15.2:0/1882401833 learned_addr learned my addr 172.21.15.2:0/1882401833 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb677fe640 1 --2- 172.21.15.2:0/1882401833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 0x560fe7c21730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb5ffff640 1 --2- 172.21.15.2:0/1882401833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe78554d0 0x560fe7c21c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb67fff640 1 -- 172.21.15.2:0/1882401833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe78554d0 msgr2=0x560fe7c21c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb67fff640 1 --2- 172.21.15.2:0/1882401833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe78554d0 0x560fe7c21c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb67fff640 1 -- 172.21.15.2:0/1882401833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 msgr2=0x560fe7c21730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb67fff640 1 --2- 172.21.15.2:0/1882401833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 0x560fe7c21730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb67fff640 1 -- 172.21.15.2:0/1882401833 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fe795a620 con 0x560fe78761b0 2024-05-14T09:43:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb677fe640 1 --2- 172.21.15.2:0/1882401833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 0x560fe7c21730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:43:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb67fff640 1 --2- 172.21.15.2:0/1882401833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c28bd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ffb60004d70 tx=0x7ffb60004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a894b15151ba6e86 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.995+0000 7ffb46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:43:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.996+0000 7ffb46ffd640 1 -- 172.21.15.2:0/1882401833 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60029020 con 0x560fe78761b0 2024-05-14T09:43:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.996+0000 7ffb46ffd640 1 -- 172.21.15.2:0/1882401833 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7ffb6000fc30 con 0x560fe78761b0 2024-05-14T09:43:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.996+0000 7ffb75019740 1 -- 172.21.15.2:0/1882401833 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb60003dd0 con 0x560fe78761b0 2024-05-14T09:43:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.996+0000 7ffb46ffd640 1 -- 172.21.15.2:0/1882401833 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ffb60034050 con 0x560fe78761b0 2024-05-14T09:43:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.996+0000 7ffb75019740 10 client.16404 fetch_fsmap learned FSMap version 36 2024-05-14T09:43:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.996+0000 7ffb75019740 1 -- 172.21.15.2:0/1882401833 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560fe7c2bb70 con 0x560fe78761b0 2024-05-14T09:43:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.996+0000 7ffb46ffd640 1 -- 172.21.15.2:0/1882401833 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb60003dd0 con 0x560fe78761b0 2024-05-14T09:43:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.996+0000 7ffb46ffd640 1 -- 172.21.15.2:0/1882401833 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7ffb60005040 con 0x560fe78761b0 2024-05-14T09:43:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.996+0000 7ffb75019740 10 client.16404 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T09:43:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.996+0000 7ffb75019740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:43:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.996+0000 7ffb75019740 20 client.16404 populate_metadata read hostname 'smithi002' 2024-05-14T09:43:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.996+0000 7ffb75019740 1 --2- 172.21.15.2:0/1882401833 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fe7834d80 0x560fe7c2f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.996+0000 7ffb75019740 4 client.16404 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:43:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.996+0000 7ffb75019740 1 -- 172.21.15.2:0/1882401833 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fe7b8a1e0 con 0x560fe7834d80 2024-05-14T09:43:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.996+0000 7ffb677fe640 1 --2- 172.21.15.2:0/1882401833 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fe7834d80 0x560fe7c2f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.997+0000 7ffb677fe640 1 --2- 172.21.15.2:0/1882401833 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fe7834d80 0x560fe7c2f0f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d34e1c570d181f62 server_cookie=7e89b12509752f7 in_seq=0 out_seq=0 2024-05-14T09:43:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.997+0000 7ffb46ffd640 10 client.16404 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.998+0000 7ffb46ffd640 1 -- 172.21.15.2:0/1882401833 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560fe7b8a1e0 con 0x560fe7834d80 2024-05-14T09:43:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.998+0000 7ffb46ffd640 10 client.16404 handle_command_reply: tid=0 2024-05-14T09:43:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.998+0000 7ffb75019740 1 client.16404 shutdown 2024-05-14T09:43:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.998+0000 7ffb75019740 1 -- 172.21.15.2:0/1882401833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 msgr2=0x560fe7c28bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.998+0000 7ffb75019740 1 --2- 172.21.15.2:0/1882401833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c28bd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ffb60004d70 tx=0x7ffb60004da0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.998+0000 7ffb75019740 1 -- 172.21.15.2:0/1882401833 shutdown_connections 2024-05-14T09:43:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.998+0000 7ffb75019740 1 --2- 172.21.15.2:0/1882401833 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fe7834d80 0x560fe7c2f0f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.998+0000 7ffb75019740 1 --2- 172.21.15.2:0/1882401833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c28bd0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.998+0000 7ffb75019740 1 --2- 172.21.15.2:0/1882401833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe78554d0 0x560fe7c21c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.998+0000 7ffb46ffd640 0 client.16404 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.998+0000 7ffb75019740 1 --2- 172.21.15.2:0/1882401833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 0x560fe7c21730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.998+0000 7ffb75019740 1 -- 172.21.15.2:0/1882401833 >> 172.21.15.2:0/1882401833 conn(0x560fe786a300 msgr2=0x560fe7c123a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.998+0000 7ffb75019740 1 -- 172.21.15.2:0/1882401833 shutdown_connections 2024-05-14T09:43:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.998+0000 7ffb75019740 1 -- 172.21.15.2:0/1882401833 wait complete. 2024-05-14T09:43:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:11.998+0000 7ffb75019740 20 client.16404 trim_cache size 0 max 16384 2024-05-14T09:43:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb75019740 1 Processor -- start 2024-05-14T09:43:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb75019740 1 -- start start 2024-05-14T09:43:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb75019740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 0x560fe7c21730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb75019740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78554d0 0x560fe7c21c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb75019740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe78761b0 0x560fe7c12cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb75019740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681af540 con 0x560fe78554d0 2024-05-14T09:43:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb75019740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681ace10 con 0x560fe784d2f0 2024-05-14T09:43:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb75019740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681aac60 con 0x560fe78761b0 2024-05-14T09:43:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb5ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78554d0 0x560fe7c21c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb5ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78554d0 0x560fe7c21c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38798/0 (socket says 172.21.15.2:38798) 2024-05-14T09:43:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb5ffff640 1 -- 172.21.15.2:0/2127843188 learned_addr learned my addr 172.21.15.2:0/2127843188 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb677fe640 1 --2- 172.21.15.2:0/2127843188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 0x560fe7c21730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb67fff640 1 --2- 172.21.15.2:0/2127843188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe78761b0 0x560fe7c12cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb5ffff640 1 -- 172.21.15.2:0/2127843188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe78761b0 msgr2=0x560fe7c12cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb5ffff640 1 --2- 172.21.15.2:0/2127843188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe78761b0 0x560fe7c12cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb5ffff640 1 -- 172.21.15.2:0/2127843188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 msgr2=0x560fe7c21730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb5ffff640 1 --2- 172.21.15.2:0/2127843188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 0x560fe7c21730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.015+0000 7ffb5ffff640 1 -- 172.21.15.2:0/2127843188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fe7c13d00 con 0x560fe78554d0 2024-05-14T09:43:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.016+0000 7ffb5ffff640 1 --2- 172.21.15.2:0/2127843188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78554d0 0x560fe7c21c70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ffb500a54a0 tx=0x7ffb50000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5936194778944746 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.016+0000 7ffb5d7fa640 1 -- 172.21.15.2:0/2127843188 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50020040 con 0x560fe78554d0 2024-05-14T09:43:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.016+0000 7ffb5d7fa640 1 -- 172.21.15.2:0/2127843188 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb500a8f30 con 0x560fe78554d0 2024-05-14T09:43:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.016+0000 7ffb5d7fa640 1 -- 172.21.15.2:0/2127843188 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb500ad7d0 con 0x560fe78554d0 2024-05-14T09:43:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.017+0000 7ffb75019740 1 -- 172.21.15.2:0/2127843188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78554d0 msgr2=0x560fe7c21c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.017+0000 7ffb75019740 1 --2- 172.21.15.2:0/2127843188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78554d0 0x560fe7c21c70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ffb500a54a0 tx=0x7ffb50000bc0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.017+0000 7ffb75019740 1 -- 172.21.15.2:0/2127843188 shutdown_connections 2024-05-14T09:43:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.017+0000 7ffb75019740 1 --2- 172.21.15.2:0/2127843188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe78761b0 0x560fe7c12cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.017+0000 7ffb75019740 1 --2- 172.21.15.2:0/2127843188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78554d0 0x560fe7c21c70 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.017+0000 7ffb75019740 1 --2- 172.21.15.2:0/2127843188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe784d2f0 0x560fe7c21730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.017+0000 7ffb75019740 1 -- 172.21.15.2:0/2127843188 >> 172.21.15.2:0/2127843188 conn(0x560fe786a300 msgr2=0x560fe7c1bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.017+0000 7ffb75019740 1 -- 172.21.15.2:0/2127843188 shutdown_connections 2024-05-14T09:43:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.017+0000 7ffb75019740 1 -- 172.21.15.2:0/2127843188 wait complete. 2024-05-14T09:43:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.017+0000 7ffb75019740 1 -- start start 2024-05-14T09:43:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.017+0000 7ffb75019740 1 Processor -- start 2024-05-14T09:43:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb75019740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe784d2f0 0x560fe7c25390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb75019740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe78554d0 0x560fe7c258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb75019740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c26800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb75019740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681ab2d0 con 0x560fe78761b0 2024-05-14T09:43:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb75019740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681ac740 con 0x560fe78554d0 2024-05-14T09:43:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb75019740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681ab9a0 con 0x560fe784d2f0 2024-05-14T09:43:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb75019740 10 client.0 _ll_register_callbacks cb 0x7ffb58009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:43:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb67fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c26800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb67fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c26800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38802/0 (socket says 172.21.15.2:38802) 2024-05-14T09:43:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb677fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe784d2f0 0x560fe7c25390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb67fff640 1 -- 172.21.15.2:0/6302444 learned_addr learned my addr 172.21.15.2:0/6302444 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb5ffff640 1 --2- 172.21.15.2:0/6302444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe78554d0 0x560fe7c258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb67fff640 1 -- 172.21.15.2:0/6302444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe784d2f0 msgr2=0x560fe7c25390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb67fff640 1 --2- 172.21.15.2:0/6302444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe784d2f0 0x560fe7c25390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb67fff640 1 -- 172.21.15.2:0/6302444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe78554d0 msgr2=0x560fe7c258d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb67fff640 1 --2- 172.21.15.2:0/6302444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe78554d0 0x560fe7c258d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb67fff640 1 -- 172.21.15.2:0/6302444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fe7c26dc0 con 0x560fe78761b0 2024-05-14T09:43:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb67fff640 1 --2- 172.21.15.2:0/6302444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c26800 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009560 tx=0x7ffb60005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfac33e686590793 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:43:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb46ffd640 1 -- 172.21.15.2:0/6302444 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6003c070 con 0x560fe78761b0 2024-05-14T09:43:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb46ffd640 1 -- 172.21.15.2:0/6302444 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7ffb60005680 con 0x560fe78761b0 2024-05-14T09:43:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.018+0000 7ffb75019740 1 -- 172.21.15.2:0/6302444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb500a8f30 con 0x560fe78761b0 2024-05-14T09:43:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.019+0000 7ffb46ffd640 1 -- 172.21.15.2:0/6302444 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ffb60044050 con 0x560fe78761b0 2024-05-14T09:43:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.019+0000 7ffb75019740 10 client.16410 fetch_fsmap learned FSMap version 36 2024-05-14T09:43:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.019+0000 7ffb75019740 1 -- 172.21.15.2:0/6302444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560fe7c28050 con 0x560fe78761b0 2024-05-14T09:43:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.019+0000 7ffb46ffd640 1 -- 172.21.15.2:0/6302444 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb500a8f30 con 0x560fe78761b0 2024-05-14T09:43:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.019+0000 7ffb46ffd640 1 -- 172.21.15.2:0/6302444 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7ffb6003f030 con 0x560fe78761b0 2024-05-14T09:43:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.019+0000 7ffb75019740 10 client.16410 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T09:43:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.019+0000 7ffb75019740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:43:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.019+0000 7ffb75019740 20 client.16410 populate_metadata read hostname 'smithi002' 2024-05-14T09:43:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.019+0000 7ffb75019740 1 --2- 172.21.15.2:0/6302444 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fe780a1c0 0x560fe7c9db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.019+0000 7ffb75019740 4 client.16410 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:43:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.019+0000 7ffb75019740 1 -- 172.21.15.2:0/6302444 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560fe7b84050 con 0x560fe780a1c0 2024-05-14T09:43:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.019+0000 7ffb677fe640 1 --2- 172.21.15.2:0/6302444 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fe780a1c0 0x560fe7c9db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.020+0000 7ffb46ffd640 10 client.16410 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.020+0000 7ffb677fe640 1 --2- 172.21.15.2:0/6302444 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fe780a1c0 0x560fe7c9db90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1b84f270fca8959 server_cookie=cc8f2e31660a1120 in_seq=0 out_seq=0 2024-05-14T09:43:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.116+0000 7ffb46ffd640 1 -- 172.21.15.2:0/6302444 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560fe7b84050 con 0x560fe780a1c0 2024-05-14T09:43:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.116+0000 7ffb46ffd640 10 client.16410 handle_command_reply: tid=0 2024-05-14T09:43:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.116+0000 7ffb75019740 1 client.16410 shutdown 2024-05-14T09:43:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.117+0000 7ffb75019740 1 -- 172.21.15.2:0/6302444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 msgr2=0x560fe7c26800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.117+0000 7ffb75019740 1 --2- 172.21.15.2:0/6302444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c26800 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009560 tx=0x7ffb60005c60 comp rx=0 tx=0).stop 2024-05-14T09:43:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.117+0000 7ffb75019740 1 -- 172.21.15.2:0/6302444 shutdown_connections 2024-05-14T09:43:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.117+0000 7ffb75019740 1 --2- 172.21.15.2:0/6302444 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fe780a1c0 0x560fe7c9db90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.117+0000 7ffb75019740 1 --2- 172.21.15.2:0/6302444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fe78761b0 0x560fe7c26800 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.117+0000 7ffb75019740 1 --2- 172.21.15.2:0/6302444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fe78554d0 0x560fe7c258d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.117+0000 7ffb75019740 1 --2- 172.21.15.2:0/6302444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fe784d2f0 0x560fe7c25390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.117+0000 7ffb46ffd640 0 client.16410 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.117+0000 7ffb75019740 1 -- 172.21.15.2:0/6302444 >> 172.21.15.2:0/6302444 conn(0x560fe786a300 msgr2=0x560fe7c21730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.117+0000 7ffb75019740 1 -- 172.21.15.2:0/6302444 shutdown_connections 2024-05-14T09:43:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.117+0000 7ffb75019740 1 -- 172.21.15.2:0/6302444 wait complete. 2024-05-14T09:43:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.117+0000 7ffb75019740 20 client.16410 trim_cache size 0 max 16384 2024-05-14T09:43:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:43:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:43:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:43:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:43:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.118+0000 7ffb6e089640 1 -- 172.21.15.2:0/1342710750 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffb540b28a0 msgr2=0x7ffb540b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.118+0000 7ffb6e089640 1 --2- 172.21.15.2:0/1342710750 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffb540b28a0 0x7ffb540b4db0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffb58005ab0 tx=0x7ffb58031400 comp rx=0 tx=0).stop 2024-05-14T09:43:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.118+0000 7ffb6e089640 1 -- 172.21.15.2:0/1342710750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb68104ea0 msgr2=0x7ffb6819a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.118+0000 7ffb6e089640 1 --2- 172.21.15.2:0/1342710750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb68104ea0 0x7ffb6819a7f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000d950 tx=0x7ffb5000dd50 comp rx=0 tx=0).stop 2024-05-14T09:43:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.118+0000 7ffb6e089640 1 -- 172.21.15.2:0/1342710750 shutdown_connections 2024-05-14T09:43:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.118+0000 7ffb6e089640 1 --2- 172.21.15.2:0/1342710750 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ffb540b28a0 0x7ffb540b4db0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.118+0000 7ffb6e089640 1 --2- 172.21.15.2:0/1342710750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb68109ba0 0x7ffb681a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.118+0000 7ffb6e089640 1 --2- 172.21.15.2:0/1342710750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb68104ea0 0x7ffb6819a7f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.118+0000 7ffb6e089640 1 --2- 172.21.15.2:0/1342710750 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb68103c60 0x7ffb6819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.118+0000 7ffb6e089640 1 -- 172.21.15.2:0/1342710750 >> 172.21.15.2:0/1342710750 conn(0x7ffb680fee30 msgr2=0x7ffb68105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.118+0000 7ffb6e089640 1 -- 172.21.15.2:0/1342710750 shutdown_connections 2024-05-14T09:43:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:12.118+0000 7ffb6e089640 1 -- 172.21.15.2:0/1342710750 wait complete. 2024-05-14T09:43:12.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 757260 10.12 MB/sec execute 278 sec latency 46.695 ms 2024-05-14T09:43:12.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:12 smithi002 ceph-mon[31557]: pgmap v671: 129 pgs: 129 active+clean; 138 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.4 MiB/s wr, 261 op/s 2024-05-14T09:43:12.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:12 smithi029 ceph-mon[36249]: pgmap v671: 129 pgs: 129 active+clean; 138 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.4 MiB/s wr, 261 op/s 2024-05-14T09:43:12.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:12 smithi026 ceph-mon[36056]: pgmap v671: 129 pgs: 129 active+clean; 138 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.4 MiB/s wr, 261 op/s 2024-05-14T09:43:13.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 759806 10.14 MB/sec execute 279 sec latency 14.221 ms 2024-05-14T09:43:14.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 761251 10.13 MB/sec execute 280 sec latency 11.316 ms 2024-05-14T09:43:14.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:14 smithi002 ceph-mon[31557]: pgmap v672: 129 pgs: 129 active+clean; 137 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 365 op/s 2024-05-14T09:43:14.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:14 smithi029 ceph-mon[36249]: pgmap v672: 129 pgs: 129 active+clean; 137 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 365 op/s 2024-05-14T09:43:14.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:14 smithi026 ceph-mon[36056]: pgmap v672: 129 pgs: 129 active+clean; 137 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 365 op/s 2024-05-14T09:43:15.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 763555 10.14 MB/sec execute 281 sec latency 52.674 ms 2024-05-14T09:43:16.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 764915 10.13 MB/sec execute 282 sec latency 37.110 ms 2024-05-14T09:43:16.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:16 smithi002 ceph-mon[31557]: pgmap v673: 129 pgs: 129 active+clean; 137 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 2.4 MiB/s wr, 242 op/s 2024-05-14T09:43:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:16 smithi029 ceph-mon[36249]: pgmap v673: 129 pgs: 129 active+clean; 137 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 2.4 MiB/s wr, 242 op/s 2024-05-14T09:43:16.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:16 smithi026 ceph-mon[36056]: pgmap v673: 129 pgs: 129 active+clean; 137 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 2.4 MiB/s wr, 242 op/s 2024-05-14T09:43:17.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 767240 10.14 MB/sec execute 283 sec latency 22.197 ms 2024-05-14T09:43:18.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 769296 10.13 MB/sec execute 284 sec latency 11.538 ms 2024-05-14T09:43:18.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:18 smithi002 ceph-mon[31557]: pgmap v674: 129 pgs: 129 active+clean; 138 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 319 op/s 2024-05-14T09:43:18.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:18 smithi029 ceph-mon[36249]: pgmap v674: 129 pgs: 129 active+clean; 138 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 319 op/s 2024-05-14T09:43:18.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:18 smithi026 ceph-mon[36056]: pgmap v674: 129 pgs: 129 active+clean; 138 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 319 op/s 2024-05-14T09:43:19.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 770945 10.14 MB/sec execute 285 sec latency 13.447 ms 2024-05-14T09:43:20.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 772929 10.13 MB/sec execute 286 sec latency 53.592 ms 2024-05-14T09:43:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:20 smithi002 ceph-mon[31557]: pgmap v675: 129 pgs: 129 active+clean; 140 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-05-14T09:43:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:20 smithi029 ceph-mon[36249]: pgmap v675: 129 pgs: 129 active+clean; 140 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-05-14T09:43:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:20 smithi026 ceph-mon[36056]: pgmap v675: 129 pgs: 129 active+clean; 140 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-05-14T09:43:21.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 774494 10.13 MB/sec execute 287 sec latency 36.700 ms 2024-05-14T09:43:22.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 776644 10.12 MB/sec execute 288 sec latency 13.949 ms 2024-05-14T09:43:22.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:22 smithi002 ceph-mon[31557]: pgmap v676: 129 pgs: 129 active+clean; 140 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-05-14T09:43:22.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:22 smithi029 ceph-mon[36249]: pgmap v676: 129 pgs: 129 active+clean; 140 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-05-14T09:43:22.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:22 smithi026 ceph-mon[36056]: pgmap v676: 129 pgs: 129 active+clean; 140 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-05-14T09:43:23.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 778312 10.13 MB/sec execute 289 sec latency 22.522 ms 2024-05-14T09:43:24.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 780947 10.14 MB/sec execute 290 sec latency 12.404 ms 2024-05-14T09:43:24.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:24 smithi002 ceph-mon[31557]: pgmap v677: 129 pgs: 129 active+clean; 143 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 373 op/s 2024-05-14T09:43:24.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:24 smithi029 ceph-mon[36249]: pgmap v677: 129 pgs: 129 active+clean; 143 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 373 op/s 2024-05-14T09:43:24.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:24 smithi026 ceph-mon[36056]: pgmap v677: 129 pgs: 129 active+clean; 143 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 373 op/s 2024-05-14T09:43:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.252+0000 7f8a84fc7640 1 -- 172.21.15.2:0/546684534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a80102380 msgr2=0x7f8a8010bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.252+0000 7f8a84fc7640 1 --2- 172.21.15.2:0/546684534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a80102380 0x7f8a8010bde0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009ca0 tx=0x7f8a7802a8c0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.252+0000 7f8a84fc7640 1 -- 172.21.15.2:0/546684534 shutdown_connections 2024-05-14T09:43:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.253+0000 7f8a84fc7640 1 --2- 172.21.15.2:0/546684534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a80102380 0x7f8a8010bde0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.253+0000 7f8a84fc7640 1 --2- 172.21.15.2:0/546684534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a80101880 0x7f8a80101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.253+0000 7f8a84fc7640 1 --2- 172.21.15.2:0/546684534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a801051d0 0x7f8a801055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.253+0000 7f8a84fc7640 1 -- 172.21.15.2:0/546684534 >> 172.21.15.2:0/546684534 conn(0x7f8a800fd480 msgr2=0x7f8a800ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.253+0000 7f8a84fc7640 1 -- 172.21.15.2:0/546684534 shutdown_connections 2024-05-14T09:43:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.253+0000 7f8a84fc7640 1 -- 172.21.15.2:0/546684534 wait complete. 2024-05-14T09:43:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.253+0000 7f8a84fc7640 1 Processor -- start 2024-05-14T09:43:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.253+0000 7f8a84fc7640 1 -- start start 2024-05-14T09:43:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a84fc7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a80101880 0x7f8a8019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a84fc7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a80102380 0x7f8a8019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a84fc7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801051d0 0x7f8a801a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a84fc7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8010f790 con 0x7f8a801051d0 2024-05-14T09:43:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a84fc7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8010f610 con 0x7f8a80101880 2024-05-14T09:43:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a84fc7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8010f910 con 0x7f8a80102380 2024-05-14T09:43:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a7ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801051d0 0x7f8a801a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a7e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a80101880 0x7f8a8019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a7ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801051d0 0x7f8a801a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50184/0 (socket says 172.21.15.2:50184) 2024-05-14T09:43:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a7ed9d640 1 -- 172.21.15.2:0/2405684112 learned_addr learned my addr 172.21.15.2:0/2405684112 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a77fff640 1 --2- 172.21.15.2:0/2405684112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a80102380 0x7f8a8019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a7e59c640 1 -- 172.21.15.2:0/2405684112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a80102380 msgr2=0x7f8a8019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a7e59c640 1 --2- 172.21.15.2:0/2405684112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a80102380 0x7f8a8019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a7e59c640 1 -- 172.21.15.2:0/2405684112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801051d0 msgr2=0x7f8a801a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a7e59c640 1 --2- 172.21.15.2:0/2405684112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801051d0 0x7f8a801a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a7e59c640 1 -- 172.21.15.2:0/2405684112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a801a1cb0 con 0x7f8a80101880 2024-05-14T09:43:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a7ed9d640 1 --2- 172.21.15.2:0/2405684112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801051d0 0x7f8a801a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:43:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.254+0000 7f8a7e59c640 1 --2- 172.21.15.2:0/2405684112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a80101880 0x7f8a8019a220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a8006a590 tx=0x7f8a7000cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.255+0000 7f8a777fe640 1 -- 172.21.15.2:0/2405684112 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70007bb0 con 0x7f8a80101880 2024-05-14T09:43:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.255+0000 7f8a777fe640 1 -- 172.21.15.2:0/2405684112 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a70007d50 con 0x7f8a80101880 2024-05-14T09:43:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.255+0000 7f8a84fc7640 1 -- 172.21.15.2:0/2405684112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a801a1fa0 con 0x7f8a80101880 2024-05-14T09:43:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.255+0000 7f8a777fe640 1 -- 172.21.15.2:0/2405684112 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a700186f0 con 0x7f8a80101880 2024-05-14T09:43:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.255+0000 7f8a84fc7640 1 -- 172.21.15.2:0/2405684112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a801a2400 con 0x7f8a80101880 2024-05-14T09:43:25.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.257+0000 7f8a8be7b740 1 Processor -- start 2024-05-14T09:43:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.257+0000 7f8a8be7b740 1 -- start start 2024-05-14T09:43:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a8be7b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca3f1b0 0x55d7acde0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a8be7b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7aca162f0 0x55d7acde0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a8be7b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca1e4d0 0x55d7acde7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a8be7b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d7acd455d0 con 0x55d7aca3f1b0 2024-05-14T09:43:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a8be7b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d7accfe890 con 0x55d7aca162f0 2024-05-14T09:43:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a8be7b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d7acd41590 con 0x55d7aca1e4d0 2024-05-14T09:43:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a7ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca3f1b0 0x55d7acde0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a7ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca3f1b0 0x55d7acde0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50194/0 (socket says 172.21.15.2:50194) 2024-05-14T09:43:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a7ed9d640 1 -- 172.21.15.2:0/3179991816 learned_addr learned my addr 172.21.15.2:0/3179991816 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a77fff640 1 --2- 172.21.15.2:0/3179991816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca1e4d0 0x55d7acde7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a777fe640 1 -- 172.21.15.2:0/2405684112 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8a7000f990 con 0x7f8a80101880 2024-05-14T09:43:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a7ed9d640 1 -- 172.21.15.2:0/3179991816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca1e4d0 msgr2=0x55d7acde7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a7ed9d640 1 --2- 172.21.15.2:0/3179991816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca1e4d0 0x55d7acde7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a7ed9d640 1 -- 172.21.15.2:0/3179991816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7aca162f0 msgr2=0x55d7acde0fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:43:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a7ed9d640 1 --2- 172.21.15.2:0/3179991816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7aca162f0 0x55d7acde0fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a7ed9d640 1 -- 172.21.15.2:0/3179991816 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7aca57210 con 0x55d7aca3f1b0 2024-05-14T09:43:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.258+0000 7f8a7ed9d640 1 --2- 172.21.15.2:0/3179991816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca3f1b0 0x55d7acde0a60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009830 tx=0x7f8a78009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e7b70cffa8d13 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.259+0000 7f8a74ff9640 1 -- 172.21.15.2:0/3179991816 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a780316d0 con 0x55d7aca3f1b0 2024-05-14T09:43:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.259+0000 7f8a74ff9640 1 -- 172.21.15.2:0/3179991816 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a78036040 con 0x55d7aca3f1b0 2024-05-14T09:43:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.259+0000 7f8a74ff9640 1 -- 172.21.15.2:0/3179991816 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7802f610 con 0x55d7aca3f1b0 2024-05-14T09:43:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.259+0000 7f8a777fe640 1 --2- 172.21.15.2:0/2405684112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8a6c0b28a0 0x7f8a6c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.259+0000 7f8a7e59c640 1 --2- 172.21.15.2:0/2405684112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8a6c0b28a0 0x7f8a6c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.259+0000 7f8a777fe640 1 -- 172.21.15.2:0/2405684112 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f8a700e2690 con 0x7f8a80101880 2024-05-14T09:43:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.260+0000 7f8a7e59c640 1 --2- 172.21.15.2:0/2405684112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8a6c0b28a0 0x7f8a6c0b4db0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8a700a9820 tx=0x7f8a700d8ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.260+0000 7f8a8be7b740 1 -- 172.21.15.2:0/3179991816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca3f1b0 msgr2=0x55d7acde0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.260+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/3179991816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca3f1b0 0x55d7acde0a60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009830 tx=0x7f8a78009860 comp rx=0 tx=0).stop 2024-05-14T09:43:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.260+0000 7f8a8be7b740 1 -- 172.21.15.2:0/3179991816 shutdown_connections 2024-05-14T09:43:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.260+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/3179991816 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca1e4d0 0x55d7acde7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.260+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/3179991816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7aca162f0 0x55d7acde0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.260+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/3179991816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca3f1b0 0x55d7acde0a60 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.260+0000 7f8a8be7b740 1 -- 172.21.15.2:0/3179991816 >> 172.21.15.2:0/3179991816 conn(0x55d7aca33300 msgr2=0x55d7acddc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.260+0000 7f8a8be7b740 1 -- 172.21.15.2:0/3179991816 shutdown_connections 2024-05-14T09:43:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.260+0000 7f8a8be7b740 1 -- 172.21.15.2:0/3179991816 wait complete. 2024-05-14T09:43:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.260+0000 7f8a8be7b740 1 -- start start 2024-05-14T09:43:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.260+0000 7f8a8be7b740 1 Processor -- start 2024-05-14T09:43:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a8be7b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca162f0 0x55d7acdea490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a8be7b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca1e4d0 0x55d7acdea9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a8be7b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7aca3f1b0 0x55d7acdf1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a8be7b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d7acde8750 con 0x55d7aca162f0 2024-05-14T09:43:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a8be7b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d7acd02370 con 0x55d7aca3f1b0 2024-05-14T09:43:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a8be7b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d7acde88d0 con 0x55d7aca1e4d0 2024-05-14T09:43:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a7ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca162f0 0x55d7acdea490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a7ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca162f0 0x55d7acdea490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50196/0 (socket says 172.21.15.2:50196) 2024-05-14T09:43:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a7ed9d640 1 -- 172.21.15.2:0/2566324946 learned_addr learned my addr 172.21.15.2:0/2566324946 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a77fff640 1 --2- 172.21.15.2:0/2566324946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca1e4d0 0x55d7acdea9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a8be7b740 10 client.0 _ll_register_callbacks cb 0x55d7aca56a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:43:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a7ed9d640 1 --2- 172.21.15.2:0/2566324946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7aca3f1b0 0x55d7acdf1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a77fff640 1 -- 172.21.15.2:0/2566324946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7aca3f1b0 msgr2=0x55d7acdf1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a77fff640 1 --2- 172.21.15.2:0/2566324946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7aca3f1b0 0x55d7acdf1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a77fff640 1 -- 172.21.15.2:0/2566324946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca162f0 msgr2=0x55d7acdea490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a77fff640 1 --2- 172.21.15.2:0/2566324946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca162f0 0x55d7acdea490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a77fff640 1 -- 172.21.15.2:0/2566324946 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7acb23620 con 0x55d7aca1e4d0 2024-05-14T09:43:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a77fff640 1 --2- 172.21.15.2:0/2566324946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca1e4d0 0x55d7acdea9d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8a8019b2c0 tx=0x7f8a68012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbe3ae7f721f10fc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.261+0000 7f8a65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:43:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.262+0000 7f8a65ffb640 1 -- 172.21.15.2:0/2566324946 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68004320 con 0x55d7aca1e4d0 2024-05-14T09:43:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.262+0000 7f8a65ffb640 1 -- 172.21.15.2:0/2566324946 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f8a680119a0 con 0x55d7aca1e4d0 2024-05-14T09:43:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.262+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2566324946 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a78036040 con 0x55d7aca1e4d0 2024-05-14T09:43:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.262+0000 7f8a65ffb640 1 -- 172.21.15.2:0/2566324946 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8a680044c0 con 0x55d7aca1e4d0 2024-05-14T09:43:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.262+0000 7f8a8be7b740 10 client.26086 fetch_fsmap learned FSMap version 36 2024-05-14T09:43:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.262+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2566324946 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d7acdf4930 con 0x55d7aca1e4d0 2024-05-14T09:43:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.262+0000 7f8a65ffb640 1 -- 172.21.15.2:0/2566324946 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a78036040 con 0x55d7aca1e4d0 2024-05-14T09:43:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.262+0000 7f8a65ffb640 1 -- 172.21.15.2:0/2566324946 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f8a6800f440 con 0x55d7aca1e4d0 2024-05-14T09:43:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.262+0000 7f8a8be7b740 10 client.26086 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T09:43:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.262+0000 7f8a8be7b740 10 client.26086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:43:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.262+0000 7f8a8be7b740 20 client.26086 populate_metadata read hostname 'smithi002' 2024-05-14T09:43:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.262+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/2566324946 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d7ac9fdd80 0x55d7acdf8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.262+0000 7f8a8be7b740 4 client.26086 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:43:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.262+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2566324946 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7acd531e0 con 0x55d7ac9fdd80 2024-05-14T09:43:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.263+0000 7f8a7ed9d640 1 --2- 172.21.15.2:0/2566324946 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d7ac9fdd80 0x55d7acdf8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.263+0000 7f8a65ffb640 10 client.26086 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.263+0000 7f8a7ed9d640 1 --2- 172.21.15.2:0/2566324946 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d7ac9fdd80 0x55d7acdf8090 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b74a87b41f0546d server_cookie=ebf135c4a240dc74 in_seq=0 out_seq=0 2024-05-14T09:43:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.264+0000 7f8a65ffb640 1 -- 172.21.15.2:0/2566324946 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d7acd531e0 con 0x55d7ac9fdd80 2024-05-14T09:43:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.264+0000 7f8a65ffb640 10 client.26086 handle_command_reply: tid=0 2024-05-14T09:43:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.264+0000 7f8a8be7b740 1 client.26086 shutdown 2024-05-14T09:43:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.264+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2566324946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca1e4d0 msgr2=0x55d7acdea9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.264+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/2566324946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca1e4d0 0x55d7acdea9d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8a8019b2c0 tx=0x7f8a68012b60 comp rx=0 tx=0).stop 2024-05-14T09:43:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.264+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2566324946 shutdown_connections 2024-05-14T09:43:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.264+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/2566324946 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d7ac9fdd80 0x55d7acdf8090 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.264+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/2566324946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7aca3f1b0 0x55d7acdf1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.264+0000 7f8a65ffb640 0 client.26086 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.264+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/2566324946 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca1e4d0 0x55d7acdea9d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.264+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/2566324946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca162f0 0x55d7acdea490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.264+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2566324946 >> 172.21.15.2:0/2566324946 conn(0x55d7aca33300 msgr2=0x55d7acddb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.264+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2566324946 shutdown_connections 2024-05-14T09:43:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.264+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2566324946 wait complete. 2024-05-14T09:43:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.264+0000 7f8a8be7b740 20 client.26086 trim_cache size 0 max 16384 2024-05-14T09:43:25.272 INFO:tasks.workunit.client.0.smithi002.stdout: 1 782041 10.13 MB/sec execute 291 sec latency 60.971 ms 2024-05-14T09:43:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.280+0000 7f8a8be7b740 1 Processor -- start 2024-05-14T09:43:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.280+0000 7f8a8be7b740 1 -- start start 2024-05-14T09:43:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.280+0000 7f8a8be7b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7ac9fdd80 0x55d7acddbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.280+0000 7f8a8be7b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca162f0 0x55d7acddc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.280+0000 7f8a8be7b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca1e4d0 0x55d7acdf1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.280+0000 7f8a8be7b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801af3c0 con 0x55d7aca1e4d0 2024-05-14T09:43:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.280+0000 7f8a8be7b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801acc90 con 0x55d7ac9fdd80 2024-05-14T09:43:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.280+0000 7f8a8be7b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801aaae0 con 0x55d7aca162f0 2024-05-14T09:43:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.280+0000 7f8a7ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca1e4d0 0x55d7acdf1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.280+0000 7f8a7ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca1e4d0 0x55d7acdf1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50202/0 (socket says 172.21.15.2:50202) 2024-05-14T09:43:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.280+0000 7f8a7ed9d640 1 -- 172.21.15.2:0/265059681 learned_addr learned my addr 172.21.15.2:0/265059681 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.280+0000 7f8a7ed9d640 1 --2- 172.21.15.2:0/265059681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7ac9fdd80 0x55d7acddbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.280+0000 7f8a77fff640 1 --2- 172.21.15.2:0/265059681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca162f0 0x55d7acddc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.281+0000 7f8a7ed9d640 1 -- 172.21.15.2:0/265059681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca162f0 msgr2=0x55d7acddc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.281+0000 7f8a7ed9d640 1 --2- 172.21.15.2:0/265059681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca162f0 0x55d7acddc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.281+0000 7f8a7ed9d640 1 -- 172.21.15.2:0/265059681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7ac9fdd80 msgr2=0x55d7acddbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.281+0000 7f8a7ed9d640 1 --2- 172.21.15.2:0/265059681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7ac9fdd80 0x55d7acddbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.281+0000 7f8a7ed9d640 1 -- 172.21.15.2:0/265059681 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7acddcd00 con 0x55d7aca1e4d0 2024-05-14T09:43:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.281+0000 7f8a7ed9d640 1 --2- 172.21.15.2:0/265059681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca1e4d0 0x55d7acdf1850 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8a78031b30 tx=0x7f8a78005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989ffdf5fd36b335 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.281+0000 7f8a67fff640 1 -- 172.21.15.2:0/265059681 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a780316b0 con 0x55d7aca1e4d0 2024-05-14T09:43:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.281+0000 7f8a67fff640 1 -- 172.21.15.2:0/265059681 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a78036030 con 0x55d7aca1e4d0 2024-05-14T09:43:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.281+0000 7f8a67fff640 1 -- 172.21.15.2:0/265059681 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7802f900 con 0x55d7aca1e4d0 2024-05-14T09:43:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.282+0000 7f8a8be7b740 1 -- 172.21.15.2:0/265059681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca1e4d0 msgr2=0x55d7acdf1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.282+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/265059681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca1e4d0 0x55d7acdf1850 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8a78031b30 tx=0x7f8a78005fd0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.282+0000 7f8a8be7b740 1 -- 172.21.15.2:0/265059681 shutdown_connections 2024-05-14T09:43:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.282+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/265059681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca1e4d0 0x55d7acdf1850 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.282+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/265059681 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca162f0 0x55d7acddc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.282+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/265059681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7ac9fdd80 0x55d7acddbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.282+0000 7f8a8be7b740 1 -- 172.21.15.2:0/265059681 >> 172.21.15.2:0/265059681 conn(0x55d7aca33300 msgr2=0x55d7acde4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.282+0000 7f8a8be7b740 1 -- 172.21.15.2:0/265059681 shutdown_connections 2024-05-14T09:43:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.282+0000 7f8a8be7b740 1 -- 172.21.15.2:0/265059681 wait complete. 2024-05-14T09:43:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.282+0000 7f8a8be7b740 1 -- start start 2024-05-14T09:43:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a8be7b740 1 Processor -- start 2024-05-14T09:43:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a8be7b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7ac9fdd80 0x55d7acded2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a8be7b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca162f0 0x55d7acded830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a8be7b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca1e4d0 0x55d7acdf0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a8be7b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801ab150 con 0x55d7aca1e4d0 2024-05-14T09:43:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a8be7b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801ac5c0 con 0x55d7ac9fdd80 2024-05-14T09:43:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a8be7b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801ab820 con 0x55d7aca162f0 2024-05-14T09:43:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a8be7b740 10 client.0 _ll_register_callbacks cb 0x7f8a680097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:43:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a7ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca1e4d0 0x55d7acdf0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a7ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca1e4d0 0x55d7acdf0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50214/0 (socket says 172.21.15.2:50214) 2024-05-14T09:43:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a7ed9d640 1 -- 172.21.15.2:0/2297263661 learned_addr learned my addr 172.21.15.2:0/2297263661 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a7ed9d640 1 --2- 172.21.15.2:0/2297263661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7ac9fdd80 0x55d7acded2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a77fff640 1 --2- 172.21.15.2:0/2297263661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca162f0 0x55d7acded830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a7ed9d640 1 -- 172.21.15.2:0/2297263661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca162f0 msgr2=0x55d7acded830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a7ed9d640 1 --2- 172.21.15.2:0/2297263661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca162f0 0x55d7acded830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a7ed9d640 1 -- 172.21.15.2:0/2297263661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7ac9fdd80 msgr2=0x55d7acded2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a7ed9d640 1 --2- 172.21.15.2:0/2297263661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7ac9fdd80 0x55d7acded2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.283+0000 7f8a7ed9d640 1 -- 172.21.15.2:0/2297263661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7acdf0d30 con 0x55d7aca1e4d0 2024-05-14T09:43:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a7ed9d640 1 --2- 172.21.15.2:0/2297263661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca1e4d0 0x55d7acdf0770 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8a780098a0 tx=0x7f8a78057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4420cb727c5c3e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:43:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a65ffb640 1 -- 172.21.15.2:0/2297263661 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78003ef0 con 0x55d7aca1e4d0 2024-05-14T09:43:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a65ffb640 1 -- 172.21.15.2:0/2297263661 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f8a7802f680 con 0x55d7aca1e4d0 2024-05-14T09:43:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2297263661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a78036030 con 0x55d7aca1e4d0 2024-05-14T09:43:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a65ffb640 1 -- 172.21.15.2:0/2297263661 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8a78004b40 con 0x55d7aca1e4d0 2024-05-14T09:43:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a8be7b740 10 client.16428 fetch_fsmap learned FSMap version 36 2024-05-14T09:43:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2297263661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d7acdf1fc0 con 0x55d7aca1e4d0 2024-05-14T09:43:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a65ffb640 1 -- 172.21.15.2:0/2297263661 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a78036030 con 0x55d7aca1e4d0 2024-05-14T09:43:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a65ffb640 1 -- 172.21.15.2:0/2297263661 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f8a78051040 con 0x55d7aca1e4d0 2024-05-14T09:43:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a8be7b740 10 client.16428 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T09:43:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a8be7b740 10 client.16428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:43:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a8be7b740 20 client.16428 populate_metadata read hostname 'smithi002' 2024-05-14T09:43:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/2297263661 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d7aca3f1b0 0x55d7ace62450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a8be7b740 4 client.16428 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:43:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.284+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2297263661 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7acd4d050 con 0x55d7aca3f1b0 2024-05-14T09:43:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.285+0000 7f8a77fff640 1 --2- 172.21.15.2:0/2297263661 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d7aca3f1b0 0x55d7ace62450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.285+0000 7f8a65ffb640 10 client.16428 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.285+0000 7f8a77fff640 1 --2- 172.21.15.2:0/2297263661 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d7aca3f1b0 0x55d7ace62450 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e3ce5d88739f367 server_cookie=3366fbae0e8f8252 in_seq=0 out_seq=0 2024-05-14T09:43:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.421+0000 7f8a65ffb640 1 -- 172.21.15.2:0/2297263661 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7acd4d050 con 0x55d7aca3f1b0 2024-05-14T09:43:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.421+0000 7f8a65ffb640 10 client.16428 handle_command_reply: tid=0 2024-05-14T09:43:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.421+0000 7f8a8be7b740 1 client.16428 shutdown 2024-05-14T09:43:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.421+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2297263661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca1e4d0 msgr2=0x55d7acdf0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.421+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/2297263661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca1e4d0 0x55d7acdf0770 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8a780098a0 tx=0x7f8a78057040 comp rx=0 tx=0).stop 2024-05-14T09:43:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.421+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2297263661 shutdown_connections 2024-05-14T09:43:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.421+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/2297263661 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d7aca3f1b0 0x55d7ace62450 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.421+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/2297263661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7aca1e4d0 0x55d7acdf0770 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.421+0000 7f8a65ffb640 0 client.16428 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.421+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/2297263661 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d7aca162f0 0x55d7acded830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.421+0000 7f8a8be7b740 1 --2- 172.21.15.2:0/2297263661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d7ac9fdd80 0x55d7acded2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.421+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2297263661 >> 172.21.15.2:0/2297263661 conn(0x55d7aca33300 msgr2=0x55d7acddbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.421+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2297263661 shutdown_connections 2024-05-14T09:43:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.421+0000 7f8a8be7b740 1 -- 172.21.15.2:0/2297263661 wait complete. 2024-05-14T09:43:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.421+0000 7f8a8be7b740 20 client.16428 trim_cache size 0 max 16384 2024-05-14T09:43:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:43:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:43:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:43:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:43:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.422+0000 7f8a84fc7640 1 -- 172.21.15.2:0/2405684112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8a6c0b28a0 msgr2=0x7f8a6c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.422+0000 7f8a84fc7640 1 --2- 172.21.15.2:0/2405684112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8a6c0b28a0 0x7f8a6c0b4db0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8a700a9820 tx=0x7f8a700d8ea0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.422+0000 7f8a84fc7640 1 -- 172.21.15.2:0/2405684112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a80101880 msgr2=0x7f8a8019a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.422+0000 7f8a84fc7640 1 --2- 172.21.15.2:0/2405684112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a80101880 0x7f8a8019a220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a8006a590 tx=0x7f8a7000cdd0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.422+0000 7f8a84fc7640 1 -- 172.21.15.2:0/2405684112 shutdown_connections 2024-05-14T09:43:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.422+0000 7f8a84fc7640 1 --2- 172.21.15.2:0/2405684112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8a6c0b28a0 0x7f8a6c0b4db0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.422+0000 7f8a84fc7640 1 --2- 172.21.15.2:0/2405684112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801051d0 0x7f8a801a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.422+0000 7f8a84fc7640 1 --2- 172.21.15.2:0/2405684112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8a80102380 0x7f8a8019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.422+0000 7f8a84fc7640 1 --2- 172.21.15.2:0/2405684112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a80101880 0x7f8a8019a220 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.422+0000 7f8a84fc7640 1 -- 172.21.15.2:0/2405684112 >> 172.21.15.2:0/2405684112 conn(0x7f8a800fd480 msgr2=0x7f8a80107d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.422+0000 7f8a84fc7640 1 -- 172.21.15.2:0/2405684112 shutdown_connections 2024-05-14T09:43:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:25.423+0000 7f8a84fc7640 1 -- 172.21.15.2:0/2405684112 wait complete. 2024-05-14T09:43:26.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 784105 10.13 MB/sec execute 292 sec latency 81.660 ms 2024-05-14T09:43:26.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:26 smithi002 ceph-mon[31557]: pgmap v678: 129 pgs: 129 active+clean; 143 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.2 MiB/s wr, 252 op/s 2024-05-14T09:43:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:26 smithi029 ceph-mon[36249]: pgmap v678: 129 pgs: 129 active+clean; 143 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.2 MiB/s wr, 252 op/s 2024-05-14T09:43:26.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:26 smithi026 ceph-mon[36056]: pgmap v678: 129 pgs: 129 active+clean; 143 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.2 MiB/s wr, 252 op/s 2024-05-14T09:43:27.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 785558 10.12 MB/sec execute 293 sec latency 24.809 ms 2024-05-14T09:43:28.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 788172 10.14 MB/sec execute 294 sec latency 21.301 ms 2024-05-14T09:43:28.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:28 smithi002 ceph-mon[31557]: pgmap v679: 129 pgs: 129 active+clean; 145 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-05-14T09:43:28.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:28 smithi029 ceph-mon[36249]: pgmap v679: 129 pgs: 129 active+clean; 145 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-05-14T09:43:28.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:28 smithi026 ceph-mon[36056]: pgmap v679: 129 pgs: 129 active+clean; 145 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-05-14T09:43:29.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 789738 10.13 MB/sec execute 295 sec latency 12.137 ms 2024-05-14T09:43:30.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 791912 10.14 MB/sec execute 296 sec latency 64.633 ms 2024-05-14T09:43:30.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:30 smithi002 ceph-mon[31557]: pgmap v680: 129 pgs: 129 active+clean; 143 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 338 op/s 2024-05-14T09:43:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:30 smithi029 ceph-mon[36249]: pgmap v680: 129 pgs: 129 active+clean; 143 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 338 op/s 2024-05-14T09:43:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:30 smithi026 ceph-mon[36056]: pgmap v680: 129 pgs: 129 active+clean; 143 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 338 op/s 2024-05-14T09:43:31.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 793281 10.13 MB/sec execute 297 sec latency 26.368 ms 2024-05-14T09:43:32.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 795565 10.14 MB/sec execute 298 sec latency 12.742 ms 2024-05-14T09:43:32.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:32 smithi002 ceph-mon[31557]: pgmap v681: 129 pgs: 129 active+clean; 143 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 291 op/s 2024-05-14T09:43:32.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:32 smithi029 ceph-mon[36249]: pgmap v681: 129 pgs: 129 active+clean; 143 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 291 op/s 2024-05-14T09:43:32.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:32 smithi026 ceph-mon[36056]: pgmap v681: 129 pgs: 129 active+clean; 143 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 291 op/s 2024-05-14T09:43:33.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 797413 10.13 MB/sec execute 299 sec latency 13.205 ms 2024-05-14T09:43:34.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 799245 10.14 MB/sec execute 300 sec latency 14.499 ms 2024-05-14T09:43:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:34 smithi002 ceph-mon[31557]: pgmap v682: 129 pgs: 129 active+clean; 147 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 383 op/s 2024-05-14T09:43:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:34 smithi029 ceph-mon[36249]: pgmap v682: 129 pgs: 129 active+clean; 147 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 383 op/s 2024-05-14T09:43:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:34 smithi026 ceph-mon[36056]: pgmap v682: 129 pgs: 129 active+clean; 147 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 383 op/s 2024-05-14T09:43:35.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 801003 10.12 MB/sec execute 301 sec latency 42.690 ms 2024-05-14T09:43:36.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 802620 10.13 MB/sec execute 302 sec latency 178.864 ms 2024-05-14T09:43:36.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:36 smithi002 ceph-mon[31557]: pgmap v683: 129 pgs: 129 active+clean; 147 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.1 MiB/s wr, 276 op/s 2024-05-14T09:43:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:36 smithi029 ceph-mon[36249]: pgmap v683: 129 pgs: 129 active+clean; 147 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.1 MiB/s wr, 276 op/s 2024-05-14T09:43:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:36 smithi026 ceph-mon[36056]: pgmap v683: 129 pgs: 129 active+clean; 147 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.1 MiB/s wr, 276 op/s 2024-05-14T09:43:37.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 804070 10.12 MB/sec execute 303 sec latency 46.215 ms 2024-05-14T09:43:38.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 806248 10.13 MB/sec execute 304 sec latency 12.406 ms 2024-05-14T09:43:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.529+0000 7f209d50d640 1 -- 172.21.15.2:0/1676773026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20981098b0 msgr2=0x7f209810bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.529+0000 7f209d50d640 1 --2- 172.21.15.2:0/1676773026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20981098b0 0x7f209810bdc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2090009ca0 tx=0x7f209002a8c0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.529+0000 7f209d50d640 1 -- 172.21.15.2:0/1676773026 shutdown_connections 2024-05-14T09:43:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.529+0000 7f209d50d640 1 --2- 172.21.15.2:0/1676773026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20981098b0 0x7f209810bdc0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.529+0000 7f209d50d640 1 --2- 172.21.15.2:0/1676773026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2098102490 0x7f2098109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.529+0000 7f209d50d640 1 --2- 172.21.15.2:0/1676773026 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2098101f90 0x7f2098106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.529+0000 7f209d50d640 1 -- 172.21.15.2:0/1676773026 >> 172.21.15.2:0/1676773026 conn(0x7f20980fd480 msgr2=0x7f20980ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.529+0000 7f209d50d640 1 -- 172.21.15.2:0/1676773026 shutdown_connections 2024-05-14T09:43:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.529+0000 7f209d50d640 1 -- 172.21.15.2:0/1676773026 wait complete. 2024-05-14T09:43:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.530+0000 7f209d50d640 1 Processor -- start 2024-05-14T09:43:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.530+0000 7f209d50d640 1 -- start start 2024-05-14T09:43:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.530+0000 7f209d50d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2098101f90 0x7f209819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.530+0000 7f209d50d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2098102490 0x7f209819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.530+0000 7f209d50d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20981098b0 0x7f20981a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.530+0000 7f209d50d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f209810f720 con 0x7f2098102490 2024-05-14T09:43:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.530+0000 7f209d50d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f209810f5a0 con 0x7f20981098b0 2024-05-14T09:43:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.530+0000 7f209d50d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f209810f8a0 con 0x7f2098101f90 2024-05-14T09:43:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.530+0000 7f20967fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2098102490 0x7f209819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.530+0000 7f20967fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2098102490 0x7f209819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47724/0 (socket says 172.21.15.2:47724) 2024-05-14T09:43:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.530+0000 7f2096ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2098101f90 0x7f209819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.530+0000 7f20967fc640 1 -- 172.21.15.2:0/2153104916 learned_addr learned my addr 172.21.15.2:0/2153104916 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.530+0000 7f20977fe640 1 --2- 172.21.15.2:0/2153104916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20981098b0 0x7f20981a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.531+0000 7f20967fc640 1 -- 172.21.15.2:0/2153104916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2098101f90 msgr2=0x7f209819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.531+0000 7f20967fc640 1 --2- 172.21.15.2:0/2153104916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2098101f90 0x7f209819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.531+0000 7f20967fc640 1 -- 172.21.15.2:0/2153104916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20981098b0 msgr2=0x7f20981a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.531+0000 7f20967fc640 1 --2- 172.21.15.2:0/2153104916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20981098b0 0x7f20981a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.531+0000 7f20967fc640 1 -- 172.21.15.2:0/2153104916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20981a1d80 con 0x7f2098102490 2024-05-14T09:43:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.531+0000 7f2096ffd640 1 --2- 172.21.15.2:0/2153104916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2098101f90 0x7f209819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:43:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.531+0000 7f20967fc640 1 --2- 172.21.15.2:0/2153104916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2098102490 0x7f209819a780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f208000e9f0 tx=0x7f208000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.531+0000 7f208ffff640 1 -- 172.21.15.2:0/2153104916 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208000cd80 con 0x7f2098102490 2024-05-14T09:43:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.531+0000 7f209d50d640 1 -- 172.21.15.2:0/2153104916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20981a2070 con 0x7f2098102490 2024-05-14T09:43:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.531+0000 7f208ffff640 1 -- 172.21.15.2:0/2153104916 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20800108e0 con 0x7f2098102490 2024-05-14T09:43:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.531+0000 7f208ffff640 1 -- 172.21.15.2:0/2153104916 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080018980 con 0x7f2098102490 2024-05-14T09:43:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.531+0000 7f209d50d640 1 -- 172.21.15.2:0/2153104916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20981a2610 con 0x7f2098102490 2024-05-14T09:43:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.534+0000 7f208ffff640 1 -- 172.21.15.2:0/2153104916 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2080010430 con 0x7f2098102490 2024-05-14T09:43:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.534+0000 7f20a43c2740 1 Processor -- start 2024-05-14T09:43:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.534+0000 7f20a43c2740 1 -- start start 2024-05-14T09:43:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.535+0000 7f20a43c2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5572337461b0 0x557233ae7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.535+0000 7f20a43c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55723371d2f0 0x557233ae7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.535+0000 7f20977fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5572337461b0 0x557233ae7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.535+0000 7f20977fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5572337461b0 0x557233ae7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47734/0 (socket says 172.21.15.2:47734) 2024-05-14T09:43:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.535+0000 7f20a43c2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 0x557233aeeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.535+0000 7f20a43c2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557233a4c5d0 con 0x5572337461b0 2024-05-14T09:43:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.535+0000 7f20a43c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557233a05890 con 0x55723371d2f0 2024-05-14T09:43:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.535+0000 7f20a43c2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557233a48590 con 0x5572337254d0 2024-05-14T09:43:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.535+0000 7f208ffff640 1 --2- 172.21.15.2:0/2153104916 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f20840b28a0 0x7f20840b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.535+0000 7f20977fe640 1 -- 172.21.15.2:0/3294378102 learned_addr learned my addr 172.21.15.2:0/3294378102 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.535+0000 7f2096ffd640 1 --2- 172.21.15.2:0/3294378102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55723371d2f0 0x557233ae7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.535+0000 7f20967fc640 1 --2- 172.21.15.2:0/3294378102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 0x557233aeeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.535+0000 7f20977fe640 1 --2- 172.21.15.2:0/2153104916 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f20840b28a0 0x7f20840b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.536+0000 7f20977fe640 1 -- 172.21.15.2:0/3294378102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 msgr2=0x557233aeeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.536+0000 7f20977fe640 1 --2- 172.21.15.2:0/3294378102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 0x557233aeeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.536+0000 7f20977fe640 1 -- 172.21.15.2:0/3294378102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55723371d2f0 msgr2=0x557233ae7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.536+0000 7f20977fe640 1 --2- 172.21.15.2:0/3294378102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55723371d2f0 0x557233ae7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.536+0000 7f208ffff640 1 -- 172.21.15.2:0/2153104916 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f20800e2930 con 0x7f2098102490 2024-05-14T09:43:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.536+0000 7f20977fe640 1 -- 172.21.15.2:0/3294378102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55723375e210 con 0x5572337461b0 2024-05-14T09:43:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.536+0000 7f20977fe640 1 --2- 172.21.15.2:0/3294378102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5572337461b0 0x557233ae7a60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2090004af0 tx=0x7f2090031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd3a53f42bfbfad server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.536+0000 7f208d7fa640 1 -- 172.21.15.2:0/3294378102 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090004240 con 0x5572337461b0 2024-05-14T09:43:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.536+0000 7f208d7fa640 1 -- 172.21.15.2:0/3294378102 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2090036070 con 0x5572337461b0 2024-05-14T09:43:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.536+0000 7f208d7fa640 1 -- 172.21.15.2:0/3294378102 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209002fca0 con 0x5572337461b0 2024-05-14T09:43:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.537+0000 7f20a43c2740 1 -- 172.21.15.2:0/3294378102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5572337461b0 msgr2=0x557233ae7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.537+0000 7f20977fe640 1 --2- 172.21.15.2:0/2153104916 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f20840b28a0 0x7f20840b4db0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f209004e000 tx=0x7f209004c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.537+0000 7f20a43c2740 1 --2- 172.21.15.2:0/3294378102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5572337461b0 0x557233ae7a60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2090004af0 tx=0x7f2090031770 comp rx=0 tx=0).stop 2024-05-14T09:43:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.537+0000 7f20a43c2740 1 -- 172.21.15.2:0/3294378102 shutdown_connections 2024-05-14T09:43:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.537+0000 7f20a43c2740 1 --2- 172.21.15.2:0/3294378102 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 0x557233aeeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.537+0000 7f20a43c2740 1 --2- 172.21.15.2:0/3294378102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55723371d2f0 0x557233ae7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.537+0000 7f20a43c2740 1 --2- 172.21.15.2:0/3294378102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5572337461b0 0x557233ae7a60 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.537+0000 7f20a43c2740 1 -- 172.21.15.2:0/3294378102 >> 172.21.15.2:0/3294378102 conn(0x55723373a300 msgr2=0x557233ae33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.537+0000 7f20a43c2740 1 -- 172.21.15.2:0/3294378102 shutdown_connections 2024-05-14T09:43:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.537+0000 7f20a43c2740 1 -- 172.21.15.2:0/3294378102 wait complete. 2024-05-14T09:43:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f20a43c2740 1 -- start start 2024-05-14T09:43:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f20a43c2740 1 Processor -- start 2024-05-14T09:43:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f20a43c2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55723371d2f0 0x557233af1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f20a43c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572337254d0 0x557233af1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f20a43c2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337461b0 0x557233af89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f20a43c2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557233aef7b0 con 0x55723371d2f0 2024-05-14T09:43:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f20a43c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557233a09370 con 0x5572337254d0 2024-05-14T09:43:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f20a43c2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557233aef930 con 0x5572337461b0 2024-05-14T09:43:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f20a43c2740 10 client.0 _ll_register_callbacks cb 0x55723375da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:43:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f2096ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55723371d2f0 0x557233af1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f2096ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55723371d2f0 0x557233af1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47742/0 (socket says 172.21.15.2:47742) 2024-05-14T09:43:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f20977fe640 1 --2- 172.21.15.2:0/2148511395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337461b0 0x557233af89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f2096ffd640 1 -- 172.21.15.2:0/2148511395 learned_addr learned my addr 172.21.15.2:0/2148511395 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f20967fc640 1 --2- 172.21.15.2:0/2148511395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572337254d0 0x557233af1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f2096ffd640 1 -- 172.21.15.2:0/2148511395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337461b0 msgr2=0x557233af89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f2096ffd640 1 --2- 172.21.15.2:0/2148511395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337461b0 0x557233af89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f2096ffd640 1 -- 172.21.15.2:0/2148511395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572337254d0 msgr2=0x557233af1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f2096ffd640 1 --2- 172.21.15.2:0/2148511395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572337254d0 0x557233af1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f2096ffd640 1 -- 172.21.15.2:0/2148511395 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55723382a620 con 0x55723371d2f0 2024-05-14T09:43:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.538+0000 7f20977fe640 1 --2- 172.21.15.2:0/2148511395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337461b0 0x557233af89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:43:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f2096ffd640 1 --2- 172.21.15.2:0/2148511395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55723371d2f0 0x557233af1520 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2088012d20 tx=0x7f208800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896c6039ae9395a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f207e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:43:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f207e7fc640 1 -- 172.21.15.2:0/2148511395 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208800fe60 con 0x55723371d2f0 2024-05-14T09:43:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f207e7fc640 1 -- 172.21.15.2:0/2148511395 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f2088012110 con 0x55723371d2f0 2024-05-14T09:43:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f20a43c2740 1 -- 172.21.15.2:0/2148511395 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2090036070 con 0x55723371d2f0 2024-05-14T09:43:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f207e7fc640 1 -- 172.21.15.2:0/2148511395 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2088004050 con 0x55723371d2f0 2024-05-14T09:43:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f20a43c2740 10 client.16440 fetch_fsmap learned FSMap version 36 2024-05-14T09:43:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f20a43c2740 1 -- 172.21.15.2:0/2148511395 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557233afb9c0 con 0x55723371d2f0 2024-05-14T09:43:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f207e7fc640 1 -- 172.21.15.2:0/2148511395 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2090036070 con 0x55723371d2f0 2024-05-14T09:43:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f207e7fc640 1 -- 172.21.15.2:0/2148511395 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f2088010430 con 0x55723371d2f0 2024-05-14T09:43:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f20a43c2740 10 client.16440 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T09:43:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f20a43c2740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:43:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f20a43c2740 20 client.16440 populate_metadata read hostname 'smithi002' 2024-05-14T09:43:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f20a43c2740 1 --2- 172.21.15.2:0/2148511395 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557233704d80 0x557233aff180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f20a43c2740 4 client.16440 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:43:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.539+0000 7f20a43c2740 1 -- 172.21.15.2:0/2148511395 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557233a5a1e0 con 0x557233704d80 2024-05-14T09:43:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.540+0000 7f20967fc640 1 --2- 172.21.15.2:0/2148511395 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557233704d80 0x557233aff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.540+0000 7f20967fc640 1 --2- 172.21.15.2:0/2148511395 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557233704d80 0x557233aff180 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3390efba1294c28 server_cookie=7454eabc94ac0f8f in_seq=0 out_seq=0 2024-05-14T09:43:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.540+0000 7f207e7fc640 10 client.16440 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.541+0000 7f207e7fc640 1 -- 172.21.15.2:0/2148511395 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557233a5a1e0 con 0x557233704d80 2024-05-14T09:43:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.541+0000 7f207e7fc640 10 client.16440 handle_command_reply: tid=0 2024-05-14T09:43:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.541+0000 7f20a43c2740 1 client.16440 shutdown 2024-05-14T09:43:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.541+0000 7f20a43c2740 1 -- 172.21.15.2:0/2148511395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55723371d2f0 msgr2=0x557233af1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.541+0000 7f20a43c2740 1 --2- 172.21.15.2:0/2148511395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55723371d2f0 0x557233af1520 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2088012d20 tx=0x7f208800f790 comp rx=0 tx=0).stop 2024-05-14T09:43:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.541+0000 7f20a43c2740 1 -- 172.21.15.2:0/2148511395 shutdown_connections 2024-05-14T09:43:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.541+0000 7f20a43c2740 1 --2- 172.21.15.2:0/2148511395 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557233704d80 0x557233aff180 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.541+0000 7f20a43c2740 1 --2- 172.21.15.2:0/2148511395 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337461b0 0x557233af89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.541+0000 7f207e7fc640 0 client.16440 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.541+0000 7f20a43c2740 1 --2- 172.21.15.2:0/2148511395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572337254d0 0x557233af1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.541+0000 7f20a43c2740 1 --2- 172.21.15.2:0/2148511395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55723371d2f0 0x557233af1520 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.541+0000 7f20a43c2740 1 -- 172.21.15.2:0/2148511395 >> 172.21.15.2:0/2148511395 conn(0x55723373a300 msgr2=0x557233ae23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.541+0000 7f20a43c2740 1 -- 172.21.15.2:0/2148511395 shutdown_connections 2024-05-14T09:43:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.541+0000 7f20a43c2740 1 -- 172.21.15.2:0/2148511395 wait complete. 2024-05-14T09:43:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.541+0000 7f20a43c2740 20 client.16440 trim_cache size 0 max 16384 2024-05-14T09:43:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.558+0000 7f20a43c2740 1 Processor -- start 2024-05-14T09:43:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.558+0000 7f20a43c2740 1 -- start start 2024-05-14T09:43:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f20a43c2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55723371d2f0 0x557233ae3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f20a43c2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 0x557233af88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f20a43c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572337461b0 0x557233af8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f20a43c2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f20981af500 con 0x55723371d2f0 2024-05-14T09:43:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f20a43c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f20981acdd0 con 0x5572337461b0 2024-05-14T09:43:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f20a43c2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f20981aac20 con 0x5572337254d0 2024-05-14T09:43:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f2096ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55723371d2f0 0x557233ae3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f20977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572337461b0 0x557233af8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f2096ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55723371d2f0 0x557233ae3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47754/0 (socket says 172.21.15.2:47754) 2024-05-14T09:43:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f20977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572337461b0 0x557233af8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:39282/0 (socket says 172.21.15.2:39282) 2024-05-14T09:43:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f2096ffd640 1 -- 172.21.15.2:0/698025318 learned_addr learned my addr 172.21.15.2:0/698025318 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f20967fc640 1 --2- 172.21.15.2:0/698025318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 0x557233af88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f2096ffd640 1 -- 172.21.15.2:0/698025318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 msgr2=0x557233af88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f2096ffd640 1 --2- 172.21.15.2:0/698025318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 0x557233af88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f2096ffd640 1 -- 172.21.15.2:0/698025318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572337461b0 msgr2=0x557233af8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f2096ffd640 1 --2- 172.21.15.2:0/698025318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572337461b0 0x557233af8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f2096ffd640 1 -- 172.21.15.2:0/698025318 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557233af25a0 con 0x55723371d2f0 2024-05-14T09:43:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f2096ffd640 1 --2- 172.21.15.2:0/698025318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55723371d2f0 0x557233ae3400 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2088012870 tx=0x7f2088012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22432a60465607f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.559+0000 7f208cff9640 1 -- 172.21.15.2:0/698025318 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088012110 con 0x55723371d2f0 2024-05-14T09:43:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.560+0000 7f208cff9640 1 -- 172.21.15.2:0/698025318 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f208802e070 con 0x55723371d2f0 2024-05-14T09:43:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.560+0000 7f208cff9640 1 -- 172.21.15.2:0/698025318 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20880035f0 con 0x55723371d2f0 2024-05-14T09:43:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.560+0000 7f20a43c2740 1 -- 172.21.15.2:0/698025318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55723371d2f0 msgr2=0x557233ae3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.560+0000 7f20a43c2740 1 --2- 172.21.15.2:0/698025318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55723371d2f0 0x557233ae3400 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2088012870 tx=0x7f2088012910 comp rx=0 tx=0).stop 2024-05-14T09:43:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.560+0000 7f20a43c2740 1 -- 172.21.15.2:0/698025318 shutdown_connections 2024-05-14T09:43:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.560+0000 7f20a43c2740 1 --2- 172.21.15.2:0/698025318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572337461b0 0x557233af8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.560+0000 7f20a43c2740 1 --2- 172.21.15.2:0/698025318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 0x557233af88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.560+0000 7f20a43c2740 1 --2- 172.21.15.2:0/698025318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55723371d2f0 0x557233ae3400 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20a43c2740 1 -- 172.21.15.2:0/698025318 >> 172.21.15.2:0/698025318 conn(0x55723373a300 msgr2=0x557233ae2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20a43c2740 1 -- 172.21.15.2:0/698025318 shutdown_connections 2024-05-14T09:43:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20a43c2740 1 -- 172.21.15.2:0/698025318 wait complete. 2024-05-14T09:43:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20a43c2740 1 -- start start 2024-05-14T09:43:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20a43c2740 1 Processor -- start 2024-05-14T09:43:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20a43c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55723371d2f0 0x557233af4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20a43c2740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 0x557233af4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20a43c2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5572337461b0 0x557233af7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20a43c2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f20981ab290 con 0x5572337461b0 2024-05-14T09:43:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20a43c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f20981ac700 con 0x55723371d2f0 2024-05-14T09:43:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20a43c2740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f20981ab960 con 0x5572337254d0 2024-05-14T09:43:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20a43c2740 10 client.0 _ll_register_callbacks cb 0x7f2088009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:43:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20977fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5572337461b0 0x557233af7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f2096ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55723371d2f0 0x557233af4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20967fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 0x557233af4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20977fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5572337461b0 0x557233af7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47758/0 (socket says 172.21.15.2:47758) 2024-05-14T09:43:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f2096ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55723371d2f0 0x557233af4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:39296/0 (socket says 172.21.15.2:39296) 2024-05-14T09:43:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20967fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 0x557233af4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:47102/0 (socket says 172.21.15.2:47102) 2024-05-14T09:43:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.561+0000 7f20977fe640 1 -- 172.21.15.2:0/3333056123 learned_addr learned my addr 172.21.15.2:0/3333056123 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.562+0000 7f20977fe640 1 -- 172.21.15.2:0/3333056123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 msgr2=0x557233af4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.562+0000 7f20977fe640 1 --2- 172.21.15.2:0/3333056123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 0x557233af4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.562+0000 7f20977fe640 1 -- 172.21.15.2:0/3333056123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55723371d2f0 msgr2=0x557233af4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.562+0000 7f20977fe640 1 --2- 172.21.15.2:0/3333056123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55723371d2f0 0x557233af4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.562+0000 7f20977fe640 1 -- 172.21.15.2:0/3333056123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557233ae3940 con 0x5572337461b0 2024-05-14T09:43:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.562+0000 7f20977fe640 1 --2- 172.21.15.2:0/3333056123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5572337461b0 0x557233af7790 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2090009ca0 tx=0x7f2090004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226718af336f562c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.562+0000 7f207e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:43:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.562+0000 7f207e7fc640 1 -- 172.21.15.2:0/3333056123 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20900047f0 con 0x5572337461b0 2024-05-14T09:43:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.562+0000 7f207e7fc640 1 -- 172.21.15.2:0/3333056123 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9282+0+0 (secure 0 0 0) 0x7f20900305c0 con 0x5572337461b0 2024-05-14T09:43:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.562+0000 7f20a43c2740 1 -- 172.21.15.2:0/3333056123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f208802e070 con 0x5572337461b0 2024-05-14T09:43:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.562+0000 7f207e7fc640 1 -- 172.21.15.2:0/3333056123 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f209006d050 con 0x5572337461b0 2024-05-14T09:43:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.562+0000 7f20a43c2740 10 client.16446 fetch_fsmap learned FSMap version 36 2024-05-14T09:43:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.562+0000 7f20a43c2740 1 -- 172.21.15.2:0/3333056123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557233af8e10 con 0x5572337461b0 2024-05-14T09:43:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.562+0000 7f207e7fc640 1 -- 172.21.15.2:0/3333056123 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f208802e070 con 0x5572337461b0 2024-05-14T09:43:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.563+0000 7f207e7fc640 1 -- 172.21.15.2:0/3333056123 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f209005f480 con 0x5572337461b0 2024-05-14T09:43:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.563+0000 7f20a43c2740 10 client.16446 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T09:43:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.563+0000 7f20a43c2740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:43:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.563+0000 7f20a43c2740 20 client.16446 populate_metadata read hostname 'smithi002' 2024-05-14T09:43:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.563+0000 7f20a43c2740 1 --2- 172.21.15.2:0/3333056123 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5572336da1c0 0x557233b69500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.563+0000 7f20a43c2740 4 client.16446 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:43:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.563+0000 7f20a43c2740 1 -- 172.21.15.2:0/3333056123 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557233a54050 con 0x5572336da1c0 2024-05-14T09:43:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.563+0000 7f2096ffd640 1 --2- 172.21.15.2:0/3333056123 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5572336da1c0 0x557233b69500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.563+0000 7f207e7fc640 10 client.16446 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.563+0000 7f2096ffd640 1 --2- 172.21.15.2:0/3333056123 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5572336da1c0 0x557233b69500 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=667783a146e920e3 server_cookie=2f7709fc22858ad4 in_seq=0 out_seq=0 2024-05-14T09:43:38.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:38 smithi002 ceph-mon[31557]: pgmap v684: 129 pgs: 129 active+clean; 146 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 300 op/s 2024-05-14T09:43:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.675+0000 7f207e7fc640 1 -- 172.21.15.2:0/3333056123 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557233a54050 con 0x5572336da1c0 2024-05-14T09:43:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.675+0000 7f207e7fc640 10 client.16446 handle_command_reply: tid=0 2024-05-14T09:43:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.675+0000 7f20a43c2740 1 client.16446 shutdown 2024-05-14T09:43:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.675+0000 7f20a43c2740 1 -- 172.21.15.2:0/3333056123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5572337461b0 msgr2=0x557233af7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.675+0000 7f20a43c2740 1 --2- 172.21.15.2:0/3333056123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5572337461b0 0x557233af7790 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2090009ca0 tx=0x7f2090004270 comp rx=0 tx=0).stop 2024-05-14T09:43:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.675+0000 7f20a43c2740 1 -- 172.21.15.2:0/3333056123 shutdown_connections 2024-05-14T09:43:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.675+0000 7f20a43c2740 1 --2- 172.21.15.2:0/3333056123 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5572336da1c0 0x557233b69500 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.675+0000 7f20a43c2740 1 --2- 172.21.15.2:0/3333056123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5572337461b0 0x557233af7790 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.675+0000 7f207e7fc640 0 client.16446 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.675+0000 7f20a43c2740 1 --2- 172.21.15.2:0/3333056123 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5572337254d0 0x557233af4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.675+0000 7f20a43c2740 1 --2- 172.21.15.2:0/3333056123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55723371d2f0 0x557233af4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.675+0000 7f20a43c2740 1 -- 172.21.15.2:0/3333056123 >> 172.21.15.2:0/3333056123 conn(0x55723373a300 msgr2=0x557233af1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.676+0000 7f20a43c2740 1 -- 172.21.15.2:0/3333056123 shutdown_connections 2024-05-14T09:43:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.676+0000 7f20a43c2740 1 -- 172.21.15.2:0/3333056123 wait complete. 2024-05-14T09:43:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.676+0000 7f20a43c2740 20 client.16446 trim_cache size 0 max 16384 2024-05-14T09:43:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:43:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:43:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:43:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:43:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.677+0000 7f209d50d640 1 -- 172.21.15.2:0/2153104916 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f20840b28a0 msgr2=0x7f20840b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.677+0000 7f209d50d640 1 --2- 172.21.15.2:0/2153104916 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f20840b28a0 0x7f20840b4db0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f209004e000 tx=0x7f209004c000 comp rx=0 tx=0).stop 2024-05-14T09:43:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.677+0000 7f209d50d640 1 -- 172.21.15.2:0/2153104916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2098102490 msgr2=0x7f209819a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.677+0000 7f209d50d640 1 --2- 172.21.15.2:0/2153104916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2098102490 0x7f209819a780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f208000e9f0 tx=0x7f208000eeb0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.677+0000 7f209d50d640 1 -- 172.21.15.2:0/2153104916 shutdown_connections 2024-05-14T09:43:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.677+0000 7f209d50d640 1 --2- 172.21.15.2:0/2153104916 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f20840b28a0 0x7f20840b4db0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.677+0000 7f209d50d640 1 --2- 172.21.15.2:0/2153104916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20981098b0 0x7f20981a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.677+0000 7f209d50d640 1 --2- 172.21.15.2:0/2153104916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2098102490 0x7f209819a780 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.677+0000 7f209d50d640 1 --2- 172.21.15.2:0/2153104916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2098101f90 0x7f209819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.677+0000 7f209d50d640 1 -- 172.21.15.2:0/2153104916 >> 172.21.15.2:0/2153104916 conn(0x7f20980fd480 msgr2=0x7f2098105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.677+0000 7f209d50d640 1 -- 172.21.15.2:0/2153104916 shutdown_connections 2024-05-14T09:43:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:38.678+0000 7f209d50d640 1 -- 172.21.15.2:0/2153104916 wait complete. 2024-05-14T09:43:38.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:38 smithi029 ceph-mon[36249]: pgmap v684: 129 pgs: 129 active+clean; 146 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 300 op/s 2024-05-14T09:43:38.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:38 smithi026 ceph-mon[36056]: pgmap v684: 129 pgs: 129 active+clean; 146 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 300 op/s 2024-05-14T09:43:39.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 807765 10.12 MB/sec execute 305 sec latency 37.704 ms 2024-05-14T09:43:39.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:39 smithi002 ceph-mon[31557]: daemon mds.e restarted 2024-05-14T09:43:39.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:39 smithi029 ceph-mon[36249]: daemon mds.e restarted 2024-05-14T09:43:39.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:39 smithi026 ceph-mon[36056]: daemon mds.e restarted 2024-05-14T09:43:39.701 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:43:39 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e[66118]: ignoring --setuser ceph since I am not root 2024-05-14T09:43:39.701 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:43:39 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e[66118]: ignoring --setgroup ceph since I am not root 2024-05-14T09:43:39.702 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:43:39 smithi026 ceph-mds[66122]: -- 172.21.15.26:0/2191089999 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555fded59a40 con 0x555fdeda1600 2024-05-14T09:43:39.702 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:43:39 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e[66118]: starting mds.e at 2024-05-14T09:43:40.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 809794 10.13 MB/sec execute 306 sec latency 23.740 ms 2024-05-14T09:43:40.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:40 smithi002 ceph-mon[31557]: pgmap v685: 129 pgs: 129 active+clean; 149 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 300 op/s 2024-05-14T09:43:40.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:40 smithi002 ceph-mon[31557]: osdmap e91: 12 total, 12 up, 12 in 2024-05-14T09:43:40.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:40 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.26:6834/303867621,v1:172.21.15.26:6835/303867621] up:boot 2024-05-14T09:43:40.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:40 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T09:43:40.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T09:43:40.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:40 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:43:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:40 smithi029 ceph-mon[36249]: pgmap v685: 129 pgs: 129 active+clean; 149 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 300 op/s 2024-05-14T09:43:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:40 smithi029 ceph-mon[36249]: osdmap e91: 12 total, 12 up, 12 in 2024-05-14T09:43:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:40 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.26:6834/303867621,v1:172.21.15.26:6835/303867621] up:boot 2024-05-14T09:43:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:40 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T09:43:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T09:43:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:40 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:43:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:40 smithi026 ceph-mon[36056]: pgmap v685: 129 pgs: 129 active+clean; 149 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 300 op/s 2024-05-14T09:43:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:40 smithi026 ceph-mon[36056]: osdmap e91: 12 total, 12 up, 12 in 2024-05-14T09:43:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:40 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.26:6834/303867621,v1:172.21.15.26:6835/303867621] up:boot 2024-05-14T09:43:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:40 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T09:43:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T09:43:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:40 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:43:41.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 811206 10.11 MB/sec execute 307 sec latency 73.289 ms 2024-05-14T09:43:42.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 813356 10.12 MB/sec execute 308 sec latency 17.373 ms 2024-05-14T09:43:42.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:42 smithi002 ceph-mon[31557]: pgmap v687: 129 pgs: 129 active+clean; 152 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 313 op/s 2024-05-14T09:43:42.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:42 smithi029 ceph-mon[36249]: pgmap v687: 129 pgs: 129 active+clean; 152 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 313 op/s 2024-05-14T09:43:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:42 smithi026 ceph-mon[36056]: pgmap v687: 129 pgs: 129 active+clean; 152 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 313 op/s 2024-05-14T09:43:43.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 815378 10.11 MB/sec execute 309 sec latency 12.571 ms 2024-05-14T09:43:44.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 816999 10.12 MB/sec execute 310 sec latency 12.190 ms 2024-05-14T09:43:44.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:44 smithi002 ceph-mon[31557]: pgmap v688: 129 pgs: 129 active+clean; 145 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 2.5 MiB/s wr, 290 op/s 2024-05-14T09:43:44.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:44 smithi029 ceph-mon[36249]: pgmap v688: 129 pgs: 129 active+clean; 145 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 2.5 MiB/s wr, 290 op/s 2024-05-14T09:43:44.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:44 smithi026 ceph-mon[36056]: pgmap v688: 129 pgs: 129 active+clean; 145 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 2.5 MiB/s wr, 290 op/s 2024-05-14T09:43:45.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 818991 10.11 MB/sec execute 311 sec latency 14.366 ms 2024-05-14T09:43:46.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 820371 10.12 MB/sec execute 312 sec latency 102.938 ms 2024-05-14T09:43:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:46 smithi002 ceph-mon[31557]: pgmap v689: 129 pgs: 129 active+clean; 145 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 2.5 MiB/s wr, 290 op/s 2024-05-14T09:43:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:46 smithi029 ceph-mon[36249]: pgmap v689: 129 pgs: 129 active+clean; 145 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 2.5 MiB/s wr, 290 op/s 2024-05-14T09:43:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:46 smithi026 ceph-mon[36056]: pgmap v689: 129 pgs: 129 active+clean; 145 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 2.5 MiB/s wr, 290 op/s 2024-05-14T09:43:47.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 822050 10.10 MB/sec execute 313 sec latency 14.841 ms 2024-05-14T09:43:48.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 823967 10.11 MB/sec execute 314 sec latency 14.921 ms 2024-05-14T09:43:48.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:48 smithi002 ceph-mon[31557]: pgmap v690: 129 pgs: 129 active+clean; 146 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 375 op/s 2024-05-14T09:43:48.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:43:48.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:48 smithi029 ceph-mon[36249]: pgmap v690: 129 pgs: 129 active+clean; 146 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 375 op/s 2024-05-14T09:43:48.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:43:48.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:48 smithi026 ceph-mon[36056]: pgmap v690: 129 pgs: 129 active+clean; 146 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 375 op/s 2024-05-14T09:43:48.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:43:49.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 825388 10.10 MB/sec execute 315 sec latency 45.935 ms 2024-05-14T09:43:50.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 827430 10.11 MB/sec execute 316 sec latency 29.400 ms 2024-05-14T09:43:50.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:50 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:50 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:50 smithi002 ceph-mon[31557]: pgmap v691: 129 pgs: 129 active+clean; 146 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-05-14T09:43:50.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:50 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:50 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:50 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:50 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:50 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:50 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:50 smithi029 ceph-mon[36249]: pgmap v691: 129 pgs: 129 active+clean; 146 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-05-14T09:43:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:50 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:50 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:50 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:50 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:50 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:50 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:50 smithi026 ceph-mon[36056]: pgmap v691: 129 pgs: 129 active+clean; 146 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-05-14T09:43:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:50 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:50 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:50 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:50.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:50 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:51.273 INFO:tasks.workunit.client.0.smithi002.stdout: 1 828286 10.08 MB/sec execute 317 sec latency 76.452 ms 2024-05-14T09:43:51.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:43:51.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:43:51.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:51.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:43:51.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:43:51.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:43:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:43:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:43:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.787+0000 7f67955d5640 1 -- 172.21.15.2:0/1489117790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6790108a70 msgr2=0x7f679010ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.787+0000 7f67955d5640 1 --2- 172.21.15.2:0/1489117790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6790108a70 0x7f679010ae90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6788009ca0 tx=0x7f678802a8c0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.787+0000 7f67955d5640 1 -- 172.21.15.2:0/1489117790 shutdown_connections 2024-05-14T09:43:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.787+0000 7f67955d5640 1 --2- 172.21.15.2:0/1489117790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6790108a70 0x7f679010ae90 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.787+0000 7f67955d5640 1 --2- 172.21.15.2:0/1489117790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6790103d70 0x7f6790104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.787+0000 7f67955d5640 1 --2- 172.21.15.2:0/1489117790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6790102b30 0x7f6790103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.787+0000 7f67955d5640 1 -- 172.21.15.2:0/1489117790 >> 172.21.15.2:0/1489117790 conn(0x7f67900fdd20 msgr2=0x7f67901001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.787+0000 7f67955d5640 1 -- 172.21.15.2:0/1489117790 shutdown_connections 2024-05-14T09:43:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.787+0000 7f67955d5640 1 -- 172.21.15.2:0/1489117790 wait complete. 2024-05-14T09:43:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.787+0000 7f67955d5640 1 Processor -- start 2024-05-14T09:43:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.787+0000 7f67955d5640 1 -- start start 2024-05-14T09:43:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f67955d5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6790102b30 0x7f679019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f67955d5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6790103d70 0x7f679019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f67955d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6790108a70 0x7f67901a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f67955d5640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f679010e6b0 con 0x7f6790103d70 2024-05-14T09:43:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f67955d5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f679010e530 con 0x7f6790108a70 2024-05-14T09:43:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f67955d5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f679010e830 con 0x7f6790102b30 2024-05-14T09:43:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f678e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6790103d70 0x7f679019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f678effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6790102b30 0x7f679019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f678f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6790108a70 0x7f67901a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f678e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6790103d70 0x7f679019a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52008/0 (socket says 172.21.15.2:52008) 2024-05-14T09:43:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f678effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6790102b30 0x7f679019a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:48714/0 (socket says 172.21.15.2:48714) 2024-05-14T09:43:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f678e7fc640 1 -- 172.21.15.2:0/4124953847 learned_addr learned my addr 172.21.15.2:0/4124953847 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f678f7fe640 1 -- 172.21.15.2:0/4124953847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6790102b30 msgr2=0x7f679019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f678f7fe640 1 --2- 172.21.15.2:0/4124953847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6790102b30 0x7f679019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f678f7fe640 1 -- 172.21.15.2:0/4124953847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6790103d70 msgr2=0x7f679019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f678f7fe640 1 --2- 172.21.15.2:0/4124953847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6790103d70 0x7f679019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f678f7fe640 1 -- 172.21.15.2:0/4124953847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67901a1da0 con 0x7f6790108a70 2024-05-14T09:43:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f678effd640 1 --2- 172.21.15.2:0/4124953847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6790102b30 0x7f679019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:43:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.788+0000 7f678f7fe640 1 --2- 172.21.15.2:0/4124953847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6790108a70 0x7f67901a1730 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f678802aeb0 tx=0x7f67880317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.789+0000 7f6777fff640 1 -- 172.21.15.2:0/4124953847 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788003b70 con 0x7f6790108a70 2024-05-14T09:43:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.789+0000 7f67955d5640 1 -- 172.21.15.2:0/4124953847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67901a2030 con 0x7f6790108a70 2024-05-14T09:43:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.789+0000 7f6777fff640 1 -- 172.21.15.2:0/4124953847 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6788003d10 con 0x7f6790108a70 2024-05-14T09:43:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.789+0000 7f6777fff640 1 -- 172.21.15.2:0/4124953847 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678802fb70 con 0x7f6790108a70 2024-05-14T09:43:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.789+0000 7f67955d5640 1 -- 172.21.15.2:0/4124953847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67901a25d0 con 0x7f6790108a70 2024-05-14T09:43:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.792+0000 7f679c563740 1 Processor -- start 2024-05-14T09:43:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.792+0000 7f6777fff640 1 -- 172.21.15.2:0/4124953847 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f67880046a0 con 0x7f6790108a70 2024-05-14T09:43:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.792+0000 7f679c563740 1 -- start start 2024-05-14T09:43:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.792+0000 7f679c563740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8961b0 0x5558fec3be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.792+0000 7f679c563740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe86d2f0 0x5558fec3c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f679c563740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe8754d0 0x5558fec432f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f679c563740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5558feb9c5d0 con 0x5558fe8961b0 2024-05-14T09:43:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f679c563740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5558feb55890 con 0x5558fe8754d0 2024-05-14T09:43:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f679c563740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5558feb98590 con 0x5558fe86d2f0 2024-05-14T09:43:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f678e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8961b0 0x5558fec3be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f678e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8961b0 0x5558fec3be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52016/0 (socket says 172.21.15.2:52016) 2024-05-14T09:43:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f678e7fc640 1 -- 172.21.15.2:0/2558808688 learned_addr learned my addr 172.21.15.2:0/2558808688 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f678effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe86d2f0 0x5558fec3c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f678e7fc640 1 --2- 172.21.15.2:0/2558808688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe8754d0 0x5558fec432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f678e7fc640 1 -- 172.21.15.2:0/2558808688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe86d2f0 msgr2=0x5558fec3c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f678e7fc640 1 --2- 172.21.15.2:0/2558808688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe86d2f0 0x5558fec3c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f678e7fc640 1 -- 172.21.15.2:0/2558808688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe8754d0 msgr2=0x5558fec432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f678e7fc640 1 --2- 172.21.15.2:0/2558808688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe8754d0 0x5558fec432f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f678e7fc640 1 -- 172.21.15.2:0/2558808688 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558fe8ae210 con 0x5558fe8961b0 2024-05-14T09:43:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f6777fff640 1 --2- 172.21.15.2:0/4124953847 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67780b28a0 0x7f67780b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f678effd640 1 --2- 172.21.15.2:0/4124953847 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67780b28a0 0x7f67780b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f678e7fc640 1 --2- 172.21.15.2:0/2558808688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8961b0 0x5558fec3be70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6784012880 tx=0x7f678400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b8d8cc3461c51f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f6777fff640 1 -- 172.21.15.2:0/4124953847 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f6788103aa0 con 0x7f6790108a70 2024-05-14T09:43:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.793+0000 7f67757fa640 1 -- 172.21.15.2:0/2558808688 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784004030 con 0x5558fe8961b0 2024-05-14T09:43:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.794+0000 7f67757fa640 1 -- 172.21.15.2:0/2558808688 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6784010c20 con 0x5558fe8961b0 2024-05-14T09:43:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.794+0000 7f67757fa640 1 -- 172.21.15.2:0/2558808688 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784003850 con 0x5558fe8961b0 2024-05-14T09:43:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.794+0000 7f679c563740 1 -- 172.21.15.2:0/2558808688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8961b0 msgr2=0x5558fec3be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.794+0000 7f678effd640 1 --2- 172.21.15.2:0/4124953847 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67780b28a0 0x7f67780b4db0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5558fec3d250 tx=0x7f6780005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.794+0000 7f679c563740 1 --2- 172.21.15.2:0/2558808688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8961b0 0x5558fec3be70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6784012880 tx=0x7f678400b8c0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 1 -- 172.21.15.2:0/2558808688 shutdown_connections 2024-05-14T09:43:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 1 --2- 172.21.15.2:0/2558808688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe8754d0 0x5558fec432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 1 --2- 172.21.15.2:0/2558808688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe86d2f0 0x5558fec3c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 1 --2- 172.21.15.2:0/2558808688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8961b0 0x5558fec3be70 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 1 -- 172.21.15.2:0/2558808688 >> 172.21.15.2:0/2558808688 conn(0x5558fe88a300 msgr2=0x5558fec333a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 1 -- 172.21.15.2:0/2558808688 shutdown_connections 2024-05-14T09:43:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 1 -- 172.21.15.2:0/2558808688 wait complete. 2024-05-14T09:43:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 1 -- start start 2024-05-14T09:43:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 1 Processor -- start 2024-05-14T09:43:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec45a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe8754d0 0x5558fec45fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8961b0 0x5558fec4cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5558fec43ce0 con 0x5558fe8961b0 2024-05-14T09:43:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5558feb59370 con 0x5558fe86d2f0 2024-05-14T09:43:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5558fec43e60 con 0x5558fe8754d0 2024-05-14T09:43:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f679c563740 10 client.0 _ll_register_callbacks cb 0x5558fe8ada00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:43:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f678f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8961b0 0x5558fec4cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f678effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec45a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f678effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec45a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:55212/0 (socket says 172.21.15.2:55212) 2024-05-14T09:43:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f678effd640 1 -- 172.21.15.2:0/3908488334 learned_addr learned my addr 172.21.15.2:0/3908488334 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f678e7fc640 1 --2- 172.21.15.2:0/3908488334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe8754d0 0x5558fec45fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.795+0000 7f678f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8961b0 0x5558fec4cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52024/0 (socket says 172.21.15.2:52024) 2024-05-14T09:43:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f678effd640 1 -- 172.21.15.2:0/3908488334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe8754d0 msgr2=0x5558fec45fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f678effd640 1 --2- 172.21.15.2:0/3908488334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe8754d0 0x5558fec45fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f678effd640 1 -- 172.21.15.2:0/3908488334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8961b0 msgr2=0x5558fec4cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f678effd640 1 --2- 172.21.15.2:0/3908488334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8961b0 0x5558fec4cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f678effd640 1 -- 172.21.15.2:0/3908488334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558fe97a620 con 0x5558fe86d2f0 2024-05-14T09:43:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f678f7fe640 1 --2- 172.21.15.2:0/3908488334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8961b0 0x5558fec4cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:43:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f678e7fc640 1 --2- 172.21.15.2:0/3908488334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe8754d0 0x5558fec45fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:43:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f678effd640 1 --2- 172.21.15.2:0/3908488334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec45a90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f67800045f0 tx=0x7f6780004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ffce9ef0ad244bb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f6772ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:43:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f6772ff5640 1 -- 172.21.15.2:0/3908488334 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678000f830 con 0x5558fe86d2f0 2024-05-14T09:43:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f6772ff5640 1 -- 172.21.15.2:0/3908488334 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f678000f9d0 con 0x5558fe86d2f0 2024-05-14T09:43:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f679c563740 1 -- 172.21.15.2:0/3908488334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6784010c20 con 0x5558fe86d2f0 2024-05-14T09:43:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f6772ff5640 1 -- 172.21.15.2:0/3908488334 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f678000fbd0 con 0x5558fe86d2f0 2024-05-14T09:43:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f679c563740 10 client.26144 fetch_fsmap learned FSMap version 38 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.796+0000 7f679c563740 1 -- 172.21.15.2:0/3908488334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5558fec4ff30 con 0x5558fe86d2f0 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.797+0000 7f6772ff5640 1 -- 172.21.15.2:0/3908488334 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6784010c20 con 0x5558fe86d2f0 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.797+0000 7f6772ff5640 1 -- 172.21.15.2:0/3908488334 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f6780027070 con 0x5558fe86d2f0 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.797+0000 7f679c563740 10 client.26144 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.797+0000 7f679c563740 10 client.26144 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.797+0000 7f679c563740 20 client.26144 populate_metadata read hostname 'smithi002' 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.797+0000 7f679c563740 1 --2- 172.21.15.2:0/3908488334 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5558fe854d80 0x5558fec53710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.797+0000 7f679c563740 4 client.26144 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.797+0000 7f679c563740 1 -- 172.21.15.2:0/3908488334 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558febaa1e0 con 0x5558fe854d80 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.797+0000 7f678e7fc640 1 --2- 172.21.15.2:0/3908488334 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5558fe854d80 0x5558fec53710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.797+0000 7f6772ff5640 10 client.26144 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.797+0000 7f678e7fc640 1 --2- 172.21.15.2:0/3908488334 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5558fe854d80 0x5558fec53710 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91fbc41f882530e4 server_cookie=5e55d738b1f3c8d7 in_seq=0 out_seq=0 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.798+0000 7f6772ff5640 1 -- 172.21.15.2:0/3908488334 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5558febaa1e0 con 0x5558fe854d80 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.798+0000 7f6772ff5640 10 client.26144 handle_command_reply: tid=0 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.798+0000 7f679c563740 1 client.26144 shutdown 2024-05-14T09:43:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.798+0000 7f679c563740 1 -- 172.21.15.2:0/3908488334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 msgr2=0x5558fec45a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.798+0000 7f679c563740 1 --2- 172.21.15.2:0/3908488334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec45a90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f67800045f0 tx=0x7f6780004730 comp rx=0 tx=0).stop 2024-05-14T09:43:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.799+0000 7f679c563740 1 -- 172.21.15.2:0/3908488334 shutdown_connections 2024-05-14T09:43:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.799+0000 7f679c563740 1 --2- 172.21.15.2:0/3908488334 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5558fe854d80 0x5558fec53710 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.799+0000 7f679c563740 1 --2- 172.21.15.2:0/3908488334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8961b0 0x5558fec4cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.799+0000 7f6772ff5640 0 client.26144 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.799+0000 7f679c563740 1 --2- 172.21.15.2:0/3908488334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe8754d0 0x5558fec45fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.799+0000 7f679c563740 1 --2- 172.21.15.2:0/3908488334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec45a90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.799+0000 7f679c563740 1 -- 172.21.15.2:0/3908488334 >> 172.21.15.2:0/3908488334 conn(0x5558fe88a300 msgr2=0x5558fec36300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.799+0000 7f679c563740 1 -- 172.21.15.2:0/3908488334 shutdown_connections 2024-05-14T09:43:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.799+0000 7f679c563740 1 -- 172.21.15.2:0/3908488334 wait complete. 2024-05-14T09:43:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.799+0000 7f679c563740 20 client.26144 trim_cache size 0 max 16384 2024-05-14T09:43:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.814+0000 7f679c563740 1 Processor -- start 2024-05-14T09:43:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.814+0000 7f679c563740 1 -- start start 2024-05-14T09:43:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.814+0000 7f679c563740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe854d80 0x5558fec33310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.814+0000 7f679c563740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec4cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.814+0000 7f679c563740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe8754d0 0x5558fec4d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.814+0000 7f679c563740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f67901ad360 con 0x5558fe854d80 2024-05-14T09:43:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.814+0000 7f679c563740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f67901aac00 con 0x5558fe86d2f0 2024-05-14T09:43:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.814+0000 7f679c563740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f67901a8a50 con 0x5558fe8754d0 2024-05-14T09:43:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.814+0000 7f678effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe854d80 0x5558fec33310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.814+0000 7f678effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe854d80 0x5558fec33310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52032/0 (socket says 172.21.15.2:52032) 2024-05-14T09:43:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.814+0000 7f678effd640 1 -- 172.21.15.2:0/4048387548 learned_addr learned my addr 172.21.15.2:0/4048387548 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.814+0000 7f678e7fc640 1 --2- 172.21.15.2:0/4048387548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec4cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.814+0000 7f678f7fe640 1 --2- 172.21.15.2:0/4048387548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe8754d0 0x5558fec4d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.815+0000 7f678effd640 1 -- 172.21.15.2:0/4048387548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe8754d0 msgr2=0x5558fec4d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.815+0000 7f678effd640 1 --2- 172.21.15.2:0/4048387548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe8754d0 0x5558fec4d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.815+0000 7f678effd640 1 -- 172.21.15.2:0/4048387548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 msgr2=0x5558fec4cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.815+0000 7f678effd640 1 --2- 172.21.15.2:0/4048387548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec4cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.815+0000 7f678effd640 1 -- 172.21.15.2:0/4048387548 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558fec46b00 con 0x5558fe854d80 2024-05-14T09:43:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.815+0000 7f678effd640 1 --2- 172.21.15.2:0/4048387548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe854d80 0x5558fec33310 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6780012f10 tx=0x7f6780003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f79a02ed6462d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.815+0000 7f6774ff9640 1 -- 172.21.15.2:0/4048387548 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678000f830 con 0x5558fe854d80 2024-05-14T09:43:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.815+0000 7f6774ff9640 1 -- 172.21.15.2:0/4048387548 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f678000f9d0 con 0x5558fe854d80 2024-05-14T09:43:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.815+0000 7f6774ff9640 1 -- 172.21.15.2:0/4048387548 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780029b90 con 0x5558fe854d80 2024-05-14T09:43:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 -- 172.21.15.2:0/4048387548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe854d80 msgr2=0x5558fec33310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 --2- 172.21.15.2:0/4048387548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe854d80 0x5558fec33310 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6780012f10 tx=0x7f6780003ed0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 -- 172.21.15.2:0/4048387548 shutdown_connections 2024-05-14T09:43:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 --2- 172.21.15.2:0/4048387548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe8754d0 0x5558fec4d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 --2- 172.21.15.2:0/4048387548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec4cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 --2- 172.21.15.2:0/4048387548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe854d80 0x5558fec33310 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 -- 172.21.15.2:0/4048387548 >> 172.21.15.2:0/4048387548 conn(0x5558fe88a300 msgr2=0x5558fec32c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 -- 172.21.15.2:0/4048387548 shutdown_connections 2024-05-14T09:43:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 -- 172.21.15.2:0/4048387548 wait complete. 2024-05-14T09:43:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 -- start start 2024-05-14T09:43:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 Processor -- start 2024-05-14T09:43:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe854d80 0x5558fec4d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec4a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8754d0 0x5558fec4a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f67901a90c0 con 0x5558fe8754d0 2024-05-14T09:43:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f67901aa530 con 0x5558fe86d2f0 2024-05-14T09:43:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f67901a9790 con 0x5558fe854d80 2024-05-14T09:43:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.816+0000 7f679c563740 10 client.0 _ll_register_callbacks cb 0x7f6780009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:43:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f678f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8754d0 0x5558fec4a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f678e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec4a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f678effd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe854d80 0x5558fec4d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f678e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec4a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:55226/0 (socket says 172.21.15.2:55226) 2024-05-14T09:43:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f678e7fc640 1 -- 172.21.15.2:0/3241920094 learned_addr learned my addr 172.21.15.2:0/3241920094 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:43:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f678f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8754d0 0x5558fec4a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52042/0 (socket says 172.21.15.2:52042) 2024-05-14T09:43:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f678f7fe640 1 -- 172.21.15.2:0/3241920094 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe854d80 msgr2=0x5558fec4d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f678f7fe640 1 --2- 172.21.15.2:0/3241920094 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe854d80 0x5558fec4d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f678f7fe640 1 -- 172.21.15.2:0/3241920094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 msgr2=0x5558fec4a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:43:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f678f7fe640 1 --2- 172.21.15.2:0/3241920094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec4a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f678f7fe640 1 -- 172.21.15.2:0/3241920094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558fecb9ae0 con 0x5558fe8754d0 2024-05-14T09:43:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f678e7fc640 1 --2- 172.21.15.2:0/3241920094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec4a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:43:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f678f7fe640 1 --2- 172.21.15.2:0/3241920094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8754d0 0x5558fec4a840 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6788005e30 tx=0x7f67880095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aada4d03189c9cb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:43:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f6772ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:43:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f6772ff5640 1 -- 172.21.15.2:0/3241920094 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67880d3440 con 0x5558fe8754d0 2024-05-14T09:43:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f6772ff5640 1 -- 172.21.15.2:0/3241920094 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f67880cb110 con 0x5558fe8754d0 2024-05-14T09:43:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.817+0000 7f679c563740 1 -- 172.21.15.2:0/3241920094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f678000f9d0 con 0x5558fe8754d0 2024-05-14T09:43:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.818+0000 7f6772ff5640 1 -- 172.21.15.2:0/3241920094 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f67880ca7b0 con 0x5558fe8754d0 2024-05-14T09:43:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.818+0000 7f679c563740 10 client.16470 fetch_fsmap learned FSMap version 38 2024-05-14T09:43:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.818+0000 7f679c563740 1 -- 172.21.15.2:0/3241920094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5558fecbae00 con 0x5558fe8754d0 2024-05-14T09:43:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.818+0000 7f6772ff5640 1 -- 172.21.15.2:0/3241920094 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f678000f9d0 con 0x5558fe8754d0 2024-05-14T09:43:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.818+0000 7f6772ff5640 1 -- 172.21.15.2:0/3241920094 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f67880d3670 con 0x5558fe8754d0 2024-05-14T09:43:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.818+0000 7f679c563740 10 client.16470 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T09:43:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.818+0000 7f679c563740 10 client.16470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:43:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.818+0000 7f679c563740 20 client.16470 populate_metadata read hostname 'smithi002' 2024-05-14T09:43:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.818+0000 7f679c563740 1 --2- 172.21.15.2:0/3241920094 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5558fe8961b0 0x5558fecbd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:43:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.818+0000 7f679c563740 4 client.16470 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:43:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.818+0000 7f679c563740 1 -- 172.21.15.2:0/3241920094 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558feba4050 con 0x5558fe8961b0 2024-05-14T09:43:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.818+0000 7f678effd640 1 --2- 172.21.15.2:0/3241920094 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5558fe8961b0 0x5558fecbd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:43:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.819+0000 7f6772ff5640 10 client.16470 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.819+0000 7f678effd640 1 --2- 172.21.15.2:0/3241920094 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5558fe8961b0 0x5558fecbd7f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3216345a57362856 server_cookie=e5d3d485d4ff7cf8 in_seq=0 out_seq=0 2024-05-14T09:43:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.921+0000 7f6772ff5640 1 -- 172.21.15.2:0/3241920094 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558feba4050 con 0x5558fe8961b0 2024-05-14T09:43:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.921+0000 7f6772ff5640 10 client.16470 handle_command_reply: tid=0 2024-05-14T09:43:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.921+0000 7f679c563740 1 client.16470 shutdown 2024-05-14T09:43:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.921+0000 7f679c563740 1 -- 172.21.15.2:0/3241920094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8754d0 msgr2=0x5558fec4a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.921+0000 7f679c563740 1 --2- 172.21.15.2:0/3241920094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8754d0 0x5558fec4a840 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6788005e30 tx=0x7f67880095c0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f679c563740 1 -- 172.21.15.2:0/3241920094 shutdown_connections 2024-05-14T09:43:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f679c563740 1 --2- 172.21.15.2:0/3241920094 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5558fe8961b0 0x5558fecbd7f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f679c563740 1 --2- 172.21.15.2:0/3241920094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558fe8754d0 0x5558fec4a840 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f679c563740 1 --2- 172.21.15.2:0/3241920094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558fe86d2f0 0x5558fec4a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f6772ff5640 0 client.16470 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:43:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f679c563740 1 --2- 172.21.15.2:0/3241920094 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5558fe854d80 0x5558fec4d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f679c563740 1 -- 172.21.15.2:0/3241920094 >> 172.21.15.2:0/3241920094 conn(0x5558fe88a300 msgr2=0x5558fec45600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f679c563740 1 -- 172.21.15.2:0/3241920094 shutdown_connections 2024-05-14T09:43:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f679c563740 1 -- 172.21.15.2:0/3241920094 wait complete. 2024-05-14T09:43:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f679c563740 20 client.16470 trim_cache size 0 max 16384 2024-05-14T09:43:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:43:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:43:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:43:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:43:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f67955d5640 1 -- 172.21.15.2:0/4124953847 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67780b28a0 msgr2=0x7f67780b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f67955d5640 1 --2- 172.21.15.2:0/4124953847 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67780b28a0 0x7f67780b4db0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5558fec3d250 tx=0x7f6780005f90 comp rx=0 tx=0).stop 2024-05-14T09:43:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f67955d5640 1 -- 172.21.15.2:0/4124953847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6790108a70 msgr2=0x7f67901a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:43:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f67955d5640 1 --2- 172.21.15.2:0/4124953847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6790108a70 0x7f67901a1730 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f678802aeb0 tx=0x7f67880317c0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f67955d5640 1 -- 172.21.15.2:0/4124953847 shutdown_connections 2024-05-14T09:43:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f67955d5640 1 --2- 172.21.15.2:0/4124953847 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67780b28a0 0x7f67780b4db0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f67955d5640 1 --2- 172.21.15.2:0/4124953847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6790108a70 0x7f67901a1730 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f67955d5640 1 --2- 172.21.15.2:0/4124953847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6790103d70 0x7f679019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f67955d5640 1 --2- 172.21.15.2:0/4124953847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6790102b30 0x7f679019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:43:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.922+0000 7f67955d5640 1 -- 172.21.15.2:0/4124953847 >> 172.21.15.2:0/4124953847 conn(0x7f67900fdd20 msgr2=0x7f6790101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:43:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.923+0000 7f67955d5640 1 -- 172.21.15.2:0/4124953847 shutdown_connections 2024-05-14T09:43:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:43:51.923+0000 7f67955d5640 1 -- 172.21.15.2:0/4124953847 wait complete. 2024-05-14T09:43:52.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 830603 10.09 MB/sec execute 318 sec latency 83.455 ms 2024-05-14T09:43:52.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:52 smithi002 ceph-mon[31557]: pgmap v692: 129 pgs: 129 active+clean; 149 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 319 op/s 2024-05-14T09:43:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:52 smithi029 ceph-mon[36249]: pgmap v692: 129 pgs: 129 active+clean; 149 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 319 op/s 2024-05-14T09:43:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:52 smithi026 ceph-mon[36056]: pgmap v692: 129 pgs: 129 active+clean; 149 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 319 op/s 2024-05-14T09:43:53.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 831963 10.08 MB/sec execute 319 sec latency 12.250 ms 2024-05-14T09:43:54.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 834132 10.09 MB/sec execute 320 sec latency 141.283 ms 2024-05-14T09:43:54.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:54 smithi002 ceph-mon[31557]: pgmap v693: 129 pgs: 129 active+clean; 149 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 338 op/s 2024-05-14T09:43:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:54 smithi029 ceph-mon[36249]: pgmap v693: 129 pgs: 129 active+clean; 149 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 338 op/s 2024-05-14T09:43:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:54 smithi026 ceph-mon[36056]: pgmap v693: 129 pgs: 129 active+clean; 149 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 338 op/s 2024-05-14T09:43:55.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 835252 10.07 MB/sec execute 321 sec latency 12.621 ms 2024-05-14T09:43:56.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 837286 10.08 MB/sec execute 322 sec latency 96.124 ms 2024-05-14T09:43:56.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:56 smithi002 ceph-mon[31557]: pgmap v694: 129 pgs: 129 active+clean; 149 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 2.3 MiB/s wr, 255 op/s 2024-05-14T09:43:56.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:56 smithi029 ceph-mon[36249]: pgmap v694: 129 pgs: 129 active+clean; 149 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 2.3 MiB/s wr, 255 op/s 2024-05-14T09:43:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:56 smithi026 ceph-mon[36056]: pgmap v694: 129 pgs: 129 active+clean; 149 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 2.3 MiB/s wr, 255 op/s 2024-05-14T09:43:57.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 838573 10.07 MB/sec execute 323 sec latency 27.518 ms 2024-05-14T09:43:58.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 840838 10.08 MB/sec execute 324 sec latency 16.798 ms 2024-05-14T09:43:58.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:43:58 smithi002 ceph-mon[31557]: pgmap v695: 129 pgs: 129 active+clean; 147 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 315 op/s 2024-05-14T09:43:58.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:43:58 smithi029 ceph-mon[36249]: pgmap v695: 129 pgs: 129 active+clean; 147 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 315 op/s 2024-05-14T09:43:58.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:43:58 smithi026 ceph-mon[36056]: pgmap v695: 129 pgs: 129 active+clean; 147 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 315 op/s 2024-05-14T09:43:59.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 842181 10.07 MB/sec execute 325 sec latency 11.347 ms 2024-05-14T09:44:00.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 844459 10.07 MB/sec execute 326 sec latency 16.818 ms 2024-05-14T09:44:00.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:00 smithi002 ceph-mon[31557]: pgmap v696: 129 pgs: 129 active+clean; 149 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 2.4 MiB/s wr, 277 op/s 2024-05-14T09:44:00.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:00 smithi029 ceph-mon[36249]: pgmap v696: 129 pgs: 129 active+clean; 149 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 2.4 MiB/s wr, 277 op/s 2024-05-14T09:44:00.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:00 smithi026 ceph-mon[36056]: pgmap v696: 129 pgs: 129 active+clean; 149 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 2.4 MiB/s wr, 277 op/s 2024-05-14T09:44:01.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 845557 10.06 MB/sec execute 327 sec latency 66.547 ms 2024-05-14T09:44:02.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 847437 10.05 MB/sec execute 328 sec latency 39.298 ms 2024-05-14T09:44:02.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:02 smithi002 ceph-mon[31557]: pgmap v697: 129 pgs: 129 active+clean; 151 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 2.4 MiB/s wr, 293 op/s 2024-05-14T09:44:02.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:02 smithi029 ceph-mon[36249]: pgmap v697: 129 pgs: 129 active+clean; 151 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 2.4 MiB/s wr, 293 op/s 2024-05-14T09:44:02.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:02 smithi026 ceph-mon[36056]: pgmap v697: 129 pgs: 129 active+clean; 151 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 2.4 MiB/s wr, 293 op/s 2024-05-14T09:44:03.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 849020 10.06 MB/sec execute 329 sec latency 13.476 ms 2024-05-14T09:44:04.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 850981 10.05 MB/sec execute 330 sec latency 34.262 ms 2024-05-14T09:44:04.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:04 smithi002 ceph-mon[31557]: pgmap v698: 129 pgs: 129 active+clean; 152 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 2.8 MiB/s wr, 292 op/s 2024-05-14T09:44:04.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:04 smithi029 ceph-mon[36249]: pgmap v698: 129 pgs: 129 active+clean; 152 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 2.8 MiB/s wr, 292 op/s 2024-05-14T09:44:04.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:04 smithi026 ceph-mon[36056]: pgmap v698: 129 pgs: 129 active+clean; 152 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 2.8 MiB/s wr, 292 op/s 2024-05-14T09:44:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.031+0000 7f42f071f640 1 -- 172.21.15.2:0/819190917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42e8109a20 msgr2=0x7f42e810be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.031+0000 7f42f071f640 1 --2- 172.21.15.2:0/819190917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42e8109a20 0x7f42e810be40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009ca0 tx=0x7f42e002a8c0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.031+0000 7f42f071f640 1 -- 172.21.15.2:0/819190917 shutdown_connections 2024-05-14T09:44:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.031+0000 7f42f071f640 1 --2- 172.21.15.2:0/819190917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42e8109a20 0x7f42e810be40 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.031+0000 7f42f071f640 1 --2- 172.21.15.2:0/819190917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42e81070c0 0x7f42e81094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.031+0000 7f42f071f640 1 --2- 172.21.15.2:0/819190917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42e8104760 0x7f42e8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.031+0000 7f42f071f640 1 -- 172.21.15.2:0/819190917 >> 172.21.15.2:0/819190917 conn(0x7f42e80fdd60 msgr2=0x7f42e8100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.031+0000 7f42f071f640 1 -- 172.21.15.2:0/819190917 shutdown_connections 2024-05-14T09:44:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.031+0000 7f42f071f640 1 -- 172.21.15.2:0/819190917 wait complete. 2024-05-14T09:44:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.031+0000 7f42f071f640 1 Processor -- start 2024-05-14T09:44:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.031+0000 7f42f071f640 1 -- start start 2024-05-14T09:44:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42f071f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42e8104760 0x7f42e819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42f071f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42e81070c0 0x7f42e819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42f071f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42e8109a20 0x7f42e81a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42f071f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f42e810f7a0 con 0x7f42e8109a20 2024-05-14T09:44:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42f071f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f42e810f620 con 0x7f42e8104760 2024-05-14T09:44:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42f071f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f42e810f920 con 0x7f42e81070c0 2024-05-14T09:44:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42eecbc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42e8109a20 0x7f42e81a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42ee4bb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42e8104760 0x7f42e819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42eecbc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42e8109a20 0x7f42e81a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34274/0 (socket says 172.21.15.2:34274) 2024-05-14T09:44:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42eecbc640 1 -- 172.21.15.2:0/2053339116 learned_addr learned my addr 172.21.15.2:0/2053339116 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42edcba640 1 --2- 172.21.15.2:0/2053339116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42e81070c0 0x7f42e819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42ee4bb640 1 -- 172.21.15.2:0/2053339116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42e81070c0 msgr2=0x7f42e819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42ee4bb640 1 --2- 172.21.15.2:0/2053339116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42e81070c0 0x7f42e819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42ee4bb640 1 -- 172.21.15.2:0/2053339116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42e8109a20 msgr2=0x7f42e81a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42ee4bb640 1 --2- 172.21.15.2:0/2053339116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42e8109a20 0x7f42e81a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42ee4bb640 1 -- 172.21.15.2:0/2053339116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e81a1da0 con 0x7f42e8104760 2024-05-14T09:44:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42eecbc640 1 --2- 172.21.15.2:0/2053339116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42e8109a20 0x7f42e81a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:44:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.032+0000 7f42ee4bb640 1 --2- 172.21.15.2:0/2053339116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42e8104760 0x7f42e819a260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f42d800e9c0 tx=0x7f42d800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.033+0000 7f42e77fe640 1 -- 172.21.15.2:0/2053339116 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d800ccc0 con 0x7f42e8104760 2024-05-14T09:44:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.033+0000 7f42e77fe640 1 -- 172.21.15.2:0/2053339116 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42d80108e0 con 0x7f42e8104760 2024-05-14T09:44:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.033+0000 7f42f071f640 1 -- 172.21.15.2:0/2053339116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e81a2090 con 0x7f42e8104760 2024-05-14T09:44:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.033+0000 7f42e77fe640 1 -- 172.21.15.2:0/2053339116 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d80187a0 con 0x7f42e8104760 2024-05-14T09:44:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.033+0000 7f42f071f640 1 -- 172.21.15.2:0/2053339116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e81a25e0 con 0x7f42e8104760 2024-05-14T09:44:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42f75d4740 1 Processor -- start 2024-05-14T09:44:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42f75d4740 1 -- start start 2024-05-14T09:44:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42e77fe640 1 -- 172.21.15.2:0/2053339116 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f42d8010430 con 0x7f42e8104760 2024-05-14T09:44:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42f75d4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbad51b0 0x559cbbe76a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42f75d4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbaac2f0 0x559cbbe76fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42f75d4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbab44d0 0x559cbbe7dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42f75d4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559cbbddb5d0 con 0x559cbbab44d0 2024-05-14T09:44:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42f75d4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559cbbd94890 con 0x559cbbad51b0 2024-05-14T09:44:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42f75d4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559cbbdd7590 con 0x559cbbaac2f0 2024-05-14T09:44:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42ee4bb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbaac2f0 0x559cbbe76fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42edcba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbab44d0 0x559cbbe7dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42eecbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbad51b0 0x559cbbe76a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42eecbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbad51b0 0x559cbbe76a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:56160/0 (socket says 172.21.15.2:56160) 2024-05-14T09:44:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42ee4bb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbaac2f0 0x559cbbe76fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:33844/0 (socket says 172.21.15.2:33844) 2024-05-14T09:44:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42edcba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbab44d0 0x559cbbe7dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34276/0 (socket says 172.21.15.2:34276) 2024-05-14T09:44:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.036+0000 7f42eecbc640 1 -- 172.21.15.2:0/937560498 learned_addr learned my addr 172.21.15.2:0/937560498 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.037+0000 7f42edcba640 1 -- 172.21.15.2:0/937560498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbaac2f0 msgr2=0x559cbbe76fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.037+0000 7f42edcba640 1 --2- 172.21.15.2:0/937560498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbaac2f0 0x559cbbe76fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.037+0000 7f42edcba640 1 -- 172.21.15.2:0/937560498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbad51b0 msgr2=0x559cbbe76a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.037+0000 7f42edcba640 1 --2- 172.21.15.2:0/937560498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbad51b0 0x559cbbe76a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.037+0000 7f42edcba640 1 -- 172.21.15.2:0/937560498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cbbaed210 con 0x559cbbab44d0 2024-05-14T09:44:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.037+0000 7f42ee4bb640 1 --2- 172.21.15.2:0/937560498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbaac2f0 0x559cbbe76fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:44:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.037+0000 7f42e77fe640 1 --2- 172.21.15.2:0/2053339116 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f42d00b28a0 0x7f42d00b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.037+0000 7f42edcba640 1 --2- 172.21.15.2:0/937560498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbab44d0 0x559cbbe7dee0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009bd0 tx=0x7f42dc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be46f774d1273730 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.037+0000 7f42eecbc640 1 --2- 172.21.15.2:0/2053339116 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f42d00b28a0 0x7f42d00b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.037+0000 7f42e77fe640 1 -- 172.21.15.2:0/2053339116 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f42d80e2990 con 0x7f42e8104760 2024-05-14T09:44:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.037+0000 7f42e4ff9640 1 -- 172.21.15.2:0/937560498 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc004230 con 0x559cbbab44d0 2024-05-14T09:44:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.037+0000 7f42e4ff9640 1 -- 172.21.15.2:0/937560498 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42dc009e40 con 0x559cbbab44d0 2024-05-14T09:44:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.037+0000 7f42e4ff9640 1 -- 172.21.15.2:0/937560498 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc004c00 con 0x559cbbab44d0 2024-05-14T09:44:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.038+0000 7f42f75d4740 1 -- 172.21.15.2:0/937560498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbab44d0 msgr2=0x559cbbe7dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.038+0000 7f42f75d4740 1 --2- 172.21.15.2:0/937560498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbab44d0 0x559cbbe7dee0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009bd0 tx=0x7f42dc011810 comp rx=0 tx=0).stop 2024-05-14T09:44:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.038+0000 7f42eecbc640 1 --2- 172.21.15.2:0/2053339116 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f42d00b28a0 0x7f42d00b4db0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f42e00094e0 tx=0x7f42e0009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.038+0000 7f42f75d4740 1 -- 172.21.15.2:0/937560498 shutdown_connections 2024-05-14T09:44:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.038+0000 7f42f75d4740 1 --2- 172.21.15.2:0/937560498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbab44d0 0x559cbbe7dee0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.038+0000 7f42f75d4740 1 --2- 172.21.15.2:0/937560498 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbaac2f0 0x559cbbe76fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.038+0000 7f42f75d4740 1 --2- 172.21.15.2:0/937560498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbad51b0 0x559cbbe76a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.038+0000 7f42f75d4740 1 -- 172.21.15.2:0/937560498 >> 172.21.15.2:0/937560498 conn(0x559cbbac9300 msgr2=0x559cbbe723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42f75d4740 1 -- 172.21.15.2:0/937560498 shutdown_connections 2024-05-14T09:44:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42f75d4740 1 -- 172.21.15.2:0/937560498 wait complete. 2024-05-14T09:44:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42f75d4740 1 -- start start 2024-05-14T09:44:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42f75d4740 1 Processor -- start 2024-05-14T09:44:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42f75d4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbaac2f0 0x559cbbe80640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42f75d4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbab44d0 0x559cbbe80b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42f75d4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbad51b0 0x559cbbe87ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42f75d4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559cbbe7e930 con 0x559cbbad51b0 2024-05-14T09:44:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42f75d4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559cbbd98370 con 0x559cbbab44d0 2024-05-14T09:44:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42f75d4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559cbbe7eab0 con 0x559cbbaac2f0 2024-05-14T09:44:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42f75d4740 10 client.0 _ll_register_callbacks cb 0x559cbbaeca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:44:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42eecbc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbad51b0 0x559cbbe87ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42ee4bb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbaac2f0 0x559cbbe80640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42edcba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbab44d0 0x559cbbe80b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42edcba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbab44d0 0x559cbbe80b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:56164/0 (socket says 172.21.15.2:56164) 2024-05-14T09:44:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42ee4bb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbaac2f0 0x559cbbe80640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:33846/0 (socket says 172.21.15.2:33846) 2024-05-14T09:44:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.039+0000 7f42edcba640 1 -- 172.21.15.2:0/1315489617 learned_addr learned my addr 172.21.15.2:0/1315489617 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.040+0000 7f42edcba640 1 -- 172.21.15.2:0/1315489617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbaac2f0 msgr2=0x559cbbe80640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.040+0000 7f42edcba640 1 --2- 172.21.15.2:0/1315489617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbaac2f0 0x559cbbe80640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.040+0000 7f42edcba640 1 -- 172.21.15.2:0/1315489617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbad51b0 msgr2=0x559cbbe87ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.040+0000 7f42edcba640 1 --2- 172.21.15.2:0/1315489617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbad51b0 0x559cbbe87ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.040+0000 7f42edcba640 1 -- 172.21.15.2:0/1315489617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cbbbb9620 con 0x559cbbab44d0 2024-05-14T09:44:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.040+0000 7f42ee4bb640 1 --2- 172.21.15.2:0/1315489617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbaac2f0 0x559cbbe80640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:44:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.040+0000 7f42edcba640 1 --2- 172.21.15.2:0/1315489617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbab44d0 0x559cbbe80b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f42dc005700 tx=0x7f42dc005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f85f3210824894d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.040+0000 7f42cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:44:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.040+0000 7f42cdffb640 1 -- 172.21.15.2:0/1315489617 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc027020 con 0x559cbbab44d0 2024-05-14T09:44:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.040+0000 7f42f75d4740 1 -- 172.21.15.2:0/1315489617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559cbbde91e0 con 0x559cbbab44d0 2024-05-14T09:44:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.040+0000 7f42cdffb640 1 -- 172.21.15.2:0/1315489617 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f42dc003870 con 0x559cbbab44d0 2024-05-14T09:44:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.040+0000 7f42cdffb640 1 -- 172.21.15.2:0/1315489617 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f42dc032050 con 0x559cbbab44d0 2024-05-14T09:44:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.040+0000 7f42f75d4740 10 client.26162 fetch_fsmap learned FSMap version 39 2024-05-14T09:44:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.040+0000 7f42f75d4740 1 -- 172.21.15.2:0/1315489617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559cbbe8aa60 con 0x559cbbab44d0 2024-05-14T09:44:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.041+0000 7f42cdffb640 1 -- 172.21.15.2:0/1315489617 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559cbbde91e0 con 0x559cbbab44d0 2024-05-14T09:44:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.041+0000 7f42cdffb640 1 -- 172.21.15.2:0/1315489617 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f42dc018070 con 0x559cbbab44d0 2024-05-14T09:44:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.041+0000 7f42f75d4740 10 client.26162 fetch_fsmap finished waiting for FSMap version 39 2024-05-14T09:44:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.041+0000 7f42f75d4740 10 client.26162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:44:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.041+0000 7f42f75d4740 20 client.26162 populate_metadata read hostname 'smithi002' 2024-05-14T09:44:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.041+0000 7f42f75d4740 1 --2- 172.21.15.2:0/1315489617 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559cbba93d80 0x559cbbe8e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.041+0000 7f42f75d4740 4 client.26162 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:44:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.041+0000 7f42f75d4740 1 -- 172.21.15.2:0/1315489617 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cbbde3050 con 0x559cbba93d80 2024-05-14T09:44:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.041+0000 7f42ee4bb640 1 --2- 172.21.15.2:0/1315489617 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559cbba93d80 0x559cbbe8e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.041+0000 7f42ee4bb640 1 --2- 172.21.15.2:0/1315489617 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559cbba93d80 0x559cbbe8e040 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11be04ab5d85c04 server_cookie=6062386d9f460343 in_seq=0 out_seq=0 2024-05-14T09:44:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.041+0000 7f42cdffb640 10 client.26162 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.042+0000 7f42cdffb640 1 -- 172.21.15.2:0/1315489617 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559cbbde3050 con 0x559cbba93d80 2024-05-14T09:44:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.042+0000 7f42cdffb640 10 client.26162 handle_command_reply: tid=0 2024-05-14T09:44:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.042+0000 7f42f75d4740 1 client.26162 shutdown 2024-05-14T09:44:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.042+0000 7f42f75d4740 1 -- 172.21.15.2:0/1315489617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbab44d0 msgr2=0x559cbbe80b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.042+0000 7f42f75d4740 1 --2- 172.21.15.2:0/1315489617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbab44d0 0x559cbbe80b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f42dc005700 tx=0x7f42dc005730 comp rx=0 tx=0).stop 2024-05-14T09:44:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.042+0000 7f42f75d4740 1 -- 172.21.15.2:0/1315489617 shutdown_connections 2024-05-14T09:44:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.042+0000 7f42f75d4740 1 --2- 172.21.15.2:0/1315489617 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559cbba93d80 0x559cbbe8e040 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.042+0000 7f42f75d4740 1 --2- 172.21.15.2:0/1315489617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbad51b0 0x559cbbe87ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.042+0000 7f42f75d4740 1 --2- 172.21.15.2:0/1315489617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbab44d0 0x559cbbe80b80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.042+0000 7f42f75d4740 1 --2- 172.21.15.2:0/1315489617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbaac2f0 0x559cbbe80640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.042+0000 7f42cdffb640 0 client.26162 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.042+0000 7f42f75d4740 1 -- 172.21.15.2:0/1315489617 >> 172.21.15.2:0/1315489617 conn(0x559cbbac9300 msgr2=0x559cbbe713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.042+0000 7f42f75d4740 1 -- 172.21.15.2:0/1315489617 shutdown_connections 2024-05-14T09:44:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.042+0000 7f42f75d4740 1 -- 172.21.15.2:0/1315489617 wait complete. 2024-05-14T09:44:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.042+0000 7f42f75d4740 20 client.26162 trim_cache size 0 max 16384 2024-05-14T09:44:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42f75d4740 1 Processor -- start 2024-05-14T09:44:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42f75d4740 1 -- start start 2024-05-14T09:44:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42f75d4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbaac2f0 0x559cbbe92830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42f75d4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbab44d0 0x559cbbe87a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42f75d4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbad51b0 0x559cbbe87f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42f75d4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81af450 con 0x559cbbaac2f0 2024-05-14T09:44:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42f75d4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81acd20 con 0x559cbbad51b0 2024-05-14T09:44:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42f75d4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81aab70 con 0x559cbbab44d0 2024-05-14T09:44:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42ee4bb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbaac2f0 0x559cbbe92830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42ee4bb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbaac2f0 0x559cbbe92830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34286/0 (socket says 172.21.15.2:34286) 2024-05-14T09:44:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42eecbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbad51b0 0x559cbbe87f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42ee4bb640 1 -- 172.21.15.2:0/2647939569 learned_addr learned my addr 172.21.15.2:0/2647939569 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42edcba640 1 --2- 172.21.15.2:0/2647939569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbab44d0 0x559cbbe87a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42ee4bb640 1 -- 172.21.15.2:0/2647939569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbab44d0 msgr2=0x559cbbe87a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42ee4bb640 1 --2- 172.21.15.2:0/2647939569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbab44d0 0x559cbbe87a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42ee4bb640 1 -- 172.21.15.2:0/2647939569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbad51b0 msgr2=0x559cbbe87f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42ee4bb640 1 --2- 172.21.15.2:0/2647939569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbad51b0 0x559cbbe87f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.059+0000 7f42ee4bb640 1 -- 172.21.15.2:0/2647939569 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cbbe76570 con 0x559cbbaac2f0 2024-05-14T09:44:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.060+0000 7f42ee4bb640 1 --2- 172.21.15.2:0/2647939569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbaac2f0 0x559cbbe92830 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f42d80a5cc0 tx=0x7f42d80dff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc713ce0eae58a86 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.060+0000 7f42cffff640 1 -- 172.21.15.2:0/2647939569 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d80a53d0 con 0x559cbbaac2f0 2024-05-14T09:44:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.060+0000 7f42cffff640 1 -- 172.21.15.2:0/2647939569 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42d8010720 con 0x559cbbaac2f0 2024-05-14T09:44:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.060+0000 7f42cffff640 1 -- 172.21.15.2:0/2647939569 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d80adf40 con 0x559cbbaac2f0 2024-05-14T09:44:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.061+0000 7f42f75d4740 1 -- 172.21.15.2:0/2647939569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbaac2f0 msgr2=0x559cbbe92830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.061+0000 7f42f75d4740 1 --2- 172.21.15.2:0/2647939569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbaac2f0 0x559cbbe92830 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f42d80a5cc0 tx=0x7f42d80dff40 comp rx=0 tx=0).stop 2024-05-14T09:44:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.061+0000 7f42f75d4740 1 -- 172.21.15.2:0/2647939569 shutdown_connections 2024-05-14T09:44:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.061+0000 7f42f75d4740 1 --2- 172.21.15.2:0/2647939569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbad51b0 0x559cbbe87f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.061+0000 7f42f75d4740 1 --2- 172.21.15.2:0/2647939569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbab44d0 0x559cbbe87a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.061+0000 7f42f75d4740 1 --2- 172.21.15.2:0/2647939569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbaac2f0 0x559cbbe92830 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.061+0000 7f42f75d4740 1 -- 172.21.15.2:0/2647939569 >> 172.21.15.2:0/2647939569 conn(0x559cbbac9300 msgr2=0x559cbbe7aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.061+0000 7f42f75d4740 1 -- 172.21.15.2:0/2647939569 shutdown_connections 2024-05-14T09:44:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.061+0000 7f42f75d4740 1 -- 172.21.15.2:0/2647939569 wait complete. 2024-05-14T09:44:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.061+0000 7f42f75d4740 1 -- start start 2024-05-14T09:44:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.061+0000 7f42f75d4740 1 Processor -- start 2024-05-14T09:44:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.061+0000 7f42f75d4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbaac2f0 0x559cbbe87fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.061+0000 7f42f75d4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbab44d0 0x559cbbe86640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42f75d4740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbad51b0 0x559cbbe86b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42f75d4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81ab1e0 con 0x559cbbab44d0 2024-05-14T09:44:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42f75d4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81ac650 con 0x559cbbaac2f0 2024-05-14T09:44:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42f75d4740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81ab8b0 con 0x559cbbad51b0 2024-05-14T09:44:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42f75d4740 10 client.0 _ll_register_callbacks cb 0x7f42dc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:44:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42edcba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbab44d0 0x559cbbe86640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42edcba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbab44d0 0x559cbbe86640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34302/0 (socket says 172.21.15.2:34302) 2024-05-14T09:44:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42ee4bb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbaac2f0 0x559cbbe87fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42edcba640 1 -- 172.21.15.2:0/1976557812 learned_addr learned my addr 172.21.15.2:0/1976557812 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42eecbc640 1 --2- 172.21.15.2:0/1976557812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbad51b0 0x559cbbe86b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42edcba640 1 -- 172.21.15.2:0/1976557812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbad51b0 msgr2=0x559cbbe86b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42edcba640 1 --2- 172.21.15.2:0/1976557812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbad51b0 0x559cbbe86b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42edcba640 1 -- 172.21.15.2:0/1976557812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbaac2f0 msgr2=0x559cbbe87fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42edcba640 1 --2- 172.21.15.2:0/1976557812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbaac2f0 0x559cbbe87fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42edcba640 1 -- 172.21.15.2:0/1976557812 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cbbe87170 con 0x559cbbab44d0 2024-05-14T09:44:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42ee4bb640 1 --2- 172.21.15.2:0/1976557812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbaac2f0 0x559cbbe87fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:44:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42edcba640 1 --2- 172.21.15.2:0/1976557812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbab44d0 0x559cbbe86640 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f42dc005e00 tx=0x7f42dc0043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d682b3a8e43e23 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:44:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42cdffb640 1 -- 172.21.15.2:0/1976557812 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc027040 con 0x559cbbab44d0 2024-05-14T09:44:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42cdffb640 1 -- 172.21.15.2:0/1976557812 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f42dc0237f0 con 0x559cbbab44d0 2024-05-14T09:44:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42f75d4740 1 -- 172.21.15.2:0/1976557812 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559cbbd5e5f0 con 0x559cbbab44d0 2024-05-14T09:44:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42cdffb640 1 -- 172.21.15.2:0/1976557812 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f42dc01e040 con 0x559cbbab44d0 2024-05-14T09:44:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42f75d4740 10 client.16488 fetch_fsmap learned FSMap version 39 2024-05-14T09:44:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.062+0000 7f42f75d4740 1 -- 172.21.15.2:0/1976557812 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559cbbef5190 con 0x559cbbab44d0 2024-05-14T09:44:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.063+0000 7f42cdffb640 1 -- 172.21.15.2:0/1976557812 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559cbbd5e5f0 con 0x559cbbab44d0 2024-05-14T09:44:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.063+0000 7f42cdffb640 1 -- 172.21.15.2:0/1976557812 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f42dc018040 con 0x559cbbab44d0 2024-05-14T09:44:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.063+0000 7f42f75d4740 10 client.16488 fetch_fsmap finished waiting for FSMap version 39 2024-05-14T09:44:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.063+0000 7f42f75d4740 10 client.16488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:44:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.063+0000 7f42f75d4740 20 client.16488 populate_metadata read hostname 'smithi002' 2024-05-14T09:44:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.063+0000 7f42f75d4740 1 --2- 172.21.15.2:0/1976557812 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559cbba691c0 0x559cbbef7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.063+0000 7f42f75d4740 4 client.16488 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:44:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.063+0000 7f42f75d4740 1 -- 172.21.15.2:0/1976557812 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559cbbdabc80 con 0x559cbba691c0 2024-05-14T09:44:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.063+0000 7f42ee4bb640 1 --2- 172.21.15.2:0/1976557812 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559cbba691c0 0x559cbbef7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.063+0000 7f42ee4bb640 1 --2- 172.21.15.2:0/1976557812 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559cbba691c0 0x559cbbef7ad0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f371a029840315 server_cookie=a902569e8d02dfde in_seq=0 out_seq=0 2024-05-14T09:44:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.063+0000 7f42cdffb640 10 client.16488 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.167+0000 7f42cdffb640 1 -- 172.21.15.2:0/1976557812 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559cbbdabc80 con 0x559cbba691c0 2024-05-14T09:44:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.167+0000 7f42cdffb640 10 client.16488 handle_command_reply: tid=0 2024-05-14T09:44:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.167+0000 7f42f75d4740 1 client.16488 shutdown 2024-05-14T09:44:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42f75d4740 1 -- 172.21.15.2:0/1976557812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbab44d0 msgr2=0x559cbbe86640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42f75d4740 1 --2- 172.21.15.2:0/1976557812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbab44d0 0x559cbbe86640 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f42dc005e00 tx=0x7f42dc0043d0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42f75d4740 1 -- 172.21.15.2:0/1976557812 shutdown_connections 2024-05-14T09:44:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42f75d4740 1 --2- 172.21.15.2:0/1976557812 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559cbba691c0 0x559cbbef7ad0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42f75d4740 1 --2- 172.21.15.2:0/1976557812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559cbbad51b0 0x559cbbe86b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42f75d4740 1 --2- 172.21.15.2:0/1976557812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559cbbab44d0 0x559cbbe86640 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42cdffb640 0 client.16488 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42f75d4740 1 --2- 172.21.15.2:0/1976557812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559cbbaac2f0 0x559cbbe87fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42f75d4740 1 -- 172.21.15.2:0/1976557812 >> 172.21.15.2:0/1976557812 conn(0x559cbbac9300 msgr2=0x559cbbe751f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42f75d4740 1 -- 172.21.15.2:0/1976557812 shutdown_connections 2024-05-14T09:44:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42f75d4740 1 -- 172.21.15.2:0/1976557812 wait complete. 2024-05-14T09:44:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42f75d4740 20 client.16488 trim_cache size 0 max 16384 2024-05-14T09:44:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:44:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:44:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:44:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:44:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42f071f640 1 -- 172.21.15.2:0/2053339116 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f42d00b28a0 msgr2=0x7f42d00b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42f071f640 1 --2- 172.21.15.2:0/2053339116 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f42d00b28a0 0x7f42d00b4db0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f42e00094e0 tx=0x7f42e0009360 comp rx=0 tx=0).stop 2024-05-14T09:44:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42f071f640 1 -- 172.21.15.2:0/2053339116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42e8104760 msgr2=0x7f42e819a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.168+0000 7f42f071f640 1 --2- 172.21.15.2:0/2053339116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42e8104760 0x7f42e819a260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f42d800e9c0 tx=0x7f42d800ee80 comp rx=0 tx=0).stop 2024-05-14T09:44:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.169+0000 7f42f071f640 1 -- 172.21.15.2:0/2053339116 shutdown_connections 2024-05-14T09:44:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.169+0000 7f42f071f640 1 --2- 172.21.15.2:0/2053339116 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f42d00b28a0 0x7f42d00b4db0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.169+0000 7f42f071f640 1 --2- 172.21.15.2:0/2053339116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42e8109a20 0x7f42e81a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.169+0000 7f42f071f640 1 --2- 172.21.15.2:0/2053339116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f42e81070c0 0x7f42e819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.169+0000 7f42f071f640 1 --2- 172.21.15.2:0/2053339116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42e8104760 0x7f42e819a260 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.169+0000 7f42f071f640 1 -- 172.21.15.2:0/2053339116 >> 172.21.15.2:0/2053339116 conn(0x7f42e80fdd60 msgr2=0x7f42e81055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.169+0000 7f42f071f640 1 -- 172.21.15.2:0/2053339116 shutdown_connections 2024-05-14T09:44:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:05.169+0000 7f42f071f640 1 -- 172.21.15.2:0/2053339116 wait complete. 2024-05-14T09:44:05.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 852506 10.06 MB/sec execute 331 sec latency 65.559 ms 2024-05-14T09:44:05.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:05 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:44:05.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:05 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:44:05.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:05 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:44:06.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 854509 10.05 MB/sec execute 332 sec latency 89.147 ms 2024-05-14T09:44:06.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:06 smithi002 ceph-mon[31557]: pgmap v699: 129 pgs: 129 active+clean; 152 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 2.1 MiB/s wr, 221 op/s 2024-05-14T09:44:06.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:06 smithi029 ceph-mon[36249]: pgmap v699: 129 pgs: 129 active+clean; 152 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 2.1 MiB/s wr, 221 op/s 2024-05-14T09:44:06.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:06 smithi026 ceph-mon[36056]: pgmap v699: 129 pgs: 129 active+clean; 152 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 2.1 MiB/s wr, 221 op/s 2024-05-14T09:44:07.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 856079 10.05 MB/sec execute 333 sec latency 31.888 ms 2024-05-14T09:44:08.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 858249 10.05 MB/sec execute 334 sec latency 12.490 ms 2024-05-14T09:44:08.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:08 smithi002 ceph-mon[31557]: pgmap v700: 129 pgs: 129 active+clean; 154 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 315 op/s 2024-05-14T09:44:08.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:08 smithi029 ceph-mon[36249]: pgmap v700: 129 pgs: 129 active+clean; 154 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 315 op/s 2024-05-14T09:44:08.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:08 smithi026 ceph-mon[36056]: pgmap v700: 129 pgs: 129 active+clean; 154 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 315 op/s 2024-05-14T09:44:09.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 859805 10.05 MB/sec execute 335 sec latency 10.553 ms 2024-05-14T09:44:10.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 862195 10.06 MB/sec execute 336 sec latency 22.430 ms 2024-05-14T09:44:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:10 smithi002 ceph-mon[31557]: pgmap v701: 129 pgs: 129 active+clean; 151 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 2.6 MiB/s wr, 303 op/s 2024-05-14T09:44:10.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:10 smithi029 ceph-mon[36249]: pgmap v701: 129 pgs: 129 active+clean; 151 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 2.6 MiB/s wr, 303 op/s 2024-05-14T09:44:10.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:10 smithi026 ceph-mon[36056]: pgmap v701: 129 pgs: 129 active+clean; 151 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 2.6 MiB/s wr, 303 op/s 2024-05-14T09:44:11.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 863525 10.05 MB/sec execute 337 sec latency 48.424 ms 2024-05-14T09:44:12.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 865932 10.06 MB/sec execute 338 sec latency 33.995 ms 2024-05-14T09:44:12.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:12 smithi002 ceph-mon[31557]: pgmap v702: 129 pgs: 129 active+clean; 151 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 2.5 MiB/s wr, 283 op/s 2024-05-14T09:44:12.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:12 smithi029 ceph-mon[36249]: pgmap v702: 129 pgs: 129 active+clean; 151 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 2.5 MiB/s wr, 283 op/s 2024-05-14T09:44:12.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:12 smithi026 ceph-mon[36056]: pgmap v702: 129 pgs: 129 active+clean; 151 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 2.5 MiB/s wr, 283 op/s 2024-05-14T09:44:13.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 867237 10.05 MB/sec execute 339 sec latency 13.076 ms 2024-05-14T09:44:14.274 INFO:tasks.workunit.client.0.smithi002.stdout: 1 869943 10.07 MB/sec execute 340 sec latency 18.748 ms 2024-05-14T09:44:14.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:14 smithi002 ceph-mon[31557]: pgmap v703: 129 pgs: 129 active+clean; 153 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-05-14T09:44:14.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:14 smithi029 ceph-mon[36249]: pgmap v703: 129 pgs: 129 active+clean; 153 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-05-14T09:44:14.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:14 smithi026 ceph-mon[36056]: pgmap v703: 129 pgs: 129 active+clean; 153 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-05-14T09:44:15.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 871480 10.06 MB/sec execute 341 sec latency 29.189 ms 2024-05-14T09:44:16.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 873626 10.07 MB/sec execute 342 sec latency 57.403 ms 2024-05-14T09:44:16.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:16 smithi002 ceph-mon[31557]: pgmap v704: 129 pgs: 129 active+clean; 153 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 2.1 MiB/s wr, 252 op/s 2024-05-14T09:44:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:16 smithi029 ceph-mon[36249]: pgmap v704: 129 pgs: 129 active+clean; 153 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 2.1 MiB/s wr, 252 op/s 2024-05-14T09:44:16.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:16 smithi026 ceph-mon[36056]: pgmap v704: 129 pgs: 129 active+clean; 153 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 2.1 MiB/s wr, 252 op/s 2024-05-14T09:44:17.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 874914 10.05 MB/sec execute 343 sec latency 24.005 ms 2024-05-14T09:44:18.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 877175 10.06 MB/sec execute 344 sec latency 17.388 ms 2024-05-14T09:44:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.275+0000 7f72641d6640 1 -- 172.21.15.2:0/3075806315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c102b50 msgr2=0x7f725c103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.275+0000 7f72641d6640 1 --2- 172.21.15.2:0/3075806315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c102b50 0x7f725c103850 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7254009900 tx=0x7f725402a410 comp rx=0 tx=0).stop 2024-05-14T09:44:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.275+0000 7f72641d6640 1 -- 172.21.15.2:0/3075806315 shutdown_connections 2024-05-14T09:44:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.275+0000 7f72641d6640 1 --2- 172.21.15.2:0/3075806315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f725c108a90 0x7f725c10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.275+0000 7f72641d6640 1 --2- 172.21.15.2:0/3075806315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f725c103d90 0x7f725c1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.275+0000 7f72641d6640 1 --2- 172.21.15.2:0/3075806315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c102b50 0x7f725c103850 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.275+0000 7f72641d6640 1 -- 172.21.15.2:0/3075806315 >> 172.21.15.2:0/3075806315 conn(0x7f725c0fdd40 msgr2=0x7f725c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.276+0000 7f72641d6640 1 -- 172.21.15.2:0/3075806315 shutdown_connections 2024-05-14T09:44:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.276+0000 7f72641d6640 1 -- 172.21.15.2:0/3075806315 wait complete. 2024-05-14T09:44:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.276+0000 7f72641d6640 1 Processor -- start 2024-05-14T09:44:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.276+0000 7f72641d6640 1 -- start start 2024-05-14T09:44:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f72641d6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c102b50 0x7f725c06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f72641d6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f725c103d90 0x7f725c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f72641d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f725c108a90 0x7f725c1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f72641d6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f725c10e650 con 0x7f725c102b50 2024-05-14T09:44:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f72641d6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f725c10e4d0 con 0x7f725c108a90 2024-05-14T09:44:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f72641d6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f725c10e7d0 con 0x7f725c103d90 2024-05-14T09:44:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f7261f72640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c102b50 0x7f725c06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f7261f72640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c102b50 0x7f725c06b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40060/0 (socket says 172.21.15.2:40060) 2024-05-14T09:44:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f7261f72640 1 -- 172.21.15.2:0/2836853893 learned_addr learned my addr 172.21.15.2:0/2836853893 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f7262773640 1 --2- 172.21.15.2:0/2836853893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f725c108a90 0x7f725c1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f7261771640 1 --2- 172.21.15.2:0/2836853893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f725c103d90 0x7f725c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f7261f72640 1 -- 172.21.15.2:0/2836853893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f725c103d90 msgr2=0x7f725c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f7261f72640 1 --2- 172.21.15.2:0/2836853893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f725c103d90 0x7f725c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f7261f72640 1 -- 172.21.15.2:0/2836853893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f725c108a90 msgr2=0x7f725c1a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f7261f72640 1 --2- 172.21.15.2:0/2836853893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f725c108a90 0x7f725c1a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f7261f72640 1 -- 172.21.15.2:0/2836853893 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c1a1de0 con 0x7f725c102b50 2024-05-14T09:44:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f7262773640 1 --2- 172.21.15.2:0/2836853893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f725c108a90 0x7f725c1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:44:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.277+0000 7f7261f72640 1 --2- 172.21.15.2:0/2836853893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c102b50 0x7f725c06b0d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7254006010 tx=0x7f7254004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.278+0000 7f725affd640 1 -- 172.21.15.2:0/2836853893 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725403f030 con 0x7f725c102b50 2024-05-14T09:44:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.278+0000 7f72641d6640 1 -- 172.21.15.2:0/2836853893 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725c1a2070 con 0x7f725c102b50 2024-05-14T09:44:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.278+0000 7f725affd640 1 -- 172.21.15.2:0/2836853893 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7254031910 con 0x7f725c102b50 2024-05-14T09:44:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.278+0000 7f725affd640 1 -- 172.21.15.2:0/2836853893 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7254035070 con 0x7f725c102b50 2024-05-14T09:44:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.278+0000 7f72641d6640 1 -- 172.21.15.2:0/2836853893 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f725c1a25b0 con 0x7f725c102b50 2024-05-14T09:44:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.280+0000 7f726b085740 1 Processor -- start 2024-05-14T09:44:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f726b085740 1 -- start start 2024-05-14T09:44:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f725affd640 1 -- 172.21.15.2:0/2836853893 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f7254042030 con 0x7f725c102b50 2024-05-14T09:44:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f726b085740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9d01b0 0x55ceffd71a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f726b085740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9a72f0 0x55ceffd71fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f726b085740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9af4d0 0x55ceffd78ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f726b085740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ceffcd65d0 con 0x55ceff9d01b0 2024-05-14T09:44:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f726b085740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ceffc8f890 con 0x55ceff9af4d0 2024-05-14T09:44:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f726b085740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ceffcd2590 con 0x55ceff9a72f0 2024-05-14T09:44:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f7262773640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9d01b0 0x55ceffd71a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f7262773640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9d01b0 0x55ceffd71a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40068/0 (socket says 172.21.15.2:40068) 2024-05-14T09:44:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f7262773640 1 -- 172.21.15.2:0/1347671262 learned_addr learned my addr 172.21.15.2:0/1347671262 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f7261f72640 1 --2- 172.21.15.2:0/1347671262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9a72f0 0x55ceffd71fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f7261771640 1 --2- 172.21.15.2:0/1347671262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9af4d0 0x55ceffd78ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f7262773640 1 -- 172.21.15.2:0/1347671262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9a72f0 msgr2=0x55ceffd71fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f7262773640 1 --2- 172.21.15.2:0/1347671262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9a72f0 0x55ceffd71fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f7262773640 1 -- 172.21.15.2:0/1347671262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9af4d0 msgr2=0x55ceffd78ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f7262773640 1 --2- 172.21.15.2:0/1347671262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9af4d0 0x55ceffd78ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.281+0000 7f7262773640 1 -- 172.21.15.2:0/1347671262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ceff9e8210 con 0x55ceff9d01b0 2024-05-14T09:44:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.282+0000 7f7262773640 1 --2- 172.21.15.2:0/1347671262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9d01b0 0x55ceffd71a60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f724c0117e0 tx=0x7f724c011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775a893454634959 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.282+0000 7f7243fff640 1 -- 172.21.15.2:0/1347671262 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c011310 con 0x55ceff9d01b0 2024-05-14T09:44:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.282+0000 7f7243fff640 1 -- 172.21.15.2:0/1347671262 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f724c003e20 con 0x55ceff9d01b0 2024-05-14T09:44:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.282+0000 7f7243fff640 1 -- 172.21.15.2:0/1347671262 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c0044f0 con 0x55ceff9d01b0 2024-05-14T09:44:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.282+0000 7f725affd640 1 --2- 172.21.15.2:0/2836853893 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f72440b2920 0x7f72440b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.282+0000 7f7261f72640 1 --2- 172.21.15.2:0/2836853893 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f72440b2920 0x7f72440b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.282+0000 7f725affd640 1 -- 172.21.15.2:0/2836853893 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f72541028d0 con 0x7f725c102b50 2024-05-14T09:44:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.283+0000 7f726b085740 1 -- 172.21.15.2:0/1347671262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9d01b0 msgr2=0x55ceffd71a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.283+0000 7f7261f72640 1 --2- 172.21.15.2:0/2836853893 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f72440b2920 0x7f72440b4e30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55ceffd72e40 tx=0x7f7254108550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.283+0000 7f726b085740 1 --2- 172.21.15.2:0/1347671262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9d01b0 0x55ceffd71a60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f724c0117e0 tx=0x7f724c011ca0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.283+0000 7f726b085740 1 -- 172.21.15.2:0/1347671262 shutdown_connections 2024-05-14T09:44:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.283+0000 7f726b085740 1 --2- 172.21.15.2:0/1347671262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9af4d0 0x55ceffd78ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.283+0000 7f726b085740 1 --2- 172.21.15.2:0/1347671262 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9a72f0 0x55ceffd71fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.283+0000 7f726b085740 1 --2- 172.21.15.2:0/1347671262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9d01b0 0x55ceffd71a60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.283+0000 7f726b085740 1 -- 172.21.15.2:0/1347671262 >> 172.21.15.2:0/1347671262 conn(0x55ceff9c4300 msgr2=0x55ceffd6d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.283+0000 7f726b085740 1 -- 172.21.15.2:0/1347671262 shutdown_connections 2024-05-14T09:44:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.283+0000 7f726b085740 1 -- 172.21.15.2:0/1347671262 wait complete. 2024-05-14T09:44:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.283+0000 7f726b085740 1 -- start start 2024-05-14T09:44:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f726b085740 1 Processor -- start 2024-05-14T09:44:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f726b085740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9a72f0 0x55ceffd7b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f726b085740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9af4d0 0x55ceffd7bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f726b085740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9d01b0 0x55ceffd82b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f726b085740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ceffd79930 con 0x55ceff9a72f0 2024-05-14T09:44:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f726b085740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ceffc93370 con 0x55ceff9d01b0 2024-05-14T09:44:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f726b085740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ceffd79ab0 con 0x55ceff9af4d0 2024-05-14T09:44:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f726b085740 10 client.0 _ll_register_callbacks cb 0x55ceff9e7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:44:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f7262773640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9a72f0 0x55ceffd7b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f7262773640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9a72f0 0x55ceffd7b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40082/0 (socket says 172.21.15.2:40082) 2024-05-14T09:44:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f7261771640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9af4d0 0x55ceffd7bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f7262773640 1 -- 172.21.15.2:0/438093422 learned_addr learned my addr 172.21.15.2:0/438093422 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f7262773640 1 --2- 172.21.15.2:0/438093422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9d01b0 0x55ceffd82b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f7261771640 1 -- 172.21.15.2:0/438093422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9d01b0 msgr2=0x55ceffd82b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f7261771640 1 --2- 172.21.15.2:0/438093422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9d01b0 0x55ceffd82b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f7261771640 1 -- 172.21.15.2:0/438093422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9a72f0 msgr2=0x55ceffd7b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f7261771640 1 --2- 172.21.15.2:0/438093422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9a72f0 0x55ceffd7b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.284+0000 7f7261771640 1 -- 172.21.15.2:0/438093422 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ceffab4620 con 0x55ceff9af4d0 2024-05-14T09:44:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.285+0000 7f7261771640 1 --2- 172.21.15.2:0/438093422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9af4d0 0x55ceffd7bbe0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f725c06aa70 tx=0x7f7250012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16db992b7adbe22c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.285+0000 7f72417fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:44:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.285+0000 7f72417fa640 1 -- 172.21.15.2:0/438093422 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725000f920 con 0x55ceff9af4d0 2024-05-14T09:44:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.285+0000 7f72417fa640 1 -- 172.21.15.2:0/438093422 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f725000fac0 con 0x55ceff9af4d0 2024-05-14T09:44:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.285+0000 7f726b085740 1 -- 172.21.15.2:0/438093422 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f724c003e20 con 0x55ceff9af4d0 2024-05-14T09:44:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.285+0000 7f72417fa640 1 -- 172.21.15.2:0/438093422 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7250025050 con 0x55ceff9af4d0 2024-05-14T09:44:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.285+0000 7f726b085740 10 client.26164 fetch_fsmap learned FSMap version 39 2024-05-14T09:44:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.285+0000 7f726b085740 1 -- 172.21.15.2:0/438093422 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ceffd85b40 con 0x55ceff9af4d0 2024-05-14T09:44:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.285+0000 7f72417fa640 1 -- 172.21.15.2:0/438093422 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f724c003e20 con 0x55ceff9af4d0 2024-05-14T09:44:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.285+0000 7f72417fa640 1 -- 172.21.15.2:0/438093422 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f7250010430 con 0x55ceff9af4d0 2024-05-14T09:44:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.286+0000 7f726b085740 10 client.26164 fetch_fsmap finished waiting for FSMap version 39 2024-05-14T09:44:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.286+0000 7f726b085740 10 client.26164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:44:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.286+0000 7f726b085740 20 client.26164 populate_metadata read hostname 'smithi002' 2024-05-14T09:44:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.286+0000 7f726b085740 1 --2- 172.21.15.2:0/438093422 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ceff98ed80 0x55ceffd892a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.286+0000 7f726b085740 4 client.26164 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:44:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.286+0000 7f726b085740 1 -- 172.21.15.2:0/438093422 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ceffce41e0 con 0x55ceff98ed80 2024-05-14T09:44:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.286+0000 7f7262773640 1 --2- 172.21.15.2:0/438093422 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ceff98ed80 0x55ceffd892a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.286+0000 7f72417fa640 10 client.26164 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.286+0000 7f7262773640 1 --2- 172.21.15.2:0/438093422 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ceff98ed80 0x55ceffd892a0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a62b9883d7a81826 server_cookie=730e64479f7b516e in_seq=0 out_seq=0 2024-05-14T09:44:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.287+0000 7f72417fa640 1 -- 172.21.15.2:0/438093422 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ceffce41e0 con 0x55ceff98ed80 2024-05-14T09:44:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.287+0000 7f72417fa640 10 client.26164 handle_command_reply: tid=0 2024-05-14T09:44:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.287+0000 7f726b085740 1 client.26164 shutdown 2024-05-14T09:44:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.287+0000 7f726b085740 1 -- 172.21.15.2:0/438093422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9af4d0 msgr2=0x55ceffd7bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.287+0000 7f726b085740 1 --2- 172.21.15.2:0/438093422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9af4d0 0x55ceffd7bbe0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f725c06aa70 tx=0x7f7250012b60 comp rx=0 tx=0).stop 2024-05-14T09:44:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.287+0000 7f726b085740 1 -- 172.21.15.2:0/438093422 shutdown_connections 2024-05-14T09:44:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.287+0000 7f726b085740 1 --2- 172.21.15.2:0/438093422 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ceff98ed80 0x55ceffd892a0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.287+0000 7f726b085740 1 --2- 172.21.15.2:0/438093422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9d01b0 0x55ceffd82b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.287+0000 7f726b085740 1 --2- 172.21.15.2:0/438093422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9af4d0 0x55ceffd7bbe0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.287+0000 7f726b085740 1 --2- 172.21.15.2:0/438093422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9a72f0 0x55ceffd7b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.287+0000 7f72417fa640 0 client.26164 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.287+0000 7f726b085740 1 -- 172.21.15.2:0/438093422 >> 172.21.15.2:0/438093422 conn(0x55ceff9c4300 msgr2=0x55ceffd70300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.287+0000 7f726b085740 1 -- 172.21.15.2:0/438093422 shutdown_connections 2024-05-14T09:44:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.287+0000 7f726b085740 1 -- 172.21.15.2:0/438093422 wait complete. 2024-05-14T09:44:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.287+0000 7f726b085740 20 client.26164 trim_cache size 0 max 16384 2024-05-14T09:44:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.304+0000 7f726b085740 1 Processor -- start 2024-05-14T09:44:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.304+0000 7f726b085740 1 -- start start 2024-05-14T09:44:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.304+0000 7f726b085740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9a72f0 0x55ceffd8da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.304+0000 7f726b085740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9af4d0 0x55ceffd82a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.304+0000 7f726b085740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9d01b0 0x55ceffd82fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.304+0000 7f726b085740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1b16d0 con 0x55ceff9a72f0 2024-05-14T09:44:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.304+0000 7f726b085740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1aefa0 con 0x55ceff9af4d0 2024-05-14T09:44:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.304+0000 7f726b085740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1acdf0 con 0x55ceff9d01b0 2024-05-14T09:44:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f7262773640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9a72f0 0x55ceffd8da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f7261771640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9af4d0 0x55ceffd82a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f7262773640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9a72f0 0x55ceffd8da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40094/0 (socket says 172.21.15.2:40094) 2024-05-14T09:44:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f7261771640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9af4d0 0x55ceffd82a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:58558/0 (socket says 172.21.15.2:58558) 2024-05-14T09:44:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f7262773640 1 -- 172.21.15.2:0/1130068898 learned_addr learned my addr 172.21.15.2:0/1130068898 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f7262773640 1 --2- 172.21.15.2:0/1130068898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9d01b0 0x55ceffd82fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f7261771640 1 -- 172.21.15.2:0/1130068898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9d01b0 msgr2=0x55ceffd82fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f7261771640 1 --2- 172.21.15.2:0/1130068898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9d01b0 0x55ceffd82fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f7261771640 1 -- 172.21.15.2:0/1130068898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9a72f0 msgr2=0x55ceffd8da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f7261771640 1 --2- 172.21.15.2:0/1130068898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9a72f0 0x55ceffd8da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f7261771640 1 -- 172.21.15.2:0/1130068898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ceffd723b0 con 0x55ceff9af4d0 2024-05-14T09:44:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f7262773640 1 --2- 172.21.15.2:0/1130068898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9a72f0 0x55ceffd8da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:44:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f7261771640 1 --2- 172.21.15.2:0/1130068898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9af4d0 0x55ceffd82a60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55ceffd72910 tx=0x7f7250009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8808653a1e3c47d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f72437fe640 1 -- 172.21.15.2:0/1130068898 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250010430 con 0x55ceff9af4d0 2024-05-14T09:44:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f72437fe640 1 -- 172.21.15.2:0/1130068898 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72500105d0 con 0x55ceff9af4d0 2024-05-14T09:44:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.305+0000 7f72437fe640 1 -- 172.21.15.2:0/1130068898 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725000fc30 con 0x55ceff9af4d0 2024-05-14T09:44:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.306+0000 7f726b085740 1 -- 172.21.15.2:0/1130068898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9af4d0 msgr2=0x55ceffd82a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.306+0000 7f726b085740 1 --2- 172.21.15.2:0/1130068898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9af4d0 0x55ceffd82a60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55ceffd72910 tx=0x7f7250009ea0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.306+0000 7f726b085740 1 -- 172.21.15.2:0/1130068898 shutdown_connections 2024-05-14T09:44:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.306+0000 7f726b085740 1 --2- 172.21.15.2:0/1130068898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9d01b0 0x55ceffd82fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.306+0000 7f726b085740 1 --2- 172.21.15.2:0/1130068898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9af4d0 0x55ceffd82a60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.306+0000 7f726b085740 1 --2- 172.21.15.2:0/1130068898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9a72f0 0x55ceffd8da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.306+0000 7f726b085740 1 -- 172.21.15.2:0/1130068898 >> 172.21.15.2:0/1130068898 conn(0x55ceff9c4300 msgr2=0x55ceffd70c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.306+0000 7f726b085740 1 -- 172.21.15.2:0/1130068898 shutdown_connections 2024-05-14T09:44:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.306+0000 7f726b085740 1 -- 172.21.15.2:0/1130068898 wait complete. 2024-05-14T09:44:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.307+0000 7f726b085740 1 -- start start 2024-05-14T09:44:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.307+0000 7f726b085740 1 Processor -- start 2024-05-14T09:44:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.307+0000 7f726b085740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9a72f0 0x55ceffd82340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.307+0000 7f726b085740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9af4d0 0x55ceffd82880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.307+0000 7f726b085740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9d01b0 0x55ceffdef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.307+0000 7f726b085740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1ad460 con 0x55ceff9d01b0 2024-05-14T09:44:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.307+0000 7f726b085740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1ae8d0 con 0x55ceff9a72f0 2024-05-14T09:44:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.307+0000 7f726b085740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1adb30 con 0x55ceff9af4d0 2024-05-14T09:44:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.307+0000 7f726b085740 10 client.0 _ll_register_callbacks cb 0x7f7250009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:44:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.307+0000 7f7262773640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9a72f0 0x55ceffd82340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.307+0000 7f7262773640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9a72f0 0x55ceffd82340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:58568/0 (socket says 172.21.15.2:58568) 2024-05-14T09:44:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.307+0000 7f7262773640 1 -- 172.21.15.2:0/448906237 learned_addr learned my addr 172.21.15.2:0/448906237 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.307+0000 7f7261771640 1 --2- 172.21.15.2:0/448906237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9af4d0 0x55ceffd82880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.307+0000 7f7262773640 1 --2- 172.21.15.2:0/448906237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9d01b0 0x55ceffdef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.308+0000 7f7262773640 1 -- 172.21.15.2:0/448906237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9af4d0 msgr2=0x55ceffd82880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.308+0000 7f7262773640 1 --2- 172.21.15.2:0/448906237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9af4d0 0x55ceffd82880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.308+0000 7f7262773640 1 -- 172.21.15.2:0/448906237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9d01b0 msgr2=0x55ceffdef5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.308+0000 7f7262773640 1 --2- 172.21.15.2:0/448906237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9d01b0 0x55ceffdef5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.308+0000 7f7262773640 1 -- 172.21.15.2:0/448906237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ceffdefb60 con 0x55ceff9a72f0 2024-05-14T09:44:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.308+0000 7f7262773640 1 --2- 172.21.15.2:0/448906237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9a72f0 0x55ceffd82340 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f724c005b40 tx=0x7f724c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b2b2a3051121593 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.308+0000 7f72417fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:44:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.308+0000 7f72417fa640 1 -- 172.21.15.2:0/448906237 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c023040 con 0x55ceff9a72f0 2024-05-14T09:44:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.308+0000 7f72417fa640 1 -- 172.21.15.2:0/448906237 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f724c00fe40 con 0x55ceff9a72f0 2024-05-14T09:44:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.308+0000 7f726b085740 1 -- 172.21.15.2:0/448906237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72500105d0 con 0x55ceff9a72f0 2024-05-14T09:44:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.308+0000 7f72417fa640 1 -- 172.21.15.2:0/448906237 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f724c005070 con 0x55ceff9a72f0 2024-05-14T09:44:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.308+0000 7f726b085740 10 client.26186 fetch_fsmap learned FSMap version 39 2024-05-14T09:44:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.308+0000 7f726b085740 1 -- 172.21.15.2:0/448906237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ceffdf0d90 con 0x55ceff9a72f0 2024-05-14T09:44:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.308+0000 7f72417fa640 1 -- 172.21.15.2:0/448906237 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72500105d0 con 0x55ceff9a72f0 2024-05-14T09:44:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.308+0000 7f72417fa640 1 -- 172.21.15.2:0/448906237 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f724c003ce0 con 0x55ceff9a72f0 2024-05-14T09:44:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.309+0000 7f726b085740 10 client.26186 fetch_fsmap finished waiting for FSMap version 39 2024-05-14T09:44:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.309+0000 7f726b085740 10 client.26186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:44:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.309+0000 7f726b085740 20 client.26186 populate_metadata read hostname 'smithi002' 2024-05-14T09:44:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.309+0000 7f726b085740 1 --2- 172.21.15.2:0/448906237 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ceff9641c0 0x55ceffdf3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.309+0000 7f726b085740 4 client.26186 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:44:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.309+0000 7f726b085740 1 -- 172.21.15.2:0/448906237 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ceffcde050 con 0x55ceff9641c0 2024-05-14T09:44:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.309+0000 7f7261771640 1 --2- 172.21.15.2:0/448906237 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ceff9641c0 0x55ceffdf3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.309+0000 7f72417fa640 10 client.26186 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.309+0000 7f7261771640 1 --2- 172.21.15.2:0/448906237 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ceff9641c0 0x55ceffdf3570 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63ff961631b0532d server_cookie=7a6a9fe072a3de35 in_seq=0 out_seq=0 2024-05-14T09:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.413+0000 7f72417fa640 1 -- 172.21.15.2:0/448906237 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ceffcde050 con 0x55ceff9641c0 2024-05-14T09:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.413+0000 7f72417fa640 10 client.26186 handle_command_reply: tid=0 2024-05-14T09:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.414+0000 7f726b085740 1 client.26186 shutdown 2024-05-14T09:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.414+0000 7f726b085740 1 -- 172.21.15.2:0/448906237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9a72f0 msgr2=0x55ceffd82340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.414+0000 7f726b085740 1 --2- 172.21.15.2:0/448906237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9a72f0 0x55ceffd82340 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f724c005b40 tx=0x7f724c009500 comp rx=0 tx=0).stop 2024-05-14T09:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.414+0000 7f726b085740 1 -- 172.21.15.2:0/448906237 shutdown_connections 2024-05-14T09:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.414+0000 7f726b085740 1 --2- 172.21.15.2:0/448906237 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ceff9641c0 0x55ceffdf3570 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.414+0000 7f726b085740 1 --2- 172.21.15.2:0/448906237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ceff9d01b0 0x55ceffdef5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.414+0000 7f726b085740 1 --2- 172.21.15.2:0/448906237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ceff9af4d0 0x55ceffd82880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.414+0000 7f726b085740 1 --2- 172.21.15.2:0/448906237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ceff9a72f0 0x55ceffd82340 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.414+0000 7f72417fa640 0 client.26186 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.414+0000 7f726b085740 1 -- 172.21.15.2:0/448906237 >> 172.21.15.2:0/448906237 conn(0x55ceff9c4300 msgr2=0x55ceffd7a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.414+0000 7f726b085740 1 -- 172.21.15.2:0/448906237 shutdown_connections 2024-05-14T09:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.414+0000 7f726b085740 1 -- 172.21.15.2:0/448906237 wait complete. 2024-05-14T09:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.414+0000 7f726b085740 20 client.26186 trim_cache size 0 max 16384 2024-05-14T09:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.415+0000 7f72641d6640 1 -- 172.21.15.2:0/2836853893 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f72440b2920 msgr2=0x7f72440b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.415+0000 7f72641d6640 1 --2- 172.21.15.2:0/2836853893 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f72440b2920 0x7f72440b4e30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55ceffd72e40 tx=0x7f7254108550 comp rx=0 tx=0).stop 2024-05-14T09:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.415+0000 7f72641d6640 1 -- 172.21.15.2:0/2836853893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c102b50 msgr2=0x7f725c06b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.415+0000 7f72641d6640 1 --2- 172.21.15.2:0/2836853893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c102b50 0x7f725c06b0d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7254006010 tx=0x7f7254004270 comp rx=0 tx=0).stop 2024-05-14T09:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.415+0000 7f72641d6640 1 -- 172.21.15.2:0/2836853893 shutdown_connections 2024-05-14T09:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.415+0000 7f72641d6640 1 --2- 172.21.15.2:0/2836853893 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f72440b2920 0x7f72440b4e30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.415+0000 7f72641d6640 1 --2- 172.21.15.2:0/2836853893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f725c108a90 0x7f725c1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.415+0000 7f72641d6640 1 --2- 172.21.15.2:0/2836853893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f725c103d90 0x7f725c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.415+0000 7f72641d6640 1 --2- 172.21.15.2:0/2836853893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c102b50 0x7f725c06b0d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.415+0000 7f72641d6640 1 -- 172.21.15.2:0/2836853893 >> 172.21.15.2:0/2836853893 conn(0x7f725c0fdd40 msgr2=0x7f725c104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.415+0000 7f72641d6640 1 -- 172.21.15.2:0/2836853893 shutdown_connections 2024-05-14T09:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:18.415+0000 7f72641d6640 1 -- 172.21.15.2:0/2836853893 wait complete. 2024-05-14T09:44:18.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:18 smithi002 ceph-mon[31557]: pgmap v705: 129 pgs: 129 active+clean; 157 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 346 op/s 2024-05-14T09:44:18.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:18 smithi029 ceph-mon[36249]: pgmap v705: 129 pgs: 129 active+clean; 157 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 346 op/s 2024-05-14T09:44:18.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:18 smithi026 ceph-mon[36056]: pgmap v705: 129 pgs: 129 active+clean; 157 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 346 op/s 2024-05-14T09:44:19.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 878619 10.05 MB/sec execute 345 sec latency 91.590 ms 2024-05-14T09:44:20.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 880743 10.06 MB/sec execute 346 sec latency 11.812 ms 2024-05-14T09:44:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:20 smithi002 ceph-mon[31557]: pgmap v706: 129 pgs: 129 active+clean; 157 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 271 op/s 2024-05-14T09:44:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:20 smithi029 ceph-mon[36249]: pgmap v706: 129 pgs: 129 active+clean; 157 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 271 op/s 2024-05-14T09:44:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:20 smithi026 ceph-mon[36056]: pgmap v706: 129 pgs: 129 active+clean; 157 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 271 op/s 2024-05-14T09:44:21.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 882180 10.05 MB/sec execute 347 sec latency 65.600 ms 2024-05-14T09:44:22.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 884234 10.06 MB/sec execute 348 sec latency 86.400 ms 2024-05-14T09:44:22.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:22 smithi002 ceph-mon[31557]: pgmap v707: 129 pgs: 129 active+clean; 158 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 298 op/s 2024-05-14T09:44:22.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:22 smithi029 ceph-mon[36249]: pgmap v707: 129 pgs: 129 active+clean; 158 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 298 op/s 2024-05-14T09:44:22.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:22 smithi026 ceph-mon[36056]: pgmap v707: 129 pgs: 129 active+clean; 158 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 298 op/s 2024-05-14T09:44:23.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 885657 10.05 MB/sec execute 349 sec latency 13.400 ms 2024-05-14T09:44:24.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 887792 10.06 MB/sec execute 350 sec latency 21.804 ms 2024-05-14T09:44:24.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:24 smithi002 ceph-mon[31557]: pgmap v708: 129 pgs: 129 active+clean; 154 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 328 op/s 2024-05-14T09:44:24.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:24 smithi029 ceph-mon[36249]: pgmap v708: 129 pgs: 129 active+clean; 154 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 328 op/s 2024-05-14T09:44:24.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:24 smithi026 ceph-mon[36056]: pgmap v708: 129 pgs: 129 active+clean; 154 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 328 op/s 2024-05-14T09:44:25.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 889258 10.04 MB/sec execute 351 sec latency 13.486 ms 2024-05-14T09:44:26.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 891372 10.05 MB/sec execute 352 sec latency 14.880 ms 2024-05-14T09:44:26.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:26 smithi002 ceph-mon[31557]: pgmap v709: 129 pgs: 129 active+clean; 154 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 2.2 MiB/s wr, 255 op/s 2024-05-14T09:44:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:26 smithi029 ceph-mon[36249]: pgmap v709: 129 pgs: 129 active+clean; 154 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 2.2 MiB/s wr, 255 op/s 2024-05-14T09:44:26.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:26 smithi026 ceph-mon[36056]: pgmap v709: 129 pgs: 129 active+clean; 154 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 2.2 MiB/s wr, 255 op/s 2024-05-14T09:44:27.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 892578 10.04 MB/sec execute 353 sec latency 61.627 ms 2024-05-14T09:44:28.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 894793 10.05 MB/sec execute 354 sec latency 22.061 ms 2024-05-14T09:44:28.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:28 smithi002 ceph-mon[31557]: pgmap v710: 129 pgs: 129 active+clean; 157 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-05-14T09:44:28.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:28 smithi029 ceph-mon[36249]: pgmap v710: 129 pgs: 129 active+clean; 157 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-05-14T09:44:28.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:28 smithi026 ceph-mon[36056]: pgmap v710: 129 pgs: 129 active+clean; 157 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-05-14T09:44:29.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 896145 10.04 MB/sec execute 355 sec latency 16.904 ms 2024-05-14T09:44:30.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 898416 10.05 MB/sec execute 356 sec latency 14.088 ms 2024-05-14T09:44:30.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:30 smithi002 ceph-mon[31557]: pgmap v711: 129 pgs: 129 active+clean; 158 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.2 MiB/s wr, 271 op/s 2024-05-14T09:44:30.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:30 smithi029 ceph-mon[36249]: pgmap v711: 129 pgs: 129 active+clean; 158 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.2 MiB/s wr, 271 op/s 2024-05-14T09:44:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:30 smithi026 ceph-mon[36056]: pgmap v711: 129 pgs: 129 active+clean; 158 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.2 MiB/s wr, 271 op/s 2024-05-14T09:44:31.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 899963 10.03 MB/sec execute 357 sec latency 13.637 ms 2024-05-14T09:44:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.550+0000 7ff06a334640 1 -- 172.21.15.2:0/1813623708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff064108a90 msgr2=0x7ff06410aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.550+0000 7ff06a334640 1 --2- 172.21.15.2:0/1813623708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff064108a90 0x7ff06410aeb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009ca0 tx=0x7ff05c02a8c0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.550+0000 7ff06a334640 1 -- 172.21.15.2:0/1813623708 shutdown_connections 2024-05-14T09:44:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.550+0000 7ff06a334640 1 --2- 172.21.15.2:0/1813623708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff064108a90 0x7ff06410aeb0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.550+0000 7ff06a334640 1 --2- 172.21.15.2:0/1813623708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff064103d90 0x7ff0641041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.550+0000 7ff06a334640 1 --2- 172.21.15.2:0/1813623708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff064102b50 0x7ff064103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.550+0000 7ff06a334640 1 -- 172.21.15.2:0/1813623708 >> 172.21.15.2:0/1813623708 conn(0x7ff0640fdd40 msgr2=0x7ff0641001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.550+0000 7ff06a334640 1 -- 172.21.15.2:0/1813623708 shutdown_connections 2024-05-14T09:44:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.550+0000 7ff06a334640 1 -- 172.21.15.2:0/1813623708 wait complete. 2024-05-14T09:44:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.551+0000 7ff06a334640 1 Processor -- start 2024-05-14T09:44:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.551+0000 7ff06a334640 1 -- start start 2024-05-14T09:44:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff06a334640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff064102b50 0x7ff064199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff06a334640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff064103d90 0x7ff064199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff06a334640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff064108a90 0x7ff0641a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff06a334640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff06410e680 con 0x7ff064102b50 2024-05-14T09:44:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff06a334640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff06410e500 con 0x7ff064103d90 2024-05-14T09:44:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff06a334640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff06410e800 con 0x7ff064108a90 2024-05-14T09:44:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff063fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff064102b50 0x7ff064199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff063fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff064102b50 0x7ff064199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56818/0 (socket says 172.21.15.2:56818) 2024-05-14T09:44:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff063fff640 1 -- 172.21.15.2:0/2007153340 learned_addr learned my addr 172.21.15.2:0/2007153340 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff0637fe640 1 --2- 172.21.15.2:0/2007153340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff064103d90 0x7ff064199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff0688d1640 1 --2- 172.21.15.2:0/2007153340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff064108a90 0x7ff0641a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff063fff640 1 -- 172.21.15.2:0/2007153340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff064108a90 msgr2=0x7ff0641a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff063fff640 1 --2- 172.21.15.2:0/2007153340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff064108a90 0x7ff0641a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff063fff640 1 -- 172.21.15.2:0/2007153340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff064103d90 msgr2=0x7ff064199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff063fff640 1 --2- 172.21.15.2:0/2007153340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff064103d90 0x7ff064199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff063fff640 1 -- 172.21.15.2:0/2007153340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0641a0c70 con 0x7ff064102b50 2024-05-14T09:44:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.552+0000 7ff0637fe640 1 --2- 172.21.15.2:0/2007153340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff064103d90 0x7ff064199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:44:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.553+0000 7ff063fff640 1 --2- 172.21.15.2:0/2007153340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff064102b50 0x7ff064199130 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff05400e9f0 tx=0x7ff05400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.553+0000 7ff0617fa640 1 -- 172.21.15.2:0/2007153340 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05400ccc0 con 0x7ff064102b50 2024-05-14T09:44:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.553+0000 7ff0617fa640 1 -- 172.21.15.2:0/2007153340 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0540035f0 con 0x7ff064102b50 2024-05-14T09:44:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.553+0000 7ff06a334640 1 -- 172.21.15.2:0/2007153340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0641a0f60 con 0x7ff064102b50 2024-05-14T09:44:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.553+0000 7ff0617fa640 1 -- 172.21.15.2:0/2007153340 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05400ce60 con 0x7ff064102b50 2024-05-14T09:44:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.553+0000 7ff06a334640 1 -- 172.21.15.2:0/2007153340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0641a13c0 con 0x7ff064102b50 2024-05-14T09:44:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff0711e6740 1 Processor -- start 2024-05-14T09:44:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff0711e6740 1 -- start start 2024-05-14T09:44:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff0617fa640 1 -- 172.21.15.2:0/2007153340 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff054003aa0 con 0x7ff064102b50 2024-05-14T09:44:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff0711e6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f629c1b0 0x5609f663fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff0711e6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f62732f0 0x5609f66401f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff0711e6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f627b4d0 0x5609f66470f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff0711e6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5609f65a25d0 con 0x5609f629c1b0 2024-05-14T09:44:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff0711e6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5609f655b890 con 0x5609f627b4d0 2024-05-14T09:44:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff0711e6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5609f659e590 con 0x5609f62732f0 2024-05-14T09:44:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff0688d1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f629c1b0 0x5609f663fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff063fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f62732f0 0x5609f66401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff0688d1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f629c1b0 0x5609f663fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56830/0 (socket says 172.21.15.2:56830) 2024-05-14T09:44:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff0688d1640 1 -- 172.21.15.2:0/2060557352 learned_addr learned my addr 172.21.15.2:0/2060557352 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff0637fe640 1 --2- 172.21.15.2:0/2060557352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f627b4d0 0x5609f66470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff0688d1640 1 -- 172.21.15.2:0/2060557352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f62732f0 msgr2=0x5609f66401f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.556+0000 7ff0688d1640 1 --2- 172.21.15.2:0/2060557352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f62732f0 0x5609f66401f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.557+0000 7ff0688d1640 1 -- 172.21.15.2:0/2060557352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f627b4d0 msgr2=0x5609f66470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.557+0000 7ff0688d1640 1 --2- 172.21.15.2:0/2060557352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f627b4d0 0x5609f66470f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.557+0000 7ff0688d1640 1 -- 172.21.15.2:0/2060557352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609f62b4210 con 0x5609f629c1b0 2024-05-14T09:44:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.557+0000 7ff0617fa640 1 --2- 172.21.15.2:0/2007153340 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff04c0b28a0 0x7ff04c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.557+0000 7ff0688d1640 1 --2- 172.21.15.2:0/2060557352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f629c1b0 0x5609f663fc70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff05c035000 tx=0x7ff05c02f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6573a90782ef62fa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.557+0000 7ff063fff640 1 --2- 172.21.15.2:0/2007153340 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff04c0b28a0 0x7ff04c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.557+0000 7ff04a7fc640 1 -- 172.21.15.2:0/2060557352 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c031650 con 0x5609f629c1b0 2024-05-14T09:44:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.557+0000 7ff0617fa640 1 -- 172.21.15.2:0/2007153340 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7ff0540e2910 con 0x7ff064102b50 2024-05-14T09:44:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.557+0000 7ff04a7fc640 1 -- 172.21.15.2:0/2060557352 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff05c036030 con 0x5609f629c1b0 2024-05-14T09:44:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.557+0000 7ff04a7fc640 1 -- 172.21.15.2:0/2060557352 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c004740 con 0x5609f629c1b0 2024-05-14T09:44:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.558+0000 7ff063fff640 1 --2- 172.21.15.2:0/2007153340 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff04c0b28a0 0x7ff04c0b4db0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5609f6641050 tx=0x7ff0540e8550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.558+0000 7ff0711e6740 1 -- 172.21.15.2:0/2060557352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f629c1b0 msgr2=0x5609f663fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.558+0000 7ff0711e6740 1 --2- 172.21.15.2:0/2060557352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f629c1b0 0x5609f663fc70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff05c035000 tx=0x7ff05c02f410 comp rx=0 tx=0).stop 2024-05-14T09:44:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.558+0000 7ff0711e6740 1 -- 172.21.15.2:0/2060557352 shutdown_connections 2024-05-14T09:44:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.558+0000 7ff0711e6740 1 --2- 172.21.15.2:0/2060557352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f627b4d0 0x5609f66470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.558+0000 7ff0711e6740 1 --2- 172.21.15.2:0/2060557352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f62732f0 0x5609f66401f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.558+0000 7ff0711e6740 1 --2- 172.21.15.2:0/2060557352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f629c1b0 0x5609f663fc70 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.558+0000 7ff0711e6740 1 -- 172.21.15.2:0/2060557352 >> 172.21.15.2:0/2060557352 conn(0x5609f6290300 msgr2=0x5609f66393a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.558+0000 7ff0711e6740 1 -- 172.21.15.2:0/2060557352 shutdown_connections 2024-05-14T09:44:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.558+0000 7ff0711e6740 1 -- 172.21.15.2:0/2060557352 wait complete. 2024-05-14T09:44:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.559+0000 7ff0711e6740 1 -- start start 2024-05-14T09:44:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.559+0000 7ff0711e6740 1 Processor -- start 2024-05-14T09:44:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.559+0000 7ff0711e6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f62732f0 0x5609f66496a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.559+0000 7ff0711e6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f627b4d0 0x5609f6649be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.559+0000 7ff0711e6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f629c1b0 0x5609f6650b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.559+0000 7ff0711e6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5609f6647960 con 0x5609f629c1b0 2024-05-14T09:44:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.559+0000 7ff0711e6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5609f655f370 con 0x5609f62732f0 2024-05-14T09:44:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.559+0000 7ff0711e6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5609f6647ae0 con 0x5609f627b4d0 2024-05-14T09:44:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.559+0000 7ff0711e6740 10 client.0 _ll_register_callbacks cb 0x5609f62b3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:44:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.559+0000 7ff0637fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f627b4d0 0x5609f6649be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.559+0000 7ff0688d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f62732f0 0x5609f66496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.559+0000 7ff0637fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f627b4d0 0x5609f6649be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:34210/0 (socket says 172.21.15.2:34210) 2024-05-14T09:44:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.559+0000 7ff0688d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f62732f0 0x5609f66496a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:55446/0 (socket says 172.21.15.2:55446) 2024-05-14T09:44:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.559+0000 7ff0637fe640 1 -- 172.21.15.2:0/3275395034 learned_addr learned my addr 172.21.15.2:0/3275395034 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.559+0000 7ff0688d1640 1 --2- 172.21.15.2:0/3275395034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f629c1b0 0x5609f6650b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.560+0000 7ff0637fe640 1 -- 172.21.15.2:0/3275395034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f62732f0 msgr2=0x5609f66496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.560+0000 7ff0637fe640 1 --2- 172.21.15.2:0/3275395034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f62732f0 0x5609f66496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.560+0000 7ff0637fe640 1 -- 172.21.15.2:0/3275395034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f629c1b0 msgr2=0x5609f6650b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.560+0000 7ff0637fe640 1 --2- 172.21.15.2:0/3275395034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f629c1b0 0x5609f6650b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.560+0000 7ff0637fe640 1 -- 172.21.15.2:0/3275395034 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609f6380620 con 0x5609f627b4d0 2024-05-14T09:44:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.560+0000 7ff0637fe640 1 --2- 172.21.15.2:0/3275395034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f627b4d0 0x5609f6649be0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff058012b10 tx=0x7ff05800f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bb9d0a1397a6848 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.560+0000 7ff0437fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:44:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.560+0000 7ff0437fe640 1 -- 172.21.15.2:0/3275395034 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05800fb40 con 0x5609f627b4d0 2024-05-14T09:44:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.560+0000 7ff0437fe640 1 -- 172.21.15.2:0/3275395034 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7ff05800fce0 con 0x5609f627b4d0 2024-05-14T09:44:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.560+0000 7ff0711e6740 1 -- 172.21.15.2:0/3275395034 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff05c036030 con 0x5609f627b4d0 2024-05-14T09:44:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.560+0000 7ff0437fe640 1 -- 172.21.15.2:0/3275395034 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff058025050 con 0x5609f627b4d0 2024-05-14T09:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.560+0000 7ff0711e6740 10 client.26182 fetch_fsmap learned FSMap version 39 2024-05-14T09:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.560+0000 7ff0711e6740 1 -- 172.21.15.2:0/3275395034 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5609f663ef50 con 0x5609f627b4d0 2024-05-14T09:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.561+0000 7ff0437fe640 1 -- 172.21.15.2:0/3275395034 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff05c036030 con 0x5609f627b4d0 2024-05-14T09:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.561+0000 7ff0437fe640 1 -- 172.21.15.2:0/3275395034 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7ff058003bb0 con 0x5609f627b4d0 2024-05-14T09:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.561+0000 7ff0711e6740 10 client.26182 fetch_fsmap finished waiting for FSMap version 39 2024-05-14T09:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.561+0000 7ff0711e6740 10 client.26182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.561+0000 7ff0711e6740 20 client.26182 populate_metadata read hostname 'smithi002' 2024-05-14T09:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.561+0000 7ff0711e6740 1 --2- 172.21.15.2:0/3275395034 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5609f625ad80 0x5609f6654210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.561+0000 7ff0711e6740 4 client.26182 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.561+0000 7ff0711e6740 1 -- 172.21.15.2:0/3275395034 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609f65b01e0 con 0x5609f625ad80 2024-05-14T09:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.561+0000 7ff0688d1640 1 --2- 172.21.15.2:0/3275395034 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5609f625ad80 0x5609f6654210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.561+0000 7ff0437fe640 10 client.26182 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.561+0000 7ff0688d1640 1 --2- 172.21.15.2:0/3275395034 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5609f625ad80 0x5609f6654210 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eb68836c878eca6 server_cookie=6f92ad11cef7f7e5 in_seq=0 out_seq=0 2024-05-14T09:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.562+0000 7ff0437fe640 1 -- 172.21.15.2:0/3275395034 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5609f65b01e0 con 0x5609f625ad80 2024-05-14T09:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.562+0000 7ff0437fe640 10 client.26182 handle_command_reply: tid=0 2024-05-14T09:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.562+0000 7ff0711e6740 1 client.26182 shutdown 2024-05-14T09:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.562+0000 7ff0711e6740 1 -- 172.21.15.2:0/3275395034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f627b4d0 msgr2=0x5609f6649be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.562+0000 7ff0711e6740 1 --2- 172.21.15.2:0/3275395034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f627b4d0 0x5609f6649be0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff058012b10 tx=0x7ff05800f3c0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.562+0000 7ff0711e6740 1 -- 172.21.15.2:0/3275395034 shutdown_connections 2024-05-14T09:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.562+0000 7ff0711e6740 1 --2- 172.21.15.2:0/3275395034 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5609f625ad80 0x5609f6654210 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.562+0000 7ff0711e6740 1 --2- 172.21.15.2:0/3275395034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f629c1b0 0x5609f6650b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.562+0000 7ff0437fe640 0 client.26182 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.562+0000 7ff0711e6740 1 --2- 172.21.15.2:0/3275395034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f627b4d0 0x5609f6649be0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.562+0000 7ff0711e6740 1 --2- 172.21.15.2:0/3275395034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f62732f0 0x5609f66496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.562+0000 7ff0711e6740 1 -- 172.21.15.2:0/3275395034 >> 172.21.15.2:0/3275395034 conn(0x5609f6290300 msgr2=0x5609f663c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.563+0000 7ff0711e6740 1 -- 172.21.15.2:0/3275395034 shutdown_connections 2024-05-14T09:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.563+0000 7ff0711e6740 1 -- 172.21.15.2:0/3275395034 wait complete. 2024-05-14T09:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.563+0000 7ff0711e6740 20 client.26182 trim_cache size 0 max 16384 2024-05-14T09:44:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.578+0000 7ff0711e6740 1 Processor -- start 2024-05-14T09:44:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.578+0000 7ff0711e6740 1 -- start start 2024-05-14T09:44:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.578+0000 7ff0711e6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f625ad80 0x5609f6643c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.578+0000 7ff0711e6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f62732f0 0x5609f66441b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.578+0000 7ff0711e6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f627b4d0 0x5609f663c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.578+0000 7ff0711e6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff0641ad2c0 con 0x5609f627b4d0 2024-05-14T09:44:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.579+0000 7ff0711e6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff0641aab90 con 0x5609f625ad80 2024-05-14T09:44:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.579+0000 7ff0711e6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff0641a89e0 con 0x5609f62732f0 2024-05-14T09:44:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.579+0000 7ff0688d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f625ad80 0x5609f6643c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.579+0000 7ff0637fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f62732f0 0x5609f66441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.579+0000 7ff0688d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f625ad80 0x5609f6643c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:55456/0 (socket says 172.21.15.2:55456) 2024-05-14T09:44:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.579+0000 7ff0688d1640 1 -- 172.21.15.2:0/4062817144 learned_addr learned my addr 172.21.15.2:0/4062817144 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.579+0000 7ff0688d1640 1 --2- 172.21.15.2:0/4062817144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f627b4d0 0x5609f663c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.579+0000 7ff0688d1640 1 -- 172.21.15.2:0/4062817144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f62732f0 msgr2=0x5609f66441b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.579+0000 7ff0688d1640 1 --2- 172.21.15.2:0/4062817144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f62732f0 0x5609f66441b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.579+0000 7ff0688d1640 1 -- 172.21.15.2:0/4062817144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f627b4d0 msgr2=0x5609f663c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.579+0000 7ff0688d1640 1 --2- 172.21.15.2:0/4062817144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f627b4d0 0x5609f663c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.579+0000 7ff0688d1640 1 -- 172.21.15.2:0/4062817144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609f6639d00 con 0x5609f625ad80 2024-05-14T09:44:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.579+0000 7ff0688d1640 1 --2- 172.21.15.2:0/4062817144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f625ad80 0x5609f6643c70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff05c031b50 tx=0x7ff05c02f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df7b523fb4bffc3a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.579+0000 7ff049ffb640 1 -- 172.21.15.2:0/4062817144 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c004040 con 0x5609f625ad80 2024-05-14T09:44:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.580+0000 7ff049ffb640 1 -- 172.21.15.2:0/4062817144 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff05c036040 con 0x5609f625ad80 2024-05-14T09:44:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.580+0000 7ff049ffb640 1 -- 172.21.15.2:0/4062817144 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c003c90 con 0x5609f625ad80 2024-05-14T09:44:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.580+0000 7ff0711e6740 1 -- 172.21.15.2:0/4062817144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f625ad80 msgr2=0x5609f6643c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.580+0000 7ff0711e6740 1 --2- 172.21.15.2:0/4062817144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f625ad80 0x5609f6643c70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff05c031b50 tx=0x7ff05c02f3a0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.580+0000 7ff0711e6740 1 -- 172.21.15.2:0/4062817144 shutdown_connections 2024-05-14T09:44:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.580+0000 7ff0711e6740 1 --2- 172.21.15.2:0/4062817144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f627b4d0 0x5609f663c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.580+0000 7ff0711e6740 1 --2- 172.21.15.2:0/4062817144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f62732f0 0x5609f66441b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.580+0000 7ff0711e6740 1 --2- 172.21.15.2:0/4062817144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f625ad80 0x5609f6643c70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.580+0000 7ff0711e6740 1 -- 172.21.15.2:0/4062817144 >> 172.21.15.2:0/4062817144 conn(0x5609f6290300 msgr2=0x5609f663fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.580+0000 7ff0711e6740 1 -- 172.21.15.2:0/4062817144 shutdown_connections 2024-05-14T09:44:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.580+0000 7ff0711e6740 1 -- 172.21.15.2:0/4062817144 wait complete. 2024-05-14T09:44:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.581+0000 7ff0711e6740 1 -- start start 2024-05-14T09:44:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.581+0000 7ff0711e6740 1 Processor -- start 2024-05-14T09:44:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.581+0000 7ff0711e6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f625ad80 0x5609f664fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.581+0000 7ff0711e6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f62732f0 0x5609f664ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.581+0000 7ff0711e6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f627b4d0 0x5609f664c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.581+0000 7ff0711e6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff0641a9050 con 0x5609f627b4d0 2024-05-14T09:44:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.581+0000 7ff0711e6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff0641aa4c0 con 0x5609f625ad80 2024-05-14T09:44:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.581+0000 7ff0711e6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff0641a9720 con 0x5609f62732f0 2024-05-14T09:44:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.581+0000 7ff0711e6740 10 client.0 _ll_register_callbacks cb 0x7ff058009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:44:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.581+0000 7ff0688d1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f627b4d0 0x5609f664c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.581+0000 7ff0688d1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f627b4d0 0x5609f664c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56858/0 (socket says 172.21.15.2:56858) 2024-05-14T09:44:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.581+0000 7ff0688d1640 1 -- 172.21.15.2:0/95264181 learned_addr learned my addr 172.21.15.2:0/95264181 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.582+0000 7ff0688d1640 1 --2- 172.21.15.2:0/95264181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f625ad80 0x5609f664fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.582+0000 7ff0637fe640 1 --2- 172.21.15.2:0/95264181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f62732f0 0x5609f664ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.582+0000 7ff0688d1640 1 -- 172.21.15.2:0/95264181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f62732f0 msgr2=0x5609f664ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.582+0000 7ff0688d1640 1 --2- 172.21.15.2:0/95264181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f62732f0 0x5609f664ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.582+0000 7ff0688d1640 1 -- 172.21.15.2:0/95264181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f625ad80 msgr2=0x5609f664fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.582+0000 7ff0688d1640 1 --2- 172.21.15.2:0/95264181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f625ad80 0x5609f664fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.582+0000 7ff0688d1640 1 -- 172.21.15.2:0/95264181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609f664cd50 con 0x5609f627b4d0 2024-05-14T09:44:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.582+0000 7ff0688d1640 1 --2- 172.21.15.2:0/95264181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f627b4d0 0x5609f664c660 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff05c02f480 tx=0x7ff05c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feafa4f1803119b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.582+0000 7ff0437fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:44:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.582+0000 7ff0437fe640 1 -- 172.21.15.2:0/95264181 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c004100 con 0x5609f627b4d0 2024-05-14T09:44:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.582+0000 7ff0437fe640 1 -- 172.21.15.2:0/95264181 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7ff05c04a040 con 0x5609f627b4d0 2024-05-14T09:44:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.582+0000 7ff0711e6740 1 -- 172.21.15.2:0/95264181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff05c036040 con 0x5609f627b4d0 2024-05-14T09:44:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.582+0000 7ff0437fe640 1 -- 172.21.15.2:0/95264181 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff05c031450 con 0x5609f627b4d0 2024-05-14T09:44:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.582+0000 7ff0711e6740 10 client.16524 fetch_fsmap learned FSMap version 39 2024-05-14T09:44:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.582+0000 7ff0711e6740 1 -- 172.21.15.2:0/95264181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5609f66c55c0 con 0x5609f627b4d0 2024-05-14T09:44:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.583+0000 7ff0437fe640 1 -- 172.21.15.2:0/95264181 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff05c036040 con 0x5609f627b4d0 2024-05-14T09:44:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.583+0000 7ff0437fe640 1 -- 172.21.15.2:0/95264181 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7ff05c062070 con 0x5609f627b4d0 2024-05-14T09:44:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.583+0000 7ff0711e6740 10 client.16524 fetch_fsmap finished waiting for FSMap version 39 2024-05-14T09:44:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.583+0000 7ff0711e6740 10 client.16524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:44:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.583+0000 7ff0711e6740 20 client.16524 populate_metadata read hostname 'smithi002' 2024-05-14T09:44:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.583+0000 7ff0711e6740 1 --2- 172.21.15.2:0/95264181 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5609f629c1b0 0x5609f66c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.583+0000 7ff0711e6740 4 client.16524 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:44:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.583+0000 7ff0711e6740 1 -- 172.21.15.2:0/95264181 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609f65aa050 con 0x5609f629c1b0 2024-05-14T09:44:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.583+0000 7ff0637fe640 1 --2- 172.21.15.2:0/95264181 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5609f629c1b0 0x5609f66c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.583+0000 7ff0437fe640 10 client.16524 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.584+0000 7ff0637fe640 1 --2- 172.21.15.2:0/95264181 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5609f629c1b0 0x5609f66c7c20 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=104fa200f6eba9ac server_cookie=1d94188d555fe165 in_seq=0 out_seq=0 2024-05-14T09:44:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.689+0000 7ff0437fe640 1 -- 172.21.15.2:0/95264181 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5609f65aa050 con 0x5609f629c1b0 2024-05-14T09:44:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.689+0000 7ff0437fe640 10 client.16524 handle_command_reply: tid=0 2024-05-14T09:44:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.690+0000 7ff0711e6740 1 client.16524 shutdown 2024-05-14T09:44:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.690+0000 7ff0711e6740 1 -- 172.21.15.2:0/95264181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f627b4d0 msgr2=0x5609f664c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.690+0000 7ff0711e6740 1 --2- 172.21.15.2:0/95264181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f627b4d0 0x5609f664c660 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff05c02f480 tx=0x7ff05c009650 comp rx=0 tx=0).stop 2024-05-14T09:44:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.690+0000 7ff0711e6740 1 -- 172.21.15.2:0/95264181 shutdown_connections 2024-05-14T09:44:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.690+0000 7ff0711e6740 1 --2- 172.21.15.2:0/95264181 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5609f629c1b0 0x5609f66c7c20 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.690+0000 7ff0711e6740 1 --2- 172.21.15.2:0/95264181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609f627b4d0 0x5609f664c660 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.690+0000 7ff0711e6740 1 --2- 172.21.15.2:0/95264181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5609f62732f0 0x5609f664ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.690+0000 7ff0437fe640 0 client.16524 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.690+0000 7ff0711e6740 1 --2- 172.21.15.2:0/95264181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609f625ad80 0x5609f664fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.690+0000 7ff0711e6740 1 -- 172.21.15.2:0/95264181 >> 172.21.15.2:0/95264181 conn(0x5609f6290300 msgr2=0x5609f663fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.690+0000 7ff0711e6740 1 -- 172.21.15.2:0/95264181 shutdown_connections 2024-05-14T09:44:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.690+0000 7ff0711e6740 1 -- 172.21.15.2:0/95264181 wait complete. 2024-05-14T09:44:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.690+0000 7ff0711e6740 20 client.16524 trim_cache size 0 max 16384 2024-05-14T09:44:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:44:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:44:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:44:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:44:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.691+0000 7ff06a334640 1 -- 172.21.15.2:0/2007153340 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff04c0b28a0 msgr2=0x7ff04c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.691+0000 7ff06a334640 1 --2- 172.21.15.2:0/2007153340 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff04c0b28a0 0x7ff04c0b4db0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5609f6641050 tx=0x7ff0540e8550 comp rx=0 tx=0).stop 2024-05-14T09:44:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.691+0000 7ff06a334640 1 -- 172.21.15.2:0/2007153340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff064102b50 msgr2=0x7ff064199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.691+0000 7ff06a334640 1 --2- 172.21.15.2:0/2007153340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff064102b50 0x7ff064199130 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff05400e9f0 tx=0x7ff05400eeb0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.691+0000 7ff06a334640 1 -- 172.21.15.2:0/2007153340 shutdown_connections 2024-05-14T09:44:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.691+0000 7ff06a334640 1 --2- 172.21.15.2:0/2007153340 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff04c0b28a0 0x7ff04c0b4db0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.691+0000 7ff06a334640 1 --2- 172.21.15.2:0/2007153340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff064108a90 0x7ff0641a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.691+0000 7ff06a334640 1 --2- 172.21.15.2:0/2007153340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff064103d90 0x7ff064199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.691+0000 7ff06a334640 1 --2- 172.21.15.2:0/2007153340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff064102b50 0x7ff064199130 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.691+0000 7ff06a334640 1 -- 172.21.15.2:0/2007153340 >> 172.21.15.2:0/2007153340 conn(0x7ff0640fdd40 msgr2=0x7ff064104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.691+0000 7ff06a334640 1 -- 172.21.15.2:0/2007153340 shutdown_connections 2024-05-14T09:44:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:31.691+0000 7ff06a334640 1 -- 172.21.15.2:0/2007153340 wait complete. 2024-05-14T09:44:32.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 901709 10.04 MB/sec execute 358 sec latency 92.087 ms 2024-05-14T09:44:32.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:32 smithi002 ceph-mon[31557]: pgmap v712: 129 pgs: 129 active+clean; 159 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 2.3 MiB/s wr, 293 op/s 2024-05-14T09:44:32.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:32 smithi029 ceph-mon[36249]: pgmap v712: 129 pgs: 129 active+clean; 159 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 2.3 MiB/s wr, 293 op/s 2024-05-14T09:44:32.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:32 smithi026 ceph-mon[36056]: pgmap v712: 129 pgs: 129 active+clean; 159 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 2.3 MiB/s wr, 293 op/s 2024-05-14T09:44:33.275 INFO:tasks.workunit.client.0.smithi002.stdout: 1 903183 10.03 MB/sec execute 359 sec latency 13.229 ms 2024-05-14T09:44:34.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 905122 10.03 MB/sec execute 360 sec latency 109.032 ms 2024-05-14T09:44:34.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:34 smithi002 ceph-mon[31557]: pgmap v713: 129 pgs: 129 active+clean; 161 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-05-14T09:44:34.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:34 smithi029 ceph-mon[36249]: pgmap v713: 129 pgs: 129 active+clean; 161 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-05-14T09:44:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:34 smithi026 ceph-mon[36056]: pgmap v713: 129 pgs: 129 active+clean; 161 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-05-14T09:44:35.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 906269 10.02 MB/sec execute 361 sec latency 63.689 ms 2024-05-14T09:44:36.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 908660 10.03 MB/sec execute 362 sec latency 19.493 ms 2024-05-14T09:44:36.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:36 smithi002 ceph-mon[31557]: pgmap v714: 129 pgs: 129 active+clean; 161 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 2.1 MiB/s wr, 255 op/s 2024-05-14T09:44:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:36 smithi029 ceph-mon[36249]: pgmap v714: 129 pgs: 129 active+clean; 161 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 2.1 MiB/s wr, 255 op/s 2024-05-14T09:44:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:36 smithi026 ceph-mon[36056]: pgmap v714: 129 pgs: 129 active+clean; 161 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 2.1 MiB/s wr, 255 op/s 2024-05-14T09:44:37.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 909571 10.01 MB/sec execute 363 sec latency 48.402 ms 2024-05-14T09:44:38.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 911825 10.02 MB/sec execute 364 sec latency 15.164 ms 2024-05-14T09:44:38.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:38 smithi002 ceph-mon[31557]: pgmap v715: 129 pgs: 129 active+clean; 160 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 357 op/s 2024-05-14T09:44:38.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:38 smithi029 ceph-mon[36249]: pgmap v715: 129 pgs: 129 active+clean; 160 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 357 op/s 2024-05-14T09:44:38.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:38 smithi026 ceph-mon[36056]: pgmap v715: 129 pgs: 129 active+clean; 160 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 357 op/s 2024-05-14T09:44:39.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 912807 10.01 MB/sec execute 365 sec latency 258.760 ms 2024-05-14T09:44:40.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 914791 10.00 MB/sec execute 366 sec latency 15.787 ms 2024-05-14T09:44:40.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:40 smithi002 ceph-mon[31557]: pgmap v716: 129 pgs: 129 active+clean; 160 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 2.5 MiB/s wr, 283 op/s 2024-05-14T09:44:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:40 smithi029 ceph-mon[36249]: pgmap v716: 129 pgs: 129 active+clean; 160 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 2.5 MiB/s wr, 283 op/s 2024-05-14T09:44:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:40 smithi026 ceph-mon[36056]: pgmap v716: 129 pgs: 129 active+clean; 160 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 2.5 MiB/s wr, 283 op/s 2024-05-14T09:44:41.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 916384 10.01 MB/sec execute 367 sec latency 12.909 ms 2024-05-14T09:44:42.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 917173 10.03 MB/sec execute 368 sec latency 169.410 ms 2024-05-14T09:44:42.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:42 smithi002 ceph-mon[31557]: pgmap v717: 129 pgs: 129 active+clean; 160 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.3 MiB/s wr, 273 op/s 2024-05-14T09:44:42.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:42 smithi029 ceph-mon[36249]: pgmap v717: 129 pgs: 129 active+clean; 160 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.3 MiB/s wr, 273 op/s 2024-05-14T09:44:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:42 smithi026 ceph-mon[36056]: pgmap v717: 129 pgs: 129 active+clean; 160 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.3 MiB/s wr, 273 op/s 2024-05-14T09:44:43.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 919284 10.03 MB/sec execute 369 sec latency 32.987 ms 2024-05-14T09:44:44.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 920811 10.03 MB/sec execute 370 sec latency 13.962 ms 2024-05-14T09:44:44.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:44 smithi002 ceph-mon[31557]: pgmap v718: 129 pgs: 129 active+clean; 159 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 324 op/s 2024-05-14T09:44:44.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:44 smithi029 ceph-mon[36249]: pgmap v718: 129 pgs: 129 active+clean; 159 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 324 op/s 2024-05-14T09:44:44.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:44 smithi026 ceph-mon[36056]: pgmap v718: 129 pgs: 129 active+clean; 159 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 324 op/s 2024-05-14T09:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.800+0000 7fc47b469640 1 -- 172.21.15.2:0/2978954730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc474108a90 msgr2=0x7fc47410aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.800+0000 7fc47b469640 1 --2- 172.21.15.2:0/2978954730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc474108a90 0x7fc47410aeb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009be0 tx=0x7fc46c02a6c0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.800+0000 7fc47b469640 1 -- 172.21.15.2:0/2978954730 shutdown_connections 2024-05-14T09:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.800+0000 7fc47b469640 1 --2- 172.21.15.2:0/2978954730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc474108a90 0x7fc47410aeb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.800+0000 7fc47b469640 1 --2- 172.21.15.2:0/2978954730 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc474103d90 0x7fc4741041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.800+0000 7fc47b469640 1 --2- 172.21.15.2:0/2978954730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc474102b50 0x7fc474103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.800+0000 7fc47b469640 1 -- 172.21.15.2:0/2978954730 >> 172.21.15.2:0/2978954730 conn(0x7fc4740fdd40 msgr2=0x7fc4741001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.800+0000 7fc47b469640 1 -- 172.21.15.2:0/2978954730 shutdown_connections 2024-05-14T09:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.800+0000 7fc47b469640 1 -- 172.21.15.2:0/2978954730 wait complete. 2024-05-14T09:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.801+0000 7fc47b469640 1 Processor -- start 2024-05-14T09:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.801+0000 7fc47b469640 1 -- start start 2024-05-14T09:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.801+0000 7fc47b469640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc474102b50 0x7fc47419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.801+0000 7fc47b469640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc474103d90 0x7fc47419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.801+0000 7fc47b469640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc474108a90 0x7fc4741a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.801+0000 7fc47b469640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc47410f780 con 0x7fc474108a90 2024-05-14T09:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.801+0000 7fc47b469640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc47410f600 con 0x7fc474102b50 2024-05-14T09:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.801+0000 7fc47b469640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc47410f900 con 0x7fc474103d90 2024-05-14T09:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.801+0000 7fc479a06640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc474108a90 0x7fc4741a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.801+0000 7fc479a06640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc474108a90 0x7fc4741a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49520/0 (socket says 172.21.15.2:49520) 2024-05-14T09:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.801+0000 7fc479a06640 1 -- 172.21.15.2:0/2466693782 learned_addr learned my addr 172.21.15.2:0/2466693782 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.802+0000 7fc479205640 1 --2- 172.21.15.2:0/2466693782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc474102b50 0x7fc47419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.802+0000 7fc478a04640 1 --2- 172.21.15.2:0/2466693782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc474103d90 0x7fc47419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.802+0000 7fc479a06640 1 -- 172.21.15.2:0/2466693782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc474103d90 msgr2=0x7fc47419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.802+0000 7fc479a06640 1 --2- 172.21.15.2:0/2466693782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc474103d90 0x7fc47419a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.802+0000 7fc479a06640 1 -- 172.21.15.2:0/2466693782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc474102b50 msgr2=0x7fc47419a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.802+0000 7fc479a06640 1 --2- 172.21.15.2:0/2466693782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc474102b50 0x7fc47419a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.802+0000 7fc479a06640 1 -- 172.21.15.2:0/2466693782 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4741a1e20 con 0x7fc474108a90 2024-05-14T09:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.802+0000 7fc479a06640 1 --2- 172.21.15.2:0/2466693782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc474108a90 0x7fc4741a17b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc46c034040 tx=0x7fc46c032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.802+0000 7fc46a7fc640 1 -- 172.21.15.2:0/2466693782 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c041020 con 0x7fc474108a90 2024-05-14T09:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.802+0000 7fc47b469640 1 -- 172.21.15.2:0/2466693782 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4741a20b0 con 0x7fc474108a90 2024-05-14T09:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.802+0000 7fc47b469640 1 -- 172.21.15.2:0/2466693782 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4741a25f0 con 0x7fc474108a90 2024-05-14T09:44:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.802+0000 7fc46a7fc640 1 -- 172.21.15.2:0/2466693782 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc46c003b70 con 0x7fc474108a90 2024-05-14T09:44:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.802+0000 7fc46a7fc640 1 -- 172.21.15.2:0/2466693782 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c036040 con 0x7fc474108a90 2024-05-14T09:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.805+0000 7fc4823f5740 1 Processor -- start 2024-05-14T09:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.805+0000 7fc46a7fc640 1 -- 172.21.15.2:0/2466693782 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc46c044020 con 0x7fc474108a90 2024-05-14T09:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc4823f5740 1 -- start start 2024-05-14T09:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc4823f5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f84b1b0 0x563e3fbeca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc4823f5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f8222f0 0x563e3fbecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc4823f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f82a4d0 0x563e3fbf3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc4823f5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563e3fb515d0 con 0x563e3f84b1b0 2024-05-14T09:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc4823f5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563e3fb0a890 con 0x563e3f8222f0 2024-05-14T09:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc4823f5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563e3fb4d590 con 0x563e3f82a4d0 2024-05-14T09:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc478a04640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f84b1b0 0x563e3fbeca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc478a04640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f84b1b0 0x563e3fbeca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49524/0 (socket says 172.21.15.2:49524) 2024-05-14T09:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc479205640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f8222f0 0x563e3fbecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc478a04640 1 -- 172.21.15.2:0/3338897619 learned_addr learned my addr 172.21.15.2:0/3338897619 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc478a04640 1 --2- 172.21.15.2:0/3338897619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f82a4d0 0x563e3fbf3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc479205640 1 -- 172.21.15.2:0/3338897619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f82a4d0 msgr2=0x563e3fbf3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc479205640 1 --2- 172.21.15.2:0/3338897619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f82a4d0 0x563e3fbf3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc479205640 1 -- 172.21.15.2:0/3338897619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f84b1b0 msgr2=0x563e3fbeca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc479205640 1 --2- 172.21.15.2:0/3338897619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f84b1b0 0x563e3fbeca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc46a7fc640 1 --2- 172.21.15.2:0/2466693782 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc4600b28a0 0x7fc4600b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc479205640 1 -- 172.21.15.2:0/3338897619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e3f863210 con 0x563e3f8222f0 2024-05-14T09:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.806+0000 7fc478a04640 1 --2- 172.21.15.2:0/3338897619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f84b1b0 0x563e3fbeca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.807+0000 7fc478a04640 1 --2- 172.21.15.2:0/2466693782 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc4600b28a0 0x7fc4600b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.807+0000 7fc46a7fc640 1 -- 172.21.15.2:0/2466693782 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fc46c102d90 con 0x7fc474108a90 2024-05-14T09:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.807+0000 7fc479205640 1 --2- 172.21.15.2:0/3338897619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f8222f0 0x563e3fbecfe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc474103750 tx=0x7fc470011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=584fad56ccbd0155 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.807+0000 7fc45f7fe640 1 -- 172.21.15.2:0/3338897619 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47001e070 con 0x563e3f8222f0 2024-05-14T09:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.807+0000 7fc45f7fe640 1 -- 172.21.15.2:0/3338897619 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4700198e0 con 0x563e3f8222f0 2024-05-14T09:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.807+0000 7fc45f7fe640 1 -- 172.21.15.2:0/3338897619 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47000e9e0 con 0x563e3f8222f0 2024-05-14T09:44:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.808+0000 7fc478a04640 1 --2- 172.21.15.2:0/2466693782 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc4600b28a0 0x7fc4600b4db0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc46400bfd0 tx=0x7fc46400b7a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.808+0000 7fc4823f5740 1 -- 172.21.15.2:0/3338897619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f8222f0 msgr2=0x563e3fbecfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.808+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3338897619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f8222f0 0x563e3fbecfe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc474103750 tx=0x7fc470011810 comp rx=0 tx=0).stop 2024-05-14T09:44:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.808+0000 7fc4823f5740 1 -- 172.21.15.2:0/3338897619 shutdown_connections 2024-05-14T09:44:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.808+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3338897619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f82a4d0 0x563e3fbf3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.808+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3338897619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f8222f0 0x563e3fbecfe0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.808+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3338897619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f84b1b0 0x563e3fbeca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.808+0000 7fc4823f5740 1 -- 172.21.15.2:0/3338897619 >> 172.21.15.2:0/3338897619 conn(0x563e3f83f300 msgr2=0x563e3fbe83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.808+0000 7fc4823f5740 1 -- 172.21.15.2:0/3338897619 shutdown_connections 2024-05-14T09:44:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.808+0000 7fc4823f5740 1 -- 172.21.15.2:0/3338897619 wait complete. 2024-05-14T09:44:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.809+0000 7fc4823f5740 1 -- start start 2024-05-14T09:44:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.809+0000 7fc4823f5740 1 Processor -- start 2024-05-14T09:44:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.809+0000 7fc4823f5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f8222f0 0x563e3fbf6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.809+0000 7fc4823f5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 0x563e3fbf6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.809+0000 7fc4823f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f84b1b0 0x563e3fbfdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.809+0000 7fc4823f5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563e3fbf4930 con 0x563e3f8222f0 2024-05-14T09:44:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.809+0000 7fc4823f5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563e3fb0e370 con 0x563e3f82a4d0 2024-05-14T09:44:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.809+0000 7fc4823f5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563e3fbf4ab0 con 0x563e3f84b1b0 2024-05-14T09:44:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc4823f5740 10 client.0 _ll_register_callbacks cb 0x563e3f862a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:44:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc478a04640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 0x563e3fbf6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc479205640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f8222f0 0x563e3fbf6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc479a06640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f84b1b0 0x563e3fbfdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc478a04640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 0x563e3fbf6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:57758/0 (socket says 172.21.15.2:57758) 2024-05-14T09:44:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc478a04640 1 -- 172.21.15.2:0/3965638801 learned_addr learned my addr 172.21.15.2:0/3965638801 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc479a06640 1 -- 172.21.15.2:0/3965638801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 msgr2=0x563e3fbf6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc479a06640 1 --2- 172.21.15.2:0/3965638801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 0x563e3fbf6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc479a06640 1 -- 172.21.15.2:0/3965638801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f8222f0 msgr2=0x563e3fbf6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc479a06640 1 --2- 172.21.15.2:0/3965638801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f8222f0 0x563e3fbf6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc479a06640 1 -- 172.21.15.2:0/3965638801 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e3f92f620 con 0x563e3f84b1b0 2024-05-14T09:44:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc479a06640 1 --2- 172.21.15.2:0/3965638801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f84b1b0 0x563e3fbfdb10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0c9c30 tx=0x7fc46c0c9a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=465c871c2fe3b0dd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc45cff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:44:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc45cff9640 1 -- 172.21.15.2:0/3965638801 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c0c9f30 con 0x563e3f84b1b0 2024-05-14T09:44:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc4823f5740 1 -- 172.21.15.2:0/3965638801 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4700198e0 con 0x563e3f84b1b0 2024-05-14T09:44:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc45cff9640 1 -- 172.21.15.2:0/3965638801 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fc46c0d2900 con 0x563e3f84b1b0 2024-05-14T09:44:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.810+0000 7fc45cff9640 1 -- 172.21.15.2:0/3965638801 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc46c0e9050 con 0x563e3f84b1b0 2024-05-14T09:44:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.811+0000 7fc4823f5740 10 client.26200 fetch_fsmap learned FSMap version 40 2024-05-14T09:44:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.811+0000 7fc4823f5740 1 -- 172.21.15.2:0/3965638801 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563e3fbf4a00 con 0x563e3f84b1b0 2024-05-14T09:44:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.811+0000 7fc45cff9640 1 -- 172.21.15.2:0/3965638801 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4700198e0 con 0x563e3f84b1b0 2024-05-14T09:44:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.811+0000 7fc45cff9640 1 -- 172.21.15.2:0/3965638801 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7fc46c0c58a0 con 0x563e3f84b1b0 2024-05-14T09:44:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.811+0000 7fc4823f5740 10 client.26200 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T09:44:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.811+0000 7fc4823f5740 10 client.26200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:44:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.811+0000 7fc4823f5740 20 client.26200 populate_metadata read hostname 'smithi002' 2024-05-14T09:44:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.811+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3965638801 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e3f809d80 0x563e3fc04300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.811+0000 7fc4823f5740 4 client.26200 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:44:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.811+0000 7fc4823f5740 1 -- 172.21.15.2:0/3965638801 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e3fb5f1e0 con 0x563e3f809d80 2024-05-14T09:44:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.811+0000 7fc479205640 1 --2- 172.21.15.2:0/3965638801 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e3f809d80 0x563e3fc04300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.812+0000 7fc45cff9640 10 client.26200 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.812+0000 7fc479205640 1 --2- 172.21.15.2:0/3965638801 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e3f809d80 0x563e3fc04300 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb6ad305b715675 server_cookie=76e3bc7167444f81 in_seq=0 out_seq=0 2024-05-14T09:44:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.812+0000 7fc45cff9640 1 -- 172.21.15.2:0/3965638801 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563e3fb5f1e0 con 0x563e3f809d80 2024-05-14T09:44:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.812+0000 7fc45cff9640 10 client.26200 handle_command_reply: tid=0 2024-05-14T09:44:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.812+0000 7fc4823f5740 1 client.26200 shutdown 2024-05-14T09:44:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.812+0000 7fc4823f5740 1 -- 172.21.15.2:0/3965638801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f84b1b0 msgr2=0x563e3fbfdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.812+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3965638801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f84b1b0 0x563e3fbfdb10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0c9c30 tx=0x7fc46c0c9a40 comp rx=0 tx=0).stop 2024-05-14T09:44:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.813+0000 7fc4823f5740 1 -- 172.21.15.2:0/3965638801 shutdown_connections 2024-05-14T09:44:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.813+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3965638801 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e3f809d80 0x563e3fc04300 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.813+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3965638801 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f84b1b0 0x563e3fbfdb10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.813+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3965638801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 0x563e3fbf6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.813+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3965638801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f8222f0 0x563e3fbf6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.813+0000 7fc45cff9640 0 client.26200 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.813+0000 7fc4823f5740 1 -- 172.21.15.2:0/3965638801 >> 172.21.15.2:0/3965638801 conn(0x563e3f83f300 msgr2=0x563e3fbeb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.813+0000 7fc4823f5740 1 -- 172.21.15.2:0/3965638801 shutdown_connections 2024-05-14T09:44:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.813+0000 7fc4823f5740 1 -- 172.21.15.2:0/3965638801 wait complete. 2024-05-14T09:44:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.813+0000 7fc4823f5740 20 client.26200 trim_cache size 0 max 16384 2024-05-14T09:44:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.829+0000 7fc4823f5740 1 Processor -- start 2024-05-14T09:44:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.829+0000 7fc4823f5740 1 -- start start 2024-05-14T09:44:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.829+0000 7fc4823f5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f8222f0 0x563e3fbf0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.829+0000 7fc4823f5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 0x563e3fbfda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.829+0000 7fc4823f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f84b1b0 0x563e3fbf0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.829+0000 7fc4823f5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc4580125f0 con 0x563e3f8222f0 2024-05-14T09:44:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.829+0000 7fc4823f5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc45800fec0 con 0x563e3f82a4d0 2024-05-14T09:44:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.829+0000 7fc4823f5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc45800dd10 con 0x563e3f84b1b0 2024-05-14T09:44:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.829+0000 7fc479205640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f8222f0 0x563e3fbf0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.829+0000 7fc479205640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f8222f0 0x563e3fbf0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49544/0 (socket says 172.21.15.2:49544) 2024-05-14T09:44:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.829+0000 7fc479205640 1 -- 172.21.15.2:0/3445960545 learned_addr learned my addr 172.21.15.2:0/3445960545 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.829+0000 7fc479a06640 1 --2- 172.21.15.2:0/3445960545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f84b1b0 0x563e3fbf0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.829+0000 7fc478a04640 1 --2- 172.21.15.2:0/3445960545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 0x563e3fbfda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.830+0000 7fc479a06640 1 -- 172.21.15.2:0/3445960545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 msgr2=0x563e3fbfda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.830+0000 7fc479a06640 1 --2- 172.21.15.2:0/3445960545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 0x563e3fbfda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.830+0000 7fc479a06640 1 -- 172.21.15.2:0/3445960545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f8222f0 msgr2=0x563e3fbf0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.830+0000 7fc479a06640 1 --2- 172.21.15.2:0/3445960545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f8222f0 0x563e3fbf0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.830+0000 7fc479a06640 1 -- 172.21.15.2:0/3445960545 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e3fbf3e20 con 0x563e3f84b1b0 2024-05-14T09:44:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.830+0000 7fc479205640 1 --2- 172.21.15.2:0/3445960545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f8222f0 0x563e3fbf0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:44:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.830+0000 7fc479a06640 1 --2- 172.21.15.2:0/3445960545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f84b1b0 0x563e3fbf0f80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc46c006010 tx=0x7fc46c003f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85751d9401ff9d38 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.830+0000 7fc45effd640 1 -- 172.21.15.2:0/3445960545 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c0ce2e0 con 0x563e3f84b1b0 2024-05-14T09:44:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.830+0000 7fc45effd640 1 -- 172.21.15.2:0/3445960545 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc46c0d2d90 con 0x563e3f84b1b0 2024-05-14T09:44:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.830+0000 7fc45effd640 1 -- 172.21.15.2:0/3445960545 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c0d2b60 con 0x563e3f84b1b0 2024-05-14T09:44:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.831+0000 7fc4823f5740 1 -- 172.21.15.2:0/3445960545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f84b1b0 msgr2=0x563e3fbf0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.831+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3445960545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f84b1b0 0x563e3fbf0f80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc46c006010 tx=0x7fc46c003f30 comp rx=0 tx=0).stop 2024-05-14T09:44:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.831+0000 7fc4823f5740 1 -- 172.21.15.2:0/3445960545 shutdown_connections 2024-05-14T09:44:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.831+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3445960545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f84b1b0 0x563e3fbf0f80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.831+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3445960545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 0x563e3fbfda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.831+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3445960545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f8222f0 0x563e3fbf0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.831+0000 7fc4823f5740 1 -- 172.21.15.2:0/3445960545 >> 172.21.15.2:0/3445960545 conn(0x563e3f83f300 msgr2=0x563e3fbe2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.831+0000 7fc4823f5740 1 -- 172.21.15.2:0/3445960545 shutdown_connections 2024-05-14T09:44:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.831+0000 7fc4823f5740 1 -- 172.21.15.2:0/3445960545 wait complete. 2024-05-14T09:44:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.831+0000 7fc4823f5740 1 -- start start 2024-05-14T09:44:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.831+0000 7fc4823f5740 1 Processor -- start 2024-05-14T09:44:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc4823f5740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f8222f0 0x563e3fbfaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc4823f5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 0x563e3fbfb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc4823f5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f84b1b0 0x563e3fbfc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc4823f5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc45800e380 con 0x563e3f84b1b0 2024-05-14T09:44:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc4823f5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc45800f7f0 con 0x563e3f82a4d0 2024-05-14T09:44:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc4823f5740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc45800ea50 con 0x563e3f8222f0 2024-05-14T09:44:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc4823f5740 10 client.0 _ll_register_callbacks cb 0x7fc46c0c5df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:44:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc479205640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f8222f0 0x563e3fbfaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc478a04640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 0x563e3fbfb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc479a06640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f84b1b0 0x563e3fbfc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc479205640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f8222f0 0x563e3fbfaf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:32948/0 (socket says 172.21.15.2:32948) 2024-05-14T09:44:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc479205640 1 -- 172.21.15.2:0/3564638369 learned_addr learned my addr 172.21.15.2:0/3564638369 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc478a04640 1 -- 172.21.15.2:0/3564638369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f8222f0 msgr2=0x563e3fbfaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc478a04640 1 --2- 172.21.15.2:0/3564638369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f8222f0 0x563e3fbfaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc478a04640 1 -- 172.21.15.2:0/3564638369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f84b1b0 msgr2=0x563e3fbfc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc478a04640 1 --2- 172.21.15.2:0/3564638369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f84b1b0 0x563e3fbfc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc478a04640 1 -- 172.21.15.2:0/3564638369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e3fbfc940 con 0x563e3f82a4d0 2024-05-14T09:44:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc479205640 1 --2- 172.21.15.2:0/3564638369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f8222f0 0x563e3fbfaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:44:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.832+0000 7fc479a06640 1 --2- 172.21.15.2:0/3564638369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f84b1b0 0x563e3fbfc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:44:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc478a04640 1 --2- 172.21.15.2:0/3564638369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 0x563e3fbfb450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x563e3fbf76e0 tx=0x7fc464004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd55e4a62f0d71f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc45cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:44:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc45cff9640 1 -- 172.21.15.2:0/3564638369 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46401fc50 con 0x563e3f82a4d0 2024-05-14T09:44:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc4823f5740 1 -- 172.21.15.2:0/3564638369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc46c0d2d90 con 0x563e3f82a4d0 2024-05-14T09:44:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc45cff9640 1 -- 172.21.15.2:0/3564638369 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fc46401fdf0 con 0x563e3f82a4d0 2024-05-14T09:44:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc45cff9640 1 -- 172.21.15.2:0/3564638369 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc464035050 con 0x563e3f82a4d0 2024-05-14T09:44:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc4823f5740 10 client.26222 fetch_fsmap learned FSMap version 40 2024-05-14T09:44:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc4823f5740 1 -- 172.21.15.2:0/3564638369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563e3fbfdd20 con 0x563e3f82a4d0 2024-05-14T09:44:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc45cff9640 1 -- 172.21.15.2:0/3564638369 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc46c0d2d90 con 0x563e3f82a4d0 2024-05-14T09:44:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc45cff9640 1 -- 172.21.15.2:0/3564638369 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7fc46401c040 con 0x563e3f82a4d0 2024-05-14T09:44:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc4823f5740 10 client.26222 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T09:44:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc4823f5740 10 client.26222 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:44:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc4823f5740 20 client.26222 populate_metadata read hostname 'smithi002' 2024-05-14T09:44:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3564638369 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e3f7df1c0 0x563e3fc775e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc4823f5740 4 client.26222 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:44:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.833+0000 7fc4823f5740 1 -- 172.21.15.2:0/3564638369 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e3fb59050 con 0x563e3f7df1c0 2024-05-14T09:44:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.834+0000 7fc479205640 1 --2- 172.21.15.2:0/3564638369 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e3f7df1c0 0x563e3fc775e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.834+0000 7fc45cff9640 10 client.26222 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.834+0000 7fc479205640 1 --2- 172.21.15.2:0/3564638369 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e3f7df1c0 0x563e3fc775e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dae84e5c27637b6 server_cookie=9a6a15c9ebd31453 in_seq=0 out_seq=0 2024-05-14T09:44:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.947+0000 7fc45cff9640 1 -- 172.21.15.2:0/3564638369 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e3fb59050 con 0x563e3f7df1c0 2024-05-14T09:44:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.947+0000 7fc45cff9640 10 client.26222 handle_command_reply: tid=0 2024-05-14T09:44:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.948+0000 7fc4823f5740 1 client.26222 shutdown 2024-05-14T09:44:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.948+0000 7fc4823f5740 1 -- 172.21.15.2:0/3564638369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 msgr2=0x563e3fbfb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.948+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3564638369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 0x563e3fbfb450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x563e3fbf76e0 tx=0x7fc464004730 comp rx=0 tx=0).stop 2024-05-14T09:44:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.948+0000 7fc4823f5740 1 -- 172.21.15.2:0/3564638369 shutdown_connections 2024-05-14T09:44:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.948+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3564638369 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e3f7df1c0 0x563e3fc775e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.948+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3564638369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e3f84b1b0 0x563e3fbfc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.948+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3564638369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e3f82a4d0 0x563e3fbfb450 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.948+0000 7fc45cff9640 0 client.26222 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.948+0000 7fc4823f5740 1 --2- 172.21.15.2:0/3564638369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e3f8222f0 0x563e3fbfaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.948+0000 7fc4823f5740 1 -- 172.21.15.2:0/3564638369 >> 172.21.15.2:0/3564638369 conn(0x563e3f83f300 msgr2=0x563e3fbf6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.948+0000 7fc4823f5740 1 -- 172.21.15.2:0/3564638369 shutdown_connections 2024-05-14T09:44:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.948+0000 7fc4823f5740 1 -- 172.21.15.2:0/3564638369 wait complete. 2024-05-14T09:44:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.948+0000 7fc4823f5740 20 client.26222 trim_cache size 0 max 16384 2024-05-14T09:44:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:44:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:44:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:44:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:44:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.949+0000 7fc47b469640 1 -- 172.21.15.2:0/2466693782 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc4600b28a0 msgr2=0x7fc4600b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.949+0000 7fc47b469640 1 --2- 172.21.15.2:0/2466693782 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc4600b28a0 0x7fc4600b4db0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc46400bfd0 tx=0x7fc46400b7a0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.949+0000 7fc47b469640 1 -- 172.21.15.2:0/2466693782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc474108a90 msgr2=0x7fc4741a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.949+0000 7fc47b469640 1 --2- 172.21.15.2:0/2466693782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc474108a90 0x7fc4741a17b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc46c034040 tx=0x7fc46c032e60 comp rx=0 tx=0).stop 2024-05-14T09:44:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.949+0000 7fc47b469640 1 -- 172.21.15.2:0/2466693782 shutdown_connections 2024-05-14T09:44:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.949+0000 7fc47b469640 1 --2- 172.21.15.2:0/2466693782 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc4600b28a0 0x7fc4600b4db0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.949+0000 7fc47b469640 1 --2- 172.21.15.2:0/2466693782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc474108a90 0x7fc4741a17b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.949+0000 7fc47b469640 1 --2- 172.21.15.2:0/2466693782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc474103d90 0x7fc47419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.949+0000 7fc47b469640 1 --2- 172.21.15.2:0/2466693782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc474102b50 0x7fc47419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.949+0000 7fc47b469640 1 -- 172.21.15.2:0/2466693782 >> 172.21.15.2:0/2466693782 conn(0x7fc4740fdd40 msgr2=0x7fc474101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.949+0000 7fc47b469640 1 -- 172.21.15.2:0/2466693782 shutdown_connections 2024-05-14T09:44:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:44.949+0000 7fc47b469640 1 -- 172.21.15.2:0/2466693782 wait complete. 2024-05-14T09:44:45.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 922725 10.03 MB/sec execute 371 sec latency 109.445 ms 2024-05-14T09:44:45.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:45 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:44:45.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:45 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:44:45.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:45 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:44:46.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 923862 10.03 MB/sec execute 372 sec latency 601.304 ms 2024-05-14T09:44:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:46 smithi002 ceph-mon[31557]: pgmap v719: 129 pgs: 129 active+clean; 159 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-05-14T09:44:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:46 smithi029 ceph-mon[36249]: pgmap v719: 129 pgs: 129 active+clean; 159 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-05-14T09:44:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:46 smithi026 ceph-mon[36056]: pgmap v719: 129 pgs: 129 active+clean; 159 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-05-14T09:44:47.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 924768 10.01 MB/sec execute 373 sec latency 604.146 ms 2024-05-14T09:44:47.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:47 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:44:47.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:47 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:44:47.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:47 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:44:48.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 927084 10.02 MB/sec execute 374 sec latency 40.988 ms 2024-05-14T09:44:48.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:48 smithi002 ceph-mon[31557]: pgmap v720: 129 pgs: 129 active+clean; 152 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.7 MiB/s wr, 324 op/s 2024-05-14T09:44:48.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:48 smithi029 ceph-mon[36249]: pgmap v720: 129 pgs: 129 active+clean; 152 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.7 MiB/s wr, 324 op/s 2024-05-14T09:44:48.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:48 smithi026 ceph-mon[36056]: pgmap v720: 129 pgs: 129 active+clean; 152 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.7 MiB/s wr, 324 op/s 2024-05-14T09:44:49.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 928599 10.01 MB/sec execute 375 sec latency 20.490 ms 2024-05-14T09:44:50.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 931108 10.03 MB/sec execute 376 sec latency 14.098 ms 2024-05-14T09:44:50.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:50 smithi002 ceph-mon[31557]: pgmap v721: 129 pgs: 129 active+clean; 152 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.3 MiB/s wr, 258 op/s 2024-05-14T09:44:50.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:50 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:44:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:50 smithi029 ceph-mon[36249]: pgmap v721: 129 pgs: 129 active+clean; 152 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.3 MiB/s wr, 258 op/s 2024-05-14T09:44:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:50 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:44:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:50 smithi026 ceph-mon[36056]: pgmap v721: 129 pgs: 129 active+clean; 152 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.3 MiB/s wr, 258 op/s 2024-05-14T09:44:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:50 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:44:51.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 932442 10.01 MB/sec execute 377 sec latency 30.342 ms 2024-05-14T09:44:51.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:44:51.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:51.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:51.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:51.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:51.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:51 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:51.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:44:51.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:51.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:51.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:51.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:51.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:51 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:44:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:51.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:51 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:52.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 934644 10.02 MB/sec execute 378 sec latency 45.590 ms 2024-05-14T09:44:52.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:52 smithi002 ceph-mon[31557]: pgmap v722: 129 pgs: 129 active+clean; 151 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 2.3 MiB/s wr, 287 op/s 2024-05-14T09:44:52.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:52.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:44:52.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:44:52.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:52 smithi029 ceph-mon[36249]: pgmap v722: 129 pgs: 129 active+clean; 151 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 2.3 MiB/s wr, 287 op/s 2024-05-14T09:44:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:52 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:52 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:44:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:52 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:44:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:52 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:52 smithi026 ceph-mon[36056]: pgmap v722: 129 pgs: 129 active+clean; 151 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 2.3 MiB/s wr, 287 op/s 2024-05-14T09:44:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:52 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:52 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:44:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:52 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:44:52.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:52 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:44:53.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 935938 10.01 MB/sec execute 379 sec latency 48.132 ms 2024-05-14T09:44:54.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 938277 10.02 MB/sec execute 380 sec latency 19.661 ms 2024-05-14T09:44:54.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:54 smithi002 ceph-mon[31557]: pgmap v723: 129 pgs: 129 active+clean; 151 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 2.9 MiB/s wr, 333 op/s 2024-05-14T09:44:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:54 smithi029 ceph-mon[36249]: pgmap v723: 129 pgs: 129 active+clean; 151 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 2.9 MiB/s wr, 333 op/s 2024-05-14T09:44:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:54 smithi026 ceph-mon[36056]: pgmap v723: 129 pgs: 129 active+clean; 151 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 2.9 MiB/s wr, 333 op/s 2024-05-14T09:44:55.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 939788 10.01 MB/sec execute 381 sec latency 19.582 ms 2024-05-14T09:44:56.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 941874 10.02 MB/sec execute 382 sec latency 25.098 ms 2024-05-14T09:44:56.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:56 smithi002 ceph-mon[31557]: pgmap v724: 129 pgs: 129 active+clean; 151 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 2.1 MiB/s wr, 241 op/s 2024-05-14T09:44:56.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:56 smithi029 ceph-mon[36249]: pgmap v724: 129 pgs: 129 active+clean; 151 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 2.1 MiB/s wr, 241 op/s 2024-05-14T09:44:56.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:56 smithi026 ceph-mon[36056]: pgmap v724: 129 pgs: 129 active+clean; 151 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 2.1 MiB/s wr, 241 op/s 2024-05-14T09:44:57.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 943267 10.01 MB/sec execute 383 sec latency 46.213 ms 2024-05-14T09:44:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.052+0000 7f4a015b7640 1 -- 172.21.15.2:0/1571749120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49fc103d90 msgr2=0x7f49fc1041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.052+0000 7f4a015b7640 1 --2- 172.21.15.2:0/1571749120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49fc103d90 0x7f49fc1041a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009900 tx=0x7f49e402a410 comp rx=0 tx=0).stop 2024-05-14T09:44:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.052+0000 7f4a015b7640 1 -- 172.21.15.2:0/1571749120 shutdown_connections 2024-05-14T09:44:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.052+0000 7f4a015b7640 1 --2- 172.21.15.2:0/1571749120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49fc108a90 0x7f49fc10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.052+0000 7f4a015b7640 1 --2- 172.21.15.2:0/1571749120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49fc103d90 0x7f49fc1041a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.052+0000 7f4a015b7640 1 --2- 172.21.15.2:0/1571749120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49fc102b50 0x7f49fc103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.052+0000 7f4a015b7640 1 -- 172.21.15.2:0/1571749120 >> 172.21.15.2:0/1571749120 conn(0x7f49fc0fdd40 msgr2=0x7f49fc1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.052+0000 7f4a015b7640 1 -- 172.21.15.2:0/1571749120 shutdown_connections 2024-05-14T09:44:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.052+0000 7f4a015b7640 1 -- 172.21.15.2:0/1571749120 wait complete. 2024-05-14T09:44:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.053+0000 7f4a015b7640 1 Processor -- start 2024-05-14T09:44:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.053+0000 7f4a015b7640 1 -- start start 2024-05-14T09:44:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.053+0000 7f4a015b7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49fc102b50 0x7f49fc06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.053+0000 7f4a015b7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49fc103d90 0x7f49fc19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f4a015b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49fc108a90 0x7f49fc1a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f4a015b7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc10e650 con 0x7f49fc103d90 2024-05-14T09:44:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f4a015b7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc10e4d0 con 0x7f49fc108a90 2024-05-14T09:44:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f4a015b7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc10e7d0 con 0x7f49fc102b50 2024-05-14T09:44:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f49fa7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49fc103d90 0x7f49fc19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f49faffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49fc102b50 0x7f49fc06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f49fa7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49fc103d90 0x7f49fc19f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60170/0 (socket says 172.21.15.2:60170) 2024-05-14T09:44:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f49fa7fc640 1 -- 172.21.15.2:0/4210236145 learned_addr learned my addr 172.21.15.2:0/4210236145 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f49faffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49fc102b50 0x7f49fc06b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:40550/0 (socket says 172.21.15.2:40550) 2024-05-14T09:44:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f49fb7fe640 1 --2- 172.21.15.2:0/4210236145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49fc108a90 0x7f49fc1a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f49fa7fc640 1 -- 172.21.15.2:0/4210236145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49fc102b50 msgr2=0x7f49fc06b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f49fa7fc640 1 --2- 172.21.15.2:0/4210236145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49fc102b50 0x7f49fc06b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f49fa7fc640 1 -- 172.21.15.2:0/4210236145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49fc108a90 msgr2=0x7f49fc1a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f49fa7fc640 1 --2- 172.21.15.2:0/4210236145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49fc108a90 0x7f49fc1a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f49fa7fc640 1 -- 172.21.15.2:0/4210236145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc1a1db0 con 0x7f49fc103d90 2024-05-14T09:44:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.054+0000 7f49fa7fc640 1 --2- 172.21.15.2:0/4210236145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49fc103d90 0x7f49fc19f040 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f49e4006010 tx=0x7f49e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.055+0000 7f49f3fff640 1 -- 172.21.15.2:0/4210236145 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e403f030 con 0x7f49fc103d90 2024-05-14T09:44:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.055+0000 7f4a015b7640 1 -- 172.21.15.2:0/4210236145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49fc1a2040 con 0x7f49fc103d90 2024-05-14T09:44:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.055+0000 7f49f3fff640 1 -- 172.21.15.2:0/4210236145 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e4031910 con 0x7f49fc103d90 2024-05-14T09:44:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.055+0000 7f49f3fff640 1 -- 172.21.15.2:0/4210236145 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4035070 con 0x7f49fc103d90 2024-05-14T09:44:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.055+0000 7f4a015b7640 1 -- 172.21.15.2:0/4210236145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49fc1a2530 con 0x7f49fc103d90 2024-05-14T09:44:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.058+0000 7f49f3fff640 1 -- 172.21.15.2:0/4210236145 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f49e4042030 con 0x7f49fc103d90 2024-05-14T09:44:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.058+0000 7f4a0846e740 1 Processor -- start 2024-05-14T09:44:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.058+0000 7f4a0846e740 1 -- start start 2024-05-14T09:44:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.058+0000 7f4a0846e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ee31b0 0x556d19284a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.058+0000 7f4a0846e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18eba2f0 0x556d19284fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.058+0000 7f4a0846e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18ec24d0 0x556d1928bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.058+0000 7f4a0846e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556d191e95d0 con 0x556d18ec24d0 2024-05-14T09:44:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.058+0000 7f4a0846e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556d191a2890 con 0x556d18eba2f0 2024-05-14T09:44:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.058+0000 7f4a0846e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556d191e5590 con 0x556d18ee31b0 2024-05-14T09:44:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.058+0000 7f49fa7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18ec24d0 0x556d1928bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.058+0000 7f49faffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18eba2f0 0x556d19284fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.058+0000 7f49fb7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ee31b0 0x556d19284a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.058+0000 7f49faffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18eba2f0 0x556d19284fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:60750/0 (socket says 172.21.15.2:60750) 2024-05-14T09:44:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.058+0000 7f49faffd640 1 -- 172.21.15.2:0/1260757592 learned_addr learned my addr 172.21.15.2:0/1260757592 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.059+0000 7f49fb7fe640 1 -- 172.21.15.2:0/1260757592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18eba2f0 msgr2=0x556d19284fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.059+0000 7f49fb7fe640 1 --2- 172.21.15.2:0/1260757592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18eba2f0 0x556d19284fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.059+0000 7f49fb7fe640 1 -- 172.21.15.2:0/1260757592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18ec24d0 msgr2=0x556d1928bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.059+0000 7f49fb7fe640 1 --2- 172.21.15.2:0/1260757592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18ec24d0 0x556d1928bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.059+0000 7f49fb7fe640 1 -- 172.21.15.2:0/1260757592 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d18efb210 con 0x556d18ee31b0 2024-05-14T09:44:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.059+0000 7f49f3fff640 1 --2- 172.21.15.2:0/4210236145 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f49e80b28a0 0x7f49e80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.059+0000 7f49fb7fe640 1 --2- 172.21.15.2:0/1260757592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ee31b0 0x556d19284a60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009b80 tx=0x7f49f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ffbcdd1191c1585 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.059+0000 7f49faffd640 1 --2- 172.21.15.2:0/4210236145 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f49e80b28a0 0x7f49e80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.059+0000 7f49f3fff640 1 -- 172.21.15.2:0/4210236145 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f49e4102b00 con 0x7f49fc103d90 2024-05-14T09:44:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.059+0000 7f49f17fa640 1 -- 172.21.15.2:0/1260757592 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4004050 con 0x556d18ee31b0 2024-05-14T09:44:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.059+0000 7f49f17fa640 1 -- 172.21.15.2:0/1260757592 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49f40054d0 con 0x556d18ee31b0 2024-05-14T09:44:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.059+0000 7f49f17fa640 1 -- 172.21.15.2:0/1260757592 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4004cd0 con 0x556d18ee31b0 2024-05-14T09:44:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.060+0000 7f4a0846e740 1 -- 172.21.15.2:0/1260757592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ee31b0 msgr2=0x556d19284a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.060+0000 7f49faffd640 1 --2- 172.21.15.2:0/4210236145 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f49e80b28a0 0x7f49e80b4db0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x556d19285e40 tx=0x7f49ec009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.060+0000 7f4a0846e740 1 --2- 172.21.15.2:0/1260757592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ee31b0 0x556d19284a60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009b80 tx=0x7f49f4011810 comp rx=0 tx=0).stop 2024-05-14T09:44:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.060+0000 7f4a0846e740 1 -- 172.21.15.2:0/1260757592 shutdown_connections 2024-05-14T09:44:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.060+0000 7f4a0846e740 1 --2- 172.21.15.2:0/1260757592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18ec24d0 0x556d1928bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.060+0000 7f4a0846e740 1 --2- 172.21.15.2:0/1260757592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18eba2f0 0x556d19284fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.060+0000 7f4a0846e740 1 --2- 172.21.15.2:0/1260757592 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ee31b0 0x556d19284a60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.060+0000 7f4a0846e740 1 -- 172.21.15.2:0/1260757592 >> 172.21.15.2:0/1260757592 conn(0x556d18ed7300 msgr2=0x556d192803a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.060+0000 7f4a0846e740 1 -- 172.21.15.2:0/1260757592 shutdown_connections 2024-05-14T09:44:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.060+0000 7f4a0846e740 1 -- 172.21.15.2:0/1260757592 wait complete. 2024-05-14T09:44:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f4a0846e740 1 -- start start 2024-05-14T09:44:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f4a0846e740 1 Processor -- start 2024-05-14T09:44:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f4a0846e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18eba2f0 0x556d1928e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f4a0846e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ec24d0 0x556d1928ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f4a0846e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18ee31b0 0x556d19295c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f4a0846e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556d1928c930 con 0x556d18ee31b0 2024-05-14T09:44:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f4a0846e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556d191a6370 con 0x556d18ec24d0 2024-05-14T09:44:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f4a0846e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556d1928cab0 con 0x556d18eba2f0 2024-05-14T09:44:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f4a0846e740 10 client.0 _ll_register_callbacks cb 0x556d18efaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:44:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f49fb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18ee31b0 0x556d19295c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f49faffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18eba2f0 0x556d1928e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f49fa7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ec24d0 0x556d1928ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f49fb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18ee31b0 0x556d19295c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60184/0 (socket says 172.21.15.2:60184) 2024-05-14T09:44:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f49faffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18eba2f0 0x556d1928e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:40562/0 (socket says 172.21.15.2:40562) 2024-05-14T09:44:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f49fa7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ec24d0 0x556d1928ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:60764/0 (socket says 172.21.15.2:60764) 2024-05-14T09:44:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f49fb7fe640 1 -- 172.21.15.2:0/2006865856 learned_addr learned my addr 172.21.15.2:0/2006865856 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f49fb7fe640 1 -- 172.21.15.2:0/2006865856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18eba2f0 msgr2=0x556d1928e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f49fb7fe640 1 --2- 172.21.15.2:0/2006865856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18eba2f0 0x556d1928e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f49fb7fe640 1 -- 172.21.15.2:0/2006865856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ec24d0 msgr2=0x556d1928ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.061+0000 7f49fb7fe640 1 --2- 172.21.15.2:0/2006865856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ec24d0 0x556d1928ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.062+0000 7f49fb7fe640 1 -- 172.21.15.2:0/2006865856 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d18fc7620 con 0x556d18ee31b0 2024-05-14T09:44:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.062+0000 7f49fb7fe640 1 --2- 172.21.15.2:0/2006865856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18ee31b0 0x556d19295c10 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f49f401b040 tx=0x7f49f4005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a3019de0f4b5ae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.062+0000 7f49e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:44:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.062+0000 7f49e27fc640 1 -- 172.21.15.2:0/2006865856 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4029070 con 0x556d18ee31b0 2024-05-14T09:44:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.062+0000 7f4a0846e740 1 -- 172.21.15.2:0/2006865856 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49f40054d0 con 0x556d18ee31b0 2024-05-14T09:44:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.062+0000 7f49e27fc640 1 -- 172.21.15.2:0/2006865856 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f49f4004a50 con 0x556d18ee31b0 2024-05-14T09:44:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.062+0000 7f49e27fc640 1 -- 172.21.15.2:0/2006865856 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f49f4035050 con 0x556d18ee31b0 2024-05-14T09:44:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.062+0000 7f4a0846e740 10 client.16554 fetch_fsmap learned FSMap version 42 2024-05-14T09:44:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.062+0000 7f4a0846e740 1 -- 172.21.15.2:0/2006865856 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556d19298b40 con 0x556d18ee31b0 2024-05-14T09:44:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.062+0000 7f49e27fc640 1 -- 172.21.15.2:0/2006865856 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49f40054d0 con 0x556d18ee31b0 2024-05-14T09:44:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.062+0000 7f49e27fc640 1 -- 172.21.15.2:0/2006865856 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f49f4023430 con 0x556d18ee31b0 2024-05-14T09:44:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.063+0000 7f4a0846e740 10 client.16554 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T09:44:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.063+0000 7f4a0846e740 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:44:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.063+0000 7f4a0846e740 20 client.16554 populate_metadata read hostname 'smithi002' 2024-05-14T09:44:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.063+0000 7f4a0846e740 1 --2- 172.21.15.2:0/2006865856 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556d18ea1d80 0x556d1929bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.063+0000 7f4a0846e740 4 client.16554 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:44:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.063+0000 7f4a0846e740 1 -- 172.21.15.2:0/2006865856 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d191f71e0 con 0x556d18ea1d80 2024-05-14T09:44:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.063+0000 7f49faffd640 1 --2- 172.21.15.2:0/2006865856 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556d18ea1d80 0x556d1929bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.063+0000 7f49e27fc640 10 client.16554 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.063+0000 7f49faffd640 1 --2- 172.21.15.2:0/2006865856 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556d18ea1d80 0x556d1929bfd0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86db0ad791dfbeaa server_cookie=452631b9887d99a6 in_seq=0 out_seq=0 2024-05-14T09:44:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.064+0000 7f49e27fc640 1 -- 172.21.15.2:0/2006865856 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556d191f71e0 con 0x556d18ea1d80 2024-05-14T09:44:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.064+0000 7f49e27fc640 10 client.16554 handle_command_reply: tid=0 2024-05-14T09:44:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.064+0000 7f4a0846e740 1 client.16554 shutdown 2024-05-14T09:44:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.064+0000 7f4a0846e740 1 -- 172.21.15.2:0/2006865856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18ee31b0 msgr2=0x556d19295c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.064+0000 7f4a0846e740 1 --2- 172.21.15.2:0/2006865856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18ee31b0 0x556d19295c10 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f49f401b040 tx=0x7f49f4005250 comp rx=0 tx=0).stop 2024-05-14T09:44:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.064+0000 7f4a0846e740 1 -- 172.21.15.2:0/2006865856 shutdown_connections 2024-05-14T09:44:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.064+0000 7f4a0846e740 1 --2- 172.21.15.2:0/2006865856 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556d18ea1d80 0x556d1929bfd0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.064+0000 7f49e27fc640 0 client.16554 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.064+0000 7f4a0846e740 1 --2- 172.21.15.2:0/2006865856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18ee31b0 0x556d19295c10 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.064+0000 7f4a0846e740 1 --2- 172.21.15.2:0/2006865856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ec24d0 0x556d1928ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.064+0000 7f4a0846e740 1 --2- 172.21.15.2:0/2006865856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18eba2f0 0x556d1928e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.064+0000 7f4a0846e740 1 -- 172.21.15.2:0/2006865856 >> 172.21.15.2:0/2006865856 conn(0x556d18ed7300 msgr2=0x556d19283300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.064+0000 7f4a0846e740 1 -- 172.21.15.2:0/2006865856 shutdown_connections 2024-05-14T09:44:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.064+0000 7f4a0846e740 1 -- 172.21.15.2:0/2006865856 wait complete. 2024-05-14T09:44:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.064+0000 7f4a0846e740 20 client.16554 trim_cache size 0 max 16384 2024-05-14T09:44:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.080+0000 7f4a0846e740 1 Processor -- start 2024-05-14T09:44:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.080+0000 7f4a0846e740 1 -- start start 2024-05-14T09:44:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f4a0846e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ea1d80 0x556d19289210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f4a0846e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18eba2f0 0x556d19280370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f4a0846e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ec24d0 0x556d19295f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f4a0846e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc1af530 con 0x556d18eba2f0 2024-05-14T09:44:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f4a0846e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc1acdd0 con 0x556d18ec24d0 2024-05-14T09:44:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f4a0846e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc1aac20 con 0x556d18ea1d80 2024-05-14T09:44:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f49fa7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18eba2f0 0x556d19280370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f49fa7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18eba2f0 0x556d19280370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60198/0 (socket says 172.21.15.2:60198) 2024-05-14T09:44:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f49fa7fc640 1 -- 172.21.15.2:0/935989131 learned_addr learned my addr 172.21.15.2:0/935989131 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f49fb7fe640 1 --2- 172.21.15.2:0/935989131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ec24d0 0x556d19295f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f49faffd640 1 --2- 172.21.15.2:0/935989131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ea1d80 0x556d19289210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f49fa7fc640 1 -- 172.21.15.2:0/935989131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ea1d80 msgr2=0x556d19289210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f49fa7fc640 1 --2- 172.21.15.2:0/935989131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ea1d80 0x556d19289210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f49fa7fc640 1 -- 172.21.15.2:0/935989131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ec24d0 msgr2=0x556d19295f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f49fa7fc640 1 --2- 172.21.15.2:0/935989131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ec24d0 0x556d19295f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f49fa7fc640 1 -- 172.21.15.2:0/935989131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d192808b0 con 0x556d18eba2f0 2024-05-14T09:44:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f49fa7fc640 1 --2- 172.21.15.2:0/935989131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18eba2f0 0x556d19280370 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f49e41001e0 tx=0x7f49e403b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4052f30b0d209 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f49f0ff9640 1 -- 172.21.15.2:0/935989131 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4049030 con 0x556d18eba2f0 2024-05-14T09:44:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.081+0000 7f49f0ff9640 1 -- 172.21.15.2:0/935989131 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e410a040 con 0x556d18eba2f0 2024-05-14T09:44:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.082+0000 7f49f0ff9640 1 -- 172.21.15.2:0/935989131 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40d2b60 con 0x556d18eba2f0 2024-05-14T09:44:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.082+0000 7f4a0846e740 1 -- 172.21.15.2:0/935989131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18eba2f0 msgr2=0x556d19280370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.082+0000 7f4a0846e740 1 --2- 172.21.15.2:0/935989131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18eba2f0 0x556d19280370 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f49e41001e0 tx=0x7f49e403b5d0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.082+0000 7f4a0846e740 1 -- 172.21.15.2:0/935989131 shutdown_connections 2024-05-14T09:44:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.082+0000 7f4a0846e740 1 --2- 172.21.15.2:0/935989131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ec24d0 0x556d19295f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.082+0000 7f4a0846e740 1 --2- 172.21.15.2:0/935989131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18eba2f0 0x556d19280370 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.082+0000 7f4a0846e740 1 --2- 172.21.15.2:0/935989131 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ea1d80 0x556d19289210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.082+0000 7f4a0846e740 1 -- 172.21.15.2:0/935989131 >> 172.21.15.2:0/935989131 conn(0x556d18ed7300 msgr2=0x556d1927fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.083+0000 7f4a0846e740 1 -- 172.21.15.2:0/935989131 shutdown_connections 2024-05-14T09:44:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.083+0000 7f4a0846e740 1 -- 172.21.15.2:0/935989131 wait complete. 2024-05-14T09:44:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.083+0000 7f4a0846e740 1 -- start start 2024-05-14T09:44:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.083+0000 7f4a0846e740 1 Processor -- start 2024-05-14T09:44:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.083+0000 7f4a0846e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ea1d80 0x556d192ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.083+0000 7f4a0846e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18eba2f0 0x556d192ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.083+0000 7f4a0846e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ec24d0 0x556d19306c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.083+0000 7f4a0846e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc1ab290 con 0x556d18eba2f0 2024-05-14T09:44:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.083+0000 7f4a0846e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc1ac700 con 0x556d18ea1d80 2024-05-14T09:44:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.083+0000 7f4a0846e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc1ab960 con 0x556d18ec24d0 2024-05-14T09:44:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.083+0000 7f4a0846e740 10 client.0 _ll_register_callbacks cb 0x7f49e41064b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:44:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.083+0000 7f49fa7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18eba2f0 0x556d192ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.083+0000 7f49faffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ea1d80 0x556d192ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f49fa7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18eba2f0 0x556d192ffd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60214/0 (socket says 172.21.15.2:60214) 2024-05-14T09:44:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f49faffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ea1d80 0x556d192ff7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:60780/0 (socket says 172.21.15.2:60780) 2024-05-14T09:44:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f49fa7fc640 1 -- 172.21.15.2:0/803338276 learned_addr learned my addr 172.21.15.2:0/803338276 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:44:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f49fb7fe640 1 --2- 172.21.15.2:0/803338276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ec24d0 0x556d19306c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f49fa7fc640 1 -- 172.21.15.2:0/803338276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ec24d0 msgr2=0x556d19306c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f49fa7fc640 1 --2- 172.21.15.2:0/803338276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ec24d0 0x556d19306c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f49fa7fc640 1 -- 172.21.15.2:0/803338276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ea1d80 msgr2=0x556d192ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:44:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f49fa7fc640 1 --2- 172.21.15.2:0/803338276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ea1d80 0x556d192ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f49fa7fc640 1 -- 172.21.15.2:0/803338276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d193071c0 con 0x556d18eba2f0 2024-05-14T09:44:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f49faffd640 1 --2- 172.21.15.2:0/803338276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ea1d80 0x556d192ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:44:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f49fa7fc640 1 --2- 172.21.15.2:0/803338276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18eba2f0 0x556d192ffd10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f49e40c9d40 tx=0x7f49e41001a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fb13fcd7d154fa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:44:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f49e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:44:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f49e27fc640 1 -- 172.21.15.2:0/803338276 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4049040 con 0x556d18eba2f0 2024-05-14T09:44:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f49e27fc640 1 -- 172.21.15.2:0/803338276 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f49e40e4260 con 0x556d18eba2f0 2024-05-14T09:44:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f4a0846e740 1 -- 172.21.15.2:0/803338276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e410a040 con 0x556d18eba2f0 2024-05-14T09:44:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f49e27fc640 1 -- 172.21.15.2:0/803338276 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f49e40c9520 con 0x556d18eba2f0 2024-05-14T09:44:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f4a0846e740 10 client.16560 fetch_fsmap learned FSMap version 42 2024-05-14T09:44:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.084+0000 7f4a0846e740 1 -- 172.21.15.2:0/803338276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556d19308450 con 0x556d18eba2f0 2024-05-14T09:44:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.085+0000 7f49e27fc640 1 -- 172.21.15.2:0/803338276 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e410a040 con 0x556d18eba2f0 2024-05-14T09:44:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.085+0000 7f49e27fc640 1 -- 172.21.15.2:0/803338276 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f49e40ca0a0 con 0x556d18eba2f0 2024-05-14T09:44:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.085+0000 7f4a0846e740 10 client.16560 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T09:44:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.085+0000 7f4a0846e740 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:44:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.085+0000 7f4a0846e740 20 client.16560 populate_metadata read hostname 'smithi002' 2024-05-14T09:44:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.085+0000 7f4a0846e740 1 --2- 172.21.15.2:0/803338276 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556d18ee31b0 0x556d1930ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:44:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.085+0000 7f4a0846e740 4 client.16560 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:44:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.085+0000 7f4a0846e740 1 -- 172.21.15.2:0/803338276 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d191f1050 con 0x556d18ee31b0 2024-05-14T09:44:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.085+0000 7f49faffd640 1 --2- 172.21.15.2:0/803338276 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556d18ee31b0 0x556d1930ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:44:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.085+0000 7f49faffd640 1 --2- 172.21.15.2:0/803338276 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556d18ee31b0 0x556d1930ad00 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75081c38facb13c4 server_cookie=581281356d791ae7 in_seq=0 out_seq=0 2024-05-14T09:44:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.085+0000 7f49e27fc640 10 client.16560 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.239+0000 7f49e27fc640 1 -- 172.21.15.2:0/803338276 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d191f1050 con 0x556d18ee31b0 2024-05-14T09:44:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.239+0000 7f49e27fc640 10 client.16560 handle_command_reply: tid=0 2024-05-14T09:44:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.239+0000 7f4a0846e740 1 client.16560 shutdown 2024-05-14T09:44:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.240+0000 7f4a0846e740 1 -- 172.21.15.2:0/803338276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18eba2f0 msgr2=0x556d192ffd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.240+0000 7f4a0846e740 1 --2- 172.21.15.2:0/803338276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18eba2f0 0x556d192ffd10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f49e40c9d40 tx=0x7f49e41001a0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.240+0000 7f4a0846e740 1 -- 172.21.15.2:0/803338276 shutdown_connections 2024-05-14T09:44:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.240+0000 7f4a0846e740 1 --2- 172.21.15.2:0/803338276 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556d18ee31b0 0x556d1930ad00 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.240+0000 7f4a0846e740 1 --2- 172.21.15.2:0/803338276 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556d18ec24d0 0x556d19306c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.240+0000 7f4a0846e740 1 --2- 172.21.15.2:0/803338276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d18eba2f0 0x556d192ffd10 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.240+0000 7f4a0846e740 1 --2- 172.21.15.2:0/803338276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d18ea1d80 0x556d192ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.240+0000 7f49e27fc640 0 client.16560 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:44:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.240+0000 7f4a0846e740 1 -- 172.21.15.2:0/803338276 >> 172.21.15.2:0/803338276 conn(0x556d18ed7300 msgr2=0x556d1928e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.240+0000 7f4a0846e740 1 -- 172.21.15.2:0/803338276 shutdown_connections 2024-05-14T09:44:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.240+0000 7f4a0846e740 1 -- 172.21.15.2:0/803338276 wait complete. 2024-05-14T09:44:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.240+0000 7f4a0846e740 20 client.16560 trim_cache size 0 max 16384 2024-05-14T09:44:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:44:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:44:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:44:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:44:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.241+0000 7f4a015b7640 1 -- 172.21.15.2:0/4210236145 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f49e80b28a0 msgr2=0x7f49e80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.241+0000 7f4a015b7640 1 --2- 172.21.15.2:0/4210236145 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f49e80b28a0 0x7f49e80b4db0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x556d19285e40 tx=0x7f49ec009500 comp rx=0 tx=0).stop 2024-05-14T09:44:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.241+0000 7f4a015b7640 1 -- 172.21.15.2:0/4210236145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49fc103d90 msgr2=0x7f49fc19f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:44:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.241+0000 7f4a015b7640 1 --2- 172.21.15.2:0/4210236145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49fc103d90 0x7f49fc19f040 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f49e4006010 tx=0x7f49e40042f0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.241+0000 7f4a015b7640 1 -- 172.21.15.2:0/4210236145 shutdown_connections 2024-05-14T09:44:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.241+0000 7f4a015b7640 1 --2- 172.21.15.2:0/4210236145 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f49e80b28a0 0x7f49e80b4db0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.241+0000 7f4a015b7640 1 --2- 172.21.15.2:0/4210236145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49fc108a90 0x7f49fc1a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.241+0000 7f4a015b7640 1 --2- 172.21.15.2:0/4210236145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49fc103d90 0x7f49fc19f040 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.241+0000 7f4a015b7640 1 --2- 172.21.15.2:0/4210236145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f49fc102b50 0x7f49fc06b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:44:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.241+0000 7f4a015b7640 1 -- 172.21.15.2:0/4210236145 >> 172.21.15.2:0/4210236145 conn(0x7f49fc0fdd40 msgr2=0x7f49fc104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:44:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.241+0000 7f4a015b7640 1 -- 172.21.15.2:0/4210236145 shutdown_connections 2024-05-14T09:44:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:44:58.241+0000 7f4a015b7640 1 -- 172.21.15.2:0/4210236145 wait complete. 2024-05-14T09:44:58.276 INFO:tasks.workunit.client.0.smithi002.stdout: 1 945320 10.02 MB/sec execute 384 sec latency 96.854 ms 2024-05-14T09:44:58.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:44:58 smithi002 ceph-mon[31557]: pgmap v725: 129 pgs: 129 active+clean; 153 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 2.8 MiB/s wr, 310 op/s 2024-05-14T09:44:58.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:44:58 smithi029 ceph-mon[36249]: pgmap v725: 129 pgs: 129 active+clean; 153 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 2.8 MiB/s wr, 310 op/s 2024-05-14T09:44:58.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:44:58 smithi026 ceph-mon[36056]: pgmap v725: 129 pgs: 129 active+clean; 153 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 2.8 MiB/s wr, 310 op/s 2024-05-14T09:44:59.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 946764 10.01 MB/sec execute 385 sec latency 17.509 ms 2024-05-14T09:45:00.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 948978 10.01 MB/sec execute 386 sec latency 14.162 ms 2024-05-14T09:45:00.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:00 smithi002 ceph-mon[31557]: pgmap v726: 129 pgs: 129 active+clean; 154 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 2.5 MiB/s wr, 260 op/s 2024-05-14T09:45:00.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:00 smithi029 ceph-mon[36249]: pgmap v726: 129 pgs: 129 active+clean; 154 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 2.5 MiB/s wr, 260 op/s 2024-05-14T09:45:00.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:00 smithi026 ceph-mon[36056]: pgmap v726: 129 pgs: 129 active+clean; 154 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 2.5 MiB/s wr, 260 op/s 2024-05-14T09:45:01.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 950793 10.00 MB/sec execute 387 sec latency 30.543 ms 2024-05-14T09:45:02.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 952487 10.01 MB/sec execute 388 sec latency 100.596 ms 2024-05-14T09:45:02.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:02 smithi002 ceph-mon[31557]: pgmap v727: 129 pgs: 129 active+clean; 150 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 2.4 MiB/s wr, 248 op/s 2024-05-14T09:45:02.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:02 smithi029 ceph-mon[36249]: pgmap v727: 129 pgs: 129 active+clean; 150 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 2.4 MiB/s wr, 248 op/s 2024-05-14T09:45:02.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:02 smithi026 ceph-mon[36056]: pgmap v727: 129 pgs: 129 active+clean; 150 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 2.4 MiB/s wr, 248 op/s 2024-05-14T09:45:03.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 953836 10.00 MB/sec execute 389 sec latency 23.315 ms 2024-05-14T09:45:04.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 955998 10.01 MB/sec execute 390 sec latency 16.862 ms 2024-05-14T09:45:04.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:04 smithi002 ceph-mon[31557]: pgmap v728: 129 pgs: 129 active+clean; 152 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 2.7 MiB/s wr, 331 op/s 2024-05-14T09:45:04.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:04 smithi029 ceph-mon[36249]: pgmap v728: 129 pgs: 129 active+clean; 152 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 2.7 MiB/s wr, 331 op/s 2024-05-14T09:45:04.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:04 smithi026 ceph-mon[36056]: pgmap v728: 129 pgs: 129 active+clean; 152 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 2.7 MiB/s wr, 331 op/s 2024-05-14T09:45:05.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 957619 10.00 MB/sec execute 391 sec latency 20.274 ms 2024-05-14T09:45:06.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 959674 10.01 MB/sec execute 392 sec latency 12.485 ms 2024-05-14T09:45:06.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:06 smithi002 ceph-mon[31557]: pgmap v729: 129 pgs: 129 active+clean; 152 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 2.0 MiB/s wr, 276 op/s 2024-05-14T09:45:06.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:06 smithi029 ceph-mon[36249]: pgmap v729: 129 pgs: 129 active+clean; 152 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 2.0 MiB/s wr, 276 op/s 2024-05-14T09:45:06.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:06 smithi026 ceph-mon[36056]: pgmap v729: 129 pgs: 129 active+clean; 152 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 2.0 MiB/s wr, 276 op/s 2024-05-14T09:45:07.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 961676 10.00 MB/sec execute 393 sec latency 17.573 ms 2024-05-14T09:45:08.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 963259 10.01 MB/sec execute 394 sec latency 50.579 ms 2024-05-14T09:45:08.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:08 smithi002 ceph-mon[31557]: pgmap v730: 129 pgs: 129 active+clean; 151 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 2.6 MiB/s wr, 312 op/s 2024-05-14T09:45:08.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:08 smithi029 ceph-mon[36249]: pgmap v730: 129 pgs: 129 active+clean; 151 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 2.6 MiB/s wr, 312 op/s 2024-05-14T09:45:08.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:08 smithi026 ceph-mon[36056]: pgmap v730: 129 pgs: 129 active+clean; 151 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 2.6 MiB/s wr, 312 op/s 2024-05-14T09:45:09.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 965325 10.00 MB/sec execute 395 sec latency 20.424 ms 2024-05-14T09:45:10.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967089 10.00 MB/sec execute 396 sec latency 18.666 ms 2024-05-14T09:45:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:10 smithi002 ceph-mon[31557]: pgmap v731: 129 pgs: 129 active+clean; 153 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.4 MiB/s wr, 315 op/s 2024-05-14T09:45:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:10 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:45:10.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:10 smithi029 ceph-mon[36249]: pgmap v731: 129 pgs: 129 active+clean; 153 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.4 MiB/s wr, 315 op/s 2024-05-14T09:45:10.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:10 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:45:10.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:10 smithi026 ceph-mon[36056]: pgmap v731: 129 pgs: 129 active+clean; 153 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.4 MiB/s wr, 315 op/s 2024-05-14T09:45:10.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:10 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:45:11.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967098 9.98 MB/sec execute 397 sec latency 992.139 ms 2024-05-14T09:45:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.357+0000 7fb5eda43640 1 -- 172.21.15.2:0/1543653782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5e8104740 msgr2=0x7fb5e8106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.357+0000 7fb5eda43640 1 --2- 172.21.15.2:0/1543653782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5e8104740 0x7fb5e8106b60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009900 tx=0x7fb5d802a410 comp rx=0 tx=0).stop 2024-05-14T09:45:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.357+0000 7fb5eda43640 1 -- 172.21.15.2:0/1543653782 shutdown_connections 2024-05-14T09:45:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.357+0000 7fb5eda43640 1 --2- 172.21.15.2:0/1543653782 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5e8109a00 0x7fb5e810be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.357+0000 7fb5eda43640 1 --2- 172.21.15.2:0/1543653782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5e81070a0 0x7fb5e81094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.357+0000 7fb5eda43640 1 --2- 172.21.15.2:0/1543653782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5e8104740 0x7fb5e8106b60 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.357+0000 7fb5eda43640 1 -- 172.21.15.2:0/1543653782 >> 172.21.15.2:0/1543653782 conn(0x7fb5e80fdd60 msgr2=0x7fb5e81001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.357+0000 7fb5eda43640 1 -- 172.21.15.2:0/1543653782 shutdown_connections 2024-05-14T09:45:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.357+0000 7fb5eda43640 1 -- 172.21.15.2:0/1543653782 wait complete. 2024-05-14T09:45:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.358+0000 7fb5eda43640 1 Processor -- start 2024-05-14T09:45:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.358+0000 7fb5eda43640 1 -- start start 2024-05-14T09:45:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.358+0000 7fb5eda43640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5e8104740 0x7fb5e819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.358+0000 7fb5eda43640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5e81070a0 0x7fb5e819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.358+0000 7fb5eda43640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5e8109a00 0x7fb5e81a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.358+0000 7fb5eda43640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e810f750 con 0x7fb5e8104740 2024-05-14T09:45:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.358+0000 7fb5eda43640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e810f5d0 con 0x7fb5e8109a00 2024-05-14T09:45:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.358+0000 7fb5eda43640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e810f8d0 con 0x7fb5e81070a0 2024-05-14T09:45:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.358+0000 7fb5e67fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5e81070a0 0x7fb5e819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5e77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5e8109a00 0x7fb5e81a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5e67fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5e81070a0 0x7fb5e819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:53280/0 (socket says 172.21.15.2:53280) 2024-05-14T09:45:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5e6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5e8104740 0x7fb5e819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5e67fc640 1 -- 172.21.15.2:0/1036853209 learned_addr learned my addr 172.21.15.2:0/1036853209 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5e67fc640 1 -- 172.21.15.2:0/1036853209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5e8109a00 msgr2=0x7fb5e81a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5e67fc640 1 --2- 172.21.15.2:0/1036853209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5e8109a00 0x7fb5e81a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5e67fc640 1 -- 172.21.15.2:0/1036853209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5e8104740 msgr2=0x7fb5e819a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5e67fc640 1 --2- 172.21.15.2:0/1036853209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5e8104740 0x7fb5e819a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5e67fc640 1 -- 172.21.15.2:0/1036853209 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e81a1de0 con 0x7fb5e81070a0 2024-05-14T09:45:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5e6ffd640 1 --2- 172.21.15.2:0/1036853209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5e8104740 0x7fb5e819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:45:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5e67fc640 1 --2- 172.21.15.2:0/1036853209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5e81070a0 0x7fb5e819a7e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00e9f0 tx=0x7fb5dc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5eca41640 1 -- 172.21.15.2:0/1036853209 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00cd80 con 0x7fb5e81070a0 2024-05-14T09:45:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5eda43640 1 -- 172.21.15.2:0/1036853209 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e81a20d0 con 0x7fb5e81070a0 2024-05-14T09:45:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5eca41640 1 -- 172.21.15.2:0/1036853209 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5dc0108e0 con 0x7fb5e81070a0 2024-05-14T09:45:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5eca41640 1 -- 172.21.15.2:0/1036853209 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc018980 con 0x7fb5e81070a0 2024-05-14T09:45:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.359+0000 7fb5eda43640 1 -- 172.21.15.2:0/1036853209 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e81a2620 con 0x7fb5e81070a0 2024-05-14T09:45:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.362+0000 7fb5f49cb740 1 Processor -- start 2024-05-14T09:45:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5f49cb740 1 -- start start 2024-05-14T09:45:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5f49cb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a44011b0 0x5620a47a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5f49cb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43d82f0 0x5620a47a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5f49cb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43e04d0 0x5620a47a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5f49cb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5620a47075d0 con 0x5620a44011b0 2024-05-14T09:45:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5f49cb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5620a46c0890 con 0x5620a43d82f0 2024-05-14T09:45:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5f49cb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5620a4703590 con 0x5620a43e04d0 2024-05-14T09:45:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5e77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a44011b0 0x5620a47a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5e77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a44011b0 0x5620a47a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49018/0 (socket says 172.21.15.2:49018) 2024-05-14T09:45:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5e77fe640 1 -- 172.21.15.2:0/4001911156 learned_addr learned my addr 172.21.15.2:0/4001911156 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5e6ffd640 1 --2- 172.21.15.2:0/4001911156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43d82f0 0x5620a47a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5e77fe640 1 -- 172.21.15.2:0/4001911156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43e04d0 msgr2=0x5620a47a9ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:45:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5e77fe640 1 --2- 172.21.15.2:0/4001911156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43e04d0 0x5620a47a9ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5eca41640 1 -- 172.21.15.2:0/1036853209 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fb5dc010430 con 0x7fb5e81070a0 2024-05-14T09:45:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5e77fe640 1 -- 172.21.15.2:0/4001911156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43d82f0 msgr2=0x5620a47a2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5e77fe640 1 --2- 172.21.15.2:0/4001911156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43d82f0 0x5620a47a2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.363+0000 7fb5e77fe640 1 -- 172.21.15.2:0/4001911156 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a4419210 con 0x5620a44011b0 2024-05-14T09:45:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.364+0000 7fb5e77fe640 1 --2- 172.21.15.2:0/4001911156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a44011b0 0x5620a47a2a60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009f90 tx=0x7fb5e00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcaded2e0adc7a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.364+0000 7fb5cdffb640 1 -- 172.21.15.2:0/4001911156 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0004d20 con 0x5620a44011b0 2024-05-14T09:45:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.364+0000 7fb5cdffb640 1 -- 172.21.15.2:0/4001911156 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e0004050 con 0x5620a44011b0 2024-05-14T09:45:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.364+0000 7fb5cdffb640 1 -- 172.21.15.2:0/4001911156 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0005260 con 0x5620a44011b0 2024-05-14T09:45:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.364+0000 7fb5eca41640 1 --2- 172.21.15.2:0/1036853209 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb5d00b28a0 0x7fb5d00b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.364+0000 7fb5e6ffd640 1 --2- 172.21.15.2:0/1036853209 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb5d00b28a0 0x7fb5d00b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.364+0000 7fb5eca41640 1 -- 172.21.15.2:0/1036853209 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fb5dc0e2950 con 0x7fb5e81070a0 2024-05-14T09:45:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.365+0000 7fb5f49cb740 1 -- 172.21.15.2:0/4001911156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a44011b0 msgr2=0x5620a47a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.365+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/4001911156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a44011b0 0x5620a47a2a60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009f90 tx=0x7fb5e00118d0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.365+0000 7fb5f49cb740 1 -- 172.21.15.2:0/4001911156 shutdown_connections 2024-05-14T09:45:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.365+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/4001911156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43e04d0 0x5620a47a9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.365+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/4001911156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43d82f0 0x5620a47a2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.365+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/4001911156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a44011b0 0x5620a47a2a60 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.365+0000 7fb5f49cb740 1 -- 172.21.15.2:0/4001911156 >> 172.21.15.2:0/4001911156 conn(0x5620a43f5300 msgr2=0x5620a479e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.365+0000 7fb5f49cb740 1 -- 172.21.15.2:0/4001911156 shutdown_connections 2024-05-14T09:45:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.365+0000 7fb5f49cb740 1 -- 172.21.15.2:0/4001911156 wait complete. 2024-05-14T09:45:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.365+0000 7fb5e6ffd640 1 --2- 172.21.15.2:0/1036853209 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb5d00b28a0 0x7fb5d00b4db0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009230 tx=0x7fb5d8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.365+0000 7fb5f49cb740 1 -- start start 2024-05-14T09:45:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.365+0000 7fb5f49cb740 1 Processor -- start 2024-05-14T09:45:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5f49cb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43d82f0 0x5620a47ac700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5f49cb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43e04d0 0x5620a47acc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5f49cb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a44011b0 0x5620a47b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5f49cb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5620a47aa930 con 0x5620a43e04d0 2024-05-14T09:45:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5f49cb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5620a46c4370 con 0x5620a44011b0 2024-05-14T09:45:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5f49cb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5620a47aaab0 con 0x5620a43d82f0 2024-05-14T09:45:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5f49cb740 10 client.0 _ll_register_callbacks cb 0x5620a4418a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:45:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5e67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43e04d0 0x5620a47acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5e67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43e04d0 0x5620a47acc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49028/0 (socket says 172.21.15.2:49028) 2024-05-14T09:45:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5e67fc640 1 -- 172.21.15.2:0/580490713 learned_addr learned my addr 172.21.15.2:0/580490713 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5e77fe640 1 --2- 172.21.15.2:0/580490713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a44011b0 0x5620a47b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5e6ffd640 1 --2- 172.21.15.2:0/580490713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43d82f0 0x5620a47ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5e67fc640 1 -- 172.21.15.2:0/580490713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43d82f0 msgr2=0x5620a47ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5e67fc640 1 --2- 172.21.15.2:0/580490713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43d82f0 0x5620a47ac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5e67fc640 1 -- 172.21.15.2:0/580490713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a44011b0 msgr2=0x5620a47b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5e67fc640 1 --2- 172.21.15.2:0/580490713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a44011b0 0x5620a47b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5e67fc640 1 -- 172.21.15.2:0/580490713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620a44e5620 con 0x5620a43e04d0 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5e77fe640 1 --2- 172.21.15.2:0/580490713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a44011b0 0x5620a47b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5e67fc640 1 --2- 172.21.15.2:0/580490713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43e04d0 0x5620a47acc40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0a9900 tx=0x7fb5dc0b22f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4099822939fe744 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.366+0000 7fb5c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.367+0000 7fb5c6ffd640 1 -- 172.21.15.2:0/580490713 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc091070 con 0x5620a43e04d0 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.367+0000 7fb5c6ffd640 1 -- 172.21.15.2:0/580490713 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fb5dc0a61f0 con 0x5620a43e04d0 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.367+0000 7fb5f49cb740 1 -- 172.21.15.2:0/580490713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5e0004050 con 0x5620a43e04d0 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.367+0000 7fb5c6ffd640 1 -- 172.21.15.2:0/580490713 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb5dc0a9c00 con 0x5620a43e04d0 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.367+0000 7fb5f49cb740 10 client.16572 fetch_fsmap learned FSMap version 43 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.367+0000 7fb5f49cb740 1 -- 172.21.15.2:0/580490713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5620a47b6b40 con 0x5620a43e04d0 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.367+0000 7fb5c6ffd640 1 -- 172.21.15.2:0/580490713 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e0004050 con 0x5620a43e04d0 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.367+0000 7fb5c6ffd640 1 -- 172.21.15.2:0/580490713 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fb5dc0a5570 con 0x5620a43e04d0 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.367+0000 7fb5f49cb740 10 client.16572 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.367+0000 7fb5f49cb740 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.367+0000 7fb5f49cb740 20 client.16572 populate_metadata read hostname 'smithi002' 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.367+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/580490713 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5620a43bfd80 0x5620a47ba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.367+0000 7fb5f49cb740 4 client.16572 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:45:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.367+0000 7fb5f49cb740 1 -- 172.21.15.2:0/580490713 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620a47151e0 con 0x5620a43bfd80 2024-05-14T09:45:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.368+0000 7fb5e6ffd640 1 --2- 172.21.15.2:0/580490713 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5620a43bfd80 0x5620a47ba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.368+0000 7fb5c6ffd640 10 client.16572 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.368+0000 7fb5e6ffd640 1 --2- 172.21.15.2:0/580490713 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5620a43bfd80 0x5620a47ba180 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d38eb80b2f88cec server_cookie=87b99454328e330b in_seq=0 out_seq=0 2024-05-14T09:45:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.369+0000 7fb5c6ffd640 1 -- 172.21.15.2:0/580490713 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5620a47151e0 con 0x5620a43bfd80 2024-05-14T09:45:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.369+0000 7fb5c6ffd640 10 client.16572 handle_command_reply: tid=0 2024-05-14T09:45:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.369+0000 7fb5f49cb740 1 client.16572 shutdown 2024-05-14T09:45:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.369+0000 7fb5f49cb740 1 -- 172.21.15.2:0/580490713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43e04d0 msgr2=0x5620a47acc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.369+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/580490713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43e04d0 0x5620a47acc40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0a9900 tx=0x7fb5dc0b22f0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.369+0000 7fb5f49cb740 1 -- 172.21.15.2:0/580490713 shutdown_connections 2024-05-14T09:45:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.369+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/580490713 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5620a43bfd80 0x5620a47ba180 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.369+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/580490713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a44011b0 0x5620a47b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.369+0000 7fb5c6ffd640 0 client.16572 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.369+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/580490713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43e04d0 0x5620a47acc40 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.369+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/580490713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43d82f0 0x5620a47ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.370+0000 7fb5f49cb740 1 -- 172.21.15.2:0/580490713 >> 172.21.15.2:0/580490713 conn(0x5620a43f5300 msgr2=0x5620a479d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.370+0000 7fb5f49cb740 1 -- 172.21.15.2:0/580490713 shutdown_connections 2024-05-14T09:45:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.370+0000 7fb5f49cb740 1 -- 172.21.15.2:0/580490713 wait complete. 2024-05-14T09:45:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.370+0000 7fb5f49cb740 20 client.16572 trim_cache size 0 max 16384 2024-05-14T09:45:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.385+0000 7fb5f49cb740 1 Processor -- start 2024-05-14T09:45:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.385+0000 7fb5f49cb740 1 -- start start 2024-05-14T09:45:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.385+0000 7fb5f49cb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43bfd80 0x5620a479dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.385+0000 7fb5f49cb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43d82f0 0x5620a47b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.385+0000 7fb5f49cb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43e04d0 0x5620a479e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.385+0000 7fb5f49cb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81af550 con 0x5620a43d82f0 2024-05-14T09:45:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.385+0000 7fb5f49cb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81acdc0 con 0x5620a43e04d0 2024-05-14T09:45:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.385+0000 7fb5f49cb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81aac10 con 0x5620a43bfd80 2024-05-14T09:45:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.385+0000 7fb5e67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43d82f0 0x5620a47b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.385+0000 7fb5e67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43d82f0 0x5620a47b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49036/0 (socket says 172.21.15.2:49036) 2024-05-14T09:45:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.385+0000 7fb5e67fc640 1 -- 172.21.15.2:0/2063245596 learned_addr learned my addr 172.21.15.2:0/2063245596 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.386+0000 7fb5e6ffd640 1 --2- 172.21.15.2:0/2063245596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43bfd80 0x5620a479dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.386+0000 7fb5e67fc640 1 -- 172.21.15.2:0/2063245596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43bfd80 msgr2=0x5620a479dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.386+0000 7fb5e77fe640 1 --2- 172.21.15.2:0/2063245596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43e04d0 0x5620a479e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.386+0000 7fb5e67fc640 1 --2- 172.21.15.2:0/2063245596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43bfd80 0x5620a479dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.386+0000 7fb5e67fc640 1 -- 172.21.15.2:0/2063245596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43e04d0 msgr2=0x5620a479e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.386+0000 7fb5e67fc640 1 --2- 172.21.15.2:0/2063245596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43e04d0 0x5620a479e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.386+0000 7fb5e67fc640 1 -- 172.21.15.2:0/2063245596 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a479ed00 con 0x5620a43d82f0 2024-05-14T09:45:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.386+0000 7fb5e67fc640 1 --2- 172.21.15.2:0/2063245596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43d82f0 0x5620a47b3ac0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00c6a0 tx=0x7fb5dc0a9790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c0f9f5a7b2e738 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.386+0000 7fb5cd7fa640 1 -- 172.21.15.2:0/2063245596 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0c8070 con 0x5620a43d82f0 2024-05-14T09:45:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.386+0000 7fb5cd7fa640 1 -- 172.21.15.2:0/2063245596 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5dc0adf00 con 0x5620a43d82f0 2024-05-14T09:45:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.386+0000 7fb5cd7fa640 1 -- 172.21.15.2:0/2063245596 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc091070 con 0x5620a43d82f0 2024-05-14T09:45:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.387+0000 7fb5f49cb740 1 -- 172.21.15.2:0/2063245596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43d82f0 msgr2=0x5620a47b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.387+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/2063245596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43d82f0 0x5620a47b3ac0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00c6a0 tx=0x7fb5dc0a9790 comp rx=0 tx=0).stop 2024-05-14T09:45:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.387+0000 7fb5f49cb740 1 -- 172.21.15.2:0/2063245596 shutdown_connections 2024-05-14T09:45:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.387+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/2063245596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43e04d0 0x5620a479e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.387+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/2063245596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43d82f0 0x5620a47b3ac0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.387+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/2063245596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43bfd80 0x5620a479dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.387+0000 7fb5f49cb740 1 -- 172.21.15.2:0/2063245596 >> 172.21.15.2:0/2063245596 conn(0x5620a43f5300 msgr2=0x5620a47a6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.387+0000 7fb5f49cb740 1 -- 172.21.15.2:0/2063245596 shutdown_connections 2024-05-14T09:45:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.387+0000 7fb5f49cb740 1 -- 172.21.15.2:0/2063245596 wait complete. 2024-05-14T09:45:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.387+0000 7fb5f49cb740 1 -- start start 2024-05-14T09:45:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5f49cb740 1 Processor -- start 2024-05-14T09:45:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5f49cb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43bfd80 0x5620a47af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5f49cb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43d82f0 0x5620a47af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5f49cb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43e04d0 0x5620a47b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5f49cb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81ab280 con 0x5620a43d82f0 2024-05-14T09:45:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5f49cb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81ac6f0 con 0x5620a43bfd80 2024-05-14T09:45:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5f49cb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81ab950 con 0x5620a43e04d0 2024-05-14T09:45:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5f49cb740 10 client.0 _ll_register_callbacks cb 0x7fb5dc009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:45:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5e67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43d82f0 0x5620a47af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5e67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43d82f0 0x5620a47af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49038/0 (socket says 172.21.15.2:49038) 2024-05-14T09:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5e67fc640 1 -- 172.21.15.2:0/3291266651 learned_addr learned my addr 172.21.15.2:0/3291266651 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5e77fe640 1 --2- 172.21.15.2:0/3291266651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43e04d0 0x5620a47b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5e6ffd640 1 --2- 172.21.15.2:0/3291266651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43bfd80 0x5620a47af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5e67fc640 1 -- 172.21.15.2:0/3291266651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43e04d0 msgr2=0x5620a47b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5e67fc640 1 --2- 172.21.15.2:0/3291266651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43e04d0 0x5620a47b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5e67fc640 1 -- 172.21.15.2:0/3291266651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43bfd80 msgr2=0x5620a47af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5e67fc640 1 --2- 172.21.15.2:0/3291266651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43bfd80 0x5620a47af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5e67fc640 1 -- 172.21.15.2:0/3291266651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620a47b29f0 con 0x5620a43d82f0 2024-05-14T09:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.388+0000 7fb5e6ffd640 1 --2- 172.21.15.2:0/3291266651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43bfd80 0x5620a47af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5e67fc640 1 --2- 172.21.15.2:0/3291266651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43d82f0 0x5620a47af540 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0a5c80 tx=0x7fb5dc0e00d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c085fba8b6854e2d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5c6ffd640 1 -- 172.21.15.2:0/3291266651 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0c8070 con 0x5620a43d82f0 2024-05-14T09:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5c6ffd640 1 -- 172.21.15.2:0/3291266651 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fb5dc0a61f0 con 0x5620a43d82f0 2024-05-14T09:45:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5f49cb740 1 -- 172.21.15.2:0/3291266651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5dc0adf00 con 0x5620a43d82f0 2024-05-14T09:45:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5c6ffd640 1 -- 172.21.15.2:0/3291266651 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb5dc091070 con 0x5620a43d82f0 2024-05-14T09:45:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5f49cb740 10 client.16578 fetch_fsmap learned FSMap version 43 2024-05-14T09:45:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5f49cb740 1 -- 172.21.15.2:0/3291266651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5620a47b3c80 con 0x5620a43d82f0 2024-05-14T09:45:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5c6ffd640 1 -- 172.21.15.2:0/3291266651 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5dc0adf00 con 0x5620a43d82f0 2024-05-14T09:45:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5c6ffd640 1 -- 172.21.15.2:0/3291266651 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fb5dc0cc0b0 con 0x5620a43d82f0 2024-05-14T09:45:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5f49cb740 10 client.16578 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T09:45:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5f49cb740 10 client.16578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:45:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5f49cb740 20 client.16578 populate_metadata read hostname 'smithi002' 2024-05-14T09:45:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/3291266651 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5620a44011b0 0x5620a4824c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5f49cb740 4 client.16578 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:45:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.389+0000 7fb5f49cb740 1 -- 172.21.15.2:0/3291266651 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620a470f050 con 0x5620a44011b0 2024-05-14T09:45:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.390+0000 7fb5e6ffd640 1 --2- 172.21.15.2:0/3291266651 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5620a44011b0 0x5620a4824c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.390+0000 7fb5c6ffd640 10 client.16578 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.390+0000 7fb5e6ffd640 1 --2- 172.21.15.2:0/3291266651 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5620a44011b0 0x5620a4824c10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e01d935deb02e83 server_cookie=bd13fa01e0b819b7 in_seq=0 out_seq=0 2024-05-14T09:45:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.528+0000 7fb5c6ffd640 1 -- 172.21.15.2:0/3291266651 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5620a470f050 con 0x5620a44011b0 2024-05-14T09:45:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.528+0000 7fb5c6ffd640 10 client.16578 handle_command_reply: tid=0 2024-05-14T09:45:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.528+0000 7fb5f49cb740 1 client.16578 shutdown 2024-05-14T09:45:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.528+0000 7fb5f49cb740 1 -- 172.21.15.2:0/3291266651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43d82f0 msgr2=0x5620a47af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.528+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/3291266651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43d82f0 0x5620a47af540 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0a5c80 tx=0x7fb5dc0e00d0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.528+0000 7fb5f49cb740 1 -- 172.21.15.2:0/3291266651 shutdown_connections 2024-05-14T09:45:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.528+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/3291266651 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5620a44011b0 0x5620a4824c10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.528+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/3291266651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5620a43e04d0 0x5620a47b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.528+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/3291266651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620a43d82f0 0x5620a47af540 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.528+0000 7fb5c6ffd640 0 client.16578 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.528+0000 7fb5f49cb740 1 --2- 172.21.15.2:0/3291266651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620a43bfd80 0x5620a47af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.528+0000 7fb5f49cb740 1 -- 172.21.15.2:0/3291266651 >> 172.21.15.2:0/3291266651 conn(0x5620a43f5300 msgr2=0x5620a479dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.529+0000 7fb5f49cb740 1 -- 172.21.15.2:0/3291266651 shutdown_connections 2024-05-14T09:45:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.529+0000 7fb5f49cb740 1 -- 172.21.15.2:0/3291266651 wait complete. 2024-05-14T09:45:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.529+0000 7fb5f49cb740 20 client.16578 trim_cache size 0 max 16384 2024-05-14T09:45:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:45:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:45:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:45:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:45:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.529+0000 7fb5eda43640 1 -- 172.21.15.2:0/1036853209 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb5d00b28a0 msgr2=0x7fb5d00b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.529+0000 7fb5eda43640 1 --2- 172.21.15.2:0/1036853209 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb5d00b28a0 0x7fb5d00b4db0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009230 tx=0x7fb5d8000bc0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.529+0000 7fb5eda43640 1 -- 172.21.15.2:0/1036853209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5e81070a0 msgr2=0x7fb5e819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.529+0000 7fb5eda43640 1 --2- 172.21.15.2:0/1036853209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5e81070a0 0x7fb5e819a7e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00e9f0 tx=0x7fb5dc00eeb0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.529+0000 7fb5eda43640 1 -- 172.21.15.2:0/1036853209 shutdown_connections 2024-05-14T09:45:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.529+0000 7fb5eda43640 1 --2- 172.21.15.2:0/1036853209 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb5d00b28a0 0x7fb5d00b4db0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.529+0000 7fb5eda43640 1 --2- 172.21.15.2:0/1036853209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5e8109a00 0x7fb5e81a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.529+0000 7fb5eda43640 1 --2- 172.21.15.2:0/1036853209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb5e81070a0 0x7fb5e819a7e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.529+0000 7fb5eda43640 1 --2- 172.21.15.2:0/1036853209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5e8104740 0x7fb5e819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.529+0000 7fb5eda43640 1 -- 172.21.15.2:0/1036853209 >> 172.21.15.2:0/1036853209 conn(0x7fb5e80fdd60 msgr2=0x7fb5e8105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.530+0000 7fb5eda43640 1 -- 172.21.15.2:0/1036853209 shutdown_connections 2024-05-14T09:45:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:11.530+0000 7fb5eda43640 1 -- 172.21.15.2:0/1036853209 wait complete. 2024-05-14T09:45:12.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967098 9.95 MB/sec execute 398 sec latency 1992.222 ms 2024-05-14T09:45:12.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:12 smithi002 ceph-mon[31557]: pgmap v732: 129 pgs: 129 active+clean; 153 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 2.1 MiB/s wr, 268 op/s 2024-05-14T09:45:12.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:12 smithi029 ceph-mon[36249]: pgmap v732: 129 pgs: 129 active+clean; 153 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 2.1 MiB/s wr, 268 op/s 2024-05-14T09:45:12.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:12 smithi026 ceph-mon[36056]: pgmap v732: 129 pgs: 129 active+clean; 153 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 2.1 MiB/s wr, 268 op/s 2024-05-14T09:45:13.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967098 9.93 MB/sec execute 399 sec latency 2992.276 ms 2024-05-14T09:45:14.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967099 9.90 MB/sec execute 400 sec latency 3921.634 ms 2024-05-14T09:45:14.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:14 smithi002 ceph-mon[31557]: pgmap v733: 129 pgs: 129 active+clean; 152 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.8 MiB/s wr, 341 op/s 2024-05-14T09:45:14.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:14 smithi029 ceph-mon[36249]: pgmap v733: 129 pgs: 129 active+clean; 152 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.8 MiB/s wr, 341 op/s 2024-05-14T09:45:14.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:14 smithi026 ceph-mon[36056]: pgmap v733: 129 pgs: 129 active+clean; 152 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.8 MiB/s wr, 341 op/s 2024-05-14T09:45:15.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967099 9.88 MB/sec execute 401 sec latency 1070.800 ms 2024-05-14T09:45:16.277 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967099 9.86 MB/sec execute 402 sec latency 2070.853 ms 2024-05-14T09:45:16.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:16 smithi002 ceph-mon[31557]: pgmap v734: 129 pgs: 129 active+clean; 152 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-05-14T09:45:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:16 smithi029 ceph-mon[36249]: pgmap v734: 129 pgs: 129 active+clean; 152 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-05-14T09:45:16.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:16 smithi026 ceph-mon[36056]: pgmap v734: 129 pgs: 129 active+clean; 152 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-05-14T09:45:17.278 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967099 9.83 MB/sec execute 403 sec latency 3070.936 ms 2024-05-14T09:45:18.278 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967099 9.81 MB/sec execute 404 sec latency 4071.017 ms 2024-05-14T09:45:18.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:18 smithi002 ceph-mon[31557]: pgmap v735: 129 pgs: 129 active+clean; 164 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 3.5 MiB/s wr, 245 op/s 2024-05-14T09:45:18.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:18 smithi029 ceph-mon[36249]: pgmap v735: 129 pgs: 129 active+clean; 164 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 3.5 MiB/s wr, 245 op/s 2024-05-14T09:45:18.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:18 smithi026 ceph-mon[36056]: pgmap v735: 129 pgs: 129 active+clean; 164 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 3.5 MiB/s wr, 245 op/s 2024-05-14T09:45:19.278 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967167 9.78 MB/sec execute 405 sec latency 4996.939 ms 2024-05-14T09:45:20.278 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967167 9.76 MB/sec execute 406 sec latency 1009.333 ms 2024-05-14T09:45:20.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:20 smithi002 ceph-mon[31557]: pgmap v736: 129 pgs: 129 active+clean; 168 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-05-14T09:45:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:20 smithi029 ceph-mon[36249]: pgmap v736: 129 pgs: 129 active+clean; 168 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-05-14T09:45:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:20 smithi026 ceph-mon[36056]: pgmap v736: 129 pgs: 129 active+clean; 168 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-05-14T09:45:21.278 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967167 9.73 MB/sec execute 407 sec latency 2009.454 ms 2024-05-14T09:45:22.278 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967167 9.71 MB/sec execute 408 sec latency 3009.501 ms 2024-05-14T09:45:22.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:22 smithi002 ceph-mon[31557]: pgmap v737: 129 pgs: 129 active+clean; 168 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 2.9 MiB/s wr, 154 op/s 2024-05-14T09:45:22.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:22 smithi029 ceph-mon[36249]: pgmap v737: 129 pgs: 129 active+clean; 168 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 2.9 MiB/s wr, 154 op/s 2024-05-14T09:45:22.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:22 smithi026 ceph-mon[36056]: pgmap v737: 129 pgs: 129 active+clean; 168 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 2.9 MiB/s wr, 154 op/s 2024-05-14T09:45:23.278 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967167 9.69 MB/sec execute 409 sec latency 4009.592 ms 2024-05-14T09:45:24.278 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967263 9.66 MB/sec execute 410 sec latency 4937.382 ms 2024-05-14T09:45:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.641+0000 7f1b9bb5e640 1 -- 172.21.15.2:0/643111863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b94102e90 msgr2=0x7f1b941032a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.641+0000 7f1b9bb5e640 1 --2- 172.21.15.2:0/643111863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b94102e90 0x7f1b941032a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009930 tx=0x7f1b8402a420 comp rx=0 tx=0).stop 2024-05-14T09:45:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.641+0000 7f1b9bb5e640 1 -- 172.21.15.2:0/643111863 shutdown_connections 2024-05-14T09:45:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.641+0000 7f1b9bb5e640 1 --2- 172.21.15.2:0/643111863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b941037e0 0x7f1b94103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.641+0000 7f1b9bb5e640 1 --2- 172.21.15.2:0/643111863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b94102e90 0x7f1b941032a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.642+0000 7f1b9bb5e640 1 --2- 172.21.15.2:0/643111863 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b94102540 0x7f1b94102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.642+0000 7f1b9bb5e640 1 -- 172.21.15.2:0/643111863 >> 172.21.15.2:0/643111863 conn(0x7f1b940fdd50 msgr2=0x7f1b94100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.642+0000 7f1b9bb5e640 1 -- 172.21.15.2:0/643111863 shutdown_connections 2024-05-14T09:45:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.642+0000 7f1b9bb5e640 1 -- 172.21.15.2:0/643111863 wait complete. 2024-05-14T09:45:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.642+0000 7f1b9bb5e640 1 Processor -- start 2024-05-14T09:45:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.642+0000 7f1b9bb5e640 1 -- start start 2024-05-14T09:45:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b9bb5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b94102540 0x7f1b9419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b9bb5e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b94102e90 0x7f1b9419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b9bb5e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b941037e0 0x7f1b941a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b9bb5e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1b94111970 con 0x7f1b94102e90 2024-05-14T09:45:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b9bb5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941117f0 con 0x7f1b94102540 2024-05-14T09:45:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b9bb5e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1b94111af0 con 0x7f1b941037e0 2024-05-14T09:45:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b990f9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b94102e90 0x7f1b9419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b990f9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b94102e90 0x7f1b9419ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42962/0 (socket says 172.21.15.2:42962) 2024-05-14T09:45:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b990f9640 1 -- 172.21.15.2:0/1345814860 learned_addr learned my addr 172.21.15.2:0/1345814860 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b998fa640 1 --2- 172.21.15.2:0/1345814860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b94102540 0x7f1b9419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b9a0fb640 1 --2- 172.21.15.2:0/1345814860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b941037e0 0x7f1b941a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b990f9640 1 -- 172.21.15.2:0/1345814860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b941037e0 msgr2=0x7f1b941a39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b990f9640 1 --2- 172.21.15.2:0/1345814860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b941037e0 0x7f1b941a39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b990f9640 1 -- 172.21.15.2:0/1345814860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b94102540 msgr2=0x7f1b9419c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b990f9640 1 --2- 172.21.15.2:0/1345814860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b94102540 0x7f1b9419c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b990f9640 1 -- 172.21.15.2:0/1345814860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b941a3f80 con 0x7f1b94102e90 2024-05-14T09:45:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b9a0fb640 1 --2- 172.21.15.2:0/1345814860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b941037e0 0x7f1b941a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:45:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b990f9640 1 --2- 172.21.15.2:0/1345814860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b94102e90 0x7f1b9419ca30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009230 tx=0x7f1b84031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b8affd640 1 -- 172.21.15.2:0/1345814860 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8403f030 con 0x7f1b94102e90 2024-05-14T09:45:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.643+0000 7f1b8affd640 1 -- 172.21.15.2:0/1345814860 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b84003aa0 con 0x7f1b94102e90 2024-05-14T09:45:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.644+0000 7f1b9bb5e640 1 -- 172.21.15.2:0/1345814860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b941a4210 con 0x7f1b94102e90 2024-05-14T09:45:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.644+0000 7f1b8affd640 1 -- 172.21.15.2:0/1345814860 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84035070 con 0x7f1b94102e90 2024-05-14T09:45:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.644+0000 7f1b9bb5e640 1 -- 172.21.15.2:0/1345814860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b941a4750 con 0x7f1b94102e90 2024-05-14T09:45:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.647+0000 7f1ba2a0f740 1 Processor -- start 2024-05-14T09:45:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.647+0000 7f1ba2a0f740 1 -- start start 2024-05-14T09:45:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.647+0000 7f1ba2a0f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c3493c1b0 0x556c34cdda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1ba2a0f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c349132f0 0x556c34cddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1ba2a0f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 0x556c34ce4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1ba2a0f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556c34c425d0 con 0x556c3493c1b0 2024-05-14T09:45:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1ba2a0f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556c34bfb890 con 0x556c3491b4d0 2024-05-14T09:45:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1ba2a0f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556c34c3e590 con 0x556c349132f0 2024-05-14T09:45:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1b8affd640 1 -- 172.21.15.2:0/1345814860 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1b84042030 con 0x7f1b94102e90 2024-05-14T09:45:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1b9a0fb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c3493c1b0 0x556c34cdda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1b9a0fb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c3493c1b0 0x556c34cdda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42970/0 (socket says 172.21.15.2:42970) 2024-05-14T09:45:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1b9a0fb640 1 -- 172.21.15.2:0/224609163 learned_addr learned my addr 172.21.15.2:0/224609163 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1b990f9640 1 --2- 172.21.15.2:0/224609163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 0x556c34ce4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1b998fa640 1 --2- 172.21.15.2:0/224609163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c349132f0 0x556c34cddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1b9a0fb640 1 -- 172.21.15.2:0/224609163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c349132f0 msgr2=0x556c34cddfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1b9a0fb640 1 --2- 172.21.15.2:0/224609163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c349132f0 0x556c34cddfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1b9a0fb640 1 -- 172.21.15.2:0/224609163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 msgr2=0x556c34ce4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1b9a0fb640 1 --2- 172.21.15.2:0/224609163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 0x556c34ce4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1b9a0fb640 1 -- 172.21.15.2:0/224609163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c34954210 con 0x556c3493c1b0 2024-05-14T09:45:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1b990f9640 1 --2- 172.21.15.2:0/224609163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 0x556c34ce4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:45:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1b9a0fb640 1 --2- 172.21.15.2:0/224609163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c3493c1b0 0x556c34cdda60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009f60 tx=0x7f1b8c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39516b24bb5da5ca server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.648+0000 7f1b8affd640 1 --2- 172.21.15.2:0/1345814860 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1b800b2920 0x7f1b800b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.649+0000 7f1b7ffff640 1 -- 172.21.15.2:0/224609163 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c016070 con 0x556c3493c1b0 2024-05-14T09:45:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.649+0000 7f1b7ffff640 1 -- 172.21.15.2:0/224609163 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b8c01bb10 con 0x556c3493c1b0 2024-05-14T09:45:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.649+0000 7f1b7ffff640 1 -- 172.21.15.2:0/224609163 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c012bc0 con 0x556c3493c1b0 2024-05-14T09:45:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.649+0000 7f1b998fa640 1 --2- 172.21.15.2:0/1345814860 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1b800b2920 0x7f1b800b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.649+0000 7f1b8affd640 1 -- 172.21.15.2:0/1345814860 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f1b84102c00 con 0x7f1b94102e90 2024-05-14T09:45:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.650+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/224609163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c3493c1b0 msgr2=0x556c34cdda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.650+0000 7f1b998fa640 1 --2- 172.21.15.2:0/1345814860 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1b800b2920 0x7f1b800b4e30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x556c34cdee40 tx=0x7f1b9000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.650+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/224609163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c3493c1b0 0x556c34cdda60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009f60 tx=0x7f1b8c009f90 comp rx=0 tx=0).stop 2024-05-14T09:45:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.650+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/224609163 shutdown_connections 2024-05-14T09:45:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.650+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/224609163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 0x556c34ce4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.650+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/224609163 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c349132f0 0x556c34cddfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.650+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/224609163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c3493c1b0 0x556c34cdda60 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.650+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/224609163 >> 172.21.15.2:0/224609163 conn(0x556c34930300 msgr2=0x556c34cd93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.650+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/224609163 shutdown_connections 2024-05-14T09:45:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.650+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/224609163 wait complete. 2024-05-14T09:45:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.650+0000 7f1ba2a0f740 1 -- start start 2024-05-14T09:45:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.650+0000 7f1ba2a0f740 1 Processor -- start 2024-05-14T09:45:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1ba2a0f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c349132f0 0x556c34ce7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1ba2a0f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 0x556c34ce7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1ba2a0f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c3493c1b0 0x556c34ceec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1ba2a0f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556c34ce5930 con 0x556c349132f0 2024-05-14T09:45:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1ba2a0f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556c34bff370 con 0x556c3491b4d0 2024-05-14T09:45:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1ba2a0f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556c34ce5ab0 con 0x556c3493c1b0 2024-05-14T09:45:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1ba2a0f740 10 client.0 _ll_register_callbacks cb 0x556c34953a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:45:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1b998fa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c349132f0 0x556c34ce7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1b998fa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c349132f0 0x556c34ce7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42984/0 (socket says 172.21.15.2:42984) 2024-05-14T09:45:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1b998fa640 1 -- 172.21.15.2:0/951205998 learned_addr learned my addr 172.21.15.2:0/951205998 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1b9a0fb640 1 --2- 172.21.15.2:0/951205998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c3493c1b0 0x556c34ceec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1b990f9640 1 --2- 172.21.15.2:0/951205998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 0x556c34ce7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1b998fa640 1 -- 172.21.15.2:0/951205998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c3493c1b0 msgr2=0x556c34ceec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1b998fa640 1 --2- 172.21.15.2:0/951205998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c3493c1b0 0x556c34ceec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1b998fa640 1 -- 172.21.15.2:0/951205998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 msgr2=0x556c34ce7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1b998fa640 1 --2- 172.21.15.2:0/951205998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 0x556c34ce7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.651+0000 7f1b998fa640 1 -- 172.21.15.2:0/951205998 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c34a20620 con 0x556c349132f0 2024-05-14T09:45:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.652+0000 7f1b998fa640 1 --2- 172.21.15.2:0/951205998 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c349132f0 0x556c34ce7760 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1b90010e40 tx=0x7f1b900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bb79e8f997330f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.652+0000 7f1b7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:45:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.652+0000 7f1b7d7fa640 1 -- 172.21.15.2:0/951205998 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90027400 con 0x556c349132f0 2024-05-14T09:45:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.652+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/951205998 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b8c01bb10 con 0x556c349132f0 2024-05-14T09:45:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.652+0000 7f1b7d7fa640 1 -- 172.21.15.2:0/951205998 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f1b900275a0 con 0x556c349132f0 2024-05-14T09:45:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.652+0000 7f1b7d7fa640 1 -- 172.21.15.2:0/951205998 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1b90030050 con 0x556c349132f0 2024-05-14T09:45:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.652+0000 7f1ba2a0f740 10 client.16590 fetch_fsmap learned FSMap version 44 2024-05-14T09:45:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.652+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/951205998 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x556c34cf1c50 con 0x556c349132f0 2024-05-14T09:45:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.652+0000 7f1b7d7fa640 1 -- 172.21.15.2:0/951205998 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b8c01bb10 con 0x556c349132f0 2024-05-14T09:45:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.653+0000 7f1b7d7fa640 1 -- 172.21.15.2:0/951205998 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f1b9001f650 con 0x556c349132f0 2024-05-14T09:45:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.653+0000 7f1ba2a0f740 10 client.16590 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T09:45:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.653+0000 7f1ba2a0f740 10 client.16590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:45:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.653+0000 7f1ba2a0f740 20 client.16590 populate_metadata read hostname 'smithi002' 2024-05-14T09:45:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.653+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/951205998 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556c348fad80 0x556c34cf5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.653+0000 7f1ba2a0f740 4 client.16590 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:45:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.653+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/951205998 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c34c501e0 con 0x556c348fad80 2024-05-14T09:45:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.653+0000 7f1b990f9640 1 --2- 172.21.15.2:0/951205998 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556c348fad80 0x556c34cf5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.653+0000 7f1b7d7fa640 10 client.16590 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.653+0000 7f1b990f9640 1 --2- 172.21.15.2:0/951205998 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556c348fad80 0x556c34cf5360 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb9d93945e157aa8 server_cookie=3b13447cef9e8307 in_seq=0 out_seq=0 2024-05-14T09:45:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.654+0000 7f1b7d7fa640 1 -- 172.21.15.2:0/951205998 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c34c501e0 con 0x556c348fad80 2024-05-14T09:45:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.654+0000 7f1b7d7fa640 10 client.16590 handle_command_reply: tid=0 2024-05-14T09:45:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.654+0000 7f1ba2a0f740 1 client.16590 shutdown 2024-05-14T09:45:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.654+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/951205998 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c349132f0 msgr2=0x556c34ce7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.654+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/951205998 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c349132f0 0x556c34ce7760 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1b90010e40 tx=0x7f1b900046c0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.654+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/951205998 shutdown_connections 2024-05-14T09:45:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.654+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/951205998 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556c348fad80 0x556c34cf5360 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.654+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/951205998 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c3493c1b0 0x556c34ceec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.654+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/951205998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 0x556c34ce7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.654+0000 7f1b7d7fa640 0 client.16590 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.654+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/951205998 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c349132f0 0x556c34ce7760 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.654+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/951205998 >> 172.21.15.2:0/951205998 conn(0x556c34930300 msgr2=0x556c34cdc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.654+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/951205998 shutdown_connections 2024-05-14T09:45:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.654+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/951205998 wait complete. 2024-05-14T09:45:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.655+0000 7f1ba2a0f740 20 client.16590 trim_cache size 0 max 16384 2024-05-14T09:45:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.670+0000 7f1ba2a0f740 1 Processor -- start 2024-05-14T09:45:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.670+0000 7f1ba2a0f740 1 -- start start 2024-05-14T09:45:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.670+0000 7f1ba2a0f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c349132f0 0x556c34cd9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.670+0000 7f1ba2a0f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c3491b4d0 0x556c34ceeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.670+0000 7f1ba2a0f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c3493c1b0 0x556c34cef0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.670+0000 7f1ba2a0f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941b1770 con 0x556c3493c1b0 2024-05-14T09:45:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.670+0000 7f1ba2a0f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941aefb0 con 0x556c349132f0 2024-05-14T09:45:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.670+0000 7f1ba2a0f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941ace00 con 0x556c3491b4d0 2024-05-14T09:45:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.670+0000 7f1b9a0fb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c3493c1b0 0x556c34cef0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.670+0000 7f1b9a0fb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c3493c1b0 0x556c34cef0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43000/0 (socket says 172.21.15.2:43000) 2024-05-14T09:45:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.670+0000 7f1b9a0fb640 1 -- 172.21.15.2:0/2702733367 learned_addr learned my addr 172.21.15.2:0/2702733367 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.670+0000 7f1b990f9640 1 --2- 172.21.15.2:0/2702733367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c3491b4d0 0x556c34ceeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.670+0000 7f1b998fa640 1 --2- 172.21.15.2:0/2702733367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c349132f0 0x556c34cd9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.671+0000 7f1b9a0fb640 1 -- 172.21.15.2:0/2702733367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c3491b4d0 msgr2=0x556c34ceeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.671+0000 7f1b9a0fb640 1 --2- 172.21.15.2:0/2702733367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c3491b4d0 0x556c34ceeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.671+0000 7f1b9a0fb640 1 -- 172.21.15.2:0/2702733367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c349132f0 msgr2=0x556c34cd9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.671+0000 7f1b9a0fb640 1 --2- 172.21.15.2:0/2702733367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c349132f0 0x556c34cd9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.671+0000 7f1b9a0fb640 1 -- 172.21.15.2:0/2702733367 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c34cd9d00 con 0x556c3493c1b0 2024-05-14T09:45:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.671+0000 7f1b9a0fb640 1 --2- 172.21.15.2:0/2702733367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c3493c1b0 0x556c34cef0d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c010c90 tx=0x7f1b8c009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313577e695d051e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.671+0000 7f1b7f7fe640 1 -- 172.21.15.2:0/2702733367 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c016050 con 0x556c3493c1b0 2024-05-14T09:45:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.671+0000 7f1b7f7fe640 1 -- 172.21.15.2:0/2702733367 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b8c029070 con 0x556c3493c1b0 2024-05-14T09:45:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.671+0000 7f1b7f7fe640 1 -- 172.21.15.2:0/2702733367 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c01b6c0 con 0x556c3493c1b0 2024-05-14T09:45:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.672+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/2702733367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c3493c1b0 msgr2=0x556c34cef0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.672+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/2702733367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c3493c1b0 0x556c34cef0d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c010c90 tx=0x7f1b8c009b70 comp rx=0 tx=0).stop 2024-05-14T09:45:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.672+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/2702733367 shutdown_connections 2024-05-14T09:45:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.672+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/2702733367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c3493c1b0 0x556c34cef0d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.672+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/2702733367 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c3491b4d0 0x556c34ceeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.672+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/2702733367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c349132f0 0x556c34cd9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.672+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/2702733367 >> 172.21.15.2:0/2702733367 conn(0x556c34930300 msgr2=0x556c34cd8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.672+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/2702733367 shutdown_connections 2024-05-14T09:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.672+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/2702733367 wait complete. 2024-05-14T09:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.672+0000 7f1ba2a0f740 1 -- start start 2024-05-14T09:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.672+0000 7f1ba2a0f740 1 Processor -- start 2024-05-14T09:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1ba2a0f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c349132f0 0x556c34cec6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1ba2a0f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 0x556c34cecbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1ba2a0f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c3493c1b0 0x556c34cedab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1ba2a0f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941ad470 con 0x556c349132f0 2024-05-14T09:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1ba2a0f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941ae8e0 con 0x556c3491b4d0 2024-05-14T09:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1ba2a0f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941adb40 con 0x556c3493c1b0 2024-05-14T09:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1ba2a0f740 10 client.0 _ll_register_callbacks cb 0x7f1b90010ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1b998fa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c349132f0 0x556c34cec6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1b998fa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c349132f0 0x556c34cec6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43012/0 (socket says 172.21.15.2:43012) 2024-05-14T09:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1b998fa640 1 -- 172.21.15.2:0/146225053 learned_addr learned my addr 172.21.15.2:0/146225053 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1b9a0fb640 1 --2- 172.21.15.2:0/146225053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c3493c1b0 0x556c34cedab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1b990f9640 1 --2- 172.21.15.2:0/146225053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 0x556c34cecbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1b998fa640 1 -- 172.21.15.2:0/146225053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c3493c1b0 msgr2=0x556c34cedab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1b998fa640 1 --2- 172.21.15.2:0/146225053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c3493c1b0 0x556c34cedab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1b998fa640 1 -- 172.21.15.2:0/146225053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 msgr2=0x556c34cecbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1b998fa640 1 --2- 172.21.15.2:0/146225053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 0x556c34cecbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1b998fa640 1 -- 172.21.15.2:0/146225053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c34cee070 con 0x556c349132f0 2024-05-14T09:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1b998fa640 1 --2- 172.21.15.2:0/146225053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c349132f0 0x556c34cec6b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000b010 tx=0x7f1b9000fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78dbec767994c1f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.673+0000 7f1b7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.674+0000 7f1b7d7fa640 1 -- 172.21.15.2:0/146225053 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9002d040 con 0x556c349132f0 2024-05-14T09:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.674+0000 7f1b7d7fa640 1 -- 172.21.15.2:0/146225053 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f1b90034400 con 0x556c349132f0 2024-05-14T09:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.674+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/146225053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b8c029070 con 0x556c349132f0 2024-05-14T09:45:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.674+0000 7f1b7d7fa640 1 -- 172.21.15.2:0/146225053 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1b90012d00 con 0x556c349132f0 2024-05-14T09:45:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.674+0000 7f1ba2a0f740 10 client.16596 fetch_fsmap learned FSMap version 44 2024-05-14T09:45:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.674+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/146225053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x556c34cef370 con 0x556c349132f0 2024-05-14T09:45:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.674+0000 7f1b7d7fa640 1 -- 172.21.15.2:0/146225053 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b8c029070 con 0x556c349132f0 2024-05-14T09:45:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.674+0000 7f1b7d7fa640 1 -- 172.21.15.2:0/146225053 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f1b9003c070 con 0x556c349132f0 2024-05-14T09:45:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.674+0000 7f1ba2a0f740 10 client.16596 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T09:45:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.674+0000 7f1ba2a0f740 10 client.16596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:45:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.674+0000 7f1ba2a0f740 20 client.16596 populate_metadata read hostname 'smithi002' 2024-05-14T09:45:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.674+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/146225053 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556c348d01c0 0x556c34d5fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.674+0000 7f1ba2a0f740 4 client.16596 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:45:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.674+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/146225053 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c34c4a050 con 0x556c348d01c0 2024-05-14T09:45:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.674+0000 7f1b990f9640 1 --2- 172.21.15.2:0/146225053 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556c348d01c0 0x556c34d5fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.675+0000 7f1b990f9640 1 --2- 172.21.15.2:0/146225053 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556c348d01c0 0x556c34d5fdf0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=348ab6e52bb0c663 server_cookie=a80eef79b8df1f32 in_seq=0 out_seq=0 2024-05-14T09:45:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.675+0000 7f1b7d7fa640 10 client.16596 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:24.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:24 smithi029 ceph-mon[36249]: pgmap v738: 129 pgs: 129 active+clean; 168 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 3.3 MiB/s wr, 170 op/s 2024-05-14T09:45:24.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:24 smithi026 ceph-mon[36056]: pgmap v738: 129 pgs: 129 active+clean; 168 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 3.3 MiB/s wr, 170 op/s 2024-05-14T09:45:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.713+0000 7f1b7d7fa640 1 -- 172.21.15.2:0/146225053 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c34c4a050 con 0x556c348d01c0 2024-05-14T09:45:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.713+0000 7f1b7d7fa640 10 client.16596 handle_command_reply: tid=0 2024-05-14T09:45:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.713+0000 7f1ba2a0f740 1 client.16596 shutdown 2024-05-14T09:45:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.713+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/146225053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c349132f0 msgr2=0x556c34cec6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.713+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/146225053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c349132f0 0x556c34cec6b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000b010 tx=0x7f1b9000fd90 comp rx=0 tx=0).stop 2024-05-14T09:45:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.713+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/146225053 shutdown_connections 2024-05-14T09:45:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.713+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/146225053 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556c348d01c0 0x556c34d5fdf0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/146225053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556c3493c1b0 0x556c34cedab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/146225053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556c3491b4d0 0x556c34cecbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1ba2a0f740 1 --2- 172.21.15.2:0/146225053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c349132f0 0x556c34cec6b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1b7d7fa640 0 client.16596 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/146225053 >> 172.21.15.2:0/146225053 conn(0x556c34930300 msgr2=0x556c34cdc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/146225053 shutdown_connections 2024-05-14T09:45:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1ba2a0f740 1 -- 172.21.15.2:0/146225053 wait complete. 2024-05-14T09:45:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1ba2a0f740 20 client.16596 trim_cache size 0 max 16384 2024-05-14T09:45:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:45:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:45:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:45:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:45:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1b9bb5e640 1 -- 172.21.15.2:0/1345814860 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1b800b2920 msgr2=0x7f1b800b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1b9bb5e640 1 --2- 172.21.15.2:0/1345814860 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1b800b2920 0x7f1b800b4e30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x556c34cdee40 tx=0x7f1b9000a2d0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1b9bb5e640 1 -- 172.21.15.2:0/1345814860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b94102e90 msgr2=0x7f1b9419ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1b9bb5e640 1 --2- 172.21.15.2:0/1345814860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b94102e90 0x7f1b9419ca30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009230 tx=0x7f1b84031a50 comp rx=0 tx=0).stop 2024-05-14T09:45:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1b9bb5e640 1 -- 172.21.15.2:0/1345814860 shutdown_connections 2024-05-14T09:45:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1b9bb5e640 1 --2- 172.21.15.2:0/1345814860 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1b800b2920 0x7f1b800b4e30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1b9bb5e640 1 --2- 172.21.15.2:0/1345814860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b941037e0 0x7f1b941a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1b9bb5e640 1 --2- 172.21.15.2:0/1345814860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b94102e90 0x7f1b9419ca30 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1b9bb5e640 1 --2- 172.21.15.2:0/1345814860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b94102540 0x7f1b9419c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1b9bb5e640 1 -- 172.21.15.2:0/1345814860 >> 172.21.15.2:0/1345814860 conn(0x7f1b940fdd50 msgr2=0x7f1b9410a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1b9bb5e640 1 -- 172.21.15.2:0/1345814860 shutdown_connections 2024-05-14T09:45:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:24.714+0000 7f1b9bb5e640 1 -- 172.21.15.2:0/1345814860 wait complete. 2024-05-14T09:45:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:24 smithi002 ceph-mon[31557]: pgmap v738: 129 pgs: 129 active+clean; 168 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 3.3 MiB/s wr, 170 op/s 2024-05-14T09:45:25.278 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967289 9.64 MB/sec execute 411 sec latency 989.657 ms 2024-05-14T09:45:25.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:25 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:45:25.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:25 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:45:25.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:25 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:45:26.278 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967289 9.62 MB/sec execute 412 sec latency 1989.791 ms 2024-05-14T09:45:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:26 smithi029 ceph-mon[36249]: pgmap v739: 129 pgs: 129 active+clean; 168 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2.4 MiB/s wr, 74 op/s 2024-05-14T09:45:26.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:26 smithi026 ceph-mon[36056]: pgmap v739: 129 pgs: 129 active+clean; 168 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2.4 MiB/s wr, 74 op/s 2024-05-14T09:45:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:26 smithi002 ceph-mon[31557]: pgmap v739: 129 pgs: 129 active+clean; 168 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2.4 MiB/s wr, 74 op/s 2024-05-14T09:45:27.278 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967289 9.59 MB/sec execute 413 sec latency 2989.867 ms 2024-05-14T09:45:28.278 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967289 9.57 MB/sec execute 414 sec latency 3989.915 ms 2024-05-14T09:45:28.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:28 smithi026 ceph-mon[36056]: pgmap v740: 129 pgs: 129 active+clean; 167 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.9 MiB/s wr, 100 op/s 2024-05-14T09:45:28.722 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:28 smithi002 ceph-mon[31557]: pgmap v740: 129 pgs: 129 active+clean; 167 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.9 MiB/s wr, 100 op/s 2024-05-14T09:45:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:28 smithi029 ceph-mon[36249]: pgmap v740: 129 pgs: 129 active+clean; 167 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.9 MiB/s wr, 100 op/s 2024-05-14T09:45:29.279 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967290 9.55 MB/sec execute 415 sec latency 4917.704 ms 2024-05-14T09:45:30.279 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967290 9.52 MB/sec execute 416 sec latency 1072.424 ms 2024-05-14T09:45:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:30 smithi026 ceph-mon[36056]: pgmap v741: 129 pgs: 129 active+clean; 166 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-05-14T09:45:30.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:30 smithi002 ceph-mon[31557]: pgmap v741: 129 pgs: 129 active+clean; 166 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-05-14T09:45:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:30 smithi029 ceph-mon[36249]: pgmap v741: 129 pgs: 129 active+clean; 166 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.7 MiB/s wr, 84 op/s 2024-05-14T09:45:31.279 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967290 9.50 MB/sec execute 417 sec latency 2072.481 ms 2024-05-14T09:45:32.279 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967290 9.48 MB/sec execute 418 sec latency 3072.542 ms 2024-05-14T09:45:32.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:32 smithi026 ceph-mon[36056]: pgmap v742: 129 pgs: 129 active+clean; 165 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-05-14T09:45:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:32 smithi002 ceph-mon[31557]: pgmap v742: 129 pgs: 129 active+clean; 165 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-05-14T09:45:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:32 smithi029 ceph-mon[36249]: pgmap v742: 129 pgs: 129 active+clean; 165 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-05-14T09:45:33.279 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967290 9.46 MB/sec execute 419 sec latency 4072.661 ms 2024-05-14T09:45:34.279 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967356 9.43 MB/sec execute 420 sec latency 4998.511 ms 2024-05-14T09:45:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:34 smithi026 ceph-mon[36056]: pgmap v743: 129 pgs: 129 active+clean; 164 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 1.8 MiB/s wr, 97 op/s 2024-05-14T09:45:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:34 smithi002 ceph-mon[31557]: pgmap v743: 129 pgs: 129 active+clean; 164 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 1.8 MiB/s wr, 97 op/s 2024-05-14T09:45:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:34 smithi029 ceph-mon[36249]: pgmap v743: 129 pgs: 129 active+clean; 164 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 1.8 MiB/s wr, 97 op/s 2024-05-14T09:45:35.279 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967356 9.41 MB/sec execute 421 sec latency 1016.033 ms 2024-05-14T09:45:36.279 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967356 9.39 MB/sec execute 422 sec latency 2016.172 ms 2024-05-14T09:45:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:36 smithi002 ceph-mon[31557]: pgmap v744: 129 pgs: 129 active+clean; 164 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-05-14T09:45:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:36 smithi029 ceph-mon[36249]: pgmap v744: 129 pgs: 129 active+clean; 164 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-05-14T09:45:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:36 smithi026 ceph-mon[36056]: pgmap v744: 129 pgs: 129 active+clean; 164 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-05-14T09:45:37.279 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967356 9.37 MB/sec execute 423 sec latency 3016.258 ms 2024-05-14T09:45:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.829+0000 7fda219ea640 1 -- 172.21.15.2:0/3573590429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda1c109ba0 msgr2=0x7fda1c10bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.829+0000 7fda219ea640 1 --2- 172.21.15.2:0/3573590429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda1c109ba0 0x7fda1c10bfc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fda14009ca0 tx=0x7fda1402a8c0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.829+0000 7fda219ea640 1 -- 172.21.15.2:0/3573590429 shutdown_connections 2024-05-14T09:45:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.829+0000 7fda219ea640 1 --2- 172.21.15.2:0/3573590429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda1c109ba0 0x7fda1c10bfc0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.829+0000 7fda219ea640 1 --2- 172.21.15.2:0/3573590429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda1c104ea0 0x7fda1c1052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.829+0000 7fda219ea640 1 --2- 172.21.15.2:0/3573590429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda1c103c60 0x7fda1c104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.829+0000 7fda219ea640 1 -- 172.21.15.2:0/3573590429 >> 172.21.15.2:0/3573590429 conn(0x7fda1c0fee30 msgr2=0x7fda1c101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.829+0000 7fda219ea640 1 -- 172.21.15.2:0/3573590429 shutdown_connections 2024-05-14T09:45:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.829+0000 7fda219ea640 1 -- 172.21.15.2:0/3573590429 wait complete. 2024-05-14T09:45:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.829+0000 7fda219ea640 1 Processor -- start 2024-05-14T09:45:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda219ea640 1 -- start start 2024-05-14T09:45:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda219ea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda1c103c60 0x7fda1c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda219ea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda1c104ea0 0x7fda1c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda219ea640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda1c109ba0 0x7fda1c1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda219ea640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c10f790 con 0x7fda1c109ba0 2024-05-14T09:45:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda219ea640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c10f610 con 0x7fda1c104ea0 2024-05-14T09:45:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda219ea640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c10f910 con 0x7fda1c103c60 2024-05-14T09:45:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda1b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda1c109ba0 0x7fda1c1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda1b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda1c109ba0 0x7fda1c1a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44498/0 (socket says 172.21.15.2:44498) 2024-05-14T09:45:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda1b7fe640 1 -- 172.21.15.2:0/972357911 learned_addr learned my addr 172.21.15.2:0/972357911 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda1affd640 1 --2- 172.21.15.2:0/972357911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda1c103c60 0x7fda1c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda13fff640 1 --2- 172.21.15.2:0/972357911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda1c104ea0 0x7fda1c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda1b7fe640 1 -- 172.21.15.2:0/972357911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda1c103c60 msgr2=0x7fda1c19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda1b7fe640 1 --2- 172.21.15.2:0/972357911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda1c103c60 0x7fda1c19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda1b7fe640 1 -- 172.21.15.2:0/972357911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda1c104ea0 msgr2=0x7fda1c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda1b7fe640 1 --2- 172.21.15.2:0/972357911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda1c104ea0 0x7fda1c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda1b7fe640 1 -- 172.21.15.2:0/972357911 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c1a1d80 con 0x7fda1c109ba0 2024-05-14T09:45:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.830+0000 7fda13fff640 1 --2- 172.21.15.2:0/972357911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda1c104ea0 0x7fda1c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:45:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.831+0000 7fda1b7fe640 1 --2- 172.21.15.2:0/972357911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda1c109ba0 0x7fda1c1a1710 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fda140097b0 tx=0x7fda14003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.831+0000 7fda209e8640 1 -- 172.21.15.2:0/972357911 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14004240 con 0x7fda1c109ba0 2024-05-14T09:45:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.831+0000 7fda209e8640 1 -- 172.21.15.2:0/972357911 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda1402f380 con 0x7fda1c109ba0 2024-05-14T09:45:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.831+0000 7fda219ea640 1 -- 172.21.15.2:0/972357911 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1c1a2010 con 0x7fda1c109ba0 2024-05-14T09:45:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.831+0000 7fda209e8640 1 -- 172.21.15.2:0/972357911 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1402fc30 con 0x7fda1c109ba0 2024-05-14T09:45:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.831+0000 7fda219ea640 1 -- 172.21.15.2:0/972357911 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda1c1a2560 con 0x7fda1c109ba0 2024-05-14T09:45:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.833+0000 7fda28973740 1 Processor -- start 2024-05-14T09:45:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.834+0000 7fda209e8640 1 -- 172.21.15.2:0/972357911 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fda140043e0 con 0x7fda1c109ba0 2024-05-14T09:45:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.834+0000 7fda28973740 1 -- start start 2024-05-14T09:45:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.834+0000 7fda28973740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b411b0 0x55ad83ee2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.834+0000 7fda28973740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83b182f0 0x55ad83ee2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.834+0000 7fda28973740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b204d0 0x55ad83ee9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.834+0000 7fda28973740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ad83e475d0 con 0x55ad83b204d0 2024-05-14T09:45:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.834+0000 7fda28973740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ad83e00890 con 0x55ad83b182f0 2024-05-14T09:45:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.834+0000 7fda28973740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ad83e43590 con 0x55ad83b411b0 2024-05-14T09:45:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.834+0000 7fda13fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b204d0 0x55ad83ee9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.834+0000 7fda1affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83b182f0 0x55ad83ee2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.834+0000 7fda13fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b204d0 0x55ad83ee9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44508/0 (socket says 172.21.15.2:44508) 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.834+0000 7fda13fff640 1 -- 172.21.15.2:0/1866038220 learned_addr learned my addr 172.21.15.2:0/1866038220 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.834+0000 7fda13fff640 1 --2- 172.21.15.2:0/1866038220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b411b0 0x55ad83ee2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.835+0000 7fda1affd640 1 -- 172.21.15.2:0/1866038220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b411b0 msgr2=0x55ad83ee2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.835+0000 7fda1affd640 1 --2- 172.21.15.2:0/1866038220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b411b0 0x55ad83ee2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.835+0000 7fda1affd640 1 -- 172.21.15.2:0/1866038220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b204d0 msgr2=0x55ad83ee9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.835+0000 7fda1affd640 1 --2- 172.21.15.2:0/1866038220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b204d0 0x55ad83ee9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.835+0000 7fda1affd640 1 -- 172.21.15.2:0/1866038220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad83b59210 con 0x55ad83b182f0 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.835+0000 7fda13fff640 1 --2- 172.21.15.2:0/1866038220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b204d0 0x55ad83ee9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.835+0000 7fda209e8640 1 --2- 172.21.15.2:0/972357911 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fda080b28a0 0x7fda080b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.835+0000 7fda13fff640 1 --2- 172.21.15.2:0/972357911 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fda080b28a0 0x7fda080b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.835+0000 7fda1affd640 1 --2- 172.21.15.2:0/1866038220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83b182f0 0x55ad83ee2fe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda1c104860 tx=0x7fda0c00bbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e1c7a3c46cab8b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.835+0000 7fda209e8640 1 -- 172.21.15.2:0/972357911 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fda14103dc0 con 0x7fda1c109ba0 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.835+0000 7fda11ffb640 1 -- 172.21.15.2:0/1866038220 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c003da0 con 0x55ad83b182f0 2024-05-14T09:45:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.835+0000 7fda11ffb640 1 -- 172.21.15.2:0/1866038220 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda0c015070 con 0x55ad83b182f0 2024-05-14T09:45:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.835+0000 7fda11ffb640 1 -- 172.21.15.2:0/1866038220 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c011640 con 0x55ad83b182f0 2024-05-14T09:45:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.836+0000 7fda28973740 1 -- 172.21.15.2:0/1866038220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83b182f0 msgr2=0x55ad83ee2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.836+0000 7fda13fff640 1 --2- 172.21.15.2:0/972357911 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fda080b28a0 0x7fda080b4db0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fda040058c0 tx=0x7fda0400c530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.836+0000 7fda28973740 1 --2- 172.21.15.2:0/1866038220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83b182f0 0x55ad83ee2fe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda1c104860 tx=0x7fda0c00bbe0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.836+0000 7fda28973740 1 -- 172.21.15.2:0/1866038220 shutdown_connections 2024-05-14T09:45:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.836+0000 7fda28973740 1 --2- 172.21.15.2:0/1866038220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b204d0 0x55ad83ee9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.836+0000 7fda28973740 1 --2- 172.21.15.2:0/1866038220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83b182f0 0x55ad83ee2fe0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.836+0000 7fda28973740 1 --2- 172.21.15.2:0/1866038220 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b411b0 0x55ad83ee2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.836+0000 7fda28973740 1 -- 172.21.15.2:0/1866038220 >> 172.21.15.2:0/1866038220 conn(0x55ad83b35300 msgr2=0x55ad83ede3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.836+0000 7fda28973740 1 -- 172.21.15.2:0/1866038220 shutdown_connections 2024-05-14T09:45:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.836+0000 7fda28973740 1 -- 172.21.15.2:0/1866038220 wait complete. 2024-05-14T09:45:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.837+0000 7fda28973740 1 -- start start 2024-05-14T09:45:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.837+0000 7fda28973740 1 Processor -- start 2024-05-14T09:45:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.837+0000 7fda28973740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83b182f0 0x55ad83eec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.837+0000 7fda28973740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b204d0 0x55ad83eecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.837+0000 7fda28973740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b411b0 0x55ad83ef3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.837+0000 7fda28973740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ad83eea930 con 0x55ad83b411b0 2024-05-14T09:45:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.837+0000 7fda28973740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ad83e04370 con 0x55ad83b182f0 2024-05-14T09:45:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.837+0000 7fda28973740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55ad83eeaab0 con 0x55ad83b204d0 2024-05-14T09:45:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.837+0000 7fda28973740 10 client.0 _ll_register_callbacks cb 0x55ad83b58a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:45:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.837+0000 7fda1b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b411b0 0x55ad83ef3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.837+0000 7fda1b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b411b0 0x55ad83ef3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44514/0 (socket says 172.21.15.2:44514) 2024-05-14T09:45:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.837+0000 7fda1b7fe640 1 -- 172.21.15.2:0/299977177 learned_addr learned my addr 172.21.15.2:0/299977177 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fda13fff640 1 --2- 172.21.15.2:0/299977177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b204d0 0x55ad83eecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fda1affd640 1 --2- 172.21.15.2:0/299977177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83b182f0 0x55ad83eec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fda1b7fe640 1 -- 172.21.15.2:0/299977177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b204d0 msgr2=0x55ad83eecbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fda1b7fe640 1 --2- 172.21.15.2:0/299977177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b204d0 0x55ad83eecbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fda1b7fe640 1 -- 172.21.15.2:0/299977177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83b182f0 msgr2=0x55ad83eec6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fda1b7fe640 1 --2- 172.21.15.2:0/299977177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83b182f0 0x55ad83eec6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fda1b7fe640 1 -- 172.21.15.2:0/299977177 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad83c25620 con 0x55ad83b411b0 2024-05-14T09:45:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fda13fff640 1 --2- 172.21.15.2:0/299977177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b204d0 0x55ad83eecbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:45:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fda1b7fe640 1 --2- 172.21.15.2:0/299977177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b411b0 0x55ad83ef3b40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fda1402f550 tx=0x7fda14031880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63838e1715be85f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fd9faffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:45:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fd9faffd640 1 -- 172.21.15.2:0/299977177 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda140cafc0 con 0x55ad83b411b0 2024-05-14T09:45:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fd9faffd640 1 -- 172.21.15.2:0/299977177 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fda140046d0 con 0x55ad83b411b0 2024-05-14T09:45:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fda28973740 1 -- 172.21.15.2:0/299977177 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda0c015070 con 0x55ad83b411b0 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fd9faffd640 1 -- 172.21.15.2:0/299977177 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fda140c6df0 con 0x55ad83b411b0 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fda28973740 10 client.16608 fetch_fsmap learned FSMap version 44 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fda28973740 1 -- 172.21.15.2:0/299977177 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ad83eea930 con 0x55ad83b411b0 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.838+0000 7fd9faffd640 1 -- 172.21.15.2:0/299977177 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda0c015070 con 0x55ad83b411b0 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.839+0000 7fd9faffd640 1 -- 172.21.15.2:0/299977177 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fda140cd070 con 0x55ad83b411b0 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.839+0000 7fda28973740 10 client.16608 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.839+0000 7fda28973740 10 client.16608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.839+0000 7fda28973740 20 client.16608 populate_metadata read hostname 'smithi002' 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.839+0000 7fda28973740 1 --2- 172.21.15.2:0/299977177 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ad83affd80 0x55ad83efa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.839+0000 7fda28973740 4 client.16608 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.839+0000 7fda28973740 1 -- 172.21.15.2:0/299977177 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad83e551e0 con 0x55ad83affd80 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.839+0000 7fda1affd640 1 --2- 172.21.15.2:0/299977177 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ad83affd80 0x55ad83efa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.839+0000 7fd9faffd640 10 client.16608 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.839+0000 7fda1affd640 1 --2- 172.21.15.2:0/299977177 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ad83affd80 0x55ad83efa3d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=437ede27ee5bb9bf server_cookie=4b570a1bd1565253 in_seq=0 out_seq=0 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.840+0000 7fd9faffd640 1 -- 172.21.15.2:0/299977177 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ad83e551e0 con 0x55ad83affd80 2024-05-14T09:45:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.840+0000 7fd9faffd640 10 client.16608 handle_command_reply: tid=0 2024-05-14T09:45:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.840+0000 7fda28973740 1 client.16608 shutdown 2024-05-14T09:45:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.841+0000 7fda28973740 1 -- 172.21.15.2:0/299977177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b411b0 msgr2=0x55ad83ef3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.841+0000 7fda28973740 1 --2- 172.21.15.2:0/299977177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b411b0 0x55ad83ef3b40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fda1402f550 tx=0x7fda14031880 comp rx=0 tx=0).stop 2024-05-14T09:45:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.841+0000 7fda28973740 1 -- 172.21.15.2:0/299977177 shutdown_connections 2024-05-14T09:45:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.841+0000 7fda28973740 1 --2- 172.21.15.2:0/299977177 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ad83affd80 0x55ad83efa3d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.841+0000 7fda28973740 1 --2- 172.21.15.2:0/299977177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b411b0 0x55ad83ef3b40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.841+0000 7fda28973740 1 --2- 172.21.15.2:0/299977177 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b204d0 0x55ad83eecbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.841+0000 7fd9faffd640 0 client.16608 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.841+0000 7fda28973740 1 --2- 172.21.15.2:0/299977177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83b182f0 0x55ad83eec6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.841+0000 7fda28973740 1 -- 172.21.15.2:0/299977177 >> 172.21.15.2:0/299977177 conn(0x55ad83b35300 msgr2=0x55ad83ee1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.841+0000 7fda28973740 1 -- 172.21.15.2:0/299977177 shutdown_connections 2024-05-14T09:45:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.841+0000 7fda28973740 1 -- 172.21.15.2:0/299977177 wait complete. 2024-05-14T09:45:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.841+0000 7fda28973740 20 client.16608 trim_cache size 0 max 16384 2024-05-14T09:45:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.856+0000 7fda28973740 1 Processor -- start 2024-05-14T09:45:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.856+0000 7fda28973740 1 -- start start 2024-05-14T09:45:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.856+0000 7fda28973740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83affd80 0x55ad83ee6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.856+0000 7fda28973740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b182f0 0x55ad83ee6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.856+0000 7fda28973740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b204d0 0x55ad83ef3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.856+0000 7fda28973740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c1af430 con 0x55ad83b204d0 2024-05-14T09:45:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.856+0000 7fda28973740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c1acc70 con 0x55ad83affd80 2024-05-14T09:45:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.856+0000 7fda28973740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c1aaac0 con 0x55ad83b182f0 2024-05-14T09:45:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.856+0000 7fda1b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b204d0 0x55ad83ef3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.857+0000 7fda1b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b204d0 0x55ad83ef3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44520/0 (socket says 172.21.15.2:44520) 2024-05-14T09:45:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.857+0000 7fda1b7fe640 1 -- 172.21.15.2:0/4079718582 learned_addr learned my addr 172.21.15.2:0/4079718582 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.857+0000 7fda1affd640 1 --2- 172.21.15.2:0/4079718582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83affd80 0x55ad83ee6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.857+0000 7fda13fff640 1 --2- 172.21.15.2:0/4079718582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b182f0 0x55ad83ee6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.857+0000 7fda1b7fe640 1 -- 172.21.15.2:0/4079718582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b182f0 msgr2=0x55ad83ee6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.857+0000 7fda1b7fe640 1 --2- 172.21.15.2:0/4079718582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b182f0 0x55ad83ee6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.857+0000 7fda1b7fe640 1 -- 172.21.15.2:0/4079718582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83affd80 msgr2=0x55ad83ee6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.857+0000 7fda1b7fe640 1 --2- 172.21.15.2:0/4079718582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83affd80 0x55ad83ee6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.857+0000 7fda1b7fe640 1 -- 172.21.15.2:0/4079718582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad83ee3c60 con 0x55ad83b204d0 2024-05-14T09:45:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.857+0000 7fda1b7fe640 1 --2- 172.21.15.2:0/4079718582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b204d0 0x55ad83ef3a60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fda140c6fb0 tx=0x7fda1402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2bd6423f50b5b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.857+0000 7fda117fa640 1 -- 172.21.15.2:0/4079718582 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda140ca910 con 0x55ad83b204d0 2024-05-14T09:45:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.857+0000 7fda117fa640 1 -- 172.21.15.2:0/4079718582 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda140e9070 con 0x55ad83b204d0 2024-05-14T09:45:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.857+0000 7fda117fa640 1 -- 172.21.15.2:0/4079718582 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda140c71c0 con 0x55ad83b204d0 2024-05-14T09:45:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.858+0000 7fda28973740 1 -- 172.21.15.2:0/4079718582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b204d0 msgr2=0x55ad83ef3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.858+0000 7fda28973740 1 --2- 172.21.15.2:0/4079718582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b204d0 0x55ad83ef3a60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fda140c6fb0 tx=0x7fda1402a8c0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.858+0000 7fda28973740 1 -- 172.21.15.2:0/4079718582 shutdown_connections 2024-05-14T09:45:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.858+0000 7fda28973740 1 --2- 172.21.15.2:0/4079718582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b204d0 0x55ad83ef3a60 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.858+0000 7fda28973740 1 --2- 172.21.15.2:0/4079718582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b182f0 0x55ad83ee6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.858+0000 7fda28973740 1 --2- 172.21.15.2:0/4079718582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83affd80 0x55ad83ee6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.858+0000 7fda28973740 1 -- 172.21.15.2:0/4079718582 >> 172.21.15.2:0/4079718582 conn(0x55ad83b35300 msgr2=0x55ad83ee29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.858+0000 7fda28973740 1 -- 172.21.15.2:0/4079718582 shutdown_connections 2024-05-14T09:45:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.858+0000 7fda28973740 1 -- 172.21.15.2:0/4079718582 wait complete. 2024-05-14T09:45:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.858+0000 7fda28973740 1 -- start start 2024-05-14T09:45:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda28973740 1 Processor -- start 2024-05-14T09:45:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda28973740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83affd80 0x55ad83ef1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda28973740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b182f0 0x55ad83ef1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda28973740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b204d0 0x55ad83ef28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda28973740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c1ab130 con 0x55ad83b182f0 2024-05-14T09:45:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda28973740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c1ac5a0 con 0x55ad83affd80 2024-05-14T09:45:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda28973740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c1ab800 con 0x55ad83b204d0 2024-05-14T09:45:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda28973740 10 client.0 _ll_register_callbacks cb 0x7fda14009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:45:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda13fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b182f0 0x55ad83ef1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda1affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83affd80 0x55ad83ef1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda13fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b182f0 0x55ad83ef1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44526/0 (socket says 172.21.15.2:44526) 2024-05-14T09:45:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda13fff640 1 -- 172.21.15.2:0/1132773856 learned_addr learned my addr 172.21.15.2:0/1132773856 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda1b7fe640 1 --2- 172.21.15.2:0/1132773856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b204d0 0x55ad83ef28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda13fff640 1 -- 172.21.15.2:0/1132773856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b204d0 msgr2=0x55ad83ef28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda13fff640 1 --2- 172.21.15.2:0/1132773856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b204d0 0x55ad83ef28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda13fff640 1 -- 172.21.15.2:0/1132773856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83affd80 msgr2=0x55ad83ef1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda13fff640 1 --2- 172.21.15.2:0/1132773856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83affd80 0x55ad83ef1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.859+0000 7fda13fff640 1 -- 172.21.15.2:0/1132773856 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad83ef2e80 con 0x55ad83b182f0 2024-05-14T09:45:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.860+0000 7fda13fff640 1 --2- 172.21.15.2:0/1132773856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b182f0 0x55ad83ef1990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fda04012e10 tx=0x7fda04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec17abca5bf027e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.860+0000 7fd9faffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:45:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.860+0000 7fd9faffd640 1 -- 172.21.15.2:0/1132773856 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda040146a0 con 0x55ad83b182f0 2024-05-14T09:45:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.860+0000 7fd9faffd640 1 -- 172.21.15.2:0/1132773856 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fda0400fb90 con 0x55ad83b182f0 2024-05-14T09:45:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.860+0000 7fda28973740 1 -- 172.21.15.2:0/1132773856 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda140e9070 con 0x55ad83b182f0 2024-05-14T09:45:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.860+0000 7fd9faffd640 1 -- 172.21.15.2:0/1132773856 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fda0400fd90 con 0x55ad83b182f0 2024-05-14T09:45:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.860+0000 7fda28973740 10 client.16614 fetch_fsmap learned FSMap version 44 2024-05-14T09:45:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.860+0000 7fda28973740 1 -- 172.21.15.2:0/1132773856 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ad83ef41b0 con 0x55ad83b182f0 2024-05-14T09:45:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.860+0000 7fd9faffd640 1 -- 172.21.15.2:0/1132773856 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda140e9070 con 0x55ad83b182f0 2024-05-14T09:45:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.860+0000 7fd9faffd640 1 -- 172.21.15.2:0/1132773856 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fda0401d070 con 0x55ad83b182f0 2024-05-14T09:45:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.860+0000 7fda28973740 10 client.16614 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T09:45:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.860+0000 7fda28973740 10 client.16614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:45:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.860+0000 7fda28973740 20 client.16614 populate_metadata read hostname 'smithi002' 2024-05-14T09:45:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.861+0000 7fda28973740 1 --2- 172.21.15.2:0/1132773856 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ad83b411b0 0x55ad83f64bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.861+0000 7fda28973740 4 client.16614 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:45:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.861+0000 7fda28973740 1 -- 172.21.15.2:0/1132773856 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad83e4f050 con 0x55ad83b411b0 2024-05-14T09:45:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.861+0000 7fda1affd640 1 --2- 172.21.15.2:0/1132773856 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ad83b411b0 0x55ad83f64bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.861+0000 7fd9faffd640 10 client.16614 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.861+0000 7fda1affd640 1 --2- 172.21.15.2:0/1132773856 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ad83b411b0 0x55ad83f64bc0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=353bbc64f4acde19 server_cookie=e8373a2862bd498a in_seq=0 out_seq=0 2024-05-14T09:45:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.896+0000 7fd9faffd640 1 -- 172.21.15.2:0/1132773856 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad83e4f050 con 0x55ad83b411b0 2024-05-14T09:45:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.896+0000 7fd9faffd640 10 client.16614 handle_command_reply: tid=0 2024-05-14T09:45:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.896+0000 7fda28973740 1 client.16614 shutdown 2024-05-14T09:45:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.896+0000 7fda28973740 1 -- 172.21.15.2:0/1132773856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b182f0 msgr2=0x55ad83ef1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.896+0000 7fda28973740 1 --2- 172.21.15.2:0/1132773856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b182f0 0x55ad83ef1990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fda04012e10 tx=0x7fda04004730 comp rx=0 tx=0).stop 2024-05-14T09:45:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.896+0000 7fda28973740 1 -- 172.21.15.2:0/1132773856 shutdown_connections 2024-05-14T09:45:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.896+0000 7fda28973740 1 --2- 172.21.15.2:0/1132773856 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55ad83b411b0 0x55ad83f64bc0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.896+0000 7fda28973740 1 --2- 172.21.15.2:0/1132773856 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ad83b204d0 0x55ad83ef28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.896+0000 7fda28973740 1 --2- 172.21.15.2:0/1132773856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ad83b182f0 0x55ad83ef1990 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.896+0000 7fd9faffd640 0 client.16614 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.896+0000 7fda28973740 1 --2- 172.21.15.2:0/1132773856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad83affd80 0x55ad83ef1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.896+0000 7fda28973740 1 -- 172.21.15.2:0/1132773856 >> 172.21.15.2:0/1132773856 conn(0x55ad83b35300 msgr2=0x55ad83ee29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.896+0000 7fda28973740 1 -- 172.21.15.2:0/1132773856 shutdown_connections 2024-05-14T09:45:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.896+0000 7fda28973740 1 -- 172.21.15.2:0/1132773856 wait complete. 2024-05-14T09:45:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.896+0000 7fda28973740 20 client.16614 trim_cache size 0 max 16384 2024-05-14T09:45:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:45:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:45:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:45:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:45:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.897+0000 7fda219ea640 1 -- 172.21.15.2:0/972357911 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fda080b28a0 msgr2=0x7fda080b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.897+0000 7fda219ea640 1 --2- 172.21.15.2:0/972357911 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fda080b28a0 0x7fda080b4db0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fda040058c0 tx=0x7fda0400c530 comp rx=0 tx=0).stop 2024-05-14T09:45:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.897+0000 7fda219ea640 1 -- 172.21.15.2:0/972357911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda1c109ba0 msgr2=0x7fda1c1a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.897+0000 7fda219ea640 1 --2- 172.21.15.2:0/972357911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda1c109ba0 0x7fda1c1a1710 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fda140097b0 tx=0x7fda14003c20 comp rx=0 tx=0).stop 2024-05-14T09:45:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.897+0000 7fda219ea640 1 -- 172.21.15.2:0/972357911 shutdown_connections 2024-05-14T09:45:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.897+0000 7fda219ea640 1 --2- 172.21.15.2:0/972357911 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fda080b28a0 0x7fda080b4db0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.897+0000 7fda219ea640 1 --2- 172.21.15.2:0/972357911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda1c109ba0 0x7fda1c1a1710 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.897+0000 7fda219ea640 1 --2- 172.21.15.2:0/972357911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda1c104ea0 0x7fda1c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.897+0000 7fda219ea640 1 --2- 172.21.15.2:0/972357911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fda1c103c60 0x7fda1c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.897+0000 7fda219ea640 1 -- 172.21.15.2:0/972357911 >> 172.21.15.2:0/972357911 conn(0x7fda1c0fee30 msgr2=0x7fda1c105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.897+0000 7fda219ea640 1 -- 172.21.15.2:0/972357911 shutdown_connections 2024-05-14T09:45:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:37.897+0000 7fda219ea640 1 -- 172.21.15.2:0/972357911 wait complete. 2024-05-14T09:45:38.279 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967356 9.35 MB/sec execute 424 sec latency 4016.336 ms 2024-05-14T09:45:38.738 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:38 smithi029 ceph-mon[36249]: pgmap v745: 129 pgs: 129 active+clean; 163 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 1.6 MiB/s wr, 92 op/s 2024-05-14T09:45:38.753 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:38 smithi026 ceph-mon[36056]: pgmap v745: 129 pgs: 129 active+clean; 163 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 1.6 MiB/s wr, 92 op/s 2024-05-14T09:45:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:38 smithi002 ceph-mon[31557]: pgmap v745: 129 pgs: 129 active+clean; 163 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 1.6 MiB/s wr, 92 op/s 2024-05-14T09:45:39.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 967541 9.33 MB/sec execute 425 sec latency 4938.987 ms 2024-05-14T09:45:40.279 INFO:tasks.workunit.client.0.smithi002.stdout: 1 969727 9.33 MB/sec execute 426 sec latency 71.441 ms 2024-05-14T09:45:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:40 smithi002 ceph-mon[31557]: pgmap v746: 129 pgs: 129 active+clean; 163 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.3 MiB/s wr, 82 op/s 2024-05-14T09:45:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:40 smithi029 ceph-mon[36249]: pgmap v746: 129 pgs: 129 active+clean; 163 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.3 MiB/s wr, 82 op/s 2024-05-14T09:45:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:40 smithi026 ceph-mon[36056]: pgmap v746: 129 pgs: 129 active+clean; 163 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.3 MiB/s wr, 82 op/s 2024-05-14T09:45:41.279 INFO:tasks.workunit.client.0.smithi002.stdout: 1 969727 9.31 MB/sec execute 427 sec latency 1071.483 ms 2024-05-14T09:45:42.279 INFO:tasks.workunit.client.0.smithi002.stdout: 1 970591 9.30 MB/sec execute 428 sec latency 1078.089 ms 2024-05-14T09:45:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:42 smithi002 ceph-mon[31557]: pgmap v747: 129 pgs: 129 active+clean; 162 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 1.2 MiB/s wr, 74 op/s 2024-05-14T09:45:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:42 smithi029 ceph-mon[36249]: pgmap v747: 129 pgs: 129 active+clean; 162 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 1.2 MiB/s wr, 74 op/s 2024-05-14T09:45:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:42 smithi026 ceph-mon[36056]: pgmap v747: 129 pgs: 129 active+clean; 162 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 1.2 MiB/s wr, 74 op/s 2024-05-14T09:45:43.279 INFO:tasks.workunit.client.0.smithi002.stdout: 1 970591 9.28 MB/sec execute 429 sec latency 1104.513 ms 2024-05-14T09:45:44.280 INFO:tasks.workunit.client.0.smithi002.stdout: 1 970657 9.26 MB/sec execute 430 sec latency 2029.022 ms 2024-05-14T09:45:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:44 smithi002 ceph-mon[31557]: pgmap v748: 129 pgs: 129 active+clean; 163 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 1.9 MiB/s wr, 122 op/s 2024-05-14T09:45:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:44 smithi029 ceph-mon[36249]: pgmap v748: 129 pgs: 129 active+clean; 163 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 1.9 MiB/s wr, 122 op/s 2024-05-14T09:45:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:44 smithi026 ceph-mon[36056]: pgmap v748: 129 pgs: 129 active+clean; 163 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 1.9 MiB/s wr, 122 op/s 2024-05-14T09:45:45.280 INFO:tasks.workunit.client.0.smithi002.stdout: 1 972638 9.26 MB/sec execute 431 sec latency 20.065 ms 2024-05-14T09:45:46.280 INFO:tasks.workunit.client.0.smithi002.stdout: 1 974000 9.26 MB/sec execute 432 sec latency 12.784 ms 2024-05-14T09:45:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:46 smithi002 ceph-mon[31557]: pgmap v749: 129 pgs: 129 active+clean; 163 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-05-14T09:45:46.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:46 smithi029 ceph-mon[36249]: pgmap v749: 129 pgs: 129 active+clean; 163 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-05-14T09:45:46.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:46 smithi026 ceph-mon[36056]: pgmap v749: 129 pgs: 129 active+clean; 163 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-05-14T09:45:47.280 INFO:tasks.workunit.client.0.smithi002.stdout: 1 974191 9.24 MB/sec execute 433 sec latency 825.685 ms 2024-05-14T09:45:48.280 INFO:tasks.workunit.client.0.smithi002.stdout: 1 974191 9.22 MB/sec execute 434 sec latency 1825.799 ms 2024-05-14T09:45:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:48 smithi002 ceph-mon[31557]: pgmap v750: 129 pgs: 129 active+clean; 163 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-14T09:45:48.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:48 smithi029 ceph-mon[36249]: pgmap v750: 129 pgs: 129 active+clean; 163 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-14T09:45:48.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:48 smithi026 ceph-mon[36056]: pgmap v750: 129 pgs: 129 active+clean; 163 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-14T09:45:49.280 INFO:tasks.workunit.client.0.smithi002.stdout: 1 974259 9.20 MB/sec execute 435 sec latency 2745.157 ms 2024-05-14T09:45:50.280 INFO:tasks.workunit.client.0.smithi002.stdout: 1 974259 9.17 MB/sec execute 436 sec latency 1011.017 ms 2024-05-14T09:45:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:50 smithi002 ceph-mon[31557]: pgmap v751: 129 pgs: 129 active+clean; 164 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-14T09:45:50.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:50 smithi029 ceph-mon[36249]: pgmap v751: 129 pgs: 129 active+clean; 164 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-14T09:45:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:50 smithi026 ceph-mon[36056]: pgmap v751: 129 pgs: 129 active+clean; 164 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-14T09:45:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.012+0000 7f4d641ce640 1 -- 172.21.15.2:0/222910056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c108a90 msgr2=0x7f4d5c10aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.012+0000 7f4d641ce640 1 --2- 172.21.15.2:0/222910056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c108a90 0x7f4d5c10aeb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009ca0 tx=0x7f4d5402a8c0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.012+0000 7f4d641ce640 1 -- 172.21.15.2:0/222910056 shutdown_connections 2024-05-14T09:45:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.012+0000 7f4d641ce640 1 --2- 172.21.15.2:0/222910056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c108a90 0x7f4d5c10aeb0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.012+0000 7f4d641ce640 1 --2- 172.21.15.2:0/222910056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c103d90 0x7f4d5c1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.012+0000 7f4d641ce640 1 --2- 172.21.15.2:0/222910056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d5c102b50 0x7f4d5c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.012+0000 7f4d641ce640 1 -- 172.21.15.2:0/222910056 >> 172.21.15.2:0/222910056 conn(0x7f4d5c0fdd40 msgr2=0x7f4d5c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.012+0000 7f4d641ce640 1 -- 172.21.15.2:0/222910056 shutdown_connections 2024-05-14T09:45:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.012+0000 7f4d641ce640 1 -- 172.21.15.2:0/222910056 wait complete. 2024-05-14T09:45:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.012+0000 7f4d641ce640 1 Processor -- start 2024-05-14T09:45:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d641ce640 1 -- start start 2024-05-14T09:45:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d641ce640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c102b50 0x7f4d5c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d641ce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d5c103d90 0x7f4d5c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d641ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c108a90 0x7f4d5c1a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d641ce640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c10e680 con 0x7f4d5c102b50 2024-05-14T09:45:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d641ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c10e500 con 0x7f4d5c108a90 2024-05-14T09:45:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d641ce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c10e800 con 0x7f4d5c103d90 2024-05-14T09:45:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d61f6a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c102b50 0x7f4d5c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d61f6a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c102b50 0x7f4d5c19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60190/0 (socket says 172.21.15.2:60190) 2024-05-14T09:45:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d61f6a640 1 -- 172.21.15.2:0/575033020 learned_addr learned my addr 172.21.15.2:0/575033020 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d61769640 1 --2- 172.21.15.2:0/575033020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d5c103d90 0x7f4d5c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d6276b640 1 --2- 172.21.15.2:0/575033020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c108a90 0x7f4d5c1a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d61f6a640 1 -- 172.21.15.2:0/575033020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d5c103d90 msgr2=0x7f4d5c19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/575033020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d5c103d90 0x7f4d5c19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d61f6a640 1 -- 172.21.15.2:0/575033020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c108a90 msgr2=0x7f4d5c1a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/575033020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c108a90 0x7f4d5c1a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d61f6a640 1 -- 172.21.15.2:0/575033020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5c1a1db0 con 0x7f4d5c102b50 2024-05-14T09:45:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d6276b640 1 --2- 172.21.15.2:0/575033020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c108a90 0x7f4d5c1a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:45:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.013+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/575033020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c102b50 0x7f4d5c19a270 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4d58005d00 tx=0x7f4d5800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.014+0000 7f4d52ffd640 1 -- 172.21.15.2:0/575033020 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58017070 con 0x7f4d5c102b50 2024-05-14T09:45:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.014+0000 7f4d641ce640 1 -- 172.21.15.2:0/575033020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5c1a20a0 con 0x7f4d5c102b50 2024-05-14T09:45:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.014+0000 7f4d52ffd640 1 -- 172.21.15.2:0/575033020 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d58009c30 con 0x7f4d5c102b50 2024-05-14T09:45:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.014+0000 7f4d52ffd640 1 -- 172.21.15.2:0/575033020 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5800f620 con 0x7f4d5c102b50 2024-05-14T09:45:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.014+0000 7f4d641ce640 1 -- 172.21.15.2:0/575033020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d5c1a25f0 con 0x7f4d5c102b50 2024-05-14T09:45:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.016+0000 7f4d6b07e740 1 Processor -- start 2024-05-14T09:45:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d6b07e740 1 -- start start 2024-05-14T09:45:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d6b07e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0921b0 0x5586bb437e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d6b07e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0692f0 0x5586bb4383f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d6b07e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 0x5586bb43f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d6b07e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5586bb3985d0 con 0x5586bb0692f0 2024-05-14T09:45:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d6b07e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5586bb351890 con 0x5586bb0714d0 2024-05-14T09:45:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d6b07e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5586bb394590 con 0x5586bb0921b0 2024-05-14T09:45:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d52ffd640 1 -- 172.21.15.2:0/575033020 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4d5801a070 con 0x7f4d5c102b50 2024-05-14T09:45:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d61f6a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0692f0 0x5586bb4383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d6276b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0921b0 0x5586bb437e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d6276b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0921b0 0x5586bb437e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:42860/0 (socket says 172.21.15.2:42860) 2024-05-14T09:45:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d61f6a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0692f0 0x5586bb4383f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60206/0 (socket says 172.21.15.2:60206) 2024-05-14T09:45:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d6276b640 1 -- 172.21.15.2:0/734551352 learned_addr learned my addr 172.21.15.2:0/734551352 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d61769640 1 --2- 172.21.15.2:0/734551352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 0x5586bb43f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d61f6a640 1 -- 172.21.15.2:0/734551352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0921b0 msgr2=0x5586bb437e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.017+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/734551352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0921b0 0x5586bb437e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.018+0000 7f4d61f6a640 1 -- 172.21.15.2:0/734551352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 msgr2=0x5586bb43f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.018+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/734551352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 0x5586bb43f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.018+0000 7f4d61f6a640 1 -- 172.21.15.2:0/734551352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586bb0aa210 con 0x5586bb0692f0 2024-05-14T09:45:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.018+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/734551352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0692f0 0x5586bb4383f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4d580df810 tx=0x7f4d580ea910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6294287a50507f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.018+0000 7f4d52ffd640 1 --2- 172.21.15.2:0/575033020 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4d480b28a0 0x7f4d480b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.018+0000 7f4d47fff640 1 -- 172.21.15.2:0/734551352 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d580f5070 con 0x5586bb0692f0 2024-05-14T09:45:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.018+0000 7f4d6276b640 1 --2- 172.21.15.2:0/575033020 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4d480b28a0 0x7f4d480b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.018+0000 7f4d47fff640 1 -- 172.21.15.2:0/734551352 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d580a8e70 con 0x5586bb0692f0 2024-05-14T09:45:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.018+0000 7f4d47fff640 1 -- 172.21.15.2:0/734551352 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d580a8ad0 con 0x5586bb0692f0 2024-05-14T09:45:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.018+0000 7f4d52ffd640 1 -- 172.21.15.2:0/575033020 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f4d580e2160 con 0x7f4d5c102b50 2024-05-14T09:45:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.019+0000 7f4d6b07e740 1 -- 172.21.15.2:0/734551352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0692f0 msgr2=0x5586bb4383f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.019+0000 7f4d6276b640 1 --2- 172.21.15.2:0/575033020 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4d480b28a0 0x7f4d480b4db0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4d540094e0 tx=0x7f4d54031a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.019+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/734551352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0692f0 0x5586bb4383f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4d580df810 tx=0x7f4d580ea910 comp rx=0 tx=0).stop 2024-05-14T09:45:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.019+0000 7f4d6b07e740 1 -- 172.21.15.2:0/734551352 shutdown_connections 2024-05-14T09:45:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.019+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/734551352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 0x5586bb43f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.019+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/734551352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0692f0 0x5586bb4383f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.019+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/734551352 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0921b0 0x5586bb437e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.019+0000 7f4d6b07e740 1 -- 172.21.15.2:0/734551352 >> 172.21.15.2:0/734551352 conn(0x5586bb086300 msgr2=0x5586bb42f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.019+0000 7f4d6b07e740 1 -- 172.21.15.2:0/734551352 shutdown_connections 2024-05-14T09:45:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.019+0000 7f4d6b07e740 1 -- 172.21.15.2:0/734551352 wait complete. 2024-05-14T09:45:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.019+0000 7f4d6b07e740 1 -- start start 2024-05-14T09:45:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.020+0000 7f4d6b07e740 1 Processor -- start 2024-05-14T09:45:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.020+0000 7f4d6b07e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0692f0 0x5586bb441b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.020+0000 7f4d6b07e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 0x5586bb442050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.020+0000 7f4d6b07e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0921b0 0x5586bb448ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.020+0000 7f4d6b07e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5586bb43fd40 con 0x5586bb0921b0 2024-05-14T09:45:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.020+0000 7f4d6b07e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5586bb355370 con 0x5586bb0714d0 2024-05-14T09:45:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.020+0000 7f4d6b07e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5586bb43fec0 con 0x5586bb0692f0 2024-05-14T09:45:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.020+0000 7f4d6b07e740 10 client.0 _ll_register_callbacks cb 0x5586bb0a9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:45:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.020+0000 7f4d6276b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0921b0 0x5586bb448ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.020+0000 7f4d6276b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0921b0 0x5586bb448ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60216/0 (socket says 172.21.15.2:60216) 2024-05-14T09:45:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.020+0000 7f4d6276b640 1 -- 172.21.15.2:0/1042703430 learned_addr learned my addr 172.21.15.2:0/1042703430 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.020+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/1042703430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0692f0 0x5586bb441b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d61769640 1 --2- 172.21.15.2:0/1042703430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 0x5586bb442050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d6276b640 1 -- 172.21.15.2:0/1042703430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0692f0 msgr2=0x5586bb441b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d6276b640 1 --2- 172.21.15.2:0/1042703430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0692f0 0x5586bb441b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d6276b640 1 -- 172.21.15.2:0/1042703430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 msgr2=0x5586bb442050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d6276b640 1 --2- 172.21.15.2:0/1042703430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 0x5586bb442050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d6276b640 1 -- 172.21.15.2:0/1042703430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586bb176620 con 0x5586bb0921b0 2024-05-14T09:45:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/1042703430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0692f0 0x5586bb441b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:45:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d61769640 1 --2- 172.21.15.2:0/1042703430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 0x5586bb442050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:45:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d6276b640 1 --2- 172.21.15.2:0/1042703430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0921b0 0x5586bb448ed0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4d5402f3d0 tx=0x7f4d5402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa85c0f2dc69b0c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d457fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:45:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d457fa640 1 -- 172.21.15.2:0/1042703430 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54003e00 con 0x5586bb0921b0 2024-05-14T09:45:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1042703430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d580a8e70 con 0x5586bb0921b0 2024-05-14T09:45:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d457fa640 1 -- 172.21.15.2:0/1042703430 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f4d540047d0 con 0x5586bb0921b0 2024-05-14T09:45:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d457fa640 1 -- 172.21.15.2:0/1042703430 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4d5402f660 con 0x5586bb0921b0 2024-05-14T09:45:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d6b07e740 10 client.16626 fetch_fsmap learned FSMap version 44 2024-05-14T09:45:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.021+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1042703430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5586bb44be00 con 0x5586bb0921b0 2024-05-14T09:45:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.022+0000 7f4d457fa640 1 -- 172.21.15.2:0/1042703430 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d580a8e70 con 0x5586bb0921b0 2024-05-14T09:45:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.022+0000 7f4d457fa640 1 -- 172.21.15.2:0/1042703430 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f4d54050020 con 0x5586bb0921b0 2024-05-14T09:45:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.022+0000 7f4d6b07e740 10 client.16626 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T09:45:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.022+0000 7f4d6b07e740 10 client.16626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:45:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.022+0000 7f4d6b07e740 20 client.16626 populate_metadata read hostname 'smithi002' 2024-05-14T09:45:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.022+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1042703430 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5586bb050d80 0x5586bb44f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.022+0000 7f4d6b07e740 4 client.16626 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:45:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.022+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1042703430 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586bb3a61e0 con 0x5586bb050d80 2024-05-14T09:45:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.022+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/1042703430 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5586bb050d80 0x5586bb44f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.023+0000 7f4d457fa640 10 client.16626 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.023+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/1042703430 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5586bb050d80 0x5586bb44f6b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffd0ff0de4f1bc96 server_cookie=383fdd2970a2e77e in_seq=0 out_seq=0 2024-05-14T09:45:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.024+0000 7f4d457fa640 1 -- 172.21.15.2:0/1042703430 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5586bb3a61e0 con 0x5586bb050d80 2024-05-14T09:45:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.024+0000 7f4d457fa640 10 client.16626 handle_command_reply: tid=0 2024-05-14T09:45:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.024+0000 7f4d6b07e740 1 client.16626 shutdown 2024-05-14T09:45:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.024+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1042703430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0921b0 msgr2=0x5586bb448ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.024+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1042703430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0921b0 0x5586bb448ed0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4d5402f3d0 tx=0x7f4d5402a8c0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.024+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1042703430 shutdown_connections 2024-05-14T09:45:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.024+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1042703430 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5586bb050d80 0x5586bb44f6b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.024+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1042703430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0921b0 0x5586bb448ed0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.024+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1042703430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 0x5586bb442050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.024+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1042703430 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0692f0 0x5586bb441b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.024+0000 7f4d457fa640 0 client.16626 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.024+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1042703430 >> 172.21.15.2:0/1042703430 conn(0x5586bb086300 msgr2=0x5586bb432300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.024+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1042703430 shutdown_connections 2024-05-14T09:45:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.024+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1042703430 wait complete. 2024-05-14T09:45:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.024+0000 7f4d6b07e740 20 client.16626 trim_cache size 0 max 16384 2024-05-14T09:45:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.040+0000 7f4d6b07e740 1 Processor -- start 2024-05-14T09:45:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.040+0000 7f4d6b07e740 1 -- start start 2024-05-14T09:45:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d6b07e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0692f0 0x5586bb4402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d6b07e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 0x5586bb440830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d6b07e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0921b0 0x5586bb4321f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d6b07e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c1ad3f0 con 0x5586bb0692f0 2024-05-14T09:45:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d6b07e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c1aac30 con 0x5586bb0714d0 2024-05-14T09:45:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d6b07e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c1a8a80 con 0x5586bb0921b0 2024-05-14T09:45:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d61f6a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0692f0 0x5586bb4402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d61f6a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0692f0 0x5586bb4402f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60218/0 (socket says 172.21.15.2:60218) 2024-05-14T09:45:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d61769640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 0x5586bb440830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d61f6a640 1 -- 172.21.15.2:0/1958935255 learned_addr learned my addr 172.21.15.2:0/1958935255 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d6276b640 1 --2- 172.21.15.2:0/1958935255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0921b0 0x5586bb4321f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d61f6a640 1 -- 172.21.15.2:0/1958935255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0921b0 msgr2=0x5586bb4321f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/1958935255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0921b0 0x5586bb4321f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d61f6a640 1 -- 172.21.15.2:0/1958935255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 msgr2=0x5586bb440830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/1958935255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 0x5586bb440830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d61f6a640 1 -- 172.21.15.2:0/1958935255 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586bb43ffb0 con 0x5586bb0692f0 2024-05-14T09:45:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/1958935255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0692f0 0x5586bb4402f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4d580df890 tx=0x7f4d58010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f64d10e0a3add3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d477fe640 1 -- 172.21.15.2:0/1958935255 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5806c3a0 con 0x5586bb0692f0 2024-05-14T09:45:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.041+0000 7f4d477fe640 1 -- 172.21.15.2:0/1958935255 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d580e5a10 con 0x5586bb0692f0 2024-05-14T09:45:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.042+0000 7f4d477fe640 1 -- 172.21.15.2:0/1958935255 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5806c600 con 0x5586bb0692f0 2024-05-14T09:45:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.042+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1958935255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0692f0 msgr2=0x5586bb4402f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.042+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1958935255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0692f0 0x5586bb4402f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4d580df890 tx=0x7f4d58010c00 comp rx=0 tx=0).stop 2024-05-14T09:45:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.042+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1958935255 shutdown_connections 2024-05-14T09:45:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.042+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1958935255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0921b0 0x5586bb4321f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.042+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1958935255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0714d0 0x5586bb440830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.042+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1958935255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0692f0 0x5586bb4402f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.042+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1958935255 >> 172.21.15.2:0/1958935255 conn(0x5586bb086300 msgr2=0x5586bb43bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.042+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1958935255 shutdown_connections 2024-05-14T09:45:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.042+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1958935255 wait complete. 2024-05-14T09:45:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.043+0000 7f4d6b07e740 1 -- start start 2024-05-14T09:45:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.043+0000 7f4d6b07e740 1 Processor -- start 2024-05-14T09:45:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.043+0000 7f4d6b07e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0692f0 0x5586bb448f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.043+0000 7f4d6b07e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0714d0 0x5586bb449480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.043+0000 7f4d6b07e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0921b0 0x5586bb445ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.043+0000 7f4d6b07e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c1a90f0 con 0x5586bb0921b0 2024-05-14T09:45:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.043+0000 7f4d6b07e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c1aa560 con 0x5586bb0692f0 2024-05-14T09:45:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.043+0000 7f4d6b07e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c1a97c0 con 0x5586bb0714d0 2024-05-14T09:45:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.043+0000 7f4d6b07e740 10 client.0 _ll_register_callbacks cb 0x7f4d540318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:45:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.043+0000 7f4d6276b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0921b0 0x5586bb445ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.043+0000 7f4d6276b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0921b0 0x5586bb445ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60228/0 (socket says 172.21.15.2:60228) 2024-05-14T09:45:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.043+0000 7f4d6276b640 1 -- 172.21.15.2:0/1606276709 learned_addr learned my addr 172.21.15.2:0/1606276709 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:45:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.043+0000 7f4d61769640 1 --2- 172.21.15.2:0/1606276709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0714d0 0x5586bb449480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.043+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/1606276709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0692f0 0x5586bb448f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.044+0000 7f4d6276b640 1 -- 172.21.15.2:0/1606276709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0714d0 msgr2=0x5586bb449480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.044+0000 7f4d6276b640 1 --2- 172.21.15.2:0/1606276709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0714d0 0x5586bb449480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.044+0000 7f4d6276b640 1 -- 172.21.15.2:0/1606276709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0692f0 msgr2=0x5586bb448f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:45:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.044+0000 7f4d6276b640 1 --2- 172.21.15.2:0/1606276709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0692f0 0x5586bb448f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.044+0000 7f4d6276b640 1 -- 172.21.15.2:0/1606276709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586bb446160 con 0x5586bb0921b0 2024-05-14T09:45:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.044+0000 7f4d6276b640 1 --2- 172.21.15.2:0/1606276709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0921b0 0x5586bb445ac0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4d54006010 tx=0x7f4d54003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b7c9e32771894f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:45:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.044+0000 7f4d457fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:45:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.044+0000 7f4d457fa640 1 -- 172.21.15.2:0/1606276709 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5402fa00 con 0x5586bb0921b0 2024-05-14T09:45:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.044+0000 7f4d457fa640 1 -- 172.21.15.2:0/1606276709 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f4d5402fba0 con 0x5586bb0921b0 2024-05-14T09:45:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.044+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1606276709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d580e5a10 con 0x5586bb0921b0 2024-05-14T09:45:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.044+0000 7f4d457fa640 1 -- 172.21.15.2:0/1606276709 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4d54030890 con 0x5586bb0921b0 2024-05-14T09:45:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.044+0000 7f4d6b07e740 10 client.16632 fetch_fsmap learned FSMap version 44 2024-05-14T09:45:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.044+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1606276709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5586bb4b65f0 con 0x5586bb0921b0 2024-05-14T09:45:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.045+0000 7f4d457fa640 1 -- 172.21.15.2:0/1606276709 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d580e5a10 con 0x5586bb0921b0 2024-05-14T09:45:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.045+0000 7f4d457fa640 1 -- 172.21.15.2:0/1606276709 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f4d54050040 con 0x5586bb0921b0 2024-05-14T09:45:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.045+0000 7f4d6b07e740 10 client.16632 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T09:45:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.045+0000 7f4d6b07e740 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:45:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.045+0000 7f4d6b07e740 20 client.16632 populate_metadata read hostname 'smithi002' 2024-05-14T09:45:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.045+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1606276709 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5586bb0261c0 0x5586bb4b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:45:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.045+0000 7f4d6b07e740 4 client.16632 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:45:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.045+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1606276709 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5586bb3a0050 con 0x5586bb0261c0 2024-05-14T09:45:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.045+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/1606276709 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5586bb0261c0 0x5586bb4b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:45:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.046+0000 7f4d61f6a640 1 --2- 172.21.15.2:0/1606276709 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5586bb0261c0 0x5586bb4b8ef0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57f520b3dc99f78 server_cookie=74a1f7a7988e0ccc in_seq=0 out_seq=0 2024-05-14T09:45:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.046+0000 7f4d457fa640 10 client.16632 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.102+0000 7f4d457fa640 1 -- 172.21.15.2:0/1606276709 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5586bb3a0050 con 0x5586bb0261c0 2024-05-14T09:45:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.102+0000 7f4d457fa640 10 client.16632 handle_command_reply: tid=0 2024-05-14T09:45:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.102+0000 7f4d6b07e740 1 client.16632 shutdown 2024-05-14T09:45:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.102+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1606276709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0921b0 msgr2=0x5586bb445ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.102+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1606276709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0921b0 0x5586bb445ac0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4d54006010 tx=0x7f4d54003e80 comp rx=0 tx=0).stop 2024-05-14T09:45:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.102+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1606276709 shutdown_connections 2024-05-14T09:45:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.102+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1606276709 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5586bb0261c0 0x5586bb4b8ef0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.102+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1606276709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586bb0921b0 0x5586bb445ac0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.102+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1606276709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5586bb0714d0 0x5586bb449480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.102+0000 7f4d6b07e740 1 --2- 172.21.15.2:0/1606276709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586bb0692f0 0x5586bb448f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.102+0000 7f4d457fa640 0 client.16632 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:45:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.102+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1606276709 >> 172.21.15.2:0/1606276709 conn(0x5586bb086300 msgr2=0x5586bb4321f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.102+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1606276709 shutdown_connections 2024-05-14T09:45:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.102+0000 7f4d6b07e740 1 -- 172.21.15.2:0/1606276709 wait complete. 2024-05-14T09:45:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.102+0000 7f4d6b07e740 20 client.16632 trim_cache size 0 max 16384 2024-05-14T09:45:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:45:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:45:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:45:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:45:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.103+0000 7f4d641ce640 1 -- 172.21.15.2:0/575033020 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4d480b28a0 msgr2=0x7f4d480b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.103+0000 7f4d641ce640 1 --2- 172.21.15.2:0/575033020 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4d480b28a0 0x7f4d480b4db0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4d540094e0 tx=0x7f4d54031a20 comp rx=0 tx=0).stop 2024-05-14T09:45:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.103+0000 7f4d641ce640 1 -- 172.21.15.2:0/575033020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c102b50 msgr2=0x7f4d5c19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:45:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.103+0000 7f4d641ce640 1 --2- 172.21.15.2:0/575033020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c102b50 0x7f4d5c19a270 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4d58005d00 tx=0x7f4d5800dde0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.103+0000 7f4d641ce640 1 -- 172.21.15.2:0/575033020 shutdown_connections 2024-05-14T09:45:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.103+0000 7f4d641ce640 1 --2- 172.21.15.2:0/575033020 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4d480b28a0 0x7f4d480b4db0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.103+0000 7f4d641ce640 1 --2- 172.21.15.2:0/575033020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d5c108a90 0x7f4d5c1a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.103+0000 7f4d641ce640 1 --2- 172.21.15.2:0/575033020 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4d5c103d90 0x7f4d5c19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.103+0000 7f4d641ce640 1 --2- 172.21.15.2:0/575033020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c102b50 0x7f4d5c19a270 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:45:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.103+0000 7f4d641ce640 1 -- 172.21.15.2:0/575033020 >> 172.21.15.2:0/575033020 conn(0x7f4d5c0fdd40 msgr2=0x7f4d5c109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:45:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.103+0000 7f4d641ce640 1 -- 172.21.15.2:0/575033020 shutdown_connections 2024-05-14T09:45:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:45:51.103+0000 7f4d641ce640 1 -- 172.21.15.2:0/575033020 wait complete. 2024-05-14T09:45:51.280 INFO:tasks.workunit.client.0.smithi002.stdout: 1 974259 9.15 MB/sec execute 437 sec latency 2011.058 ms 2024-05-14T09:45:52.280 INFO:tasks.workunit.client.0.smithi002.stdout: 1 974259 9.13 MB/sec execute 438 sec latency 3011.129 ms 2024-05-14T09:45:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:52 smithi002 ceph-mon[31557]: pgmap v752: 129 pgs: 129 active+clean; 164 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-05-14T09:45:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:52 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:45:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:52 smithi029 ceph-mon[36249]: pgmap v752: 129 pgs: 129 active+clean; 164 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-05-14T09:45:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:52 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:45:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:52 smithi026 ceph-mon[36056]: pgmap v752: 129 pgs: 129 active+clean; 164 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-05-14T09:45:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:52 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:45:53.280 INFO:tasks.workunit.client.0.smithi002.stdout: 1 974259 9.11 MB/sec execute 439 sec latency 4011.224 ms 2024-05-14T09:45:53.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:45:53.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:45:53.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:45:53.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:53 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:45:53.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:53 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:45:53.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:53 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:45:53.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:53 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:45:53.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:53 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:45:53.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:53 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:45:54.280 INFO:tasks.workunit.client.0.smithi002.stdout: 1 974371 9.09 MB/sec execute 440 sec latency 4932.398 ms 2024-05-14T09:45:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:54 smithi002 ceph-mon[31557]: pgmap v753: 129 pgs: 129 active+clean; 164 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-05-14T09:45:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:54 smithi029 ceph-mon[36249]: pgmap v753: 129 pgs: 129 active+clean; 164 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-05-14T09:45:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:54 smithi026 ceph-mon[36056]: pgmap v753: 129 pgs: 129 active+clean; 164 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-05-14T09:45:55.280 INFO:tasks.workunit.client.0.smithi002.stdout: 1 976483 9.10 MB/sec execute 441 sec latency 12.157 ms 2024-05-14T09:45:56.280 INFO:tasks.workunit.client.0.smithi002.stdout: 1 977739 9.09 MB/sec execute 442 sec latency 52.973 ms 2024-05-14T09:45:56.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:56 smithi002 ceph-mon[31557]: pgmap v754: 129 pgs: 129 active+clean; 164 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 496 KiB/s wr, 71 op/s 2024-05-14T09:45:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:56 smithi029 ceph-mon[36249]: pgmap v754: 129 pgs: 129 active+clean; 164 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 496 KiB/s wr, 71 op/s 2024-05-14T09:45:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:56 smithi026 ceph-mon[36056]: pgmap v754: 129 pgs: 129 active+clean; 164 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 496 KiB/s wr, 71 op/s 2024-05-14T09:45:57.280 INFO:tasks.workunit.client.0.smithi002.stdout: 1 977739 9.07 MB/sec execute 443 sec latency 1053.036 ms 2024-05-14T09:45:58.280 INFO:tasks.workunit.client.0.smithi002.stdout: 1 977739 9.05 MB/sec execute 444 sec latency 2053.135 ms 2024-05-14T09:45:58.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:45:58 smithi002 ceph-mon[31557]: pgmap v755: 129 pgs: 129 active+clean; 165 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 570 KiB/s wr, 85 op/s 2024-05-14T09:45:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:45:58 smithi029 ceph-mon[36249]: pgmap v755: 129 pgs: 129 active+clean; 165 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 570 KiB/s wr, 85 op/s 2024-05-14T09:45:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:45:58 smithi026 ceph-mon[36056]: pgmap v755: 129 pgs: 129 active+clean; 165 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 570 KiB/s wr, 85 op/s 2024-05-14T09:45:59.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 977807 9.03 MB/sec execute 445 sec latency 2970.971 ms 2024-05-14T09:46:00.281 INFO:tasks.workunit.client.0.smithi002.stdout: 1 979916 9.04 MB/sec execute 446 sec latency 12.426 ms 2024-05-14T09:46:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:00 smithi002 ceph-mon[31557]: pgmap v756: 129 pgs: 129 active+clean; 166 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 421 KiB/s wr, 95 op/s 2024-05-14T09:46:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:00 smithi029 ceph-mon[36249]: pgmap v756: 129 pgs: 129 active+clean; 166 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 421 KiB/s wr, 95 op/s 2024-05-14T09:46:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:00 smithi026 ceph-mon[36056]: pgmap v756: 129 pgs: 129 active+clean; 166 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 421 KiB/s wr, 95 op/s 2024-05-14T09:46:01.281 INFO:tasks.workunit.client.0.smithi002.stdout: 1 980391 9.02 MB/sec execute 447 sec latency 756.210 ms 2024-05-14T09:46:02.281 INFO:tasks.workunit.client.0.smithi002.stdout: 1 980391 9.00 MB/sec execute 448 sec latency 1756.356 ms 2024-05-14T09:46:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:02 smithi002 ceph-mon[31557]: pgmap v757: 129 pgs: 129 active+clean; 166 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 255 KiB/s wr, 66 op/s 2024-05-14T09:46:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:02 smithi029 ceph-mon[36249]: pgmap v757: 129 pgs: 129 active+clean; 166 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 255 KiB/s wr, 66 op/s 2024-05-14T09:46:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:02 smithi026 ceph-mon[36056]: pgmap v757: 129 pgs: 129 active+clean; 166 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 255 KiB/s wr, 66 op/s 2024-05-14T09:46:03.281 INFO:tasks.workunit.client.0.smithi002.stdout: 1 980391 8.98 MB/sec execute 449 sec latency 2756.471 ms 2024-05-14T09:46:04.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.216+0000 7f438e022640 1 -- 172.21.15.2:0/1320692589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4388107000 msgr2=0x7f4388109420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:04.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.216+0000 7f438e022640 1 --2- 172.21.15.2:0/1320692589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4388107000 0x7f4388109420 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f437c009900 tx=0x7f437c02a410 comp rx=0 tx=0).stop 2024-05-14T09:46:04.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.216+0000 7f438e022640 1 -- 172.21.15.2:0/1320692589 shutdown_connections 2024-05-14T09:46:04.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.216+0000 7f438e022640 1 --2- 172.21.15.2:0/1320692589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4388109960 0x7f438810bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.216+0000 7f438e022640 1 --2- 172.21.15.2:0/1320692589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4388107000 0x7f4388109420 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.216+0000 7f438e022640 1 --2- 172.21.15.2:0/1320692589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4388101860 0x7f4388101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.216+0000 7f438e022640 1 -- 172.21.15.2:0/1320692589 >> 172.21.15.2:0/1320692589 conn(0x7f43880fd050 msgr2=0x7f43880ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:04.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.216+0000 7f438e022640 1 -- 172.21.15.2:0/1320692589 shutdown_connections 2024-05-14T09:46:04.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.216+0000 7f438e022640 1 -- 172.21.15.2:0/1320692589 wait complete. 2024-05-14T09:46:04.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.217+0000 7f438e022640 1 Processor -- start 2024-05-14T09:46:04.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.217+0000 7f438e022640 1 -- start start 2024-05-14T09:46:04.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.217+0000 7f438e022640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4388101860 0x7f438819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.217+0000 7f438e022640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4388107000 0x7f438819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f438e022640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4388109960 0x7f43881a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f438e022640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f438810f7a0 con 0x7f4388107000 2024-05-14T09:46:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f438e022640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f438810f620 con 0x7f4388101860 2024-05-14T09:46:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f438e022640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f438810f920 con 0x7f4388109960 2024-05-14T09:46:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f43877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4388101860 0x7f438819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f4386ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4388107000 0x7f438819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f43877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4388101860 0x7f438819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:49776/0 (socket says 172.21.15.2:49776) 2024-05-14T09:46:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f43877fe640 1 -- 172.21.15.2:0/3647083653 learned_addr learned my addr 172.21.15.2:0/3647083653 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f4386ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4388107000 0x7f438819a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50164/0 (socket says 172.21.15.2:50164) 2024-05-14T09:46:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f4387fff640 1 --2- 172.21.15.2:0/3647083653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4388109960 0x7f43881a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f43877fe640 1 -- 172.21.15.2:0/3647083653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4388109960 msgr2=0x7f43881a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f43877fe640 1 --2- 172.21.15.2:0/3647083653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4388109960 0x7f43881a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f43877fe640 1 -- 172.21.15.2:0/3647083653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4388107000 msgr2=0x7f438819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f43877fe640 1 --2- 172.21.15.2:0/3647083653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4388107000 0x7f438819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f43877fe640 1 -- 172.21.15.2:0/3647083653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43881a1df0 con 0x7f4388101860 2024-05-14T09:46:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f4386ffd640 1 --2- 172.21.15.2:0/3647083653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4388107000 0x7f438819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:46:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f43877fe640 1 --2- 172.21.15.2:0/3647083653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4388101860 0x7f438819a2b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f437800e9f0 tx=0x7f437800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f4384ff9640 1 -- 172.21.15.2:0/3647083653 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437800ccc0 con 0x7f4388101860 2024-05-14T09:46:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f4384ff9640 1 -- 172.21.15.2:0/3647083653 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43780054d0 con 0x7f4388101860 2024-05-14T09:46:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f4384ff9640 1 -- 172.21.15.2:0/3647083653 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4378010430 con 0x7f4388101860 2024-05-14T09:46:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.218+0000 7f438e022640 1 -- 172.21.15.2:0/3647083653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43881a20e0 con 0x7f4388101860 2024-05-14T09:46:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.219+0000 7f438e022640 1 -- 172.21.15.2:0/3647083653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43881a25b0 con 0x7f4388101860 2024-05-14T09:46:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.221+0000 7f4394fae740 1 Processor -- start 2024-05-14T09:46:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.221+0000 7f4384ff9640 1 -- 172.21.15.2:0/3647083653 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f43780105d0 con 0x7f4388101860 2024-05-14T09:46:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.221+0000 7f4394fae740 1 -- start start 2024-05-14T09:46:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4394fae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26e41b0 0x55a7f2a85a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4394fae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a85fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4394fae740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26c34d0 0x55a7f2a8cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4394fae740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a7f29ea5d0 con 0x55a7f26c34d0 2024-05-14T09:46:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4394fae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a7f29a3890 con 0x55a7f26bb2f0 2024-05-14T09:46:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4394fae740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a7f29e6590 con 0x55a7f26e41b0 2024-05-14T09:46:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4387fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26e41b0 0x55a7f2a85a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f43877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a85fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4386ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26c34d0 0x55a7f2a8cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4387fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26e41b0 0x55a7f2a85a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:44282/0 (socket says 172.21.15.2:44282) 2024-05-14T09:46:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4387fff640 1 -- 172.21.15.2:0/3146296959 learned_addr learned my addr 172.21.15.2:0/3146296959 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4387fff640 1 -- 172.21.15.2:0/3146296959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26bb2f0 msgr2=0x55a7f2a85fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4387fff640 1 --2- 172.21.15.2:0/3146296959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a85fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4387fff640 1 -- 172.21.15.2:0/3146296959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26c34d0 msgr2=0x55a7f2a8cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4387fff640 1 --2- 172.21.15.2:0/3146296959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26c34d0 0x55a7f2a8cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4387fff640 1 -- 172.21.15.2:0/3146296959 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f26fc210 con 0x55a7f26e41b0 2024-05-14T09:46:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4384ff9640 1 --2- 172.21.15.2:0/3647083653 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f43700b28a0 0x7f43700b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f4386ffd640 1 --2- 172.21.15.2:0/3146296959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26c34d0 0x55a7f2a8cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:46:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.222+0000 7f43877fe640 1 --2- 172.21.15.2:0/3146296959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a85fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:46:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.223+0000 7f4384ff9640 1 -- 172.21.15.2:0/3647083653 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f43780e29a0 con 0x7f4388101860 2024-05-14T09:46:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.223+0000 7f4387fff640 1 --2- 172.21.15.2:0/3647083653 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f43700b28a0 0x7f43700b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.223+0000 7f4387fff640 1 --2- 172.21.15.2:0/3146296959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26e41b0 0x55a7f2a85a60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4380009fd0 tx=0x7f43800118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=842a50365b4924b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.223+0000 7f436dffb640 1 -- 172.21.15.2:0/3146296959 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380004050 con 0x55a7f26e41b0 2024-05-14T09:46:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.223+0000 7f436dffb640 1 -- 172.21.15.2:0/3146296959 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4380004720 con 0x55a7f26e41b0 2024-05-14T09:46:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.223+0000 7f436dffb640 1 -- 172.21.15.2:0/3146296959 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380019400 con 0x55a7f26e41b0 2024-05-14T09:46:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.224+0000 7f4394fae740 1 -- 172.21.15.2:0/3146296959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26e41b0 msgr2=0x55a7f2a85a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.224+0000 7f4387fff640 1 --2- 172.21.15.2:0/3647083653 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f43700b28a0 0x7f43700b4db0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f438001d000 tx=0x7f4380019660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.224+0000 7f4394fae740 1 --2- 172.21.15.2:0/3146296959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26e41b0 0x55a7f2a85a60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4380009fd0 tx=0x7f43800118d0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.224+0000 7f4394fae740 1 -- 172.21.15.2:0/3146296959 shutdown_connections 2024-05-14T09:46:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.224+0000 7f4394fae740 1 --2- 172.21.15.2:0/3146296959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26c34d0 0x55a7f2a8cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.224+0000 7f4394fae740 1 --2- 172.21.15.2:0/3146296959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a85fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.224+0000 7f4394fae740 1 --2- 172.21.15.2:0/3146296959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26e41b0 0x55a7f2a85a60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.224+0000 7f4394fae740 1 -- 172.21.15.2:0/3146296959 >> 172.21.15.2:0/3146296959 conn(0x55a7f26d8300 msgr2=0x55a7f2a813a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.224+0000 7f4394fae740 1 -- 172.21.15.2:0/3146296959 shutdown_connections 2024-05-14T09:46:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.224+0000 7f4394fae740 1 -- 172.21.15.2:0/3146296959 wait complete. 2024-05-14T09:46:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.224+0000 7f4394fae740 1 -- start start 2024-05-14T09:46:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f4394fae740 1 Processor -- start 2024-05-14T09:46:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f4394fae740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a8f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f4394fae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26c34d0 0x55a7f2a8fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f4394fae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26e41b0 0x55a7f2a96ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f4394fae740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a7f2a8d930 con 0x55a7f26bb2f0 2024-05-14T09:46:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f4394fae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a7f29a7370 con 0x55a7f26e41b0 2024-05-14T09:46:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f4394fae740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a7f2a8dab0 con 0x55a7f26c34d0 2024-05-14T09:46:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f4394fae740 10 client.0 _ll_register_callbacks cb 0x55a7f26fba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:46:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f43877fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a8f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f43877fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a8f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50182/0 (socket says 172.21.15.2:50182) 2024-05-14T09:46:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f43877fe640 1 -- 172.21.15.2:0/246509021 learned_addr learned my addr 172.21.15.2:0/246509021 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f4386ffd640 1 --2- 172.21.15.2:0/246509021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26c34d0 0x55a7f2a8fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f4387fff640 1 --2- 172.21.15.2:0/246509021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26e41b0 0x55a7f2a96ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f43877fe640 1 -- 172.21.15.2:0/246509021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26c34d0 msgr2=0x55a7f2a8fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f43877fe640 1 --2- 172.21.15.2:0/246509021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26c34d0 0x55a7f2a8fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f43877fe640 1 -- 172.21.15.2:0/246509021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26e41b0 msgr2=0x55a7f2a96ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f43877fe640 1 --2- 172.21.15.2:0/246509021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26e41b0 0x55a7f2a96ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f43877fe640 1 -- 172.21.15.2:0/246509021 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f27c8620 con 0x55a7f26bb2f0 2024-05-14T09:46:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f43877fe640 1 --2- 172.21.15.2:0/246509021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a8f620 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f43780e0080 tx=0x7f43780a5a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0c6fca0ed00f28 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.225+0000 7f4366ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:46:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.226+0000 7f4366ffd640 1 -- 172.21.15.2:0/246509021 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43780a9310 con 0x55a7f26bb2f0 2024-05-14T09:46:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.226+0000 7f4366ffd640 1 -- 172.21.15.2:0/246509021 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f43780b2780 con 0x55a7f26bb2f0 2024-05-14T09:46:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.226+0000 7f4394fae740 1 -- 172.21.15.2:0/246509021 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4380004720 con 0x55a7f26bb2f0 2024-05-14T09:46:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.226+0000 7f4366ffd640 1 -- 172.21.15.2:0/246509021 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f43780a5330 con 0x55a7f26bb2f0 2024-05-14T09:46:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.226+0000 7f4394fae740 10 client.16644 fetch_fsmap learned FSMap version 44 2024-05-14T09:46:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.226+0000 7f4394fae740 1 -- 172.21.15.2:0/246509021 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a7f2a99ac0 con 0x55a7f26bb2f0 2024-05-14T09:46:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.226+0000 7f4366ffd640 1 -- 172.21.15.2:0/246509021 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4380004720 con 0x55a7f26bb2f0 2024-05-14T09:46:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.226+0000 7f4366ffd640 1 -- 172.21.15.2:0/246509021 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f4378005000 con 0x55a7f26bb2f0 2024-05-14T09:46:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.226+0000 7f4394fae740 10 client.16644 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T09:46:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.226+0000 7f4394fae740 10 client.16644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:46:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.226+0000 7f4394fae740 20 client.16644 populate_metadata read hostname 'smithi002' 2024-05-14T09:46:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.226+0000 7f4394fae740 1 --2- 172.21.15.2:0/246509021 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a7f26a2d80 0x55a7f2a9d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.226+0000 7f4394fae740 4 client.16644 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:46:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.226+0000 7f4394fae740 1 -- 172.21.15.2:0/246509021 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7f29f81e0 con 0x55a7f26a2d80 2024-05-14T09:46:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.227+0000 7f4386ffd640 1 --2- 172.21.15.2:0/246509021 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a7f26a2d80 0x55a7f2a9d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.227+0000 7f4366ffd640 10 client.16644 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.227+0000 7f4386ffd640 1 --2- 172.21.15.2:0/246509021 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a7f26a2d80 0x55a7f2a9d1b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d6f140cb9608dbb server_cookie=4ff7a31ad60d4f6e in_seq=0 out_seq=0 2024-05-14T09:46:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.228+0000 7f4366ffd640 1 -- 172.21.15.2:0/246509021 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a7f29f81e0 con 0x55a7f26a2d80 2024-05-14T09:46:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.228+0000 7f4366ffd640 10 client.16644 handle_command_reply: tid=0 2024-05-14T09:46:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.228+0000 7f4394fae740 1 client.16644 shutdown 2024-05-14T09:46:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.228+0000 7f4394fae740 1 -- 172.21.15.2:0/246509021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26bb2f0 msgr2=0x55a7f2a8f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.228+0000 7f4394fae740 1 --2- 172.21.15.2:0/246509021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a8f620 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f43780e0080 tx=0x7f43780a5a60 comp rx=0 tx=0).stop 2024-05-14T09:46:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.228+0000 7f4394fae740 1 -- 172.21.15.2:0/246509021 shutdown_connections 2024-05-14T09:46:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.228+0000 7f4394fae740 1 --2- 172.21.15.2:0/246509021 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a7f26a2d80 0x55a7f2a9d1b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.228+0000 7f4394fae740 1 --2- 172.21.15.2:0/246509021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26e41b0 0x55a7f2a96ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.228+0000 7f4394fae740 1 --2- 172.21.15.2:0/246509021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26c34d0 0x55a7f2a8fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.228+0000 7f4394fae740 1 --2- 172.21.15.2:0/246509021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a8f620 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.228+0000 7f4366ffd640 0 client.16644 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.228+0000 7f4394fae740 1 -- 172.21.15.2:0/246509021 >> 172.21.15.2:0/246509021 conn(0x55a7f26d8300 msgr2=0x55a7f2a84300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.228+0000 7f4394fae740 1 -- 172.21.15.2:0/246509021 shutdown_connections 2024-05-14T09:46:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.228+0000 7f4394fae740 1 -- 172.21.15.2:0/246509021 wait complete. 2024-05-14T09:46:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.228+0000 7f4394fae740 20 client.16644 trim_cache size 0 max 16384 2024-05-14T09:46:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f4394fae740 1 Processor -- start 2024-05-14T09:46:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f4394fae740 1 -- start start 2024-05-14T09:46:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f4394fae740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a841f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f4394fae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26c34d0 0x55a7f2a84730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f4394fae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26e41b0 0x55a7f2a8f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f4394fae740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f43881af450 con 0x55a7f26bb2f0 2024-05-14T09:46:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f4394fae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f43881acc90 con 0x55a7f26e41b0 2024-05-14T09:46:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f4394fae740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f43881aaae0 con 0x55a7f26c34d0 2024-05-14T09:46:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f43877fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a841f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f43877fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a841f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50190/0 (socket says 172.21.15.2:50190) 2024-05-14T09:46:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f4386ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26c34d0 0x55a7f2a84730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f43877fe640 1 -- 172.21.15.2:0/1461650377 learned_addr learned my addr 172.21.15.2:0/1461650377 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f4387fff640 1 --2- 172.21.15.2:0/1461650377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26e41b0 0x55a7f2a8f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f4386ffd640 1 -- 172.21.15.2:0/1461650377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26e41b0 msgr2=0x55a7f2a8f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f4386ffd640 1 --2- 172.21.15.2:0/1461650377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26e41b0 0x55a7f2a8f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f4386ffd640 1 -- 172.21.15.2:0/1461650377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26bb2f0 msgr2=0x55a7f2a841f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.245+0000 7f4386ffd640 1 --2- 172.21.15.2:0/1461650377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a841f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.246+0000 7f4386ffd640 1 -- 172.21.15.2:0/1461650377 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f2a81d00 con 0x55a7f26c34d0 2024-05-14T09:46:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.246+0000 7f43877fe640 1 --2- 172.21.15.2:0/1461650377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a841f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:46:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.246+0000 7f4386ffd640 1 --2- 172.21.15.2:0/1461650377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26c34d0 0x55a7f2a84730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f437c02aa00 tx=0x7f437c0038e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad352bd1c6cd9dea server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.246+0000 7f436d7fa640 1 -- 172.21.15.2:0/1461650377 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c035040 con 0x55a7f26c34d0 2024-05-14T09:46:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.246+0000 7f436d7fa640 1 -- 172.21.15.2:0/1461650377 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f437c048e50 con 0x55a7f26c34d0 2024-05-14T09:46:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.246+0000 7f436d7fa640 1 -- 172.21.15.2:0/1461650377 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c0317e0 con 0x55a7f26c34d0 2024-05-14T09:46:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.247+0000 7f4394fae740 1 -- 172.21.15.2:0/1461650377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26c34d0 msgr2=0x55a7f2a84730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.247+0000 7f4394fae740 1 --2- 172.21.15.2:0/1461650377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26c34d0 0x55a7f2a84730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f437c02aa00 tx=0x7f437c0038e0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.247+0000 7f4394fae740 1 -- 172.21.15.2:0/1461650377 shutdown_connections 2024-05-14T09:46:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.247+0000 7f4394fae740 1 --2- 172.21.15.2:0/1461650377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26e41b0 0x55a7f2a8f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.247+0000 7f4394fae740 1 --2- 172.21.15.2:0/1461650377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26c34d0 0x55a7f2a84730 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.247+0000 7f4394fae740 1 --2- 172.21.15.2:0/1461650377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a841f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.247+0000 7f4394fae740 1 -- 172.21.15.2:0/1461650377 >> 172.21.15.2:0/1461650377 conn(0x55a7f26d8300 msgr2=0x55a7f2a89a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.247+0000 7f4394fae740 1 -- 172.21.15.2:0/1461650377 shutdown_connections 2024-05-14T09:46:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.247+0000 7f4394fae740 1 -- 172.21.15.2:0/1461650377 wait complete. 2024-05-14T09:46:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.248+0000 7f4394fae740 1 -- start start 2024-05-14T09:46:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.248+0000 7f4394fae740 1 Processor -- start 2024-05-14T09:46:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.248+0000 7f4394fae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a934e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.248+0000 7f4394fae740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26c34d0 0x55a7f2a93a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.248+0000 7f4394fae740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26e41b0 0x55a7f2a94900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.248+0000 7f4394fae740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f43881ab150 con 0x55a7f26c34d0 2024-05-14T09:46:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.248+0000 7f4394fae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f43881ac5c0 con 0x55a7f26bb2f0 2024-05-14T09:46:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.248+0000 7f4394fae740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f43881ab820 con 0x55a7f26e41b0 2024-05-14T09:46:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.248+0000 7f4394fae740 10 client.0 _ll_register_callbacks cb 0x7f43780e8240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:46:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.248+0000 7f4386ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26c34d0 0x55a7f2a93a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.248+0000 7f4386ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26c34d0 0x55a7f2a93a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50192/0 (socket says 172.21.15.2:50192) 2024-05-14T09:46:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.248+0000 7f43877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a934e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.248+0000 7f4386ffd640 1 -- 172.21.15.2:0/1011684442 learned_addr learned my addr 172.21.15.2:0/1011684442 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.248+0000 7f4387fff640 1 --2- 172.21.15.2:0/1011684442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26e41b0 0x55a7f2a94900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.249+0000 7f4386ffd640 1 -- 172.21.15.2:0/1011684442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26e41b0 msgr2=0x55a7f2a94900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.249+0000 7f4386ffd640 1 --2- 172.21.15.2:0/1011684442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26e41b0 0x55a7f2a94900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.249+0000 7f4386ffd640 1 -- 172.21.15.2:0/1011684442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26bb2f0 msgr2=0x55a7f2a934e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.249+0000 7f4386ffd640 1 --2- 172.21.15.2:0/1011684442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a934e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.249+0000 7f4386ffd640 1 -- 172.21.15.2:0/1011684442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f2a94ec0 con 0x55a7f26c34d0 2024-05-14T09:46:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.249+0000 7f43877fe640 1 --2- 172.21.15.2:0/1011684442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a934e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:46:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.249+0000 7f4386ffd640 1 --2- 172.21.15.2:0/1011684442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26c34d0 0x55a7f2a93a20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f437c009850 tx=0x7f437c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcbf5e70fce0178 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.249+0000 7f4366ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:46:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.249+0000 7f4366ffd640 1 -- 172.21.15.2:0/1011684442 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c035050 con 0x55a7f26c34d0 2024-05-14T09:46:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.249+0000 7f4394fae740 1 -- 172.21.15.2:0/1011684442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f437c048e50 con 0x55a7f26c34d0 2024-05-14T09:46:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.249+0000 7f4366ffd640 1 -- 172.21.15.2:0/1011684442 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f437c048680 con 0x55a7f26c34d0 2024-05-14T09:46:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.249+0000 7f4366ffd640 1 -- 172.21.15.2:0/1011684442 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f437c062050 con 0x55a7f26c34d0 2024-05-14T09:46:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.249+0000 7f4394fae740 10 client.16650 fetch_fsmap learned FSMap version 44 2024-05-14T09:46:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.249+0000 7f4394fae740 1 -- 172.21.15.2:0/1011684442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a7f2a96180 con 0x55a7f26c34d0 2024-05-14T09:46:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.250+0000 7f4366ffd640 1 -- 172.21.15.2:0/1011684442 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f437c048e50 con 0x55a7f26c34d0 2024-05-14T09:46:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.250+0000 7f4366ffd640 1 -- 172.21.15.2:0/1011684442 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f437c03c440 con 0x55a7f26c34d0 2024-05-14T09:46:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.250+0000 7f4394fae740 10 client.16650 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T09:46:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.250+0000 7f4394fae740 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:46:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.250+0000 7f4394fae740 20 client.16650 populate_metadata read hostname 'smithi002' 2024-05-14T09:46:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.250+0000 7f4394fae740 1 --2- 172.21.15.2:0/1011684442 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a7f26781c0 0x55a7f2b07a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.250+0000 7f4394fae740 4 client.16650 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:46:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.250+0000 7f4394fae740 1 -- 172.21.15.2:0/1011684442 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7f29f2050 con 0x55a7f26781c0 2024-05-14T09:46:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.250+0000 7f43877fe640 1 --2- 172.21.15.2:0/1011684442 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a7f26781c0 0x55a7f2b07a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.250+0000 7f4366ffd640 10 client.16650 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.250+0000 7f43877fe640 1 --2- 172.21.15.2:0/1011684442 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a7f26781c0 0x55a7f2b07a30 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2796a4408799280d server_cookie=60a839ef8647f91f in_seq=0 out_seq=0 2024-05-14T09:46:04.281 INFO:tasks.workunit.client.0.smithi002.stdout: 1 980953 8.97 MB/sec execute 450 sec latency 3215.961 ms 2024-05-14T09:46:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.312+0000 7f4366ffd640 1 -- 172.21.15.2:0/1011684442 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7f29f2050 con 0x55a7f26781c0 2024-05-14T09:46:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.312+0000 7f4366ffd640 10 client.16650 handle_command_reply: tid=0 2024-05-14T09:46:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.312+0000 7f4394fae740 1 client.16650 shutdown 2024-05-14T09:46:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.313+0000 7f4394fae740 1 -- 172.21.15.2:0/1011684442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26c34d0 msgr2=0x55a7f2a93a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.313+0000 7f4394fae740 1 --2- 172.21.15.2:0/1011684442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26c34d0 0x55a7f2a93a20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f437c009850 tx=0x7f437c000bc0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.313+0000 7f4394fae740 1 -- 172.21.15.2:0/1011684442 shutdown_connections 2024-05-14T09:46:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.313+0000 7f4394fae740 1 --2- 172.21.15.2:0/1011684442 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a7f26781c0 0x55a7f2b07a30 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.313+0000 7f4394fae740 1 --2- 172.21.15.2:0/1011684442 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a7f26e41b0 0x55a7f2a94900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.313+0000 7f4394fae740 1 --2- 172.21.15.2:0/1011684442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f26c34d0 0x55a7f2a93a20 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.313+0000 7f4394fae740 1 --2- 172.21.15.2:0/1011684442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7f26bb2f0 0x55a7f2a934e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.313+0000 7f4366ffd640 0 client.16650 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.313+0000 7f4394fae740 1 -- 172.21.15.2:0/1011684442 >> 172.21.15.2:0/1011684442 conn(0x55a7f26d8300 msgr2=0x55a7f2a841f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.313+0000 7f4394fae740 1 -- 172.21.15.2:0/1011684442 shutdown_connections 2024-05-14T09:46:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.313+0000 7f4394fae740 1 -- 172.21.15.2:0/1011684442 wait complete. 2024-05-14T09:46:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.313+0000 7f4394fae740 20 client.16650 trim_cache size 0 max 16384 2024-05-14T09:46:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:46:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:46:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:46:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:46:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.314+0000 7f438e022640 1 -- 172.21.15.2:0/3647083653 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f43700b28a0 msgr2=0x7f43700b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.314+0000 7f438e022640 1 --2- 172.21.15.2:0/3647083653 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f43700b28a0 0x7f43700b4db0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f438001d000 tx=0x7f4380019660 comp rx=0 tx=0).stop 2024-05-14T09:46:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.314+0000 7f438e022640 1 -- 172.21.15.2:0/3647083653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4388101860 msgr2=0x7f438819a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.314+0000 7f438e022640 1 --2- 172.21.15.2:0/3647083653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4388101860 0x7f438819a2b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f437800e9f0 tx=0x7f437800eeb0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.314+0000 7f438e022640 1 -- 172.21.15.2:0/3647083653 shutdown_connections 2024-05-14T09:46:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.314+0000 7f438e022640 1 --2- 172.21.15.2:0/3647083653 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f43700b28a0 0x7f43700b4db0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.314+0000 7f438e022640 1 --2- 172.21.15.2:0/3647083653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4388109960 0x7f43881a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.314+0000 7f438e022640 1 --2- 172.21.15.2:0/3647083653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4388107000 0x7f438819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.314+0000 7f438e022640 1 --2- 172.21.15.2:0/3647083653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4388101860 0x7f438819a2b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.314+0000 7f438e022640 1 -- 172.21.15.2:0/3647083653 >> 172.21.15.2:0/3647083653 conn(0x7f43880fd050 msgr2=0x7f4388107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.314+0000 7f438e022640 1 -- 172.21.15.2:0/3647083653 shutdown_connections 2024-05-14T09:46:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:04.314+0000 7f438e022640 1 -- 172.21.15.2:0/3647083653 wait complete. 2024-05-14T09:46:04.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:04 smithi002 ceph-mon[31557]: pgmap v758: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 336 KiB/s wr, 90 op/s 2024-05-14T09:46:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:04 smithi029 ceph-mon[36249]: pgmap v758: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 336 KiB/s wr, 90 op/s 2024-05-14T09:46:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:04 smithi026 ceph-mon[36056]: pgmap v758: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 336 KiB/s wr, 90 op/s 2024-05-14T09:46:05.281 INFO:tasks.workunit.client.0.smithi002.stdout: 1 981224 8.95 MB/sec execute 451 sec latency 676.764 ms 2024-05-14T09:46:06.281 INFO:tasks.workunit.client.0.smithi002.stdout: 1 981224 8.93 MB/sec execute 452 sec latency 1676.865 ms 2024-05-14T09:46:06.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:06 smithi002 ceph-mon[31557]: pgmap v759: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 310 KiB/s wr, 75 op/s 2024-05-14T09:46:06.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:06 smithi029 ceph-mon[36249]: pgmap v759: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 310 KiB/s wr, 75 op/s 2024-05-14T09:46:06.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:06 smithi026 ceph-mon[36056]: pgmap v759: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 310 KiB/s wr, 75 op/s 2024-05-14T09:46:07.281 INFO:tasks.workunit.client.0.smithi002.stdout: 1 981224 8.91 MB/sec execute 453 sec latency 2676.918 ms 2024-05-14T09:46:08.281 INFO:tasks.workunit.client.0.smithi002.stdout: 1 981224 8.89 MB/sec execute 454 sec latency 3677.014 ms 2024-05-14T09:46:08.754 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:08 smithi026 ceph-mon[36056]: pgmap v760: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 339 KiB/s wr, 92 op/s 2024-05-14T09:46:08.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:08 smithi002 ceph-mon[31557]: pgmap v760: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 339 KiB/s wr, 92 op/s 2024-05-14T09:46:08.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:08 smithi029 ceph-mon[36249]: pgmap v760: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 339 KiB/s wr, 92 op/s 2024-05-14T09:46:09.281 INFO:tasks.workunit.client.0.smithi002.stdout: 1 981365 8.88 MB/sec execute 455 sec latency 4276.467 ms 2024-05-14T09:46:10.281 INFO:tasks.workunit.client.0.smithi002.stdout: 1 983117 8.87 MB/sec execute 456 sec latency 29.552 ms 2024-05-14T09:46:10.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:10 smithi002 ceph-mon[31557]: pgmap v761: 129 pgs: 129 active+clean; 163 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 287 KiB/s wr, 90 op/s 2024-05-14T09:46:10.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:10 smithi029 ceph-mon[36249]: pgmap v761: 129 pgs: 129 active+clean; 163 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 287 KiB/s wr, 90 op/s 2024-05-14T09:46:10.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:10 smithi026 ceph-mon[36056]: pgmap v761: 129 pgs: 129 active+clean; 163 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 287 KiB/s wr, 90 op/s 2024-05-14T09:46:11.281 INFO:tasks.workunit.client.0.smithi002.stdout: 1 984571 8.88 MB/sec execute 457 sec latency 24.882 ms 2024-05-14T09:46:12.281 INFO:tasks.workunit.client.0.smithi002.stdout: 1 984831 8.86 MB/sec execute 458 sec latency 778.356 ms 2024-05-14T09:46:12.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:12 smithi002 ceph-mon[31557]: pgmap v762: 129 pgs: 129 active+clean; 164 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 460 KiB/s wr, 87 op/s 2024-05-14T09:46:12.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:12 smithi029 ceph-mon[36249]: pgmap v762: 129 pgs: 129 active+clean; 164 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 460 KiB/s wr, 87 op/s 2024-05-14T09:46:12.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:12 smithi026 ceph-mon[36056]: pgmap v762: 129 pgs: 129 active+clean; 164 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 460 KiB/s wr, 87 op/s 2024-05-14T09:46:13.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 984831 8.84 MB/sec execute 459 sec latency 1778.404 ms 2024-05-14T09:46:14.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 984899 8.82 MB/sec execute 460 sec latency 2696.691 ms 2024-05-14T09:46:14.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:14 smithi002 ceph-mon[31557]: pgmap v763: 129 pgs: 129 active+clean; 163 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-05-14T09:46:14.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:14 smithi029 ceph-mon[36249]: pgmap v763: 129 pgs: 129 active+clean; 163 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-05-14T09:46:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:14 smithi026 ceph-mon[36056]: pgmap v763: 129 pgs: 129 active+clean; 163 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-05-14T09:46:15.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 984899 8.80 MB/sec execute 461 sec latency 1009.661 ms 2024-05-14T09:46:16.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 984899 8.78 MB/sec execute 462 sec latency 2009.698 ms 2024-05-14T09:46:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:16 smithi002 ceph-mon[31557]: pgmap v764: 129 pgs: 129 active+clean; 163 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-05-14T09:46:16.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:16 smithi029 ceph-mon[36249]: pgmap v764: 129 pgs: 129 active+clean; 163 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-05-14T09:46:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:16 smithi026 ceph-mon[36056]: pgmap v764: 129 pgs: 129 active+clean; 163 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-05-14T09:46:17.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 984899 8.76 MB/sec execute 463 sec latency 3009.800 ms 2024-05-14T09:46:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.429+0000 7f196b98d640 1 -- 172.21.15.2:0/1038062922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19641037b0 msgr2=0x7f1964103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.429+0000 7f196b98d640 1 --2- 172.21.15.2:0/1038062922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19641037b0 0x7f1964103bc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f195c009cd0 tx=0x7f195c02a8e0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.429+0000 7f196b98d640 1 -- 172.21.15.2:0/1038062922 shutdown_connections 2024-05-14T09:46:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.429+0000 7f196b98d640 1 --2- 172.21.15.2:0/1038062922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19641037b0 0x7f1964103bc0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.429+0000 7f196b98d640 1 --2- 172.21.15.2:0/1038062922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1964102e60 0x7f1964103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.429+0000 7f196b98d640 1 --2- 172.21.15.2:0/1038062922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1964102510 0x7f1964102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.429+0000 7f196b98d640 1 -- 172.21.15.2:0/1038062922 >> 172.21.15.2:0/1038062922 conn(0x7f19640fdd20 msgr2=0x7f19641001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.429+0000 7f196b98d640 1 -- 172.21.15.2:0/1038062922 shutdown_connections 2024-05-14T09:46:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.429+0000 7f196b98d640 1 -- 172.21.15.2:0/1038062922 wait complete. 2024-05-14T09:46:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f196b98d640 1 Processor -- start 2024-05-14T09:46:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f196b98d640 1 -- start start 2024-05-14T09:46:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f196b98d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1964102510 0x7f196419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f196b98d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1964102e60 0x7f196419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f196b98d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19641037b0 0x7f19641a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f196b98d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f19641119c0 con 0x7f19641037b0 2024-05-14T09:46:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f196b98d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1964111840 con 0x7f1964102510 2024-05-14T09:46:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f196b98d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1964111b40 con 0x7f1964102e60 2024-05-14T09:46:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f1969f2a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19641037b0 0x7f19641a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f1969f2a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19641037b0 0x7f19641a3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53478/0 (socket says 172.21.15.2:53478) 2024-05-14T09:46:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f1969f2a640 1 -- 172.21.15.2:0/3704266345 learned_addr learned my addr 172.21.15.2:0/3704266345 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f1968f28640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1964102e60 0x7f196419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f1969729640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1964102510 0x7f196419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f1969f2a640 1 -- 172.21.15.2:0/3704266345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1964102e60 msgr2=0x7f196419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f1969f2a640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1964102e60 0x7f196419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f1969f2a640 1 -- 172.21.15.2:0/3704266345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1964102510 msgr2=0x7f196419c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f1969f2a640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1964102510 0x7f196419c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f1969f2a640 1 -- 172.21.15.2:0/3704266345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19641a4070 con 0x7f19641037b0 2024-05-14T09:46:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f1968f28640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1964102e60 0x7f196419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:46:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.430+0000 7f1969729640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1964102510 0x7f196419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:46:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.431+0000 7f1969f2a640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19641037b0 0x7f19641a3a00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f195c005e30 tx=0x7f195c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.431+0000 7f195a7fc640 1 -- 172.21.15.2:0/3704266345 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c004240 con 0x7f19641037b0 2024-05-14T09:46:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.431+0000 7f195a7fc640 1 -- 172.21.15.2:0/3704266345 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195c0305a0 con 0x7f19641037b0 2024-05-14T09:46:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.431+0000 7f196b98d640 1 -- 172.21.15.2:0/3704266345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19641a4300 con 0x7f19641037b0 2024-05-14T09:46:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.431+0000 7f195a7fc640 1 -- 172.21.15.2:0/3704266345 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c0466c0 con 0x7f19641037b0 2024-05-14T09:46:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.431+0000 7f196b98d640 1 -- 172.21.15.2:0/3704266345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19641a48d0 con 0x7f19641037b0 2024-05-14T09:46:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.435+0000 7f197284a740 1 Processor -- start 2024-05-14T09:46:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.435+0000 7f195a7fc640 1 -- 172.21.15.2:0/3704266345 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f195c030a50 con 0x7f19641037b0 2024-05-14T09:46:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.435+0000 7f197284a740 1 -- start start 2024-05-14T09:46:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.435+0000 7f197284a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea967d1b0 0x555ea9a1ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.435+0000 7f197284a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea96542f0 0x555ea9a1efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.435+0000 7f197284a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea965c4d0 0x555ea9a25ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.435+0000 7f197284a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555ea99835d0 con 0x555ea967d1b0 2024-05-14T09:46:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.435+0000 7f197284a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555ea993c890 con 0x555ea965c4d0 2024-05-14T09:46:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.435+0000 7f197284a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555ea997f590 con 0x555ea96542f0 2024-05-14T09:46:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.435+0000 7f1968f28640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea967d1b0 0x555ea9a1ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.435+0000 7f1968f28640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea967d1b0 0x555ea9a1ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53480/0 (socket says 172.21.15.2:53480) 2024-05-14T09:46:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.435+0000 7f1968f28640 1 -- 172.21.15.2:0/2439853322 learned_addr learned my addr 172.21.15.2:0/2439853322 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.435+0000 7f1969729640 1 --2- 172.21.15.2:0/2439853322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea96542f0 0x555ea9a1efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.436+0000 7f1968f28640 1 --2- 172.21.15.2:0/2439853322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea965c4d0 0x555ea9a25ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.436+0000 7f1968f28640 1 -- 172.21.15.2:0/2439853322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea96542f0 msgr2=0x555ea9a1efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.436+0000 7f1968f28640 1 --2- 172.21.15.2:0/2439853322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea96542f0 0x555ea9a1efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.436+0000 7f1968f28640 1 -- 172.21.15.2:0/2439853322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea965c4d0 msgr2=0x555ea9a25ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.436+0000 7f1968f28640 1 --2- 172.21.15.2:0/2439853322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea965c4d0 0x555ea9a25ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.436+0000 7f1968f28640 1 -- 172.21.15.2:0/2439853322 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea9695210 con 0x555ea967d1b0 2024-05-14T09:46:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.436+0000 7f195a7fc640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f19500b28a0 0x7f19500b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.436+0000 7f1969729640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f19500b28a0 0x7f19500b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.436+0000 7f1968f28640 1 --2- 172.21.15.2:0/2439853322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea967d1b0 0x555ea9a1ea60 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f195400bfa0 tx=0x7f195400b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b76c46976d7834a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.436+0000 7f195a7fc640 1 -- 172.21.15.2:0/3704266345 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f195c02e030 con 0x7f19641037b0 2024-05-14T09:46:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.436+0000 7f194f7fe640 1 -- 172.21.15.2:0/2439853322 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19540110d0 con 0x555ea967d1b0 2024-05-14T09:46:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.436+0000 7f194f7fe640 1 -- 172.21.15.2:0/2439853322 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1954017020 con 0x555ea967d1b0 2024-05-14T09:46:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.436+0000 7f194f7fe640 1 -- 172.21.15.2:0/2439853322 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954003cd0 con 0x555ea967d1b0 2024-05-14T09:46:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.437+0000 7f197284a740 1 -- 172.21.15.2:0/2439853322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea967d1b0 msgr2=0x555ea9a1ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.437+0000 7f1969729640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f19500b28a0 0x7f19500b4db0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x555ea9a1fe40 tx=0x7f1960008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.437+0000 7f197284a740 1 --2- 172.21.15.2:0/2439853322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea967d1b0 0x555ea9a1ea60 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f195400bfa0 tx=0x7f195400b750 comp rx=0 tx=0).stop 2024-05-14T09:46:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.437+0000 7f197284a740 1 -- 172.21.15.2:0/2439853322 shutdown_connections 2024-05-14T09:46:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.437+0000 7f197284a740 1 --2- 172.21.15.2:0/2439853322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea965c4d0 0x555ea9a25ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.437+0000 7f197284a740 1 --2- 172.21.15.2:0/2439853322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea96542f0 0x555ea9a1efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.437+0000 7f197284a740 1 --2- 172.21.15.2:0/2439853322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea967d1b0 0x555ea9a1ea60 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.437+0000 7f197284a740 1 -- 172.21.15.2:0/2439853322 >> 172.21.15.2:0/2439853322 conn(0x555ea9671300 msgr2=0x555ea9a1a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.437+0000 7f197284a740 1 -- 172.21.15.2:0/2439853322 shutdown_connections 2024-05-14T09:46:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.437+0000 7f197284a740 1 -- 172.21.15.2:0/2439853322 wait complete. 2024-05-14T09:46:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.438+0000 7f197284a740 1 -- start start 2024-05-14T09:46:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.438+0000 7f197284a740 1 Processor -- start 2024-05-14T09:46:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.438+0000 7f197284a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea96542f0 0x555ea9a287c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.438+0000 7f197284a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea965c4d0 0x555ea9a28d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.438+0000 7f197284a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea967d1b0 0x555ea9a2fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.438+0000 7f197284a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555ea9a26930 con 0x555ea96542f0 2024-05-14T09:46:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.438+0000 7f197284a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555ea9940370 con 0x555ea967d1b0 2024-05-14T09:46:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.438+0000 7f197284a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555ea9a26ab0 con 0x555ea965c4d0 2024-05-14T09:46:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.438+0000 7f197284a740 10 client.0 _ll_register_callbacks cb 0x555ea9694a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:46:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f1969729640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea96542f0 0x555ea9a287c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f1969729640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea96542f0 0x555ea9a287c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53496/0 (socket says 172.21.15.2:53496) 2024-05-14T09:46:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f1969729640 1 -- 172.21.15.2:0/1980793669 learned_addr learned my addr 172.21.15.2:0/1980793669 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f1968f28640 1 --2- 172.21.15.2:0/1980793669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea965c4d0 0x555ea9a28d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f1969f2a640 1 --2- 172.21.15.2:0/1980793669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea967d1b0 0x555ea9a2fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f1969729640 1 -- 172.21.15.2:0/1980793669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea965c4d0 msgr2=0x555ea9a28d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f1969729640 1 --2- 172.21.15.2:0/1980793669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea965c4d0 0x555ea9a28d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f1969729640 1 -- 172.21.15.2:0/1980793669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea967d1b0 msgr2=0x555ea9a2fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f1969729640 1 --2- 172.21.15.2:0/1980793669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea967d1b0 0x555ea9a2fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f1969729640 1 -- 172.21.15.2:0/1980793669 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea9761620 con 0x555ea96542f0 2024-05-14T09:46:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f1969729640 1 --2- 172.21.15.2:0/1980793669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea96542f0 0x555ea9a287c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f19600167b0 tx=0x7f19600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a239b5e86cd03c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f194cff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:46:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f194cff9640 1 -- 172.21.15.2:0/1980793669 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196001c050 con 0x555ea96542f0 2024-05-14T09:46:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f194cff9640 1 -- 172.21.15.2:0/1980793669 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f1960014930 con 0x555ea96542f0 2024-05-14T09:46:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f197284a740 1 -- 172.21.15.2:0/1980793669 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1954017020 con 0x555ea96542f0 2024-05-14T09:46:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f194cff9640 1 -- 172.21.15.2:0/1980793669 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1960014b30 con 0x555ea96542f0 2024-05-14T09:46:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f197284a740 10 client.16662 fetch_fsmap learned FSMap version 44 2024-05-14T09:46:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.439+0000 7f197284a740 1 -- 172.21.15.2:0/1980793669 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555ea9a32bf0 con 0x555ea96542f0 2024-05-14T09:46:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.440+0000 7f194cff9640 1 -- 172.21.15.2:0/1980793669 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1954017020 con 0x555ea96542f0 2024-05-14T09:46:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.440+0000 7f194cff9640 1 -- 172.21.15.2:0/1980793669 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f1960006430 con 0x555ea96542f0 2024-05-14T09:46:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.440+0000 7f197284a740 10 client.16662 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T09:46:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.440+0000 7f197284a740 10 client.16662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:46:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.440+0000 7f197284a740 20 client.16662 populate_metadata read hostname 'smithi002' 2024-05-14T09:46:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.440+0000 7f197284a740 1 --2- 172.21.15.2:0/1980793669 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555ea963bd80 0x555ea9a363f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.440+0000 7f197284a740 4 client.16662 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:46:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.440+0000 7f197284a740 1 -- 172.21.15.2:0/1980793669 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ea99911e0 con 0x555ea963bd80 2024-05-14T09:46:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.440+0000 7f1968f28640 1 --2- 172.21.15.2:0/1980793669 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555ea963bd80 0x555ea9a363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.441+0000 7f1968f28640 1 --2- 172.21.15.2:0/1980793669 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555ea963bd80 0x555ea9a363f0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b3f3ae79b77967 server_cookie=f05b355026a921ce in_seq=0 out_seq=0 2024-05-14T09:46:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.441+0000 7f194cff9640 10 client.16662 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.442+0000 7f194cff9640 1 -- 172.21.15.2:0/1980793669 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ea99911e0 con 0x555ea963bd80 2024-05-14T09:46:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.442+0000 7f194cff9640 10 client.16662 handle_command_reply: tid=0 2024-05-14T09:46:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.442+0000 7f197284a740 1 client.16662 shutdown 2024-05-14T09:46:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.442+0000 7f197284a740 1 -- 172.21.15.2:0/1980793669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea96542f0 msgr2=0x555ea9a287c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.442+0000 7f197284a740 1 --2- 172.21.15.2:0/1980793669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea96542f0 0x555ea9a287c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f19600167b0 tx=0x7f19600046c0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.442+0000 7f197284a740 1 -- 172.21.15.2:0/1980793669 shutdown_connections 2024-05-14T09:46:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.442+0000 7f197284a740 1 --2- 172.21.15.2:0/1980793669 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555ea963bd80 0x555ea9a363f0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.442+0000 7f197284a740 1 --2- 172.21.15.2:0/1980793669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea967d1b0 0x555ea9a2fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.442+0000 7f197284a740 1 --2- 172.21.15.2:0/1980793669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea965c4d0 0x555ea9a28d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.442+0000 7f197284a740 1 --2- 172.21.15.2:0/1980793669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea96542f0 0x555ea9a287c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.442+0000 7f194cff9640 0 client.16662 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.442+0000 7f197284a740 1 -- 172.21.15.2:0/1980793669 >> 172.21.15.2:0/1980793669 conn(0x555ea9671300 msgr2=0x555ea9a1d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.442+0000 7f197284a740 1 -- 172.21.15.2:0/1980793669 shutdown_connections 2024-05-14T09:46:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.442+0000 7f197284a740 1 -- 172.21.15.2:0/1980793669 wait complete. 2024-05-14T09:46:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.442+0000 7f197284a740 20 client.16662 trim_cache size 0 max 16384 2024-05-14T09:46:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.457+0000 7f197284a740 1 Processor -- start 2024-05-14T09:46:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.457+0000 7f197284a740 1 -- start start 2024-05-14T09:46:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f197284a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea96542f0 0x555ea9a23250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f197284a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea965c4d0 0x555ea9a28aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f197284a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea967d1b0 0x555ea9a2fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f197284a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f19641b1850 con 0x555ea967d1b0 2024-05-14T09:46:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f197284a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f19641af090 con 0x555ea965c4d0 2024-05-14T09:46:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f197284a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f19641acee0 con 0x555ea96542f0 2024-05-14T09:46:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f1969f2a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea967d1b0 0x555ea9a2fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f1969f2a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea967d1b0 0x555ea9a2fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53512/0 (socket says 172.21.15.2:53512) 2024-05-14T09:46:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f1969f2a640 1 -- 172.21.15.2:0/4199224653 learned_addr learned my addr 172.21.15.2:0/4199224653 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f1969729640 1 --2- 172.21.15.2:0/4199224653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea96542f0 0x555ea9a23250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f1969f2a640 1 -- 172.21.15.2:0/4199224653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea96542f0 msgr2=0x555ea9a23250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f1969f2a640 1 --2- 172.21.15.2:0/4199224653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea96542f0 0x555ea9a23250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f1968f28640 1 --2- 172.21.15.2:0/4199224653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea965c4d0 0x555ea9a28aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f1969f2a640 1 -- 172.21.15.2:0/4199224653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea965c4d0 msgr2=0x555ea9a28aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f1969f2a640 1 --2- 172.21.15.2:0/4199224653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea965c4d0 0x555ea9a28aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f1969f2a640 1 -- 172.21.15.2:0/4199224653 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea9a1ad00 con 0x555ea967d1b0 2024-05-14T09:46:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f1969f2a640 1 --2- 172.21.15.2:0/4199224653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea967d1b0 0x555ea9a2fb80 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f195c006010 tx=0x7f195c030df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b46f93c340e5e9f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.458+0000 7f194effd640 1 -- 172.21.15.2:0/4199224653 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c0cb440 con 0x555ea967d1b0 2024-05-14T09:46:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.459+0000 7f194effd640 1 -- 172.21.15.2:0/4199224653 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195c036070 con 0x555ea967d1b0 2024-05-14T09:46:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.459+0000 7f194effd640 1 -- 172.21.15.2:0/4199224653 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c0c6fd0 con 0x555ea967d1b0 2024-05-14T09:46:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.459+0000 7f197284a740 1 -- 172.21.15.2:0/4199224653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea967d1b0 msgr2=0x555ea9a2fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.459+0000 7f197284a740 1 --2- 172.21.15.2:0/4199224653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea967d1b0 0x555ea9a2fb80 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f195c006010 tx=0x7f195c030df0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.459+0000 7f197284a740 1 -- 172.21.15.2:0/4199224653 shutdown_connections 2024-05-14T09:46:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.459+0000 7f197284a740 1 --2- 172.21.15.2:0/4199224653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea967d1b0 0x555ea9a2fb80 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.459+0000 7f197284a740 1 --2- 172.21.15.2:0/4199224653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea965c4d0 0x555ea9a28aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.459+0000 7f197284a740 1 --2- 172.21.15.2:0/4199224653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea96542f0 0x555ea9a23250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.459+0000 7f197284a740 1 -- 172.21.15.2:0/4199224653 >> 172.21.15.2:0/4199224653 conn(0x555ea9671300 msgr2=0x555ea9a28390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.459+0000 7f197284a740 1 -- 172.21.15.2:0/4199224653 shutdown_connections 2024-05-14T09:46:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.459+0000 7f197284a740 1 -- 172.21.15.2:0/4199224653 wait complete. 2024-05-14T09:46:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.460+0000 7f197284a740 1 -- start start 2024-05-14T09:46:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.460+0000 7f197284a740 1 Processor -- start 2024-05-14T09:46:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.460+0000 7f197284a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea96542f0 0x555ea9a2d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.460+0000 7f197284a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea965c4d0 0x555ea9a2d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.460+0000 7f197284a740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea967d1b0 0x555ea9a2e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.460+0000 7f197284a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f19641ad550 con 0x555ea965c4d0 2024-05-14T09:46:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.460+0000 7f197284a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f19641ae9c0 con 0x555ea96542f0 2024-05-14T09:46:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.460+0000 7f197284a740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f19641adc20 con 0x555ea967d1b0 2024-05-14T09:46:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.460+0000 7f197284a740 10 client.0 _ll_register_callbacks cb 0x7f1960007700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:46:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.460+0000 7f1968f28640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea965c4d0 0x555ea9a2d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.460+0000 7f1968f28640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea965c4d0 0x555ea9a2d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53514/0 (socket says 172.21.15.2:53514) 2024-05-14T09:46:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.460+0000 7f1968f28640 1 -- 172.21.15.2:0/705062037 learned_addr learned my addr 172.21.15.2:0/705062037 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.461+0000 7f1969729640 1 --2- 172.21.15.2:0/705062037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea96542f0 0x555ea9a2d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.461+0000 7f1969f2a640 1 --2- 172.21.15.2:0/705062037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea967d1b0 0x555ea9a2e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.461+0000 7f1968f28640 1 -- 172.21.15.2:0/705062037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea967d1b0 msgr2=0x555ea9a2e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.461+0000 7f1968f28640 1 --2- 172.21.15.2:0/705062037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea967d1b0 0x555ea9a2e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.461+0000 7f1968f28640 1 -- 172.21.15.2:0/705062037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea96542f0 msgr2=0x555ea9a2d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.461+0000 7f1968f28640 1 --2- 172.21.15.2:0/705062037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea96542f0 0x555ea9a2d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.461+0000 7f1968f28640 1 -- 172.21.15.2:0/705062037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea9a2eed0 con 0x555ea965c4d0 2024-05-14T09:46:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.461+0000 7f1968f28640 1 --2- 172.21.15.2:0/705062037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea965c4d0 0x555ea9a2d9e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f195400bfa0 tx=0x7f1954011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea13ac11badca92 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.461+0000 7f194cff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:46:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.461+0000 7f194cff9640 1 -- 172.21.15.2:0/705062037 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954004440 con 0x555ea965c4d0 2024-05-14T09:46:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.461+0000 7f194cff9640 1 -- 172.21.15.2:0/705062037 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f1954005040 con 0x555ea965c4d0 2024-05-14T09:46:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.461+0000 7f197284a740 1 -- 172.21.15.2:0/705062037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f195c036070 con 0x555ea965c4d0 2024-05-14T09:46:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.461+0000 7f194cff9640 1 -- 172.21.15.2:0/705062037 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1954031050 con 0x555ea965c4d0 2024-05-14T09:46:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.461+0000 7f197284a740 10 client.16668 fetch_fsmap learned FSMap version 44 2024-05-14T09:46:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.461+0000 7f197284a740 1 -- 172.21.15.2:0/705062037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555ea9a30260 con 0x555ea965c4d0 2024-05-14T09:46:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.462+0000 7f194cff9640 1 -- 172.21.15.2:0/705062037 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195c036070 con 0x555ea965c4d0 2024-05-14T09:46:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.462+0000 7f194cff9640 1 -- 172.21.15.2:0/705062037 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f195402c030 con 0x555ea965c4d0 2024-05-14T09:46:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.462+0000 7f197284a740 10 client.16668 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T09:46:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.462+0000 7f197284a740 10 client.16668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:46:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.462+0000 7f197284a740 20 client.16668 populate_metadata read hostname 'smithi002' 2024-05-14T09:46:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.462+0000 7f197284a740 1 --2- 172.21.15.2:0/705062037 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555ea96111c0 0x555ea9aa0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.462+0000 7f197284a740 4 client.16668 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:46:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.462+0000 7f197284a740 1 -- 172.21.15.2:0/705062037 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ea998b050 con 0x555ea96111c0 2024-05-14T09:46:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.462+0000 7f1969729640 1 --2- 172.21.15.2:0/705062037 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555ea96111c0 0x555ea9aa0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.463+0000 7f1969729640 1 --2- 172.21.15.2:0/705062037 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555ea96111c0 0x555ea9aa0890 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68aff8e04f3a4460 server_cookie=3b84a36d13f90681 in_seq=0 out_seq=0 2024-05-14T09:46:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.463+0000 7f194cff9640 10 client.16668 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.516+0000 7f194cff9640 1 -- 172.21.15.2:0/705062037 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555ea998b050 con 0x555ea96111c0 2024-05-14T09:46:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.516+0000 7f194cff9640 10 client.16668 handle_command_reply: tid=0 2024-05-14T09:46:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.516+0000 7f197284a740 1 client.16668 shutdown 2024-05-14T09:46:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.516+0000 7f197284a740 1 -- 172.21.15.2:0/705062037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea965c4d0 msgr2=0x555ea9a2d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.516+0000 7f197284a740 1 --2- 172.21.15.2:0/705062037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea965c4d0 0x555ea9a2d9e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f195400bfa0 tx=0x7f1954011da0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.516+0000 7f197284a740 1 -- 172.21.15.2:0/705062037 shutdown_connections 2024-05-14T09:46:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.516+0000 7f197284a740 1 --2- 172.21.15.2:0/705062037 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555ea96111c0 0x555ea9aa0890 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.516+0000 7f197284a740 1 --2- 172.21.15.2:0/705062037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555ea967d1b0 0x555ea9a2e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.516+0000 7f197284a740 1 --2- 172.21.15.2:0/705062037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ea965c4d0 0x555ea9a2d9e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.516+0000 7f194cff9640 0 client.16668 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.516+0000 7f197284a740 1 --2- 172.21.15.2:0/705062037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555ea96542f0 0x555ea9a2d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.516+0000 7f197284a740 1 -- 172.21.15.2:0/705062037 >> 172.21.15.2:0/705062037 conn(0x555ea9671300 msgr2=0x555ea9a193a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.517+0000 7f197284a740 1 -- 172.21.15.2:0/705062037 shutdown_connections 2024-05-14T09:46:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.517+0000 7f197284a740 1 -- 172.21.15.2:0/705062037 wait complete. 2024-05-14T09:46:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.517+0000 7f197284a740 20 client.16668 trim_cache size 0 max 16384 2024-05-14T09:46:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:46:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:46:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:46:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:46:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.517+0000 7f196b98d640 1 -- 172.21.15.2:0/3704266345 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f19500b28a0 msgr2=0x7f19500b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.517+0000 7f196b98d640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f19500b28a0 0x7f19500b4db0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x555ea9a1fe40 tx=0x7f1960008040 comp rx=0 tx=0).stop 2024-05-14T09:46:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.517+0000 7f196b98d640 1 -- 172.21.15.2:0/3704266345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19641037b0 msgr2=0x7f19641a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.517+0000 7f196b98d640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19641037b0 0x7f19641a3a00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f195c005e30 tx=0x7f195c003ba0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.517+0000 7f196b98d640 1 -- 172.21.15.2:0/3704266345 shutdown_connections 2024-05-14T09:46:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.517+0000 7f196b98d640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f19500b28a0 0x7f19500b4db0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.517+0000 7f196b98d640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19641037b0 0x7f19641a3a00 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.517+0000 7f196b98d640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1964102e60 0x7f196419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.517+0000 7f196b98d640 1 --2- 172.21.15.2:0/3704266345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1964102510 0x7f196419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.517+0000 7f196b98d640 1 -- 172.21.15.2:0/3704266345 >> 172.21.15.2:0/3704266345 conn(0x7f19640fdd20 msgr2=0x7f196410a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.517+0000 7f196b98d640 1 -- 172.21.15.2:0/3704266345 shutdown_connections 2024-05-14T09:46:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:17.518+0000 7f196b98d640 1 -- 172.21.15.2:0/3704266345 wait complete. 2024-05-14T09:46:18.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 984899 8.74 MB/sec execute 464 sec latency 4009.901 ms 2024-05-14T09:46:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:18 smithi002 ceph-mon[31557]: pgmap v765: 129 pgs: 129 active+clean; 163 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-05-14T09:46:18.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:18 smithi029 ceph-mon[36249]: pgmap v765: 129 pgs: 129 active+clean; 163 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-05-14T09:46:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:18 smithi026 ceph-mon[36056]: pgmap v765: 129 pgs: 129 active+clean; 163 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-05-14T09:46:19.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 985011 8.73 MB/sec execute 465 sec latency 4931.839 ms 2024-05-14T09:46:20.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 987105 8.73 MB/sec execute 466 sec latency 13.015 ms 2024-05-14T09:46:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:20 smithi002 ceph-mon[31557]: pgmap v766: 129 pgs: 129 active+clean; 164 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-05-14T09:46:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:20 smithi029 ceph-mon[36249]: pgmap v766: 129 pgs: 129 active+clean; 164 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-05-14T09:46:20.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:20 smithi026 ceph-mon[36056]: pgmap v766: 129 pgs: 129 active+clean; 164 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-05-14T09:46:21.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 988317 8.73 MB/sec execute 467 sec latency 79.680 ms 2024-05-14T09:46:22.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 988317 8.71 MB/sec execute 468 sec latency 1079.781 ms 2024-05-14T09:46:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:22 smithi002 ceph-mon[31557]: pgmap v767: 129 pgs: 129 active+clean; 164 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-05-14T09:46:22.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:22 smithi029 ceph-mon[36249]: pgmap v767: 129 pgs: 129 active+clean; 164 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-05-14T09:46:22.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:22 smithi026 ceph-mon[36056]: pgmap v767: 129 pgs: 129 active+clean; 164 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-05-14T09:46:23.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 988317 8.69 MB/sec execute 469 sec latency 2079.874 ms 2024-05-14T09:46:24.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 988383 8.67 MB/sec execute 470 sec latency 3003.562 ms 2024-05-14T09:46:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:24 smithi002 ceph-mon[31557]: pgmap v768: 129 pgs: 129 active+clean; 165 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-05-14T09:46:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:24 smithi029 ceph-mon[36249]: pgmap v768: 129 pgs: 129 active+clean; 165 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-05-14T09:46:24.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:24 smithi026 ceph-mon[36056]: pgmap v768: 129 pgs: 129 active+clean; 165 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-05-14T09:46:25.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 990291 8.67 MB/sec execute 471 sec latency 17.798 ms 2024-05-14T09:46:26.282 INFO:tasks.workunit.client.0.smithi002.stdout: 1 991695 8.68 MB/sec execute 472 sec latency 14.462 ms 2024-05-14T09:46:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:26 smithi002 ceph-mon[31557]: pgmap v769: 129 pgs: 129 active+clean; 165 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 189 KiB/s wr, 56 op/s 2024-05-14T09:46:26.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:26 smithi029 ceph-mon[36249]: pgmap v769: 129 pgs: 129 active+clean; 165 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 189 KiB/s wr, 56 op/s 2024-05-14T09:46:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:26 smithi026 ceph-mon[36056]: pgmap v769: 129 pgs: 129 active+clean; 165 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 189 KiB/s wr, 56 op/s 2024-05-14T09:46:27.283 INFO:tasks.workunit.client.0.smithi002.stdout: 1 991898 8.66 MB/sec execute 473 sec latency 836.014 ms 2024-05-14T09:46:28.283 INFO:tasks.workunit.client.0.smithi002.stdout: 1 991898 8.64 MB/sec execute 474 sec latency 1836.160 ms 2024-05-14T09:46:28.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:28 smithi002 ceph-mon[31557]: pgmap v770: 129 pgs: 129 active+clean; 166 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 281 KiB/s wr, 80 op/s 2024-05-14T09:46:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:28 smithi029 ceph-mon[36249]: pgmap v770: 129 pgs: 129 active+clean; 166 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 281 KiB/s wr, 80 op/s 2024-05-14T09:46:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:28 smithi026 ceph-mon[36056]: pgmap v770: 129 pgs: 129 active+clean; 166 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 281 KiB/s wr, 80 op/s 2024-05-14T09:46:29.283 INFO:tasks.workunit.client.0.smithi002.stdout: 1 991954 8.62 MB/sec execute 475 sec latency 2757.320 ms 2024-05-14T09:46:30.283 INFO:tasks.workunit.client.0.smithi002.stdout: 1 991954 8.60 MB/sec execute 476 sec latency 1007.085 ms 2024-05-14T09:46:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.631+0000 7f64d675e640 1 -- 172.21.15.2:0/2844117717 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d01099e0 msgr2=0x7f64d010be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.631+0000 7f64d675e640 1 --2- 172.21.15.2:0/2844117717 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d01099e0 0x7f64d010be00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009ca0 tx=0x7f64c802a8c0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.631+0000 7f64d675e640 1 -- 172.21.15.2:0/2844117717 shutdown_connections 2024-05-14T09:46:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.631+0000 7f64d675e640 1 --2- 172.21.15.2:0/2844117717 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d01099e0 0x7f64d010be00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.631+0000 7f64d675e640 1 --2- 172.21.15.2:0/2844117717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64d0107080 0x7f64d01094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.631+0000 7f64d675e640 1 --2- 172.21.15.2:0/2844117717 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d0104720 0x7f64d0106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.631+0000 7f64d675e640 1 -- 172.21.15.2:0/2844117717 >> 172.21.15.2:0/2844117717 conn(0x7f64d00fdd60 msgr2=0x7f64d01001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.631+0000 7f64d675e640 1 -- 172.21.15.2:0/2844117717 shutdown_connections 2024-05-14T09:46:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.631+0000 7f64d675e640 1 -- 172.21.15.2:0/2844117717 wait complete. 2024-05-14T09:46:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.632+0000 7f64d675e640 1 Processor -- start 2024-05-14T09:46:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.632+0000 7f64d675e640 1 -- start start 2024-05-14T09:46:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.632+0000 7f64d675e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0104720 0x7f64d019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.632+0000 7f64d675e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d0107080 0x7f64d019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.632+0000 7f64d675e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64d01099e0 0x7f64d01a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.632+0000 7f64d675e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f64d010f760 con 0x7f64d0104720 2024-05-14T09:46:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.632+0000 7f64d675e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f64d010f5e0 con 0x7f64d01099e0 2024-05-14T09:46:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.632+0000 7f64d675e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f64d010f8e0 con 0x7f64d0107080 2024-05-14T09:46:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.633+0000 7f64cffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0104720 0x7f64d019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.633+0000 7f64cffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0104720 0x7f64d019a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33902/0 (socket says 172.21.15.2:33902) 2024-05-14T09:46:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.633+0000 7f64cffff640 1 -- 172.21.15.2:0/2965149652 learned_addr learned my addr 172.21.15.2:0/2965149652 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.633+0000 7f64cf7fe640 1 --2- 172.21.15.2:0/2965149652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d0107080 0x7f64d019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.633+0000 7f64d4cfb640 1 --2- 172.21.15.2:0/2965149652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64d01099e0 0x7f64d01a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.633+0000 7f64cffff640 1 -- 172.21.15.2:0/2965149652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d0107080 msgr2=0x7f64d019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.633+0000 7f64cffff640 1 --2- 172.21.15.2:0/2965149652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d0107080 0x7f64d019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.633+0000 7f64cffff640 1 -- 172.21.15.2:0/2965149652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64d01099e0 msgr2=0x7f64d01a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.633+0000 7f64cffff640 1 --2- 172.21.15.2:0/2965149652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64d01099e0 0x7f64d01a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.633+0000 7f64cffff640 1 -- 172.21.15.2:0/2965149652 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d01a1de0 con 0x7f64d0104720 2024-05-14T09:46:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.633+0000 7f64cffff640 1 --2- 172.21.15.2:0/2965149652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0104720 0x7f64d019a2a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f64c000e9f0 tx=0x7f64c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.633+0000 7f64cd7fa640 1 -- 172.21.15.2:0/2965149652 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c000ccc0 con 0x7f64d0104720 2024-05-14T09:46:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.633+0000 7f64cd7fa640 1 -- 172.21.15.2:0/2965149652 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c00054d0 con 0x7f64d0104720 2024-05-14T09:46:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.633+0000 7f64d675e640 1 -- 172.21.15.2:0/2965149652 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d01a20d0 con 0x7f64d0104720 2024-05-14T09:46:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.634+0000 7f64cd7fa640 1 -- 172.21.15.2:0/2965149652 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c0010430 con 0x7f64d0104720 2024-05-14T09:46:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.634+0000 7f64d675e640 1 -- 172.21.15.2:0/2965149652 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d01a25d0 con 0x7f64d0104720 2024-05-14T09:46:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.636+0000 7f64dd60e740 1 Processor -- start 2024-05-14T09:46:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.636+0000 7f64dd60e740 1 -- start start 2024-05-14T09:46:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64cd7fa640 1 -- 172.21.15.2:0/2965149652 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f64c00046c0 con 0x7f64d0104720 2024-05-14T09:46:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64dd60e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef17e1b0 0x5590ef523e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64dd60e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef1552f0 0x5590ef5243f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64dd60e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef15d4d0 0x5590ef52b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64dd60e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5590ef4845d0 con 0x5590ef17e1b0 2024-05-14T09:46:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64dd60e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5590ef43d890 con 0x5590ef15d4d0 2024-05-14T09:46:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64dd60e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5590ef480590 con 0x5590ef1552f0 2024-05-14T09:46:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64d4cfb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef17e1b0 0x5590ef523e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64d4cfb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef17e1b0 0x5590ef523e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33904/0 (socket says 172.21.15.2:33904) 2024-05-14T09:46:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64d4cfb640 1 -- 172.21.15.2:0/923157837 learned_addr learned my addr 172.21.15.2:0/923157837 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64cf7fe640 1 --2- 172.21.15.2:0/923157837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef15d4d0 0x5590ef52b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64cffff640 1 --2- 172.21.15.2:0/923157837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef1552f0 0x5590ef5243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64d4cfb640 1 -- 172.21.15.2:0/923157837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef1552f0 msgr2=0x5590ef5243f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64d4cfb640 1 --2- 172.21.15.2:0/923157837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef1552f0 0x5590ef5243f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64d4cfb640 1 -- 172.21.15.2:0/923157837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef15d4d0 msgr2=0x5590ef52b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64d4cfb640 1 --2- 172.21.15.2:0/923157837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef15d4d0 0x5590ef52b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64d4cfb640 1 -- 172.21.15.2:0/923157837 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590ef196210 con 0x5590ef17e1b0 2024-05-14T09:46:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.637+0000 7f64cf7fe640 1 --2- 172.21.15.2:0/923157837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef15d4d0 0x5590ef52b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:46:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.638+0000 7f64cd7fa640 1 --2- 172.21.15.2:0/2965149652 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64b80b28a0 0x7f64b80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.638+0000 7f64d4cfb640 1 --2- 172.21.15.2:0/923157837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef17e1b0 0x5590ef523e70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f64d0075fe0 tx=0x7f64c802f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff073f92eda47139 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.638+0000 7f64cffff640 1 --2- 172.21.15.2:0/2965149652 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64b80b28a0 0x7f64b80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.638+0000 7f64b67fc640 1 -- 172.21.15.2:0/923157837 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8031650 con 0x5590ef17e1b0 2024-05-14T09:46:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.638+0000 7f64cd7fa640 1 -- 172.21.15.2:0/2965149652 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f64c00e2960 con 0x7f64d0104720 2024-05-14T09:46:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.638+0000 7f64b67fc640 1 -- 172.21.15.2:0/923157837 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c8036030 con 0x5590ef17e1b0 2024-05-14T09:46:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.638+0000 7f64b67fc640 1 -- 172.21.15.2:0/923157837 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c80046d0 con 0x5590ef17e1b0 2024-05-14T09:46:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.639+0000 7f64dd60e740 1 -- 172.21.15.2:0/923157837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef17e1b0 msgr2=0x5590ef523e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.639+0000 7f64dd60e740 1 --2- 172.21.15.2:0/923157837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef17e1b0 0x5590ef523e70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f64d0075fe0 tx=0x7f64c802f410 comp rx=0 tx=0).stop 2024-05-14T09:46:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.639+0000 7f64cffff640 1 --2- 172.21.15.2:0/2965149652 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64b80b28a0 0x7f64b80b4db0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x5590ef525250 tx=0x7f64c000ce60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.639+0000 7f64dd60e740 1 -- 172.21.15.2:0/923157837 shutdown_connections 2024-05-14T09:46:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.639+0000 7f64dd60e740 1 --2- 172.21.15.2:0/923157837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef15d4d0 0x5590ef52b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.639+0000 7f64dd60e740 1 --2- 172.21.15.2:0/923157837 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef1552f0 0x5590ef5243f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.639+0000 7f64dd60e740 1 --2- 172.21.15.2:0/923157837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef17e1b0 0x5590ef523e70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.639+0000 7f64dd60e740 1 -- 172.21.15.2:0/923157837 >> 172.21.15.2:0/923157837 conn(0x5590ef172300 msgr2=0x5590ef51b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.639+0000 7f64dd60e740 1 -- 172.21.15.2:0/923157837 shutdown_connections 2024-05-14T09:46:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.639+0000 7f64dd60e740 1 -- 172.21.15.2:0/923157837 wait complete. 2024-05-14T09:46:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.639+0000 7f64dd60e740 1 -- start start 2024-05-14T09:46:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.639+0000 7f64dd60e740 1 Processor -- start 2024-05-14T09:46:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64dd60e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef1552f0 0x5590ef52d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64dd60e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef15d4d0 0x5590ef52dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64dd60e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef17e1b0 0x5590ef534d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64dd60e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5590ef52bb60 con 0x5590ef15d4d0 2024-05-14T09:46:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64dd60e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5590ef441370 con 0x5590ef17e1b0 2024-05-14T09:46:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64dd60e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5590ef52bce0 con 0x5590ef1552f0 2024-05-14T09:46:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64dd60e740 10 client.0 _ll_register_callbacks cb 0x5590ef195a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:46:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64cf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef15d4d0 0x5590ef52dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64cf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef15d4d0 0x5590ef52dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33906/0 (socket says 172.21.15.2:33906) 2024-05-14T09:46:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64cf7fe640 1 -- 172.21.15.2:0/2903493082 learned_addr learned my addr 172.21.15.2:0/2903493082 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64d4cfb640 1 --2- 172.21.15.2:0/2903493082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef1552f0 0x5590ef52d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64cf7fe640 1 -- 172.21.15.2:0/2903493082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef1552f0 msgr2=0x5590ef52d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64cf7fe640 1 --2- 172.21.15.2:0/2903493082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef1552f0 0x5590ef52d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64cf7fe640 1 -- 172.21.15.2:0/2903493082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef17e1b0 msgr2=0x5590ef534d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64cf7fe640 1 --2- 172.21.15.2:0/2903493082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef17e1b0 0x5590ef534d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64cf7fe640 1 -- 172.21.15.2:0/2903493082 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590ef262620 con 0x5590ef15d4d0 2024-05-14T09:46:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64d4cfb640 1 --2- 172.21.15.2:0/2903493082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef1552f0 0x5590ef52d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:46:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64cf7fe640 1 --2- 172.21.15.2:0/2903493082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef15d4d0 0x5590ef52dde0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f64c4012700 tx=0x7f64c4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333d6d18be960c16 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.640+0000 7f64af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:46:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.641+0000 7f64af7fe640 1 -- 172.21.15.2:0/2903493082 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4004030 con 0x5590ef15d4d0 2024-05-14T09:46:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.641+0000 7f64af7fe640 1 -- 172.21.15.2:0/2903493082 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f64c40041d0 con 0x5590ef15d4d0 2024-05-14T09:46:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.641+0000 7f64dd60e740 1 -- 172.21.15.2:0/2903493082 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64c8036030 con 0x5590ef15d4d0 2024-05-14T09:46:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.641+0000 7f64af7fe640 1 -- 172.21.15.2:0/2903493082 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f64c4025050 con 0x5590ef15d4d0 2024-05-14T09:46:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.641+0000 7f64dd60e740 10 client.16680 fetch_fsmap learned FSMap version 44 2024-05-14T09:46:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.641+0000 7f64dd60e740 1 -- 172.21.15.2:0/2903493082 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5590ef537d40 con 0x5590ef15d4d0 2024-05-14T09:46:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.641+0000 7f64af7fe640 1 -- 172.21.15.2:0/2903493082 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c8036030 con 0x5590ef15d4d0 2024-05-14T09:46:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.641+0000 7f64af7fe640 1 -- 172.21.15.2:0/2903493082 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f64c4003990 con 0x5590ef15d4d0 2024-05-14T09:46:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.641+0000 7f64dd60e740 10 client.16680 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T09:46:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.641+0000 7f64dd60e740 10 client.16680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:46:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.641+0000 7f64dd60e740 20 client.16680 populate_metadata read hostname 'smithi002' 2024-05-14T09:46:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.641+0000 7f64dd60e740 1 --2- 172.21.15.2:0/2903493082 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590ef13cd80 0x5590ef53b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.641+0000 7f64dd60e740 4 client.16680 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:46:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.641+0000 7f64dd60e740 1 -- 172.21.15.2:0/2903493082 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590ef4921e0 con 0x5590ef13cd80 2024-05-14T09:46:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.642+0000 7f64d4cfb640 1 --2- 172.21.15.2:0/2903493082 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590ef13cd80 0x5590ef53b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.642+0000 7f64d4cfb640 1 --2- 172.21.15.2:0/2903493082 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590ef13cd80 0x5590ef53b560 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=729642b4039321d0 server_cookie=8233293a50e4e9dc in_seq=0 out_seq=0 2024-05-14T09:46:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.642+0000 7f64af7fe640 10 client.16680 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.643+0000 7f64af7fe640 1 -- 172.21.15.2:0/2903493082 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5590ef4921e0 con 0x5590ef13cd80 2024-05-14T09:46:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.643+0000 7f64af7fe640 10 client.16680 handle_command_reply: tid=0 2024-05-14T09:46:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.643+0000 7f64dd60e740 1 client.16680 shutdown 2024-05-14T09:46:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.643+0000 7f64dd60e740 1 -- 172.21.15.2:0/2903493082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef15d4d0 msgr2=0x5590ef52dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.643+0000 7f64dd60e740 1 --2- 172.21.15.2:0/2903493082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef15d4d0 0x5590ef52dde0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f64c4012700 tx=0x7f64c4012bc0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.643+0000 7f64dd60e740 1 -- 172.21.15.2:0/2903493082 shutdown_connections 2024-05-14T09:46:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.643+0000 7f64dd60e740 1 --2- 172.21.15.2:0/2903493082 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590ef13cd80 0x5590ef53b560 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.643+0000 7f64dd60e740 1 --2- 172.21.15.2:0/2903493082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef17e1b0 0x5590ef534d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.643+0000 7f64dd60e740 1 --2- 172.21.15.2:0/2903493082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef15d4d0 0x5590ef52dde0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.643+0000 7f64dd60e740 1 --2- 172.21.15.2:0/2903493082 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef1552f0 0x5590ef52d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.643+0000 7f64af7fe640 0 client.16680 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.643+0000 7f64dd60e740 1 -- 172.21.15.2:0/2903493082 >> 172.21.15.2:0/2903493082 conn(0x5590ef172300 msgr2=0x5590ef51e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.643+0000 7f64dd60e740 1 -- 172.21.15.2:0/2903493082 shutdown_connections 2024-05-14T09:46:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.643+0000 7f64dd60e740 1 -- 172.21.15.2:0/2903493082 wait complete. 2024-05-14T09:46:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.644+0000 7f64dd60e740 20 client.16680 trim_cache size 0 max 16384 2024-05-14T09:46:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64dd60e740 1 Processor -- start 2024-05-14T09:46:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64dd60e740 1 -- start start 2024-05-14T09:46:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64dd60e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef1552f0 0x5590ef51e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64dd60e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef15d4d0 0x5590ef534c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64dd60e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef17e1b0 0x5590ef51ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64dd60e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01ad2e0 con 0x5590ef1552f0 2024-05-14T09:46:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64dd60e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01aab20 con 0x5590ef15d4d0 2024-05-14T09:46:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64dd60e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01a8970 con 0x5590ef17e1b0 2024-05-14T09:46:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64d4cfb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef1552f0 0x5590ef51e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64d4cfb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef1552f0 0x5590ef51e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33920/0 (socket says 172.21.15.2:33920) 2024-05-14T09:46:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64d4cfb640 1 -- 172.21.15.2:0/3015070234 learned_addr learned my addr 172.21.15.2:0/3015070234 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64cf7fe640 1 --2- 172.21.15.2:0/3015070234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef15d4d0 0x5590ef534c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64d4cfb640 1 -- 172.21.15.2:0/3015070234 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef17e1b0 msgr2=0x5590ef51ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64d4cfb640 1 --2- 172.21.15.2:0/3015070234 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef17e1b0 0x5590ef51ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64d4cfb640 1 -- 172.21.15.2:0/3015070234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef15d4d0 msgr2=0x5590ef534c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64d4cfb640 1 --2- 172.21.15.2:0/3015070234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef15d4d0 0x5590ef534c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64d4cfb640 1 -- 172.21.15.2:0/3015070234 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590ef51bd00 con 0x5590ef1552f0 2024-05-14T09:46:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.659+0000 7f64cf7fe640 1 --2- 172.21.15.2:0/3015070234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef15d4d0 0x5590ef534c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:46:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.660+0000 7f64d4cfb640 1 --2- 172.21.15.2:0/3015070234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef1552f0 0x5590ef51e900 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009620 tx=0x7f64c8031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5991890f5d35ef74 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.660+0000 7f64b5ffb640 1 -- 172.21.15.2:0/3015070234 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8004650 con 0x5590ef1552f0 2024-05-14T09:46:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.660+0000 7f64b5ffb640 1 -- 172.21.15.2:0/3015070234 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c8059020 con 0x5590ef1552f0 2024-05-14T09:46:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.660+0000 7f64b5ffb640 1 -- 172.21.15.2:0/3015070234 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8003a60 con 0x5590ef1552f0 2024-05-14T09:46:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.661+0000 7f64dd60e740 1 -- 172.21.15.2:0/3015070234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef1552f0 msgr2=0x5590ef51e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.661+0000 7f64dd60e740 1 --2- 172.21.15.2:0/3015070234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef1552f0 0x5590ef51e900 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009620 tx=0x7f64c8031b10 comp rx=0 tx=0).stop 2024-05-14T09:46:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.661+0000 7f64dd60e740 1 -- 172.21.15.2:0/3015070234 shutdown_connections 2024-05-14T09:46:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.661+0000 7f64dd60e740 1 --2- 172.21.15.2:0/3015070234 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef17e1b0 0x5590ef51ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.661+0000 7f64dd60e740 1 --2- 172.21.15.2:0/3015070234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef15d4d0 0x5590ef534c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.661+0000 7f64dd60e740 1 --2- 172.21.15.2:0/3015070234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef1552f0 0x5590ef51e900 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.661+0000 7f64dd60e740 1 -- 172.21.15.2:0/3015070234 >> 172.21.15.2:0/3015070234 conn(0x5590ef172300 msgr2=0x5590ef51e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.661+0000 7f64dd60e740 1 -- 172.21.15.2:0/3015070234 shutdown_connections 2024-05-14T09:46:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.661+0000 7f64dd60e740 1 -- 172.21.15.2:0/3015070234 wait complete. 2024-05-14T09:46:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.661+0000 7f64dd60e740 1 -- start start 2024-05-14T09:46:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.661+0000 7f64dd60e740 1 Processor -- start 2024-05-14T09:46:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64dd60e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef1552f0 0x5590ef532250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64dd60e740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef15d4d0 0x5590ef532790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64dd60e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef17e1b0 0x5590ef5336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64dd60e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01a8fe0 con 0x5590ef1552f0 2024-05-14T09:46:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64dd60e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01aa450 con 0x5590ef17e1b0 2024-05-14T09:46:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64dd60e740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01a96b0 con 0x5590ef15d4d0 2024-05-14T09:46:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64dd60e740 10 client.0 _ll_register_callbacks cb 0x7f64c4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:46:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64d4cfb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef1552f0 0x5590ef532250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64d4cfb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef1552f0 0x5590ef532250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33928/0 (socket says 172.21.15.2:33928) 2024-05-14T09:46:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64d4cfb640 1 -- 172.21.15.2:0/2867767414 learned_addr learned my addr 172.21.15.2:0/2867767414 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64cf7fe640 1 --2- 172.21.15.2:0/2867767414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef15d4d0 0x5590ef532790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64d4cfb640 1 --2- 172.21.15.2:0/2867767414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef17e1b0 0x5590ef5336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64d4cfb640 1 -- 172.21.15.2:0/2867767414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef15d4d0 msgr2=0x5590ef532790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64d4cfb640 1 --2- 172.21.15.2:0/2867767414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef15d4d0 0x5590ef532790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64d4cfb640 1 -- 172.21.15.2:0/2867767414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef17e1b0 msgr2=0x5590ef5336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64d4cfb640 1 --2- 172.21.15.2:0/2867767414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef17e1b0 0x5590ef5336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64d4cfb640 1 -- 172.21.15.2:0/2867767414 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590ef533c80 con 0x5590ef1552f0 2024-05-14T09:46:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64d4cfb640 1 --2- 172.21.15.2:0/2867767414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef1552f0 0x5590ef532250 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f64c8006010 tx=0x7f64c8009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c719e8f5cfca967 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:46:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.662+0000 7f64af7fe640 1 -- 172.21.15.2:0/2867767414 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8062020 con 0x5590ef1552f0 2024-05-14T09:46:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.663+0000 7f64af7fe640 1 -- 172.21.15.2:0/2867767414 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f64c804a040 con 0x5590ef1552f0 2024-05-14T09:46:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.663+0000 7f64dd60e740 1 -- 172.21.15.2:0/2867767414 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64c8059020 con 0x5590ef1552f0 2024-05-14T09:46:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.663+0000 7f64af7fe640 1 -- 172.21.15.2:0/2867767414 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f64c80317c0 con 0x5590ef1552f0 2024-05-14T09:46:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.663+0000 7f64dd60e740 10 client.16686 fetch_fsmap learned FSMap version 44 2024-05-14T09:46:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.663+0000 7f64dd60e740 1 -- 172.21.15.2:0/2867767414 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5590ef534f10 con 0x5590ef1552f0 2024-05-14T09:46:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.663+0000 7f64af7fe640 1 -- 172.21.15.2:0/2867767414 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c8059020 con 0x5590ef1552f0 2024-05-14T09:46:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.663+0000 7f64af7fe640 1 -- 172.21.15.2:0/2867767414 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f64c8043070 con 0x5590ef1552f0 2024-05-14T09:46:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.663+0000 7f64dd60e740 10 client.16686 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T09:46:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.663+0000 7f64dd60e740 10 client.16686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:46:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.663+0000 7f64dd60e740 20 client.16686 populate_metadata read hostname 'smithi002' 2024-05-14T09:46:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.663+0000 7f64dd60e740 1 --2- 172.21.15.2:0/2867767414 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590ef1121c0 0x5590ef5a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.663+0000 7f64dd60e740 4 client.16686 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:46:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.663+0000 7f64dd60e740 1 -- 172.21.15.2:0/2867767414 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590ef48c050 con 0x5590ef1121c0 2024-05-14T09:46:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.664+0000 7f64cf7fe640 1 --2- 172.21.15.2:0/2867767414 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590ef1121c0 0x5590ef5a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.664+0000 7f64cf7fe640 1 --2- 172.21.15.2:0/2867767414 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590ef1121c0 0x5590ef5a5d80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48b98ea0f9a0eebc server_cookie=2a6afb0581610ee6 in_seq=0 out_seq=0 2024-05-14T09:46:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.664+0000 7f64af7fe640 10 client.16686 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.729+0000 7f64af7fe640 1 -- 172.21.15.2:0/2867767414 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590ef48c050 con 0x5590ef1121c0 2024-05-14T09:46:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.729+0000 7f64af7fe640 10 client.16686 handle_command_reply: tid=0 2024-05-14T09:46:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.729+0000 7f64dd60e740 1 client.16686 shutdown 2024-05-14T09:46:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.729+0000 7f64dd60e740 1 -- 172.21.15.2:0/2867767414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef1552f0 msgr2=0x5590ef532250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.729+0000 7f64dd60e740 1 --2- 172.21.15.2:0/2867767414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef1552f0 0x5590ef532250 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f64c8006010 tx=0x7f64c8009960 comp rx=0 tx=0).stop 2024-05-14T09:46:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.729+0000 7f64dd60e740 1 -- 172.21.15.2:0/2867767414 shutdown_connections 2024-05-14T09:46:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.729+0000 7f64dd60e740 1 --2- 172.21.15.2:0/2867767414 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5590ef1121c0 0x5590ef5a5d80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.729+0000 7f64dd60e740 1 --2- 172.21.15.2:0/2867767414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5590ef17e1b0 0x5590ef5336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.729+0000 7f64dd60e740 1 --2- 172.21.15.2:0/2867767414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5590ef15d4d0 0x5590ef532790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.729+0000 7f64dd60e740 1 --2- 172.21.15.2:0/2867767414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590ef1552f0 0x5590ef532250 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.729+0000 7f64af7fe640 0 client.16686 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.729+0000 7f64dd60e740 1 -- 172.21.15.2:0/2867767414 >> 172.21.15.2:0/2867767414 conn(0x5590ef172300 msgr2=0x5590ef51ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.729+0000 7f64dd60e740 1 -- 172.21.15.2:0/2867767414 shutdown_connections 2024-05-14T09:46:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.729+0000 7f64dd60e740 1 -- 172.21.15.2:0/2867767414 wait complete. 2024-05-14T09:46:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.729+0000 7f64dd60e740 20 client.16686 trim_cache size 0 max 16384 2024-05-14T09:46:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:46:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:46:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:46:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:46:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.730+0000 7f64d675e640 1 -- 172.21.15.2:0/2965149652 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64b80b28a0 msgr2=0x7f64b80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.730+0000 7f64d675e640 1 --2- 172.21.15.2:0/2965149652 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64b80b28a0 0x7f64b80b4db0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x5590ef525250 tx=0x7f64c000ce60 comp rx=0 tx=0).stop 2024-05-14T09:46:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.730+0000 7f64d675e640 1 -- 172.21.15.2:0/2965149652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0104720 msgr2=0x7f64d019a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.730+0000 7f64d675e640 1 --2- 172.21.15.2:0/2965149652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0104720 0x7f64d019a2a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f64c000e9f0 tx=0x7f64c000eeb0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.730+0000 7f64d675e640 1 -- 172.21.15.2:0/2965149652 shutdown_connections 2024-05-14T09:46:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.730+0000 7f64d675e640 1 --2- 172.21.15.2:0/2965149652 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64b80b28a0 0x7f64b80b4db0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.730+0000 7f64d675e640 1 --2- 172.21.15.2:0/2965149652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64d01099e0 0x7f64d01a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.730+0000 7f64d675e640 1 --2- 172.21.15.2:0/2965149652 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64d0107080 0x7f64d019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.730+0000 7f64d675e640 1 --2- 172.21.15.2:0/2965149652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0104720 0x7f64d019a2a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.730+0000 7f64d675e640 1 -- 172.21.15.2:0/2965149652 >> 172.21.15.2:0/2965149652 conn(0x7f64d00fdd60 msgr2=0x7f64d0105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.730+0000 7f64d675e640 1 -- 172.21.15.2:0/2965149652 shutdown_connections 2024-05-14T09:46:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:30.730+0000 7f64d675e640 1 -- 172.21.15.2:0/2965149652 wait complete. 2024-05-14T09:46:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:30 smithi002 ceph-mon[31557]: pgmap v771: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 345 KiB/s wr, 87 op/s 2024-05-14T09:46:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:30 smithi029 ceph-mon[36249]: pgmap v771: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 345 KiB/s wr, 87 op/s 2024-05-14T09:46:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:30 smithi026 ceph-mon[36056]: pgmap v771: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 345 KiB/s wr, 87 op/s 2024-05-14T09:46:31.283 INFO:tasks.workunit.client.0.smithi002.stdout: 1 991954 8.59 MB/sec execute 477 sec latency 2007.136 ms 2024-05-14T09:46:32.283 INFO:tasks.workunit.client.0.smithi002.stdout: 1 991954 8.57 MB/sec execute 478 sec latency 3007.249 ms 2024-05-14T09:46:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:32 smithi002 ceph-mon[31557]: pgmap v772: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 357 KiB/s wr, 91 op/s 2024-05-14T09:46:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:32 smithi029 ceph-mon[36249]: pgmap v772: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 357 KiB/s wr, 91 op/s 2024-05-14T09:46:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:32 smithi026 ceph-mon[36056]: pgmap v772: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 357 KiB/s wr, 91 op/s 2024-05-14T09:46:33.283 INFO:tasks.workunit.client.0.smithi002.stdout: 1 991954 8.55 MB/sec execute 479 sec latency 4007.323 ms 2024-05-14T09:46:34.283 INFO:tasks.workunit.client.0.smithi002.stdout: 1 992022 8.53 MB/sec execute 480 sec latency 4927.262 ms 2024-05-14T09:46:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:34 smithi002 ceph-mon[31557]: pgmap v773: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 310 KiB/s wr, 86 op/s 2024-05-14T09:46:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:34 smithi029 ceph-mon[36249]: pgmap v773: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 310 KiB/s wr, 86 op/s 2024-05-14T09:46:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:34 smithi026 ceph-mon[36056]: pgmap v773: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 310 KiB/s wr, 86 op/s 2024-05-14T09:46:35.283 INFO:tasks.workunit.client.0.smithi002.stdout: 1 994128 8.54 MB/sec execute 481 sec latency 15.289 ms 2024-05-14T09:46:36.283 INFO:tasks.workunit.client.0.smithi002.stdout: 1 995310 8.54 MB/sec execute 482 sec latency 14.593 ms 2024-05-14T09:46:36.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:36 smithi002 ceph-mon[31557]: pgmap v774: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 217 KiB/s wr, 62 op/s 2024-05-14T09:46:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:36 smithi029 ceph-mon[36249]: pgmap v774: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 217 KiB/s wr, 62 op/s 2024-05-14T09:46:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:36 smithi026 ceph-mon[36056]: pgmap v774: 129 pgs: 129 active+clean; 167 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 217 KiB/s wr, 62 op/s 2024-05-14T09:46:37.283 INFO:tasks.workunit.client.0.smithi002.stdout: 1 995441 8.52 MB/sec execute 483 sec latency 885.976 ms 2024-05-14T09:46:38.283 INFO:tasks.workunit.client.0.smithi002.stdout: 1 995441 8.50 MB/sec execute 484 sec latency 1886.079 ms 2024-05-14T09:46:38.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:38 smithi002 ceph-mon[31557]: pgmap v775: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 325 KiB/s wr, 85 op/s 2024-05-14T09:46:38.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:38 smithi029 ceph-mon[36249]: pgmap v775: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 325 KiB/s wr, 85 op/s 2024-05-14T09:46:38.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:38 smithi026 ceph-mon[36056]: pgmap v775: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 325 KiB/s wr, 85 op/s 2024-05-14T09:46:39.283 INFO:tasks.workunit.client.0.smithi002.stdout: 1 995497 8.48 MB/sec execute 485 sec latency 2807.264 ms 2024-05-14T09:46:40.304 INFO:tasks.workunit.client.0.smithi002.stdout: 1 995565 8.47 MB/sec execute 486 sec latency 846.798 ms 2024-05-14T09:46:40.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:40 smithi002 ceph-mon[31557]: pgmap v776: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 279 KiB/s wr, 73 op/s 2024-05-14T09:46:40.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:40 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:46:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:40 smithi029 ceph-mon[36249]: pgmap v776: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 279 KiB/s wr, 73 op/s 2024-05-14T09:46:40.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:40 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:46:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:40 smithi026 ceph-mon[36056]: pgmap v776: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 279 KiB/s wr, 73 op/s 2024-05-14T09:46:40.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:40 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:46:41.284 INFO:tasks.workunit.client.0.smithi002.stdout: 1 995565 8.45 MB/sec execute 487 sec latency 1108.218 ms 2024-05-14T09:46:42.284 INFO:tasks.workunit.client.0.smithi002.stdout: 1 995565 8.43 MB/sec execute 488 sec latency 2108.305 ms 2024-05-14T09:46:42.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:42 smithi002 ceph-mon[31557]: pgmap v777: 129 pgs: 129 active+clean; 169 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 444 KiB/s wr, 71 op/s 2024-05-14T09:46:42.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:42 smithi029 ceph-mon[36249]: pgmap v777: 129 pgs: 129 active+clean; 169 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 444 KiB/s wr, 71 op/s 2024-05-14T09:46:42.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:42 smithi026 ceph-mon[36056]: pgmap v777: 129 pgs: 129 active+clean; 169 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 444 KiB/s wr, 71 op/s 2024-05-14T09:46:43.285 INFO:tasks.workunit.client.0.smithi002.stdout: 1 995565 8.42 MB/sec execute 489 sec latency 3108.421 ms 2024-05-14T09:46:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.844+0000 7f9e6da67640 1 -- 172.21.15.2:0/2288865586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e68109bd0 msgr2=0x7f9e6810bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.844+0000 7f9e6da67640 1 --2- 172.21.15.2:0/2288865586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e68109bd0 0x7f9e6810bff0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009ca0 tx=0x7f9e6002a8c0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.844+0000 7f9e6da67640 1 -- 172.21.15.2:0/2288865586 shutdown_connections 2024-05-14T09:46:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.844+0000 7f9e6da67640 1 --2- 172.21.15.2:0/2288865586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e68109bd0 0x7f9e6810bff0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.844+0000 7f9e6da67640 1 --2- 172.21.15.2:0/2288865586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e68104ed0 0x7f9e681052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.844+0000 7f9e6da67640 1 --2- 172.21.15.2:0/2288865586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e68103c90 0x7f9e68104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.844+0000 7f9e6da67640 1 -- 172.21.15.2:0/2288865586 >> 172.21.15.2:0/2288865586 conn(0x7f9e680feea0 msgr2=0x7f9e68101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.844+0000 7f9e6da67640 1 -- 172.21.15.2:0/2288865586 shutdown_connections 2024-05-14T09:46:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.844+0000 7f9e6da67640 1 -- 172.21.15.2:0/2288865586 wait complete. 2024-05-14T09:46:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.845+0000 7f9e6da67640 1 Processor -- start 2024-05-14T09:46:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.845+0000 7f9e6da67640 1 -- start start 2024-05-14T09:46:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e6da67640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e68103c90 0x7f9e6819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e6da67640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e68104ed0 0x7f9e6819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e6da67640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e68109bd0 0x7f9e681a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e6da67640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6810f7a0 con 0x7f9e68109bd0 2024-05-14T09:46:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e6da67640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6810f620 con 0x7f9e68104ed0 2024-05-14T09:46:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e6da67640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6810f920 con 0x7f9e68103c90 2024-05-14T09:46:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e68109bd0 0x7f9e681a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e68109bd0 0x7f9e681a3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34854/0 (socket says 172.21.15.2:34854) 2024-05-14T09:46:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e66ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e68104ed0 0x7f9e6819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e67fff640 1 -- 172.21.15.2:0/2703671164 learned_addr learned my addr 172.21.15.2:0/2703671164 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e677fe640 1 --2- 172.21.15.2:0/2703671164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e68103c90 0x7f9e6819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e67fff640 1 -- 172.21.15.2:0/2703671164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e68103c90 msgr2=0x7f9e6819c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e67fff640 1 --2- 172.21.15.2:0/2703671164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e68103c90 0x7f9e6819c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e67fff640 1 -- 172.21.15.2:0/2703671164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e68104ed0 msgr2=0x7f9e6819c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e67fff640 1 --2- 172.21.15.2:0/2703671164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e68104ed0 0x7f9e6819c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e67fff640 1 -- 172.21.15.2:0/2703671164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e681a3fe0 con 0x7f9e68109bd0 2024-05-14T09:46:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e66ffd640 1 --2- 172.21.15.2:0/2703671164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e68104ed0 0x7f9e6819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:46:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.846+0000 7f9e67fff640 1 --2- 172.21.15.2:0/2703671164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e68109bd0 0x7f9e681a3970 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9e60005e30 tx=0x7f9e6002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.847+0000 7f9e64ff9640 1 -- 172.21.15.2:0/2703671164 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e60003bf0 con 0x7f9e68109bd0 2024-05-14T09:46:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.847+0000 7f9e64ff9640 1 -- 172.21.15.2:0/2703671164 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e60003d90 con 0x7f9e68109bd0 2024-05-14T09:46:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.847+0000 7f9e6da67640 1 -- 172.21.15.2:0/2703671164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e681a4270 con 0x7f9e68109bd0 2024-05-14T09:46:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.847+0000 7f9e64ff9640 1 -- 172.21.15.2:0/2703671164 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6002fb90 con 0x7f9e68109bd0 2024-05-14T09:46:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.847+0000 7f9e6da67640 1 -- 172.21.15.2:0/2703671164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e681a47c0 con 0x7f9e68109bd0 2024-05-14T09:46:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.850+0000 7f9e749f7740 1 Processor -- start 2024-05-14T09:46:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.850+0000 7f9e64ff9640 1 -- 172.21.15.2:0/2703671164 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9e6002fd30 con 0x7f9e68109bd0 2024-05-14T09:46:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.850+0000 7f9e749f7740 1 -- start start 2024-05-14T09:46:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e749f7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14911b0 0x55b9c1832a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e749f7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14682f0 0x55b9c1832fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e749f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14704d0 0x55b9c1839ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e749f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b9c17975d0 con 0x55b9c14704d0 2024-05-14T09:46:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e749f7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b9c1750890 con 0x55b9c14682f0 2024-05-14T09:46:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e749f7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b9c1793590 con 0x55b9c14911b0 2024-05-14T09:46:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e66ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14704d0 0x55b9c1839ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e66ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14704d0 0x55b9c1839ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34856/0 (socket says 172.21.15.2:34856) 2024-05-14T09:46:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e66ffd640 1 -- 172.21.15.2:0/797787481 learned_addr learned my addr 172.21.15.2:0/797787481 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e677fe640 1 --2- 172.21.15.2:0/797787481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14682f0 0x55b9c1832fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e66ffd640 1 --2- 172.21.15.2:0/797787481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14911b0 0x55b9c1832a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e677fe640 1 -- 172.21.15.2:0/797787481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14911b0 msgr2=0x55b9c1832a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e677fe640 1 --2- 172.21.15.2:0/797787481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14911b0 0x55b9c1832a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e677fe640 1 -- 172.21.15.2:0/797787481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14704d0 msgr2=0x55b9c1839ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e677fe640 1 --2- 172.21.15.2:0/797787481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14704d0 0x55b9c1839ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e677fe640 1 -- 172.21.15.2:0/797787481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c14a9210 con 0x55b9c14682f0 2024-05-14T09:46:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.851+0000 7f9e64ff9640 1 --2- 172.21.15.2:0/2703671164 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9e500b28a0 0x7f9e500b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.852+0000 7f9e677fe640 1 --2- 172.21.15.2:0/797787481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14682f0 0x55b9c1832fe0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9e68103560 tx=0x7f9e58011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71972b00b0b56c67 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.852+0000 7f9e66ffd640 1 --2- 172.21.15.2:0/2703671164 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9e500b28a0 0x7f9e500b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.852+0000 7f9e64ff9640 1 -- 172.21.15.2:0/2703671164 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f9e60103d20 con 0x7f9e68109bd0 2024-05-14T09:46:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.852+0000 7f9e4dffb640 1 -- 172.21.15.2:0/797787481 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58004030 con 0x55b9c14682f0 2024-05-14T09:46:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.852+0000 7f9e4dffb640 1 -- 172.21.15.2:0/797787481 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e580054d0 con 0x55b9c14682f0 2024-05-14T09:46:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.852+0000 7f9e4dffb640 1 -- 172.21.15.2:0/797787481 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58004c60 con 0x55b9c14682f0 2024-05-14T09:46:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.853+0000 7f9e749f7740 1 -- 172.21.15.2:0/797787481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14682f0 msgr2=0x55b9c1832fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.853+0000 7f9e749f7740 1 --2- 172.21.15.2:0/797787481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14682f0 0x55b9c1832fe0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9e68103560 tx=0x7f9e58011810 comp rx=0 tx=0).stop 2024-05-14T09:46:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.853+0000 7f9e66ffd640 1 --2- 172.21.15.2:0/2703671164 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9e500b28a0 0x7f9e500b4db0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55b9c1833c30 tx=0x7f9e5c00bbb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.853+0000 7f9e749f7740 1 -- 172.21.15.2:0/797787481 shutdown_connections 2024-05-14T09:46:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.853+0000 7f9e749f7740 1 --2- 172.21.15.2:0/797787481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14704d0 0x55b9c1839ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.853+0000 7f9e749f7740 1 --2- 172.21.15.2:0/797787481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14682f0 0x55b9c1832fe0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.853+0000 7f9e749f7740 1 --2- 172.21.15.2:0/797787481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14911b0 0x55b9c1832a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.853+0000 7f9e749f7740 1 -- 172.21.15.2:0/797787481 >> 172.21.15.2:0/797787481 conn(0x55b9c1485300 msgr2=0x55b9c182e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.853+0000 7f9e749f7740 1 -- 172.21.15.2:0/797787481 shutdown_connections 2024-05-14T09:46:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.853+0000 7f9e749f7740 1 -- 172.21.15.2:0/797787481 wait complete. 2024-05-14T09:46:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.853+0000 7f9e749f7740 1 -- start start 2024-05-14T09:46:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.853+0000 7f9e749f7740 1 Processor -- start 2024-05-14T09:46:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e749f7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14682f0 0x55b9c183c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e749f7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14704d0 0x55b9c183cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e749f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14911b0 0x55b9c1843ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e749f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b9c183a930 con 0x55b9c14911b0 2024-05-14T09:46:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e749f7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b9c1754370 con 0x55b9c14682f0 2024-05-14T09:46:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e749f7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b9c183aab0 con 0x55b9c14704d0 2024-05-14T09:46:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e749f7740 10 client.0 _ll_register_callbacks cb 0x55b9c14a8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:46:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14911b0 0x55b9c1843ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14911b0 0x55b9c1843ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34862/0 (socket says 172.21.15.2:34862) 2024-05-14T09:46:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e67fff640 1 -- 172.21.15.2:0/3642307792 learned_addr learned my addr 172.21.15.2:0/3642307792 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e677fe640 1 --2- 172.21.15.2:0/3642307792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14682f0 0x55b9c183c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e66ffd640 1 --2- 172.21.15.2:0/3642307792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14704d0 0x55b9c183cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e67fff640 1 -- 172.21.15.2:0/3642307792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14704d0 msgr2=0x55b9c183cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e67fff640 1 --2- 172.21.15.2:0/3642307792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14704d0 0x55b9c183cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e67fff640 1 -- 172.21.15.2:0/3642307792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14682f0 msgr2=0x55b9c183c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e67fff640 1 --2- 172.21.15.2:0/3642307792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14682f0 0x55b9c183c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e67fff640 1 -- 172.21.15.2:0/3642307792 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c1575620 con 0x55b9c14911b0 2024-05-14T09:46:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e677fe640 1 --2- 172.21.15.2:0/3642307792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14682f0 0x55b9c183c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:46:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e66ffd640 1 --2- 172.21.15.2:0/3642307792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14704d0 0x55b9c183cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:46:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e67fff640 1 --2- 172.21.15.2:0/3642307792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14911b0 0x55b9c1843ab0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9e60006010 tx=0x7f9e600cb680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f83c22be88acfa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.854+0000 7f9e46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:46:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.855+0000 7f9e46ffd640 1 -- 172.21.15.2:0/3642307792 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e600ed040 con 0x55b9c14911b0 2024-05-14T09:46:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.855+0000 7f9e46ffd640 1 -- 172.21.15.2:0/3642307792 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f9e60004660 con 0x55b9c14911b0 2024-05-14T09:46:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.855+0000 7f9e749f7740 1 -- 172.21.15.2:0/3642307792 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e580054d0 con 0x55b9c14911b0 2024-05-14T09:46:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.855+0000 7f9e46ffd640 1 -- 172.21.15.2:0/3642307792 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9e600cf270 con 0x55b9c14911b0 2024-05-14T09:46:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.855+0000 7f9e749f7740 10 client.16698 fetch_fsmap learned FSMap version 45 2024-05-14T09:46:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.855+0000 7f9e749f7740 1 -- 172.21.15.2:0/3642307792 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b9c183a930 con 0x55b9c14911b0 2024-05-14T09:46:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.855+0000 7f9e46ffd640 1 -- 172.21.15.2:0/3642307792 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e580054d0 con 0x55b9c14911b0 2024-05-14T09:46:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.855+0000 7f9e46ffd640 1 -- 172.21.15.2:0/3642307792 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f9e600ca120 con 0x55b9c14911b0 2024-05-14T09:46:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.855+0000 7f9e749f7740 10 client.16698 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T09:46:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.855+0000 7f9e749f7740 10 client.16698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:46:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.855+0000 7f9e749f7740 20 client.16698 populate_metadata read hostname 'smithi002' 2024-05-14T09:46:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.855+0000 7f9e749f7740 1 --2- 172.21.15.2:0/3642307792 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b9c144fd80 0x55b9c184a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.855+0000 7f9e749f7740 4 client.16698 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:46:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.855+0000 7f9e749f7740 1 -- 172.21.15.2:0/3642307792 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9c17a51e0 con 0x55b9c144fd80 2024-05-14T09:46:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.856+0000 7f9e677fe640 1 --2- 172.21.15.2:0/3642307792 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b9c144fd80 0x55b9c184a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.856+0000 7f9e46ffd640 10 client.16698 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.856+0000 7f9e677fe640 1 --2- 172.21.15.2:0/3642307792 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b9c144fd80 0x55b9c184a220 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f02d94c4c0753da server_cookie=d709c0fb2695e844 in_seq=0 out_seq=0 2024-05-14T09:46:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.857+0000 7f9e46ffd640 1 -- 172.21.15.2:0/3642307792 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b9c17a51e0 con 0x55b9c144fd80 2024-05-14T09:46:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.857+0000 7f9e46ffd640 10 client.16698 handle_command_reply: tid=0 2024-05-14T09:46:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.857+0000 7f9e749f7740 1 client.16698 shutdown 2024-05-14T09:46:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.857+0000 7f9e749f7740 1 -- 172.21.15.2:0/3642307792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14911b0 msgr2=0x55b9c1843ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.857+0000 7f9e749f7740 1 --2- 172.21.15.2:0/3642307792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14911b0 0x55b9c1843ab0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9e60006010 tx=0x7f9e600cb680 comp rx=0 tx=0).stop 2024-05-14T09:46:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.857+0000 7f9e749f7740 1 -- 172.21.15.2:0/3642307792 shutdown_connections 2024-05-14T09:46:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.857+0000 7f9e749f7740 1 --2- 172.21.15.2:0/3642307792 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b9c144fd80 0x55b9c184a220 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.857+0000 7f9e749f7740 1 --2- 172.21.15.2:0/3642307792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14911b0 0x55b9c1843ab0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.857+0000 7f9e749f7740 1 --2- 172.21.15.2:0/3642307792 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14704d0 0x55b9c183cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.857+0000 7f9e749f7740 1 --2- 172.21.15.2:0/3642307792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14682f0 0x55b9c183c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.857+0000 7f9e46ffd640 0 client.16698 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.857+0000 7f9e749f7740 1 -- 172.21.15.2:0/3642307792 >> 172.21.15.2:0/3642307792 conn(0x55b9c1485300 msgr2=0x55b9c1831300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.858+0000 7f9e749f7740 1 -- 172.21.15.2:0/3642307792 shutdown_connections 2024-05-14T09:46:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.858+0000 7f9e749f7740 1 -- 172.21.15.2:0/3642307792 wait complete. 2024-05-14T09:46:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.858+0000 7f9e749f7740 20 client.16698 trim_cache size 0 max 16384 2024-05-14T09:46:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.873+0000 7f9e749f7740 1 Processor -- start 2024-05-14T09:46:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.873+0000 7f9e749f7740 1 -- start start 2024-05-14T09:46:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.873+0000 7f9e749f7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14682f0 0x55b9c1844780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.873+0000 7f9e749f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14704d0 0x55b9c18311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.873+0000 7f9e749f7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14911b0 0x55b9c1831730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.873+0000 7f9e749f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9e681b16c0 con 0x55b9c14704d0 2024-05-14T09:46:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.873+0000 7f9e749f7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e681aef00 con 0x55b9c14911b0 2024-05-14T09:46:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.873+0000 7f9e749f7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9e681acd50 con 0x55b9c14682f0 2024-05-14T09:46:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.873+0000 7f9e66ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14704d0 0x55b9c18311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.873+0000 7f9e66ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14704d0 0x55b9c18311f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34868/0 (socket says 172.21.15.2:34868) 2024-05-14T09:46:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.873+0000 7f9e66ffd640 1 -- 172.21.15.2:0/802214500 learned_addr learned my addr 172.21.15.2:0/802214500 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.874+0000 7f9e677fe640 1 --2- 172.21.15.2:0/802214500 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14682f0 0x55b9c1844780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.874+0000 7f9e67fff640 1 --2- 172.21.15.2:0/802214500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14911b0 0x55b9c1831730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.874+0000 7f9e66ffd640 1 -- 172.21.15.2:0/802214500 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14682f0 msgr2=0x55b9c1844780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.874+0000 7f9e66ffd640 1 --2- 172.21.15.2:0/802214500 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14682f0 0x55b9c1844780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.874+0000 7f9e66ffd640 1 -- 172.21.15.2:0/802214500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14911b0 msgr2=0x55b9c1831730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.874+0000 7f9e66ffd640 1 --2- 172.21.15.2:0/802214500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14911b0 0x55b9c1831730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.874+0000 7f9e66ffd640 1 -- 172.21.15.2:0/802214500 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c1833c60 con 0x55b9c14704d0 2024-05-14T09:46:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.874+0000 7f9e67fff640 1 --2- 172.21.15.2:0/802214500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14911b0 0x55b9c1831730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:46:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.874+0000 7f9e66ffd640 1 --2- 172.21.15.2:0/802214500 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14704d0 0x55b9c18311f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00efa0 tx=0x7f9e5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6610db37a10e757b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.874+0000 7f9e4d7fa640 1 -- 172.21.15.2:0/802214500 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c012d10 con 0x55b9c14704d0 2024-05-14T09:46:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.874+0000 7f9e4d7fa640 1 -- 172.21.15.2:0/802214500 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e5c0035f0 con 0x55b9c14704d0 2024-05-14T09:46:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.874+0000 7f9e4d7fa640 1 -- 172.21.15.2:0/802214500 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c020690 con 0x55b9c14704d0 2024-05-14T09:46:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.875+0000 7f9e749f7740 1 -- 172.21.15.2:0/802214500 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14704d0 msgr2=0x55b9c18311f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.875+0000 7f9e749f7740 1 --2- 172.21.15.2:0/802214500 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14704d0 0x55b9c18311f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00efa0 tx=0x7f9e5c0046c0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.875+0000 7f9e749f7740 1 -- 172.21.15.2:0/802214500 shutdown_connections 2024-05-14T09:46:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.875+0000 7f9e749f7740 1 --2- 172.21.15.2:0/802214500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14911b0 0x55b9c1831730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.875+0000 7f9e749f7740 1 --2- 172.21.15.2:0/802214500 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14704d0 0x55b9c18311f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.875+0000 7f9e749f7740 1 --2- 172.21.15.2:0/802214500 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14682f0 0x55b9c1844780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.875+0000 7f9e749f7740 1 -- 172.21.15.2:0/802214500 >> 172.21.15.2:0/802214500 conn(0x55b9c1485300 msgr2=0x55b9c1844070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.875+0000 7f9e749f7740 1 -- 172.21.15.2:0/802214500 shutdown_connections 2024-05-14T09:46:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.875+0000 7f9e749f7740 1 -- 172.21.15.2:0/802214500 wait complete. 2024-05-14T09:46:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.875+0000 7f9e749f7740 1 -- start start 2024-05-14T09:46:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e749f7740 1 Processor -- start 2024-05-14T09:46:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e749f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14682f0 0x55b9c18401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e749f7740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14704d0 0x55b9c1840710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e749f7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14911b0 0x55b9c1841640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e749f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9e681ad3c0 con 0x55b9c14682f0 2024-05-14T09:46:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e749f7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e681ae830 con 0x55b9c14911b0 2024-05-14T09:46:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e749f7740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9e681ada90 con 0x55b9c14704d0 2024-05-14T09:46:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e749f7740 10 client.0 _ll_register_callbacks cb 0x7f9e60009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:46:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e677fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14682f0 0x55b9c18401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e677fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14682f0 0x55b9c18401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34876/0 (socket says 172.21.15.2:34876) 2024-05-14T09:46:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e677fe640 1 -- 172.21.15.2:0/2595009684 learned_addr learned my addr 172.21.15.2:0/2595009684 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e67fff640 1 --2- 172.21.15.2:0/2595009684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14911b0 0x55b9c1841640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e66ffd640 1 --2- 172.21.15.2:0/2595009684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14704d0 0x55b9c1840710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e677fe640 1 -- 172.21.15.2:0/2595009684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14704d0 msgr2=0x55b9c1840710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e677fe640 1 --2- 172.21.15.2:0/2595009684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14704d0 0x55b9c1840710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e677fe640 1 -- 172.21.15.2:0/2595009684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14911b0 msgr2=0x55b9c1841640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e677fe640 1 --2- 172.21.15.2:0/2595009684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14911b0 0x55b9c1841640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e677fe640 1 -- 172.21.15.2:0/2595009684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c1841c00 con 0x55b9c14682f0 2024-05-14T09:46:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.876+0000 7f9e66ffd640 1 --2- 172.21.15.2:0/2595009684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14704d0 0x55b9c1840710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:46:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.877+0000 7f9e677fe640 1 --2- 172.21.15.2:0/2595009684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14682f0 0x55b9c18401d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009b40 tx=0x7f9e58009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbba2e2389704e83 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.877+0000 7f9e46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:46:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.877+0000 7f9e46ffd640 1 -- 172.21.15.2:0/2595009684 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58004250 con 0x55b9c14682f0 2024-05-14T09:46:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.877+0000 7f9e46ffd640 1 -- 172.21.15.2:0/2595009684 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f9e580043f0 con 0x55b9c14682f0 2024-05-14T09:46:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.877+0000 7f9e749f7740 1 -- 172.21.15.2:0/2595009684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e5c0035f0 con 0x55b9c14682f0 2024-05-14T09:46:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.877+0000 7f9e46ffd640 1 -- 172.21.15.2:0/2595009684 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9e580035f0 con 0x55b9c14682f0 2024-05-14T09:46:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.877+0000 7f9e749f7740 10 client.16704 fetch_fsmap learned FSMap version 45 2024-05-14T09:46:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.877+0000 7f9e749f7740 1 -- 172.21.15.2:0/2595009684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b9c1842e90 con 0x55b9c14682f0 2024-05-14T09:46:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.877+0000 7f9e46ffd640 1 -- 172.21.15.2:0/2595009684 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e5c0035f0 con 0x55b9c14682f0 2024-05-14T09:46:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.877+0000 7f9e46ffd640 1 -- 172.21.15.2:0/2595009684 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f9e58024040 con 0x55b9c14682f0 2024-05-14T09:46:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.877+0000 7f9e749f7740 10 client.16704 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T09:46:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.877+0000 7f9e749f7740 10 client.16704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:46:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.877+0000 7f9e749f7740 20 client.16704 populate_metadata read hostname 'smithi002' 2024-05-14T09:46:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.878+0000 7f9e749f7740 1 --2- 172.21.15.2:0/2595009684 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b9c14251c0 0x55b9c18b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.878+0000 7f9e749f7740 4 client.16704 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:46:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.878+0000 7f9e749f7740 1 -- 172.21.15.2:0/2595009684 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9c179f050 con 0x55b9c14251c0 2024-05-14T09:46:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.878+0000 7f9e66ffd640 1 --2- 172.21.15.2:0/2595009684 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b9c14251c0 0x55b9c18b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.878+0000 7f9e66ffd640 1 --2- 172.21.15.2:0/2595009684 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b9c14251c0 0x55b9c18b47b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b29e6804b11c6c1 server_cookie=9ec8527ce8203d8c in_seq=0 out_seq=0 2024-05-14T09:46:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.878+0000 7f9e46ffd640 10 client.16704 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.919+0000 7f9e46ffd640 1 -- 172.21.15.2:0/2595009684 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b9c179f050 con 0x55b9c14251c0 2024-05-14T09:46:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.919+0000 7f9e46ffd640 10 client.16704 handle_command_reply: tid=0 2024-05-14T09:46:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.919+0000 7f9e749f7740 1 client.16704 shutdown 2024-05-14T09:46:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.919+0000 7f9e749f7740 1 -- 172.21.15.2:0/2595009684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14682f0 msgr2=0x55b9c18401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.919+0000 7f9e749f7740 1 --2- 172.21.15.2:0/2595009684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14682f0 0x55b9c18401d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009b40 tx=0x7f9e58009530 comp rx=0 tx=0).stop 2024-05-14T09:46:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.919+0000 7f9e749f7740 1 -- 172.21.15.2:0/2595009684 shutdown_connections 2024-05-14T09:46:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.919+0000 7f9e749f7740 1 --2- 172.21.15.2:0/2595009684 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b9c14251c0 0x55b9c18b47b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.919+0000 7f9e749f7740 1 --2- 172.21.15.2:0/2595009684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b9c14911b0 0x55b9c1841640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.919+0000 7f9e749f7740 1 --2- 172.21.15.2:0/2595009684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b9c14704d0 0x55b9c1840710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.919+0000 7f9e46ffd640 0 client.16704 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.919+0000 7f9e749f7740 1 --2- 172.21.15.2:0/2595009684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9c14682f0 0x55b9c18401d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.919+0000 7f9e749f7740 1 -- 172.21.15.2:0/2595009684 >> 172.21.15.2:0/2595009684 conn(0x55b9c1485300 msgr2=0x55b9c18311f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.919+0000 7f9e749f7740 1 -- 172.21.15.2:0/2595009684 shutdown_connections 2024-05-14T09:46:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.919+0000 7f9e749f7740 1 -- 172.21.15.2:0/2595009684 wait complete. 2024-05-14T09:46:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.919+0000 7f9e749f7740 20 client.16704 trim_cache size 0 max 16384 2024-05-14T09:46:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:46:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:46:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:46:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:46:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.920+0000 7f9e6da67640 1 -- 172.21.15.2:0/2703671164 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9e500b28a0 msgr2=0x7f9e500b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.920+0000 7f9e6da67640 1 --2- 172.21.15.2:0/2703671164 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9e500b28a0 0x7f9e500b4db0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55b9c1833c30 tx=0x7f9e5c00bbb0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.920+0000 7f9e6da67640 1 -- 172.21.15.2:0/2703671164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e68109bd0 msgr2=0x7f9e681a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.920+0000 7f9e6da67640 1 --2- 172.21.15.2:0/2703671164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e68109bd0 0x7f9e681a3970 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9e60005e30 tx=0x7f9e6002f380 comp rx=0 tx=0).stop 2024-05-14T09:46:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.920+0000 7f9e6da67640 1 -- 172.21.15.2:0/2703671164 shutdown_connections 2024-05-14T09:46:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.920+0000 7f9e6da67640 1 --2- 172.21.15.2:0/2703671164 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9e500b28a0 0x7f9e500b4db0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.920+0000 7f9e6da67640 1 --2- 172.21.15.2:0/2703671164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e68109bd0 0x7f9e681a3970 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.920+0000 7f9e6da67640 1 --2- 172.21.15.2:0/2703671164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e68104ed0 0x7f9e6819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.920+0000 7f9e6da67640 1 --2- 172.21.15.2:0/2703671164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e68103c90 0x7f9e6819c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.920+0000 7f9e6da67640 1 -- 172.21.15.2:0/2703671164 >> 172.21.15.2:0/2703671164 conn(0x7f9e680feea0 msgr2=0x7f9e680ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.920+0000 7f9e6da67640 1 -- 172.21.15.2:0/2703671164 shutdown_connections 2024-05-14T09:46:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:43.920+0000 7f9e6da67640 1 -- 172.21.15.2:0/2703671164 wait complete. 2024-05-14T09:46:44.284 INFO:tasks.workunit.client.0.smithi002.stdout: 1 995658 8.40 MB/sec execute 490 sec latency 4031.554 ms 2024-05-14T09:46:44.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:44 smithi002 ceph-mon[31557]: pgmap v778: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-05-14T09:46:44.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:44 smithi029 ceph-mon[36249]: pgmap v778: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-05-14T09:46:44.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:44 smithi026 ceph-mon[36056]: pgmap v778: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-05-14T09:46:45.284 INFO:tasks.workunit.client.0.smithi002.stdout: 1 997702 8.41 MB/sec execute 491 sec latency 13.850 ms 2024-05-14T09:46:46.284 INFO:tasks.workunit.client.0.smithi002.stdout: 1 998943 8.40 MB/sec execute 492 sec latency 14.279 ms 2024-05-14T09:46:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:46 smithi002 ceph-mon[31557]: pgmap v779: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.2 MiB/s wr, 85 op/s 2024-05-14T09:46:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:46 smithi029 ceph-mon[36249]: pgmap v779: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.2 MiB/s wr, 85 op/s 2024-05-14T09:46:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:46 smithi026 ceph-mon[36056]: pgmap v779: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.2 MiB/s wr, 85 op/s 2024-05-14T09:46:47.284 INFO:tasks.workunit.client.0.smithi002.stdout: 1 999046 8.39 MB/sec execute 493 sec latency 911.415 ms 2024-05-14T09:46:48.284 INFO:tasks.workunit.client.0.smithi002.stdout: 1 999046 8.37 MB/sec execute 494 sec latency 1911.538 ms 2024-05-14T09:46:48.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:48 smithi002 ceph-mon[31557]: pgmap v780: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-05-14T09:46:48.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:48 smithi029 ceph-mon[36249]: pgmap v780: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-05-14T09:46:48.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:48 smithi026 ceph-mon[36056]: pgmap v780: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-05-14T09:46:49.284 INFO:tasks.workunit.client.0.smithi002.stdout: 1 999115 8.35 MB/sec execute 495 sec latency 2830.502 ms 2024-05-14T09:46:50.285 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1001082 8.36 MB/sec execute 496 sec latency 14.704 ms 2024-05-14T09:46:50.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:50 smithi002 ceph-mon[31557]: pgmap v781: 129 pgs: 129 active+clean; 169 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-05-14T09:46:50.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:50 smithi029 ceph-mon[36249]: pgmap v781: 129 pgs: 129 active+clean; 169 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-05-14T09:46:50.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:50 smithi026 ceph-mon[36056]: pgmap v781: 129 pgs: 129 active+clean; 169 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-05-14T09:46:51.284 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1002308 8.36 MB/sec execute 497 sec latency 14.172 ms 2024-05-14T09:46:52.285 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1002655 8.34 MB/sec execute 498 sec latency 671.191 ms 2024-05-14T09:46:52.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:52 smithi002 ceph-mon[31557]: pgmap v782: 129 pgs: 129 active+clean; 169 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-05-14T09:46:52.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:52 smithi029 ceph-mon[36249]: pgmap v782: 129 pgs: 129 active+clean; 169 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-05-14T09:46:52.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:52 smithi026 ceph-mon[36056]: pgmap v782: 129 pgs: 129 active+clean; 169 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-05-14T09:46:53.285 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1002655 8.32 MB/sec execute 499 sec latency 1671.246 ms 2024-05-14T09:46:53.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:53 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:46:53.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:53 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:46:53.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:53 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:46:54.285 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1002777 8.31 MB/sec execute 500 sec latency 2589.034 ms 2024-05-14T09:46:54.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:54 smithi002 ceph-mon[31557]: pgmap v783: 129 pgs: 129 active+clean; 170 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-05-14T09:46:54.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:54 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:46:54.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:54 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:46:54.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:54 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:46:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:54 smithi029 ceph-mon[36249]: pgmap v783: 129 pgs: 129 active+clean; 170 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-05-14T09:46:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:54 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:46:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:54 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:46:54.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:54 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:46:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:54 smithi026 ceph-mon[36056]: pgmap v783: 129 pgs: 129 active+clean; 170 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-05-14T09:46:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:54 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:46:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:54 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:46:54.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:54 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:46:55.285 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1004781 8.31 MB/sec execute 501 sec latency 20.058 ms 2024-05-14T09:46:56.285 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1006080 8.31 MB/sec execute 502 sec latency 74.320 ms 2024-05-14T09:46:56.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:56 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:46:56.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:56 smithi002 ceph-mon[31557]: pgmap v784: 129 pgs: 129 active+clean; 170 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 363 KiB/s wr, 86 op/s 2024-05-14T09:46:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:56 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:46:56.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:56 smithi029 ceph-mon[36249]: pgmap v784: 129 pgs: 129 active+clean; 170 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 363 KiB/s wr, 86 op/s 2024-05-14T09:46:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:56 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:46:56.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:56 smithi026 ceph-mon[36056]: pgmap v784: 129 pgs: 129 active+clean; 170 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 363 KiB/s wr, 86 op/s 2024-05-14T09:46:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.023+0000 7f543122b640 1 -- 172.21.15.2:0/1036372172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f542c1022d0 msgr2=0x7f542c102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.023+0000 7f543122b640 1 --2- 172.21.15.2:0/1036372172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f542c1022d0 0x7f542c102fd0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f541c009900 tx=0x7f541c02a410 comp rx=0 tx=0).stop 2024-05-14T09:46:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.023+0000 7f543122b640 1 -- 172.21.15.2:0/1036372172 shutdown_connections 2024-05-14T09:46:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.023+0000 7f543122b640 1 --2- 172.21.15.2:0/1036372172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542c108210 0x7f542c10a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.023+0000 7f543122b640 1 --2- 172.21.15.2:0/1036372172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f542c103510 0x7f542c103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.023+0000 7f543122b640 1 --2- 172.21.15.2:0/1036372172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f542c1022d0 0x7f542c102fd0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.023+0000 7f543122b640 1 -- 172.21.15.2:0/1036372172 >> 172.21.15.2:0/1036372172 conn(0x7f542c0fd4e0 msgr2=0x7f542c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.023+0000 7f543122b640 1 -- 172.21.15.2:0/1036372172 shutdown_connections 2024-05-14T09:46:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.023+0000 7f543122b640 1 -- 172.21.15.2:0/1036372172 wait complete. 2024-05-14T09:46:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.024+0000 7f543122b640 1 Processor -- start 2024-05-14T09:46:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.024+0000 7f543122b640 1 -- start start 2024-05-14T09:46:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.024+0000 7f543122b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f542c1022d0 0x7f542c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.024+0000 7f543122b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f542c103510 0x7f542c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.024+0000 7f543122b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542c108210 0x7f542c1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f543122b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f542c10d560 con 0x7f542c103510 2024-05-14T09:46:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f543122b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f542c10d3e0 con 0x7f542c108210 2024-05-14T09:46:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f543122b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f542c10d6e0 con 0x7f542c1022d0 2024-05-14T09:46:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f542259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f542c103510 0x7f542c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f542259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f542c103510 0x7f542c19a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42840/0 (socket says 172.21.15.2:42840) 2024-05-14T09:46:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f542259c640 1 -- 172.21.15.2:0/3888418686 learned_addr learned my addr 172.21.15.2:0/3888418686 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f542ad9d640 1 --2- 172.21.15.2:0/3888418686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f542c1022d0 0x7f542c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f542b59e640 1 --2- 172.21.15.2:0/3888418686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542c108210 0x7f542c1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f542259c640 1 -- 172.21.15.2:0/3888418686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f542c1022d0 msgr2=0x7f542c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f542259c640 1 --2- 172.21.15.2:0/3888418686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f542c1022d0 0x7f542c19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f542259c640 1 -- 172.21.15.2:0/3888418686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542c108210 msgr2=0x7f542c1a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f542259c640 1 --2- 172.21.15.2:0/3888418686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542c108210 0x7f542c1a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f542259c640 1 -- 172.21.15.2:0/3888418686 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542c1a1e50 con 0x7f542c103510 2024-05-14T09:46:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f542259c640 1 --2- 172.21.15.2:0/3888418686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f542c103510 0x7f542c19a850 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f541400d770 tx=0x7f541400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f5428d99640 1 -- 172.21.15.2:0/3888418686 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414017070 con 0x7f542c103510 2024-05-14T09:46:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f5428d99640 1 -- 172.21.15.2:0/3888418686 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54140054d0 con 0x7f542c103510 2024-05-14T09:46:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f543122b640 1 -- 172.21.15.2:0/3888418686 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542c1a2140 con 0x7f542c103510 2024-05-14T09:46:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f5428d99640 1 -- 172.21.15.2:0/3888418686 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414004c30 con 0x7f542c103510 2024-05-14T09:46:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.025+0000 7f543122b640 1 -- 172.21.15.2:0/3888418686 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f542c1a2640 con 0x7f542c103510 2024-05-14T09:46:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.028+0000 7f54381be740 1 Processor -- start 2024-05-14T09:46:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.028+0000 7f54381be740 1 -- start start 2024-05-14T09:46:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.028+0000 7f5428d99640 1 -- 172.21.15.2:0/3888418686 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5414004e10 con 0x7f542c103510 2024-05-14T09:46:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f54381be740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 0x55e3ebc2de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f54381be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc2e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f54381be740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8674d0 0x55e3ebc352f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f54381be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e3ebb8e5d0 con 0x55e3eb85f2f0 2024-05-14T09:46:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f54381be740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e3ebb47890 con 0x55e3eb8881b0 2024-05-14T09:46:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f54381be740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e3ebb8a590 con 0x55e3eb8674d0 2024-05-14T09:46:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f542ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc2e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f542ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc2e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42850/0 (socket says 172.21.15.2:42850) 2024-05-14T09:46:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f542ad9d640 1 -- 172.21.15.2:0/4265135510 learned_addr learned my addr 172.21.15.2:0/4265135510 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f542259c640 1 --2- 172.21.15.2:0/4265135510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8674d0 0x55e3ebc352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f542b59e640 1 --2- 172.21.15.2:0/4265135510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 0x55e3ebc2de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f542ad9d640 1 -- 172.21.15.2:0/4265135510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8674d0 msgr2=0x55e3ebc352f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f542ad9d640 1 --2- 172.21.15.2:0/4265135510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8674d0 0x55e3ebc352f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f542ad9d640 1 -- 172.21.15.2:0/4265135510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 msgr2=0x55e3ebc2de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f542ad9d640 1 --2- 172.21.15.2:0/4265135510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 0x55e3ebc2de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f542ad9d640 1 -- 172.21.15.2:0/4265135510 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3eb8a0210 con 0x55e3eb85f2f0 2024-05-14T09:46:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f542ad9d640 1 --2- 172.21.15.2:0/4265135510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc2e3f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f541c009230 tx=0x7f541c031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85193d3963a2318b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f5428d99640 1 --2- 172.21.15.2:0/3888418686 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f54180b28a0 0x7f54180b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.029+0000 7f542159a640 1 -- 172.21.15.2:0/4265135510 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c035030 con 0x55e3eb85f2f0 2024-05-14T09:46:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.030+0000 7f542159a640 1 -- 172.21.15.2:0/4265135510 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f541c040070 con 0x55e3eb85f2f0 2024-05-14T09:46:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.030+0000 7f542159a640 1 -- 172.21.15.2:0/4265135510 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c003a20 con 0x55e3eb85f2f0 2024-05-14T09:46:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.030+0000 7f542b59e640 1 --2- 172.21.15.2:0/3888418686 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f54180b28a0 0x7f54180b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.030+0000 7f5428d99640 1 -- 172.21.15.2:0/3888418686 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f54140e20a0 con 0x7f542c103510 2024-05-14T09:46:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.030+0000 7f54381be740 1 -- 172.21.15.2:0/4265135510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb85f2f0 msgr2=0x55e3ebc2e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.030+0000 7f542b59e640 1 --2- 172.21.15.2:0/3888418686 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f54180b28a0 0x7f54180b4db0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55e3ebc2f040 tx=0x7f542400a6b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.030+0000 7f54381be740 1 --2- 172.21.15.2:0/4265135510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc2e3f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f541c009230 tx=0x7f541c031bc0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.031+0000 7f54381be740 1 -- 172.21.15.2:0/4265135510 shutdown_connections 2024-05-14T09:46:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.031+0000 7f54381be740 1 --2- 172.21.15.2:0/4265135510 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8674d0 0x55e3ebc352f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.031+0000 7f54381be740 1 --2- 172.21.15.2:0/4265135510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc2e3f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.031+0000 7f54381be740 1 --2- 172.21.15.2:0/4265135510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 0x55e3ebc2de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.031+0000 7f54381be740 1 -- 172.21.15.2:0/4265135510 >> 172.21.15.2:0/4265135510 conn(0x55e3eb87c300 msgr2=0x55e3ebc253a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.031+0000 7f54381be740 1 -- 172.21.15.2:0/4265135510 shutdown_connections 2024-05-14T09:46:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.031+0000 7f54381be740 1 -- 172.21.15.2:0/4265135510 wait complete. 2024-05-14T09:46:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.031+0000 7f54381be740 1 -- start start 2024-05-14T09:46:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.031+0000 7f54381be740 1 Processor -- start 2024-05-14T09:46:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.031+0000 7f54381be740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc37a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.031+0000 7f54381be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb8674d0 0x55e3ebc37f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.031+0000 7f54381be740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 0x55e3ebc3eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.031+0000 7f54381be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e3ebc35ce0 con 0x55e3eb8674d0 2024-05-14T09:46:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.031+0000 7f54381be740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e3ebb4b370 con 0x55e3eb8881b0 2024-05-14T09:46:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.031+0000 7f54381be740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e3ebc35e60 con 0x55e3eb85f2f0 2024-05-14T09:46:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f542259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb8674d0 0x55e3ebc37f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f54381be740 10 client.0 _ll_register_callbacks cb 0x55e3eb89fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:46:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f542259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb8674d0 0x55e3ebc37f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42864/0 (socket says 172.21.15.2:42864) 2024-05-14T09:46:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f542259c640 1 -- 172.21.15.2:0/2087808081 learned_addr learned my addr 172.21.15.2:0/2087808081 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f542ad9d640 1 --2- 172.21.15.2:0/2087808081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc37a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f542b59e640 1 --2- 172.21.15.2:0/2087808081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 0x55e3ebc3eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f542259c640 1 -- 172.21.15.2:0/2087808081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb85f2f0 msgr2=0x55e3ebc37a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f542259c640 1 --2- 172.21.15.2:0/2087808081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc37a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f542259c640 1 -- 172.21.15.2:0/2087808081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 msgr2=0x55e3ebc3eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f542259c640 1 --2- 172.21.15.2:0/2087808081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 0x55e3ebc3eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f542259c640 1 -- 172.21.15.2:0/2087808081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3eb96c620 con 0x55e3eb8674d0 2024-05-14T09:46:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f542259c640 1 --2- 172.21.15.2:0/2087808081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb8674d0 0x55e3ebc37f90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f541400d740 tx=0x7f54140a52d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0df997d0c3017c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f54127fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:46:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f54127fc640 1 -- 172.21.15.2:0/2087808081 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414020070 con 0x55e3eb8674d0 2024-05-14T09:46:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f54127fc640 1 -- 172.21.15.2:0/2087808081 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f54140a55e0 con 0x55e3eb8674d0 2024-05-14T09:46:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f54381be740 1 -- 172.21.15.2:0/2087808081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f541c040070 con 0x55e3eb8674d0 2024-05-14T09:46:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f54127fc640 1 -- 172.21.15.2:0/2087808081 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f54140a8e60 con 0x55e3eb8674d0 2024-05-14T09:46:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f54381be740 10 client.16716 fetch_fsmap learned FSMap version 46 2024-05-14T09:46:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.032+0000 7f54381be740 1 -- 172.21.15.2:0/2087808081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e3ebc41e90 con 0x55e3eb8674d0 2024-05-14T09:46:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.033+0000 7f54127fc640 1 -- 172.21.15.2:0/2087808081 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f541c040070 con 0x55e3eb8674d0 2024-05-14T09:46:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.033+0000 7f54127fc640 1 -- 172.21.15.2:0/2087808081 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f54140b1c70 con 0x55e3eb8674d0 2024-05-14T09:46:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.033+0000 7f54381be740 10 client.16716 fetch_fsmap finished waiting for FSMap version 46 2024-05-14T09:46:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.033+0000 7f54381be740 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:46:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.033+0000 7f54381be740 20 client.16716 populate_metadata read hostname 'smithi002' 2024-05-14T09:46:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.033+0000 7f54381be740 1 --2- 172.21.15.2:0/2087808081 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e3eb846d80 0x55e3ebc45660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.033+0000 7f54381be740 4 client.16716 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:46:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.033+0000 7f54381be740 1 -- 172.21.15.2:0/2087808081 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3ebb9c1e0 con 0x55e3eb846d80 2024-05-14T09:46:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.033+0000 7f542ad9d640 1 --2- 172.21.15.2:0/2087808081 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e3eb846d80 0x55e3ebc45660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.034+0000 7f542ad9d640 1 --2- 172.21.15.2:0/2087808081 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e3eb846d80 0x55e3ebc45660 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e46d02504fb55791 server_cookie=d111db97928b673c in_seq=0 out_seq=0 2024-05-14T09:46:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.034+0000 7f54127fc640 10 client.16716 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.034+0000 7f54127fc640 1 -- 172.21.15.2:0/2087808081 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e3ebb9c1e0 con 0x55e3eb846d80 2024-05-14T09:46:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.034+0000 7f54127fc640 10 client.16716 handle_command_reply: tid=0 2024-05-14T09:46:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.034+0000 7f54381be740 1 client.16716 shutdown 2024-05-14T09:46:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.034+0000 7f54381be740 1 -- 172.21.15.2:0/2087808081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb8674d0 msgr2=0x55e3ebc37f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.034+0000 7f54381be740 1 --2- 172.21.15.2:0/2087808081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb8674d0 0x55e3ebc37f90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f541400d740 tx=0x7f54140a52d0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.034+0000 7f54381be740 1 -- 172.21.15.2:0/2087808081 shutdown_connections 2024-05-14T09:46:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.034+0000 7f54381be740 1 --2- 172.21.15.2:0/2087808081 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e3eb846d80 0x55e3ebc45660 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.034+0000 7f54381be740 1 --2- 172.21.15.2:0/2087808081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 0x55e3ebc3eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.034+0000 7f54381be740 1 --2- 172.21.15.2:0/2087808081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb8674d0 0x55e3ebc37f90 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.034+0000 7f54127fc640 0 client.16716 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.034+0000 7f54381be740 1 --2- 172.21.15.2:0/2087808081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc37a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.034+0000 7f54381be740 1 -- 172.21.15.2:0/2087808081 >> 172.21.15.2:0/2087808081 conn(0x55e3eb87c300 msgr2=0x55e3ebc28300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.035+0000 7f54381be740 1 -- 172.21.15.2:0/2087808081 shutdown_connections 2024-05-14T09:46:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.035+0000 7f54381be740 1 -- 172.21.15.2:0/2087808081 wait complete. 2024-05-14T09:46:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.035+0000 7f54381be740 20 client.16716 trim_cache size 0 max 16384 2024-05-14T09:46:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.050+0000 7f54381be740 1 Processor -- start 2024-05-14T09:46:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.050+0000 7f54381be740 1 -- start start 2024-05-14T09:46:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.050+0000 7f54381be740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc24b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.050+0000 7f54381be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb8674d0 0x55e3ebc3ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.050+0000 7f54381be740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8881b0 0x55e3ebc25060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.050+0000 7f54381be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f542c1ad3e0 con 0x55e3eb8674d0 2024-05-14T09:46:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.050+0000 7f54381be740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f542c1aac20 con 0x55e3eb85f2f0 2024-05-14T09:46:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.050+0000 7f54381be740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f542c1a8a70 con 0x55e3eb8881b0 2024-05-14T09:46:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.050+0000 7f542259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb8674d0 0x55e3ebc3ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.050+0000 7f542259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb8674d0 0x55e3ebc3ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42870/0 (socket says 172.21.15.2:42870) 2024-05-14T09:46:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.050+0000 7f542259c640 1 -- 172.21.15.2:0/2930389624 learned_addr learned my addr 172.21.15.2:0/2930389624 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.050+0000 7f542b59e640 1 --2- 172.21.15.2:0/2930389624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8881b0 0x55e3ebc25060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.050+0000 7f542259c640 1 -- 172.21.15.2:0/2930389624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8881b0 msgr2=0x55e3ebc25060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.050+0000 7f542259c640 1 --2- 172.21.15.2:0/2930389624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8881b0 0x55e3ebc25060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.051+0000 7f542259c640 1 -- 172.21.15.2:0/2930389624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb85f2f0 msgr2=0x55e3ebc24b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.051+0000 7f542ad9d640 1 --2- 172.21.15.2:0/2930389624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc24b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.051+0000 7f542259c640 1 --2- 172.21.15.2:0/2930389624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc24b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.051+0000 7f542259c640 1 -- 172.21.15.2:0/2930389624 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3ebc32580 con 0x55e3eb8674d0 2024-05-14T09:46:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.051+0000 7f542b59e640 1 --2- 172.21.15.2:0/2930389624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8881b0 0x55e3ebc25060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:46:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.051+0000 7f542259c640 1 --2- 172.21.15.2:0/2930389624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb8674d0 0x55e3ebc3ee10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f54140a92e0 tx=0x7f54140e5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f328292156abd3bd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.051+0000 7f5420d99640 1 -- 172.21.15.2:0/2930389624 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414020070 con 0x55e3eb8674d0 2024-05-14T09:46:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.051+0000 7f5420d99640 1 -- 172.21.15.2:0/2930389624 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54140a9350 con 0x55e3eb8674d0 2024-05-14T09:46:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.051+0000 7f5420d99640 1 -- 172.21.15.2:0/2930389624 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54140a8810 con 0x55e3eb8674d0 2024-05-14T09:46:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.052+0000 7f54381be740 1 -- 172.21.15.2:0/2930389624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb8674d0 msgr2=0x55e3ebc3ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.052+0000 7f54381be740 1 --2- 172.21.15.2:0/2930389624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb8674d0 0x55e3ebc3ee10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f54140a92e0 tx=0x7f54140e5920 comp rx=0 tx=0).stop 2024-05-14T09:46:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.052+0000 7f54381be740 1 -- 172.21.15.2:0/2930389624 shutdown_connections 2024-05-14T09:46:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.052+0000 7f54381be740 1 --2- 172.21.15.2:0/2930389624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8881b0 0x55e3ebc25060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.052+0000 7f54381be740 1 --2- 172.21.15.2:0/2930389624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb8674d0 0x55e3ebc3ee10 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.052+0000 7f54381be740 1 --2- 172.21.15.2:0/2930389624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc24b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.052+0000 7f54381be740 1 -- 172.21.15.2:0/2930389624 >> 172.21.15.2:0/2930389624 conn(0x55e3eb87c300 msgr2=0x55e3ebc31e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.052+0000 7f54381be740 1 -- 172.21.15.2:0/2930389624 shutdown_connections 2024-05-14T09:46:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.052+0000 7f54381be740 1 -- 172.21.15.2:0/2930389624 wait complete. 2024-05-14T09:46:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.052+0000 7f54381be740 1 -- start start 2024-05-14T09:46:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.052+0000 7f54381be740 1 Processor -- start 2024-05-14T09:46:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f54381be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc3c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f54381be740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8674d0 0x55e3ebc3c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f54381be740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 0x55e3ebc3d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f54381be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f542c1a90e0 con 0x55e3eb85f2f0 2024-05-14T09:46:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f54381be740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f542c1aa550 con 0x55e3eb8881b0 2024-05-14T09:46:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f54381be740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f542c1a97b0 con 0x55e3eb8674d0 2024-05-14T09:46:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f54381be740 10 client.0 _ll_register_callbacks cb 0x7f54140e5870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:46:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f542ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc3c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f542ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc3c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42884/0 (socket says 172.21.15.2:42884) 2024-05-14T09:46:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f542ad9d640 1 -- 172.21.15.2:0/3672766170 learned_addr learned my addr 172.21.15.2:0/3672766170 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:46:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f542259c640 1 --2- 172.21.15.2:0/3672766170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8674d0 0x55e3ebc3c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f542b59e640 1 --2- 172.21.15.2:0/3672766170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 0x55e3ebc3d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f542ad9d640 1 -- 172.21.15.2:0/3672766170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8674d0 msgr2=0x55e3ebc3c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f542ad9d640 1 --2- 172.21.15.2:0/3672766170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8674d0 0x55e3ebc3c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f542ad9d640 1 -- 172.21.15.2:0/3672766170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 msgr2=0x55e3ebc3d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:46:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f542ad9d640 1 --2- 172.21.15.2:0/3672766170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 0x55e3ebc3d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f542ad9d640 1 -- 172.21.15.2:0/3672766170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3ebc3db10 con 0x55e3eb85f2f0 2024-05-14T09:46:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f542b59e640 1 --2- 172.21.15.2:0/3672766170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 0x55e3ebc3d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:46:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f542ad9d640 1 --2- 172.21.15.2:0/3672766170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc3c0e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f541c005f70 tx=0x7f541c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632bb5d688b10179 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:46:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f54127fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:46:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f54127fc640 1 -- 172.21.15.2:0/3672766170 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c035040 con 0x55e3eb85f2f0 2024-05-14T09:46:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f54127fc640 1 -- 172.21.15.2:0/3672766170 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f541c004450 con 0x55e3eb85f2f0 2024-05-14T09:46:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.053+0000 7f54381be740 1 -- 172.21.15.2:0/3672766170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54140a9350 con 0x55e3eb85f2f0 2024-05-14T09:46:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.054+0000 7f54127fc640 1 -- 172.21.15.2:0/3672766170 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f541c05f050 con 0x55e3eb85f2f0 2024-05-14T09:46:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.054+0000 7f54381be740 10 client.16722 fetch_fsmap learned FSMap version 46 2024-05-14T09:46:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.054+0000 7f54381be740 1 -- 172.21.15.2:0/3672766170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e3ebc3eda0 con 0x55e3eb85f2f0 2024-05-14T09:46:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.054+0000 7f54127fc640 1 -- 172.21.15.2:0/3672766170 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54140a9350 con 0x55e3eb85f2f0 2024-05-14T09:46:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.054+0000 7f54127fc640 1 -- 172.21.15.2:0/3672766170 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f541c05a030 con 0x55e3eb85f2f0 2024-05-14T09:46:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.054+0000 7f54381be740 10 client.16722 fetch_fsmap finished waiting for FSMap version 46 2024-05-14T09:46:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.054+0000 7f54381be740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:46:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.054+0000 7f54381be740 20 client.16722 populate_metadata read hostname 'smithi002' 2024-05-14T09:46:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.054+0000 7f54381be740 1 --2- 172.21.15.2:0/3672766170 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e3eb81c1c0 0x55e3ebcafb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:46:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.054+0000 7f54381be740 4 client.16722 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:46:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.054+0000 7f54381be740 1 -- 172.21.15.2:0/3672766170 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3ebb96050 con 0x55e3eb81c1c0 2024-05-14T09:46:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.054+0000 7f542259c640 1 --2- 172.21.15.2:0/3672766170 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e3eb81c1c0 0x55e3ebcafb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:46:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.055+0000 7f54127fc640 10 client.16722 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.055+0000 7f542259c640 1 --2- 172.21.15.2:0/3672766170 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e3eb81c1c0 0x55e3ebcafb80 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=add28ea07ddc1b62 server_cookie=42893437716287be in_seq=0 out_seq=0 2024-05-14T09:46:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.115+0000 7f54127fc640 1 -- 172.21.15.2:0/3672766170 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e3ebb96050 con 0x55e3eb81c1c0 2024-05-14T09:46:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.115+0000 7f54127fc640 10 client.16722 handle_command_reply: tid=0 2024-05-14T09:46:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.115+0000 7f54381be740 1 client.16722 shutdown 2024-05-14T09:46:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f54381be740 1 -- 172.21.15.2:0/3672766170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb85f2f0 msgr2=0x55e3ebc3c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f54381be740 1 --2- 172.21.15.2:0/3672766170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc3c0e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f541c005f70 tx=0x7f541c0046c0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f54381be740 1 -- 172.21.15.2:0/3672766170 shutdown_connections 2024-05-14T09:46:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f54381be740 1 --2- 172.21.15.2:0/3672766170 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e3eb81c1c0 0x55e3ebcafb80 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f54381be740 1 --2- 172.21.15.2:0/3672766170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e3eb8881b0 0x55e3ebc3d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f54381be740 1 --2- 172.21.15.2:0/3672766170 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e3eb8674d0 0x55e3ebc3c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f54381be740 1 --2- 172.21.15.2:0/3672766170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e3eb85f2f0 0x55e3ebc3c0e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f54127fc640 0 client.16722 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:46:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f54381be740 1 -- 172.21.15.2:0/3672766170 >> 172.21.15.2:0/3672766170 conn(0x55e3eb87c300 msgr2=0x55e3ebc31e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f54381be740 1 -- 172.21.15.2:0/3672766170 shutdown_connections 2024-05-14T09:46:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f54381be740 1 -- 172.21.15.2:0/3672766170 wait complete. 2024-05-14T09:46:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f54381be740 20 client.16722 trim_cache size 0 max 16384 2024-05-14T09:46:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:46:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:46:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:46:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:46:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f543122b640 1 -- 172.21.15.2:0/3888418686 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f54180b28a0 msgr2=0x7f54180b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f543122b640 1 --2- 172.21.15.2:0/3888418686 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f54180b28a0 0x7f54180b4db0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55e3ebc2f040 tx=0x7f542400a6b0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f543122b640 1 -- 172.21.15.2:0/3888418686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f542c103510 msgr2=0x7f542c19a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:46:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f543122b640 1 --2- 172.21.15.2:0/3888418686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f542c103510 0x7f542c19a850 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f541400d770 tx=0x7f541400dc30 comp rx=0 tx=0).stop 2024-05-14T09:46:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f543122b640 1 -- 172.21.15.2:0/3888418686 shutdown_connections 2024-05-14T09:46:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f543122b640 1 --2- 172.21.15.2:0/3888418686 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f54180b28a0 0x7f54180b4db0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f543122b640 1 --2- 172.21.15.2:0/3888418686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542c108210 0x7f542c1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f543122b640 1 --2- 172.21.15.2:0/3888418686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f542c103510 0x7f542c19a850 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f543122b640 1 --2- 172.21.15.2:0/3888418686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f542c1022d0 0x7f542c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:46:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.116+0000 7f543122b640 1 -- 172.21.15.2:0/3888418686 >> 172.21.15.2:0/3888418686 conn(0x7f542c0fd4e0 msgr2=0x7f542c106200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:46:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.117+0000 7f543122b640 1 -- 172.21.15.2:0/3888418686 shutdown_connections 2024-05-14T09:46:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:46:57.117+0000 7f543122b640 1 -- 172.21.15.2:0/3888418686 wait complete. 2024-05-14T09:46:57.285 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1006080 8.29 MB/sec execute 503 sec latency 1074.419 ms 2024-05-14T09:46:58.285 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1006080 8.28 MB/sec execute 504 sec latency 2074.489 ms 2024-05-14T09:46:58.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:46:58 smithi002 ceph-mon[31557]: pgmap v785: 129 pgs: 129 active+clean; 171 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 450 KiB/s wr, 107 op/s 2024-05-14T09:46:58.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:46:58 smithi029 ceph-mon[36249]: pgmap v785: 129 pgs: 129 active+clean; 171 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 450 KiB/s wr, 107 op/s 2024-05-14T09:46:58.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:46:58 smithi026 ceph-mon[36056]: pgmap v785: 129 pgs: 129 active+clean; 171 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 450 KiB/s wr, 107 op/s 2024-05-14T09:46:59.285 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1006136 8.26 MB/sec execute 505 sec latency 2991.895 ms 2024-05-14T09:47:00.285 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1006136 8.25 MB/sec execute 506 sec latency 1026.137 ms 2024-05-14T09:47:00.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:00 smithi002 ceph-mon[31557]: pgmap v786: 129 pgs: 129 active+clean; 172 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 454 KiB/s wr, 114 op/s 2024-05-14T09:47:00.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:00 smithi029 ceph-mon[36249]: pgmap v786: 129 pgs: 129 active+clean; 172 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 454 KiB/s wr, 114 op/s 2024-05-14T09:47:00.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:00 smithi026 ceph-mon[36056]: pgmap v786: 129 pgs: 129 active+clean; 172 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 454 KiB/s wr, 114 op/s 2024-05-14T09:47:01.285 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1006136 8.23 MB/sec execute 507 sec latency 2026.207 ms 2024-05-14T09:47:02.286 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1006136 8.21 MB/sec execute 508 sec latency 3026.246 ms 2024-05-14T09:47:02.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:02 smithi002 ceph-mon[31557]: pgmap v787: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 374 KiB/s wr, 99 op/s 2024-05-14T09:47:02.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:02 smithi029 ceph-mon[36249]: pgmap v787: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 374 KiB/s wr, 99 op/s 2024-05-14T09:47:02.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:02 smithi026 ceph-mon[36056]: pgmap v787: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 374 KiB/s wr, 99 op/s 2024-05-14T09:47:03.286 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1006136 8.20 MB/sec execute 509 sec latency 4026.345 ms 2024-05-14T09:47:04.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1006204 8.18 MB/sec execute 510 sec latency 4943.607 ms 2024-05-14T09:47:04.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:04 smithi002 ceph-mon[31557]: pgmap v788: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 346 KiB/s wr, 91 op/s 2024-05-14T09:47:04.688 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:04 smithi029 ceph-mon[36249]: pgmap v788: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 346 KiB/s wr, 91 op/s 2024-05-14T09:47:04.702 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:04 smithi026 ceph-mon[36056]: pgmap v788: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 346 KiB/s wr, 91 op/s 2024-05-14T09:47:05.286 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1006204 8.16 MB/sec execute 511 sec latency 1011.540 ms 2024-05-14T09:47:06.286 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1006300 8.15 MB/sec execute 512 sec latency 1925.287 ms 2024-05-14T09:47:06.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:06 smithi002 ceph-mon[31557]: pgmap v789: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 206 KiB/s wr, 66 op/s 2024-05-14T09:47:06.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:06 smithi029 ceph-mon[36249]: pgmap v789: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 206 KiB/s wr, 66 op/s 2024-05-14T09:47:06.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:06 smithi026 ceph-mon[36056]: pgmap v789: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 206 KiB/s wr, 66 op/s 2024-05-14T09:47:07.286 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1008188 8.15 MB/sec execute 513 sec latency 24.739 ms 2024-05-14T09:47:08.286 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1009577 8.15 MB/sec execute 514 sec latency 13.888 ms 2024-05-14T09:47:08.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:08 smithi002 ceph-mon[31557]: pgmap v790: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 586 KiB/s wr, 80 op/s 2024-05-14T09:47:08.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:08 smithi029 ceph-mon[36249]: pgmap v790: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 586 KiB/s wr, 80 op/s 2024-05-14T09:47:08.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:08 smithi026 ceph-mon[36056]: pgmap v790: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 586 KiB/s wr, 80 op/s 2024-05-14T09:47:09.286 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1009680 8.14 MB/sec execute 515 sec latency 898.802 ms 2024-05-14T09:47:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.230+0000 7f81fbdd5640 1 -- 172.21.15.2:0/2293472879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4102e80 msgr2=0x7f81f4103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.230+0000 7f81fbdd5640 1 --2- 172.21.15.2:0/2293472879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4102e80 0x7f81f4103290 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009930 tx=0x7f81e802a420 comp rx=0 tx=0).stop 2024-05-14T09:47:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.230+0000 7f81fbdd5640 1 -- 172.21.15.2:0/2293472879 shutdown_connections 2024-05-14T09:47:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.230+0000 7f81fbdd5640 1 --2- 172.21.15.2:0/2293472879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f41037d0 0x7f81f4103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.230+0000 7f81fbdd5640 1 --2- 172.21.15.2:0/2293472879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4102e80 0x7f81f4103290 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.230+0000 7f81fbdd5640 1 --2- 172.21.15.2:0/2293472879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81f4102530 0x7f81f4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.230+0000 7f81fbdd5640 1 -- 172.21.15.2:0/2293472879 >> 172.21.15.2:0/2293472879 conn(0x7f81f40fdd40 msgr2=0x7f81f41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.230+0000 7f81fbdd5640 1 -- 172.21.15.2:0/2293472879 shutdown_connections 2024-05-14T09:47:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.230+0000 7f81fbdd5640 1 -- 172.21.15.2:0/2293472879 wait complete. 2024-05-14T09:47:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.230+0000 7f81fbdd5640 1 Processor -- start 2024-05-14T09:47:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81fbdd5640 1 -- start start 2024-05-14T09:47:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81fbdd5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f4102530 0x7f81f419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81fbdd5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4102e80 0x7f81f419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81fbdd5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81f41037d0 0x7f81f41a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81fbdd5640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f81f4111980 con 0x7f81f4102e80 2024-05-14T09:47:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81fbdd5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f81f4111800 con 0x7f81f4102530 2024-05-14T09:47:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81fbdd5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f81f4111b00 con 0x7f81f41037d0 2024-05-14T09:47:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81f9370640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4102e80 0x7f81f419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81f9370640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4102e80 0x7f81f419ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60438/0 (socket says 172.21.15.2:60438) 2024-05-14T09:47:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81f9370640 1 -- 172.21.15.2:0/319402819 learned_addr learned my addr 172.21.15.2:0/319402819 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81f9b71640 1 --2- 172.21.15.2:0/319402819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f4102530 0x7f81f419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81fa372640 1 --2- 172.21.15.2:0/319402819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81f41037d0 0x7f81f41a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81f9370640 1 -- 172.21.15.2:0/319402819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81f41037d0 msgr2=0x7f81f41a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81f9370640 1 --2- 172.21.15.2:0/319402819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81f41037d0 0x7f81f41a39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81f9370640 1 -- 172.21.15.2:0/319402819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f4102530 msgr2=0x7f81f419c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81f9370640 1 --2- 172.21.15.2:0/319402819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f4102530 0x7f81f419c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.231+0000 7f81f9370640 1 -- 172.21.15.2:0/319402819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f41a4060 con 0x7f81f4102e80 2024-05-14T09:47:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.232+0000 7f81f9370640 1 --2- 172.21.15.2:0/319402819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4102e80 0x7f81f419ca60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009230 tx=0x7f81e8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.232+0000 7f81e6ffd640 1 -- 172.21.15.2:0/319402819 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e803f030 con 0x7f81f4102e80 2024-05-14T09:47:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.232+0000 7f81e6ffd640 1 -- 172.21.15.2:0/319402819 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e8003aa0 con 0x7f81f4102e80 2024-05-14T09:47:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.232+0000 7f81fbdd5640 1 -- 172.21.15.2:0/319402819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f41a42f0 con 0x7f81f4102e80 2024-05-14T09:47:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.232+0000 7f81e6ffd640 1 -- 172.21.15.2:0/319402819 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8035070 con 0x7f81f4102e80 2024-05-14T09:47:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.232+0000 7f81fbdd5640 1 -- 172.21.15.2:0/319402819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f41a4830 con 0x7f81f4102e80 2024-05-14T09:47:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.235+0000 7f8202d65740 1 Processor -- start 2024-05-14T09:47:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.235+0000 7f8202d65740 1 -- start start 2024-05-14T09:47:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.235+0000 7f8202d65740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.235+0000 7f8202d65740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacdd72f0 0x562aad1a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.235+0000 7f8202d65740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacddf4d0 0x562aad1a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.235+0000 7f81e6ffd640 1 -- 172.21.15.2:0/319402819 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f81e8042030 con 0x7f81f4102e80 2024-05-14T09:47:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.235+0000 7f8202d65740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562aad1065d0 con 0x562aace001b0 2024-05-14T09:47:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.235+0000 7f8202d65740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562aad0bf890 con 0x562aacdd72f0 2024-05-14T09:47:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.235+0000 7f8202d65740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562aad102590 con 0x562aacddf4d0 2024-05-14T09:47:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.235+0000 7f81fa372640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.235+0000 7f81fa372640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60450/0 (socket says 172.21.15.2:60450) 2024-05-14T09:47:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.235+0000 7f81fa372640 1 -- 172.21.15.2:0/222216949 learned_addr learned my addr 172.21.15.2:0/222216949 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.236+0000 7f81f9370640 1 --2- 172.21.15.2:0/222216949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacddf4d0 0x562aad1a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.236+0000 7f81f9b71640 1 --2- 172.21.15.2:0/222216949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacdd72f0 0x562aad1a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.236+0000 7f81f9370640 1 -- 172.21.15.2:0/222216949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacdd72f0 msgr2=0x562aad1a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.236+0000 7f81f9370640 1 --2- 172.21.15.2:0/222216949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacdd72f0 0x562aad1a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.236+0000 7f81f9370640 1 -- 172.21.15.2:0/222216949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 msgr2=0x562aad1a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.236+0000 7f81f9370640 1 --2- 172.21.15.2:0/222216949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.236+0000 7f81f9370640 1 -- 172.21.15.2:0/222216949 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aace18210 con 0x562aacddf4d0 2024-05-14T09:47:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.236+0000 7f81fa372640 1 --2- 172.21.15.2:0/222216949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:47:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.236+0000 7f81e6ffd640 1 --2- 172.21.15.2:0/319402819 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81e00b2920 0x7f81e00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.236+0000 7f81f9370640 1 --2- 172.21.15.2:0/222216949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacddf4d0 0x562aad1a8ee0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f81e8100210 tx=0x7f81e8106dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=341c7ab7a308184b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.236+0000 7f81fa372640 1 --2- 172.21.15.2:0/319402819 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81e00b2920 0x7f81e00b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.236+0000 7f81dffff640 1 -- 172.21.15.2:0/222216949 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8113030 con 0x562aacddf4d0 2024-05-14T09:47:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.237+0000 7f81e6ffd640 1 -- 172.21.15.2:0/319402819 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f81e8102b60 con 0x7f81f4102e80 2024-05-14T09:47:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.237+0000 7f81dffff640 1 -- 172.21.15.2:0/222216949 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e803b460 con 0x562aacddf4d0 2024-05-14T09:47:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.237+0000 7f81dffff640 1 -- 172.21.15.2:0/222216949 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e80c9be0 con 0x562aacddf4d0 2024-05-14T09:47:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.238+0000 7f8202d65740 1 -- 172.21.15.2:0/222216949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacddf4d0 msgr2=0x562aad1a8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.238+0000 7f81fa372640 1 --2- 172.21.15.2:0/319402819 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81e00b2920 0x7f81e00b4e30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x562aad1a2c30 tx=0x7f81ec009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.238+0000 7f8202d65740 1 --2- 172.21.15.2:0/222216949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacddf4d0 0x562aad1a8ee0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f81e8100210 tx=0x7f81e8106dc0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.238+0000 7f8202d65740 1 -- 172.21.15.2:0/222216949 shutdown_connections 2024-05-14T09:47:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.238+0000 7f8202d65740 1 --2- 172.21.15.2:0/222216949 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacddf4d0 0x562aad1a8ee0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.238+0000 7f8202d65740 1 --2- 172.21.15.2:0/222216949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacdd72f0 0x562aad1a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.238+0000 7f8202d65740 1 --2- 172.21.15.2:0/222216949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.238+0000 7f8202d65740 1 -- 172.21.15.2:0/222216949 >> 172.21.15.2:0/222216949 conn(0x562aacdf4300 msgr2=0x562aad19d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.238+0000 7f8202d65740 1 -- 172.21.15.2:0/222216949 shutdown_connections 2024-05-14T09:47:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.238+0000 7f8202d65740 1 -- 172.21.15.2:0/222216949 wait complete. 2024-05-14T09:47:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.238+0000 7f8202d65740 1 -- start start 2024-05-14T09:47:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.238+0000 7f8202d65740 1 Processor -- start 2024-05-14T09:47:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f8202d65740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacdd72f0 0x562aad1ab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f8202d65740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacddf4d0 0x562aad1ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f8202d65740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f8202d65740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562aad102590 con 0x562aace001b0 2024-05-14T09:47:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f8202d65740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562aad0c3370 con 0x562aacddf4d0 2024-05-14T09:47:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f8202d65740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562aad1a96f0 con 0x562aacdd72f0 2024-05-14T09:47:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f8202d65740 10 client.0 _ll_register_callbacks cb 0x562aace17a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:47:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f81fa372640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f81fa372640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60458/0 (socket says 172.21.15.2:60458) 2024-05-14T09:47:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f81fa372640 1 -- 172.21.15.2:0/1124149884 learned_addr learned my addr 172.21.15.2:0/1124149884 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f81f9370640 1 --2- 172.21.15.2:0/1124149884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacddf4d0 0x562aad1ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f81f9b71640 1 --2- 172.21.15.2:0/1124149884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacdd72f0 0x562aad1ab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f81fa372640 1 -- 172.21.15.2:0/1124149884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacdd72f0 msgr2=0x562aad1ab1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f81fa372640 1 --2- 172.21.15.2:0/1124149884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacdd72f0 0x562aad1ab1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f81fa372640 1 -- 172.21.15.2:0/1124149884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacddf4d0 msgr2=0x562aad1ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f81fa372640 1 --2- 172.21.15.2:0/1124149884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacddf4d0 0x562aad1ab6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f81fa372640 1 -- 172.21.15.2:0/1124149884 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aacee4620 con 0x562aace001b0 2024-05-14T09:47:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f81f9370640 1 --2- 172.21.15.2:0/1124149884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacddf4d0 0x562aad1ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:47:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f81fa372640 1 --2- 172.21.15.2:0/1124149884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1b2650 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f81ec011f40 tx=0x7f81ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2377ea0c9d96f7a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.239+0000 7f81dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:47:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.240+0000 7f81dd7fa640 1 -- 172.21.15.2:0/1124149884 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec003b70 con 0x562aace001b0 2024-05-14T09:47:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.240+0000 7f81dd7fa640 1 -- 172.21.15.2:0/1124149884 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f81ec003d10 con 0x562aace001b0 2024-05-14T09:47:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.240+0000 7f8202d65740 1 -- 172.21.15.2:0/1124149884 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562aad1141e0 con 0x562aace001b0 2024-05-14T09:47:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.240+0000 7f81dd7fa640 1 -- 172.21.15.2:0/1124149884 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f81ec032050 con 0x562aace001b0 2024-05-14T09:47:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.240+0000 7f8202d65740 10 client.16734 fetch_fsmap learned FSMap version 46 2024-05-14T09:47:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.240+0000 7f8202d65740 1 -- 172.21.15.2:0/1124149884 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562aad1b5650 con 0x562aace001b0 2024-05-14T09:47:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.240+0000 7f81dd7fa640 1 -- 172.21.15.2:0/1124149884 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562aad1141e0 con 0x562aace001b0 2024-05-14T09:47:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.240+0000 7f81dd7fa640 1 -- 172.21.15.2:0/1124149884 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f81ec00f040 con 0x562aace001b0 2024-05-14T09:47:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.240+0000 7f8202d65740 10 client.16734 fetch_fsmap finished waiting for FSMap version 46 2024-05-14T09:47:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.240+0000 7f8202d65740 10 client.16734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:47:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.240+0000 7f8202d65740 20 client.16734 populate_metadata read hostname 'smithi002' 2024-05-14T09:47:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.241+0000 7f8202d65740 1 --2- 172.21.15.2:0/1124149884 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562aacdbed80 0x562aad1b8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.241+0000 7f8202d65740 4 client.16734 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:47:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.241+0000 7f8202d65740 1 -- 172.21.15.2:0/1124149884 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562aad10e050 con 0x562aacdbed80 2024-05-14T09:47:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.241+0000 7f81f9b71640 1 --2- 172.21.15.2:0/1124149884 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562aacdbed80 0x562aad1b8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.241+0000 7f81f9b71640 1 --2- 172.21.15.2:0/1124149884 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562aacdbed80 0x562aad1b8dc0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18d71e3a42a7078b server_cookie=645cff9719add5d7 in_seq=0 out_seq=0 2024-05-14T09:47:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.241+0000 7f81dd7fa640 10 client.16734 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.242+0000 7f81dd7fa640 1 -- 172.21.15.2:0/1124149884 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562aad10e050 con 0x562aacdbed80 2024-05-14T09:47:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.242+0000 7f81dd7fa640 10 client.16734 handle_command_reply: tid=0 2024-05-14T09:47:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.242+0000 7f8202d65740 1 client.16734 shutdown 2024-05-14T09:47:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.242+0000 7f8202d65740 1 -- 172.21.15.2:0/1124149884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 msgr2=0x562aad1b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.242+0000 7f8202d65740 1 --2- 172.21.15.2:0/1124149884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1b2650 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f81ec011f40 tx=0x7f81ec004a40 comp rx=0 tx=0).stop 2024-05-14T09:47:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.242+0000 7f8202d65740 1 -- 172.21.15.2:0/1124149884 shutdown_connections 2024-05-14T09:47:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.242+0000 7f8202d65740 1 --2- 172.21.15.2:0/1124149884 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562aacdbed80 0x562aad1b8dc0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.242+0000 7f8202d65740 1 --2- 172.21.15.2:0/1124149884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1b2650 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.242+0000 7f8202d65740 1 --2- 172.21.15.2:0/1124149884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacddf4d0 0x562aad1ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.242+0000 7f81dd7fa640 0 client.16734 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.242+0000 7f8202d65740 1 --2- 172.21.15.2:0/1124149884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacdd72f0 0x562aad1ab1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.242+0000 7f8202d65740 1 -- 172.21.15.2:0/1124149884 >> 172.21.15.2:0/1124149884 conn(0x562aacdf4300 msgr2=0x562aad1a0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.242+0000 7f8202d65740 1 -- 172.21.15.2:0/1124149884 shutdown_connections 2024-05-14T09:47:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.242+0000 7f8202d65740 1 -- 172.21.15.2:0/1124149884 wait complete. 2024-05-14T09:47:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.243+0000 7f8202d65740 20 client.16734 trim_cache size 0 max 16384 2024-05-14T09:47:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f8202d65740 1 Processor -- start 2024-05-14T09:47:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f8202d65740 1 -- start start 2024-05-14T09:47:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f8202d65740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacdd72f0 0x562aad1a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f8202d65740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aacddf4d0 0x562aad1b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f8202d65740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aace001b0 0x562aad1b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f8202d65740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f81f41b1800 con 0x562aacddf4d0 2024-05-14T09:47:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f8202d65740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f81f41af040 con 0x562aacdd72f0 2024-05-14T09:47:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f8202d65740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f81f41ace90 con 0x562aace001b0 2024-05-14T09:47:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f81f9370640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aacddf4d0 0x562aad1b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f81f9370640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aacddf4d0 0x562aad1b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60466/0 (socket says 172.21.15.2:60466) 2024-05-14T09:47:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f81f9370640 1 -- 172.21.15.2:0/1062749573 learned_addr learned my addr 172.21.15.2:0/1062749573 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f81f9370640 1 -- 172.21.15.2:0/1062749573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aace001b0 msgr2=0x562aad1b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f81f9370640 1 --2- 172.21.15.2:0/1062749573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aace001b0 0x562aad1b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f81f9370640 1 -- 172.21.15.2:0/1062749573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacdd72f0 msgr2=0x562aad1a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f81f9370640 1 --2- 172.21.15.2:0/1062749573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacdd72f0 0x562aad1a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.258+0000 7f81f9370640 1 -- 172.21.15.2:0/1062749573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aad1a8e20 con 0x562aacddf4d0 2024-05-14T09:47:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.259+0000 7f81f9370640 1 --2- 172.21.15.2:0/1062749573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aacddf4d0 0x562aad1b2570 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f81e8100290 tx=0x7f81e8106c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07d025dbc5beddf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.259+0000 7f81df7fe640 1 -- 172.21.15.2:0/1062749573 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8113020 con 0x562aacddf4d0 2024-05-14T09:47:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.259+0000 7f81df7fe640 1 -- 172.21.15.2:0/1062749573 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e80c9920 con 0x562aacddf4d0 2024-05-14T09:47:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.259+0000 7f81df7fe640 1 -- 172.21.15.2:0/1062749573 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8049040 con 0x562aacddf4d0 2024-05-14T09:47:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.260+0000 7f8202d65740 1 -- 172.21.15.2:0/1062749573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aacddf4d0 msgr2=0x562aad1b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.260+0000 7f8202d65740 1 --2- 172.21.15.2:0/1062749573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aacddf4d0 0x562aad1b2570 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f81e8100290 tx=0x7f81e8106c30 comp rx=0 tx=0).stop 2024-05-14T09:47:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.260+0000 7f8202d65740 1 -- 172.21.15.2:0/1062749573 shutdown_connections 2024-05-14T09:47:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.260+0000 7f8202d65740 1 --2- 172.21.15.2:0/1062749573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aace001b0 0x562aad1b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.260+0000 7f8202d65740 1 --2- 172.21.15.2:0/1062749573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aacddf4d0 0x562aad1b2570 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.260+0000 7f8202d65740 1 --2- 172.21.15.2:0/1062749573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacdd72f0 0x562aad1a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.260+0000 7f8202d65740 1 -- 172.21.15.2:0/1062749573 >> 172.21.15.2:0/1062749573 conn(0x562aacdf4300 msgr2=0x562aad1a02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.260+0000 7f8202d65740 1 -- 172.21.15.2:0/1062749573 shutdown_connections 2024-05-14T09:47:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.260+0000 7f8202d65740 1 -- 172.21.15.2:0/1062749573 wait complete. 2024-05-14T09:47:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.260+0000 7f8202d65740 1 -- start start 2024-05-14T09:47:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.260+0000 7f8202d65740 1 Processor -- start 2024-05-14T09:47:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.260+0000 7f8202d65740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacdd72f0 0x562aad1aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f8202d65740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacddf4d0 0x562aad1b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f8202d65740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f8202d65740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f81f41ad500 con 0x562aace001b0 2024-05-14T09:47:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f8202d65740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f81f41ae970 con 0x562aacddf4d0 2024-05-14T09:47:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f8202d65740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f81f41adbd0 con 0x562aacdd72f0 2024-05-14T09:47:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f8202d65740 10 client.0 _ll_register_callbacks cb 0x7f81ec009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:47:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f81fa372640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f81fa372640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1b13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60468/0 (socket says 172.21.15.2:60468) 2024-05-14T09:47:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f81fa372640 1 -- 172.21.15.2:0/3456227058 learned_addr learned my addr 172.21.15.2:0/3456227058 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f81f9370640 1 --2- 172.21.15.2:0/3456227058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacddf4d0 0x562aad1b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f81f9b71640 1 --2- 172.21.15.2:0/3456227058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacdd72f0 0x562aad1aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f81fa372640 1 -- 172.21.15.2:0/3456227058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacdd72f0 msgr2=0x562aad1aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f81fa372640 1 --2- 172.21.15.2:0/3456227058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacdd72f0 0x562aad1aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f81fa372640 1 -- 172.21.15.2:0/3456227058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacddf4d0 msgr2=0x562aad1b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f81fa372640 1 --2- 172.21.15.2:0/3456227058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacddf4d0 0x562aad1b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f81fa372640 1 -- 172.21.15.2:0/3456227058 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aad1b19a0 con 0x562aace001b0 2024-05-14T09:47:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f81fa372640 1 --2- 172.21.15.2:0/3456227058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1b13e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f81ec020990 tx=0x7f81ec020cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5692d9132b89198a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f81dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:47:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.261+0000 7f81dd7fa640 1 -- 172.21.15.2:0/3456227058 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec03b060 con 0x562aace001b0 2024-05-14T09:47:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.262+0000 7f8202d65740 1 -- 172.21.15.2:0/3456227058 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81e80c9920 con 0x562aace001b0 2024-05-14T09:47:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.262+0000 7f81dd7fa640 1 -- 172.21.15.2:0/3456227058 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f81ec003d80 con 0x562aace001b0 2024-05-14T09:47:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.262+0000 7f81dd7fa640 1 -- 172.21.15.2:0/3456227058 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f81ec011430 con 0x562aace001b0 2024-05-14T09:47:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.262+0000 7f8202d65740 10 client.16740 fetch_fsmap learned FSMap version 46 2024-05-14T09:47:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.262+0000 7f8202d65740 1 -- 172.21.15.2:0/3456227058 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562aad1b2c90 con 0x562aace001b0 2024-05-14T09:47:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.262+0000 7f81dd7fa640 1 -- 172.21.15.2:0/3456227058 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e80c9920 con 0x562aace001b0 2024-05-14T09:47:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.262+0000 7f81dd7fa640 1 -- 172.21.15.2:0/3456227058 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f81ec035070 con 0x562aace001b0 2024-05-14T09:47:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.262+0000 7f8202d65740 10 client.16740 fetch_fsmap finished waiting for FSMap version 46 2024-05-14T09:47:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.262+0000 7f8202d65740 10 client.16740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:47:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.262+0000 7f8202d65740 20 client.16740 populate_metadata read hostname 'smithi002' 2024-05-14T09:47:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.262+0000 7f8202d65740 1 --2- 172.21.15.2:0/3456227058 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562aacd941c0 0x562aad2234e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.262+0000 7f8202d65740 4 client.16740 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:47:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.262+0000 7f8202d65740 1 -- 172.21.15.2:0/3456227058 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562aad0895f0 con 0x562aacd941c0 2024-05-14T09:47:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.262+0000 7f81f9b71640 1 --2- 172.21.15.2:0/3456227058 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562aacd941c0 0x562aad2234e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.263+0000 7f81f9b71640 1 --2- 172.21.15.2:0/3456227058 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562aacd941c0 0x562aad2234e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=659a3fc4cd39acc3 server_cookie=a1200f48fb1936eb in_seq=0 out_seq=0 2024-05-14T09:47:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.263+0000 7f81dd7fa640 10 client.16740 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:10.287 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1009680 8.12 MB/sec execute 516 sec latency 1031.153 ms 2024-05-14T09:47:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.306+0000 7f81dd7fa640 1 -- 172.21.15.2:0/3456227058 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562aad0895f0 con 0x562aacd941c0 2024-05-14T09:47:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.306+0000 7f81dd7fa640 10 client.16740 handle_command_reply: tid=0 2024-05-14T09:47:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.306+0000 7f8202d65740 1 client.16740 shutdown 2024-05-14T09:47:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.306+0000 7f8202d65740 1 -- 172.21.15.2:0/3456227058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 msgr2=0x562aad1b13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.306+0000 7f8202d65740 1 --2- 172.21.15.2:0/3456227058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1b13e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f81ec020990 tx=0x7f81ec020cc0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.306+0000 7f8202d65740 1 -- 172.21.15.2:0/3456227058 shutdown_connections 2024-05-14T09:47:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.306+0000 7f8202d65740 1 --2- 172.21.15.2:0/3456227058 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562aacd941c0 0x562aad2234e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.306+0000 7f8202d65740 1 --2- 172.21.15.2:0/3456227058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aace001b0 0x562aad1b13e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.306+0000 7f8202d65740 1 --2- 172.21.15.2:0/3456227058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aacddf4d0 0x562aad1b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.306+0000 7f81dd7fa640 0 client.16740 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.306+0000 7f8202d65740 1 --2- 172.21.15.2:0/3456227058 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562aacdd72f0 0x562aad1aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.306+0000 7f8202d65740 1 -- 172.21.15.2:0/3456227058 >> 172.21.15.2:0/3456227058 conn(0x562aacdf4300 msgr2=0x562aad19b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.306+0000 7f8202d65740 1 -- 172.21.15.2:0/3456227058 shutdown_connections 2024-05-14T09:47:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.306+0000 7f8202d65740 1 -- 172.21.15.2:0/3456227058 wait complete. 2024-05-14T09:47:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.307+0000 7f8202d65740 20 client.16740 trim_cache size 0 max 16384 2024-05-14T09:47:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:47:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:47:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:47:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:47:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.307+0000 7f81fbdd5640 1 -- 172.21.15.2:0/319402819 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81e00b2920 msgr2=0x7f81e00b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.307+0000 7f81fbdd5640 1 --2- 172.21.15.2:0/319402819 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81e00b2920 0x7f81e00b4e30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x562aad1a2c30 tx=0x7f81ec009600 comp rx=0 tx=0).stop 2024-05-14T09:47:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.307+0000 7f81fbdd5640 1 -- 172.21.15.2:0/319402819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4102e80 msgr2=0x7f81f419ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.307+0000 7f81fbdd5640 1 --2- 172.21.15.2:0/319402819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4102e80 0x7f81f419ca60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009230 tx=0x7f81e8031a50 comp rx=0 tx=0).stop 2024-05-14T09:47:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.307+0000 7f81fbdd5640 1 -- 172.21.15.2:0/319402819 shutdown_connections 2024-05-14T09:47:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.307+0000 7f81fbdd5640 1 --2- 172.21.15.2:0/319402819 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81e00b2920 0x7f81e00b4e30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.307+0000 7f81fbdd5640 1 --2- 172.21.15.2:0/319402819 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f81f41037d0 0x7f81f41a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.307+0000 7f81fbdd5640 1 --2- 172.21.15.2:0/319402819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81f4102e80 0x7f81f419ca60 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.307+0000 7f81fbdd5640 1 --2- 172.21.15.2:0/319402819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81f4102530 0x7f81f419c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.307+0000 7f81fbdd5640 1 -- 172.21.15.2:0/319402819 >> 172.21.15.2:0/319402819 conn(0x7f81f40fdd40 msgr2=0x7f81f410a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.307+0000 7f81fbdd5640 1 -- 172.21.15.2:0/319402819 shutdown_connections 2024-05-14T09:47:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:10.307+0000 7f81fbdd5640 1 -- 172.21.15.2:0/319402819 wait complete. 2024-05-14T09:47:10.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:10 smithi002 ceph-mon[31557]: pgmap v791: 129 pgs: 129 active+clean; 169 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 890 KiB/s wr, 100 op/s 2024-05-14T09:47:10.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:10 smithi029 ceph-mon[36249]: pgmap v791: 129 pgs: 129 active+clean; 169 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 890 KiB/s wr, 100 op/s 2024-05-14T09:47:10.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:10 smithi026 ceph-mon[36056]: pgmap v791: 129 pgs: 129 active+clean; 169 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 890 KiB/s wr, 100 op/s 2024-05-14T09:47:11.287 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1009680 8.11 MB/sec execute 517 sec latency 2031.241 ms 2024-05-14T09:47:12.287 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1009680 8.09 MB/sec execute 518 sec latency 3031.315 ms 2024-05-14T09:47:12.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:12 smithi002 ceph-mon[31557]: pgmap v792: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 981 KiB/s wr, 79 op/s 2024-05-14T09:47:12.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:12 smithi029 ceph-mon[36249]: pgmap v792: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 981 KiB/s wr, 79 op/s 2024-05-14T09:47:12.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:12 smithi026 ceph-mon[36056]: pgmap v792: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 981 KiB/s wr, 79 op/s 2024-05-14T09:47:13.287 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1009680 8.08 MB/sec execute 519 sec latency 4031.414 ms 2024-05-14T09:47:14.287 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1009748 8.06 MB/sec execute 520 sec latency 4948.593 ms 2024-05-14T09:47:14.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:14 smithi002 ceph-mon[31557]: pgmap v793: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.0 MiB/s wr, 96 op/s 2024-05-14T09:47:14.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:14 smithi029 ceph-mon[36249]: pgmap v793: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.0 MiB/s wr, 96 op/s 2024-05-14T09:47:14.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:14 smithi026 ceph-mon[36056]: pgmap v793: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.0 MiB/s wr, 96 op/s 2024-05-14T09:47:15.287 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1009748 8.04 MB/sec execute 521 sec latency 1013.439 ms 2024-05-14T09:47:15.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:15 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:47:15.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:15 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:47:15.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:15 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:47:16.287 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1009748 8.03 MB/sec execute 522 sec latency 2013.475 ms 2024-05-14T09:47:16.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:16 smithi002 ceph-mon[31557]: pgmap v794: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1017 KiB/s wr, 88 op/s 2024-05-14T09:47:16.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:16 smithi029 ceph-mon[36249]: pgmap v794: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1017 KiB/s wr, 88 op/s 2024-05-14T09:47:16.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:16 smithi026 ceph-mon[36056]: pgmap v794: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1017 KiB/s wr, 88 op/s 2024-05-14T09:47:17.287 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1009748 8.01 MB/sec execute 523 sec latency 3013.537 ms 2024-05-14T09:47:18.288 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1009748 8.00 MB/sec execute 524 sec latency 4013.626 ms 2024-05-14T09:47:18.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:18 smithi029 ceph-mon[36249]: pgmap v795: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.0 MiB/s wr, 89 op/s 2024-05-14T09:47:18.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:18 smithi026 ceph-mon[36056]: pgmap v795: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.0 MiB/s wr, 89 op/s 2024-05-14T09:47:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:18 smithi002 ceph-mon[31557]: pgmap v795: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.0 MiB/s wr, 89 op/s 2024-05-14T09:47:19.288 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1009860 7.98 MB/sec execute 525 sec latency 4929.189 ms 2024-05-14T09:47:20.288 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1011967 7.99 MB/sec execute 526 sec latency 14.008 ms 2024-05-14T09:47:20.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:20 smithi029 ceph-mon[36249]: pgmap v796: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 653 KiB/s wr, 81 op/s 2024-05-14T09:47:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:20 smithi026 ceph-mon[36056]: pgmap v796: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 653 KiB/s wr, 81 op/s 2024-05-14T09:47:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:20 smithi002 ceph-mon[31557]: pgmap v796: 129 pgs: 129 active+clean; 168 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 653 KiB/s wr, 81 op/s 2024-05-14T09:47:21.288 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1013171 7.99 MB/sec execute 527 sec latency 154.744 ms 2024-05-14T09:47:22.288 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1013171 7.97 MB/sec execute 528 sec latency 1154.839 ms 2024-05-14T09:47:22.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:22 smithi029 ceph-mon[36249]: pgmap v797: 129 pgs: 129 active+clean; 169 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 281 KiB/s wr, 41 op/s 2024-05-14T09:47:22.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:22 smithi026 ceph-mon[36056]: pgmap v797: 129 pgs: 129 active+clean; 169 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 281 KiB/s wr, 41 op/s 2024-05-14T09:47:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:22 smithi002 ceph-mon[31557]: pgmap v797: 129 pgs: 129 active+clean; 169 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 281 KiB/s wr, 41 op/s 2024-05-14T09:47:23.288 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1013171 7.96 MB/sec execute 529 sec latency 2154.937 ms 2024-05-14T09:47:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.420+0000 7fd93fbce640 1 -- 172.21.15.2:0/1524540904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9381037d0 msgr2=0x7fd938103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.420+0000 7fd93fbce640 1 --2- 172.21.15.2:0/1524540904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9381037d0 0x7fd938103be0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd930009cd0 tx=0x7fd93002a8e0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.420+0000 7fd93fbce640 1 -- 172.21.15.2:0/1524540904 shutdown_connections 2024-05-14T09:47:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.420+0000 7fd93fbce640 1 --2- 172.21.15.2:0/1524540904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9381037d0 0x7fd938103be0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.420+0000 7fd93fbce640 1 --2- 172.21.15.2:0/1524540904 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd938102e80 0x7fd938103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.420+0000 7fd93fbce640 1 --2- 172.21.15.2:0/1524540904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd938102530 0x7fd938102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.420+0000 7fd93fbce640 1 -- 172.21.15.2:0/1524540904 >> 172.21.15.2:0/1524540904 conn(0x7fd9380fdd60 msgr2=0x7fd9381001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.420+0000 7fd93fbce640 1 -- 172.21.15.2:0/1524540904 shutdown_connections 2024-05-14T09:47:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.420+0000 7fd93fbce640 1 -- 172.21.15.2:0/1524540904 wait complete. 2024-05-14T09:47:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.421+0000 7fd93fbce640 1 Processor -- start 2024-05-14T09:47:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.421+0000 7fd93fbce640 1 -- start start 2024-05-14T09:47:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.421+0000 7fd93fbce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd938102530 0x7fd93819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.421+0000 7fd93fbce640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd938102e80 0x7fd93819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.421+0000 7fd93fbce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9381037d0 0x7fd9381a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.421+0000 7fd93fbce640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd938111960 con 0x7fd938102e80 2024-05-14T09:47:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.421+0000 7fd93fbce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9381117e0 con 0x7fd938102530 2024-05-14T09:47:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.421+0000 7fd93fbce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd938111ae0 con 0x7fd9381037d0 2024-05-14T09:47:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.421+0000 7fd93d169640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd938102e80 0x7fd93819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.421+0000 7fd93d169640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd938102e80 0x7fd93819c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52870/0 (socket says 172.21.15.2:52870) 2024-05-14T09:47:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.421+0000 7fd93d169640 1 -- 172.21.15.2:0/3601738037 learned_addr learned my addr 172.21.15.2:0/3601738037 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.421+0000 7fd93d96a640 1 --2- 172.21.15.2:0/3601738037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd938102530 0x7fd93819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.421+0000 7fd93e16b640 1 --2- 172.21.15.2:0/3601738037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9381037d0 0x7fd9381a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.422+0000 7fd93d169640 1 -- 172.21.15.2:0/3601738037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9381037d0 msgr2=0x7fd9381a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.422+0000 7fd93d169640 1 --2- 172.21.15.2:0/3601738037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9381037d0 0x7fd9381a3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.422+0000 7fd93d169640 1 -- 172.21.15.2:0/3601738037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd938102530 msgr2=0x7fd93819c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.422+0000 7fd93d169640 1 --2- 172.21.15.2:0/3601738037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd938102530 0x7fd93819c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.422+0000 7fd93d169640 1 -- 172.21.15.2:0/3601738037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9381a3fe0 con 0x7fd938102e80 2024-05-14T09:47:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.422+0000 7fd93d96a640 1 --2- 172.21.15.2:0/3601738037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd938102530 0x7fd93819c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:47:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.422+0000 7fd93d169640 1 --2- 172.21.15.2:0/3601738037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd938102e80 0x7fd93819c9e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd92800e9f0 tx=0x7fd92800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.422+0000 7fd92effd640 1 -- 172.21.15.2:0/3601738037 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92800cd80 con 0x7fd938102e80 2024-05-14T09:47:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.422+0000 7fd92effd640 1 -- 172.21.15.2:0/3601738037 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9280108e0 con 0x7fd938102e80 2024-05-14T09:47:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.422+0000 7fd93fbce640 1 -- 172.21.15.2:0/3601738037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9381a42d0 con 0x7fd938102e80 2024-05-14T09:47:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.422+0000 7fd92effd640 1 -- 172.21.15.2:0/3601738037 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928018980 con 0x7fd938102e80 2024-05-14T09:47:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.422+0000 7fd93fbce640 1 -- 172.21.15.2:0/3601738037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9381a4870 con 0x7fd938102e80 2024-05-14T09:47:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.425+0000 7fd946a82740 1 Processor -- start 2024-05-14T09:47:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.425+0000 7fd946a82740 1 -- start start 2024-05-14T09:47:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.425+0000 7fd946a82740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c577f1b0 0x55e0c5b20a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.425+0000 7fd946a82740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c57562f0 0x55e0c5b20fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.425+0000 7fd946a82740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b27ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.425+0000 7fd946a82740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e0c5a855d0 con 0x55e0c57562f0 2024-05-14T09:47:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.425+0000 7fd946a82740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e0c5a3e890 con 0x55e0c575e4d0 2024-05-14T09:47:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.425+0000 7fd946a82740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e0c5a81590 con 0x55e0c577f1b0 2024-05-14T09:47:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.425+0000 7fd92effd640 1 -- 172.21.15.2:0/3601738037 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd928010430 con 0x7fd938102e80 2024-05-14T09:47:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.425+0000 7fd93d96a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c57562f0 0x55e0c5b20fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.426+0000 7fd93d96a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c57562f0 0x55e0c5b20fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52884/0 (socket says 172.21.15.2:52884) 2024-05-14T09:47:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.426+0000 7fd93d96a640 1 -- 172.21.15.2:0/2233194267 learned_addr learned my addr 172.21.15.2:0/2233194267 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.426+0000 7fd93e16b640 1 --2- 172.21.15.2:0/2233194267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c577f1b0 0x55e0c5b20a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.426+0000 7fd93d169640 1 --2- 172.21.15.2:0/2233194267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b27ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.426+0000 7fd93d96a640 1 -- 172.21.15.2:0/2233194267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c577f1b0 msgr2=0x55e0c5b20a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.426+0000 7fd93d96a640 1 --2- 172.21.15.2:0/2233194267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c577f1b0 0x55e0c5b20a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.426+0000 7fd93d96a640 1 -- 172.21.15.2:0/2233194267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 msgr2=0x55e0c5b27ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.426+0000 7fd93d96a640 1 --2- 172.21.15.2:0/2233194267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b27ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.426+0000 7fd93d96a640 1 -- 172.21.15.2:0/2233194267 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c5797210 con 0x55e0c57562f0 2024-05-14T09:47:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.426+0000 7fd93d96a640 1 --2- 172.21.15.2:0/2233194267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c57562f0 0x55e0c5b20fe0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd93400bd30 tx=0x7fd93400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec63aca75655e80 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.426+0000 7fd923fff640 1 -- 172.21.15.2:0/2233194267 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934015070 con 0x55e0c57562f0 2024-05-14T09:47:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.426+0000 7fd923fff640 1 -- 172.21.15.2:0/2233194267 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd93400be20 con 0x55e0c57562f0 2024-05-14T09:47:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.426+0000 7fd923fff640 1 -- 172.21.15.2:0/2233194267 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934010750 con 0x55e0c57562f0 2024-05-14T09:47:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.427+0000 7fd92effd640 1 --2- 172.21.15.2:0/3601738037 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9240b2920 0x7fd9240b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.427+0000 7fd93e16b640 1 --2- 172.21.15.2:0/3601738037 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9240b2920 0x7fd9240b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.427+0000 7fd92effd640 1 -- 172.21.15.2:0/3601738037 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fd9280e2a30 con 0x7fd938102e80 2024-05-14T09:47:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.427+0000 7fd946a82740 1 -- 172.21.15.2:0/2233194267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c57562f0 msgr2=0x55e0c5b20fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.427+0000 7fd93e16b640 1 --2- 172.21.15.2:0/3601738037 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9240b2920 0x7fd9240b4e30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd930005cc0 tx=0x7fd930031850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.427+0000 7fd946a82740 1 --2- 172.21.15.2:0/2233194267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c57562f0 0x55e0c5b20fe0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd93400bd30 tx=0x7fd93400f730 comp rx=0 tx=0).stop 2024-05-14T09:47:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.428+0000 7fd946a82740 1 -- 172.21.15.2:0/2233194267 shutdown_connections 2024-05-14T09:47:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.428+0000 7fd946a82740 1 --2- 172.21.15.2:0/2233194267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b27ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.428+0000 7fd946a82740 1 --2- 172.21.15.2:0/2233194267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c57562f0 0x55e0c5b20fe0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.428+0000 7fd946a82740 1 --2- 172.21.15.2:0/2233194267 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c577f1b0 0x55e0c5b20a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.428+0000 7fd946a82740 1 -- 172.21.15.2:0/2233194267 >> 172.21.15.2:0/2233194267 conn(0x55e0c5773300 msgr2=0x55e0c5b1c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.428+0000 7fd946a82740 1 -- 172.21.15.2:0/2233194267 shutdown_connections 2024-05-14T09:47:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.428+0000 7fd946a82740 1 -- 172.21.15.2:0/2233194267 wait complete. 2024-05-14T09:47:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.428+0000 7fd946a82740 1 -- start start 2024-05-14T09:47:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.428+0000 7fd946a82740 1 Processor -- start 2024-05-14T09:47:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd946a82740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c57562f0 0x55e0c5b2a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd946a82740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b2abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd946a82740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c577f1b0 0x55e0c5b31b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd946a82740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e0c5b28930 con 0x55e0c577f1b0 2024-05-14T09:47:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd946a82740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e0c5a42370 con 0x55e0c575e4d0 2024-05-14T09:47:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd946a82740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e0c5b28ab0 con 0x55e0c57562f0 2024-05-14T09:47:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd946a82740 10 client.0 _ll_register_callbacks cb 0x55e0c5796a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:47:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd93e16b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c577f1b0 0x55e0c5b31b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd93d96a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c57562f0 0x55e0c5b2a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd93d96a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c57562f0 0x55e0c5b2a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:39656/0 (socket says 172.21.15.2:39656) 2024-05-14T09:47:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd93e16b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c577f1b0 0x55e0c5b31b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52888/0 (socket says 172.21.15.2:52888) 2024-05-14T09:47:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd93d96a640 1 -- 172.21.15.2:0/2280974558 learned_addr learned my addr 172.21.15.2:0/2280974558 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd93d169640 1 --2- 172.21.15.2:0/2280974558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b2abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd93d96a640 1 -- 172.21.15.2:0/2280974558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 msgr2=0x55e0c5b2abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd93d96a640 1 --2- 172.21.15.2:0/2280974558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b2abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd93d96a640 1 -- 172.21.15.2:0/2280974558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c577f1b0 msgr2=0x55e0c5b31b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd93d96a640 1 --2- 172.21.15.2:0/2280974558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c577f1b0 0x55e0c5b31b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.429+0000 7fd93d96a640 1 -- 172.21.15.2:0/2280974558 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c5863620 con 0x55e0c57562f0 2024-05-14T09:47:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.430+0000 7fd93d96a640 1 --2- 172.21.15.2:0/2280974558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c57562f0 0x55e0c5b2a670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd9340057a0 tx=0x7fd934004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f60d832f8a0d08c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.430+0000 7fd9217fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:47:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.433+0000 7fd9217fa640 1 -- 172.21.15.2:0/2280974558 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934015050 con 0x55e0c57562f0 2024-05-14T09:47:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.433+0000 7fd9217fa640 1 -- 172.21.15.2:0/2280974558 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fd934005030 con 0x55e0c57562f0 2024-05-14T09:47:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.433+0000 7fd946a82740 1 -- 172.21.15.2:0/2280974558 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd93400be20 con 0x55e0c57562f0 2024-05-14T09:47:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.433+0000 7fd9217fa640 1 -- 172.21.15.2:0/2280974558 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd934005230 con 0x55e0c57562f0 2024-05-14T09:47:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.433+0000 7fd946a82740 10 client.26395 fetch_fsmap learned FSMap version 47 2024-05-14T09:47:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.433+0000 7fd946a82740 1 -- 172.21.15.2:0/2280974558 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e0c5b34a40 con 0x55e0c57562f0 2024-05-14T09:47:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.433+0000 7fd9217fa640 1 -- 172.21.15.2:0/2280974558 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd93400be20 con 0x55e0c57562f0 2024-05-14T09:47:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.433+0000 7fd9217fa640 1 -- 172.21.15.2:0/2280974558 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7fd934003680 con 0x55e0c57562f0 2024-05-14T09:47:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.434+0000 7fd946a82740 10 client.26395 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T09:47:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.434+0000 7fd946a82740 10 client.26395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:47:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.434+0000 7fd946a82740 20 client.26395 populate_metadata read hostname 'smithi002' 2024-05-14T09:47:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.434+0000 7fd946a82740 1 --2- 172.21.15.2:0/2280974558 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e0c573dd80 0x55e0c5b380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.434+0000 7fd946a82740 4 client.26395 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:47:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.434+0000 7fd946a82740 1 -- 172.21.15.2:0/2280974558 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0c5a931e0 con 0x55e0c573dd80 2024-05-14T09:47:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.434+0000 7fd93d169640 1 --2- 172.21.15.2:0/2280974558 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e0c573dd80 0x55e0c5b380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.434+0000 7fd93d169640 1 --2- 172.21.15.2:0/2280974558 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e0c573dd80 0x55e0c5b380e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=983c2939ca7f0077 server_cookie=b7e31eb76061915b in_seq=0 out_seq=0 2024-05-14T09:47:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.434+0000 7fd9217fa640 10 client.26395 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.435+0000 7fd9217fa640 1 -- 172.21.15.2:0/2280974558 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e0c5a931e0 con 0x55e0c573dd80 2024-05-14T09:47:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.435+0000 7fd9217fa640 10 client.26395 handle_command_reply: tid=0 2024-05-14T09:47:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.435+0000 7fd946a82740 1 client.26395 shutdown 2024-05-14T09:47:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.435+0000 7fd946a82740 1 -- 172.21.15.2:0/2280974558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c57562f0 msgr2=0x55e0c5b2a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.435+0000 7fd946a82740 1 --2- 172.21.15.2:0/2280974558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c57562f0 0x55e0c5b2a670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd9340057a0 tx=0x7fd934004270 comp rx=0 tx=0).stop 2024-05-14T09:47:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.435+0000 7fd946a82740 1 -- 172.21.15.2:0/2280974558 shutdown_connections 2024-05-14T09:47:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.435+0000 7fd946a82740 1 --2- 172.21.15.2:0/2280974558 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e0c573dd80 0x55e0c5b380e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.435+0000 7fd946a82740 1 --2- 172.21.15.2:0/2280974558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c577f1b0 0x55e0c5b31b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.435+0000 7fd946a82740 1 --2- 172.21.15.2:0/2280974558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b2abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.435+0000 7fd946a82740 1 --2- 172.21.15.2:0/2280974558 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c57562f0 0x55e0c5b2a670 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.435+0000 7fd9217fa640 0 client.26395 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.435+0000 7fd946a82740 1 -- 172.21.15.2:0/2280974558 >> 172.21.15.2:0/2280974558 conn(0x55e0c5773300 msgr2=0x55e0c5b1f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.435+0000 7fd946a82740 1 -- 172.21.15.2:0/2280974558 shutdown_connections 2024-05-14T09:47:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.435+0000 7fd946a82740 1 -- 172.21.15.2:0/2280974558 wait complete. 2024-05-14T09:47:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.436+0000 7fd946a82740 20 client.26395 trim_cache size 0 max 16384 2024-05-14T09:47:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.451+0000 7fd946a82740 1 Processor -- start 2024-05-14T09:47:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.451+0000 7fd946a82740 1 -- start start 2024-05-14T09:47:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.451+0000 7fd946a82740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c57562f0 0x55e0c5b2a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.451+0000 7fd946a82740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b2a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.451+0000 7fd946a82740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c577f1b0 0x55e0c5b31a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.451+0000 7fd946a82740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd9381b17c0 con 0x55e0c57562f0 2024-05-14T09:47:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.451+0000 7fd946a82740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9381af030 con 0x55e0c575e4d0 2024-05-14T09:47:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.451+0000 7fd946a82740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9381ace80 con 0x55e0c577f1b0 2024-05-14T09:47:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.451+0000 7fd93d96a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c57562f0 0x55e0c5b2a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.451+0000 7fd93d96a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c57562f0 0x55e0c5b2a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52898/0 (socket says 172.21.15.2:52898) 2024-05-14T09:47:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.451+0000 7fd93d96a640 1 -- 172.21.15.2:0/3327413255 learned_addr learned my addr 172.21.15.2:0/3327413255 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.452+0000 7fd93e16b640 1 --2- 172.21.15.2:0/3327413255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c577f1b0 0x55e0c5b31a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.452+0000 7fd93d169640 1 --2- 172.21.15.2:0/3327413255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b2a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.452+0000 7fd93d96a640 1 -- 172.21.15.2:0/3327413255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c577f1b0 msgr2=0x55e0c5b31a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.452+0000 7fd93d96a640 1 --2- 172.21.15.2:0/3327413255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c577f1b0 0x55e0c5b31a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.452+0000 7fd93d96a640 1 -- 172.21.15.2:0/3327413255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 msgr2=0x55e0c5b2a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.452+0000 7fd93d96a640 1 --2- 172.21.15.2:0/3327413255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b2a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.452+0000 7fd93d96a640 1 -- 172.21.15.2:0/3327413255 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c5b164d0 con 0x55e0c57562f0 2024-05-14T09:47:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.452+0000 7fd93d96a640 1 --2- 172.21.15.2:0/3327413255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c57562f0 0x55e0c5b2a240 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd934011460 tx=0x7fd93400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5167c54b21f5c2c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.452+0000 7fd9237fe640 1 -- 172.21.15.2:0/3327413255 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934015070 con 0x55e0c57562f0 2024-05-14T09:47:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.452+0000 7fd9237fe640 1 -- 172.21.15.2:0/3327413255 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd93400be20 con 0x55e0c57562f0 2024-05-14T09:47:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.452+0000 7fd9237fe640 1 -- 172.21.15.2:0/3327413255 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934003850 con 0x55e0c57562f0 2024-05-14T09:47:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.453+0000 7fd946a82740 1 -- 172.21.15.2:0/3327413255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c57562f0 msgr2=0x55e0c5b2a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.453+0000 7fd946a82740 1 --2- 172.21.15.2:0/3327413255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c57562f0 0x55e0c5b2a240 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd934011460 tx=0x7fd93400b8b0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.453+0000 7fd946a82740 1 -- 172.21.15.2:0/3327413255 shutdown_connections 2024-05-14T09:47:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.453+0000 7fd946a82740 1 --2- 172.21.15.2:0/3327413255 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c577f1b0 0x55e0c5b31a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.453+0000 7fd946a82740 1 --2- 172.21.15.2:0/3327413255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b2a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.453+0000 7fd946a82740 1 --2- 172.21.15.2:0/3327413255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c57562f0 0x55e0c5b2a240 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.453+0000 7fd946a82740 1 -- 172.21.15.2:0/3327413255 >> 172.21.15.2:0/3327413255 conn(0x55e0c5773300 msgr2=0x55e0c5b24a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.453+0000 7fd946a82740 1 -- 172.21.15.2:0/3327413255 shutdown_connections 2024-05-14T09:47:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.453+0000 7fd946a82740 1 -- 172.21.15.2:0/3327413255 wait complete. 2024-05-14T09:47:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.453+0000 7fd946a82740 1 -- start start 2024-05-14T09:47:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.453+0000 7fd946a82740 1 Processor -- start 2024-05-14T09:47:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd946a82740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c57562f0 0x55e0c5b2f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd946a82740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b2f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd946a82740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c577f1b0 0x55e0c5b9e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd946a82740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd9381ad4f0 con 0x55e0c577f1b0 2024-05-14T09:47:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd946a82740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd9381ae960 con 0x55e0c575e4d0 2024-05-14T09:47:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd946a82740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd9381adbc0 con 0x55e0c57562f0 2024-05-14T09:47:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd946a82740 10 client.0 _ll_register_callbacks cb 0x7fd928005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:47:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd93e16b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c577f1b0 0x55e0c5b9e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd93e16b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c577f1b0 0x55e0c5b9e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52912/0 (socket says 172.21.15.2:52912) 2024-05-14T09:47:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd93e16b640 1 -- 172.21.15.2:0/3431416814 learned_addr learned my addr 172.21.15.2:0/3431416814 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd93d96a640 1 --2- 172.21.15.2:0/3431416814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c57562f0 0x55e0c5b2f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd93d169640 1 --2- 172.21.15.2:0/3431416814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b2f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd93e16b640 1 -- 172.21.15.2:0/3431416814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c57562f0 msgr2=0x55e0c5b2f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd93e16b640 1 --2- 172.21.15.2:0/3431416814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c57562f0 0x55e0c5b2f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd93e16b640 1 -- 172.21.15.2:0/3431416814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 msgr2=0x55e0c5b2f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd93e16b640 1 --2- 172.21.15.2:0/3431416814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b2f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd93e16b640 1 -- 172.21.15.2:0/3431416814 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c5b2fc40 con 0x55e0c577f1b0 2024-05-14T09:47:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.454+0000 7fd93d169640 1 --2- 172.21.15.2:0/3431416814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b2f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:47:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd93e16b640 1 --2- 172.21.15.2:0/3431416814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c577f1b0 0x55e0c5b9e480 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd938075fe0 tx=0x7fd9300095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d17790d8bbc9e8c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd9217fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:47:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd9217fa640 1 -- 172.21.15.2:0/3431416814 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930057060 con 0x55e0c577f1b0 2024-05-14T09:47:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd9217fa640 1 -- 172.21.15.2:0/3431416814 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fd930004250 con 0x55e0c577f1b0 2024-05-14T09:47:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd946a82740 1 -- 172.21.15.2:0/3431416814 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd93400be20 con 0x55e0c577f1b0 2024-05-14T09:47:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd9217fa640 1 -- 172.21.15.2:0/3431416814 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd93005e050 con 0x55e0c577f1b0 2024-05-14T09:47:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd946a82740 10 client.16758 fetch_fsmap learned FSMap version 47 2024-05-14T09:47:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd946a82740 1 -- 172.21.15.2:0/3431416814 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e0c5b9fa90 con 0x55e0c577f1b0 2024-05-14T09:47:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd9217fa640 1 -- 172.21.15.2:0/3431416814 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd93400be20 con 0x55e0c577f1b0 2024-05-14T09:47:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd9217fa640 1 -- 172.21.15.2:0/3431416814 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7fd930030070 con 0x55e0c577f1b0 2024-05-14T09:47:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd946a82740 10 client.16758 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T09:47:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd946a82740 10 client.16758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:47:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd946a82740 20 client.16758 populate_metadata read hostname 'smithi002' 2024-05-14T09:47:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd946a82740 1 --2- 172.21.15.2:0/3431416814 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e0c57131c0 0x55e0c5ba2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd946a82740 4 client.16758 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:47:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.455+0000 7fd946a82740 1 -- 172.21.15.2:0/3431416814 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0c5a8d050 con 0x55e0c57131c0 2024-05-14T09:47:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.456+0000 7fd93d96a640 1 --2- 172.21.15.2:0/3431416814 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e0c57131c0 0x55e0c5ba2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.456+0000 7fd9217fa640 10 client.16758 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.456+0000 7fd93d96a640 1 --2- 172.21.15.2:0/3431416814 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e0c57131c0 0x55e0c5ba2510 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4b6c54b83ce1501 server_cookie=4b25bd0188a1ba9d in_seq=0 out_seq=0 2024-05-14T09:47:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.507+0000 7fd9217fa640 1 -- 172.21.15.2:0/3431416814 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e0c5a8d050 con 0x55e0c57131c0 2024-05-14T09:47:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.507+0000 7fd9217fa640 10 client.16758 handle_command_reply: tid=0 2024-05-14T09:47:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.507+0000 7fd946a82740 1 client.16758 shutdown 2024-05-14T09:47:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.507+0000 7fd946a82740 1 -- 172.21.15.2:0/3431416814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c577f1b0 msgr2=0x55e0c5b9e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.507+0000 7fd946a82740 1 --2- 172.21.15.2:0/3431416814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c577f1b0 0x55e0c5b9e480 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd938075fe0 tx=0x7fd9300095c0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.507+0000 7fd946a82740 1 -- 172.21.15.2:0/3431416814 shutdown_connections 2024-05-14T09:47:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.507+0000 7fd946a82740 1 --2- 172.21.15.2:0/3431416814 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e0c57131c0 0x55e0c5ba2510 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.507+0000 7fd946a82740 1 --2- 172.21.15.2:0/3431416814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0c577f1b0 0x55e0c5b9e480 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.507+0000 7fd946a82740 1 --2- 172.21.15.2:0/3431416814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e0c575e4d0 0x55e0c5b2f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.507+0000 7fd946a82740 1 --2- 172.21.15.2:0/3431416814 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e0c57562f0 0x55e0c5b2f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.507+0000 7fd9217fa640 0 client.16758 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.507+0000 7fd946a82740 1 -- 172.21.15.2:0/3431416814 >> 172.21.15.2:0/3431416814 conn(0x55e0c5773300 msgr2=0x55e0c5b2a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.507+0000 7fd946a82740 1 -- 172.21.15.2:0/3431416814 shutdown_connections 2024-05-14T09:47:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.507+0000 7fd946a82740 1 -- 172.21.15.2:0/3431416814 wait complete. 2024-05-14T09:47:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.508+0000 7fd946a82740 20 client.16758 trim_cache size 0 max 16384 2024-05-14T09:47:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:47:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:47:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:47:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:47:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.508+0000 7fd93fbce640 1 -- 172.21.15.2:0/3601738037 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9240b2920 msgr2=0x7fd9240b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.508+0000 7fd93fbce640 1 --2- 172.21.15.2:0/3601738037 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9240b2920 0x7fd9240b4e30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd930005cc0 tx=0x7fd930031850 comp rx=0 tx=0).stop 2024-05-14T09:47:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.508+0000 7fd93fbce640 1 -- 172.21.15.2:0/3601738037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd938102e80 msgr2=0x7fd93819c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.508+0000 7fd93fbce640 1 --2- 172.21.15.2:0/3601738037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd938102e80 0x7fd93819c9e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd92800e9f0 tx=0x7fd92800eeb0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.508+0000 7fd93fbce640 1 -- 172.21.15.2:0/3601738037 shutdown_connections 2024-05-14T09:47:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.508+0000 7fd93fbce640 1 --2- 172.21.15.2:0/3601738037 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd9240b2920 0x7fd9240b4e30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.508+0000 7fd93fbce640 1 --2- 172.21.15.2:0/3601738037 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd9381037d0 0x7fd9381a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.508+0000 7fd93fbce640 1 --2- 172.21.15.2:0/3601738037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd938102e80 0x7fd93819c9e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.508+0000 7fd93fbce640 1 --2- 172.21.15.2:0/3601738037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd938102530 0x7fd93819c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.508+0000 7fd93fbce640 1 -- 172.21.15.2:0/3601738037 >> 172.21.15.2:0/3601738037 conn(0x7fd9380fdd60 msgr2=0x7fd93810a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.508+0000 7fd93fbce640 1 -- 172.21.15.2:0/3601738037 shutdown_connections 2024-05-14T09:47:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:23.508+0000 7fd93fbce640 1 -- 172.21.15.2:0/3601738037 wait complete. 2024-05-14T09:47:24.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1013227 7.94 MB/sec execute 530 sec latency 3069.362 ms 2024-05-14T09:47:24.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:24 smithi029 ceph-mon[36249]: pgmap v798: 129 pgs: 129 active+clean; 170 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 230 KiB/s wr, 64 op/s 2024-05-14T09:47:24.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:24 smithi026 ceph-mon[36056]: pgmap v798: 129 pgs: 129 active+clean; 170 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 230 KiB/s wr, 64 op/s 2024-05-14T09:47:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:24 smithi002 ceph-mon[31557]: pgmap v798: 129 pgs: 129 active+clean; 170 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 230 KiB/s wr, 64 op/s 2024-05-14T09:47:25.288 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1013227 7.93 MB/sec execute 531 sec latency 1028.566 ms 2024-05-14T09:47:26.288 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1013227 7.91 MB/sec execute 532 sec latency 2028.642 ms 2024-05-14T09:47:26.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:26 smithi029 ceph-mon[36249]: pgmap v799: 129 pgs: 129 active+clean; 170 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 171 KiB/s wr, 42 op/s 2024-05-14T09:47:26.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:26 smithi026 ceph-mon[36056]: pgmap v799: 129 pgs: 129 active+clean; 170 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 171 KiB/s wr, 42 op/s 2024-05-14T09:47:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:26 smithi002 ceph-mon[31557]: pgmap v799: 129 pgs: 129 active+clean; 170 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 171 KiB/s wr, 42 op/s 2024-05-14T09:47:27.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1013295 7.90 MB/sec execute 533 sec latency 2899.752 ms 2024-05-14T09:47:28.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1013295 7.88 MB/sec execute 534 sec latency 1064.889 ms 2024-05-14T09:47:28.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:28 smithi026 ceph-mon[36056]: pgmap v800: 129 pgs: 129 active+clean; 171 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 334 KiB/s wr, 54 op/s 2024-05-14T09:47:28.724 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:28 smithi002 ceph-mon[31557]: pgmap v800: 129 pgs: 129 active+clean; 171 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 334 KiB/s wr, 54 op/s 2024-05-14T09:47:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:28 smithi029 ceph-mon[36249]: pgmap v800: 129 pgs: 129 active+clean; 171 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 334 KiB/s wr, 54 op/s 2024-05-14T09:47:29.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1013407 7.87 MB/sec execute 535 sec latency 1979.831 ms 2024-05-14T09:47:30.288 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1015503 7.88 MB/sec execute 536 sec latency 16.823 ms 2024-05-14T09:47:30.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:30 smithi026 ceph-mon[36056]: pgmap v801: 129 pgs: 129 active+clean; 173 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 632 KiB/s wr, 71 op/s 2024-05-14T09:47:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:30 smithi002 ceph-mon[31557]: pgmap v801: 129 pgs: 129 active+clean; 173 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 632 KiB/s wr, 71 op/s 2024-05-14T09:47:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:30 smithi029 ceph-mon[36249]: pgmap v801: 129 pgs: 129 active+clean; 173 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 632 KiB/s wr, 71 op/s 2024-05-14T09:47:31.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1016804 7.88 MB/sec execute 537 sec latency 16.824 ms 2024-05-14T09:47:32.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1016840 7.86 MB/sec execute 538 sec latency 974.620 ms 2024-05-14T09:47:32.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:32 smithi026 ceph-mon[36056]: pgmap v802: 129 pgs: 129 active+clean; 173 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 682 KiB/s wr, 73 op/s 2024-05-14T09:47:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:32 smithi002 ceph-mon[31557]: pgmap v802: 129 pgs: 129 active+clean; 173 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 682 KiB/s wr, 73 op/s 2024-05-14T09:47:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:32 smithi029 ceph-mon[36249]: pgmap v802: 129 pgs: 129 active+clean; 173 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 682 KiB/s wr, 73 op/s 2024-05-14T09:47:33.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1016840 7.85 MB/sec execute 539 sec latency 1974.733 ms 2024-05-14T09:47:34.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1016952 7.83 MB/sec execute 540 sec latency 2888.208 ms 2024-05-14T09:47:34.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:34 smithi026 ceph-mon[36056]: pgmap v803: 129 pgs: 129 active+clean; 171 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 921 KiB/s wr, 119 op/s 2024-05-14T09:47:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:34 smithi002 ceph-mon[31557]: pgmap v803: 129 pgs: 129 active+clean; 171 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 921 KiB/s wr, 119 op/s 2024-05-14T09:47:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:34 smithi029 ceph-mon[36249]: pgmap v803: 129 pgs: 129 active+clean; 171 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 921 KiB/s wr, 119 op/s 2024-05-14T09:47:35.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1019041 7.84 MB/sec execute 541 sec latency 14.259 ms 2024-05-14T09:47:36.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1020337 7.84 MB/sec execute 542 sec latency 23.526 ms 2024-05-14T09:47:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.612+0000 7f8175442640 1 -- 172.21.15.2:0/9569534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8170102b30 msgr2=0x7f8170103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.612+0000 7f8175442640 1 --2- 172.21.15.2:0/9569534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8170102b30 0x7f8170103830 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8160009900 tx=0x7f816002a410 comp rx=0 tx=0).stop 2024-05-14T09:47:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.612+0000 7f8175442640 1 -- 172.21.15.2:0/9569534 shutdown_connections 2024-05-14T09:47:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.612+0000 7f8175442640 1 --2- 172.21.15.2:0/9569534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8170108a70 0x7f817010ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.612+0000 7f8175442640 1 --2- 172.21.15.2:0/9569534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8170103d70 0x7f8170104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.612+0000 7f8175442640 1 --2- 172.21.15.2:0/9569534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8170102b30 0x7f8170103830 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.612+0000 7f8175442640 1 -- 172.21.15.2:0/9569534 >> 172.21.15.2:0/9569534 conn(0x7f81700fdd60 msgr2=0x7f81701001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.612+0000 7f8175442640 1 -- 172.21.15.2:0/9569534 shutdown_connections 2024-05-14T09:47:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.612+0000 7f8175442640 1 -- 172.21.15.2:0/9569534 wait complete. 2024-05-14T09:47:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.613+0000 7f8175442640 1 Processor -- start 2024-05-14T09:47:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.613+0000 7f8175442640 1 -- start start 2024-05-14T09:47:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.613+0000 7f8175442640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8170102b30 0x7f817019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.613+0000 7f8175442640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8170103d70 0x7f817019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.613+0000 7f8175442640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8170108a70 0x7f81701a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.613+0000 7f8175442640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f817010e630 con 0x7f8170108a70 2024-05-14T09:47:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.613+0000 7f8175442640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f817010e4b0 con 0x7f8170102b30 2024-05-14T09:47:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.613+0000 7f8175442640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f817010e7b0 con 0x7f8170103d70 2024-05-14T09:47:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.613+0000 7f816e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8170103d70 0x7f817019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.613+0000 7f816f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8170108a70 0x7f81701a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.613+0000 7f816e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8170103d70 0x7f817019ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:57442/0 (socket says 172.21.15.2:57442) 2024-05-14T09:47:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.614+0000 7f816e7fc640 1 -- 172.21.15.2:0/3001316036 learned_addr learned my addr 172.21.15.2:0/3001316036 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.614+0000 7f816effd640 1 --2- 172.21.15.2:0/3001316036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8170102b30 0x7f817019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.614+0000 7f816e7fc640 1 -- 172.21.15.2:0/3001316036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8170102b30 msgr2=0x7f817019c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.614+0000 7f816e7fc640 1 --2- 172.21.15.2:0/3001316036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8170102b30 0x7f817019c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.614+0000 7f816e7fc640 1 -- 172.21.15.2:0/3001316036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8170108a70 msgr2=0x7f81701a3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.614+0000 7f816e7fc640 1 --2- 172.21.15.2:0/3001316036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8170108a70 0x7f81701a3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.614+0000 7f816e7fc640 1 -- 172.21.15.2:0/3001316036 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81701a4000 con 0x7f8170103d70 2024-05-14T09:47:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.614+0000 7f816effd640 1 --2- 172.21.15.2:0/3001316036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8170102b30 0x7f817019c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:47:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.614+0000 7f816e7fc640 1 --2- 172.21.15.2:0/3001316036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8170103d70 0x7f817019ca00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f816400e9f0 tx=0x7f816400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.614+0000 7f8157fff640 1 -- 172.21.15.2:0/3001316036 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816400cd80 con 0x7f8170103d70 2024-05-14T09:47:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.614+0000 7f8175442640 1 -- 172.21.15.2:0/3001316036 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81701a42f0 con 0x7f8170103d70 2024-05-14T09:47:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.614+0000 7f8157fff640 1 -- 172.21.15.2:0/3001316036 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81640108e0 con 0x7f8170103d70 2024-05-14T09:47:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.614+0000 7f8157fff640 1 -- 172.21.15.2:0/3001316036 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164018980 con 0x7f8170103d70 2024-05-14T09:47:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.614+0000 7f8175442640 1 -- 172.21.15.2:0/3001316036 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81701a4890 con 0x7f8170103d70 2024-05-14T09:47:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.617+0000 7f817c3cf740 1 Processor -- start 2024-05-14T09:47:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.617+0000 7f817c3cf740 1 -- start start 2024-05-14T09:47:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.617+0000 7f8157fff640 1 -- 172.21.15.2:0/3001316036 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8164010430 con 0x7f8170103d70 2024-05-14T09:47:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.617+0000 7f817c3cf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948cc1b0 0x561f94c6da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.617+0000 7f817c3cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948a32f0 0x561f94c6dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.617+0000 7f817c3cf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948ab4d0 0x561f94c74ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.617+0000 7f817c3cf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561f94bd25d0 con 0x561f948cc1b0 2024-05-14T09:47:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.617+0000 7f817c3cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561f94b8b890 con 0x561f948a32f0 2024-05-14T09:47:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.617+0000 7f817c3cf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561f94bce590 con 0x561f948ab4d0 2024-05-14T09:47:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.617+0000 7f816f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948cc1b0 0x561f94c6da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.618+0000 7f816effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948a32f0 0x561f94c6dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.618+0000 7f816e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948ab4d0 0x561f94c74ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.618+0000 7f816f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948cc1b0 0x561f94c6da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44714/0 (socket says 172.21.15.2:44714) 2024-05-14T09:47:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.618+0000 7f816effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948a32f0 0x561f94c6dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:45230/0 (socket says 172.21.15.2:45230) 2024-05-14T09:47:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.618+0000 7f816f7fe640 1 -- 172.21.15.2:0/4206078013 learned_addr learned my addr 172.21.15.2:0/4206078013 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.618+0000 7f816f7fe640 1 -- 172.21.15.2:0/4206078013 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948ab4d0 msgr2=0x561f94c74ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.618+0000 7f816f7fe640 1 --2- 172.21.15.2:0/4206078013 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948ab4d0 0x561f94c74ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.618+0000 7f816f7fe640 1 -- 172.21.15.2:0/4206078013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948a32f0 msgr2=0x561f94c6dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.618+0000 7f816f7fe640 1 --2- 172.21.15.2:0/4206078013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948a32f0 0x561f94c6dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.618+0000 7f816f7fe640 1 -- 172.21.15.2:0/4206078013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f948e4210 con 0x561f948cc1b0 2024-05-14T09:47:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.618+0000 7f816e7fc640 1 --2- 172.21.15.2:0/4206078013 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948ab4d0 0x561f94c74ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:47:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.618+0000 7f816f7fe640 1 --2- 172.21.15.2:0/4206078013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948cc1b0 0x561f94c6da60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8168010370 tx=0x7f8168010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f6d097ce198401 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.618+0000 7f8157fff640 1 --2- 172.21.15.2:0/3001316036 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81580b28a0 0x7f81580b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.618+0000 7f81557fa640 1 -- 172.21.15.2:0/4206078013 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168016070 con 0x561f948cc1b0 2024-05-14T09:47:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.618+0000 7f816effd640 1 --2- 172.21.15.2:0/3001316036 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81580b28a0 0x7f81580b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.619+0000 7f8157fff640 1 -- 172.21.15.2:0/3001316036 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f81640e2950 con 0x7f8170103d70 2024-05-14T09:47:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.619+0000 7f81557fa640 1 -- 172.21.15.2:0/4206078013 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8168004d20 con 0x561f948cc1b0 2024-05-14T09:47:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.619+0000 7f81557fa640 1 -- 172.21.15.2:0/4206078013 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168005350 con 0x561f948cc1b0 2024-05-14T09:47:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.619+0000 7f817c3cf740 1 -- 172.21.15.2:0/4206078013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948cc1b0 msgr2=0x561f94c6da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.619+0000 7f816effd640 1 --2- 172.21.15.2:0/3001316036 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81580b28a0 0x7f81580b4db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8160005ab0 tx=0x7f8160000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.619+0000 7f817c3cf740 1 --2- 172.21.15.2:0/4206078013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948cc1b0 0x561f94c6da60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8168010370 tx=0x7f8168010830 comp rx=0 tx=0).stop 2024-05-14T09:47:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 1 -- 172.21.15.2:0/4206078013 shutdown_connections 2024-05-14T09:47:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 1 --2- 172.21.15.2:0/4206078013 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948ab4d0 0x561f94c74ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 1 --2- 172.21.15.2:0/4206078013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948a32f0 0x561f94c6dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 1 --2- 172.21.15.2:0/4206078013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948cc1b0 0x561f94c6da60 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 1 -- 172.21.15.2:0/4206078013 >> 172.21.15.2:0/4206078013 conn(0x561f948c0300 msgr2=0x561f94c693a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 1 -- 172.21.15.2:0/4206078013 shutdown_connections 2024-05-14T09:47:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 1 -- 172.21.15.2:0/4206078013 wait complete. 2024-05-14T09:47:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 1 -- start start 2024-05-14T09:47:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 1 Processor -- start 2024-05-14T09:47:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c77730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948ab4d0 0x561f94c77c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948cc1b0 0x561f94c7ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561f94c75930 con 0x561f948a32f0 2024-05-14T09:47:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561f94b8f370 con 0x561f948ab4d0 2024-05-14T09:47:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561f94c75ab0 con 0x561f948cc1b0 2024-05-14T09:47:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f817c3cf740 10 client.0 _ll_register_callbacks cb 0x561f948e3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:47:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.620+0000 7f816effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c77730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f816e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948ab4d0 0x561f94c77c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f816effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c77730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44720/0 (socket says 172.21.15.2:44720) 2024-05-14T09:47:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f816effd640 1 -- 172.21.15.2:0/1838740625 learned_addr learned my addr 172.21.15.2:0/1838740625 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f816f7fe640 1 --2- 172.21.15.2:0/1838740625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948cc1b0 0x561f94c7ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f816e7fc640 1 -- 172.21.15.2:0/1838740625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948cc1b0 msgr2=0x561f94c7ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f816e7fc640 1 --2- 172.21.15.2:0/1838740625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948cc1b0 0x561f94c7ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f816e7fc640 1 -- 172.21.15.2:0/1838740625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 msgr2=0x561f94c77730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f816e7fc640 1 --2- 172.21.15.2:0/1838740625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c77730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f816e7fc640 1 -- 172.21.15.2:0/1838740625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f949b0620 con 0x561f948ab4d0 2024-05-14T09:47:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f816effd640 1 --2- 172.21.15.2:0/1838740625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c77730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:47:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f816e7fc640 1 --2- 172.21.15.2:0/1838740625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948ab4d0 0x561f94c77c70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f81640e0080 tx=0x7f8164009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5f2ccf47713b7b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f8152ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:47:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f8152ff5640 1 -- 172.21.15.2:0/1838740625 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164091070 con 0x561f948ab4d0 2024-05-14T09:47:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f817c3cf740 1 -- 172.21.15.2:0/1838740625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8168004d20 con 0x561f948ab4d0 2024-05-14T09:47:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f8152ff5640 1 -- 172.21.15.2:0/1838740625 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f81640cae00 con 0x561f948ab4d0 2024-05-14T09:47:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f8152ff5640 1 -- 172.21.15.2:0/1838740625 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f81640a9550 con 0x561f948ab4d0 2024-05-14T09:47:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f817c3cf740 10 client.26417 fetch_fsmap learned FSMap version 47 2024-05-14T09:47:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.621+0000 7f817c3cf740 1 -- 172.21.15.2:0/1838740625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561f94c81b70 con 0x561f948ab4d0 2024-05-14T09:47:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.622+0000 7f8152ff5640 1 -- 172.21.15.2:0/1838740625 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8168004d20 con 0x561f948ab4d0 2024-05-14T09:47:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.622+0000 7f8152ff5640 1 -- 172.21.15.2:0/1838740625 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7f81640a5570 con 0x561f948ab4d0 2024-05-14T09:47:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.622+0000 7f817c3cf740 10 client.26417 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T09:47:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.622+0000 7f817c3cf740 10 client.26417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:47:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.622+0000 7f817c3cf740 20 client.26417 populate_metadata read hostname 'smithi002' 2024-05-14T09:47:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.622+0000 7f817c3cf740 1 --2- 172.21.15.2:0/1838740625 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561f9488ad80 0x561f94c85210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.622+0000 7f817c3cf740 4 client.26417 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:47:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.622+0000 7f817c3cf740 1 -- 172.21.15.2:0/1838740625 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f94be01e0 con 0x561f9488ad80 2024-05-14T09:47:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.622+0000 7f816effd640 1 --2- 172.21.15.2:0/1838740625 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561f9488ad80 0x561f94c85210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.622+0000 7f8152ff5640 10 client.26417 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.622+0000 7f816effd640 1 --2- 172.21.15.2:0/1838740625 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561f9488ad80 0x561f94c85210 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d20919b4decc7541 server_cookie=7444352582eb4cd8 in_seq=0 out_seq=0 2024-05-14T09:47:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.624+0000 7f8152ff5640 1 -- 172.21.15.2:0/1838740625 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561f94be01e0 con 0x561f9488ad80 2024-05-14T09:47:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.624+0000 7f8152ff5640 10 client.26417 handle_command_reply: tid=0 2024-05-14T09:47:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.624+0000 7f817c3cf740 1 client.26417 shutdown 2024-05-14T09:47:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.624+0000 7f817c3cf740 1 -- 172.21.15.2:0/1838740625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948ab4d0 msgr2=0x561f94c77c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.624+0000 7f817c3cf740 1 --2- 172.21.15.2:0/1838740625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948ab4d0 0x561f94c77c70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f81640e0080 tx=0x7f8164009310 comp rx=0 tx=0).stop 2024-05-14T09:47:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.624+0000 7f817c3cf740 1 -- 172.21.15.2:0/1838740625 shutdown_connections 2024-05-14T09:47:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.624+0000 7f817c3cf740 1 --2- 172.21.15.2:0/1838740625 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561f9488ad80 0x561f94c85210 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.624+0000 7f817c3cf740 1 --2- 172.21.15.2:0/1838740625 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948cc1b0 0x561f94c7ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.624+0000 7f817c3cf740 1 --2- 172.21.15.2:0/1838740625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948ab4d0 0x561f94c77c70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.624+0000 7f817c3cf740 1 --2- 172.21.15.2:0/1838740625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c77730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.624+0000 7f8152ff5640 0 client.26417 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.624+0000 7f817c3cf740 1 -- 172.21.15.2:0/1838740625 >> 172.21.15.2:0/1838740625 conn(0x561f948c0300 msgr2=0x561f94c683a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.624+0000 7f817c3cf740 1 -- 172.21.15.2:0/1838740625 shutdown_connections 2024-05-14T09:47:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.624+0000 7f817c3cf740 1 -- 172.21.15.2:0/1838740625 wait complete. 2024-05-14T09:47:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.624+0000 7f817c3cf740 20 client.26417 trim_cache size 0 max 16384 2024-05-14T09:47:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.640+0000 7f817c3cf740 1 Processor -- start 2024-05-14T09:47:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.640+0000 7f817c3cf740 1 -- start start 2024-05-14T09:47:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.640+0000 7f817c3cf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c70c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.640+0000 7f817c3cf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948ab4d0 0x561f94c711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.640+0000 7f817c3cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948cc1b0 0x561f94c68cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.640+0000 7f817c3cf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f81701b1800 con 0x561f948a32f0 2024-05-14T09:47:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.640+0000 7f817c3cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f81701af070 con 0x561f948cc1b0 2024-05-14T09:47:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.640+0000 7f817c3cf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f81701acec0 con 0x561f948ab4d0 2024-05-14T09:47:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.640+0000 7f816e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948ab4d0 0x561f94c711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.640+0000 7f816effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c70c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.640+0000 7f816f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948cc1b0 0x561f94c68cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.640+0000 7f816e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948ab4d0 0x561f94c711b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:57460/0 (socket says 172.21.15.2:57460) 2024-05-14T09:47:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.640+0000 7f816e7fc640 1 -- 172.21.15.2:0/160123318 learned_addr learned my addr 172.21.15.2:0/160123318 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.641+0000 7f816f7fe640 1 -- 172.21.15.2:0/160123318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948ab4d0 msgr2=0x561f94c711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.641+0000 7f816f7fe640 1 --2- 172.21.15.2:0/160123318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948ab4d0 0x561f94c711b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.641+0000 7f816f7fe640 1 -- 172.21.15.2:0/160123318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 msgr2=0x561f94c70c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.641+0000 7f816f7fe640 1 --2- 172.21.15.2:0/160123318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c70c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.641+0000 7f816f7fe640 1 -- 172.21.15.2:0/160123318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f94c69d00 con 0x561f948cc1b0 2024-05-14T09:47:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.641+0000 7f816e7fc640 1 --2- 172.21.15.2:0/160123318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948ab4d0 0x561f94c711b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:47:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.641+0000 7f816effd640 1 --2- 172.21.15.2:0/160123318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c70c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:47:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.641+0000 7f816f7fe640 1 --2- 172.21.15.2:0/160123318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948cc1b0 0x561f94c68cf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8168006010 tx=0x7f8168013f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=447e0b564c3d95b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.641+0000 7f8154ff9640 1 -- 172.21.15.2:0/160123318 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168016040 con 0x561f948cc1b0 2024-05-14T09:47:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.641+0000 7f8154ff9640 1 -- 172.21.15.2:0/160123318 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8168003c70 con 0x561f948cc1b0 2024-05-14T09:47:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.641+0000 7f8154ff9640 1 -- 172.21.15.2:0/160123318 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816801a6d0 con 0x561f948cc1b0 2024-05-14T09:47:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.642+0000 7f817c3cf740 1 -- 172.21.15.2:0/160123318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948cc1b0 msgr2=0x561f94c68cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.642+0000 7f817c3cf740 1 --2- 172.21.15.2:0/160123318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948cc1b0 0x561f94c68cf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8168006010 tx=0x7f8168013f50 comp rx=0 tx=0).stop 2024-05-14T09:47:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.642+0000 7f817c3cf740 1 -- 172.21.15.2:0/160123318 shutdown_connections 2024-05-14T09:47:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.642+0000 7f817c3cf740 1 --2- 172.21.15.2:0/160123318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948cc1b0 0x561f94c68cf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.642+0000 7f817c3cf740 1 --2- 172.21.15.2:0/160123318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948ab4d0 0x561f94c711b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.642+0000 7f817c3cf740 1 --2- 172.21.15.2:0/160123318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c70c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.642+0000 7f817c3cf740 1 -- 172.21.15.2:0/160123318 >> 172.21.15.2:0/160123318 conn(0x561f948c0300 msgr2=0x561f94c71bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.642+0000 7f817c3cf740 1 -- 172.21.15.2:0/160123318 shutdown_connections 2024-05-14T09:47:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.642+0000 7f817c3cf740 1 -- 172.21.15.2:0/160123318 wait complete. 2024-05-14T09:47:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f817c3cf740 1 -- start start 2024-05-14T09:47:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f817c3cf740 1 Processor -- start 2024-05-14T09:47:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f817c3cf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c7b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f817c3cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948ab4d0 0x561f94c7b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f817c3cf740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948cc1b0 0x561f94c7c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f817c3cf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f81701ad530 con 0x561f948a32f0 2024-05-14T09:47:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f817c3cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f81701ae9a0 con 0x561f948ab4d0 2024-05-14T09:47:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f817c3cf740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f81701adc00 con 0x561f948cc1b0 2024-05-14T09:47:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f817c3cf740 10 client.0 _ll_register_callbacks cb 0x7f81640e8240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:47:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f816effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c7b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f816effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c7b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44732/0 (socket says 172.21.15.2:44732) 2024-05-14T09:47:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f816effd640 1 -- 172.21.15.2:0/681587572 learned_addr learned my addr 172.21.15.2:0/681587572 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f816e7fc640 1 --2- 172.21.15.2:0/681587572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948ab4d0 0x561f94c7b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f816f7fe640 1 --2- 172.21.15.2:0/681587572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948cc1b0 0x561f94c7c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f816effd640 1 -- 172.21.15.2:0/681587572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948cc1b0 msgr2=0x561f94c7c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f816effd640 1 --2- 172.21.15.2:0/681587572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948cc1b0 0x561f94c7c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f816effd640 1 -- 172.21.15.2:0/681587572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948ab4d0 msgr2=0x561f94c7b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f816effd640 1 --2- 172.21.15.2:0/681587572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948ab4d0 0x561f94c7b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f816effd640 1 -- 172.21.15.2:0/681587572 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f94c7cea0 con 0x561f948a32f0 2024-05-14T09:47:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.643+0000 7f816f7fe640 1 --2- 172.21.15.2:0/681587572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948cc1b0 0x561f94c7c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:47:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f816effd640 1 --2- 172.21.15.2:0/681587572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c7b470 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8160009230 tx=0x7f8160009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5609cb23e253fe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f8152ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:47:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f8152ff5640 1 -- 172.21.15.2:0/681587572 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160035040 con 0x561f948a32f0 2024-05-14T09:47:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f8152ff5640 1 -- 172.21.15.2:0/681587572 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f81600545f0 con 0x561f948a32f0 2024-05-14T09:47:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f817c3cf740 1 -- 172.21.15.2:0/681587572 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8168003c70 con 0x561f948a32f0 2024-05-14T09:47:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f8152ff5640 1 -- 172.21.15.2:0/681587572 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8160065050 con 0x561f948a32f0 2024-05-14T09:47:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f817c3cf740 10 client.16776 fetch_fsmap learned FSMap version 47 2024-05-14T09:47:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f817c3cf740 1 -- 172.21.15.2:0/681587572 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561f94c7e200 con 0x561f948a32f0 2024-05-14T09:47:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f8152ff5640 1 -- 172.21.15.2:0/681587572 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8168003c70 con 0x561f948a32f0 2024-05-14T09:47:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f8152ff5640 1 -- 172.21.15.2:0/681587572 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7f8160044070 con 0x561f948a32f0 2024-05-14T09:47:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f817c3cf740 10 client.16776 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T09:47:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f817c3cf740 10 client.16776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:47:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f817c3cf740 20 client.16776 populate_metadata read hostname 'smithi002' 2024-05-14T09:47:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f817c3cf740 1 --2- 172.21.15.2:0/681587572 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561f948601c0 0x561f94cefab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f817c3cf740 4 client.16776 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:47:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.644+0000 7f817c3cf740 1 -- 172.21.15.2:0/681587572 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f94bda050 con 0x561f948601c0 2024-05-14T09:47:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.645+0000 7f816e7fc640 1 --2- 172.21.15.2:0/681587572 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561f948601c0 0x561f94cefab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.645+0000 7f8152ff5640 10 client.16776 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.645+0000 7f816e7fc640 1 --2- 172.21.15.2:0/681587572 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561f948601c0 0x561f94cefab0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a7929888749348a server_cookie=f8f9ed43dcac332e in_seq=0 out_seq=0 2024-05-14T09:47:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:36 smithi026 ceph-mon[36056]: pgmap v804: 129 pgs: 129 active+clean; 171 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 784 KiB/s wr, 86 op/s 2024-05-14T09:47:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.704+0000 7f8152ff5640 1 -- 172.21.15.2:0/681587572 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f94bda050 con 0x561f948601c0 2024-05-14T09:47:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.704+0000 7f8152ff5640 10 client.16776 handle_command_reply: tid=0 2024-05-14T09:47:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.704+0000 7f817c3cf740 1 client.16776 shutdown 2024-05-14T09:47:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.704+0000 7f817c3cf740 1 -- 172.21.15.2:0/681587572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 msgr2=0x561f94c7b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.704+0000 7f817c3cf740 1 --2- 172.21.15.2:0/681587572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c7b470 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8160009230 tx=0x7f8160009260 comp rx=0 tx=0).stop 2024-05-14T09:47:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.704+0000 7f817c3cf740 1 -- 172.21.15.2:0/681587572 shutdown_connections 2024-05-14T09:47:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.704+0000 7f817c3cf740 1 --2- 172.21.15.2:0/681587572 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561f948601c0 0x561f94cefab0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.704+0000 7f817c3cf740 1 --2- 172.21.15.2:0/681587572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561f948cc1b0 0x561f94c7c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.704+0000 7f8152ff5640 0 client.16776 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.704+0000 7f817c3cf740 1 --2- 172.21.15.2:0/681587572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561f948ab4d0 0x561f94c7b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.704+0000 7f817c3cf740 1 --2- 172.21.15.2:0/681587572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f948a32f0 0x561f94c7b470 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.704+0000 7f817c3cf740 1 -- 172.21.15.2:0/681587572 >> 172.21.15.2:0/681587572 conn(0x561f948c0300 msgr2=0x561f94c71bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.704+0000 7f817c3cf740 1 -- 172.21.15.2:0/681587572 shutdown_connections 2024-05-14T09:47:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.704+0000 7f817c3cf740 1 -- 172.21.15.2:0/681587572 wait complete. 2024-05-14T09:47:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.704+0000 7f817c3cf740 20 client.16776 trim_cache size 0 max 16384 2024-05-14T09:47:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:47:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:47:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:47:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:47:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.705+0000 7f8175442640 1 -- 172.21.15.2:0/3001316036 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81580b28a0 msgr2=0x7f81580b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.705+0000 7f8175442640 1 --2- 172.21.15.2:0/3001316036 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81580b28a0 0x7f81580b4db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8160005ab0 tx=0x7f8160000bc0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.705+0000 7f8175442640 1 -- 172.21.15.2:0/3001316036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8170103d70 msgr2=0x7f817019ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.705+0000 7f8175442640 1 --2- 172.21.15.2:0/3001316036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8170103d70 0x7f817019ca00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f816400e9f0 tx=0x7f816400eeb0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.705+0000 7f8175442640 1 -- 172.21.15.2:0/3001316036 shutdown_connections 2024-05-14T09:47:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.705+0000 7f8175442640 1 --2- 172.21.15.2:0/3001316036 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81580b28a0 0x7f81580b4db0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.705+0000 7f8175442640 1 --2- 172.21.15.2:0/3001316036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8170108a70 0x7f81701a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.705+0000 7f8175442640 1 --2- 172.21.15.2:0/3001316036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8170103d70 0x7f817019ca00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.705+0000 7f8175442640 1 --2- 172.21.15.2:0/3001316036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8170102b30 0x7f817019c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.705+0000 7f8175442640 1 -- 172.21.15.2:0/3001316036 >> 172.21.15.2:0/3001316036 conn(0x7f81700fdd60 msgr2=0x7f8170100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.705+0000 7f8175442640 1 -- 172.21.15.2:0/3001316036 shutdown_connections 2024-05-14T09:47:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:36.705+0000 7f8175442640 1 -- 172.21.15.2:0/3001316036 wait complete. 2024-05-14T09:47:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:36 smithi002 ceph-mon[31557]: pgmap v804: 129 pgs: 129 active+clean; 171 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 784 KiB/s wr, 86 op/s 2024-05-14T09:47:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:36 smithi029 ceph-mon[36249]: pgmap v804: 129 pgs: 129 active+clean; 171 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 784 KiB/s wr, 86 op/s 2024-05-14T09:47:37.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1022189 7.84 MB/sec execute 543 sec latency 55.675 ms 2024-05-14T09:47:38.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1023681 7.84 MB/sec execute 544 sec latency 13.819 ms 2024-05-14T09:47:38.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:38 smithi026 ceph-mon[36056]: pgmap v805: 129 pgs: 129 active+clean; 172 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 877 KiB/s wr, 98 op/s 2024-05-14T09:47:38.738 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:38 smithi029 ceph-mon[36249]: pgmap v805: 129 pgs: 129 active+clean; 172 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 877 KiB/s wr, 98 op/s 2024-05-14T09:47:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:38 smithi002 ceph-mon[31557]: pgmap v805: 129 pgs: 129 active+clean; 172 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 877 KiB/s wr, 98 op/s 2024-05-14T09:47:39.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1023878 7.83 MB/sec execute 545 sec latency 815.842 ms 2024-05-14T09:47:40.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1025908 7.84 MB/sec execute 546 sec latency 19.019 ms 2024-05-14T09:47:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:40 smithi002 ceph-mon[31557]: pgmap v806: 129 pgs: 129 active+clean; 174 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 911 KiB/s wr, 146 op/s 2024-05-14T09:47:40.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:40 smithi029 ceph-mon[36249]: pgmap v806: 129 pgs: 129 active+clean; 174 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 911 KiB/s wr, 146 op/s 2024-05-14T09:47:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:40 smithi026 ceph-mon[36056]: pgmap v806: 129 pgs: 129 active+clean; 174 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 911 KiB/s wr, 146 op/s 2024-05-14T09:47:41.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1027245 7.84 MB/sec execute 547 sec latency 17.325 ms 2024-05-14T09:47:42.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1027355 7.82 MB/sec execute 548 sec latency 910.600 ms 2024-05-14T09:47:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:42 smithi002 ceph-mon[31557]: pgmap v807: 129 pgs: 129 active+clean; 174 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 628 KiB/s wr, 136 op/s 2024-05-14T09:47:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:42 smithi029 ceph-mon[36249]: pgmap v807: 129 pgs: 129 active+clean; 174 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 628 KiB/s wr, 136 op/s 2024-05-14T09:47:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:42 smithi026 ceph-mon[36056]: pgmap v807: 129 pgs: 129 active+clean; 174 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 628 KiB/s wr, 136 op/s 2024-05-14T09:47:43.290 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1027355 7.81 MB/sec execute 549 sec latency 1910.733 ms 2024-05-14T09:47:44.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1027426 7.79 MB/sec execute 550 sec latency 2826.086 ms 2024-05-14T09:47:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:44 smithi002 ceph-mon[31557]: pgmap v808: 129 pgs: 129 active+clean; 176 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 799 KiB/s wr, 178 op/s 2024-05-14T09:47:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:44 smithi029 ceph-mon[36249]: pgmap v808: 129 pgs: 129 active+clean; 176 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 799 KiB/s wr, 178 op/s 2024-05-14T09:47:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:44 smithi026 ceph-mon[36056]: pgmap v808: 129 pgs: 129 active+clean; 176 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 799 KiB/s wr, 178 op/s 2024-05-14T09:47:45.289 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1027479 7.78 MB/sec execute 551 sec latency 952.241 ms 2024-05-14T09:47:46.290 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1027479 7.77 MB/sec execute 552 sec latency 1952.318 ms 2024-05-14T09:47:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:46 smithi002 ceph-mon[31557]: pgmap v809: 129 pgs: 129 active+clean; 176 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 541 KiB/s wr, 129 op/s 2024-05-14T09:47:46.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:46 smithi029 ceph-mon[36249]: pgmap v809: 129 pgs: 129 active+clean; 176 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 541 KiB/s wr, 129 op/s 2024-05-14T09:47:46.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:46 smithi026 ceph-mon[36056]: pgmap v809: 129 pgs: 129 active+clean; 176 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 541 KiB/s wr, 129 op/s 2024-05-14T09:47:47.290 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1027479 7.75 MB/sec execute 553 sec latency 2952.427 ms 2024-05-14T09:47:48.290 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1027479 7.74 MB/sec execute 554 sec latency 3952.534 ms 2024-05-14T09:47:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:48 smithi002 ceph-mon[31557]: pgmap v810: 129 pgs: 129 active+clean; 176 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 542 KiB/s wr, 131 op/s 2024-05-14T09:47:48.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:48 smithi029 ceph-mon[36249]: pgmap v810: 129 pgs: 129 active+clean; 176 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 542 KiB/s wr, 131 op/s 2024-05-14T09:47:48.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:48 smithi026 ceph-mon[36056]: pgmap v810: 129 pgs: 129 active+clean; 176 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 542 KiB/s wr, 131 op/s 2024-05-14T09:47:49.290 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1027591 7.72 MB/sec execute 555 sec latency 4868.087 ms 2024-05-14T09:47:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.811+0000 7f6dc9d9d640 1 -- 172.21.15.2:0/988836572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc4106e50 msgr2=0x7f6dc41092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.811+0000 7f6dc9d9d640 1 --2- 172.21.15.2:0/988836572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc4106e50 0x7f6dc41092f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009900 tx=0x7f6dac02a410 comp rx=0 tx=0).stop 2024-05-14T09:47:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.811+0000 7f6dc9d9d640 1 -- 172.21.15.2:0/988836572 shutdown_connections 2024-05-14T09:47:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.811+0000 7f6dc9d9d640 1 --2- 172.21.15.2:0/988836572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc4109980 0x7f6dc410be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.811+0000 7f6dc9d9d640 1 --2- 172.21.15.2:0/988836572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc4106e50 0x7f6dc41092f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.811+0000 7f6dc9d9d640 1 --2- 172.21.15.2:0/988836572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dc4104460 0x7f6dc4106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.811+0000 7f6dc9d9d640 1 -- 172.21.15.2:0/988836572 >> 172.21.15.2:0/988836572 conn(0x7f6dc40fddc0 msgr2=0x7f6dc4100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.811+0000 7f6dc9d9d640 1 -- 172.21.15.2:0/988836572 shutdown_connections 2024-05-14T09:47:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.811+0000 7f6dc9d9d640 1 -- 172.21.15.2:0/988836572 wait complete. 2024-05-14T09:47:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.811+0000 7f6dc9d9d640 1 Processor -- start 2024-05-14T09:47:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.811+0000 7f6dc9d9d640 1 -- start start 2024-05-14T09:47:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc9d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc4104460 0x7f6dc419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc9d9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc4106e50 0x7f6dc419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc9d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dc4109980 0x7f6dc41a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc9d9d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc410f7f0 con 0x7f6dc4104460 2024-05-14T09:47:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc9d9d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc410f670 con 0x7f6dc4109980 2024-05-14T09:47:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc9d9d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc410f970 con 0x7f6dc4106e50 2024-05-14T09:47:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc4104460 0x7f6dc419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc4104460 0x7f6dc419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54454/0 (socket says 172.21.15.2:54454) 2024-05-14T09:47:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc37fe640 1 -- 172.21.15.2:0/1779591460 learned_addr learned my addr 172.21.15.2:0/1779591460 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc3fff640 1 --2- 172.21.15.2:0/1779591460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dc4109980 0x7f6dc41a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dbaffd640 1 --2- 172.21.15.2:0/1779591460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc4106e50 0x7f6dc419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc37fe640 1 -- 172.21.15.2:0/1779591460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc4106e50 msgr2=0x7f6dc419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc37fe640 1 --2- 172.21.15.2:0/1779591460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc4106e50 0x7f6dc419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc37fe640 1 -- 172.21.15.2:0/1779591460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dc4109980 msgr2=0x7f6dc41a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc37fe640 1 --2- 172.21.15.2:0/1779591460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dc4109980 0x7f6dc41a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc37fe640 1 -- 172.21.15.2:0/1779591460 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc41a1e70 con 0x7f6dc4104460 2024-05-14T09:47:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.812+0000 7f6dc37fe640 1 --2- 172.21.15.2:0/1779591460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc4104460 0x7f6dc419a330 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6db400e9c0 tx=0x7f6db400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.813+0000 7f6dc17fa640 1 -- 172.21.15.2:0/1779591460 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db400ccc0 con 0x7f6dc4104460 2024-05-14T09:47:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.813+0000 7f6dc17fa640 1 -- 172.21.15.2:0/1779591460 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6db40054d0 con 0x7f6dc4104460 2024-05-14T09:47:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.813+0000 7f6dc17fa640 1 -- 172.21.15.2:0/1779591460 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4010430 con 0x7f6dc4104460 2024-05-14T09:47:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.813+0000 7f6dc9d9d640 1 -- 172.21.15.2:0/1779591460 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc41a2160 con 0x7f6dc4104460 2024-05-14T09:47:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.813+0000 7f6dc9d9d640 1 -- 172.21.15.2:0/1779591460 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc41a2700 con 0x7f6dc4104460 2024-05-14T09:47:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.815+0000 7f6dd0c51740 1 Processor -- start 2024-05-14T09:47:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.815+0000 7f6dd0c51740 1 -- start start 2024-05-14T09:47:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dd0c51740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 0x555744461a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dd0c51740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 0x555744461fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dd0c51740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 0x555744468ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dc17fa640 1 -- 172.21.15.2:0/1779591460 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6db4004f20 con 0x7f6dc4104460 2024-05-14T09:47:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dd0c51740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5557443c65d0 con 0x5557440972f0 2024-05-14T09:47:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dd0c51740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55574437f890 con 0x55574409f4d0 2024-05-14T09:47:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dd0c51740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5557443c2590 con 0x5557440c01b0 2024-05-14T09:47:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dc3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 0x555744461a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dbaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 0x555744468ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dc3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 0x555744461a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:52562/0 (socket says 172.21.15.2:52562) 2024-05-14T09:47:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dc37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 0x555744461fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dc3fff640 1 -- 172.21.15.2:0/2549529115 learned_addr learned my addr 172.21.15.2:0/2549529115 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dc3fff640 1 -- 172.21.15.2:0/2549529115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 msgr2=0x555744468ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dc3fff640 1 --2- 172.21.15.2:0/2549529115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 0x555744468ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dc3fff640 1 -- 172.21.15.2:0/2549529115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 msgr2=0x555744461fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dc3fff640 1 --2- 172.21.15.2:0/2549529115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 0x555744461fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dc3fff640 1 -- 172.21.15.2:0/2549529115 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557440d8210 con 0x5557440c01b0 2024-05-14T09:47:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.816+0000 7f6dc3fff640 1 --2- 172.21.15.2:0/2549529115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 0x555744461a60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc011440 tx=0x7f6dbc011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13bd1b95f9c81142 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.817+0000 7f6dc17fa640 1 --2- 172.21.15.2:0/1779591460 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6db00b2920 0x7f6db00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.817+0000 7f6db9ffb640 1 -- 172.21.15.2:0/2549529115 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc003c30 con 0x5557440c01b0 2024-05-14T09:47:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.817+0000 7f6db9ffb640 1 -- 172.21.15.2:0/2549529115 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dbc003dd0 con 0x5557440c01b0 2024-05-14T09:47:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.817+0000 7f6db9ffb640 1 -- 172.21.15.2:0/2549529115 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00fae0 con 0x5557440c01b0 2024-05-14T09:47:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.817+0000 7f6dc37fe640 1 --2- 172.21.15.2:0/1779591460 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6db00b2920 0x7f6db00b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.817+0000 7f6dc17fa640 1 -- 172.21.15.2:0/1779591460 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f6db40e2be0 con 0x7f6dc4104460 2024-05-14T09:47:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.818+0000 7f6dd0c51740 1 -- 172.21.15.2:0/2549529115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 msgr2=0x555744461a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.818+0000 7f6dc37fe640 1 --2- 172.21.15.2:0/1779591460 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6db00b2920 0x7f6db00b4e30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x555744462e40 tx=0x7f6db40e66c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.818+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/2549529115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 0x555744461a60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc011440 tx=0x7f6dbc011900 comp rx=0 tx=0).stop 2024-05-14T09:47:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.818+0000 7f6dd0c51740 1 -- 172.21.15.2:0/2549529115 shutdown_connections 2024-05-14T09:47:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.818+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/2549529115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 0x555744468ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.818+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/2549529115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 0x555744461fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.818+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/2549529115 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 0x555744461a60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.818+0000 7f6dd0c51740 1 -- 172.21.15.2:0/2549529115 >> 172.21.15.2:0/2549529115 conn(0x5557440b4300 msgr2=0x55574445d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.818+0000 7f6dd0c51740 1 -- 172.21.15.2:0/2549529115 shutdown_connections 2024-05-14T09:47:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.818+0000 7f6dd0c51740 1 -- 172.21.15.2:0/2549529115 wait complete. 2024-05-14T09:47:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.818+0000 7f6dd0c51740 1 -- start start 2024-05-14T09:47:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.818+0000 7f6dd0c51740 1 Processor -- start 2024-05-14T09:47:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dd0c51740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 0x55574446b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dd0c51740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 0x55574446bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dd0c51740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 0x555744472b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dd0c51740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555744469930 con 0x5557440972f0 2024-05-14T09:47:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dd0c51740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555744383370 con 0x55574409f4d0 2024-05-14T09:47:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dd0c51740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555744469ab0 con 0x5557440c01b0 2024-05-14T09:47:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dd0c51740 10 client.0 _ll_register_callbacks cb 0x5557440d7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:47:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dc3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 0x55574446b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dc3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 0x55574446b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54466/0 (socket says 172.21.15.2:54466) 2024-05-14T09:47:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dc3fff640 1 -- 172.21.15.2:0/3655319906 learned_addr learned my addr 172.21.15.2:0/3655319906 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dbaffd640 1 --2- 172.21.15.2:0/3655319906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 0x55574446bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dc3fff640 1 --2- 172.21.15.2:0/3655319906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 0x555744472b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dbaffd640 1 -- 172.21.15.2:0/3655319906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 msgr2=0x555744472b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dbaffd640 1 --2- 172.21.15.2:0/3655319906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 0x555744472b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dbaffd640 1 -- 172.21.15.2:0/3655319906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 msgr2=0x55574446b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dbaffd640 1 --2- 172.21.15.2:0/3655319906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 0x55574446b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dbaffd640 1 -- 172.21.15.2:0/3655319906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557441a4620 con 0x55574409f4d0 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6dbaffd640 1 --2- 172.21.15.2:0/3655319906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 0x55574446bc20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6dc419b3d0 tx=0x7f6dac0317d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3ab1266d9e180cb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.819+0000 7f6da2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.820+0000 7f6da2ffd640 1 -- 172.21.15.2:0/3655319906 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac035040 con 0x55574409f4d0 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.820+0000 7f6da2ffd640 1 -- 172.21.15.2:0/3655319906 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f6dac004240 con 0x55574409f4d0 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.820+0000 7f6dd0c51740 1 -- 172.21.15.2:0/3655319906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dbc003dd0 con 0x55574409f4d0 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.820+0000 7f6da2ffd640 1 -- 172.21.15.2:0/3655319906 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6dac004440 con 0x55574409f4d0 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.820+0000 7f6dd0c51740 10 client.26435 fetch_fsmap learned FSMap version 48 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.820+0000 7f6dd0c51740 1 -- 172.21.15.2:0/3655319906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x555744475ab0 con 0x55574409f4d0 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.820+0000 7f6da2ffd640 1 -- 172.21.15.2:0/3655319906 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dbc003dd0 con 0x55574409f4d0 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.820+0000 7f6da2ffd640 1 -- 172.21.15.2:0/3655319906 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 48) ==== 8350+0+0 (secure 0 0 0) 0x7f6dac0048c0 con 0x55574409f4d0 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.820+0000 7f6dd0c51740 10 client.26435 fetch_fsmap finished waiting for FSMap version 48 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.820+0000 7f6dd0c51740 10 client.26435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.820+0000 7f6dd0c51740 20 client.26435 populate_metadata read hostname 'smithi002' 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.820+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/3655319906 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55574407ed80 0x5557444791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.820+0000 7f6dd0c51740 4 client.26435 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:47:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.820+0000 7f6dd0c51740 1 -- 172.21.15.2:0/3655319906 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557443d41e0 con 0x55574407ed80 2024-05-14T09:47:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.820+0000 7f6dc3fff640 1 --2- 172.21.15.2:0/3655319906 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55574407ed80 0x5557444791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.821+0000 7f6dc3fff640 1 --2- 172.21.15.2:0/3655319906 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55574407ed80 0x5557444791d0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed2e8b7760ddedf server_cookie=62f2bcca50c79e21 in_seq=0 out_seq=0 2024-05-14T09:47:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.821+0000 7f6da2ffd640 10 client.26435 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.822+0000 7f6da2ffd640 1 -- 172.21.15.2:0/3655319906 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5557443d41e0 con 0x55574407ed80 2024-05-14T09:47:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.822+0000 7f6da2ffd640 10 client.26435 handle_command_reply: tid=0 2024-05-14T09:47:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.822+0000 7f6dd0c51740 1 client.26435 shutdown 2024-05-14T09:47:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.822+0000 7f6dd0c51740 1 -- 172.21.15.2:0/3655319906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 msgr2=0x55574446bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.822+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/3655319906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 0x55574446bc20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6dc419b3d0 tx=0x7f6dac0317d0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.822+0000 7f6dd0c51740 1 -- 172.21.15.2:0/3655319906 shutdown_connections 2024-05-14T09:47:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.822+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/3655319906 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55574407ed80 0x5557444791d0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.822+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/3655319906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 0x555744472b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.822+0000 7f6da2ffd640 0 client.26435 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.822+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/3655319906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 0x55574446bc20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.822+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/3655319906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 0x55574446b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.822+0000 7f6dd0c51740 1 -- 172.21.15.2:0/3655319906 >> 172.21.15.2:0/3655319906 conn(0x5557440b4300 msgr2=0x55574445c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.822+0000 7f6dd0c51740 1 -- 172.21.15.2:0/3655319906 shutdown_connections 2024-05-14T09:47:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.822+0000 7f6dd0c51740 1 -- 172.21.15.2:0/3655319906 wait complete. 2024-05-14T09:47:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.823+0000 7f6dd0c51740 20 client.26435 trim_cache size 0 max 16384 2024-05-14T09:47:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.838+0000 7f6dd0c51740 1 Processor -- start 2024-05-14T09:47:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.838+0000 7f6dd0c51740 1 -- start start 2024-05-14T09:47:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dd0c51740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 0x55574445cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dd0c51740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 0x555744472a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dd0c51740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 0x55574445d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dd0c51740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41af5e0 con 0x5557440972f0 2024-05-14T09:47:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dd0c51740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41ace80 con 0x55574409f4d0 2024-05-14T09:47:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dd0c51740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41aacd0 con 0x5557440c01b0 2024-05-14T09:47:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dc3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 0x55574445cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dc3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 0x55574445cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54474/0 (socket says 172.21.15.2:54474) 2024-05-14T09:47:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dc3fff640 1 -- 172.21.15.2:0/3791203368 learned_addr learned my addr 172.21.15.2:0/3791203368 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dbaffd640 1 --2- 172.21.15.2:0/3791203368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 0x555744472a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dc3fff640 1 --2- 172.21.15.2:0/3791203368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 0x55574445d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dc3fff640 1 -- 172.21.15.2:0/3791203368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 msgr2=0x55574445d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dc3fff640 1 --2- 172.21.15.2:0/3791203368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 0x55574445d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dc3fff640 1 -- 172.21.15.2:0/3791203368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 msgr2=0x555744472a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dc3fff640 1 --2- 172.21.15.2:0/3791203368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 0x555744472a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dc3fff640 1 -- 172.21.15.2:0/3791203368 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55574445dd00 con 0x5557440972f0 2024-05-14T09:47:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6dc3fff640 1 --2- 172.21.15.2:0/3791203368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 0x55574445cd60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc000f80 tx=0x7f6dbc02b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd42b59e183c0ec4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.839+0000 7f6db97fa640 1 -- 172.21.15.2:0/3791203368 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc004ac0 con 0x5557440972f0 2024-05-14T09:47:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.840+0000 7f6db97fa640 1 -- 172.21.15.2:0/3791203368 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dbc035070 con 0x5557440972f0 2024-05-14T09:47:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.840+0000 7f6db97fa640 1 -- 172.21.15.2:0/3791203368 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc004000 con 0x5557440972f0 2024-05-14T09:47:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.840+0000 7f6dd0c51740 1 -- 172.21.15.2:0/3791203368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 msgr2=0x55574445cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.840+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/3791203368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 0x55574445cd60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc000f80 tx=0x7f6dbc02b040 comp rx=0 tx=0).stop 2024-05-14T09:47:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.840+0000 7f6dd0c51740 1 -- 172.21.15.2:0/3791203368 shutdown_connections 2024-05-14T09:47:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.840+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/3791203368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5557440c01b0 0x55574445d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.840+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/3791203368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55574409f4d0 0x555744472a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.840+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/3791203368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440972f0 0x55574445cd60 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.841+0000 7f6dd0c51740 1 -- 172.21.15.2:0/3791203368 >> 172.21.15.2:0/3791203368 conn(0x5557440b4300 msgr2=0x555744465b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.841+0000 7f6dd0c51740 1 -- 172.21.15.2:0/3791203368 shutdown_connections 2024-05-14T09:47:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.841+0000 7f6dd0c51740 1 -- 172.21.15.2:0/3791203368 wait complete. 2024-05-14T09:47:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.841+0000 7f6dd0c51740 1 -- start start 2024-05-14T09:47:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.841+0000 7f6dd0c51740 1 Processor -- start 2024-05-14T09:47:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.841+0000 7f6dd0c51740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557440972f0 0x55574446feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.841+0000 7f6dd0c51740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55574409f4d0 0x5557444703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.841+0000 7f6dd0c51740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440c01b0 0x555744471320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.841+0000 7f6dd0c51740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41ab340 con 0x5557440c01b0 2024-05-14T09:47:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.841+0000 7f6dd0c51740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41ac7b0 con 0x5557440972f0 2024-05-14T09:47:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.841+0000 7f6dd0c51740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41aba10 con 0x55574409f4d0 2024-05-14T09:47:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.841+0000 7f6dd0c51740 10 client.0 _ll_register_callbacks cb 0x7f6dbc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:47:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6dbaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55574409f4d0 0x5557444703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6dc3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557440972f0 0x55574446feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6dbaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55574409f4d0 0x5557444703f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:52580/0 (socket says 172.21.15.2:52580) 2024-05-14T09:47:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6dbaffd640 1 -- 172.21.15.2:0/2303984924 learned_addr learned my addr 172.21.15.2:0/2303984924 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:47:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6dc3fff640 1 --2- 172.21.15.2:0/2303984924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440c01b0 0x555744471320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6dbaffd640 1 -- 172.21.15.2:0/2303984924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557440972f0 msgr2=0x55574446feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6dbaffd640 1 --2- 172.21.15.2:0/2303984924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557440972f0 0x55574446feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6dbaffd640 1 -- 172.21.15.2:0/2303984924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440c01b0 msgr2=0x555744471320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:47:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6dbaffd640 1 --2- 172.21.15.2:0/2303984924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440c01b0 0x555744471320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6dbaffd640 1 -- 172.21.15.2:0/2303984924 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557444718e0 con 0x55574409f4d0 2024-05-14T09:47:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6dbaffd640 1 --2- 172.21.15.2:0/2303984924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55574409f4d0 0x5557444703f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6dac02a450 tx=0x7f6dac003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d53217a04297aff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:47:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6da2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:47:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6da2ffd640 1 -- 172.21.15.2:0/2303984924 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac058060 con 0x55574409f4d0 2024-05-14T09:47:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6da2ffd640 1 -- 172.21.15.2:0/2303984924 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f6dac031c50 con 0x55574409f4d0 2024-05-14T09:47:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6dd0c51740 1 -- 172.21.15.2:0/2303984924 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dbc035070 con 0x55574409f4d0 2024-05-14T09:47:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.842+0000 7f6da2ffd640 1 -- 172.21.15.2:0/2303984924 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6dac035040 con 0x55574409f4d0 2024-05-14T09:47:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.843+0000 7f6dd0c51740 10 client.26434 fetch_fsmap learned FSMap version 48 2024-05-14T09:47:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.843+0000 7f6dd0c51740 1 -- 172.21.15.2:0/2303984924 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x555744472c40 con 0x55574409f4d0 2024-05-14T09:47:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.843+0000 7f6da2ffd640 1 -- 172.21.15.2:0/2303984924 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dbc035070 con 0x55574409f4d0 2024-05-14T09:47:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.843+0000 7f6da2ffd640 1 -- 172.21.15.2:0/2303984924 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 48) ==== 8350+0+0 (secure 0 0 0) 0x7f6dac032040 con 0x55574409f4d0 2024-05-14T09:47:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.843+0000 7f6dd0c51740 10 client.26434 fetch_fsmap finished waiting for FSMap version 48 2024-05-14T09:47:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.843+0000 7f6dd0c51740 10 client.26434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:47:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.843+0000 7f6dd0c51740 20 client.26434 populate_metadata read hostname 'smithi002' 2024-05-14T09:47:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.843+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/2303984924 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5557440541c0 0x5557444e3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:47:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.843+0000 7f6dd0c51740 4 client.26434 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:47:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.843+0000 7f6dd0c51740 1 -- 172.21.15.2:0/2303984924 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557443ce050 con 0x5557440541c0 2024-05-14T09:47:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.843+0000 7f6dc3fff640 1 --2- 172.21.15.2:0/2303984924 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5557440541c0 0x5557444e3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:47:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.844+0000 7f6dc3fff640 1 --2- 172.21.15.2:0/2303984924 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5557440541c0 0x5557444e3c30 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37dc85d8fe3d7853 server_cookie=678ef997190e1d1e in_seq=0 out_seq=0 2024-05-14T09:47:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.844+0000 7f6da2ffd640 10 client.26434 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.900+0000 7f6da2ffd640 1 -- 172.21.15.2:0/2303984924 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557443ce050 con 0x5557440541c0 2024-05-14T09:47:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.900+0000 7f6da2ffd640 10 client.26434 handle_command_reply: tid=0 2024-05-14T09:47:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.900+0000 7f6dd0c51740 1 client.26434 shutdown 2024-05-14T09:47:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.900+0000 7f6dd0c51740 1 -- 172.21.15.2:0/2303984924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55574409f4d0 msgr2=0x5557444703f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.900+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/2303984924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55574409f4d0 0x5557444703f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6dac02a450 tx=0x7f6dac003820 comp rx=0 tx=0).stop 2024-05-14T09:47:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.900+0000 7f6dd0c51740 1 -- 172.21.15.2:0/2303984924 shutdown_connections 2024-05-14T09:47:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.900+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/2303984924 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5557440541c0 0x5557444e3c30 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.900+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/2303984924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557440c01b0 0x555744471320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.900+0000 7f6da2ffd640 0 client.26434 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:47:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.900+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/2303984924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55574409f4d0 0x5557444703f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.900+0000 7f6dd0c51740 1 --2- 172.21.15.2:0/2303984924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5557440972f0 0x55574446feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.900+0000 7f6dd0c51740 1 -- 172.21.15.2:0/2303984924 >> 172.21.15.2:0/2303984924 conn(0x5557440b4300 msgr2=0x55574445ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.900+0000 7f6dd0c51740 1 -- 172.21.15.2:0/2303984924 shutdown_connections 2024-05-14T09:47:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.900+0000 7f6dd0c51740 1 -- 172.21.15.2:0/2303984924 wait complete. 2024-05-14T09:47:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.900+0000 7f6dd0c51740 20 client.26434 trim_cache size 0 max 16384 2024-05-14T09:47:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:47:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:47:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:47:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:47:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.901+0000 7f6dc9d9d640 1 -- 172.21.15.2:0/1779591460 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6db00b2920 msgr2=0x7f6db00b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.901+0000 7f6dc9d9d640 1 --2- 172.21.15.2:0/1779591460 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6db00b2920 0x7f6db00b4e30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x555744462e40 tx=0x7f6db40e66c0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.901+0000 7f6dc9d9d640 1 -- 172.21.15.2:0/1779591460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc4104460 msgr2=0x7f6dc419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:47:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.901+0000 7f6dc9d9d640 1 --2- 172.21.15.2:0/1779591460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc4104460 0x7f6dc419a330 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6db400e9c0 tx=0x7f6db400edc0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.901+0000 7f6dc9d9d640 1 -- 172.21.15.2:0/1779591460 shutdown_connections 2024-05-14T09:47:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.901+0000 7f6dc9d9d640 1 --2- 172.21.15.2:0/1779591460 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f6db00b2920 0x7f6db00b4e30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.901+0000 7f6dc9d9d640 1 --2- 172.21.15.2:0/1779591460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dc4109980 0x7f6dc41a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.901+0000 7f6dc9d9d640 1 --2- 172.21.15.2:0/1779591460 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6dc4106e50 0x7f6dc419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.901+0000 7f6dc9d9d640 1 --2- 172.21.15.2:0/1779591460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc4104460 0x7f6dc419a330 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:47:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.901+0000 7f6dc9d9d640 1 -- 172.21.15.2:0/1779591460 >> 172.21.15.2:0/1779591460 conn(0x7f6dc40fddc0 msgr2=0x7f6dc4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:47:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.901+0000 7f6dc9d9d640 1 -- 172.21.15.2:0/1779591460 shutdown_connections 2024-05-14T09:47:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:47:49.901+0000 7f6dc9d9d640 1 -- 172.21.15.2:0/1779591460 wait complete. 2024-05-14T09:47:50.290 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1029659 7.73 MB/sec execute 556 sec latency 12.929 ms 2024-05-14T09:47:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:50 smithi002 ceph-mon[31557]: pgmap v811: 129 pgs: 129 active+clean; 176 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 480 KiB/s wr, 126 op/s 2024-05-14T09:47:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:50 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:47:50.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:50 smithi029 ceph-mon[36249]: pgmap v811: 129 pgs: 129 active+clean; 176 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 480 KiB/s wr, 126 op/s 2024-05-14T09:47:50.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:50 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:47:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:50 smithi026 ceph-mon[36056]: pgmap v811: 129 pgs: 129 active+clean; 176 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 480 KiB/s wr, 126 op/s 2024-05-14T09:47:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:50 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:47:51.290 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1030953 7.73 MB/sec execute 557 sec latency 20.824 ms 2024-05-14T09:47:52.290 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1032722 7.73 MB/sec execute 558 sec latency 15.644 ms 2024-05-14T09:47:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:52 smithi002 ceph-mon[31557]: pgmap v812: 129 pgs: 129 active+clean; 176 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 306 KiB/s wr, 70 op/s 2024-05-14T09:47:52.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:52 smithi029 ceph-mon[36249]: pgmap v812: 129 pgs: 129 active+clean; 176 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 306 KiB/s wr, 70 op/s 2024-05-14T09:47:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:52 smithi026 ceph-mon[36056]: pgmap v812: 129 pgs: 129 active+clean; 176 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 306 KiB/s wr, 70 op/s 2024-05-14T09:47:53.290 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1034171 7.74 MB/sec execute 559 sec latency 24.903 ms 2024-05-14T09:47:54.290 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1034578 7.72 MB/sec execute 560 sec latency 573.191 ms 2024-05-14T09:47:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:54 smithi002 ceph-mon[31557]: pgmap v813: 129 pgs: 129 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 546 KiB/s wr, 131 op/s 2024-05-14T09:47:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:54 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:47:54.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:54 smithi029 ceph-mon[36249]: pgmap v813: 129 pgs: 129 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 546 KiB/s wr, 131 op/s 2024-05-14T09:47:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:54 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:47:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:54 smithi026 ceph-mon[36056]: pgmap v813: 129 pgs: 129 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 546 KiB/s wr, 131 op/s 2024-05-14T09:47:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:54 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:47:55.290 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1036360 7.72 MB/sec execute 561 sec latency 20.467 ms 2024-05-14T09:47:55.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:55 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:47:55.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:55 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:47:55.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:55 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:47:55.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:55 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:47:55.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:55 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:47:55.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:55 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:47:55.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:55 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:47:55.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:55 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:47:55.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:55 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:47:56.290 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1037771 7.73 MB/sec execute 562 sec latency 13.227 ms 2024-05-14T09:47:56.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:56 smithi002 ceph-mon[31557]: pgmap v814: 129 pgs: 129 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 317 KiB/s wr, 83 op/s 2024-05-14T09:47:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:56 smithi029 ceph-mon[36249]: pgmap v814: 129 pgs: 129 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 317 KiB/s wr, 83 op/s 2024-05-14T09:47:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:56 smithi026 ceph-mon[36056]: pgmap v814: 129 pgs: 129 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 317 KiB/s wr, 83 op/s 2024-05-14T09:47:57.290 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1039461 7.73 MB/sec execute 563 sec latency 35.264 ms 2024-05-14T09:47:58.290 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1041114 7.74 MB/sec execute 564 sec latency 25.229 ms 2024-05-14T09:47:58.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:47:58 smithi002 ceph-mon[31557]: pgmap v815: 129 pgs: 129 active+clean; 180 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 419 KiB/s wr, 92 op/s 2024-05-14T09:47:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:47:58 smithi029 ceph-mon[36249]: pgmap v815: 129 pgs: 129 active+clean; 180 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 419 KiB/s wr, 92 op/s 2024-05-14T09:47:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:47:58 smithi026 ceph-mon[36056]: pgmap v815: 129 pgs: 129 active+clean; 180 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 419 KiB/s wr, 92 op/s 2024-05-14T09:47:59.291 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1041661 7.72 MB/sec execute 565 sec latency 286.886 ms 2024-05-14T09:48:00.291 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1041661 7.71 MB/sec execute 566 sec latency 1017.423 ms 2024-05-14T09:48:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:00 smithi002 ceph-mon[31557]: pgmap v816: 129 pgs: 129 active+clean; 182 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 676 KiB/s wr, 155 op/s 2024-05-14T09:48:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:00 smithi029 ceph-mon[36249]: pgmap v816: 129 pgs: 129 active+clean; 182 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 676 KiB/s wr, 155 op/s 2024-05-14T09:48:01.060 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:00 smithi026 ceph-mon[36056]: pgmap v816: 129 pgs: 129 active+clean; 182 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 676 KiB/s wr, 155 op/s 2024-05-14T09:48:01.291 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1041661 7.70 MB/sec execute 567 sec latency 2017.506 ms 2024-05-14T09:48:02.291 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1041661 7.68 MB/sec execute 568 sec latency 3017.628 ms 2024-05-14T09:48:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:02 smithi002 ceph-mon[31557]: pgmap v817: 129 pgs: 129 active+clean; 182 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 652 KiB/s wr, 149 op/s 2024-05-14T09:48:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:02 smithi029 ceph-mon[36249]: pgmap v817: 129 pgs: 129 active+clean; 182 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 652 KiB/s wr, 149 op/s 2024-05-14T09:48:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:02 smithi026 ceph-mon[36056]: pgmap v817: 129 pgs: 129 active+clean; 182 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 652 KiB/s wr, 149 op/s 2024-05-14T09:48:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.038+0000 7f0447ef2640 1 -- 172.21.15.2:0/4289261446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0440108a70 msgr2=0x7f044010ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.038+0000 7f0447ef2640 1 --2- 172.21.15.2:0/4289261446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0440108a70 0x7f044010ae90 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0438009ca0 tx=0x7f043802a8c0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.038+0000 7f0447ef2640 1 -- 172.21.15.2:0/4289261446 shutdown_connections 2024-05-14T09:48:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.039+0000 7f0447ef2640 1 --2- 172.21.15.2:0/4289261446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0440108a70 0x7f044010ae90 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.039+0000 7f0447ef2640 1 --2- 172.21.15.2:0/4289261446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0440103d70 0x7f0440104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.039+0000 7f0447ef2640 1 --2- 172.21.15.2:0/4289261446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0440102b30 0x7f0440103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.039+0000 7f0447ef2640 1 -- 172.21.15.2:0/4289261446 >> 172.21.15.2:0/4289261446 conn(0x7f04400fdd20 msgr2=0x7f04401001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.039+0000 7f0447ef2640 1 -- 172.21.15.2:0/4289261446 shutdown_connections 2024-05-14T09:48:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.039+0000 7f0447ef2640 1 -- 172.21.15.2:0/4289261446 wait complete. 2024-05-14T09:48:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.039+0000 7f0447ef2640 1 Processor -- start 2024-05-14T09:48:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.039+0000 7f0447ef2640 1 -- start start 2024-05-14T09:48:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f0447ef2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0440102b30 0x7f044019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f0447ef2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0440103d70 0x7f044019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f0447ef2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0440108a70 0x7f04401a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f0447ef2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f044010e660 con 0x7f0440108a70 2024-05-14T09:48:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f0447ef2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f044010e4e0 con 0x7f0440103d70 2024-05-14T09:48:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f0447ef2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f044010e7e0 con 0x7f0440102b30 2024-05-14T09:48:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f044648f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0440108a70 0x7f04401a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f044648f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0440108a70 0x7f04401a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48906/0 (socket says 172.21.15.2:48906) 2024-05-14T09:48:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f044648f640 1 -- 172.21.15.2:0/3471940216 learned_addr learned my addr 172.21.15.2:0/3471940216 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f044548d640 1 --2- 172.21.15.2:0/3471940216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0440103d70 0x7f044019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f0445c8e640 1 --2- 172.21.15.2:0/3471940216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0440102b30 0x7f044019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f044648f640 1 -- 172.21.15.2:0/3471940216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0440102b30 msgr2=0x7f044019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f044648f640 1 --2- 172.21.15.2:0/3471940216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0440102b30 0x7f044019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f044648f640 1 -- 172.21.15.2:0/3471940216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0440103d70 msgr2=0x7f044019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f044648f640 1 --2- 172.21.15.2:0/3471940216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0440103d70 0x7f044019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f044648f640 1 -- 172.21.15.2:0/3471940216 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04401a1dd0 con 0x7f0440108a70 2024-05-14T09:48:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f044648f640 1 --2- 172.21.15.2:0/3471940216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0440108a70 0x7f04401a1760 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f04380097b0 tx=0x7f0438003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f0432ffd640 1 -- 172.21.15.2:0/3471940216 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438004240 con 0x7f0440108a70 2024-05-14T09:48:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f0432ffd640 1 -- 172.21.15.2:0/3471940216 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f043802f380 con 0x7f0440108a70 2024-05-14T09:48:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f0447ef2640 1 -- 172.21.15.2:0/3471940216 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04401a2060 con 0x7f0440108a70 2024-05-14T09:48:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.040+0000 7f0432ffd640 1 -- 172.21.15.2:0/3471940216 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043802fc30 con 0x7f0440108a70 2024-05-14T09:48:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.041+0000 7f0447ef2640 1 -- 172.21.15.2:0/3471940216 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04401a25b0 con 0x7f0440108a70 2024-05-14T09:48:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.044+0000 7f044eda3740 1 Processor -- start 2024-05-14T09:48:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.044+0000 7f0432ffd640 1 -- 172.21.15.2:0/3471940216 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f04380043e0 con 0x7f0440108a70 2024-05-14T09:48:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.044+0000 7f044eda3740 1 -- start start 2024-05-14T09:48:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.044+0000 7f044eda3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e40805a1b0 0x55e4083fba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.044+0000 7f044eda3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080312f0 0x55e4083fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.044+0000 7f044eda3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080394d0 0x55e408402ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.044+0000 7f044eda3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e4083605d0 con 0x55e4080312f0 2024-05-14T09:48:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.044+0000 7f044eda3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e408319890 con 0x55e4080394d0 2024-05-14T09:48:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.044+0000 7f044eda3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e40835c590 con 0x55e40805a1b0 2024-05-14T09:48:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.044+0000 7f0445c8e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080312f0 0x55e4083fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.044+0000 7f0445c8e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080312f0 0x55e4083fbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48910/0 (socket says 172.21.15.2:48910) 2024-05-14T09:48:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.044+0000 7f0445c8e640 1 -- 172.21.15.2:0/1425976839 learned_addr learned my addr 172.21.15.2:0/1425976839 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.044+0000 7f044548d640 1 --2- 172.21.15.2:0/1425976839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e40805a1b0 0x55e4083fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.045+0000 7f044548d640 1 --2- 172.21.15.2:0/1425976839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080394d0 0x55e408402ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.045+0000 7f0445c8e640 1 -- 172.21.15.2:0/1425976839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e40805a1b0 msgr2=0x55e4083fba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.045+0000 7f0445c8e640 1 --2- 172.21.15.2:0/1425976839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e40805a1b0 0x55e4083fba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.045+0000 7f0445c8e640 1 -- 172.21.15.2:0/1425976839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080394d0 msgr2=0x55e408402ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.045+0000 7f0445c8e640 1 --2- 172.21.15.2:0/1425976839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080394d0 0x55e408402ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.045+0000 7f0445c8e640 1 -- 172.21.15.2:0/1425976839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e408072210 con 0x55e4080312f0 2024-05-14T09:48:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.045+0000 7f044548d640 1 --2- 172.21.15.2:0/1425976839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080394d0 0x55e408402ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:48:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.045+0000 7f0432ffd640 1 --2- 172.21.15.2:0/3471940216 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f042c0b28a0 0x7f042c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.045+0000 7f0445c8e640 1 --2- 172.21.15.2:0/1425976839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080312f0 0x55e4083fbfe0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f043c00b750 tx=0x7f043c00bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec393b75922fe1f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.045+0000 7f044548d640 1 --2- 172.21.15.2:0/3471940216 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f042c0b28a0 0x7f042c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.045+0000 7f0432ffd640 1 -- 172.21.15.2:0/3471940216 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f0438103d70 con 0x7f0440108a70 2024-05-14T09:48:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.045+0000 7f042bfff640 1 -- 172.21.15.2:0/1425976839 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c00be60 con 0x55e4080312f0 2024-05-14T09:48:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.045+0000 7f042bfff640 1 -- 172.21.15.2:0/1425976839 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f043c015070 con 0x55e4080312f0 2024-05-14T09:48:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.045+0000 7f042bfff640 1 -- 172.21.15.2:0/1425976839 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c0106a0 con 0x55e4080312f0 2024-05-14T09:48:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.046+0000 7f044eda3740 1 -- 172.21.15.2:0/1425976839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080312f0 msgr2=0x55e4083fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.046+0000 7f044548d640 1 --2- 172.21.15.2:0/3471940216 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f042c0b28a0 0x7f042c0b4db0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0434004650 tx=0x7f0434004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.046+0000 7f044eda3740 1 --2- 172.21.15.2:0/1425976839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080312f0 0x55e4083fbfe0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f043c00b750 tx=0x7f043c00bc10 comp rx=0 tx=0).stop 2024-05-14T09:48:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.046+0000 7f044eda3740 1 -- 172.21.15.2:0/1425976839 shutdown_connections 2024-05-14T09:48:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.046+0000 7f044eda3740 1 --2- 172.21.15.2:0/1425976839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080394d0 0x55e408402ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.046+0000 7f044eda3740 1 --2- 172.21.15.2:0/1425976839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080312f0 0x55e4083fbfe0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.046+0000 7f044eda3740 1 --2- 172.21.15.2:0/1425976839 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e40805a1b0 0x55e4083fba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.046+0000 7f044eda3740 1 -- 172.21.15.2:0/1425976839 >> 172.21.15.2:0/1425976839 conn(0x55e40804e300 msgr2=0x55e4083f73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.046+0000 7f044eda3740 1 -- 172.21.15.2:0/1425976839 shutdown_connections 2024-05-14T09:48:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.046+0000 7f044eda3740 1 -- 172.21.15.2:0/1425976839 wait complete. 2024-05-14T09:48:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.047+0000 7f044eda3740 1 -- start start 2024-05-14T09:48:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.047+0000 7f044eda3740 1 Processor -- start 2024-05-14T09:48:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.047+0000 7f044eda3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080312f0 0x55e4084056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.047+0000 7f044eda3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080394d0 0x55e408405be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.047+0000 7f044eda3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e40805a1b0 0x55e40840cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.047+0000 7f044eda3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e408403930 con 0x55e4080394d0 2024-05-14T09:48:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.047+0000 7f044eda3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e40831d370 con 0x55e4080312f0 2024-05-14T09:48:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.047+0000 7f044eda3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e408403ab0 con 0x55e40805a1b0 2024-05-14T09:48:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.047+0000 7f044eda3740 10 client.0 _ll_register_callbacks cb 0x55e408071a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:48:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.047+0000 7f044548d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080394d0 0x55e408405be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.047+0000 7f044648f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e40805a1b0 0x55e40840cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.047+0000 7f044548d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080394d0 0x55e408405be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48918/0 (socket says 172.21.15.2:48918) 2024-05-14T09:48:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.047+0000 7f044548d640 1 -- 172.21.15.2:0/2356269345 learned_addr learned my addr 172.21.15.2:0/2356269345 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.048+0000 7f0445c8e640 1 --2- 172.21.15.2:0/2356269345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080312f0 0x55e4084056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.048+0000 7f044648f640 1 -- 172.21.15.2:0/2356269345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080312f0 msgr2=0x55e4084056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.048+0000 7f044648f640 1 --2- 172.21.15.2:0/2356269345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080312f0 0x55e4084056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.048+0000 7f044648f640 1 -- 172.21.15.2:0/2356269345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080394d0 msgr2=0x55e408405be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.048+0000 7f044648f640 1 --2- 172.21.15.2:0/2356269345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080394d0 0x55e408405be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.048+0000 7f044648f640 1 -- 172.21.15.2:0/2356269345 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40813e620 con 0x55e40805a1b0 2024-05-14T09:48:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.048+0000 7f044548d640 1 --2- 172.21.15.2:0/2356269345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080394d0 0x55e408405be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:48:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.048+0000 7f044648f640 1 --2- 172.21.15.2:0/2356269345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e40805a1b0 0x55e40840cb40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f04381013f0 tx=0x7f0438031880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c95e6ddf87ac255 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.048+0000 7f04297fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:48:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.048+0000 7f04297fa640 1 -- 172.21.15.2:0/2356269345 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04380caf10 con 0x55e40805a1b0 2024-05-14T09:48:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.048+0000 7f04297fa640 1 -- 172.21.15.2:0/2356269345 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f04380046d0 con 0x55e40805a1b0 2024-05-14T09:48:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.048+0000 7f044eda3740 1 -- 172.21.15.2:0/2356269345 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f043c015070 con 0x55e40805a1b0 2024-05-14T09:48:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.048+0000 7f04297fa640 1 -- 172.21.15.2:0/2356269345 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f04380cf2c0 con 0x55e40805a1b0 2024-05-14T09:48:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.048+0000 7f044eda3740 10 client.26446 fetch_fsmap learned FSMap version 48 2024-05-14T09:48:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.048+0000 7f044eda3740 1 -- 172.21.15.2:0/2356269345 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e408403930 con 0x55e40805a1b0 2024-05-14T09:48:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.049+0000 7f04297fa640 1 -- 172.21.15.2:0/2356269345 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f043c015070 con 0x55e40805a1b0 2024-05-14T09:48:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.049+0000 7f04297fa640 1 -- 172.21.15.2:0/2356269345 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 48) ==== 8350+0+0 (secure 0 0 0) 0x7f04380d38e0 con 0x55e40805a1b0 2024-05-14T09:48:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.049+0000 7f044eda3740 10 client.26446 fetch_fsmap finished waiting for FSMap version 48 2024-05-14T09:48:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.049+0000 7f044eda3740 10 client.26446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:48:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.049+0000 7f044eda3740 20 client.26446 populate_metadata read hostname 'smithi002' 2024-05-14T09:48:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.049+0000 7f044eda3740 1 --2- 172.21.15.2:0/2356269345 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e408018d80 0x55e408413140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.049+0000 7f044eda3740 4 client.26446 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:48:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.049+0000 7f044eda3740 1 -- 172.21.15.2:0/2356269345 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e40836e1e0 con 0x55e408018d80 2024-05-14T09:48:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.049+0000 7f0445c8e640 1 --2- 172.21.15.2:0/2356269345 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e408018d80 0x55e408413140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.049+0000 7f0445c8e640 1 --2- 172.21.15.2:0/2356269345 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e408018d80 0x55e408413140 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=564acb25b7948a19 server_cookie=e2df0a8ab6e1a662 in_seq=0 out_seq=0 2024-05-14T09:48:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.049+0000 7f04297fa640 10 client.26446 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.050+0000 7f04297fa640 1 -- 172.21.15.2:0/2356269345 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e40836e1e0 con 0x55e408018d80 2024-05-14T09:48:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.050+0000 7f04297fa640 10 client.26446 handle_command_reply: tid=0 2024-05-14T09:48:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.050+0000 7f044eda3740 1 client.26446 shutdown 2024-05-14T09:48:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.050+0000 7f044eda3740 1 -- 172.21.15.2:0/2356269345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e40805a1b0 msgr2=0x55e40840cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.050+0000 7f044eda3740 1 --2- 172.21.15.2:0/2356269345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e40805a1b0 0x55e40840cb40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f04381013f0 tx=0x7f0438031880 comp rx=0 tx=0).stop 2024-05-14T09:48:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.050+0000 7f044eda3740 1 -- 172.21.15.2:0/2356269345 shutdown_connections 2024-05-14T09:48:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.050+0000 7f044eda3740 1 --2- 172.21.15.2:0/2356269345 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e408018d80 0x55e408413140 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.050+0000 7f044eda3740 1 --2- 172.21.15.2:0/2356269345 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e40805a1b0 0x55e40840cb40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.050+0000 7f044eda3740 1 --2- 172.21.15.2:0/2356269345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080394d0 0x55e408405be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.050+0000 7f044eda3740 1 --2- 172.21.15.2:0/2356269345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080312f0 0x55e4084056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.050+0000 7f04297fa640 0 client.26446 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.050+0000 7f044eda3740 1 -- 172.21.15.2:0/2356269345 >> 172.21.15.2:0/2356269345 conn(0x55e40804e300 msgr2=0x55e4083fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.050+0000 7f044eda3740 1 -- 172.21.15.2:0/2356269345 shutdown_connections 2024-05-14T09:48:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.051+0000 7f044eda3740 1 -- 172.21.15.2:0/2356269345 wait complete. 2024-05-14T09:48:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.051+0000 7f044eda3740 20 client.26446 trim_cache size 0 max 16384 2024-05-14T09:48:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.065+0000 7f044eda3740 1 Processor -- start 2024-05-14T09:48:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044eda3740 1 -- start start 2024-05-14T09:48:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044eda3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080312f0 0x55e408403eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044eda3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e4080394d0 0x55e40840ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044eda3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e40805a1b0 0x55e4084043f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044eda3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f04401af450 con 0x55e40805a1b0 2024-05-14T09:48:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044eda3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04401accf0 con 0x55e4080312f0 2024-05-14T09:48:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044eda3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f04401aab40 con 0x55e4080394d0 2024-05-14T09:48:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044648f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e40805a1b0 0x55e4084043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044648f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e40805a1b0 0x55e4084043f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48920/0 (socket says 172.21.15.2:48920) 2024-05-14T09:48:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044648f640 1 -- 172.21.15.2:0/634251494 learned_addr learned my addr 172.21.15.2:0/634251494 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f0445c8e640 1 --2- 172.21.15.2:0/634251494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080312f0 0x55e408403eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044548d640 1 --2- 172.21.15.2:0/634251494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e4080394d0 0x55e40840ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044648f640 1 -- 172.21.15.2:0/634251494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e4080394d0 msgr2=0x55e40840ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044648f640 1 --2- 172.21.15.2:0/634251494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e4080394d0 0x55e40840ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044648f640 1 -- 172.21.15.2:0/634251494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080312f0 msgr2=0x55e408403eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044648f640 1 --2- 172.21.15.2:0/634251494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080312f0 0x55e408403eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044648f640 1 -- 172.21.15.2:0/634251494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e408404930 con 0x55e40805a1b0 2024-05-14T09:48:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.066+0000 7f044648f640 1 --2- 172.21.15.2:0/634251494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e40805a1b0 0x55e4084043f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f04380e9040 tx=0x7f043802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714a9454e9781c06 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.067+0000 7f042b7fe640 1 -- 172.21.15.2:0/634251494 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04380c6d40 con 0x55e40805a1b0 2024-05-14T09:48:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.067+0000 7f042b7fe640 1 -- 172.21.15.2:0/634251494 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04380a9070 con 0x55e40805a1b0 2024-05-14T09:48:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.067+0000 7f042b7fe640 1 -- 172.21.15.2:0/634251494 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04380d3d70 con 0x55e40805a1b0 2024-05-14T09:48:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.067+0000 7f044eda3740 1 -- 172.21.15.2:0/634251494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e40805a1b0 msgr2=0x55e4084043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.067+0000 7f044eda3740 1 --2- 172.21.15.2:0/634251494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e40805a1b0 0x55e4084043f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f04380e9040 tx=0x7f043802a8c0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.067+0000 7f044eda3740 1 -- 172.21.15.2:0/634251494 shutdown_connections 2024-05-14T09:48:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.067+0000 7f044eda3740 1 --2- 172.21.15.2:0/634251494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e40805a1b0 0x55e4084043f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.067+0000 7f044eda3740 1 --2- 172.21.15.2:0/634251494 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e4080394d0 0x55e40840ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.067+0000 7f044eda3740 1 --2- 172.21.15.2:0/634251494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4080312f0 0x55e408403eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.067+0000 7f044eda3740 1 -- 172.21.15.2:0/634251494 >> 172.21.15.2:0/634251494 conn(0x55e40804e300 msgr2=0x55e4083fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.067+0000 7f044eda3740 1 -- 172.21.15.2:0/634251494 shutdown_connections 2024-05-14T09:48:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.067+0000 7f044eda3740 1 -- 172.21.15.2:0/634251494 wait complete. 2024-05-14T09:48:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.068+0000 7f044eda3740 1 -- start start 2024-05-14T09:48:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.068+0000 7f044eda3740 1 Processor -- start 2024-05-14T09:48:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.068+0000 7f044eda3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e4080312f0 0x55e40840a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.068+0000 7f044eda3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080394d0 0x55e40840a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.068+0000 7f044eda3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e40805a1b0 0x55e40840b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.068+0000 7f044eda3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f04401ab1b0 con 0x55e4080394d0 2024-05-14T09:48:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.068+0000 7f044eda3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04401ac620 con 0x55e40805a1b0 2024-05-14T09:48:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.068+0000 7f044eda3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f04401ab880 con 0x55e4080312f0 2024-05-14T09:48:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.068+0000 7f044eda3740 10 client.0 _ll_register_callbacks cb 0x7f0438009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:48:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.068+0000 7f044548d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080394d0 0x55e40840a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.068+0000 7f044548d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080394d0 0x55e40840a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48936/0 (socket says 172.21.15.2:48936) 2024-05-14T09:48:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.068+0000 7f044548d640 1 -- 172.21.15.2:0/2453438546 learned_addr learned my addr 172.21.15.2:0/2453438546 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.068+0000 7f044648f640 1 --2- 172.21.15.2:0/2453438546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e40805a1b0 0x55e40840b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.068+0000 7f0445c8e640 1 --2- 172.21.15.2:0/2453438546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e4080312f0 0x55e40840a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.069+0000 7f044548d640 1 -- 172.21.15.2:0/2453438546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e4080312f0 msgr2=0x55e40840a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.069+0000 7f044548d640 1 --2- 172.21.15.2:0/2453438546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e4080312f0 0x55e40840a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.069+0000 7f044548d640 1 -- 172.21.15.2:0/2453438546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e40805a1b0 msgr2=0x55e40840b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.069+0000 7f044548d640 1 --2- 172.21.15.2:0/2453438546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e40805a1b0 0x55e40840b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.069+0000 7f044548d640 1 -- 172.21.15.2:0/2453438546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40840bc20 con 0x55e4080394d0 2024-05-14T09:48:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.069+0000 7f0445c8e640 1 --2- 172.21.15.2:0/2453438546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e4080312f0 0x55e40840a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:48:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.069+0000 7f044548d640 1 --2- 172.21.15.2:0/2453438546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080394d0 0x55e40840a730 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f04340118f0 tx=0x7f04340040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5483d411e3d630d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.069+0000 7f04297fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:48:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.069+0000 7f04297fa640 1 -- 172.21.15.2:0/2453438546 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043402c060 con 0x55e4080394d0 2024-05-14T09:48:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.069+0000 7f04297fa640 1 -- 172.21.15.2:0/2453438546 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f043401da10 con 0x55e4080394d0 2024-05-14T09:48:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.069+0000 7f044eda3740 1 -- 172.21.15.2:0/2453438546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04380a9070 con 0x55e4080394d0 2024-05-14T09:48:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.069+0000 7f04297fa640 1 -- 172.21.15.2:0/2453438546 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f043401dc10 con 0x55e4080394d0 2024-05-14T09:48:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.069+0000 7f044eda3740 10 client.16812 fetch_fsmap learned FSMap version 48 2024-05-14T09:48:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.069+0000 7f044eda3740 1 -- 172.21.15.2:0/2453438546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e40847b4c0 con 0x55e4080394d0 2024-05-14T09:48:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.069+0000 7f04297fa640 1 -- 172.21.15.2:0/2453438546 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04380a9070 con 0x55e4080394d0 2024-05-14T09:48:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.070+0000 7f04297fa640 1 -- 172.21.15.2:0/2453438546 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 48) ==== 8350+0+0 (secure 0 0 0) 0x7f0434026070 con 0x55e4080394d0 2024-05-14T09:48:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.070+0000 7f044eda3740 10 client.16812 fetch_fsmap finished waiting for FSMap version 48 2024-05-14T09:48:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.070+0000 7f044eda3740 10 client.16812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:48:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.070+0000 7f044eda3740 20 client.16812 populate_metadata read hostname 'smithi002' 2024-05-14T09:48:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.070+0000 7f044eda3740 1 --2- 172.21.15.2:0/2453438546 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e407fee1c0 0x55e40847deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.070+0000 7f044eda3740 4 client.16812 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:48:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.070+0000 7f044eda3740 1 -- 172.21.15.2:0/2453438546 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e408368050 con 0x55e407fee1c0 2024-05-14T09:48:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.070+0000 7f0445c8e640 1 --2- 172.21.15.2:0/2453438546 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e407fee1c0 0x55e40847deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.070+0000 7f04297fa640 10 client.16812 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.070+0000 7f0445c8e640 1 --2- 172.21.15.2:0/2453438546 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e407fee1c0 0x55e40847deb0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=586c98f321aa3230 server_cookie=62858937cfd2bdd8 in_seq=0 out_seq=0 2024-05-14T09:48:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.152+0000 7f04297fa640 1 -- 172.21.15.2:0/2453438546 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e408368050 con 0x55e407fee1c0 2024-05-14T09:48:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.152+0000 7f04297fa640 10 client.16812 handle_command_reply: tid=0 2024-05-14T09:48:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.152+0000 7f044eda3740 1 client.16812 shutdown 2024-05-14T09:48:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.152+0000 7f044eda3740 1 -- 172.21.15.2:0/2453438546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080394d0 msgr2=0x55e40840a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.152+0000 7f044eda3740 1 --2- 172.21.15.2:0/2453438546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080394d0 0x55e40840a730 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f04340118f0 tx=0x7f04340040f0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.152+0000 7f044eda3740 1 -- 172.21.15.2:0/2453438546 shutdown_connections 2024-05-14T09:48:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.152+0000 7f044eda3740 1 --2- 172.21.15.2:0/2453438546 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e407fee1c0 0x55e40847deb0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.152+0000 7f044eda3740 1 --2- 172.21.15.2:0/2453438546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e40805a1b0 0x55e40840b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.152+0000 7f044eda3740 1 --2- 172.21.15.2:0/2453438546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4080394d0 0x55e40840a730 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.152+0000 7f044eda3740 1 --2- 172.21.15.2:0/2453438546 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e4080312f0 0x55e40840a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.152+0000 7f04297fa640 0 client.16812 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.152+0000 7f044eda3740 1 -- 172.21.15.2:0/2453438546 >> 172.21.15.2:0/2453438546 conn(0x55e40804e300 msgr2=0x55e4083ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.152+0000 7f044eda3740 1 -- 172.21.15.2:0/2453438546 shutdown_connections 2024-05-14T09:48:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.152+0000 7f044eda3740 1 -- 172.21.15.2:0/2453438546 wait complete. 2024-05-14T09:48:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.152+0000 7f044eda3740 20 client.16812 trim_cache size 0 max 16384 2024-05-14T09:48:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:48:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:48:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:48:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:48:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.153+0000 7f0447ef2640 1 -- 172.21.15.2:0/3471940216 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f042c0b28a0 msgr2=0x7f042c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.153+0000 7f0447ef2640 1 --2- 172.21.15.2:0/3471940216 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f042c0b28a0 0x7f042c0b4db0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0434004650 tx=0x7f0434004730 comp rx=0 tx=0).stop 2024-05-14T09:48:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.153+0000 7f0447ef2640 1 -- 172.21.15.2:0/3471940216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0440108a70 msgr2=0x7f04401a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.153+0000 7f0447ef2640 1 --2- 172.21.15.2:0/3471940216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0440108a70 0x7f04401a1760 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f04380097b0 tx=0x7f0438003c20 comp rx=0 tx=0).stop 2024-05-14T09:48:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.153+0000 7f0447ef2640 1 -- 172.21.15.2:0/3471940216 shutdown_connections 2024-05-14T09:48:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.153+0000 7f0447ef2640 1 --2- 172.21.15.2:0/3471940216 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f042c0b28a0 0x7f042c0b4db0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.153+0000 7f0447ef2640 1 --2- 172.21.15.2:0/3471940216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0440108a70 0x7f04401a1760 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.153+0000 7f0447ef2640 1 --2- 172.21.15.2:0/3471940216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0440103d70 0x7f044019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.153+0000 7f0447ef2640 1 --2- 172.21.15.2:0/3471940216 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0440102b30 0x7f044019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.153+0000 7f0447ef2640 1 -- 172.21.15.2:0/3471940216 >> 172.21.15.2:0/3471940216 conn(0x7f04400fdd20 msgr2=0x7f0440109ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.153+0000 7f0447ef2640 1 -- 172.21.15.2:0/3471940216 shutdown_connections 2024-05-14T09:48:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:03.153+0000 7f0447ef2640 1 -- 172.21.15.2:0/3471940216 wait complete. 2024-05-14T09:48:03.291 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1041661 7.67 MB/sec execute 569 sec latency 4017.729 ms 2024-05-14T09:48:04.291 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1041757 7.66 MB/sec execute 570 sec latency 4934.072 ms 2024-05-14T09:48:04.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:04 smithi002 ceph-mon[31557]: pgmap v818: 129 pgs: 129 active+clean; 184 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 785 KiB/s wr, 166 op/s 2024-05-14T09:48:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:04 smithi029 ceph-mon[36249]: pgmap v818: 129 pgs: 129 active+clean; 184 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 785 KiB/s wr, 166 op/s 2024-05-14T09:48:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:04 smithi026 ceph-mon[36056]: pgmap v818: 129 pgs: 129 active+clean; 184 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 785 KiB/s wr, 166 op/s 2024-05-14T09:48:05.291 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1043647 7.66 MB/sec execute 571 sec latency 16.917 ms 2024-05-14T09:48:06.293 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1044868 7.66 MB/sec execute 572 sec latency 14.644 ms 2024-05-14T09:48:06.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:06 smithi002 ceph-mon[31557]: pgmap v819: 129 pgs: 129 active+clean; 184 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 522 KiB/s wr, 96 op/s 2024-05-14T09:48:06.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:06 smithi029 ceph-mon[36249]: pgmap v819: 129 pgs: 129 active+clean; 184 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 522 KiB/s wr, 96 op/s 2024-05-14T09:48:06.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:06 smithi026 ceph-mon[36056]: pgmap v819: 129 pgs: 129 active+clean; 184 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 522 KiB/s wr, 96 op/s 2024-05-14T09:48:07.291 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1045206 7.65 MB/sec execute 573 sec latency 634.771 ms 2024-05-14T09:48:08.291 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1045206 7.64 MB/sec execute 574 sec latency 1634.819 ms 2024-05-14T09:48:08.753 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:08 smithi026 ceph-mon[36056]: pgmap v820: 129 pgs: 129 active+clean; 184 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 684 KiB/s wr, 118 op/s 2024-05-14T09:48:08.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:08 smithi002 ceph-mon[31557]: pgmap v820: 129 pgs: 129 active+clean; 184 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 684 KiB/s wr, 118 op/s 2024-05-14T09:48:08.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:08 smithi029 ceph-mon[36249]: pgmap v820: 129 pgs: 129 active+clean; 184 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 684 KiB/s wr, 118 op/s 2024-05-14T09:48:09.291 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1045318 7.62 MB/sec execute 575 sec latency 2550.780 ms 2024-05-14T09:48:10.291 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1047176 7.63 MB/sec execute 576 sec latency 22.691 ms 2024-05-14T09:48:10.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:10 smithi002 ceph-mon[31557]: pgmap v821: 129 pgs: 129 active+clean; 180 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 860 KiB/s wr, 158 op/s 2024-05-14T09:48:10.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:10 smithi029 ceph-mon[36249]: pgmap v821: 129 pgs: 129 active+clean; 180 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 860 KiB/s wr, 158 op/s 2024-05-14T09:48:10.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:10 smithi026 ceph-mon[36056]: pgmap v821: 129 pgs: 129 active+clean; 180 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 860 KiB/s wr, 158 op/s 2024-05-14T09:48:11.291 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1048411 7.63 MB/sec execute 577 sec latency 20.200 ms 2024-05-14T09:48:12.292 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1048755 7.62 MB/sec execute 578 sec latency 647.294 ms 2024-05-14T09:48:12.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:12 smithi002 ceph-mon[31557]: pgmap v822: 129 pgs: 129 active+clean; 180 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 799 KiB/s wr, 101 op/s 2024-05-14T09:48:12.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:12 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:48:12.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:12 smithi029 ceph-mon[36249]: pgmap v822: 129 pgs: 129 active+clean; 180 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 799 KiB/s wr, 101 op/s 2024-05-14T09:48:12.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:12 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:48:12.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:12 smithi026 ceph-mon[36056]: pgmap v822: 129 pgs: 129 active+clean; 180 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 799 KiB/s wr, 101 op/s 2024-05-14T09:48:12.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:12 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:48:13.292 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1048755 7.60 MB/sec execute 579 sec latency 1647.398 ms 2024-05-14T09:48:14.292 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1048867 7.59 MB/sec execute 580 sec latency 2563.064 ms 2024-05-14T09:48:14.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:14 smithi002 ceph-mon[31557]: pgmap v823: 129 pgs: 129 active+clean; 181 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-05-14T09:48:14.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:14 smithi029 ceph-mon[36249]: pgmap v823: 129 pgs: 129 active+clean; 181 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-05-14T09:48:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:14 smithi026 ceph-mon[36056]: pgmap v823: 129 pgs: 129 active+clean; 181 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-05-14T09:48:15.292 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1050724 7.59 MB/sec execute 581 sec latency 18.843 ms 2024-05-14T09:48:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.255+0000 7fc8b9d2e640 1 -- 172.21.15.2:0/521809347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b4109a20 msgr2=0x7fc8b410be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.255+0000 7fc8b9d2e640 1 --2- 172.21.15.2:0/521809347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b4109a20 0x7fc8b410be40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009ca0 tx=0x7fc8ac02a8c0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.255+0000 7fc8b9d2e640 1 -- 172.21.15.2:0/521809347 shutdown_connections 2024-05-14T09:48:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.255+0000 7fc8b9d2e640 1 --2- 172.21.15.2:0/521809347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b4109a20 0x7fc8b410be40 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.255+0000 7fc8b9d2e640 1 --2- 172.21.15.2:0/521809347 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b41070c0 0x7fc8b41094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.255+0000 7fc8b9d2e640 1 --2- 172.21.15.2:0/521809347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b4104760 0x7fc8b4106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.255+0000 7fc8b9d2e640 1 -- 172.21.15.2:0/521809347 >> 172.21.15.2:0/521809347 conn(0x7fc8b40fdd60 msgr2=0x7fc8b4100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.255+0000 7fc8b9d2e640 1 -- 172.21.15.2:0/521809347 shutdown_connections 2024-05-14T09:48:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.255+0000 7fc8b9d2e640 1 -- 172.21.15.2:0/521809347 wait complete. 2024-05-14T09:48:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.256+0000 7fc8b9d2e640 1 Processor -- start 2024-05-14T09:48:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.256+0000 7fc8b9d2e640 1 -- start start 2024-05-14T09:48:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.256+0000 7fc8b9d2e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b4104760 0x7fc8b4103bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.256+0000 7fc8b9d2e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b41070c0 0x7fc8b4102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.256+0000 7fc8b9d2e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b4109a20 0x7fc8b4102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.256+0000 7fc8b9d2e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b410f7a0 con 0x7fc8b41070c0 2024-05-14T09:48:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.256+0000 7fc8b9d2e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b410f620 con 0x7fc8b4104760 2024-05-14T09:48:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.256+0000 7fc8b9d2e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b410f920 con 0x7fc8b4109a20 2024-05-14T09:48:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.256+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b41070c0 0x7fc8b4102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.256+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b41070c0 0x7fc8b4102250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38202/0 (socket says 172.21.15.2:38202) 2024-05-14T09:48:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.256+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/495387898 learned_addr learned my addr 172.21.15.2:0/495387898 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.256+0000 7fc8b37fe640 1 --2- 172.21.15.2:0/495387898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b4104760 0x7fc8b4103bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.256+0000 7fc8b3fff640 1 --2- 172.21.15.2:0/495387898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b4109a20 0x7fc8b4102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.257+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/495387898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b4109a20 msgr2=0x7fc8b4102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.257+0000 7fc8b2ffd640 1 --2- 172.21.15.2:0/495387898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b4109a20 0x7fc8b4102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.257+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/495387898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b4104760 msgr2=0x7fc8b4103bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.257+0000 7fc8b2ffd640 1 --2- 172.21.15.2:0/495387898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b4104760 0x7fc8b4103bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.257+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/495387898 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b4102ec0 con 0x7fc8b41070c0 2024-05-14T09:48:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.257+0000 7fc8b2ffd640 1 --2- 172.21.15.2:0/495387898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b41070c0 0x7fc8b4102250 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800e9f0 tx=0x7fc8a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.257+0000 7fc8b0ff9640 1 -- 172.21.15.2:0/495387898 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800cd80 con 0x7fc8b41070c0 2024-05-14T09:48:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.257+0000 7fc8b9d2e640 1 -- 172.21.15.2:0/495387898 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b41a4300 con 0x7fc8b41070c0 2024-05-14T09:48:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.257+0000 7fc8b0ff9640 1 -- 172.21.15.2:0/495387898 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a80054d0 con 0x7fc8b41070c0 2024-05-14T09:48:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.257+0000 7fc8b0ff9640 1 -- 172.21.15.2:0/495387898 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8010690 con 0x7fc8b41070c0 2024-05-14T09:48:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.257+0000 7fc8b9d2e640 1 -- 172.21.15.2:0/495387898 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b41a4770 con 0x7fc8b41070c0 2024-05-14T09:48:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8c0bdc740 1 Processor -- start 2024-05-14T09:48:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8c0bdc740 1 -- start start 2024-05-14T09:48:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8c0bdc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 0x55b15e7eba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8c0bdc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4212f0 0x55b15e7ebfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8c0bdc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e4294d0 0x55b15e7f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8c0bdc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b15e7505d0 con 0x55b15e4212f0 2024-05-14T09:48:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8c0bdc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b15e709890 con 0x55b15e44a1b0 2024-05-14T09:48:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8c0bdc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b15e74c590 con 0x55b15e4294d0 2024-05-14T09:48:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8b37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4212f0 0x55b15e7ebfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8b37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4212f0 0x55b15e7ebfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38216/0 (socket says 172.21.15.2:38216) 2024-05-14T09:48:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8b37fe640 1 -- 172.21.15.2:0/921948780 learned_addr learned my addr 172.21.15.2:0/921948780 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8b3fff640 1 --2- 172.21.15.2:0/921948780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 0x55b15e7eba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8b0ff9640 1 -- 172.21.15.2:0/495387898 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc8a8010830 con 0x7fc8b41070c0 2024-05-14T09:48:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8b37fe640 1 -- 172.21.15.2:0/921948780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e4294d0 msgr2=0x55b15e7f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8b37fe640 1 --2- 172.21.15.2:0/921948780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e4294d0 0x55b15e7f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8b37fe640 1 -- 172.21.15.2:0/921948780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 msgr2=0x55b15e7eba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8b37fe640 1 --2- 172.21.15.2:0/921948780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 0x55b15e7eba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.260+0000 7fc8b37fe640 1 -- 172.21.15.2:0/921948780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b15e462210 con 0x55b15e4212f0 2024-05-14T09:48:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.261+0000 7fc8b37fe640 1 --2- 172.21.15.2:0/921948780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4212f0 0x55b15e7ebfe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009c90 tx=0x7fc8a4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c83af5e60cece30 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.261+0000 7fc899ffb640 1 -- 172.21.15.2:0/921948780 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a401e070 con 0x55b15e4212f0 2024-05-14T09:48:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.261+0000 7fc899ffb640 1 -- 172.21.15.2:0/921948780 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a4019ae0 con 0x55b15e4212f0 2024-05-14T09:48:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.261+0000 7fc899ffb640 1 -- 172.21.15.2:0/921948780 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a400e940 con 0x55b15e4212f0 2024-05-14T09:48:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.261+0000 7fc8b0ff9640 1 --2- 172.21.15.2:0/495387898 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc89c0b28a0 0x7fc89c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.261+0000 7fc8b3fff640 1 --2- 172.21.15.2:0/495387898 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc89c0b28a0 0x7fc89c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.261+0000 7fc8b0ff9640 1 -- 172.21.15.2:0/495387898 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fc8a80e2a40 con 0x7fc8b41070c0 2024-05-14T09:48:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.262+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/921948780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4212f0 msgr2=0x55b15e7ebfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.262+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/921948780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4212f0 0x55b15e7ebfe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009c90 tx=0x7fc8a4011810 comp rx=0 tx=0).stop 2024-05-14T09:48:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.262+0000 7fc8b3fff640 1 --2- 172.21.15.2:0/495387898 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc89c0b28a0 0x7fc89c0b4db0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac004af0 tx=0x7fc8ac009380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.262+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/921948780 shutdown_connections 2024-05-14T09:48:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.262+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/921948780 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e4294d0 0x55b15e7f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.262+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/921948780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4212f0 0x55b15e7ebfe0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.262+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/921948780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 0x55b15e7eba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.262+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/921948780 >> 172.21.15.2:0/921948780 conn(0x55b15e43e300 msgr2=0x55b15e7e73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.262+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/921948780 shutdown_connections 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.262+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/921948780 wait complete. 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.262+0000 7fc8c0bdc740 1 -- start start 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.262+0000 7fc8c0bdc740 1 Processor -- start 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8c0bdc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e4212f0 0x55b15e7f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8c0bdc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8c0bdc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 0x55b15e7fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8c0bdc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b15e7f3930 con 0x55b15e4294d0 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8c0bdc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b15e70d370 con 0x55b15e44a1b0 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8c0bdc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b15e7f3ab0 con 0x55b15e4212f0 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8c0bdc740 10 client.0 _ll_register_callbacks cb 0x55b15e461a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8b37fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e4212f0 0x55b15e7f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7f5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38218/0 (socket says 172.21.15.2:38218) 2024-05-14T09:48:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/2440290084 learned_addr learned my addr 172.21.15.2:0/2440290084 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8b3fff640 1 --2- 172.21.15.2:0/2440290084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 0x55b15e7fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/2440290084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e4212f0 msgr2=0x55b15e7f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8b2ffd640 1 --2- 172.21.15.2:0/2440290084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e4212f0 0x55b15e7f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/2440290084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 msgr2=0x55b15e7fcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8b2ffd640 1 --2- 172.21.15.2:0/2440290084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 0x55b15e7fcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/2440290084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b15e52e620 con 0x55b15e4294d0 2024-05-14T09:48:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8b3fff640 1 --2- 172.21.15.2:0/2440290084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 0x55b15e7fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:48:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8b2ffd640 1 --2- 172.21.15.2:0/2440290084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7f5c30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80a6eb0 tx=0x7fc8a80a93b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a9e1cb5fa8d7ab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:48:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc892ffd640 1 -- 172.21.15.2:0/2440290084 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a80aa340 con 0x55b15e4294d0 2024-05-14T09:48:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc892ffd640 1 -- 172.21.15.2:0/2440290084 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fc8a80b2610 con 0x55b15e4294d0 2024-05-14T09:48:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.263+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/2440290084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8a4019ae0 con 0x55b15e4294d0 2024-05-14T09:48:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.264+0000 7fc892ffd640 1 -- 172.21.15.2:0/2440290084 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc8a80a9c40 con 0x55b15e4294d0 2024-05-14T09:48:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.264+0000 7fc8c0bdc740 10 client.16824 fetch_fsmap learned FSMap version 49 2024-05-14T09:48:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.264+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/2440290084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55b15e7ffb90 con 0x55b15e4294d0 2024-05-14T09:48:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.264+0000 7fc892ffd640 1 -- 172.21.15.2:0/2440290084 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a4019ae0 con 0x55b15e4294d0 2024-05-14T09:48:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.264+0000 7fc892ffd640 1 -- 172.21.15.2:0/2440290084 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 49) ==== 8346+0+0 (secure 0 0 0) 0x7fc8a8010b20 con 0x55b15e4294d0 2024-05-14T09:48:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.264+0000 7fc8c0bdc740 10 client.16824 fetch_fsmap finished waiting for FSMap version 49 2024-05-14T09:48:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.264+0000 7fc8c0bdc740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:48:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.264+0000 7fc8c0bdc740 20 client.16824 populate_metadata read hostname 'smithi002' 2024-05-14T09:48:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.264+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/2440290084 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b15e408d80 0x55b15e8032e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.264+0000 7fc8c0bdc740 4 client.16824 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:48:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.264+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/2440290084 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b15e75e1e0 con 0x55b15e408d80 2024-05-14T09:48:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.264+0000 7fc8b37fe640 1 --2- 172.21.15.2:0/2440290084 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b15e408d80 0x55b15e8032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.265+0000 7fc892ffd640 10 client.16824 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.265+0000 7fc8b37fe640 1 --2- 172.21.15.2:0/2440290084 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b15e408d80 0x55b15e8032e0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5b5618d775cf6b1 server_cookie=9e4bd1058de2eba7 in_seq=0 out_seq=0 2024-05-14T09:48:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.265+0000 7fc892ffd640 1 -- 172.21.15.2:0/2440290084 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b15e75e1e0 con 0x55b15e408d80 2024-05-14T09:48:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.265+0000 7fc892ffd640 10 client.16824 handle_command_reply: tid=0 2024-05-14T09:48:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.265+0000 7fc8c0bdc740 1 client.16824 shutdown 2024-05-14T09:48:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.266+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/2440290084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 msgr2=0x55b15e7f5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.266+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/2440290084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7f5c30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80a6eb0 tx=0x7fc8a80a93b0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.266+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/2440290084 shutdown_connections 2024-05-14T09:48:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.266+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/2440290084 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b15e408d80 0x55b15e8032e0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.266+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/2440290084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 0x55b15e7fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.266+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/2440290084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7f5c30 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.266+0000 7fc892ffd640 0 client.16824 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.266+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/2440290084 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e4212f0 0x55b15e7f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.266+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/2440290084 >> 172.21.15.2:0/2440290084 conn(0x55b15e43e300 msgr2=0x55b15e7ea300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.266+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/2440290084 shutdown_connections 2024-05-14T09:48:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.266+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/2440290084 wait complete. 2024-05-14T09:48:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.266+0000 7fc8c0bdc740 20 client.16824 trim_cache size 0 max 16384 2024-05-14T09:48:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.281+0000 7fc8c0bdc740 1 Processor -- start 2024-05-14T09:48:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.281+0000 7fc8c0bdc740 1 -- start start 2024-05-14T09:48:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.281+0000 7fc8c0bdc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e4212f0 0x55b15e7e6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.281+0000 7fc8c0bdc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7e72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.281+0000 7fc8c0bdc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 0x55b15e7fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.281+0000 7fc8c0bdc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b41b1600 con 0x55b15e4294d0 2024-05-14T09:48:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.281+0000 7fc8c0bdc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b41aeed0 con 0x55b15e44a1b0 2024-05-14T09:48:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.281+0000 7fc8c0bdc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b41acd20 con 0x55b15e4212f0 2024-05-14T09:48:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.282+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7e72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.282+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7e72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38232/0 (socket says 172.21.15.2:38232) 2024-05-14T09:48:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.282+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/1843627933 learned_addr learned my addr 172.21.15.2:0/1843627933 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.282+0000 7fc8b37fe640 1 --2- 172.21.15.2:0/1843627933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e4212f0 0x55b15e7e6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.282+0000 7fc8b3fff640 1 --2- 172.21.15.2:0/1843627933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 0x55b15e7fcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.282+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/1843627933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e4212f0 msgr2=0x55b15e7e6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.282+0000 7fc8b2ffd640 1 --2- 172.21.15.2:0/1843627933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e4212f0 0x55b15e7e6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.282+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/1843627933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 msgr2=0x55b15e7fcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.282+0000 7fc8b2ffd640 1 --2- 172.21.15.2:0/1843627933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 0x55b15e7fcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.282+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/1843627933 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b15e7f2f40 con 0x55b15e4294d0 2024-05-14T09:48:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.282+0000 7fc8b2ffd640 1 --2- 172.21.15.2:0/1843627933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7e72a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80a9940 tx=0x7fc8a800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410a73ae2638c39c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.282+0000 7fc8997fa640 1 -- 172.21.15.2:0/1843627933 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a80a53d0 con 0x55b15e4294d0 2024-05-14T09:48:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.282+0000 7fc8997fa640 1 -- 172.21.15.2:0/1843627933 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a80c2060 con 0x55b15e4294d0 2024-05-14T09:48:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.282+0000 7fc8997fa640 1 -- 172.21.15.2:0/1843627933 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8010b20 con 0x55b15e4294d0 2024-05-14T09:48:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.283+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/1843627933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 msgr2=0x55b15e7e72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.283+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/1843627933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7e72a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80a9940 tx=0x7fc8a800cf90 comp rx=0 tx=0).stop 2024-05-14T09:48:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.283+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/1843627933 shutdown_connections 2024-05-14T09:48:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.283+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/1843627933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e44a1b0 0x55b15e7fcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.283+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/1843627933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7e72a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.283+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/1843627933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e4212f0 0x55b15e7e6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.283+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/1843627933 >> 172.21.15.2:0/1843627933 conn(0x55b15e43e300 msgr2=0x55b15e7efb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.283+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/1843627933 shutdown_connections 2024-05-14T09:48:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.283+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/1843627933 wait complete. 2024-05-14T09:48:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.283+0000 7fc8c0bdc740 1 -- start start 2024-05-14T09:48:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8c0bdc740 1 Processor -- start 2024-05-14T09:48:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8c0bdc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e4212f0 0x55b15e7fa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8c0bdc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7fac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8c0bdc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e44a1b0 0x55b15e7fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8c0bdc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b41ad390 con 0x55b15e4294d0 2024-05-14T09:48:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8c0bdc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b41ae800 con 0x55b15e4212f0 2024-05-14T09:48:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8c0bdc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b41ada60 con 0x55b15e44a1b0 2024-05-14T09:48:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8c0bdc740 10 client.0 _ll_register_callbacks cb 0x7fc8a800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:48:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7fac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8b37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e4212f0 0x55b15e7fa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7fac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38234/0 (socket says 172.21.15.2:38234) 2024-05-14T09:48:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/1693497455 learned_addr learned my addr 172.21.15.2:0/1693497455 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8b3fff640 1 --2- 172.21.15.2:0/1693497455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e44a1b0 0x55b15e7fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/1693497455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e44a1b0 msgr2=0x55b15e7fbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8b2ffd640 1 --2- 172.21.15.2:0/1693497455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e44a1b0 0x55b15e7fbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/1693497455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e4212f0 msgr2=0x55b15e7fa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.284+0000 7fc8b2ffd640 1 --2- 172.21.15.2:0/1693497455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e4212f0 0x55b15e7fa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.285+0000 7fc8b2ffd640 1 -- 172.21.15.2:0/1693497455 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b15e7fc0a0 con 0x55b15e4294d0 2024-05-14T09:48:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.285+0000 7fc8b2ffd640 1 --2- 172.21.15.2:0/1693497455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7fac20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80a9e00 tx=0x7fc8a8004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0c0675b8c8fffa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.285+0000 7fc892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:48:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.285+0000 7fc892ffd640 1 -- 172.21.15.2:0/1693497455 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a80a53d0 con 0x55b15e4294d0 2024-05-14T09:48:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.285+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/1693497455 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8a80c2060 con 0x55b15e4294d0 2024-05-14T09:48:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.285+0000 7fc892ffd640 1 -- 172.21.15.2:0/1693497455 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fc8a80b2820 con 0x55b15e4294d0 2024-05-14T09:48:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.285+0000 7fc892ffd640 1 -- 172.21.15.2:0/1693497455 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc8a80a55b0 con 0x55b15e4294d0 2024-05-14T09:48:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.285+0000 7fc8c0bdc740 10 client.16830 fetch_fsmap learned FSMap version 49 2024-05-14T09:48:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.285+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/1693497455 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55b15e7fd330 con 0x55b15e4294d0 2024-05-14T09:48:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.285+0000 7fc892ffd640 1 -- 172.21.15.2:0/1693497455 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a80c2060 con 0x55b15e4294d0 2024-05-14T09:48:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.285+0000 7fc892ffd640 1 -- 172.21.15.2:0/1693497455 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 49) ==== 8346+0+0 (secure 0 0 0) 0x7fc8a8091cb0 con 0x55b15e4294d0 2024-05-14T09:48:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.285+0000 7fc8c0bdc740 10 client.16830 fetch_fsmap finished waiting for FSMap version 49 2024-05-14T09:48:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.285+0000 7fc8c0bdc740 10 client.16830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:48:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.285+0000 7fc8c0bdc740 20 client.16830 populate_metadata read hostname 'smithi002' 2024-05-14T09:48:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.285+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/1693497455 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b15e3de1c0 0x55b15e86d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.286+0000 7fc8c0bdc740 4 client.16830 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:48:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.286+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/1693497455 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b15e758050 con 0x55b15e3de1c0 2024-05-14T09:48:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.286+0000 7fc8b37fe640 1 --2- 172.21.15.2:0/1693497455 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b15e3de1c0 0x55b15e86d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.286+0000 7fc8b37fe640 1 --2- 172.21.15.2:0/1693497455 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b15e3de1c0 0x55b15e86d8b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66bf6bbb277786b1 server_cookie=941d15f7b2a8a529 in_seq=0 out_seq=0 2024-05-14T09:48:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.286+0000 7fc892ffd640 10 client.16830 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:16.292 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1051941 7.60 MB/sec execute 582 sec latency 50.033 ms 2024-05-14T09:48:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.370+0000 7fc892ffd640 1 -- 172.21.15.2:0/1693497455 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b15e758050 con 0x55b15e3de1c0 2024-05-14T09:48:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.370+0000 7fc892ffd640 10 client.16830 handle_command_reply: tid=0 2024-05-14T09:48:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.370+0000 7fc8c0bdc740 1 client.16830 shutdown 2024-05-14T09:48:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/1693497455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 msgr2=0x55b15e7fac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/1693497455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7fac20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80a9e00 tx=0x7fc8a8004a90 comp rx=0 tx=0).stop 2024-05-14T09:48:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/1693497455 shutdown_connections 2024-05-14T09:48:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/1693497455 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b15e3de1c0 0x55b15e86d8b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/1693497455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b15e44a1b0 0x55b15e7fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/1693497455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b15e4294d0 0x55b15e7fac20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc8c0bdc740 1 --2- 172.21.15.2:0/1693497455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b15e4212f0 0x55b15e7fa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc892ffd640 0 client.16830 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/1693497455 >> 172.21.15.2:0/1693497455 conn(0x55b15e43e300 msgr2=0x55b15e7ea1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/1693497455 shutdown_connections 2024-05-14T09:48:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc8c0bdc740 1 -- 172.21.15.2:0/1693497455 wait complete. 2024-05-14T09:48:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc8c0bdc740 20 client.16830 trim_cache size 0 max 16384 2024-05-14T09:48:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:48:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:48:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:48:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:48:16.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc8b9d2e640 1 -- 172.21.15.2:0/495387898 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc89c0b28a0 msgr2=0x7fc89c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:16.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc8b9d2e640 1 --2- 172.21.15.2:0/495387898 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc89c0b28a0 0x7fc89c0b4db0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac004af0 tx=0x7fc8ac009380 comp rx=0 tx=0).stop 2024-05-14T09:48:16.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc8b9d2e640 1 -- 172.21.15.2:0/495387898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b41070c0 msgr2=0x7fc8b4102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.371+0000 7fc8b9d2e640 1 --2- 172.21.15.2:0/495387898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b41070c0 0x7fc8b4102250 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800e9f0 tx=0x7fc8a800edf0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.372+0000 7fc8b9d2e640 1 -- 172.21.15.2:0/495387898 shutdown_connections 2024-05-14T09:48:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.372+0000 7fc8b9d2e640 1 --2- 172.21.15.2:0/495387898 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc89c0b28a0 0x7fc89c0b4db0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.372+0000 7fc8b9d2e640 1 --2- 172.21.15.2:0/495387898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8b4109a20 0x7fc8b4102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.372+0000 7fc8b9d2e640 1 --2- 172.21.15.2:0/495387898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b41070c0 0x7fc8b4102250 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.372+0000 7fc8b9d2e640 1 --2- 172.21.15.2:0/495387898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8b4104760 0x7fc8b4103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.372+0000 7fc8b9d2e640 1 -- 172.21.15.2:0/495387898 >> 172.21.15.2:0/495387898 conn(0x7fc8b40fdd60 msgr2=0x7fc8b41051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.372+0000 7fc8b9d2e640 1 -- 172.21.15.2:0/495387898 shutdown_connections 2024-05-14T09:48:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:16.372+0000 7fc8b9d2e640 1 -- 172.21.15.2:0/495387898 wait complete. 2024-05-14T09:48:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:16 smithi002 ceph-mon[31557]: pgmap v824: 129 pgs: 129 active+clean; 181 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-05-14T09:48:16.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:16 smithi029 ceph-mon[36249]: pgmap v824: 129 pgs: 129 active+clean; 181 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-05-14T09:48:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:16 smithi026 ceph-mon[36056]: pgmap v824: 129 pgs: 129 active+clean; 181 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-05-14T09:48:17.292 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1052175 7.58 MB/sec execute 583 sec latency 730.477 ms 2024-05-14T09:48:18.292 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1052175 7.57 MB/sec execute 584 sec latency 1730.579 ms 2024-05-14T09:48:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:18 smithi002 ceph-mon[31557]: pgmap v825: 129 pgs: 129 active+clean; 181 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-05-14T09:48:18.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:18 smithi029 ceph-mon[36249]: pgmap v825: 129 pgs: 129 active+clean; 181 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-05-14T09:48:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:18 smithi026 ceph-mon[36056]: pgmap v825: 129 pgs: 129 active+clean; 181 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-05-14T09:48:19.292 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1052231 7.56 MB/sec execute 585 sec latency 2646.809 ms 2024-05-14T09:48:20.292 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1052231 7.54 MB/sec execute 586 sec latency 1024.685 ms 2024-05-14T09:48:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:20 smithi002 ceph-mon[31557]: pgmap v826: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-05-14T09:48:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:20 smithi029 ceph-mon[36249]: pgmap v826: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-05-14T09:48:20.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:20 smithi026 ceph-mon[36056]: pgmap v826: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-05-14T09:48:21.292 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1052231 7.53 MB/sec execute 587 sec latency 2024.779 ms 2024-05-14T09:48:22.293 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1052231 7.52 MB/sec execute 588 sec latency 3024.908 ms 2024-05-14T09:48:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:22 smithi002 ceph-mon[31557]: pgmap v827: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-05-14T09:48:22.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:22 smithi029 ceph-mon[36249]: pgmap v827: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-05-14T09:48:22.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:22 smithi026 ceph-mon[36056]: pgmap v827: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-05-14T09:48:23.293 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1052231 7.51 MB/sec execute 589 sec latency 4025.028 ms 2024-05-14T09:48:24.293 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1052299 7.49 MB/sec execute 590 sec latency 4943.396 ms 2024-05-14T09:48:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:24 smithi002 ceph-mon[31557]: pgmap v828: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 929 KiB/s wr, 134 op/s 2024-05-14T09:48:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:24 smithi029 ceph-mon[36249]: pgmap v828: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 929 KiB/s wr, 134 op/s 2024-05-14T09:48:24.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:24 smithi026 ceph-mon[36056]: pgmap v828: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 929 KiB/s wr, 134 op/s 2024-05-14T09:48:25.293 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1054110 7.49 MB/sec execute 591 sec latency 18.156 ms 2024-05-14T09:48:26.293 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1055488 7.50 MB/sec execute 592 sec latency 22.842 ms 2024-05-14T09:48:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:26 smithi002 ceph-mon[31557]: pgmap v829: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 450 KiB/s wr, 81 op/s 2024-05-14T09:48:26.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:26 smithi029 ceph-mon[36249]: pgmap v829: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 450 KiB/s wr, 81 op/s 2024-05-14T09:48:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:26 smithi026 ceph-mon[36056]: pgmap v829: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 450 KiB/s wr, 81 op/s 2024-05-14T09:48:27.293 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1055722 7.49 MB/sec execute 593 sec latency 774.346 ms 2024-05-14T09:48:27.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:27 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:48:27.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:27 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:48:27.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:27 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:48:28.293 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1055722 7.47 MB/sec execute 594 sec latency 1774.455 ms 2024-05-14T09:48:28.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:28 smithi002 ceph-mon[31557]: pgmap v830: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 839 KiB/s wr, 104 op/s 2024-05-14T09:48:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:28 smithi029 ceph-mon[36249]: pgmap v830: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 839 KiB/s wr, 104 op/s 2024-05-14T09:48:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:28 smithi026 ceph-mon[36056]: pgmap v830: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 839 KiB/s wr, 104 op/s 2024-05-14T09:48:29.293 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1055788 7.46 MB/sec execute 595 sec latency 2692.315 ms 2024-05-14T09:48:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.482+0000 7f4632ce0640 1 -- 172.21.15.2:0/3791916975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f462c104e90 msgr2=0x7f462c1052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.482+0000 7f4632ce0640 1 --2- 172.21.15.2:0/3791916975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f462c104e90 0x7f462c1052a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4618009900 tx=0x7f461802a410 comp rx=0 tx=0).stop 2024-05-14T09:48:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.483+0000 7f4632ce0640 1 -- 172.21.15.2:0/3791916975 shutdown_connections 2024-05-14T09:48:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.483+0000 7f4632ce0640 1 --2- 172.21.15.2:0/3791916975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c109b90 0x7f462c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.483+0000 7f4632ce0640 1 --2- 172.21.15.2:0/3791916975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f462c104e90 0x7f462c1052a0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.483+0000 7f4632ce0640 1 --2- 172.21.15.2:0/3791916975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f462c103c50 0x7f462c104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.483+0000 7f4632ce0640 1 -- 172.21.15.2:0/3791916975 >> 172.21.15.2:0/3791916975 conn(0x7f462c0fee40 msgr2=0x7f462c1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.483+0000 7f4632ce0640 1 -- 172.21.15.2:0/3791916975 shutdown_connections 2024-05-14T09:48:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.483+0000 7f4632ce0640 1 -- 172.21.15.2:0/3791916975 wait complete. 2024-05-14T09:48:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.483+0000 7f4632ce0640 1 Processor -- start 2024-05-14T09:48:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.483+0000 7f4632ce0640 1 -- start start 2024-05-14T09:48:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.483+0000 7f4632ce0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c103c50 0x7f462c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f4632ce0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f462c104e90 0x7f462c19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f4632ce0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f462c109b90 0x7f462c1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f4632ce0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f462c10f7c0 con 0x7f462c109b90 2024-05-14T09:48:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f4632ce0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f462c10f640 con 0x7f462c103c50 2024-05-14T09:48:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f4632ce0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f462c10f940 con 0x7f462c104e90 2024-05-14T09:48:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f463127d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f462c109b90 0x7f462c1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f462bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f462c104e90 0x7f462c19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f463127d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f462c109b90 0x7f462c1a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38052/0 (socket says 172.21.15.2:38052) 2024-05-14T09:48:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f463127d640 1 -- 172.21.15.2:0/735413301 learned_addr learned my addr 172.21.15.2:0/735413301 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f4630a7c640 1 --2- 172.21.15.2:0/735413301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c103c50 0x7f462c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f463127d640 1 -- 172.21.15.2:0/735413301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f462c104e90 msgr2=0x7f462c19a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f463127d640 1 --2- 172.21.15.2:0/735413301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f462c104e90 0x7f462c19a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f463127d640 1 -- 172.21.15.2:0/735413301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c103c50 msgr2=0x7f462c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f463127d640 1 --2- 172.21.15.2:0/735413301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c103c50 0x7f462c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f463127d640 1 -- 172.21.15.2:0/735413301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c1a1dd0 con 0x7f462c109b90 2024-05-14T09:48:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f462bfff640 1 --2- 172.21.15.2:0/735413301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f462c104e90 0x7f462c19a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:48:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f463127d640 1 --2- 172.21.15.2:0/735413301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f462c109b90 0x7f462c1a1760 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f462400eaa0 tx=0x7f462400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f4629ffb640 1 -- 172.21.15.2:0/735413301 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462400cdb0 con 0x7f462c109b90 2024-05-14T09:48:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f4632ce0640 1 -- 172.21.15.2:0/735413301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c1a20c0 con 0x7f462c109b90 2024-05-14T09:48:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f4629ffb640 1 -- 172.21.15.2:0/735413301 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4624005020 con 0x7f462c109b90 2024-05-14T09:48:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.484+0000 7f4629ffb640 1 -- 172.21.15.2:0/735413301 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46240116d0 con 0x7f462c109b90 2024-05-14T09:48:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.485+0000 7f4632ce0640 1 -- 172.21.15.2:0/735413301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c1a2590 con 0x7f462c109b90 2024-05-14T09:48:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.487+0000 7f4639b99740 1 Processor -- start 2024-05-14T09:48:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.487+0000 7f4629ffb640 1 -- 172.21.15.2:0/735413301 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f46240051d0 con 0x7f462c109b90 2024-05-14T09:48:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f4639b99740 1 -- start start 2024-05-14T09:48:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f4639b99740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bd91b0 0x5592b3f7aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f4639b99740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb02f0 0x5592b3f7afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f4639b99740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3f81ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f4639b99740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5592b3edf5d0 con 0x5592b3bb84d0 2024-05-14T09:48:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f4639b99740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5592b3e98890 con 0x5592b3bd91b0 2024-05-14T09:48:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f4639b99740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5592b3edb590 con 0x5592b3bb02f0 2024-05-14T09:48:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f462bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3f81ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f462bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3f81ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38064/0 (socket says 172.21.15.2:38064) 2024-05-14T09:48:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f462bfff640 1 -- 172.21.15.2:0/958078777 learned_addr learned my addr 172.21.15.2:0/958078777 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f462bfff640 1 --2- 172.21.15.2:0/958078777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bd91b0 0x5592b3f7aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f4630a7c640 1 --2- 172.21.15.2:0/958078777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb02f0 0x5592b3f7afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f462bfff640 1 -- 172.21.15.2:0/958078777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb02f0 msgr2=0x5592b3f7afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f462bfff640 1 --2- 172.21.15.2:0/958078777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb02f0 0x5592b3f7afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f462bfff640 1 -- 172.21.15.2:0/958078777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bd91b0 msgr2=0x5592b3f7aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f462bfff640 1 --2- 172.21.15.2:0/958078777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bd91b0 0x5592b3f7aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.488+0000 7f4629ffb640 1 --2- 172.21.15.2:0/735413301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f461c0b28a0 0x7f461c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.489+0000 7f462bfff640 1 -- 172.21.15.2:0/958078777 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b3bf1210 con 0x5592b3bb84d0 2024-05-14T09:48:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.489+0000 7f463127d640 1 --2- 172.21.15.2:0/735413301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f461c0b28a0 0x7f461c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.489+0000 7f462bfff640 1 --2- 172.21.15.2:0/958078777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3f81ee0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4618005f70 tx=0x7f4618005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d56caa084db24b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.489+0000 7f4629ffb640 1 -- 172.21.15.2:0/735413301 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f46240e2780 con 0x7f462c109b90 2024-05-14T09:48:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.489+0000 7f4616ffd640 1 -- 172.21.15.2:0/958078777 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618035040 con 0x5592b3bb84d0 2024-05-14T09:48:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.489+0000 7f4616ffd640 1 -- 172.21.15.2:0/958078777 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46180315a0 con 0x5592b3bb84d0 2024-05-14T09:48:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.489+0000 7f4616ffd640 1 -- 172.21.15.2:0/958078777 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461803d680 con 0x5592b3bb84d0 2024-05-14T09:48:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.490+0000 7f4639b99740 1 -- 172.21.15.2:0/958078777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 msgr2=0x5592b3f81ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.490+0000 7f463127d640 1 --2- 172.21.15.2:0/735413301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f461c0b28a0 0x7f461c0b4db0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f46240a6ca0 tx=0x7f4624073320 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.490+0000 7f4639b99740 1 --2- 172.21.15.2:0/958078777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3f81ee0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4618005f70 tx=0x7f4618005d90 comp rx=0 tx=0).stop 2024-05-14T09:48:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.490+0000 7f4639b99740 1 -- 172.21.15.2:0/958078777 shutdown_connections 2024-05-14T09:48:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.490+0000 7f4639b99740 1 --2- 172.21.15.2:0/958078777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3f81ee0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.490+0000 7f4639b99740 1 --2- 172.21.15.2:0/958078777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb02f0 0x5592b3f7afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.490+0000 7f4639b99740 1 --2- 172.21.15.2:0/958078777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bd91b0 0x5592b3f7aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.490+0000 7f4639b99740 1 -- 172.21.15.2:0/958078777 >> 172.21.15.2:0/958078777 conn(0x5592b3bcd300 msgr2=0x5592b3f763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.490+0000 7f4639b99740 1 -- 172.21.15.2:0/958078777 shutdown_connections 2024-05-14T09:48:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.490+0000 7f4639b99740 1 -- 172.21.15.2:0/958078777 wait complete. 2024-05-14T09:48:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.490+0000 7f4639b99740 1 -- start start 2024-05-14T09:48:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.490+0000 7f4639b99740 1 Processor -- start 2024-05-14T09:48:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f4639b99740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb02f0 0x5592b3f84650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f4639b99740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3f84b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f4639b99740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bd91b0 0x5592b3f8baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f4639b99740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5592b3f828d0 con 0x5592b3bb84d0 2024-05-14T09:48:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f4639b99740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5592b3e9c370 con 0x5592b3bd91b0 2024-05-14T09:48:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f4639b99740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5592b3f82a50 con 0x5592b3bb02f0 2024-05-14T09:48:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f4639b99740 10 client.0 _ll_register_callbacks cb 0x5592b3bf0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:48:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f4630a7c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3f84b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f462bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb02f0 0x5592b3f84650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f462bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb02f0 0x5592b3f84650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:45704/0 (socket says 172.21.15.2:45704) 2024-05-14T09:48:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f462bfff640 1 -- 172.21.15.2:0/3065007144 learned_addr learned my addr 172.21.15.2:0/3065007144 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f4630a7c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3f84b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38074/0 (socket says 172.21.15.2:38074) 2024-05-14T09:48:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f462bfff640 1 --2- 172.21.15.2:0/3065007144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bd91b0 0x5592b3f8baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f4630a7c640 1 -- 172.21.15.2:0/3065007144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb02f0 msgr2=0x5592b3f84650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f4630a7c640 1 --2- 172.21.15.2:0/3065007144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb02f0 0x5592b3f84650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f4630a7c640 1 -- 172.21.15.2:0/3065007144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bd91b0 msgr2=0x5592b3f8baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.491+0000 7f4630a7c640 1 --2- 172.21.15.2:0/3065007144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bd91b0 0x5592b3f8baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.492+0000 7f4630a7c640 1 -- 172.21.15.2:0/3065007144 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b3cbd620 con 0x5592b3bb84d0 2024-05-14T09:48:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.492+0000 7f462bfff640 1 --2- 172.21.15.2:0/3065007144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb02f0 0x5592b3f84650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:48:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.492+0000 7f4630a7c640 1 --2- 172.21.15.2:0/3065007144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3f84b90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x5592b3f7be40 tx=0x7f4620011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209468645c51fd9f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.492+0000 7f460bfff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:48:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.492+0000 7f460bfff640 1 -- 172.21.15.2:0/3065007144 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620005020 con 0x5592b3bb84d0 2024-05-14T09:48:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.492+0000 7f460bfff640 1 -- 172.21.15.2:0/3065007144 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f46200051c0 con 0x5592b3bb84d0 2024-05-14T09:48:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.492+0000 7f4639b99740 1 -- 172.21.15.2:0/3065007144 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46180315a0 con 0x5592b3bb84d0 2024-05-14T09:48:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.492+0000 7f460bfff640 1 -- 172.21.15.2:0/3065007144 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f4620025050 con 0x5592b3bb84d0 2024-05-14T09:48:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.492+0000 7f4639b99740 10 client.16842 fetch_fsmap learned FSMap version 50 2024-05-14T09:48:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.492+0000 7f4639b99740 1 -- 172.21.15.2:0/3065007144 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5592b3f8ea90 con 0x5592b3bb84d0 2024-05-14T09:48:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.492+0000 7f460bfff640 1 -- 172.21.15.2:0/3065007144 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46180315a0 con 0x5592b3bb84d0 2024-05-14T09:48:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.492+0000 7f460bfff640 1 -- 172.21.15.2:0/3065007144 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) ==== 8350+0+0 (secure 0 0 0) 0x7f4620010760 con 0x5592b3bb84d0 2024-05-14T09:48:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.493+0000 7f4639b99740 10 client.16842 fetch_fsmap finished waiting for FSMap version 50 2024-05-14T09:48:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.493+0000 7f4639b99740 10 client.16842 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:48:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.493+0000 7f4639b99740 20 client.16842 populate_metadata read hostname 'smithi002' 2024-05-14T09:48:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.493+0000 7f4639b99740 1 --2- 172.21.15.2:0/3065007144 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b3b97d80 0x5592b3f921f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.493+0000 7f4639b99740 4 client.16842 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:48:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.493+0000 7f4639b99740 1 -- 172.21.15.2:0/3065007144 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592b3eed1e0 con 0x5592b3b97d80 2024-05-14T09:48:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.493+0000 7f462bfff640 1 --2- 172.21.15.2:0/3065007144 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b3b97d80 0x5592b3f921f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.493+0000 7f460bfff640 10 client.16842 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.493+0000 7f462bfff640 1 --2- 172.21.15.2:0/3065007144 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b3b97d80 0x5592b3f921f0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ff1a25de1f178c7 server_cookie=ce1f763e2e9d4bd7 in_seq=0 out_seq=0 2024-05-14T09:48:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.494+0000 7f460bfff640 1 -- 172.21.15.2:0/3065007144 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592b3eed1e0 con 0x5592b3b97d80 2024-05-14T09:48:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.494+0000 7f460bfff640 10 client.16842 handle_command_reply: tid=0 2024-05-14T09:48:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.494+0000 7f4639b99740 1 client.16842 shutdown 2024-05-14T09:48:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.494+0000 7f4639b99740 1 -- 172.21.15.2:0/3065007144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 msgr2=0x5592b3f84b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.494+0000 7f4639b99740 1 --2- 172.21.15.2:0/3065007144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3f84b90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x5592b3f7be40 tx=0x7f4620011810 comp rx=0 tx=0).stop 2024-05-14T09:48:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.494+0000 7f4639b99740 1 -- 172.21.15.2:0/3065007144 shutdown_connections 2024-05-14T09:48:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.494+0000 7f4639b99740 1 --2- 172.21.15.2:0/3065007144 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b3b97d80 0x5592b3f921f0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.494+0000 7f4639b99740 1 --2- 172.21.15.2:0/3065007144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bd91b0 0x5592b3f8baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.494+0000 7f460bfff640 0 client.16842 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.494+0000 7f4639b99740 1 --2- 172.21.15.2:0/3065007144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3f84b90 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.494+0000 7f4639b99740 1 --2- 172.21.15.2:0/3065007144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb02f0 0x5592b3f84650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.494+0000 7f4639b99740 1 -- 172.21.15.2:0/3065007144 >> 172.21.15.2:0/3065007144 conn(0x5592b3bcd300 msgr2=0x5592b3f79300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.494+0000 7f4639b99740 1 -- 172.21.15.2:0/3065007144 shutdown_connections 2024-05-14T09:48:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.494+0000 7f4639b99740 1 -- 172.21.15.2:0/3065007144 wait complete. 2024-05-14T09:48:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.494+0000 7f4639b99740 20 client.16842 trim_cache size 0 max 16384 2024-05-14T09:48:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f4639b99740 1 Processor -- start 2024-05-14T09:48:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f4639b99740 1 -- start start 2024-05-14T09:48:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f4639b99740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3b97d80 0x5592b3f75dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f4639b99740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bb02f0 0x5592b3f76310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f4639b99740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb84d0 0x5592b3f74970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f4639b99740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1af540 con 0x5592b3b97d80 2024-05-14T09:48:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f4639b99740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1acde0 con 0x5592b3bb02f0 2024-05-14T09:48:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f4639b99740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1aac30 con 0x5592b3bb84d0 2024-05-14T09:48:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f462bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3b97d80 0x5592b3f75dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f462bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3b97d80 0x5592b3f75dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38078/0 (socket says 172.21.15.2:38078) 2024-05-14T09:48:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f462bfff640 1 -- 172.21.15.2:0/559505524 learned_addr learned my addr 172.21.15.2:0/559505524 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f4630a7c640 1 --2- 172.21.15.2:0/559505524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bb02f0 0x5592b3f76310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f462bfff640 1 -- 172.21.15.2:0/559505524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb84d0 msgr2=0x5592b3f74970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f462bfff640 1 --2- 172.21.15.2:0/559505524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb84d0 0x5592b3f74970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f462bfff640 1 -- 172.21.15.2:0/559505524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bb02f0 msgr2=0x5592b3f76310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f462bfff640 1 --2- 172.21.15.2:0/559505524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bb02f0 0x5592b3f76310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f462bfff640 1 -- 172.21.15.2:0/559505524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b3f82270 con 0x5592b3b97d80 2024-05-14T09:48:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.510+0000 7f4630a7c640 1 --2- 172.21.15.2:0/559505524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bb02f0 0x5592b3f76310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:48:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.511+0000 7f462bfff640 1 --2- 172.21.15.2:0/559505524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3b97d80 0x5592b3f75dd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4618005e00 tx=0x7f4618000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10acf54253803f7b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.511+0000 7f46167fc640 1 -- 172.21.15.2:0/559505524 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618035040 con 0x5592b3b97d80 2024-05-14T09:48:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.511+0000 7f46167fc640 1 -- 172.21.15.2:0/559505524 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4618003740 con 0x5592b3b97d80 2024-05-14T09:48:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.511+0000 7f46167fc640 1 -- 172.21.15.2:0/559505524 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46180564f0 con 0x5592b3b97d80 2024-05-14T09:48:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.511+0000 7f4639b99740 1 -- 172.21.15.2:0/559505524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3b97d80 msgr2=0x5592b3f75dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.511+0000 7f4639b99740 1 --2- 172.21.15.2:0/559505524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3b97d80 0x5592b3f75dd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4618005e00 tx=0x7f4618000bc0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.512+0000 7f4639b99740 1 -- 172.21.15.2:0/559505524 shutdown_connections 2024-05-14T09:48:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.512+0000 7f4639b99740 1 --2- 172.21.15.2:0/559505524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3bb84d0 0x5592b3f74970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.512+0000 7f4639b99740 1 --2- 172.21.15.2:0/559505524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bb02f0 0x5592b3f76310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.512+0000 7f4639b99740 1 --2- 172.21.15.2:0/559505524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3b97d80 0x5592b3f75dd0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.512+0000 7f4639b99740 1 -- 172.21.15.2:0/559505524 >> 172.21.15.2:0/559505524 conn(0x5592b3bcd300 msgr2=0x5592b3f8be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.512+0000 7f4639b99740 1 -- 172.21.15.2:0/559505524 shutdown_connections 2024-05-14T09:48:29.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.512+0000 7f4639b99740 1 -- 172.21.15.2:0/559505524 wait complete. 2024-05-14T09:48:29.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.512+0000 7f4639b99740 1 -- start start 2024-05-14T09:48:29.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.512+0000 7f4639b99740 1 Processor -- start 2024-05-14T09:48:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.512+0000 7f4639b99740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3b97d80 0x5592b3ff5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.512+0000 7f4639b99740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bb02f0 0x5592b3ff61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.512+0000 7f4639b99740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3ffd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.512+0000 7f4639b99740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1ab2a0 con 0x5592b3bb84d0 2024-05-14T09:48:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f4639b99740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1ac710 con 0x5592b3bb02f0 2024-05-14T09:48:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f4639b99740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1ab970 con 0x5592b3b97d80 2024-05-14T09:48:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f4639b99740 10 client.0 _ll_register_callbacks cb 0x7f462000b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:48:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f462bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3ffd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f4630a7c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bb02f0 0x5592b3ff61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f462bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3ffd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38082/0 (socket says 172.21.15.2:38082) 2024-05-14T09:48:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f462bfff640 1 -- 172.21.15.2:0/2000323041 learned_addr learned my addr 172.21.15.2:0/2000323041 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f462bfff640 1 --2- 172.21.15.2:0/2000323041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3b97d80 0x5592b3ff5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f4630a7c640 1 -- 172.21.15.2:0/2000323041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3b97d80 msgr2=0x5592b3ff5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f4630a7c640 1 --2- 172.21.15.2:0/2000323041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3b97d80 0x5592b3ff5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f4630a7c640 1 -- 172.21.15.2:0/2000323041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 msgr2=0x5592b3ffd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f4630a7c640 1 --2- 172.21.15.2:0/2000323041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3ffd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f4630a7c640 1 -- 172.21.15.2:0/2000323041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b3ffd6d0 con 0x5592b3bb02f0 2024-05-14T09:48:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f462bfff640 1 --2- 172.21.15.2:0/2000323041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3b97d80 0x5592b3ff5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:48:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f4630a7c640 1 --2- 172.21.15.2:0/2000323041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bb02f0 0x5592b3ff61b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x5592b3f706d0 tx=0x7f462000b400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a57c66207e3db00 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f460bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:48:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f460bfff640 1 -- 172.21.15.2:0/2000323041 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462002d070 con 0x5592b3bb02f0 2024-05-14T09:48:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f4639b99740 1 -- 172.21.15.2:0/2000323041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4618003740 con 0x5592b3bb02f0 2024-05-14T09:48:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.513+0000 7f460bfff640 1 -- 172.21.15.2:0/2000323041 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f4620010640 con 0x5592b3bb02f0 2024-05-14T09:48:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.514+0000 7f460bfff640 1 -- 172.21.15.2:0/2000323041 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f4620010840 con 0x5592b3bb02f0 2024-05-14T09:48:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.514+0000 7f4639b99740 10 client.26492 fetch_fsmap learned FSMap version 50 2024-05-14T09:48:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.514+0000 7f4639b99740 1 -- 172.21.15.2:0/2000323041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5592b3ffe9c0 con 0x5592b3bb02f0 2024-05-14T09:48:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.514+0000 7f460bfff640 1 -- 172.21.15.2:0/2000323041 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4618003740 con 0x5592b3bb02f0 2024-05-14T09:48:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.514+0000 7f460bfff640 1 -- 172.21.15.2:0/2000323041 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 50) ==== 8350+0+0 (secure 0 0 0) 0x7f46200324d0 con 0x5592b3bb02f0 2024-05-14T09:48:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.514+0000 7f4639b99740 10 client.26492 fetch_fsmap finished waiting for FSMap version 50 2024-05-14T09:48:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.514+0000 7f4639b99740 10 client.26492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:48:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.514+0000 7f4639b99740 20 client.26492 populate_metadata read hostname 'smithi002' 2024-05-14T09:48:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.514+0000 7f4639b99740 1 --2- 172.21.15.2:0/2000323041 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b3bd91b0 0x5592b40013b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.514+0000 7f4639b99740 4 client.26492 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:48:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.514+0000 7f4639b99740 1 -- 172.21.15.2:0/2000323041 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592b3ee7050 con 0x5592b3bd91b0 2024-05-14T09:48:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.514+0000 7f462bfff640 1 --2- 172.21.15.2:0/2000323041 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b3bd91b0 0x5592b40013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.515+0000 7f460bfff640 10 client.26492 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.515+0000 7f462bfff640 1 --2- 172.21.15.2:0/2000323041 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b3bd91b0 0x5592b40013b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7336495a1564c69 server_cookie=61c4a2ce62282613 in_seq=0 out_seq=0 2024-05-14T09:48:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.578+0000 7f460bfff640 1 -- 172.21.15.2:0/2000323041 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592b3ee7050 con 0x5592b3bd91b0 2024-05-14T09:48:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.578+0000 7f460bfff640 10 client.26492 handle_command_reply: tid=0 2024-05-14T09:48:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.578+0000 7f4639b99740 1 client.26492 shutdown 2024-05-14T09:48:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.578+0000 7f4639b99740 1 -- 172.21.15.2:0/2000323041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bb02f0 msgr2=0x5592b3ff61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.578+0000 7f4639b99740 1 --2- 172.21.15.2:0/2000323041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bb02f0 0x5592b3ff61b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x5592b3f706d0 tx=0x7f462000b400 comp rx=0 tx=0).stop 2024-05-14T09:48:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.578+0000 7f4639b99740 1 -- 172.21.15.2:0/2000323041 shutdown_connections 2024-05-14T09:48:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.578+0000 7f4639b99740 1 --2- 172.21.15.2:0/2000323041 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b3bd91b0 0x5592b40013b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.578+0000 7f4639b99740 1 --2- 172.21.15.2:0/2000323041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b3bb84d0 0x5592b3ffd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.578+0000 7f4639b99740 1 --2- 172.21.15.2:0/2000323041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b3bb02f0 0x5592b3ff61b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.578+0000 7f4639b99740 1 --2- 172.21.15.2:0/2000323041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b3b97d80 0x5592b3ff5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.578+0000 7f460bfff640 0 client.26492 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.578+0000 7f4639b99740 1 -- 172.21.15.2:0/2000323041 >> 172.21.15.2:0/2000323041 conn(0x5592b3bcd300 msgr2=0x5592b3f791f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.578+0000 7f4639b99740 1 -- 172.21.15.2:0/2000323041 shutdown_connections 2024-05-14T09:48:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.578+0000 7f4639b99740 1 -- 172.21.15.2:0/2000323041 wait complete. 2024-05-14T09:48:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.578+0000 7f4639b99740 20 client.26492 trim_cache size 0 max 16384 2024-05-14T09:48:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:48:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:48:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:48:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:48:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.579+0000 7f4632ce0640 1 -- 172.21.15.2:0/735413301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f461c0b28a0 msgr2=0x7f461c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.579+0000 7f4632ce0640 1 --2- 172.21.15.2:0/735413301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f461c0b28a0 0x7f461c0b4db0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f46240a6ca0 tx=0x7f4624073320 comp rx=0 tx=0).stop 2024-05-14T09:48:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.579+0000 7f4632ce0640 1 -- 172.21.15.2:0/735413301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f462c109b90 msgr2=0x7f462c1a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.579+0000 7f4632ce0640 1 --2- 172.21.15.2:0/735413301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f462c109b90 0x7f462c1a1760 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f462400eaa0 tx=0x7f462400ef60 comp rx=0 tx=0).stop 2024-05-14T09:48:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.579+0000 7f4632ce0640 1 -- 172.21.15.2:0/735413301 shutdown_connections 2024-05-14T09:48:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.579+0000 7f4632ce0640 1 --2- 172.21.15.2:0/735413301 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f461c0b28a0 0x7f461c0b4db0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.579+0000 7f4632ce0640 1 --2- 172.21.15.2:0/735413301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f462c109b90 0x7f462c1a1760 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.579+0000 7f4632ce0640 1 --2- 172.21.15.2:0/735413301 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f462c104e90 0x7f462c19a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.579+0000 7f4632ce0640 1 --2- 172.21.15.2:0/735413301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c103c50 0x7f462c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.579+0000 7f4632ce0640 1 -- 172.21.15.2:0/735413301 >> 172.21.15.2:0/735413301 conn(0x7f462c0fee40 msgr2=0x7f462c105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.579+0000 7f4632ce0640 1 -- 172.21.15.2:0/735413301 shutdown_connections 2024-05-14T09:48:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:29.579+0000 7f4632ce0640 1 -- 172.21.15.2:0/735413301 wait complete. 2024-05-14T09:48:30.293 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1055846 7.45 MB/sec execute 596 sec latency 940.887 ms 2024-05-14T09:48:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:30 smithi002 ceph-mon[31557]: pgmap v831: 129 pgs: 129 active+clean; 183 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 864 KiB/s wr, 115 op/s 2024-05-14T09:48:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:30 smithi029 ceph-mon[36249]: pgmap v831: 129 pgs: 129 active+clean; 183 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 864 KiB/s wr, 115 op/s 2024-05-14T09:48:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:30 smithi026 ceph-mon[36056]: pgmap v831: 129 pgs: 129 active+clean; 183 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 864 KiB/s wr, 115 op/s 2024-05-14T09:48:31.293 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1055846 7.44 MB/sec execute 597 sec latency 1940.977 ms 2024-05-14T09:48:32.294 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1055846 7.42 MB/sec execute 598 sec latency 2941.088 ms 2024-05-14T09:48:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:32 smithi002 ceph-mon[31557]: pgmap v832: 129 pgs: 129 active+clean; 183 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 746 KiB/s wr, 82 op/s 2024-05-14T09:48:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:32 smithi029 ceph-mon[36249]: pgmap v832: 129 pgs: 129 active+clean; 183 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 746 KiB/s wr, 82 op/s 2024-05-14T09:48:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:32 smithi026 ceph-mon[36056]: pgmap v832: 129 pgs: 129 active+clean; 183 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 746 KiB/s wr, 82 op/s 2024-05-14T09:48:33.294 INFO:tasks.workunit.client.0.smithi002.stdout: 1 1055846 7.41 MB/sec execute 599 sec latency 3941.192 ms 2024-05-14T09:48:34.294 INFO:tasks.workunit.client.0.smithi002.stdout: 1 cleanup 600 sec 2024-05-14T09:48:34.495 INFO:tasks.workunit.client.0.smithi002.stdout: 0 cleanup 600 sec 2024-05-14T09:48:34.495 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-05-14T09:48:34.495 INFO:tasks.workunit.client.0.smithi002.stdout: Operation Count AvgLat MaxLat 2024-05-14T09:48:34.495 INFO:tasks.workunit.client.0.smithi002.stdout: ---------------------------------------- 2024-05-14T09:48:34.495 INFO:tasks.workunit.client.0.smithi002.stdout: NTCreateX 141258 2.315 4998.494 2024-05-14T09:48:34.495 INFO:tasks.workunit.client.0.smithi002.stdout: Close 103762 0.002 0.096 2024-05-14T09:48:34.495 INFO:tasks.workunit.client.0.smithi002.stdout: Rename 5981 6.705 4917.686 2024-05-14T09:48:34.495 INFO:tasks.workunit.client.0.smithi002.stdout: Unlink 28525 1.945 152.504 2024-05-14T09:48:34.495 INFO:tasks.workunit.client.0.smithi002.stdout: Deltree 4 118.027 169.403 2024-05-14T09:48:34.496 INFO:tasks.workunit.client.0.smithi002.stdout: Mkdir 2 0.005 0.006 2024-05-14T09:48:34.496 INFO:tasks.workunit.client.0.smithi002.stdout: Qpathinfo 128037 0.919 258.753 2024-05-14T09:48:34.496 INFO:tasks.workunit.client.0.smithi002.stdout: Qfileinfo 22453 0.001 0.053 2024-05-14T09:48:34.496 INFO:tasks.workunit.client.0.smithi002.stdout: Qfsinfo 23478 0.185 5.617 2024-05-14T09:48:34.496 INFO:tasks.workunit.client.0.smithi002.stdout: Sfileinfo 11503 0.026 21.442 2024-05-14T09:48:34.496 INFO:tasks.workunit.client.0.smithi002.stdout: Find 49504 0.166 48.385 2024-05-14T09:48:34.496 INFO:tasks.workunit.client.0.smithi002.stdout: WriteX 70510 0.015 57.397 2024-05-14T09:48:34.496 INFO:tasks.workunit.client.0.smithi002.stdout: ReadX 221429 0.011 54.456 2024-05-14T09:48:34.496 INFO:tasks.workunit.client.0.smithi002.stdout: LockX 460 0.846 2.681 2024-05-14T09:48:34.496 INFO:tasks.workunit.client.0.smithi002.stdout: UnlockX 460 0.736 3.382 2024-05-14T09:48:34.496 INFO:tasks.workunit.client.0.smithi002.stdout: Flush 9899 4.035 260.688 2024-05-14T09:48:34.496 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-05-14T09:48:34.496 INFO:tasks.workunit.client.0.smithi002.stdout:Throughput 7.41233 MB/sec 1 clients 1 procs max_latency=4998.511 ms 2024-05-14T09:48:34.497 INFO:tasks.workunit.client.0.smithi002.stderr:+ dbench 10 2024-05-14T09:48:34.497 INFO:tasks.workunit.client.0.smithi002.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-14T09:48:34.497 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-05-14T09:48:34.497 INFO:tasks.workunit.client.0.smithi002.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-14T09:48:34.497 INFO:tasks.workunit.client.0.smithi002.stdout:8 of 10 processes prepared for launch 0 sec 2024-05-14T09:48:34.597 INFO:tasks.workunit.client.0.smithi002.stdout:10 of 10 processes prepared for launch 0 sec 2024-05-14T09:48:34.597 INFO:tasks.workunit.client.0.smithi002.stdout:releasing clients 2024-05-14T09:48:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:34 smithi002 ceph-mon[31557]: pgmap v833: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-05-14T09:48:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:34 smithi029 ceph-mon[36249]: pgmap v833: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-05-14T09:48:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:34 smithi026 ceph-mon[36056]: pgmap v833: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-05-14T09:48:35.597 INFO:tasks.workunit.client.0.smithi002.stdout: 10 338 122.38 MB/sec warmup 1 sec latency 56.894 ms 2024-05-14T09:48:36.597 INFO:tasks.workunit.client.0.smithi002.stdout: 10 687 102.30 MB/sec warmup 2 sec latency 462.501 ms 2024-05-14T09:48:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:36 smithi002 ceph-mon[31557]: pgmap v834: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-05-14T09:48:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:36 smithi029 ceph-mon[36249]: pgmap v834: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-05-14T09:48:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:36 smithi026 ceph-mon[36056]: pgmap v834: 129 pgs: 129 active+clean; 182 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-05-14T09:48:37.597 INFO:tasks.workunit.client.0.smithi002.stdout: 10 1004 74.04 MB/sec warmup 3 sec latency 130.402 ms 2024-05-14T09:48:38.597 INFO:tasks.workunit.client.0.smithi002.stdout: 10 1554 64.31 MB/sec warmup 4 sec latency 90.987 ms 2024-05-14T09:48:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:38 smithi002 ceph-mon[31557]: pgmap v835: 129 pgs: 129 active+clean; 178 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-05-14T09:48:38.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:38 smithi029 ceph-mon[36249]: pgmap v835: 129 pgs: 129 active+clean; 178 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-05-14T09:48:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:38 smithi026 ceph-mon[36056]: pgmap v835: 129 pgs: 129 active+clean; 178 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-05-14T09:48:39.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 2241 54.68 MB/sec warmup 5 sec latency 102.566 ms 2024-05-14T09:48:40.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 2777 53.69 MB/sec warmup 6 sec latency 59.734 ms 2024-05-14T09:48:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:40 smithi002 ceph-mon[31557]: pgmap v836: 129 pgs: 129 active+clean; 180 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-14T09:48:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:40 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:48:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:40 smithi029 ceph-mon[36249]: pgmap v836: 129 pgs: 129 active+clean; 180 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-14T09:48:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:40 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:48:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:40 smithi026 ceph-mon[36056]: pgmap v836: 129 pgs: 129 active+clean; 180 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-14T09:48:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:40 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:48:41.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 3426 53.00 MB/sec warmup 7 sec latency 64.905 ms 2024-05-14T09:48:42.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 3695 48.35 MB/sec warmup 8 sec latency 116.044 ms 2024-05-14T09:48:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.685+0000 7f39a50af640 1 -- 172.21.15.2:0/1960968754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39a0108a90 msgr2=0x7f39a010aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.685+0000 7f39a50af640 1 --2- 172.21.15.2:0/1960968754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39a0108a90 0x7f39a010aeb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3998009ca0 tx=0x7f399802a8c0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.685+0000 7f39a50af640 1 -- 172.21.15.2:0/1960968754 shutdown_connections 2024-05-14T09:48:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.685+0000 7f39a50af640 1 --2- 172.21.15.2:0/1960968754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39a0108a90 0x7f39a010aeb0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.685+0000 7f39a50af640 1 --2- 172.21.15.2:0/1960968754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a0103d90 0x7f39a01041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.685+0000 7f39a50af640 1 --2- 172.21.15.2:0/1960968754 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39a0102b50 0x7f39a0103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.685+0000 7f39a50af640 1 -- 172.21.15.2:0/1960968754 >> 172.21.15.2:0/1960968754 conn(0x7f39a00fdd40 msgr2=0x7f39a01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.685+0000 7f39a50af640 1 -- 172.21.15.2:0/1960968754 shutdown_connections 2024-05-14T09:48:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.685+0000 7f39a50af640 1 -- 172.21.15.2:0/1960968754 wait complete. 2024-05-14T09:48:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.686+0000 7f39a50af640 1 Processor -- start 2024-05-14T09:48:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.686+0000 7f39a50af640 1 -- start start 2024-05-14T09:48:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.686+0000 7f39a50af640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39a0102b50 0x7f39a006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.686+0000 7f39a50af640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39a0103d90 0x7f39a019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.686+0000 7f39a50af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a0108a90 0x7f39a01a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f39a50af640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f39a010e6b0 con 0x7f39a0102b50 2024-05-14T09:48:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f39a50af640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f39a010e530 con 0x7f39a0108a90 2024-05-14T09:48:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f39a50af640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f39a010e830 con 0x7f39a0103d90 2024-05-14T09:48:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f399ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39a0102b50 0x7f39a006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f399ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39a0102b50 0x7f39a006b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43848/0 (socket says 172.21.15.2:43848) 2024-05-14T09:48:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f399ed9d640 1 -- 172.21.15.2:0/4013441451 learned_addr learned my addr 172.21.15.2:0/4013441451 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f3997fff640 1 --2- 172.21.15.2:0/4013441451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39a0103d90 0x7f39a019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f399f59e640 1 --2- 172.21.15.2:0/4013441451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a0108a90 0x7f39a01a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f3997fff640 1 -- 172.21.15.2:0/4013441451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a0108a90 msgr2=0x7f39a01a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f3997fff640 1 --2- 172.21.15.2:0/4013441451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a0108a90 0x7f39a01a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f3997fff640 1 -- 172.21.15.2:0/4013441451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39a0102b50 msgr2=0x7f39a006b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f3997fff640 1 --2- 172.21.15.2:0/4013441451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39a0102b50 0x7f39a006b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f3997fff640 1 -- 172.21.15.2:0/4013441451 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a01a1db0 con 0x7f39a0103d90 2024-05-14T09:48:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f399ed9d640 1 --2- 172.21.15.2:0/4013441451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39a0102b50 0x7f39a006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:48:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f3997fff640 1 --2- 172.21.15.2:0/4013441451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39a0103d90 0x7f39a019f040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f398800e9f0 tx=0x7f398800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f399cd99640 1 -- 172.21.15.2:0/4013441451 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398800cd80 con 0x7f39a0103d90 2024-05-14T09:48:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f399cd99640 1 -- 172.21.15.2:0/4013441451 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39880035f0 con 0x7f39a0103d90 2024-05-14T09:48:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f39a50af640 1 -- 172.21.15.2:0/4013441451 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a01a20a0 con 0x7f39a0103d90 2024-05-14T09:48:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.687+0000 7f399cd99640 1 -- 172.21.15.2:0/4013441451 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988010670 con 0x7f39a0103d90 2024-05-14T09:48:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.688+0000 7f39a50af640 1 -- 172.21.15.2:0/4013441451 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a01a2640 con 0x7f39a0103d90 2024-05-14T09:48:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.690+0000 7f39abf66740 1 Processor -- start 2024-05-14T09:48:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.690+0000 7f399cd99640 1 -- 172.21.15.2:0/4013441451 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f39880037a0 con 0x7f39a0103d90 2024-05-14T09:48:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.690+0000 7f39abf66740 1 -- start start 2024-05-14T09:48:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f39abf66740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3f41b0 0x55e88c795a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f39abf66740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3cb2f0 0x55e88c795fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f39abf66740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c79cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f39abf66740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e88c6fa5d0 con 0x55e88c3cb2f0 2024-05-14T09:48:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f39abf66740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e88c6b3890 con 0x55e88c3f41b0 2024-05-14T09:48:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f39abf66740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e88c6f6590 con 0x55e88c3d34d0 2024-05-14T09:48:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f399f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3f41b0 0x55e88c795a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f399ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3cb2f0 0x55e88c795fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f399f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3f41b0 0x55e88c795a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:49070/0 (socket says 172.21.15.2:49070) 2024-05-14T09:48:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f399ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3cb2f0 0x55e88c795fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43860/0 (socket says 172.21.15.2:43860) 2024-05-14T09:48:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f399f59e640 1 -- 172.21.15.2:0/4136472847 learned_addr learned my addr 172.21.15.2:0/4136472847 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f3997fff640 1 --2- 172.21.15.2:0/4136472847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c79cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f399f59e640 1 -- 172.21.15.2:0/4136472847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 msgr2=0x55e88c79cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f399f59e640 1 --2- 172.21.15.2:0/4136472847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c79cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f399f59e640 1 -- 172.21.15.2:0/4136472847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3cb2f0 msgr2=0x55e88c795fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f399f59e640 1 --2- 172.21.15.2:0/4136472847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3cb2f0 0x55e88c795fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f399f59e640 1 -- 172.21.15.2:0/4136472847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88c40c210 con 0x55e88c3f41b0 2024-05-14T09:48:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.691+0000 7f399f59e640 1 --2- 172.21.15.2:0/4136472847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3f41b0 0x55e88c795a60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3998035000 tx=0x7f399802f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=217150a52706644c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.692+0000 7f399cd99640 1 --2- 172.21.15.2:0/4013441451 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f398c0b2920 0x7f398c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.692+0000 7f39957fa640 1 -- 172.21.15.2:0/4136472847 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998031650 con 0x55e88c3f41b0 2024-05-14T09:48:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.692+0000 7f399ed9d640 1 --2- 172.21.15.2:0/4013441451 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f398c0b2920 0x7f398c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.692+0000 7f39957fa640 1 -- 172.21.15.2:0/4136472847 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3998036030 con 0x55e88c3f41b0 2024-05-14T09:48:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.692+0000 7f39957fa640 1 -- 172.21.15.2:0/4136472847 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998040e60 con 0x55e88c3f41b0 2024-05-14T09:48:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.692+0000 7f399cd99640 1 -- 172.21.15.2:0/4013441451 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f39880e2810 con 0x7f39a0103d90 2024-05-14T09:48:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.693+0000 7f39abf66740 1 -- 172.21.15.2:0/4136472847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3f41b0 msgr2=0x55e88c795a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.693+0000 7f39abf66740 1 --2- 172.21.15.2:0/4136472847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3f41b0 0x55e88c795a60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3998035000 tx=0x7f399802f3a0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.693+0000 7f399ed9d640 1 --2- 172.21.15.2:0/4013441451 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f398c0b2920 0x7f398c0b4e30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55e88c796e40 tx=0x7f3990005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.693+0000 7f39abf66740 1 -- 172.21.15.2:0/4136472847 shutdown_connections 2024-05-14T09:48:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.693+0000 7f39abf66740 1 --2- 172.21.15.2:0/4136472847 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c79cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.693+0000 7f39abf66740 1 --2- 172.21.15.2:0/4136472847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3cb2f0 0x55e88c795fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.693+0000 7f39abf66740 1 --2- 172.21.15.2:0/4136472847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3f41b0 0x55e88c795a60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.693+0000 7f39abf66740 1 -- 172.21.15.2:0/4136472847 >> 172.21.15.2:0/4136472847 conn(0x55e88c3e8300 msgr2=0x55e88c7913a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.693+0000 7f39abf66740 1 -- 172.21.15.2:0/4136472847 shutdown_connections 2024-05-14T09:48:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.693+0000 7f39abf66740 1 -- 172.21.15.2:0/4136472847 wait complete. 2024-05-14T09:48:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.693+0000 7f39abf66740 1 -- start start 2024-05-14T09:48:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f39abf66740 1 Processor -- start 2024-05-14T09:48:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f39abf66740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3cb2f0 0x55e88c79f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f39abf66740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c79fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f39abf66740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3f41b0 0x55e88c7a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f39abf66740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e88c79d750 con 0x55e88c3cb2f0 2024-05-14T09:48:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f39abf66740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e88c6b7370 con 0x55e88c3f41b0 2024-05-14T09:48:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f39abf66740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e88c79d8d0 con 0x55e88c3d34d0 2024-05-14T09:48:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f39abf66740 10 client.0 _ll_register_callbacks cb 0x55e88c40ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:48:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f399ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3cb2f0 0x55e88c79f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f399ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3cb2f0 0x55e88c79f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43870/0 (socket says 172.21.15.2:43870) 2024-05-14T09:48:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f399ed9d640 1 -- 172.21.15.2:0/2761707127 learned_addr learned my addr 172.21.15.2:0/2761707127 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f3997fff640 1 --2- 172.21.15.2:0/2761707127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c79fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f399f59e640 1 --2- 172.21.15.2:0/2761707127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3f41b0 0x55e88c7a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f399ed9d640 1 -- 172.21.15.2:0/2761707127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 msgr2=0x55e88c79fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f399ed9d640 1 --2- 172.21.15.2:0/2761707127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c79fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f399ed9d640 1 -- 172.21.15.2:0/2761707127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3f41b0 msgr2=0x55e88c7a69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f399ed9d640 1 --2- 172.21.15.2:0/2761707127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3f41b0 0x55e88c7a69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.694+0000 7f399ed9d640 1 -- 172.21.15.2:0/2761707127 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88c4d8620 con 0x55e88c3cb2f0 2024-05-14T09:48:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f399ed9d640 1 --2- 172.21.15.2:0/2761707127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3cb2f0 0x55e88c79f550 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f399000fb90 tx=0x7f39900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f3f7d4a6581a8a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f39867fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:48:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f39867fc640 1 -- 172.21.15.2:0/2761707127 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399000fd40 con 0x55e88c3cb2f0 2024-05-14T09:48:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f39abf66740 1 -- 172.21.15.2:0/2761707127 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3998036030 con 0x55e88c3cb2f0 2024-05-14T09:48:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f39867fc640 1 -- 172.21.15.2:0/2761707127 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f3990018030 con 0x55e88c3cb2f0 2024-05-14T09:48:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f39867fc640 1 -- 172.21.15.2:0/2761707127 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f3990034050 con 0x55e88c3cb2f0 2024-05-14T09:48:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f39abf66740 10 client.16860 fetch_fsmap learned FSMap version 51 2024-05-14T09:48:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f39abf66740 1 -- 172.21.15.2:0/2761707127 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55e88c7a9a40 con 0x55e88c3cb2f0 2024-05-14T09:48:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f39867fc640 1 -- 172.21.15.2:0/2761707127 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3998036030 con 0x55e88c3cb2f0 2024-05-14T09:48:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f39867fc640 1 -- 172.21.15.2:0/2761707127 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f3990029970 con 0x55e88c3cb2f0 2024-05-14T09:48:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f39abf66740 10 client.16860 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T09:48:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f39abf66740 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:48:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f39abf66740 20 client.16860 populate_metadata read hostname 'smithi002' 2024-05-14T09:48:42.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f39abf66740 1 --2- 172.21.15.2:0/2761707127 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e88c3b2d80 0x55e88c7ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f39abf66740 4 client.16860 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:48:42.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.695+0000 7f39abf66740 1 -- 172.21.15.2:0/2761707127 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e88c7081e0 con 0x55e88c3b2d80 2024-05-14T09:48:42.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.696+0000 7f3997fff640 1 --2- 172.21.15.2:0/2761707127 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e88c3b2d80 0x55e88c7ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.696+0000 7f39867fc640 10 client.16860 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:42.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.696+0000 7f3997fff640 1 --2- 172.21.15.2:0/2761707127 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e88c3b2d80 0x55e88c7ad150 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=888d0ac4a54a01e9 server_cookie=19fd5dff2a5746ab in_seq=0 out_seq=0 2024-05-14T09:48:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.699+0000 7f39867fc640 1 -- 172.21.15.2:0/2761707127 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e88c7081e0 con 0x55e88c3b2d80 2024-05-14T09:48:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.699+0000 7f39867fc640 10 client.16860 handle_command_reply: tid=0 2024-05-14T09:48:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.699+0000 7f39abf66740 1 client.16860 shutdown 2024-05-14T09:48:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.699+0000 7f39abf66740 1 -- 172.21.15.2:0/2761707127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3cb2f0 msgr2=0x55e88c79f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.699+0000 7f39abf66740 1 --2- 172.21.15.2:0/2761707127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3cb2f0 0x55e88c79f550 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f399000fb90 tx=0x7f39900046c0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.699+0000 7f39abf66740 1 -- 172.21.15.2:0/2761707127 shutdown_connections 2024-05-14T09:48:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.699+0000 7f39abf66740 1 --2- 172.21.15.2:0/2761707127 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e88c3b2d80 0x55e88c7ad150 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.699+0000 7f39abf66740 1 --2- 172.21.15.2:0/2761707127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3f41b0 0x55e88c7a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.699+0000 7f39abf66740 1 --2- 172.21.15.2:0/2761707127 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c79fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.699+0000 7f39867fc640 0 client.16860 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.699+0000 7f39abf66740 1 --2- 172.21.15.2:0/2761707127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3cb2f0 0x55e88c79f550 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.699+0000 7f39abf66740 1 -- 172.21.15.2:0/2761707127 >> 172.21.15.2:0/2761707127 conn(0x55e88c3e8300 msgr2=0x55e88c7903a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.699+0000 7f39abf66740 1 -- 172.21.15.2:0/2761707127 shutdown_connections 2024-05-14T09:48:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.699+0000 7f39abf66740 1 -- 172.21.15.2:0/2761707127 wait complete. 2024-05-14T09:48:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.700+0000 7f39abf66740 20 client.16860 trim_cache size 0 max 16384 2024-05-14T09:48:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.716+0000 7f39abf66740 1 Processor -- start 2024-05-14T09:48:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f39abf66740 1 -- start start 2024-05-14T09:48:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f39abf66740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3cb2f0 0x55e88c79f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f39abf66740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c7a6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f39abf66740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3f41b0 0x55e88c7a6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f39abf66740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01ad320 con 0x55e88c3f41b0 2024-05-14T09:48:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f39abf66740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01aabc0 con 0x55e88c3cb2f0 2024-05-14T09:48:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f39abf66740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01a8a10 con 0x55e88c3d34d0 2024-05-14T09:48:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f3997fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c7a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f399f59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3f41b0 0x55e88c7a6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f3997fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c7a6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:42856/0 (socket says 172.21.15.2:42856) 2024-05-14T09:48:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f399f59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3f41b0 0x55e88c7a6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43872/0 (socket says 172.21.15.2:43872) 2024-05-14T09:48:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f3997fff640 1 -- 172.21.15.2:0/2086328934 learned_addr learned my addr 172.21.15.2:0/2086328934 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f399ed9d640 1 --2- 172.21.15.2:0/2086328934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3cb2f0 0x55e88c79f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f3997fff640 1 -- 172.21.15.2:0/2086328934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3cb2f0 msgr2=0x55e88c79f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f3997fff640 1 --2- 172.21.15.2:0/2086328934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3cb2f0 0x55e88c79f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f3997fff640 1 -- 172.21.15.2:0/2086328934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3f41b0 msgr2=0x55e88c7a6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f3997fff640 1 --2- 172.21.15.2:0/2086328934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3f41b0 0x55e88c7a6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.717+0000 7f3997fff640 1 -- 172.21.15.2:0/2086328934 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88c791d00 con 0x55e88c3d34d0 2024-05-14T09:48:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.718+0000 7f3997fff640 1 --2- 172.21.15.2:0/2086328934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c7a6980 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f398800e9c0 tx=0x7f39880dfdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f4e6999ecd8bdf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.718+0000 7f3994ff9640 1 -- 172.21.15.2:0/2086328934 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39880aa110 con 0x55e88c3d34d0 2024-05-14T09:48:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.718+0000 7f3994ff9640 1 -- 172.21.15.2:0/2086328934 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39880a9a10 con 0x55e88c3d34d0 2024-05-14T09:48:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.718+0000 7f3994ff9640 1 -- 172.21.15.2:0/2086328934 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988009310 con 0x55e88c3d34d0 2024-05-14T09:48:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.719+0000 7f39abf66740 1 -- 172.21.15.2:0/2086328934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 msgr2=0x55e88c7a6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.719+0000 7f39abf66740 1 --2- 172.21.15.2:0/2086328934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c7a6980 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f398800e9c0 tx=0x7f39880dfdc0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.719+0000 7f39abf66740 1 -- 172.21.15.2:0/2086328934 shutdown_connections 2024-05-14T09:48:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.719+0000 7f39abf66740 1 --2- 172.21.15.2:0/2086328934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3f41b0 0x55e88c7a6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.719+0000 7f39abf66740 1 --2- 172.21.15.2:0/2086328934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c7a6980 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.719+0000 7f39abf66740 1 --2- 172.21.15.2:0/2086328934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3cb2f0 0x55e88c79f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.719+0000 7f39abf66740 1 -- 172.21.15.2:0/2086328934 >> 172.21.15.2:0/2086328934 conn(0x55e88c3e8300 msgr2=0x55e88c79f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.719+0000 7f39abf66740 1 -- 172.21.15.2:0/2086328934 shutdown_connections 2024-05-14T09:48:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.720+0000 7f39abf66740 1 -- 172.21.15.2:0/2086328934 wait complete. 2024-05-14T09:48:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.720+0000 7f39abf66740 1 -- start start 2024-05-14T09:48:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.720+0000 7f39abf66740 1 Processor -- start 2024-05-14T09:48:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.720+0000 7f39abf66740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3cb2f0 0x55e88c7a4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.720+0000 7f39abf66740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c7a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.720+0000 7f39abf66740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3f41b0 0x55e88c7a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.720+0000 7f39abf66740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01a9080 con 0x55e88c3f41b0 2024-05-14T09:48:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.720+0000 7f39abf66740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01aa4f0 con 0x55e88c3cb2f0 2024-05-14T09:48:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.720+0000 7f39abf66740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01a9750 con 0x55e88c3d34d0 2024-05-14T09:48:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.720+0000 7f39abf66740 10 client.0 _ll_register_callbacks cb 0x7f3990009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:48:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.721+0000 7f399ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3cb2f0 0x55e88c7a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.721+0000 7f399f59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3f41b0 0x55e88c7a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.721+0000 7f3997fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c7a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.721+0000 7f399ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3cb2f0 0x55e88c7a4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:49094/0 (socket says 172.21.15.2:49094) 2024-05-14T09:48:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.721+0000 7f399ed9d640 1 -- 172.21.15.2:0/2342159398 learned_addr learned my addr 172.21.15.2:0/2342159398 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.721+0000 7f399ed9d640 1 -- 172.21.15.2:0/2342159398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 msgr2=0x55e88c7a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.721+0000 7f399ed9d640 1 --2- 172.21.15.2:0/2342159398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c7a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.721+0000 7f399ed9d640 1 -- 172.21.15.2:0/2342159398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3f41b0 msgr2=0x55e88c7a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.721+0000 7f399ed9d640 1 --2- 172.21.15.2:0/2342159398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3f41b0 0x55e88c7a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.721+0000 7f399ed9d640 1 -- 172.21.15.2:0/2342159398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88c7a5b90 con 0x55e88c3cb2f0 2024-05-14T09:48:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.721+0000 7f399ed9d640 1 --2- 172.21.15.2:0/2342159398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3cb2f0 0x55e88c7a4160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3990012ec0 tx=0x7f399000fac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2fb1493d81ec5a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.721+0000 7f39867fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:48:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.721+0000 7f39867fc640 1 -- 172.21.15.2:0/2342159398 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39900052d0 con 0x55e88c3cb2f0 2024-05-14T09:48:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.721+0000 7f39867fc640 1 -- 172.21.15.2:0/2342159398 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f3990005470 con 0x55e88c3cb2f0 2024-05-14T09:48:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.721+0000 7f39abf66740 1 -- 172.21.15.2:0/2342159398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39880a9a10 con 0x55e88c3cb2f0 2024-05-14T09:48:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.722+0000 7f39867fc640 1 -- 172.21.15.2:0/2342159398 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f3990046050 con 0x55e88c3cb2f0 2024-05-14T09:48:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.722+0000 7f39abf66740 10 client.26510 fetch_fsmap learned FSMap version 51 2024-05-14T09:48:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.722+0000 7f39abf66740 1 -- 172.21.15.2:0/2342159398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55e88c7a6e90 con 0x55e88c3cb2f0 2024-05-14T09:48:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.722+0000 7f39867fc640 1 -- 172.21.15.2:0/2342159398 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39880a9a10 con 0x55e88c3cb2f0 2024-05-14T09:48:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.722+0000 7f39867fc640 1 -- 172.21.15.2:0/2342159398 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f3990004a20 con 0x55e88c3cb2f0 2024-05-14T09:48:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.722+0000 7f39abf66740 10 client.26510 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T09:48:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.722+0000 7f39abf66740 10 client.26510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:48:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.722+0000 7f39abf66740 20 client.26510 populate_metadata read hostname 'smithi002' 2024-05-14T09:48:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.722+0000 7f39abf66740 1 --2- 172.21.15.2:0/2342159398 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e88c3881c0 0x55e88c8178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.722+0000 7f39abf66740 4 client.26510 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:48:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.722+0000 7f39abf66740 1 -- 172.21.15.2:0/2342159398 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e88c702050 con 0x55e88c3881c0 2024-05-14T09:48:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.722+0000 7f3997fff640 1 --2- 172.21.15.2:0/2342159398 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e88c3881c0 0x55e88c8178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.723+0000 7f39867fc640 10 client.26510 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:42.723+0000 7f3997fff640 1 --2- 172.21.15.2:0/2342159398 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e88c3881c0 0x55e88c8178b0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25d52678824d7270 server_cookie=9cb46e2ba08ac901 in_seq=0 out_seq=0 2024-05-14T09:48:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:42 smithi002 ceph-mon[31557]: pgmap v837: 129 pgs: 129 active+clean; 179 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-05-14T09:48:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:42 smithi029 ceph-mon[36249]: pgmap v837: 129 pgs: 129 active+clean; 179 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-05-14T09:48:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:42 smithi026 ceph-mon[36056]: pgmap v837: 129 pgs: 129 active+clean; 179 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-05-14T09:48:43.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 3756 43.35 MB/sec warmup 9 sec latency 531.704 ms 2024-05-14T09:48:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.796+0000 7f39867fc640 1 -- 172.21.15.2:0/2342159398 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e88c702050 con 0x55e88c3881c0 2024-05-14T09:48:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.796+0000 7f39867fc640 10 client.26510 handle_command_reply: tid=0 2024-05-14T09:48:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.796+0000 7f39abf66740 1 client.26510 shutdown 2024-05-14T09:48:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.796+0000 7f39abf66740 1 -- 172.21.15.2:0/2342159398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3cb2f0 msgr2=0x55e88c7a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.796+0000 7f39abf66740 1 --2- 172.21.15.2:0/2342159398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3cb2f0 0x55e88c7a4160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3990012ec0 tx=0x7f399000fac0 comp rx=0 tx=0).stop 2024-05-14T09:48:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.796+0000 7f39abf66740 1 -- 172.21.15.2:0/2342159398 shutdown_connections 2024-05-14T09:48:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.796+0000 7f39abf66740 1 --2- 172.21.15.2:0/2342159398 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e88c3881c0 0x55e88c8178b0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.796+0000 7f39abf66740 1 --2- 172.21.15.2:0/2342159398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e88c3f41b0 0x55e88c7a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.796+0000 7f39abf66740 1 --2- 172.21.15.2:0/2342159398 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e88c3d34d0 0x55e88c7a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.796+0000 7f39abf66740 1 --2- 172.21.15.2:0/2342159398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e88c3cb2f0 0x55e88c7a4160 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.796+0000 7f39867fc640 0 client.26510 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.796+0000 7f39abf66740 1 -- 172.21.15.2:0/2342159398 >> 172.21.15.2:0/2342159398 conn(0x55e88c3e8300 msgr2=0x55e88c78f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.796+0000 7f39abf66740 1 -- 172.21.15.2:0/2342159398 shutdown_connections 2024-05-14T09:48:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.796+0000 7f39abf66740 1 -- 172.21.15.2:0/2342159398 wait complete. 2024-05-14T09:48:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.796+0000 7f39abf66740 20 client.26510 trim_cache size 0 max 16384 2024-05-14T09:48:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:48:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:48:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:48:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:48:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.797+0000 7f39a50af640 1 -- 172.21.15.2:0/4013441451 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f398c0b2920 msgr2=0x7f398c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.797+0000 7f39a50af640 1 --2- 172.21.15.2:0/4013441451 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f398c0b2920 0x7f398c0b4e30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55e88c796e40 tx=0x7f3990005f90 comp rx=0 tx=0).stop 2024-05-14T09:48:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.797+0000 7f39a50af640 1 -- 172.21.15.2:0/4013441451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39a0103d90 msgr2=0x7f39a019f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.797+0000 7f39a50af640 1 --2- 172.21.15.2:0/4013441451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39a0103d90 0x7f39a019f040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f398800e9f0 tx=0x7f398800eeb0 comp rx=0 tx=0).stop 2024-05-14T09:48:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.797+0000 7f39a50af640 1 -- 172.21.15.2:0/4013441451 shutdown_connections 2024-05-14T09:48:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.797+0000 7f39a50af640 1 --2- 172.21.15.2:0/4013441451 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f398c0b2920 0x7f398c0b4e30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.797+0000 7f39a50af640 1 --2- 172.21.15.2:0/4013441451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39a0108a90 0x7f39a01a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.797+0000 7f39a50af640 1 --2- 172.21.15.2:0/4013441451 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39a0103d90 0x7f39a019f040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.797+0000 7f39a50af640 1 --2- 172.21.15.2:0/4013441451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39a0102b50 0x7f39a006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.797+0000 7f39a50af640 1 -- 172.21.15.2:0/4013441451 >> 172.21.15.2:0/4013441451 conn(0x7f39a00fdd40 msgr2=0x7f39a0100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.797+0000 7f39a50af640 1 -- 172.21.15.2:0/4013441451 shutdown_connections 2024-05-14T09:48:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:43.797+0000 7f39a50af640 1 -- 172.21.15.2:0/4013441451 wait complete. 2024-05-14T09:48:44.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 3881 39.41 MB/sec warmup 10 sec latency 430.182 ms 2024-05-14T09:48:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:44 smithi002 ceph-mon[31557]: pgmap v838: 129 pgs: 129 active+clean; 178 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-05-14T09:48:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:44 smithi029 ceph-mon[36249]: pgmap v838: 129 pgs: 129 active+clean; 178 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-05-14T09:48:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:44 smithi026 ceph-mon[36056]: pgmap v838: 129 pgs: 129 active+clean; 178 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-05-14T09:48:45.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 4061 35.93 MB/sec warmup 11 sec latency 122.467 ms 2024-05-14T09:48:46.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 4304 33.67 MB/sec warmup 12 sec latency 164.867 ms 2024-05-14T09:48:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:46 smithi002 ceph-mon[31557]: pgmap v839: 129 pgs: 129 active+clean; 178 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-05-14T09:48:46.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:46 smithi029 ceph-mon[36249]: pgmap v839: 129 pgs: 129 active+clean; 178 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-05-14T09:48:46.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:46 smithi026 ceph-mon[36056]: pgmap v839: 129 pgs: 129 active+clean; 178 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-05-14T09:48:47.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 4681 32.19 MB/sec warmup 13 sec latency 70.433 ms 2024-05-14T09:48:48.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 5303 32.12 MB/sec warmup 14 sec latency 55.734 ms 2024-05-14T09:48:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:48 smithi002 ceph-mon[31557]: pgmap v840: 129 pgs: 129 active+clean; 181 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 2.4 MiB/s wr, 451 op/s 2024-05-14T09:48:48.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:48 smithi029 ceph-mon[36249]: pgmap v840: 129 pgs: 129 active+clean; 181 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 2.4 MiB/s wr, 451 op/s 2024-05-14T09:48:48.955 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:48 smithi026 ceph-mon[36056]: pgmap v840: 129 pgs: 129 active+clean; 181 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 2.4 MiB/s wr, 451 op/s 2024-05-14T09:48:49.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 5823 31.26 MB/sec warmup 15 sec latency 63.759 ms 2024-05-14T09:48:50.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 6288 31.85 MB/sec warmup 16 sec latency 85.494 ms 2024-05-14T09:48:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:50 smithi002 ceph-mon[31557]: pgmap v841: 129 pgs: 129 active+clean; 177 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 595 op/s 2024-05-14T09:48:50.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:50 smithi029 ceph-mon[36249]: pgmap v841: 129 pgs: 129 active+clean; 177 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 595 op/s 2024-05-14T09:48:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:50 smithi026 ceph-mon[36056]: pgmap v841: 129 pgs: 129 active+clean; 177 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 595 op/s 2024-05-14T09:48:51.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 6858 32.69 MB/sec warmup 17 sec latency 66.708 ms 2024-05-14T09:48:52.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 7130 31.89 MB/sec warmup 18 sec latency 93.750 ms 2024-05-14T09:48:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:52 smithi002 ceph-mon[31557]: pgmap v842: 129 pgs: 129 active+clean; 178 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 2.6 MiB/s wr, 545 op/s 2024-05-14T09:48:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:52 smithi029 ceph-mon[36249]: pgmap v842: 129 pgs: 129 active+clean; 178 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 2.6 MiB/s wr, 545 op/s 2024-05-14T09:48:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:52 smithi026 ceph-mon[36056]: pgmap v842: 129 pgs: 129 active+clean; 178 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 2.6 MiB/s wr, 545 op/s 2024-05-14T09:48:53.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 7322 30.42 MB/sec warmup 19 sec latency 107.032 ms 2024-05-14T09:48:54.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 7471 29.00 MB/sec warmup 20 sec latency 362.047 ms 2024-05-14T09:48:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:54 smithi002 ceph-mon[31557]: pgmap v843: 129 pgs: 129 active+clean; 187 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 602 op/s 2024-05-14T09:48:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:54 smithi029 ceph-mon[36249]: pgmap v843: 129 pgs: 129 active+clean; 187 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 602 op/s 2024-05-14T09:48:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:54 smithi026 ceph-mon[36056]: pgmap v843: 129 pgs: 129 active+clean; 187 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 602 op/s 2024-05-14T09:48:55.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 7552 27.65 MB/sec warmup 21 sec latency 1042.679 ms 2024-05-14T09:48:55.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:55 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:48:55.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:55 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:48:55.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:55 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:48:55.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:55 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:48:55.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:55 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:48:55.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:55 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:48:56.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 7873 26.85 MB/sec warmup 22 sec latency 1054.085 ms 2024-05-14T09:48:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.914+0000 7f5c04dce640 1 -- 172.21.15.2:0/3201058708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c00104ea0 msgr2=0x7f5c001052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.914+0000 7f5c04dce640 1 --2- 172.21.15.2:0/3201058708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c00104ea0 0x7f5c001052b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009900 tx=0x7f5bf402a410 comp rx=0 tx=0).stop 2024-05-14T09:48:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.914+0000 7f5c04dce640 1 -- 172.21.15.2:0/3201058708 shutdown_connections 2024-05-14T09:48:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.914+0000 7f5c04dce640 1 --2- 172.21.15.2:0/3201058708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c00109ba0 0x7f5c0010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.914+0000 7f5c04dce640 1 --2- 172.21.15.2:0/3201058708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c00104ea0 0x7f5c001052b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.914+0000 7f5c04dce640 1 --2- 172.21.15.2:0/3201058708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c00103c60 0x7f5c00104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.914+0000 7f5c04dce640 1 -- 172.21.15.2:0/3201058708 >> 172.21.15.2:0/3201058708 conn(0x7f5c000fee50 msgr2=0x7f5c00101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.914+0000 7f5c04dce640 1 -- 172.21.15.2:0/3201058708 shutdown_connections 2024-05-14T09:48:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.914+0000 7f5c04dce640 1 -- 172.21.15.2:0/3201058708 wait complete. 2024-05-14T09:48:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.915+0000 7f5c04dce640 1 Processor -- start 2024-05-14T09:48:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.915+0000 7f5c04dce640 1 -- start start 2024-05-14T09:48:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.915+0000 7f5c04dce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c00103c60 0x7f5c0019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.915+0000 7f5c04dce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c00104ea0 0x7f5c0019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.916+0000 7f5c04dce640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c00109ba0 0x7f5c001a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.916+0000 7f5c04dce640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0010f760 con 0x7f5c00109ba0 2024-05-14T09:48:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.916+0000 7f5c04dce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0010f5e0 con 0x7f5c00104ea0 2024-05-14T09:48:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.916+0000 7f5c04dce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0010f8e0 con 0x7f5c00103c60 2024-05-14T09:48:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.916+0000 7f5bfe59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c00103c60 0x7f5c0019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.916+0000 7f5bfe59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c00103c60 0x7f5c0019a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:58660/0 (socket says 172.21.15.2:58660) 2024-05-14T09:48:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.916+0000 7f5bfe59c640 1 -- 172.21.15.2:0/813282584 learned_addr learned my addr 172.21.15.2:0/813282584 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.916+0000 7f5bfed9d640 1 --2- 172.21.15.2:0/813282584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c00109ba0 0x7f5c001a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.916+0000 7f5bfdd9b640 1 --2- 172.21.15.2:0/813282584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c00104ea0 0x7f5c0019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.916+0000 7f5bfe59c640 1 -- 172.21.15.2:0/813282584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c00104ea0 msgr2=0x7f5c0019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.916+0000 7f5bfe59c640 1 --2- 172.21.15.2:0/813282584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c00104ea0 0x7f5c0019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.916+0000 7f5bfe59c640 1 -- 172.21.15.2:0/813282584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c00109ba0 msgr2=0x7f5c001a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.916+0000 7f5bfe59c640 1 --2- 172.21.15.2:0/813282584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c00109ba0 0x7f5c001a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.916+0000 7f5bfe59c640 1 -- 172.21.15.2:0/813282584 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c001a1db0 con 0x7f5c00103c60 2024-05-14T09:48:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.916+0000 7f5bfed9d640 1 --2- 172.21.15.2:0/813282584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c00109ba0 0x7f5c001a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:48:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.917+0000 7f5bfe59c640 1 --2- 172.21.15.2:0/813282584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c00103c60 0x7f5c0019a270 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000e980 tx=0x7f5bf000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.917+0000 7f5bef7fe640 1 -- 172.21.15.2:0/813282584 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf000cd80 con 0x7f5c00103c60 2024-05-14T09:48:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.917+0000 7f5bef7fe640 1 -- 172.21.15.2:0/813282584 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bf00108e0 con 0x7f5c00103c60 2024-05-14T09:48:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.917+0000 7f5bef7fe640 1 -- 172.21.15.2:0/813282584 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0018990 con 0x7f5c00103c60 2024-05-14T09:48:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.917+0000 7f5c04dce640 1 -- 172.21.15.2:0/813282584 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c001a20a0 con 0x7f5c00103c60 2024-05-14T09:48:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.917+0000 7f5c04dce640 1 -- 172.21.15.2:0/813282584 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c001a25a0 con 0x7f5c00103c60 2024-05-14T09:48:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.920+0000 7f5c0bc83740 1 Processor -- start 2024-05-14T09:48:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.920+0000 7f5c0bc83740 1 -- start start 2024-05-14T09:48:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5c0bc83740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c39a0281b0 0x55c39a3c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5c0bc83740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 0x55c39a3c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5c0bc83740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c39a0074d0 0x55c39a3d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5c0bc83740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c39a32e5d0 con 0x55c399fff2f0 2024-05-14T09:48:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5c0bc83740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c39a2e7890 con 0x55c39a0281b0 2024-05-14T09:48:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5c0bc83740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c39a32a590 con 0x55c39a0074d0 2024-05-14T09:48:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5bfdd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c39a0074d0 0x55c39a3d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5bfed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c39a0281b0 0x55c39a3c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5bfed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c39a0281b0 0x55c39a3c9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:45580/0 (socket says 172.21.15.2:45580) 2024-05-14T09:48:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5bfdd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c39a0074d0 0x55c39a3d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:58668/0 (socket says 172.21.15.2:58668) 2024-05-14T09:48:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5bfed9d640 1 -- 172.21.15.2:0/3939462044 learned_addr learned my addr 172.21.15.2:0/3939462044 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:56.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:56 smithi002 ceph-mon[31557]: pgmap v844: 129 pgs: 129 active+clean; 187 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 518 op/s 2024-05-14T09:48:56.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:56 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:48:56.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:56 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:48:56.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:56 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:48:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5bfdd9b640 1 -- 172.21.15.2:0/3939462044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c39a0281b0 msgr2=0x55c39a3c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5bfdd9b640 1 --2- 172.21.15.2:0/3939462044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c39a0281b0 0x55c39a3c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5bfdd9b640 1 -- 172.21.15.2:0/3939462044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 msgr2=0x55c39a3c9fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:48:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5bfdd9b640 1 --2- 172.21.15.2:0/3939462044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 0x55c39a3c9fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5bfdd9b640 1 -- 172.21.15.2:0/3939462044 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c39a040210 con 0x55c39a0074d0 2024-05-14T09:48:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.921+0000 7f5bef7fe640 1 -- 172.21.15.2:0/813282584 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5bf001f020 con 0x7f5c00103c60 2024-05-14T09:48:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.922+0000 7f5bfdd9b640 1 --2- 172.21.15.2:0/3939462044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c39a0074d0 0x55c39a3d0ee0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5c0019b310 tx=0x7f5bf4032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce1f6a20a866b2d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.922+0000 7f5becff9640 1 -- 172.21.15.2:0/3939462044 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf403f070 con 0x55c39a0074d0 2024-05-14T09:48:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.922+0000 7f5becff9640 1 -- 172.21.15.2:0/3939462044 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bf4004040 con 0x55c39a0074d0 2024-05-14T09:48:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.922+0000 7f5becff9640 1 -- 172.21.15.2:0/3939462044 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4035070 con 0x55c39a0074d0 2024-05-14T09:48:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.922+0000 7f5bef7fe640 1 --2- 172.21.15.2:0/813282584 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5be80b2920 0x7f5be80b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.923+0000 7f5bfed9d640 1 --2- 172.21.15.2:0/813282584 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5be80b2920 0x7f5be80b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.923+0000 7f5bef7fe640 1 -- 172.21.15.2:0/813282584 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f5bf00e27e0 con 0x7f5c00103c60 2024-05-14T09:48:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.923+0000 7f5c0bc83740 1 -- 172.21.15.2:0/3939462044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c39a0074d0 msgr2=0x55c39a3d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.923+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/3939462044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c39a0074d0 0x55c39a3d0ee0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5c0019b310 tx=0x7f5bf4032b20 comp rx=0 tx=0).stop 2024-05-14T09:48:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.923+0000 7f5c0bc83740 1 -- 172.21.15.2:0/3939462044 shutdown_connections 2024-05-14T09:48:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.923+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/3939462044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c39a0074d0 0x55c39a3d0ee0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.923+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/3939462044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 0x55c39a3c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.923+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/3939462044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c39a0281b0 0x55c39a3c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.923+0000 7f5c0bc83740 1 -- 172.21.15.2:0/3939462044 >> 172.21.15.2:0/3939462044 conn(0x55c39a01c300 msgr2=0x55c39a3c53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.923+0000 7f5bfed9d640 1 --2- 172.21.15.2:0/813282584 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5be80b2920 0x7f5be80b4e30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55c39a3cac30 tx=0x7f5bf80096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.923+0000 7f5c0bc83740 1 -- 172.21.15.2:0/3939462044 shutdown_connections 2024-05-14T09:48:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.923+0000 7f5c0bc83740 1 -- 172.21.15.2:0/3939462044 wait complete. 2024-05-14T09:48:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5c0bc83740 1 -- start start 2024-05-14T09:48:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5c0bc83740 1 Processor -- start 2024-05-14T09:48:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5c0bc83740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c399fff2f0 0x55c39a3d36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5c0bc83740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c39a0074d0 0x55c39a3d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5c0bc83740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c39a0281b0 0x55c39a3dab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5c0bc83740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c39a3d18d0 con 0x55c39a0281b0 2024-05-14T09:48:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5c0bc83740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c39a2eb370 con 0x55c399fff2f0 2024-05-14T09:48:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5c0bc83740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c39a3d1a50 con 0x55c39a0074d0 2024-05-14T09:48:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5c0bc83740 10 client.0 _ll_register_callbacks cb 0x55c39a03fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:48:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5bfdd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c39a0074d0 0x55c39a3d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5bfed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c39a0281b0 0x55c39a3dab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5bfe59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c399fff2f0 0x55c39a3d36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5bfdd9b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c39a0074d0 0x55c39a3d3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:58684/0 (socket says 172.21.15.2:58684) 2024-05-14T09:48:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5bfed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c39a0281b0 0x55c39a3dab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60264/0 (socket says 172.21.15.2:60264) 2024-05-14T09:48:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5bfdd9b640 1 -- 172.21.15.2:0/1457648733 learned_addr learned my addr 172.21.15.2:0/1457648733 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.924+0000 7f5bfe59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c399fff2f0 0x55c39a3d36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:45590/0 (socket says 172.21.15.2:45590) 2024-05-14T09:48:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.925+0000 7f5bfdd9b640 1 -- 172.21.15.2:0/1457648733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c399fff2f0 msgr2=0x55c39a3d36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.925+0000 7f5bfdd9b640 1 --2- 172.21.15.2:0/1457648733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c399fff2f0 0x55c39a3d36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.925+0000 7f5bfdd9b640 1 -- 172.21.15.2:0/1457648733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c39a0281b0 msgr2=0x55c39a3dab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.925+0000 7f5bfdd9b640 1 --2- 172.21.15.2:0/1457648733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c39a0281b0 0x55c39a3dab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.925+0000 7f5bfdd9b640 1 -- 172.21.15.2:0/1457648733 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c39a10c620 con 0x55c39a0074d0 2024-05-14T09:48:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.925+0000 7f5bfed9d640 1 --2- 172.21.15.2:0/1457648733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c39a0281b0 0x55c39a3dab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:48:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.925+0000 7f5bfdd9b640 1 --2- 172.21.15.2:0/1457648733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c39a0074d0 0x55c39a3d3be0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4006010 tx=0x7f5bf40041f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cf1c54a75d97c8f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.925+0000 7f5be5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:48:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.925+0000 7f5be5ffb640 1 -- 172.21.15.2:0/1457648733 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4035070 con 0x55c39a0074d0 2024-05-14T09:48:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.925+0000 7f5be5ffb640 1 -- 172.21.15.2:0/1457648733 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f5bf40317a0 con 0x55c39a0074d0 2024-05-14T09:48:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.925+0000 7f5c0bc83740 1 -- 172.21.15.2:0/1457648733 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bf4004040 con 0x55c39a0074d0 2024-05-14T09:48:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.926+0000 7f5be5ffb640 1 -- 172.21.15.2:0/1457648733 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5bf4058050 con 0x55c39a0074d0 2024-05-14T09:48:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.926+0000 7f5c0bc83740 10 client.26509 fetch_fsmap learned FSMap version 52 2024-05-14T09:48:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.926+0000 7f5c0bc83740 1 -- 172.21.15.2:0/1457648733 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c39a3d18d0 con 0x55c39a0074d0 2024-05-14T09:48:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.926+0000 7f5be5ffb640 1 -- 172.21.15.2:0/1457648733 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bf4004040 con 0x55c39a0074d0 2024-05-14T09:48:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.926+0000 7f5be5ffb640 1 -- 172.21.15.2:0/1457648733 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f5bf40048f0 con 0x55c39a0074d0 2024-05-14T09:48:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.926+0000 7f5c0bc83740 10 client.26509 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T09:48:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.926+0000 7f5c0bc83740 10 client.26509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:48:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.926+0000 7f5c0bc83740 20 client.26509 populate_metadata read hostname 'smithi002' 2024-05-14T09:48:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.926+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/1457648733 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c399fe6d80 0x55c39a3e0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.926+0000 7f5c0bc83740 4 client.26509 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:48:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.926+0000 7f5c0bc83740 1 -- 172.21.15.2:0/1457648733 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c39a33c1e0 con 0x55c399fe6d80 2024-05-14T09:48:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.927+0000 7f5bfe59c640 1 --2- 172.21.15.2:0/1457648733 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c399fe6d80 0x55c39a3e0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.927+0000 7f5bfe59c640 1 --2- 172.21.15.2:0/1457648733 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c399fe6d80 0x55c39a3e0cf0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=989dcc10dd0c1801 server_cookie=59560076d51a69a2 in_seq=0 out_seq=0 2024-05-14T09:48:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.927+0000 7f5be5ffb640 10 client.26509 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.930+0000 7f5be5ffb640 1 -- 172.21.15.2:0/1457648733 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c39a33c1e0 con 0x55c399fe6d80 2024-05-14T09:48:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.930+0000 7f5be5ffb640 10 client.26509 handle_command_reply: tid=0 2024-05-14T09:48:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.930+0000 7f5c0bc83740 1 client.26509 shutdown 2024-05-14T09:48:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.930+0000 7f5c0bc83740 1 -- 172.21.15.2:0/1457648733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c39a0074d0 msgr2=0x55c39a3d3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.930+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/1457648733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c39a0074d0 0x55c39a3d3be0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4006010 tx=0x7f5bf40041f0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.930+0000 7f5c0bc83740 1 -- 172.21.15.2:0/1457648733 shutdown_connections 2024-05-14T09:48:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.930+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/1457648733 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c399fe6d80 0x55c39a3e0cf0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.930+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/1457648733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c39a0281b0 0x55c39a3dab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.930+0000 7f5be5ffb640 0 client.26509 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.931+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/1457648733 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c39a0074d0 0x55c39a3d3be0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.931+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/1457648733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c399fff2f0 0x55c39a3d36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.931+0000 7f5c0bc83740 1 -- 172.21.15.2:0/1457648733 >> 172.21.15.2:0/1457648733 conn(0x55c39a01c300 msgr2=0x55c39a3c43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.931+0000 7f5c0bc83740 1 -- 172.21.15.2:0/1457648733 shutdown_connections 2024-05-14T09:48:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.931+0000 7f5c0bc83740 1 -- 172.21.15.2:0/1457648733 wait complete. 2024-05-14T09:48:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.931+0000 7f5c0bc83740 20 client.26509 trim_cache size 0 max 16384 2024-05-14T09:48:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:56 smithi029 ceph-mon[36249]: pgmap v844: 129 pgs: 129 active+clean; 187 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 518 op/s 2024-05-14T09:48:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:56 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:48:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:56 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:48:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:56 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:48:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:56 smithi026 ceph-mon[36056]: pgmap v844: 129 pgs: 129 active+clean; 187 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 518 op/s 2024-05-14T09:48:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:56 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:48:56.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:56 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:48:56.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:56 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:48:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.955+0000 7f5c0bc83740 1 Processor -- start 2024-05-14T09:48:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.955+0000 7f5c0bc83740 1 -- start start 2024-05-14T09:48:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.955+0000 7f5c0bc83740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c399fe6d80 0x55c39a3ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.955+0000 7f5c0bc83740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 0x55c39a3d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.955+0000 7f5c0bc83740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c39a0074d0 0x55c39a3daa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.955+0000 7f5c0bc83740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5c001af5b0 con 0x55c399fff2f0 2024-05-14T09:48:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.955+0000 7f5c0bc83740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5c001ace20 con 0x55c39a0074d0 2024-05-14T09:48:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.955+0000 7f5c0bc83740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5c001aac70 con 0x55c399fe6d80 2024-05-14T09:48:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.956+0000 7f5bfdd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 0x55c39a3d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.956+0000 7f5bfdd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 0x55c39a3d2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60272/0 (socket says 172.21.15.2:60272) 2024-05-14T09:48:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.956+0000 7f5bfdd9b640 1 -- 172.21.15.2:0/2076987034 learned_addr learned my addr 172.21.15.2:0/2076987034 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.956+0000 7f5bfe59c640 1 --2- 172.21.15.2:0/2076987034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c399fe6d80 0x55c39a3ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.956+0000 7f5bfdd9b640 1 -- 172.21.15.2:0/2076987034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c399fe6d80 msgr2=0x55c39a3ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.956+0000 7f5bfdd9b640 1 --2- 172.21.15.2:0/2076987034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c399fe6d80 0x55c39a3ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.956+0000 7f5bfdd9b640 1 -- 172.21.15.2:0/2076987034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c39a0074d0 msgr2=0x55c39a3daa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.956+0000 7f5bfdd9b640 1 --2- 172.21.15.2:0/2076987034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c39a0074d0 0x55c39a3daa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.957+0000 7f5bfdd9b640 1 -- 172.21.15.2:0/2076987034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c39a3c5d00 con 0x55c399fff2f0 2024-05-14T09:48:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.957+0000 7f5bfe59c640 1 --2- 172.21.15.2:0/2076987034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c399fe6d80 0x55c39a3ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:48:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.957+0000 7f5bfdd9b640 1 --2- 172.21.15.2:0/2076987034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 0x55c39a3d2530 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4004d10 tx=0x7f5bf4005140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7c117bf8d356f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.957+0000 7f5be7fff640 1 -- 172.21.15.2:0/2076987034 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4035070 con 0x55c399fff2f0 2024-05-14T09:48:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.957+0000 7f5be7fff640 1 -- 172.21.15.2:0/2076987034 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bf40048c0 con 0x55c399fff2f0 2024-05-14T09:48:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.957+0000 7f5be7fff640 1 -- 172.21.15.2:0/2076987034 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf403f040 con 0x55c399fff2f0 2024-05-14T09:48:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.958+0000 7f5c0bc83740 1 -- 172.21.15.2:0/2076987034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 msgr2=0x55c39a3d2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.958+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/2076987034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 0x55c39a3d2530 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4004d10 tx=0x7f5bf4005140 comp rx=0 tx=0).stop 2024-05-14T09:48:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.958+0000 7f5c0bc83740 1 -- 172.21.15.2:0/2076987034 shutdown_connections 2024-05-14T09:48:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.958+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/2076987034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c39a0074d0 0x55c39a3daa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.958+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/2076987034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 0x55c39a3d2530 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.958+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/2076987034 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c399fe6d80 0x55c39a3ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.958+0000 7f5c0bc83740 1 -- 172.21.15.2:0/2076987034 >> 172.21.15.2:0/2076987034 conn(0x55c39a01c300 msgr2=0x55c39a3d1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.958+0000 7f5c0bc83740 1 -- 172.21.15.2:0/2076987034 shutdown_connections 2024-05-14T09:48:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.958+0000 7f5c0bc83740 1 -- 172.21.15.2:0/2076987034 wait complete. 2024-05-14T09:48:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.959+0000 7f5c0bc83740 1 -- start start 2024-05-14T09:48:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.959+0000 7f5c0bc83740 1 Processor -- start 2024-05-14T09:48:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.959+0000 7f5c0bc83740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c399fe6d80 0x55c39a3d8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.959+0000 7f5c0bc83740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 0x55c39a3d8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.959+0000 7f5c0bc83740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c39a0074d0 0x55c39a447030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.959+0000 7f5c0bc83740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5c001ab2e0 con 0x55c399fff2f0 2024-05-14T09:48:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.959+0000 7f5c0bc83740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5c001ac750 con 0x55c39a0074d0 2024-05-14T09:48:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.959+0000 7f5c0bc83740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f5c001ab9b0 con 0x55c399fe6d80 2024-05-14T09:48:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.959+0000 7f5c0bc83740 10 client.0 _ll_register_callbacks cb 0x7f5bf8009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:48:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.959+0000 7f5bfdd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 0x55c39a3d8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.959+0000 7f5bfe59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c399fe6d80 0x55c39a3d8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.959+0000 7f5bfdd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 0x55c39a3d8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60280/0 (socket says 172.21.15.2:60280) 2024-05-14T09:48:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.959+0000 7f5bfdd9b640 1 -- 172.21.15.2:0/997341250 learned_addr learned my addr 172.21.15.2:0/997341250 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:48:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.959+0000 7f5bfed9d640 1 --2- 172.21.15.2:0/997341250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c39a0074d0 0x55c39a447030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5bfe59c640 1 -- 172.21.15.2:0/997341250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c39a0074d0 msgr2=0x55c39a447030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5bfe59c640 1 --2- 172.21.15.2:0/997341250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c39a0074d0 0x55c39a447030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5bfe59c640 1 -- 172.21.15.2:0/997341250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 msgr2=0x55c39a3d8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:48:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5bfe59c640 1 --2- 172.21.15.2:0/997341250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 0x55c39a3d8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5bfe59c640 1 -- 172.21.15.2:0/997341250 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c39a447570 con 0x55c399fe6d80 2024-05-14T09:48:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5bfdd9b640 1 --2- 172.21.15.2:0/997341250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 0x55c39a3d8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:48:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5bfe59c640 1 --2- 172.21.15.2:0/997341250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c399fe6d80 0x55c39a3d8430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55c39a3cae40 tx=0x7f5bf0009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82f695e509fdd513 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:48:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5be5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5be5ffb640 1 -- 172.21.15.2:0/997341250 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf008f020 con 0x55c399fe6d80 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5be5ffb640 1 -- 172.21.15.2:0/997341250 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f5bf0010a90 con 0x55c399fe6d80 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5c0bc83740 1 -- 172.21.15.2:0/997341250 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bf40048c0 con 0x55c399fe6d80 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5be5ffb640 1 -- 172.21.15.2:0/997341250 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5bf00a99e0 con 0x55c399fe6d80 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5c0bc83740 10 client.26512 fetch_fsmap learned FSMap version 52 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5c0bc83740 1 -- 172.21.15.2:0/997341250 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c39a4487f0 con 0x55c399fe6d80 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5be5ffb640 1 -- 172.21.15.2:0/997341250 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bf40048c0 con 0x55c399fe6d80 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.960+0000 7f5be5ffb640 1 -- 172.21.15.2:0/997341250 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7f5bf00cc0b0 con 0x55c399fe6d80 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.961+0000 7f5c0bc83740 10 client.26512 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.961+0000 7f5c0bc83740 10 client.26512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.961+0000 7f5c0bc83740 20 client.26512 populate_metadata read hostname 'smithi002' 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.961+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/997341250 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c39a0281b0 0x55c39a44b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.961+0000 7f5c0bc83740 4 client.26512 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.961+0000 7f5c0bc83740 1 -- 172.21.15.2:0/997341250 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c39a336050 con 0x55c39a0281b0 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.961+0000 7f5bfdd9b640 1 --2- 172.21.15.2:0/997341250 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c39a0281b0 0x55c39a44b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:48:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.961+0000 7f5bfdd9b640 1 --2- 172.21.15.2:0/997341250 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c39a0281b0 0x55c39a44b1c0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2279d57c4fb4df7 server_cookie=7c3764804072964e in_seq=0 out_seq=0 2024-05-14T09:48:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:56.961+0000 7f5be5ffb640 10 client.26512 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:57.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 8171 26.53 MB/sec warmup 23 sec latency 469.632 ms 2024-05-14T09:48:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.660+0000 7f5be5ffb640 1 -- 172.21.15.2:0/997341250 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c39a336050 con 0x55c39a0281b0 2024-05-14T09:48:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.660+0000 7f5be5ffb640 10 client.26512 handle_command_reply: tid=0 2024-05-14T09:48:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.660+0000 7f5c0bc83740 1 client.26512 shutdown 2024-05-14T09:48:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.660+0000 7f5c0bc83740 1 -- 172.21.15.2:0/997341250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c399fe6d80 msgr2=0x55c39a3d8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.660+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/997341250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c399fe6d80 0x55c39a3d8430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55c39a3cae40 tx=0x7f5bf0009490 comp rx=0 tx=0).stop 2024-05-14T09:48:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.660+0000 7f5c0bc83740 1 -- 172.21.15.2:0/997341250 shutdown_connections 2024-05-14T09:48:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.660+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/997341250 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c39a0281b0 0x55c39a44b1c0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.661+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/997341250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c39a0074d0 0x55c39a447030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.661+0000 7f5be5ffb640 0 client.26512 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:48:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.661+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/997341250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c399fff2f0 0x55c39a3d8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.661+0000 7f5c0bc83740 1 --2- 172.21.15.2:0/997341250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c399fe6d80 0x55c39a3d8430 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.661+0000 7f5c0bc83740 1 -- 172.21.15.2:0/997341250 >> 172.21.15.2:0/997341250 conn(0x55c39a01c300 msgr2=0x55c39a3c3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.661+0000 7f5c0bc83740 1 -- 172.21.15.2:0/997341250 shutdown_connections 2024-05-14T09:48:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.661+0000 7f5c0bc83740 1 -- 172.21.15.2:0/997341250 wait complete. 2024-05-14T09:48:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.661+0000 7f5c0bc83740 20 client.26512 trim_cache size 0 max 16384 2024-05-14T09:48:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:48:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:48:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:48:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:48:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.662+0000 7f5c04dce640 1 -- 172.21.15.2:0/813282584 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5be80b2920 msgr2=0x7f5be80b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.662+0000 7f5c04dce640 1 --2- 172.21.15.2:0/813282584 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5be80b2920 0x7f5be80b4e30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55c39a3cac30 tx=0x7f5bf80096a0 comp rx=0 tx=0).stop 2024-05-14T09:48:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.662+0000 7f5c04dce640 1 -- 172.21.15.2:0/813282584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c00103c60 msgr2=0x7f5c0019a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:48:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.662+0000 7f5c04dce640 1 --2- 172.21.15.2:0/813282584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c00103c60 0x7f5c0019a270 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000e980 tx=0x7f5bf000ee40 comp rx=0 tx=0).stop 2024-05-14T09:48:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.662+0000 7f5c04dce640 1 -- 172.21.15.2:0/813282584 shutdown_connections 2024-05-14T09:48:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.662+0000 7f5c04dce640 1 --2- 172.21.15.2:0/813282584 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f5be80b2920 0x7f5be80b4e30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.662+0000 7f5c04dce640 1 --2- 172.21.15.2:0/813282584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c00109ba0 0x7f5c001a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.662+0000 7f5c04dce640 1 --2- 172.21.15.2:0/813282584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c00104ea0 0x7f5c0019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.662+0000 7f5c04dce640 1 --2- 172.21.15.2:0/813282584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5c00103c60 0x7f5c0019a270 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:48:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.662+0000 7f5c04dce640 1 -- 172.21.15.2:0/813282584 >> 172.21.15.2:0/813282584 conn(0x7f5c000fee50 msgr2=0x7f5c00105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:48:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.662+0000 7f5c04dce640 1 -- 172.21.15.2:0/813282584 shutdown_connections 2024-05-14T09:48:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:48:57.662+0000 7f5c04dce640 1 -- 172.21.15.2:0/813282584 wait complete. 2024-05-14T09:48:58.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 8825 26.98 MB/sec warmup 24 sec latency 93.225 ms 2024-05-14T09:48:58.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:48:58 smithi002 ceph-mon[31557]: pgmap v845: 129 pgs: 129 active+clean; 306 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 13 MiB/s wr, 607 op/s 2024-05-14T09:48:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:48:58 smithi029 ceph-mon[36249]: pgmap v845: 129 pgs: 129 active+clean; 306 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 13 MiB/s wr, 607 op/s 2024-05-14T09:48:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:48:58 smithi026 ceph-mon[36056]: pgmap v845: 129 pgs: 129 active+clean; 306 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 13 MiB/s wr, 607 op/s 2024-05-14T09:48:59.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 9489 27.22 MB/sec warmup 25 sec latency 95.890 ms 2024-05-14T09:49:00.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 10118 27.73 MB/sec warmup 26 sec latency 124.009 ms 2024-05-14T09:49:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:00 smithi002 ceph-mon[31557]: pgmap v846: 129 pgs: 129 active+clean; 364 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 18 MiB/s wr, 487 op/s 2024-05-14T09:49:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:00 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:49:00.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:00 smithi029 ceph-mon[36249]: pgmap v846: 129 pgs: 129 active+clean; 364 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 18 MiB/s wr, 487 op/s 2024-05-14T09:49:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:00 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:49:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:00 smithi026 ceph-mon[36056]: pgmap v846: 129 pgs: 129 active+clean; 364 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 18 MiB/s wr, 487 op/s 2024-05-14T09:49:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:00 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:49:01.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 10926 28.69 MB/sec warmup 27 sec latency 87.302 ms 2024-05-14T09:49:02.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 11589 28.48 MB/sec warmup 28 sec latency 71.272 ms 2024-05-14T09:49:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:02 smithi002 ceph-mon[31557]: pgmap v847: 129 pgs: 129 active+clean; 360 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 17 MiB/s wr, 390 op/s 2024-05-14T09:49:02.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:02 smithi029 ceph-mon[36249]: pgmap v847: 129 pgs: 129 active+clean; 360 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 17 MiB/s wr, 390 op/s 2024-05-14T09:49:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:02 smithi026 ceph-mon[36056]: pgmap v847: 129 pgs: 129 active+clean; 360 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 17 MiB/s wr, 390 op/s 2024-05-14T09:49:03.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 12242 28.87 MB/sec warmup 29 sec latency 133.430 ms 2024-05-14T09:49:04.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 13180 29.15 MB/sec warmup 30 sec latency 60.987 ms 2024-05-14T09:49:04.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:04 smithi002 ceph-mon[31557]: pgmap v848: 129 pgs: 129 active+clean; 367 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 18 MiB/s wr, 580 op/s 2024-05-14T09:49:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:04 smithi029 ceph-mon[36249]: pgmap v848: 129 pgs: 129 active+clean; 367 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 18 MiB/s wr, 580 op/s 2024-05-14T09:49:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:04 smithi026 ceph-mon[36056]: pgmap v848: 129 pgs: 129 active+clean; 367 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 18 MiB/s wr, 580 op/s 2024-05-14T09:49:05.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 13980 30.05 MB/sec warmup 31 sec latency 170.626 ms 2024-05-14T09:49:06.598 INFO:tasks.workunit.client.0.smithi002.stdout: 10 14685 30.35 MB/sec warmup 32 sec latency 47.654 ms 2024-05-14T09:49:06.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:06 smithi002 ceph-mon[31557]: pgmap v849: 129 pgs: 129 active+clean; 367 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 17 MiB/s wr, 498 op/s 2024-05-14T09:49:06.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:06 smithi029 ceph-mon[36249]: pgmap v849: 129 pgs: 129 active+clean; 367 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 17 MiB/s wr, 498 op/s 2024-05-14T09:49:06.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:06 smithi026 ceph-mon[36056]: pgmap v849: 129 pgs: 129 active+clean; 367 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 17 MiB/s wr, 498 op/s 2024-05-14T09:49:07.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 15318 30.26 MB/sec warmup 33 sec latency 60.562 ms 2024-05-14T09:49:08.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 16096 30.57 MB/sec warmup 34 sec latency 50.487 ms 2024-05-14T09:49:08.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:08 smithi029 ceph-mon[36249]: pgmap v850: 129 pgs: 129 active+clean; 375 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 18 MiB/s wr, 593 op/s 2024-05-14T09:49:08.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:08 smithi026 ceph-mon[36056]: pgmap v850: 129 pgs: 129 active+clean; 375 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 18 MiB/s wr, 593 op/s 2024-05-14T09:49:08.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:08 smithi002 ceph-mon[31557]: pgmap v850: 129 pgs: 129 active+clean; 375 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 18 MiB/s wr, 593 op/s 2024-05-14T09:49:09.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 16863 30.82 MB/sec warmup 35 sec latency 119.943 ms 2024-05-14T09:49:10.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 17468 31.21 MB/sec warmup 36 sec latency 317.724 ms 2024-05-14T09:49:10.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:10 smithi029 ceph-mon[36249]: pgmap v851: 129 pgs: 129 active+clean; 384 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 8.8 MiB/s wr, 618 op/s 2024-05-14T09:49:10.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:10 smithi026 ceph-mon[36056]: pgmap v851: 129 pgs: 129 active+clean; 384 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 8.8 MiB/s wr, 618 op/s 2024-05-14T09:49:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.809+0000 7f142c213640 1 -- 172.21.15.2:0/2407993312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1424104740 msgr2=0x7f1424106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.809+0000 7f142c213640 1 --2- 172.21.15.2:0/2407993312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1424104740 0x7f1424106b60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1420009900 tx=0x7f142002a3f0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.810+0000 7f142c213640 1 -- 172.21.15.2:0/2407993312 shutdown_connections 2024-05-14T09:49:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.810+0000 7f142c213640 1 --2- 172.21.15.2:0/2407993312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1424109a00 0x7f142410be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.810+0000 7f142c213640 1 --2- 172.21.15.2:0/2407993312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14241070a0 0x7f14241094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.810+0000 7f142c213640 1 --2- 172.21.15.2:0/2407993312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1424104740 0x7f1424106b60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.810+0000 7f142c213640 1 -- 172.21.15.2:0/2407993312 >> 172.21.15.2:0/2407993312 conn(0x7f14240fdd40 msgr2=0x7f14241001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.810+0000 7f142c213640 1 -- 172.21.15.2:0/2407993312 shutdown_connections 2024-05-14T09:49:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.810+0000 7f142c213640 1 -- 172.21.15.2:0/2407993312 wait complete. 2024-05-14T09:49:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.811+0000 7f142c213640 1 Processor -- start 2024-05-14T09:49:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.811+0000 7f142c213640 1 -- start start 2024-05-14T09:49:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.811+0000 7f142c213640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1424104740 0x7f14241a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.811+0000 7f142c213640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14241070a0 0x7f14241a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.811+0000 7f142c213640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424109a00 0x7f14241a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.811+0000 7f142c213640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1424103c90 con 0x7f1424104740 2024-05-14T09:49:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.811+0000 7f142c213640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1424103b10 con 0x7f1424109a00 2024-05-14T09:49:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.811+0000 7f142c213640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1424103e10 con 0x7f14241070a0 2024-05-14T09:49:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.812+0000 7f142a7b0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424109a00 0x7f14241a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.812+0000 7f14297ae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14241070a0 0x7f14241a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.812+0000 7f1429faf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1424104740 0x7f14241a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.812+0000 7f142a7b0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424109a00 0x7f14241a7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:33852/0 (socket says 172.21.15.2:33852) 2024-05-14T09:49:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.812+0000 7f142a7b0640 1 -- 172.21.15.2:0/2024900907 learned_addr learned my addr 172.21.15.2:0/2024900907 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.812+0000 7f14297ae640 1 -- 172.21.15.2:0/2024900907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424109a00 msgr2=0x7f14241a7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.812+0000 7f14297ae640 1 --2- 172.21.15.2:0/2024900907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424109a00 0x7f14241a7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.812+0000 7f14297ae640 1 -- 172.21.15.2:0/2024900907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1424104740 msgr2=0x7f14241a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.812+0000 7f14297ae640 1 --2- 172.21.15.2:0/2024900907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1424104740 0x7f14241a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.812+0000 7f14297ae640 1 -- 172.21.15.2:0/2024900907 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1424112160 con 0x7f14241070a0 2024-05-14T09:49:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.812+0000 7f14297ae640 1 --2- 172.21.15.2:0/2024900907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14241070a0 0x7f14241a0e50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f141400e9f0 tx=0x7f141400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.812+0000 7f141affd640 1 -- 172.21.15.2:0/2024900907 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141400ccc0 con 0x7f14241070a0 2024-05-14T09:49:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.812+0000 7f141affd640 1 -- 172.21.15.2:0/2024900907 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1414005020 con 0x7f14241070a0 2024-05-14T09:49:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.812+0000 7f141affd640 1 -- 172.21.15.2:0/2024900907 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414004a00 con 0x7f14241070a0 2024-05-14T09:49:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.812+0000 7f142c213640 1 -- 172.21.15.2:0/2024900907 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1424112450 con 0x7f14241070a0 2024-05-14T09:49:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.813+0000 7f142c213640 1 -- 172.21.15.2:0/2024900907 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1424112950 con 0x7f14241070a0 2024-05-14T09:49:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.816+0000 7f141affd640 1 -- 172.21.15.2:0/2024900907 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1414004ba0 con 0x7f14241070a0 2024-05-14T09:49:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.816+0000 7f14331a0740 1 Processor -- start 2024-05-14T09:49:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.817+0000 7f14331a0740 1 -- start start 2024-05-14T09:49:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.817+0000 7f14331a0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915a11b0 0x55c191942a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.817+0000 7f14331a0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915782f0 0x55c191942fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.817+0000 7f14331a0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915804d0 0x55c191949ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.817+0000 7f14331a0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c1918a75d0 con 0x55c1915782f0 2024-05-14T09:49:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.817+0000 7f14331a0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c191860890 con 0x55c1915a11b0 2024-05-14T09:49:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.817+0000 7f14331a0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c1918a3590 con 0x55c1915804d0 2024-05-14T09:49:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.817+0000 7f1429faf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915782f0 0x55c191942fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.817+0000 7f14297ae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915804d0 0x55c191949ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.817+0000 7f14297ae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915804d0 0x55c191949ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:40382/0 (socket says 172.21.15.2:40382) 2024-05-14T09:49:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.817+0000 7f14297ae640 1 -- 172.21.15.2:0/1495596458 learned_addr learned my addr 172.21.15.2:0/1495596458 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.817+0000 7f142a7b0640 1 --2- 172.21.15.2:0/1495596458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915a11b0 0x55c191942a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.818+0000 7f14297ae640 1 -- 172.21.15.2:0/1495596458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915a11b0 msgr2=0x55c191942a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.818+0000 7f14297ae640 1 --2- 172.21.15.2:0/1495596458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915a11b0 0x55c191942a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.818+0000 7f14297ae640 1 -- 172.21.15.2:0/1495596458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915782f0 msgr2=0x55c191942fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.818+0000 7f14297ae640 1 --2- 172.21.15.2:0/1495596458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915782f0 0x55c191942fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.818+0000 7f14297ae640 1 -- 172.21.15.2:0/1495596458 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1915b9210 con 0x55c1915804d0 2024-05-14T09:49:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.818+0000 7f141affd640 1 --2- 172.21.15.2:0/2024900907 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14100b28a0 0x7f14100b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.818+0000 7f14297ae640 1 --2- 172.21.15.2:0/1495596458 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915804d0 0x55c191949ee0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f14140e8720 tx=0x7f14140e8810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84e061c0bae416fc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.818+0000 7f141affd640 1 -- 172.21.15.2:0/2024900907 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f14140e25b0 con 0x7f14241070a0 2024-05-14T09:49:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.818+0000 7f140ffff640 1 -- 172.21.15.2:0/1495596458 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14140a5670 con 0x55c1915804d0 2024-05-14T09:49:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.819+0000 7f140ffff640 1 -- 172.21.15.2:0/1495596458 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14140a92c0 con 0x55c1915804d0 2024-05-14T09:49:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.819+0000 7f140ffff640 1 -- 172.21.15.2:0/1495596458 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14140a5670 con 0x55c1915804d0 2024-05-14T09:49:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.819+0000 7f142a7b0640 1 --2- 172.21.15.2:0/2024900907 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14100b28a0 0x7f14100b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.820+0000 7f14331a0740 1 -- 172.21.15.2:0/1495596458 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915804d0 msgr2=0x55c191949ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.820+0000 7f142a7b0640 1 --2- 172.21.15.2:0/2024900907 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14100b28a0 0x7f14100b4db0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55c191943c30 tx=0x7f141c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.820+0000 7f14331a0740 1 --2- 172.21.15.2:0/1495596458 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915804d0 0x55c191949ee0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f14140e8720 tx=0x7f14140e8810 comp rx=0 tx=0).stop 2024-05-14T09:49:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.820+0000 7f14331a0740 1 -- 172.21.15.2:0/1495596458 shutdown_connections 2024-05-14T09:49:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.820+0000 7f14331a0740 1 --2- 172.21.15.2:0/1495596458 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915804d0 0x55c191949ee0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.820+0000 7f14331a0740 1 --2- 172.21.15.2:0/1495596458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915782f0 0x55c191942fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.820+0000 7f14331a0740 1 --2- 172.21.15.2:0/1495596458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915a11b0 0x55c191942a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.820+0000 7f14331a0740 1 -- 172.21.15.2:0/1495596458 >> 172.21.15.2:0/1495596458 conn(0x55c191595300 msgr2=0x55c19193e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.820+0000 7f14331a0740 1 -- 172.21.15.2:0/1495596458 shutdown_connections 2024-05-14T09:49:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.820+0000 7f14331a0740 1 -- 172.21.15.2:0/1495596458 wait complete. 2024-05-14T09:49:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.821+0000 7f14331a0740 1 -- start start 2024-05-14T09:49:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.821+0000 7f14331a0740 1 Processor -- start 2024-05-14T09:49:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.821+0000 7f14331a0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915782f0 0x55c19194c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.821+0000 7f14331a0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915804d0 0x55c19194cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.821+0000 7f14331a0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915a11b0 0x55c191953a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.821+0000 7f14331a0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c19194a930 con 0x55c1915804d0 2024-05-14T09:49:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.821+0000 7f14331a0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c191864370 con 0x55c1915782f0 2024-05-14T09:49:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.821+0000 7f14331a0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55c19194aab0 con 0x55c1915a11b0 2024-05-14T09:49:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.821+0000 7f14331a0740 10 client.0 _ll_register_callbacks cb 0x55c1915b8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:49:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.821+0000 7f1429faf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915782f0 0x55c19194c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.822+0000 7f142a7b0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915a11b0 0x55c191953a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.822+0000 7f1429faf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915782f0 0x55c19194c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:33862/0 (socket says 172.21.15.2:33862) 2024-05-14T09:49:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.822+0000 7f142a7b0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915a11b0 0x55c191953a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:40394/0 (socket says 172.21.15.2:40394) 2024-05-14T09:49:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.822+0000 7f1429faf640 1 -- 172.21.15.2:0/4267497873 learned_addr learned my addr 172.21.15.2:0/4267497873 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.822+0000 7f14297ae640 1 --2- 172.21.15.2:0/4267497873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915804d0 0x55c19194cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.822+0000 7f142a7b0640 1 -- 172.21.15.2:0/4267497873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915782f0 msgr2=0x55c19194c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.822+0000 7f142a7b0640 1 --2- 172.21.15.2:0/4267497873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915782f0 0x55c19194c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.822+0000 7f142a7b0640 1 -- 172.21.15.2:0/4267497873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915804d0 msgr2=0x55c19194cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.822+0000 7f142a7b0640 1 --2- 172.21.15.2:0/4267497873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915804d0 0x55c19194cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.822+0000 7f142a7b0640 1 -- 172.21.15.2:0/4267497873 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c191685620 con 0x55c1915a11b0 2024-05-14T09:49:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.822+0000 7f142a7b0640 1 --2- 172.21.15.2:0/4267497873 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915a11b0 0x55c191953a80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f141c011eb0 tx=0x7f141c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ded4c187488bbcba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.822+0000 7f140d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:49:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.822+0000 7f140d7fa640 1 -- 172.21.15.2:0/4267497873 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c004b20 con 0x55c1915a11b0 2024-05-14T09:49:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.822+0000 7f140d7fa640 1 -- 172.21.15.2:0/4267497873 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f141c004cc0 con 0x55c1915a11b0 2024-05-14T09:49:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.822+0000 7f14331a0740 1 -- 172.21.15.2:0/4267497873 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14140a92c0 con 0x55c1915a11b0 2024-05-14T09:49:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.823+0000 7f140d7fa640 1 -- 172.21.15.2:0/4267497873 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f141c00f8d0 con 0x55c1915a11b0 2024-05-14T09:49:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.823+0000 7f14331a0740 10 client.26524 fetch_fsmap learned FSMap version 53 2024-05-14T09:49:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.823+0000 7f14331a0740 1 -- 172.21.15.2:0/4267497873 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55c191956a80 con 0x55c1915a11b0 2024-05-14T09:49:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.823+0000 7f140d7fa640 1 -- 172.21.15.2:0/4267497873 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14140a92c0 con 0x55c1915a11b0 2024-05-14T09:49:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.823+0000 7f140d7fa640 1 -- 172.21.15.2:0/4267497873 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 53) ==== 8362+0+0 (secure 0 0 0) 0x7f141c026070 con 0x55c1915a11b0 2024-05-14T09:49:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.823+0000 7f14331a0740 10 client.26524 fetch_fsmap finished waiting for FSMap version 53 2024-05-14T09:49:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.823+0000 7f14331a0740 10 client.26524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:49:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.823+0000 7f14331a0740 20 client.26524 populate_metadata read hostname 'smithi002' 2024-05-14T09:49:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.823+0000 7f14331a0740 1 --2- 172.21.15.2:0/4267497873 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c19155fd80 0x55c19195a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.823+0000 7f14331a0740 4 client.26524 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:49:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.823+0000 7f14331a0740 1 -- 172.21.15.2:0/4267497873 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1918b51e0 con 0x55c19155fd80 2024-05-14T09:49:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.823+0000 7f1429faf640 1 --2- 172.21.15.2:0/4267497873 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c19155fd80 0x55c19195a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.824+0000 7f140d7fa640 10 client.26524 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.824+0000 7f1429faf640 1 --2- 172.21.15.2:0/4267497873 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c19155fd80 0x55c19195a250 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f9997bd5b23df7 server_cookie=19fd015bfad4d4e7 in_seq=0 out_seq=0 2024-05-14T09:49:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.825+0000 7f140d7fa640 1 -- 172.21.15.2:0/4267497873 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c1918b51e0 con 0x55c19155fd80 2024-05-14T09:49:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.825+0000 7f140d7fa640 10 client.26524 handle_command_reply: tid=0 2024-05-14T09:49:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.825+0000 7f14331a0740 1 client.26524 shutdown 2024-05-14T09:49:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.825+0000 7f14331a0740 1 -- 172.21.15.2:0/4267497873 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915a11b0 msgr2=0x55c191953a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.825+0000 7f14331a0740 1 --2- 172.21.15.2:0/4267497873 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915a11b0 0x55c191953a80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f141c011eb0 tx=0x7f141c004ab0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.826+0000 7f14331a0740 1 -- 172.21.15.2:0/4267497873 shutdown_connections 2024-05-14T09:49:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.826+0000 7f14331a0740 1 --2- 172.21.15.2:0/4267497873 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c19155fd80 0x55c19195a250 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.826+0000 7f14331a0740 1 --2- 172.21.15.2:0/4267497873 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915a11b0 0x55c191953a80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.826+0000 7f14331a0740 1 --2- 172.21.15.2:0/4267497873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915804d0 0x55c19194cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.826+0000 7f140d7fa640 0 client.26524 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.826+0000 7f14331a0740 1 --2- 172.21.15.2:0/4267497873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915782f0 0x55c19194c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.826+0000 7f14331a0740 1 -- 172.21.15.2:0/4267497873 >> 172.21.15.2:0/4267497873 conn(0x55c191595300 msgr2=0x55c191941300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.826+0000 7f14331a0740 1 -- 172.21.15.2:0/4267497873 shutdown_connections 2024-05-14T09:49:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.826+0000 7f14331a0740 1 -- 172.21.15.2:0/4267497873 wait complete. 2024-05-14T09:49:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.826+0000 7f14331a0740 20 client.26524 trim_cache size 0 max 16384 2024-05-14T09:49:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.847+0000 7f14331a0740 1 Processor -- start 2024-05-14T09:49:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.847+0000 7f14331a0740 1 -- start start 2024-05-14T09:49:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f14331a0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19155fd80 0x55c19193dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f14331a0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915782f0 0x55c19193e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f14331a0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915804d0 0x55c1919539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f14331a0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14241be4b0 con 0x55c1915804d0 2024-05-14T09:49:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f14331a0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f14241bbcf0 con 0x55c19155fd80 2024-05-14T09:49:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f14331a0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f14241b9c40 con 0x55c1915782f0 2024-05-14T09:49:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f142a7b0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915804d0 0x55c1919539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f1429faf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19155fd80 0x55c19193dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f1429faf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19155fd80 0x55c19193dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:33874/0 (socket says 172.21.15.2:33874) 2024-05-14T09:49:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f1429faf640 1 -- 172.21.15.2:0/2587930039 learned_addr learned my addr 172.21.15.2:0/2587930039 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f142a7b0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915804d0 0x55c1919539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40948/0 (socket says 172.21.15.2:40948) 2024-05-14T09:49:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f14297ae640 1 --2- 172.21.15.2:0/2587930039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915782f0 0x55c19193e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f142a7b0640 1 -- 172.21.15.2:0/2587930039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915782f0 msgr2=0x55c19193e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f142a7b0640 1 --2- 172.21.15.2:0/2587930039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915782f0 0x55c19193e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f142a7b0640 1 -- 172.21.15.2:0/2587930039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19155fd80 msgr2=0x55c19193dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f142a7b0640 1 --2- 172.21.15.2:0/2587930039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19155fd80 0x55c19193dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.848+0000 7f142a7b0640 1 -- 172.21.15.2:0/2587930039 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c191956870 con 0x55c1915804d0 2024-05-14T09:49:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.849+0000 7f142a7b0640 1 --2- 172.21.15.2:0/2587930039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915804d0 0x55c1919539a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f141c018040 tx=0x7f141c0095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1e53efdbb97770 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.849+0000 7f140f7fe640 1 -- 172.21.15.2:0/2587930039 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c03a070 con 0x55c1915804d0 2024-05-14T09:49:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.849+0000 7f140f7fe640 1 -- 172.21.15.2:0/2587930039 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f141c035480 con 0x55c1915804d0 2024-05-14T09:49:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.849+0000 7f140f7fe640 1 -- 172.21.15.2:0/2587930039 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c01e690 con 0x55c1915804d0 2024-05-14T09:49:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.850+0000 7f14331a0740 1 -- 172.21.15.2:0/2587930039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915804d0 msgr2=0x55c1919539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.850+0000 7f14331a0740 1 --2- 172.21.15.2:0/2587930039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915804d0 0x55c1919539a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f141c018040 tx=0x7f141c0095d0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.850+0000 7f14331a0740 1 -- 172.21.15.2:0/2587930039 shutdown_connections 2024-05-14T09:49:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.850+0000 7f14331a0740 1 --2- 172.21.15.2:0/2587930039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1915804d0 0x55c1919539a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.850+0000 7f14331a0740 1 --2- 172.21.15.2:0/2587930039 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915782f0 0x55c19193e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.850+0000 7f14331a0740 1 --2- 172.21.15.2:0/2587930039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19155fd80 0x55c19193dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.850+0000 7f14331a0740 1 -- 172.21.15.2:0/2587930039 >> 172.21.15.2:0/2587930039 conn(0x55c191595300 msgr2=0x55c191946a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.850+0000 7f14331a0740 1 -- 172.21.15.2:0/2587930039 shutdown_connections 2024-05-14T09:49:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.850+0000 7f14331a0740 1 -- 172.21.15.2:0/2587930039 wait complete. 2024-05-14T09:49:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.850+0000 7f14331a0740 1 -- start start 2024-05-14T09:49:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.851+0000 7f14331a0740 1 Processor -- start 2024-05-14T09:49:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.851+0000 7f14331a0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c19155fd80 0x55c191951780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.851+0000 7f14331a0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915782f0 0x55c191951cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.851+0000 7f14331a0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915804d0 0x55c191952ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.851+0000 7f14331a0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14241ba1b0 con 0x55c19155fd80 2024-05-14T09:49:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.851+0000 7f14331a0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f14241bb620 con 0x55c1915804d0 2024-05-14T09:49:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.851+0000 7f14331a0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f14241ba880 con 0x55c1915782f0 2024-05-14T09:49:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.851+0000 7f14331a0740 10 client.0 _ll_register_callbacks cb 0x7f141c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:49:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.851+0000 7f14297ae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915782f0 0x55c191951cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.851+0000 7f1429faf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c19155fd80 0x55c191951780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.851+0000 7f14297ae640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915782f0 0x55c191951cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:40408/0 (socket says 172.21.15.2:40408) 2024-05-14T09:49:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.851+0000 7f14297ae640 1 -- 172.21.15.2:0/913945984 learned_addr learned my addr 172.21.15.2:0/913945984 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.851+0000 7f142a7b0640 1 --2- 172.21.15.2:0/913945984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915804d0 0x55c191952ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.852+0000 7f14297ae640 1 -- 172.21.15.2:0/913945984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915804d0 msgr2=0x55c191952ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.852+0000 7f14297ae640 1 --2- 172.21.15.2:0/913945984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915804d0 0x55c191952ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.852+0000 7f14297ae640 1 -- 172.21.15.2:0/913945984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c19155fd80 msgr2=0x55c191951780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.852+0000 7f14297ae640 1 --2- 172.21.15.2:0/913945984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c19155fd80 0x55c191951780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.852+0000 7f14297ae640 1 -- 172.21.15.2:0/913945984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c191953160 con 0x55c1915782f0 2024-05-14T09:49:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.852+0000 7f14297ae640 1 --2- 172.21.15.2:0/913945984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915782f0 0x55c191951cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f14140e0000 tx=0x7f14140096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1500ffaa951e7dfb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.852+0000 7f140d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:49:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.852+0000 7f140d7fa640 1 -- 172.21.15.2:0/913945984 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141401c070 con 0x55c1915782f0 2024-05-14T09:49:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.852+0000 7f140d7fa640 1 -- 172.21.15.2:0/913945984 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f14140e82a0 con 0x55c1915782f0 2024-05-14T09:49:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.852+0000 7f14331a0740 1 -- 172.21.15.2:0/913945984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f141c035480 con 0x55c1915782f0 2024-05-14T09:49:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.852+0000 7f140d7fa640 1 -- 172.21.15.2:0/913945984 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f14140e5e60 con 0x55c1915782f0 2024-05-14T09:49:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.852+0000 7f14331a0740 10 client.26530 fetch_fsmap learned FSMap version 53 2024-05-14T09:49:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.852+0000 7f14331a0740 1 -- 172.21.15.2:0/913945984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55c1919543f0 con 0x55c1915782f0 2024-05-14T09:49:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.853+0000 7f140d7fa640 1 -- 172.21.15.2:0/913945984 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f141c035480 con 0x55c1915782f0 2024-05-14T09:49:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.853+0000 7f140d7fa640 1 -- 172.21.15.2:0/913945984 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 53) ==== 8362+0+0 (secure 0 0 0) 0x7f14140b2180 con 0x55c1915782f0 2024-05-14T09:49:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.853+0000 7f14331a0740 10 client.26530 fetch_fsmap finished waiting for FSMap version 53 2024-05-14T09:49:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.853+0000 7f14331a0740 10 client.26530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:49:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.853+0000 7f14331a0740 20 client.26530 populate_metadata read hostname 'smithi002' 2024-05-14T09:49:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.853+0000 7f14331a0740 1 --2- 172.21.15.2:0/913945984 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c1915a11b0 0x55c1919c4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.853+0000 7f14331a0740 4 client.26530 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:49:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.853+0000 7f14331a0740 1 -- 172.21.15.2:0/913945984 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1918af050 con 0x55c1915a11b0 2024-05-14T09:49:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.853+0000 7f1429faf640 1 --2- 172.21.15.2:0/913945984 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c1915a11b0 0x55c1919c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.854+0000 7f1429faf640 1 --2- 172.21.15.2:0/913945984 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c1915a11b0 0x55c1919c4820 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec1ef86f9a18a311 server_cookie=7100d14b9557de7b in_seq=0 out_seq=0 2024-05-14T09:49:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:10.854+0000 7f140d7fa640 10 client.26530 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:10.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:10 smithi002 ceph-mon[31557]: pgmap v851: 129 pgs: 129 active+clean; 384 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 8.8 MiB/s wr, 618 op/s 2024-05-14T09:49:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.352+0000 7f140d7fa640 1 -- 172.21.15.2:0/913945984 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c1918af050 con 0x55c1915a11b0 2024-05-14T09:49:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.352+0000 7f140d7fa640 10 client.26530 handle_command_reply: tid=0 2024-05-14T09:49:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.352+0000 7f14331a0740 1 client.26530 shutdown 2024-05-14T09:49:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.352+0000 7f14331a0740 1 -- 172.21.15.2:0/913945984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915782f0 msgr2=0x55c191951cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.352+0000 7f14331a0740 1 --2- 172.21.15.2:0/913945984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915782f0 0x55c191951cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f14140e0000 tx=0x7f14140096e0 comp rx=0 tx=0).stop 2024-05-14T09:49:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.352+0000 7f14331a0740 1 -- 172.21.15.2:0/913945984 shutdown_connections 2024-05-14T09:49:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.352+0000 7f14331a0740 1 --2- 172.21.15.2:0/913945984 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55c1915a11b0 0x55c1919c4820 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.352+0000 7f14331a0740 1 --2- 172.21.15.2:0/913945984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c1915804d0 0x55c191952ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.352+0000 7f140d7fa640 0 client.26530 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.352+0000 7f14331a0740 1 --2- 172.21.15.2:0/913945984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c1915782f0 0x55c191951cc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.352+0000 7f14331a0740 1 --2- 172.21.15.2:0/913945984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c19155fd80 0x55c191951780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.352+0000 7f14331a0740 1 -- 172.21.15.2:0/913945984 >> 172.21.15.2:0/913945984 conn(0x55c191595300 msgr2=0x55c1919411f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.353+0000 7f14331a0740 1 -- 172.21.15.2:0/913945984 shutdown_connections 2024-05-14T09:49:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.353+0000 7f14331a0740 1 -- 172.21.15.2:0/913945984 wait complete. 2024-05-14T09:49:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.353+0000 7f14331a0740 20 client.26530 trim_cache size 0 max 16384 2024-05-14T09:49:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:49:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:49:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:49:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:49:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.353+0000 7f142c213640 1 -- 172.21.15.2:0/2024900907 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14100b28a0 msgr2=0x7f14100b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.353+0000 7f142c213640 1 --2- 172.21.15.2:0/2024900907 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14100b28a0 0x7f14100b4db0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55c191943c30 tx=0x7f141c009500 comp rx=0 tx=0).stop 2024-05-14T09:49:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.354+0000 7f142c213640 1 -- 172.21.15.2:0/2024900907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14241070a0 msgr2=0x7f14241a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.354+0000 7f142c213640 1 --2- 172.21.15.2:0/2024900907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14241070a0 0x7f14241a0e50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f141400e9f0 tx=0x7f141400eeb0 comp rx=0 tx=0).stop 2024-05-14T09:49:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.354+0000 7f142c213640 1 -- 172.21.15.2:0/2024900907 shutdown_connections 2024-05-14T09:49:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.354+0000 7f142c213640 1 --2- 172.21.15.2:0/2024900907 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f14100b28a0 0x7f14100b4db0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.354+0000 7f142c213640 1 --2- 172.21.15.2:0/2024900907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1424109a00 0x7f14241a7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.354+0000 7f142c213640 1 --2- 172.21.15.2:0/2024900907 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14241070a0 0x7f14241a0e50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.354+0000 7f142c213640 1 --2- 172.21.15.2:0/2024900907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1424104740 0x7f14241a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.354+0000 7f142c213640 1 -- 172.21.15.2:0/2024900907 >> 172.21.15.2:0/2024900907 conn(0x7f14240fdd40 msgr2=0x7f1424105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.354+0000 7f142c213640 1 -- 172.21.15.2:0/2024900907 shutdown_connections 2024-05-14T09:49:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:11.354+0000 7f142c213640 1 -- 172.21.15.2:0/2024900907 wait complete. 2024-05-14T09:49:11.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 17804 30.85 MB/sec warmup 37 sec latency 367.595 ms 2024-05-14T09:49:12.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 18516 30.67 MB/sec warmup 38 sec latency 96.773 ms 2024-05-14T09:49:12.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:12 smithi029 ceph-mon[36249]: pgmap v852: 129 pgs: 129 active+clean; 394 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.2 MiB/s wr, 542 op/s 2024-05-14T09:49:12.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:12 smithi026 ceph-mon[36056]: pgmap v852: 129 pgs: 129 active+clean; 394 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.2 MiB/s wr, 542 op/s 2024-05-14T09:49:12.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:12 smithi002 ceph-mon[31557]: pgmap v852: 129 pgs: 129 active+clean; 394 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.2 MiB/s wr, 542 op/s 2024-05-14T09:49:13.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 19416 31.16 MB/sec warmup 39 sec latency 75.915 ms 2024-05-14T09:49:14.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 20319 31.76 MB/sec warmup 40 sec latency 107.772 ms 2024-05-14T09:49:14.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:14 smithi029 ceph-mon[36249]: pgmap v853: 129 pgs: 129 active+clean; 411 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.7 MiB/s wr, 763 op/s 2024-05-14T09:49:14.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:14 smithi026 ceph-mon[36056]: pgmap v853: 129 pgs: 129 active+clean; 411 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.7 MiB/s wr, 763 op/s 2024-05-14T09:49:14.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:14 smithi002 ceph-mon[31557]: pgmap v853: 129 pgs: 129 active+clean; 411 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.7 MiB/s wr, 763 op/s 2024-05-14T09:49:15.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 20725 31.62 MB/sec warmup 41 sec latency 357.903 ms 2024-05-14T09:49:16.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 21160 31.39 MB/sec warmup 42 sec latency 77.668 ms 2024-05-14T09:49:16.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:16 smithi026 ceph-mon[36056]: pgmap v854: 129 pgs: 129 active+clean; 411 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.5 MiB/s wr, 564 op/s 2024-05-14T09:49:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:16 smithi002 ceph-mon[31557]: pgmap v854: 129 pgs: 129 active+clean; 411 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.5 MiB/s wr, 564 op/s 2024-05-14T09:49:16.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:16 smithi029 ceph-mon[36249]: pgmap v854: 129 pgs: 129 active+clean; 411 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.5 MiB/s wr, 564 op/s 2024-05-14T09:49:17.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 21789 31.27 MB/sec warmup 43 sec latency 83.266 ms 2024-05-14T09:49:18.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 22548 31.61 MB/sec warmup 44 sec latency 144.460 ms 2024-05-14T09:49:18.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:18 smithi026 ceph-mon[36056]: pgmap v855: 129 pgs: 129 active+clean; 425 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.1 MiB/s wr, 751 op/s 2024-05-14T09:49:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:18 smithi002 ceph-mon[31557]: pgmap v855: 129 pgs: 129 active+clean; 425 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.1 MiB/s wr, 751 op/s 2024-05-14T09:49:18.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:18 smithi029 ceph-mon[36249]: pgmap v855: 129 pgs: 129 active+clean; 425 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.1 MiB/s wr, 751 op/s 2024-05-14T09:49:19.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 23240 31.62 MB/sec warmup 45 sec latency 137.847 ms 2024-05-14T09:49:20.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 23615 31.43 MB/sec warmup 46 sec latency 309.374 ms 2024-05-14T09:49:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:20 smithi026 ceph-mon[36056]: pgmap v856: 129 pgs: 129 active+clean; 429 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.3 MiB/s wr, 743 op/s 2024-05-14T09:49:20.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:20 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:49:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:20 smithi002 ceph-mon[31557]: pgmap v856: 129 pgs: 129 active+clean; 429 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.3 MiB/s wr, 743 op/s 2024-05-14T09:49:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:20 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:49:20.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:20 smithi029 ceph-mon[36249]: pgmap v856: 129 pgs: 129 active+clean; 429 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.3 MiB/s wr, 743 op/s 2024-05-14T09:49:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:20 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:49:21.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 24248 31.45 MB/sec warmup 47 sec latency 322.114 ms 2024-05-14T09:49:22.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 24975 31.61 MB/sec warmup 48 sec latency 70.956 ms 2024-05-14T09:49:22.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:22 smithi026 ceph-mon[36056]: pgmap v857: 129 pgs: 129 active+clean; 430 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.1 MiB/s wr, 694 op/s 2024-05-14T09:49:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:22 smithi002 ceph-mon[31557]: pgmap v857: 129 pgs: 129 active+clean; 430 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.1 MiB/s wr, 694 op/s 2024-05-14T09:49:22.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:22 smithi029 ceph-mon[36249]: pgmap v857: 129 pgs: 129 active+clean; 430 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.1 MiB/s wr, 694 op/s 2024-05-14T09:49:23.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 25761 32.00 MB/sec warmup 49 sec latency 87.815 ms 2024-05-14T09:49:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.464+0000 7f476a09a640 1 -- 172.21.15.2:0/4144662303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4764102b50 msgr2=0x7f4764103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.464+0000 7f476a09a640 1 --2- 172.21.15.2:0/4144662303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4764102b50 0x7f4764103850 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4754009900 tx=0x7f475402a410 comp rx=0 tx=0).stop 2024-05-14T09:49:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.464+0000 7f476a09a640 1 -- 172.21.15.2:0/4144662303 shutdown_connections 2024-05-14T09:49:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.464+0000 7f476a09a640 1 --2- 172.21.15.2:0/4144662303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4764108a90 0x7f476410aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.464+0000 7f476a09a640 1 --2- 172.21.15.2:0/4144662303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4764103d90 0x7f47641041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.464+0000 7f476a09a640 1 --2- 172.21.15.2:0/4144662303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4764102b50 0x7f4764103850 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.464+0000 7f476a09a640 1 -- 172.21.15.2:0/4144662303 >> 172.21.15.2:0/4144662303 conn(0x7f47640fdd40 msgr2=0x7f47641001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.464+0000 7f476a09a640 1 -- 172.21.15.2:0/4144662303 shutdown_connections 2024-05-14T09:49:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.464+0000 7f476a09a640 1 -- 172.21.15.2:0/4144662303 wait complete. 2024-05-14T09:49:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f476a09a640 1 Processor -- start 2024-05-14T09:49:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f476a09a640 1 -- start start 2024-05-14T09:49:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f476a09a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4764102b50 0x7f47641991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f476a09a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4764103d90 0x7f4764199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f476a09a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4764108a90 0x7f47641a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f476a09a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f476410e650 con 0x7f4764108a90 2024-05-14T09:49:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f476a09a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f476410e4d0 con 0x7f4764102b50 2024-05-14T09:49:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f476a09a640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f476410e7d0 con 0x7f4764103d90 2024-05-14T09:49:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f4763fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4764108a90 0x7f47641a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f475bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4764103d90 0x7f4764199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f4763fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4764108a90 0x7f47641a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37350/0 (socket says 172.21.15.2:37350) 2024-05-14T09:49:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f4763fff640 1 -- 172.21.15.2:0/3920236004 learned_addr learned my addr 172.21.15.2:0/3920236004 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f47637fe640 1 --2- 172.21.15.2:0/3920236004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4764102b50 0x7f47641991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f4763fff640 1 -- 172.21.15.2:0/3920236004 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4764103d90 msgr2=0x7f4764199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f4763fff640 1 --2- 172.21.15.2:0/3920236004 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4764103d90 0x7f4764199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f4763fff640 1 -- 172.21.15.2:0/3920236004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4764102b50 msgr2=0x7f47641991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f4763fff640 1 --2- 172.21.15.2:0/3920236004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4764102b50 0x7f47641991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f4763fff640 1 -- 172.21.15.2:0/3920236004 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47641a0d00 con 0x7f4764108a90 2024-05-14T09:49:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.465+0000 7f4763fff640 1 --2- 172.21.15.2:0/3920236004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4764108a90 0x7f47641a0690 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f475c00eb10 tx=0x7f475c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.466+0000 7f47617fa640 1 -- 172.21.15.2:0/3920236004 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c004b20 con 0x7f4764108a90 2024-05-14T09:49:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.466+0000 7f47617fa640 1 -- 172.21.15.2:0/3920236004 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475c004cc0 con 0x7f4764108a90 2024-05-14T09:49:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.466+0000 7f476a09a640 1 -- 172.21.15.2:0/3920236004 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47641a0ff0 con 0x7f4764108a90 2024-05-14T09:49:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.466+0000 7f47617fa640 1 -- 172.21.15.2:0/3920236004 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c010670 con 0x7f4764108a90 2024-05-14T09:49:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.466+0000 7f476a09a640 1 -- 172.21.15.2:0/3920236004 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47641a1460 con 0x7f4764108a90 2024-05-14T09:49:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.469+0000 7f477101f740 1 Processor -- start 2024-05-14T09:49:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.469+0000 7f47617fa640 1 -- 172.21.15.2:0/3920236004 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f475c005020 con 0x7f4764108a90 2024-05-14T09:49:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f477101f740 1 -- start start 2024-05-14T09:49:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f477101f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d9f1b0 0x556bf3142c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f477101f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d762f0 0x556bf31431f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f477101f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d7e4d0 0x556bf314a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f477101f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556bf30a55d0 con 0x556bf2d9f1b0 2024-05-14T09:49:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f477101f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556bf305e890 con 0x556bf2d7e4d0 2024-05-14T09:49:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f477101f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556bf30a1590 con 0x556bf2d762f0 2024-05-14T09:49:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f475bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d9f1b0 0x556bf3142c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f475bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d9f1b0 0x556bf3142c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37356/0 (socket says 172.21.15.2:37356) 2024-05-14T09:49:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f475bfff640 1 -- 172.21.15.2:0/2405946462 learned_addr learned my addr 172.21.15.2:0/2405946462 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f47637fe640 1 --2- 172.21.15.2:0/2405946462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d762f0 0x556bf31431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f475bfff640 1 --2- 172.21.15.2:0/2405946462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d7e4d0 0x556bf314a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f475bfff640 1 -- 172.21.15.2:0/2405946462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d762f0 msgr2=0x556bf31431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f475bfff640 1 --2- 172.21.15.2:0/2405946462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d762f0 0x556bf31431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f475bfff640 1 -- 172.21.15.2:0/2405946462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d7e4d0 msgr2=0x556bf314a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.470+0000 7f475bfff640 1 --2- 172.21.15.2:0/2405946462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d7e4d0 0x556bf314a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.471+0000 7f475bfff640 1 -- 172.21.15.2:0/2405946462 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bf2db7210 con 0x556bf2d9f1b0 2024-05-14T09:49:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.471+0000 7f47617fa640 1 --2- 172.21.15.2:0/3920236004 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f47500b2920 0x7f47500b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.471+0000 7f475bfff640 1 --2- 172.21.15.2:0/2405946462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d9f1b0 0x556bf3142c70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f474c013620 tx=0x7f474c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd4e73625de28bf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.471+0000 7f47637fe640 1 --2- 172.21.15.2:0/3920236004 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f47500b2920 0x7f47500b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.471+0000 7f47617fa640 1 -- 172.21.15.2:0/3920236004 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f475c0e2ae0 con 0x7f4764108a90 2024-05-14T09:49:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.471+0000 7f4759ffb640 1 -- 172.21.15.2:0/2405946462 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c00f580 con 0x556bf2d9f1b0 2024-05-14T09:49:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.471+0000 7f4759ffb640 1 -- 172.21.15.2:0/2405946462 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f474c00fbd0 con 0x556bf2d9f1b0 2024-05-14T09:49:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.471+0000 7f4759ffb640 1 -- 172.21.15.2:0/2405946462 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c010aa0 con 0x556bf2d9f1b0 2024-05-14T09:49:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.472+0000 7f477101f740 1 -- 172.21.15.2:0/2405946462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d9f1b0 msgr2=0x556bf3142c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.472+0000 7f477101f740 1 --2- 172.21.15.2:0/2405946462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d9f1b0 0x556bf3142c70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f474c013620 tx=0x7f474c009850 comp rx=0 tx=0).stop 2024-05-14T09:49:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.472+0000 7f47637fe640 1 --2- 172.21.15.2:0/3920236004 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f47500b2920 0x7f47500b4e30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4754005ab0 tx=0x7f4754031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.472+0000 7f477101f740 1 -- 172.21.15.2:0/2405946462 shutdown_connections 2024-05-14T09:49:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.472+0000 7f477101f740 1 --2- 172.21.15.2:0/2405946462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d7e4d0 0x556bf314a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.472+0000 7f477101f740 1 --2- 172.21.15.2:0/2405946462 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d762f0 0x556bf31431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.472+0000 7f477101f740 1 --2- 172.21.15.2:0/2405946462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d9f1b0 0x556bf3142c70 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.472+0000 7f477101f740 1 -- 172.21.15.2:0/2405946462 >> 172.21.15.2:0/2405946462 conn(0x556bf2d93300 msgr2=0x556bf313e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.472+0000 7f477101f740 1 -- 172.21.15.2:0/2405946462 shutdown_connections 2024-05-14T09:49:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.472+0000 7f477101f740 1 -- 172.21.15.2:0/2405946462 wait complete. 2024-05-14T09:49:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.472+0000 7f477101f740 1 -- start start 2024-05-14T09:49:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.473+0000 7f477101f740 1 Processor -- start 2024-05-14T09:49:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.473+0000 7f477101f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.473+0000 7f477101f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d7e4d0 0x556bf314cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.473+0000 7f477101f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d9f1b0 0x556bf3153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.473+0000 7f477101f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556bf314aae0 con 0x556bf2d762f0 2024-05-14T09:49:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.473+0000 7f477101f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556bf3062370 con 0x556bf2d7e4d0 2024-05-14T09:49:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.473+0000 7f477101f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556bf314ac60 con 0x556bf2d9f1b0 2024-05-14T09:49:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.473+0000 7f47637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.473+0000 7f477101f740 10 client.0 _ll_register_callbacks cb 0x556bf2db6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:49:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.473+0000 7f47637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37364/0 (socket says 172.21.15.2:37364) 2024-05-14T09:49:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.473+0000 7f47637fe640 1 -- 172.21.15.2:0/2726081105 learned_addr learned my addr 172.21.15.2:0/2726081105 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.473+0000 7f4763fff640 1 --2- 172.21.15.2:0/2726081105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d9f1b0 0x556bf3153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.473+0000 7f475bfff640 1 --2- 172.21.15.2:0/2726081105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d7e4d0 0x556bf314cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.474+0000 7f47637fe640 1 -- 172.21.15.2:0/2726081105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d9f1b0 msgr2=0x556bf3153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.474+0000 7f47637fe640 1 --2- 172.21.15.2:0/2726081105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d9f1b0 0x556bf3153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.474+0000 7f47637fe640 1 -- 172.21.15.2:0/2726081105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d7e4d0 msgr2=0x556bf314cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.474+0000 7f47637fe640 1 --2- 172.21.15.2:0/2726081105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d7e4d0 0x556bf314cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.474+0000 7f47637fe640 1 -- 172.21.15.2:0/2726081105 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bf2e83620 con 0x556bf2d762f0 2024-05-14T09:49:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.474+0000 7f4763fff640 1 --2- 172.21.15.2:0/2726081105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d9f1b0 0x556bf3153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:49:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.474+0000 7f47637fe640 1 --2- 172.21.15.2:0/2726081105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314c850 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4754009900 tx=0x7f475402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a56e82395ebae9e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.474+0000 7f4742ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:49:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.474+0000 7f4742ffd640 1 -- 172.21.15.2:0/2726081105 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754035040 con 0x556bf2d762f0 2024-05-14T09:49:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.474+0000 7f4742ffd640 1 -- 172.21.15.2:0/2726081105 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f4754004240 con 0x556bf2d762f0 2024-05-14T09:49:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.474+0000 7f477101f740 1 -- 172.21.15.2:0/2726081105 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f474c00fbd0 con 0x556bf2d762f0 2024-05-14T09:49:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.474+0000 7f4742ffd640 1 -- 172.21.15.2:0/2726081105 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f4754004440 con 0x556bf2d762f0 2024-05-14T09:49:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.474+0000 7f477101f740 10 client.16911 fetch_fsmap learned FSMap version 54 2024-05-14T09:49:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.474+0000 7f477101f740 1 -- 172.21.15.2:0/2726081105 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x556bf3156c90 con 0x556bf2d762f0 2024-05-14T09:49:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.475+0000 7f4742ffd640 1 -- 172.21.15.2:0/2726081105 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f474c00fbd0 con 0x556bf2d762f0 2024-05-14T09:49:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.475+0000 7f4742ffd640 1 -- 172.21.15.2:0/2726081105 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f4754044070 con 0x556bf2d762f0 2024-05-14T09:49:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.475+0000 7f477101f740 10 client.16911 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T09:49:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.475+0000 7f477101f740 10 client.16911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:49:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.475+0000 7f477101f740 20 client.16911 populate_metadata read hostname 'smithi002' 2024-05-14T09:49:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.475+0000 7f477101f740 1 --2- 172.21.15.2:0/2726081105 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556bf2d5dd80 0x556bf315a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.475+0000 7f477101f740 4 client.16911 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:49:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.475+0000 7f477101f740 1 -- 172.21.15.2:0/2726081105 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bf30b31e0 con 0x556bf2d5dd80 2024-05-14T09:49:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.475+0000 7f475bfff640 1 --2- 172.21.15.2:0/2726081105 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556bf2d5dd80 0x556bf315a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.476+0000 7f4742ffd640 10 client.16911 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.476+0000 7f475bfff640 1 --2- 172.21.15.2:0/2726081105 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556bf2d5dd80 0x556bf315a470 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90c18042dc93bc65 server_cookie=fc2b5f181636db62 in_seq=0 out_seq=0 2024-05-14T09:49:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.482+0000 7f4742ffd640 1 -- 172.21.15.2:0/2726081105 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556bf30b31e0 con 0x556bf2d5dd80 2024-05-14T09:49:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.482+0000 7f4742ffd640 10 client.16911 handle_command_reply: tid=0 2024-05-14T09:49:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.482+0000 7f477101f740 1 client.16911 shutdown 2024-05-14T09:49:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.483+0000 7f477101f740 1 -- 172.21.15.2:0/2726081105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 msgr2=0x556bf314c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.483+0000 7f477101f740 1 --2- 172.21.15.2:0/2726081105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314c850 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4754009900 tx=0x7f475402a410 comp rx=0 tx=0).stop 2024-05-14T09:49:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.483+0000 7f477101f740 1 -- 172.21.15.2:0/2726081105 shutdown_connections 2024-05-14T09:49:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.483+0000 7f477101f740 1 --2- 172.21.15.2:0/2726081105 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556bf2d5dd80 0x556bf315a470 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.483+0000 7f477101f740 1 --2- 172.21.15.2:0/2726081105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d9f1b0 0x556bf3153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.483+0000 7f477101f740 1 --2- 172.21.15.2:0/2726081105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d7e4d0 0x556bf314cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.483+0000 7f477101f740 1 --2- 172.21.15.2:0/2726081105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314c850 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.483+0000 7f4742ffd640 0 client.16911 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.483+0000 7f477101f740 1 -- 172.21.15.2:0/2726081105 >> 172.21.15.2:0/2726081105 conn(0x556bf2d93300 msgr2=0x556bf3141510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.483+0000 7f477101f740 1 -- 172.21.15.2:0/2726081105 shutdown_connections 2024-05-14T09:49:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.483+0000 7f477101f740 1 -- 172.21.15.2:0/2726081105 wait complete. 2024-05-14T09:49:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.483+0000 7f477101f740 20 client.16911 trim_cache size 0 max 16384 2024-05-14T09:49:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.498+0000 7f477101f740 1 Processor -- start 2024-05-14T09:49:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f477101f740 1 -- start start 2024-05-14T09:49:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f477101f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f477101f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d7e4d0 0x556bf3153c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f477101f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d9f1b0 0x556bf31541c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f477101f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f47641ad460 con 0x556bf2d762f0 2024-05-14T09:49:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f477101f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f47641aaca0 con 0x556bf2d9f1b0 2024-05-14T09:49:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f477101f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f47641a8af0 con 0x556bf2d7e4d0 2024-05-14T09:49:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f47637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f47637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37366/0 (socket says 172.21.15.2:37366) 2024-05-14T09:49:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f47637fe640 1 -- 172.21.15.2:0/2972289061 learned_addr learned my addr 172.21.15.2:0/2972289061 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f4763fff640 1 --2- 172.21.15.2:0/2972289061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d9f1b0 0x556bf31541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f475bfff640 1 --2- 172.21.15.2:0/2972289061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d7e4d0 0x556bf3153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f47637fe640 1 -- 172.21.15.2:0/2972289061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d7e4d0 msgr2=0x556bf3153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f47637fe640 1 --2- 172.21.15.2:0/2972289061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d7e4d0 0x556bf3153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f47637fe640 1 -- 172.21.15.2:0/2972289061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d9f1b0 msgr2=0x556bf31541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f47637fe640 1 --2- 172.21.15.2:0/2972289061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d9f1b0 0x556bf31541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f47637fe640 1 -- 172.21.15.2:0/2972289061 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bf31364d0 con 0x556bf2d762f0 2024-05-14T09:49:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.499+0000 7f47637fe640 1 --2- 172.21.15.2:0/2972289061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314cb30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4754005f70 tx=0x7f4754009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4bdb1b9f0f265f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.500+0000 7f47597fa640 1 -- 172.21.15.2:0/2972289061 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754035040 con 0x556bf2d762f0 2024-05-14T09:49:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.500+0000 7f47597fa640 1 -- 172.21.15.2:0/2972289061 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475403db50 con 0x556bf2d762f0 2024-05-14T09:49:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.500+0000 7f47597fa640 1 -- 172.21.15.2:0/2972289061 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475405e6a0 con 0x556bf2d762f0 2024-05-14T09:49:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.501+0000 7f477101f740 1 -- 172.21.15.2:0/2972289061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 msgr2=0x556bf314cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.501+0000 7f477101f740 1 --2- 172.21.15.2:0/2972289061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314cb30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4754005f70 tx=0x7f4754009560 comp rx=0 tx=0).stop 2024-05-14T09:49:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.501+0000 7f477101f740 1 -- 172.21.15.2:0/2972289061 shutdown_connections 2024-05-14T09:49:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.501+0000 7f477101f740 1 --2- 172.21.15.2:0/2972289061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d9f1b0 0x556bf31541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.501+0000 7f477101f740 1 --2- 172.21.15.2:0/2972289061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d7e4d0 0x556bf3153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.501+0000 7f477101f740 1 --2- 172.21.15.2:0/2972289061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314cb30 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.501+0000 7f477101f740 1 -- 172.21.15.2:0/2972289061 >> 172.21.15.2:0/2972289061 conn(0x556bf2d93300 msgr2=0x556bf314c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.501+0000 7f477101f740 1 -- 172.21.15.2:0/2972289061 shutdown_connections 2024-05-14T09:49:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.501+0000 7f477101f740 1 -- 172.21.15.2:0/2972289061 wait complete. 2024-05-14T09:49:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.501+0000 7f477101f740 1 -- start start 2024-05-14T09:49:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.501+0000 7f477101f740 1 Processor -- start 2024-05-14T09:49:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f477101f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f477101f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d7e4d0 0x556bf3151d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f477101f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d9f1b0 0x556bf31c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f477101f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f47641a9160 con 0x556bf2d762f0 2024-05-14T09:49:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f477101f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f47641aa5d0 con 0x556bf2d9f1b0 2024-05-14T09:49:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f477101f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f47641a9830 con 0x556bf2d7e4d0 2024-05-14T09:49:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f477101f740 10 client.0 _ll_register_callbacks cb 0x7f4754009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:49:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f47637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f475bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d7e4d0 0x556bf3151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f47637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37370/0 (socket says 172.21.15.2:37370) 2024-05-14T09:49:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f475bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d7e4d0 0x556bf3151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:51776/0 (socket says 172.21.15.2:51776) 2024-05-14T09:49:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f4763fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d9f1b0 0x556bf31c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f475bfff640 1 -- 172.21.15.2:0/177096817 learned_addr learned my addr 172.21.15.2:0/177096817 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f47637fe640 1 -- 172.21.15.2:0/177096817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d7e4d0 msgr2=0x556bf3151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f47637fe640 1 --2- 172.21.15.2:0/177096817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d7e4d0 0x556bf3151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f47637fe640 1 -- 172.21.15.2:0/177096817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d9f1b0 msgr2=0x556bf31c2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f47637fe640 1 --2- 172.21.15.2:0/177096817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d9f1b0 0x556bf31c2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f47637fe640 1 -- 172.21.15.2:0/177096817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bf31c3110 con 0x556bf2d762f0 2024-05-14T09:49:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f47637fe640 1 --2- 172.21.15.2:0/177096817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314f830 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f475403d6e0 tx=0x7f475402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae52d592473e73da server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.502+0000 7f4742ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:49:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.503+0000 7f4742ffd640 1 -- 172.21.15.2:0/177096817 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754035040 con 0x556bf2d762f0 2024-05-14T09:49:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.503+0000 7f4742ffd640 1 -- 172.21.15.2:0/177096817 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f47540035f0 con 0x556bf2d762f0 2024-05-14T09:49:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.503+0000 7f477101f740 1 -- 172.21.15.2:0/177096817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f475403db50 con 0x556bf2d762f0 2024-05-14T09:49:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.503+0000 7f4742ffd640 1 -- 172.21.15.2:0/177096817 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f4754003df0 con 0x556bf2d762f0 2024-05-14T09:49:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.503+0000 7f477101f740 10 client.16917 fetch_fsmap learned FSMap version 54 2024-05-14T09:49:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.503+0000 7f477101f740 1 -- 172.21.15.2:0/177096817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x556bf31c41f0 con 0x556bf2d762f0 2024-05-14T09:49:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.503+0000 7f4742ffd640 1 -- 172.21.15.2:0/177096817 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475403db50 con 0x556bf2d762f0 2024-05-14T09:49:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.503+0000 7f4742ffd640 1 -- 172.21.15.2:0/177096817 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f475405c020 con 0x556bf2d762f0 2024-05-14T09:49:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.503+0000 7f477101f740 10 client.16917 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T09:49:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.503+0000 7f477101f740 10 client.16917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:49:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.503+0000 7f477101f740 20 client.16917 populate_metadata read hostname 'smithi002' 2024-05-14T09:49:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.503+0000 7f477101f740 1 --2- 172.21.15.2:0/177096817 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556bf2d331c0 0x556bf31c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.503+0000 7f477101f740 4 client.16917 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:49:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.503+0000 7f477101f740 1 -- 172.21.15.2:0/177096817 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556bf30ad050 con 0x556bf2d331c0 2024-05-14T09:49:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.503+0000 7f475bfff640 1 --2- 172.21.15.2:0/177096817 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556bf2d331c0 0x556bf31c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.504+0000 7f475bfff640 1 --2- 172.21.15.2:0/177096817 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556bf2d331c0 0x556bf31c6b80 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1320a6ca3cca2b4f server_cookie=c21bbbdf4d2ac30e in_seq=0 out_seq=0 2024-05-14T09:49:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:24.504+0000 7f4742ffd640 10 client.16917 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:24.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 26320 31.88 MB/sec warmup 50 sec latency 83.743 ms 2024-05-14T09:49:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:24 smithi002 ceph-mon[31557]: pgmap v858: 129 pgs: 129 active+clean; 437 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.6 MiB/s wr, 807 op/s 2024-05-14T09:49:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:24 smithi029 ceph-mon[36249]: pgmap v858: 129 pgs: 129 active+clean; 437 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.6 MiB/s wr, 807 op/s 2024-05-14T09:49:24.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:24 smithi026 ceph-mon[36056]: pgmap v858: 129 pgs: 129 active+clean; 437 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.6 MiB/s wr, 807 op/s 2024-05-14T09:49:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.338+0000 7f4742ffd640 1 -- 172.21.15.2:0/177096817 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556bf30ad050 con 0x556bf2d331c0 2024-05-14T09:49:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.338+0000 7f4742ffd640 10 client.16917 handle_command_reply: tid=0 2024-05-14T09:49:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.338+0000 7f477101f740 1 client.16917 shutdown 2024-05-14T09:49:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.338+0000 7f477101f740 1 -- 172.21.15.2:0/177096817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 msgr2=0x556bf314f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.338+0000 7f477101f740 1 --2- 172.21.15.2:0/177096817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314f830 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f475403d6e0 tx=0x7f475402a410 comp rx=0 tx=0).stop 2024-05-14T09:49:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.338+0000 7f477101f740 1 -- 172.21.15.2:0/177096817 shutdown_connections 2024-05-14T09:49:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.338+0000 7f477101f740 1 --2- 172.21.15.2:0/177096817 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556bf2d331c0 0x556bf31c6b80 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.338+0000 7f477101f740 1 --2- 172.21.15.2:0/177096817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf2d9f1b0 0x556bf31c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.338+0000 7f477101f740 1 --2- 172.21.15.2:0/177096817 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556bf2d7e4d0 0x556bf3151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.338+0000 7f477101f740 1 --2- 172.21.15.2:0/177096817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556bf2d762f0 0x556bf314f830 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.339+0000 7f477101f740 1 -- 172.21.15.2:0/177096817 >> 172.21.15.2:0/177096817 conn(0x556bf2d93300 msgr2=0x556bf314cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.339+0000 7f4742ffd640 0 client.16917 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.339+0000 7f477101f740 1 -- 172.21.15.2:0/177096817 shutdown_connections 2024-05-14T09:49:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.339+0000 7f477101f740 1 -- 172.21.15.2:0/177096817 wait complete. 2024-05-14T09:49:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.339+0000 7f477101f740 20 client.16917 trim_cache size 0 max 16384 2024-05-14T09:49:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:49:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:49:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:49:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:49:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.340+0000 7f476a09a640 1 -- 172.21.15.2:0/3920236004 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f47500b2920 msgr2=0x7f47500b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.340+0000 7f476a09a640 1 --2- 172.21.15.2:0/3920236004 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f47500b2920 0x7f47500b4e30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4754005ab0 tx=0x7f4754031400 comp rx=0 tx=0).stop 2024-05-14T09:49:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.340+0000 7f476a09a640 1 -- 172.21.15.2:0/3920236004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4764108a90 msgr2=0x7f47641a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.340+0000 7f476a09a640 1 --2- 172.21.15.2:0/3920236004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4764108a90 0x7f47641a0690 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f475c00eb10 tx=0x7f475c00c2d0 comp rx=0 tx=0).stop 2024-05-14T09:49:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.340+0000 7f476a09a640 1 -- 172.21.15.2:0/3920236004 shutdown_connections 2024-05-14T09:49:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.340+0000 7f476a09a640 1 --2- 172.21.15.2:0/3920236004 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f47500b2920 0x7f47500b4e30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.340+0000 7f476a09a640 1 --2- 172.21.15.2:0/3920236004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4764108a90 0x7f47641a0690 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.340+0000 7f476a09a640 1 --2- 172.21.15.2:0/3920236004 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4764103d90 0x7f4764199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.340+0000 7f476a09a640 1 --2- 172.21.15.2:0/3920236004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4764102b50 0x7f47641991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.340+0000 7f476a09a640 1 -- 172.21.15.2:0/3920236004 >> 172.21.15.2:0/3920236004 conn(0x7f47640fdd40 msgr2=0x7f4764104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.340+0000 7f476a09a640 1 -- 172.21.15.2:0/3920236004 shutdown_connections 2024-05-14T09:49:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:25.340+0000 7f476a09a640 1 -- 172.21.15.2:0/3920236004 wait complete. 2024-05-14T09:49:25.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 26533 31.44 MB/sec warmup 51 sec latency 526.126 ms 2024-05-14T09:49:26.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 26833 31.04 MB/sec warmup 52 sec latency 532.053 ms 2024-05-14T09:49:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:26 smithi002 ceph-mon[31557]: pgmap v859: 129 pgs: 129 active+clean; 437 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.0 MiB/s wr, 526 op/s 2024-05-14T09:49:26.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:26 smithi029 ceph-mon[36249]: pgmap v859: 129 pgs: 129 active+clean; 437 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.0 MiB/s wr, 526 op/s 2024-05-14T09:49:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:26 smithi026 ceph-mon[36056]: pgmap v859: 129 pgs: 129 active+clean; 437 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.0 MiB/s wr, 526 op/s 2024-05-14T09:49:27.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 27582 31.17 MB/sec warmup 53 sec latency 83.888 ms 2024-05-14T09:49:28.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 28447 31.65 MB/sec warmup 54 sec latency 90.662 ms 2024-05-14T09:49:28.722 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:28 smithi002 ceph-mon[31557]: pgmap v860: 129 pgs: 129 active+clean; 435 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.6 MiB/s wr, 704 op/s 2024-05-14T09:49:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:28 smithi029 ceph-mon[36249]: pgmap v860: 129 pgs: 129 active+clean; 435 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.6 MiB/s wr, 704 op/s 2024-05-14T09:49:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:28 smithi026 ceph-mon[36056]: pgmap v860: 129 pgs: 129 active+clean; 435 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.6 MiB/s wr, 704 op/s 2024-05-14T09:49:29.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 29053 31.69 MB/sec warmup 55 sec latency 103.032 ms 2024-05-14T09:49:30.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 29380 31.41 MB/sec warmup 56 sec latency 348.730 ms 2024-05-14T09:49:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:30 smithi002 ceph-mon[31557]: pgmap v861: 129 pgs: 129 active+clean; 440 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.5 MiB/s wr, 704 op/s 2024-05-14T09:49:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:30 smithi029 ceph-mon[36249]: pgmap v861: 129 pgs: 129 active+clean; 440 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.5 MiB/s wr, 704 op/s 2024-05-14T09:49:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:30 smithi026 ceph-mon[36056]: pgmap v861: 129 pgs: 129 active+clean; 440 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.5 MiB/s wr, 704 op/s 2024-05-14T09:49:31.599 INFO:tasks.workunit.client.0.smithi002.stdout: 10 29544 30.94 MB/sec warmup 57 sec latency 470.710 ms 2024-05-14T09:49:32.600 INFO:tasks.workunit.client.0.smithi002.stdout: 10 30363 31.16 MB/sec warmup 58 sec latency 83.758 ms 2024-05-14T09:49:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:32 smithi002 ceph-mon[31557]: pgmap v862: 129 pgs: 129 active+clean; 442 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.9 MiB/s wr, 678 op/s 2024-05-14T09:49:32.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:32 smithi029 ceph-mon[36249]: pgmap v862: 129 pgs: 129 active+clean; 442 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.9 MiB/s wr, 678 op/s 2024-05-14T09:49:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:32 smithi026 ceph-mon[36056]: pgmap v862: 129 pgs: 129 active+clean; 442 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.9 MiB/s wr, 678 op/s 2024-05-14T09:49:33.600 INFO:tasks.workunit.client.0.smithi002.stdout: 10 31280 31.45 MB/sec warmup 59 sec latency 64.443 ms 2024-05-14T09:49:34.600 INFO:tasks.workunit.client.0.smithi002.stdout: 10 32018 31.65 MB/sec warmup 60 sec latency 113.640 ms 2024-05-14T09:49:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:34 smithi002 ceph-mon[31557]: pgmap v863: 129 pgs: 129 active+clean; 449 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.3 MiB/s wr, 785 op/s 2024-05-14T09:49:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:34 smithi029 ceph-mon[36249]: pgmap v863: 129 pgs: 129 active+clean; 449 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.3 MiB/s wr, 785 op/s 2024-05-14T09:49:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:34 smithi026 ceph-mon[36056]: pgmap v863: 129 pgs: 129 active+clean; 449 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.3 MiB/s wr, 785 op/s 2024-05-14T09:49:35.600 INFO:tasks.workunit.client.0.smithi002.stdout: 10 32322 31.48 MB/sec warmup 61 sec latency 305.256 ms 2024-05-14T09:49:36.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 32530 31.16 MB/sec warmup 62 sec latency 495.912 ms 2024-05-14T09:49:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:36 smithi002 ceph-mon[31557]: pgmap v864: 129 pgs: 129 active+clean; 449 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.8 MiB/s wr, 598 op/s 2024-05-14T09:49:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:36 smithi029 ceph-mon[36249]: pgmap v864: 129 pgs: 129 active+clean; 449 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.8 MiB/s wr, 598 op/s 2024-05-14T09:49:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:36 smithi026 ceph-mon[36056]: pgmap v864: 129 pgs: 129 active+clean; 449 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.8 MiB/s wr, 598 op/s 2024-05-14T09:49:37.600 INFO:tasks.workunit.client.0.smithi002.stdout: 10 33082 30.99 MB/sec warmup 63 sec latency 414.363 ms 2024-05-14T09:49:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.481+0000 7f7616f1c640 1 -- 172.21.15.2:0/1968404280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7610109b80 msgr2=0x7f761010bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.481+0000 7f7616f1c640 1 --2- 172.21.15.2:0/1968404280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7610109b80 0x7f761010bfa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7608009be0 tx=0x7f760802a6c0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.481+0000 7f7616f1c640 1 -- 172.21.15.2:0/1968404280 shutdown_connections 2024-05-14T09:49:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.481+0000 7f7616f1c640 1 --2- 172.21.15.2:0/1968404280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7610109b80 0x7f761010bfa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.481+0000 7f7616f1c640 1 --2- 172.21.15.2:0/1968404280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7610104e80 0x7f7610105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.481+0000 7f7616f1c640 1 --2- 172.21.15.2:0/1968404280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7610103c40 0x7f7610104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.481+0000 7f7616f1c640 1 -- 172.21.15.2:0/1968404280 >> 172.21.15.2:0/1968404280 conn(0x7f76100fee70 msgr2=0x7f76101012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.481+0000 7f7616f1c640 1 -- 172.21.15.2:0/1968404280 shutdown_connections 2024-05-14T09:49:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.481+0000 7f7616f1c640 1 -- 172.21.15.2:0/1968404280 wait complete. 2024-05-14T09:49:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.482+0000 7f7616f1c640 1 Processor -- start 2024-05-14T09:49:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.482+0000 7f7616f1c640 1 -- start start 2024-05-14T09:49:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.482+0000 7f7616f1c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7610103c40 0x7f7610198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.482+0000 7f7616f1c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7610104e80 0x7f7610198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.482+0000 7f7616f1c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7610109b80 0x7f76101951d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.482+0000 7f7616f1c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f761010f770 con 0x7f7610103c40 2024-05-14T09:49:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.482+0000 7f7616f1c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f761010f5f0 con 0x7f7610109b80 2024-05-14T09:49:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.482+0000 7f7616f1c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f761010f8f0 con 0x7f7610104e80 2024-05-14T09:49:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.482+0000 7f7614cb8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7610103c40 0x7f7610198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.483+0000 7f7614cb8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7610103c40 0x7f7610198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60654/0 (socket says 172.21.15.2:60654) 2024-05-14T09:49:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.483+0000 7f7614cb8640 1 -- 172.21.15.2:0/4005921514 learned_addr learned my addr 172.21.15.2:0/4005921514 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.483+0000 7f760ffff640 1 --2- 172.21.15.2:0/4005921514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7610104e80 0x7f7610198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.483+0000 7f76154b9640 1 --2- 172.21.15.2:0/4005921514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7610109b80 0x7f76101951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.483+0000 7f760ffff640 1 -- 172.21.15.2:0/4005921514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7610109b80 msgr2=0x7f76101951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.483+0000 7f760ffff640 1 --2- 172.21.15.2:0/4005921514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7610109b80 0x7f76101951d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.483+0000 7f760ffff640 1 -- 172.21.15.2:0/4005921514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7610103c40 msgr2=0x7f7610198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.483+0000 7f760ffff640 1 --2- 172.21.15.2:0/4005921514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7610103c40 0x7f7610198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.483+0000 7f760ffff640 1 -- 172.21.15.2:0/4005921514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76101959f0 con 0x7f7610104e80 2024-05-14T09:49:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.483+0000 7f7614cb8640 1 --2- 172.21.15.2:0/4005921514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7610103c40 0x7f7610198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:49:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.483+0000 7f760ffff640 1 --2- 172.21.15.2:0/4005921514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7610104e80 0x7f7610198bc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f75fc00e9f0 tx=0x7f75fc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.483+0000 7f760dffb640 1 -- 172.21.15.2:0/4005921514 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc00ccc0 con 0x7f7610104e80 2024-05-14T09:49:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.484+0000 7f760dffb640 1 -- 172.21.15.2:0/4005921514 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75fc005020 con 0x7f7610104e80 2024-05-14T09:49:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.484+0000 7f7616f1c640 1 -- 172.21.15.2:0/4005921514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7610195ce0 con 0x7f7610104e80 2024-05-14T09:49:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.484+0000 7f760dffb640 1 -- 172.21.15.2:0/4005921514 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc004a00 con 0x7f7610104e80 2024-05-14T09:49:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.484+0000 7f7616f1c640 1 -- 172.21.15.2:0/4005921514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76101a8ba0 con 0x7f7610104e80 2024-05-14T09:49:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.487+0000 7f760dffb640 1 -- 172.21.15.2:0/4005921514 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f75fc004ba0 con 0x7f7610104e80 2024-05-14T09:49:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.488+0000 7f760dffb640 1 --2- 172.21.15.2:0/4005921514 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f76000b2920 0x7f76000b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.488+0000 7f761ddcc740 1 Processor -- start 2024-05-14T09:49:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.488+0000 7f7614cb8640 1 --2- 172.21.15.2:0/4005921514 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f76000b2920 0x7f76000b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.488+0000 7f760dffb640 1 -- 172.21.15.2:0/4005921514 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f75fc0e24f0 con 0x7f7610104e80 2024-05-14T09:49:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.488+0000 7f761ddcc740 1 -- start start 2024-05-14T09:49:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.488+0000 7f761ddcc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e49ab1b0 0x5583e4d4ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.488+0000 7f761ddcc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49822f0 0x5583e4d4cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.488+0000 7f761ddcc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e498a4d0 0x5583e4d53ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f761ddcc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5583e4cb15d0 con 0x5583e49822f0 2024-05-14T09:49:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f761ddcc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5583e4c6a890 con 0x5583e498a4d0 2024-05-14T09:49:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f761ddcc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5583e4cad590 con 0x5583e49ab1b0 2024-05-14T09:49:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f7614cb8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e49ab1b0 0x5583e4d4ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f7614cb8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e49ab1b0 0x5583e4d4ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:53166/0 (socket says 172.21.15.2:53166) 2024-05-14T09:49:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f7614cb8640 1 -- 172.21.15.2:0/1413137676 learned_addr learned my addr 172.21.15.2:0/1413137676 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f760ffff640 1 --2- 172.21.15.2:0/1413137676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49822f0 0x5583e4d4cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f7614cb8640 1 --2- 172.21.15.2:0/4005921514 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f76000b2920 0x7f76000b4e30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7604005c80 tx=0x7f7604009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f76154b9640 1 --2- 172.21.15.2:0/1413137676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e498a4d0 0x5583e4d53ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f7614cb8640 1 -- 172.21.15.2:0/1413137676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e498a4d0 msgr2=0x5583e4d53ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f7614cb8640 1 --2- 172.21.15.2:0/1413137676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e498a4d0 0x5583e4d53ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f7614cb8640 1 -- 172.21.15.2:0/1413137676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49822f0 msgr2=0x5583e4d4cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f7614cb8640 1 --2- 172.21.15.2:0/1413137676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49822f0 0x5583e4d4cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f7614cb8640 1 -- 172.21.15.2:0/1413137676 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583e49c3210 con 0x5583e49ab1b0 2024-05-14T09:49:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f7614cb8640 1 --2- 172.21.15.2:0/1413137676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e49ab1b0 0x5583e4d4ca60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f760400f700 tx=0x7f7604019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82996409be8a18c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.489+0000 7f75faffd640 1 -- 172.21.15.2:0/1413137676 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760402a070 con 0x5583e49ab1b0 2024-05-14T09:49:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.490+0000 7f75faffd640 1 -- 172.21.15.2:0/1413137676 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f760401dda0 con 0x5583e49ab1b0 2024-05-14T09:49:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.490+0000 7f75faffd640 1 -- 172.21.15.2:0/1413137676 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76040253f0 con 0x5583e49ab1b0 2024-05-14T09:49:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.491+0000 7f761ddcc740 1 -- 172.21.15.2:0/1413137676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e49ab1b0 msgr2=0x5583e4d4ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.491+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1413137676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e49ab1b0 0x5583e4d4ca60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f760400f700 tx=0x7f7604019000 comp rx=0 tx=0).stop 2024-05-14T09:49:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.491+0000 7f761ddcc740 1 -- 172.21.15.2:0/1413137676 shutdown_connections 2024-05-14T09:49:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.491+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1413137676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e498a4d0 0x5583e4d53ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.491+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1413137676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49822f0 0x5583e4d4cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.491+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1413137676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e49ab1b0 0x5583e4d4ca60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.491+0000 7f761ddcc740 1 -- 172.21.15.2:0/1413137676 >> 172.21.15.2:0/1413137676 conn(0x5583e499f300 msgr2=0x5583e4d483a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.491+0000 7f761ddcc740 1 -- 172.21.15.2:0/1413137676 shutdown_connections 2024-05-14T09:49:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.491+0000 7f761ddcc740 1 -- 172.21.15.2:0/1413137676 wait complete. 2024-05-14T09:49:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.491+0000 7f761ddcc740 1 -- start start 2024-05-14T09:49:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.492+0000 7f761ddcc740 1 Processor -- start 2024-05-14T09:49:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.492+0000 7f761ddcc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e49822f0 0x5583e4d56620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.492+0000 7f761ddcc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e498a4d0 0x5583e4d56b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.492+0000 7f761ddcc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e49ab1b0 0x5583e4d5dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.492+0000 7f761ddcc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5583e4d549d0 con 0x5583e498a4d0 2024-05-14T09:49:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.492+0000 7f761ddcc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5583e4c6e370 con 0x5583e49822f0 2024-05-14T09:49:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.492+0000 7f761ddcc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5583e4d54b50 con 0x5583e49ab1b0 2024-05-14T09:49:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.492+0000 7f761ddcc740 10 client.0 _ll_register_callbacks cb 0x5583e49c2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:49:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.492+0000 7f760ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e498a4d0 0x5583e4d56b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.492+0000 7f76154b9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e49ab1b0 0x5583e4d5dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.492+0000 7f7614cb8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e49822f0 0x5583e4d56620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.492+0000 7f760ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e498a4d0 0x5583e4d56b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60670/0 (socket says 172.21.15.2:60670) 2024-05-14T09:49:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.492+0000 7f76154b9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e49ab1b0 0x5583e4d5dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:53170/0 (socket says 172.21.15.2:53170) 2024-05-14T09:49:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.492+0000 7f760ffff640 1 -- 172.21.15.2:0/1483230931 learned_addr learned my addr 172.21.15.2:0/1483230931 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.493+0000 7f760ffff640 1 -- 172.21.15.2:0/1483230931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e49ab1b0 msgr2=0x5583e4d5dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.493+0000 7f760ffff640 1 --2- 172.21.15.2:0/1483230931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e49ab1b0 0x5583e4d5dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.493+0000 7f760ffff640 1 -- 172.21.15.2:0/1483230931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e49822f0 msgr2=0x5583e4d56620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.493+0000 7f760ffff640 1 --2- 172.21.15.2:0/1483230931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e49822f0 0x5583e4d56620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.493+0000 7f760ffff640 1 -- 172.21.15.2:0/1483230931 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583e4a8f620 con 0x5583e498a4d0 2024-05-14T09:49:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.493+0000 7f760ffff640 1 --2- 172.21.15.2:0/1483230931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e498a4d0 0x5583e4d56b60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0dff40 tx=0x7f75fc0a8f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2633809365af0df9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.493+0000 7f75effff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:49:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.493+0000 7f75effff640 1 -- 172.21.15.2:0/1483230931 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc0c3060 con 0x5583e498a4d0 2024-05-14T09:49:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.493+0000 7f75effff640 1 -- 172.21.15.2:0/1483230931 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f75fc0e2170 con 0x5583e498a4d0 2024-05-14T09:49:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.493+0000 7f761ddcc740 1 -- 172.21.15.2:0/1483230931 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f760401dda0 con 0x5583e498a4d0 2024-05-14T09:49:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.494+0000 7f75effff640 1 -- 172.21.15.2:0/1483230931 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f75fc0c8050 con 0x5583e498a4d0 2024-05-14T09:49:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.494+0000 7f761ddcc740 10 client.16929 fetch_fsmap learned FSMap version 54 2024-05-14T09:49:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.494+0000 7f761ddcc740 1 -- 172.21.15.2:0/1483230931 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5583e4d60aa0 con 0x5583e498a4d0 2024-05-14T09:49:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.494+0000 7f75effff640 1 -- 172.21.15.2:0/1483230931 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f760401dda0 con 0x5583e498a4d0 2024-05-14T09:49:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.494+0000 7f75effff640 1 -- 172.21.15.2:0/1483230931 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f75fc091150 con 0x5583e498a4d0 2024-05-14T09:49:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.494+0000 7f761ddcc740 10 client.16929 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T09:49:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.494+0000 7f761ddcc740 10 client.16929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:49:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.494+0000 7f761ddcc740 20 client.16929 populate_metadata read hostname 'smithi002' 2024-05-14T09:49:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.494+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1483230931 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5583e4969d80 0x5583e4d641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.494+0000 7f761ddcc740 4 client.16929 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:49:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.494+0000 7f761ddcc740 1 -- 172.21.15.2:0/1483230931 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583e4cbf1e0 con 0x5583e4969d80 2024-05-14T09:49:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.494+0000 7f7614cb8640 1 --2- 172.21.15.2:0/1483230931 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5583e4969d80 0x5583e4d641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.495+0000 7f75effff640 10 client.16929 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.495+0000 7f7614cb8640 1 --2- 172.21.15.2:0/1483230931 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5583e4969d80 0x5583e4d641c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eff79bad1259b28 server_cookie=ce66caeec39218a5 in_seq=0 out_seq=0 2024-05-14T09:49:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.497+0000 7f75effff640 1 -- 172.21.15.2:0/1483230931 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583e4cbf1e0 con 0x5583e4969d80 2024-05-14T09:49:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.497+0000 7f75effff640 10 client.16929 handle_command_reply: tid=0 2024-05-14T09:49:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.498+0000 7f761ddcc740 1 client.16929 shutdown 2024-05-14T09:49:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.498+0000 7f761ddcc740 1 -- 172.21.15.2:0/1483230931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e498a4d0 msgr2=0x5583e4d56b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.498+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1483230931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e498a4d0 0x5583e4d56b60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0dff40 tx=0x7f75fc0a8f50 comp rx=0 tx=0).stop 2024-05-14T09:49:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.498+0000 7f761ddcc740 1 -- 172.21.15.2:0/1483230931 shutdown_connections 2024-05-14T09:49:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.498+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1483230931 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5583e4969d80 0x5583e4d641c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.498+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1483230931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e49ab1b0 0x5583e4d5dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.498+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1483230931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e498a4d0 0x5583e4d56b60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.498+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1483230931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e49822f0 0x5583e4d56620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.498+0000 7f761ddcc740 1 -- 172.21.15.2:0/1483230931 >> 172.21.15.2:0/1483230931 conn(0x5583e499f300 msgr2=0x5583e4d4b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.498+0000 7f75effff640 0 client.16929 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.499+0000 7f761ddcc740 1 -- 172.21.15.2:0/1483230931 shutdown_connections 2024-05-14T09:49:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.499+0000 7f761ddcc740 1 -- 172.21.15.2:0/1483230931 wait complete. 2024-05-14T09:49:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.499+0000 7f761ddcc740 20 client.16929 trim_cache size 0 max 16384 2024-05-14T09:49:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.519+0000 7f761ddcc740 1 Processor -- start 2024-05-14T09:49:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.519+0000 7f761ddcc740 1 -- start start 2024-05-14T09:49:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f761ddcc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e49822f0 0x5583e4d47d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f761ddcc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e498a4d0 0x5583e4d5d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f761ddcc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49ab1b0 0x5583e4d482a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f761ddcc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f76101b5c00 con 0x5583e49ab1b0 2024-05-14T09:49:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f761ddcc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76101b3440 con 0x5583e49822f0 2024-05-14T09:49:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f761ddcc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f76101b1290 con 0x5583e498a4d0 2024-05-14T09:49:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f76154b9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49ab1b0 0x5583e4d482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f760ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e498a4d0 0x5583e4d5d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f7614cb8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e49822f0 0x5583e4d47d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f76154b9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49ab1b0 0x5583e4d482a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60676/0 (socket says 172.21.15.2:60676) 2024-05-14T09:49:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f76154b9640 1 -- 172.21.15.2:0/1206115355 learned_addr learned my addr 172.21.15.2:0/1206115355 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f76154b9640 1 -- 172.21.15.2:0/1206115355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e498a4d0 msgr2=0x5583e4d5d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f76154b9640 1 --2- 172.21.15.2:0/1206115355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e498a4d0 0x5583e4d5d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f76154b9640 1 -- 172.21.15.2:0/1206115355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e49822f0 msgr2=0x5583e4d47d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f76154b9640 1 --2- 172.21.15.2:0/1206115355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e49822f0 0x5583e4d47d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.520+0000 7f76154b9640 1 -- 172.21.15.2:0/1206115355 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583e4d48d00 con 0x5583e49ab1b0 2024-05-14T09:49:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.521+0000 7f76154b9640 1 --2- 172.21.15.2:0/1206115355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49ab1b0 0x5583e4d482a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f760802acb0 tx=0x7f76080093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84193fbf1b0d09 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.521+0000 7f75fa7fc640 1 -- 172.21.15.2:0/1206115355 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608036040 con 0x5583e49ab1b0 2024-05-14T09:49:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.521+0000 7f75fa7fc640 1 -- 172.21.15.2:0/1206115355 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7608003e20 con 0x5583e49ab1b0 2024-05-14T09:49:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.521+0000 7f75fa7fc640 1 -- 172.21.15.2:0/1206115355 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76080045d0 con 0x5583e49ab1b0 2024-05-14T09:49:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.522+0000 7f761ddcc740 1 -- 172.21.15.2:0/1206115355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49ab1b0 msgr2=0x5583e4d482a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.522+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1206115355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49ab1b0 0x5583e4d482a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f760802acb0 tx=0x7f76080093e0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.522+0000 7f761ddcc740 1 -- 172.21.15.2:0/1206115355 shutdown_connections 2024-05-14T09:49:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.522+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1206115355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49ab1b0 0x5583e4d482a0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.522+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1206115355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e498a4d0 0x5583e4d5d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.522+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1206115355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e49822f0 0x5583e4d47d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.522+0000 7f761ddcc740 1 -- 172.21.15.2:0/1206115355 >> 172.21.15.2:0/1206115355 conn(0x5583e499f300 msgr2=0x5583e4d50960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.522+0000 7f761ddcc740 1 -- 172.21.15.2:0/1206115355 shutdown_connections 2024-05-14T09:49:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.522+0000 7f761ddcc740 1 -- 172.21.15.2:0/1206115355 wait complete. 2024-05-14T09:49:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.522+0000 7f761ddcc740 1 -- start start 2024-05-14T09:49:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.522+0000 7f761ddcc740 1 Processor -- start 2024-05-14T09:49:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f761ddcc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49822f0 0x5583e4d5ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f761ddcc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e498a4d0 0x5583e4d5b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f761ddcc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e49ab1b0 0x5583e4d5c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f761ddcc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f76101b1900 con 0x5583e49822f0 2024-05-14T09:49:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f761ddcc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76101b2d70 con 0x5583e49ab1b0 2024-05-14T09:49:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f761ddcc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f76101b1fd0 con 0x5583e498a4d0 2024-05-14T09:49:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f761ddcc740 10 client.0 _ll_register_callbacks cb 0x7f75fc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:49:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f7614cb8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49822f0 0x5583e4d5ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f7614cb8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49822f0 0x5583e4d5ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60690/0 (socket says 172.21.15.2:60690) 2024-05-14T09:49:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f7614cb8640 1 -- 172.21.15.2:0/1805340913 learned_addr learned my addr 172.21.15.2:0/1805340913 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f760ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e498a4d0 0x5583e4d5b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f76154b9640 1 --2- 172.21.15.2:0/1805340913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e49ab1b0 0x5583e4d5c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f7614cb8640 1 -- 172.21.15.2:0/1805340913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e498a4d0 msgr2=0x5583e4d5b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f7614cb8640 1 --2- 172.21.15.2:0/1805340913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e498a4d0 0x5583e4d5b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f7614cb8640 1 -- 172.21.15.2:0/1805340913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e49ab1b0 msgr2=0x5583e4d5c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f7614cb8640 1 --2- 172.21.15.2:0/1805340913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e49ab1b0 0x5583e4d5c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.523+0000 7f7614cb8640 1 -- 172.21.15.2:0/1805340913 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583e4d5c750 con 0x5583e49822f0 2024-05-14T09:49:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.524+0000 7f7614cb8640 1 --2- 172.21.15.2:0/1805340913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49822f0 0x5583e4d5ad20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7604013010 tx=0x7f7604005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be533155164dd3a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.524+0000 7f75effff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:49:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.524+0000 7f75effff640 1 -- 172.21.15.2:0/1805340913 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760402a040 con 0x5583e49822f0 2024-05-14T09:49:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.524+0000 7f75effff640 1 -- 172.21.15.2:0/1805340913 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f760401b460 con 0x5583e49822f0 2024-05-14T09:49:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.524+0000 7f761ddcc740 1 -- 172.21.15.2:0/1805340913 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7608003e20 con 0x5583e49822f0 2024-05-14T09:49:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.524+0000 7f75effff640 1 -- 172.21.15.2:0/1805340913 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f760401b660 con 0x5583e49822f0 2024-05-14T09:49:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.524+0000 7f761ddcc740 10 client.16935 fetch_fsmap learned FSMap version 54 2024-05-14T09:49:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.524+0000 7f761ddcc740 1 -- 172.21.15.2:0/1805340913 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5583e4d5da50 con 0x5583e49822f0 2024-05-14T09:49:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.525+0000 7f75effff640 1 -- 172.21.15.2:0/1805340913 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7608003e20 con 0x5583e49822f0 2024-05-14T09:49:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.525+0000 7f75effff640 1 -- 172.21.15.2:0/1805340913 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f7604024030 con 0x5583e49822f0 2024-05-14T09:49:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.525+0000 7f761ddcc740 10 client.16935 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T09:49:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.525+0000 7f761ddcc740 10 client.16935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:49:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.525+0000 7f761ddcc740 20 client.16935 populate_metadata read hostname 'smithi002' 2024-05-14T09:49:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.525+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1805340913 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5583e493f1c0 0x5583e4dce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.525+0000 7f761ddcc740 4 client.16935 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:49:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.525+0000 7f761ddcc740 1 -- 172.21.15.2:0/1805340913 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5583e4cb9050 con 0x5583e493f1c0 2024-05-14T09:49:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.525+0000 7f760ffff640 1 --2- 172.21.15.2:0/1805340913 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5583e493f1c0 0x5583e4dce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.526+0000 7f75effff640 10 client.16935 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.526+0000 7f760ffff640 1 --2- 172.21.15.2:0/1805340913 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5583e493f1c0 0x5583e4dce830 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b24aad77c05d50c3 server_cookie=4583684b565bd494 in_seq=0 out_seq=0 2024-05-14T09:49:38.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 33875 31.22 MB/sec warmup 64 sec latency 85.242 ms 2024-05-14T09:49:38.738 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:38 smithi029 ceph-mon[36249]: pgmap v865: 129 pgs: 129 active+clean; 451 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 713 op/s 2024-05-14T09:49:38.752 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:38 smithi026 ceph-mon[36056]: pgmap v865: 129 pgs: 129 active+clean; 451 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 713 op/s 2024-05-14T09:49:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:38 smithi002 ceph-mon[31557]: pgmap v865: 129 pgs: 129 active+clean; 451 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 713 op/s 2024-05-14T09:49:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.922+0000 7f75effff640 1 -- 172.21.15.2:0/1805340913 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5583e4cb9050 con 0x5583e493f1c0 2024-05-14T09:49:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.922+0000 7f75effff640 10 client.16935 handle_command_reply: tid=0 2024-05-14T09:49:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.922+0000 7f761ddcc740 1 client.16935 shutdown 2024-05-14T09:49:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.923+0000 7f761ddcc740 1 -- 172.21.15.2:0/1805340913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49822f0 msgr2=0x5583e4d5ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.923+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1805340913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49822f0 0x5583e4d5ad20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7604013010 tx=0x7f7604005e10 comp rx=0 tx=0).stop 2024-05-14T09:49:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.923+0000 7f761ddcc740 1 -- 172.21.15.2:0/1805340913 shutdown_connections 2024-05-14T09:49:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.923+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1805340913 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5583e493f1c0 0x5583e4dce830 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.923+0000 7f75effff640 0 client.16935 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.923+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1805340913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e49ab1b0 0x5583e4d5c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.923+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1805340913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5583e498a4d0 0x5583e4d5b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.923+0000 7f761ddcc740 1 --2- 172.21.15.2:0/1805340913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583e49822f0 0x5583e4d5ad20 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.923+0000 7f761ddcc740 1 -- 172.21.15.2:0/1805340913 >> 172.21.15.2:0/1805340913 conn(0x5583e499f300 msgr2=0x5583e4d47d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.923+0000 7f761ddcc740 1 -- 172.21.15.2:0/1805340913 shutdown_connections 2024-05-14T09:49:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.924+0000 7f761ddcc740 1 -- 172.21.15.2:0/1805340913 wait complete. 2024-05-14T09:49:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.924+0000 7f761ddcc740 20 client.16935 trim_cache size 0 max 16384 2024-05-14T09:49:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:49:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:49:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:49:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:49:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.925+0000 7f7616f1c640 1 -- 172.21.15.2:0/4005921514 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f76000b2920 msgr2=0x7f76000b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.925+0000 7f7616f1c640 1 --2- 172.21.15.2:0/4005921514 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f76000b2920 0x7f76000b4e30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7604005c80 tx=0x7f7604009260 comp rx=0 tx=0).stop 2024-05-14T09:49:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.925+0000 7f7616f1c640 1 -- 172.21.15.2:0/4005921514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7610104e80 msgr2=0x7f7610198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.925+0000 7f7616f1c640 1 --2- 172.21.15.2:0/4005921514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7610104e80 0x7f7610198bc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f75fc00e9f0 tx=0x7f75fc00eeb0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.925+0000 7f7616f1c640 1 -- 172.21.15.2:0/4005921514 shutdown_connections 2024-05-14T09:49:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.925+0000 7f7616f1c640 1 --2- 172.21.15.2:0/4005921514 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f76000b2920 0x7f76000b4e30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.925+0000 7f7616f1c640 1 --2- 172.21.15.2:0/4005921514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7610109b80 0x7f76101951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.925+0000 7f7616f1c640 1 --2- 172.21.15.2:0/4005921514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7610104e80 0x7f7610198bc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.925+0000 7f7616f1c640 1 --2- 172.21.15.2:0/4005921514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7610103c40 0x7f7610198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.925+0000 7f7616f1c640 1 -- 172.21.15.2:0/4005921514 >> 172.21.15.2:0/4005921514 conn(0x7f76100fee70 msgr2=0x7f76100ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.927+0000 7f7616f1c640 1 -- 172.21.15.2:0/4005921514 shutdown_connections 2024-05-14T09:49:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:38.927+0000 7f7616f1c640 1 -- 172.21.15.2:0/4005921514 wait complete. 2024-05-14T09:49:39.601 INFO:tasks.workunit.client.0.smithi002.stdout: 10 34449 31.11 MB/sec warmup 65 sec latency 309.565 ms 2024-05-14T09:49:40.601 INFO:tasks.workunit.client.0.smithi002.stdout: 10 34978 31.18 MB/sec warmup 66 sec latency 397.034 ms 2024-05-14T09:49:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:40 smithi002 ceph-mon[31557]: pgmap v866: 129 pgs: 129 active+clean; 460 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.5 MiB/s wr, 690 op/s 2024-05-14T09:49:40.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:40 smithi029 ceph-mon[36249]: pgmap v866: 129 pgs: 129 active+clean; 460 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.5 MiB/s wr, 690 op/s 2024-05-14T09:49:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:40 smithi026 ceph-mon[36056]: pgmap v866: 129 pgs: 129 active+clean; 460 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.5 MiB/s wr, 690 op/s 2024-05-14T09:49:41.601 INFO:tasks.workunit.client.0.smithi002.stdout: 10 35361 31.07 MB/sec warmup 67 sec latency 409.711 ms 2024-05-14T09:49:42.601 INFO:tasks.workunit.client.0.smithi002.stdout: 10 35938 31.08 MB/sec warmup 68 sec latency 81.300 ms 2024-05-14T09:49:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:42 smithi002 ceph-mon[31557]: pgmap v867: 129 pgs: 129 active+clean; 461 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.8 MiB/s wr, 571 op/s 2024-05-14T09:49:42.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:42 smithi029 ceph-mon[36249]: pgmap v867: 129 pgs: 129 active+clean; 461 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.8 MiB/s wr, 571 op/s 2024-05-14T09:49:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:42 smithi026 ceph-mon[36056]: pgmap v867: 129 pgs: 129 active+clean; 461 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.8 MiB/s wr, 571 op/s 2024-05-14T09:49:43.601 INFO:tasks.workunit.client.0.smithi002.stdout: 10 36653 31.20 MB/sec warmup 69 sec latency 46.821 ms 2024-05-14T09:49:44.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 37374 31.22 MB/sec warmup 70 sec latency 257.588 ms 2024-05-14T09:49:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:44 smithi002 ceph-mon[31557]: pgmap v868: 129 pgs: 129 active+clean; 459 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 834 op/s 2024-05-14T09:49:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:44 smithi029 ceph-mon[36249]: pgmap v868: 129 pgs: 129 active+clean; 459 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 834 op/s 2024-05-14T09:49:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:44 smithi026 ceph-mon[36056]: pgmap v868: 129 pgs: 129 active+clean; 459 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 834 op/s 2024-05-14T09:49:45.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 38042 31.33 MB/sec warmup 71 sec latency 290.092 ms 2024-05-14T09:49:46.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 38582 31.32 MB/sec warmup 72 sec latency 357.942 ms 2024-05-14T09:49:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:46 smithi002 ceph-mon[31557]: pgmap v869: 129 pgs: 129 active+clean; 459 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.4 MiB/s wr, 663 op/s 2024-05-14T09:49:46.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:46 smithi029 ceph-mon[36249]: pgmap v869: 129 pgs: 129 active+clean; 459 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.4 MiB/s wr, 663 op/s 2024-05-14T09:49:46.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:46 smithi026 ceph-mon[36056]: pgmap v869: 129 pgs: 129 active+clean; 459 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.4 MiB/s wr, 663 op/s 2024-05-14T09:49:47.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 39101 31.23 MB/sec warmup 73 sec latency 96.047 ms 2024-05-14T09:49:48.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 39908 31.35 MB/sec warmup 74 sec latency 52.537 ms 2024-05-14T09:49:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:48 smithi002 ceph-mon[31557]: pgmap v870: 129 pgs: 129 active+clean; 464 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 783 op/s 2024-05-14T09:49:48.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:48 smithi029 ceph-mon[36249]: pgmap v870: 129 pgs: 129 active+clean; 464 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 783 op/s 2024-05-14T09:49:48.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:48 smithi026 ceph-mon[36056]: pgmap v870: 129 pgs: 129 active+clean; 464 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 783 op/s 2024-05-14T09:49:49.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 40736 31.63 MB/sec warmup 75 sec latency 60.510 ms 2024-05-14T09:49:50.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 41420 31.70 MB/sec warmup 76 sec latency 167.094 ms 2024-05-14T09:49:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:50 smithi002 ceph-mon[31557]: pgmap v871: 129 pgs: 129 active+clean; 473 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 826 op/s 2024-05-14T09:49:50.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:50 smithi029 ceph-mon[36249]: pgmap v871: 129 pgs: 129 active+clean; 473 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 826 op/s 2024-05-14T09:49:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:50 smithi026 ceph-mon[36056]: pgmap v871: 129 pgs: 129 active+clean; 473 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 826 op/s 2024-05-14T09:49:51.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 41772 31.52 MB/sec warmup 77 sec latency 308.776 ms 2024-05-14T09:49:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.105+0000 7f2199d1c640 1 -- 172.21.15.2:0/3845627149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2194104e90 msgr2=0x7f21941052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.105+0000 7f2199d1c640 1 --2- 172.21.15.2:0/3845627149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2194104e90 0x7f21941052a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2188013f60 tx=0x7f21880328b0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.105+0000 7f2199d1c640 1 -- 172.21.15.2:0/3845627149 shutdown_connections 2024-05-14T09:49:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.105+0000 7f2199d1c640 1 --2- 172.21.15.2:0/3845627149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2194109b90 0x7f219410bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.105+0000 7f2199d1c640 1 --2- 172.21.15.2:0/3845627149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2194104e90 0x7f21941052a0 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2188013f60 tx=0x7f21880328b0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.105+0000 7f2199d1c640 1 --2- 172.21.15.2:0/3845627149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2194103c50 0x7f2194104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.105+0000 7f2199d1c640 1 -- 172.21.15.2:0/3845627149 >> 172.21.15.2:0/3845627149 conn(0x7f21940fee60 msgr2=0x7f21941012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.105+0000 7f2199d1c640 1 -- 172.21.15.2:0/3845627149 shutdown_connections 2024-05-14T09:49:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.105+0000 7f2199d1c640 1 -- 172.21.15.2:0/3845627149 wait complete. 2024-05-14T09:49:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.106+0000 7f2199d1c640 1 Processor -- start 2024-05-14T09:49:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.106+0000 7f2199d1c640 1 -- start start 2024-05-14T09:49:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.106+0000 7f2199d1c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2194103c50 0x7f2194076730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.106+0000 7f2199d1c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2194109b90 0x7f2194074dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.106+0000 7f2199d1c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2194075310 0x7f2194075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.107+0000 7f2199d1c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f219406f180 con 0x7f2194109b90 2024-05-14T09:49:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.107+0000 7f2199d1c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f219406f000 con 0x7f2194103c50 2024-05-14T09:49:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.107+0000 7f2199d1c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f219406f300 con 0x7f2194075310 2024-05-14T09:49:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.107+0000 7f2192ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2194109b90 0x7f2194074dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.107+0000 7f2192ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2194109b90 0x7f2194074dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39126/0 (socket says 172.21.15.2:39126) 2024-05-14T09:49:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.107+0000 7f2192ffd640 1 -- 172.21.15.2:0/1742087439 learned_addr learned my addr 172.21.15.2:0/1742087439 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.107+0000 7f21937fe640 1 --2- 172.21.15.2:0/1742087439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2194103c50 0x7f2194076730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.107+0000 7f2193fff640 1 --2- 172.21.15.2:0/1742087439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2194075310 0x7f2194075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.108+0000 7f2192ffd640 1 -- 172.21.15.2:0/1742087439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2194075310 msgr2=0x7f2194075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.108+0000 7f2192ffd640 1 --2- 172.21.15.2:0/1742087439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2194075310 0x7f2194075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.108+0000 7f2192ffd640 1 -- 172.21.15.2:0/1742087439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2194103c50 msgr2=0x7f2194076730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.108+0000 7f2192ffd640 1 --2- 172.21.15.2:0/1742087439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2194103c50 0x7f2194076730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.108+0000 7f2192ffd640 1 -- 172.21.15.2:0/1742087439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2194197a70 con 0x7f2194109b90 2024-05-14T09:49:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.108+0000 7f2192ffd640 1 --2- 172.21.15.2:0/1742087439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2194109b90 0x7f2194074dd0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f218803d000 tx=0x7f2188038d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.110+0000 7f2190ff9640 1 -- 172.21.15.2:0/1742087439 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188005550 con 0x7f2194109b90 2024-05-14T09:49:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.110+0000 7f2190ff9640 1 -- 172.21.15.2:0/1742087439 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2188048030 con 0x7f2194109b90 2024-05-14T09:49:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.110+0000 7f2190ff9640 1 -- 172.21.15.2:0/1742087439 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188039940 con 0x7f2194109b90 2024-05-14T09:49:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.110+0000 7f2199d1c640 1 -- 172.21.15.2:0/1742087439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2194197c40 con 0x7f2194109b90 2024-05-14T09:49:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.110+0000 7f2199d1c640 1 -- 172.21.15.2:0/1742087439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2194198110 con 0x7f2194109b90 2024-05-14T09:49:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.115+0000 7f21a0bcc740 1 Processor -- start 2024-05-14T09:49:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.118+0000 7f21a0bcc740 1 -- start start 2024-05-14T09:49:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.118+0000 7f21a0bcc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb81151b0 0x560fb84b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.118+0000 7f21a0bcc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80ec2f0 0x560fb84b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.119+0000 7f21a0bcc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80f44d0 0x560fb84bdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.119+0000 7f2190ff9640 1 -- 172.21.15.2:0/1742087439 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2188039ae0 con 0x7f2194109b90 2024-05-14T09:49:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.119+0000 7f21a0bcc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560fb841b5d0 con 0x560fb80ec2f0 2024-05-14T09:49:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.119+0000 7f21a0bcc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560fb83d4890 con 0x560fb81151b0 2024-05-14T09:49:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.119+0000 7f21a0bcc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560fb8417590 con 0x560fb80f44d0 2024-05-14T09:49:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.120+0000 7f2190ff9640 1 --2- 172.21.15.2:0/1742087439 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f217c0b2a70 0x7f217c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.120+0000 7f2193fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb81151b0 0x560fb84b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.120+0000 7f2193fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb81151b0 0x560fb84b6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:54574/0 (socket says 172.21.15.2:54574) 2024-05-14T09:49:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.120+0000 7f2193fff640 1 -- 172.21.15.2:0/237575727 learned_addr learned my addr 172.21.15.2:0/237575727 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.120+0000 7f2190ff9640 1 -- 172.21.15.2:0/1742087439 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f218810b450 con 0x7f2194109b90 2024-05-14T09:49:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.121+0000 7f2192ffd640 1 --2- 172.21.15.2:0/237575727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80f44d0 0x560fb84bdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.121+0000 7f21937fe640 1 --2- 172.21.15.2:0/237575727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80ec2f0 0x560fb84b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.121+0000 7f2193fff640 1 -- 172.21.15.2:0/237575727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80f44d0 msgr2=0x560fb84bdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.121+0000 7f2193fff640 1 --2- 172.21.15.2:0/237575727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80f44d0 0x560fb84bdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.121+0000 7f2193fff640 1 -- 172.21.15.2:0/237575727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80ec2f0 msgr2=0x560fb84b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.121+0000 7f2193fff640 1 --2- 172.21.15.2:0/237575727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80ec2f0 0x560fb84b6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.121+0000 7f2193fff640 1 -- 172.21.15.2:0/237575727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb812d210 con 0x560fb81151b0 2024-05-14T09:49:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.121+0000 7f2193fff640 1 --2- 172.21.15.2:0/1742087439 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f217c0b2a70 0x7f217c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.124+0000 7f2193fff640 1 --2- 172.21.15.2:0/237575727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb81151b0 0x560fb84b6a60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f218400de60 tx=0x7f218400b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb3bc30894177d62 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.125+0000 7f2179ffb640 1 -- 172.21.15.2:0/237575727 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2184019cf0 con 0x560fb81151b0 2024-05-14T09:49:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.126+0000 7f2193fff640 1 --2- 172.21.15.2:0/1742087439 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f217c0b2a70 0x7f217c0b4f80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2184018000 tx=0x7f218401a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.126+0000 7f2179ffb640 1 -- 172.21.15.2:0/237575727 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21840238a0 con 0x560fb81151b0 2024-05-14T09:49:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.127+0000 7f21a0bcc740 1 -- 172.21.15.2:0/237575727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb81151b0 msgr2=0x560fb84b6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.127+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/237575727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb81151b0 0x560fb84b6a60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f218400de60 tx=0x7f218400b630 comp rx=0 tx=0).stop 2024-05-14T09:49:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.129+0000 7f21a0bcc740 1 -- 172.21.15.2:0/237575727 shutdown_connections 2024-05-14T09:49:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.129+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/237575727 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80f44d0 0x560fb84bdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.129+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/237575727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80ec2f0 0x560fb84b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.129+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/237575727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb81151b0 0x560fb84b6a60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.129+0000 7f21a0bcc740 1 -- 172.21.15.2:0/237575727 >> 172.21.15.2:0/237575727 conn(0x560fb8109300 msgr2=0x560fb84b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.130+0000 7f21a0bcc740 1 -- 172.21.15.2:0/237575727 shutdown_connections 2024-05-14T09:49:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.131+0000 7f21a0bcc740 1 -- 172.21.15.2:0/237575727 wait complete. 2024-05-14T09:49:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.132+0000 7f21a0bcc740 1 -- start start 2024-05-14T09:49:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.132+0000 7f21a0bcc740 1 Processor -- start 2024-05-14T09:49:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.133+0000 7f21a0bcc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80ec2f0 0x560fb84d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.133+0000 7f21a0bcc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb80f44d0 0x560fb84d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.133+0000 7f21a0bcc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb81151b0 0x560fb84d8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.133+0000 7f21a0bcc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560fb84cf9e0 con 0x560fb80ec2f0 2024-05-14T09:49:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.133+0000 7f21a0bcc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560fb83d8370 con 0x560fb80f44d0 2024-05-14T09:49:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.133+0000 7f21a0bcc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560fb84cfb60 con 0x560fb81151b0 2024-05-14T09:49:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.133+0000 7f21a0bcc740 10 client.0 _ll_register_callbacks cb 0x560fb812ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:49:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.133+0000 7f21937fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80ec2f0 0x560fb84d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.133+0000 7f21937fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80ec2f0 0x560fb84d1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39142/0 (socket says 172.21.15.2:39142) 2024-05-14T09:49:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.133+0000 7f21937fe640 1 -- 172.21.15.2:0/4273382807 learned_addr learned my addr 172.21.15.2:0/4273382807 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.133+0000 7f2192ffd640 1 --2- 172.21.15.2:0/4273382807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb80f44d0 0x560fb84d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.133+0000 7f2193fff640 1 --2- 172.21.15.2:0/4273382807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb81151b0 0x560fb84d8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.134+0000 7f21937fe640 1 -- 172.21.15.2:0/4273382807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb81151b0 msgr2=0x560fb84d8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.134+0000 7f21937fe640 1 --2- 172.21.15.2:0/4273382807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb81151b0 0x560fb84d8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.134+0000 7f21937fe640 1 -- 172.21.15.2:0/4273382807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb80f44d0 msgr2=0x560fb84d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.134+0000 7f21937fe640 1 --2- 172.21.15.2:0/4273382807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb80f44d0 0x560fb84d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.134+0000 7f21937fe640 1 -- 172.21.15.2:0/4273382807 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fb81f9620 con 0x560fb80ec2f0 2024-05-14T09:49:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.134+0000 7f21937fe640 1 --2- 172.21.15.2:0/4273382807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80ec2f0 0x560fb84d1740 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f218c005fd0 tx=0x7f218c01ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770bbc67eacf706b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.135+0000 7f2172ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:49:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.135+0000 7f2172ffd640 1 -- 172.21.15.2:0/4273382807 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c01ccc0 con 0x560fb80ec2f0 2024-05-14T09:49:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.135+0000 7f21a0bcc740 1 -- 172.21.15.2:0/4273382807 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21840238a0 con 0x560fb80ec2f0 2024-05-14T09:49:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.135+0000 7f2172ffd640 1 -- 172.21.15.2:0/4273382807 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f218c003970 con 0x560fb80ec2f0 2024-05-14T09:49:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.135+0000 7f2172ffd640 1 -- 172.21.15.2:0/4273382807 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f218c035050 con 0x560fb80ec2f0 2024-05-14T09:49:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.135+0000 7f21a0bcc740 10 client.16947 fetch_fsmap learned FSMap version 54 2024-05-14T09:49:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.135+0000 7f21a0bcc740 1 -- 172.21.15.2:0/4273382807 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x560fb84dbbe0 con 0x560fb80ec2f0 2024-05-14T09:49:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.136+0000 7f2172ffd640 1 -- 172.21.15.2:0/4273382807 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21840238a0 con 0x560fb80ec2f0 2024-05-14T09:49:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.136+0000 7f2172ffd640 1 -- 172.21.15.2:0/4273382807 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f218c029070 con 0x560fb80ec2f0 2024-05-14T09:49:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.137+0000 7f21a0bcc740 10 client.16947 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T09:49:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.137+0000 7f21a0bcc740 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:49:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.137+0000 7f21a0bcc740 20 client.16947 populate_metadata read hostname 'smithi002' 2024-05-14T09:49:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.137+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/4273382807 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fb80d3d80 0x560fb84df8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.137+0000 7f21a0bcc740 4 client.16947 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:49:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.137+0000 7f21a0bcc740 1 -- 172.21.15.2:0/4273382807 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fb84291e0 con 0x560fb80d3d80 2024-05-14T09:49:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.139+0000 7f2192ffd640 1 --2- 172.21.15.2:0/4273382807 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fb80d3d80 0x560fb84df8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.148+0000 7f2192ffd640 1 --2- 172.21.15.2:0/4273382807 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fb80d3d80 0x560fb84df8b0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ab34aead72c42bf server_cookie=8c0fd17514c3c608 in_seq=0 out_seq=0 2024-05-14T09:49:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.148+0000 7f2172ffd640 10 client.16947 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.156+0000 7f2172ffd640 1 -- 172.21.15.2:0/4273382807 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560fb84291e0 con 0x560fb80d3d80 2024-05-14T09:49:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.156+0000 7f2172ffd640 10 client.16947 handle_command_reply: tid=0 2024-05-14T09:49:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.156+0000 7f21a0bcc740 1 client.16947 shutdown 2024-05-14T09:49:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.158+0000 7f21a0bcc740 1 -- 172.21.15.2:0/4273382807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80ec2f0 msgr2=0x560fb84d1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.158+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/4273382807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80ec2f0 0x560fb84d1740 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f218c005fd0 tx=0x7f218c01ef90 comp rx=0 tx=0).stop 2024-05-14T09:49:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.159+0000 7f21a0bcc740 1 -- 172.21.15.2:0/4273382807 shutdown_connections 2024-05-14T09:49:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.159+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/4273382807 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fb80d3d80 0x560fb84df8b0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.159+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/4273382807 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb81151b0 0x560fb84d8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.159+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/4273382807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb80f44d0 0x560fb84d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.159+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/4273382807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80ec2f0 0x560fb84d1740 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.159+0000 7f21a0bcc740 1 -- 172.21.15.2:0/4273382807 >> 172.21.15.2:0/4273382807 conn(0x560fb8109300 msgr2=0x560fb84b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.159+0000 7f2172ffd640 0 client.16947 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.160+0000 7f21a0bcc740 1 -- 172.21.15.2:0/4273382807 shutdown_connections 2024-05-14T09:49:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.160+0000 7f21a0bcc740 1 -- 172.21.15.2:0/4273382807 wait complete. 2024-05-14T09:49:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.160+0000 7f21a0bcc740 20 client.16947 trim_cache size 0 max 16384 2024-05-14T09:49:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.188+0000 7f21a0bcc740 1 Processor -- start 2024-05-14T09:49:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.188+0000 7f21a0bcc740 1 -- start start 2024-05-14T09:49:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.189+0000 7f21a0bcc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80ec2f0 0x560fb84d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.189+0000 7f21a0bcc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb80f44d0 0x560fb84d8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.189+0000 7f21937fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80ec2f0 0x560fb84d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.189+0000 7f21937fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80ec2f0 0x560fb84d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:47612/0 (socket says 172.21.15.2:47612) 2024-05-14T09:49:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.189+0000 7f21937fe640 1 -- 172.21.15.2:0/1409319093 learned_addr learned my addr 172.21.15.2:0/1409319093 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.189+0000 7f2192ffd640 1 --2- 172.21.15.2:0/1409319093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb80f44d0 0x560fb84d8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.190+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/1409319093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb81151b0 0x560fb84d9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.190+0000 7f21a0bcc740 1 -- 172.21.15.2:0/1409319093 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2180012830 con 0x560fb81151b0 2024-05-14T09:49:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.190+0000 7f2193fff640 1 --2- 172.21.15.2:0/1409319093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb81151b0 0x560fb84d9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.190+0000 7f21a0bcc740 1 -- 172.21.15.2:0/1409319093 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2180010070 con 0x560fb80f44d0 2024-05-14T09:49:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.190+0000 7f21a0bcc740 1 -- 172.21.15.2:0/1409319093 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f218000dec0 con 0x560fb80ec2f0 2024-05-14T09:49:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.191+0000 7f21937fe640 1 -- 172.21.15.2:0/1409319093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb80f44d0 msgr2=0x560fb84d8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.191+0000 7f21937fe640 1 --2- 172.21.15.2:0/1409319093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb80f44d0 0x560fb84d8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.191+0000 7f21937fe640 1 -- 172.21.15.2:0/1409319093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb81151b0 msgr2=0x560fb84d9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.191+0000 7f21937fe640 1 --2- 172.21.15.2:0/1409319093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb81151b0 0x560fb84d9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.191+0000 7f21937fe640 1 -- 172.21.15.2:0/1409319093 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb84d27b0 con 0x560fb80ec2f0 2024-05-14T09:49:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.191+0000 7f21937fe640 1 --2- 172.21.15.2:0/1409319093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80ec2f0 0x560fb84d06f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f218c01a9e0 tx=0x7f218c01e3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea5a1705d7d67c42 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.191+0000 7f21797fa640 1 -- 172.21.15.2:0/1409319093 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c029660 con 0x560fb80ec2f0 2024-05-14T09:49:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.192+0000 7f21797fa640 1 -- 172.21.15.2:0/1409319093 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f218c003e20 con 0x560fb80ec2f0 2024-05-14T09:49:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.192+0000 7f21797fa640 1 -- 172.21.15.2:0/1409319093 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c005310 con 0x560fb80ec2f0 2024-05-14T09:49:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.193+0000 7f21a0bcc740 1 -- 172.21.15.2:0/1409319093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80ec2f0 msgr2=0x560fb84d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.193+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/1409319093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80ec2f0 0x560fb84d06f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f218c01a9e0 tx=0x7f218c01e3f0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.193+0000 7f21a0bcc740 1 -- 172.21.15.2:0/1409319093 shutdown_connections 2024-05-14T09:49:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.193+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/1409319093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb81151b0 0x560fb84d9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.193+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/1409319093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb80f44d0 0x560fb84d8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.193+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/1409319093 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80ec2f0 0x560fb84d06f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.193+0000 7f21a0bcc740 1 -- 172.21.15.2:0/1409319093 >> 172.21.15.2:0/1409319093 conn(0x560fb8109300 msgr2=0x560fb84cffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.193+0000 7f21a0bcc740 1 -- 172.21.15.2:0/1409319093 shutdown_connections 2024-05-14T09:49:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.193+0000 7f21a0bcc740 1 -- 172.21.15.2:0/1409319093 wait complete. 2024-05-14T09:49:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.194+0000 7f21a0bcc740 1 -- start start 2024-05-14T09:49:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.194+0000 7f21a0bcc740 1 Processor -- start 2024-05-14T09:49:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.195+0000 7f21a0bcc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80ec2f0 0x560fb84d3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.195+0000 7f21a0bcc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80f44d0 0x560fb84d41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.195+0000 7f21a0bcc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb81151b0 0x560fb84d9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.195+0000 7f21a0bcc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f218000e530 con 0x560fb80f44d0 2024-05-14T09:49:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.195+0000 7f21a0bcc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f218000f9a0 con 0x560fb81151b0 2024-05-14T09:49:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.195+0000 7f21a0bcc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f218000ec00 con 0x560fb80ec2f0 2024-05-14T09:49:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.195+0000 7f21a0bcc740 10 client.0 _ll_register_callbacks cb 0x7f218c01aa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:49:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.196+0000 7f21937fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80ec2f0 0x560fb84d3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.196+0000 7f21937fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80ec2f0 0x560fb84d3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:47622/0 (socket says 172.21.15.2:47622) 2024-05-14T09:49:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.196+0000 7f21937fe640 1 -- 172.21.15.2:0/3835027368 learned_addr learned my addr 172.21.15.2:0/3835027368 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:49:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.196+0000 7f2192ffd640 1 --2- 172.21.15.2:0/3835027368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80f44d0 0x560fb84d41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.196+0000 7f21937fe640 1 -- 172.21.15.2:0/3835027368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb81151b0 msgr2=0x560fb84d9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.196+0000 7f21937fe640 1 --2- 172.21.15.2:0/3835027368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb81151b0 0x560fb84d9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.196+0000 7f21937fe640 1 -- 172.21.15.2:0/3835027368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80f44d0 msgr2=0x560fb84d41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:49:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.196+0000 7f21937fe640 1 --2- 172.21.15.2:0/3835027368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80f44d0 0x560fb84d41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.196+0000 7f21937fe640 1 -- 172.21.15.2:0/3835027368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fb84d96f0 con 0x560fb80ec2f0 2024-05-14T09:49:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.196+0000 7f2192ffd640 1 --2- 172.21.15.2:0/3835027368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80f44d0 0x560fb84d41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:49:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.196+0000 7f21937fe640 1 --2- 172.21.15.2:0/3835027368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80ec2f0 0x560fb84d3ca0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x560fb84d0fa0 tx=0x7f218c01ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d970dcb39bb3a75 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:49:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.197+0000 7f2172ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:49:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.197+0000 7f2172ffd640 1 -- 172.21.15.2:0/3835027368 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c048060 con 0x560fb80ec2f0 2024-05-14T09:49:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.197+0000 7f2172ffd640 1 -- 172.21.15.2:0/3835027368 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f218c029870 con 0x560fb80ec2f0 2024-05-14T09:49:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.197+0000 7f21a0bcc740 1 -- 172.21.15.2:0/3835027368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f218c003e20 con 0x560fb80ec2f0 2024-05-14T09:49:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.199+0000 7f2172ffd640 1 -- 172.21.15.2:0/3835027368 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f218c04f050 con 0x560fb80ec2f0 2024-05-14T09:49:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.199+0000 7f21a0bcc740 10 client.26584 fetch_fsmap learned FSMap version 54 2024-05-14T09:49:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.199+0000 7f21a0bcc740 1 -- 172.21.15.2:0/3835027368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x560fb8546bd0 con 0x560fb80ec2f0 2024-05-14T09:49:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.200+0000 7f2172ffd640 1 -- 172.21.15.2:0/3835027368 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f218c029ad0 con 0x560fb80ec2f0 2024-05-14T09:49:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.200+0000 7f2172ffd640 1 -- 172.21.15.2:0/3835027368 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f218c029050 con 0x560fb80ec2f0 2024-05-14T09:49:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.201+0000 7f21a0bcc740 10 client.26584 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T09:49:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.201+0000 7f21a0bcc740 10 client.26584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:49:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.201+0000 7f21a0bcc740 20 client.26584 populate_metadata read hostname 'smithi002' 2024-05-14T09:49:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.201+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/3835027368 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fb80a91c0 0x560fb8549540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:49:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.201+0000 7f21a0bcc740 4 client.26584 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:49:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.201+0000 7f21a0bcc740 1 -- 172.21.15.2:0/3835027368 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f218c003e20 con 0x560fb80a91c0 2024-05-14T09:49:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.201+0000 7f2192ffd640 1 --2- 172.21.15.2:0/3835027368 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fb80a91c0 0x560fb8549540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:49:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.202+0000 7f2172ffd640 10 client.26584 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.202+0000 7f2192ffd640 1 --2- 172.21.15.2:0/3835027368 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fb80a91c0 0x560fb8549540 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fd7492e1eafbe52 server_cookie=b11dddfa7b3c1cc7 in_seq=0 out_seq=0 2024-05-14T09:49:52.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 42027 31.30 MB/sec warmup 78 sec latency 352.405 ms 2024-05-14T09:49:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.805+0000 7f2172ffd640 1 -- 172.21.15.2:0/3835027368 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f218c003e20 con 0x560fb80a91c0 2024-05-14T09:49:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.806+0000 7f2172ffd640 10 client.26584 handle_command_reply: tid=0 2024-05-14T09:49:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.806+0000 7f21a0bcc740 1 client.26584 shutdown 2024-05-14T09:49:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.806+0000 7f21a0bcc740 1 -- 172.21.15.2:0/3835027368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80ec2f0 msgr2=0x560fb84d3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.806+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/3835027368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80ec2f0 0x560fb84d3ca0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x560fb84d0fa0 tx=0x7f218c01ef90 comp rx=0 tx=0).stop 2024-05-14T09:49:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.806+0000 7f21a0bcc740 1 -- 172.21.15.2:0/3835027368 shutdown_connections 2024-05-14T09:49:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.806+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/3835027368 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x560fb80a91c0 0x560fb8549540 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.807+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/3835027368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fb81151b0 0x560fb84d9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.807+0000 7f2172ffd640 0 client.26584 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:49:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.807+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/3835027368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fb80f44d0 0x560fb84d41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.807+0000 7f21a0bcc740 1 --2- 172.21.15.2:0/3835027368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560fb80ec2f0 0x560fb84d3ca0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.807+0000 7f21a0bcc740 1 -- 172.21.15.2:0/3835027368 >> 172.21.15.2:0/3835027368 conn(0x560fb8109300 msgr2=0x560fb84b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.807+0000 7f21a0bcc740 1 -- 172.21.15.2:0/3835027368 shutdown_connections 2024-05-14T09:49:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.807+0000 7f21a0bcc740 1 -- 172.21.15.2:0/3835027368 wait complete. 2024-05-14T09:49:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.807+0000 7f21a0bcc740 20 client.26584 trim_cache size 0 max 16384 2024-05-14T09:49:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:49:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:49:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:49:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:49:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.808+0000 7f2199d1c640 1 -- 172.21.15.2:0/1742087439 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f217c0b2a70 msgr2=0x7f217c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.808+0000 7f2199d1c640 1 --2- 172.21.15.2:0/1742087439 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f217c0b2a70 0x7f217c0b4f80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2184018000 tx=0x7f218401a000 comp rx=0 tx=0).stop 2024-05-14T09:49:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.808+0000 7f2199d1c640 1 -- 172.21.15.2:0/1742087439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2194109b90 msgr2=0x7f2194074dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:49:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.808+0000 7f2199d1c640 1 --2- 172.21.15.2:0/1742087439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2194109b90 0x7f2194074dd0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f218803d000 tx=0x7f2188038d50 comp rx=0 tx=0).stop 2024-05-14T09:49:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.808+0000 7f2199d1c640 1 -- 172.21.15.2:0/1742087439 shutdown_connections 2024-05-14T09:49:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.808+0000 7f2199d1c640 1 --2- 172.21.15.2:0/1742087439 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f217c0b2a70 0x7f217c0b4f80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.808+0000 7f2199d1c640 1 --2- 172.21.15.2:0/1742087439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2194075310 0x7f2194075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.808+0000 7f2199d1c640 1 --2- 172.21.15.2:0/1742087439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2194109b90 0x7f2194074dd0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.808+0000 7f2199d1c640 1 --2- 172.21.15.2:0/1742087439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2194103c50 0x7f2194076730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:49:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.808+0000 7f2199d1c640 1 -- 172.21.15.2:0/1742087439 >> 172.21.15.2:0/1742087439 conn(0x7f21940fee60 msgr2=0x7f2194105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:49:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.808+0000 7f2199d1c640 1 -- 172.21.15.2:0/1742087439 shutdown_connections 2024-05-14T09:49:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:49:52.808+0000 7f2199d1c640 1 -- 172.21.15.2:0/1742087439 wait complete. 2024-05-14T09:49:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:52 smithi002 ceph-mon[31557]: pgmap v872: 129 pgs: 129 active+clean; 474 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.9 MiB/s wr, 696 op/s 2024-05-14T09:49:52.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:52 smithi029 ceph-mon[36249]: pgmap v872: 129 pgs: 129 active+clean; 474 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.9 MiB/s wr, 696 op/s 2024-05-14T09:49:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:52 smithi026 ceph-mon[36056]: pgmap v872: 129 pgs: 129 active+clean; 474 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.9 MiB/s wr, 696 op/s 2024-05-14T09:49:53.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 42642 31.30 MB/sec warmup 79 sec latency 323.039 ms 2024-05-14T09:49:54.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 43376 31.33 MB/sec warmup 80 sec latency 72.641 ms 2024-05-14T09:49:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:54 smithi002 ceph-mon[31557]: pgmap v873: 129 pgs: 129 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 876 op/s 2024-05-14T09:49:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:54 smithi029 ceph-mon[36249]: pgmap v873: 129 pgs: 129 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 876 op/s 2024-05-14T09:49:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:54 smithi026 ceph-mon[36056]: pgmap v873: 129 pgs: 129 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 876 op/s 2024-05-14T09:49:55.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 43982 31.34 MB/sec warmup 81 sec latency 570.649 ms 2024-05-14T09:49:56.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 44499 31.36 MB/sec warmup 82 sec latency 365.576 ms 2024-05-14T09:49:56.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:56 smithi002 ceph-mon[31557]: pgmap v874: 129 pgs: 129 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 552 op/s 2024-05-14T09:49:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:56 smithi029 ceph-mon[36249]: pgmap v874: 129 pgs: 129 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 552 op/s 2024-05-14T09:49:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:56 smithi026 ceph-mon[36056]: pgmap v874: 129 pgs: 129 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 552 op/s 2024-05-14T09:49:57.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 44765 31.17 MB/sec warmup 83 sec latency 414.552 ms 2024-05-14T09:49:57.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:49:57.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:57.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:57.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:57.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:57 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:57.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:49:57.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:57.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:57.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:57.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:57 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:57.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:49:57.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:57.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:57.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:57.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:57 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:58.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 45453 31.26 MB/sec warmup 84 sec latency 63.388 ms 2024-05-14T09:49:58.821 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:58 smithi002 ceph-mon[31557]: pgmap v875: 129 pgs: 129 active+clean; 482 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.9 MiB/s wr, 724 op/s 2024-05-14T09:49:58.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:58 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T09:49:58.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:58 smithi002 ceph-mon[31557]: Unable to set osd_memory_target on smithi002 to 185877708: error parsing value: Value '185877708' is below minimum 939524096 2024-05-14T09:49:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:58 smithi029 ceph-mon[36249]: pgmap v875: 129 pgs: 129 active+clean; 482 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.9 MiB/s wr, 724 op/s 2024-05-14T09:49:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:58 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T09:49:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:58 smithi029 ceph-mon[36249]: Unable to set osd_memory_target on smithi002 to 185877708: error parsing value: Value '185877708' is below minimum 939524096 2024-05-14T09:49:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:58 smithi026 ceph-mon[36056]: pgmap v875: 129 pgs: 129 active+clean; 482 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.9 MiB/s wr, 724 op/s 2024-05-14T09:49:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:58 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T09:49:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:58 smithi026 ceph-mon[36056]: Unable to set osd_memory_target on smithi002 to 185877708: error parsing value: Value '185877708' is below minimum 939524096 2024-05-14T09:49:59.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 45883 31.21 MB/sec warmup 85 sec latency 347.528 ms 2024-05-14T09:49:59.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:59.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:59.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:59.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:59.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:59 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-05-14T09:49:59.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:59 smithi002 ceph-mon[31557]: Unable to set osd_memory_target on smithi026 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-14T09:49:59.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:49:59.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:49:59.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:49:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:49:59.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:59.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:59.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:59.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:59.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:59 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-05-14T09:49:59.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:59 smithi029 ceph-mon[36249]: Unable to set osd_memory_target on smithi026 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-14T09:49:59.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:49:59.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:49:59.939 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:49:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:49:59.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:59.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:59.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:59.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:49:59.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:59 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-05-14T09:49:59.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:59 smithi026 ceph-mon[36056]: Unable to set osd_memory_target on smithi026 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-14T09:49:59.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:49:59.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:49:59.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:49:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:50:00.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a[31553]: 2024-05-14T09:49:59.999+0000 7f82afa15700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-14T09:50:00.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 46430 31.15 MB/sec warmup 86 sec latency 70.111 ms 2024-05-14T09:50:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:00 smithi002 ceph-mon[31557]: pgmap v876: 129 pgs: 129 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.7 MiB/s wr, 749 op/s 2024-05-14T09:50:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:00 smithi002 ceph-mon[31557]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-14T09:50:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:00 smithi029 ceph-mon[36249]: pgmap v876: 129 pgs: 129 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.7 MiB/s wr, 749 op/s 2024-05-14T09:50:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:00 smithi029 ceph-mon[36249]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-14T09:50:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:00 smithi026 ceph-mon[36056]: pgmap v876: 129 pgs: 129 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.7 MiB/s wr, 749 op/s 2024-05-14T09:50:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:00 smithi026 ceph-mon[36056]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-14T09:50:01.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 46856 31.06 MB/sec warmup 87 sec latency 245.208 ms 2024-05-14T09:50:02.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 47427 30.93 MB/sec warmup 88 sec latency 85.684 ms 2024-05-14T09:50:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:02 smithi002 ceph-mon[31557]: pgmap v877: 129 pgs: 129 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.0 MiB/s wr, 635 op/s 2024-05-14T09:50:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:02 smithi029 ceph-mon[36249]: pgmap v877: 129 pgs: 129 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.0 MiB/s wr, 635 op/s 2024-05-14T09:50:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:02 smithi026 ceph-mon[36056]: pgmap v877: 129 pgs: 129 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.0 MiB/s wr, 635 op/s 2024-05-14T09:50:03.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 48409 31.22 MB/sec warmup 89 sec latency 57.213 ms 2024-05-14T09:50:04.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 49194 31.43 MB/sec warmup 90 sec latency 136.642 ms 2024-05-14T09:50:04.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:04 smithi002 ceph-mon[31557]: pgmap v878: 129 pgs: 129 active+clean; 488 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.1 MiB/s wr, 801 op/s 2024-05-14T09:50:04.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:04 smithi029 ceph-mon[36249]: pgmap v878: 129 pgs: 129 active+clean; 488 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.1 MiB/s wr, 801 op/s 2024-05-14T09:50:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:04 smithi026 ceph-mon[36056]: pgmap v878: 129 pgs: 129 active+clean; 488 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.1 MiB/s wr, 801 op/s 2024-05-14T09:50:05.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 49709 31.37 MB/sec warmup 91 sec latency 134.944 ms 2024-05-14T09:50:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:05.986+0000 7f83b21c0640 1 -- 172.21.15.2:0/2769763091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac1034b0 msgr2=0x7f83ac103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:05.986+0000 7f83b21c0640 1 --2- 172.21.15.2:0/2769763091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac1034b0 0x7f83ac103d80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f839c009900 tx=0x7f839c02a440 comp rx=0 tx=0).stop 2024-05-14T09:50:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:05.991+0000 7f83b21c0640 1 -- 172.21.15.2:0/2769763091 shutdown_connections 2024-05-14T09:50:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:05.991+0000 7f83b21c0640 1 --2- 172.21.15.2:0/2769763091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac1034b0 0x7f83ac103d80 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:05.991+0000 7f83b21c0640 1 --2- 172.21.15.2:0/2769763091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac102b60 0x7f83ac102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:05.991+0000 7f83b21c0640 1 --2- 172.21.15.2:0/2769763091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac101a40 0x7f83ac101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:05.991+0000 7f83b21c0640 1 -- 172.21.15.2:0/2769763091 >> 172.21.15.2:0/2769763091 conn(0x7f83ac0fd050 msgr2=0x7f83ac0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:05.997+0000 7f83b21c0640 1 -- 172.21.15.2:0/2769763091 shutdown_connections 2024-05-14T09:50:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:05.998+0000 7f83b21c0640 1 -- 172.21.15.2:0/2769763091 wait complete. 2024-05-14T09:50:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:05.999+0000 7f83b21c0640 1 Processor -- start 2024-05-14T09:50:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:05.999+0000 7f83b21c0640 1 -- start start 2024-05-14T09:50:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.000+0000 7f83b21c0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac101a40 0x7f83ac071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.000+0000 7f83b21c0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac102b60 0x7f83ac072050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.000+0000 7f83b21c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac1034b0 0x7f83ac072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.000+0000 7f83b21c0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac111ad0 con 0x7f83ac102b60 2024-05-14T09:50:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.000+0000 7f83b21c0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac111950 con 0x7f83ac1034b0 2024-05-14T09:50:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.000+0000 7f83b21c0640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac111c50 con 0x7f83ac101a40 2024-05-14T09:50:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.000+0000 7f83ab7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac101a40 0x7f83ac071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.000+0000 7f83ab7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac101a40 0x7f83ac071ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:41218/0 (socket says 172.21.15.2:41218) 2024-05-14T09:50:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.000+0000 7f83ab7fe640 1 -- 172.21.15.2:0/2722484197 learned_addr learned my addr 172.21.15.2:0/2722484197 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.001+0000 7f83ab7fe640 1 -- 172.21.15.2:0/2722484197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac1034b0 msgr2=0x7f83ac072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.001+0000 7f83ab7fe640 1 --2- 172.21.15.2:0/2722484197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac1034b0 0x7f83ac072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.001+0000 7f83ab7fe640 1 -- 172.21.15.2:0/2722484197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac102b60 msgr2=0x7f83ac072050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.001+0000 7f83ab7fe640 1 --2- 172.21.15.2:0/2722484197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac102b60 0x7f83ac072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.001+0000 7f83ab7fe640 1 -- 172.21.15.2:0/2722484197 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ac073660 con 0x7f83ac101a40 2024-05-14T09:50:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.001+0000 7f83ab7fe640 1 --2- 172.21.15.2:0/2722484197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac101a40 0x7f83ac071ad0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f83a000b450 tx=0x7f83a000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.004+0000 7f83a8ff9640 1 -- 172.21.15.2:0/2722484197 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0015020 con 0x7f83ac101a40 2024-05-14T09:50:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.004+0000 7f83a8ff9640 1 -- 172.21.15.2:0/2722484197 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a0011d40 con 0x7f83ac101a40 2024-05-14T09:50:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.004+0000 7f83a8ff9640 1 -- 172.21.15.2:0/2722484197 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a000f340 con 0x7f83ac101a40 2024-05-14T09:50:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.004+0000 7f83b21c0640 1 -- 172.21.15.2:0/2722484197 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ac073950 con 0x7f83ac101a40 2024-05-14T09:50:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.004+0000 7f83b21c0640 1 -- 172.21.15.2:0/2722484197 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ac073ed0 con 0x7f83ac101a40 2024-05-14T09:50:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.010+0000 7f83a8ff9640 1 -- 172.21.15.2:0/2722484197 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f83a0018020 con 0x7f83ac101a40 2024-05-14T09:50:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.012+0000 7f83a8ff9640 1 --2- 172.21.15.2:0/2722484197 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83940b29c0 0x7f83940b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.012+0000 7f83a8ff9640 1 -- 172.21.15.2:0/2722484197 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f83a00e2140 con 0x7f83ac101a40 2024-05-14T09:50:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.013+0000 7f83aaffd640 1 --2- 172.21.15.2:0/2722484197 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83940b29c0 0x7f83940b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.014+0000 7f83b9070740 1 Processor -- start 2024-05-14T09:50:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.014+0000 7f83aaffd640 1 --2- 172.21.15.2:0/2722484197 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83940b29c0 0x7f83940b4ed0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f83ac072bf0 tx=0x7f83a4009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.015+0000 7f83b9070740 1 -- start start 2024-05-14T09:50:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.015+0000 7f83b9070740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46eb1b0 0x5592b4a95270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.015+0000 7f83b9070740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46c22f0 0x5592b4a957f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.015+0000 7f83b9070740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 0x5592b4a9c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.015+0000 7f83b9070740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5592b49f15d0 con 0x5592b46ca4d0 2024-05-14T09:50:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.015+0000 7f83b9070740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5592b49aa890 con 0x5592b46c22f0 2024-05-14T09:50:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.015+0000 7f83b9070740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5592b49ed590 con 0x5592b46eb1b0 2024-05-14T09:50:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.018+0000 7f83abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 0x5592b4a9c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.018+0000 7f83abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 0x5592b4a9c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35188/0 (socket says 172.21.15.2:35188) 2024-05-14T09:50:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.018+0000 7f83abfff640 1 -- 172.21.15.2:0/1406756384 learned_addr learned my addr 172.21.15.2:0/1406756384 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.018+0000 7f83ab7fe640 1 --2- 172.21.15.2:0/1406756384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46eb1b0 0x5592b4a95270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.019+0000 7f83ab7fe640 1 -- 172.21.15.2:0/1406756384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46c22f0 msgr2=0x5592b4a957f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:50:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.019+0000 7f83ab7fe640 1 --2- 172.21.15.2:0/1406756384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46c22f0 0x5592b4a957f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.019+0000 7f83ab7fe640 1 -- 172.21.15.2:0/1406756384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 msgr2=0x5592b4a9c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.019+0000 7f83ab7fe640 1 --2- 172.21.15.2:0/1406756384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 0x5592b4a9c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.019+0000 7f83ab7fe640 1 -- 172.21.15.2:0/1406756384 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b4703210 con 0x5592b46eb1b0 2024-05-14T09:50:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.020+0000 7f83ab7fe640 1 --2- 172.21.15.2:0/1406756384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46eb1b0 0x5592b4a95270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f83a00a8fe0 tx=0x7f83a00a8df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d2184afd2983379 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.021+0000 7f8391ffb640 1 -- 172.21.15.2:0/1406756384 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00ad690 con 0x5592b46eb1b0 2024-05-14T09:50:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.021+0000 7f8391ffb640 1 -- 172.21.15.2:0/1406756384 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a00b2140 con 0x5592b46eb1b0 2024-05-14T09:50:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.021+0000 7f8391ffb640 1 -- 172.21.15.2:0/1406756384 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00a8540 con 0x5592b46eb1b0 2024-05-14T09:50:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.024+0000 7f83b9070740 1 -- 172.21.15.2:0/1406756384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46eb1b0 msgr2=0x5592b4a95270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.024+0000 7f83b9070740 1 --2- 172.21.15.2:0/1406756384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46eb1b0 0x5592b4a95270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f83a00a8fe0 tx=0x7f83a00a8df0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.025+0000 7f83b9070740 1 -- 172.21.15.2:0/1406756384 shutdown_connections 2024-05-14T09:50:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.025+0000 7f83b9070740 1 --2- 172.21.15.2:0/1406756384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 0x5592b4a9c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.025+0000 7f83b9070740 1 --2- 172.21.15.2:0/1406756384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46c22f0 0x5592b4a957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.025+0000 7f83b9070740 1 --2- 172.21.15.2:0/1406756384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46eb1b0 0x5592b4a95270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.025+0000 7f83b9070740 1 -- 172.21.15.2:0/1406756384 >> 172.21.15.2:0/1406756384 conn(0x5592b46df300 msgr2=0x5592b4a883a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.027+0000 7f83b9070740 1 -- 172.21.15.2:0/1406756384 shutdown_connections 2024-05-14T09:50:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.029+0000 7f83b9070740 1 -- 172.21.15.2:0/1406756384 wait complete. 2024-05-14T09:50:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.029+0000 7f83b9070740 1 -- start start 2024-05-14T09:50:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.029+0000 7f83b9070740 1 Processor -- start 2024-05-14T09:50:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.030+0000 7f83b9070740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46c22f0 0x5592b4a8e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.030+0000 7f83b9070740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 0x5592b4a8eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.030+0000 7f83ab7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46c22f0 0x5592b4a8e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.030+0000 7f83ab7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46c22f0 0x5592b4a8e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:41226/0 (socket says 172.21.15.2:41226) 2024-05-14T09:50:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.030+0000 7f83ab7fe640 1 -- 172.21.15.2:0/2585066473 learned_addr learned my addr 172.21.15.2:0/2585066473 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.030+0000 7f83b9070740 1 --2- 172.21.15.2:0/2585066473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46eb1b0 0x5592b4a93e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.030+0000 7f83b9070740 1 -- 172.21.15.2:0/2585066473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5592b4a8cab0 con 0x5592b46ca4d0 2024-05-14T09:50:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.030+0000 7f83b9070740 1 -- 172.21.15.2:0/2585066473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5592b49ae370 con 0x5592b46eb1b0 2024-05-14T09:50:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.030+0000 7f83b9070740 1 -- 172.21.15.2:0/2585066473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5592b4a8cc30 con 0x5592b46c22f0 2024-05-14T09:50:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.031+0000 7f83abfff640 1 --2- 172.21.15.2:0/2585066473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46eb1b0 0x5592b4a93e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.031+0000 7f83b9070740 10 client.0 _ll_register_callbacks cb 0x5592b4702a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:50:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.031+0000 7f83ab7fe640 1 -- 172.21.15.2:0/2585066473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46eb1b0 msgr2=0x5592b4a93e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.031+0000 7f83ab7fe640 1 --2- 172.21.15.2:0/2585066473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46eb1b0 0x5592b4a93e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.031+0000 7f83ab7fe640 1 -- 172.21.15.2:0/2585066473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 msgr2=0x5592b4a8eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.031+0000 7f83ab7fe640 1 --2- 172.21.15.2:0/2585066473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 0x5592b4a8eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.031+0000 7f83ab7fe640 1 -- 172.21.15.2:0/2585066473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b47cf620 con 0x5592b46c22f0 2024-05-14T09:50:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.031+0000 7f83b9070740 1 -- 172.21.15.2:0/2585066473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83a00b2140 con 0x5592b46c22f0 2024-05-14T09:50:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.031+0000 7f83ab7fe640 1 --2- 172.21.15.2:0/2585066473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46c22f0 0x5592b4a8e9b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f83a0013000 tx=0x7f83a00df7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ac4b4ebc57221dd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.034+0000 7f838affd640 10 client.26593 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:50:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.036+0000 7f838affd640 1 -- 172.21.15.2:0/2585066473 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00ad690 con 0x5592b46c22f0 2024-05-14T09:50:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.036+0000 7f838affd640 1 -- 172.21.15.2:0/2585066473 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f83a00ca300 con 0x5592b46c22f0 2024-05-14T09:50:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.036+0000 7f838affd640 1 -- 172.21.15.2:0/2585066473 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f83a0011aa0 con 0x5592b46c22f0 2024-05-14T09:50:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.038+0000 7f83b9070740 10 client.26593 fetch_fsmap learned FSMap version 54 2024-05-14T09:50:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.038+0000 7f83b9070740 1 -- 172.21.15.2:0/2585066473 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({fsmap=54}) -- 0x5592b4a8cae0 con 0x5592b46c22f0 2024-05-14T09:50:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.040+0000 7f838affd640 1 -- 172.21.15.2:0/2585066473 <== mon.1 v2:172.21.15.29:3300/0 4 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f83a00c3bd0 con 0x5592b46c22f0 2024-05-14T09:50:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.041+0000 7f83b9070740 10 client.26593 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T09:50:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.041+0000 7f83b9070740 10 client.26593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:50:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.041+0000 7f83b9070740 20 client.26593 populate_metadata read hostname 'smithi002' 2024-05-14T09:50:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.041+0000 7f83b9070740 1 --2- 172.21.15.2:0/2585066473 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b46a9d80 0x5592b4ab4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.041+0000 7f83b9070740 4 client.26593 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:50:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.041+0000 7f83b9070740 1 -- 172.21.15.2:0/2585066473 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592b49ff1e0 con 0x5592b46a9d80 2024-05-14T09:50:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.043+0000 7f83aaffd640 1 --2- 172.21.15.2:0/2585066473 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b46a9d80 0x5592b4ab4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.054+0000 7f83aaffd640 1 --2- 172.21.15.2:0/2585066473 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b46a9d80 0x5592b4ab4c20 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e8502d962e986ec server_cookie=5a3a3917600cea71 in_seq=0 out_seq=0 2024-05-14T09:50:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.055+0000 7f838affd640 10 client.26593 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.058+0000 7f838affd640 1 -- 172.21.15.2:0/2585066473 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592b49ff1e0 con 0x5592b46a9d80 2024-05-14T09:50:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.058+0000 7f838affd640 10 client.26593 handle_command_reply: tid=0 2024-05-14T09:50:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.059+0000 7f83b9070740 1 client.26593 shutdown 2024-05-14T09:50:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.067+0000 7f83b9070740 1 -- 172.21.15.2:0/2585066473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46c22f0 msgr2=0x5592b4a8e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.067+0000 7f83b9070740 1 --2- 172.21.15.2:0/2585066473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46c22f0 0x5592b4a8e9b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f83a0013000 tx=0x7f83a00df7d0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.067+0000 7f83b9070740 1 -- 172.21.15.2:0/2585066473 shutdown_connections 2024-05-14T09:50:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.067+0000 7f83b9070740 1 --2- 172.21.15.2:0/2585066473 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b46a9d80 0x5592b4ab4c20 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.067+0000 7f83b9070740 1 --2- 172.21.15.2:0/2585066473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46eb1b0 0x5592b4a93e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.067+0000 7f83b9070740 1 --2- 172.21.15.2:0/2585066473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 0x5592b4a8eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.067+0000 7f83b9070740 1 --2- 172.21.15.2:0/2585066473 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46c22f0 0x5592b4a8e9b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.067+0000 7f838affd640 0 client.26593 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.067+0000 7f83b9070740 1 -- 172.21.15.2:0/2585066473 >> 172.21.15.2:0/2585066473 conn(0x5592b46df300 msgr2=0x5592b4a873a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.070+0000 7f83b9070740 1 -- 172.21.15.2:0/2585066473 shutdown_connections 2024-05-14T09:50:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.070+0000 7f83b9070740 1 -- 172.21.15.2:0/2585066473 wait complete. 2024-05-14T09:50:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.071+0000 7f83b9070740 20 client.26593 trim_cache size 0 max 16384 2024-05-14T09:50:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.113+0000 7f83b9070740 1 Processor -- start 2024-05-14T09:50:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.113+0000 7f83b9070740 1 -- start start 2024-05-14T09:50:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.114+0000 7f83b9070740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46c22f0 0x5592b4a8efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.114+0000 7f83b9070740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 0x5592b4a87080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.114+0000 7f83b9070740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46eb1b0 0x5592b4a8d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.114+0000 7f83b9070740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1a4eb0 con 0x5592b46ca4d0 2024-05-14T09:50:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.114+0000 7f83b9070740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1a26f0 con 0x5592b46c22f0 2024-05-14T09:50:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.114+0000 7f83b9070740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1a0560 con 0x5592b46eb1b0 2024-05-14T09:50:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.115+0000 7f83abfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46eb1b0 0x5592b4a8d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.115+0000 7f83abfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46eb1b0 0x5592b4a8d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:41242/0 (socket says 172.21.15.2:41242) 2024-05-14T09:50:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.115+0000 7f83abfff640 1 -- 172.21.15.2:0/2441813491 learned_addr learned my addr 172.21.15.2:0/2441813491 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.115+0000 7f83aaffd640 1 --2- 172.21.15.2:0/2441813491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 0x5592b4a87080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.115+0000 7f83ab7fe640 1 --2- 172.21.15.2:0/2441813491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46c22f0 0x5592b4a8efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.116+0000 7f83aaffd640 1 -- 172.21.15.2:0/2441813491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46eb1b0 msgr2=0x5592b4a8d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.116+0000 7f83aaffd640 1 --2- 172.21.15.2:0/2441813491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46eb1b0 0x5592b4a8d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.116+0000 7f83aaffd640 1 -- 172.21.15.2:0/2441813491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46c22f0 msgr2=0x5592b4a8efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.116+0000 7f83aaffd640 1 --2- 172.21.15.2:0/2441813491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46c22f0 0x5592b4a8efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.116+0000 7f83aaffd640 1 -- 172.21.15.2:0/2441813491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b4a95420 con 0x5592b46ca4d0 2024-05-14T09:50:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.116+0000 7f83aaffd640 1 --2- 172.21.15.2:0/2441813491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 0x5592b4a87080 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x5592b4ab5360 tx=0x7f83a400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84304ee26e56c676 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.117+0000 7f83917fa640 1 -- 172.21.15.2:0/2441813491 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4026070 con 0x5592b46ca4d0 2024-05-14T09:50:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.117+0000 7f83917fa640 1 -- 172.21.15.2:0/2441813491 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a4021780 con 0x5592b46ca4d0 2024-05-14T09:50:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.117+0000 7f83917fa640 1 -- 172.21.15.2:0/2441813491 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4019ba0 con 0x5592b46ca4d0 2024-05-14T09:50:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.119+0000 7f83b9070740 1 -- 172.21.15.2:0/2441813491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 msgr2=0x5592b4a87080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.119+0000 7f83b9070740 1 --2- 172.21.15.2:0/2441813491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 0x5592b4a87080 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x5592b4ab5360 tx=0x7f83a400f000 comp rx=0 tx=0).stop 2024-05-14T09:50:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.119+0000 7f83b9070740 1 -- 172.21.15.2:0/2441813491 shutdown_connections 2024-05-14T09:50:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.119+0000 7f83b9070740 1 --2- 172.21.15.2:0/2441813491 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46eb1b0 0x5592b4a8d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.119+0000 7f83b9070740 1 --2- 172.21.15.2:0/2441813491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46ca4d0 0x5592b4a87080 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.119+0000 7f83b9070740 1 --2- 172.21.15.2:0/2441813491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46c22f0 0x5592b4a8efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.119+0000 7f83b9070740 1 -- 172.21.15.2:0/2441813491 >> 172.21.15.2:0/2441813491 conn(0x5592b46df300 msgr2=0x5592b4a86970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.125+0000 7f83b9070740 1 -- 172.21.15.2:0/2441813491 shutdown_connections 2024-05-14T09:50:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.128+0000 7f83b9070740 1 -- 172.21.15.2:0/2441813491 wait complete. 2024-05-14T09:50:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.128+0000 7f83b9070740 1 -- start start 2024-05-14T09:50:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.129+0000 7f83b9070740 1 Processor -- start 2024-05-14T09:50:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.129+0000 7f83b9070740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46c22f0 0x5592b4ac3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.130+0000 7f83b9070740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46ca4d0 0x5592b4ac3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.130+0000 7f83b9070740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46eb1b0 0x5592b4ac4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.130+0000 7f83b9070740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1a0bb0 con 0x5592b46c22f0 2024-05-14T09:50:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.130+0000 7f83b9070740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1a2020 con 0x5592b46eb1b0 2024-05-14T09:50:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.130+0000 7f83b9070740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1a1280 con 0x5592b46ca4d0 2024-05-14T09:50:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.130+0000 7f83b9070740 10 client.0 _ll_register_callbacks cb 0x7f83a00ad070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:50:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.130+0000 7f83abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46eb1b0 0x5592b4ac4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.130+0000 7f83ab7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46c22f0 0x5592b4ac3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.130+0000 7f83ab7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46c22f0 0x5592b4ac3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35208/0 (socket says 172.21.15.2:35208) 2024-05-14T09:50:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.130+0000 7f83ab7fe640 1 -- 172.21.15.2:0/2209243906 learned_addr learned my addr 172.21.15.2:0/2209243906 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.131+0000 7f83aaffd640 1 --2- 172.21.15.2:0/2209243906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46ca4d0 0x5592b4ac3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.133+0000 7f83aaffd640 1 -- 172.21.15.2:0/2209243906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46eb1b0 msgr2=0x5592b4ac4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.133+0000 7f83aaffd640 1 --2- 172.21.15.2:0/2209243906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46eb1b0 0x5592b4ac4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.133+0000 7f83aaffd640 1 -- 172.21.15.2:0/2209243906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46c22f0 msgr2=0x5592b4ac3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.133+0000 7f83aaffd640 1 --2- 172.21.15.2:0/2209243906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46c22f0 0x5592b4ac3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.133+0000 7f83aaffd640 1 -- 172.21.15.2:0/2209243906 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b4ac54e0 con 0x5592b46ca4d0 2024-05-14T09:50:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.135+0000 7f83aaffd640 1 --2- 172.21.15.2:0/2209243906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46ca4d0 0x5592b4ac3ff0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x5592b4a949a0 tx=0x7f83a4019650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20bcd6f598966858 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.135+0000 7f838affd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:50:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.138+0000 7f838affd640 1 -- 172.21.15.2:0/2209243906 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4026040 con 0x5592b46ca4d0 2024-05-14T09:50:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.138+0000 7f83b9070740 1 -- 172.21.15.2:0/2209243906 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83a4021780 con 0x5592b46ca4d0 2024-05-14T09:50:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.138+0000 7f838affd640 1 -- 172.21.15.2:0/2209243906 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f83a40212d0 con 0x5592b46ca4d0 2024-05-14T09:50:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.139+0000 7f838affd640 1 -- 172.21.15.2:0/2209243906 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f83a4038050 con 0x5592b46ca4d0 2024-05-14T09:50:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.140+0000 7f83b9070740 10 client.26599 fetch_fsmap learned FSMap version 54 2024-05-14T09:50:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.140+0000 7f83b9070740 1 -- 172.21.15.2:0/2209243906 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5592b4ac6770 con 0x5592b46ca4d0 2024-05-14T09:50:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.143+0000 7f838affd640 1 -- 172.21.15.2:0/2209243906 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a4021780 con 0x5592b46ca4d0 2024-05-14T09:50:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.143+0000 7f838affd640 1 -- 172.21.15.2:0/2209243906 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f83a4029070 con 0x5592b46ca4d0 2024-05-14T09:50:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.143+0000 7f83b9070740 10 client.26599 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T09:50:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.143+0000 7f83b9070740 10 client.26599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:50:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.143+0000 7f83b9070740 20 client.26599 populate_metadata read hostname 'smithi002' 2024-05-14T09:50:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.143+0000 7f83b9070740 1 --2- 172.21.15.2:0/2209243906 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b467f1c0 0x5592b4b1f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.143+0000 7f83b9070740 4 client.26599 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:50:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.143+0000 7f83b9070740 1 -- 172.21.15.2:0/2209243906 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592b49f9050 con 0x5592b467f1c0 2024-05-14T09:50:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.147+0000 7f83ab7fe640 1 --2- 172.21.15.2:0/2209243906 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b467f1c0 0x5592b4b1f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.152+0000 7f83ab7fe640 1 --2- 172.21.15.2:0/2209243906 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b467f1c0 0x5592b4b1f380 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ecdf13429f7b30e server_cookie=3f1f2c8432cb6fd9 in_seq=0 out_seq=0 2024-05-14T09:50:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.152+0000 7f838affd640 10 client.26599 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:06.602 INFO:tasks.workunit.client.0.smithi002.stdout: 10 49892 31.14 MB/sec warmup 92 sec latency 418.908 ms 2024-05-14T09:50:06.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:06 smithi002 ceph-mon[31557]: pgmap v879: 129 pgs: 129 active+clean; 488 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 551 op/s 2024-05-14T09:50:06.686 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:06 smithi029 ceph-mon[36249]: pgmap v879: 129 pgs: 129 active+clean; 488 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 551 op/s 2024-05-14T09:50:06.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:06 smithi026 ceph-mon[36056]: pgmap v879: 129 pgs: 129 active+clean; 488 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 551 op/s 2024-05-14T09:50:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.825+0000 7f838affd640 1 -- 172.21.15.2:0/2209243906 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592b49f9050 con 0x5592b467f1c0 2024-05-14T09:50:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.825+0000 7f838affd640 10 client.26599 handle_command_reply: tid=0 2024-05-14T09:50:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.825+0000 7f83b9070740 1 client.26599 shutdown 2024-05-14T09:50:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.826+0000 7f83b9070740 1 -- 172.21.15.2:0/2209243906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46ca4d0 msgr2=0x5592b4ac3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.826+0000 7f83b9070740 1 --2- 172.21.15.2:0/2209243906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46ca4d0 0x5592b4ac3ff0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x5592b4a949a0 tx=0x7f83a4019650 comp rx=0 tx=0).stop 2024-05-14T09:50:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.826+0000 7f83b9070740 1 -- 172.21.15.2:0/2209243906 shutdown_connections 2024-05-14T09:50:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.826+0000 7f83b9070740 1 --2- 172.21.15.2:0/2209243906 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5592b467f1c0 0x5592b4b1f380 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.826+0000 7f83b9070740 1 --2- 172.21.15.2:0/2209243906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592b46eb1b0 0x5592b4ac4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.826+0000 7f83b9070740 1 --2- 172.21.15.2:0/2209243906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5592b46ca4d0 0x5592b4ac3ff0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.826+0000 7f83b9070740 1 --2- 172.21.15.2:0/2209243906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592b46c22f0 0x5592b4ac3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.826+0000 7f83b9070740 1 -- 172.21.15.2:0/2209243906 >> 172.21.15.2:0/2209243906 conn(0x5592b46df300 msgr2=0x5592b4a8ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.826+0000 7f838affd640 0 client.26599 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.826+0000 7f83b9070740 1 -- 172.21.15.2:0/2209243906 shutdown_connections 2024-05-14T09:50:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.826+0000 7f83b9070740 1 -- 172.21.15.2:0/2209243906 wait complete. 2024-05-14T09:50:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.826+0000 7f83b9070740 20 client.26599 trim_cache size 0 max 16384 2024-05-14T09:50:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:50:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:50:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:50:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:50:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.827+0000 7f83b21c0640 1 -- 172.21.15.2:0/2722484197 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83940b29c0 msgr2=0x7f83940b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.827+0000 7f83b21c0640 1 --2- 172.21.15.2:0/2722484197 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83940b29c0 0x7f83940b4ed0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f83ac072bf0 tx=0x7f83a4009250 comp rx=0 tx=0).stop 2024-05-14T09:50:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.827+0000 7f83b21c0640 1 -- 172.21.15.2:0/2722484197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac101a40 msgr2=0x7f83ac071ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.827+0000 7f83b21c0640 1 --2- 172.21.15.2:0/2722484197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac101a40 0x7f83ac071ad0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f83a000b450 tx=0x7f83a000b910 comp rx=0 tx=0).stop 2024-05-14T09:50:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.827+0000 7f83b21c0640 1 -- 172.21.15.2:0/2722484197 shutdown_connections 2024-05-14T09:50:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.827+0000 7f83b21c0640 1 --2- 172.21.15.2:0/2722484197 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83940b29c0 0x7f83940b4ed0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.827+0000 7f83b21c0640 1 --2- 172.21.15.2:0/2722484197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac1034b0 0x7f83ac072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.827+0000 7f83b21c0640 1 --2- 172.21.15.2:0/2722484197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac102b60 0x7f83ac072050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.827+0000 7f83b21c0640 1 --2- 172.21.15.2:0/2722484197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac101a40 0x7f83ac071ad0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.827+0000 7f83b21c0640 1 -- 172.21.15.2:0/2722484197 >> 172.21.15.2:0/2722484197 conn(0x7f83ac0fd050 msgr2=0x7f83ac1090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.827+0000 7f83b21c0640 1 -- 172.21.15.2:0/2722484197 shutdown_connections 2024-05-14T09:50:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:06.827+0000 7f83b21c0640 1 -- 172.21.15.2:0/2722484197 wait complete. 2024-05-14T09:50:07.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 50225 31.01 MB/sec warmup 93 sec latency 500.651 ms 2024-05-14T09:50:08.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 50927 30.93 MB/sec warmup 94 sec latency 99.251 ms 2024-05-14T09:50:08.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:08 smithi026 ceph-mon[36056]: pgmap v880: 129 pgs: 129 active+clean; 496 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.1 MiB/s wr, 706 op/s 2024-05-14T09:50:08.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:08 smithi002 ceph-mon[31557]: pgmap v880: 129 pgs: 129 active+clean; 496 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.1 MiB/s wr, 706 op/s 2024-05-14T09:50:08.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:08 smithi029 ceph-mon[36249]: pgmap v880: 129 pgs: 129 active+clean; 496 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.1 MiB/s wr, 706 op/s 2024-05-14T09:50:09.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 51729 31.11 MB/sec warmup 95 sec latency 124.399 ms 2024-05-14T09:50:10.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 52480 31.30 MB/sec warmup 96 sec latency 76.774 ms 2024-05-14T09:50:10.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:10 smithi026 ceph-mon[36056]: pgmap v881: 129 pgs: 129 active+clean; 503 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.0 MiB/s wr, 760 op/s 2024-05-14T09:50:10.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:10 smithi002 ceph-mon[31557]: pgmap v881: 129 pgs: 129 active+clean; 503 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.0 MiB/s wr, 760 op/s 2024-05-14T09:50:10.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:10 smithi029 ceph-mon[36249]: pgmap v881: 129 pgs: 129 active+clean; 503 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.0 MiB/s wr, 760 op/s 2024-05-14T09:50:11.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 52808 31.17 MB/sec warmup 97 sec latency 358.300 ms 2024-05-14T09:50:12.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 53243 31.06 MB/sec warmup 98 sec latency 80.671 ms 2024-05-14T09:50:12.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:12 smithi002 ceph-mon[31557]: pgmap v882: 129 pgs: 129 active+clean; 502 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.4 MiB/s wr, 634 op/s 2024-05-14T09:50:12.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:12 smithi029 ceph-mon[36249]: pgmap v882: 129 pgs: 129 active+clean; 502 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.4 MiB/s wr, 634 op/s 2024-05-14T09:50:12.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:12 smithi026 ceph-mon[36056]: pgmap v882: 129 pgs: 129 active+clean; 502 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.4 MiB/s wr, 634 op/s 2024-05-14T09:50:13.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 53984 31.12 MB/sec warmup 99 sec latency 45.833 ms 2024-05-14T09:50:14.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 54631 31.12 MB/sec warmup 100 sec latency 334.196 ms 2024-05-14T09:50:14.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:14 smithi002 ceph-mon[31557]: pgmap v883: 129 pgs: 129 active+clean; 503 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 856 op/s 2024-05-14T09:50:14.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:14 smithi029 ceph-mon[36249]: pgmap v883: 129 pgs: 129 active+clean; 503 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 856 op/s 2024-05-14T09:50:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:14 smithi026 ceph-mon[36056]: pgmap v883: 129 pgs: 129 active+clean; 503 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 856 op/s 2024-05-14T09:50:15.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 55518 31.40 MB/sec warmup 101 sec latency 66.114 ms 2024-05-14T09:50:16.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 55946 31.28 MB/sec warmup 102 sec latency 315.969 ms 2024-05-14T09:50:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:16 smithi002 ceph-mon[31557]: pgmap v884: 129 pgs: 129 active+clean; 503 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.4 MiB/s wr, 667 op/s 2024-05-14T09:50:16.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:16 smithi029 ceph-mon[36249]: pgmap v884: 129 pgs: 129 active+clean; 503 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.4 MiB/s wr, 667 op/s 2024-05-14T09:50:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:16 smithi026 ceph-mon[36056]: pgmap v884: 129 pgs: 129 active+clean; 503 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.4 MiB/s wr, 667 op/s 2024-05-14T09:50:17.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 56301 31.13 MB/sec warmup 103 sec latency 102.024 ms 2024-05-14T09:50:18.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 57032 31.27 MB/sec warmup 104 sec latency 162.434 ms 2024-05-14T09:50:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:18 smithi002 ceph-mon[31557]: pgmap v885: 129 pgs: 129 active+clean; 510 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.2 MiB/s wr, 798 op/s 2024-05-14T09:50:18.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:18 smithi029 ceph-mon[36249]: pgmap v885: 129 pgs: 129 active+clean; 510 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.2 MiB/s wr, 798 op/s 2024-05-14T09:50:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:18 smithi026 ceph-mon[36056]: pgmap v885: 129 pgs: 129 active+clean; 510 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.2 MiB/s wr, 798 op/s 2024-05-14T09:50:19.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 57638 31.20 MB/sec warmup 105 sec latency 97.016 ms 2024-05-14T09:50:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.951+0000 7f05e5e13640 1 -- 172.21.15.2:0/1346806240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05e01121f0 msgr2=0x7f05e0114690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.951+0000 7f05e5e13640 1 --2- 172.21.15.2:0/1346806240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05e01121f0 0x7f05e0114690 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009be0 tx=0x7f05d802a6c0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.951+0000 7f05e5e13640 1 -- 172.21.15.2:0/1346806240 shutdown_connections 2024-05-14T09:50:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.951+0000 7f05e5e13640 1 --2- 172.21.15.2:0/1346806240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05e01121f0 0x7f05e0114690 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.951+0000 7f05e5e13640 1 --2- 172.21.15.2:0/1346806240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e010f890 0x7f05e0111cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.951+0000 7f05e5e13640 1 --2- 172.21.15.2:0/1346806240 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05e010a0f0 0x7f05e010a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.951+0000 7f05e5e13640 1 -- 172.21.15.2:0/1346806240 >> 172.21.15.2:0/1346806240 conn(0x7f05e0070640 msgr2=0x7f05e0072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.952+0000 7f05e5e13640 1 -- 172.21.15.2:0/1346806240 shutdown_connections 2024-05-14T09:50:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.952+0000 7f05e5e13640 1 -- 172.21.15.2:0/1346806240 wait complete. 2024-05-14T09:50:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.952+0000 7f05e5e13640 1 Processor -- start 2024-05-14T09:50:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.952+0000 7f05e5e13640 1 -- start start 2024-05-14T09:50:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05e5e13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e010a0f0 0x7f05e010d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05e5e13640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05e010f890 0x7f05e010b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05e5e13640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05e01121f0 0x7f05e010bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05e5e13640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f05e0118020 con 0x7f05e01121f0 2024-05-14T09:50:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05e5e13640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05e0117ea0 con 0x7f05e010a0f0 2024-05-14T09:50:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05e5e13640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f05e01181a0 con 0x7f05e010f890 2024-05-14T09:50:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05dffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05e01121f0 0x7f05e010bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05dffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05e01121f0 0x7f05e010bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43332/0 (socket says 172.21.15.2:43332) 2024-05-14T09:50:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05deffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05e010f890 0x7f05e010b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05dffff640 1 -- 172.21.15.2:0/1283814486 learned_addr learned my addr 172.21.15.2:0/1283814486 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05df7fe640 1 --2- 172.21.15.2:0/1283814486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e010a0f0 0x7f05e010d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05dffff640 1 -- 172.21.15.2:0/1283814486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05e010f890 msgr2=0x7f05e010b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05dffff640 1 --2- 172.21.15.2:0/1283814486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05e010f890 0x7f05e010b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05dffff640 1 -- 172.21.15.2:0/1283814486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e010a0f0 msgr2=0x7f05e010d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05dffff640 1 --2- 172.21.15.2:0/1283814486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e010a0f0 0x7f05e010d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.953+0000 7f05dffff640 1 -- 172.21.15.2:0/1283814486 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e010c400 con 0x7f05e01121f0 2024-05-14T09:50:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.954+0000 7f05dffff640 1 --2- 172.21.15.2:0/1283814486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05e01121f0 0x7f05e010bcd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f05d8034040 tx=0x7f05d8030d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.954+0000 7f05dcff9640 1 -- 172.21.15.2:0/1283814486 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d8041020 con 0x7f05e01121f0 2024-05-14T09:50:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.954+0000 7f05dcff9640 1 -- 172.21.15.2:0/1283814486 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05d80318f0 con 0x7f05e01121f0 2024-05-14T09:50:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.954+0000 7f05e5e13640 1 -- 172.21.15.2:0/1283814486 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e01acbc0 con 0x7f05e01121f0 2024-05-14T09:50:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.955+0000 7f05e5e13640 1 -- 172.21.15.2:0/1283814486 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e01acf70 con 0x7f05e01121f0 2024-05-14T09:50:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.955+0000 7f05dcff9640 1 -- 172.21.15.2:0/1283814486 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d8036040 con 0x7f05e01121f0 2024-05-14T09:50:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.959+0000 7f05dcff9640 1 -- 172.21.15.2:0/1283814486 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f05d8044020 con 0x7f05e01121f0 2024-05-14T09:50:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.959+0000 7f05ecd9f740 1 Processor -- start 2024-05-14T09:50:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.959+0000 7f05ecd9f740 1 -- start start 2024-05-14T09:50:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.959+0000 7f05ecd9f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f451b0 0x5647732e6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.959+0000 7f05ecd9f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 0x5647732e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.959+0000 7f05ecd9f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f244d0 0x5647732edee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.959+0000 7f05ecd9f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56477324b5d0 con 0x564772f244d0 2024-05-14T09:50:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.959+0000 7f05ecd9f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564773204890 con 0x564772f451b0 2024-05-14T09:50:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.959+0000 7f05ecd9f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564773247590 con 0x564772f1c2f0 2024-05-14T09:50:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05df7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 0x5647732e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05deffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f244d0 0x5647732edee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05df7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 0x5647732e6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:57252/0 (socket says 172.21.15.2:57252) 2024-05-14T09:50:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05df7fe640 1 -- 172.21.15.2:0/2896905372 learned_addr learned my addr 172.21.15.2:0/2896905372 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05dcff9640 1 --2- 172.21.15.2:0/1283814486 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f05c80b2920 0x7f05c80b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05dffff640 1 --2- 172.21.15.2:0/1283814486 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f05c80b2920 0x7f05c80b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05df7fe640 1 -- 172.21.15.2:0/2896905372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f451b0 msgr2=0x5647732e6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05df7fe640 1 --2- 172.21.15.2:0/2896905372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f451b0 0x5647732e6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05df7fe640 1 -- 172.21.15.2:0/2896905372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f244d0 msgr2=0x5647732edee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05df7fe640 1 --2- 172.21.15.2:0/2896905372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f244d0 0x5647732edee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05df7fe640 1 -- 172.21.15.2:0/2896905372 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564772f5d210 con 0x564772f1c2f0 2024-05-14T09:50:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05deffd640 1 --2- 172.21.15.2:0/2896905372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f244d0 0x5647732edee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:50:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05dcff9640 1 -- 172.21.15.2:0/1283814486 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f05d8102f10 con 0x7f05e01121f0 2024-05-14T09:50:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05df7fe640 1 --2- 172.21.15.2:0/2896905372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 0x5647732e6fe0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f05d0009c60 tx=0x7f05d0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48787e929b9adec8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05c5ffb640 1 -- 172.21.15.2:0/2896905372 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0004230 con 0x564772f1c2f0 2024-05-14T09:50:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05dffff640 1 --2- 172.21.15.2:0/1283814486 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f05c80b2920 0x7f05c80b4e30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f05d80c7380 tx=0x7f05d8093a00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05c5ffb640 1 -- 172.21.15.2:0/2896905372 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05d00043d0 con 0x564772f1c2f0 2024-05-14T09:50:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.960+0000 7f05c5ffb640 1 -- 172.21.15.2:0/2896905372 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0019710 con 0x564772f1c2f0 2024-05-14T09:50:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.961+0000 7f05ecd9f740 1 -- 172.21.15.2:0/2896905372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 msgr2=0x5647732e6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.961+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/2896905372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 0x5647732e6fe0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f05d0009c60 tx=0x7f05d0011810 comp rx=0 tx=0).stop 2024-05-14T09:50:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.962+0000 7f05ecd9f740 1 -- 172.21.15.2:0/2896905372 shutdown_connections 2024-05-14T09:50:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.962+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/2896905372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f244d0 0x5647732edee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.962+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/2896905372 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 0x5647732e6fe0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.962+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/2896905372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f451b0 0x5647732e6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.962+0000 7f05ecd9f740 1 -- 172.21.15.2:0/2896905372 >> 172.21.15.2:0/2896905372 conn(0x564772f39300 msgr2=0x5647732e23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.962+0000 7f05ecd9f740 1 -- 172.21.15.2:0/2896905372 shutdown_connections 2024-05-14T09:50:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.962+0000 7f05ecd9f740 1 -- 172.21.15.2:0/2896905372 wait complete. 2024-05-14T09:50:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.962+0000 7f05ecd9f740 1 -- start start 2024-05-14T09:50:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.962+0000 7f05ecd9f740 1 Processor -- start 2024-05-14T09:50:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.962+0000 7f05ecd9f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 0x5647732f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05ecd9f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f244d0 0x5647732f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05ecd9f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f451b0 0x5647732f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05ecd9f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5647732ee930 con 0x564772f244d0 2024-05-14T09:50:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05ecd9f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564773208370 con 0x564772f451b0 2024-05-14T09:50:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05ecd9f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5647732eeab0 con 0x564772f1c2f0 2024-05-14T09:50:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05ecd9f740 10 client.0 _ll_register_callbacks cb 0x564772f5ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:50:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05deffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 0x5647732f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05deffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 0x5647732f06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:57254/0 (socket says 172.21.15.2:57254) 2024-05-14T09:50:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05deffd640 1 -- 172.21.15.2:0/763520064 learned_addr learned my addr 172.21.15.2:0/763520064 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05df7fe640 1 --2- 172.21.15.2:0/763520064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f244d0 0x5647732f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05deffd640 1 --2- 172.21.15.2:0/763520064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f451b0 0x5647732f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05df7fe640 1 -- 172.21.15.2:0/763520064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 msgr2=0x5647732f06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05df7fe640 1 --2- 172.21.15.2:0/763520064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 0x5647732f06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05df7fe640 1 -- 172.21.15.2:0/763520064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f451b0 msgr2=0x5647732f7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05df7fe640 1 --2- 172.21.15.2:0/763520064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f451b0 0x5647732f7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05df7fe640 1 -- 172.21.15.2:0/763520064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564773029620 con 0x564772f244d0 2024-05-14T09:50:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05df7fe640 1 --2- 172.21.15.2:0/763520064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f244d0 0x5647732f0be0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x5647732e3180 tx=0x7f05d00052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f469637694696bf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05beffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:50:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.963+0000 7f05beffd640 1 -- 172.21.15.2:0/763520064 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0028020 con 0x564772f244d0 2024-05-14T09:50:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.964+0000 7f05beffd640 1 -- 172.21.15.2:0/763520064 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f05d00055e0 con 0x564772f244d0 2024-05-14T09:50:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.964+0000 7f05ecd9f740 1 -- 172.21.15.2:0/763520064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05d00043d0 con 0x564772f244d0 2024-05-14T09:50:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.964+0000 7f05beffd640 1 -- 172.21.15.2:0/763520064 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f05d0033050 con 0x564772f244d0 2024-05-14T09:50:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.964+0000 7f05ecd9f740 10 client.16977 fetch_fsmap learned FSMap version 54 2024-05-14T09:50:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.964+0000 7f05ecd9f740 1 -- 172.21.15.2:0/763520064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5647732faa70 con 0x564772f244d0 2024-05-14T09:50:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.964+0000 7f05beffd640 1 -- 172.21.15.2:0/763520064 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05d00043d0 con 0x564772f244d0 2024-05-14T09:50:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.964+0000 7f05beffd640 1 -- 172.21.15.2:0/763520064 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f05d0018070 con 0x564772f244d0 2024-05-14T09:50:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.965+0000 7f05ecd9f740 10 client.16977 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T09:50:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.965+0000 7f05ecd9f740 10 client.16977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:50:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.965+0000 7f05ecd9f740 20 client.16977 populate_metadata read hostname 'smithi002' 2024-05-14T09:50:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.965+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/763520064 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564772f03d80 0x5647732fe020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.965+0000 7f05ecd9f740 4 client.16977 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:50:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.965+0000 7f05ecd9f740 1 -- 172.21.15.2:0/763520064 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647732591e0 con 0x564772f03d80 2024-05-14T09:50:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.965+0000 7f05deffd640 1 --2- 172.21.15.2:0/763520064 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564772f03d80 0x5647732fe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.965+0000 7f05deffd640 1 --2- 172.21.15.2:0/763520064 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564772f03d80 0x5647732fe020 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf819548aaffb74 server_cookie=4676e45a35395ee4 in_seq=0 out_seq=0 2024-05-14T09:50:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.965+0000 7f05beffd640 10 client.16977 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.968+0000 7f05beffd640 1 -- 172.21.15.2:0/763520064 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5647732591e0 con 0x564772f03d80 2024-05-14T09:50:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.968+0000 7f05beffd640 10 client.16977 handle_command_reply: tid=0 2024-05-14T09:50:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.968+0000 7f05ecd9f740 1 client.16977 shutdown 2024-05-14T09:50:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.968+0000 7f05ecd9f740 1 -- 172.21.15.2:0/763520064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f244d0 msgr2=0x5647732f0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.968+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/763520064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f244d0 0x5647732f0be0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x5647732e3180 tx=0x7f05d00052f0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.968+0000 7f05ecd9f740 1 -- 172.21.15.2:0/763520064 shutdown_connections 2024-05-14T09:50:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.968+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/763520064 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564772f03d80 0x5647732fe020 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.968+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/763520064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f451b0 0x5647732f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.968+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/763520064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f244d0 0x5647732f0be0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.968+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/763520064 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 0x5647732f06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.968+0000 7f05beffd640 0 client.16977 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.968+0000 7f05ecd9f740 1 -- 172.21.15.2:0/763520064 >> 172.21.15.2:0/763520064 conn(0x564772f39300 msgr2=0x5647732e5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.968+0000 7f05ecd9f740 1 -- 172.21.15.2:0/763520064 shutdown_connections 2024-05-14T09:50:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.968+0000 7f05ecd9f740 1 -- 172.21.15.2:0/763520064 wait complete. 2024-05-14T09:50:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.968+0000 7f05ecd9f740 20 client.16977 trim_cache size 0 max 16384 2024-05-14T09:50:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05ecd9f740 1 Processor -- start 2024-05-14T09:50:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05ecd9f740 1 -- start start 2024-05-14T09:50:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05ecd9f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f03d80 0x5647732e2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05ecd9f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f1c2f0 0x5647732f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05ecd9f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f244d0 0x5647732f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05ecd9f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f05e01b9ec0 con 0x564772f1c2f0 2024-05-14T09:50:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05ecd9f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05e01b7700 con 0x564772f244d0 2024-05-14T09:50:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05ecd9f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f05e01b5550 con 0x564772f03d80 2024-05-14T09:50:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05df7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f1c2f0 0x5647732f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05df7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f1c2f0 0x5647732f7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43368/0 (socket says 172.21.15.2:43368) 2024-05-14T09:50:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05df7fe640 1 -- 172.21.15.2:0/1886285787 learned_addr learned my addr 172.21.15.2:0/1886285787 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05deffd640 1 --2- 172.21.15.2:0/1886285787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f03d80 0x5647732e2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05deffd640 1 --2- 172.21.15.2:0/1886285787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f244d0 0x5647732f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05df7fe640 1 -- 172.21.15.2:0/1886285787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f03d80 msgr2=0x5647732e2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05df7fe640 1 --2- 172.21.15.2:0/1886285787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f03d80 0x5647732e2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05df7fe640 1 -- 172.21.15.2:0/1886285787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f244d0 msgr2=0x5647732f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05df7fe640 1 --2- 172.21.15.2:0/1886285787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f244d0 0x5647732f7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.987+0000 7f05df7fe640 1 -- 172.21.15.2:0/1886285787 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647732e1b60 con 0x564772f1c2f0 2024-05-14T09:50:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.988+0000 7f05df7fe640 1 --2- 172.21.15.2:0/1886285787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f1c2f0 0x5647732f7a60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f05d0003f50 tx=0x7f05d0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d02eb3de14ee58 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.988+0000 7f05c57fa640 1 -- 172.21.15.2:0/1886285787 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0028040 con 0x564772f1c2f0 2024-05-14T09:50:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.988+0000 7f05c57fa640 1 -- 172.21.15.2:0/1886285787 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05d00097c0 con 0x564772f1c2f0 2024-05-14T09:50:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.988+0000 7f05c57fa640 1 -- 172.21.15.2:0/1886285787 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d001f040 con 0x564772f1c2f0 2024-05-14T09:50:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.989+0000 7f05ecd9f740 1 -- 172.21.15.2:0/1886285787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f1c2f0 msgr2=0x5647732f7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.989+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/1886285787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f1c2f0 0x5647732f7a60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f05d0003f50 tx=0x7f05d0009b30 comp rx=0 tx=0).stop 2024-05-14T09:50:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.989+0000 7f05ecd9f740 1 -- 172.21.15.2:0/1886285787 shutdown_connections 2024-05-14T09:50:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.989+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/1886285787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f244d0 0x5647732f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.989+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/1886285787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f1c2f0 0x5647732f7a60 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.989+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/1886285787 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f03d80 0x5647732e2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.989+0000 7f05ecd9f740 1 -- 172.21.15.2:0/1886285787 >> 172.21.15.2:0/1886285787 conn(0x564772f39300 msgr2=0x5647732e1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.989+0000 7f05ecd9f740 1 -- 172.21.15.2:0/1886285787 shutdown_connections 2024-05-14T09:50:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.989+0000 7f05ecd9f740 1 -- 172.21.15.2:0/1886285787 wait complete. 2024-05-14T09:50:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.989+0000 7f05ecd9f740 1 -- start start 2024-05-14T09:50:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05ecd9f740 1 Processor -- start 2024-05-14T09:50:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05ecd9f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f03d80 0x5647732f5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05ecd9f740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 0x5647732f5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05ecd9f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f244d0 0x5647732f6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05ecd9f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f05e01b5bc0 con 0x564772f03d80 2024-05-14T09:50:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05ecd9f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05e01b7030 con 0x564772f244d0 2024-05-14T09:50:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05ecd9f740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f05e01b6290 con 0x564772f1c2f0 2024-05-14T09:50:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05ecd9f740 10 client.0 _ll_register_callbacks cb 0x7f05d40097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:50:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05deffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f03d80 0x5647732f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05deffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f03d80 0x5647732f5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43372/0 (socket says 172.21.15.2:43372) 2024-05-14T09:50:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05deffd640 1 -- 172.21.15.2:0/820676098 learned_addr learned my addr 172.21.15.2:0/820676098 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05df7fe640 1 --2- 172.21.15.2:0/820676098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 0x5647732f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05deffd640 1 --2- 172.21.15.2:0/820676098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f244d0 0x5647732f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05deffd640 1 -- 172.21.15.2:0/820676098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 msgr2=0x5647732f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05deffd640 1 --2- 172.21.15.2:0/820676098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 0x5647732f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05deffd640 1 -- 172.21.15.2:0/820676098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f244d0 msgr2=0x5647732f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05deffd640 1 --2- 172.21.15.2:0/820676098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f244d0 0x5647732f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.990+0000 7f05deffd640 1 -- 172.21.15.2:0/820676098 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647732f6e40 con 0x564772f03d80 2024-05-14T09:50:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.991+0000 7f05deffd640 1 --2- 172.21.15.2:0/820676098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f03d80 0x5647732f5410 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f05d4011d20 tx=0x7f05d0004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a710cda16417cdbe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.991+0000 7f05beffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:50:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.991+0000 7f05beffd640 1 -- 172.21.15.2:0/820676098 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d401a040 con 0x564772f03d80 2024-05-14T09:50:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.991+0000 7f05beffd640 1 -- 172.21.15.2:0/820676098 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f05d4031430 con 0x564772f03d80 2024-05-14T09:50:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.991+0000 7f05ecd9f740 1 -- 172.21.15.2:0/820676098 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05d00097c0 con 0x564772f03d80 2024-05-14T09:50:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.991+0000 7f05beffd640 1 -- 172.21.15.2:0/820676098 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f05d403a050 con 0x564772f03d80 2024-05-14T09:50:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.991+0000 7f05ecd9f740 10 client.16983 fetch_fsmap learned FSMap version 54 2024-05-14T09:50:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.991+0000 7f05ecd9f740 1 -- 172.21.15.2:0/820676098 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5647732f81d0 con 0x564772f03d80 2024-05-14T09:50:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.991+0000 7f05beffd640 1 -- 172.21.15.2:0/820676098 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05d00097c0 con 0x564772f03d80 2024-05-14T09:50:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.991+0000 7f05beffd640 1 -- 172.21.15.2:0/820676098 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f05d40225a0 con 0x564772f03d80 2024-05-14T09:50:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.992+0000 7f05ecd9f740 10 client.16983 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T09:50:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.992+0000 7f05ecd9f740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:50:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.992+0000 7f05ecd9f740 20 client.16983 populate_metadata read hostname 'smithi002' 2024-05-14T09:50:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.992+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/820676098 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564772f451b0 0x564773368790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.992+0000 7f05ecd9f740 4 client.16983 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:50:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.992+0000 7f05ecd9f740 1 -- 172.21.15.2:0/820676098 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564773253050 con 0x564772f451b0 2024-05-14T09:50:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.992+0000 7f05df7fe640 1 --2- 172.21.15.2:0/820676098 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564772f451b0 0x564773368790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.992+0000 7f05beffd640 10 client.16983 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:19.992+0000 7f05df7fe640 1 --2- 172.21.15.2:0/820676098 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564772f451b0 0x564773368790 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed1cb2263978b656 server_cookie=50c4428f46cc056d in_seq=0 out_seq=0 2024-05-14T09:50:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.439+0000 7f05beffd640 1 -- 172.21.15.2:0/820676098 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564773253050 con 0x564772f451b0 2024-05-14T09:50:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.439+0000 7f05beffd640 10 client.16983 handle_command_reply: tid=0 2024-05-14T09:50:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.439+0000 7f05ecd9f740 1 client.16983 shutdown 2024-05-14T09:50:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.439+0000 7f05ecd9f740 1 -- 172.21.15.2:0/820676098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f03d80 msgr2=0x5647732f5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.439+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/820676098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f03d80 0x5647732f5410 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f05d4011d20 tx=0x7f05d0004290 comp rx=0 tx=0).stop 2024-05-14T09:50:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.439+0000 7f05ecd9f740 1 -- 172.21.15.2:0/820676098 shutdown_connections 2024-05-14T09:50:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.439+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/820676098 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564772f451b0 0x564773368790 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.439+0000 7f05beffd640 0 client.16983 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.439+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/820676098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564772f244d0 0x5647732f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.439+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/820676098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564772f1c2f0 0x5647732f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.439+0000 7f05ecd9f740 1 --2- 172.21.15.2:0/820676098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564772f03d80 0x5647732f5410 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.439+0000 7f05ecd9f740 1 -- 172.21.15.2:0/820676098 >> 172.21.15.2:0/820676098 conn(0x564772f39300 msgr2=0x5647732eeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.439+0000 7f05ecd9f740 1 -- 172.21.15.2:0/820676098 shutdown_connections 2024-05-14T09:50:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.439+0000 7f05ecd9f740 1 -- 172.21.15.2:0/820676098 wait complete. 2024-05-14T09:50:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.440+0000 7f05ecd9f740 20 client.16983 trim_cache size 0 max 16384 2024-05-14T09:50:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:50:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:50:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:50:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.440+0000 7f05e5e13640 1 -- 172.21.15.2:0/1283814486 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f05c80b2920 msgr2=0x7f05c80b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.440+0000 7f05e5e13640 1 --2- 172.21.15.2:0/1283814486 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f05c80b2920 0x7f05c80b4e30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f05d80c7380 tx=0x7f05d8093a00 comp rx=0 tx=0).stop 2024-05-14T09:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.440+0000 7f05e5e13640 1 -- 172.21.15.2:0/1283814486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05e01121f0 msgr2=0x7f05e010bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.440+0000 7f05e5e13640 1 --2- 172.21.15.2:0/1283814486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05e01121f0 0x7f05e010bcd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f05d8034040 tx=0x7f05d8030d20 comp rx=0 tx=0).stop 2024-05-14T09:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.440+0000 7f05e5e13640 1 -- 172.21.15.2:0/1283814486 shutdown_connections 2024-05-14T09:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.440+0000 7f05e5e13640 1 --2- 172.21.15.2:0/1283814486 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f05c80b2920 0x7f05c80b4e30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.440+0000 7f05e5e13640 1 --2- 172.21.15.2:0/1283814486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05e01121f0 0x7f05e010bcd0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.440+0000 7f05e5e13640 1 --2- 172.21.15.2:0/1283814486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f05e010f890 0x7f05e010b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.440+0000 7f05e5e13640 1 --2- 172.21.15.2:0/1283814486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e010a0f0 0x7f05e010d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.440+0000 7f05e5e13640 1 -- 172.21.15.2:0/1283814486 >> 172.21.15.2:0/1283814486 conn(0x7f05e0070640 msgr2=0x7f05e01106d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.441+0000 7f05e5e13640 1 -- 172.21.15.2:0/1283814486 shutdown_connections 2024-05-14T09:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:20.441+0000 7f05e5e13640 1 -- 172.21.15.2:0/1283814486 wait complete. 2024-05-14T09:50:20.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 58134 31.18 MB/sec warmup 106 sec latency 359.070 ms 2024-05-14T09:50:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:20 smithi002 ceph-mon[31557]: pgmap v886: 129 pgs: 129 active+clean; 518 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.4 MiB/s wr, 765 op/s 2024-05-14T09:50:20.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:20 smithi029 ceph-mon[36249]: pgmap v886: 129 pgs: 129 active+clean; 518 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.4 MiB/s wr, 765 op/s 2024-05-14T09:50:20.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:20 smithi026 ceph-mon[36056]: pgmap v886: 129 pgs: 129 active+clean; 518 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.4 MiB/s wr, 765 op/s 2024-05-14T09:50:21.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 58624 31.12 MB/sec warmup 107 sec latency 328.294 ms 2024-05-14T09:50:22.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 59119 31.14 MB/sec warmup 108 sec latency 141.761 ms 2024-05-14T09:50:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:22 smithi002 ceph-mon[31557]: pgmap v887: 129 pgs: 129 active+clean; 519 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.3 MiB/s wr, 573 op/s 2024-05-14T09:50:22.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:22 smithi029 ceph-mon[36249]: pgmap v887: 129 pgs: 129 active+clean; 519 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.3 MiB/s wr, 573 op/s 2024-05-14T09:50:22.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:22 smithi026 ceph-mon[36056]: pgmap v887: 129 pgs: 129 active+clean; 519 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.3 MiB/s wr, 573 op/s 2024-05-14T09:50:23.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 59731 31.14 MB/sec warmup 109 sec latency 86.002 ms 2024-05-14T09:50:24.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 60438 31.21 MB/sec warmup 110 sec latency 147.287 ms 2024-05-14T09:50:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:24 smithi002 ceph-mon[31557]: pgmap v888: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 883 op/s 2024-05-14T09:50:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:24 smithi029 ceph-mon[36249]: pgmap v888: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 883 op/s 2024-05-14T09:50:24.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:24 smithi026 ceph-mon[36056]: pgmap v888: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 883 op/s 2024-05-14T09:50:25.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 60988 31.15 MB/sec warmup 111 sec latency 317.296 ms 2024-05-14T09:50:26.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 61546 31.20 MB/sec warmup 112 sec latency 301.037 ms 2024-05-14T09:50:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:26 smithi002 ceph-mon[31557]: pgmap v889: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.5 MiB/s wr, 617 op/s 2024-05-14T09:50:26.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:26 smithi029 ceph-mon[36249]: pgmap v889: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.5 MiB/s wr, 617 op/s 2024-05-14T09:50:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:26 smithi026 ceph-mon[36056]: pgmap v889: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.5 MiB/s wr, 617 op/s 2024-05-14T09:50:27.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 62194 31.20 MB/sec warmup 113 sec latency 126.231 ms 2024-05-14T09:50:28.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 62881 31.25 MB/sec warmup 114 sec latency 82.054 ms 2024-05-14T09:50:28.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:28 smithi002 ceph-mon[31557]: pgmap v890: 129 pgs: 129 active+clean; 526 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.9 MiB/s wr, 737 op/s 2024-05-14T09:50:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:28 smithi029 ceph-mon[36249]: pgmap v890: 129 pgs: 129 active+clean; 526 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.9 MiB/s wr, 737 op/s 2024-05-14T09:50:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:28 smithi026 ceph-mon[36056]: pgmap v890: 129 pgs: 129 active+clean; 526 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.9 MiB/s wr, 737 op/s 2024-05-14T09:50:29.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 63637 31.32 MB/sec warmup 115 sec latency 53.646 ms 2024-05-14T09:50:30.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 64039 31.21 MB/sec warmup 116 sec latency 455.171 ms 2024-05-14T09:50:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:30 smithi002 ceph-mon[31557]: pgmap v891: 129 pgs: 129 active+clean; 516 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.7 MiB/s wr, 722 op/s 2024-05-14T09:50:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:30 smithi029 ceph-mon[36249]: pgmap v891: 129 pgs: 129 active+clean; 516 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.7 MiB/s wr, 722 op/s 2024-05-14T09:50:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:30 smithi026 ceph-mon[36056]: pgmap v891: 129 pgs: 129 active+clean; 516 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.7 MiB/s wr, 722 op/s 2024-05-14T09:50:31.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 64687 31.29 MB/sec warmup 117 sec latency 219.209 ms 2024-05-14T09:50:32.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 65354 31.35 MB/sec warmup 118 sec latency 255.807 ms 2024-05-14T09:50:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:32 smithi002 ceph-mon[31557]: pgmap v892: 129 pgs: 129 active+clean; 518 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.5 MiB/s wr, 668 op/s 2024-05-14T09:50:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:32 smithi029 ceph-mon[36249]: pgmap v892: 129 pgs: 129 active+clean; 518 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.5 MiB/s wr, 668 op/s 2024-05-14T09:50:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:32 smithi026 ceph-mon[36056]: pgmap v892: 129 pgs: 129 active+clean; 518 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.5 MiB/s wr, 668 op/s 2024-05-14T09:50:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.575+0000 7f03cd401640 1 -- 172.21.15.2:0/3487625769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c8104d00 msgr2=0x7f03c8105110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.575+0000 7f03cd401640 1 --2- 172.21.15.2:0/3487625769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c8104d00 0x7f03c8105110 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009900 tx=0x7f03b002a410 comp rx=0 tx=0).stop 2024-05-14T09:50:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.575+0000 7f03cd401640 1 -- 172.21.15.2:0/3487625769 shutdown_connections 2024-05-14T09:50:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.575+0000 7f03cd401640 1 --2- 172.21.15.2:0/3487625769 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03c8109a00 0x7f03c810be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.575+0000 7f03cd401640 1 --2- 172.21.15.2:0/3487625769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c8104d00 0x7f03c8105110 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.575+0000 7f03cd401640 1 --2- 172.21.15.2:0/3487625769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03c8103ac0 0x7f03c81047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.575+0000 7f03cd401640 1 -- 172.21.15.2:0/3487625769 >> 172.21.15.2:0/3487625769 conn(0x7f03c80fecb0 msgr2=0x7f03c8101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.575+0000 7f03cd401640 1 -- 172.21.15.2:0/3487625769 shutdown_connections 2024-05-14T09:50:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.575+0000 7f03cd401640 1 -- 172.21.15.2:0/3487625769 wait complete. 2024-05-14T09:50:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.576+0000 7f03cd401640 1 Processor -- start 2024-05-14T09:50:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.576+0000 7f03cd401640 1 -- start start 2024-05-14T09:50:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.576+0000 7f03cd401640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c8103ac0 0x7f03c819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.576+0000 7f03cd401640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03c8104d00 0x7f03c819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.576+0000 7f03cd401640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03c8109a00 0x7f03c81a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.576+0000 7f03cd401640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f03c810f660 con 0x7f03c8103ac0 2024-05-14T09:50:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.576+0000 7f03cd401640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f03c810f4e0 con 0x7f03c8109a00 2024-05-14T09:50:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.576+0000 7f03cd401640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f03c810f7e0 con 0x7f03c8104d00 2024-05-14T09:50:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.577+0000 7f03c6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c8103ac0 0x7f03c819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.577+0000 7f03c6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c8103ac0 0x7f03c819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37814/0 (socket says 172.21.15.2:37814) 2024-05-14T09:50:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.577+0000 7f03c6ffd640 1 -- 172.21.15.2:0/1747368074 learned_addr learned my addr 172.21.15.2:0/1747368074 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.577+0000 7f03bffff640 1 --2- 172.21.15.2:0/1747368074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03c8104d00 0x7f03c819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.577+0000 7f03c77fe640 1 --2- 172.21.15.2:0/1747368074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03c8109a00 0x7f03c81a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.577+0000 7f03c6ffd640 1 -- 172.21.15.2:0/1747368074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03c8104d00 msgr2=0x7f03c819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.577+0000 7f03c6ffd640 1 --2- 172.21.15.2:0/1747368074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03c8104d00 0x7f03c819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.577+0000 7f03c6ffd640 1 -- 172.21.15.2:0/1747368074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03c8109a00 msgr2=0x7f03c81a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.577+0000 7f03c6ffd640 1 --2- 172.21.15.2:0/1747368074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03c8109a00 0x7f03c81a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.577+0000 7f03c6ffd640 1 -- 172.21.15.2:0/1747368074 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c81a1ce0 con 0x7f03c8103ac0 2024-05-14T09:50:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.577+0000 7f03c6ffd640 1 --2- 172.21.15.2:0/1747368074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c8103ac0 0x7f03c819a1a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f03b800e9f0 tx=0x7f03b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.577+0000 7f03c4ff9640 1 -- 172.21.15.2:0/1747368074 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b800ccc0 con 0x7f03c8103ac0 2024-05-14T09:50:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.577+0000 7f03c4ff9640 1 -- 172.21.15.2:0/1747368074 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b80035f0 con 0x7f03c8103ac0 2024-05-14T09:50:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.577+0000 7f03cd401640 1 -- 172.21.15.2:0/1747368074 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c81a1fd0 con 0x7f03c8103ac0 2024-05-14T09:50:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.577+0000 7f03c4ff9640 1 -- 172.21.15.2:0/1747368074 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b800ce60 con 0x7f03c8103ac0 2024-05-14T09:50:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.578+0000 7f03cd401640 1 -- 172.21.15.2:0/1747368074 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c81a2430 con 0x7f03c8103ac0 2024-05-14T09:50:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.580+0000 7f03d438b740 1 Processor -- start 2024-05-14T09:50:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.580+0000 7f03d438b740 1 -- start start 2024-05-14T09:50:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03d438b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 0x55fa38a5b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03d438b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 0x55fa38a5b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03d438b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386944d0 0x55fa38a624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03d438b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fa389bb5d0 con 0x55fa3868c2f0 2024-05-14T09:50:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03d438b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fa38974890 con 0x55fa386944d0 2024-05-14T09:50:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03d438b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fa389b7590 con 0x55fa386b51b0 2024-05-14T09:50:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03c77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 0x55fa38a5b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03c77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 0x55fa38a5b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:51784/0 (socket says 172.21.15.2:51784) 2024-05-14T09:50:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03c77fe640 1 -- 172.21.15.2:0/1600079096 learned_addr learned my addr 172.21.15.2:0/1600079096 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03bffff640 1 --2- 172.21.15.2:0/1600079096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386944d0 0x55fa38a624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03c77fe640 1 -- 172.21.15.2:0/1600079096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386944d0 msgr2=0x55fa38a624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03c77fe640 1 --2- 172.21.15.2:0/1600079096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386944d0 0x55fa38a624c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03c77fe640 1 -- 172.21.15.2:0/1600079096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 msgr2=0x55fa38a5b5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:50:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03c77fe640 1 --2- 172.21.15.2:0/1600079096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 0x55fa38a5b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03c4ff9640 1 -- 172.21.15.2:0/1747368074 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f03b8004020 con 0x7f03c8103ac0 2024-05-14T09:50:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.581+0000 7f03c77fe640 1 -- 172.21.15.2:0/1600079096 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa386cd210 con 0x55fa386b51b0 2024-05-14T09:50:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.582+0000 7f03c77fe640 1 --2- 172.21.15.2:0/1600079096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 0x55fa38a5b040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f03c00039f0 tx=0x7f03c0003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3832213ee5819627 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.582+0000 7f03bd7fa640 1 -- 172.21.15.2:0/1600079096 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c001a470 con 0x55fa386b51b0 2024-05-14T09:50:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.582+0000 7f03bd7fa640 1 -- 172.21.15.2:0/1600079096 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c0015070 con 0x55fa386b51b0 2024-05-14T09:50:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.582+0000 7f03bd7fa640 1 -- 172.21.15.2:0/1600079096 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c00107e0 con 0x55fa386b51b0 2024-05-14T09:50:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.583+0000 7f03c4ff9640 1 --2- 172.21.15.2:0/1747368074 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f03b40b28a0 0x7f03b40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.583+0000 7f03c4ff9640 1 -- 172.21.15.2:0/1747368074 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f03b80e2900 con 0x7f03c8103ac0 2024-05-14T09:50:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.583+0000 7f03c6ffd640 1 --2- 172.21.15.2:0/1747368074 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f03b40b28a0 0x7f03b40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.583+0000 7f03d438b740 1 -- 172.21.15.2:0/1600079096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 msgr2=0x55fa38a5b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.583+0000 7f03d438b740 1 --2- 172.21.15.2:0/1600079096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 0x55fa38a5b040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f03c00039f0 tx=0x7f03c0003eb0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.584+0000 7f03d438b740 1 -- 172.21.15.2:0/1600079096 shutdown_connections 2024-05-14T09:50:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.584+0000 7f03d438b740 1 --2- 172.21.15.2:0/1600079096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386944d0 0x55fa38a624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.584+0000 7f03d438b740 1 --2- 172.21.15.2:0/1600079096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 0x55fa38a5b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.584+0000 7f03d438b740 1 --2- 172.21.15.2:0/1600079096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 0x55fa38a5b040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.584+0000 7f03d438b740 1 -- 172.21.15.2:0/1600079096 >> 172.21.15.2:0/1600079096 conn(0x55fa386a9300 msgr2=0x55fa38a525c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.584+0000 7f03d438b740 1 -- 172.21.15.2:0/1600079096 shutdown_connections 2024-05-14T09:50:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.584+0000 7f03d438b740 1 -- 172.21.15.2:0/1600079096 wait complete. 2024-05-14T09:50:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.584+0000 7f03c6ffd640 1 --2- 172.21.15.2:0/1747368074 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f03b40b28a0 0x7f03b40b4db0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f03b80a9a90 tx=0x7f03b8003a10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.585+0000 7f03d438b740 1 -- start start 2024-05-14T09:50:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.585+0000 7f03d438b740 1 Processor -- start 2024-05-14T09:50:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.586+0000 7f03d438b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 0x55fa38a64c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.586+0000 7f03d438b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386944d0 0x55fa38a651c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.586+0000 7f03c77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 0x55fa38a64c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.586+0000 7f03c77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 0x55fa38a64c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37824/0 (socket says 172.21.15.2:37824) 2024-05-14T09:50:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.586+0000 7f03c77fe640 1 -- 172.21.15.2:0/1152205872 learned_addr learned my addr 172.21.15.2:0/1152205872 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.586+0000 7f03d438b740 1 --2- 172.21.15.2:0/1152205872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 0x55fa38a6c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.586+0000 7f03bffff640 1 --2- 172.21.15.2:0/1152205872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386944d0 0x55fa38a651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.586+0000 7f03d438b740 1 -- 172.21.15.2:0/1152205872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fa38a62f10 con 0x55fa3868c2f0 2024-05-14T09:50:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.586+0000 7f03d438b740 1 -- 172.21.15.2:0/1152205872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fa38978370 con 0x55fa386944d0 2024-05-14T09:50:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.587+0000 7f03d438b740 1 -- 172.21.15.2:0/1152205872 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55fa38a63090 con 0x55fa386b51b0 2024-05-14T09:50:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.587+0000 7f03c77fe640 1 --2- 172.21.15.2:0/1152205872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 0x55fa38a6c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.587+0000 7f03c77fe640 1 -- 172.21.15.2:0/1152205872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 msgr2=0x55fa38a6c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.587+0000 7f03c77fe640 1 --2- 172.21.15.2:0/1152205872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 0x55fa38a6c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.587+0000 7f03c77fe640 1 -- 172.21.15.2:0/1152205872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386944d0 msgr2=0x55fa38a651c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.587+0000 7f03c77fe640 1 --2- 172.21.15.2:0/1152205872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386944d0 0x55fa38a651c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.587+0000 7f03c77fe640 1 -- 172.21.15.2:0/1152205872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa38799620 con 0x55fa3868c2f0 2024-05-14T09:50:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.587+0000 7f03c77fe640 1 --2- 172.21.15.2:0/1152205872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 0x55fa38a64c80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f03c0005b40 tx=0x7f03c0005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df25e5fb65a7a295 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.588+0000 7f03ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:50:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.588+0000 7f03d438b740 10 client.0 _ll_register_callbacks cb 0x55fa386cca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:50:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.588+0000 7f03d438b740 1 -- 172.21.15.2:0/1152205872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c0015070 con 0x55fa3868c2f0 2024-05-14T09:50:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.588+0000 7f03ae7fc640 1 -- 172.21.15.2:0/1152205872 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0010720 con 0x55fa3868c2f0 2024-05-14T09:50:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.588+0000 7f03ae7fc640 1 -- 172.21.15.2:0/1152205872 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f03c00108c0 con 0x55fa3868c2f0 2024-05-14T09:50:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.589+0000 7f03ae7fc640 1 -- 172.21.15.2:0/1152205872 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f03c0034050 con 0x55fa3868c2f0 2024-05-14T09:50:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.589+0000 7f03d438b740 10 client.16992 fetch_fsmap learned FSMap version 54 2024-05-14T09:50:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.589+0000 7f03d438b740 1 -- 172.21.15.2:0/1152205872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({fsmap=54}) -- 0x55fa38a6edb0 con 0x55fa3868c2f0 2024-05-14T09:50:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.589+0000 7f03ae7fc640 1 -- 172.21.15.2:0/1152205872 <== mon.0 v2:172.21.15.2:3300/0 4 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f03c0010ac0 con 0x55fa3868c2f0 2024-05-14T09:50:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.589+0000 7f03d438b740 10 client.16992 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T09:50:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.589+0000 7f03d438b740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:50:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.589+0000 7f03d438b740 20 client.16992 populate_metadata read hostname 'smithi002' 2024-05-14T09:50:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.589+0000 7f03d438b740 1 --2- 172.21.15.2:0/1152205872 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55fa38673d80 0x55fa38a72a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.589+0000 7f03d438b740 4 client.16992 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:50:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.589+0000 7f03d438b740 1 -- 172.21.15.2:0/1152205872 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa389c91e0 con 0x55fa38673d80 2024-05-14T09:50:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.590+0000 7f03bffff640 1 --2- 172.21.15.2:0/1152205872 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55fa38673d80 0x55fa38a72a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.590+0000 7f03bffff640 1 --2- 172.21.15.2:0/1152205872 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55fa38673d80 0x55fa38a72a60 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d1a4665fba043a4 server_cookie=361fc6e967f9b6d0 in_seq=0 out_seq=0 2024-05-14T09:50:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.590+0000 7f03ae7fc640 10 client.16992 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.591+0000 7f03ae7fc640 1 -- 172.21.15.2:0/1152205872 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa389c91e0 con 0x55fa38673d80 2024-05-14T09:50:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.591+0000 7f03ae7fc640 10 client.16992 handle_command_reply: tid=0 2024-05-14T09:50:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.591+0000 7f03d438b740 1 client.16992 shutdown 2024-05-14T09:50:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.592+0000 7f03d438b740 1 -- 172.21.15.2:0/1152205872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 msgr2=0x55fa38a64c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.592+0000 7f03d438b740 1 --2- 172.21.15.2:0/1152205872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 0x55fa38a64c80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f03c0005b40 tx=0x7f03c0005a70 comp rx=0 tx=0).stop 2024-05-14T09:50:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.592+0000 7f03d438b740 1 -- 172.21.15.2:0/1152205872 shutdown_connections 2024-05-14T09:50:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.592+0000 7f03d438b740 1 --2- 172.21.15.2:0/1152205872 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55fa38673d80 0x55fa38a72a60 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.592+0000 7f03d438b740 1 --2- 172.21.15.2:0/1152205872 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 0x55fa38a6c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.592+0000 7f03d438b740 1 --2- 172.21.15.2:0/1152205872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386944d0 0x55fa38a651c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.592+0000 7f03d438b740 1 --2- 172.21.15.2:0/1152205872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 0x55fa38a64c80 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.592+0000 7f03d438b740 1 -- 172.21.15.2:0/1152205872 >> 172.21.15.2:0/1152205872 conn(0x55fa386a9300 msgr2=0x55fa38a51580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.592+0000 7f03ae7fc640 0 client.16992 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.592+0000 7f03d438b740 1 -- 172.21.15.2:0/1152205872 shutdown_connections 2024-05-14T09:50:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.592+0000 7f03d438b740 1 -- 172.21.15.2:0/1152205872 wait complete. 2024-05-14T09:50:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.593+0000 7f03d438b740 20 client.16992 trim_cache size 0 max 16384 2024-05-14T09:50:33.603 INFO:tasks.workunit.client.0.smithi002.stdout: 10 66048 31.37 MB/sec warmup 119 sec latency 51.174 ms 2024-05-14T09:50:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.618+0000 7f03d438b740 1 Processor -- start 2024-05-14T09:50:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.618+0000 7f03d438b740 1 -- start start 2024-05-14T09:50:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.619+0000 7f03d438b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 0x55fa38a64c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.619+0000 7f03d438b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386944d0 0x55fa38a65160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.619+0000 7f03d438b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 0x55fa38a6c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.619+0000 7f03c77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 0x55fa38a64c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.619+0000 7f03c77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 0x55fa38a64c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37830/0 (socket says 172.21.15.2:37830) 2024-05-14T09:50:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.619+0000 7f03c77fe640 1 -- 172.21.15.2:0/622738016 learned_addr learned my addr 172.21.15.2:0/622738016 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.619+0000 7f03d438b740 1 -- 172.21.15.2:0/622738016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f03c81ad330 con 0x55fa3868c2f0 2024-05-14T09:50:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.619+0000 7f03d438b740 1 -- 172.21.15.2:0/622738016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f03c81aab70 con 0x55fa386944d0 2024-05-14T09:50:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.619+0000 7f03d438b740 1 -- 172.21.15.2:0/622738016 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f03c81a89c0 con 0x55fa386b51b0 2024-05-14T09:50:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.620+0000 7f03c77fe640 1 -- 172.21.15.2:0/622738016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 msgr2=0x55fa38a6c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.620+0000 7f03c77fe640 1 --2- 172.21.15.2:0/622738016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 0x55fa38a6c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.620+0000 7f03c77fe640 1 -- 172.21.15.2:0/622738016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386944d0 msgr2=0x55fa38a65160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:50:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.620+0000 7f03c77fe640 1 --2- 172.21.15.2:0/622738016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386944d0 0x55fa38a65160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.620+0000 7f03c77fe640 1 -- 172.21.15.2:0/622738016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa38a624b0 con 0x55fa3868c2f0 2024-05-14T09:50:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.620+0000 7f03c77fe640 1 --2- 172.21.15.2:0/622738016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 0x55fa38a64c20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009b60 tx=0x7f03c00096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e575ae2b8c331dd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.620+0000 7f03bcff9640 1 -- 172.21.15.2:0/622738016 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0011440 con 0x55fa3868c2f0 2024-05-14T09:50:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.621+0000 7f03bcff9640 1 -- 172.21.15.2:0/622738016 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c0015070 con 0x55fa3868c2f0 2024-05-14T09:50:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.621+0000 7f03bcff9640 1 -- 172.21.15.2:0/622738016 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c001ab90 con 0x55fa3868c2f0 2024-05-14T09:50:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.622+0000 7f03d438b740 1 -- 172.21.15.2:0/622738016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 msgr2=0x55fa38a64c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.622+0000 7f03d438b740 1 --2- 172.21.15.2:0/622738016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 0x55fa38a64c20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009b60 tx=0x7f03c00096a0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.623+0000 7f03d438b740 1 -- 172.21.15.2:0/622738016 shutdown_connections 2024-05-14T09:50:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.623+0000 7f03d438b740 1 --2- 172.21.15.2:0/622738016 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa386b51b0 0x55fa38a6c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.623+0000 7f03d438b740 1 --2- 172.21.15.2:0/622738016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386944d0 0x55fa38a65160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.623+0000 7f03d438b740 1 --2- 172.21.15.2:0/622738016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa3868c2f0 0x55fa38a64c20 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.623+0000 7f03d438b740 1 -- 172.21.15.2:0/622738016 >> 172.21.15.2:0/622738016 conn(0x55fa386a9300 msgr2=0x55fa38a5e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.623+0000 7f03d438b740 1 -- 172.21.15.2:0/622738016 shutdown_connections 2024-05-14T09:50:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.623+0000 7f03d438b740 1 -- 172.21.15.2:0/622738016 wait complete. 2024-05-14T09:50:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.624+0000 7f03d438b740 1 -- start start 2024-05-14T09:50:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.624+0000 7f03d438b740 1 Processor -- start 2024-05-14T09:50:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.624+0000 7f03d438b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa3868c2f0 0x55fa38a5a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.624+0000 7f03d438b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa386944d0 0x55fa38a570f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03d438b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386b51b0 0x55fa38a57630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03c77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa3868c2f0 0x55fa38a5a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03c77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa3868c2f0 0x55fa38a5a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:51802/0 (socket says 172.21.15.2:51802) 2024-05-14T09:50:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03bffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa386944d0 0x55fa38a570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03bffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa386944d0 0x55fa38a570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37840/0 (socket says 172.21.15.2:37840) 2024-05-14T09:50:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03c77fe640 1 -- 172.21.15.2:0/3675558100 learned_addr learned my addr 172.21.15.2:0/3675558100 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03d438b740 1 -- 172.21.15.2:0/3675558100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f03c81a9030 con 0x55fa386944d0 2024-05-14T09:50:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03d438b740 1 -- 172.21.15.2:0/3675558100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f03c81aa4a0 con 0x55fa386b51b0 2024-05-14T09:50:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03d438b740 1 -- 172.21.15.2:0/3675558100 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f03c81a9700 con 0x55fa3868c2f0 2024-05-14T09:50:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03c77fe640 1 --2- 172.21.15.2:0/3675558100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386b51b0 0x55fa38a57630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03d438b740 10 client.0 _ll_register_callbacks cb 0x7f03b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:50:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03c77fe640 1 -- 172.21.15.2:0/3675558100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386b51b0 msgr2=0x55fa38a57630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03c77fe640 1 --2- 172.21.15.2:0/3675558100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386b51b0 0x55fa38a57630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03c77fe640 1 -- 172.21.15.2:0/3675558100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa386944d0 msgr2=0x55fa38a570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03c77fe640 1 --2- 172.21.15.2:0/3675558100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa386944d0 0x55fa38a570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03c77fe640 1 -- 172.21.15.2:0/3675558100 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa38a57ca0 con 0x55fa3868c2f0 2024-05-14T09:50:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03c77fe640 1 --2- 172.21.15.2:0/3675558100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa3868c2f0 0x55fa38a5a520 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f03c0010fc0 tx=0x7f03c0003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d55c97d80e92f2c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.625+0000 7f03ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:50:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.626+0000 7f03ae7fc640 1 -- 172.21.15.2:0/3675558100 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0030040 con 0x55fa3868c2f0 2024-05-14T09:50:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.626+0000 7f03d438b740 1 -- 172.21.15.2:0/3675558100 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c0015070 con 0x55fa3868c2f0 2024-05-14T09:50:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.626+0000 7f03ae7fc640 1 -- 172.21.15.2:0/3675558100 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f03c0010700 con 0x55fa3868c2f0 2024-05-14T09:50:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.626+0000 7f03ae7fc640 1 -- 172.21.15.2:0/3675558100 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f03c0010900 con 0x55fa3868c2f0 2024-05-14T09:50:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.626+0000 7f03d438b740 10 client.26632 fetch_fsmap learned FSMap version 54 2024-05-14T09:50:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.626+0000 7f03d438b740 1 -- 172.21.15.2:0/3675558100 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55fa38a6b060 con 0x55fa3868c2f0 2024-05-14T09:50:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.626+0000 7f03ae7fc640 1 -- 172.21.15.2:0/3675558100 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c0015070 con 0x55fa3868c2f0 2024-05-14T09:50:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.627+0000 7f03ae7fc640 1 -- 172.21.15.2:0/3675558100 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f03c0038c90 con 0x55fa3868c2f0 2024-05-14T09:50:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.627+0000 7f03d438b740 10 client.26632 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T09:50:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.627+0000 7f03d438b740 10 client.26632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:50:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.627+0000 7f03d438b740 20 client.26632 populate_metadata read hostname 'smithi002' 2024-05-14T09:50:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.627+0000 7f03d438b740 1 --2- 172.21.15.2:0/3675558100 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55fa386491c0 0x55fa38a6b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.627+0000 7f03d438b740 4 client.26632 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:50:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.628+0000 7f03d438b740 1 -- 172.21.15.2:0/3675558100 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa389c3050 con 0x55fa386491c0 2024-05-14T09:50:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.628+0000 7f03bffff640 1 --2- 172.21.15.2:0/3675558100 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55fa386491c0 0x55fa38a6b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.628+0000 7f03bffff640 1 --2- 172.21.15.2:0/3675558100 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55fa386491c0 0x55fa38a6b790 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9b7a842aa33b80f server_cookie=851c2a7763e17f7a in_seq=0 out_seq=0 2024-05-14T09:50:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:33.630+0000 7f03ae7fc640 10 client.26632 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.150+0000 7f03ae7fc640 1 -- 172.21.15.2:0/3675558100 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa389c3050 con 0x55fa386491c0 2024-05-14T09:50:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.150+0000 7f03ae7fc640 10 client.26632 handle_command_reply: tid=0 2024-05-14T09:50:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.150+0000 7f03d438b740 1 client.26632 shutdown 2024-05-14T09:50:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.150+0000 7f03d438b740 1 -- 172.21.15.2:0/3675558100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa3868c2f0 msgr2=0x55fa38a5a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.150+0000 7f03d438b740 1 --2- 172.21.15.2:0/3675558100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa3868c2f0 0x55fa38a5a520 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f03c0010fc0 tx=0x7f03c0003eb0 comp rx=0 tx=0).stop 2024-05-14T09:50:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.151+0000 7f03d438b740 1 -- 172.21.15.2:0/3675558100 shutdown_connections 2024-05-14T09:50:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.151+0000 7f03d438b740 1 --2- 172.21.15.2:0/3675558100 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55fa386491c0 0x55fa38a6b790 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.151+0000 7f03ae7fc640 0 client.26632 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.151+0000 7f03d438b740 1 --2- 172.21.15.2:0/3675558100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa386b51b0 0x55fa38a57630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.151+0000 7f03d438b740 1 --2- 172.21.15.2:0/3675558100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa386944d0 0x55fa38a570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.151+0000 7f03d438b740 1 --2- 172.21.15.2:0/3675558100 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fa3868c2f0 0x55fa38a5a520 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.151+0000 7f03d438b740 1 -- 172.21.15.2:0/3675558100 >> 172.21.15.2:0/3675558100 conn(0x55fa386a9300 msgr2=0x55fa38a5f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.151+0000 7f03d438b740 1 -- 172.21.15.2:0/3675558100 shutdown_connections 2024-05-14T09:50:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.151+0000 7f03d438b740 1 -- 172.21.15.2:0/3675558100 wait complete. 2024-05-14T09:50:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.152+0000 7f03d438b740 20 client.26632 trim_cache size 0 max 16384 2024-05-14T09:50:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:50:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:50:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:50:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:50:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.153+0000 7f03cd401640 1 -- 172.21.15.2:0/1747368074 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f03b40b28a0 msgr2=0x7f03b40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.153+0000 7f03cd401640 1 --2- 172.21.15.2:0/1747368074 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f03b40b28a0 0x7f03b40b4db0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f03b80a9a90 tx=0x7f03b8003a10 comp rx=0 tx=0).stop 2024-05-14T09:50:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.153+0000 7f03cd401640 1 -- 172.21.15.2:0/1747368074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c8103ac0 msgr2=0x7f03c819a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.153+0000 7f03cd401640 1 --2- 172.21.15.2:0/1747368074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c8103ac0 0x7f03c819a1a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f03b800e9f0 tx=0x7f03b800eeb0 comp rx=0 tx=0).stop 2024-05-14T09:50:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.153+0000 7f03cd401640 1 -- 172.21.15.2:0/1747368074 shutdown_connections 2024-05-14T09:50:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.153+0000 7f03cd401640 1 --2- 172.21.15.2:0/1747368074 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f03b40b28a0 0x7f03b40b4db0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.153+0000 7f03cd401640 1 --2- 172.21.15.2:0/1747368074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03c8109a00 0x7f03c81a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.153+0000 7f03cd401640 1 --2- 172.21.15.2:0/1747368074 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f03c8104d00 0x7f03c819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.153+0000 7f03cd401640 1 --2- 172.21.15.2:0/1747368074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c8103ac0 0x7f03c819a1a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.153+0000 7f03cd401640 1 -- 172.21.15.2:0/1747368074 >> 172.21.15.2:0/1747368074 conn(0x7f03c80fecb0 msgr2=0x7f03c81059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.153+0000 7f03cd401640 1 -- 172.21.15.2:0/1747368074 shutdown_connections 2024-05-14T09:50:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:34.153+0000 7f03cd401640 1 -- 172.21.15.2:0/1747368074 wait complete. 2024-05-14T09:50:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:34 smithi002 ceph-mon[31557]: pgmap v893: 129 pgs: 129 active+clean; 526 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 836 op/s 2024-05-14T09:50:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:34 smithi029 ceph-mon[36249]: pgmap v893: 129 pgs: 129 active+clean; 526 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 836 op/s 2024-05-14T09:50:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:34 smithi026 ceph-mon[36056]: pgmap v893: 129 pgs: 129 active+clean; 526 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 836 op/s 2024-05-14T09:50:35.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 66929 17.97 MB/sec execute 1 sec latency 348.360 ms 2024-05-14T09:50:36.462 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:36 smithi026 ceph-mon[36056]: pgmap v894: 129 pgs: 129 active+clean; 526 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.3 MiB/s wr, 505 op/s 2024-05-14T09:50:36.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 67502 26.62 MB/sec execute 2 sec latency 164.803 ms 2024-05-14T09:50:36.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:36 smithi002 ceph-mon[31557]: pgmap v894: 129 pgs: 129 active+clean; 526 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.3 MiB/s wr, 505 op/s 2024-05-14T09:50:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:36 smithi029 ceph-mon[36249]: pgmap v894: 129 pgs: 129 active+clean; 526 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.3 MiB/s wr, 505 op/s 2024-05-14T09:50:37.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 67543 18.81 MB/sec execute 3 sec latency 1008.449 ms 2024-05-14T09:50:38.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 67549 14.13 MB/sec execute 4 sec latency 2008.491 ms 2024-05-14T09:50:38.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:38 smithi026 ceph-mon[36056]: pgmap v895: 129 pgs: 129 active+clean; 521 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.4 MiB/s wr, 606 op/s 2024-05-14T09:50:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:38 smithi002 ceph-mon[31557]: pgmap v895: 129 pgs: 129 active+clean; 521 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.4 MiB/s wr, 606 op/s 2024-05-14T09:50:38.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:38 smithi029 ceph-mon[36249]: pgmap v895: 129 pgs: 129 active+clean; 521 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.4 MiB/s wr, 606 op/s 2024-05-14T09:50:39.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 67771 13.54 MB/sec execute 5 sec latency 2177.922 ms 2024-05-14T09:50:40.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 68017 13.10 MB/sec execute 6 sec latency 469.351 ms 2024-05-14T09:50:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:40 smithi002 ceph-mon[31557]: pgmap v896: 129 pgs: 129 active+clean; 594 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 13 MiB/s wr, 810 op/s 2024-05-14T09:50:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:40 smithi029 ceph-mon[36249]: pgmap v896: 129 pgs: 129 active+clean; 594 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 13 MiB/s wr, 810 op/s 2024-05-14T09:50:40.981 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:40 smithi026 ceph-mon[36056]: pgmap v896: 129 pgs: 129 active+clean; 594 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 13 MiB/s wr, 810 op/s 2024-05-14T09:50:41.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 68429 13.39 MB/sec execute 7 sec latency 147.557 ms 2024-05-14T09:50:42.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 68748 14.59 MB/sec execute 8 sec latency 106.970 ms 2024-05-14T09:50:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:42 smithi002 ceph-mon[31557]: pgmap v897: 129 pgs: 129 active+clean; 600 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 729 op/s 2024-05-14T09:50:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:42 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:50:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:42 smithi029 ceph-mon[36249]: pgmap v897: 129 pgs: 129 active+clean; 600 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 729 op/s 2024-05-14T09:50:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:42 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:50:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:42 smithi026 ceph-mon[36056]: pgmap v897: 129 pgs: 129 active+clean; 600 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 729 op/s 2024-05-14T09:50:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:42 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:50:43.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 69255 16.63 MB/sec execute 9 sec latency 161.835 ms 2024-05-14T09:50:44.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 69823 18.09 MB/sec execute 10 sec latency 68.435 ms 2024-05-14T09:50:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:44 smithi002 ceph-mon[31557]: pgmap v898: 129 pgs: 129 active+clean; 530 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 20 MiB/s wr, 1.11k op/s 2024-05-14T09:50:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:44 smithi029 ceph-mon[36249]: pgmap v898: 129 pgs: 129 active+clean; 530 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 20 MiB/s wr, 1.11k op/s 2024-05-14T09:50:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:44 smithi026 ceph-mon[36056]: pgmap v898: 129 pgs: 129 active+clean; 530 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 20 MiB/s wr, 1.11k op/s 2024-05-14T09:50:45.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 70433 19.18 MB/sec execute 11 sec latency 166.008 ms 2024-05-14T09:50:46.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 71062 20.36 MB/sec execute 12 sec latency 167.138 ms 2024-05-14T09:50:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:46 smithi002 ceph-mon[31557]: pgmap v899: 129 pgs: 129 active+clean; 530 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 17 MiB/s wr, 906 op/s 2024-05-14T09:50:46.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:46 smithi029 ceph-mon[36249]: pgmap v899: 129 pgs: 129 active+clean; 530 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 17 MiB/s wr, 906 op/s 2024-05-14T09:50:46.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:46 smithi026 ceph-mon[36056]: pgmap v899: 129 pgs: 129 active+clean; 530 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 17 MiB/s wr, 906 op/s 2024-05-14T09:50:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.292+0000 7f30155f2640 1 -- 172.21.15.2:0/3235215603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30101101e0 msgr2=0x7f3010112600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.292+0000 7f30155f2640 1 --2- 172.21.15.2:0/3235215603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30101101e0 0x7f3010112600 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009900 tx=0x7f2ff802a410 comp rx=0 tx=0).stop 2024-05-14T09:50:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.293+0000 7f30155f2640 1 -- 172.21.15.2:0/3235215603 shutdown_connections 2024-05-14T09:50:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.293+0000 7f30155f2640 1 --2- 172.21.15.2:0/3235215603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30101101e0 0x7f3010112600 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.293+0000 7f30155f2640 1 --2- 172.21.15.2:0/3235215603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f301010b4e0 0x7f301010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.293+0000 7f30155f2640 1 --2- 172.21.15.2:0/3235215603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f301010a2a0 0x7f301010afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.293+0000 7f30155f2640 1 -- 172.21.15.2:0/3235215603 >> 172.21.15.2:0/3235215603 conn(0x7f3010078eb0 msgr2=0x7f3010107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.293+0000 7f30155f2640 1 -- 172.21.15.2:0/3235215603 shutdown_connections 2024-05-14T09:50:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.293+0000 7f30155f2640 1 -- 172.21.15.2:0/3235215603 wait complete. 2024-05-14T09:50:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.294+0000 7f30155f2640 1 Processor -- start 2024-05-14T09:50:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.295+0000 7f30155f2640 1 -- start start 2024-05-14T09:50:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.295+0000 7f30155f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f301010a2a0 0x7f30101a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.295+0000 7f30155f2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f301010b4e0 0x7f30101a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.295+0000 7f30155f2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30101101e0 0x7f30101a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.295+0000 7f30155f2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3010115dc0 con 0x7f301010b4e0 2024-05-14T09:50:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.295+0000 7f30155f2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3010115c40 con 0x7f301010a2a0 2024-05-14T09:50:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.295+0000 7f30067fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f301010b4e0 0x7f30101a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.295+0000 7f300effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f301010a2a0 0x7f30101a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.295+0000 7f300effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f301010a2a0 0x7f30101a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:39612/0 (socket says 172.21.15.2:39612) 2024-05-14T09:50:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.295+0000 7f30067fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f301010b4e0 0x7f30101a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58418/0 (socket says 172.21.15.2:58418) 2024-05-14T09:50:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.295+0000 7f300effd640 1 -- 172.21.15.2:0/3262372304 learned_addr learned my addr 172.21.15.2:0/3262372304 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.295+0000 7f30155f2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3010115f40 con 0x7f30101101e0 2024-05-14T09:50:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.295+0000 7f300f7fe640 1 --2- 172.21.15.2:0/3262372304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30101101e0 0x7f30101a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.296+0000 7f300effd640 1 -- 172.21.15.2:0/3262372304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30101101e0 msgr2=0x7f30101a7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.296+0000 7f300effd640 1 --2- 172.21.15.2:0/3262372304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30101101e0 0x7f30101a7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.296+0000 7f300effd640 1 -- 172.21.15.2:0/3262372304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f301010b4e0 msgr2=0x7f30101a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.296+0000 7f300effd640 1 --2- 172.21.15.2:0/3262372304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f301010b4e0 0x7f30101a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.296+0000 7f300effd640 1 -- 172.21.15.2:0/3262372304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30101a83d0 con 0x7f301010a2a0 2024-05-14T09:50:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.296+0000 7f30067fc640 1 --2- 172.21.15.2:0/3262372304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f301010b4e0 0x7f30101a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:50:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.296+0000 7f300effd640 1 --2- 172.21.15.2:0/3262372304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f301010a2a0 0x7f30101a0890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f300800eb90 tx=0x7f300800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.296+0000 7f300cff9640 1 -- 172.21.15.2:0/3262372304 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008003b70 con 0x7f301010a2a0 2024-05-14T09:50:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.296+0000 7f300cff9640 1 -- 172.21.15.2:0/3262372304 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f300800ee90 con 0x7f301010a2a0 2024-05-14T09:50:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.296+0000 7f300cff9640 1 -- 172.21.15.2:0/3262372304 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30080197b0 con 0x7f301010a2a0 2024-05-14T09:50:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.296+0000 7f30155f2640 1 -- 172.21.15.2:0/3262372304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30101a86c0 con 0x7f301010a2a0 2024-05-14T09:50:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.297+0000 7f30155f2640 1 -- 172.21.15.2:0/3262372304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30101a8b60 con 0x7f301010a2a0 2024-05-14T09:50:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.300+0000 7f301c578740 1 Processor -- start 2024-05-14T09:50:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.300+0000 7f301c578740 1 -- start start 2024-05-14T09:50:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.301+0000 7f301c578740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f353f1b0 0x55f1f38e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.301+0000 7f301c578740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f35162f0 0x55f1f38e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.301+0000 7f301c578740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f351e4d0 0x55f1f38e7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.301+0000 7f301c578740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f1f38455d0 con 0x55f1f353f1b0 2024-05-14T09:50:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.301+0000 7f301c578740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f1f37fe890 con 0x55f1f351e4d0 2024-05-14T09:50:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.301+0000 7f301c578740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f1f3841590 con 0x55f1f35162f0 2024-05-14T09:50:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.301+0000 7f300f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f353f1b0 0x55f1f38e0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.301+0000 7f300f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f353f1b0 0x55f1f38e0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58428/0 (socket says 172.21.15.2:58428) 2024-05-14T09:50:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.301+0000 7f300f7fe640 1 -- 172.21.15.2:0/209176465 learned_addr learned my addr 172.21.15.2:0/209176465 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.301+0000 7f30067fc640 1 --2- 172.21.15.2:0/209176465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f351e4d0 0x55f1f38e7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.301+0000 7f300cff9640 1 -- 172.21.15.2:0/3262372304 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3008004030 con 0x7f301010a2a0 2024-05-14T09:50:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.302+0000 7f300effd640 1 --2- 172.21.15.2:0/209176465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f35162f0 0x55f1f38e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.302+0000 7f300f7fe640 1 -- 172.21.15.2:0/209176465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f35162f0 msgr2=0x55f1f38e0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.302+0000 7f300f7fe640 1 --2- 172.21.15.2:0/209176465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f35162f0 0x55f1f38e0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.302+0000 7f300f7fe640 1 -- 172.21.15.2:0/209176465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f351e4d0 msgr2=0x55f1f38e7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.302+0000 7f300f7fe640 1 --2- 172.21.15.2:0/209176465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f351e4d0 0x55f1f38e7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.302+0000 7f300f7fe640 1 -- 172.21.15.2:0/209176465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1f3557210 con 0x55f1f353f1b0 2024-05-14T09:50:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.302+0000 7f300effd640 1 --2- 172.21.15.2:0/209176465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f35162f0 0x55f1f38e0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:50:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.302+0000 7f30067fc640 1 --2- 172.21.15.2:0/209176465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f351e4d0 0x55f1f38e7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:50:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.302+0000 7f300f7fe640 1 --2- 172.21.15.2:0/209176465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f353f1b0 0x55f1f38e0a60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009900 tx=0x7f2ff8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9a7401ae05af4b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.302+0000 7f30057fa640 1 -- 172.21.15.2:0/209176465 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8040070 con 0x55f1f353f1b0 2024-05-14T09:50:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.303+0000 7f30057fa640 1 -- 172.21.15.2:0/209176465 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff8004030 con 0x55f1f353f1b0 2024-05-14T09:50:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.303+0000 7f300cff9640 1 --2- 172.21.15.2:0/3262372304 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2ffc0b2920 0x7f2ffc0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.303+0000 7f300cff9640 1 -- 172.21.15.2:0/3262372304 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f30080e2e60 con 0x7f301010a2a0 2024-05-14T09:50:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.303+0000 7f300effd640 1 --2- 172.21.15.2:0/3262372304 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2ffc0b2920 0x7f2ffc0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.303+0000 7f30057fa640 1 -- 172.21.15.2:0/209176465 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8035040 con 0x55f1f353f1b0 2024-05-14T09:50:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.304+0000 7f301c578740 1 -- 172.21.15.2:0/209176465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f353f1b0 msgr2=0x55f1f38e0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.304+0000 7f301c578740 1 --2- 172.21.15.2:0/209176465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f353f1b0 0x55f1f38e0a60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009900 tx=0x7f2ff8005e80 comp rx=0 tx=0).stop 2024-05-14T09:50:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.304+0000 7f300effd640 1 --2- 172.21.15.2:0/3262372304 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2ffc0b2920 0x7f2ffc0b4e30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3008000f80 tx=0x7f300800cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.304+0000 7f301c578740 1 -- 172.21.15.2:0/209176465 shutdown_connections 2024-05-14T09:50:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.304+0000 7f301c578740 1 --2- 172.21.15.2:0/209176465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f351e4d0 0x55f1f38e7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.304+0000 7f301c578740 1 --2- 172.21.15.2:0/209176465 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f35162f0 0x55f1f38e0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.304+0000 7f301c578740 1 --2- 172.21.15.2:0/209176465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f353f1b0 0x55f1f38e0a60 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.304+0000 7f301c578740 1 -- 172.21.15.2:0/209176465 >> 172.21.15.2:0/209176465 conn(0x55f1f3533300 msgr2=0x55f1f38dc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.304+0000 7f301c578740 1 -- 172.21.15.2:0/209176465 shutdown_connections 2024-05-14T09:50:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.305+0000 7f301c578740 1 -- 172.21.15.2:0/209176465 wait complete. 2024-05-14T09:50:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.305+0000 7f301c578740 1 -- start start 2024-05-14T09:50:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.305+0000 7f301c578740 1 Processor -- start 2024-05-14T09:50:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.305+0000 7f301c578740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f35162f0 0x55f1f38ea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.305+0000 7f301c578740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f351e4d0 0x55f1f38eab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.305+0000 7f301c578740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f353f1b0 0x55f1f38f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f301c578740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f1f38e8970 con 0x55f1f353f1b0 2024-05-14T09:50:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f301c578740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f1f3802370 con 0x55f1f35162f0 2024-05-14T09:50:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f301c578740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f1f38e8af0 con 0x55f1f351e4d0 2024-05-14T09:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f301c578740 10 client.0 _ll_register_callbacks cb 0x55f1f3556a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f300f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f35162f0 0x55f1f38ea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f300f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f35162f0 0x55f1f38ea650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:39634/0 (socket says 172.21.15.2:39634) 2024-05-14T09:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f300f7fe640 1 -- 172.21.15.2:0/926628869 learned_addr learned my addr 172.21.15.2:0/926628869 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f300f7fe640 1 --2- 172.21.15.2:0/926628869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f353f1b0 0x55f1f38f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f30067fc640 1 --2- 172.21.15.2:0/926628869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f351e4d0 0x55f1f38eab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f300f7fe640 1 -- 172.21.15.2:0/926628869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f351e4d0 msgr2=0x55f1f38eab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f300f7fe640 1 --2- 172.21.15.2:0/926628869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f351e4d0 0x55f1f38eab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f300f7fe640 1 -- 172.21.15.2:0/926628869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f353f1b0 msgr2=0x55f1f38f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f300f7fe640 1 --2- 172.21.15.2:0/926628869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f353f1b0 0x55f1f38f1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f300f7fe640 1 -- 172.21.15.2:0/926628869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1f3623620 con 0x55f1f35162f0 2024-05-14T09:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f300f7fe640 1 --2- 172.21.15.2:0/926628869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f35162f0 0x55f1f38ea650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8005560 tx=0x7f2ff802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3e18de68ad3518b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.306+0000 7f2ff67fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.307+0000 7f2ff67fc640 1 -- 172.21.15.2:0/926628869 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8040070 con 0x55f1f35162f0 2024-05-14T09:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.307+0000 7f301c578740 1 -- 172.21.15.2:0/926628869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff8004030 con 0x55f1f35162f0 2024-05-14T09:50:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.307+0000 7f2ff67fc640 1 -- 172.21.15.2:0/926628869 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f2ff80042a0 con 0x55f1f35162f0 2024-05-14T09:50:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.307+0000 7f2ff67fc640 1 -- 172.21.15.2:0/926628869 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2ff8059050 con 0x55f1f35162f0 2024-05-14T09:50:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.307+0000 7f301c578740 10 client.26645 fetch_fsmap learned FSMap version 55 2024-05-14T09:50:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.307+0000 7f301c578740 1 -- 172.21.15.2:0/926628869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f1f38f4990 con 0x55f1f35162f0 2024-05-14T09:50:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.307+0000 7f2ff67fc640 1 -- 172.21.15.2:0/926628869 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff8004030 con 0x55f1f35162f0 2024-05-14T09:50:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.307+0000 7f2ff67fc640 1 -- 172.21.15.2:0/926628869 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) ==== 8362+0+0 (secure 0 0 0) 0x7f2ff8004cd0 con 0x55f1f35162f0 2024-05-14T09:50:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.307+0000 7f301c578740 10 client.26645 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T09:50:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.307+0000 7f301c578740 10 client.26645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:50:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.307+0000 7f301c578740 20 client.26645 populate_metadata read hostname 'smithi002' 2024-05-14T09:50:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.308+0000 7f301c578740 1 --2- 172.21.15.2:0/926628869 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f1f34fdd80 0x55f1f38f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.308+0000 7f301c578740 4 client.26645 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:50:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.308+0000 7f301c578740 1 -- 172.21.15.2:0/926628869 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1f38531e0 con 0x55f1f34fdd80 2024-05-14T09:50:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.308+0000 7f30067fc640 1 --2- 172.21.15.2:0/926628869 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f1f34fdd80 0x55f1f38f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.308+0000 7f30067fc640 1 --2- 172.21.15.2:0/926628869 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f1f34fdd80 0x55f1f38f7cf0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a9b7e33828794cd server_cookie=f38ff1b0104cb8d8 in_seq=0 out_seq=0 2024-05-14T09:50:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.308+0000 7f2ff67fc640 10 client.26645 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.309+0000 7f2ff67fc640 1 -- 172.21.15.2:0/926628869 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f1f38531e0 con 0x55f1f34fdd80 2024-05-14T09:50:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.309+0000 7f2ff67fc640 10 client.26645 handle_command_reply: tid=0 2024-05-14T09:50:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.309+0000 7f301c578740 1 client.26645 shutdown 2024-05-14T09:50:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.309+0000 7f301c578740 1 -- 172.21.15.2:0/926628869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f35162f0 msgr2=0x55f1f38ea650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.309+0000 7f301c578740 1 --2- 172.21.15.2:0/926628869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f35162f0 0x55f1f38ea650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8005560 tx=0x7f2ff802a410 comp rx=0 tx=0).stop 2024-05-14T09:50:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.309+0000 7f301c578740 1 -- 172.21.15.2:0/926628869 shutdown_connections 2024-05-14T09:50:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.309+0000 7f301c578740 1 --2- 172.21.15.2:0/926628869 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f1f34fdd80 0x55f1f38f7cf0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.309+0000 7f301c578740 1 --2- 172.21.15.2:0/926628869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f353f1b0 0x55f1f38f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.309+0000 7f301c578740 1 --2- 172.21.15.2:0/926628869 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f351e4d0 0x55f1f38eab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.309+0000 7f301c578740 1 --2- 172.21.15.2:0/926628869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f35162f0 0x55f1f38ea650 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.309+0000 7f301c578740 1 -- 172.21.15.2:0/926628869 >> 172.21.15.2:0/926628869 conn(0x55f1f3533300 msgr2=0x55f1f38df300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.309+0000 7f2ff67fc640 0 client.26645 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.310+0000 7f301c578740 1 -- 172.21.15.2:0/926628869 shutdown_connections 2024-05-14T09:50:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.310+0000 7f301c578740 1 -- 172.21.15.2:0/926628869 wait complete. 2024-05-14T09:50:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.310+0000 7f301c578740 20 client.26645 trim_cache size 0 max 16384 2024-05-14T09:50:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.333+0000 7f301c578740 1 Processor -- start 2024-05-14T09:50:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.333+0000 7f301c578740 1 -- start start 2024-05-14T09:50:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f301c578740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f35162f0 0x55f1f38df900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f301c578740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f351e4d0 0x55f1f38f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f301c578740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f353f1b0 0x55f1f38fd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f301c578740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f30101b5bb0 con 0x55f1f351e4d0 2024-05-14T09:50:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f301c578740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f30101b3420 con 0x55f1f353f1b0 2024-05-14T09:50:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f301c578740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30101b1270 con 0x55f1f35162f0 2024-05-14T09:50:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f30067fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f351e4d0 0x55f1f38f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f30067fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f351e4d0 0x55f1f38f1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58444/0 (socket says 172.21.15.2:58444) 2024-05-14T09:50:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f30067fc640 1 -- 172.21.15.2:0/1298698461 learned_addr learned my addr 172.21.15.2:0/1298698461 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f300f7fe640 1 --2- 172.21.15.2:0/1298698461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f353f1b0 0x55f1f38fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f300f7fe640 1 --2- 172.21.15.2:0/1298698461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f35162f0 0x55f1f38df900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f30067fc640 1 -- 172.21.15.2:0/1298698461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f35162f0 msgr2=0x55f1f38df900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f30067fc640 1 --2- 172.21.15.2:0/1298698461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f35162f0 0x55f1f38df900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f30067fc640 1 -- 172.21.15.2:0/1298698461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f353f1b0 msgr2=0x55f1f38fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f30067fc640 1 --2- 172.21.15.2:0/1298698461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f353f1b0 0x55f1f38fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f30067fc640 1 -- 172.21.15.2:0/1298698461 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1f38dcd00 con 0x55f1f351e4d0 2024-05-14T09:50:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.334+0000 7f30067fc640 1 --2- 172.21.15.2:0/1298698461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f351e4d0 0x55f1f38f1a80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x55f1f38eb6c0 tx=0x7f3000005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd76e81f3d2a5934 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.335+0000 7f3004ff9640 1 -- 172.21.15.2:0/1298698461 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000010ae0 con 0x55f1f351e4d0 2024-05-14T09:50:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.335+0000 7f3004ff9640 1 -- 172.21.15.2:0/1298698461 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30000124e0 con 0x55f1f351e4d0 2024-05-14T09:50:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.335+0000 7f3004ff9640 1 -- 172.21.15.2:0/1298698461 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300001f8c0 con 0x55f1f351e4d0 2024-05-14T09:50:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.336+0000 7f301c578740 1 -- 172.21.15.2:0/1298698461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f351e4d0 msgr2=0x55f1f38f1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.336+0000 7f301c578740 1 --2- 172.21.15.2:0/1298698461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f351e4d0 0x55f1f38f1a80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x55f1f38eb6c0 tx=0x7f3000005f90 comp rx=0 tx=0).stop 2024-05-14T09:50:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.336+0000 7f301c578740 1 -- 172.21.15.2:0/1298698461 shutdown_connections 2024-05-14T09:50:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.336+0000 7f301c578740 1 --2- 172.21.15.2:0/1298698461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f353f1b0 0x55f1f38fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.336+0000 7f301c578740 1 --2- 172.21.15.2:0/1298698461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f351e4d0 0x55f1f38f1a80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.336+0000 7f301c578740 1 --2- 172.21.15.2:0/1298698461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f35162f0 0x55f1f38df900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.336+0000 7f301c578740 1 -- 172.21.15.2:0/1298698461 >> 172.21.15.2:0/1298698461 conn(0x55f1f3533300 msgr2=0x55f1f38df1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.336+0000 7f301c578740 1 -- 172.21.15.2:0/1298698461 shutdown_connections 2024-05-14T09:50:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.336+0000 7f301c578740 1 -- 172.21.15.2:0/1298698461 wait complete. 2024-05-14T09:50:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.336+0000 7f301c578740 1 -- start start 2024-05-14T09:50:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.336+0000 7f301c578740 1 Processor -- start 2024-05-14T09:50:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.337+0000 7f301c578740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f35162f0 0x55f1f38f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.337+0000 7f301c578740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f351e4d0 0x55f1f38f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.337+0000 7f301c578740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f353f1b0 0x55f1f395dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.337+0000 7f301c578740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f30101b18e0 con 0x55f1f35162f0 2024-05-14T09:50:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.337+0000 7f301c578740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f30101b2d50 con 0x55f1f351e4d0 2024-05-14T09:50:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.337+0000 7f301c578740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f30101b1fb0 con 0x55f1f353f1b0 2024-05-14T09:50:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.337+0000 7f301c578740 10 client.0 _ll_register_callbacks cb 0x7f3000009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:50:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.337+0000 7f300f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f35162f0 0x55f1f38f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.337+0000 7f300f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f35162f0 0x55f1f38f0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58446/0 (socket says 172.21.15.2:58446) 2024-05-14T09:50:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.337+0000 7f300f7fe640 1 -- 172.21.15.2:0/1602968236 learned_addr learned my addr 172.21.15.2:0/1602968236 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:50:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.337+0000 7f300f7fe640 1 --2- 172.21.15.2:0/1602968236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f353f1b0 0x55f1f395dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.337+0000 7f30067fc640 1 --2- 172.21.15.2:0/1602968236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f351e4d0 0x55f1f38f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.338+0000 7f300f7fe640 1 -- 172.21.15.2:0/1602968236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f353f1b0 msgr2=0x55f1f395dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.338+0000 7f300f7fe640 1 --2- 172.21.15.2:0/1602968236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f353f1b0 0x55f1f395dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.338+0000 7f300f7fe640 1 -- 172.21.15.2:0/1602968236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f351e4d0 msgr2=0x55f1f38f0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:50:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.338+0000 7f300f7fe640 1 --2- 172.21.15.2:0/1602968236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f351e4d0 0x55f1f38f0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.338+0000 7f300f7fe640 1 -- 172.21.15.2:0/1602968236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1f395e540 con 0x55f1f35162f0 2024-05-14T09:50:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.338+0000 7f30067fc640 1 --2- 172.21.15.2:0/1602968236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f351e4d0 0x55f1f38f0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:50:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.338+0000 7f300f7fe640 1 --2- 172.21.15.2:0/1602968236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f35162f0 0x55f1f38f0ab0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8005590 tx=0x7f2ff802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e390ede4f26c73 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:50:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.338+0000 7f2ff67fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:50:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.338+0000 7f2ff67fc640 1 -- 172.21.15.2:0/1602968236 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8040040 con 0x55f1f35162f0 2024-05-14T09:50:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.338+0000 7f2ff67fc640 1 -- 172.21.15.2:0/1602968236 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f2ff80315a0 con 0x55f1f35162f0 2024-05-14T09:50:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.338+0000 7f301c578740 1 -- 172.21.15.2:0/1602968236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30000124e0 con 0x55f1f35162f0 2024-05-14T09:50:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.338+0000 7f2ff67fc640 1 -- 172.21.15.2:0/1602968236 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2ff806e050 con 0x55f1f35162f0 2024-05-14T09:50:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.338+0000 7f301c578740 10 client.17016 fetch_fsmap learned FSMap version 55 2024-05-14T09:50:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.338+0000 7f301c578740 1 -- 172.21.15.2:0/1602968236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f1f395f770 con 0x55f1f35162f0 2024-05-14T09:50:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.339+0000 7f2ff67fc640 1 -- 172.21.15.2:0/1602968236 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30000124e0 con 0x55f1f35162f0 2024-05-14T09:50:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.339+0000 7f2ff67fc640 1 -- 172.21.15.2:0/1602968236 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 55) ==== 8362+0+0 (secure 0 0 0) 0x7f2ff80047a0 con 0x55f1f35162f0 2024-05-14T09:50:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.339+0000 7f301c578740 10 client.17016 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T09:50:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.339+0000 7f301c578740 10 client.17016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:50:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.339+0000 7f301c578740 20 client.17016 populate_metadata read hostname 'smithi002' 2024-05-14T09:50:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.339+0000 7f301c578740 1 --2- 172.21.15.2:0/1602968236 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f1f34d31c0 0x55f1f3962160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:50:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.339+0000 7f301c578740 4 client.17016 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:50:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.339+0000 7f301c578740 1 -- 172.21.15.2:0/1602968236 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1f38e9710 con 0x55f1f34d31c0 2024-05-14T09:50:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.339+0000 7f30067fc640 1 --2- 172.21.15.2:0/1602968236 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f1f34d31c0 0x55f1f3962160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:50:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.340+0000 7f2ff67fc640 10 client.17016 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.340+0000 7f30067fc640 1 --2- 172.21.15.2:0/1602968236 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f1f34d31c0 0x55f1f3962160 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=146907aa25488bba server_cookie=fc31cacd612f9cba in_seq=0 out_seq=0 2024-05-14T09:50:47.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 71657 21.34 MB/sec execute 13 sec latency 297.868 ms 2024-05-14T09:50:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.696+0000 7f2ff67fc640 1 -- 172.21.15.2:0/1602968236 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f1f38e9710 con 0x55f1f34d31c0 2024-05-14T09:50:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.696+0000 7f2ff67fc640 10 client.17016 handle_command_reply: tid=0 2024-05-14T09:50:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.696+0000 7f301c578740 1 client.17016 shutdown 2024-05-14T09:50:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.697+0000 7f301c578740 1 -- 172.21.15.2:0/1602968236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f35162f0 msgr2=0x55f1f38f0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.697+0000 7f301c578740 1 --2- 172.21.15.2:0/1602968236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f35162f0 0x55f1f38f0ab0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8005590 tx=0x7f2ff802a410 comp rx=0 tx=0).stop 2024-05-14T09:50:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.697+0000 7f301c578740 1 -- 172.21.15.2:0/1602968236 shutdown_connections 2024-05-14T09:50:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.697+0000 7f301c578740 1 --2- 172.21.15.2:0/1602968236 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f1f34d31c0 0x55f1f3962160 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.697+0000 7f2ff67fc640 0 client.17016 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:50:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.697+0000 7f301c578740 1 --2- 172.21.15.2:0/1602968236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f1f353f1b0 0x55f1f395dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.697+0000 7f301c578740 1 --2- 172.21.15.2:0/1602968236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f1f351e4d0 0x55f1f38f0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.697+0000 7f301c578740 1 --2- 172.21.15.2:0/1602968236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1f35162f0 0x55f1f38f0ab0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.697+0000 7f301c578740 1 -- 172.21.15.2:0/1602968236 >> 172.21.15.2:0/1602968236 conn(0x55f1f3533300 msgr2=0x55f1f38ea220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.697+0000 7f301c578740 1 -- 172.21.15.2:0/1602968236 shutdown_connections 2024-05-14T09:50:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.698+0000 7f301c578740 1 -- 172.21.15.2:0/1602968236 wait complete. 2024-05-14T09:50:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.698+0000 7f301c578740 20 client.17016 trim_cache size 0 max 16384 2024-05-14T09:50:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:50:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:50:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:50:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:50:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.699+0000 7f30155f2640 1 -- 172.21.15.2:0/3262372304 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2ffc0b2920 msgr2=0x7f2ffc0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.699+0000 7f30155f2640 1 --2- 172.21.15.2:0/3262372304 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2ffc0b2920 0x7f2ffc0b4e30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3008000f80 tx=0x7f300800cf90 comp rx=0 tx=0).stop 2024-05-14T09:50:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.699+0000 7f30155f2640 1 -- 172.21.15.2:0/3262372304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f301010a2a0 msgr2=0x7f30101a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:50:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.699+0000 7f30155f2640 1 --2- 172.21.15.2:0/3262372304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f301010a2a0 0x7f30101a0890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f300800eb90 tx=0x7f300800c6a0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.699+0000 7f30155f2640 1 -- 172.21.15.2:0/3262372304 shutdown_connections 2024-05-14T09:50:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.699+0000 7f30155f2640 1 --2- 172.21.15.2:0/3262372304 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2ffc0b2920 0x7f2ffc0b4e30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.699+0000 7f30155f2640 1 --2- 172.21.15.2:0/3262372304 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f30101101e0 0x7f30101a7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.699+0000 7f30155f2640 1 --2- 172.21.15.2:0/3262372304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f301010b4e0 0x7f30101a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.699+0000 7f30155f2640 1 --2- 172.21.15.2:0/3262372304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f301010a2a0 0x7f30101a0890 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:50:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.699+0000 7f30155f2640 1 -- 172.21.15.2:0/3262372304 >> 172.21.15.2:0/3262372304 conn(0x7f3010078eb0 msgr2=0x7f301010c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:50:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.699+0000 7f30155f2640 1 -- 172.21.15.2:0/3262372304 shutdown_connections 2024-05-14T09:50:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:50:47.699+0000 7f30155f2640 1 -- 172.21.15.2:0/3262372304 wait complete. 2024-05-14T09:50:48.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 72384 22.65 MB/sec execute 14 sec latency 383.717 ms 2024-05-14T09:50:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:48 smithi002 ceph-mon[31557]: pgmap v900: 129 pgs: 129 active+clean; 516 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 17 MiB/s wr, 990 op/s 2024-05-14T09:50:48.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:48 smithi029 ceph-mon[36249]: pgmap v900: 129 pgs: 129 active+clean; 516 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 17 MiB/s wr, 990 op/s 2024-05-14T09:50:48.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:48 smithi026 ceph-mon[36056]: pgmap v900: 129 pgs: 129 active+clean; 516 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 17 MiB/s wr, 990 op/s 2024-05-14T09:50:49.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 73071 23.28 MB/sec execute 15 sec latency 99.185 ms 2024-05-14T09:50:50.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 73749 24.12 MB/sec execute 16 sec latency 143.160 ms 2024-05-14T09:50:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:50 smithi002 ceph-mon[31557]: pgmap v901: 129 pgs: 129 active+clean; 503 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 17 MiB/s wr, 1.05k op/s 2024-05-14T09:50:50.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:50 smithi029 ceph-mon[36249]: pgmap v901: 129 pgs: 129 active+clean; 503 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 17 MiB/s wr, 1.05k op/s 2024-05-14T09:50:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:50 smithi026 ceph-mon[36056]: pgmap v901: 129 pgs: 129 active+clean; 503 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 17 MiB/s wr, 1.05k op/s 2024-05-14T09:50:51.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 74456 25.35 MB/sec execute 17 sec latency 109.998 ms 2024-05-14T09:50:52.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 75223 26.40 MB/sec execute 18 sec latency 67.253 ms 2024-05-14T09:50:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:52 smithi002 ceph-mon[31557]: pgmap v902: 129 pgs: 129 active+clean; 509 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 10 MiB/s wr, 810 op/s 2024-05-14T09:50:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:52 smithi029 ceph-mon[36249]: pgmap v902: 129 pgs: 129 active+clean; 509 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 10 MiB/s wr, 810 op/s 2024-05-14T09:50:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:52 smithi026 ceph-mon[36056]: pgmap v902: 129 pgs: 129 active+clean; 509 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 10 MiB/s wr, 810 op/s 2024-05-14T09:50:53.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 75817 26.48 MB/sec execute 19 sec latency 64.762 ms 2024-05-14T09:50:54.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 76539 26.64 MB/sec execute 20 sec latency 83.846 ms 2024-05-14T09:50:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:54 smithi002 ceph-mon[31557]: pgmap v903: 129 pgs: 129 active+clean; 523 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 974 op/s 2024-05-14T09:50:54.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:54 smithi029 ceph-mon[36249]: pgmap v903: 129 pgs: 129 active+clean; 523 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 974 op/s 2024-05-14T09:50:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:54 smithi026 ceph-mon[36056]: pgmap v903: 129 pgs: 129 active+clean; 523 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 974 op/s 2024-05-14T09:50:55.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 77312 27.59 MB/sec execute 21 sec latency 160.222 ms 2024-05-14T09:50:56.604 INFO:tasks.workunit.client.0.smithi002.stdout: 10 77879 28.00 MB/sec execute 22 sec latency 190.608 ms 2024-05-14T09:50:56.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:56 smithi002 ceph-mon[31557]: pgmap v904: 129 pgs: 129 active+clean; 523 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.3 MiB/s wr, 528 op/s 2024-05-14T09:50:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:56 smithi029 ceph-mon[36249]: pgmap v904: 129 pgs: 129 active+clean; 523 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.3 MiB/s wr, 528 op/s 2024-05-14T09:50:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:56 smithi026 ceph-mon[36056]: pgmap v904: 129 pgs: 129 active+clean; 523 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.3 MiB/s wr, 528 op/s 2024-05-14T09:50:57.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 78581 28.26 MB/sec execute 23 sec latency 67.884 ms 2024-05-14T09:50:58.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 79230 28.47 MB/sec execute 24 sec latency 123.340 ms 2024-05-14T09:50:58.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:58 smithi002 ceph-mon[31557]: pgmap v905: 129 pgs: 129 active+clean; 524 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 550 op/s 2024-05-14T09:50:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:58 smithi029 ceph-mon[36249]: pgmap v905: 129 pgs: 129 active+clean; 524 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 550 op/s 2024-05-14T09:50:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:58 smithi026 ceph-mon[36056]: pgmap v905: 129 pgs: 129 active+clean; 524 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 550 op/s 2024-05-14T09:50:59.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 79893 28.49 MB/sec execute 25 sec latency 84.752 ms 2024-05-14T09:50:59.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:50:59 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:50:59.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:50:59 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:50:59.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:50:59 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:51:00.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 80570 29.05 MB/sec execute 26 sec latency 156.586 ms 2024-05-14T09:51:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.832+0000 7f181d245640 1 -- 172.21.15.2:0/4017394289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18100a72c0 msgr2=0x7f18100a76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.832+0000 7f181d245640 1 --2- 172.21.15.2:0/4017394289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18100a72c0 0x7f18100a76d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f180c00aa10 tx=0x7f180c02b730 comp rx=0 tx=0).stop 2024-05-14T09:51:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.833+0000 7f181d245640 1 -- 172.21.15.2:0/4017394289 shutdown_connections 2024-05-14T09:51:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.833+0000 7f181d245640 1 --2- 172.21.15.2:0/4017394289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18100abfc0 0x7f18100ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.833+0000 7f181d245640 1 --2- 172.21.15.2:0/4017394289 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18100a72c0 0x7f18100a76d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.833+0000 7f181d245640 1 --2- 172.21.15.2:0/4017394289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18100a61c0 0x7f18100a6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.833+0000 7f181d245640 1 -- 172.21.15.2:0/4017394289 >> 172.21.15.2:0/4017394289 conn(0x7f18100a09a0 msgr2=0x7f18100a2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.833+0000 7f181d245640 1 -- 172.21.15.2:0/4017394289 shutdown_connections 2024-05-14T09:51:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.834+0000 7f181d245640 1 -- 172.21.15.2:0/4017394289 wait complete. 2024-05-14T09:51:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.835+0000 7f181d245640 1 Processor -- start 2024-05-14T09:51:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.835+0000 7f181d245640 1 -- start start 2024-05-14T09:51:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.835+0000 7f181d245640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18100a61c0 0x7f1810144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.835+0000 7f181d245640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18100a72c0 0x7f1810144cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.836+0000 7f181d245640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18100abfc0 0x7f181014bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.836+0000 7f181d245640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f18100b0540 con 0x7f18100abfc0 2024-05-14T09:51:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.836+0000 7f181d245640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f18100b03c0 con 0x7f18100a72c0 2024-05-14T09:51:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.836+0000 7f181d245640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f18100b06c0 con 0x7f18100a61c0 2024-05-14T09:51:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.836+0000 7f181ca44640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18100abfc0 0x7f181014bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.837+0000 7f181ca44640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18100abfc0 0x7f181014bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39912/0 (socket says 172.21.15.2:39912) 2024-05-14T09:51:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.837+0000 7f181ca44640 1 -- 172.21.15.2:0/3988984092 learned_addr learned my addr 172.21.15.2:0/3988984092 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.836+0000 7f1817fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18100a61c0 0x7f1810144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.837+0000 7f18177fe640 1 --2- 172.21.15.2:0/3988984092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18100a72c0 0x7f1810144cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.837+0000 7f1817fff640 1 -- 172.21.15.2:0/3988984092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18100a72c0 msgr2=0x7f1810144cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.837+0000 7f1817fff640 1 --2- 172.21.15.2:0/3988984092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18100a72c0 0x7f1810144cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.837+0000 7f1817fff640 1 -- 172.21.15.2:0/3988984092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18100abfc0 msgr2=0x7f181014bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.837+0000 7f1817fff640 1 --2- 172.21.15.2:0/3988984092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18100abfc0 0x7f181014bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.837+0000 7f1817fff640 1 -- 172.21.15.2:0/3988984092 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181014c2c0 con 0x7f18100a61c0 2024-05-14T09:51:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.838+0000 7f1817fff640 1 --2- 172.21.15.2:0/3988984092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18100a61c0 0x7f1810144780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1818000b60 tx=0x7f1818071890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.838+0000 7f18157fa640 1 -- 172.21.15.2:0/3988984092 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818076650 con 0x7f18100a61c0 2024-05-14T09:51:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.838+0000 7f18157fa640 1 -- 172.21.15.2:0/3988984092 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1818076ca0 con 0x7f18100a61c0 2024-05-14T09:51:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.838+0000 7f18157fa640 1 -- 172.21.15.2:0/3988984092 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818077d60 con 0x7f18100a61c0 2024-05-14T09:51:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.839+0000 7f181d245640 1 -- 172.21.15.2:0/3988984092 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181014c5b0 con 0x7f18100a61c0 2024-05-14T09:51:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.839+0000 7f181d245640 1 -- 172.21.15.2:0/3988984092 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181014cb00 con 0x7f18100a61c0 2024-05-14T09:51:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.843+0000 7f1825431740 1 Processor -- start 2024-05-14T09:51:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.844+0000 7f1825431740 1 -- start start 2024-05-14T09:51:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.844+0000 7f1825431740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 0x561259bccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.844+0000 7f1825431740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612598022f0 0x561259bcd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.844+0000 7f1825431740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 0x561259bd40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.845+0000 7f1825431740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561259b315d0 con 0x5612598022f0 2024-05-14T09:51:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.845+0000 7f1825431740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561259aea890 con 0x56125982b1b0 2024-05-14T09:51:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.845+0000 7f1825431740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561259b2d590 con 0x56125980a4d0 2024-05-14T09:51:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.845+0000 7f18177fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 0x561259bd40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.845+0000 7f18177fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 0x561259bd40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:35734/0 (socket says 172.21.15.2:35734) 2024-05-14T09:51:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.845+0000 7f18177fe640 1 -- 172.21.15.2:0/4269767237 learned_addr learned my addr 172.21.15.2:0/4269767237 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.845+0000 7f18157fa640 1 -- 172.21.15.2:0/3988984092 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f181807e030 con 0x7f18100a61c0 2024-05-14T09:51:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.845+0000 7f181ca44640 1 --2- 172.21.15.2:0/4269767237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 0x561259bccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.846+0000 7f18177fe640 1 -- 172.21.15.2:0/4269767237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 msgr2=0x561259bccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.846+0000 7f18177fe640 1 --2- 172.21.15.2:0/4269767237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 0x561259bccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.846+0000 7f18177fe640 1 -- 172.21.15.2:0/4269767237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612598022f0 msgr2=0x561259bcd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.846+0000 7f18177fe640 1 --2- 172.21.15.2:0/4269767237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612598022f0 0x561259bcd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.846+0000 7f18177fe640 1 -- 172.21.15.2:0/4269767237 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561259843210 con 0x56125980a4d0 2024-05-14T09:51:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.846+0000 7f18177fe640 1 --2- 172.21.15.2:0/4269767237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 0x561259bd40b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f180c030610 tx=0x7f180c030cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae6b5857d93f518a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.847+0000 7f18027fc640 1 -- 172.21.15.2:0/4269767237 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c036040 con 0x56125980a4d0 2024-05-14T09:51:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.847+0000 7f18027fc640 1 -- 172.21.15.2:0/4269767237 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f180c0308b0 con 0x56125980a4d0 2024-05-14T09:51:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.847+0000 7f18027fc640 1 -- 172.21.15.2:0/4269767237 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c003ce0 con 0x56125980a4d0 2024-05-14T09:51:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.847+0000 7f18157fa640 1 --2- 172.21.15.2:0/3988984092 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f18080b28f0 0x7f18080b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.847+0000 7f18157fa640 1 -- 172.21.15.2:0/3988984092 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f18181461a0 con 0x7f18100a61c0 2024-05-14T09:51:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.849+0000 7f1825431740 1 -- 172.21.15.2:0/4269767237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 msgr2=0x561259bd40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.849+0000 7f1825431740 1 --2- 172.21.15.2:0/4269767237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 0x561259bd40b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f180c030610 tx=0x7f180c030cb0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.849+0000 7f181ca44640 1 --2- 172.21.15.2:0/3988984092 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f18080b28f0 0x7f18080b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.849+0000 7f1825431740 1 -- 172.21.15.2:0/4269767237 shutdown_connections 2024-05-14T09:51:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.849+0000 7f1825431740 1 --2- 172.21.15.2:0/4269767237 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 0x561259bd40b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.849+0000 7f1825431740 1 --2- 172.21.15.2:0/4269767237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612598022f0 0x561259bcd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.849+0000 7f1825431740 1 --2- 172.21.15.2:0/4269767237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 0x561259bccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.849+0000 7f1825431740 1 -- 172.21.15.2:0/4269767237 >> 172.21.15.2:0/4269767237 conn(0x56125981f300 msgr2=0x561259bc85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.849+0000 7f181ca44640 1 --2- 172.21.15.2:0/3988984092 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f18080b28f0 0x7f18080b4e00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x561259bcde00 tx=0x7f1804005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.852+0000 7f1825431740 1 -- 172.21.15.2:0/4269767237 shutdown_connections 2024-05-14T09:51:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.852+0000 7f1825431740 1 -- 172.21.15.2:0/4269767237 wait complete. 2024-05-14T09:51:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.853+0000 7f1825431740 1 -- start start 2024-05-14T09:51:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.853+0000 7f1825431740 1 Processor -- start 2024-05-14T09:51:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.854+0000 7f1825431740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5612598022f0 0x561259be78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.854+0000 7f1825431740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56125980a4d0 0x561259be7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.854+0000 7f1825431740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 0x561259beed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.854+0000 7f1825431740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561259be5ab0 con 0x56125980a4d0 2024-05-14T09:51:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.854+0000 7f1825431740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561259aee370 con 0x56125982b1b0 2024-05-14T09:51:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.854+0000 7f1825431740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561259be5c30 con 0x5612598022f0 2024-05-14T09:51:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.854+0000 7f1825431740 10 client.0 _ll_register_callbacks cb 0x561259842a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:51:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.855+0000 7f18177fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56125980a4d0 0x561259be7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.855+0000 7f18177fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56125980a4d0 0x561259be7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39934/0 (socket says 172.21.15.2:39934) 2024-05-14T09:51:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.855+0000 7f18177fe640 1 -- 172.21.15.2:0/3220201669 learned_addr learned my addr 172.21.15.2:0/3220201669 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.855+0000 7f1817fff640 1 --2- 172.21.15.2:0/3220201669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5612598022f0 0x561259be78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.855+0000 7f181ca44640 1 --2- 172.21.15.2:0/3220201669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 0x561259beed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.856+0000 7f18177fe640 1 -- 172.21.15.2:0/3220201669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5612598022f0 msgr2=0x561259be78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.856+0000 7f18177fe640 1 --2- 172.21.15.2:0/3220201669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5612598022f0 0x561259be78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.856+0000 7f18177fe640 1 -- 172.21.15.2:0/3220201669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 msgr2=0x561259beed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.856+0000 7f18177fe640 1 --2- 172.21.15.2:0/3220201669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 0x561259beed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.856+0000 7f18177fe640 1 -- 172.21.15.2:0/3220201669 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56125990f620 con 0x56125980a4d0 2024-05-14T09:51:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.856+0000 7f181ca44640 1 --2- 172.21.15.2:0/3220201669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 0x561259beed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:51:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.856+0000 7f18177fe640 1 --2- 172.21.15.2:0/3220201669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56125980a4d0 0x561259be7e00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f180c030ba0 tx=0x7f180c030ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd51204abcfc8610 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.856+0000 7f17f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:51:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.857+0000 7f17f77fe640 1 -- 172.21.15.2:0/3220201669 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c036040 con 0x56125980a4d0 2024-05-14T09:51:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.857+0000 7f17f77fe640 1 -- 172.21.15.2:0/3220201669 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f180c003e70 con 0x56125980a4d0 2024-05-14T09:51:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.857+0000 7f1825431740 1 -- 172.21.15.2:0/3220201669 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f180c0308b0 con 0x56125980a4d0 2024-05-14T09:51:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.858+0000 7f17f77fe640 1 -- 172.21.15.2:0/3220201669 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f180c004070 con 0x56125980a4d0 2024-05-14T09:51:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.859+0000 7f1825431740 10 client.17025 fetch_fsmap learned FSMap version 55 2024-05-14T09:51:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.859+0000 7f1825431740 1 -- 172.21.15.2:0/3220201669 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x561259bf1c90 con 0x56125980a4d0 2024-05-14T09:51:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.860+0000 7f17f77fe640 1 -- 172.21.15.2:0/3220201669 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f180c0308b0 con 0x56125980a4d0 2024-05-14T09:51:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.860+0000 7f17f77fe640 1 -- 172.21.15.2:0/3220201669 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 55) ==== 8362+0+0 (secure 0 0 0) 0x7f180c00b040 con 0x56125980a4d0 2024-05-14T09:51:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.860+0000 7f1825431740 10 client.17025 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T09:51:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.860+0000 7f1825431740 10 client.17025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:51:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.860+0000 7f1825431740 20 client.17025 populate_metadata read hostname 'smithi002' 2024-05-14T09:51:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.860+0000 7f1825431740 1 --2- 172.21.15.2:0/3220201669 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5612597e9d80 0x561259bf55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.860+0000 7f1825431740 4 client.17025 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:51:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.860+0000 7f1825431740 1 -- 172.21.15.2:0/3220201669 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561259b3f1e0 con 0x5612597e9d80 2024-05-14T09:51:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.861+0000 7f1817fff640 1 --2- 172.21.15.2:0/3220201669 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5612597e9d80 0x561259bf55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.862+0000 7f17f77fe640 10 client.17025 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.862+0000 7f1817fff640 1 --2- 172.21.15.2:0/3220201669 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5612597e9d80 0x561259bf55a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d99ba2538b265f3 server_cookie=55deecb665932eef in_seq=0 out_seq=0 2024-05-14T09:51:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.864+0000 7f17f77fe640 1 -- 172.21.15.2:0/3220201669 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561259b3f1e0 con 0x5612597e9d80 2024-05-14T09:51:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.864+0000 7f17f77fe640 10 client.17025 handle_command_reply: tid=0 2024-05-14T09:51:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.864+0000 7f1825431740 1 client.17025 shutdown 2024-05-14T09:51:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.864+0000 7f1825431740 1 -- 172.21.15.2:0/3220201669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56125980a4d0 msgr2=0x561259be7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.864+0000 7f1825431740 1 --2- 172.21.15.2:0/3220201669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56125980a4d0 0x561259be7e00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f180c030ba0 tx=0x7f180c030ee0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.864+0000 7f1825431740 1 -- 172.21.15.2:0/3220201669 shutdown_connections 2024-05-14T09:51:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.864+0000 7f1825431740 1 --2- 172.21.15.2:0/3220201669 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5612597e9d80 0x561259bf55a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.864+0000 7f1825431740 1 --2- 172.21.15.2:0/3220201669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 0x561259beed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.864+0000 7f1825431740 1 --2- 172.21.15.2:0/3220201669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56125980a4d0 0x561259be7e00 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.864+0000 7f1825431740 1 --2- 172.21.15.2:0/3220201669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5612598022f0 0x561259be78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.864+0000 7f1825431740 1 -- 172.21.15.2:0/3220201669 >> 172.21.15.2:0/3220201669 conn(0x56125981f300 msgr2=0x561259bc7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.864+0000 7f17f77fe640 0 client.17025 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.864+0000 7f1825431740 1 -- 172.21.15.2:0/3220201669 shutdown_connections 2024-05-14T09:51:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.864+0000 7f1825431740 1 -- 172.21.15.2:0/3220201669 wait complete. 2024-05-14T09:51:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.864+0000 7f1825431740 20 client.17025 trim_cache size 0 max 16384 2024-05-14T09:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.882+0000 7f1825431740 1 Processor -- start 2024-05-14T09:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.882+0000 7f1825431740 1 -- start start 2024-05-14T09:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.882+0000 7f1825431740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612598022f0 0x561259bc8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.882+0000 7f1825431740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 0x561259beec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.882+0000 7f1825431740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56125982b1b0 0x561259bc6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.882+0000 7f1825431740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1810151210 con 0x56125982b1b0 2024-05-14T09:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.882+0000 7f1825431740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f181014ea80 con 0x5612598022f0 2024-05-14T09:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.882+0000 7f1825431740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f18100052a0 con 0x56125980a4d0 2024-05-14T09:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.882+0000 7f181ca44640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56125982b1b0 0x561259bc6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.882+0000 7f181ca44640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56125982b1b0 0x561259bc6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39946/0 (socket says 172.21.15.2:39946) 2024-05-14T09:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.882+0000 7f18177fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 0x561259beec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.882+0000 7f181ca44640 1 -- 172.21.15.2:0/3468846346 learned_addr learned my addr 172.21.15.2:0/3468846346 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.883+0000 7f1817fff640 1 --2- 172.21.15.2:0/3468846346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612598022f0 0x561259bc8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.883+0000 7f181ca44640 1 -- 172.21.15.2:0/3468846346 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 msgr2=0x561259beec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.883+0000 7f181ca44640 1 --2- 172.21.15.2:0/3468846346 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 0x561259beec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.883+0000 7f181ca44640 1 -- 172.21.15.2:0/3468846346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612598022f0 msgr2=0x561259bc8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.883+0000 7f181ca44640 1 --2- 172.21.15.2:0/3468846346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612598022f0 0x561259bc8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.883+0000 7f181ca44640 1 -- 172.21.15.2:0/3468846346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561259bcde30 con 0x56125982b1b0 2024-05-14T09:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.883+0000 7f181ca44640 1 --2- 172.21.15.2:0/3468846346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56125982b1b0 0x561259bc6b50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1804005eb0 tx=0x7f1804016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed74b4965c28fcb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.883+0000 7f1801ffb640 1 -- 172.21.15.2:0/3468846346 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180401c430 con 0x56125982b1b0 2024-05-14T09:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.883+0000 7f1801ffb640 1 -- 172.21.15.2:0/3468846346 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f180401ca80 con 0x56125982b1b0 2024-05-14T09:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.883+0000 7f1801ffb640 1 -- 172.21.15.2:0/3468846346 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804026a70 con 0x56125982b1b0 2024-05-14T09:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.884+0000 7f1825431740 1 -- 172.21.15.2:0/3468846346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56125982b1b0 msgr2=0x561259bc6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.884+0000 7f1825431740 1 --2- 172.21.15.2:0/3468846346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56125982b1b0 0x561259bc6b50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1804005eb0 tx=0x7f1804016000 comp rx=0 tx=0).stop 2024-05-14T09:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.884+0000 7f1825431740 1 -- 172.21.15.2:0/3468846346 shutdown_connections 2024-05-14T09:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.884+0000 7f1825431740 1 --2- 172.21.15.2:0/3468846346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56125982b1b0 0x561259bc6b50 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.884+0000 7f1825431740 1 --2- 172.21.15.2:0/3468846346 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 0x561259beec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.884+0000 7f1825431740 1 --2- 172.21.15.2:0/3468846346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612598022f0 0x561259bc8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.884+0000 7f1825431740 1 -- 172.21.15.2:0/3468846346 >> 172.21.15.2:0/3468846346 conn(0x56125981f300 msgr2=0x561259bc7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.884+0000 7f1825431740 1 -- 172.21.15.2:0/3468846346 shutdown_connections 2024-05-14T09:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.884+0000 7f1825431740 1 -- 172.21.15.2:0/3468846346 wait complete. 2024-05-14T09:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.885+0000 7f1825431740 1 -- start start 2024-05-14T09:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.885+0000 7f1825431740 1 Processor -- start 2024-05-14T09:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.885+0000 7f1825431740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612598022f0 0x561259bee920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.885+0000 7f1825431740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 0x561259beb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.885+0000 7f1825431740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 0x561259bebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.885+0000 7f1825431740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f181014cf40 con 0x5612598022f0 2024-05-14T09:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.885+0000 7f1825431740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f181014e3b0 con 0x56125982b1b0 2024-05-14T09:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.885+0000 7f1825431740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f181014d610 con 0x56125980a4d0 2024-05-14T09:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.885+0000 7f1825431740 10 client.0 _ll_register_callbacks cb 0x7f180c00a890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.885+0000 7f1817fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612598022f0 0x561259bee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.885+0000 7f18177fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 0x561259beb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.885+0000 7f1817fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612598022f0 0x561259bee920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39958/0 (socket says 172.21.15.2:39958) 2024-05-14T09:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.885+0000 7f1817fff640 1 -- 172.21.15.2:0/2965163342 learned_addr learned my addr 172.21.15.2:0/2965163342 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.885+0000 7f181ca44640 1 --2- 172.21.15.2:0/2965163342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 0x561259bebaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.886+0000 7f1817fff640 1 -- 172.21.15.2:0/2965163342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 msgr2=0x561259beb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.886+0000 7f1817fff640 1 --2- 172.21.15.2:0/2965163342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 0x561259beb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.886+0000 7f1817fff640 1 -- 172.21.15.2:0/2965163342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 msgr2=0x561259bebaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.886+0000 7f1817fff640 1 --2- 172.21.15.2:0/2965163342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 0x561259bebaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.886+0000 7f1817fff640 1 -- 172.21.15.2:0/2965163342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561259bebfe0 con 0x5612598022f0 2024-05-14T09:51:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.886+0000 7f18177fe640 1 --2- 172.21.15.2:0/2965163342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 0x561259beb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:51:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.886+0000 7f1817fff640 1 --2- 172.21.15.2:0/2965163342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612598022f0 0x561259bee920 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f181810a640 tx=0x7f1818077850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c985ab89a76f167 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.886+0000 7f17f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:51:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.886+0000 7f17f77fe640 1 -- 172.21.15.2:0/2965163342 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181810dad0 con 0x5612598022f0 2024-05-14T09:51:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.886+0000 7f17f77fe640 1 -- 172.21.15.2:0/2965163342 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f18180774b0 con 0x5612598022f0 2024-05-14T09:51:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.886+0000 7f1825431740 1 -- 172.21.15.2:0/2965163342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f180401ca80 con 0x5612598022f0 2024-05-14T09:51:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.886+0000 7f17f77fe640 1 -- 172.21.15.2:0/2965163342 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f181810d3d0 con 0x5612598022f0 2024-05-14T09:51:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.886+0000 7f1825431740 10 client.17031 fetch_fsmap learned FSMap version 55 2024-05-14T09:51:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.886+0000 7f1825431740 1 -- 172.21.15.2:0/2965163342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x561259c5c860 con 0x5612598022f0 2024-05-14T09:51:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.886+0000 7f17f77fe640 1 -- 172.21.15.2:0/2965163342 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f180401ca80 con 0x5612598022f0 2024-05-14T09:51:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.887+0000 7f17f77fe640 1 -- 172.21.15.2:0/2965163342 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 55) ==== 8362+0+0 (secure 0 0 0) 0x7f1818143ce0 con 0x5612598022f0 2024-05-14T09:51:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.887+0000 7f1825431740 10 client.17031 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T09:51:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.887+0000 7f1825431740 10 client.17031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:51:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.887+0000 7f1825431740 20 client.17031 populate_metadata read hostname 'smithi002' 2024-05-14T09:51:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.887+0000 7f1825431740 1 --2- 172.21.15.2:0/2965163342 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5612597bf1c0 0x561259c5f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.887+0000 7f1825431740 4 client.17031 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:51:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.887+0000 7f1825431740 1 -- 172.21.15.2:0/2965163342 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561259b39050 con 0x5612597bf1c0 2024-05-14T09:51:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.887+0000 7f18177fe640 1 --2- 172.21.15.2:0/2965163342 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5612597bf1c0 0x561259c5f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.887+0000 7f17f77fe640 10 client.17031 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:00.888+0000 7f18177fe640 1 --2- 172.21.15.2:0/2965163342 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5612597bf1c0 0x561259c5f230 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6c1254809eb570d server_cookie=79c8889751dec775 in_seq=0 out_seq=0 2024-05-14T09:51:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:00 smithi002 ceph-mon[31557]: pgmap v906: 129 pgs: 129 active+clean; 538 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 638 op/s 2024-05-14T09:51:00.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:00 smithi029 ceph-mon[36249]: pgmap v906: 129 pgs: 129 active+clean; 538 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 638 op/s 2024-05-14T09:51:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:00 smithi026 ceph-mon[36056]: pgmap v906: 129 pgs: 129 active+clean; 538 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 638 op/s 2024-05-14T09:51:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.431+0000 7f17f77fe640 1 -- 172.21.15.2:0/2965163342 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561259b39050 con 0x5612597bf1c0 2024-05-14T09:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.431+0000 7f17f77fe640 10 client.17031 handle_command_reply: tid=0 2024-05-14T09:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.431+0000 7f1825431740 1 client.17031 shutdown 2024-05-14T09:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.431+0000 7f1825431740 1 -- 172.21.15.2:0/2965163342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612598022f0 msgr2=0x561259bee920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.431+0000 7f1825431740 1 --2- 172.21.15.2:0/2965163342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612598022f0 0x561259bee920 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f181810a640 tx=0x7f1818077850 comp rx=0 tx=0).stop 2024-05-14T09:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.431+0000 7f1825431740 1 -- 172.21.15.2:0/2965163342 shutdown_connections 2024-05-14T09:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.431+0000 7f1825431740 1 --2- 172.21.15.2:0/2965163342 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5612597bf1c0 0x561259c5f230 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.432+0000 7f1825431740 1 --2- 172.21.15.2:0/2965163342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56125982b1b0 0x561259bebaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.432+0000 7f17f77fe640 0 client.17031 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.432+0000 7f1825431740 1 --2- 172.21.15.2:0/2965163342 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56125980a4d0 0x561259beb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.432+0000 7f1825431740 1 --2- 172.21.15.2:0/2965163342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612598022f0 0x561259bee920 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.432+0000 7f1825431740 1 -- 172.21.15.2:0/2965163342 >> 172.21.15.2:0/2965163342 conn(0x56125981f300 msgr2=0x561259bcb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.432+0000 7f1825431740 1 -- 172.21.15.2:0/2965163342 shutdown_connections 2024-05-14T09:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.432+0000 7f1825431740 1 -- 172.21.15.2:0/2965163342 wait complete. 2024-05-14T09:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.432+0000 7f1825431740 20 client.17031 trim_cache size 0 max 16384 2024-05-14T09:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:51:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.433+0000 7f181d245640 1 -- 172.21.15.2:0/3988984092 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f18080b28f0 msgr2=0x7f18080b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.433+0000 7f181d245640 1 --2- 172.21.15.2:0/3988984092 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f18080b28f0 0x7f18080b4e00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x561259bcde00 tx=0x7f1804005f90 comp rx=0 tx=0).stop 2024-05-14T09:51:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.433+0000 7f181d245640 1 -- 172.21.15.2:0/3988984092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18100a61c0 msgr2=0x7f1810144780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.433+0000 7f181d245640 1 --2- 172.21.15.2:0/3988984092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18100a61c0 0x7f1810144780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1818000b60 tx=0x7f1818071890 comp rx=0 tx=0).stop 2024-05-14T09:51:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.433+0000 7f181d245640 1 -- 172.21.15.2:0/3988984092 shutdown_connections 2024-05-14T09:51:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.433+0000 7f181d245640 1 --2- 172.21.15.2:0/3988984092 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f18080b28f0 0x7f18080b4e00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.433+0000 7f181d245640 1 --2- 172.21.15.2:0/3988984092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18100abfc0 0x7f181014bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.433+0000 7f181d245640 1 --2- 172.21.15.2:0/3988984092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18100a72c0 0x7f1810144cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.433+0000 7f181d245640 1 --2- 172.21.15.2:0/3988984092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f18100a61c0 0x7f1810144780 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.433+0000 7f181d245640 1 -- 172.21.15.2:0/3988984092 >> 172.21.15.2:0/3988984092 conn(0x7f18100a09a0 msgr2=0x7f18100ac440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.433+0000 7f181d245640 1 -- 172.21.15.2:0/3988984092 shutdown_connections 2024-05-14T09:51:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:01.433+0000 7f181d245640 1 -- 172.21.15.2:0/3988984092 wait complete. 2024-05-14T09:51:01.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 81037 29.11 MB/sec execute 27 sec latency 448.974 ms 2024-05-14T09:51:02.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 81591 29.16 MB/sec execute 28 sec latency 75.266 ms 2024-05-14T09:51:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:02 smithi002 ceph-mon[31557]: pgmap v907: 129 pgs: 129 active+clean; 539 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.7 MiB/s wr, 476 op/s 2024-05-14T09:51:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:02 smithi029 ceph-mon[36249]: pgmap v907: 129 pgs: 129 active+clean; 539 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.7 MiB/s wr, 476 op/s 2024-05-14T09:51:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:02 smithi026 ceph-mon[36056]: pgmap v907: 129 pgs: 129 active+clean; 539 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.7 MiB/s wr, 476 op/s 2024-05-14T09:51:03.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 82201 29.00 MB/sec execute 29 sec latency 60.662 ms 2024-05-14T09:51:04.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 82708 29.00 MB/sec execute 30 sec latency 111.157 ms 2024-05-14T09:51:04.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:04 smithi002 ceph-mon[31557]: pgmap v908: 129 pgs: 129 active+clean; 545 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.7 MiB/s wr, 684 op/s 2024-05-14T09:51:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:04 smithi029 ceph-mon[36249]: pgmap v908: 129 pgs: 129 active+clean; 545 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.7 MiB/s wr, 684 op/s 2024-05-14T09:51:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:04 smithi026 ceph-mon[36056]: pgmap v908: 129 pgs: 129 active+clean; 545 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.7 MiB/s wr, 684 op/s 2024-05-14T09:51:05.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 83210 29.02 MB/sec execute 31 sec latency 339.707 ms 2024-05-14T09:51:06.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 83798 28.94 MB/sec execute 32 sec latency 117.437 ms 2024-05-14T09:51:06.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:06 smithi002 ceph-mon[31557]: pgmap v909: 129 pgs: 129 active+clean; 545 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.2 MiB/s wr, 485 op/s 2024-05-14T09:51:06.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:06 smithi029 ceph-mon[36249]: pgmap v909: 129 pgs: 129 active+clean; 545 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.2 MiB/s wr, 485 op/s 2024-05-14T09:51:06.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:06 smithi026 ceph-mon[36056]: pgmap v909: 129 pgs: 129 active+clean; 545 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.2 MiB/s wr, 485 op/s 2024-05-14T09:51:07.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 84239 28.84 MB/sec execute 33 sec latency 375.554 ms 2024-05-14T09:51:08.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 84280 28.02 MB/sec execute 34 sec latency 1145.947 ms 2024-05-14T09:51:08.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:08 smithi002 ceph-mon[31557]: pgmap v910: 129 pgs: 129 active+clean; 539 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.4 MiB/s wr, 514 op/s 2024-05-14T09:51:08.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:08 smithi029 ceph-mon[36249]: pgmap v910: 129 pgs: 129 active+clean; 539 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.4 MiB/s wr, 514 op/s 2024-05-14T09:51:08.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:08 smithi026 ceph-mon[36056]: pgmap v910: 129 pgs: 129 active+clean; 539 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.4 MiB/s wr, 514 op/s 2024-05-14T09:51:09.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 84374 27.43 MB/sec execute 35 sec latency 1952.208 ms 2024-05-14T09:51:10.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 84714 27.23 MB/sec execute 36 sec latency 2059.179 ms 2024-05-14T09:51:10.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:10 smithi002 ceph-mon[31557]: pgmap v911: 129 pgs: 129 active+clean; 580 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.1 MiB/s wr, 695 op/s 2024-05-14T09:51:10.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:10 smithi029 ceph-mon[36249]: pgmap v911: 129 pgs: 129 active+clean; 580 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.1 MiB/s wr, 695 op/s 2024-05-14T09:51:10.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:10 smithi026 ceph-mon[36056]: pgmap v911: 129 pgs: 129 active+clean; 580 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.1 MiB/s wr, 695 op/s 2024-05-14T09:51:11.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 85064 26.96 MB/sec execute 37 sec latency 114.697 ms 2024-05-14T09:51:11.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:11 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:11.950 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:11 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:11.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:11 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:12.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 85490 26.99 MB/sec execute 38 sec latency 76.419 ms 2024-05-14T09:51:12.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:12 smithi002 ceph-mon[31557]: pgmap v912: 129 pgs: 129 active+clean; 584 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.5 MiB/s wr, 546 op/s 2024-05-14T09:51:12.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:12 smithi029 ceph-mon[36249]: pgmap v912: 129 pgs: 129 active+clean; 584 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.5 MiB/s wr, 546 op/s 2024-05-14T09:51:12.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:12 smithi026 ceph-mon[36056]: pgmap v912: 129 pgs: 129 active+clean; 584 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.5 MiB/s wr, 546 op/s 2024-05-14T09:51:13.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 86095 27.02 MB/sec execute 39 sec latency 74.684 ms 2024-05-14T09:51:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.563+0000 7f0c7e096640 1 -- 172.21.15.2:0/2234393826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c78104740 msgr2=0x7f0c78106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.563+0000 7f0c7e096640 1 --2- 172.21.15.2:0/2234393826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c78104740 0x7f0c78106b60 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009900 tx=0x7f0c6802a410 comp rx=0 tx=0).stop 2024-05-14T09:51:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.563+0000 7f0c7e096640 1 -- 172.21.15.2:0/2234393826 shutdown_connections 2024-05-14T09:51:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.563+0000 7f0c7e096640 1 --2- 172.21.15.2:0/2234393826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c78109a00 0x7f0c7810be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.563+0000 7f0c7e096640 1 --2- 172.21.15.2:0/2234393826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c781070a0 0x7f0c781094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.563+0000 7f0c7e096640 1 --2- 172.21.15.2:0/2234393826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c78104740 0x7f0c78106b60 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.563+0000 7f0c7e096640 1 -- 172.21.15.2:0/2234393826 >> 172.21.15.2:0/2234393826 conn(0x7f0c780fdd40 msgr2=0x7f0c781001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.563+0000 7f0c7e096640 1 -- 172.21.15.2:0/2234393826 shutdown_connections 2024-05-14T09:51:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.563+0000 7f0c7e096640 1 -- 172.21.15.2:0/2234393826 wait complete. 2024-05-14T09:51:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.564+0000 7f0c7e096640 1 Processor -- start 2024-05-14T09:51:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.564+0000 7f0c7e096640 1 -- start start 2024-05-14T09:51:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.564+0000 7f0c7e096640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c78104740 0x7f0c7819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.564+0000 7f0c7e096640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c781070a0 0x7f0c7819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.564+0000 7f0c7e096640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c78109a00 0x7f0c781a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.564+0000 7f0c7e096640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7810f750 con 0x7f0c78109a00 2024-05-14T09:51:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.564+0000 7f0c7e096640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7810f5d0 con 0x7f0c781070a0 2024-05-14T09:51:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.564+0000 7f0c7e096640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7810f8d0 con 0x7f0c78104740 2024-05-14T09:51:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.564+0000 7f0c777fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c78104740 0x7f0c7819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.564+0000 7f0c77fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c78109a00 0x7f0c781a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.564+0000 7f0c777fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c78104740 0x7f0c7819a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:53950/0 (socket says 172.21.15.2:53950) 2024-05-14T09:51:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.564+0000 7f0c777fe640 1 -- 172.21.15.2:0/1156435086 learned_addr learned my addr 172.21.15.2:0/1156435086 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.565+0000 7f0c6ffff640 1 --2- 172.21.15.2:0/1156435086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c781070a0 0x7f0c7819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.565+0000 7f0c777fe640 1 -- 172.21.15.2:0/1156435086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c781070a0 msgr2=0x7f0c7819a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.565+0000 7f0c777fe640 1 --2- 172.21.15.2:0/1156435086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c781070a0 0x7f0c7819a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.565+0000 7f0c777fe640 1 -- 172.21.15.2:0/1156435086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c78109a00 msgr2=0x7f0c781a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.565+0000 7f0c777fe640 1 --2- 172.21.15.2:0/1156435086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c78109a00 0x7f0c781a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.565+0000 7f0c777fe640 1 -- 172.21.15.2:0/1156435086 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c781a1e60 con 0x7f0c78104740 2024-05-14T09:51:14.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.565+0000 7f0c77fff640 1 --2- 172.21.15.2:0/1156435086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c78109a00 0x7f0c781a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:51:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.565+0000 7f0c777fe640 1 --2- 172.21.15.2:0/1156435086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c78104740 0x7f0c7819a320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0c68006010 tx=0x7f0c68004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.565+0000 7f0c757fa640 1 -- 172.21.15.2:0/1156435086 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6803f030 con 0x7f0c78104740 2024-05-14T09:51:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.565+0000 7f0c757fa640 1 -- 172.21.15.2:0/1156435086 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c68031910 con 0x7f0c78104740 2024-05-14T09:51:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.565+0000 7f0c7e096640 1 -- 172.21.15.2:0/1156435086 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c781a20f0 con 0x7f0c78104740 2024-05-14T09:51:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.565+0000 7f0c757fa640 1 -- 172.21.15.2:0/1156435086 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68035070 con 0x7f0c78104740 2024-05-14T09:51:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.566+0000 7f0c7e096640 1 -- 172.21.15.2:0/1156435086 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c781a2590 con 0x7f0c78104740 2024-05-14T09:51:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.568+0000 7f0c85029740 1 Processor -- start 2024-05-14T09:51:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.569+0000 7f0c85029740 1 -- start start 2024-05-14T09:51:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.569+0000 7f0c85029740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038606f1b0 0x560386410a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.569+0000 7f0c85029740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 0x560386410fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.569+0000 7f0c85029740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038604e4d0 0x560386417ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.569+0000 7f0c85029740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5603863755d0 con 0x5603860462f0 2024-05-14T09:51:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.569+0000 7f0c85029740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56038632e890 con 0x56038604e4d0 2024-05-14T09:51:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.569+0000 7f0c85029740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560386371590 con 0x56038606f1b0 2024-05-14T09:51:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.569+0000 7f0c757fa640 1 -- 172.21.15.2:0/1156435086 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0c68042030 con 0x7f0c78104740 2024-05-14T09:51:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.569+0000 7f0c77fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038606f1b0 0x560386410a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.569+0000 7f0c6ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038604e4d0 0x560386417ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.569+0000 7f0c77fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038606f1b0 0x560386410a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:53954/0 (socket says 172.21.15.2:53954) 2024-05-14T09:51:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.569+0000 7f0c77fff640 1 -- 172.21.15.2:0/4085290111 learned_addr learned my addr 172.21.15.2:0/4085290111 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.569+0000 7f0c6ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038604e4d0 0x560386417ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:59252/0 (socket says 172.21.15.2:59252) 2024-05-14T09:51:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.569+0000 7f0c777fe640 1 --2- 172.21.15.2:0/4085290111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 0x560386410fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.570+0000 7f0c77fff640 1 -- 172.21.15.2:0/4085290111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038604e4d0 msgr2=0x560386417ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.570+0000 7f0c77fff640 1 --2- 172.21.15.2:0/4085290111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038604e4d0 0x560386417ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.570+0000 7f0c77fff640 1 -- 172.21.15.2:0/4085290111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 msgr2=0x560386410fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.570+0000 7f0c77fff640 1 --2- 172.21.15.2:0/4085290111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 0x560386410fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.570+0000 7f0c77fff640 1 -- 172.21.15.2:0/4085290111 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560386087210 con 0x56038606f1b0 2024-05-14T09:51:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.570+0000 7f0c77fff640 1 --2- 172.21.15.2:0/4085290111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038606f1b0 0x560386410a60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0c700117a0 tx=0x7f0c70011c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3990d3e9c078491 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.570+0000 7f0c757fa640 1 --2- 172.21.15.2:0/1156435086 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c640b28a0 0x7f0c640b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.570+0000 7f0c6dffb640 1 -- 172.21.15.2:0/4085290111 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70011220 con 0x56038606f1b0 2024-05-14T09:51:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.570+0000 7f0c6dffb640 1 -- 172.21.15.2:0/4085290111 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c7000f9a0 con 0x56038606f1b0 2024-05-14T09:51:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.570+0000 7f0c6dffb640 1 -- 172.21.15.2:0/4085290111 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c700045d0 con 0x56038606f1b0 2024-05-14T09:51:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.570+0000 7f0c757fa640 1 -- 172.21.15.2:0/1156435086 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f0c681029a0 con 0x7f0c78104740 2024-05-14T09:51:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.571+0000 7f0c777fe640 1 --2- 172.21.15.2:0/1156435086 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c640b28a0 0x7f0c640b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.571+0000 7f0c85029740 1 -- 172.21.15.2:0/4085290111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038606f1b0 msgr2=0x560386410a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.571+0000 7f0c85029740 1 --2- 172.21.15.2:0/4085290111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038606f1b0 0x560386410a60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0c700117a0 tx=0x7f0c70011c60 comp rx=0 tx=0).stop 2024-05-14T09:51:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.571+0000 7f0c777fe640 1 --2- 172.21.15.2:0/1156435086 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c640b28a0 0x7f0c640b4db0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x560386411e40 tx=0x7f0c68105f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.572+0000 7f0c85029740 1 -- 172.21.15.2:0/4085290111 shutdown_connections 2024-05-14T09:51:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.572+0000 7f0c85029740 1 --2- 172.21.15.2:0/4085290111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038604e4d0 0x560386417ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.572+0000 7f0c85029740 1 --2- 172.21.15.2:0/4085290111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 0x560386410fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.572+0000 7f0c85029740 1 --2- 172.21.15.2:0/4085290111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038606f1b0 0x560386410a60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.572+0000 7f0c85029740 1 -- 172.21.15.2:0/4085290111 >> 172.21.15.2:0/4085290111 conn(0x560386063300 msgr2=0x56038640c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.572+0000 7f0c85029740 1 -- 172.21.15.2:0/4085290111 shutdown_connections 2024-05-14T09:51:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.572+0000 7f0c85029740 1 -- 172.21.15.2:0/4085290111 wait complete. 2024-05-14T09:51:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.572+0000 7f0c85029740 1 -- start start 2024-05-14T09:51:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.572+0000 7f0c85029740 1 Processor -- start 2024-05-14T09:51:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c85029740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 0x56038641a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c85029740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038604e4d0 0x56038641abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c85029740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038606f1b0 0x560386421b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c85029740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560386418930 con 0x5603860462f0 2024-05-14T09:51:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c85029740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560386332370 con 0x56038604e4d0 2024-05-14T09:51:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c85029740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x560386418ab0 con 0x56038606f1b0 2024-05-14T09:51:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c77fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 0x56038641a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c77fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 0x56038641a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56242/0 (socket says 172.21.15.2:56242) 2024-05-14T09:51:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c77fff640 1 -- 172.21.15.2:0/1860628023 learned_addr learned my addr 172.21.15.2:0/1860628023 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c6ffff640 1 --2- 172.21.15.2:0/1860628023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038604e4d0 0x56038641abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c77fff640 1 --2- 172.21.15.2:0/1860628023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038606f1b0 0x560386421b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c77fff640 1 -- 172.21.15.2:0/1860628023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038606f1b0 msgr2=0x560386421b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c77fff640 1 --2- 172.21.15.2:0/1860628023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038606f1b0 0x560386421b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c77fff640 1 -- 172.21.15.2:0/1860628023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038604e4d0 msgr2=0x56038641abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c77fff640 1 --2- 172.21.15.2:0/1860628023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038604e4d0 0x56038641abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c77fff640 1 -- 172.21.15.2:0/1860628023 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560386153620 con 0x5603860462f0 2024-05-14T09:51:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.573+0000 7f0c85029740 10 client.0 _ll_register_callbacks cb 0x560386086a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:51:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.574+0000 7f0c85029740 1 -- 172.21.15.2:0/1860628023 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c7000f9a0 con 0x5603860462f0 2024-05-14T09:51:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.574+0000 7f0c77fff640 1 --2- 172.21.15.2:0/1860628023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 0x56038641a6b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0c70003fb0 tx=0x7f0c70004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46bed3c5998896c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.574+0000 7f0c56ffd640 10 client.17043 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:51:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.575+0000 7f0c56ffd640 1 -- 172.21.15.2:0/1860628023 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7002e060 con 0x5603860462f0 2024-05-14T09:51:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.575+0000 7f0c56ffd640 1 -- 172.21.15.2:0/1860628023 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f0c70004340 con 0x5603860462f0 2024-05-14T09:51:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.575+0000 7f0c56ffd640 1 -- 172.21.15.2:0/1860628023 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f0c70004540 con 0x5603860462f0 2024-05-14T09:51:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.575+0000 7f0c85029740 10 client.17043 fetch_fsmap learned FSMap version 56 2024-05-14T09:51:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.575+0000 7f0c85029740 1 -- 172.21.15.2:0/1860628023 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({fsmap=56}) -- 0x560386424990 con 0x5603860462f0 2024-05-14T09:51:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.575+0000 7f0c56ffd640 1 -- 172.21.15.2:0/1860628023 <== mon.0 v2:172.21.15.2:3300/0 4 ==== fsmap(e 56) ==== 8358+0+0 (secure 0 0 0) 0x7f0c70018070 con 0x5603860462f0 2024-05-14T09:51:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.575+0000 7f0c85029740 10 client.17043 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T09:51:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.575+0000 7f0c85029740 10 client.17043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:51:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.575+0000 7f0c85029740 20 client.17043 populate_metadata read hostname 'smithi002' 2024-05-14T09:51:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.575+0000 7f0c85029740 1 --2- 172.21.15.2:0/1860628023 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56038602dd80 0x560386427da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.575+0000 7f0c85029740 4 client.17043 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:51:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.575+0000 7f0c85029740 1 -- 172.21.15.2:0/1860628023 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603863831e0 con 0x56038602dd80 2024-05-14T09:51:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.576+0000 7f0c6ffff640 1 --2- 172.21.15.2:0/1860628023 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56038602dd80 0x560386427da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.576+0000 7f0c6ffff640 1 --2- 172.21.15.2:0/1860628023 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56038602dd80 0x560386427da0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f501e3a8ca0f174e server_cookie=7fd74a1f1f2b56fc in_seq=0 out_seq=0 2024-05-14T09:51:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.576+0000 7f0c56ffd640 10 client.17043 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.579+0000 7f0c56ffd640 1 -- 172.21.15.2:0/1860628023 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5603863831e0 con 0x56038602dd80 2024-05-14T09:51:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.579+0000 7f0c56ffd640 10 client.17043 handle_command_reply: tid=0 2024-05-14T09:51:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.579+0000 7f0c85029740 1 client.17043 shutdown 2024-05-14T09:51:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.580+0000 7f0c85029740 1 -- 172.21.15.2:0/1860628023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 msgr2=0x56038641a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.580+0000 7f0c85029740 1 --2- 172.21.15.2:0/1860628023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 0x56038641a6b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0c70003fb0 tx=0x7f0c70004050 comp rx=0 tx=0).stop 2024-05-14T09:51:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.580+0000 7f0c85029740 1 -- 172.21.15.2:0/1860628023 shutdown_connections 2024-05-14T09:51:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.580+0000 7f0c85029740 1 --2- 172.21.15.2:0/1860628023 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56038602dd80 0x560386427da0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.580+0000 7f0c85029740 1 --2- 172.21.15.2:0/1860628023 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038606f1b0 0x560386421b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.580+0000 7f0c56ffd640 0 client.17043 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.580+0000 7f0c85029740 1 --2- 172.21.15.2:0/1860628023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038604e4d0 0x56038641abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.580+0000 7f0c85029740 1 --2- 172.21.15.2:0/1860628023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 0x56038641a6b0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.580+0000 7f0c85029740 1 -- 172.21.15.2:0/1860628023 >> 172.21.15.2:0/1860628023 conn(0x560386063300 msgr2=0x56038640b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.580+0000 7f0c85029740 1 -- 172.21.15.2:0/1860628023 shutdown_connections 2024-05-14T09:51:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.580+0000 7f0c85029740 1 -- 172.21.15.2:0/1860628023 wait complete. 2024-05-14T09:51:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.580+0000 7f0c85029740 20 client.17043 trim_cache size 0 max 16384 2024-05-14T09:51:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.600+0000 7f0c85029740 1 Processor -- start 2024-05-14T09:51:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.600+0000 7f0c85029740 1 -- start start 2024-05-14T09:51:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c85029740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603860462f0 0x56038641a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c85029740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038604e4d0 0x560386421ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c85029740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56038606f1b0 0x560386422020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c85029740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781af5f0 con 0x56038606f1b0 2024-05-14T09:51:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c85029740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781ace60 con 0x5603860462f0 2024-05-14T09:51:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c85029740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781aacb0 con 0x56038604e4d0 2024-05-14T09:51:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c6ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038604e4d0 0x560386421ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c77fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56038606f1b0 0x560386422020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c6ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038604e4d0 0x560386421ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:53966/0 (socket says 172.21.15.2:53966) 2024-05-14T09:51:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c6ffff640 1 -- 172.21.15.2:0/1057343967 learned_addr learned my addr 172.21.15.2:0/1057343967 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c77fff640 1 --2- 172.21.15.2:0/1057343967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603860462f0 0x56038641a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c77fff640 1 -- 172.21.15.2:0/1057343967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038604e4d0 msgr2=0x560386421ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c77fff640 1 --2- 172.21.15.2:0/1057343967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038604e4d0 0x560386421ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c77fff640 1 -- 172.21.15.2:0/1057343967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603860462f0 msgr2=0x56038641a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c77fff640 1 --2- 172.21.15.2:0/1057343967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603860462f0 0x56038641a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c77fff640 1 -- 172.21.15.2:0/1057343967 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56038641b720 con 0x56038606f1b0 2024-05-14T09:51:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c6ffff640 1 --2- 172.21.15.2:0/1057343967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038604e4d0 0x560386421ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:51:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.601+0000 7f0c77fff640 1 --2- 172.21.15.2:0/1057343967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56038606f1b0 0x560386422020 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0c70003fb0 tx=0x7f0c7002a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48402c63b9450ff8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.602+0000 7f0c6d7fa640 1 -- 172.21.15.2:0/1057343967 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70003fe0 con 0x56038606f1b0 2024-05-14T09:51:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.602+0000 7f0c6d7fa640 1 -- 172.21.15.2:0/1057343967 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c7000f9a0 con 0x56038606f1b0 2024-05-14T09:51:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.602+0000 7f0c6d7fa640 1 -- 172.21.15.2:0/1057343967 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70024a50 con 0x56038606f1b0 2024-05-14T09:51:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.603+0000 7f0c85029740 1 -- 172.21.15.2:0/1057343967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56038606f1b0 msgr2=0x560386422020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.603+0000 7f0c85029740 1 --2- 172.21.15.2:0/1057343967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56038606f1b0 0x560386422020 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0c70003fb0 tx=0x7f0c7002a040 comp rx=0 tx=0).stop 2024-05-14T09:51:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.603+0000 7f0c85029740 1 -- 172.21.15.2:0/1057343967 shutdown_connections 2024-05-14T09:51:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.603+0000 7f0c85029740 1 --2- 172.21.15.2:0/1057343967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56038606f1b0 0x560386422020 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.603+0000 7f0c85029740 1 --2- 172.21.15.2:0/1057343967 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038604e4d0 0x560386421ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.603+0000 7f0c85029740 1 --2- 172.21.15.2:0/1057343967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5603860462f0 0x56038641a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.603+0000 7f0c85029740 1 -- 172.21.15.2:0/1057343967 >> 172.21.15.2:0/1057343967 conn(0x560386063300 msgr2=0x56038641a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.603+0000 7f0c85029740 1 -- 172.21.15.2:0/1057343967 shutdown_connections 2024-05-14T09:51:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.603+0000 7f0c85029740 1 -- 172.21.15.2:0/1057343967 wait complete. 2024-05-14T09:51:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.603+0000 7f0c85029740 1 -- start start 2024-05-14T09:51:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.603+0000 7f0c85029740 1 Processor -- start 2024-05-14T09:51:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c85029740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 0x56038641d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c85029740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038604e4d0 0x56038641d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c85029740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038606f1b0 0x5603864208d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c85029740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781ab320 con 0x5603860462f0 2024-05-14T09:51:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c85029740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781ac790 con 0x56038606f1b0 2024-05-14T09:51:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c85029740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781ab9f0 con 0x56038604e4d0 2024-05-14T09:51:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c85029740 10 client.0 _ll_register_callbacks cb 0x7f0c68108420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:51:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c77fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 0x56038641d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c6ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038604e4d0 0x56038641d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c6ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038604e4d0 0x56038641d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:53976/0 (socket says 172.21.15.2:53976) 2024-05-14T09:51:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c77fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 0x56038641d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56258/0 (socket says 172.21.15.2:56258) 2024-05-14T09:51:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c6ffff640 1 -- 172.21.15.2:0/3357898118 learned_addr learned my addr 172.21.15.2:0/3357898118 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c77fff640 1 --2- 172.21.15.2:0/3357898118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038606f1b0 0x5603864208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c6ffff640 1 -- 172.21.15.2:0/3357898118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038606f1b0 msgr2=0x5603864208d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c6ffff640 1 --2- 172.21.15.2:0/3357898118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038606f1b0 0x5603864208d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c6ffff640 1 -- 172.21.15.2:0/3357898118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 msgr2=0x56038641d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c6ffff640 1 --2- 172.21.15.2:0/3357898118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 0x56038641d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.604+0000 7f0c6ffff640 1 -- 172.21.15.2:0/3357898118 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560386420e90 con 0x56038604e4d0 2024-05-14T09:51:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.605+0000 7f0c6ffff640 1 --2- 172.21.15.2:0/3357898118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038604e4d0 0x56038641d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0c60010600 tx=0x7f0c60010ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dc3c52a687d769f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.605+0000 7f0c56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:51:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.605+0000 7f0c56ffd640 1 -- 172.21.15.2:0/3357898118 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60026060 con 0x56038604e4d0 2024-05-14T09:51:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.605+0000 7f0c56ffd640 1 -- 172.21.15.2:0/3357898118 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f0c600035f0 con 0x56038604e4d0 2024-05-14T09:51:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.605+0000 7f0c85029740 1 -- 172.21.15.2:0/3357898118 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c7000f9a0 con 0x56038604e4d0 2024-05-14T09:51:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.605+0000 7f0c56ffd640 1 -- 172.21.15.2:0/3357898118 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f0c600037f0 con 0x56038604e4d0 2024-05-14T09:51:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.605+0000 7f0c85029740 10 client.26683 fetch_fsmap learned FSMap version 56 2024-05-14T09:51:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.605+0000 7f0c85029740 1 -- 172.21.15.2:0/3357898118 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5603864221f0 con 0x56038604e4d0 2024-05-14T09:51:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.605+0000 7f0c56ffd640 1 -- 172.21.15.2:0/3357898118 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c7000f9a0 con 0x56038604e4d0 2024-05-14T09:51:14.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 86566 26.93 MB/sec execute 40 sec latency 191.303 ms 2024-05-14T09:51:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.606+0000 7f0c56ffd640 1 -- 172.21.15.2:0/3357898118 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 56) ==== 8358+0+0 (secure 0 0 0) 0x7f0c6000fb40 con 0x56038604e4d0 2024-05-14T09:51:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.606+0000 7f0c85029740 10 client.26683 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T09:51:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.606+0000 7f0c85029740 10 client.26683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:51:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.606+0000 7f0c85029740 20 client.26683 populate_metadata read hostname 'smithi002' 2024-05-14T09:51:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.606+0000 7f0c85029740 1 --2- 172.21.15.2:0/3357898118 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5603860031c0 0x560386492500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.606+0000 7f0c85029740 4 client.26683 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:51:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.606+0000 7f0c85029740 1 -- 172.21.15.2:0/3357898118 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56038637d050 con 0x5603860031c0 2024-05-14T09:51:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.606+0000 7f0c77fff640 1 --2- 172.21.15.2:0/3357898118 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5603860031c0 0x560386492500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.607+0000 7f0c56ffd640 10 client.26683 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:14.607+0000 7f0c77fff640 1 --2- 172.21.15.2:0/3357898118 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5603860031c0 0x560386492500 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df5f235fc09dcf5d server_cookie=223398d94a21e01b in_seq=0 out_seq=0 2024-05-14T09:51:14.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:14 smithi002 ceph-mon[31557]: pgmap v913: 129 pgs: 129 active+clean; 574 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 14 MiB/s wr, 829 op/s 2024-05-14T09:51:14.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:14 smithi029 ceph-mon[36249]: pgmap v913: 129 pgs: 129 active+clean; 574 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 14 MiB/s wr, 829 op/s 2024-05-14T09:51:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:14 smithi026 ceph-mon[36056]: pgmap v913: 129 pgs: 129 active+clean; 574 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 14 MiB/s wr, 829 op/s 2024-05-14T09:51:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.037+0000 7f0c56ffd640 1 -- 172.21.15.2:0/3357898118 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56038637d050 con 0x5603860031c0 2024-05-14T09:51:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.037+0000 7f0c56ffd640 10 client.26683 handle_command_reply: tid=0 2024-05-14T09:51:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.037+0000 7f0c85029740 1 client.26683 shutdown 2024-05-14T09:51:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.037+0000 7f0c85029740 1 -- 172.21.15.2:0/3357898118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038604e4d0 msgr2=0x56038641d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.037+0000 7f0c85029740 1 --2- 172.21.15.2:0/3357898118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038604e4d0 0x56038641d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0c60010600 tx=0x7f0c60010ac0 comp rx=0 tx=0).stop 2024-05-14T09:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.037+0000 7f0c85029740 1 -- 172.21.15.2:0/3357898118 shutdown_connections 2024-05-14T09:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.037+0000 7f0c85029740 1 --2- 172.21.15.2:0/3357898118 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5603860031c0 0x560386492500 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.037+0000 7f0c85029740 1 --2- 172.21.15.2:0/3357898118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56038606f1b0 0x5603864208d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.037+0000 7f0c85029740 1 --2- 172.21.15.2:0/3357898118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56038604e4d0 0x56038641d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.037+0000 7f0c56ffd640 0 client.26683 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.037+0000 7f0c85029740 1 --2- 172.21.15.2:0/3357898118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603860462f0 0x56038641d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.037+0000 7f0c85029740 1 -- 172.21.15.2:0/3357898118 >> 172.21.15.2:0/3357898118 conn(0x560386063300 msgr2=0x56038640a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.037+0000 7f0c85029740 1 -- 172.21.15.2:0/3357898118 shutdown_connections 2024-05-14T09:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.037+0000 7f0c85029740 1 -- 172.21.15.2:0/3357898118 wait complete. 2024-05-14T09:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.037+0000 7f0c85029740 20 client.26683 trim_cache size 0 max 16384 2024-05-14T09:51:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.038+0000 7f0c7e096640 1 -- 172.21.15.2:0/1156435086 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c640b28a0 msgr2=0x7f0c640b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.038+0000 7f0c7e096640 1 --2- 172.21.15.2:0/1156435086 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c640b28a0 0x7f0c640b4db0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x560386411e40 tx=0x7f0c68105f90 comp rx=0 tx=0).stop 2024-05-14T09:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.038+0000 7f0c7e096640 1 -- 172.21.15.2:0/1156435086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c78104740 msgr2=0x7f0c7819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.038+0000 7f0c7e096640 1 --2- 172.21.15.2:0/1156435086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c78104740 0x7f0c7819a320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0c68006010 tx=0x7f0c68004270 comp rx=0 tx=0).stop 2024-05-14T09:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.038+0000 7f0c7e096640 1 -- 172.21.15.2:0/1156435086 shutdown_connections 2024-05-14T09:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.038+0000 7f0c7e096640 1 --2- 172.21.15.2:0/1156435086 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f0c640b28a0 0x7f0c640b4db0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.038+0000 7f0c7e096640 1 --2- 172.21.15.2:0/1156435086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c78109a00 0x7f0c781a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.038+0000 7f0c7e096640 1 --2- 172.21.15.2:0/1156435086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c781070a0 0x7f0c7819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.038+0000 7f0c7e096640 1 --2- 172.21.15.2:0/1156435086 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0c78104740 0x7f0c7819a320 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.038+0000 7f0c7e096640 1 -- 172.21.15.2:0/1156435086 >> 172.21.15.2:0/1156435086 conn(0x7f0c780fdd40 msgr2=0x7f0c78105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.038+0000 7f0c7e096640 1 -- 172.21.15.2:0/1156435086 shutdown_connections 2024-05-14T09:51:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:15.038+0000 7f0c7e096640 1 -- 172.21.15.2:0/1156435086 wait complete. 2024-05-14T09:51:15.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 87158 27.19 MB/sec execute 41 sec latency 212.824 ms 2024-05-14T09:51:16.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 87748 27.24 MB/sec execute 42 sec latency 107.537 ms 2024-05-14T09:51:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:16 smithi002 ceph-mon[31557]: pgmap v914: 129 pgs: 129 active+clean; 542 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 686 op/s 2024-05-14T09:51:16.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:16 smithi029 ceph-mon[36249]: pgmap v914: 129 pgs: 129 active+clean; 542 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 686 op/s 2024-05-14T09:51:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:16 smithi026 ceph-mon[36056]: pgmap v914: 129 pgs: 129 active+clean; 542 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 686 op/s 2024-05-14T09:51:17.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 88600 27.65 MB/sec execute 43 sec latency 65.938 ms 2024-05-14T09:51:18.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 89342 27.91 MB/sec execute 44 sec latency 50.792 ms 2024-05-14T09:51:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:18 smithi002 ceph-mon[31557]: pgmap v915: 129 pgs: 129 active+clean; 531 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 778 op/s 2024-05-14T09:51:18.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:18 smithi029 ceph-mon[36249]: pgmap v915: 129 pgs: 129 active+clean; 531 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 778 op/s 2024-05-14T09:51:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:18 smithi026 ceph-mon[36056]: pgmap v915: 129 pgs: 129 active+clean; 531 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 778 op/s 2024-05-14T09:51:19.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 90025 28.14 MB/sec execute 45 sec latency 64.882 ms 2024-05-14T09:51:20.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 90633 28.38 MB/sec execute 46 sec latency 80.823 ms 2024-05-14T09:51:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:20 smithi002 ceph-mon[31557]: pgmap v916: 129 pgs: 129 active+clean; 537 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 15 MiB/s wr, 969 op/s 2024-05-14T09:51:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:20 smithi029 ceph-mon[36249]: pgmap v916: 129 pgs: 129 active+clean; 537 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 15 MiB/s wr, 969 op/s 2024-05-14T09:51:20.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:20 smithi026 ceph-mon[36056]: pgmap v916: 129 pgs: 129 active+clean; 537 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 15 MiB/s wr, 969 op/s 2024-05-14T09:51:21.605 INFO:tasks.workunit.client.0.smithi002.stdout: 10 91238 28.33 MB/sec execute 47 sec latency 272.729 ms 2024-05-14T09:51:22.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 91975 28.51 MB/sec execute 48 sec latency 68.358 ms 2024-05-14T09:51:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:22 smithi002 ceph-mon[31557]: pgmap v917: 129 pgs: 129 active+clean; 538 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 771 op/s 2024-05-14T09:51:22.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:22 smithi029 ceph-mon[36249]: pgmap v917: 129 pgs: 129 active+clean; 538 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 771 op/s 2024-05-14T09:51:22.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:22 smithi026 ceph-mon[36056]: pgmap v917: 129 pgs: 129 active+clean; 538 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 771 op/s 2024-05-14T09:51:23.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 92717 28.86 MB/sec execute 49 sec latency 416.107 ms 2024-05-14T09:51:24.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 93287 28.85 MB/sec execute 50 sec latency 161.518 ms 2024-05-14T09:51:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:24 smithi002 ceph-mon[31557]: pgmap v918: 129 pgs: 129 active+clean; 548 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 889 op/s 2024-05-14T09:51:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:24 smithi029 ceph-mon[36249]: pgmap v918: 129 pgs: 129 active+clean; 548 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 889 op/s 2024-05-14T09:51:24.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:24 smithi026 ceph-mon[36056]: pgmap v918: 129 pgs: 129 active+clean; 548 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 889 op/s 2024-05-14T09:51:25.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 94022 28.96 MB/sec execute 51 sec latency 82.304 ms 2024-05-14T09:51:26.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 94768 29.25 MB/sec execute 52 sec latency 102.028 ms 2024-05-14T09:51:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:26 smithi002 ceph-mon[31557]: pgmap v919: 129 pgs: 129 active+clean; 552 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 5.1 MiB/s wr, 643 op/s 2024-05-14T09:51:26.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:26 smithi029 ceph-mon[36249]: pgmap v919: 129 pgs: 129 active+clean; 552 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 5.1 MiB/s wr, 643 op/s 2024-05-14T09:51:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:26 smithi026 ceph-mon[36056]: pgmap v919: 129 pgs: 129 active+clean; 552 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 5.1 MiB/s wr, 643 op/s 2024-05-14T09:51:27.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 95403 29.41 MB/sec execute 53 sec latency 74.627 ms 2024-05-14T09:51:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.157+0000 7f83b3451640 1 -- 172.21.15.2:0/3214876647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac108a90 msgr2=0x7f83ac10aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.157+0000 7f83b3451640 1 --2- 172.21.15.2:0/3214876647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac108a90 0x7f83ac10aeb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009ca0 tx=0x7f83a402a8c0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.157+0000 7f83b3451640 1 -- 172.21.15.2:0/3214876647 shutdown_connections 2024-05-14T09:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.157+0000 7f83b3451640 1 --2- 172.21.15.2:0/3214876647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac108a90 0x7f83ac10aeb0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.157+0000 7f83b3451640 1 --2- 172.21.15.2:0/3214876647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac103d90 0x7f83ac1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.157+0000 7f83b3451640 1 --2- 172.21.15.2:0/3214876647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac102b50 0x7f83ac103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.157+0000 7f83b3451640 1 -- 172.21.15.2:0/3214876647 >> 172.21.15.2:0/3214876647 conn(0x7f83ac0fdd40 msgr2=0x7f83ac1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.157+0000 7f83b3451640 1 -- 172.21.15.2:0/3214876647 shutdown_connections 2024-05-14T09:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.157+0000 7f83b3451640 1 -- 172.21.15.2:0/3214876647 wait complete. 2024-05-14T09:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.158+0000 7f83b3451640 1 Processor -- start 2024-05-14T09:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.158+0000 7f83b3451640 1 -- start start 2024-05-14T09:51:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.158+0000 7f83b3451640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac102b50 0x7f83ac06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.158+0000 7f83b3451640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac103d90 0x7f83ac19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.158+0000 7f83b3451640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac108a90 0x7f83ac1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.158+0000 7f83b3451640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac10e680 con 0x7f83ac103d90 2024-05-14T09:51:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.158+0000 7f83b3451640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac10e500 con 0x7f83ac108a90 2024-05-14T09:51:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.158+0000 7f83b3451640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac10e800 con 0x7f83ac102b50 2024-05-14T09:51:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.158+0000 7f83b09ec640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac103d90 0x7f83ac19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.158+0000 7f83b09ec640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac103d90 0x7f83ac19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55360/0 (socket says 172.21.15.2:55360) 2024-05-14T09:51:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.158+0000 7f83b11ed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac102b50 0x7f83ac06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.158+0000 7f83b09ec640 1 -- 172.21.15.2:0/1302361295 learned_addr learned my addr 172.21.15.2:0/1302361295 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.159+0000 7f83b19ee640 1 --2- 172.21.15.2:0/1302361295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac108a90 0x7f83ac1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.159+0000 7f83b09ec640 1 -- 172.21.15.2:0/1302361295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac102b50 msgr2=0x7f83ac06b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.159+0000 7f83b09ec640 1 --2- 172.21.15.2:0/1302361295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac102b50 0x7f83ac06b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.159+0000 7f83b09ec640 1 -- 172.21.15.2:0/1302361295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac108a90 msgr2=0x7f83ac1a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.159+0000 7f83b09ec640 1 --2- 172.21.15.2:0/1302361295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac108a90 0x7f83ac1a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.159+0000 7f83b09ec640 1 -- 172.21.15.2:0/1302361295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ac1a1d80 con 0x7f83ac103d90 2024-05-14T09:51:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.159+0000 7f83b09ec640 1 --2- 172.21.15.2:0/1302361295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac103d90 0x7f83ac19f010 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f839c00e9f0 tx=0x7f839c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.159+0000 7f83a27fc640 1 -- 172.21.15.2:0/1302361295 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c00cd80 con 0x7f83ac103d90 2024-05-14T09:51:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.159+0000 7f83a27fc640 1 -- 172.21.15.2:0/1302361295 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f839c0054d0 con 0x7f83ac103d90 2024-05-14T09:51:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.159+0000 7f83b3451640 1 -- 172.21.15.2:0/1302361295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ac1a2070 con 0x7f83ac103d90 2024-05-14T09:51:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.159+0000 7f83a27fc640 1 -- 172.21.15.2:0/1302361295 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c010690 con 0x7f83ac103d90 2024-05-14T09:51:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.159+0000 7f83b3451640 1 -- 172.21.15.2:0/1302361295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ac1a2570 con 0x7f83ac103d90 2024-05-14T09:51:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.162+0000 7f83a27fc640 1 -- 172.21.15.2:0/1302361295 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f839c004e90 con 0x7f83ac103d90 2024-05-14T09:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.163+0000 7f83ba3e6740 1 Processor -- start 2024-05-14T09:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.163+0000 7f83ba3e6740 1 -- start start 2024-05-14T09:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.163+0000 7f83ba3e6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c7021b0 0x55587caa7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.163+0000 7f83ba3e6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587caa83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.163+0000 7f83ba3e6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6e14d0 0x55587caaf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.163+0000 7f83ba3e6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55587ca085d0 con 0x55587c7021b0 2024-05-14T09:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.163+0000 7f83ba3e6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55587c9c1890 con 0x55587c6d92f0 2024-05-14T09:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.163+0000 7f83ba3e6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55587ca04590 con 0x55587c6e14d0 2024-05-14T09:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.163+0000 7f83b19ee640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c7021b0 0x55587caa7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.163+0000 7f83b09ec640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6e14d0 0x55587caaf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.163+0000 7f83b19ee640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c7021b0 0x55587caa7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55374/0 (socket says 172.21.15.2:55374) 2024-05-14T09:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.163+0000 7f83b19ee640 1 -- 172.21.15.2:0/3490107344 learned_addr learned my addr 172.21.15.2:0/3490107344 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.163+0000 7f83b11ed640 1 --2- 172.21.15.2:0/3490107344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587caa83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.163+0000 7f83a27fc640 1 --2- 172.21.15.2:0/1302361295 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83980b28a0 0x7f83980b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.164+0000 7f83b11ed640 1 -- 172.21.15.2:0/3490107344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6e14d0 msgr2=0x55587caaf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.164+0000 7f83b11ed640 1 --2- 172.21.15.2:0/3490107344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6e14d0 0x55587caaf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.164+0000 7f83b11ed640 1 -- 172.21.15.2:0/3490107344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c7021b0 msgr2=0x55587caa7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.164+0000 7f83b11ed640 1 --2- 172.21.15.2:0/3490107344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c7021b0 0x55587caa7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.164+0000 7f83b11ed640 1 -- 172.21.15.2:0/3490107344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55587c71a210 con 0x55587c6d92f0 2024-05-14T09:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.164+0000 7f83b19ee640 1 --2- 172.21.15.2:0/3490107344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c7021b0 0x55587caa7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.164+0000 7f83b19ee640 1 --2- 172.21.15.2:0/1302361295 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83980b28a0 0x7f83980b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.164+0000 7f83a27fc640 1 -- 172.21.15.2:0/1302361295 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f839c0e2a50 con 0x7f83ac103d90 2024-05-14T09:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.164+0000 7f83b11ed640 1 --2- 172.21.15.2:0/3490107344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587caa83f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f83ac102420 tx=0x7f83a8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa8b3910aa76050c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.164+0000 7f83977fe640 1 -- 172.21.15.2:0/3490107344 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8004050 con 0x55587c6d92f0 2024-05-14T09:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.164+0000 7f83977fe640 1 -- 172.21.15.2:0/3490107344 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a80054d0 con 0x55587c6d92f0 2024-05-14T09:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.164+0000 7f83977fe640 1 -- 172.21.15.2:0/3490107344 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8004bc0 con 0x55587c6d92f0 2024-05-14T09:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.165+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3490107344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 msgr2=0x55587caa83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.165+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/3490107344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587caa83f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f83ac102420 tx=0x7f83a8011810 comp rx=0 tx=0).stop 2024-05-14T09:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.165+0000 7f83b19ee640 1 --2- 172.21.15.2:0/1302361295 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83980b28a0 0x7f83980b4db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009a60 tx=0x7f83a40095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.165+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3490107344 shutdown_connections 2024-05-14T09:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.165+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/3490107344 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6e14d0 0x55587caaf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.165+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/3490107344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587caa83f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.165+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/3490107344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c7021b0 0x55587caa7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.165+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3490107344 >> 172.21.15.2:0/3490107344 conn(0x55587c6f6300 msgr2=0x55587ca9f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.165+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3490107344 shutdown_connections 2024-05-14T09:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.165+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3490107344 wait complete. 2024-05-14T09:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.166+0000 7f83ba3e6740 1 -- start start 2024-05-14T09:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.166+0000 7f83ba3e6740 1 Processor -- start 2024-05-14T09:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.167+0000 7f83ba3e6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.167+0000 7f83ba3e6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6e14d0 0x55587cab1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.167+0000 7f83ba3e6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c7021b0 0x55587cab8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.167+0000 7f83b11ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.167+0000 7f83b11ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:36982/0 (socket says 172.21.15.2:36982) 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.167+0000 7f83b11ed640 1 -- 172.21.15.2:0/3392803317 learned_addr learned my addr 172.21.15.2:0/3392803317 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.167+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3392803317 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55587caafd40 con 0x55587c6e14d0 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.167+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3392803317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55587c9c5370 con 0x55587c6d92f0 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.167+0000 7f83b09ec640 1 --2- 172.21.15.2:0/3392803317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6e14d0 0x55587cab1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.167+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3392803317 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55587caafec0 con 0x55587c7021b0 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.167+0000 7f83b19ee640 1 --2- 172.21.15.2:0/3392803317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c7021b0 0x55587cab8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.167+0000 7f83ba3e6740 10 client.0 _ll_register_callbacks cb 0x55587c719a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.168+0000 7f83b11ed640 1 -- 172.21.15.2:0/3392803317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c7021b0 msgr2=0x55587cab8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.168+0000 7f83b11ed640 1 --2- 172.21.15.2:0/3392803317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c7021b0 0x55587cab8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.168+0000 7f83b11ed640 1 -- 172.21.15.2:0/3392803317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6e14d0 msgr2=0x55587cab1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.168+0000 7f83b11ed640 1 --2- 172.21.15.2:0/3392803317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6e14d0 0x55587cab1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.168+0000 7f83b11ed640 1 -- 172.21.15.2:0/3392803317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55587c7e6620 con 0x55587c6d92f0 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.168+0000 7f83b09ec640 1 --2- 172.21.15.2:0/3392803317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6e14d0 0x55587cab1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.168+0000 7f83b11ed640 1 --2- 172.21.15.2:0/3392803317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab1a80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f83a801b040 tx=0x7f83a8009fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cec4a1cde3a69421 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.168+0000 7f8394ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.168+0000 7f8394ff9640 1 -- 172.21.15.2:0/3392803317 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8029070 con 0x55587c6d92f0 2024-05-14T09:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.168+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3392803317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83a80054d0 con 0x55587c6d92f0 2024-05-14T09:51:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.168+0000 7f8394ff9640 1 -- 172.21.15.2:0/3392803317 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f83a8003a00 con 0x55587c6d92f0 2024-05-14T09:51:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.168+0000 7f8394ff9640 1 -- 172.21.15.2:0/3392803317 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f83a8033050 con 0x55587c6d92f0 2024-05-14T09:51:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.169+0000 7f83ba3e6740 10 client.26693 fetch_fsmap learned FSMap version 56 2024-05-14T09:51:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.169+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3392803317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55587cabbe60 con 0x55587c6d92f0 2024-05-14T09:51:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.169+0000 7f8394ff9640 1 -- 172.21.15.2:0/3392803317 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a8003c60 con 0x55587c6d92f0 2024-05-14T09:51:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.170+0000 7f8394ff9640 1 -- 172.21.15.2:0/3392803317 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 56) ==== 8358+0+0 (secure 0 0 0) 0x7f83a8023430 con 0x55587c6d92f0 2024-05-14T09:51:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.170+0000 7f83ba3e6740 10 client.26693 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T09:51:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.170+0000 7f83ba3e6740 10 client.26693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:51:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.170+0000 7f83ba3e6740 20 client.26693 populate_metadata read hostname 'smithi002' 2024-05-14T09:51:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.170+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/3392803317 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55587c6c0d80 0x55587cabf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.170+0000 7f83ba3e6740 4 client.26693 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:51:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.170+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3392803317 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f83a80054d0 con 0x55587c6c0d80 2024-05-14T09:51:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.170+0000 7f83b09ec640 1 --2- 172.21.15.2:0/3392803317 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55587c6c0d80 0x55587cabf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.170+0000 7f8394ff9640 10 client.26693 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.170+0000 7f83b09ec640 1 --2- 172.21.15.2:0/3392803317 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55587c6c0d80 0x55587cabf3a0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdc38ddfec237297 server_cookie=461ccf5f0272dcfa in_seq=0 out_seq=0 2024-05-14T09:51:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.175+0000 7f8394ff9640 1 -- 172.21.15.2:0/3392803317 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f83a80054d0 con 0x55587c6c0d80 2024-05-14T09:51:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.175+0000 7f8394ff9640 10 client.26693 handle_command_reply: tid=0 2024-05-14T09:51:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.175+0000 7f83ba3e6740 1 client.26693 shutdown 2024-05-14T09:51:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.175+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3392803317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 msgr2=0x55587cab1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.175+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/3392803317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab1a80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f83a801b040 tx=0x7f83a8009fa0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.175+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3392803317 shutdown_connections 2024-05-14T09:51:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.175+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/3392803317 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55587c6c0d80 0x55587cabf3a0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.175+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/3392803317 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c7021b0 0x55587cab8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.175+0000 7f8394ff9640 0 client.26693 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.175+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/3392803317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6e14d0 0x55587cab1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.175+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/3392803317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab1a80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.175+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3392803317 >> 172.21.15.2:0/3392803317 conn(0x55587c6f6300 msgr2=0x55587caa2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.175+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3392803317 shutdown_connections 2024-05-14T09:51:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.175+0000 7f83ba3e6740 1 -- 172.21.15.2:0/3392803317 wait complete. 2024-05-14T09:51:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.175+0000 7f83ba3e6740 20 client.26693 trim_cache size 0 max 16384 2024-05-14T09:51:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.195+0000 7f83ba3e6740 1 Processor -- start 2024-05-14T09:51:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.195+0000 7f83ba3e6740 1 -- start start 2024-05-14T09:51:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.196+0000 7f83ba3e6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6c0d80 0x55587cab1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.196+0000 7f83ba3e6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.196+0000 7f83ba3e6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6e14d0 0x55587cab9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.196+0000 7f83ba3e6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1ad3b0 con 0x55587c6c0d80 2024-05-14T09:51:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.196+0000 7f83ba3e6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1aac20 con 0x55587c6d92f0 2024-05-14T09:51:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.196+0000 7f83ba3e6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1a8a70 con 0x55587c6e14d0 2024-05-14T09:51:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.197+0000 7f83b11ed640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6c0d80 0x55587cab1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.197+0000 7f83b11ed640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6c0d80 0x55587cab1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55392/0 (socket says 172.21.15.2:55392) 2024-05-14T09:51:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.197+0000 7f83b19ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6e14d0 0x55587cab9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.197+0000 7f83b11ed640 1 -- 172.21.15.2:0/374744298 learned_addr learned my addr 172.21.15.2:0/374744298 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.197+0000 7f83b09ec640 1 --2- 172.21.15.2:0/374744298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.197+0000 7f83b11ed640 1 -- 172.21.15.2:0/374744298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6e14d0 msgr2=0x55587cab9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.197+0000 7f83b11ed640 1 --2- 172.21.15.2:0/374744298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6e14d0 0x55587cab9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.197+0000 7f83b11ed640 1 -- 172.21.15.2:0/374744298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 msgr2=0x55587cab8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.197+0000 7f83b11ed640 1 --2- 172.21.15.2:0/374744298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.197+0000 7f83b11ed640 1 -- 172.21.15.2:0/374744298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55587cab98c0 con 0x55587c6c0d80 2024-05-14T09:51:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.197+0000 7f83b19ee640 1 --2- 172.21.15.2:0/374744298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6e14d0 0x55587cab9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:51:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.197+0000 7f83b09ec640 1 --2- 172.21.15.2:0/374744298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:51:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.198+0000 7f83b11ed640 1 --2- 172.21.15.2:0/374744298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6c0d80 0x55587cab1d60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f83a80239c0 tx=0x7f83a8023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a430dfba78c898 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.198+0000 7f8396ffd640 1 -- 172.21.15.2:0/374744298 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a803c070 con 0x55587c6c0d80 2024-05-14T09:51:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.198+0000 7f8396ffd640 1 -- 172.21.15.2:0/374744298 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a8023a40 con 0x55587c6c0d80 2024-05-14T09:51:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.198+0000 7f8396ffd640 1 -- 172.21.15.2:0/374744298 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8029070 con 0x55587c6c0d80 2024-05-14T09:51:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.199+0000 7f83ba3e6740 1 -- 172.21.15.2:0/374744298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6c0d80 msgr2=0x55587cab1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.199+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/374744298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6c0d80 0x55587cab1d60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f83a80239c0 tx=0x7f83a8023000 comp rx=0 tx=0).stop 2024-05-14T09:51:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.199+0000 7f83ba3e6740 1 -- 172.21.15.2:0/374744298 shutdown_connections 2024-05-14T09:51:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.199+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/374744298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6e14d0 0x55587cab9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.199+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/374744298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.199+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/374744298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6c0d80 0x55587cab1d60 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.200+0000 7f83ba3e6740 1 -- 172.21.15.2:0/374744298 >> 172.21.15.2:0/374744298 conn(0x55587c6f6300 msgr2=0x55587cab1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.200+0000 7f83ba3e6740 1 -- 172.21.15.2:0/374744298 shutdown_connections 2024-05-14T09:51:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.200+0000 7f83ba3e6740 1 -- 172.21.15.2:0/374744298 wait complete. 2024-05-14T09:51:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.200+0000 7f83ba3e6740 1 -- start start 2024-05-14T09:51:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.200+0000 7f83ba3e6740 1 Processor -- start 2024-05-14T09:51:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.201+0000 7f83ba3e6740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6c0d80 0x55587cab6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.201+0000 7f83ba3e6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.201+0000 7f83ba3e6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6e14d0 0x55587cab7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.201+0000 7f83ba3e6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1a90e0 con 0x55587c6e14d0 2024-05-14T09:51:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.201+0000 7f83ba3e6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1aa550 con 0x55587c6d92f0 2024-05-14T09:51:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.201+0000 7f83ba3e6740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1a97b0 con 0x55587c6c0d80 2024-05-14T09:51:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.201+0000 7f83b19ee640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6e14d0 0x55587cab7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.201+0000 7f83b11ed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6c0d80 0x55587cab6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.201+0000 7f83b19ee640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6e14d0 0x55587cab7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55396/0 (socket says 172.21.15.2:55396) 2024-05-14T09:51:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.202+0000 7f83b11ed640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6c0d80 0x55587cab6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:59560/0 (socket says 172.21.15.2:59560) 2024-05-14T09:51:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.202+0000 7f83b19ee640 1 -- 172.21.15.2:0/543345450 learned_addr learned my addr 172.21.15.2:0/543345450 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.202+0000 7f83ba3e6740 10 client.0 _ll_register_callbacks cb 0x7f839c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:51:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.202+0000 7f83b19ee640 1 -- 172.21.15.2:0/543345450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6c0d80 msgr2=0x55587cab6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.202+0000 7f83b19ee640 1 --2- 172.21.15.2:0/543345450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6c0d80 0x55587cab6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.202+0000 7f83b09ec640 1 --2- 172.21.15.2:0/543345450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.202+0000 7f83b19ee640 1 -- 172.21.15.2:0/543345450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 msgr2=0x55587cab6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.202+0000 7f83b19ee640 1 --2- 172.21.15.2:0/543345450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.202+0000 7f83b19ee640 1 -- 172.21.15.2:0/543345450 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55587cab7fc0 con 0x55587c6e14d0 2024-05-14T09:51:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.202+0000 7f83b09ec640 1 --2- 172.21.15.2:0/543345450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:51:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.202+0000 7f83b19ee640 1 --2- 172.21.15.2:0/543345450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6e14d0 0x55587cab7a00 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f83a4004af0 tx=0x7f83a402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf407fc16a085783 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.203+0000 7f8394ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:51:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.203+0000 7f8394ff9640 1 -- 172.21.15.2:0/543345450 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4004710 con 0x55587c6e14d0 2024-05-14T09:51:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.203+0000 7f8394ff9640 1 -- 172.21.15.2:0/543345450 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f83a402f660 con 0x55587c6e14d0 2024-05-14T09:51:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.203+0000 7f83ba3e6740 1 -- 172.21.15.2:0/543345450 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83a8023a40 con 0x55587c6e14d0 2024-05-14T09:51:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.203+0000 7f8394ff9640 1 -- 172.21.15.2:0/543345450 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f83a402f860 con 0x55587c6e14d0 2024-05-14T09:51:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.203+0000 7f83ba3e6740 10 client.17067 fetch_fsmap learned FSMap version 56 2024-05-14T09:51:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.203+0000 7f83ba3e6740 1 -- 172.21.15.2:0/543345450 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55587cab9280 con 0x55587c6e14d0 2024-05-14T09:51:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.204+0000 7f8394ff9640 1 -- 172.21.15.2:0/543345450 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a8023a40 con 0x55587c6e14d0 2024-05-14T09:51:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.204+0000 7f8394ff9640 1 -- 172.21.15.2:0/543345450 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 56) ==== 8358+0+0 (secure 0 0 0) 0x7f83a4030b60 con 0x55587c6e14d0 2024-05-14T09:51:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.204+0000 7f83ba3e6740 10 client.17067 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T09:51:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.204+0000 7f83ba3e6740 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:51:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.204+0000 7f83ba3e6740 20 client.17067 populate_metadata read hostname 'smithi002' 2024-05-14T09:51:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.204+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/543345450 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55587c7021b0 0x55587cb29c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.204+0000 7f83ba3e6740 4 client.17067 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:51:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.204+0000 7f83ba3e6740 1 -- 172.21.15.2:0/543345450 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55587ca161e0 con 0x55587c7021b0 2024-05-14T09:51:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.205+0000 7f83b11ed640 1 --2- 172.21.15.2:0/543345450 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55587c7021b0 0x55587cb29c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.205+0000 7f8394ff9640 10 client.17067 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.205+0000 7f83b11ed640 1 --2- 172.21.15.2:0/543345450 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55587c7021b0 0x55587cb29c30 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d1426db17dfeb6 server_cookie=4dcb2ff6f64898cb in_seq=0 out_seq=0 2024-05-14T09:51:28.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 95872 29.37 MB/sec execute 54 sec latency 308.637 ms 2024-05-14T09:51:28.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:28 smithi002 ceph-mon[31557]: pgmap v920: 129 pgs: 129 active+clean; 561 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 600 op/s 2024-05-14T09:51:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:28 smithi029 ceph-mon[36249]: pgmap v920: 129 pgs: 129 active+clean; 561 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 600 op/s 2024-05-14T09:51:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:28 smithi026 ceph-mon[36056]: pgmap v920: 129 pgs: 129 active+clean; 561 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 600 op/s 2024-05-14T09:51:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.961+0000 7f8394ff9640 1 -- 172.21.15.2:0/543345450 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55587ca161e0 con 0x55587c7021b0 2024-05-14T09:51:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.962+0000 7f8394ff9640 10 client.17067 handle_command_reply: tid=0 2024-05-14T09:51:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.962+0000 7f83ba3e6740 1 client.17067 shutdown 2024-05-14T09:51:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.962+0000 7f83ba3e6740 1 -- 172.21.15.2:0/543345450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6e14d0 msgr2=0x55587cab7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.962+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/543345450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6e14d0 0x55587cab7a00 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f83a4004af0 tx=0x7f83a402a8c0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.962+0000 7f83ba3e6740 1 -- 172.21.15.2:0/543345450 shutdown_connections 2024-05-14T09:51:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.962+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/543345450 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55587c7021b0 0x55587cb29c30 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.962+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/543345450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55587c6e14d0 0x55587cab7a00 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.962+0000 7f8394ff9640 0 client.17067 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.962+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/543345450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55587c6d92f0 0x55587cab6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.962+0000 7f83ba3e6740 1 --2- 172.21.15.2:0/543345450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55587c6c0d80 0x55587cab6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.962+0000 7f83ba3e6740 1 -- 172.21.15.2:0/543345450 >> 172.21.15.2:0/543345450 conn(0x55587c6f6300 msgr2=0x55587cab1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.962+0000 7f83ba3e6740 1 -- 172.21.15.2:0/543345450 shutdown_connections 2024-05-14T09:51:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.963+0000 7f83ba3e6740 1 -- 172.21.15.2:0/543345450 wait complete. 2024-05-14T09:51:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.963+0000 7f83ba3e6740 20 client.17067 trim_cache size 0 max 16384 2024-05-14T09:51:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:51:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:51:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:51:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:51:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.964+0000 7f83b3451640 1 -- 172.21.15.2:0/1302361295 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83980b28a0 msgr2=0x7f83980b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.964+0000 7f83b3451640 1 --2- 172.21.15.2:0/1302361295 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83980b28a0 0x7f83980b4db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009a60 tx=0x7f83a40095c0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.964+0000 7f83b3451640 1 -- 172.21.15.2:0/1302361295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac103d90 msgr2=0x7f83ac19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.964+0000 7f83b3451640 1 --2- 172.21.15.2:0/1302361295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac103d90 0x7f83ac19f010 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f839c00e9f0 tx=0x7f839c00edf0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.964+0000 7f83b3451640 1 -- 172.21.15.2:0/1302361295 shutdown_connections 2024-05-14T09:51:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.964+0000 7f83b3451640 1 --2- 172.21.15.2:0/1302361295 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83980b28a0 0x7f83980b4db0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.964+0000 7f83b3451640 1 --2- 172.21.15.2:0/1302361295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac108a90 0x7f83ac1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.964+0000 7f83b3451640 1 --2- 172.21.15.2:0/1302361295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac103d90 0x7f83ac19f010 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.964+0000 7f83b3451640 1 --2- 172.21.15.2:0/1302361295 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83ac102b50 0x7f83ac06b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.964+0000 7f83b3451640 1 -- 172.21.15.2:0/1302361295 >> 172.21.15.2:0/1302361295 conn(0x7f83ac0fdd40 msgr2=0x7f83ac104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.964+0000 7f83b3451640 1 -- 172.21.15.2:0/1302361295 shutdown_connections 2024-05-14T09:51:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:28.964+0000 7f83b3451640 1 -- 172.21.15.2:0/1302361295 wait complete. 2024-05-14T09:51:29.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 96343 29.25 MB/sec execute 55 sec latency 577.768 ms 2024-05-14T09:51:30.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 96868 29.23 MB/sec execute 56 sec latency 174.611 ms 2024-05-14T09:51:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:30 smithi002 ceph-mon[31557]: pgmap v921: 129 pgs: 129 active+clean; 568 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 653 op/s 2024-05-14T09:51:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:30 smithi029 ceph-mon[36249]: pgmap v921: 129 pgs: 129 active+clean; 568 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 653 op/s 2024-05-14T09:51:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:30 smithi026 ceph-mon[36056]: pgmap v921: 129 pgs: 129 active+clean; 568 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 653 op/s 2024-05-14T09:51:31.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 97552 29.27 MB/sec execute 57 sec latency 145.014 ms 2024-05-14T09:51:32.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 98260 29.61 MB/sec execute 58 sec latency 65.536 ms 2024-05-14T09:51:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:32 smithi002 ceph-mon[31557]: pgmap v922: 129 pgs: 129 active+clean; 568 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.3 MiB/s wr, 448 op/s 2024-05-14T09:51:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:32 smithi029 ceph-mon[36249]: pgmap v922: 129 pgs: 129 active+clean; 568 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.3 MiB/s wr, 448 op/s 2024-05-14T09:51:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:32 smithi026 ceph-mon[36056]: pgmap v922: 129 pgs: 129 active+clean; 568 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.3 MiB/s wr, 448 op/s 2024-05-14T09:51:33.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 98453 29.19 MB/sec execute 59 sec latency 769.177 ms 2024-05-14T09:51:34.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 98466 28.72 MB/sec execute 60 sec latency 1736.748 ms 2024-05-14T09:51:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:34 smithi002 ceph-mon[31557]: pgmap v923: 129 pgs: 129 active+clean; 570 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 595 op/s 2024-05-14T09:51:34.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:34 smithi029 ceph-mon[36249]: pgmap v923: 129 pgs: 129 active+clean; 570 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 595 op/s 2024-05-14T09:51:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:34 smithi026 ceph-mon[36056]: pgmap v923: 129 pgs: 129 active+clean; 570 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 595 op/s 2024-05-14T09:51:35.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 98663 28.41 MB/sec execute 61 sec latency 2657.153 ms 2024-05-14T09:51:36.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 98896 28.14 MB/sec execute 62 sec latency 304.926 ms 2024-05-14T09:51:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:36 smithi002 ceph-mon[31557]: pgmap v924: 129 pgs: 129 active+clean; 605 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.7 MiB/s wr, 630 op/s 2024-05-14T09:51:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:51:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:51:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:51:36.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:51:36.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:51:36.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:51:36.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:36 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:51:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:36 smithi029 ceph-mon[36249]: pgmap v924: 129 pgs: 129 active+clean; 605 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.7 MiB/s wr, 630 op/s 2024-05-14T09:51:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:51:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:51:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:51:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:51:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:51:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:51:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:36 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:51:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:36 smithi026 ceph-mon[36056]: pgmap v924: 129 pgs: 129 active+clean; 605 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.7 MiB/s wr, 630 op/s 2024-05-14T09:51:36.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:51:36.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:51:36.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:51:36.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T09:51:36.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:51:36.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:51:36.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:36 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:51:37.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 99195 27.86 MB/sec execute 63 sec latency 142.481 ms 2024-05-14T09:51:37.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:37 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi029 to 177.2M 2024-05-14T09:51:37.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:37 smithi002 ceph-mon[31557]: Unable to set osd_memory_target on smithi029 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-14T09:51:37.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:37 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi029 to 177.2M 2024-05-14T09:51:37.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:37 smithi029 ceph-mon[36249]: Unable to set osd_memory_target on smithi029 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-14T09:51:37.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:37 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi029 to 177.2M 2024-05-14T09:51:37.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:37 smithi026 ceph-mon[36056]: Unable to set osd_memory_target on smithi029 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-14T09:51:38.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 99640 27.78 MB/sec execute 64 sec latency 119.571 ms 2024-05-14T09:51:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:38 smithi002 ceph-mon[31557]: pgmap v925: 129 pgs: 129 active+clean; 613 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 720 op/s 2024-05-14T09:51:38.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:38 smithi029 ceph-mon[36249]: pgmap v925: 129 pgs: 129 active+clean; 613 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 720 op/s 2024-05-14T09:51:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:38 smithi026 ceph-mon[36056]: pgmap v925: 129 pgs: 129 active+clean; 613 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 720 op/s 2024-05-14T09:51:39.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 100260 27.99 MB/sec execute 65 sec latency 80.049 ms 2024-05-14T09:51:40.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 100869 28.16 MB/sec execute 66 sec latency 87.544 ms 2024-05-14T09:51:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:40 smithi002 ceph-mon[31557]: pgmap v926: 129 pgs: 129 active+clean; 571 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 859 op/s 2024-05-14T09:51:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:40 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:40 smithi029 ceph-mon[36249]: pgmap v926: 129 pgs: 129 active+clean; 571 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 859 op/s 2024-05-14T09:51:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:40 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:40 smithi026 ceph-mon[36056]: pgmap v926: 129 pgs: 129 active+clean; 571 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 859 op/s 2024-05-14T09:51:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:40 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:41.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 101343 28.04 MB/sec execute 67 sec latency 215.244 ms 2024-05-14T09:51:41.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:41 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:41.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:41 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:41.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:41 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.093+0000 7f8bbe56c640 1 -- 172.21.15.2:0/4209216057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bb8108b30 msgr2=0x7f8bb810af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.093+0000 7f8bbe56c640 1 --2- 172.21.15.2:0/4209216057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bb8108b30 0x7f8bb810af50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009ca0 tx=0x7f8bb002a8c0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.093+0000 7f8bbe56c640 1 -- 172.21.15.2:0/4209216057 shutdown_connections 2024-05-14T09:51:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.093+0000 7f8bbe56c640 1 --2- 172.21.15.2:0/4209216057 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bb8108b30 0x7f8bb810af50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.093+0000 7f8bbe56c640 1 --2- 172.21.15.2:0/4209216057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb8103e30 0x7f8bb8104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.093+0000 7f8bbe56c640 1 --2- 172.21.15.2:0/4209216057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb8102bf0 0x7f8bb81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.093+0000 7f8bbe56c640 1 -- 172.21.15.2:0/4209216057 >> 172.21.15.2:0/4209216057 conn(0x7f8bb80fddc0 msgr2=0x7f8bb8100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.093+0000 7f8bbe56c640 1 -- 172.21.15.2:0/4209216057 shutdown_connections 2024-05-14T09:51:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.093+0000 7f8bbe56c640 1 -- 172.21.15.2:0/4209216057 wait complete. 2024-05-14T09:51:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.094+0000 7f8bbe56c640 1 Processor -- start 2024-05-14T09:51:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.094+0000 7f8bbe56c640 1 -- start start 2024-05-14T09:51:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.094+0000 7f8bbe56c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb8102bf0 0x7f8bb819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.094+0000 7f8bbe56c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bb8103e30 0x7f8bb819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.094+0000 7f8bbe56c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb8108b30 0x7f8bb81a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.094+0000 7f8bbe56c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb810e750 con 0x7f8bb8108b30 2024-05-14T09:51:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.094+0000 7f8bbe56c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb810e5d0 con 0x7f8bb8102bf0 2024-05-14T09:51:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.094+0000 7f8bbe56c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb810e8d0 con 0x7f8bb8103e30 2024-05-14T09:51:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.094+0000 7f8bbcb09640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb8108b30 0x7f8bb81a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.094+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb8102bf0 0x7f8bb819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.094+0000 7f8bbcb09640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb8108b30 0x7f8bb81a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35602/0 (socket says 172.21.15.2:35602) 2024-05-14T09:51:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.094+0000 7f8bbcb09640 1 -- 172.21.15.2:0/1966173883 learned_addr learned my addr 172.21.15.2:0/1966173883 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.094+0000 7f8bb77fe640 1 --2- 172.21.15.2:0/1966173883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bb8103e30 0x7f8bb819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.095+0000 7f8bbcb09640 1 -- 172.21.15.2:0/1966173883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bb8103e30 msgr2=0x7f8bb819a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.095+0000 7f8bbcb09640 1 --2- 172.21.15.2:0/1966173883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bb8103e30 0x7f8bb819a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.095+0000 7f8bbcb09640 1 -- 172.21.15.2:0/1966173883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb8102bf0 msgr2=0x7f8bb819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.095+0000 7f8bbcb09640 1 --2- 172.21.15.2:0/1966173883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb8102bf0 0x7f8bb819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.095+0000 7f8bbcb09640 1 -- 172.21.15.2:0/1966173883 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb81a1e20 con 0x7f8bb8108b30 2024-05-14T09:51:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.095+0000 7f8bb7fff640 1 --2- 172.21.15.2:0/1966173883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb8102bf0 0x7f8bb819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:51:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.095+0000 7f8bbcb09640 1 --2- 172.21.15.2:0/1966173883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb8108b30 0x7f8bb81a17b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009430 tx=0x7f8bb00317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.095+0000 7f8bb57fa640 1 -- 172.21.15.2:0/1966173883 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0003bf0 con 0x7f8bb8108b30 2024-05-14T09:51:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.095+0000 7f8bb57fa640 1 -- 172.21.15.2:0/1966173883 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb0003d90 con 0x7f8bb8108b30 2024-05-14T09:51:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.096+0000 7f8bb57fa640 1 -- 172.21.15.2:0/1966173883 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb002fc80 con 0x7f8bb8108b30 2024-05-14T09:51:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.096+0000 7f8bbe56c640 1 -- 172.21.15.2:0/1966173883 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb81a20b0 con 0x7f8bb8108b30 2024-05-14T09:51:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.096+0000 7f8bbe56c640 1 -- 172.21.15.2:0/1966173883 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb81a2650 con 0x7f8bb8108b30 2024-05-14T09:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.098+0000 7f8bc541c740 1 Processor -- start 2024-05-14T09:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.098+0000 7f8bb57fa640 1 -- 172.21.15.2:0/1966173883 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8bb0004710 con 0x7f8bb8108b30 2024-05-14T09:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bc541c740 1 -- start start 2024-05-14T09:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bc541c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 0x561da059aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bc541c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d02f0 0x561da059afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bc541c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d84d0 0x561da05a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bc541c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561da04ff5d0 con 0x561da01d02f0 2024-05-14T09:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bc541c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561da04b8890 con 0x561da01f91b0 2024-05-14T09:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bc541c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561da04fb590 con 0x561da01d84d0 2024-05-14T09:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d02f0 0x561da059afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 0x561da059aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 0x561da059aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:38056/0 (socket says 172.21.15.2:38056) 2024-05-14T09:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d02f0 0x561da059afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35604/0 (socket says 172.21.15.2:35604) 2024-05-14T09:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bb77fe640 1 -- 172.21.15.2:0/1235514589 learned_addr learned my addr 172.21.15.2:0/1235514589 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bb77fe640 1 --2- 172.21.15.2:0/1235514589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d84d0 0x561da05a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bb77fe640 1 -- 172.21.15.2:0/1235514589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d84d0 msgr2=0x561da05a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bb77fe640 1 --2- 172.21.15.2:0/1235514589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d84d0 0x561da05a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bb77fe640 1 -- 172.21.15.2:0/1235514589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d02f0 msgr2=0x561da059afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bb77fe640 1 --2- 172.21.15.2:0/1235514589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d02f0 0x561da059afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.099+0000 7f8bb77fe640 1 -- 172.21.15.2:0/1235514589 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561da0211210 con 0x561da01f91b0 2024-05-14T09:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.100+0000 7f8bb77fe640 1 --2- 172.21.15.2:0/1235514589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 0x561da059aa60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8bac005bc0 tx=0x7f8bac004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be1424fe40c3bd3b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.100+0000 7f8b9e7fc640 1 -- 172.21.15.2:0/1235514589 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac004030 con 0x561da01f91b0 2024-05-14T09:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.100+0000 7f8b9e7fc640 1 -- 172.21.15.2:0/1235514589 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bac005020 con 0x561da01f91b0 2024-05-14T09:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.100+0000 7f8b9e7fc640 1 -- 172.21.15.2:0/1235514589 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac004bc0 con 0x561da01f91b0 2024-05-14T09:51:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.100+0000 7f8bb57fa640 1 --2- 172.21.15.2:0/1966173883 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8ba00b2920 0x7f8ba00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.100+0000 7f8bb7fff640 1 --2- 172.21.15.2:0/1966173883 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8ba00b2920 0x7f8ba00b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.100+0000 7f8bb57fa640 1 -- 172.21.15.2:0/1966173883 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f8bb0103ce0 con 0x7f8bb8108b30 2024-05-14T09:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.101+0000 7f8bc541c740 1 -- 172.21.15.2:0/1235514589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 msgr2=0x561da059aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.101+0000 7f8bc541c740 1 --2- 172.21.15.2:0/1235514589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 0x561da059aa60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8bac005bc0 tx=0x7f8bac004730 comp rx=0 tx=0).stop 2024-05-14T09:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.101+0000 7f8bb7fff640 1 --2- 172.21.15.2:0/1966173883 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8ba00b2920 0x7f8ba00b4e30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x561da059be40 tx=0x7f8ba800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.101+0000 7f8bc541c740 1 -- 172.21.15.2:0/1235514589 shutdown_connections 2024-05-14T09:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.101+0000 7f8bc541c740 1 --2- 172.21.15.2:0/1235514589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d84d0 0x561da05a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.101+0000 7f8bc541c740 1 --2- 172.21.15.2:0/1235514589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d02f0 0x561da059afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.101+0000 7f8bc541c740 1 --2- 172.21.15.2:0/1235514589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 0x561da059aa60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.101+0000 7f8bc541c740 1 -- 172.21.15.2:0/1235514589 >> 172.21.15.2:0/1235514589 conn(0x561da01ed300 msgr2=0x561da05963a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.101+0000 7f8bc541c740 1 -- 172.21.15.2:0/1235514589 shutdown_connections 2024-05-14T09:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.101+0000 7f8bc541c740 1 -- 172.21.15.2:0/1235514589 wait complete. 2024-05-14T09:51:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.101+0000 7f8bc541c740 1 -- start start 2024-05-14T09:51:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.101+0000 7f8bc541c740 1 Processor -- start 2024-05-14T09:51:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bc541c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 0x561da05a4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bc541c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d84d0 0x561da05a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bc541c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 0x561da05abba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bc541c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561da05a2930 con 0x561da01d84d0 2024-05-14T09:51:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bc541c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561da04bc370 con 0x561da01f91b0 2024-05-14T09:51:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bc541c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x561da05a2ab0 con 0x561da01d02f0 2024-05-14T09:51:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d84d0 0x561da05a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 0x561da05a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d84d0 0x561da05a4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35608/0 (socket says 172.21.15.2:35608) 2024-05-14T09:51:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bc541c740 10 client.0 _ll_register_callbacks cb 0x561da0210a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:51:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 0x561da05a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:41504/0 (socket says 172.21.15.2:41504) 2024-05-14T09:51:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bb77fe640 1 -- 172.21.15.2:0/3263175589 learned_addr learned my addr 172.21.15.2:0/3263175589 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bbcb09640 1 --2- 172.21.15.2:0/3263175589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 0x561da05abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bb77fe640 1 -- 172.21.15.2:0/3263175589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 msgr2=0x561da05a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bb77fe640 1 --2- 172.21.15.2:0/3263175589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 0x561da05a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bb77fe640 1 -- 172.21.15.2:0/3263175589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 msgr2=0x561da05abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bb77fe640 1 --2- 172.21.15.2:0/3263175589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 0x561da05abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.102+0000 7f8bb77fe640 1 -- 172.21.15.2:0/3263175589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561da02dd620 con 0x561da01d84d0 2024-05-14T09:51:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.103+0000 7f8bb77fe640 1 --2- 172.21.15.2:0/3263175589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d84d0 0x561da05a4c40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8bac005540 tx=0x7f8bac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74eea0de694e1c01 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.103+0000 7f8b977fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:51:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.103+0000 7f8b977fe640 1 -- 172.21.15.2:0/3263175589 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac004a00 con 0x561da01d84d0 2024-05-14T09:51:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.103+0000 7f8bc541c740 1 -- 172.21.15.2:0/3263175589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bac005020 con 0x561da01d84d0 2024-05-14T09:51:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.103+0000 7f8b977fe640 1 -- 172.21.15.2:0/3263175589 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f8bac004ba0 con 0x561da01d84d0 2024-05-14T09:51:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.103+0000 7f8b977fe640 1 -- 172.21.15.2:0/3263175589 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8bac004de0 con 0x561da01d84d0 2024-05-14T09:51:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.103+0000 7f8bc541c740 10 client.17079 fetch_fsmap learned FSMap version 58 2024-05-14T09:51:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.103+0000 7f8bc541c740 1 -- 172.21.15.2:0/3263175589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561da05aead0 con 0x561da01d84d0 2024-05-14T09:51:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.103+0000 7f8b977fe640 1 -- 172.21.15.2:0/3263175589 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bac005020 con 0x561da01d84d0 2024-05-14T09:51:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.103+0000 7f8b977fe640 1 -- 172.21.15.2:0/3263175589 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 58) ==== 8350+0+0 (secure 0 0 0) 0x7f8bac02a6e0 con 0x561da01d84d0 2024-05-14T09:51:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.103+0000 7f8bc541c740 10 client.17079 fetch_fsmap finished waiting for FSMap version 58 2024-05-14T09:51:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.103+0000 7f8bc541c740 10 client.17079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:51:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.103+0000 7f8bc541c740 20 client.17079 populate_metadata read hostname 'smithi002' 2024-05-14T09:51:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.103+0000 7f8bc541c740 1 --2- 172.21.15.2:0/3263175589 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561da01b7d80 0x561da05b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.104+0000 7f8bc541c740 4 client.17079 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:51:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.104+0000 7f8bc541c740 1 -- 172.21.15.2:0/3263175589 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561da050d1e0 con 0x561da01b7d80 2024-05-14T09:51:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.104+0000 7f8bb7fff640 1 --2- 172.21.15.2:0/3263175589 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561da01b7d80 0x561da05b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.104+0000 7f8bb7fff640 1 --2- 172.21.15.2:0/3263175589 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561da01b7d80 0x561da05b2030 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c9b60e0077d6237 server_cookie=3c94843d029a3212 in_seq=0 out_seq=0 2024-05-14T09:51:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.104+0000 7f8b977fe640 10 client.17079 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.107+0000 7f8b977fe640 1 -- 172.21.15.2:0/3263175589 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561da050d1e0 con 0x561da01b7d80 2024-05-14T09:51:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.107+0000 7f8b977fe640 10 client.17079 handle_command_reply: tid=0 2024-05-14T09:51:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.107+0000 7f8bc541c740 1 client.17079 shutdown 2024-05-14T09:51:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.107+0000 7f8bc541c740 1 -- 172.21.15.2:0/3263175589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d84d0 msgr2=0x561da05a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.107+0000 7f8bc541c740 1 --2- 172.21.15.2:0/3263175589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d84d0 0x561da05a4c40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8bac005540 tx=0x7f8bac0046c0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.107+0000 7f8bc541c740 1 -- 172.21.15.2:0/3263175589 shutdown_connections 2024-05-14T09:51:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.107+0000 7f8bc541c740 1 --2- 172.21.15.2:0/3263175589 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561da01b7d80 0x561da05b2030 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.107+0000 7f8bc541c740 1 --2- 172.21.15.2:0/3263175589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 0x561da05abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.107+0000 7f8b977fe640 0 client.17079 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.107+0000 7f8bc541c740 1 --2- 172.21.15.2:0/3263175589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d84d0 0x561da05a4c40 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.107+0000 7f8bc541c740 1 --2- 172.21.15.2:0/3263175589 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 0x561da05a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.107+0000 7f8bc541c740 1 -- 172.21.15.2:0/3263175589 >> 172.21.15.2:0/3263175589 conn(0x561da01ed300 msgr2=0x561da0599300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.107+0000 7f8bc541c740 1 -- 172.21.15.2:0/3263175589 shutdown_connections 2024-05-14T09:51:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.107+0000 7f8bc541c740 1 -- 172.21.15.2:0/3263175589 wait complete. 2024-05-14T09:51:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.107+0000 7f8bc541c740 20 client.17079 trim_cache size 0 max 16384 2024-05-14T09:51:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.130+0000 7f8bc541c740 1 Processor -- start 2024-05-14T09:51:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.131+0000 7f8bc541c740 1 -- start start 2024-05-14T09:51:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.131+0000 7f8bc541c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 0x561da0595d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.131+0000 7f8bc541c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01d84d0 0x561da05abac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.132+0000 7f8bc541c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01f91b0 0x561da05962a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.132+0000 7f8bc541c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81b1730 con 0x561da01f91b0 2024-05-14T09:51:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.132+0000 7f8bc541c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81aefd0 con 0x561da01d84d0 2024-05-14T09:51:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.132+0000 7f8bc541c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81ace20 con 0x561da01d02f0 2024-05-14T09:51:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.132+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01d84d0 0x561da05abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.132+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01d84d0 0x561da05abac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:38070/0 (socket says 172.21.15.2:38070) 2024-05-14T09:51:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.132+0000 7f8bb77fe640 1 -- 172.21.15.2:0/82670300 learned_addr learned my addr 172.21.15.2:0/82670300 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.133+0000 7f8bb7fff640 1 --2- 172.21.15.2:0/82670300 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 0x561da0595d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.133+0000 7f8bbcb09640 1 --2- 172.21.15.2:0/82670300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01f91b0 0x561da05962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.134+0000 7f8bb77fe640 1 -- 172.21.15.2:0/82670300 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 msgr2=0x561da0595d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.134+0000 7f8bb77fe640 1 --2- 172.21.15.2:0/82670300 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 0x561da0595d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.134+0000 7f8bb77fe640 1 -- 172.21.15.2:0/82670300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01f91b0 msgr2=0x561da05962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.134+0000 7f8bb77fe640 1 --2- 172.21.15.2:0/82670300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01f91b0 0x561da05962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.134+0000 7f8bb77fe640 1 -- 172.21.15.2:0/82670300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561da0596d00 con 0x561da01d84d0 2024-05-14T09:51:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.135+0000 7f8bb77fe640 1 --2- 172.21.15.2:0/82670300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01d84d0 0x561da05abac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8bac02aab0 tx=0x7f8bac005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4dc005c3bc9e4f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.136+0000 7f8b9dffb640 1 -- 172.21.15.2:0/82670300 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac03c070 con 0x561da01d84d0 2024-05-14T09:51:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.136+0000 7f8b9dffb640 1 -- 172.21.15.2:0/82670300 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bac004a00 con 0x561da01d84d0 2024-05-14T09:51:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.136+0000 7f8b9dffb640 1 -- 172.21.15.2:0/82670300 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00fa80 con 0x561da01d84d0 2024-05-14T09:51:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.138+0000 7f8bc541c740 1 -- 172.21.15.2:0/82670300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01d84d0 msgr2=0x561da05abac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.138+0000 7f8bc541c740 1 --2- 172.21.15.2:0/82670300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01d84d0 0x561da05abac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8bac02aab0 tx=0x7f8bac005fd0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.138+0000 7f8bc541c740 1 -- 172.21.15.2:0/82670300 shutdown_connections 2024-05-14T09:51:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.138+0000 7f8bc541c740 1 --2- 172.21.15.2:0/82670300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01f91b0 0x561da05962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.138+0000 7f8bc541c740 1 --2- 172.21.15.2:0/82670300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01d84d0 0x561da05abac0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.138+0000 7f8bc541c740 1 --2- 172.21.15.2:0/82670300 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 0x561da0595d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.138+0000 7f8bc541c740 1 -- 172.21.15.2:0/82670300 >> 172.21.15.2:0/82670300 conn(0x561da01ed300 msgr2=0x561da059eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.139+0000 7f8bc541c740 1 -- 172.21.15.2:0/82670300 shutdown_connections 2024-05-14T09:51:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.139+0000 7f8bc541c740 1 -- 172.21.15.2:0/82670300 wait complete. 2024-05-14T09:51:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.139+0000 7f8bc541c740 1 -- start start 2024-05-14T09:51:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.140+0000 7f8bc541c740 1 Processor -- start 2024-05-14T09:51:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.140+0000 7f8bc541c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 0x561da05a8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.140+0000 7f8bc541c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d84d0 0x561da05a8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.140+0000 7f8bc541c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 0x561da06183d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.140+0000 7f8bc541c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81ad490 con 0x561da01d84d0 2024-05-14T09:51:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.140+0000 7f8bc541c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81ae900 con 0x561da01f91b0 2024-05-14T09:51:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.140+0000 7f8bc541c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81adb60 con 0x561da01d02f0 2024-05-14T09:51:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.140+0000 7f8bc541c740 10 client.0 _ll_register_callbacks cb 0x7f8ba800a960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:51:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.141+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d84d0 0x561da05a8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.141+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 0x561da05a8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.141+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 0x561da05a8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:41526/0 (socket says 172.21.15.2:41526) 2024-05-14T09:51:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.141+0000 7f8bb77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d84d0 0x561da05a8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35628/0 (socket says 172.21.15.2:35628) 2024-05-14T09:51:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.141+0000 7f8bb7fff640 1 -- 172.21.15.2:0/948644895 learned_addr learned my addr 172.21.15.2:0/948644895 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.141+0000 7f8bbcb09640 1 --2- 172.21.15.2:0/948644895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 0x561da06183d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.141+0000 7f8bb7fff640 1 -- 172.21.15.2:0/948644895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 msgr2=0x561da06183d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.141+0000 7f8bb7fff640 1 --2- 172.21.15.2:0/948644895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 0x561da06183d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.141+0000 7f8bb7fff640 1 -- 172.21.15.2:0/948644895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d84d0 msgr2=0x561da05a8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.141+0000 7f8bb7fff640 1 --2- 172.21.15.2:0/948644895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d84d0 0x561da05a8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.141+0000 7f8bb7fff640 1 -- 172.21.15.2:0/948644895 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561da05abd80 con 0x561da01d02f0 2024-05-14T09:51:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.142+0000 7f8bb7fff640 1 --2- 172.21.15.2:0/948644895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 0x561da05a8880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bb81037f0 tx=0x7f8ba80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=183234542f4ab866 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.142+0000 7f8b977fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:51:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.143+0000 7f8b977fe640 1 -- 172.21.15.2:0/948644895 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba800fe40 con 0x561da01d02f0 2024-05-14T09:51:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.143+0000 7f8b977fe640 1 -- 172.21.15.2:0/948644895 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f8ba8005030 con 0x561da01d02f0 2024-05-14T09:51:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.143+0000 7f8bc541c740 1 -- 172.21.15.2:0/948644895 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bac004a00 con 0x561da01d02f0 2024-05-14T09:51:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.144+0000 7f8b977fe640 1 -- 172.21.15.2:0/948644895 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8ba8005230 con 0x561da01d02f0 2024-05-14T09:51:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.144+0000 7f8bc541c740 10 client.26719 fetch_fsmap learned FSMap version 58 2024-05-14T09:51:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.144+0000 7f8bc541c740 1 -- 172.21.15.2:0/948644895 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561da062a9e0 con 0x561da01d02f0 2024-05-14T09:51:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.144+0000 7f8b977fe640 1 -- 172.21.15.2:0/948644895 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bac004a00 con 0x561da01d02f0 2024-05-14T09:51:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.145+0000 7f8b977fe640 1 -- 172.21.15.2:0/948644895 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 58) ==== 8350+0+0 (secure 0 0 0) 0x7f8ba80344f0 con 0x561da01d02f0 2024-05-14T09:51:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.145+0000 7f8bc541c740 10 client.26719 fetch_fsmap finished waiting for FSMap version 58 2024-05-14T09:51:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.145+0000 7f8bc541c740 10 client.26719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:51:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.145+0000 7f8bc541c740 20 client.26719 populate_metadata read hostname 'smithi002' 2024-05-14T09:51:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.145+0000 7f8bc541c740 1 --2- 172.21.15.2:0/948644895 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561da018d1c0 0x561da062d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.145+0000 7f8bc541c740 4 client.26719 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:51:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.145+0000 7f8bc541c740 1 -- 172.21.15.2:0/948644895 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561da0507050 con 0x561da018d1c0 2024-05-14T09:51:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.148+0000 7f8bb77fe640 1 --2- 172.21.15.2:0/948644895 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561da018d1c0 0x561da062d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.149+0000 7f8bb77fe640 1 --2- 172.21.15.2:0/948644895 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561da018d1c0 0x561da062d390 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27dc564945d8f849 server_cookie=98d611db3906e444 in_seq=0 out_seq=0 2024-05-14T09:51:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.149+0000 7f8b977fe640 10 client.26719 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.553+0000 7f8b977fe640 1 -- 172.21.15.2:0/948644895 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561da0507050 con 0x561da018d1c0 2024-05-14T09:51:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.553+0000 7f8b977fe640 10 client.26719 handle_command_reply: tid=0 2024-05-14T09:51:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.553+0000 7f8bc541c740 1 client.26719 shutdown 2024-05-14T09:51:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.553+0000 7f8bc541c740 1 -- 172.21.15.2:0/948644895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 msgr2=0x561da05a8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.553+0000 7f8bc541c740 1 --2- 172.21.15.2:0/948644895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 0x561da05a8880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bb81037f0 tx=0x7f8ba80046c0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.553+0000 7f8bc541c740 1 -- 172.21.15.2:0/948644895 shutdown_connections 2024-05-14T09:51:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.553+0000 7f8bc541c740 1 --2- 172.21.15.2:0/948644895 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x561da018d1c0 0x561da062d390 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.553+0000 7f8bc541c740 1 --2- 172.21.15.2:0/948644895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561da01f91b0 0x561da06183d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.553+0000 7f8bc541c740 1 --2- 172.21.15.2:0/948644895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561da01d84d0 0x561da05a8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.553+0000 7f8bc541c740 1 --2- 172.21.15.2:0/948644895 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561da01d02f0 0x561da05a8880 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.553+0000 7f8b977fe640 0 client.26719 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.553+0000 7f8bc541c740 1 -- 172.21.15.2:0/948644895 >> 172.21.15.2:0/948644895 conn(0x561da01ed300 msgr2=0x561da05991f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.553+0000 7f8bc541c740 1 -- 172.21.15.2:0/948644895 shutdown_connections 2024-05-14T09:51:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.553+0000 7f8bc541c740 1 -- 172.21.15.2:0/948644895 wait complete. 2024-05-14T09:51:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.553+0000 7f8bc541c740 20 client.26719 trim_cache size 0 max 16384 2024-05-14T09:51:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:51:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:51:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:51:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:51:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.554+0000 7f8bbe56c640 1 -- 172.21.15.2:0/1966173883 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8ba00b2920 msgr2=0x7f8ba00b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.554+0000 7f8bbe56c640 1 --2- 172.21.15.2:0/1966173883 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8ba00b2920 0x7f8ba00b4e30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x561da059be40 tx=0x7f8ba800a2d0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.554+0000 7f8bbe56c640 1 -- 172.21.15.2:0/1966173883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb8108b30 msgr2=0x7f8bb81a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.554+0000 7f8bbe56c640 1 --2- 172.21.15.2:0/1966173883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb8108b30 0x7f8bb81a17b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009430 tx=0x7f8bb00317c0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.554+0000 7f8bbe56c640 1 -- 172.21.15.2:0/1966173883 shutdown_connections 2024-05-14T09:51:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.554+0000 7f8bbe56c640 1 --2- 172.21.15.2:0/1966173883 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f8ba00b2920 0x7f8ba00b4e30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.554+0000 7f8bbe56c640 1 --2- 172.21.15.2:0/1966173883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb8108b30 0x7f8bb81a17b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.554+0000 7f8bbe56c640 1 --2- 172.21.15.2:0/1966173883 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8bb8103e30 0x7f8bb819a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.554+0000 7f8bbe56c640 1 --2- 172.21.15.2:0/1966173883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bb8102bf0 0x7f8bb819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.554+0000 7f8bbe56c640 1 -- 172.21.15.2:0/1966173883 >> 172.21.15.2:0/1966173883 conn(0x7f8bb80fddc0 msgr2=0x7f8bb8109ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.554+0000 7f8bbe56c640 1 -- 172.21.15.2:0/1966173883 shutdown_connections 2024-05-14T09:51:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:42.554+0000 7f8bbe56c640 1 -- 172.21.15.2:0/1966173883 wait complete. 2024-05-14T09:51:42.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 101840 28.04 MB/sec execute 68 sec latency 280.551 ms 2024-05-14T09:51:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:42 smithi002 ceph-mon[31557]: pgmap v927: 129 pgs: 129 active+clean; 563 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 12 MiB/s wr, 728 op/s 2024-05-14T09:51:43.272 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:42 smithi026 ceph-mon[36056]: pgmap v927: 129 pgs: 129 active+clean; 563 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 12 MiB/s wr, 728 op/s 2024-05-14T09:51:43.272 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:42 smithi029 ceph-mon[36249]: pgmap v927: 129 pgs: 129 active+clean; 563 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 12 MiB/s wr, 728 op/s 2024-05-14T09:51:43.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 102585 28.16 MB/sec execute 69 sec latency 86.821 ms 2024-05-14T09:51:44.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 103367 28.31 MB/sec execute 70 sec latency 84.022 ms 2024-05-14T09:51:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:44 smithi002 ceph-mon[31557]: pgmap v928: 129 pgs: 129 active+clean; 559 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 855 op/s 2024-05-14T09:51:44.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:44 smithi029 ceph-mon[36249]: pgmap v928: 129 pgs: 129 active+clean; 559 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 855 op/s 2024-05-14T09:51:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:44 smithi026 ceph-mon[36056]: pgmap v928: 129 pgs: 129 active+clean; 559 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 855 op/s 2024-05-14T09:51:45.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 104078 28.58 MB/sec execute 71 sec latency 69.321 ms 2024-05-14T09:51:45.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:45 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:45.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:45 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:45.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:45 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:46.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 104744 28.63 MB/sec execute 72 sec latency 191.625 ms 2024-05-14T09:51:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:46 smithi002 ceph-mon[31557]: pgmap v929: 129 pgs: 129 active+clean; 556 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 13 MiB/s wr, 818 op/s 2024-05-14T09:51:46.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:46 smithi029 ceph-mon[36249]: pgmap v929: 129 pgs: 129 active+clean; 556 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 13 MiB/s wr, 818 op/s 2024-05-14T09:51:46.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:46 smithi026 ceph-mon[36056]: pgmap v929: 129 pgs: 129 active+clean; 556 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 13 MiB/s wr, 818 op/s 2024-05-14T09:51:47.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 105382 28.72 MB/sec execute 73 sec latency 58.536 ms 2024-05-14T09:51:48.606 INFO:tasks.workunit.client.0.smithi002.stdout: 10 106025 28.75 MB/sec execute 74 sec latency 118.417 ms 2024-05-14T09:51:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:48 smithi002 ceph-mon[31557]: pgmap v930: 129 pgs: 129 active+clean; 558 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.6 MiB/s wr, 695 op/s 2024-05-14T09:51:48.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:48 smithi029 ceph-mon[36249]: pgmap v930: 129 pgs: 129 active+clean; 558 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.6 MiB/s wr, 695 op/s 2024-05-14T09:51:48.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:48 smithi026 ceph-mon[36056]: pgmap v930: 129 pgs: 129 active+clean; 558 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.6 MiB/s wr, 695 op/s 2024-05-14T09:51:49.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 106651 28.83 MB/sec execute 75 sec latency 99.233 ms 2024-05-14T09:51:50.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 107289 28.93 MB/sec execute 76 sec latency 73.830 ms 2024-05-14T09:51:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:50 smithi002 ceph-mon[31557]: pgmap v931: 129 pgs: 129 active+clean; 569 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.0 MiB/s wr, 708 op/s 2024-05-14T09:51:50.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:50 smithi029 ceph-mon[36249]: pgmap v931: 129 pgs: 129 active+clean; 569 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.0 MiB/s wr, 708 op/s 2024-05-14T09:51:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:50 smithi026 ceph-mon[36056]: pgmap v931: 129 pgs: 129 active+clean; 569 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.0 MiB/s wr, 708 op/s 2024-05-14T09:51:51.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 107846 28.95 MB/sec execute 77 sec latency 203.945 ms 2024-05-14T09:51:51.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:51 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:51.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:51 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:52.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:51 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:52.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 108582 29.09 MB/sec execute 78 sec latency 55.804 ms 2024-05-14T09:51:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:52 smithi029 ceph-mon[36249]: pgmap v932: 129 pgs: 129 active+clean; 569 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.0 MiB/s wr, 470 op/s 2024-05-14T09:51:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:52 smithi026 ceph-mon[36056]: pgmap v932: 129 pgs: 129 active+clean; 569 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.0 MiB/s wr, 470 op/s 2024-05-14T09:51:53.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:52 smithi002 ceph-mon[31557]: pgmap v932: 129 pgs: 129 active+clean; 569 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.0 MiB/s wr, 470 op/s 2024-05-14T09:51:53.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 109143 29.08 MB/sec execute 79 sec latency 224.691 ms 2024-05-14T09:51:54.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 109884 29.19 MB/sec execute 80 sec latency 66.316 ms 2024-05-14T09:51:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:54 smithi026 ceph-mon[36056]: pgmap v933: 129 pgs: 129 active+clean; 578 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 569 op/s 2024-05-14T09:51:55.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:54 smithi002 ceph-mon[31557]: pgmap v933: 129 pgs: 129 active+clean; 578 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 569 op/s 2024-05-14T09:51:55.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:54 smithi029 ceph-mon[36249]: pgmap v933: 129 pgs: 129 active+clean; 578 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 569 op/s 2024-05-14T09:51:55.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 110566 29.30 MB/sec execute 81 sec latency 66.279 ms 2024-05-14T09:51:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.702+0000 7f64c3772640 1 -- 172.21.15.2:0/1850863749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64bc102b60 msgr2=0x7f64bc103860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.702+0000 7f64c3772640 1 --2- 172.21.15.2:0/1850863749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64bc102b60 0x7f64bc103860 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f64b800dd10 tx=0x7f64b8032970 comp rx=0 tx=0).stop 2024-05-14T09:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.702+0000 7f64c3772640 1 -- 172.21.15.2:0/1850863749 shutdown_connections 2024-05-14T09:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.702+0000 7f64c3772640 1 --2- 172.21.15.2:0/1850863749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64bc108aa0 0x7f64bc10aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.702+0000 7f64c3772640 1 --2- 172.21.15.2:0/1850863749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64bc103da0 0x7f64bc1041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.702+0000 7f64c3772640 1 --2- 172.21.15.2:0/1850863749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64bc102b60 0x7f64bc103860 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.702+0000 7f64c3772640 1 -- 172.21.15.2:0/1850863749 >> 172.21.15.2:0/1850863749 conn(0x7f64bc0fdd50 msgr2=0x7f64bc100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.702+0000 7f64c3772640 1 -- 172.21.15.2:0/1850863749 shutdown_connections 2024-05-14T09:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.702+0000 7f64c3772640 1 -- 172.21.15.2:0/1850863749 wait complete. 2024-05-14T09:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.703+0000 7f64c3772640 1 Processor -- start 2024-05-14T09:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.703+0000 7f64c3772640 1 -- start start 2024-05-14T09:51:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.703+0000 7f64c3772640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64bc102b60 0x7f64bc198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.703+0000 7f64c3772640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64bc103da0 0x7f64bc1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.703+0000 7f64c3772640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64bc108aa0 0x7f64bc19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.703+0000 7f64c3772640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc10d570 con 0x7f64bc102b60 2024-05-14T09:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.704+0000 7f64c3772640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc10d3f0 con 0x7f64bc108aa0 2024-05-14T09:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.704+0000 7f64c3772640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc10d6f0 con 0x7f64bc103da0 2024-05-14T09:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.704+0000 7f64c150e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64bc102b60 0x7f64bc198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.704+0000 7f64c0d0d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64bc103da0 0x7f64bc1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.704+0000 7f64c0d0d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64bc103da0 0x7f64bc1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:55872/0 (socket says 172.21.15.2:55872) 2024-05-14T09:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.704+0000 7f64c0d0d640 1 -- 172.21.15.2:0/242697549 learned_addr learned my addr 172.21.15.2:0/242697549 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.704+0000 7f64c150e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64bc102b60 0x7f64bc198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56356/0 (socket says 172.21.15.2:56356) 2024-05-14T09:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.704+0000 7f64c150e640 1 -- 172.21.15.2:0/242697549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64bc103da0 msgr2=0x7f64bc1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.704+0000 7f64c150e640 1 --2- 172.21.15.2:0/242697549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64bc103da0 0x7f64bc1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.704+0000 7f64c150e640 1 -- 172.21.15.2:0/242697549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64bc108aa0 msgr2=0x7f64bc19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.704+0000 7f64c150e640 1 --2- 172.21.15.2:0/242697549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64bc108aa0 0x7f64bc19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.704+0000 7f64c150e640 1 -- 172.21.15.2:0/242697549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc19fbb0 con 0x7f64bc102b60 2024-05-14T09:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.704+0000 7f64c0d0d640 1 --2- 172.21.15.2:0/242697549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64bc103da0 0x7f64bc1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.705+0000 7f64c150e640 1 --2- 172.21.15.2:0/242697549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64bc102b60 0x7f64bc198070 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f64b8032e90 tx=0x7f64b8005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.705+0000 7f64b27fc640 1 -- 172.21.15.2:0/242697549 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b8046030 con 0x7f64bc102b60 2024-05-14T09:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.705+0000 7f64c3772640 1 -- 172.21.15.2:0/242697549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64bc19fe40 con 0x7f64bc102b60 2024-05-14T09:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.705+0000 7f64b27fc640 1 -- 172.21.15.2:0/242697549 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64b80052b0 con 0x7f64bc102b60 2024-05-14T09:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.705+0000 7f64b27fc640 1 -- 172.21.15.2:0/242697549 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b80385e0 con 0x7f64bc102b60 2024-05-14T09:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.705+0000 7f64c3772640 1 -- 172.21.15.2:0/242697549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64bc1a02b0 con 0x7f64bc102b60 2024-05-14T09:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.708+0000 7f64b27fc640 1 -- 172.21.15.2:0/242697549 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f64b8049030 con 0x7f64bc102b60 2024-05-14T09:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.708+0000 7f64ca629740 1 Processor -- start 2024-05-14T09:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.708+0000 7f64ca629740 1 -- start start 2024-05-14T09:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.708+0000 7f64ca629740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b0831b0 0x55bb4b424a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.708+0000 7f64ca629740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b05a2f0 0x55bb4b424fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.708+0000 7f64ca629740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b0624d0 0x55bb4b42bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.708+0000 7f64ca629740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b3895d0 con 0x55bb4b0624d0 2024-05-14T09:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.708+0000 7f64ca629740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b342890 con 0x55bb4b05a2f0 2024-05-14T09:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.708+0000 7f64ca629740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b385590 con 0x55bb4b0831b0 2024-05-14T09:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.709+0000 7f64c0d0d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b0624d0 0x55bb4b42bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.709+0000 7f64c150e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b05a2f0 0x55bb4b424fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.709+0000 7f64c1d0f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b0831b0 0x55bb4b424a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.709+0000 7f64c0d0d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b0624d0 0x55bb4b42bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56372/0 (socket says 172.21.15.2:56372) 2024-05-14T09:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.709+0000 7f64c0d0d640 1 -- 172.21.15.2:0/1887417051 learned_addr learned my addr 172.21.15.2:0/1887417051 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.709+0000 7f64c0d0d640 1 -- 172.21.15.2:0/1887417051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b0831b0 msgr2=0x55bb4b424a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.709+0000 7f64c0d0d640 1 --2- 172.21.15.2:0/1887417051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b0831b0 0x55bb4b424a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.709+0000 7f64b27fc640 1 --2- 172.21.15.2:0/242697549 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64a80b28a0 0x7f64a80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.709+0000 7f64c0d0d640 1 -- 172.21.15.2:0/1887417051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b05a2f0 msgr2=0x55bb4b424fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.709+0000 7f64c0d0d640 1 --2- 172.21.15.2:0/1887417051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b05a2f0 0x55bb4b424fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.709+0000 7f64c0d0d640 1 -- 172.21.15.2:0/1887417051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4b09b210 con 0x55bb4b0624d0 2024-05-14T09:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.709+0000 7f64c1d0f640 1 --2- 172.21.15.2:0/242697549 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64a80b28a0 0x7f64a80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.709+0000 7f64b27fc640 1 -- 172.21.15.2:0/242697549 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f64b810a8e0 con 0x7f64bc102b60 2024-05-14T09:51:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.709+0000 7f64c0d0d640 1 --2- 172.21.15.2:0/1887417051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b0624d0 0x55bb4b42bee0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0115f0 tx=0x7f64ac011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09649b6d32d6ac2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.710+0000 7f64a77fe640 1 -- 172.21.15.2:0/1887417051 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac01e070 con 0x55bb4b0624d0 2024-05-14T09:51:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.710+0000 7f64a77fe640 1 -- 172.21.15.2:0/1887417051 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64ac00f660 con 0x55bb4b0624d0 2024-05-14T09:51:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.710+0000 7f64a77fe640 1 -- 172.21.15.2:0/1887417051 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac003850 con 0x55bb4b0624d0 2024-05-14T09:51:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.710+0000 7f64c1d0f640 1 --2- 172.21.15.2:0/242697549 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64a80b28a0 0x7f64a80b4db0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55bb4b425c30 tx=0x7f64b4009200 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.711+0000 7f64ca629740 1 -- 172.21.15.2:0/1887417051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b0624d0 msgr2=0x55bb4b42bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.711+0000 7f64ca629740 1 --2- 172.21.15.2:0/1887417051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b0624d0 0x55bb4b42bee0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0115f0 tx=0x7f64ac011ab0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.711+0000 7f64ca629740 1 -- 172.21.15.2:0/1887417051 shutdown_connections 2024-05-14T09:51:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.711+0000 7f64ca629740 1 --2- 172.21.15.2:0/1887417051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b0624d0 0x55bb4b42bee0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.711+0000 7f64ca629740 1 --2- 172.21.15.2:0/1887417051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b05a2f0 0x55bb4b424fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.711+0000 7f64ca629740 1 --2- 172.21.15.2:0/1887417051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b0831b0 0x55bb4b424a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.711+0000 7f64ca629740 1 -- 172.21.15.2:0/1887417051 >> 172.21.15.2:0/1887417051 conn(0x55bb4b077300 msgr2=0x55bb4b4203a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.711+0000 7f64ca629740 1 -- 172.21.15.2:0/1887417051 shutdown_connections 2024-05-14T09:51:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.711+0000 7f64ca629740 1 -- 172.21.15.2:0/1887417051 wait complete. 2024-05-14T09:51:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.711+0000 7f64ca629740 1 -- start start 2024-05-14T09:51:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.711+0000 7f64ca629740 1 Processor -- start 2024-05-14T09:51:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.712+0000 7f64ca629740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 0x55bb4b42e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.712+0000 7f64ca629740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b0624d0 0x55bb4b42ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.712+0000 7f64ca629740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b0831b0 0x55bb4b435b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.712+0000 7f64c150e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 0x55bb4b42e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64c150e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 0x55bb4b42e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:55892/0 (socket says 172.21.15.2:55892) 2024-05-14T09:51:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64ca629740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b42c930 con 0x55bb4b0831b0 2024-05-14T09:51:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64c150e640 1 -- 172.21.15.2:0/1295307651 learned_addr learned my addr 172.21.15.2:0/1295307651 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64ca629740 1 -- 172.21.15.2:0/1295307651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b346370 con 0x55bb4b0624d0 2024-05-14T09:51:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64ca629740 1 -- 172.21.15.2:0/1295307651 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b42cab0 con 0x55bb4b05a2f0 2024-05-14T09:51:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64ca629740 10 client.0 _ll_register_callbacks cb 0x55bb4b09aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:51:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64c0d0d640 1 --2- 172.21.15.2:0/1295307651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b0624d0 0x55bb4b42ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64c1d0f640 1 --2- 172.21.15.2:0/1295307651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b0831b0 0x55bb4b435b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64c1d0f640 1 -- 172.21.15.2:0/1295307651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 msgr2=0x55bb4b42e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64c1d0f640 1 --2- 172.21.15.2:0/1295307651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 0x55bb4b42e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64c1d0f640 1 -- 172.21.15.2:0/1295307651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b0624d0 msgr2=0x55bb4b42ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64c1d0f640 1 --2- 172.21.15.2:0/1295307651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b0624d0 0x55bb4b42ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64c1d0f640 1 -- 172.21.15.2:0/1295307651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4b167620 con 0x55bb4b0831b0 2024-05-14T09:51:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64c0d0d640 1 --2- 172.21.15.2:0/1295307651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b0624d0 0x55bb4b42ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:51:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64c1d0f640 1 --2- 172.21.15.2:0/1295307651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b0831b0 0x55bb4b435b10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009270 tx=0x7f64b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a688f1f637698b1f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.713+0000 7f64a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:51:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.714+0000 7f64a4ff9640 1 -- 172.21.15.2:0/1295307651 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b402b060 con 0x55bb4b0831b0 2024-05-14T09:51:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.714+0000 7f64a4ff9640 1 -- 172.21.15.2:0/1295307651 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f64b400eb00 con 0x55bb4b0831b0 2024-05-14T09:51:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.714+0000 7f64ca629740 1 -- 172.21.15.2:0/1295307651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64ac00f660 con 0x55bb4b0831b0 2024-05-14T09:51:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.714+0000 7f64a4ff9640 1 -- 172.21.15.2:0/1295307651 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f64b4032050 con 0x55bb4b0831b0 2024-05-14T09:51:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.714+0000 7f64ca629740 10 client.17097 fetch_fsmap learned FSMap version 61 2024-05-14T09:51:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.714+0000 7f64ca629740 1 -- 172.21.15.2:0/1295307651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55bb4b438b10 con 0x55bb4b0831b0 2024-05-14T09:51:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.715+0000 7f64a4ff9640 1 -- 172.21.15.2:0/1295307651 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64ac00f660 con 0x55bb4b0831b0 2024-05-14T09:51:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.715+0000 7f64a4ff9640 1 -- 172.21.15.2:0/1295307651 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7f64b400f3c0 con 0x55bb4b0831b0 2024-05-14T09:51:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.715+0000 7f64ca629740 10 client.17097 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:51:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.715+0000 7f64ca629740 10 client.17097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:51:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.715+0000 7f64ca629740 20 client.17097 populate_metadata read hostname 'smithi002' 2024-05-14T09:51:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.715+0000 7f64ca629740 1 --2- 172.21.15.2:0/1295307651 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb4b041d80 0x55bb4b43c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.715+0000 7f64ca629740 4 client.17097 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:51:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.715+0000 7f64ca629740 1 -- 172.21.15.2:0/1295307651 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb4b3971e0 con 0x55bb4b041d80 2024-05-14T09:51:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.715+0000 7f64c150e640 1 --2- 172.21.15.2:0/1295307651 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb4b041d80 0x55bb4b43c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.716+0000 7f64a4ff9640 10 client.17097 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.716+0000 7f64c150e640 1 --2- 172.21.15.2:0/1295307651 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb4b041d80 0x55bb4b43c230 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f94a06cf70e5f853 server_cookie=7b5bd7dba803c060 in_seq=0 out_seq=0 2024-05-14T09:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.718+0000 7f64a4ff9640 1 -- 172.21.15.2:0/1295307651 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bb4b3971e0 con 0x55bb4b041d80 2024-05-14T09:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.718+0000 7f64a4ff9640 10 client.17097 handle_command_reply: tid=0 2024-05-14T09:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.719+0000 7f64ca629740 1 client.17097 shutdown 2024-05-14T09:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.719+0000 7f64ca629740 1 -- 172.21.15.2:0/1295307651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b0831b0 msgr2=0x55bb4b435b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.719+0000 7f64ca629740 1 --2- 172.21.15.2:0/1295307651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b0831b0 0x55bb4b435b10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009270 tx=0x7f64b4004a40 comp rx=0 tx=0).stop 2024-05-14T09:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.719+0000 7f64ca629740 1 -- 172.21.15.2:0/1295307651 shutdown_connections 2024-05-14T09:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.719+0000 7f64ca629740 1 --2- 172.21.15.2:0/1295307651 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb4b041d80 0x55bb4b43c230 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.719+0000 7f64a4ff9640 0 client.17097 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.719+0000 7f64ca629740 1 --2- 172.21.15.2:0/1295307651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b0831b0 0x55bb4b435b10 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.719+0000 7f64ca629740 1 --2- 172.21.15.2:0/1295307651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b0624d0 0x55bb4b42ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.719+0000 7f64ca629740 1 --2- 172.21.15.2:0/1295307651 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 0x55bb4b42e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.719+0000 7f64ca629740 1 -- 172.21.15.2:0/1295307651 >> 172.21.15.2:0/1295307651 conn(0x55bb4b077300 msgr2=0x55bb4b41f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.719+0000 7f64ca629740 1 -- 172.21.15.2:0/1295307651 shutdown_connections 2024-05-14T09:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.719+0000 7f64ca629740 1 -- 172.21.15.2:0/1295307651 wait complete. 2024-05-14T09:51:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.719+0000 7f64ca629740 20 client.17097 trim_cache size 0 max 16384 2024-05-14T09:51:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64ca629740 1 Processor -- start 2024-05-14T09:51:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64ca629740 1 -- start start 2024-05-14T09:51:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64ca629740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b077300 0x55bb4b420400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64ca629740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b041d80 0x55bb4b435a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64ca629740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 0x55bb4b435f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64ca629740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1ad390 con 0x55bb4b041d80 2024-05-14T09:51:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64ca629740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1aac60 con 0x55bb4b077300 2024-05-14T09:51:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64ca629740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1a8ab0 con 0x55bb4b05a2f0 2024-05-14T09:51:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64c0d0d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b041d80 0x55bb4b435a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64c0d0d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b041d80 0x55bb4b435a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56390/0 (socket says 172.21.15.2:56390) 2024-05-14T09:51:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64c0d0d640 1 -- 172.21.15.2:0/3144637648 learned_addr learned my addr 172.21.15.2:0/3144637648 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64c150e640 1 --2- 172.21.15.2:0/3144637648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b077300 0x55bb4b420400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64c1d0f640 1 --2- 172.21.15.2:0/3144637648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 0x55bb4b435f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64c0d0d640 1 -- 172.21.15.2:0/3144637648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 msgr2=0x55bb4b435f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64c0d0d640 1 --2- 172.21.15.2:0/3144637648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 0x55bb4b435f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64c0d0d640 1 -- 172.21.15.2:0/3144637648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b077300 msgr2=0x55bb4b420400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64c0d0d640 1 --2- 172.21.15.2:0/3144637648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b077300 0x55bb4b420400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.739+0000 7f64c0d0d640 1 -- 172.21.15.2:0/3144637648 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4b42c2a0 con 0x55bb4b041d80 2024-05-14T09:51:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.740+0000 7f64c0d0d640 1 --2- 172.21.15.2:0/3144637648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b041d80 0x55bb4b435a30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x55bb4b421180 tx=0x7f64ac009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4b4d95d47c075d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.740+0000 7f64a6ffd640 1 -- 172.21.15.2:0/3144637648 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac01e070 con 0x55bb4b041d80 2024-05-14T09:51:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.740+0000 7f64a6ffd640 1 -- 172.21.15.2:0/3144637648 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64ac01f040 con 0x55bb4b041d80 2024-05-14T09:51:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.740+0000 7f64a6ffd640 1 -- 172.21.15.2:0/3144637648 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac003970 con 0x55bb4b041d80 2024-05-14T09:51:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.741+0000 7f64ca629740 1 -- 172.21.15.2:0/3144637648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b041d80 msgr2=0x55bb4b435a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.741+0000 7f64ca629740 1 --2- 172.21.15.2:0/3144637648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b041d80 0x55bb4b435a30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x55bb4b421180 tx=0x7f64ac009dc0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.741+0000 7f64ca629740 1 -- 172.21.15.2:0/3144637648 shutdown_connections 2024-05-14T09:51:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.741+0000 7f64ca629740 1 --2- 172.21.15.2:0/3144637648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 0x55bb4b435f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.741+0000 7f64ca629740 1 --2- 172.21.15.2:0/3144637648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b041d80 0x55bb4b435a30 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.741+0000 7f64ca629740 1 --2- 172.21.15.2:0/3144637648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b077300 0x55bb4b420400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.741+0000 7f64ca629740 1 -- 172.21.15.2:0/3144637648 >> 172.21.15.2:0/3144637648 conn(0x55bb4b4283f0 msgr2=0x55bb4b41fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.741+0000 7f64ca629740 1 -- 172.21.15.2:0/3144637648 shutdown_connections 2024-05-14T09:51:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.741+0000 7f64ca629740 1 -- 172.21.15.2:0/3144637648 wait complete. 2024-05-14T09:51:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.741+0000 7f64ca629740 1 -- start start 2024-05-14T09:51:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.741+0000 7f64ca629740 1 Processor -- start 2024-05-14T09:51:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.742+0000 7f64ca629740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b041d80 0x55bb4b433c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.742+0000 7f64ca629740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 0x55bb4b434170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.742+0000 7f64ca629740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b077300 0x55bb4b4350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.742+0000 7f64ca629740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1a9120 con 0x55bb4b041d80 2024-05-14T09:51:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.742+0000 7f64ca629740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1aa590 con 0x55bb4b077300 2024-05-14T09:51:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.742+0000 7f64ca629740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1a97f0 con 0x55bb4b05a2f0 2024-05-14T09:51:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.742+0000 7f64c150e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b041d80 0x55bb4b433c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.742+0000 7f64ca629740 10 client.0 _ll_register_callbacks cb 0x7f64b4009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:51:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.742+0000 7f64c150e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b041d80 0x55bb4b433c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56404/0 (socket says 172.21.15.2:56404) 2024-05-14T09:51:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.742+0000 7f64c150e640 1 -- 172.21.15.2:0/4014917514 learned_addr learned my addr 172.21.15.2:0/4014917514 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:51:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.742+0000 7f64c0d0d640 1 --2- 172.21.15.2:0/4014917514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 0x55bb4b434170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.742+0000 7f64c1d0f640 1 --2- 172.21.15.2:0/4014917514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b077300 0x55bb4b4350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.742+0000 7f64c150e640 1 -- 172.21.15.2:0/4014917514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 msgr2=0x55bb4b434170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.742+0000 7f64c150e640 1 --2- 172.21.15.2:0/4014917514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 0x55bb4b434170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.743+0000 7f64c150e640 1 -- 172.21.15.2:0/4014917514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b077300 msgr2=0x55bb4b4350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:51:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.743+0000 7f64c150e640 1 --2- 172.21.15.2:0/4014917514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b077300 0x55bb4b4350a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.743+0000 7f64c150e640 1 -- 172.21.15.2:0/4014917514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4b420940 con 0x55bb4b041d80 2024-05-14T09:51:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.743+0000 7f64c150e640 1 --2- 172.21.15.2:0/4014917514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b041d80 0x55bb4b433c30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f64b800d690 tx=0x7f64b800d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9273febbb0e479d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:51:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.743+0000 7f64a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:51:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.743+0000 7f64a4ff9640 1 -- 172.21.15.2:0/4014917514 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b8050040 con 0x55bb4b041d80 2024-05-14T09:51:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.743+0000 7f64ca629740 1 -- 172.21.15.2:0/4014917514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64ac01f040 con 0x55bb4b041d80 2024-05-14T09:51:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.743+0000 7f64a4ff9640 1 -- 172.21.15.2:0/4014917514 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f64b80cdd70 con 0x55bb4b041d80 2024-05-14T09:51:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.743+0000 7f64a4ff9640 1 -- 172.21.15.2:0/4014917514 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f64b80da280 con 0x55bb4b041d80 2024-05-14T09:51:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.743+0000 7f64ca629740 10 client.17103 fetch_fsmap learned FSMap version 61 2024-05-14T09:51:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.743+0000 7f64ca629740 1 -- 172.21.15.2:0/4014917514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55bb4b4a45e0 con 0x55bb4b041d80 2024-05-14T09:51:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.743+0000 7f64a4ff9640 1 -- 172.21.15.2:0/4014917514 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64ac01f040 con 0x55bb4b041d80 2024-05-14T09:51:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.744+0000 7f64a4ff9640 1 -- 172.21.15.2:0/4014917514 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7f64b800bad0 con 0x55bb4b041d80 2024-05-14T09:51:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.744+0000 7f64ca629740 10 client.17103 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:51:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.744+0000 7f64ca629740 10 client.17103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:51:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.744+0000 7f64ca629740 20 client.17103 populate_metadata read hostname 'smithi002' 2024-05-14T09:51:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.744+0000 7f64ca629740 1 --2- 172.21.15.2:0/4014917514 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb4b0624d0 0x55bb4b4a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:51:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.744+0000 7f64ca629740 4 client.17103 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:51:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.744+0000 7f64ca629740 1 -- 172.21.15.2:0/4014917514 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb4b391050 con 0x55bb4b0624d0 2024-05-14T09:51:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.744+0000 7f64c0d0d640 1 --2- 172.21.15.2:0/4014917514 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb4b0624d0 0x55bb4b4a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:51:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.744+0000 7f64c0d0d640 1 --2- 172.21.15.2:0/4014917514 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb4b0624d0 0x55bb4b4a6e70 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de763152496aee07 server_cookie=a326df600d8d664b in_seq=0 out_seq=0 2024-05-14T09:51:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:55.744+0000 7f64a4ff9640 10 client.17103 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.531+0000 7f64a4ff9640 1 -- 172.21.15.2:0/4014917514 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb4b391050 con 0x55bb4b0624d0 2024-05-14T09:51:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.531+0000 7f64a4ff9640 10 client.17103 handle_command_reply: tid=0 2024-05-14T09:51:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.531+0000 7f64ca629740 1 client.17103 shutdown 2024-05-14T09:51:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.531+0000 7f64ca629740 1 -- 172.21.15.2:0/4014917514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b041d80 msgr2=0x55bb4b433c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.531+0000 7f64ca629740 1 --2- 172.21.15.2:0/4014917514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b041d80 0x55bb4b433c30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f64b800d690 tx=0x7f64b800d040 comp rx=0 tx=0).stop 2024-05-14T09:51:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.531+0000 7f64ca629740 1 -- 172.21.15.2:0/4014917514 shutdown_connections 2024-05-14T09:51:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.531+0000 7f64ca629740 1 --2- 172.21.15.2:0/4014917514 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb4b0624d0 0x55bb4b4a6e70 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.531+0000 7f64ca629740 1 --2- 172.21.15.2:0/4014917514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb4b077300 0x55bb4b4350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.531+0000 7f64ca629740 1 --2- 172.21.15.2:0/4014917514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb4b05a2f0 0x55bb4b434170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.531+0000 7f64a4ff9640 0 client.17103 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:51:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.531+0000 7f64ca629740 1 --2- 172.21.15.2:0/4014917514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb4b041d80 0x55bb4b433c30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.531+0000 7f64ca629740 1 -- 172.21.15.2:0/4014917514 >> 172.21.15.2:0/4014917514 conn(0x55bb4b4283f0 msgr2=0x55bb4b41e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.532+0000 7f64ca629740 1 -- 172.21.15.2:0/4014917514 shutdown_connections 2024-05-14T09:51:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.532+0000 7f64ca629740 1 -- 172.21.15.2:0/4014917514 wait complete. 2024-05-14T09:51:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.532+0000 7f64ca629740 20 client.17103 trim_cache size 0 max 16384 2024-05-14T09:51:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:51:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:51:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:51:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:51:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.532+0000 7f64c3772640 1 -- 172.21.15.2:0/242697549 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64a80b28a0 msgr2=0x7f64a80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.532+0000 7f64c3772640 1 --2- 172.21.15.2:0/242697549 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64a80b28a0 0x7f64a80b4db0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55bb4b425c30 tx=0x7f64b4009200 comp rx=0 tx=0).stop 2024-05-14T09:51:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.532+0000 7f64c3772640 1 -- 172.21.15.2:0/242697549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64bc102b60 msgr2=0x7f64bc198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:51:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.532+0000 7f64c3772640 1 --2- 172.21.15.2:0/242697549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64bc102b60 0x7f64bc198070 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f64b8032e90 tx=0x7f64b8005e40 comp rx=0 tx=0).stop 2024-05-14T09:51:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.532+0000 7f64c3772640 1 -- 172.21.15.2:0/242697549 shutdown_connections 2024-05-14T09:51:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.532+0000 7f64c3772640 1 --2- 172.21.15.2:0/242697549 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f64a80b28a0 0x7f64a80b4db0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.532+0000 7f64c3772640 1 --2- 172.21.15.2:0/242697549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64bc108aa0 0x7f64bc19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.532+0000 7f64c3772640 1 --2- 172.21.15.2:0/242697549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f64bc103da0 0x7f64bc1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.532+0000 7f64c3772640 1 --2- 172.21.15.2:0/242697549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64bc102b60 0x7f64bc198070 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:51:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.532+0000 7f64c3772640 1 -- 172.21.15.2:0/242697549 >> 172.21.15.2:0/242697549 conn(0x7f64bc0fdd50 msgr2=0x7f64bc104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:51:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.533+0000 7f64c3772640 1 -- 172.21.15.2:0/242697549 shutdown_connections 2024-05-14T09:51:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:51:56.533+0000 7f64c3772640 1 -- 172.21.15.2:0/242697549 wait complete. 2024-05-14T09:51:56.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 110809 29.13 MB/sec execute 82 sec latency 713.391 ms 2024-05-14T09:51:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:56 smithi026 ceph-mon[36056]: pgmap v934: 129 pgs: 129 active+clean; 583 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.7 MiB/s wr, 500 op/s 2024-05-14T09:51:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:56 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:57.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:56 smithi002 ceph-mon[31557]: pgmap v934: 129 pgs: 129 active+clean; 583 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.7 MiB/s wr, 500 op/s 2024-05-14T09:51:57.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:56 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:57.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:56 smithi029 ceph-mon[36249]: pgmap v934: 129 pgs: 129 active+clean; 583 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.7 MiB/s wr, 500 op/s 2024-05-14T09:51:57.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:56 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:51:57.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 111600 29.24 MB/sec execute 83 sec latency 49.816 ms 2024-05-14T09:51:58.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 112214 29.45 MB/sec execute 84 sec latency 170.569 ms 2024-05-14T09:51:58.972 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:51:58 smithi002 ceph-mon[31557]: pgmap v935: 129 pgs: 129 active+clean; 591 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 530 op/s 2024-05-14T09:51:59.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:51:58 smithi029 ceph-mon[36249]: pgmap v935: 129 pgs: 129 active+clean; 591 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 530 op/s 2024-05-14T09:51:59.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:51:58 smithi026 ceph-mon[36056]: pgmap v935: 129 pgs: 129 active+clean; 591 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 530 op/s 2024-05-14T09:51:59.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 112233 29.10 MB/sec execute 85 sec latency 1168.962 ms 2024-05-14T09:52:00.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 112303 28.80 MB/sec execute 86 sec latency 1641.177 ms 2024-05-14T09:52:01.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:00 smithi002 ceph-mon[31557]: pgmap v936: 129 pgs: 129 active+clean; 598 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 632 op/s 2024-05-14T09:52:01.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:00 smithi029 ceph-mon[36249]: pgmap v936: 129 pgs: 129 active+clean; 598 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 632 op/s 2024-05-14T09:52:01.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:00 smithi026 ceph-mon[36056]: pgmap v936: 129 pgs: 129 active+clean; 598 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 632 op/s 2024-05-14T09:52:01.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 112562 28.56 MB/sec execute 87 sec latency 455.176 ms 2024-05-14T09:52:02.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 112826 28.37 MB/sec execute 88 sec latency 107.557 ms 2024-05-14T09:52:03.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:02 smithi002 ceph-mon[31557]: pgmap v937: 129 pgs: 129 active+clean; 619 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.2 MiB/s wr, 570 op/s 2024-05-14T09:52:03.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:02 smithi029 ceph-mon[36249]: pgmap v937: 129 pgs: 129 active+clean; 619 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.2 MiB/s wr, 570 op/s 2024-05-14T09:52:03.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:02 smithi026 ceph-mon[36056]: pgmap v937: 129 pgs: 129 active+clean; 619 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.2 MiB/s wr, 570 op/s 2024-05-14T09:52:03.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 113251 28.28 MB/sec execute 89 sec latency 80.052 ms 2024-05-14T09:52:04.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 113866 28.38 MB/sec execute 90 sec latency 90.313 ms 2024-05-14T09:52:05.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:04 smithi002 ceph-mon[31557]: pgmap v938: 129 pgs: 129 active+clean; 635 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 11 MiB/s wr, 730 op/s 2024-05-14T09:52:05.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:04 smithi029 ceph-mon[36249]: pgmap v938: 129 pgs: 129 active+clean; 635 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 11 MiB/s wr, 730 op/s 2024-05-14T09:52:05.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:04 smithi026 ceph-mon[36056]: pgmap v938: 129 pgs: 129 active+clean; 635 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 11 MiB/s wr, 730 op/s 2024-05-14T09:52:05.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 114728 28.48 MB/sec execute 91 sec latency 92.772 ms 2024-05-14T09:52:06.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 115288 28.61 MB/sec execute 92 sec latency 172.444 ms 2024-05-14T09:52:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:06 smithi002 ceph-mon[31557]: pgmap v939: 129 pgs: 129 active+clean; 610 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 833 op/s 2024-05-14T09:52:07.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:06 smithi029 ceph-mon[36249]: pgmap v939: 129 pgs: 129 active+clean; 610 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 833 op/s 2024-05-14T09:52:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:06 smithi026 ceph-mon[36056]: pgmap v939: 129 pgs: 129 active+clean; 610 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 833 op/s 2024-05-14T09:52:07.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 115946 28.73 MB/sec execute 93 sec latency 49.885 ms 2024-05-14T09:52:08.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 116544 28.69 MB/sec execute 94 sec latency 279.111 ms 2024-05-14T09:52:08.988 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:08 smithi029 ceph-mon[36249]: pgmap v940: 129 pgs: 129 active+clean; 591 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 849 op/s 2024-05-14T09:52:09.002 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:08 smithi026 ceph-mon[36056]: pgmap v940: 129 pgs: 129 active+clean; 591 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 849 op/s 2024-05-14T09:52:09.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:08 smithi002 ceph-mon[31557]: pgmap v940: 129 pgs: 129 active+clean; 591 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 849 op/s 2024-05-14T09:52:09.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 117172 28.74 MB/sec execute 95 sec latency 74.263 ms 2024-05-14T09:52:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.691+0000 7fc8a566e640 1 -- 172.21.15.2:0/3083122099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8a0110220 msgr2=0x7fc8a0112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.691+0000 7fc8a566e640 1 --2- 172.21.15.2:0/3083122099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8a0110220 0x7fc8a0112640 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc89801acb0 tx=0x7fc89803b920 comp rx=0 tx=0).stop 2024-05-14T09:52:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.691+0000 7fc8a566e640 1 -- 172.21.15.2:0/3083122099 shutdown_connections 2024-05-14T09:52:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.691+0000 7fc8a566e640 1 --2- 172.21.15.2:0/3083122099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8a0110220 0x7fc8a0112640 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.691+0000 7fc8a566e640 1 --2- 172.21.15.2:0/3083122099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8a010b520 0x7fc8a010b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.691+0000 7fc8a566e640 1 --2- 172.21.15.2:0/3083122099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a010a450 0x7fc8a010afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.691+0000 7fc8a566e640 1 -- 172.21.15.2:0/3083122099 >> 172.21.15.2:0/3083122099 conn(0x7fc8a0078f00 msgr2=0x7fc8a007b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.691+0000 7fc8a566e640 1 -- 172.21.15.2:0/3083122099 shutdown_connections 2024-05-14T09:52:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.691+0000 7fc8a566e640 1 -- 172.21.15.2:0/3083122099 wait complete. 2024-05-14T09:52:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.692+0000 7fc8a566e640 1 Processor -- start 2024-05-14T09:52:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.692+0000 7fc8a566e640 1 -- start start 2024-05-14T09:52:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.692+0000 7fc8a566e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8a010a450 0x7fc8a0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.693+0000 7fc8a566e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8a010b520 0x7fc8a0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.693+0000 7fc8a566e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a0110220 0x7fc8a019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.693+0000 7fc8a566e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a007f0e0 con 0x7fc8a0110220 2024-05-14T09:52:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.693+0000 7fc89f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a0110220 0x7fc8a019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.693+0000 7fc89f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a0110220 0x7fc8a019f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41654/0 (socket says 172.21.15.2:41654) 2024-05-14T09:52:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.693+0000 7fc89f7fe640 1 -- 172.21.15.2:0/2011504033 learned_addr learned my addr 172.21.15.2:0/2011504033 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.693+0000 7fc8a566e640 1 -- 172.21.15.2:0/2011504033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a007ef60 con 0x7fc8a010a450 2024-05-14T09:52:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.693+0000 7fc8a566e640 1 -- 172.21.15.2:0/2011504033 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a007f260 con 0x7fc8a010b520 2024-05-14T09:52:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.694+0000 7fc89effd640 1 --2- 172.21.15.2:0/2011504033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8a010a450 0x7fc8a0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.694+0000 7fc89e7fc640 1 --2- 172.21.15.2:0/2011504033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8a010b520 0x7fc8a0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.694+0000 7fc89f7fe640 1 -- 172.21.15.2:0/2011504033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8a010b520 msgr2=0x7fc8a0198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.694+0000 7fc89f7fe640 1 --2- 172.21.15.2:0/2011504033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8a010b520 0x7fc8a0198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.694+0000 7fc89f7fe640 1 -- 172.21.15.2:0/2011504033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8a010a450 msgr2=0x7fc8a0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.694+0000 7fc89f7fe640 1 --2- 172.21.15.2:0/2011504033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8a010a450 0x7fc8a0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.694+0000 7fc89f7fe640 1 -- 172.21.15.2:0/2011504033 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a019fc90 con 0x7fc8a0110220 2024-05-14T09:52:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.695+0000 7fc89f7fe640 1 --2- 172.21.15.2:0/2011504033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a0110220 0x7fc8a019f620 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc89801add0 tx=0x7fc898042fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.695+0000 7fc887fff640 1 -- 172.21.15.2:0/2011504033 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc898045070 con 0x7fc8a0110220 2024-05-14T09:52:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.695+0000 7fc887fff640 1 -- 172.21.15.2:0/2011504033 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc898003e20 con 0x7fc8a0110220 2024-05-14T09:52:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.695+0000 7fc887fff640 1 -- 172.21.15.2:0/2011504033 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89804a430 con 0x7fc8a0110220 2024-05-14T09:52:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.696+0000 7fc8a566e640 1 -- 172.21.15.2:0/2011504033 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a019ff20 con 0x7fc8a0110220 2024-05-14T09:52:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.696+0000 7fc8a566e640 1 -- 172.21.15.2:0/2011504033 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a01a03b0 con 0x7fc8a0110220 2024-05-14T09:52:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.705+0000 7fc887fff640 1 -- 172.21.15.2:0/2011504033 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc898048020 con 0x7fc8a0110220 2024-05-14T09:52:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.705+0000 7fc8ac601740 1 Processor -- start 2024-05-14T09:52:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.705+0000 7fc8ac601740 1 -- start start 2024-05-14T09:52:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.706+0000 7fc8ac601740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6fa1b0 0x55f4eda9ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.706+0000 7fc8ac601740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 0x55f4eda9bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.706+0000 7fc8ac601740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6d94d0 0x55f4edaa2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.706+0000 7fc8ac601740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f4eda005d0 con 0x55f4ed6d94d0 2024-05-14T09:52:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.706+0000 7fc8ac601740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f4ed9b9890 con 0x55f4ed6d12f0 2024-05-14T09:52:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.706+0000 7fc8ac601740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f4ed9fc590 con 0x55f4ed6fa1b0 2024-05-14T09:52:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.706+0000 7fc89e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6fa1b0 0x55f4eda9ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.706+0000 7fc89e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6fa1b0 0x55f4eda9ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:48780/0 (socket says 172.21.15.2:48780) 2024-05-14T09:52:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.706+0000 7fc89e7fc640 1 -- 172.21.15.2:0/402448062 learned_addr learned my addr 172.21.15.2:0/402448062 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.706+0000 7fc89e7fc640 1 --2- 172.21.15.2:0/402448062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6d94d0 0x55f4edaa2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.707+0000 7fc89e7fc640 1 -- 172.21.15.2:0/402448062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 msgr2=0x55f4eda9bfe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:52:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.707+0000 7fc89e7fc640 1 --2- 172.21.15.2:0/402448062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 0x55f4eda9bfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.707+0000 7fc89e7fc640 1 -- 172.21.15.2:0/402448062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6d94d0 msgr2=0x55f4edaa2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.707+0000 7fc89e7fc640 1 --2- 172.21.15.2:0/402448062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6d94d0 0x55f4edaa2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.707+0000 7fc89e7fc640 1 -- 172.21.15.2:0/402448062 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4ed712210 con 0x55f4ed6fa1b0 2024-05-14T09:52:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.707+0000 7fc89e7fc640 1 --2- 172.21.15.2:0/402448062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6fa1b0 0x55f4eda9ba60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc894016530 tx=0x7fc89400a2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb64964d443bb8d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.707+0000 7fc8857fa640 1 -- 172.21.15.2:0/402448062 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894018430 con 0x55f4ed6fa1b0 2024-05-14T09:52:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.707+0000 7fc8857fa640 1 -- 172.21.15.2:0/402448062 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc894018a80 con 0x55f4ed6fa1b0 2024-05-14T09:52:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.707+0000 7fc8857fa640 1 -- 172.21.15.2:0/402448062 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8940178b0 con 0x55f4ed6fa1b0 2024-05-14T09:52:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.708+0000 7fc887fff640 1 --2- 172.21.15.2:0/2011504033 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc8880b2a70 0x7fc8880b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.708+0000 7fc887fff640 1 -- 172.21.15.2:0/2011504033 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fc898114700 con 0x7fc8a0110220 2024-05-14T09:52:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.708+0000 7fc89effd640 1 --2- 172.21.15.2:0/2011504033 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc8880b2a70 0x7fc8880b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.709+0000 7fc8ac601740 1 -- 172.21.15.2:0/402448062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6fa1b0 msgr2=0x55f4eda9ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.709+0000 7fc8ac601740 1 --2- 172.21.15.2:0/402448062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6fa1b0 0x55f4eda9ba60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc894016530 tx=0x7fc89400a2b0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.709+0000 7fc8ac601740 1 -- 172.21.15.2:0/402448062 shutdown_connections 2024-05-14T09:52:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.709+0000 7fc8ac601740 1 --2- 172.21.15.2:0/402448062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6d94d0 0x55f4edaa2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.709+0000 7fc8ac601740 1 --2- 172.21.15.2:0/402448062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 0x55f4eda9bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.709+0000 7fc8ac601740 1 --2- 172.21.15.2:0/402448062 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6fa1b0 0x55f4eda9ba60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.709+0000 7fc8ac601740 1 -- 172.21.15.2:0/402448062 >> 172.21.15.2:0/402448062 conn(0x55f4ed6ee300 msgr2=0x55f4eda973a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.709+0000 7fc8ac601740 1 -- 172.21.15.2:0/402448062 shutdown_connections 2024-05-14T09:52:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.709+0000 7fc89effd640 1 --2- 172.21.15.2:0/2011504033 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc8880b2a70 0x7fc8880b4f80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55f4eda9ce40 tx=0x7fc890008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.709+0000 7fc8ac601740 1 -- 172.21.15.2:0/402448062 wait complete. 2024-05-14T09:52:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.710+0000 7fc8ac601740 1 -- start start 2024-05-14T09:52:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.710+0000 7fc8ac601740 1 Processor -- start 2024-05-14T09:52:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.710+0000 7fc8ac601740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6d12f0 0x55f4edaa56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.710+0000 7fc8ac601740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 0x55f4edaa5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.710+0000 7fc8ac601740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6fa1b0 0x55f4edaacb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.710+0000 7fc8ac601740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f4edaa3870 con 0x55f4ed6d12f0 2024-05-14T09:52:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.710+0000 7fc8ac601740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f4ed9bd370 con 0x55f4ed6fa1b0 2024-05-14T09:52:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.710+0000 7fc8ac601740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f4edaa39f0 con 0x55f4ed6d94d0 2024-05-14T09:52:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.710+0000 7fc8ac601740 10 client.0 _ll_register_callbacks cb 0x55f4ed711a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:52:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.710+0000 7fc89e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 0x55f4edaa5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.710+0000 7fc89e7fc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 0x55f4edaa5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:48784/0 (socket says 172.21.15.2:48784) 2024-05-14T09:52:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.710+0000 7fc89e7fc640 1 -- 172.21.15.2:0/3766318698 learned_addr learned my addr 172.21.15.2:0/3766318698 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.711+0000 7fc89effd640 1 --2- 172.21.15.2:0/3766318698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6d12f0 0x55f4edaa56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.711+0000 7fc89f7fe640 1 --2- 172.21.15.2:0/3766318698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6fa1b0 0x55f4edaacb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.711+0000 7fc89e7fc640 1 -- 172.21.15.2:0/3766318698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6fa1b0 msgr2=0x55f4edaacb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.711+0000 7fc89e7fc640 1 --2- 172.21.15.2:0/3766318698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6fa1b0 0x55f4edaacb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.711+0000 7fc89e7fc640 1 -- 172.21.15.2:0/3766318698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6d12f0 msgr2=0x55f4edaa56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.711+0000 7fc89e7fc640 1 --2- 172.21.15.2:0/3766318698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6d12f0 0x55f4edaa56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.711+0000 7fc89e7fc640 1 -- 172.21.15.2:0/3766318698 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4ed7de620 con 0x55f4ed6d94d0 2024-05-14T09:52:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.711+0000 7fc89f7fe640 1 --2- 172.21.15.2:0/3766318698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6fa1b0 0x55f4edaacb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:52:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.711+0000 7fc89e7fc640 1 --2- 172.21.15.2:0/3766318698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 0x55f4edaa5bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc894018ee0 tx=0x7fc894018f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ec274bea5f43fe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.711+0000 7fc882ff5640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:52:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.712+0000 7fc882ff5640 1 -- 172.21.15.2:0/3766318698 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894027020 con 0x55f4ed6d94d0 2024-05-14T09:52:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.712+0000 7fc8ac601740 1 -- 172.21.15.2:0/3766318698 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc894018a80 con 0x55f4ed6d94d0 2024-05-14T09:52:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.712+0000 7fc882ff5640 1 -- 172.21.15.2:0/3766318698 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fc894018840 con 0x55f4ed6d94d0 2024-05-14T09:52:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.712+0000 7fc882ff5640 1 -- 172.21.15.2:0/3766318698 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fc894017630 con 0x55f4ed6d94d0 2024-05-14T09:52:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.712+0000 7fc8ac601740 10 client.26746 fetch_fsmap learned FSMap version 61 2024-05-14T09:52:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.712+0000 7fc8ac601740 1 -- 172.21.15.2:0/3766318698 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55f4edac0ae0 con 0x55f4ed6d94d0 2024-05-14T09:52:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.712+0000 7fc882ff5640 1 -- 172.21.15.2:0/3766318698 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc894018a80 con 0x55f4ed6d94d0 2024-05-14T09:52:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.713+0000 7fc882ff5640 1 -- 172.21.15.2:0/3766318698 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7fc894012050 con 0x55f4ed6d94d0 2024-05-14T09:52:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.713+0000 7fc8ac601740 10 client.26746 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:52:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.713+0000 7fc8ac601740 10 client.26746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:52:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.713+0000 7fc8ac601740 20 client.26746 populate_metadata read hostname 'smithi002' 2024-05-14T09:52:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.713+0000 7fc8ac601740 1 --2- 172.21.15.2:0/3766318698 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4ed6b8d80 0x55f4edac3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.713+0000 7fc8ac601740 4 client.26746 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:52:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.713+0000 7fc8ac601740 1 -- 172.21.15.2:0/3766318698 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4eda0e1e0 con 0x55f4ed6b8d80 2024-05-14T09:52:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.713+0000 7fc89effd640 1 --2- 172.21.15.2:0/3766318698 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4ed6b8d80 0x55f4edac3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.713+0000 7fc882ff5640 10 client.26746 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.713+0000 7fc89effd640 1 --2- 172.21.15.2:0/3766318698 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4ed6b8d80 0x55f4edac3f60 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7d3788a39d22f3a server_cookie=b556dc6a5de8fd7c in_seq=0 out_seq=0 2024-05-14T09:52:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.716+0000 7fc882ff5640 1 -- 172.21.15.2:0/3766318698 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4eda0e1e0 con 0x55f4ed6b8d80 2024-05-14T09:52:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.716+0000 7fc882ff5640 10 client.26746 handle_command_reply: tid=0 2024-05-14T09:52:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.716+0000 7fc8ac601740 1 client.26746 shutdown 2024-05-14T09:52:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.716+0000 7fc8ac601740 1 -- 172.21.15.2:0/3766318698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 msgr2=0x55f4edaa5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.716+0000 7fc8ac601740 1 --2- 172.21.15.2:0/3766318698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 0x55f4edaa5bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc894018ee0 tx=0x7fc894018f10 comp rx=0 tx=0).stop 2024-05-14T09:52:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.716+0000 7fc8ac601740 1 -- 172.21.15.2:0/3766318698 shutdown_connections 2024-05-14T09:52:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.716+0000 7fc8ac601740 1 --2- 172.21.15.2:0/3766318698 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4ed6b8d80 0x55f4edac3f60 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.716+0000 7fc8ac601740 1 --2- 172.21.15.2:0/3766318698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6fa1b0 0x55f4edaacb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.716+0000 7fc8ac601740 1 --2- 172.21.15.2:0/3766318698 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 0x55f4edaa5bf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.716+0000 7fc882ff5640 0 client.26746 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.716+0000 7fc8ac601740 1 --2- 172.21.15.2:0/3766318698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6d12f0 0x55f4edaa56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.716+0000 7fc8ac601740 1 -- 172.21.15.2:0/3766318698 >> 172.21.15.2:0/3766318698 conn(0x55f4ed6ee300 msgr2=0x55f4eda9a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.716+0000 7fc8ac601740 1 -- 172.21.15.2:0/3766318698 shutdown_connections 2024-05-14T09:52:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.717+0000 7fc8ac601740 1 -- 172.21.15.2:0/3766318698 wait complete. 2024-05-14T09:52:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.717+0000 7fc8ac601740 20 client.26746 trim_cache size 0 max 16384 2024-05-14T09:52:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.740+0000 7fc8ac601740 1 Processor -- start 2024-05-14T09:52:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.741+0000 7fc8ac601740 1 -- start start 2024-05-14T09:52:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.741+0000 7fc8ac601740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 0x55f4eda9fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.741+0000 7fc8ac601740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 0x55f4edaa00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.741+0000 7fc8ac601740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6fa1b0 0x55f4edac9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.741+0000 7fc8ac601740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01a4b20 con 0x55f4ed6fa1b0 2024-05-14T09:52:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.741+0000 7fc8ac601740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01a23f0 con 0x55f4ed6d12f0 2024-05-14T09:52:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.741+0000 7fc8ac601740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a0074aa0 con 0x55f4ed6d94d0 2024-05-14T09:52:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.741+0000 7fc89f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6fa1b0 0x55f4edac9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.741+0000 7fc89f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6fa1b0 0x55f4edac9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41676/0 (socket says 172.21.15.2:41676) 2024-05-14T09:52:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.741+0000 7fc89f7fe640 1 -- 172.21.15.2:0/481474824 learned_addr learned my addr 172.21.15.2:0/481474824 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.742+0000 7fc89effd640 1 --2- 172.21.15.2:0/481474824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 0x55f4eda9fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.742+0000 7fc89e7fc640 1 --2- 172.21.15.2:0/481474824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 0x55f4edaa00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.742+0000 7fc89f7fe640 1 -- 172.21.15.2:0/481474824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 msgr2=0x55f4edaa00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.742+0000 7fc89f7fe640 1 --2- 172.21.15.2:0/481474824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 0x55f4edaa00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.742+0000 7fc89f7fe640 1 -- 172.21.15.2:0/481474824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 msgr2=0x55f4eda9fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.742+0000 7fc89f7fe640 1 --2- 172.21.15.2:0/481474824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 0x55f4eda9fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.742+0000 7fc89f7fe640 1 -- 172.21.15.2:0/481474824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4eda97d00 con 0x55f4ed6fa1b0 2024-05-14T09:52:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.742+0000 7fc89e7fc640 1 --2- 172.21.15.2:0/481474824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 0x55f4edaa00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:52:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.742+0000 7fc89effd640 1 --2- 172.21.15.2:0/481474824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 0x55f4eda9fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:52:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.742+0000 7fc89f7fe640 1 --2- 172.21.15.2:0/481474824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6fa1b0 0x55f4edac9690 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc898111d80 tx=0x7fc89801a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d03a84a422d2473 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.743+0000 7fc884ff9640 1 -- 172.21.15.2:0/481474824 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8980f5070 con 0x55f4ed6fa1b0 2024-05-14T09:52:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.743+0000 7fc884ff9640 1 -- 172.21.15.2:0/481474824 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8980db3b0 con 0x55f4ed6fa1b0 2024-05-14T09:52:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.743+0000 7fc884ff9640 1 -- 172.21.15.2:0/481474824 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8980e4700 con 0x55f4ed6fa1b0 2024-05-14T09:52:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.744+0000 7fc8ac601740 1 -- 172.21.15.2:0/481474824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6fa1b0 msgr2=0x55f4edac9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.744+0000 7fc8ac601740 1 --2- 172.21.15.2:0/481474824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6fa1b0 0x55f4edac9690 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc898111d80 tx=0x7fc89801a5d0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.744+0000 7fc8ac601740 1 -- 172.21.15.2:0/481474824 shutdown_connections 2024-05-14T09:52:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.744+0000 7fc8ac601740 1 --2- 172.21.15.2:0/481474824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6fa1b0 0x55f4edac9690 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.744+0000 7fc8ac601740 1 --2- 172.21.15.2:0/481474824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 0x55f4edaa00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.744+0000 7fc8ac601740 1 --2- 172.21.15.2:0/481474824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 0x55f4eda9fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.744+0000 7fc8ac601740 1 -- 172.21.15.2:0/481474824 >> 172.21.15.2:0/481474824 conn(0x55f4ed6ee300 msgr2=0x55f4edaa3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.744+0000 7fc8ac601740 1 -- 172.21.15.2:0/481474824 shutdown_connections 2024-05-14T09:52:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.744+0000 7fc8ac601740 1 -- 172.21.15.2:0/481474824 wait complete. 2024-05-14T09:52:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.745+0000 7fc8ac601740 1 -- start start 2024-05-14T09:52:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.745+0000 7fc8ac601740 1 Processor -- start 2024-05-14T09:52:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.745+0000 7fc8ac601740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 0x55f4edaab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.745+0000 7fc8ac601740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 0x55f4edaab680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.746+0000 7fc8ac601740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6fa1b0 0x55f4edb2a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.746+0000 7fc89effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 0x55f4edaab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.746+0000 7fc89effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 0x55f4edaab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:54548/0 (socket says 172.21.15.2:54548) 2024-05-14T09:52:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.746+0000 7fc8ac601740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01a08b0 con 0x55f4ed6fa1b0 2024-05-14T09:52:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.746+0000 7fc89effd640 1 -- 172.21.15.2:0/4163632377 learned_addr learned my addr 172.21.15.2:0/4163632377 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.746+0000 7fc89e7fc640 1 --2- 172.21.15.2:0/4163632377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 0x55f4edaab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.746+0000 7fc8ac601740 1 -- 172.21.15.2:0/4163632377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01a1d20 con 0x55f4ed6d12f0 2024-05-14T09:52:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.746+0000 7fc8ac601740 1 -- 172.21.15.2:0/4163632377 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01a0f80 con 0x55f4ed6d94d0 2024-05-14T09:52:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.746+0000 7fc89f7fe640 1 --2- 172.21.15.2:0/4163632377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6fa1b0 0x55f4edb2a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.746+0000 7fc8ac601740 10 client.0 _ll_register_callbacks cb 0x7fc89400ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:52:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.746+0000 7fc89effd640 1 -- 172.21.15.2:0/4163632377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 msgr2=0x55f4edaab680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.746+0000 7fc89effd640 1 --2- 172.21.15.2:0/4163632377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 0x55f4edaab680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.746+0000 7fc89effd640 1 -- 172.21.15.2:0/4163632377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6fa1b0 msgr2=0x55f4edb2a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.746+0000 7fc89effd640 1 --2- 172.21.15.2:0/4163632377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6fa1b0 0x55f4edb2a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.746+0000 7fc89effd640 1 -- 172.21.15.2:0/4163632377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4edb2a770 con 0x55f4ed6d12f0 2024-05-14T09:52:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.747+0000 7fc89effd640 1 --2- 172.21.15.2:0/4163632377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 0x55f4edaab140 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc89000bfd0 tx=0x7fc8900047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4458d91f3900c6e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.747+0000 7fc882ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:52:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.747+0000 7fc882ff5640 1 -- 172.21.15.2:0/4163632377 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890026130 con 0x55f4ed6d12f0 2024-05-14T09:52:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.747+0000 7fc8ac601740 1 -- 172.21.15.2:0/4163632377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8980db3b0 con 0x55f4ed6d12f0 2024-05-14T09:52:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.747+0000 7fc882ff5640 1 -- 172.21.15.2:0/4163632377 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fc89000f040 con 0x55f4ed6d12f0 2024-05-14T09:52:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.747+0000 7fc882ff5640 1 -- 172.21.15.2:0/4163632377 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fc8900262d0 con 0x55f4ed6d12f0 2024-05-14T09:52:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.747+0000 7fc8ac601740 10 client.26741 fetch_fsmap learned FSMap version 61 2024-05-14T09:52:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.747+0000 7fc8ac601740 1 -- 172.21.15.2:0/4163632377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55f4edb2b980 con 0x55f4ed6d12f0 2024-05-14T09:52:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.747+0000 7fc882ff5640 1 -- 172.21.15.2:0/4163632377 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8980db3b0 con 0x55f4ed6d12f0 2024-05-14T09:52:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.747+0000 7fc882ff5640 1 -- 172.21.15.2:0/4163632377 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7fc890013070 con 0x55f4ed6d12f0 2024-05-14T09:52:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.748+0000 7fc8ac601740 10 client.26741 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:52:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.748+0000 7fc8ac601740 10 client.26741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:52:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.748+0000 7fc8ac601740 20 client.26741 populate_metadata read hostname 'smithi002' 2024-05-14T09:52:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.748+0000 7fc8ac601740 1 --2- 172.21.15.2:0/4163632377 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4ed68e1c0 0x55f4edb2e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.748+0000 7fc8ac601740 4 client.26741 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:52:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.748+0000 7fc8ac601740 1 -- 172.21.15.2:0/4163632377 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4eda08050 con 0x55f4ed68e1c0 2024-05-14T09:52:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.748+0000 7fc89e7fc640 1 --2- 172.21.15.2:0/4163632377 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4ed68e1c0 0x55f4edb2e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.748+0000 7fc882ff5640 10 client.26741 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:09.748+0000 7fc89e7fc640 1 --2- 172.21.15.2:0/4163632377 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4ed68e1c0 0x55f4edb2e3a0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d958d6e849aa8824 server_cookie=debdf40696cb93fe in_seq=0 out_seq=0 2024-05-14T09:52:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.181+0000 7fc882ff5640 1 -- 172.21.15.2:0/4163632377 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f4eda08050 con 0x55f4ed68e1c0 2024-05-14T09:52:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.181+0000 7fc882ff5640 10 client.26741 handle_command_reply: tid=0 2024-05-14T09:52:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.181+0000 7fc8ac601740 1 client.26741 shutdown 2024-05-14T09:52:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.182+0000 7fc8ac601740 1 -- 172.21.15.2:0/4163632377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 msgr2=0x55f4edaab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.182+0000 7fc8ac601740 1 --2- 172.21.15.2:0/4163632377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 0x55f4edaab140 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc89000bfd0 tx=0x7fc8900047d0 comp rx=0 tx=0).stop 2024-05-14T09:52:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.182+0000 7fc8ac601740 1 -- 172.21.15.2:0/4163632377 shutdown_connections 2024-05-14T09:52:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.182+0000 7fc8ac601740 1 --2- 172.21.15.2:0/4163632377 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4ed68e1c0 0x55f4edb2e3a0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.182+0000 7fc8ac601740 1 --2- 172.21.15.2:0/4163632377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4ed6fa1b0 0x55f4edb2a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.182+0000 7fc882ff5640 0 client.26741 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.182+0000 7fc8ac601740 1 --2- 172.21.15.2:0/4163632377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4ed6d94d0 0x55f4edaab680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.182+0000 7fc8ac601740 1 --2- 172.21.15.2:0/4163632377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ed6d12f0 0x55f4edaab140 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.182+0000 7fc8ac601740 1 -- 172.21.15.2:0/4163632377 >> 172.21.15.2:0/4163632377 conn(0x55f4ed6ee300 msgr2=0x55f4edaa2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.182+0000 7fc8ac601740 1 -- 172.21.15.2:0/4163632377 shutdown_connections 2024-05-14T09:52:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.182+0000 7fc8ac601740 1 -- 172.21.15.2:0/4163632377 wait complete. 2024-05-14T09:52:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.182+0000 7fc8ac601740 20 client.26741 trim_cache size 0 max 16384 2024-05-14T09:52:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:52:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:52:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:52:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:52:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.183+0000 7fc8a566e640 1 -- 172.21.15.2:0/2011504033 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc8880b2a70 msgr2=0x7fc8880b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.183+0000 7fc8a566e640 1 --2- 172.21.15.2:0/2011504033 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc8880b2a70 0x7fc8880b4f80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55f4eda9ce40 tx=0x7fc890008040 comp rx=0 tx=0).stop 2024-05-14T09:52:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.183+0000 7fc8a566e640 1 -- 172.21.15.2:0/2011504033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a0110220 msgr2=0x7fc8a019f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.183+0000 7fc8a566e640 1 --2- 172.21.15.2:0/2011504033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a0110220 0x7fc8a019f620 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc89801add0 tx=0x7fc898042fb0 comp rx=0 tx=0).stop 2024-05-14T09:52:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.183+0000 7fc8a566e640 1 -- 172.21.15.2:0/2011504033 shutdown_connections 2024-05-14T09:52:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.183+0000 7fc8a566e640 1 --2- 172.21.15.2:0/2011504033 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc8880b2a70 0x7fc8880b4f80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.183+0000 7fc8a566e640 1 --2- 172.21.15.2:0/2011504033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a0110220 0x7fc8a019f620 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.183+0000 7fc8a566e640 1 --2- 172.21.15.2:0/2011504033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc8a010b520 0x7fc8a0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.183+0000 7fc8a566e640 1 --2- 172.21.15.2:0/2011504033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8a010a450 0x7fc8a0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.183+0000 7fc8a566e640 1 -- 172.21.15.2:0/2011504033 >> 172.21.15.2:0/2011504033 conn(0x7fc8a0078f00 msgr2=0x7fc8a010c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.183+0000 7fc8a566e640 1 -- 172.21.15.2:0/2011504033 shutdown_connections 2024-05-14T09:52:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:10.184+0000 7fc8a566e640 1 -- 172.21.15.2:0/2011504033 wait complete. 2024-05-14T09:52:10.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 117746 28.78 MB/sec execute 96 sec latency 308.982 ms 2024-05-14T09:52:11.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:10 smithi002 ceph-mon[31557]: pgmap v941: 129 pgs: 129 active+clean; 590 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 13 MiB/s wr, 786 op/s 2024-05-14T09:52:11.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:10 smithi029 ceph-mon[36249]: pgmap v941: 129 pgs: 129 active+clean; 590 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 13 MiB/s wr, 786 op/s 2024-05-14T09:52:11.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:10 smithi026 ceph-mon[36056]: pgmap v941: 129 pgs: 129 active+clean; 590 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 13 MiB/s wr, 786 op/s 2024-05-14T09:52:11.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 118237 28.77 MB/sec execute 97 sec latency 302.526 ms 2024-05-14T09:52:12.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:11 smithi002 ceph-mon[31557]: pgmap v942: 129 pgs: 129 active+clean; 592 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 730 op/s 2024-05-14T09:52:12.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:11 smithi029 ceph-mon[36249]: pgmap v942: 129 pgs: 129 active+clean; 592 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 730 op/s 2024-05-14T09:52:12.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:11 smithi026 ceph-mon[36056]: pgmap v942: 129 pgs: 129 active+clean; 592 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 730 op/s 2024-05-14T09:52:12.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 118797 28.75 MB/sec execute 98 sec latency 346.317 ms 2024-05-14T09:52:13.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 119371 28.75 MB/sec execute 99 sec latency 104.920 ms 2024-05-14T09:52:14.607 INFO:tasks.workunit.client.0.smithi002.stdout: 10 120169 28.86 MB/sec execute 100 sec latency 58.793 ms 2024-05-14T09:52:14.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:14 smithi002 ceph-mon[31557]: pgmap v943: 129 pgs: 129 active+clean; 592 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 10 MiB/s wr, 747 op/s 2024-05-14T09:52:14.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:14 smithi029 ceph-mon[36249]: pgmap v943: 129 pgs: 129 active+clean; 592 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 10 MiB/s wr, 747 op/s 2024-05-14T09:52:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:14 smithi026 ceph-mon[36056]: pgmap v943: 129 pgs: 129 active+clean; 592 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 10 MiB/s wr, 747 op/s 2024-05-14T09:52:15.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 120892 29.05 MB/sec execute 101 sec latency 100.053 ms 2024-05-14T09:52:16.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 121515 29.09 MB/sec execute 102 sec latency 162.911 ms 2024-05-14T09:52:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:16 smithi002 ceph-mon[31557]: pgmap v944: 129 pgs: 129 active+clean; 590 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.4 MiB/s wr, 687 op/s 2024-05-14T09:52:16.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:16 smithi029 ceph-mon[36249]: pgmap v944: 129 pgs: 129 active+clean; 590 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.4 MiB/s wr, 687 op/s 2024-05-14T09:52:16.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:16 smithi026 ceph-mon[36056]: pgmap v944: 129 pgs: 129 active+clean; 590 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.4 MiB/s wr, 687 op/s 2024-05-14T09:52:17.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 121999 29.10 MB/sec execute 103 sec latency 286.205 ms 2024-05-14T09:52:18.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 122555 29.04 MB/sec execute 104 sec latency 93.653 ms 2024-05-14T09:52:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:18 smithi002 ceph-mon[31557]: pgmap v945: 129 pgs: 129 active+clean; 590 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.6 MiB/s wr, 569 op/s 2024-05-14T09:52:18.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:18 smithi029 ceph-mon[36249]: pgmap v945: 129 pgs: 129 active+clean; 590 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.6 MiB/s wr, 569 op/s 2024-05-14T09:52:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:18 smithi026 ceph-mon[36056]: pgmap v945: 129 pgs: 129 active+clean; 590 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.6 MiB/s wr, 569 op/s 2024-05-14T09:52:19.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 122792 28.85 MB/sec execute 105 sec latency 686.761 ms 2024-05-14T09:52:20.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 122879 28.61 MB/sec execute 106 sec latency 1048.920 ms 2024-05-14T09:52:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:20 smithi002 ceph-mon[31557]: pgmap v946: 129 pgs: 129 active+clean; 594 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 530 op/s 2024-05-14T09:52:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:20 smithi029 ceph-mon[36249]: pgmap v946: 129 pgs: 129 active+clean; 594 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 530 op/s 2024-05-14T09:52:20.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:20 smithi026 ceph-mon[36056]: pgmap v946: 129 pgs: 129 active+clean; 594 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 530 op/s 2024-05-14T09:52:21.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 123187 28.49 MB/sec execute 107 sec latency 184.819 ms 2024-05-14T09:52:22.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 123581 28.43 MB/sec execute 108 sec latency 211.894 ms 2024-05-14T09:52:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:22 smithi002 ceph-mon[31557]: pgmap v947: 129 pgs: 129 active+clean; 627 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.4 MiB/s wr, 561 op/s 2024-05-14T09:52:22.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:22 smithi029 ceph-mon[36249]: pgmap v947: 129 pgs: 129 active+clean; 627 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.4 MiB/s wr, 561 op/s 2024-05-14T09:52:22.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:22 smithi026 ceph-mon[36056]: pgmap v947: 129 pgs: 129 active+clean; 627 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.4 MiB/s wr, 561 op/s 2024-05-14T09:52:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.316+0000 7f983a94d640 1 -- 172.21.15.2:0/2853821676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9834104740 msgr2=0x7f9834106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.316+0000 7f983a94d640 1 --2- 172.21.15.2:0/2853821676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9834104740 0x7f9834106b60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9824009900 tx=0x7f982402a410 comp rx=0 tx=0).stop 2024-05-14T09:52:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.317+0000 7f983a94d640 1 -- 172.21.15.2:0/2853821676 shutdown_connections 2024-05-14T09:52:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.317+0000 7f983a94d640 1 --2- 172.21.15.2:0/2853821676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9834109a00 0x7f983410be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.317+0000 7f983a94d640 1 --2- 172.21.15.2:0/2853821676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98341070a0 0x7f98341094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.317+0000 7f983a94d640 1 --2- 172.21.15.2:0/2853821676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9834104740 0x7f9834106b60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.317+0000 7f983a94d640 1 -- 172.21.15.2:0/2853821676 >> 172.21.15.2:0/2853821676 conn(0x7f98340fdd40 msgr2=0x7f98341001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.317+0000 7f983a94d640 1 -- 172.21.15.2:0/2853821676 shutdown_connections 2024-05-14T09:52:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.317+0000 7f983a94d640 1 -- 172.21.15.2:0/2853821676 wait complete. 2024-05-14T09:52:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.318+0000 7f983a94d640 1 Processor -- start 2024-05-14T09:52:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.318+0000 7f983a94d640 1 -- start start 2024-05-14T09:52:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.318+0000 7f983a94d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9834104740 0x7f983419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.318+0000 7f983a94d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98341070a0 0x7f983419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.318+0000 7f983a94d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9834109a00 0x7f98341a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.319+0000 7f983a94d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f983410f730 con 0x7f9834104740 2024-05-14T09:52:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.319+0000 7f983a94d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f983410f5b0 con 0x7f98341070a0 2024-05-14T09:52:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.319+0000 7f983a94d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f983410f8b0 con 0x7f9834109a00 2024-05-14T09:52:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.319+0000 7f9833fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9834104740 0x7f983419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.319+0000 7f9833fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9834104740 0x7f983419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36910/0 (socket says 172.21.15.2:36910) 2024-05-14T09:52:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.319+0000 7f9833fff640 1 -- 172.21.15.2:0/3156670141 learned_addr learned my addr 172.21.15.2:0/3156670141 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.319+0000 7f98337fe640 1 --2- 172.21.15.2:0/3156670141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98341070a0 0x7f983419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.319+0000 7f9838eea640 1 --2- 172.21.15.2:0/3156670141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9834109a00 0x7f98341a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.319+0000 7f9833fff640 1 -- 172.21.15.2:0/3156670141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9834109a00 msgr2=0x7f98341a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.319+0000 7f9833fff640 1 --2- 172.21.15.2:0/3156670141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9834109a00 0x7f98341a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.319+0000 7f9833fff640 1 -- 172.21.15.2:0/3156670141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98341070a0 msgr2=0x7f983419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.319+0000 7f9833fff640 1 --2- 172.21.15.2:0/3156670141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98341070a0 0x7f983419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.319+0000 7f9833fff640 1 -- 172.21.15.2:0/3156670141 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98341a1e40 con 0x7f9834104740 2024-05-14T09:52:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.319+0000 7f9838eea640 1 --2- 172.21.15.2:0/3156670141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9834109a00 0x7f98341a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:52:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.319+0000 7f9833fff640 1 --2- 172.21.15.2:0/3156670141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9834104740 0x7f983419a300 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f98240057c0 tx=0x7f9824032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.320+0000 7f98317fa640 1 -- 172.21.15.2:0/3156670141 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982403f030 con 0x7f9834104740 2024-05-14T09:52:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.320+0000 7f98317fa640 1 -- 172.21.15.2:0/3156670141 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9824003cb0 con 0x7f9834104740 2024-05-14T09:52:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.320+0000 7f98317fa640 1 -- 172.21.15.2:0/3156670141 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824035070 con 0x7f9834104740 2024-05-14T09:52:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.320+0000 7f983a94d640 1 -- 172.21.15.2:0/3156670141 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98341a20d0 con 0x7f9834104740 2024-05-14T09:52:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.320+0000 7f983a94d640 1 -- 172.21.15.2:0/3156670141 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98341a2610 con 0x7f9834104740 2024-05-14T09:52:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.323+0000 7f98317fa640 1 -- 172.21.15.2:0/3156670141 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9824042030 con 0x7f9834104740 2024-05-14T09:52:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.323+0000 7f9841804740 1 Processor -- start 2024-05-14T09:52:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.323+0000 7f9841804740 1 -- start start 2024-05-14T09:52:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f9841804740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd67c21b0 0x55bdd6b67e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f9841804740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 0x55bdd6b683f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f9841804740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6b6f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f9841804740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55bdd6ac85d0 con 0x55bdd67992f0 2024-05-14T09:52:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f9841804740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bdd6a81890 con 0x55bdd67c21b0 2024-05-14T09:52:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f9841804740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bdd6ac4590 con 0x55bdd67a14d0 2024-05-14T09:52:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f9833fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 0x55bdd6b683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f9838eea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd67c21b0 0x55bdd6b67e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f9838eea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd67c21b0 0x55bdd6b67e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:45070/0 (socket says 172.21.15.2:45070) 2024-05-14T09:52:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f9838eea640 1 -- 172.21.15.2:0/1728960934 learned_addr learned my addr 172.21.15.2:0/1728960934 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f98337fe640 1 --2- 172.21.15.2:0/1728960934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6b6f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f9833fff640 1 -- 172.21.15.2:0/1728960934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 msgr2=0x55bdd6b6f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f9833fff640 1 --2- 172.21.15.2:0/1728960934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6b6f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f9833fff640 1 -- 172.21.15.2:0/1728960934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd67c21b0 msgr2=0x55bdd6b67e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f9833fff640 1 --2- 172.21.15.2:0/1728960934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd67c21b0 0x55bdd6b67e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.324+0000 7f9833fff640 1 -- 172.21.15.2:0/1728960934 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdd67da210 con 0x55bdd67992f0 2024-05-14T09:52:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.325+0000 7f98317fa640 1 --2- 172.21.15.2:0/3156670141 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f981c0b2920 0x7f981c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.325+0000 7f9838eea640 1 --2- 172.21.15.2:0/3156670141 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f981c0b2920 0x7f981c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.325+0000 7f98317fa640 1 -- 172.21.15.2:0/3156670141 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f9824102ab0 con 0x7f9834104740 2024-05-14T09:52:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.325+0000 7f9838eea640 1 --2- 172.21.15.2:0/3156670141 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f981c0b2920 0x7f981c0b4e30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55bdd6b69040 tx=0x7f982c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.325+0000 7f9833fff640 1 --2- 172.21.15.2:0/1728960934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 0x55bdd6b683f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9824100160 tx=0x7f9824106d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93c3cde5e93459b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.326+0000 7f981a7fc640 1 -- 172.21.15.2:0/1728960934 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824049070 con 0x55bdd67992f0 2024-05-14T09:52:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.326+0000 7f981a7fc640 1 -- 172.21.15.2:0/1728960934 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98240d2460 con 0x55bdd67992f0 2024-05-14T09:52:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.326+0000 7f981a7fc640 1 -- 172.21.15.2:0/1728960934 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98240c5b70 con 0x55bdd67992f0 2024-05-14T09:52:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.327+0000 7f9841804740 1 -- 172.21.15.2:0/1728960934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 msgr2=0x55bdd6b683f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.327+0000 7f9841804740 1 --2- 172.21.15.2:0/1728960934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 0x55bdd6b683f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9824100160 tx=0x7f9824106d10 comp rx=0 tx=0).stop 2024-05-14T09:52:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.327+0000 7f9841804740 1 -- 172.21.15.2:0/1728960934 shutdown_connections 2024-05-14T09:52:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.327+0000 7f9841804740 1 --2- 172.21.15.2:0/1728960934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6b6f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.327+0000 7f9841804740 1 --2- 172.21.15.2:0/1728960934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 0x55bdd6b683f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.327+0000 7f9841804740 1 --2- 172.21.15.2:0/1728960934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd67c21b0 0x55bdd6b67e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.327+0000 7f9841804740 1 -- 172.21.15.2:0/1728960934 >> 172.21.15.2:0/1728960934 conn(0x55bdd67b6300 msgr2=0x55bdd6b5f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.327+0000 7f9841804740 1 -- 172.21.15.2:0/1728960934 shutdown_connections 2024-05-14T09:52:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.327+0000 7f9841804740 1 -- 172.21.15.2:0/1728960934 wait complete. 2024-05-14T09:52:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.327+0000 7f9841804740 1 -- start start 2024-05-14T09:52:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.328+0000 7f9841804740 1 Processor -- start 2024-05-14T09:52:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.328+0000 7f9841804740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd67992f0 0x55bdd6b6cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.328+0000 7f9841804740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6b6d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.328+0000 7f9841804740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67c21b0 0x55bdd6b6e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.328+0000 7f9841804740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55bdd6b6fb00 con 0x55bdd67c21b0 2024-05-14T09:52:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.328+0000 7f9841804740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bdd6a85370 con 0x55bdd67992f0 2024-05-14T09:52:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.328+0000 7f9841804740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bdd6b6fc80 con 0x55bdd67a14d0 2024-05-14T09:52:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.328+0000 7f9841804740 10 client.0 _ll_register_callbacks cb 0x55bdd67d9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:52:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.328+0000 7f9833fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd67992f0 0x55bdd6b6cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.328+0000 7f9833fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd67992f0 0x55bdd6b6cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:45074/0 (socket says 172.21.15.2:45074) 2024-05-14T09:52:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.328+0000 7f9833fff640 1 -- 172.21.15.2:0/2867340280 learned_addr learned my addr 172.21.15.2:0/2867340280 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.329+0000 7f9838eea640 1 --2- 172.21.15.2:0/2867340280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67c21b0 0x55bdd6b6e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.329+0000 7f98337fe640 1 --2- 172.21.15.2:0/2867340280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6b6d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.329+0000 7f9833fff640 1 -- 172.21.15.2:0/2867340280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 msgr2=0x55bdd6b6d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.329+0000 7f9833fff640 1 --2- 172.21.15.2:0/2867340280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6b6d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.329+0000 7f9833fff640 1 -- 172.21.15.2:0/2867340280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67c21b0 msgr2=0x55bdd6b6e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.329+0000 7f9833fff640 1 --2- 172.21.15.2:0/2867340280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67c21b0 0x55bdd6b6e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.329+0000 7f9833fff640 1 -- 172.21.15.2:0/2867340280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdd68a6620 con 0x55bdd67992f0 2024-05-14T09:52:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.329+0000 7f9838eea640 1 --2- 172.21.15.2:0/2867340280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67c21b0 0x55bdd6b6e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:52:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.329+0000 7f9833fff640 1 --2- 172.21.15.2:0/2867340280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd67992f0 0x55bdd6b6cbd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9824100190 tx=0x7f9824005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96d44040363dfa67 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.329+0000 7f98137fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:52:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.329+0000 7f98137fe640 1 -- 172.21.15.2:0/2867340280 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824049040 con 0x55bdd67992f0 2024-05-14T09:52:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.329+0000 7f98137fe640 1 -- 172.21.15.2:0/2867340280 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f98240d2680 con 0x55bdd67992f0 2024-05-14T09:52:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.329+0000 7f9841804740 1 -- 172.21.15.2:0/2867340280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98240d2460 con 0x55bdd67992f0 2024-05-14T09:52:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.330+0000 7f98137fe640 1 -- 172.21.15.2:0/2867340280 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f9824003e60 con 0x55bdd67992f0 2024-05-14T09:52:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.330+0000 7f9841804740 10 client.26753 fetch_fsmap learned FSMap version 61 2024-05-14T09:52:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.330+0000 7f9841804740 1 -- 172.21.15.2:0/2867340280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55bdd6b6fb00 con 0x55bdd67992f0 2024-05-14T09:52:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.330+0000 7f98137fe640 1 -- 172.21.15.2:0/2867340280 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98240d2460 con 0x55bdd67992f0 2024-05-14T09:52:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.330+0000 7f98137fe640 1 -- 172.21.15.2:0/2867340280 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7f98240c8f10 con 0x55bdd67992f0 2024-05-14T09:52:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.330+0000 7f9841804740 10 client.26753 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:52:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.330+0000 7f9841804740 10 client.26753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:52:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.330+0000 7f9841804740 20 client.26753 populate_metadata read hostname 'smithi002' 2024-05-14T09:52:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.330+0000 7f9841804740 1 --2- 172.21.15.2:0/2867340280 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bdd6780d80 0x55bdd6b831f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.330+0000 7f9841804740 4 client.26753 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:52:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.330+0000 7f9841804740 1 -- 172.21.15.2:0/2867340280 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdd6ad61e0 con 0x55bdd6780d80 2024-05-14T09:52:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.331+0000 7f98337fe640 1 --2- 172.21.15.2:0/2867340280 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bdd6780d80 0x55bdd6b831f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.331+0000 7f98337fe640 1 --2- 172.21.15.2:0/2867340280 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bdd6780d80 0x55bdd6b831f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f89d18d84123c93c server_cookie=4bde058f8b075e66 in_seq=0 out_seq=0 2024-05-14T09:52:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.331+0000 7f98137fe640 10 client.26753 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.333+0000 7f98137fe640 1 -- 172.21.15.2:0/2867340280 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bdd6ad61e0 con 0x55bdd6780d80 2024-05-14T09:52:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.333+0000 7f98137fe640 10 client.26753 handle_command_reply: tid=0 2024-05-14T09:52:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.333+0000 7f9841804740 1 client.26753 shutdown 2024-05-14T09:52:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.333+0000 7f9841804740 1 -- 172.21.15.2:0/2867340280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd67992f0 msgr2=0x55bdd6b6cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.333+0000 7f9841804740 1 --2- 172.21.15.2:0/2867340280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd67992f0 0x55bdd6b6cbd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9824100190 tx=0x7f9824005c90 comp rx=0 tx=0).stop 2024-05-14T09:52:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.333+0000 7f9841804740 1 -- 172.21.15.2:0/2867340280 shutdown_connections 2024-05-14T09:52:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.333+0000 7f9841804740 1 --2- 172.21.15.2:0/2867340280 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bdd6780d80 0x55bdd6b831f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.333+0000 7f9841804740 1 --2- 172.21.15.2:0/2867340280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67c21b0 0x55bdd6b6e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.333+0000 7f9841804740 1 --2- 172.21.15.2:0/2867340280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6b6d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.333+0000 7f98137fe640 0 client.26753 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.333+0000 7f9841804740 1 --2- 172.21.15.2:0/2867340280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd67992f0 0x55bdd6b6cbd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.333+0000 7f9841804740 1 -- 172.21.15.2:0/2867340280 >> 172.21.15.2:0/2867340280 conn(0x55bdd67b6300 msgr2=0x55bdd6b62300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.333+0000 7f9841804740 1 -- 172.21.15.2:0/2867340280 shutdown_connections 2024-05-14T09:52:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.333+0000 7f9841804740 1 -- 172.21.15.2:0/2867340280 wait complete. 2024-05-14T09:52:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.334+0000 7f9841804740 20 client.26753 trim_cache size 0 max 16384 2024-05-14T09:52:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.358+0000 7f9841804740 1 Processor -- start 2024-05-14T09:52:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.358+0000 7f9841804740 1 -- start start 2024-05-14T09:52:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.358+0000 7f9841804740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd6780d80 0x55bdd6b5ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.358+0000 7f9841804740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 0x55bdd6b5f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.358+0000 7f9841804740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6b6c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.358+0000 7f9841804740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f98341ad370 con 0x55bdd67992f0 2024-05-14T09:52:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.358+0000 7f9841804740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f98341aac40 con 0x55bdd6780d80 2024-05-14T09:52:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.358+0000 7f9841804740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f98341a8a90 con 0x55bdd67a14d0 2024-05-14T09:52:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.358+0000 7f98337fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 0x55bdd6b5f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.358+0000 7f98337fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 0x55bdd6b5f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36958/0 (socket says 172.21.15.2:36958) 2024-05-14T09:52:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.358+0000 7f98337fe640 1 -- 172.21.15.2:0/3389942228 learned_addr learned my addr 172.21.15.2:0/3389942228 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.358+0000 7f9833fff640 1 --2- 172.21.15.2:0/3389942228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd6780d80 0x55bdd6b5ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.359+0000 7f98337fe640 1 -- 172.21.15.2:0/3389942228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 msgr2=0x55bdd6b6c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.359+0000 7f98337fe640 1 --2- 172.21.15.2:0/3389942228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6b6c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.359+0000 7f98337fe640 1 -- 172.21.15.2:0/3389942228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd6780d80 msgr2=0x55bdd6b5ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.359+0000 7f98337fe640 1 --2- 172.21.15.2:0/3389942228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd6780d80 0x55bdd6b5ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.359+0000 7f98337fe640 1 -- 172.21.15.2:0/3389942228 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdd6b6fb00 con 0x55bdd67992f0 2024-05-14T09:52:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.359+0000 7f98337fe640 1 --2- 172.21.15.2:0/3389942228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 0x55bdd6b5f140 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x55bdd6b60180 tx=0x7f9828005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158c272d728ff368 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.359+0000 7f9819ffb640 1 -- 172.21.15.2:0/3389942228 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828004020 con 0x55bdd67992f0 2024-05-14T09:52:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.359+0000 7f9819ffb640 1 -- 172.21.15.2:0/3389942228 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9828024030 con 0x55bdd67992f0 2024-05-14T09:52:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.359+0000 7f9819ffb640 1 -- 172.21.15.2:0/3389942228 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828020350 con 0x55bdd67992f0 2024-05-14T09:52:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.360+0000 7f9841804740 1 -- 172.21.15.2:0/3389942228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 msgr2=0x55bdd6b5f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.360+0000 7f9841804740 1 --2- 172.21.15.2:0/3389942228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 0x55bdd6b5f140 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x55bdd6b60180 tx=0x7f9828005f90 comp rx=0 tx=0).stop 2024-05-14T09:52:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.361+0000 7f9841804740 1 -- 172.21.15.2:0/3389942228 shutdown_connections 2024-05-14T09:52:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.361+0000 7f9841804740 1 --2- 172.21.15.2:0/3389942228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6b6c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.361+0000 7f9841804740 1 --2- 172.21.15.2:0/3389942228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 0x55bdd6b5f140 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.361+0000 7f9841804740 1 --2- 172.21.15.2:0/3389942228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd6780d80 0x55bdd6b5ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.361+0000 7f9841804740 1 -- 172.21.15.2:0/3389942228 >> 172.21.15.2:0/3389942228 conn(0x55bdd67b6300 msgr2=0x55bdd6b6b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.361+0000 7f9841804740 1 -- 172.21.15.2:0/3389942228 shutdown_connections 2024-05-14T09:52:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.361+0000 7f9841804740 1 -- 172.21.15.2:0/3389942228 wait complete. 2024-05-14T09:52:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.361+0000 7f9841804740 1 -- start start 2024-05-14T09:52:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.362+0000 7f9841804740 1 Processor -- start 2024-05-14T09:52:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.362+0000 7f9841804740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd6780d80 0x55bdd6b7b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.362+0000 7f9841804740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 0x55bdd6b7b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.362+0000 7f9833fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd6780d80 0x55bdd6b7b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.362+0000 7f9833fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd6780d80 0x55bdd6b7b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:45094/0 (socket says 172.21.15.2:45094) 2024-05-14T09:52:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.362+0000 7f98337fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 0x55bdd6b7b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.362+0000 7f9833fff640 1 -- 172.21.15.2:0/688920559 learned_addr learned my addr 172.21.15.2:0/688920559 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f9841804740 1 --2- 172.21.15.2:0/688920559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6be9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f9841804740 1 -- 172.21.15.2:0/688920559 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f98341a9100 con 0x55bdd67992f0 2024-05-14T09:52:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f9841804740 1 -- 172.21.15.2:0/688920559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f98341aa570 con 0x55bdd6780d80 2024-05-14T09:52:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f9841804740 1 -- 172.21.15.2:0/688920559 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f98341a97d0 con 0x55bdd67a14d0 2024-05-14T09:52:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f9838eea640 1 --2- 172.21.15.2:0/688920559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6be9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f9841804740 10 client.0 _ll_register_callbacks cb 0x7f982c0099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:52:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f9833fff640 1 -- 172.21.15.2:0/688920559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 msgr2=0x55bdd6be9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f9833fff640 1 --2- 172.21.15.2:0/688920559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6be9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f9833fff640 1 -- 172.21.15.2:0/688920559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 msgr2=0x55bdd6b7b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f9833fff640 1 --2- 172.21.15.2:0/688920559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 0x55bdd6b7b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f9833fff640 1 -- 172.21.15.2:0/688920559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdd6be9ad0 con 0x55bdd6780d80 2024-05-14T09:52:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f9838eea640 1 --2- 172.21.15.2:0/688920559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6be9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:52:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f9841804740 1 -- 172.21.15.2:0/688920559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9828024030 con 0x55bdd6780d80 2024-05-14T09:52:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f9833fff640 1 --2- 172.21.15.2:0/688920559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd6780d80 0x55bdd6b7b2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9824005ea0 tx=0x7f98241061b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17de6943b62893cb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f98137fe640 10 client.26759 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:52:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.363+0000 7f98137fe640 1 -- 172.21.15.2:0/688920559 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824049040 con 0x55bdd6780d80 2024-05-14T09:52:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.364+0000 7f98137fe640 1 -- 172.21.15.2:0/688920559 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f98240d2680 con 0x55bdd6780d80 2024-05-14T09:52:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.364+0000 7f98137fe640 1 -- 172.21.15.2:0/688920559 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f98240c9420 con 0x55bdd6780d80 2024-05-14T09:52:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.364+0000 7f9841804740 10 client.26759 fetch_fsmap learned FSMap version 61 2024-05-14T09:52:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.364+0000 7f9841804740 1 -- 172.21.15.2:0/688920559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=61}) -- 0x55bdd6be9ad0 con 0x55bdd6780d80 2024-05-14T09:52:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.364+0000 7f98137fe640 1 -- 172.21.15.2:0/688920559 <== mon.2 v2:172.21.15.26:3300/0 4 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7f9824003800 con 0x55bdd6780d80 2024-05-14T09:52:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.364+0000 7f9841804740 10 client.26759 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:52:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.364+0000 7f9841804740 10 client.26759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:52:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.364+0000 7f9841804740 20 client.26759 populate_metadata read hostname 'smithi002' 2024-05-14T09:52:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.364+0000 7f9841804740 1 --2- 172.21.15.2:0/688920559 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bdd67c21b0 0x55bdd6bed050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.364+0000 7f9841804740 4 client.26759 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:52:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.364+0000 7f9841804740 1 -- 172.21.15.2:0/688920559 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bdd6ad0050 con 0x55bdd67c21b0 2024-05-14T09:52:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.364+0000 7f98337fe640 1 --2- 172.21.15.2:0/688920559 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bdd67c21b0 0x55bdd6bed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.365+0000 7f98337fe640 1 --2- 172.21.15.2:0/688920559 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bdd67c21b0 0x55bdd6bed050 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6a797e2ef2dc8c1 server_cookie=3f465c604b9cd999 in_seq=0 out_seq=0 2024-05-14T09:52:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.365+0000 7f98137fe640 10 client.26759 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.604+0000 7f98137fe640 1 -- 172.21.15.2:0/688920559 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bdd6ad0050 con 0x55bdd67c21b0 2024-05-14T09:52:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.604+0000 7f98137fe640 10 client.26759 handle_command_reply: tid=0 2024-05-14T09:52:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.604+0000 7f9841804740 1 client.26759 shutdown 2024-05-14T09:52:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.605+0000 7f9841804740 1 -- 172.21.15.2:0/688920559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd6780d80 msgr2=0x55bdd6b7b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.605+0000 7f9841804740 1 --2- 172.21.15.2:0/688920559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd6780d80 0x55bdd6b7b2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9824005ea0 tx=0x7f98241061b0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.605+0000 7f9841804740 1 -- 172.21.15.2:0/688920559 shutdown_connections 2024-05-14T09:52:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.605+0000 7f9841804740 1 --2- 172.21.15.2:0/688920559 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bdd67c21b0 0x55bdd6bed050 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.605+0000 7f9841804740 1 --2- 172.21.15.2:0/688920559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bdd67a14d0 0x55bdd6be9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.605+0000 7f98137fe640 0 client.26759 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.605+0000 7f9841804740 1 --2- 172.21.15.2:0/688920559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdd67992f0 0x55bdd6b7b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.605+0000 7f9841804740 1 --2- 172.21.15.2:0/688920559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdd6780d80 0x55bdd6b7b2f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.605+0000 7f9841804740 1 -- 172.21.15.2:0/688920559 >> 172.21.15.2:0/688920559 conn(0x55bdd67b6300 msgr2=0x55bdd6b6adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.605+0000 7f9841804740 1 -- 172.21.15.2:0/688920559 shutdown_connections 2024-05-14T09:52:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.605+0000 7f9841804740 1 -- 172.21.15.2:0/688920559 wait complete. 2024-05-14T09:52:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.605+0000 7f9841804740 20 client.26759 trim_cache size 0 max 16384 2024-05-14T09:52:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:52:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:52:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:52:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:52:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.606+0000 7f983a94d640 1 -- 172.21.15.2:0/3156670141 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f981c0b2920 msgr2=0x7f981c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.606+0000 7f983a94d640 1 --2- 172.21.15.2:0/3156670141 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f981c0b2920 0x7f981c0b4e30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55bdd6b69040 tx=0x7f982c009500 comp rx=0 tx=0).stop 2024-05-14T09:52:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.606+0000 7f983a94d640 1 -- 172.21.15.2:0/3156670141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9834104740 msgr2=0x7f983419a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.606+0000 7f983a94d640 1 --2- 172.21.15.2:0/3156670141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9834104740 0x7f983419a300 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f98240057c0 tx=0x7f9824032d00 comp rx=0 tx=0).stop 2024-05-14T09:52:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.606+0000 7f983a94d640 1 -- 172.21.15.2:0/3156670141 shutdown_connections 2024-05-14T09:52:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.606+0000 7f983a94d640 1 --2- 172.21.15.2:0/3156670141 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f981c0b2920 0x7f981c0b4e30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.606+0000 7f983a94d640 1 --2- 172.21.15.2:0/3156670141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9834109a00 0x7f98341a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.606+0000 7f983a94d640 1 --2- 172.21.15.2:0/3156670141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98341070a0 0x7f983419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.606+0000 7f983a94d640 1 --2- 172.21.15.2:0/3156670141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9834104740 0x7f983419a300 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.606+0000 7f983a94d640 1 -- 172.21.15.2:0/3156670141 >> 172.21.15.2:0/3156670141 conn(0x7f98340fdd40 msgr2=0x7f9834105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.607+0000 7f983a94d640 1 -- 172.21.15.2:0/3156670141 shutdown_connections 2024-05-14T09:52:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:23.607+0000 7f983a94d640 1 -- 172.21.15.2:0/3156670141 wait complete. 2024-05-14T09:52:23.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 124104 28.48 MB/sec execute 109 sec latency 188.068 ms 2024-05-14T09:52:24.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 124687 28.56 MB/sec execute 110 sec latency 333.209 ms 2024-05-14T09:52:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:24 smithi002 ceph-mon[31557]: pgmap v948: 129 pgs: 129 active+clean; 635 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.5 MiB/s wr, 590 op/s 2024-05-14T09:52:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:24 smithi029 ceph-mon[36249]: pgmap v948: 129 pgs: 129 active+clean; 635 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.5 MiB/s wr, 590 op/s 2024-05-14T09:52:24.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:24 smithi026 ceph-mon[36056]: pgmap v948: 129 pgs: 129 active+clean; 635 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.5 MiB/s wr, 590 op/s 2024-05-14T09:52:25.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 125219 28.54 MB/sec execute 111 sec latency 392.763 ms 2024-05-14T09:52:26.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 125852 28.55 MB/sec execute 112 sec latency 102.555 ms 2024-05-14T09:52:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:26 smithi002 ceph-mon[31557]: pgmap v949: 129 pgs: 129 active+clean; 611 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 715 op/s 2024-05-14T09:52:26.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:26 smithi029 ceph-mon[36249]: pgmap v949: 129 pgs: 129 active+clean; 611 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 715 op/s 2024-05-14T09:52:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:26 smithi026 ceph-mon[36056]: pgmap v949: 129 pgs: 129 active+clean; 611 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 715 op/s 2024-05-14T09:52:27.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 126552 28.64 MB/sec execute 113 sec latency 98.589 ms 2024-05-14T09:52:28.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 127316 28.74 MB/sec execute 114 sec latency 84.276 ms 2024-05-14T09:52:28.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:28 smithi002 ceph-mon[31557]: pgmap v950: 129 pgs: 129 active+clean; 589 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 676 op/s 2024-05-14T09:52:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:28 smithi029 ceph-mon[36249]: pgmap v950: 129 pgs: 129 active+clean; 589 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 676 op/s 2024-05-14T09:52:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:28 smithi026 ceph-mon[36056]: pgmap v950: 129 pgs: 129 active+clean; 589 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 676 op/s 2024-05-14T09:52:29.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 127939 28.82 MB/sec execute 115 sec latency 59.268 ms 2024-05-14T09:52:30.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 128558 28.87 MB/sec execute 116 sec latency 68.277 ms 2024-05-14T09:52:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:30 smithi002 ceph-mon[31557]: pgmap v951: 129 pgs: 129 active+clean; 588 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 9.7 MiB/s wr, 604 op/s 2024-05-14T09:52:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:30 smithi029 ceph-mon[36249]: pgmap v951: 129 pgs: 129 active+clean; 588 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 9.7 MiB/s wr, 604 op/s 2024-05-14T09:52:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:30 smithi026 ceph-mon[36056]: pgmap v951: 129 pgs: 129 active+clean; 588 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 9.7 MiB/s wr, 604 op/s 2024-05-14T09:52:31.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 129153 28.83 MB/sec execute 117 sec latency 92.667 ms 2024-05-14T09:52:32.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 129662 28.82 MB/sec execute 118 sec latency 203.205 ms 2024-05-14T09:52:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:32 smithi002 ceph-mon[31557]: pgmap v952: 129 pgs: 129 active+clean; 601 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 11 MiB/s wr, 712 op/s 2024-05-14T09:52:32.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:32 smithi029 ceph-mon[36249]: pgmap v952: 129 pgs: 129 active+clean; 601 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 11 MiB/s wr, 712 op/s 2024-05-14T09:52:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:32 smithi026 ceph-mon[36056]: pgmap v952: 129 pgs: 129 active+clean; 601 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 11 MiB/s wr, 712 op/s 2024-05-14T09:52:33.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 130430 28.96 MB/sec execute 119 sec latency 155.927 ms 2024-05-14T09:52:34.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 131074 28.98 MB/sec execute 120 sec latency 86.629 ms 2024-05-14T09:52:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:34 smithi002 ceph-mon[31557]: pgmap v953: 129 pgs: 129 active+clean; 606 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.7 MiB/s wr, 672 op/s 2024-05-14T09:52:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:34 smithi029 ceph-mon[36249]: pgmap v953: 129 pgs: 129 active+clean; 606 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.7 MiB/s wr, 672 op/s 2024-05-14T09:52:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:34 smithi026 ceph-mon[36056]: pgmap v953: 129 pgs: 129 active+clean; 606 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.7 MiB/s wr, 672 op/s 2024-05-14T09:52:35.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 131729 29.09 MB/sec execute 121 sec latency 92.275 ms 2024-05-14T09:52:36.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 132370 29.12 MB/sec execute 122 sec latency 71.891 ms 2024-05-14T09:52:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.749+0000 7f8431b6c640 1 -- 172.21.15.2:0/885736531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f842c109a00 msgr2=0x7f842c10be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.749+0000 7f8431b6c640 1 --2- 172.21.15.2:0/885736531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f842c109a00 0x7f842c10be20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f8424009ca0 tx=0x7f842402a8c0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.749+0000 7f8431b6c640 1 -- 172.21.15.2:0/885736531 shutdown_connections 2024-05-14T09:52:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.749+0000 7f8431b6c640 1 --2- 172.21.15.2:0/885736531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f842c109a00 0x7f842c10be20 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.749+0000 7f8431b6c640 1 --2- 172.21.15.2:0/885736531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f842c1070a0 0x7f842c1094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.749+0000 7f8431b6c640 1 --2- 172.21.15.2:0/885736531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c104740 0x7f842c106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.749+0000 7f8431b6c640 1 -- 172.21.15.2:0/885736531 >> 172.21.15.2:0/885736531 conn(0x7f842c0fdd40 msgr2=0x7f842c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.749+0000 7f8431b6c640 1 -- 172.21.15.2:0/885736531 shutdown_connections 2024-05-14T09:52:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.749+0000 7f8431b6c640 1 -- 172.21.15.2:0/885736531 wait complete. 2024-05-14T09:52:36.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.750+0000 7f8431b6c640 1 Processor -- start 2024-05-14T09:52:36.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.750+0000 7f8431b6c640 1 -- start start 2024-05-14T09:52:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.750+0000 7f8431b6c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c104740 0x7f842c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.751+0000 7f8431b6c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f842c1070a0 0x7f842c19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.751+0000 7f8431b6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f842c109a00 0x7f842c1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.751+0000 7f8431b6c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f842c10f7d0 con 0x7f842c1070a0 2024-05-14T09:52:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.751+0000 7f8431b6c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f842c10f650 con 0x7f842c109a00 2024-05-14T09:52:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.751+0000 7f8431b6c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f842c10f950 con 0x7f842c104740 2024-05-14T09:52:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.751+0000 7f842affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f842c1070a0 0x7f842c19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.751+0000 7f842affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f842c1070a0 0x7f842c19a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39828/0 (socket says 172.21.15.2:39828) 2024-05-14T09:52:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.751+0000 7f842b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c104740 0x7f842c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.751+0000 7f842affd640 1 -- 172.21.15.2:0/2856395532 learned_addr learned my addr 172.21.15.2:0/2856395532 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.751+0000 7f842bfff640 1 --2- 172.21.15.2:0/2856395532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f842c109a00 0x7f842c1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.751+0000 7f842affd640 1 -- 172.21.15.2:0/2856395532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c104740 msgr2=0x7f842c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.751+0000 7f842affd640 1 --2- 172.21.15.2:0/2856395532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c104740 0x7f842c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.751+0000 7f842affd640 1 -- 172.21.15.2:0/2856395532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f842c109a00 msgr2=0x7f842c1a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.751+0000 7f842affd640 1 --2- 172.21.15.2:0/2856395532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f842c109a00 0x7f842c1a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.751+0000 7f842affd640 1 -- 172.21.15.2:0/2856395532 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842c1a1dd0 con 0x7f842c1070a0 2024-05-14T09:52:36.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.752+0000 7f842affd640 1 --2- 172.21.15.2:0/2856395532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f842c1070a0 0x7f842c19a7d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f842000e9f0 tx=0x7f842000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:36.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.754+0000 7f8428ff9640 1 -- 172.21.15.2:0/2856395532 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842000cd80 con 0x7f842c1070a0 2024-05-14T09:52:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.754+0000 7f8428ff9640 1 -- 172.21.15.2:0/2856395532 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84200108e0 con 0x7f842c1070a0 2024-05-14T09:52:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.754+0000 7f8428ff9640 1 -- 172.21.15.2:0/2856395532 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420018880 con 0x7f842c1070a0 2024-05-14T09:52:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.754+0000 7f8431b6c640 1 -- 172.21.15.2:0/2856395532 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842c102170 con 0x7f842c1070a0 2024-05-14T09:52:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.755+0000 7f8431b6c640 1 -- 172.21.15.2:0/2856395532 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842c102670 con 0x7f842c1070a0 2024-05-14T09:52:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.760+0000 7f8438a1c740 1 Processor -- start 2024-05-14T09:52:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.761+0000 7f8428ff9640 1 -- 172.21.15.2:0/2856395532 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8420010430 con 0x7f842c1070a0 2024-05-14T09:52:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.762+0000 7f8438a1c740 1 -- start start 2024-05-14T09:52:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.762+0000 7f8438a1c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244671b0 0x564024808a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.762+0000 7f8438a1c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 0x564024808fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.762+0000 7f8438a1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 0x56402480fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.762+0000 7f842bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244671b0 0x564024808a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.762+0000 7f842bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244671b0 0x564024808a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39832/0 (socket says 172.21.15.2:39832) 2024-05-14T09:52:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.763+0000 7f842bfff640 1 -- 172.21.15.2:0/4058180697 learned_addr learned my addr 172.21.15.2:0/4058180697 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.763+0000 7f8438a1c740 1 -- 172.21.15.2:0/4058180697 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56402476d5d0 con 0x5640244671b0 2024-05-14T09:52:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.763+0000 7f8438a1c740 1 -- 172.21.15.2:0/4058180697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564024726890 con 0x5640244464d0 2024-05-14T09:52:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.763+0000 7f8438a1c740 1 -- 172.21.15.2:0/4058180697 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564024769590 con 0x56402443e2f0 2024-05-14T09:52:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.763+0000 7f8428ff9640 1 --2- 172.21.15.2:0/2856395532 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f84140b6e80 0x7f84140b9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.763+0000 7f842affd640 1 --2- 172.21.15.2:0/4058180697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 0x56402480fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.763+0000 7f8428ff9640 1 -- 172.21.15.2:0/2856395532 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f84200e2a50 con 0x7f842c1070a0 2024-05-14T09:52:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.764+0000 7f842b7fe640 1 --2- 172.21.15.2:0/4058180697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 0x564024808fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.764+0000 7f842affd640 1 -- 172.21.15.2:0/4058180697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 msgr2=0x564024808fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.764+0000 7f842affd640 1 --2- 172.21.15.2:0/4058180697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 0x564024808fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.764+0000 7f842affd640 1 -- 172.21.15.2:0/4058180697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244671b0 msgr2=0x564024808a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.764+0000 7f842affd640 1 --2- 172.21.15.2:0/4058180697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244671b0 0x564024808a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.764+0000 7f842affd640 1 -- 172.21.15.2:0/4058180697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56402447f210 con 0x5640244464d0 2024-05-14T09:52:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.764+0000 7f842bfff640 1 --2- 172.21.15.2:0/4058180697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244671b0 0x564024808a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:52:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.764+0000 7f842bfff640 1 --2- 172.21.15.2:0/2856395532 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f84140b6e80 0x7f84140b9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.764+0000 7f842affd640 1 --2- 172.21.15.2:0/4058180697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 0x56402480fee0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f842001e000 tx=0x7f84200d9200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dee46fb5d335e1bb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.764+0000 7f8411ffb640 1 -- 172.21.15.2:0/4058180697 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84200a9700 con 0x5640244464d0 2024-05-14T09:52:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.764+0000 7f8411ffb640 1 -- 172.21.15.2:0/4058180697 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84200aa2f0 con 0x5640244464d0 2024-05-14T09:52:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.764+0000 7f8411ffb640 1 -- 172.21.15.2:0/4058180697 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84200b25c0 con 0x5640244464d0 2024-05-14T09:52:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.765+0000 7f8438a1c740 1 -- 172.21.15.2:0/4058180697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 msgr2=0x56402480fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.765+0000 7f8438a1c740 1 --2- 172.21.15.2:0/4058180697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 0x56402480fee0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f842001e000 tx=0x7f84200d9200 comp rx=0 tx=0).stop 2024-05-14T09:52:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.765+0000 7f842bfff640 1 --2- 172.21.15.2:0/2856395532 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f84140b6e80 0x7f84140b9390 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x564024809c30 tx=0x7f84240095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.765+0000 7f8438a1c740 1 -- 172.21.15.2:0/4058180697 shutdown_connections 2024-05-14T09:52:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.765+0000 7f8438a1c740 1 --2- 172.21.15.2:0/4058180697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 0x56402480fee0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.765+0000 7f8438a1c740 1 --2- 172.21.15.2:0/4058180697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 0x564024808fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.765+0000 7f8438a1c740 1 --2- 172.21.15.2:0/4058180697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244671b0 0x564024808a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.765+0000 7f8438a1c740 1 -- 172.21.15.2:0/4058180697 >> 172.21.15.2:0/4058180697 conn(0x56402445b300 msgr2=0x5640248043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.765+0000 7f8438a1c740 1 -- 172.21.15.2:0/4058180697 shutdown_connections 2024-05-14T09:52:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.766+0000 7f8438a1c740 1 -- 172.21.15.2:0/4058180697 wait complete. 2024-05-14T09:52:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.766+0000 7f8438a1c740 1 -- start start 2024-05-14T09:52:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.766+0000 7f8438a1c740 1 Processor -- start 2024-05-14T09:52:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.766+0000 7f8438a1c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 0x5640248126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.766+0000 7f8438a1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 0x564024812bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.766+0000 7f8438a1c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244671b0 0x564024819b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.766+0000 7f8438a1c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5640248108d0 con 0x5640244671b0 2024-05-14T09:52:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f8438a1c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56402472a370 con 0x5640244464d0 2024-05-14T09:52:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f8438a1c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x564024810a50 con 0x56402443e2f0 2024-05-14T09:52:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f8438a1c740 10 client.0 _ll_register_callbacks cb 0x56402447ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:52:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f842b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 0x5640248126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f842affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 0x564024812bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f842b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 0x5640248126b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:43756/0 (socket says 172.21.15.2:43756) 2024-05-14T09:52:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f842affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 0x564024812bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:38612/0 (socket says 172.21.15.2:38612) 2024-05-14T09:52:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f842b7fe640 1 -- 172.21.15.2:0/1606896913 learned_addr learned my addr 172.21.15.2:0/1606896913 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f842bfff640 1 --2- 172.21.15.2:0/1606896913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244671b0 0x564024819b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f842affd640 1 -- 172.21.15.2:0/1606896913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 msgr2=0x5640248126b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f842affd640 1 --2- 172.21.15.2:0/1606896913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 0x5640248126b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f842affd640 1 -- 172.21.15.2:0/1606896913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244671b0 msgr2=0x564024819b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f842affd640 1 --2- 172.21.15.2:0/1606896913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244671b0 0x564024819b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f842affd640 1 -- 172.21.15.2:0/1606896913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56402454b620 con 0x5640244464d0 2024-05-14T09:52:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f842affd640 1 --2- 172.21.15.2:0/1606896913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 0x564024812bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x564024810580 tx=0x7f84200b2870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ccc703690850481 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.767+0000 7f840affd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:52:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.768+0000 7f840affd640 1 -- 172.21.15.2:0/1606896913 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84200a9700 con 0x5640244464d0 2024-05-14T09:52:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.768+0000 7f840affd640 1 -- 172.21.15.2:0/1606896913 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f84200a9060 con 0x5640244464d0 2024-05-14T09:52:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.768+0000 7f8438a1c740 1 -- 172.21.15.2:0/1606896913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84200aa2f0 con 0x5640244464d0 2024-05-14T09:52:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.768+0000 7f840affd640 1 -- 172.21.15.2:0/1606896913 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f84200c9050 con 0x5640244464d0 2024-05-14T09:52:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.768+0000 7f8438a1c740 10 client.26771 fetch_fsmap learned FSMap version 61 2024-05-14T09:52:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.768+0000 7f8438a1c740 1 -- 172.21.15.2:0/1606896913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x56402481ca80 con 0x5640244464d0 2024-05-14T09:52:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.768+0000 7f840affd640 1 -- 172.21.15.2:0/1606896913 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84200aa2f0 con 0x5640244464d0 2024-05-14T09:52:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.768+0000 7f840affd640 1 -- 172.21.15.2:0/1606896913 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7f84200a88c0 con 0x5640244464d0 2024-05-14T09:52:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.769+0000 7f8438a1c740 10 client.26771 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:52:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.769+0000 7f8438a1c740 10 client.26771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:52:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.769+0000 7f8438a1c740 20 client.26771 populate_metadata read hostname 'smithi002' 2024-05-14T09:52:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.769+0000 7f8438a1c740 1 --2- 172.21.15.2:0/1606896913 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564024425d80 0x56402481fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.769+0000 7f8438a1c740 4 client.26771 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:52:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.769+0000 7f8438a1c740 1 -- 172.21.15.2:0/1606896913 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56402477b1e0 con 0x564024425d80 2024-05-14T09:52:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.769+0000 7f842b7fe640 1 --2- 172.21.15.2:0/1606896913 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564024425d80 0x56402481fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.769+0000 7f840affd640 10 client.26771 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.769+0000 7f842b7fe640 1 --2- 172.21.15.2:0/1606896913 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564024425d80 0x56402481fde0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6a06f75a36cc51b server_cookie=b237e1f39cc9aee2 in_seq=0 out_seq=0 2024-05-14T09:52:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.771+0000 7f840affd640 1 -- 172.21.15.2:0/1606896913 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56402477b1e0 con 0x564024425d80 2024-05-14T09:52:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.771+0000 7f840affd640 10 client.26771 handle_command_reply: tid=0 2024-05-14T09:52:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.771+0000 7f8438a1c740 1 client.26771 shutdown 2024-05-14T09:52:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.771+0000 7f8438a1c740 1 -- 172.21.15.2:0/1606896913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 msgr2=0x564024812bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.771+0000 7f8438a1c740 1 --2- 172.21.15.2:0/1606896913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 0x564024812bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x564024810580 tx=0x7f84200b2870 comp rx=0 tx=0).stop 2024-05-14T09:52:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.771+0000 7f8438a1c740 1 -- 172.21.15.2:0/1606896913 shutdown_connections 2024-05-14T09:52:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.771+0000 7f8438a1c740 1 --2- 172.21.15.2:0/1606896913 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x564024425d80 0x56402481fde0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.771+0000 7f840affd640 0 client.26771 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.771+0000 7f8438a1c740 1 --2- 172.21.15.2:0/1606896913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244671b0 0x564024819b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.771+0000 7f8438a1c740 1 --2- 172.21.15.2:0/1606896913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 0x564024812bf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.771+0000 7f8438a1c740 1 --2- 172.21.15.2:0/1606896913 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 0x5640248126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.771+0000 7f8438a1c740 1 -- 172.21.15.2:0/1606896913 >> 172.21.15.2:0/1606896913 conn(0x56402445b300 msgr2=0x564024807300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.771+0000 7f8438a1c740 1 -- 172.21.15.2:0/1606896913 shutdown_connections 2024-05-14T09:52:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.771+0000 7f8438a1c740 1 -- 172.21.15.2:0/1606896913 wait complete. 2024-05-14T09:52:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.772+0000 7f8438a1c740 20 client.26771 trim_cache size 0 max 16384 2024-05-14T09:52:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.797+0000 7f8438a1c740 1 Processor -- start 2024-05-14T09:52:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.797+0000 7f8438a1c740 1 -- start start 2024-05-14T09:52:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.797+0000 7f8438a1c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564024425d80 0x564024803c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.798+0000 7f8438a1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56402443e2f0 0x564024804140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.798+0000 7f8438a1c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244464d0 0x564024802970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.798+0000 7f8438a1c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f842c1ad320 con 0x5640244464d0 2024-05-14T09:52:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.798+0000 7f8438a1c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f842c1aabf0 con 0x56402443e2f0 2024-05-14T09:52:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.798+0000 7f8438a1c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f842c1a8a40 con 0x564024425d80 2024-05-14T09:52:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.798+0000 7f842b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564024425d80 0x564024803c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.798+0000 7f842b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564024425d80 0x564024803c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:43768/0 (socket says 172.21.15.2:43768) 2024-05-14T09:52:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.798+0000 7f842b7fe640 1 -- 172.21.15.2:0/221394329 learned_addr learned my addr 172.21.15.2:0/221394329 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.798+0000 7f842bfff640 1 --2- 172.21.15.2:0/221394329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244464d0 0x564024802970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.798+0000 7f842affd640 1 --2- 172.21.15.2:0/221394329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56402443e2f0 0x564024804140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.798+0000 7f842b7fe640 1 -- 172.21.15.2:0/221394329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56402443e2f0 msgr2=0x564024804140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.798+0000 7f842b7fe640 1 --2- 172.21.15.2:0/221394329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56402443e2f0 0x564024804140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.798+0000 7f842b7fe640 1 -- 172.21.15.2:0/221394329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244464d0 msgr2=0x564024802970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.798+0000 7f842b7fe640 1 --2- 172.21.15.2:0/221394329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244464d0 0x564024802970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.798+0000 7f842b7fe640 1 -- 172.21.15.2:0/221394329 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640248105b0 con 0x564024425d80 2024-05-14T09:52:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.799+0000 7f842affd640 1 --2- 172.21.15.2:0/221394329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56402443e2f0 0x564024804140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:52:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.799+0000 7f842b7fe640 1 --2- 172.21.15.2:0/221394329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564024425d80 0x564024803c00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f841c009d00 tx=0x7f841c014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c86240b4b3866fb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.799+0000 7f84117fa640 1 -- 172.21.15.2:0/221394329 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c024070 con 0x564024425d80 2024-05-14T09:52:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.799+0000 7f84117fa640 1 -- 172.21.15.2:0/221394329 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56402477b1e0 con 0x564024425d80 2024-05-14T09:52:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.799+0000 7f84117fa640 1 -- 172.21.15.2:0/221394329 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c004b90 con 0x564024425d80 2024-05-14T09:52:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.800+0000 7f8438a1c740 1 -- 172.21.15.2:0/221394329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564024425d80 msgr2=0x564024803c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.800+0000 7f8438a1c740 1 --2- 172.21.15.2:0/221394329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564024425d80 0x564024803c00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f841c009d00 tx=0x7f841c014040 comp rx=0 tx=0).stop 2024-05-14T09:52:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.800+0000 7f8438a1c740 1 -- 172.21.15.2:0/221394329 shutdown_connections 2024-05-14T09:52:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.800+0000 7f8438a1c740 1 --2- 172.21.15.2:0/221394329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5640244464d0 0x564024802970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.800+0000 7f8438a1c740 1 --2- 172.21.15.2:0/221394329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56402443e2f0 0x564024804140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.800+0000 7f8438a1c740 1 --2- 172.21.15.2:0/221394329 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564024425d80 0x564024803c00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.800+0000 7f8438a1c740 1 -- 172.21.15.2:0/221394329 >> 172.21.15.2:0/221394329 conn(0x56402445b300 msgr2=0x5640248089e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.801+0000 7f8438a1c740 1 -- 172.21.15.2:0/221394329 shutdown_connections 2024-05-14T09:52:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.801+0000 7f8438a1c740 1 -- 172.21.15.2:0/221394329 wait complete. 2024-05-14T09:52:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.801+0000 7f8438a1c740 1 -- start start 2024-05-14T09:52:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.802+0000 7f8438a1c740 1 Processor -- start 2024-05-14T09:52:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.802+0000 7f8438a1c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564024425d80 0x5640248168f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.802+0000 7f8438a1c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 0x564024816e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.803+0000 7f8438a1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 0x564024817d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.803+0000 7f842b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564024425d80 0x5640248168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.803+0000 7f842affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 0x564024816e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.803+0000 7f842affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 0x564024816e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:43774/0 (socket says 172.21.15.2:43774) 2024-05-14T09:52:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.803+0000 7f842b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564024425d80 0x5640248168f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39854/0 (socket says 172.21.15.2:39854) 2024-05-14T09:52:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.803+0000 7f842affd640 1 -- 172.21.15.2:0/1112764992 learned_addr learned my addr 172.21.15.2:0/1112764992 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.803+0000 7f842bfff640 1 --2- 172.21.15.2:0/1112764992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 0x564024817d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.803+0000 7f8438a1c740 1 -- 172.21.15.2:0/1112764992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f842c1a90b0 con 0x564024425d80 2024-05-14T09:52:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.803+0000 7f8438a1c740 1 -- 172.21.15.2:0/1112764992 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f842c1aa520 con 0x5640244464d0 2024-05-14T09:52:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.803+0000 7f8438a1c740 1 -- 172.21.15.2:0/1112764992 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f842c1a9780 con 0x56402443e2f0 2024-05-14T09:52:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.804+0000 7f842affd640 1 -- 172.21.15.2:0/1112764992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 msgr2=0x564024817d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.804+0000 7f842affd640 1 --2- 172.21.15.2:0/1112764992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 0x564024817d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.804+0000 7f842affd640 1 -- 172.21.15.2:0/1112764992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564024425d80 msgr2=0x5640248168f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.804+0000 7f842affd640 1 --2- 172.21.15.2:0/1112764992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564024425d80 0x5640248168f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.804+0000 7f842affd640 1 -- 172.21.15.2:0/1112764992 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564024818320 con 0x56402443e2f0 2024-05-14T09:52:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.804+0000 7f8438a1c740 10 client.0 _ll_register_callbacks cb 0x7f841c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:52:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.804+0000 7f8438a1c740 1 -- 172.21.15.2:0/1112764992 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56402477b1e0 con 0x56402443e2f0 2024-05-14T09:52:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.804+0000 7f842affd640 1 --2- 172.21.15.2:0/1112764992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 0x564024816e30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f84200a6190 tx=0x7f84200a9290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7d84f2ff50fd15b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.804+0000 7f840affd640 10 client.26782 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:52:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.804+0000 7f840affd640 1 -- 172.21.15.2:0/1112764992 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84200a9700 con 0x56402443e2f0 2024-05-14T09:52:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.804+0000 7f840affd640 1 -- 172.21.15.2:0/1112764992 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f8420010a90 con 0x56402443e2f0 2024-05-14T09:52:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.805+0000 7f840affd640 1 -- 172.21.15.2:0/1112764992 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f84200aa2f0 con 0x56402443e2f0 2024-05-14T09:52:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.805+0000 7f8438a1c740 10 client.26782 fetch_fsmap learned FSMap version 61 2024-05-14T09:52:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.805+0000 7f8438a1c740 1 -- 172.21.15.2:0/1112764992 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({fsmap=61}) -- 0x5640248193c0 con 0x56402443e2f0 2024-05-14T09:52:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.806+0000 7f840affd640 1 -- 172.21.15.2:0/1112764992 <== mon.1 v2:172.21.15.29:3300/0 4 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7f84200af130 con 0x56402443e2f0 2024-05-14T09:52:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.806+0000 7f8438a1c740 10 client.26782 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:52:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.806+0000 7f8438a1c740 10 client.26782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:52:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.806+0000 7f8438a1c740 20 client.26782 populate_metadata read hostname 'smithi002' 2024-05-14T09:52:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.806+0000 7f8438a1c740 1 --2- 172.21.15.2:0/1112764992 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5640244671b0 0x56402488a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.806+0000 7f8438a1c740 4 client.26782 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:52:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.806+0000 7f8438a1c740 1 -- 172.21.15.2:0/1112764992 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564024775050 con 0x5640244671b0 2024-05-14T09:52:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.806+0000 7f842b7fe640 1 --2- 172.21.15.2:0/1112764992 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5640244671b0 0x56402488a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.806+0000 7f840affd640 10 client.26782 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:36.806+0000 7f842b7fe640 1 --2- 172.21.15.2:0/1112764992 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5640244671b0 0x56402488a500 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b327a9671533306 server_cookie=bbc6face99df3aa4 in_seq=0 out_seq=0 2024-05-14T09:52:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:36 smithi002 ceph-mon[31557]: pgmap v954: 129 pgs: 129 active+clean; 609 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.3 MiB/s wr, 578 op/s 2024-05-14T09:52:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:36 smithi029 ceph-mon[36249]: pgmap v954: 129 pgs: 129 active+clean; 609 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.3 MiB/s wr, 578 op/s 2024-05-14T09:52:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:36 smithi026 ceph-mon[36056]: pgmap v954: 129 pgs: 129 active+clean; 609 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.3 MiB/s wr, 578 op/s 2024-05-14T09:52:37.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 132626 28.98 MB/sec execute 123 sec latency 506.945 ms 2024-05-14T09:52:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.753+0000 7f840affd640 1 -- 172.21.15.2:0/1112764992 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564024775050 con 0x5640244671b0 2024-05-14T09:52:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.753+0000 7f840affd640 10 client.26782 handle_command_reply: tid=0 2024-05-14T09:52:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.753+0000 7f8438a1c740 1 client.26782 shutdown 2024-05-14T09:52:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.754+0000 7f8438a1c740 1 -- 172.21.15.2:0/1112764992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 msgr2=0x564024816e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.754+0000 7f8438a1c740 1 --2- 172.21.15.2:0/1112764992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 0x564024816e30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f84200a6190 tx=0x7f84200a9290 comp rx=0 tx=0).stop 2024-05-14T09:52:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.754+0000 7f8438a1c740 1 -- 172.21.15.2:0/1112764992 shutdown_connections 2024-05-14T09:52:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.754+0000 7f8438a1c740 1 --2- 172.21.15.2:0/1112764992 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5640244671b0 0x56402488a500 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.754+0000 7f8438a1c740 1 --2- 172.21.15.2:0/1112764992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640244464d0 0x564024817d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.754+0000 7f8438a1c740 1 --2- 172.21.15.2:0/1112764992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56402443e2f0 0x564024816e30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.754+0000 7f8438a1c740 1 --2- 172.21.15.2:0/1112764992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564024425d80 0x5640248168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.754+0000 7f8438a1c740 1 -- 172.21.15.2:0/1112764992 >> 172.21.15.2:0/1112764992 conn(0x56402445b300 msgr2=0x5640248071f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.754+0000 7f840affd640 0 client.26782 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.754+0000 7f8438a1c740 1 -- 172.21.15.2:0/1112764992 shutdown_connections 2024-05-14T09:52:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.754+0000 7f8438a1c740 1 -- 172.21.15.2:0/1112764992 wait complete. 2024-05-14T09:52:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.754+0000 7f8438a1c740 20 client.26782 trim_cache size 0 max 16384 2024-05-14T09:52:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:52:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:52:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:52:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:52:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.755+0000 7f8431b6c640 1 -- 172.21.15.2:0/2856395532 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f84140b6e80 msgr2=0x7f84140b9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.755+0000 7f8431b6c640 1 --2- 172.21.15.2:0/2856395532 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f84140b6e80 0x7f84140b9390 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x564024809c30 tx=0x7f84240095c0 comp rx=0 tx=0).stop 2024-05-14T09:52:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.755+0000 7f8431b6c640 1 -- 172.21.15.2:0/2856395532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f842c1070a0 msgr2=0x7f842c19a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.755+0000 7f8431b6c640 1 --2- 172.21.15.2:0/2856395532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f842c1070a0 0x7f842c19a7d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f842000e9f0 tx=0x7f842000eeb0 comp rx=0 tx=0).stop 2024-05-14T09:52:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.756+0000 7f8431b6c640 1 -- 172.21.15.2:0/2856395532 shutdown_connections 2024-05-14T09:52:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.756+0000 7f8431b6c640 1 --2- 172.21.15.2:0/2856395532 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f84140b6e80 0x7f84140b9390 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.756+0000 7f8431b6c640 1 --2- 172.21.15.2:0/2856395532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f842c109a00 0x7f842c1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.756+0000 7f8431b6c640 1 --2- 172.21.15.2:0/2856395532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f842c1070a0 0x7f842c19a7d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.756+0000 7f8431b6c640 1 --2- 172.21.15.2:0/2856395532 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f842c104740 0x7f842c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.756+0000 7f8431b6c640 1 -- 172.21.15.2:0/2856395532 >> 172.21.15.2:0/2856395532 conn(0x7f842c0fdd40 msgr2=0x7f842c105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.756+0000 7f8431b6c640 1 -- 172.21.15.2:0/2856395532 shutdown_connections 2024-05-14T09:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:37.756+0000 7f8431b6c640 1 -- 172.21.15.2:0/2856395532 wait complete. 2024-05-14T09:52:37.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:37 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:52:37.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:37 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:52:37.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:37 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:52:38.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 133262 29.06 MB/sec execute 124 sec latency 408.019 ms 2024-05-14T09:52:38.865 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:38 smithi029 ceph-mon[36249]: pgmap v955: 129 pgs: 129 active+clean; 620 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.7 MiB/s wr, 501 op/s 2024-05-14T09:52:38.865 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:52:38.865 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:52:38.865 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:52:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:38 smithi002 ceph-mon[31557]: pgmap v955: 129 pgs: 129 active+clean; 620 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.7 MiB/s wr, 501 op/s 2024-05-14T09:52:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:52:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:52:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:52:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:38 smithi026 ceph-mon[36056]: pgmap v955: 129 pgs: 129 active+clean; 620 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.7 MiB/s wr, 501 op/s 2024-05-14T09:52:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:52:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:52:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:52:39.608 INFO:tasks.workunit.client.0.smithi002.stdout: 10 133706 29.01 MB/sec execute 125 sec latency 223.031 ms 2024-05-14T09:52:40.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 133724 28.79 MB/sec execute 126 sec latency 1121.474 ms 2024-05-14T09:52:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:40 smithi002 ceph-mon[31557]: pgmap v956: 129 pgs: 129 active+clean; 623 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.8 MiB/s wr, 517 op/s 2024-05-14T09:52:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:40 smithi029 ceph-mon[36249]: pgmap v956: 129 pgs: 129 active+clean; 623 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.8 MiB/s wr, 517 op/s 2024-05-14T09:52:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:40 smithi026 ceph-mon[36056]: pgmap v956: 129 pgs: 129 active+clean; 623 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.8 MiB/s wr, 517 op/s 2024-05-14T09:52:41.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 133873 28.62 MB/sec execute 127 sec latency 1336.971 ms 2024-05-14T09:52:42.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 134181 28.45 MB/sec execute 128 sec latency 148.966 ms 2024-05-14T09:52:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:42 smithi002 ceph-mon[31557]: pgmap v957: 129 pgs: 129 active+clean; 648 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.6 MiB/s wr, 629 op/s 2024-05-14T09:52:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:42 smithi029 ceph-mon[36249]: pgmap v957: 129 pgs: 129 active+clean; 648 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.6 MiB/s wr, 629 op/s 2024-05-14T09:52:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:42 smithi026 ceph-mon[36056]: pgmap v957: 129 pgs: 129 active+clean; 648 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.6 MiB/s wr, 629 op/s 2024-05-14T09:52:43.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 134412 28.33 MB/sec execute 129 sec latency 333.693 ms 2024-05-14T09:52:44.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 134912 28.36 MB/sec execute 130 sec latency 126.054 ms 2024-05-14T09:52:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:44 smithi002 ceph-mon[31557]: pgmap v958: 129 pgs: 129 active+clean; 667 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.8 MiB/s wr, 659 op/s 2024-05-14T09:52:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:44 smithi029 ceph-mon[36249]: pgmap v958: 129 pgs: 129 active+clean; 667 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.8 MiB/s wr, 659 op/s 2024-05-14T09:52:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:44 smithi026 ceph-mon[36056]: pgmap v958: 129 pgs: 129 active+clean; 667 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.8 MiB/s wr, 659 op/s 2024-05-14T09:52:45.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 135621 28.42 MB/sec execute 131 sec latency 97.163 ms 2024-05-14T09:52:46.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 136338 28.58 MB/sec execute 132 sec latency 77.268 ms 2024-05-14T09:52:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:46 smithi002 ceph-mon[31557]: pgmap v959: 129 pgs: 129 active+clean; 658 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 10 MiB/s wr, 700 op/s 2024-05-14T09:52:46.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:46 smithi029 ceph-mon[36249]: pgmap v959: 129 pgs: 129 active+clean; 658 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 10 MiB/s wr, 700 op/s 2024-05-14T09:52:46.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:46 smithi026 ceph-mon[36056]: pgmap v959: 129 pgs: 129 active+clean; 658 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 10 MiB/s wr, 700 op/s 2024-05-14T09:52:47.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 136848 28.56 MB/sec execute 133 sec latency 70.738 ms 2024-05-14T09:52:48.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 137296 28.49 MB/sec execute 134 sec latency 227.737 ms 2024-05-14T09:52:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:48 smithi002 ceph-mon[31557]: pgmap v960: 129 pgs: 129 active+clean; 623 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 825 op/s 2024-05-14T09:52:48.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:48 smithi029 ceph-mon[36249]: pgmap v960: 129 pgs: 129 active+clean; 623 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 825 op/s 2024-05-14T09:52:48.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:48 smithi026 ceph-mon[36056]: pgmap v960: 129 pgs: 129 active+clean; 623 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 825 op/s 2024-05-14T09:52:49.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 137921 28.47 MB/sec execute 135 sec latency 377.188 ms 2024-05-14T09:52:50.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 138687 28.58 MB/sec execute 136 sec latency 379.975 ms 2024-05-14T09:52:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.883+0000 7f181428b640 1 -- 172.21.15.2:0/2767543676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f180c108fb0 msgr2=0x7f180c10b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.883+0000 7f181428b640 1 --2- 172.21.15.2:0/2767543676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f180c108fb0 0x7f180c10b450 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1800009900 tx=0x7f180002a410 comp rx=0 tx=0).stop 2024-05-14T09:52:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.884+0000 7f181428b640 1 -- 172.21.15.2:0/2767543676 shutdown_connections 2024-05-14T09:52:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.884+0000 7f181428b640 1 --2- 172.21.15.2:0/2767543676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f180c10bae0 0x7f180c10dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.884+0000 7f181428b640 1 --2- 172.21.15.2:0/2767543676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f180c108fb0 0x7f180c10b450 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.884+0000 7f181428b640 1 --2- 172.21.15.2:0/2767543676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180c104100 0x7f180c104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.884+0000 7f181428b640 1 -- 172.21.15.2:0/2767543676 >> 172.21.15.2:0/2767543676 conn(0x7f180c0fd480 msgr2=0x7f180c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.884+0000 7f181428b640 1 -- 172.21.15.2:0/2767543676 shutdown_connections 2024-05-14T09:52:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.884+0000 7f181428b640 1 -- 172.21.15.2:0/2767543676 wait complete. 2024-05-14T09:52:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.884+0000 7f181428b640 1 Processor -- start 2024-05-14T09:52:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.884+0000 7f181428b640 1 -- start start 2024-05-14T09:52:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f181428b640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f180c104100 0x7f180c19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f181428b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180c108fb0 0x7f180c19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f181428b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f180c10bae0 0x7f180c1a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f181428b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f180c111980 con 0x7f180c10bae0 2024-05-14T09:52:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f181428b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f180c111800 con 0x7f180c108fb0 2024-05-14T09:52:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f181428b640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f180c111b00 con 0x7f180c104100 2024-05-14T09:52:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f1812027640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f180c104100 0x7f180c19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f1811826640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180c108fb0 0x7f180c19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f1812027640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f180c104100 0x7f180c19c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:49110/0 (socket says 172.21.15.2:49110) 2024-05-14T09:52:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f1812027640 1 -- 172.21.15.2:0/3133553846 learned_addr learned my addr 172.21.15.2:0/3133553846 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f1812828640 1 --2- 172.21.15.2:0/3133553846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f180c10bae0 0x7f180c1a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f1811826640 1 -- 172.21.15.2:0/3133553846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f180c104100 msgr2=0x7f180c19c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f1811826640 1 --2- 172.21.15.2:0/3133553846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f180c104100 0x7f180c19c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f1811826640 1 -- 172.21.15.2:0/3133553846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f180c10bae0 msgr2=0x7f180c1a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f1811826640 1 --2- 172.21.15.2:0/3133553846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f180c10bae0 0x7f180c1a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f1811826640 1 -- 172.21.15.2:0/3133553846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180c1a3fb0 con 0x7f180c108fb0 2024-05-14T09:52:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.885+0000 7f1812027640 1 --2- 172.21.15.2:0/3133553846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f180c104100 0x7f180c19c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:52:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.886+0000 7f1811826640 1 --2- 172.21.15.2:0/3133553846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180c108fb0 0x7f180c19c9b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1800006010 tx=0x7f1800004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.886+0000 7f180b7fe640 1 -- 172.21.15.2:0/3133553846 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180003f070 con 0x7f180c108fb0 2024-05-14T09:52:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.886+0000 7f181428b640 1 -- 172.21.15.2:0/3133553846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180c1a4240 con 0x7f180c108fb0 2024-05-14T09:52:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.886+0000 7f180b7fe640 1 -- 172.21.15.2:0/3133553846 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1800031910 con 0x7f180c108fb0 2024-05-14T09:52:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.886+0000 7f180b7fe640 1 -- 172.21.15.2:0/3133553846 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800035070 con 0x7f180c108fb0 2024-05-14T09:52:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.886+0000 7f181428b640 1 -- 172.21.15.2:0/3133553846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180c1a4780 con 0x7f180c108fb0 2024-05-14T09:52:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.889+0000 7f180b7fe640 1 -- 172.21.15.2:0/3133553846 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1800043400 con 0x7f180c108fb0 2024-05-14T09:52:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.890+0000 7f181b212740 1 Processor -- start 2024-05-14T09:52:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.891+0000 7f181b212740 1 -- start start 2024-05-14T09:52:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.891+0000 7f180b7fe640 1 --2- 172.21.15.2:0/3133553846 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f17f40b28a0 0x7f17f40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.891+0000 7f1812828640 1 --2- 172.21.15.2:0/3133553846 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f17f40b28a0 0x7f17f40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.891+0000 7f181b212740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6c11b0 0x55b17fa62a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.891+0000 7f181b212740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6982f0 0x55b17fa62fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.891+0000 7f181b212740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6a04d0 0x55b17fa69ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.891+0000 7f181b212740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b17f9c75d0 con 0x55b17f6a04d0 2024-05-14T09:52:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.891+0000 7f181b212740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b17f980890 con 0x55b17f6982f0 2024-05-14T09:52:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.891+0000 7f181b212740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b17f9c3590 con 0x55b17f6c11b0 2024-05-14T09:52:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.891+0000 7f180b7fe640 1 -- 172.21.15.2:0/3133553846 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f1800102b50 con 0x7f180c108fb0 2024-05-14T09:52:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.891+0000 7f1812828640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6a04d0 0x55b17fa69ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.891+0000 7f1812828640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6a04d0 0x55b17fa69ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51126/0 (socket says 172.21.15.2:51126) 2024-05-14T09:52:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.891+0000 7f1812027640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6c11b0 0x55b17fa62a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.891+0000 7f1812828640 1 -- 172.21.15.2:0/2622870867 learned_addr learned my addr 172.21.15.2:0/2622870867 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.891+0000 7f1811826640 1 --2- 172.21.15.2:0/2622870867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6982f0 0x55b17fa62fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.892+0000 7f1812828640 1 --2- 172.21.15.2:0/3133553846 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f17f40b28a0 0x7f17f40b4db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f18040134b0 tx=0x7f1804013970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.892+0000 7f1812027640 1 -- 172.21.15.2:0/2622870867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6982f0 msgr2=0x55b17fa62fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.892+0000 7f1812027640 1 --2- 172.21.15.2:0/2622870867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6982f0 0x55b17fa62fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.892+0000 7f1812027640 1 -- 172.21.15.2:0/2622870867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6a04d0 msgr2=0x55b17fa69ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.892+0000 7f1812027640 1 --2- 172.21.15.2:0/2622870867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6a04d0 0x55b17fa69ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.892+0000 7f1812027640 1 -- 172.21.15.2:0/2622870867 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b17f6d9210 con 0x55b17f6c11b0 2024-05-14T09:52:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.892+0000 7f1812828640 1 --2- 172.21.15.2:0/2622870867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6a04d0 0x55b17fa69ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:52:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.892+0000 7f1812027640 1 --2- 172.21.15.2:0/2622870867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6c11b0 0x55b17fa62a60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f17fc011810 tx=0x7f17fc011c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbc60123b9d4e045 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.892+0000 7f1808bfe640 1 -- 172.21.15.2:0/2622870867 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc01e070 con 0x55b17f6c11b0 2024-05-14T09:52:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.892+0000 7f1808bfe640 1 -- 172.21.15.2:0/2622870867 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17fc00fc60 con 0x55b17f6c11b0 2024-05-14T09:52:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.892+0000 7f1808bfe640 1 -- 172.21.15.2:0/2622870867 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc011e60 con 0x55b17f6c11b0 2024-05-14T09:52:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.893+0000 7f181b212740 1 -- 172.21.15.2:0/2622870867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6c11b0 msgr2=0x55b17fa62a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.893+0000 7f181b212740 1 --2- 172.21.15.2:0/2622870867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6c11b0 0x55b17fa62a60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f17fc011810 tx=0x7f17fc011c10 comp rx=0 tx=0).stop 2024-05-14T09:52:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.893+0000 7f181b212740 1 -- 172.21.15.2:0/2622870867 shutdown_connections 2024-05-14T09:52:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.893+0000 7f181b212740 1 --2- 172.21.15.2:0/2622870867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6a04d0 0x55b17fa69ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.893+0000 7f181b212740 1 --2- 172.21.15.2:0/2622870867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6982f0 0x55b17fa62fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.893+0000 7f181b212740 1 --2- 172.21.15.2:0/2622870867 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6c11b0 0x55b17fa62a60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.894+0000 7f181b212740 1 -- 172.21.15.2:0/2622870867 >> 172.21.15.2:0/2622870867 conn(0x55b17f6b5300 msgr2=0x55b17fa5e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.894+0000 7f181b212740 1 -- 172.21.15.2:0/2622870867 shutdown_connections 2024-05-14T09:52:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.894+0000 7f181b212740 1 -- 172.21.15.2:0/2622870867 wait complete. 2024-05-14T09:52:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.894+0000 7f181b212740 1 -- start start 2024-05-14T09:52:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.894+0000 7f181b212740 1 Processor -- start 2024-05-14T09:52:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.894+0000 7f181b212740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6982f0 0x55b17fa6c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.894+0000 7f181b212740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6a04d0 0x55b17fa6cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.894+0000 7f181b212740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6c11b0 0x55b17fa73ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.894+0000 7f181b212740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b17fa6a930 con 0x55b17f6982f0 2024-05-14T09:52:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.894+0000 7f181b212740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b17f984370 con 0x55b17f6c11b0 2024-05-14T09:52:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.894+0000 7f181b212740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55b17fa6aab0 con 0x55b17f6a04d0 2024-05-14T09:52:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.894+0000 7f181b212740 10 client.0 _ll_register_callbacks cb 0x55b17f6d8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:52:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f1811826640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6a04d0 0x55b17fa6cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f1812027640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6982f0 0x55b17fa6c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f1812027640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6982f0 0x55b17fa6c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51130/0 (socket says 172.21.15.2:51130) 2024-05-14T09:52:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f1812828640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6c11b0 0x55b17fa73ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f1812027640 1 -- 172.21.15.2:0/2233407709 learned_addr learned my addr 172.21.15.2:0/2233407709 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f1812828640 1 -- 172.21.15.2:0/2233407709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6a04d0 msgr2=0x55b17fa6cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f1812828640 1 --2- 172.21.15.2:0/2233407709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6a04d0 0x55b17fa6cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f1812828640 1 -- 172.21.15.2:0/2233407709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6982f0 msgr2=0x55b17fa6c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f1812828640 1 --2- 172.21.15.2:0/2233407709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6982f0 0x55b17fa6c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f1812828640 1 -- 172.21.15.2:0/2233407709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b17f7a5620 con 0x55b17f6c11b0 2024-05-14T09:52:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f1811826640 1 --2- 172.21.15.2:0/2233407709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6a04d0 0x55b17fa6cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:52:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f1812828640 1 --2- 172.21.15.2:0/2233407709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6c11b0 0x55b17fa73ab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f180400f930 tx=0x7f1804004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be85b893001d641c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f17f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:52:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f17f1ffb640 1 -- 172.21.15.2:0/2233407709 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804010d40 con 0x55b17f6c11b0 2024-05-14T09:52:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f181b212740 1 -- 172.21.15.2:0/2233407709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17fc00fc60 con 0x55b17f6c11b0 2024-05-14T09:52:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.895+0000 7f17f1ffb640 1 -- 172.21.15.2:0/2233407709 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f1804005020 con 0x55b17f6c11b0 2024-05-14T09:52:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.896+0000 7f17f1ffb640 1 -- 172.21.15.2:0/2233407709 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1804005220 con 0x55b17f6c11b0 2024-05-14T09:52:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.896+0000 7f181b212740 10 client.26786 fetch_fsmap learned FSMap version 61 2024-05-14T09:52:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.896+0000 7f181b212740 1 -- 172.21.15.2:0/2233407709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55b17fa76a50 con 0x55b17f6c11b0 2024-05-14T09:52:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.896+0000 7f17f1ffb640 1 -- 172.21.15.2:0/2233407709 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17fc00fc60 con 0x55b17f6c11b0 2024-05-14T09:52:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.896+0000 7f17f1ffb640 1 -- 172.21.15.2:0/2233407709 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7f180401f070 con 0x55b17f6c11b0 2024-05-14T09:52:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.896+0000 7f181b212740 10 client.26786 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:52:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.896+0000 7f181b212740 10 client.26786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:52:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.896+0000 7f181b212740 20 client.26786 populate_metadata read hostname 'smithi002' 2024-05-14T09:52:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.896+0000 7f181b212740 1 --2- 172.21.15.2:0/2233407709 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b17f67fd80 0x55b17fa7a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.896+0000 7f181b212740 4 client.26786 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:52:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.896+0000 7f181b212740 1 -- 172.21.15.2:0/2233407709 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b17f9d51e0 con 0x55b17f67fd80 2024-05-14T09:52:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.896+0000 7f1812027640 1 --2- 172.21.15.2:0/2233407709 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b17f67fd80 0x55b17fa7a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.897+0000 7f1812027640 1 --2- 172.21.15.2:0/2233407709 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b17f67fd80 0x55b17fa7a1a0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f13e9e714731aae7 server_cookie=56e81103dfcb56e6 in_seq=0 out_seq=0 2024-05-14T09:52:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.897+0000 7f17f1ffb640 10 client.26786 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.899+0000 7f17f1ffb640 1 -- 172.21.15.2:0/2233407709 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b17f9d51e0 con 0x55b17f67fd80 2024-05-14T09:52:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.899+0000 7f17f1ffb640 10 client.26786 handle_command_reply: tid=0 2024-05-14T09:52:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.899+0000 7f181b212740 1 client.26786 shutdown 2024-05-14T09:52:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.899+0000 7f181b212740 1 -- 172.21.15.2:0/2233407709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6c11b0 msgr2=0x55b17fa73ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.899+0000 7f181b212740 1 --2- 172.21.15.2:0/2233407709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6c11b0 0x55b17fa73ab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f180400f930 tx=0x7f1804004730 comp rx=0 tx=0).stop 2024-05-14T09:52:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.899+0000 7f181b212740 1 -- 172.21.15.2:0/2233407709 shutdown_connections 2024-05-14T09:52:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.899+0000 7f181b212740 1 --2- 172.21.15.2:0/2233407709 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b17f67fd80 0x55b17fa7a1a0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.899+0000 7f17f1ffb640 0 client.26786 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.899+0000 7f181b212740 1 --2- 172.21.15.2:0/2233407709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6c11b0 0x55b17fa73ab0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.899+0000 7f181b212740 1 --2- 172.21.15.2:0/2233407709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6a04d0 0x55b17fa6cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.899+0000 7f181b212740 1 --2- 172.21.15.2:0/2233407709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6982f0 0x55b17fa6c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.899+0000 7f181b212740 1 -- 172.21.15.2:0/2233407709 >> 172.21.15.2:0/2233407709 conn(0x55b17f6b5300 msgr2=0x55b17fa61300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.899+0000 7f181b212740 1 -- 172.21.15.2:0/2233407709 shutdown_connections 2024-05-14T09:52:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.900+0000 7f181b212740 1 -- 172.21.15.2:0/2233407709 wait complete. 2024-05-14T09:52:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.900+0000 7f181b212740 20 client.26786 trim_cache size 0 max 16384 2024-05-14T09:52:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:50 smithi002 ceph-mon[31557]: pgmap v961: 129 pgs: 129 active+clean; 625 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 741 op/s 2024-05-14T09:52:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.924+0000 7f181b212740 1 Processor -- start 2024-05-14T09:52:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.925+0000 7f181b212740 1 -- start start 2024-05-14T09:52:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.925+0000 7f181b212740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6982f0 0x55b17fa5dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.925+0000 7f181b212740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6a04d0 0x55b17fa739d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.926+0000 7f181b212740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6c11b0 0x55b17fa5e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.926+0000 7f1812027640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6982f0 0x55b17fa5dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.926+0000 7f1812027640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6982f0 0x55b17fa5dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:49134/0 (socket says 172.21.15.2:49134) 2024-05-14T09:52:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.926+0000 7f1812027640 1 -- 172.21.15.2:0/2234683022 learned_addr learned my addr 172.21.15.2:0/2234683022 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.926+0000 7f1812828640 1 --2- 172.21.15.2:0/2234683022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6c11b0 0x55b17fa5e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.926+0000 7f181b212740 1 -- 172.21.15.2:0/2234683022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1ad310 con 0x55b17f6a04d0 2024-05-14T09:52:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.926+0000 7f1811826640 1 --2- 172.21.15.2:0/2234683022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6a04d0 0x55b17fa739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.926+0000 7f181b212740 1 -- 172.21.15.2:0/2234683022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1aabe0 con 0x55b17f6c11b0 2024-05-14T09:52:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.926+0000 7f181b212740 1 -- 172.21.15.2:0/2234683022 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1a8a30 con 0x55b17f6982f0 2024-05-14T09:52:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.927+0000 7f1812027640 1 -- 172.21.15.2:0/2234683022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6c11b0 msgr2=0x55b17fa5e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.927+0000 7f1812027640 1 --2- 172.21.15.2:0/2234683022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6c11b0 0x55b17fa5e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.927+0000 7f1812027640 1 -- 172.21.15.2:0/2234683022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6a04d0 msgr2=0x55b17fa739d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.927+0000 7f1812027640 1 --2- 172.21.15.2:0/2234683022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6a04d0 0x55b17fa739d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.927+0000 7f1812027640 1 -- 172.21.15.2:0/2234683022 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b17fa5ed00 con 0x55b17f6982f0 2024-05-14T09:52:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.928+0000 7f1812027640 1 --2- 172.21.15.2:0/2234683022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6982f0 0x55b17fa5dc00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f17fc005c40 tx=0x7f17fc00f9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c108dd6f55e32594 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.928+0000 7f17f3fff640 1 -- 172.21.15.2:0/2234683022 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc01e040 con 0x55b17f6982f0 2024-05-14T09:52:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.929+0000 7f17f3fff640 1 -- 172.21.15.2:0/2234683022 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17fc003e20 con 0x55b17f6982f0 2024-05-14T09:52:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.929+0000 7f17f3fff640 1 -- 172.21.15.2:0/2234683022 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc019790 con 0x55b17f6982f0 2024-05-14T09:52:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.930+0000 7f181b212740 1 -- 172.21.15.2:0/2234683022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6982f0 msgr2=0x55b17fa5dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.930+0000 7f181b212740 1 --2- 172.21.15.2:0/2234683022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6982f0 0x55b17fa5dc00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f17fc005c40 tx=0x7f17fc00f9a0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.930+0000 7f181b212740 1 -- 172.21.15.2:0/2234683022 shutdown_connections 2024-05-14T09:52:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.930+0000 7f181b212740 1 --2- 172.21.15.2:0/2234683022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6c11b0 0x55b17fa5e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.930+0000 7f181b212740 1 --2- 172.21.15.2:0/2234683022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6a04d0 0x55b17fa739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.930+0000 7f181b212740 1 --2- 172.21.15.2:0/2234683022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6982f0 0x55b17fa5dc00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.930+0000 7f181b212740 1 -- 172.21.15.2:0/2234683022 >> 172.21.15.2:0/2234683022 conn(0x55b17f6b5300 msgr2=0x55b17fa66a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.930+0000 7f181b212740 1 -- 172.21.15.2:0/2234683022 shutdown_connections 2024-05-14T09:52:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.930+0000 7f181b212740 1 -- 172.21.15.2:0/2234683022 wait complete. 2024-05-14T09:52:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.930+0000 7f181b212740 1 -- start start 2024-05-14T09:52:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.931+0000 7f181b212740 1 Processor -- start 2024-05-14T09:52:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.931+0000 7f181b212740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6982f0 0x55b17fa70d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.931+0000 7f181b212740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6a04d0 0x55b17fa712d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.931+0000 7f1812027640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6982f0 0x55b17fa70d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.931+0000 7f1812027640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6982f0 0x55b17fa70d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:48602/0 (socket says 172.21.15.2:48602) 2024-05-14T09:52:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.931+0000 7f181b212740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6c11b0 0x55b17fa72200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.931+0000 7f1811826640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6a04d0 0x55b17fa712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.931+0000 7f1811826640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6a04d0 0x55b17fa712d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:49142/0 (socket says 172.21.15.2:49142) 2024-05-14T09:52:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.931+0000 7f1812027640 1 -- 172.21.15.2:0/566089615 learned_addr learned my addr 172.21.15.2:0/566089615 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:52:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.931+0000 7f181b212740 1 -- 172.21.15.2:0/566089615 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1a90a0 con 0x55b17f6c11b0 2024-05-14T09:52:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.931+0000 7f181b212740 1 -- 172.21.15.2:0/566089615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1aa510 con 0x55b17f6982f0 2024-05-14T09:52:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.931+0000 7f181b212740 1 -- 172.21.15.2:0/566089615 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1a9770 con 0x55b17f6a04d0 2024-05-14T09:52:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.932+0000 7f181b212740 10 client.0 _ll_register_callbacks cb 0x7f1804012360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.932+0000 7f1812828640 1 --2- 172.21.15.2:0/566089615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6c11b0 0x55b17fa72200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.933+0000 7f1812828640 1 -- 172.21.15.2:0/566089615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6a04d0 msgr2=0x55b17fa712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.933+0000 7f1812828640 1 --2- 172.21.15.2:0/566089615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6a04d0 0x55b17fa712d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.933+0000 7f1812828640 1 -- 172.21.15.2:0/566089615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6982f0 msgr2=0x55b17fa70d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.933+0000 7f1812828640 1 --2- 172.21.15.2:0/566089615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6982f0 0x55b17fa70d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.933+0000 7f1812828640 1 -- 172.21.15.2:0/566089615 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b17fa727c0 con 0x55b17f6c11b0 2024-05-14T09:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.933+0000 7f1812828640 1 --2- 172.21.15.2:0/566089615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6c11b0 0x55b17fa72200 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f180400ffd0 tx=0x7f1804010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120a146e5f5ec799 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.933+0000 7f17f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:52:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.934+0000 7f17f1ffb640 1 -- 172.21.15.2:0/566089615 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180403a060 con 0x55b17f6c11b0 2024-05-14T09:52:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.934+0000 7f17f1ffb640 1 -- 172.21.15.2:0/566089615 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f18040287f0 con 0x55b17f6c11b0 2024-05-14T09:52:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.934+0000 7f181b212740 1 -- 172.21.15.2:0/566089615 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17fc003e20 con 0x55b17f6c11b0 2024-05-14T09:52:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.935+0000 7f17f1ffb640 1 -- 172.21.15.2:0/566089615 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f18040289f0 con 0x55b17f6c11b0 2024-05-14T09:52:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.935+0000 7f181b212740 10 client.17172 fetch_fsmap learned FSMap version 61 2024-05-14T09:52:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.935+0000 7f181b212740 1 -- 172.21.15.2:0/566089615 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55b17fa73ac0 con 0x55b17f6c11b0 2024-05-14T09:52:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.936+0000 7f17f1ffb640 1 -- 172.21.15.2:0/566089615 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1804028b90 con 0x55b17f6c11b0 2024-05-14T09:52:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.936+0000 7f17f1ffb640 1 -- 172.21.15.2:0/566089615 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7f180401f040 con 0x55b17f6c11b0 2024-05-14T09:52:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.936+0000 7f181b212740 10 client.17172 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:52:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.936+0000 7f181b212740 10 client.17172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:52:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.936+0000 7f181b212740 20 client.17172 populate_metadata read hostname 'smithi002' 2024-05-14T09:52:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.937+0000 7f181b212740 1 --2- 172.21.15.2:0/566089615 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b17f6551c0 0x55b17faf5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:52:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.937+0000 7f181b212740 4 client.17172 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:52:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.937+0000 7f181b212740 1 -- 172.21.15.2:0/566089615 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f17fc003e20 con 0x55b17f6551c0 2024-05-14T09:52:50.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:50 smithi029 ceph-mon[36249]: pgmap v961: 129 pgs: 129 active+clean; 625 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 741 op/s 2024-05-14T09:52:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.938+0000 7f1812027640 1 --2- 172.21.15.2:0/566089615 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b17f6551c0 0x55b17faf5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:52:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.940+0000 7f1812027640 1 --2- 172.21.15.2:0/566089615 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b17f6551c0 0x55b17faf5b00 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36da10b07193e015 server_cookie=31b37fe53856e42 in_seq=0 out_seq=0 2024-05-14T09:52:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:50.940+0000 7f17f1ffb640 10 client.17172 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:50 smithi026 ceph-mon[36056]: pgmap v961: 129 pgs: 129 active+clean; 625 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 741 op/s 2024-05-14T09:52:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.407+0000 7f17f1ffb640 1 -- 172.21.15.2:0/566089615 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f17fc003e20 con 0x55b17f6551c0 2024-05-14T09:52:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.407+0000 7f17f1ffb640 10 client.17172 handle_command_reply: tid=0 2024-05-14T09:52:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.407+0000 7f181b212740 1 client.17172 shutdown 2024-05-14T09:52:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.408+0000 7f181b212740 1 -- 172.21.15.2:0/566089615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6c11b0 msgr2=0x55b17fa72200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.408+0000 7f181b212740 1 --2- 172.21.15.2:0/566089615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6c11b0 0x55b17fa72200 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f180400ffd0 tx=0x7f1804010d70 comp rx=0 tx=0).stop 2024-05-14T09:52:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.408+0000 7f181b212740 1 -- 172.21.15.2:0/566089615 shutdown_connections 2024-05-14T09:52:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.408+0000 7f181b212740 1 --2- 172.21.15.2:0/566089615 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55b17f6551c0 0x55b17faf5b00 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.408+0000 7f17f1ffb640 0 client.17172 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:52:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.408+0000 7f181b212740 1 --2- 172.21.15.2:0/566089615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17f6c11b0 0x55b17fa72200 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.408+0000 7f181b212740 1 --2- 172.21.15.2:0/566089615 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55b17f6a04d0 0x55b17fa712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.408+0000 7f181b212740 1 --2- 172.21.15.2:0/566089615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b17f6982f0 0x55b17fa70d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.409+0000 7f181b212740 1 -- 172.21.15.2:0/566089615 >> 172.21.15.2:0/566089615 conn(0x55b17f6b5300 msgr2=0x55b17fa611f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.409+0000 7f181b212740 1 -- 172.21.15.2:0/566089615 shutdown_connections 2024-05-14T09:52:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.409+0000 7f181b212740 1 -- 172.21.15.2:0/566089615 wait complete. 2024-05-14T09:52:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.409+0000 7f181b212740 20 client.17172 trim_cache size 0 max 16384 2024-05-14T09:52:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:52:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:52:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:52:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:52:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.410+0000 7f181428b640 1 -- 172.21.15.2:0/3133553846 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f17f40b28a0 msgr2=0x7f17f40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.410+0000 7f181428b640 1 --2- 172.21.15.2:0/3133553846 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f17f40b28a0 0x7f17f40b4db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f18040134b0 tx=0x7f1804013970 comp rx=0 tx=0).stop 2024-05-14T09:52:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.410+0000 7f181428b640 1 -- 172.21.15.2:0/3133553846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180c108fb0 msgr2=0x7f180c19c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:52:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.410+0000 7f181428b640 1 --2- 172.21.15.2:0/3133553846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180c108fb0 0x7f180c19c9b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1800006010 tx=0x7f1800004270 comp rx=0 tx=0).stop 2024-05-14T09:52:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.410+0000 7f181428b640 1 -- 172.21.15.2:0/3133553846 shutdown_connections 2024-05-14T09:52:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.410+0000 7f181428b640 1 --2- 172.21.15.2:0/3133553846 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f17f40b28a0 0x7f17f40b4db0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.410+0000 7f181428b640 1 --2- 172.21.15.2:0/3133553846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f180c10bae0 0x7f180c1a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.410+0000 7f181428b640 1 --2- 172.21.15.2:0/3133553846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180c108fb0 0x7f180c19c9b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.410+0000 7f181428b640 1 --2- 172.21.15.2:0/3133553846 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f180c104100 0x7f180c19c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:52:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.410+0000 7f181428b640 1 -- 172.21.15.2:0/3133553846 >> 172.21.15.2:0/3133553846 conn(0x7f180c0fd480 msgr2=0x7f180c109f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:52:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.410+0000 7f181428b640 1 -- 172.21.15.2:0/3133553846 shutdown_connections 2024-05-14T09:52:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:52:51.411+0000 7f181428b640 1 -- 172.21.15.2:0/3133553846 wait complete. 2024-05-14T09:52:51.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 139269 28.61 MB/sec execute 137 sec latency 365.090 ms 2024-05-14T09:52:52.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 139920 28.69 MB/sec execute 138 sec latency 116.809 ms 2024-05-14T09:52:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:52 smithi002 ceph-mon[31557]: pgmap v962: 129 pgs: 129 active+clean; 628 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 696 op/s 2024-05-14T09:52:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:52 smithi029 ceph-mon[36249]: pgmap v962: 129 pgs: 129 active+clean; 628 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 696 op/s 2024-05-14T09:52:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:52 smithi026 ceph-mon[36056]: pgmap v962: 129 pgs: 129 active+clean; 628 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 696 op/s 2024-05-14T09:52:53.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 140445 28.68 MB/sec execute 139 sec latency 102.122 ms 2024-05-14T09:52:54.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 140807 28.62 MB/sec execute 140 sec latency 406.466 ms 2024-05-14T09:52:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:54 smithi002 ceph-mon[31557]: pgmap v963: 129 pgs: 129 active+clean; 622 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.5 MiB/s wr, 694 op/s 2024-05-14T09:52:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:54 smithi029 ceph-mon[36249]: pgmap v963: 129 pgs: 129 active+clean; 622 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.5 MiB/s wr, 694 op/s 2024-05-14T09:52:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:54 smithi026 ceph-mon[36056]: pgmap v963: 129 pgs: 129 active+clean; 622 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.5 MiB/s wr, 694 op/s 2024-05-14T09:52:55.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 141442 28.62 MB/sec execute 141 sec latency 101.377 ms 2024-05-14T09:52:56.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 142206 28.75 MB/sec execute 142 sec latency 47.595 ms 2024-05-14T09:52:56.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:56 smithi002 ceph-mon[31557]: pgmap v964: 129 pgs: 129 active+clean; 615 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.6 MiB/s wr, 603 op/s 2024-05-14T09:52:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:56 smithi029 ceph-mon[36249]: pgmap v964: 129 pgs: 129 active+clean; 615 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.6 MiB/s wr, 603 op/s 2024-05-14T09:52:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:56 smithi026 ceph-mon[36056]: pgmap v964: 129 pgs: 129 active+clean; 615 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.6 MiB/s wr, 603 op/s 2024-05-14T09:52:57.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 142873 28.82 MB/sec execute 143 sec latency 86.481 ms 2024-05-14T09:52:58.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 143432 28.84 MB/sec execute 144 sec latency 84.146 ms 2024-05-14T09:52:58.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:52:58 smithi002 ceph-mon[31557]: pgmap v965: 129 pgs: 129 active+clean; 626 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.5 MiB/s wr, 650 op/s 2024-05-14T09:52:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:52:58 smithi029 ceph-mon[36249]: pgmap v965: 129 pgs: 129 active+clean; 626 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.5 MiB/s wr, 650 op/s 2024-05-14T09:52:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:52:58 smithi026 ceph-mon[36056]: pgmap v965: 129 pgs: 129 active+clean; 626 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.5 MiB/s wr, 650 op/s 2024-05-14T09:52:59.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 143796 28.72 MB/sec execute 145 sec latency 326.801 ms 2024-05-14T09:53:00.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 143902 28.55 MB/sec execute 146 sec latency 689.469 ms 2024-05-14T09:53:00.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:00 smithi002 ceph-mon[31557]: pgmap v966: 129 pgs: 129 active+clean; 628 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.3 MiB/s wr, 506 op/s 2024-05-14T09:53:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:00 smithi029 ceph-mon[36249]: pgmap v966: 129 pgs: 129 active+clean; 628 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.3 MiB/s wr, 506 op/s 2024-05-14T09:53:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:00 smithi026 ceph-mon[36056]: pgmap v966: 129 pgs: 129 active+clean; 628 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.3 MiB/s wr, 506 op/s 2024-05-14T09:53:01.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 143994 28.40 MB/sec execute 147 sec latency 1559.234 ms 2024-05-14T09:53:02.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 144332 28.30 MB/sec execute 148 sec latency 1802.238 ms 2024-05-14T09:53:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:02 smithi002 ceph-mon[31557]: pgmap v967: 129 pgs: 129 active+clean; 647 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.1 MiB/s wr, 504 op/s 2024-05-14T09:53:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:02 smithi029 ceph-mon[36249]: pgmap v967: 129 pgs: 129 active+clean; 647 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.1 MiB/s wr, 504 op/s 2024-05-14T09:53:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:02 smithi026 ceph-mon[36056]: pgmap v967: 129 pgs: 129 active+clean; 647 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.1 MiB/s wr, 504 op/s 2024-05-14T09:53:03.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 144612 28.25 MB/sec execute 149 sec latency 119.543 ms 2024-05-14T09:53:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.555+0000 7f8842100640 1 -- 172.21.15.2:0/256656215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f883c103db0 msgr2=0x7f883c1041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.555+0000 7f8842100640 1 --2- 172.21.15.2:0/256656215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f883c103db0 0x7f883c1041c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8830009860 tx=0x7f883002a340 comp rx=0 tx=0).stop 2024-05-14T09:53:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.555+0000 7f8842100640 1 -- 172.21.15.2:0/256656215 shutdown_connections 2024-05-14T09:53:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.555+0000 7f8842100640 1 --2- 172.21.15.2:0/256656215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f883c108ab0 0x7f883c10aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.555+0000 7f8842100640 1 --2- 172.21.15.2:0/256656215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f883c103db0 0x7f883c1041c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.555+0000 7f8842100640 1 --2- 172.21.15.2:0/256656215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f883c102b70 0x7f883c103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.555+0000 7f8842100640 1 -- 172.21.15.2:0/256656215 >> 172.21.15.2:0/256656215 conn(0x7f883c0fdd60 msgr2=0x7f883c100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.555+0000 7f8842100640 1 -- 172.21.15.2:0/256656215 shutdown_connections 2024-05-14T09:53:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.555+0000 7f8842100640 1 -- 172.21.15.2:0/256656215 wait complete. 2024-05-14T09:53:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.556+0000 7f8842100640 1 Processor -- start 2024-05-14T09:53:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.556+0000 7f8842100640 1 -- start start 2024-05-14T09:53:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f8842100640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f883c102b70 0x7f883c19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f8842100640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f883c103db0 0x7f883c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f8842100640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f883c108ab0 0x7f883c1a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f8842100640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f883c10e690 con 0x7f883c108ab0 2024-05-14T09:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f8842100640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f883c10e510 con 0x7f883c103db0 2024-05-14T09:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f8842100640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f883c10e810 con 0x7f883c102b70 2024-05-14T09:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f883bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f883c108ab0 0x7f883c1a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f883b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f883c102b70 0x7f883c19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f883b7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f883c102b70 0x7f883c19c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:44990/0 (socket says 172.21.15.2:44990) 2024-05-14T09:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f883bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f883c108ab0 0x7f883c1a3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33946/0 (socket says 172.21.15.2:33946) 2024-05-14T09:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f883bfff640 1 -- 172.21.15.2:0/966277902 learned_addr learned my addr 172.21.15.2:0/966277902 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f883affd640 1 --2- 172.21.15.2:0/966277902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f883c103db0 0x7f883c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f883b7fe640 1 -- 172.21.15.2:0/966277902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f883c103db0 msgr2=0x7f883c19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f883b7fe640 1 --2- 172.21.15.2:0/966277902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f883c103db0 0x7f883c19c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f883b7fe640 1 -- 172.21.15.2:0/966277902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f883c108ab0 msgr2=0x7f883c1a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f883b7fe640 1 --2- 172.21.15.2:0/966277902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f883c108ab0 0x7f883c1a3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f883b7fe640 1 -- 172.21.15.2:0/966277902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883c1a3fe0 con 0x7f883c102b70 2024-05-14T09:53:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.557+0000 7f883b7fe640 1 --2- 172.21.15.2:0/966277902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f883c102b70 0x7f883c19c4a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f883c102440 tx=0x7f882c00ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.558+0000 7f8838ff9640 1 -- 172.21.15.2:0/966277902 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c00cc90 con 0x7f883c102b70 2024-05-14T09:53:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.558+0000 7f8842100640 1 -- 172.21.15.2:0/966277902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883c1a42d0 con 0x7f883c102b70 2024-05-14T09:53:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.558+0000 7f8842100640 1 -- 172.21.15.2:0/966277902 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883c1a4820 con 0x7f883c102b70 2024-05-14T09:53:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.558+0000 7f8838ff9640 1 -- 172.21.15.2:0/966277902 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f882c00ee70 con 0x7f883c102b70 2024-05-14T09:53:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.558+0000 7f8838ff9640 1 -- 172.21.15.2:0/966277902 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c00cc90 con 0x7f883c102b70 2024-05-14T09:53:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.560+0000 7f8848fbb740 1 Processor -- start 2024-05-14T09:53:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.560+0000 7f8848fbb740 1 -- start start 2024-05-14T09:53:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.560+0000 7f8848fbb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0d81b0 0x563fee479a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.560+0000 7f8848fbb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 0x563fee479fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.560+0000 7f8848fbb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 0x563fee480ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.561+0000 7f8848fbb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563fee3de5d0 con 0x563fee0af2f0 2024-05-14T09:53:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.561+0000 7f8848fbb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563fee397890 con 0x563fee0d81b0 2024-05-14T09:53:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.561+0000 7f8848fbb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563fee3da590 con 0x563fee0b74d0 2024-05-14T09:53:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.561+0000 7f883bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0d81b0 0x563fee479a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.561+0000 7f883affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 0x563fee480ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.561+0000 7f883bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0d81b0 0x563fee479a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:56808/0 (socket says 172.21.15.2:56808) 2024-05-14T09:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.561+0000 7f883bfff640 1 -- 172.21.15.2:0/2269217235 learned_addr learned my addr 172.21.15.2:0/2269217235 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.561+0000 7f883bfff640 1 -- 172.21.15.2:0/2269217235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 msgr2=0x563fee480ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.561+0000 7f883bfff640 1 --2- 172.21.15.2:0/2269217235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 0x563fee480ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.561+0000 7f883bfff640 1 -- 172.21.15.2:0/2269217235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 msgr2=0x563fee479fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.561+0000 7f883bfff640 1 --2- 172.21.15.2:0/2269217235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 0x563fee479fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.561+0000 7f883bfff640 1 -- 172.21.15.2:0/2269217235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fee0f0210 con 0x563fee0d81b0 2024-05-14T09:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.561+0000 7f883affd640 1 --2- 172.21.15.2:0/2269217235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 0x563fee480ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.561+0000 7f883bfff640 1 --2- 172.21.15.2:0/2269217235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0d81b0 0x563fee479a60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8834011570 tx=0x7f8834011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30757f48a2199976 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.562+0000 7f8821ffb640 1 -- 172.21.15.2:0/2269217235 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834004b20 con 0x563fee0d81b0 2024-05-14T09:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.562+0000 7f8821ffb640 1 -- 172.21.15.2:0/2269217235 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8834004cc0 con 0x563fee0d81b0 2024-05-14T09:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.562+0000 7f8821ffb640 1 -- 172.21.15.2:0/2269217235 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834005490 con 0x563fee0d81b0 2024-05-14T09:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.562+0000 7f8838ff9640 1 -- 172.21.15.2:0/966277902 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f882c0035f0 con 0x7f883c102b70 2024-05-14T09:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.563+0000 7f8838ff9640 1 --2- 172.21.15.2:0/966277902 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f88240b2a90 0x7f88240b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.563+0000 7f8848fbb740 1 -- 172.21.15.2:0/2269217235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0d81b0 msgr2=0x563fee479a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.563+0000 7f8848fbb740 1 --2- 172.21.15.2:0/2269217235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0d81b0 0x563fee479a60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8834011570 tx=0x7f8834011a30 comp rx=0 tx=0).stop 2024-05-14T09:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.563+0000 7f883b7fe640 1 --2- 172.21.15.2:0/966277902 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f88240b2a90 0x7f88240b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.563+0000 7f8838ff9640 1 -- 172.21.15.2:0/966277902 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f882c0e2360 con 0x7f883c102b70 2024-05-14T09:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.564+0000 7f8848fbb740 1 -- 172.21.15.2:0/2269217235 shutdown_connections 2024-05-14T09:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.564+0000 7f8848fbb740 1 --2- 172.21.15.2:0/2269217235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 0x563fee480ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.564+0000 7f8848fbb740 1 --2- 172.21.15.2:0/2269217235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 0x563fee479fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.564+0000 7f8848fbb740 1 --2- 172.21.15.2:0/2269217235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0d81b0 0x563fee479a60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.564+0000 7f8848fbb740 1 -- 172.21.15.2:0/2269217235 >> 172.21.15.2:0/2269217235 conn(0x563fee0cc300 msgr2=0x563fee4753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.564+0000 7f8848fbb740 1 -- 172.21.15.2:0/2269217235 shutdown_connections 2024-05-14T09:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.564+0000 7f883b7fe640 1 --2- 172.21.15.2:0/966277902 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f88240b2a90 0x7f88240b4fa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f882c0a67d0 tx=0x7f882c00ce30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.564+0000 7f8848fbb740 1 -- 172.21.15.2:0/2269217235 wait complete. 2024-05-14T09:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.564+0000 7f8848fbb740 1 -- start start 2024-05-14T09:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.565+0000 7f8848fbb740 1 Processor -- start 2024-05-14T09:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.565+0000 7f8848fbb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0af2f0 0x563fee483550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.565+0000 7f8848fbb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 0x563fee483a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f8848fbb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0d81b0 0x563fee48a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f8848fbb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563fee481930 con 0x563fee0d81b0 2024-05-14T09:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f8848fbb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563fee39b370 con 0x563fee0af2f0 2024-05-14T09:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f8848fbb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563fee481ab0 con 0x563fee0b74d0 2024-05-14T09:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f883bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0af2f0 0x563fee483550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f883affd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 0x563fee483a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f883bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0af2f0 0x563fee483550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:56820/0 (socket says 172.21.15.2:56820) 2024-05-14T09:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f883bfff640 1 -- 172.21.15.2:0/868062124 learned_addr learned my addr 172.21.15.2:0/868062124 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f883bfff640 1 --2- 172.21.15.2:0/868062124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0d81b0 0x563fee48a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f8848fbb740 10 client.0 _ll_register_callbacks cb 0x563fee0efa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f883affd640 1 -- 172.21.15.2:0/868062124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0af2f0 msgr2=0x563fee483550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f883affd640 1 --2- 172.21.15.2:0/868062124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0af2f0 0x563fee483550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f883affd640 1 -- 172.21.15.2:0/868062124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0d81b0 msgr2=0x563fee48a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f883affd640 1 --2- 172.21.15.2:0/868062124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0d81b0 0x563fee48a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f883affd640 1 -- 172.21.15.2:0/868062124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fee1bc620 con 0x563fee0b74d0 2024-05-14T09:53:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f883bfff640 1 --2- 172.21.15.2:0/868062124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0d81b0 0x563fee48a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:53:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f8848fbb740 1 -- 172.21.15.2:0/868062124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8834004cc0 con 0x563fee0b74d0 2024-05-14T09:53:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f883affd640 1 --2- 172.21.15.2:0/868062124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 0x563fee483a90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f883c19d540 tx=0x7f8830030fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71f574d98bc2ddc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.566+0000 7f881affd640 10 client.26812 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:53:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.567+0000 7f881affd640 1 -- 172.21.15.2:0/868062124 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883004b050 con 0x563fee0b74d0 2024-05-14T09:53:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.567+0000 7f881affd640 1 -- 172.21.15.2:0/868062124 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f8830004240 con 0x563fee0b74d0 2024-05-14T09:53:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.567+0000 7f881affd640 1 -- 172.21.15.2:0/868062124 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8830041070 con 0x563fee0b74d0 2024-05-14T09:53:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.567+0000 7f8848fbb740 10 client.26812 fetch_fsmap learned FSMap version 61 2024-05-14T09:53:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.567+0000 7f8848fbb740 1 -- 172.21.15.2:0/868062124 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({fsmap=61}) -- 0x563fee48d850 con 0x563fee0b74d0 2024-05-14T09:53:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.567+0000 7f881affd640 1 -- 172.21.15.2:0/868062124 <== mon.1 v2:172.21.15.29:3300/0 4 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7f88300035f0 con 0x563fee0b74d0 2024-05-14T09:53:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.567+0000 7f8848fbb740 10 client.26812 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.568+0000 7f8848fbb740 10 client.26812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.568+0000 7f8848fbb740 20 client.26812 populate_metadata read hostname 'smithi002' 2024-05-14T09:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.568+0000 7f8848fbb740 1 --2- 172.21.15.2:0/868062124 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563fee096d80 0x563fee490f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.568+0000 7f8848fbb740 4 client.26812 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.568+0000 7f8848fbb740 1 -- 172.21.15.2:0/868062124 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fee3ec1e0 con 0x563fee096d80 2024-05-14T09:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.568+0000 7f883bfff640 1 --2- 172.21.15.2:0/868062124 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563fee096d80 0x563fee490f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.568+0000 7f883bfff640 1 --2- 172.21.15.2:0/868062124 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563fee096d80 0x563fee490f50 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a85549624d53ced server_cookie=64f7d3d201096084 in_seq=0 out_seq=0 2024-05-14T09:53:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.568+0000 7f881affd640 10 client.26812 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.573+0000 7f881affd640 1 -- 172.21.15.2:0/868062124 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563fee3ec1e0 con 0x563fee096d80 2024-05-14T09:53:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.573+0000 7f881affd640 10 client.26812 handle_command_reply: tid=0 2024-05-14T09:53:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.574+0000 7f8848fbb740 1 client.26812 shutdown 2024-05-14T09:53:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.574+0000 7f8848fbb740 1 -- 172.21.15.2:0/868062124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 msgr2=0x563fee483a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.574+0000 7f8848fbb740 1 --2- 172.21.15.2:0/868062124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 0x563fee483a90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f883c19d540 tx=0x7f8830030fc0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.574+0000 7f8848fbb740 1 -- 172.21.15.2:0/868062124 shutdown_connections 2024-05-14T09:53:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.574+0000 7f8848fbb740 1 --2- 172.21.15.2:0/868062124 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563fee096d80 0x563fee490f50 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.574+0000 7f8848fbb740 1 --2- 172.21.15.2:0/868062124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0d81b0 0x563fee48a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.574+0000 7f881affd640 0 client.26812 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.574+0000 7f8848fbb740 1 --2- 172.21.15.2:0/868062124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 0x563fee483a90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.574+0000 7f8848fbb740 1 --2- 172.21.15.2:0/868062124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0af2f0 0x563fee483550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.574+0000 7f8848fbb740 1 -- 172.21.15.2:0/868062124 >> 172.21.15.2:0/868062124 conn(0x563fee0cc300 msgr2=0x563fee4743a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.575+0000 7f8848fbb740 1 -- 172.21.15.2:0/868062124 shutdown_connections 2024-05-14T09:53:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.575+0000 7f8848fbb740 1 -- 172.21.15.2:0/868062124 wait complete. 2024-05-14T09:53:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.575+0000 7f8848fbb740 20 client.26812 trim_cache size 0 max 16384 2024-05-14T09:53:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.593+0000 7f8848fbb740 1 Processor -- start 2024-05-14T09:53:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.593+0000 7f8848fbb740 1 -- start start 2024-05-14T09:53:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.593+0000 7f8848fbb740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee096d80 0x563fee4827e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.593+0000 7f8848fbb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 0x563fee48a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.593+0000 7f8848fbb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 0x563fee495740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.593+0000 7f8848fbb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8828012600 con 0x563fee0af2f0 2024-05-14T09:53:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.593+0000 7f8848fbb740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f882800fed0 con 0x563fee096d80 2024-05-14T09:53:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.593+0000 7f8848fbb740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f882800dd20 con 0x563fee0b74d0 2024-05-14T09:53:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.594+0000 7f883affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 0x563fee48a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.594+0000 7f883bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee096d80 0x563fee4827e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.594+0000 7f883affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 0x563fee48a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33964/0 (socket says 172.21.15.2:33964) 2024-05-14T09:53:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.594+0000 7f883affd640 1 -- 172.21.15.2:0/669693586 learned_addr learned my addr 172.21.15.2:0/669693586 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.594+0000 7f883bfff640 1 --2- 172.21.15.2:0/669693586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 0x563fee495740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.594+0000 7f883affd640 1 -- 172.21.15.2:0/669693586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 msgr2=0x563fee495740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.594+0000 7f883affd640 1 --2- 172.21.15.2:0/669693586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 0x563fee495740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.594+0000 7f883affd640 1 -- 172.21.15.2:0/669693586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee096d80 msgr2=0x563fee4827e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.594+0000 7f883affd640 1 --2- 172.21.15.2:0/669693586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee096d80 0x563fee4827e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.594+0000 7f883affd640 1 -- 172.21.15.2:0/669693586 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fee475d00 con 0x563fee0af2f0 2024-05-14T09:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.594+0000 7f883affd640 1 --2- 172.21.15.2:0/669693586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 0x563fee48a910 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f88300303f0 tx=0x7f883003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc62aa5791ed9c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.594+0000 7f88217fa640 1 -- 172.21.15.2:0/669693586 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883004b030 con 0x563fee0af2f0 2024-05-14T09:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.594+0000 7f88217fa640 1 -- 172.21.15.2:0/669693586 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8834004cc0 con 0x563fee0af2f0 2024-05-14T09:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.595+0000 7f88217fa640 1 -- 172.21.15.2:0/669693586 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830037040 con 0x563fee0af2f0 2024-05-14T09:53:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.596+0000 7f8848fbb740 1 -- 172.21.15.2:0/669693586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 msgr2=0x563fee48a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.596+0000 7f8848fbb740 1 --2- 172.21.15.2:0/669693586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 0x563fee48a910 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f88300303f0 tx=0x7f883003d000 comp rx=0 tx=0).stop 2024-05-14T09:53:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.596+0000 7f8848fbb740 1 -- 172.21.15.2:0/669693586 shutdown_connections 2024-05-14T09:53:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.596+0000 7f8848fbb740 1 --2- 172.21.15.2:0/669693586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee0b74d0 0x563fee495740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.596+0000 7f8848fbb740 1 --2- 172.21.15.2:0/669693586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 0x563fee48a910 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.596+0000 7f8848fbb740 1 --2- 172.21.15.2:0/669693586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee096d80 0x563fee4827e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.596+0000 7f8848fbb740 1 -- 172.21.15.2:0/669693586 >> 172.21.15.2:0/669693586 conn(0x563fee0cc300 msgr2=0x563fee47d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.596+0000 7f8848fbb740 1 -- 172.21.15.2:0/669693586 shutdown_connections 2024-05-14T09:53:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.596+0000 7f8848fbb740 1 -- 172.21.15.2:0/669693586 wait complete. 2024-05-14T09:53:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.596+0000 7f8848fbb740 1 -- start start 2024-05-14T09:53:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.596+0000 7f8848fbb740 1 Processor -- start 2024-05-14T09:53:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.597+0000 7f8848fbb740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee096d80 0x563fee488130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.597+0000 7f8848fbb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 0x563fee488670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.597+0000 7f883bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee096d80 0x563fee488130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f883bfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee096d80 0x563fee488130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:45020/0 (socket says 172.21.15.2:45020) 2024-05-14T09:53:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f883bfff640 1 -- 172.21.15.2:0/2765626644 learned_addr learned my addr 172.21.15.2:0/2765626644 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f8848fbb740 1 --2- 172.21.15.2:0/2765626644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0b74d0 0x563fee4895a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f8848fbb740 1 -- 172.21.15.2:0/2765626644 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f882800e390 con 0x563fee0af2f0 2024-05-14T09:53:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f8848fbb740 1 -- 172.21.15.2:0/2765626644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f882800f800 con 0x563fee0b74d0 2024-05-14T09:53:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f883affd640 1 --2- 172.21.15.2:0/2765626644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 0x563fee488670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f8848fbb740 1 -- 172.21.15.2:0/2765626644 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f882800ea60 con 0x563fee096d80 2024-05-14T09:53:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f8848fbb740 10 client.0 _ll_register_callbacks cb 0x7f8830009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:53:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f883bfff640 1 --2- 172.21.15.2:0/2765626644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0b74d0 0x563fee4895a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f883bfff640 1 -- 172.21.15.2:0/2765626644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0b74d0 msgr2=0x563fee4895a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f883bfff640 1 --2- 172.21.15.2:0/2765626644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0b74d0 0x563fee4895a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f883bfff640 1 -- 172.21.15.2:0/2765626644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 msgr2=0x563fee488670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f883bfff640 1 --2- 172.21.15.2:0/2765626644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 0x563fee488670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f883bfff640 1 -- 172.21.15.2:0/2765626644 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fee489b60 con 0x563fee096d80 2024-05-14T09:53:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f883affd640 1 --2- 172.21.15.2:0/2765626644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 0x563fee488670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:53:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f883bfff640 1 --2- 172.21.15.2:0/2765626644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee096d80 0x563fee488130 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8834011570 tx=0x7f883402ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9d89d4f5120976d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f881affd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:53:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.598+0000 7f881affd640 1 -- 172.21.15.2:0/2765626644 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883400fcd0 con 0x563fee096d80 2024-05-14T09:53:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.599+0000 7f8848fbb740 1 -- 172.21.15.2:0/2765626644 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8834004cc0 con 0x563fee096d80 2024-05-14T09:53:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.599+0000 7f881affd640 1 -- 172.21.15.2:0/2765626644 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f8834016040 con 0x563fee096d80 2024-05-14T09:53:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.599+0000 7f881affd640 1 -- 172.21.15.2:0/2765626644 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f883402e3b0 con 0x563fee096d80 2024-05-14T09:53:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.599+0000 7f8848fbb740 10 client.26818 fetch_fsmap learned FSMap version 61 2024-05-14T09:53:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.599+0000 7f8848fbb740 1 -- 172.21.15.2:0/2765626644 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x563fee501b10 con 0x563fee096d80 2024-05-14T09:53:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.599+0000 7f881affd640 1 -- 172.21.15.2:0/2765626644 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8834004cc0 con 0x563fee096d80 2024-05-14T09:53:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.600+0000 7f881affd640 1 -- 172.21.15.2:0/2765626644 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7f883402ece0 con 0x563fee096d80 2024-05-14T09:53:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.600+0000 7f8848fbb740 10 client.26818 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:53:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.600+0000 7f8848fbb740 10 client.26818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:53:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.600+0000 7f8848fbb740 20 client.26818 populate_metadata read hostname 'smithi002' 2024-05-14T09:53:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.600+0000 7f8848fbb740 1 --2- 172.21.15.2:0/2765626644 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563fee0d81b0 0x563fee5041b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.600+0000 7f8848fbb740 4 client.26818 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:53:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.600+0000 7f8848fbb740 1 -- 172.21.15.2:0/2765626644 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563fee3e6050 con 0x563fee0d81b0 2024-05-14T09:53:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.600+0000 7f883affd640 1 --2- 172.21.15.2:0/2765626644 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563fee0d81b0 0x563fee5041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.601+0000 7f881affd640 10 client.26818 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.601+0000 7f883affd640 1 --2- 172.21.15.2:0/2765626644 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563fee0d81b0 0x563fee5041b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a681d63f122ce332 server_cookie=819796a507ea4b2b in_seq=0 out_seq=0 2024-05-14T09:53:04.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 145112 28.24 MB/sec execute 150 sec latency 284.242 ms 2024-05-14T09:53:04.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:04 smithi002 ceph-mon[31557]: pgmap v968: 129 pgs: 129 active+clean; 671 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.0 MiB/s wr, 624 op/s 2024-05-14T09:53:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:04 smithi029 ceph-mon[36249]: pgmap v968: 129 pgs: 129 active+clean; 671 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.0 MiB/s wr, 624 op/s 2024-05-14T09:53:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:04 smithi026 ceph-mon[36056]: pgmap v968: 129 pgs: 129 active+clean; 671 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.0 MiB/s wr, 624 op/s 2024-05-14T09:53:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.956+0000 7f881affd640 1 -- 172.21.15.2:0/2765626644 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563fee3e6050 con 0x563fee0d81b0 2024-05-14T09:53:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.956+0000 7f881affd640 10 client.26818 handle_command_reply: tid=0 2024-05-14T09:53:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.956+0000 7f8848fbb740 1 client.26818 shutdown 2024-05-14T09:53:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.956+0000 7f8848fbb740 1 -- 172.21.15.2:0/2765626644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee096d80 msgr2=0x563fee488130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.956+0000 7f8848fbb740 1 --2- 172.21.15.2:0/2765626644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee096d80 0x563fee488130 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8834011570 tx=0x7f883402ec70 comp rx=0 tx=0).stop 2024-05-14T09:53:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.956+0000 7f8848fbb740 1 -- 172.21.15.2:0/2765626644 shutdown_connections 2024-05-14T09:53:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.956+0000 7f8848fbb740 1 --2- 172.21.15.2:0/2765626644 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563fee0d81b0 0x563fee5041b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.956+0000 7f8848fbb740 1 --2- 172.21.15.2:0/2765626644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fee0b74d0 0x563fee4895a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.956+0000 7f8848fbb740 1 --2- 172.21.15.2:0/2765626644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563fee0af2f0 0x563fee488670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.956+0000 7f881affd640 0 client.26818 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.956+0000 7f8848fbb740 1 --2- 172.21.15.2:0/2765626644 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563fee096d80 0x563fee488130 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.956+0000 7f8848fbb740 1 -- 172.21.15.2:0/2765626644 >> 172.21.15.2:0/2765626644 conn(0x563fee0cc300 msgr2=0x563fee4781f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.957+0000 7f8848fbb740 1 -- 172.21.15.2:0/2765626644 shutdown_connections 2024-05-14T09:53:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.957+0000 7f8848fbb740 1 -- 172.21.15.2:0/2765626644 wait complete. 2024-05-14T09:53:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.957+0000 7f8848fbb740 20 client.26818 trim_cache size 0 max 16384 2024-05-14T09:53:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:53:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:53:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:53:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:53:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.957+0000 7f8842100640 1 -- 172.21.15.2:0/966277902 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f88240b2a90 msgr2=0x7f88240b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.957+0000 7f8842100640 1 --2- 172.21.15.2:0/966277902 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f88240b2a90 0x7f88240b4fa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f882c0a67d0 tx=0x7f882c00ce30 comp rx=0 tx=0).stop 2024-05-14T09:53:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.957+0000 7f8842100640 1 -- 172.21.15.2:0/966277902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f883c102b70 msgr2=0x7f883c19c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.957+0000 7f8842100640 1 --2- 172.21.15.2:0/966277902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f883c102b70 0x7f883c19c4a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f883c102440 tx=0x7f882c00ec20 comp rx=0 tx=0).stop 2024-05-14T09:53:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.958+0000 7f8842100640 1 -- 172.21.15.2:0/966277902 shutdown_connections 2024-05-14T09:53:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.958+0000 7f8842100640 1 --2- 172.21.15.2:0/966277902 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f88240b2a90 0x7f88240b4fa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.958+0000 7f8842100640 1 --2- 172.21.15.2:0/966277902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f883c108ab0 0x7f883c1a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.958+0000 7f8842100640 1 --2- 172.21.15.2:0/966277902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f883c103db0 0x7f883c19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.958+0000 7f8842100640 1 --2- 172.21.15.2:0/966277902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f883c102b70 0x7f883c19c4a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.958+0000 7f8842100640 1 -- 172.21.15.2:0/966277902 >> 172.21.15.2:0/966277902 conn(0x7f883c0fdd60 msgr2=0x7f883c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.958+0000 7f8842100640 1 -- 172.21.15.2:0/966277902 shutdown_connections 2024-05-14T09:53:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:04.958+0000 7f8842100640 1 -- 172.21.15.2:0/966277902 wait complete. 2024-05-14T09:53:05.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 145692 28.21 MB/sec execute 151 sec latency 222.883 ms 2024-05-14T09:53:06.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 146422 28.40 MB/sec execute 152 sec latency 53.821 ms 2024-05-14T09:53:06.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:06 smithi002 ceph-mon[31557]: pgmap v969: 129 pgs: 129 active+clean; 672 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.5 MiB/s wr, 594 op/s 2024-05-14T09:53:06.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:06 smithi029 ceph-mon[36249]: pgmap v969: 129 pgs: 129 active+clean; 672 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.5 MiB/s wr, 594 op/s 2024-05-14T09:53:06.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:06 smithi026 ceph-mon[36056]: pgmap v969: 129 pgs: 129 active+clean; 672 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.5 MiB/s wr, 594 op/s 2024-05-14T09:53:07.609 INFO:tasks.workunit.client.0.smithi002.stdout: 10 146962 28.38 MB/sec execute 153 sec latency 78.906 ms 2024-05-14T09:53:08.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 147446 28.33 MB/sec execute 154 sec latency 78.151 ms 2024-05-14T09:53:08.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:08 smithi002 ceph-mon[31557]: pgmap v970: 129 pgs: 129 active+clean; 624 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 667 op/s 2024-05-14T09:53:08.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:08 smithi029 ceph-mon[36249]: pgmap v970: 129 pgs: 129 active+clean; 624 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 667 op/s 2024-05-14T09:53:08.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:08 smithi026 ceph-mon[36056]: pgmap v970: 129 pgs: 129 active+clean; 624 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 667 op/s 2024-05-14T09:53:09.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 147806 28.26 MB/sec execute 155 sec latency 369.259 ms 2024-05-14T09:53:10.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 148626 28.37 MB/sec execute 156 sec latency 87.107 ms 2024-05-14T09:53:10.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:10 smithi002 ceph-mon[31557]: pgmap v971: 129 pgs: 129 active+clean; 627 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.8 MiB/s wr, 605 op/s 2024-05-14T09:53:10.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:10 smithi029 ceph-mon[36249]: pgmap v971: 129 pgs: 129 active+clean; 627 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.8 MiB/s wr, 605 op/s 2024-05-14T09:53:10.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:10 smithi026 ceph-mon[36056]: pgmap v971: 129 pgs: 129 active+clean; 627 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.8 MiB/s wr, 605 op/s 2024-05-14T09:53:11.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 149421 28.42 MB/sec execute 157 sec latency 112.223 ms 2024-05-14T09:53:12.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 150053 28.51 MB/sec execute 158 sec latency 66.508 ms 2024-05-14T09:53:12.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:12 smithi002 ceph-mon[31557]: pgmap v972: 129 pgs: 129 active+clean; 631 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 659 op/s 2024-05-14T09:53:12.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:12 smithi029 ceph-mon[36249]: pgmap v972: 129 pgs: 129 active+clean; 631 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 659 op/s 2024-05-14T09:53:12.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:12 smithi026 ceph-mon[36056]: pgmap v972: 129 pgs: 129 active+clean; 631 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 659 op/s 2024-05-14T09:53:13.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 150693 28.59 MB/sec execute 159 sec latency 72.658 ms 2024-05-14T09:53:14.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 151247 28.54 MB/sec execute 160 sec latency 192.766 ms 2024-05-14T09:53:14.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:14 smithi002 ceph-mon[31557]: pgmap v973: 129 pgs: 129 active+clean; 640 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.9 MiB/s wr, 704 op/s 2024-05-14T09:53:14.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:14 smithi029 ceph-mon[36249]: pgmap v973: 129 pgs: 129 active+clean; 640 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.9 MiB/s wr, 704 op/s 2024-05-14T09:53:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:14 smithi026 ceph-mon[36056]: pgmap v973: 129 pgs: 129 active+clean; 640 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.9 MiB/s wr, 704 op/s 2024-05-14T09:53:15.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 151886 28.59 MB/sec execute 161 sec latency 83.173 ms 2024-05-14T09:53:16.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 152573 28.62 MB/sec execute 162 sec latency 79.272 ms 2024-05-14T09:53:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:16 smithi002 ceph-mon[31557]: pgmap v974: 129 pgs: 129 active+clean; 643 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.0 MiB/s wr, 608 op/s 2024-05-14T09:53:16.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:16 smithi029 ceph-mon[36249]: pgmap v974: 129 pgs: 129 active+clean; 643 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.0 MiB/s wr, 608 op/s 2024-05-14T09:53:16.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:16 smithi026 ceph-mon[36056]: pgmap v974: 129 pgs: 129 active+clean; 643 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.0 MiB/s wr, 608 op/s 2024-05-14T09:53:17.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 153259 28.69 MB/sec execute 163 sec latency 88.971 ms 2024-05-14T09:53:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.080+0000 7f4e5db55640 1 -- 172.21.15.2:0/1967420447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e58108a90 msgr2=0x7f4e5810aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.080+0000 7f4e5db55640 1 --2- 172.21.15.2:0/1967420447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e58108a90 0x7f4e5810aeb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009ca0 tx=0x7f4e5002a8c0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.080+0000 7f4e5db55640 1 -- 172.21.15.2:0/1967420447 shutdown_connections 2024-05-14T09:53:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.080+0000 7f4e5db55640 1 --2- 172.21.15.2:0/1967420447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e58108a90 0x7f4e5810aeb0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.080+0000 7f4e5db55640 1 --2- 172.21.15.2:0/1967420447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58103d90 0x7f4e581041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.080+0000 7f4e5db55640 1 --2- 172.21.15.2:0/1967420447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e58102b50 0x7f4e58103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.080+0000 7f4e5db55640 1 -- 172.21.15.2:0/1967420447 >> 172.21.15.2:0/1967420447 conn(0x7f4e580fdd40 msgr2=0x7f4e581001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.080+0000 7f4e5db55640 1 -- 172.21.15.2:0/1967420447 shutdown_connections 2024-05-14T09:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.080+0000 7f4e5db55640 1 -- 172.21.15.2:0/1967420447 wait complete. 2024-05-14T09:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.081+0000 7f4e5db55640 1 Processor -- start 2024-05-14T09:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.081+0000 7f4e5db55640 1 -- start start 2024-05-14T09:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.081+0000 7f4e5db55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e58102b50 0x7f4e5810d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.081+0000 7f4e5db55640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e58103d90 0x7f4e5810bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.081+0000 7f4e5db55640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58108a90 0x7f4e5810c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.081+0000 7f4e5db55640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5810f780 con 0x7f4e58103d90 2024-05-14T09:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.081+0000 7f4e5db55640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5810f600 con 0x7f4e58102b50 2024-05-14T09:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.081+0000 7f4e5db55640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5810f900 con 0x7f4e58108a90 2024-05-14T09:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.081+0000 7f4e4ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e58103d90 0x7f4e5810bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.081+0000 7f4e4ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e58103d90 0x7f4e5810bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49012/0 (socket says 172.21.15.2:49012) 2024-05-14T09:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.081+0000 7f4e4ffff640 1 -- 172.21.15.2:0/3181959436 learned_addr learned my addr 172.21.15.2:0/3181959436 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.081+0000 7f4e577fe640 1 --2- 172.21.15.2:0/3181959436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e58102b50 0x7f4e5810d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.081+0000 7f4e57fff640 1 --2- 172.21.15.2:0/3181959436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58108a90 0x7f4e5810c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.082+0000 7f4e4ffff640 1 -- 172.21.15.2:0/3181959436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58108a90 msgr2=0x7f4e5810c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.082+0000 7f4e4ffff640 1 --2- 172.21.15.2:0/3181959436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58108a90 0x7f4e5810c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.082+0000 7f4e4ffff640 1 -- 172.21.15.2:0/3181959436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e58102b50 msgr2=0x7f4e5810d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.082+0000 7f4e4ffff640 1 --2- 172.21.15.2:0/3181959436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e58102b50 0x7f4e5810d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.082+0000 7f4e4ffff640 1 -- 172.21.15.2:0/3181959436 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5810c6e0 con 0x7f4e58103d90 2024-05-14T09:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.082+0000 7f4e4ffff640 1 --2- 172.21.15.2:0/3181959436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e58103d90 0x7f4e5810bac0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4e4000e9f0 tx=0x7f4e4000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.082+0000 7f4e557fa640 1 -- 172.21.15.2:0/3181959436 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4000cd80 con 0x7f4e58103d90 2024-05-14T09:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.082+0000 7f4e557fa640 1 -- 172.21.15.2:0/3181959436 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e400054d0 con 0x7f4e58103d90 2024-05-14T09:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.082+0000 7f4e5db55640 1 -- 172.21.15.2:0/3181959436 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e581a4380 con 0x7f4e58103d90 2024-05-14T09:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.082+0000 7f4e557fa640 1 -- 172.21.15.2:0/3181959436 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40010690 con 0x7f4e58103d90 2024-05-14T09:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.082+0000 7f4e5db55640 1 -- 172.21.15.2:0/3181959436 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e581a4790 con 0x7f4e58103d90 2024-05-14T09:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.085+0000 7f4e557fa640 1 -- 172.21.15.2:0/3181959436 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4e40010830 con 0x7f4e58103d90 2024-05-14T09:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.085+0000 7f4e64a05740 1 Processor -- start 2024-05-14T09:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.085+0000 7f4e64a05740 1 -- start start 2024-05-14T09:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e64a05740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ce000b1b0 0x558ce03aca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e64a05740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffe22f0 0x558ce03acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e64a05740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffea4d0 0x558ce03b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e64a05740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558ce03115d0 con 0x558cdffea4d0 2024-05-14T09:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e64a05740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558ce02ca890 con 0x558ce000b1b0 2024-05-14T09:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e64a05740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558ce030d590 con 0x558cdffe22f0 2024-05-14T09:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e4ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffea4d0 0x558ce03b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffe22f0 0x558ce03acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e57fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ce000b1b0 0x558ce03aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e4ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffea4d0 0x558ce03b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49014/0 (socket says 172.21.15.2:49014) 2024-05-14T09:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e4ffff640 1 -- 172.21.15.2:0/2931900423 learned_addr learned my addr 172.21.15.2:0/2931900423 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e57fff640 1 -- 172.21.15.2:0/2931900423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffe22f0 msgr2=0x558ce03acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e57fff640 1 --2- 172.21.15.2:0/2931900423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffe22f0 0x558ce03acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e57fff640 1 -- 172.21.15.2:0/2931900423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffea4d0 msgr2=0x558ce03b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e57fff640 1 --2- 172.21.15.2:0/2931900423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffea4d0 0x558ce03b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e57fff640 1 -- 172.21.15.2:0/2931900423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce0023210 con 0x558ce000b1b0 2024-05-14T09:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e577fe640 1 --2- 172.21.15.2:0/2931900423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffe22f0 0x558ce03acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.086+0000 7f4e4ffff640 1 --2- 172.21.15.2:0/2931900423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffea4d0 0x558ce03b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:53:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.087+0000 7f4e57fff640 1 --2- 172.21.15.2:0/2931900423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ce000b1b0 0x558ce03aca60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4e50031b50 tx=0x7f4e5002f410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16f6d2f6f14db99b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.087+0000 7f4e4dffb640 1 -- 172.21.15.2:0/2931900423 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50031650 con 0x558ce000b1b0 2024-05-14T09:53:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.087+0000 7f4e4dffb640 1 -- 172.21.15.2:0/2931900423 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e50036030 con 0x558ce000b1b0 2024-05-14T09:53:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.087+0000 7f4e557fa640 1 --2- 172.21.15.2:0/3181959436 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4e440b28a0 0x7f4e440b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.087+0000 7f4e4dffb640 1 -- 172.21.15.2:0/2931900423 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50004660 con 0x558ce000b1b0 2024-05-14T09:53:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.087+0000 7f4e577fe640 1 --2- 172.21.15.2:0/3181959436 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4e440b28a0 0x7f4e440b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.087+0000 7f4e557fa640 1 -- 172.21.15.2:0/3181959436 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f4e400e2a40 con 0x7f4e58103d90 2024-05-14T09:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.088+0000 7f4e64a05740 1 -- 172.21.15.2:0/2931900423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ce000b1b0 msgr2=0x558ce03aca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.088+0000 7f4e577fe640 1 --2- 172.21.15.2:0/3181959436 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4e440b28a0 0x7f4e440b4db0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x558ce03ade40 tx=0x7f4e48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.088+0000 7f4e64a05740 1 --2- 172.21.15.2:0/2931900423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ce000b1b0 0x558ce03aca60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4e50031b50 tx=0x7f4e5002f410 comp rx=0 tx=0).stop 2024-05-14T09:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.088+0000 7f4e64a05740 1 -- 172.21.15.2:0/2931900423 shutdown_connections 2024-05-14T09:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.088+0000 7f4e64a05740 1 --2- 172.21.15.2:0/2931900423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffea4d0 0x558ce03b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.088+0000 7f4e64a05740 1 --2- 172.21.15.2:0/2931900423 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffe22f0 0x558ce03acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.088+0000 7f4e64a05740 1 --2- 172.21.15.2:0/2931900423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ce000b1b0 0x558ce03aca60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.088+0000 7f4e64a05740 1 -- 172.21.15.2:0/2931900423 >> 172.21.15.2:0/2931900423 conn(0x558cdffff300 msgr2=0x558ce03a83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.088+0000 7f4e64a05740 1 -- 172.21.15.2:0/2931900423 shutdown_connections 2024-05-14T09:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.088+0000 7f4e64a05740 1 -- 172.21.15.2:0/2931900423 wait complete. 2024-05-14T09:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.088+0000 7f4e64a05740 1 -- start start 2024-05-14T09:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e64a05740 1 Processor -- start 2024-05-14T09:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e64a05740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558cdffe22f0 0x558ce03b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e64a05740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffea4d0 0x558ce03b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e64a05740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558ce000b1b0 0x558ce03bd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e64a05740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558ce03b4750 con 0x558cdffea4d0 2024-05-14T09:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e64a05740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558ce02ce370 con 0x558cdffe22f0 2024-05-14T09:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e64a05740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558ce03b48d0 con 0x558ce000b1b0 2024-05-14T09:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e4ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffea4d0 0x558ce03b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e4ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffea4d0 0x558ce03b6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49026/0 (socket says 172.21.15.2:49026) 2024-05-14T09:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e64a05740 10 client.0 _ll_register_callbacks cb 0x558ce0022a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e4ffff640 1 -- 172.21.15.2:0/1545815927 learned_addr learned my addr 172.21.15.2:0/1545815927 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e577fe640 1 --2- 172.21.15.2:0/1545815927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558cdffe22f0 0x558ce03b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e57fff640 1 --2- 172.21.15.2:0/1545815927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558ce000b1b0 0x558ce03bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e4ffff640 1 -- 172.21.15.2:0/1545815927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558ce000b1b0 msgr2=0x558ce03bd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e4ffff640 1 --2- 172.21.15.2:0/1545815927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558ce000b1b0 0x558ce03bd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e4ffff640 1 -- 172.21.15.2:0/1545815927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558cdffe22f0 msgr2=0x558ce03b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.089+0000 7f4e4ffff640 1 --2- 172.21.15.2:0/1545815927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558cdffe22f0 0x558ce03b64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.090+0000 7f4e4ffff640 1 -- 172.21.15.2:0/1545815927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce00ef620 con 0x558cdffea4d0 2024-05-14T09:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.090+0000 7f4e4ffff640 1 --2- 172.21.15.2:0/1545815927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffea4d0 0x558ce03b6a30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4e400a5da0 tx=0x7f4e400a5be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de688a1c48e7f65d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.090+0000 7f4e36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.090+0000 7f4e36ffd640 1 -- 172.21.15.2:0/1545815927 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e400a9490 con 0x558cdffea4d0 2024-05-14T09:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.090+0000 7f4e36ffd640 1 -- 172.21.15.2:0/1545815927 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f4e400a9190 con 0x558cdffea4d0 2024-05-14T09:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.090+0000 7f4e64a05740 1 -- 172.21.15.2:0/1545815927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e50036030 con 0x558cdffea4d0 2024-05-14T09:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.090+0000 7f4e36ffd640 1 -- 172.21.15.2:0/1545815927 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f4e400b23e0 con 0x558cdffea4d0 2024-05-14T09:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.090+0000 7f4e64a05740 10 client.17199 fetch_fsmap learned FSMap version 61 2024-05-14T09:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.090+0000 7f4e64a05740 1 -- 172.21.15.2:0/1545815927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x558ce03c0970 con 0x558cdffea4d0 2024-05-14T09:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.090+0000 7f4e36ffd640 1 -- 172.21.15.2:0/1545815927 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e50036030 con 0x558cdffea4d0 2024-05-14T09:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.091+0000 7f4e36ffd640 1 -- 172.21.15.2:0/1545815927 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7f4e400a5690 con 0x558cdffea4d0 2024-05-14T09:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.091+0000 7f4e64a05740 10 client.17199 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.091+0000 7f4e64a05740 10 client.17199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.091+0000 7f4e64a05740 20 client.17199 populate_metadata read hostname 'smithi002' 2024-05-14T09:53:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.091+0000 7f4e64a05740 1 --2- 172.21.15.2:0/1545815927 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558cdffc9d80 0x558ce03c4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.091+0000 7f4e64a05740 4 client.17199 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:53:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.091+0000 7f4e64a05740 1 -- 172.21.15.2:0/1545815927 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ce031f1e0 con 0x558cdffc9d80 2024-05-14T09:53:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.091+0000 7f4e577fe640 1 --2- 172.21.15.2:0/1545815927 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558cdffc9d80 0x558ce03c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.092+0000 7f4e36ffd640 10 client.17199 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.092+0000 7f4e577fe640 1 --2- 172.21.15.2:0/1545815927 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558cdffc9d80 0x558ce03c4550 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=275e1039037e8728 server_cookie=b7171186baa5ef05 in_seq=0 out_seq=0 2024-05-14T09:53:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.093+0000 7f4e36ffd640 1 -- 172.21.15.2:0/1545815927 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ce031f1e0 con 0x558cdffc9d80 2024-05-14T09:53:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.093+0000 7f4e36ffd640 10 client.17199 handle_command_reply: tid=0 2024-05-14T09:53:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.093+0000 7f4e64a05740 1 client.17199 shutdown 2024-05-14T09:53:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.094+0000 7f4e64a05740 1 -- 172.21.15.2:0/1545815927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffea4d0 msgr2=0x558ce03b6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.094+0000 7f4e64a05740 1 --2- 172.21.15.2:0/1545815927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffea4d0 0x558ce03b6a30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4e400a5da0 tx=0x7f4e400a5be0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.094+0000 7f4e64a05740 1 -- 172.21.15.2:0/1545815927 shutdown_connections 2024-05-14T09:53:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.094+0000 7f4e64a05740 1 --2- 172.21.15.2:0/1545815927 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558cdffc9d80 0x558ce03c4550 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.094+0000 7f4e64a05740 1 --2- 172.21.15.2:0/1545815927 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558ce000b1b0 0x558ce03bd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.094+0000 7f4e64a05740 1 --2- 172.21.15.2:0/1545815927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffea4d0 0x558ce03b6a30 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.094+0000 7f4e36ffd640 0 client.17199 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.094+0000 7f4e64a05740 1 --2- 172.21.15.2:0/1545815927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558cdffe22f0 0x558ce03b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.094+0000 7f4e64a05740 1 -- 172.21.15.2:0/1545815927 >> 172.21.15.2:0/1545815927 conn(0x558cdffff300 msgr2=0x558ce03ab300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.094+0000 7f4e64a05740 1 -- 172.21.15.2:0/1545815927 shutdown_connections 2024-05-14T09:53:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.094+0000 7f4e64a05740 1 -- 172.21.15.2:0/1545815927 wait complete. 2024-05-14T09:53:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.094+0000 7f4e64a05740 20 client.17199 trim_cache size 0 max 16384 2024-05-14T09:53:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.116+0000 7f4e64a05740 1 Processor -- start 2024-05-14T09:53:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.116+0000 7f4e64a05740 1 -- start start 2024-05-14T09:53:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.116+0000 7f4e64a05740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffe22f0 0x558ce03c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.116+0000 7f4e64a05740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffea4d0 0x558ce03bd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.116+0000 7f4e64a05740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ce000b1b0 0x558ce03c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.116+0000 7f4e64a05740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581b16e0 con 0x558cdffe22f0 2024-05-14T09:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.116+0000 7f4e64a05740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581aefb0 con 0x558ce000b1b0 2024-05-14T09:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.116+0000 7f4e64a05740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581ace00 con 0x558cdffea4d0 2024-05-14T09:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.116+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffe22f0 0x558ce03c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.116+0000 7f4e57fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ce000b1b0 0x558ce03c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.116+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffe22f0 0x558ce03c8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49038/0 (socket says 172.21.15.2:49038) 2024-05-14T09:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.116+0000 7f4e577fe640 1 -- 172.21.15.2:0/2713110272 learned_addr learned my addr 172.21.15.2:0/2713110272 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.116+0000 7f4e4ffff640 1 --2- 172.21.15.2:0/2713110272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffea4d0 0x558ce03bd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.117+0000 7f4e57fff640 1 -- 172.21.15.2:0/2713110272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffea4d0 msgr2=0x558ce03bd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.117+0000 7f4e57fff640 1 --2- 172.21.15.2:0/2713110272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffea4d0 0x558ce03bd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.117+0000 7f4e57fff640 1 -- 172.21.15.2:0/2713110272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffe22f0 msgr2=0x558ce03c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.117+0000 7f4e57fff640 1 --2- 172.21.15.2:0/2713110272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffe22f0 0x558ce03c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.117+0000 7f4e57fff640 1 -- 172.21.15.2:0/2713110272 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce03a8d00 con 0x558ce000b1b0 2024-05-14T09:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.117+0000 7f4e577fe640 1 --2- 172.21.15.2:0/2713110272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffe22f0 0x558ce03c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.117+0000 7f4e57fff640 1 --2- 172.21.15.2:0/2713110272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ce000b1b0 0x558ce03c9280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4e50031b50 tx=0x7f4e5002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88dc9e0f74a9f6e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.118+0000 7f4e4d7fa640 1 -- 172.21.15.2:0/2713110272 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50055020 con 0x558ce000b1b0 2024-05-14T09:53:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.118+0000 7f4e4d7fa640 1 -- 172.21.15.2:0/2713110272 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e50036040 con 0x558ce000b1b0 2024-05-14T09:53:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.118+0000 7f4e4d7fa640 1 -- 172.21.15.2:0/2713110272 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5004a040 con 0x558ce000b1b0 2024-05-14T09:53:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.119+0000 7f4e64a05740 1 -- 172.21.15.2:0/2713110272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ce000b1b0 msgr2=0x558ce03c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.119+0000 7f4e64a05740 1 --2- 172.21.15.2:0/2713110272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ce000b1b0 0x558ce03c9280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4e50031b50 tx=0x7f4e5002a8c0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.119+0000 7f4e64a05740 1 -- 172.21.15.2:0/2713110272 shutdown_connections 2024-05-14T09:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.119+0000 7f4e64a05740 1 --2- 172.21.15.2:0/2713110272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558ce000b1b0 0x558ce03c9280 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.119+0000 7f4e64a05740 1 --2- 172.21.15.2:0/2713110272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffea4d0 0x558ce03bd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.119+0000 7f4e64a05740 1 --2- 172.21.15.2:0/2713110272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cdffe22f0 0x558ce03c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.119+0000 7f4e64a05740 1 -- 172.21.15.2:0/2713110272 >> 172.21.15.2:0/2713110272 conn(0x558cdffff300 msgr2=0x558ce03b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.119+0000 7f4e64a05740 1 -- 172.21.15.2:0/2713110272 shutdown_connections 2024-05-14T09:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.119+0000 7f4e64a05740 1 -- 172.21.15.2:0/2713110272 wait complete. 2024-05-14T09:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.119+0000 7f4e64a05740 1 -- start start 2024-05-14T09:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.119+0000 7f4e64a05740 1 Processor -- start 2024-05-14T09:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.120+0000 7f4e64a05740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558cdffe22f0 0x558ce03bc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.120+0000 7f4e64a05740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffea4d0 0x558ce03bcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.120+0000 7f4e64a05740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce000b1b0 0x558ce042a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.120+0000 7f4e64a05740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581ad470 con 0x558ce000b1b0 2024-05-14T09:53:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.120+0000 7f4e64a05740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581ae8e0 con 0x558cdffe22f0 2024-05-14T09:53:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.120+0000 7f4e64a05740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581adb40 con 0x558cdffea4d0 2024-05-14T09:53:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.120+0000 7f4e64a05740 10 client.0 _ll_register_callbacks cb 0x7f4e40009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:53:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.120+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558cdffe22f0 0x558ce03bc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.120+0000 7f4e57fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce000b1b0 0x558ce042a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.120+0000 7f4e4ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffea4d0 0x558ce03bcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.120+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558cdffe22f0 0x558ce03bc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:39824/0 (socket says 172.21.15.2:39824) 2024-05-14T09:53:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.120+0000 7f4e577fe640 1 -- 172.21.15.2:0/2460760955 learned_addr learned my addr 172.21.15.2:0/2460760955 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.120+0000 7f4e57fff640 1 -- 172.21.15.2:0/2460760955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffea4d0 msgr2=0x558ce03bcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.120+0000 7f4e57fff640 1 --2- 172.21.15.2:0/2460760955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffea4d0 0x558ce03bcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.121+0000 7f4e57fff640 1 -- 172.21.15.2:0/2460760955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558cdffe22f0 msgr2=0x558ce03bc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.121+0000 7f4e57fff640 1 --2- 172.21.15.2:0/2460760955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558cdffe22f0 0x558ce03bc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.121+0000 7f4e57fff640 1 -- 172.21.15.2:0/2460760955 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce042ae30 con 0x558ce000b1b0 2024-05-14T09:53:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.121+0000 7f4e577fe640 1 --2- 172.21.15.2:0/2460760955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558cdffe22f0 0x558ce03bc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:53:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.121+0000 7f4e57fff640 1 --2- 172.21.15.2:0/2460760955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce000b1b0 0x558ce042a8f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x558ce03b6fe0 tx=0x7f4e50003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25ba169f219b160 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.121+0000 7f4e36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:53:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.121+0000 7f4e36ffd640 1 -- 172.21.15.2:0/2460760955 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50055030 con 0x558ce000b1b0 2024-05-14T09:53:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.121+0000 7f4e64a05740 1 -- 172.21.15.2:0/2460760955 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e50036040 con 0x558ce000b1b0 2024-05-14T09:53:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.121+0000 7f4e36ffd640 1 -- 172.21.15.2:0/2460760955 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f4e50004240 con 0x558ce000b1b0 2024-05-14T09:53:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.122+0000 7f4e36ffd640 1 -- 172.21.15.2:0/2460760955 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f4e50068050 con 0x558ce000b1b0 2024-05-14T09:53:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.122+0000 7f4e64a05740 10 client.17205 fetch_fsmap learned FSMap version 61 2024-05-14T09:53:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.122+0000 7f4e64a05740 1 -- 172.21.15.2:0/2460760955 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x558ce042c060 con 0x558ce000b1b0 2024-05-14T09:53:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.122+0000 7f4e36ffd640 1 -- 172.21.15.2:0/2460760955 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e50036040 con 0x558ce000b1b0 2024-05-14T09:53:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.122+0000 7f4e36ffd640 1 -- 172.21.15.2:0/2460760955 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7f4e50043070 con 0x558ce000b1b0 2024-05-14T09:53:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.122+0000 7f4e64a05740 10 client.17205 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:53:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.122+0000 7f4e64a05740 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:53:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.122+0000 7f4e64a05740 20 client.17205 populate_metadata read hostname 'smithi002' 2024-05-14T09:53:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.122+0000 7f4e64a05740 1 --2- 172.21.15.2:0/2460760955 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558cdff9f1c0 0x558ce042e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.122+0000 7f4e64a05740 4 client.17205 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:53:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.123+0000 7f4e64a05740 1 -- 172.21.15.2:0/2460760955 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ce0319050 con 0x558cdff9f1c0 2024-05-14T09:53:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.123+0000 7f4e577fe640 1 --2- 172.21.15.2:0/2460760955 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558cdff9f1c0 0x558ce042e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.123+0000 7f4e36ffd640 10 client.17205 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.123+0000 7f4e577fe640 1 --2- 172.21.15.2:0/2460760955 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558cdff9f1c0 0x558ce042e820 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=479107bce6e343a4 server_cookie=270c4a33938678bd in_seq=0 out_seq=0 2024-05-14T09:53:18.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 153656 28.66 MB/sec execute 164 sec latency 466.100 ms 2024-05-14T09:53:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:18 smithi002 ceph-mon[31557]: pgmap v975: 129 pgs: 129 active+clean; 650 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.5 MiB/s wr, 578 op/s 2024-05-14T09:53:18.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:18 smithi029 ceph-mon[36249]: pgmap v975: 129 pgs: 129 active+clean; 650 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.5 MiB/s wr, 578 op/s 2024-05-14T09:53:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:18 smithi026 ceph-mon[36056]: pgmap v975: 129 pgs: 129 active+clean; 650 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.5 MiB/s wr, 578 op/s 2024-05-14T09:53:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.979+0000 7f4e36ffd640 1 -- 172.21.15.2:0/2460760955 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ce0319050 con 0x558cdff9f1c0 2024-05-14T09:53:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.979+0000 7f4e36ffd640 10 client.17205 handle_command_reply: tid=0 2024-05-14T09:53:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.980+0000 7f4e64a05740 1 client.17205 shutdown 2024-05-14T09:53:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.980+0000 7f4e64a05740 1 -- 172.21.15.2:0/2460760955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce000b1b0 msgr2=0x558ce042a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.980+0000 7f4e64a05740 1 --2- 172.21.15.2:0/2460760955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce000b1b0 0x558ce042a8f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x558ce03b6fe0 tx=0x7f4e50003dd0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.980+0000 7f4e64a05740 1 -- 172.21.15.2:0/2460760955 shutdown_connections 2024-05-14T09:53:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.981+0000 7f4e64a05740 1 --2- 172.21.15.2:0/2460760955 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558cdff9f1c0 0x558ce042e820 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.981+0000 7f4e64a05740 1 --2- 172.21.15.2:0/2460760955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce000b1b0 0x558ce042a8f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.981+0000 7f4e36ffd640 0 client.17205 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.981+0000 7f4e64a05740 1 --2- 172.21.15.2:0/2460760955 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558cdffea4d0 0x558ce03bcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.981+0000 7f4e64a05740 1 --2- 172.21.15.2:0/2460760955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558cdffe22f0 0x558ce03bc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.981+0000 7f4e64a05740 1 -- 172.21.15.2:0/2460760955 >> 172.21.15.2:0/2460760955 conn(0x558cdffff300 msgr2=0x558ce03ab1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.981+0000 7f4e64a05740 1 -- 172.21.15.2:0/2460760955 shutdown_connections 2024-05-14T09:53:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.981+0000 7f4e64a05740 1 -- 172.21.15.2:0/2460760955 wait complete. 2024-05-14T09:53:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.981+0000 7f4e64a05740 20 client.17205 trim_cache size 0 max 16384 2024-05-14T09:53:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:53:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:53:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:53:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:53:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.982+0000 7f4e5db55640 1 -- 172.21.15.2:0/3181959436 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4e440b28a0 msgr2=0x7f4e440b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.982+0000 7f4e5db55640 1 --2- 172.21.15.2:0/3181959436 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4e440b28a0 0x7f4e440b4db0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x558ce03ade40 tx=0x7f4e48005f90 comp rx=0 tx=0).stop 2024-05-14T09:53:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.982+0000 7f4e5db55640 1 -- 172.21.15.2:0/3181959436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e58103d90 msgr2=0x7f4e5810bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.982+0000 7f4e5db55640 1 --2- 172.21.15.2:0/3181959436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e58103d90 0x7f4e5810bac0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4e4000e9f0 tx=0x7f4e4000edf0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.982+0000 7f4e5db55640 1 -- 172.21.15.2:0/3181959436 shutdown_connections 2024-05-14T09:53:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.982+0000 7f4e5db55640 1 --2- 172.21.15.2:0/3181959436 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f4e440b28a0 0x7f4e440b4db0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.982+0000 7f4e5db55640 1 --2- 172.21.15.2:0/3181959436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e58108a90 0x7f4e5810c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.982+0000 7f4e5db55640 1 --2- 172.21.15.2:0/3181959436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e58103d90 0x7f4e5810bac0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.982+0000 7f4e5db55640 1 --2- 172.21.15.2:0/3181959436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e58102b50 0x7f4e5810d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.982+0000 7f4e5db55640 1 -- 172.21.15.2:0/3181959436 >> 172.21.15.2:0/3181959436 conn(0x7f4e580fdd40 msgr2=0x7f4e58104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.983+0000 7f4e5db55640 1 -- 172.21.15.2:0/3181959436 shutdown_connections 2024-05-14T09:53:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:18.983+0000 7f4e5db55640 1 -- 172.21.15.2:0/3181959436 wait complete. 2024-05-14T09:53:19.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 154198 28.67 MB/sec execute 165 sec latency 600.826 ms 2024-05-14T09:53:20.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 154775 28.67 MB/sec execute 166 sec latency 77.774 ms 2024-05-14T09:53:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:20 smithi002 ceph-mon[31557]: pgmap v976: 129 pgs: 129 active+clean; 653 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.5 MiB/s wr, 470 op/s 2024-05-14T09:53:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:20 smithi029 ceph-mon[36249]: pgmap v976: 129 pgs: 129 active+clean; 653 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.5 MiB/s wr, 470 op/s 2024-05-14T09:53:20.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:20 smithi026 ceph-mon[36056]: pgmap v976: 129 pgs: 129 active+clean; 653 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.5 MiB/s wr, 470 op/s 2024-05-14T09:53:21.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 155394 28.68 MB/sec execute 167 sec latency 77.382 ms 2024-05-14T09:53:22.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 156120 28.74 MB/sec execute 168 sec latency 71.314 ms 2024-05-14T09:53:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:22 smithi002 ceph-mon[31557]: pgmap v977: 129 pgs: 129 active+clean; 651 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 562 op/s 2024-05-14T09:53:22.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:22 smithi029 ceph-mon[36249]: pgmap v977: 129 pgs: 129 active+clean; 651 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 562 op/s 2024-05-14T09:53:22.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:22 smithi026 ceph-mon[36056]: pgmap v977: 129 pgs: 129 active+clean; 651 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 562 op/s 2024-05-14T09:53:23.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 156836 28.84 MB/sec execute 169 sec latency 55.474 ms 2024-05-14T09:53:24.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 157323 28.84 MB/sec execute 170 sec latency 233.703 ms 2024-05-14T09:53:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:24 smithi002 ceph-mon[31557]: pgmap v978: 129 pgs: 129 active+clean; 647 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.8 MiB/s wr, 618 op/s 2024-05-14T09:53:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:24 smithi029 ceph-mon[36249]: pgmap v978: 129 pgs: 129 active+clean; 647 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.8 MiB/s wr, 618 op/s 2024-05-14T09:53:24.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:24 smithi026 ceph-mon[36056]: pgmap v978: 129 pgs: 129 active+clean; 647 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.8 MiB/s wr, 618 op/s 2024-05-14T09:53:25.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 157766 28.79 MB/sec execute 171 sec latency 113.986 ms 2024-05-14T09:53:26.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 158344 28.76 MB/sec execute 172 sec latency 79.562 ms 2024-05-14T09:53:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:26 smithi002 ceph-mon[31557]: pgmap v979: 129 pgs: 129 active+clean; 648 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.1 MiB/s wr, 532 op/s 2024-05-14T09:53:26.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:26 smithi029 ceph-mon[36249]: pgmap v979: 129 pgs: 129 active+clean; 648 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.1 MiB/s wr, 532 op/s 2024-05-14T09:53:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:26 smithi026 ceph-mon[36056]: pgmap v979: 129 pgs: 129 active+clean; 648 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.1 MiB/s wr, 532 op/s 2024-05-14T09:53:27.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 158880 28.75 MB/sec execute 173 sec latency 178.398 ms 2024-05-14T09:53:28.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 159460 28.80 MB/sec execute 174 sec latency 115.791 ms 2024-05-14T09:53:28.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:28 smithi002 ceph-mon[31557]: pgmap v980: 129 pgs: 129 active+clean; 657 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 586 op/s 2024-05-14T09:53:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:28 smithi029 ceph-mon[36249]: pgmap v980: 129 pgs: 129 active+clean; 657 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 586 op/s 2024-05-14T09:53:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:28 smithi026 ceph-mon[36056]: pgmap v980: 129 pgs: 129 active+clean; 657 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 586 op/s 2024-05-14T09:53:29.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 159939 28.78 MB/sec execute 175 sec latency 154.656 ms 2024-05-14T09:53:30.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 160265 28.74 MB/sec execute 176 sec latency 348.134 ms 2024-05-14T09:53:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:30 smithi002 ceph-mon[31557]: pgmap v981: 129 pgs: 129 active+clean; 662 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.4 MiB/s wr, 581 op/s 2024-05-14T09:53:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:30 smithi029 ceph-mon[36249]: pgmap v981: 129 pgs: 129 active+clean; 662 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.4 MiB/s wr, 581 op/s 2024-05-14T09:53:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:30 smithi026 ceph-mon[36056]: pgmap v981: 129 pgs: 129 active+clean; 662 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.4 MiB/s wr, 581 op/s 2024-05-14T09:53:31.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 160284 28.58 MB/sec execute 177 sec latency 1023.049 ms 2024-05-14T09:53:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.152+0000 7efc04b6c640 1 -- 172.21.15.2:0/3814985414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc00118b80 msgr2=0x7efc0011afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.152+0000 7efc04b6c640 1 --2- 172.21.15.2:0/3814985414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc00118b80 0x7efc0011afa0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7efbf401a910 tx=0x7efbf403b470 comp rx=0 tx=0).stop 2024-05-14T09:53:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.157+0000 7efc04b6c640 1 -- 172.21.15.2:0/3814985414 shutdown_connections 2024-05-14T09:53:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.157+0000 7efc04b6c640 1 --2- 172.21.15.2:0/3814985414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc00118b80 0x7efc0011afa0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.157+0000 7efc04b6c640 1 --2- 172.21.15.2:0/3814985414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0006d460 0x7efc00118640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.157+0000 7efc04b6c640 1 --2- 172.21.15.2:0/3814985414 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0006cb10 0x7efc0006cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.157+0000 7efc04b6c640 1 -- 172.21.15.2:0/3814985414 >> 172.21.15.2:0/3814985414 conn(0x7efc00072840 msgr2=0x7efc00109cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.164+0000 7efc04b6c640 1 -- 172.21.15.2:0/3814985414 shutdown_connections 2024-05-14T09:53:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.166+0000 7efc04b6c640 1 -- 172.21.15.2:0/3814985414 wait complete. 2024-05-14T09:53:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.167+0000 7efc04b6c640 1 Processor -- start 2024-05-14T09:53:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.168+0000 7efc04b6c640 1 -- start start 2024-05-14T09:53:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.168+0000 7efc04b6c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0006cb10 0x7efc001a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.168+0000 7efc04b6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0006d460 0x7efc001a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.168+0000 7efc04b6c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc00118b80 0x7efc0007af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.168+0000 7efc04b6c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efc00112950 con 0x7efc00118b80 2024-05-14T09:53:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.168+0000 7efc04b6c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc001127d0 con 0x7efc0006d460 2024-05-14T09:53:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.168+0000 7efc04b6c640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efc00112ad0 con 0x7efc0006cb10 2024-05-14T09:53:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.169+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc00118b80 0x7efc0007af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.169+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc00118b80 0x7efc0007af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37168/0 (socket says 172.21.15.2:37168) 2024-05-14T09:53:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.169+0000 7efbfed9d640 1 -- 172.21.15.2:0/1391582849 learned_addr learned my addr 172.21.15.2:0/1391582849 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.170+0000 7efbfdd9b640 1 --2- 172.21.15.2:0/1391582849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0006d460 0x7efc001a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.171+0000 7efbfed9d640 1 -- 172.21.15.2:0/1391582849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0006cb10 msgr2=0x7efc001a0a10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T09:53:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.171+0000 7efbfed9d640 1 --2- 172.21.15.2:0/1391582849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0006cb10 0x7efc001a0a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.171+0000 7efbfed9d640 1 -- 172.21.15.2:0/1391582849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0006d460 msgr2=0x7efc001a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.171+0000 7efbfed9d640 1 --2- 172.21.15.2:0/1391582849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0006d460 0x7efc001a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.171+0000 7efbfed9d640 1 -- 172.21.15.2:0/1391582849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0007b660 con 0x7efc00118b80 2024-05-14T09:53:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.173+0000 7efbfed9d640 1 --2- 172.21.15.2:0/1391582849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc00118b80 0x7efc0007af30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005ab0 tx=0x7efbf40053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.178+0000 7efbef7fe640 1 -- 172.21.15.2:0/1391582849 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf404f070 con 0x7efc00118b80 2024-05-14T09:53:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.178+0000 7efbef7fe640 1 -- 172.21.15.2:0/1391582849 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf4004b10 con 0x7efc00118b80 2024-05-14T09:53:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.178+0000 7efbef7fe640 1 -- 172.21.15.2:0/1391582849 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4046030 con 0x7efc00118b80 2024-05-14T09:53:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.178+0000 7efc04b6c640 1 -- 172.21.15.2:0/1391582849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0007b890 con 0x7efc00118b80 2024-05-14T09:53:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.181+0000 7efc04b6c640 1 -- 172.21.15.2:0/1391582849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0007bdb0 con 0x7efc00118b80 2024-05-14T09:53:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.192+0000 7efc0ba1c740 1 Processor -- start 2024-05-14T09:53:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.193+0000 7efc0ba1c740 1 -- start start 2024-05-14T09:53:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.193+0000 7efc0ba1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9751b0 0x56024cd16a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.193+0000 7efc0ba1c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c94c2f0 0x56024cd16fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.193+0000 7efc0ba1c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c9544d0 0x56024cd1dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.193+0000 7efc0ba1c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56024cc7b5d0 con 0x56024c94c2f0 2024-05-14T09:53:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.193+0000 7efc0ba1c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56024cc34890 con 0x56024c9751b0 2024-05-14T09:53:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.193+0000 7efc0ba1c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56024cc77590 con 0x56024c9544d0 2024-05-14T09:53:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.196+0000 7efbfe59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c94c2f0 0x56024cd16fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.196+0000 7efbfe59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c94c2f0 0x56024cd16fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37172/0 (socket says 172.21.15.2:37172) 2024-05-14T09:53:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.196+0000 7efbfe59c640 1 -- 172.21.15.2:0/2073722992 learned_addr learned my addr 172.21.15.2:0/2073722992 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.198+0000 7efbfe59c640 1 -- 172.21.15.2:0/2073722992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c9544d0 msgr2=0x56024cd1dec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:53:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.198+0000 7efbfe59c640 1 --2- 172.21.15.2:0/2073722992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c9544d0 0x56024cd1dec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.198+0000 7efbfe59c640 1 -- 172.21.15.2:0/2073722992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9751b0 msgr2=0x56024cd16a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:53:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.198+0000 7efbfe59c640 1 --2- 172.21.15.2:0/2073722992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9751b0 0x56024cd16a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.198+0000 7efbfe59c640 1 -- 172.21.15.2:0/2073722992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56024c98d210 con 0x56024c94c2f0 2024-05-14T09:53:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.199+0000 7efbef7fe640 1 -- 172.21.15.2:0/1391582849 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7efbf4052070 con 0x7efc00118b80 2024-05-14T09:53:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.199+0000 7efbfe59c640 1 --2- 172.21.15.2:0/2073722992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c94c2f0 0x56024cd16fc0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009740 tx=0x7efbf0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123f43e3a6075ea1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.201+0000 7efbef7fe640 1 --2- 172.21.15.2:0/1391582849 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7efbe80b2a70 0x7efbe80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.201+0000 7efbef7fe640 1 -- 172.21.15.2:0/1391582849 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7efbf41137e0 con 0x7efc00118b80 2024-05-14T09:53:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.203+0000 7efbfdd9b640 1 --2- 172.21.15.2:0/1391582849 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7efbe80b2a70 0x7efbe80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.204+0000 7efc04b6c640 1 -- 172.21.15.2:0/2073722992 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf0015020 con 0x56024c94c2f0 2024-05-14T09:53:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.204+0000 7efc04b6c640 1 -- 172.21.15.2:0/2073722992 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf0009e50 con 0x56024c94c2f0 2024-05-14T09:53:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.205+0000 7efc0ba1c740 1 -- 172.21.15.2:0/2073722992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c94c2f0 msgr2=0x56024cd16fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.205+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/2073722992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c94c2f0 0x56024cd16fc0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009740 tx=0x7efbf0009c00 comp rx=0 tx=0).stop 2024-05-14T09:53:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.211+0000 7efc0ba1c740 1 -- 172.21.15.2:0/2073722992 shutdown_connections 2024-05-14T09:53:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.211+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/2073722992 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c9544d0 0x56024cd1dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.211+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/2073722992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c94c2f0 0x56024cd16fc0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.211+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/2073722992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9751b0 0x56024cd16a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.211+0000 7efc0ba1c740 1 -- 172.21.15.2:0/2073722992 >> 172.21.15.2:0/2073722992 conn(0x56024c969300 msgr2=0x56024cd12670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.215+0000 7efc0ba1c740 1 -- 172.21.15.2:0/2073722992 shutdown_connections 2024-05-14T09:53:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.218+0000 7efc0ba1c740 1 -- 172.21.15.2:0/2073722992 wait complete. 2024-05-14T09:53:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.219+0000 7efc0ba1c740 1 -- start start 2024-05-14T09:53:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.219+0000 7efc0ba1c740 1 Processor -- start 2024-05-14T09:53:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.220+0000 7efc0ba1c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c94c2f0 0x56024cd20600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.220+0000 7efc0ba1c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9544d0 0x56024cd20b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.220+0000 7efc0ba1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9751b0 0x56024cd27aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.220+0000 7efc0ba1c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56024cd1e960 con 0x56024c9544d0 2024-05-14T09:53:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.220+0000 7efc0ba1c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56024cc38370 con 0x56024c9751b0 2024-05-14T09:53:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.220+0000 7efc0ba1c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x56024cd1eae0 con 0x56024c94c2f0 2024-05-14T09:53:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.220+0000 7efc0ba1c740 10 client.0 _ll_register_callbacks cb 0x56024c98ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:53:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.221+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9751b0 0x56024cd27aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.221+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9751b0 0x56024cd27aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:48908/0 (socket says 172.21.15.2:48908) 2024-05-14T09:53:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.221+0000 7efbfed9d640 1 -- 172.21.15.2:0/433254327 learned_addr learned my addr 172.21.15.2:0/433254327 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.221+0000 7efbfdd9b640 1 --2- 172.21.15.2:0/433254327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9544d0 0x56024cd20b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.221+0000 7efbfdd9b640 1 --2- 172.21.15.2:0/1391582849 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7efbe80b2a70 0x7efbe80b4f80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x56024cd17c10 tx=0x7efbf80094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.223+0000 7efbfed9d640 1 -- 172.21.15.2:0/433254327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c94c2f0 msgr2=0x56024cd20600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:53:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.223+0000 7efbfed9d640 1 --2- 172.21.15.2:0/433254327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c94c2f0 0x56024cd20600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.223+0000 7efbfed9d640 1 -- 172.21.15.2:0/433254327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9544d0 msgr2=0x56024cd20b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.223+0000 7efbfed9d640 1 --2- 172.21.15.2:0/433254327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9544d0 0x56024cd20b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.223+0000 7efbfed9d640 1 -- 172.21.15.2:0/433254327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56024ca59620 con 0x56024c9751b0 2024-05-14T09:53:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.227+0000 7efbfed9d640 1 --2- 172.21.15.2:0/433254327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9751b0 0x56024cd27aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efbf40da680 tx=0x7efbf40da170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3905a6545ffddd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.227+0000 7efbe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:53:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.230+0000 7efbe5ffb640 1 -- 172.21.15.2:0/433254327 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf40f4060 con 0x56024c9751b0 2024-05-14T09:53:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.230+0000 7efbe5ffb640 1 -- 172.21.15.2:0/433254327 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7efbf40d6c10 con 0x56024c9751b0 2024-05-14T09:53:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.231+0000 7efc0ba1c740 1 -- 172.21.15.2:0/433254327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf0009e50 con 0x56024c9751b0 2024-05-14T09:53:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.235+0000 7efbe5ffb640 1 -- 172.21.15.2:0/433254327 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7efbf4059040 con 0x56024c9751b0 2024-05-14T09:53:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.237+0000 7efc0ba1c740 10 client.26834 fetch_fsmap learned FSMap version 61 2024-05-14T09:53:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.237+0000 7efc0ba1c740 1 -- 172.21.15.2:0/433254327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x56024cd2a9d0 con 0x56024c9751b0 2024-05-14T09:53:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.239+0000 7efbe5ffb640 1 -- 172.21.15.2:0/433254327 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf0009e50 con 0x56024c9751b0 2024-05-14T09:53:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.239+0000 7efbe5ffb640 1 -- 172.21.15.2:0/433254327 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7efbf405c060 con 0x56024c9751b0 2024-05-14T09:53:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.241+0000 7efc0ba1c740 10 client.26834 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:53:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.241+0000 7efc0ba1c740 10 client.26834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:53:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.241+0000 7efc0ba1c740 20 client.26834 populate_metadata read hostname 'smithi002' 2024-05-14T09:53:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.241+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/433254327 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56024c933d80 0x56024cd2e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.241+0000 7efc0ba1c740 4 client.26834 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:53:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.241+0000 7efc0ba1c740 1 -- 172.21.15.2:0/433254327 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56024cc891e0 con 0x56024c933d80 2024-05-14T09:53:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.243+0000 7efbfe59c640 1 --2- 172.21.15.2:0/433254327 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56024c933d80 0x56024cd2e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.270+0000 7efbfe59c640 1 --2- 172.21.15.2:0/433254327 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56024c933d80 0x56024cd2e230 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f035e45903721a4 server_cookie=b44eec6f81d02cdf in_seq=0 out_seq=0 2024-05-14T09:53:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.270+0000 7efbe5ffb640 10 client.26834 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.286+0000 7efbe5ffb640 1 -- 172.21.15.2:0/433254327 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56024cc891e0 con 0x56024c933d80 2024-05-14T09:53:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.286+0000 7efbe5ffb640 10 client.26834 handle_command_reply: tid=0 2024-05-14T09:53:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.288+0000 7efc0ba1c740 1 client.26834 shutdown 2024-05-14T09:53:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.295+0000 7efc0ba1c740 1 -- 172.21.15.2:0/433254327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9751b0 msgr2=0x56024cd27aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.295+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/433254327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9751b0 0x56024cd27aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efbf40da680 tx=0x7efbf40da170 comp rx=0 tx=0).stop 2024-05-14T09:53:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.298+0000 7efc0ba1c740 1 -- 172.21.15.2:0/433254327 shutdown_connections 2024-05-14T09:53:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.298+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/433254327 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56024c933d80 0x56024cd2e230 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.298+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/433254327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9751b0 0x56024cd27aa0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.298+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/433254327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9544d0 0x56024cd20b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.298+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/433254327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c94c2f0 0x56024cd20600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.298+0000 7efc0ba1c740 1 -- 172.21.15.2:0/433254327 >> 172.21.15.2:0/433254327 conn(0x56024c969300 msgr2=0x56024cd15510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.298+0000 7efbe5ffb640 0 client.26834 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.303+0000 7efc0ba1c740 1 -- 172.21.15.2:0/433254327 shutdown_connections 2024-05-14T09:53:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.303+0000 7efc0ba1c740 1 -- 172.21.15.2:0/433254327 wait complete. 2024-05-14T09:53:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.308+0000 7efc0ba1c740 20 client.26834 trim_cache size 0 max 16384 2024-05-14T09:53:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.340+0000 7efc0ba1c740 1 Processor -- start 2024-05-14T09:53:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.342+0000 7efc0ba1c740 1 -- start start 2024-05-14T09:53:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.342+0000 7efc0ba1c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c94c2f0 0x56024cd1f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.342+0000 7efc0ba1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9544d0 0x56024cd12050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.342+0000 7efc0ba1c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9751b0 0x56024cd12590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.342+0000 7efc0ba1c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efc001c6ca0 con 0x56024c9751b0 2024-05-14T09:53:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.342+0000 7efc0ba1c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc001c4570 con 0x56024c9544d0 2024-05-14T09:53:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.342+0000 7efc0ba1c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efc000844a0 con 0x56024c94c2f0 2024-05-14T09:53:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.344+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9751b0 0x56024cd12590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.344+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9751b0 0x56024cd12590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37202/0 (socket says 172.21.15.2:37202) 2024-05-14T09:53:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.344+0000 7efbfed9d640 1 -- 172.21.15.2:0/1266203523 learned_addr learned my addr 172.21.15.2:0/1266203523 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.344+0000 7efbfe59c640 1 --2- 172.21.15.2:0/1266203523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c94c2f0 0x56024cd1f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.347+0000 7efbfed9d640 1 -- 172.21.15.2:0/1266203523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c94c2f0 msgr2=0x56024cd1f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.347+0000 7efbfed9d640 1 --2- 172.21.15.2:0/1266203523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c94c2f0 0x56024cd1f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.347+0000 7efbfed9d640 1 -- 172.21.15.2:0/1266203523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9544d0 msgr2=0x56024cd12050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:53:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.347+0000 7efbfed9d640 1 --2- 172.21.15.2:0/1266203523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9544d0 0x56024cd12050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.347+0000 7efbfed9d640 1 -- 172.21.15.2:0/1266203523 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56024cd10a40 con 0x56024c9751b0 2024-05-14T09:53:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.352+0000 7efbfed9d640 1 --2- 172.21.15.2:0/1266203523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9751b0 0x56024cd12590 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7efbf40da260 tx=0x7efbf40da5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9c606cd48e7631 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.354+0000 7efbe7fff640 1 -- 172.21.15.2:0/1266203523 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf40d6840 con 0x56024c9751b0 2024-05-14T09:53:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.355+0000 7efbe7fff640 1 -- 172.21.15.2:0/1266203523 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf40db080 con 0x56024c9751b0 2024-05-14T09:53:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.355+0000 7efbe7fff640 1 -- 172.21.15.2:0/1266203523 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf40ded30 con 0x56024c9751b0 2024-05-14T09:53:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.357+0000 7efc0ba1c740 1 -- 172.21.15.2:0/1266203523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9751b0 msgr2=0x56024cd12590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.357+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/1266203523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9751b0 0x56024cd12590 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7efbf40da260 tx=0x7efbf40da5d0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.358+0000 7efc0ba1c740 1 -- 172.21.15.2:0/1266203523 shutdown_connections 2024-05-14T09:53:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.358+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/1266203523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9751b0 0x56024cd12590 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.358+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/1266203523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9544d0 0x56024cd12050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.358+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/1266203523 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c94c2f0 0x56024cd1f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.358+0000 7efc0ba1c740 1 -- 172.21.15.2:0/1266203523 >> 172.21.15.2:0/1266203523 conn(0x56024c969300 msgr2=0x56024cd1ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.364+0000 7efc0ba1c740 1 -- 172.21.15.2:0/1266203523 shutdown_connections 2024-05-14T09:53:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.365+0000 7efc0ba1c740 1 -- 172.21.15.2:0/1266203523 wait complete. 2024-05-14T09:53:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.365+0000 7efc0ba1c740 1 -- start start 2024-05-14T09:53:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.366+0000 7efc0ba1c740 1 Processor -- start 2024-05-14T09:53:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.366+0000 7efc0ba1c740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c94c2f0 0x56024cd24a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.366+0000 7efc0ba1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9544d0 0x56024cd24f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.366+0000 7efc0ba1c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9751b0 0x56024cd25eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.366+0000 7efc0ba1c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efc00084b10 con 0x56024c9751b0 2024-05-14T09:53:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.366+0000 7efc0ba1c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efc001c3ea0 con 0x56024c9544d0 2024-05-14T09:53:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.366+0000 7efc0ba1c740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7efc000851e0 con 0x56024c94c2f0 2024-05-14T09:53:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.366+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9751b0 0x56024cd25eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.367+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9751b0 0x56024cd25eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37206/0 (socket says 172.21.15.2:37206) 2024-05-14T09:53:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.367+0000 7efbfed9d640 1 -- 172.21.15.2:0/3638020272 learned_addr learned my addr 172.21.15.2:0/3638020272 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.367+0000 7efc0ba1c740 10 client.0 _ll_register_callbacks cb 0x7efbf401a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:53:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.367+0000 7efbfed9d640 1 -- 172.21.15.2:0/3638020272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c94c2f0 msgr2=0x56024cd24a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:53:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.367+0000 7efbfed9d640 1 --2- 172.21.15.2:0/3638020272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c94c2f0 0x56024cd24a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.367+0000 7efbfed9d640 1 -- 172.21.15.2:0/3638020272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9544d0 msgr2=0x56024cd24f80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:53:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.367+0000 7efbfed9d640 1 --2- 172.21.15.2:0/3638020272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9544d0 0x56024cd24f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.367+0000 7efbfed9d640 1 -- 172.21.15.2:0/3638020272 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56024cd26470 con 0x56024c9751b0 2024-05-14T09:53:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.367+0000 7efbfed9d640 1 --2- 172.21.15.2:0/3638020272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9751b0 0x56024cd25eb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7efbf4006010 tx=0x7efbf40e3580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c59027bbe008a93 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.367+0000 7efbe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:53:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.368+0000 7efbe5ffb640 1 -- 172.21.15.2:0/3638020272 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf40d6840 con 0x56024c9751b0 2024-05-14T09:53:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.369+0000 7efc0ba1c740 1 -- 172.21.15.2:0/3638020272 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf40db080 con 0x56024c9751b0 2024-05-14T09:53:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.369+0000 7efbe5ffb640 1 -- 172.21.15.2:0/3638020272 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7efbf40dadf0 con 0x56024c9751b0 2024-05-14T09:53:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.369+0000 7efbe5ffb640 1 -- 172.21.15.2:0/3638020272 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7efbf40f4020 con 0x56024c9751b0 2024-05-14T09:53:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.371+0000 7efc0ba1c740 10 client.17223 fetch_fsmap learned FSMap version 61 2024-05-14T09:53:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.371+0000 7efc0ba1c740 1 -- 172.21.15.2:0/3638020272 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x56024cd966a0 con 0x56024c9751b0 2024-05-14T09:53:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.375+0000 7efbe5ffb640 1 -- 172.21.15.2:0/3638020272 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf40db080 con 0x56024c9751b0 2024-05-14T09:53:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.375+0000 7efbe5ffb640 1 -- 172.21.15.2:0/3638020272 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 61) ==== 8354+0+0 (secure 0 0 0) 0x7efbf40d62f0 con 0x56024c9751b0 2024-05-14T09:53:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.375+0000 7efc0ba1c740 10 client.17223 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T09:53:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.375+0000 7efc0ba1c740 10 client.17223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:53:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.375+0000 7efc0ba1c740 20 client.17223 populate_metadata read hostname 'smithi002' 2024-05-14T09:53:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.376+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/3638020272 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56024c9091c0 0x56024cd98ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.376+0000 7efc0ba1c740 4 client.17223 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:53:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.376+0000 7efc0ba1c740 1 -- 172.21.15.2:0/3638020272 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56024cc83050 con 0x56024c9091c0 2024-05-14T09:53:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.380+0000 7efbfe59c640 1 --2- 172.21.15.2:0/3638020272 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56024c9091c0 0x56024cd98ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.393+0000 7efbfe59c640 1 --2- 172.21.15.2:0/3638020272 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56024c9091c0 0x56024cd98ce0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c94031e2062f84a server_cookie=7c92758d747f2b6d in_seq=0 out_seq=0 2024-05-14T09:53:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:32.394+0000 7efbe5ffb640 10 client.17223 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:32.610 INFO:tasks.workunit.client.0.smithi002.stdout: 10 160303 28.43 MB/sec execute 178 sec latency 1933.930 ms 2024-05-14T09:53:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:32 smithi002 ceph-mon[31557]: pgmap v982: 129 pgs: 129 active+clean; 669 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.9 MiB/s wr, 578 op/s 2024-05-14T09:53:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:32 smithi029 ceph-mon[36249]: pgmap v982: 129 pgs: 129 active+clean; 669 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.9 MiB/s wr, 578 op/s 2024-05-14T09:53:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:32 smithi026 ceph-mon[36056]: pgmap v982: 129 pgs: 129 active+clean; 669 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.9 MiB/s wr, 578 op/s 2024-05-14T09:53:33.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 160413 28.30 MB/sec execute 179 sec latency 2014.017 ms 2024-05-14T09:53:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.911+0000 7efbe5ffb640 1 -- 172.21.15.2:0/3638020272 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56024cc83050 con 0x56024c9091c0 2024-05-14T09:53:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.911+0000 7efbe5ffb640 10 client.17223 handle_command_reply: tid=0 2024-05-14T09:53:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.911+0000 7efc0ba1c740 1 client.17223 shutdown 2024-05-14T09:53:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.912+0000 7efc0ba1c740 1 -- 172.21.15.2:0/3638020272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9751b0 msgr2=0x56024cd25eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.912+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/3638020272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9751b0 0x56024cd25eb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7efbf4006010 tx=0x7efbf40e3580 comp rx=0 tx=0).stop 2024-05-14T09:53:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.912+0000 7efc0ba1c740 1 -- 172.21.15.2:0/3638020272 shutdown_connections 2024-05-14T09:53:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.912+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/3638020272 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56024c9091c0 0x56024cd98ce0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.912+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/3638020272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56024c9751b0 0x56024cd25eb0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.912+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/3638020272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56024c9544d0 0x56024cd24f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.912+0000 7efc0ba1c740 1 --2- 172.21.15.2:0/3638020272 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56024c94c2f0 0x56024cd24a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.912+0000 7efc0ba1c740 1 -- 172.21.15.2:0/3638020272 >> 172.21.15.2:0/3638020272 conn(0x56024c969300 msgr2=0x56024cd12050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.912+0000 7efbe5ffb640 0 client.17223 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.912+0000 7efc0ba1c740 1 -- 172.21.15.2:0/3638020272 shutdown_connections 2024-05-14T09:53:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.912+0000 7efc0ba1c740 1 -- 172.21.15.2:0/3638020272 wait complete. 2024-05-14T09:53:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.912+0000 7efc0ba1c740 20 client.17223 trim_cache size 0 max 16384 2024-05-14T09:53:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:53:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:53:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:53:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:53:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.913+0000 7efbed7fa640 1 -- 172.21.15.2:0/1391582849 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7efbe80b2a70 msgr2=0x7efbe80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.913+0000 7efbed7fa640 1 --2- 172.21.15.2:0/1391582849 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7efbe80b2a70 0x7efbe80b4f80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x56024cd17c10 tx=0x7efbf80094f0 comp rx=0 tx=0).stop 2024-05-14T09:53:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.913+0000 7efbed7fa640 1 -- 172.21.15.2:0/1391582849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc00118b80 msgr2=0x7efc0007af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.913+0000 7efbed7fa640 1 --2- 172.21.15.2:0/1391582849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc00118b80 0x7efc0007af30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005ab0 tx=0x7efbf40053f0 comp rx=0 tx=0).stop 2024-05-14T09:53:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.913+0000 7efbed7fa640 1 -- 172.21.15.2:0/1391582849 shutdown_connections 2024-05-14T09:53:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.913+0000 7efbed7fa640 1 --2- 172.21.15.2:0/1391582849 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7efbe80b2a70 0x7efbe80b4f80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.913+0000 7efbed7fa640 1 --2- 172.21.15.2:0/1391582849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc00118b80 0x7efc0007af30 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.913+0000 7efbed7fa640 1 --2- 172.21.15.2:0/1391582849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc0006d460 0x7efc001a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.913+0000 7efbed7fa640 1 --2- 172.21.15.2:0/1391582849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efc0006cb10 0x7efc001a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.913+0000 7efbed7fa640 1 -- 172.21.15.2:0/1391582849 >> 172.21.15.2:0/1391582849 conn(0x7efc00072840 msgr2=0x7efc0010b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.913+0000 7efbed7fa640 1 -- 172.21.15.2:0/1391582849 shutdown_connections 2024-05-14T09:53:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:33.913+0000 7efbed7fa640 1 -- 172.21.15.2:0/1391582849 wait complete. 2024-05-14T09:53:34.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 160700 28.23 MB/sec execute 180 sec latency 461.880 ms 2024-05-14T09:53:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:34 smithi029 ceph-mon[36249]: pgmap v983: 129 pgs: 129 active+clean; 729 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.5 MiB/s wr, 531 op/s 2024-05-14T09:53:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:34 smithi026 ceph-mon[36056]: pgmap v983: 129 pgs: 129 active+clean; 729 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.5 MiB/s wr, 531 op/s 2024-05-14T09:53:35.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:34 smithi002 ceph-mon[31557]: pgmap v983: 129 pgs: 129 active+clean; 729 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.5 MiB/s wr, 531 op/s 2024-05-14T09:53:35.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 160918 28.14 MB/sec execute 181 sec latency 161.436 ms 2024-05-14T09:53:36.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 161106 28.04 MB/sec execute 182 sec latency 163.017 ms 2024-05-14T09:53:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:36 smithi029 ceph-mon[36249]: pgmap v984: 129 pgs: 129 active+clean; 754 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 484 op/s 2024-05-14T09:53:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:36 smithi026 ceph-mon[36056]: pgmap v984: 129 pgs: 129 active+clean; 754 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 484 op/s 2024-05-14T09:53:37.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:36 smithi002 ceph-mon[31557]: pgmap v984: 129 pgs: 129 active+clean; 754 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 484 op/s 2024-05-14T09:53:37.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 161366 27.97 MB/sec execute 183 sec latency 113.333 ms 2024-05-14T09:53:38.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 161816 27.92 MB/sec execute 184 sec latency 154.645 ms 2024-05-14T09:53:38.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:38 smithi029 ceph-mon[36249]: pgmap v985: 129 pgs: 129 active+clean; 717 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 15 MiB/s wr, 748 op/s 2024-05-14T09:53:38.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:38 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:53:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:38 smithi026 ceph-mon[36056]: pgmap v985: 129 pgs: 129 active+clean; 717 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 15 MiB/s wr, 748 op/s 2024-05-14T09:53:38.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:38 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:53:39.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:38 smithi002 ceph-mon[31557]: pgmap v985: 129 pgs: 129 active+clean; 717 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 15 MiB/s wr, 748 op/s 2024-05-14T09:53:39.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:38 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:53:39.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 162380 27.91 MB/sec execute 185 sec latency 79.609 ms 2024-05-14T09:53:40.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 163063 28.00 MB/sec execute 186 sec latency 110.720 ms 2024-05-14T09:53:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:40 smithi002 ceph-mon[31557]: pgmap v986: 129 pgs: 129 active+clean; 678 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 14 MiB/s wr, 711 op/s 2024-05-14T09:53:40.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:53:40.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:53:40.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:53:40.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:53:40.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:53:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:40 smithi029 ceph-mon[36249]: pgmap v986: 129 pgs: 129 active+clean; 678 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 14 MiB/s wr, 711 op/s 2024-05-14T09:53:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:53:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:53:40.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:53:40.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:53:40.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:53:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:40 smithi026 ceph-mon[36056]: pgmap v986: 129 pgs: 129 active+clean; 678 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 14 MiB/s wr, 711 op/s 2024-05-14T09:53:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:53:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:53:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:53:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:53:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:53:41.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 163743 28.03 MB/sec execute 187 sec latency 105.805 ms 2024-05-14T09:53:42.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 164392 28.07 MB/sec execute 188 sec latency 79.717 ms 2024-05-14T09:53:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:42 smithi002 ceph-mon[31557]: pgmap v987: 129 pgs: 129 active+clean; 657 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 734 op/s 2024-05-14T09:53:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:42 smithi029 ceph-mon[36249]: pgmap v987: 129 pgs: 129 active+clean; 657 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 734 op/s 2024-05-14T09:53:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:42 smithi026 ceph-mon[36056]: pgmap v987: 129 pgs: 129 active+clean; 657 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 734 op/s 2024-05-14T09:53:43.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 165014 28.14 MB/sec execute 189 sec latency 96.269 ms 2024-05-14T09:53:44.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 165668 28.12 MB/sec execute 190 sec latency 205.507 ms 2024-05-14T09:53:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:44 smithi002 ceph-mon[31557]: pgmap v988: 129 pgs: 129 active+clean; 647 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 800 op/s 2024-05-14T09:53:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:44 smithi029 ceph-mon[36249]: pgmap v988: 129 pgs: 129 active+clean; 647 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 800 op/s 2024-05-14T09:53:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:44 smithi026 ceph-mon[36056]: pgmap v988: 129 pgs: 129 active+clean; 647 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 800 op/s 2024-05-14T09:53:45.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 166112 28.14 MB/sec execute 191 sec latency 238.316 ms 2024-05-14T09:53:46.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 166698 28.14 MB/sec execute 192 sec latency 83.965 ms 2024-05-14T09:53:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:46 smithi002 ceph-mon[31557]: pgmap v989: 129 pgs: 129 active+clean; 650 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.6 MiB/s wr, 712 op/s 2024-05-14T09:53:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:46 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:53:46.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:46 smithi029 ceph-mon[36249]: pgmap v989: 129 pgs: 129 active+clean; 650 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.6 MiB/s wr, 712 op/s 2024-05-14T09:53:46.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:46 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:53:46.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:46 smithi026 ceph-mon[36056]: pgmap v989: 129 pgs: 129 active+clean; 650 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.6 MiB/s wr, 712 op/s 2024-05-14T09:53:46.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:46 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:53:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.068+0000 7f33db1ee640 1 -- 172.21.15.2:0/835198671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33d4102b40 msgr2=0x7f33d4103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.068+0000 7f33db1ee640 1 --2- 172.21.15.2:0/835198671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33d4102b40 0x7f33d4103840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009900 tx=0x7f33c802a400 comp rx=0 tx=0).stop 2024-05-14T09:53:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.068+0000 7f33db1ee640 1 -- 172.21.15.2:0/835198671 shutdown_connections 2024-05-14T09:53:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.068+0000 7f33db1ee640 1 --2- 172.21.15.2:0/835198671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33d4108a80 0x7f33d410aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.068+0000 7f33db1ee640 1 --2- 172.21.15.2:0/835198671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33d4103d80 0x7f33d4104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.068+0000 7f33db1ee640 1 --2- 172.21.15.2:0/835198671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33d4102b40 0x7f33d4103840 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.068+0000 7f33db1ee640 1 -- 172.21.15.2:0/835198671 >> 172.21.15.2:0/835198671 conn(0x7f33d40fdd30 msgr2=0x7f33d41001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.068+0000 7f33db1ee640 1 -- 172.21.15.2:0/835198671 shutdown_connections 2024-05-14T09:53:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.068+0000 7f33db1ee640 1 -- 172.21.15.2:0/835198671 wait complete. 2024-05-14T09:53:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.069+0000 7f33db1ee640 1 Processor -- start 2024-05-14T09:53:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.069+0000 7f33db1ee640 1 -- start start 2024-05-14T09:53:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.069+0000 7f33db1ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33d4102b40 0x7f33d41a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.069+0000 7f33db1ee640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33d4103d80 0x7f33d41a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.070+0000 7f33db1ee640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33d4108a80 0x7f33d41a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.070+0000 7f33db1ee640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f33d410e620 con 0x7f33d4103d80 2024-05-14T09:53:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.070+0000 7f33db1ee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33d410e4a0 con 0x7f33d4102b40 2024-05-14T09:53:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.070+0000 7f33db1ee640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f33d410e7a0 con 0x7f33d4108a80 2024-05-14T09:53:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.070+0000 7f33d8f8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33d4102b40 0x7f33d41a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.070+0000 7f33d8f8a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33d4102b40 0x7f33d41a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:38004/0 (socket says 172.21.15.2:38004) 2024-05-14T09:53:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.070+0000 7f33d8f8a640 1 -- 172.21.15.2:0/2881419112 learned_addr learned my addr 172.21.15.2:0/2881419112 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.070+0000 7f33d978b640 1 --2- 172.21.15.2:0/2881419112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33d4108a80 0x7f33d41a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.070+0000 7f33d3fff640 1 --2- 172.21.15.2:0/2881419112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33d4103d80 0x7f33d41a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.070+0000 7f33d8f8a640 1 -- 172.21.15.2:0/2881419112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33d4108a80 msgr2=0x7f33d41a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.070+0000 7f33d8f8a640 1 --2- 172.21.15.2:0/2881419112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33d4108a80 0x7f33d41a7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.070+0000 7f33d8f8a640 1 -- 172.21.15.2:0/2881419112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33d4103d80 msgr2=0x7f33d41a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.070+0000 7f33d8f8a640 1 --2- 172.21.15.2:0/2881419112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33d4103d80 0x7f33d41a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.070+0000 7f33d8f8a640 1 -- 172.21.15.2:0/2881419112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d41a8410 con 0x7f33d4102b40 2024-05-14T09:53:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.071+0000 7f33d8f8a640 1 --2- 172.21.15.2:0/2881419112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33d4102b40 0x7f33d41a0b80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f33c8006010 tx=0x7f33c8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.071+0000 7f33d1ffb640 1 -- 172.21.15.2:0/2881419112 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c803f030 con 0x7f33d4102b40 2024-05-14T09:53:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.071+0000 7f33d1ffb640 1 -- 172.21.15.2:0/2881419112 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c8031910 con 0x7f33d4102b40 2024-05-14T09:53:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.071+0000 7f33d1ffb640 1 -- 172.21.15.2:0/2881419112 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8035070 con 0x7f33d4102b40 2024-05-14T09:53:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.071+0000 7f33db1ee640 1 -- 172.21.15.2:0/2881419112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d41a86a0 con 0x7f33d4102b40 2024-05-14T09:53:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.071+0000 7f33db1ee640 1 -- 172.21.15.2:0/2881419112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d41a8b40 con 0x7f33d4102b40 2024-05-14T09:53:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.075+0000 7f33e217b740 1 Processor -- start 2024-05-14T09:53:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.075+0000 7f33e217b740 1 -- start start 2024-05-14T09:53:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.075+0000 7f33e217b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c991b0 0x563a8303aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.075+0000 7f33e217b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c702f0 0x563a8303afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.076+0000 7f33e217b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c784d0 0x563a83041ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.076+0000 7f33e217b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563a82f9f5d0 con 0x563a82c702f0 2024-05-14T09:53:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.076+0000 7f33e217b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563a82f58890 con 0x563a82c991b0 2024-05-14T09:53:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.076+0000 7f33e217b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563a82f9b590 con 0x563a82c784d0 2024-05-14T09:53:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.076+0000 7f33d978b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c991b0 0x563a8303aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.076+0000 7f33d978b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c991b0 0x563a8303aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:38016/0 (socket says 172.21.15.2:38016) 2024-05-14T09:53:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.076+0000 7f33d978b640 1 -- 172.21.15.2:0/3323081708 learned_addr learned my addr 172.21.15.2:0/3323081708 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.076+0000 7f33d1ffb640 1 -- 172.21.15.2:0/2881419112 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f33c8042030 con 0x7f33d4102b40 2024-05-14T09:53:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.076+0000 7f33d978b640 1 -- 172.21.15.2:0/3323081708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c784d0 msgr2=0x563a83041ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:53:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.076+0000 7f33d978b640 1 --2- 172.21.15.2:0/3323081708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c784d0 0x563a83041ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.076+0000 7f33d978b640 1 -- 172.21.15.2:0/3323081708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c702f0 msgr2=0x563a8303afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.076+0000 7f33d978b640 1 --2- 172.21.15.2:0/3323081708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c702f0 0x563a8303afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.076+0000 7f33d978b640 1 -- 172.21.15.2:0/3323081708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a82cb1210 con 0x563a82c991b0 2024-05-14T09:53:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.076+0000 7f33d978b640 1 --2- 172.21.15.2:0/3323081708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c991b0 0x563a8303aa60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009fd0 tx=0x7f33cc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea3ef1006b88ae0f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.077+0000 7f33beffd640 1 -- 172.21.15.2:0/3323081708 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc004d20 con 0x563a82c991b0 2024-05-14T09:53:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.077+0000 7f33beffd640 1 -- 172.21.15.2:0/3323081708 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33cc004050 con 0x563a82c991b0 2024-05-14T09:53:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.077+0000 7f33beffd640 1 -- 172.21.15.2:0/3323081708 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc005260 con 0x563a82c991b0 2024-05-14T09:53:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.077+0000 7f33d1ffb640 1 --2- 172.21.15.2:0/2881419112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f33c40b2920 0x7f33c40b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.078+0000 7f33d1ffb640 1 -- 172.21.15.2:0/2881419112 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f33c8102940 con 0x7f33d4102b40 2024-05-14T09:53:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.078+0000 7f33d8f8a640 1 --2- 172.21.15.2:0/2881419112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f33c40b2920 0x7f33c40b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.078+0000 7f33e217b740 1 -- 172.21.15.2:0/3323081708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c991b0 msgr2=0x563a8303aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.078+0000 7f33e217b740 1 --2- 172.21.15.2:0/3323081708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c991b0 0x563a8303aa60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009fd0 tx=0x7f33cc0118d0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.078+0000 7f33e217b740 1 -- 172.21.15.2:0/3323081708 shutdown_connections 2024-05-14T09:53:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.078+0000 7f33e217b740 1 --2- 172.21.15.2:0/3323081708 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c784d0 0x563a83041ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.078+0000 7f33e217b740 1 --2- 172.21.15.2:0/3323081708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c702f0 0x563a8303afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.078+0000 7f33e217b740 1 --2- 172.21.15.2:0/3323081708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c991b0 0x563a8303aa60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.078+0000 7f33e217b740 1 -- 172.21.15.2:0/3323081708 >> 172.21.15.2:0/3323081708 conn(0x563a82c8d300 msgr2=0x563a830363a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.078+0000 7f33e217b740 1 -- 172.21.15.2:0/3323081708 shutdown_connections 2024-05-14T09:53:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.078+0000 7f33e217b740 1 -- 172.21.15.2:0/3323081708 wait complete. 2024-05-14T09:53:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.079+0000 7f33d8f8a640 1 --2- 172.21.15.2:0/2881419112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f33c40b2920 0x7f33c40b4e30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f33c80c9ad0 tx=0x7f33c803b460 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.079+0000 7f33e217b740 1 -- start start 2024-05-14T09:53:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.079+0000 7f33e217b740 1 Processor -- start 2024-05-14T09:53:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.080+0000 7f33e217b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c702f0 0x563a830446e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.080+0000 7f33e217b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c784d0 0x563a83044c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.080+0000 7f33d978b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c702f0 0x563a830446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.080+0000 7f33d978b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c702f0 0x563a830446e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:38028/0 (socket says 172.21.15.2:38028) 2024-05-14T09:53:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.081+0000 7f33d3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c784d0 0x563a83044c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.081+0000 7f33d3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c784d0 0x563a83044c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:58522/0 (socket says 172.21.15.2:58522) 2024-05-14T09:53:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.081+0000 7f33d978b640 1 -- 172.21.15.2:0/830958833 learned_addr learned my addr 172.21.15.2:0/830958833 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.081+0000 7f33e217b740 1 --2- 172.21.15.2:0/830958833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c991b0 0x563a8304bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.081+0000 7f33e217b740 1 -- 172.21.15.2:0/830958833 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563a83042930 con 0x563a82c991b0 2024-05-14T09:53:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.081+0000 7f33e217b740 1 -- 172.21.15.2:0/830958833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563a82f5c370 con 0x563a82c702f0 2024-05-14T09:53:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.081+0000 7f33e217b740 1 -- 172.21.15.2:0/830958833 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563a83042ab0 con 0x563a82c784d0 2024-05-14T09:53:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.081+0000 7f33e217b740 10 client.0 _ll_register_callbacks cb 0x563a82cb0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:53:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.081+0000 7f33d978b640 1 --2- 172.21.15.2:0/830958833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c991b0 0x563a8304bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.082+0000 7f33d978b640 1 -- 172.21.15.2:0/830958833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c784d0 msgr2=0x563a83044c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.082+0000 7f33d978b640 1 --2- 172.21.15.2:0/830958833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c784d0 0x563a83044c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.082+0000 7f33d978b640 1 -- 172.21.15.2:0/830958833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c991b0 msgr2=0x563a8304bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.082+0000 7f33d978b640 1 --2- 172.21.15.2:0/830958833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c991b0 0x563a8304bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.082+0000 7f33d978b640 1 -- 172.21.15.2:0/830958833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a82d7d620 con 0x563a82c702f0 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.082+0000 7f33e217b740 1 -- 172.21.15.2:0/830958833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33cc004050 con 0x563a82c702f0 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.082+0000 7f33d978b640 1 --2- 172.21.15.2:0/830958833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c702f0 0x563a830446e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f33cc006010 tx=0x7f33cc009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbd9b8b77678bd8b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.082+0000 7f33b3fff640 10 client.26846 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.082+0000 7f33b3fff640 1 -- 172.21.15.2:0/830958833 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc003e60 con 0x563a82c702f0 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.083+0000 7f33b3fff640 1 -- 172.21.15.2:0/830958833 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f33cc004610 con 0x563a82c702f0 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.083+0000 7f33b3fff640 1 -- 172.21.15.2:0/830958833 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f33cc005040 con 0x563a82c702f0 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.083+0000 7f33e217b740 10 client.26846 fetch_fsmap learned FSMap version 62 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.083+0000 7f33e217b740 1 -- 172.21.15.2:0/830958833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=62}) -- 0x563a8304e950 con 0x563a82c702f0 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.083+0000 7f33b3fff640 1 -- 172.21.15.2:0/830958833 <== mon.2 v2:172.21.15.26:3300/0 4 ==== fsmap(e 62) ==== 8350+0+0 (secure 0 0 0) 0x7f33cc0051e0 con 0x563a82c702f0 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.083+0000 7f33e217b740 10 client.26846 fetch_fsmap finished waiting for FSMap version 62 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.083+0000 7f33e217b740 10 client.26846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.083+0000 7f33e217b740 20 client.26846 populate_metadata read hostname 'smithi002' 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.083+0000 7f33e217b740 1 --2- 172.21.15.2:0/830958833 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563a82c57d80 0x563a83052410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.083+0000 7f33e217b740 4 client.26846 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:53:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.084+0000 7f33e217b740 1 -- 172.21.15.2:0/830958833 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a82fad1e0 con 0x563a82c57d80 2024-05-14T09:53:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.084+0000 7f33d3fff640 1 --2- 172.21.15.2:0/830958833 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563a82c57d80 0x563a83052410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.084+0000 7f33d3fff640 1 --2- 172.21.15.2:0/830958833 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563a82c57d80 0x563a83052410 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7adff80c20ae62c server_cookie=273300d76e2d304d in_seq=0 out_seq=0 2024-05-14T09:53:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.084+0000 7f33b3fff640 10 client.26846 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.088+0000 7f33b3fff640 1 -- 172.21.15.2:0/830958833 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563a82fad1e0 con 0x563a82c57d80 2024-05-14T09:53:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.088+0000 7f33b3fff640 10 client.26846 handle_command_reply: tid=0 2024-05-14T09:53:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.088+0000 7f33e217b740 1 client.26846 shutdown 2024-05-14T09:53:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.088+0000 7f33e217b740 1 -- 172.21.15.2:0/830958833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c702f0 msgr2=0x563a830446e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.088+0000 7f33e217b740 1 --2- 172.21.15.2:0/830958833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c702f0 0x563a830446e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f33cc006010 tx=0x7f33cc009cd0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.088+0000 7f33e217b740 1 -- 172.21.15.2:0/830958833 shutdown_connections 2024-05-14T09:53:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.088+0000 7f33e217b740 1 --2- 172.21.15.2:0/830958833 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563a82c57d80 0x563a83052410 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.088+0000 7f33e217b740 1 --2- 172.21.15.2:0/830958833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c991b0 0x563a8304bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.088+0000 7f33e217b740 1 --2- 172.21.15.2:0/830958833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c784d0 0x563a83044c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.088+0000 7f33e217b740 1 --2- 172.21.15.2:0/830958833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c702f0 0x563a830446e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.089+0000 7f33b3fff640 0 client.26846 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.089+0000 7f33e217b740 1 -- 172.21.15.2:0/830958833 >> 172.21.15.2:0/830958833 conn(0x563a82c8d300 msgr2=0x563a830353a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.089+0000 7f33e217b740 1 -- 172.21.15.2:0/830958833 shutdown_connections 2024-05-14T09:53:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.089+0000 7f33e217b740 1 -- 172.21.15.2:0/830958833 wait complete. 2024-05-14T09:53:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.089+0000 7f33e217b740 20 client.26846 trim_cache size 0 max 16384 2024-05-14T09:53:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.117+0000 7f33e217b740 1 Processor -- start 2024-05-14T09:53:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.117+0000 7f33e217b740 1 -- start start 2024-05-14T09:53:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.117+0000 7f33e217b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c57d80 0x563a83035cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33e217b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c702f0 0x563a83036230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33e217b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c784d0 0x563a8304baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33d978b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c57d80 0x563a83035cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33d978b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c57d80 0x563a83035cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48974/0 (socket says 172.21.15.2:48974) 2024-05-14T09:53:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c702f0 0x563a83036230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c702f0 0x563a83036230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:38032/0 (socket says 172.21.15.2:38032) 2024-05-14T09:53:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33d978b640 1 -- 172.21.15.2:0/1707497753 learned_addr learned my addr 172.21.15.2:0/1707497753 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33e217b740 1 -- 172.21.15.2:0/1707497753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f33d41b5bb0 con 0x563a82c57d80 2024-05-14T09:53:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33e217b740 1 -- 172.21.15.2:0/1707497753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33d41b3480 con 0x563a82c702f0 2024-05-14T09:53:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33e217b740 1 -- 172.21.15.2:0/1707497753 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f33d41b12d0 con 0x563a82c784d0 2024-05-14T09:53:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33d978b640 1 --2- 172.21.15.2:0/1707497753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c784d0 0x563a8304baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33d978b640 1 -- 172.21.15.2:0/1707497753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c784d0 msgr2=0x563a8304baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33d978b640 1 --2- 172.21.15.2:0/1707497753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c784d0 0x563a8304baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33d978b640 1 -- 172.21.15.2:0/1707497753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c702f0 msgr2=0x563a83036230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33d978b640 1 --2- 172.21.15.2:0/1707497753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c702f0 0x563a83036230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.118+0000 7f33d978b640 1 -- 172.21.15.2:0/1707497753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a83036d00 con 0x563a82c57d80 2024-05-14T09:53:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.119+0000 7f33d978b640 1 --2- 172.21.15.2:0/1707497753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c57d80 0x563a83035cf0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f33cc011f50 tx=0x7f33cc021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62b6d7081918fc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.119+0000 7f33be7fc640 1 -- 172.21.15.2:0/1707497753 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc03c040 con 0x563a82c57d80 2024-05-14T09:53:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.119+0000 7f33be7fc640 1 -- 172.21.15.2:0/1707497753 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33cc004050 con 0x563a82c57d80 2024-05-14T09:53:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.119+0000 7f33be7fc640 1 -- 172.21.15.2:0/1707497753 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc00f740 con 0x563a82c57d80 2024-05-14T09:53:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.120+0000 7f33e217b740 1 -- 172.21.15.2:0/1707497753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c57d80 msgr2=0x563a83035cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.120+0000 7f33e217b740 1 --2- 172.21.15.2:0/1707497753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c57d80 0x563a83035cf0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f33cc011f50 tx=0x7f33cc021000 comp rx=0 tx=0).stop 2024-05-14T09:53:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.120+0000 7f33e217b740 1 -- 172.21.15.2:0/1707497753 shutdown_connections 2024-05-14T09:53:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.120+0000 7f33e217b740 1 --2- 172.21.15.2:0/1707497753 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c784d0 0x563a8304baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.120+0000 7f33e217b740 1 --2- 172.21.15.2:0/1707497753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c702f0 0x563a83036230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.120+0000 7f33e217b740 1 --2- 172.21.15.2:0/1707497753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c57d80 0x563a83035cf0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.120+0000 7f33e217b740 1 -- 172.21.15.2:0/1707497753 >> 172.21.15.2:0/1707497753 conn(0x563a82c8d300 msgr2=0x563a8303eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.121+0000 7f33e217b740 1 -- 172.21.15.2:0/1707497753 shutdown_connections 2024-05-14T09:53:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.121+0000 7f33e217b740 1 -- 172.21.15.2:0/1707497753 wait complete. 2024-05-14T09:53:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.121+0000 7f33e217b740 1 -- start start 2024-05-14T09:53:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.121+0000 7f33e217b740 1 Processor -- start 2024-05-14T09:53:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.122+0000 7f33e217b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c57d80 0x563a83048f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.122+0000 7f33e217b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c702f0 0x563a83049450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.122+0000 7f33e217b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c784d0 0x563a830b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.122+0000 7f33d978b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c57d80 0x563a83048f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.122+0000 7f33e217b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f33d41b1940 con 0x563a82c57d80 2024-05-14T09:53:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.122+0000 7f33d978b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c57d80 0x563a83048f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48988/0 (socket says 172.21.15.2:48988) 2024-05-14T09:53:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.122+0000 7f33d978b640 1 -- 172.21.15.2:0/1151146368 learned_addr learned my addr 172.21.15.2:0/1151146368 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:53:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.122+0000 7f33d3fff640 1 --2- 172.21.15.2:0/1151146368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c702f0 0x563a83049450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.122+0000 7f33e217b740 1 -- 172.21.15.2:0/1151146368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33d41b2db0 con 0x563a82c784d0 2024-05-14T09:53:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.122+0000 7f33e217b740 1 -- 172.21.15.2:0/1151146368 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f33d41b2010 con 0x563a82c702f0 2024-05-14T09:53:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.123+0000 7f33e217b740 10 client.0 _ll_register_callbacks cb 0x7f33cc009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:53:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.123+0000 7f33d978b640 1 -- 172.21.15.2:0/1151146368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c702f0 msgr2=0x563a83049450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.123+0000 7f33d978b640 1 --2- 172.21.15.2:0/1151146368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c702f0 0x563a83049450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.123+0000 7f33d978b640 1 -- 172.21.15.2:0/1151146368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c784d0 msgr2=0x563a830b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:53:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.123+0000 7f33d978b640 1 --2- 172.21.15.2:0/1151146368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c784d0 0x563a830b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.123+0000 7f33d978b640 1 -- 172.21.15.2:0/1151146368 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a8304c410 con 0x563a82c57d80 2024-05-14T09:53:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.123+0000 7f33d978b640 1 --2- 172.21.15.2:0/1151146368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c57d80 0x563a83048f10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f33cc011910 tx=0x7f33cc004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2348af7de4d09a44 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:53:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.123+0000 7f33b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:53:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.124+0000 7f33b3fff640 1 -- 172.21.15.2:0/1151146368 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc00f4e0 con 0x563a82c57d80 2024-05-14T09:53:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.124+0000 7f33b3fff640 1 -- 172.21.15.2:0/1151146368 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7f33cc00f680 con 0x563a82c57d80 2024-05-14T09:53:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.124+0000 7f33e217b740 1 -- 172.21.15.2:0/1151146368 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33cc004050 con 0x563a82c57d80 2024-05-14T09:53:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.124+0000 7f33b3fff640 1 -- 172.21.15.2:0/1151146368 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f33cc04f050 con 0x563a82c57d80 2024-05-14T09:53:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.124+0000 7f33e217b740 10 client.17238 fetch_fsmap learned FSMap version 62 2024-05-14T09:53:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.124+0000 7f33e217b740 1 -- 172.21.15.2:0/1151146368 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x563a830b9980 con 0x563a82c57d80 2024-05-14T09:53:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.124+0000 7f33b3fff640 1 -- 172.21.15.2:0/1151146368 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33cc004050 con 0x563a82c57d80 2024-05-14T09:53:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.125+0000 7f33b3fff640 1 -- 172.21.15.2:0/1151146368 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 62) ==== 8350+0+0 (secure 0 0 0) 0x7f33cc028510 con 0x563a82c57d80 2024-05-14T09:53:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.125+0000 7f33e217b740 10 client.17238 fetch_fsmap finished waiting for FSMap version 62 2024-05-14T09:53:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.125+0000 7f33e217b740 10 client.17238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:53:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.125+0000 7f33e217b740 20 client.17238 populate_metadata read hostname 'smithi002' 2024-05-14T09:53:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.125+0000 7f33e217b740 1 --2- 172.21.15.2:0/1151146368 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563a82c991b0 0x563a830bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:53:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.125+0000 7f33e217b740 4 client.17238 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:53:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.125+0000 7f33e217b740 1 -- 172.21.15.2:0/1151146368 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a82fa7050 con 0x563a82c991b0 2024-05-14T09:53:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.126+0000 7f33d3fff640 1 --2- 172.21.15.2:0/1151146368 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563a82c991b0 0x563a830bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:53:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.126+0000 7f33d3fff640 1 --2- 172.21.15.2:0/1151146368 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563a82c991b0 0x563a830bc220 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=263ab08c154f56fb server_cookie=1dd4039c1d2280fd in_seq=0 out_seq=0 2024-05-14T09:53:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.126+0000 7f33b3fff640 10 client.17238 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:47.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 167127 28.09 MB/sec execute 193 sec latency 402.098 ms 2024-05-14T09:53:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.825+0000 7f33b3fff640 1 -- 172.21.15.2:0/1151146368 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563a82fa7050 con 0x563a82c991b0 2024-05-14T09:53:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.825+0000 7f33b3fff640 10 client.17238 handle_command_reply: tid=0 2024-05-14T09:53:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.825+0000 7f33e217b740 1 client.17238 shutdown 2024-05-14T09:53:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.825+0000 7f33e217b740 1 -- 172.21.15.2:0/1151146368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c57d80 msgr2=0x563a83048f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.825+0000 7f33e217b740 1 --2- 172.21.15.2:0/1151146368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c57d80 0x563a83048f10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f33cc011910 tx=0x7f33cc004a00 comp rx=0 tx=0).stop 2024-05-14T09:53:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.825+0000 7f33e217b740 1 -- 172.21.15.2:0/1151146368 shutdown_connections 2024-05-14T09:53:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.825+0000 7f33e217b740 1 --2- 172.21.15.2:0/1151146368 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563a82c991b0 0x563a830bc220 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.825+0000 7f33b3fff640 0 client.17238 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:53:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.825+0000 7f33e217b740 1 --2- 172.21.15.2:0/1151146368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a82c784d0 0x563a830b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.825+0000 7f33e217b740 1 --2- 172.21.15.2:0/1151146368 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563a82c702f0 0x563a83049450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.825+0000 7f33e217b740 1 --2- 172.21.15.2:0/1151146368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a82c57d80 0x563a83048f10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.825+0000 7f33e217b740 1 -- 172.21.15.2:0/1151146368 >> 172.21.15.2:0/1151146368 conn(0x563a82c8d300 msgr2=0x563a83035cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.826+0000 7f33e217b740 1 -- 172.21.15.2:0/1151146368 shutdown_connections 2024-05-14T09:53:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.826+0000 7f33e217b740 1 -- 172.21.15.2:0/1151146368 wait complete. 2024-05-14T09:53:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.826+0000 7f33e217b740 20 client.17238 trim_cache size 0 max 16384 2024-05-14T09:53:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:53:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:53:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:53:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:53:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.827+0000 7f33db1ee640 1 -- 172.21.15.2:0/2881419112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f33c40b2920 msgr2=0x7f33c40b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.827+0000 7f33db1ee640 1 --2- 172.21.15.2:0/2881419112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f33c40b2920 0x7f33c40b4e30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f33c80c9ad0 tx=0x7f33c803b460 comp rx=0 tx=0).stop 2024-05-14T09:53:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.827+0000 7f33db1ee640 1 -- 172.21.15.2:0/2881419112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33d4102b40 msgr2=0x7f33d41a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:53:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.827+0000 7f33db1ee640 1 --2- 172.21.15.2:0/2881419112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33d4102b40 0x7f33d41a0b80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f33c8006010 tx=0x7f33c8004270 comp rx=0 tx=0).stop 2024-05-14T09:53:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.827+0000 7f33db1ee640 1 -- 172.21.15.2:0/2881419112 shutdown_connections 2024-05-14T09:53:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.827+0000 7f33db1ee640 1 --2- 172.21.15.2:0/2881419112 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f33c40b2920 0x7f33c40b4e30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.827+0000 7f33db1ee640 1 --2- 172.21.15.2:0/2881419112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f33d4108a80 0x7f33d41a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.827+0000 7f33db1ee640 1 --2- 172.21.15.2:0/2881419112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33d4103d80 0x7f33d41a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.827+0000 7f33db1ee640 1 --2- 172.21.15.2:0/2881419112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33d4102b40 0x7f33d41a0b80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:53:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.827+0000 7f33db1ee640 1 -- 172.21.15.2:0/2881419112 >> 172.21.15.2:0/2881419112 conn(0x7f33d40fdd30 msgr2=0x7f33d4100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:53:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.827+0000 7f33db1ee640 1 -- 172.21.15.2:0/2881419112 shutdown_connections 2024-05-14T09:53:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:53:47.827+0000 7f33db1ee640 1 -- 172.21.15.2:0/2881419112 wait complete. 2024-05-14T09:53:48.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 167717 28.13 MB/sec execute 194 sec latency 441.527 ms 2024-05-14T09:53:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:48 smithi002 ceph-mon[31557]: pgmap v990: 129 pgs: 129 active+clean; 655 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.3 MiB/s wr, 695 op/s 2024-05-14T09:53:48.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:48 smithi029 ceph-mon[36249]: pgmap v990: 129 pgs: 129 active+clean; 655 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.3 MiB/s wr, 695 op/s 2024-05-14T09:53:48.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:48 smithi026 ceph-mon[36056]: pgmap v990: 129 pgs: 129 active+clean; 655 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.3 MiB/s wr, 695 op/s 2024-05-14T09:53:49.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 168412 28.15 MB/sec execute 195 sec latency 96.749 ms 2024-05-14T09:53:50.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 169011 28.16 MB/sec execute 196 sec latency 233.913 ms 2024-05-14T09:53:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:50 smithi002 ceph-mon[31557]: pgmap v991: 129 pgs: 129 active+clean; 660 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 2.8 MiB/s wr, 522 op/s 2024-05-14T09:53:50.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:50 smithi029 ceph-mon[36249]: pgmap v991: 129 pgs: 129 active+clean; 660 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 2.8 MiB/s wr, 522 op/s 2024-05-14T09:53:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:50 smithi026 ceph-mon[36056]: pgmap v991: 129 pgs: 129 active+clean; 660 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 2.8 MiB/s wr, 522 op/s 2024-05-14T09:53:51.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 169760 28.27 MB/sec execute 197 sec latency 103.697 ms 2024-05-14T09:53:52.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 170379 28.29 MB/sec execute 198 sec latency 64.699 ms 2024-05-14T09:53:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:52 smithi002 ceph-mon[31557]: pgmap v992: 129 pgs: 129 active+clean; 667 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.4 MiB/s wr, 576 op/s 2024-05-14T09:53:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:52 smithi029 ceph-mon[36249]: pgmap v992: 129 pgs: 129 active+clean; 667 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.4 MiB/s wr, 576 op/s 2024-05-14T09:53:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:52 smithi026 ceph-mon[36056]: pgmap v992: 129 pgs: 129 active+clean; 667 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.4 MiB/s wr, 576 op/s 2024-05-14T09:53:53.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 170983 28.32 MB/sec execute 199 sec latency 83.288 ms 2024-05-14T09:53:54.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 171363 28.27 MB/sec execute 200 sec latency 332.935 ms 2024-05-14T09:53:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:54 smithi002 ceph-mon[31557]: pgmap v993: 129 pgs: 129 active+clean; 665 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 594 op/s 2024-05-14T09:53:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:54 smithi029 ceph-mon[36249]: pgmap v993: 129 pgs: 129 active+clean; 665 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 594 op/s 2024-05-14T09:53:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:54 smithi026 ceph-mon[36056]: pgmap v993: 129 pgs: 129 active+clean; 665 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 594 op/s 2024-05-14T09:53:55.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 171930 28.22 MB/sec execute 201 sec latency 279.312 ms 2024-05-14T09:53:56.611 INFO:tasks.workunit.client.0.smithi002.stdout: 10 172693 28.32 MB/sec execute 202 sec latency 75.776 ms 2024-05-14T09:53:56.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:56 smithi002 ceph-mon[31557]: pgmap v994: 129 pgs: 129 active+clean; 670 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.4 MiB/s wr, 537 op/s 2024-05-14T09:53:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:56 smithi029 ceph-mon[36249]: pgmap v994: 129 pgs: 129 active+clean; 670 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.4 MiB/s wr, 537 op/s 2024-05-14T09:53:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:56 smithi026 ceph-mon[36056]: pgmap v994: 129 pgs: 129 active+clean; 670 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.4 MiB/s wr, 537 op/s 2024-05-14T09:53:57.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 173413 28.43 MB/sec execute 203 sec latency 75.790 ms 2024-05-14T09:53:58.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 173921 28.39 MB/sec execute 204 sec latency 69.493 ms 2024-05-14T09:53:58.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:53:58 smithi002 ceph-mon[31557]: pgmap v995: 129 pgs: 129 active+clean; 678 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.8 MiB/s wr, 573 op/s 2024-05-14T09:53:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:53:58 smithi029 ceph-mon[36249]: pgmap v995: 129 pgs: 129 active+clean; 678 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.8 MiB/s wr, 573 op/s 2024-05-14T09:53:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:53:58 smithi026 ceph-mon[36056]: pgmap v995: 129 pgs: 129 active+clean; 678 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.8 MiB/s wr, 573 op/s 2024-05-14T09:53:59.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 174497 28.40 MB/sec execute 205 sec latency 74.723 ms 2024-05-14T09:54:00.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 174905 28.38 MB/sec execute 206 sec latency 384.032 ms 2024-05-14T09:54:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:00 smithi002 ceph-mon[31557]: pgmap v996: 129 pgs: 129 active+clean; 679 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.4 MiB/s wr, 532 op/s 2024-05-14T09:54:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:00 smithi029 ceph-mon[36249]: pgmap v996: 129 pgs: 129 active+clean; 679 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.4 MiB/s wr, 532 op/s 2024-05-14T09:54:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:00 smithi026 ceph-mon[36056]: pgmap v996: 129 pgs: 129 active+clean; 679 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.4 MiB/s wr, 532 op/s 2024-05-14T09:54:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.954+0000 7fd45695f640 1 -- 172.21.15.2:0/1348199773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4501034f0 msgr2=0x7fd450103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.954+0000 7fd45695f640 1 --2- 172.21.15.2:0/1348199773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4501034f0 0x7fd450103900 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd44400aa10 tx=0x7fd44402b720 comp rx=0 tx=0).stop 2024-05-14T09:54:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.955+0000 7fd45695f640 1 -- 172.21.15.2:0/1348199773 shutdown_connections 2024-05-14T09:54:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.955+0000 7fd45695f640 1 --2- 172.21.15.2:0/1348199773 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4501081f0 0x7fd45010a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.955+0000 7fd45695f640 1 --2- 172.21.15.2:0/1348199773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4501034f0 0x7fd450103900 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.955+0000 7fd45695f640 1 --2- 172.21.15.2:0/1348199773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4501022b0 0x7fd450102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.955+0000 7fd45695f640 1 -- 172.21.15.2:0/1348199773 >> 172.21.15.2:0/1348199773 conn(0x7fd4500fd480 msgr2=0x7fd4500ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.955+0000 7fd45695f640 1 -- 172.21.15.2:0/1348199773 shutdown_connections 2024-05-14T09:54:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.955+0000 7fd45695f640 1 -- 172.21.15.2:0/1348199773 wait complete. 2024-05-14T09:54:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.955+0000 7fd45695f640 1 Processor -- start 2024-05-14T09:54:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.955+0000 7fd45695f640 1 -- start start 2024-05-14T09:54:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd45695f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4501022b0 0x7fd4501980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd45695f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4501034f0 0x7fd4501985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd45695f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4501081f0 0x7fd45019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd45695f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd45010d540 con 0x7fd4501081f0 2024-05-14T09:54:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd45695f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd45010d3c0 con 0x7fd4501022b0 2024-05-14T09:54:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd45695f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd45010d6c0 con 0x7fd4501034f0 2024-05-14T09:54:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd44ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4501022b0 0x7fd4501980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd454efc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4501081f0 0x7fd45019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd44ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4501022b0 0x7fd4501980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:60518/0 (socket says 172.21.15.2:60518) 2024-05-14T09:54:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd44f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4501034f0 0x7fd4501985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd44ffff640 1 -- 172.21.15.2:0/2850363060 learned_addr learned my addr 172.21.15.2:0/2850363060 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd44ffff640 1 -- 172.21.15.2:0/2850363060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4501034f0 msgr2=0x7fd4501985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd44ffff640 1 --2- 172.21.15.2:0/2850363060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4501034f0 0x7fd4501985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd44ffff640 1 -- 172.21.15.2:0/2850363060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4501081f0 msgr2=0x7fd45019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd44ffff640 1 --2- 172.21.15.2:0/2850363060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4501081f0 0x7fd45019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd44ffff640 1 -- 172.21.15.2:0/2850363060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45019fbe0 con 0x7fd4501022b0 2024-05-14T09:54:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.956+0000 7fd454efc640 1 --2- 172.21.15.2:0/2850363060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4501081f0 0x7fd45019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:54:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.957+0000 7fd44ffff640 1 --2- 172.21.15.2:0/2850363060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4501022b0 0x7fd4501980a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd44000eb00 tx=0x7fd440009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.957+0000 7fd44d7fa640 1 -- 172.21.15.2:0/2850363060 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44000bbd0 con 0x7fd4501022b0 2024-05-14T09:54:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.957+0000 7fd44d7fa640 1 -- 172.21.15.2:0/2850363060 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd44000bd70 con 0x7fd4501022b0 2024-05-14T09:54:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.957+0000 7fd45695f640 1 -- 172.21.15.2:0/2850363060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45019fed0 con 0x7fd4501022b0 2024-05-14T09:54:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.957+0000 7fd44d7fa640 1 -- 172.21.15.2:0/2850363060 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440017410 con 0x7fd4501022b0 2024-05-14T09:54:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.957+0000 7fd45695f640 1 -- 172.21.15.2:0/2850363060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4501a0330 con 0x7fd4501022b0 2024-05-14T09:54:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.959+0000 7fd45d80b740 1 Processor -- start 2024-05-14T09:54:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd45d80b740 1 -- start start 2024-05-14T09:54:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd45d80b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 0x55f0e0c1fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd45d80b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 0x55f0e0c1ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd45d80b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c26ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd45d80b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f0e0b845d0 con 0x55f0e087e1b0 2024-05-14T09:54:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd45d80b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f0e0b3d890 con 0x55f0e085d4d0 2024-05-14T09:54:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd45d80b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f0e0b80590 con 0x55f0e08552f0 2024-05-14T09:54:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd454efc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 0x55f0e0c1fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd454efc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 0x55f0e0c1fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54046/0 (socket says 172.21.15.2:54046) 2024-05-14T09:54:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd454efc640 1 -- 172.21.15.2:0/3506424205 learned_addr learned my addr 172.21.15.2:0/3506424205 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd44f7fe640 1 --2- 172.21.15.2:0/3506424205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c26ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd44d7fa640 1 -- 172.21.15.2:0/2850363060 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd44001e020 con 0x7fd4501022b0 2024-05-14T09:54:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd454efc640 1 -- 172.21.15.2:0/3506424205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 msgr2=0x55f0e0c1ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd454efc640 1 --2- 172.21.15.2:0/3506424205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 0x55f0e0c1ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd454efc640 1 -- 172.21.15.2:0/3506424205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 msgr2=0x55f0e0c26ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd454efc640 1 --2- 172.21.15.2:0/3506424205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c26ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.960+0000 7fd454efc640 1 -- 172.21.15.2:0/3506424205 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0e0896210 con 0x55f0e087e1b0 2024-05-14T09:54:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.961+0000 7fd454efc640 1 --2- 172.21.15.2:0/3506424205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 0x55f0e0c1fa60 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd4480117e0 tx=0x7fd448011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55b54fca8e7944c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.961+0000 7fd4367fc640 1 -- 172.21.15.2:0/3506424205 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448011310 con 0x55f0e087e1b0 2024-05-14T09:54:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.961+0000 7fd4367fc640 1 -- 172.21.15.2:0/3506424205 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd448003e20 con 0x55f0e087e1b0 2024-05-14T09:54:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.961+0000 7fd4367fc640 1 -- 172.21.15.2:0/3506424205 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480044f0 con 0x55f0e087e1b0 2024-05-14T09:54:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.961+0000 7fd44d7fa640 1 --2- 172.21.15.2:0/2850363060 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd4380b28a0 0x7fd4380b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.961+0000 7fd44ffff640 1 --2- 172.21.15.2:0/2850363060 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd4380b28a0 0x7fd4380b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.961+0000 7fd44d7fa640 1 -- 172.21.15.2:0/2850363060 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fd4400e3b60 con 0x7fd4501022b0 2024-05-14T09:54:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.962+0000 7fd44ffff640 1 --2- 172.21.15.2:0/2850363060 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd4380b28a0 0x7fd4380b4db0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd44000ead0 tx=0x7fd4400e5040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.962+0000 7fd45d80b740 1 -- 172.21.15.2:0/3506424205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 msgr2=0x55f0e0c1fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.962+0000 7fd45d80b740 1 --2- 172.21.15.2:0/3506424205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 0x55f0e0c1fa60 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd4480117e0 tx=0x7fd448011ca0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.962+0000 7fd45d80b740 1 -- 172.21.15.2:0/3506424205 shutdown_connections 2024-05-14T09:54:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.962+0000 7fd45d80b740 1 --2- 172.21.15.2:0/3506424205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c26ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.962+0000 7fd45d80b740 1 --2- 172.21.15.2:0/3506424205 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 0x55f0e0c1ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.962+0000 7fd45d80b740 1 --2- 172.21.15.2:0/3506424205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 0x55f0e0c1fa60 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.962+0000 7fd45d80b740 1 -- 172.21.15.2:0/3506424205 >> 172.21.15.2:0/3506424205 conn(0x55f0e0872300 msgr2=0x55f0e0c1b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.962+0000 7fd45d80b740 1 -- 172.21.15.2:0/3506424205 shutdown_connections 2024-05-14T09:54:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.962+0000 7fd45d80b740 1 -- 172.21.15.2:0/3506424205 wait complete. 2024-05-14T09:54:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.962+0000 7fd45d80b740 1 -- start start 2024-05-14T09:54:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.963+0000 7fd45d80b740 1 Processor -- start 2024-05-14T09:54:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.963+0000 7fd45d80b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 0x55f0e0c29640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.963+0000 7fd45d80b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c29b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.963+0000 7fd45d80b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 0x55f0e0c30ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.963+0000 7fd45d80b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f0e0c27930 con 0x55f0e087e1b0 2024-05-14T09:54:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.963+0000 7fd45d80b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f0e0b41370 con 0x55f0e085d4d0 2024-05-14T09:54:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.963+0000 7fd45d80b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f0e0c27ab0 con 0x55f0e08552f0 2024-05-14T09:54:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.963+0000 7fd45d80b740 10 client.0 _ll_register_callbacks cb 0x55f0e0895a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:54:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.963+0000 7fd454efc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 0x55f0e0c29640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.963+0000 7fd44f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c29b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.963+0000 7fd454efc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 0x55f0e0c30ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.963+0000 7fd44f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c29b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:60536/0 (socket says 172.21.15.2:60536) 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.963+0000 7fd44f7fe640 1 -- 172.21.15.2:0/3993953862 learned_addr learned my addr 172.21.15.2:0/3993953862 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.964+0000 7fd44f7fe640 1 -- 172.21.15.2:0/3993953862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 msgr2=0x55f0e0c29640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.964+0000 7fd44f7fe640 1 --2- 172.21.15.2:0/3993953862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 0x55f0e0c29640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.964+0000 7fd44f7fe640 1 -- 172.21.15.2:0/3993953862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 msgr2=0x55f0e0c30ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.964+0000 7fd44f7fe640 1 --2- 172.21.15.2:0/3993953862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 0x55f0e0c30ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.964+0000 7fd44f7fe640 1 -- 172.21.15.2:0/3993953862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0e0962620 con 0x55f0e085d4d0 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.964+0000 7fd44f7fe640 1 --2- 172.21.15.2:0/3993953862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c29b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd44402bc40 tx=0x7fd444030f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d1879434f22f449 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.964+0000 7fd42f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.964+0000 7fd42f7fe640 1 -- 172.21.15.2:0/3993953862 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444036040 con 0x55f0e085d4d0 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.964+0000 7fd42f7fe640 1 -- 172.21.15.2:0/3993953862 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fd444004040 con 0x55f0e085d4d0 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.964+0000 7fd45d80b740 1 -- 172.21.15.2:0/3993953862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd448003e20 con 0x55f0e085d4d0 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.964+0000 7fd42f7fe640 1 -- 172.21.15.2:0/3993953862 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fd44404f050 con 0x55f0e085d4d0 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.964+0000 7fd45d80b740 10 client.26861 fetch_fsmap learned FSMap version 62 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.964+0000 7fd45d80b740 1 -- 172.21.15.2:0/3993953862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55f0e0c33a80 con 0x55f0e085d4d0 2024-05-14T09:54:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.965+0000 7fd42f7fe640 1 -- 172.21.15.2:0/3993953862 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd448003e20 con 0x55f0e085d4d0 2024-05-14T09:54:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.965+0000 7fd42f7fe640 1 -- 172.21.15.2:0/3993953862 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 62) ==== 8350+0+0 (secure 0 0 0) 0x7fd44400b040 con 0x55f0e085d4d0 2024-05-14T09:54:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.965+0000 7fd45d80b740 10 client.26861 fetch_fsmap finished waiting for FSMap version 62 2024-05-14T09:54:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.965+0000 7fd45d80b740 10 client.26861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:54:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.965+0000 7fd45d80b740 20 client.26861 populate_metadata read hostname 'smithi002' 2024-05-14T09:54:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.965+0000 7fd45d80b740 1 --2- 172.21.15.2:0/3993953862 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f0e083cd80 0x55f0e0c37180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.965+0000 7fd45d80b740 4 client.26861 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:54:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.965+0000 7fd45d80b740 1 -- 172.21.15.2:0/3993953862 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0e0b921e0 con 0x55f0e083cd80 2024-05-14T09:54:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.965+0000 7fd454efc640 1 --2- 172.21.15.2:0/3993953862 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f0e083cd80 0x55f0e0c37180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.966+0000 7fd42f7fe640 10 client.26861 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.966+0000 7fd454efc640 1 --2- 172.21.15.2:0/3993953862 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f0e083cd80 0x55f0e0c37180 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f7efec43721b8c server_cookie=293048911d17da76 in_seq=0 out_seq=0 2024-05-14T09:54:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.969+0000 7fd42f7fe640 1 -- 172.21.15.2:0/3993953862 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f0e0b921e0 con 0x55f0e083cd80 2024-05-14T09:54:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.969+0000 7fd42f7fe640 10 client.26861 handle_command_reply: tid=0 2024-05-14T09:54:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.969+0000 7fd45d80b740 1 client.26861 shutdown 2024-05-14T09:54:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.970+0000 7fd45d80b740 1 -- 172.21.15.2:0/3993953862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 msgr2=0x55f0e0c29b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.970+0000 7fd45d80b740 1 --2- 172.21.15.2:0/3993953862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c29b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd44402bc40 tx=0x7fd444030f90 comp rx=0 tx=0).stop 2024-05-14T09:54:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.970+0000 7fd45d80b740 1 -- 172.21.15.2:0/3993953862 shutdown_connections 2024-05-14T09:54:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.970+0000 7fd45d80b740 1 --2- 172.21.15.2:0/3993953862 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f0e083cd80 0x55f0e0c37180 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.970+0000 7fd45d80b740 1 --2- 172.21.15.2:0/3993953862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 0x55f0e0c30ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.970+0000 7fd45d80b740 1 --2- 172.21.15.2:0/3993953862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c29b80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.970+0000 7fd45d80b740 1 --2- 172.21.15.2:0/3993953862 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 0x55f0e0c29640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.970+0000 7fd45d80b740 1 -- 172.21.15.2:0/3993953862 >> 172.21.15.2:0/3993953862 conn(0x55f0e0872300 msgr2=0x55f0e0c1a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.970+0000 7fd42f7fe640 0 client.26861 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.970+0000 7fd45d80b740 1 -- 172.21.15.2:0/3993953862 shutdown_connections 2024-05-14T09:54:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.970+0000 7fd45d80b740 1 -- 172.21.15.2:0/3993953862 wait complete. 2024-05-14T09:54:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.970+0000 7fd45d80b740 20 client.26861 trim_cache size 0 max 16384 2024-05-14T09:54:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.988+0000 7fd45d80b740 1 Processor -- start 2024-05-14T09:54:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.988+0000 7fd45d80b740 1 -- start start 2024-05-14T09:54:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd45d80b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 0x55f0e0c23aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd45d80b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e085d4d0 0x55f0e0c23fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd45d80b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e087e1b0 0x55f0e0c1acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd45d80b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501ad3a0 con 0x55f0e085d4d0 2024-05-14T09:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd45d80b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501aac70 con 0x55f0e087e1b0 2024-05-14T09:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd45d80b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501a8ac0 con 0x55f0e08552f0 2024-05-14T09:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd44f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e085d4d0 0x55f0e0c23fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd44f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e085d4d0 0x55f0e0c23fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54050/0 (socket says 172.21.15.2:54050) 2024-05-14T09:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd44f7fe640 1 -- 172.21.15.2:0/3091023582 learned_addr learned my addr 172.21.15.2:0/3091023582 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd454efc640 1 --2- 172.21.15.2:0/3091023582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 0x55f0e0c23aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd454efc640 1 --2- 172.21.15.2:0/3091023582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e087e1b0 0x55f0e0c1acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd44f7fe640 1 -- 172.21.15.2:0/3091023582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 msgr2=0x55f0e0c23aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd44f7fe640 1 --2- 172.21.15.2:0/3091023582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 0x55f0e0c23aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd44f7fe640 1 -- 172.21.15.2:0/3091023582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e087e1b0 msgr2=0x55f0e0c1acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd44f7fe640 1 --2- 172.21.15.2:0/3091023582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e087e1b0 0x55f0e0c1acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd44f7fe640 1 -- 172.21.15.2:0/3091023582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0e0c1bd00 con 0x55f0e085d4d0 2024-05-14T09:54:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd44f7fe640 1 --2- 172.21.15.2:0/3091023582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e085d4d0 0x55f0e0c23fe0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd444003f20 tx=0x7fd444001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf41e4f2a137ec6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.989+0000 7fd435ffb640 1 -- 172.21.15.2:0/3091023582 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444036070 con 0x55f0e085d4d0 2024-05-14T09:54:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.990+0000 7fd435ffb640 1 -- 172.21.15.2:0/3091023582 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd444042070 con 0x55f0e085d4d0 2024-05-14T09:54:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.990+0000 7fd435ffb640 1 -- 172.21.15.2:0/3091023582 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444004260 con 0x55f0e085d4d0 2024-05-14T09:54:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.990+0000 7fd45d80b740 1 -- 172.21.15.2:0/3091023582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e085d4d0 msgr2=0x55f0e0c23fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.990+0000 7fd45d80b740 1 --2- 172.21.15.2:0/3091023582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e085d4d0 0x55f0e0c23fe0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd444003f20 tx=0x7fd444001e70 comp rx=0 tx=0).stop 2024-05-14T09:54:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.990+0000 7fd45d80b740 1 -- 172.21.15.2:0/3091023582 shutdown_connections 2024-05-14T09:54:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.990+0000 7fd45d80b740 1 --2- 172.21.15.2:0/3091023582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e087e1b0 0x55f0e0c1acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.990+0000 7fd45d80b740 1 --2- 172.21.15.2:0/3091023582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e085d4d0 0x55f0e0c23fe0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.990+0000 7fd45d80b740 1 --2- 172.21.15.2:0/3091023582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 0x55f0e0c23aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.990+0000 7fd45d80b740 1 -- 172.21.15.2:0/3091023582 >> 172.21.15.2:0/3091023582 conn(0x55f0e0872300 msgr2=0x55f0e0c30df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.991+0000 7fd45d80b740 1 -- 172.21.15.2:0/3091023582 shutdown_connections 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.991+0000 7fd45d80b740 1 -- 172.21.15.2:0/3091023582 wait complete. 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.991+0000 7fd45d80b740 1 -- start start 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.991+0000 7fd45d80b740 1 Processor -- start 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.991+0000 7fd45d80b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 0x55f0e0c30270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.991+0000 7fd45d80b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c307b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.991+0000 7fd45d80b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 0x55f0e0ca15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.991+0000 7fd45d80b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501a9130 con 0x55f0e087e1b0 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.991+0000 7fd45d80b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501aa5a0 con 0x55f0e085d4d0 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.991+0000 7fd45d80b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501a9800 con 0x55f0e08552f0 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.991+0000 7fd45d80b740 10 client.0 _ll_register_callbacks cb 0x7fd44400a890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd454efc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 0x55f0e0ca15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd44f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c307b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd44f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c307b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:60560/0 (socket says 172.21.15.2:60560) 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd44f7fe640 1 -- 172.21.15.2:0/693078328 learned_addr learned my addr 172.21.15.2:0/693078328 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd454efc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 0x55f0e0ca15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54056/0 (socket says 172.21.15.2:54056) 2024-05-14T09:54:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd454efc640 1 --2- 172.21.15.2:0/693078328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 0x55f0e0c30270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd44f7fe640 1 -- 172.21.15.2:0/693078328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 msgr2=0x55f0e0c30270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd44f7fe640 1 --2- 172.21.15.2:0/693078328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 0x55f0e0c30270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd44f7fe640 1 -- 172.21.15.2:0/693078328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 msgr2=0x55f0e0ca15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd44f7fe640 1 --2- 172.21.15.2:0/693078328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 0x55f0e0ca15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd44f7fe640 1 -- 172.21.15.2:0/693078328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0e0ca1bc0 con 0x55f0e085d4d0 2024-05-14T09:54:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd44f7fe640 1 --2- 172.21.15.2:0/693078328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c307b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd44402b760 tx=0x7fd44400a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b3a79516afc75b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd42f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:54:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd42f7fe640 1 -- 172.21.15.2:0/693078328 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444036070 con 0x55f0e085d4d0 2024-05-14T09:54:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd45d80b740 1 -- 172.21.15.2:0/693078328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd444042070 con 0x55f0e085d4d0 2024-05-14T09:54:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.992+0000 7fd42f7fe640 1 -- 172.21.15.2:0/693078328 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7fd444004000 con 0x55f0e085d4d0 2024-05-14T09:54:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.993+0000 7fd42f7fe640 1 -- 172.21.15.2:0/693078328 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fd44404c040 con 0x55f0e085d4d0 2024-05-14T09:54:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.993+0000 7fd45d80b740 10 client.26867 fetch_fsmap learned FSMap version 62 2024-05-14T09:54:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.993+0000 7fd45d80b740 1 -- 172.21.15.2:0/693078328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55f0e0ca2e30 con 0x55f0e085d4d0 2024-05-14T09:54:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.993+0000 7fd42f7fe640 1 -- 172.21.15.2:0/693078328 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd444042070 con 0x55f0e085d4d0 2024-05-14T09:54:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.993+0000 7fd42f7fe640 1 -- 172.21.15.2:0/693078328 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 62) ==== 8350+0+0 (secure 0 0 0) 0x7fd444004200 con 0x55f0e085d4d0 2024-05-14T09:54:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.993+0000 7fd45d80b740 10 client.26867 fetch_fsmap finished waiting for FSMap version 62 2024-05-14T09:54:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.993+0000 7fd45d80b740 10 client.26867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:54:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.993+0000 7fd45d80b740 20 client.26867 populate_metadata read hostname 'smithi002' 2024-05-14T09:54:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.993+0000 7fd45d80b740 1 --2- 172.21.15.2:0/693078328 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f0e08121c0 0x55f0e0ca5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.993+0000 7fd45d80b740 4 client.26867 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:54:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.993+0000 7fd45d80b740 1 -- 172.21.15.2:0/693078328 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f0e0b8c050 con 0x55f0e08121c0 2024-05-14T09:54:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.993+0000 7fd454efc640 1 --2- 172.21.15.2:0/693078328 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f0e08121c0 0x55f0e0ca5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.994+0000 7fd42f7fe640 10 client.26867 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:00.994+0000 7fd454efc640 1 --2- 172.21.15.2:0/693078328 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f0e08121c0 0x55f0e0ca5730 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=567440197a68f326 server_cookie=1a2cecd58e2e12b7 in_seq=0 out_seq=0 2024-05-14T09:54:01.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 175221 28.32 MB/sec execute 207 sec latency 522.529 ms 2024-05-14T09:54:02.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 175241 28.19 MB/sec execute 208 sec latency 1522.574 ms 2024-05-14T09:54:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:02 smithi002 ceph-mon[31557]: pgmap v997: 129 pgs: 129 active+clean; 684 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.4 MiB/s wr, 492 op/s 2024-05-14T09:54:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:02 smithi029 ceph-mon[36249]: pgmap v997: 129 pgs: 129 active+clean; 684 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.4 MiB/s wr, 492 op/s 2024-05-14T09:54:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:02 smithi026 ceph-mon[36056]: pgmap v997: 129 pgs: 129 active+clean; 684 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.4 MiB/s wr, 492 op/s 2024-05-14T09:54:03.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 175261 28.06 MB/sec execute 209 sec latency 2522.622 ms 2024-05-14T09:54:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.833+0000 7fd42f7fe640 1 -- 172.21.15.2:0/693078328 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f0e0b8c050 con 0x55f0e08121c0 2024-05-14T09:54:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.833+0000 7fd42f7fe640 10 client.26867 handle_command_reply: tid=0 2024-05-14T09:54:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.833+0000 7fd45d80b740 1 client.26867 shutdown 2024-05-14T09:54:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.833+0000 7fd45d80b740 1 -- 172.21.15.2:0/693078328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 msgr2=0x55f0e0c307b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.834+0000 7fd45d80b740 1 --2- 172.21.15.2:0/693078328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c307b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd44402b760 tx=0x7fd44400a6a0 comp rx=0 tx=0).stop 2024-05-14T09:54:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.834+0000 7fd45d80b740 1 -- 172.21.15.2:0/693078328 shutdown_connections 2024-05-14T09:54:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.834+0000 7fd45d80b740 1 --2- 172.21.15.2:0/693078328 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f0e08121c0 0x55f0e0ca5730 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.834+0000 7fd45d80b740 1 --2- 172.21.15.2:0/693078328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f0e087e1b0 0x55f0e0ca15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.834+0000 7fd42f7fe640 0 client.26867 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.834+0000 7fd45d80b740 1 --2- 172.21.15.2:0/693078328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e085d4d0 0x55f0e0c307b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.834+0000 7fd45d80b740 1 --2- 172.21.15.2:0/693078328 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f0e08552f0 0x55f0e0c30270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.834+0000 7fd45d80b740 1 -- 172.21.15.2:0/693078328 >> 172.21.15.2:0/693078328 conn(0x55f0e0872300 msgr2=0x55f0e0c1acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.834+0000 7fd45d80b740 1 -- 172.21.15.2:0/693078328 shutdown_connections 2024-05-14T09:54:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.835+0000 7fd45d80b740 1 -- 172.21.15.2:0/693078328 wait complete. 2024-05-14T09:54:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.835+0000 7fd45d80b740 20 client.26867 trim_cache size 0 max 16384 2024-05-14T09:54:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:54:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:54:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:54:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:54:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.836+0000 7fd45695f640 1 -- 172.21.15.2:0/2850363060 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd4380b28a0 msgr2=0x7fd4380b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.836+0000 7fd45695f640 1 --2- 172.21.15.2:0/2850363060 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd4380b28a0 0x7fd4380b4db0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd44000ead0 tx=0x7fd4400e5040 comp rx=0 tx=0).stop 2024-05-14T09:54:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.836+0000 7fd45695f640 1 -- 172.21.15.2:0/2850363060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4501022b0 msgr2=0x7fd4501980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.836+0000 7fd45695f640 1 --2- 172.21.15.2:0/2850363060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4501022b0 0x7fd4501980a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd44000eb00 tx=0x7fd440009ec0 comp rx=0 tx=0).stop 2024-05-14T09:54:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.836+0000 7fd45695f640 1 -- 172.21.15.2:0/2850363060 shutdown_connections 2024-05-14T09:54:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.836+0000 7fd45695f640 1 --2- 172.21.15.2:0/2850363060 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd4380b28a0 0x7fd4380b4db0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.836+0000 7fd45695f640 1 --2- 172.21.15.2:0/2850363060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4501081f0 0x7fd45019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.837+0000 7fd45695f640 1 --2- 172.21.15.2:0/2850363060 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd4501034f0 0x7fd4501985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.837+0000 7fd45695f640 1 --2- 172.21.15.2:0/2850363060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4501022b0 0x7fd4501980a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.837+0000 7fd45695f640 1 -- 172.21.15.2:0/2850363060 >> 172.21.15.2:0/2850363060 conn(0x7fd4500fd480 msgr2=0x7fd4501041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.837+0000 7fd45695f640 1 -- 172.21.15.2:0/2850363060 shutdown_connections 2024-05-14T09:54:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:03.837+0000 7fd45695f640 1 -- 172.21.15.2:0/2850363060 wait complete. 2024-05-14T09:54:04.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 175420 27.94 MB/sec execute 210 sec latency 2756.463 ms 2024-05-14T09:54:04.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:04 smithi002 ceph-mon[31557]: pgmap v998: 129 pgs: 129 active+clean; 726 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.9 MiB/s wr, 602 op/s 2024-05-14T09:54:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:04 smithi029 ceph-mon[36249]: pgmap v998: 129 pgs: 129 active+clean; 726 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.9 MiB/s wr, 602 op/s 2024-05-14T09:54:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:04 smithi026 ceph-mon[36056]: pgmap v998: 129 pgs: 129 active+clean; 726 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.9 MiB/s wr, 602 op/s 2024-05-14T09:54:05.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 175649 27.84 MB/sec execute 211 sec latency 280.372 ms 2024-05-14T09:54:06.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 175995 27.79 MB/sec execute 212 sec latency 378.889 ms 2024-05-14T09:54:06.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:06 smithi029 ceph-mon[36249]: pgmap v999: 129 pgs: 129 active+clean; 752 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 9.0 MiB/s wr, 587 op/s 2024-05-14T09:54:06.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:06 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:54:06.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:06 smithi026 ceph-mon[36056]: pgmap v999: 129 pgs: 129 active+clean; 752 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 9.0 MiB/s wr, 587 op/s 2024-05-14T09:54:06.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:06 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:54:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:06 smithi002 ceph-mon[31557]: pgmap v999: 129 pgs: 129 active+clean; 752 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 9.0 MiB/s wr, 587 op/s 2024-05-14T09:54:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:06 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:54:07.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 176381 27.79 MB/sec execute 213 sec latency 278.629 ms 2024-05-14T09:54:08.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 176855 27.81 MB/sec execute 214 sec latency 173.632 ms 2024-05-14T09:54:08.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:08 smithi029 ceph-mon[36249]: pgmap v1000: 129 pgs: 129 active+clean; 744 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 619 op/s 2024-05-14T09:54:08.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:08 smithi026 ceph-mon[36056]: pgmap v1000: 129 pgs: 129 active+clean; 744 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 619 op/s 2024-05-14T09:54:09.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:08 smithi002 ceph-mon[31557]: pgmap v1000: 129 pgs: 129 active+clean; 744 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 619 op/s 2024-05-14T09:54:09.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 177266 27.81 MB/sec execute 215 sec latency 105.478 ms 2024-05-14T09:54:10.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 177698 27.76 MB/sec execute 216 sec latency 290.091 ms 2024-05-14T09:54:10.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:10 smithi026 ceph-mon[36056]: pgmap v1001: 129 pgs: 129 active+clean; 710 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 13 MiB/s wr, 736 op/s 2024-05-14T09:54:11.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:10 smithi002 ceph-mon[31557]: pgmap v1001: 129 pgs: 129 active+clean; 710 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 13 MiB/s wr, 736 op/s 2024-05-14T09:54:11.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:10 smithi029 ceph-mon[36249]: pgmap v1001: 129 pgs: 129 active+clean; 710 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 13 MiB/s wr, 736 op/s 2024-05-14T09:54:11.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 178172 27.74 MB/sec execute 217 sec latency 200.737 ms 2024-05-14T09:54:12.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 178824 27.74 MB/sec execute 218 sec latency 47.806 ms 2024-05-14T09:54:12.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:12 smithi026 ceph-mon[36056]: pgmap v1002: 129 pgs: 129 active+clean; 684 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 13 MiB/s wr, 795 op/s 2024-05-14T09:54:13.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:12 smithi002 ceph-mon[31557]: pgmap v1002: 129 pgs: 129 active+clean; 684 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 13 MiB/s wr, 795 op/s 2024-05-14T09:54:13.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:12 smithi029 ceph-mon[36249]: pgmap v1002: 129 pgs: 129 active+clean; 684 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 13 MiB/s wr, 795 op/s 2024-05-14T09:54:13.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 179527 27.84 MB/sec execute 219 sec latency 106.167 ms 2024-05-14T09:54:14.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 180090 27.85 MB/sec execute 220 sec latency 176.925 ms 2024-05-14T09:54:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:14 smithi026 ceph-mon[36056]: pgmap v1003: 129 pgs: 129 active+clean; 663 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 869 op/s 2024-05-14T09:54:15.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:14 smithi002 ceph-mon[31557]: pgmap v1003: 129 pgs: 129 active+clean; 663 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 869 op/s 2024-05-14T09:54:15.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:14 smithi029 ceph-mon[36249]: pgmap v1003: 129 pgs: 129 active+clean; 663 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 869 op/s 2024-05-14T09:54:15.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 180655 27.85 MB/sec execute 221 sec latency 105.785 ms 2024-05-14T09:54:16.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 181139 27.85 MB/sec execute 222 sec latency 264.920 ms 2024-05-14T09:54:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:16 smithi026 ceph-mon[36056]: pgmap v1004: 129 pgs: 129 active+clean; 662 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 9.1 MiB/s wr, 654 op/s 2024-05-14T09:54:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.965+0000 7ff2f5f2e640 1 -- 172.21.15.2:0/3562190176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2f0104740 msgr2=0x7ff2f0106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.965+0000 7ff2f5f2e640 1 --2- 172.21.15.2:0/3562190176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2f0104740 0x7ff2f0106b60 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009900 tx=0x7ff2e002a410 comp rx=0 tx=0).stop 2024-05-14T09:54:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.966+0000 7ff2f5f2e640 1 -- 172.21.15.2:0/3562190176 shutdown_connections 2024-05-14T09:54:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.966+0000 7ff2f5f2e640 1 --2- 172.21.15.2:0/3562190176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2f0109a00 0x7ff2f010be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.966+0000 7ff2f5f2e640 1 --2- 172.21.15.2:0/3562190176 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2f01070a0 0x7ff2f01094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.966+0000 7ff2f5f2e640 1 --2- 172.21.15.2:0/3562190176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2f0104740 0x7ff2f0106b60 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.966+0000 7ff2f5f2e640 1 -- 172.21.15.2:0/3562190176 >> 172.21.15.2:0/3562190176 conn(0x7ff2f00fdd40 msgr2=0x7ff2f01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.966+0000 7ff2f5f2e640 1 -- 172.21.15.2:0/3562190176 shutdown_connections 2024-05-14T09:54:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.966+0000 7ff2f5f2e640 1 -- 172.21.15.2:0/3562190176 wait complete. 2024-05-14T09:54:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.967+0000 7ff2f5f2e640 1 Processor -- start 2024-05-14T09:54:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.967+0000 7ff2f5f2e640 1 -- start start 2024-05-14T09:54:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.967+0000 7ff2f5f2e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2f0104740 0x7ff2f0103b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.967+0000 7ff2f5f2e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2f01070a0 0x7ff2f0102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.967+0000 7ff2f5f2e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2f0109a00 0x7ff2f0102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.967+0000 7ff2f5f2e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f010f730 con 0x7ff2f01070a0 2024-05-14T09:54:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.967+0000 7ff2f5f2e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f010f5b0 con 0x7ff2f0104740 2024-05-14T09:54:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.967+0000 7ff2f5f2e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f010f8b0 con 0x7ff2f0109a00 2024-05-14T09:54:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.968+0000 7ff2eeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2f01070a0 0x7ff2f0102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.968+0000 7ff2eeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2f01070a0 0x7ff2f0102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58998/0 (socket says 172.21.15.2:58998) 2024-05-14T09:54:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.968+0000 7ff2eeffd640 1 -- 172.21.15.2:0/740460042 learned_addr learned my addr 172.21.15.2:0/740460042 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.968+0000 7ff2ef7fe640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2f0104740 0x7ff2f0103b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.968+0000 7ff2effff640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2f0109a00 0x7ff2f0102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.968+0000 7ff2ef7fe640 1 -- 172.21.15.2:0/740460042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2f0109a00 msgr2=0x7ff2f0102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.968+0000 7ff2ef7fe640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2f0109a00 0x7ff2f0102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.968+0000 7ff2ef7fe640 1 -- 172.21.15.2:0/740460042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2f01070a0 msgr2=0x7ff2f0102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.968+0000 7ff2ef7fe640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2f01070a0 0x7ff2f0102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.968+0000 7ff2ef7fe640 1 -- 172.21.15.2:0/740460042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f019e190 con 0x7ff2f0104740 2024-05-14T09:54:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.968+0000 7ff2eeffd640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2f01070a0 0x7ff2f0102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:54:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.968+0000 7ff2effff640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2f0109a00 0x7ff2f0102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:54:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.969+0000 7ff2ef7fe640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2f0104740 0x7ff2f0103b90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0006010 tx=0x7ff2e0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.969+0000 7ff2ecff9640 1 -- 172.21.15.2:0/740460042 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e003f070 con 0x7ff2f0104740 2024-05-14T09:54:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.969+0000 7ff2f5f2e640 1 -- 172.21.15.2:0/740460042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f0102d70 con 0x7ff2f0104740 2024-05-14T09:54:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.969+0000 7ff2ecff9640 1 -- 172.21.15.2:0/740460042 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e0031910 con 0x7ff2f0104740 2024-05-14T09:54:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.969+0000 7ff2ecff9640 1 -- 172.21.15.2:0/740460042 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0035070 con 0x7ff2f0104740 2024-05-14T09:54:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.969+0000 7ff2f5f2e640 1 -- 172.21.15.2:0/740460042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f01a8d90 con 0x7ff2f0104740 2024-05-14T09:54:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.972+0000 7ff2ecff9640 1 -- 172.21.15.2:0/740460042 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff2e00037f0 con 0x7ff2f0104740 2024-05-14T09:54:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.972+0000 7ff2fcddc740 1 Processor -- start 2024-05-14T09:54:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.972+0000 7ff2fcddc740 1 -- start start 2024-05-14T09:54:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2fcddc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db521b0 0x55720def3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2fcddc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db292f0 0x55720def3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2fcddc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db314d0 0x55720defaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2fcddc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55720de585d0 con 0x55720db292f0 2024-05-14T09:54:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2fcddc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55720de11890 con 0x55720db521b0 2024-05-14T09:54:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2fcddc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55720de54590 con 0x55720db314d0 2024-05-14T09:54:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2eeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db314d0 0x55720defaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2eeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db314d0 0x55720defaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:45398/0 (socket says 172.21.15.2:45398) 2024-05-14T09:54:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2eeffd640 1 -- 172.21.15.2:0/2722661905 learned_addr learned my addr 172.21.15.2:0/2722661905 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2effff640 1 --2- 172.21.15.2:0/2722661905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db521b0 0x55720def3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2eeffd640 1 -- 172.21.15.2:0/2722661905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db521b0 msgr2=0x55720def3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2eeffd640 1 --2- 172.21.15.2:0/2722661905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db521b0 0x55720def3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2ef7fe640 1 --2- 172.21.15.2:0/2722661905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db292f0 0x55720def3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2eeffd640 1 -- 172.21.15.2:0/2722661905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db292f0 msgr2=0x55720def3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2eeffd640 1 --2- 172.21.15.2:0/2722661905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db292f0 0x55720def3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2eeffd640 1 -- 172.21.15.2:0/2722661905 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55720db6a210 con 0x55720db314d0 2024-05-14T09:54:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.973+0000 7ff2ef7fe640 1 --2- 172.21.15.2:0/2722661905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db292f0 0x55720def3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:54:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.974+0000 7ff2eeffd640 1 --2- 172.21.15.2:0/2722661905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db314d0 0x55720defaee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40115f0 tx=0x7ff2e4011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bc2feb6ceb7b137 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.974+0000 7ff2d5ffb640 1 -- 172.21.15.2:0/2722661905 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4004020 con 0x55720db314d0 2024-05-14T09:54:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.974+0000 7ff2ecff9640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff2d80b2920 0x7ff2d80b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.974+0000 7ff2d5ffb640 1 -- 172.21.15.2:0/2722661905 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e40054d0 con 0x55720db314d0 2024-05-14T09:54:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.974+0000 7ff2d5ffb640 1 -- 172.21.15.2:0/2722661905 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4004020 con 0x55720db314d0 2024-05-14T09:54:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.974+0000 7ff2ecff9640 1 -- 172.21.15.2:0/740460042 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7ff2e01026b0 con 0x7ff2f0104740 2024-05-14T09:54:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.974+0000 7ff2effff640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff2d80b2920 0x7ff2d80b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.975+0000 7ff2fcddc740 1 -- 172.21.15.2:0/2722661905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db314d0 msgr2=0x55720defaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.975+0000 7ff2effff640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff2d80b2920 0x7ff2d80b4e30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55720def4c30 tx=0x7ff2e8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.975+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/2722661905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db314d0 0x55720defaee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40115f0 tx=0x7ff2e4011ab0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.975+0000 7ff2fcddc740 1 -- 172.21.15.2:0/2722661905 shutdown_connections 2024-05-14T09:54:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.975+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/2722661905 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db314d0 0x55720defaee0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.975+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/2722661905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db292f0 0x55720def3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.975+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/2722661905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db521b0 0x55720def3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.975+0000 7ff2fcddc740 1 -- 172.21.15.2:0/2722661905 >> 172.21.15.2:0/2722661905 conn(0x55720db46300 msgr2=0x55720deef3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.975+0000 7ff2fcddc740 1 -- 172.21.15.2:0/2722661905 shutdown_connections 2024-05-14T09:54:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.975+0000 7ff2fcddc740 1 -- 172.21.15.2:0/2722661905 wait complete. 2024-05-14T09:54:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.975+0000 7ff2fcddc740 1 -- start start 2024-05-14T09:54:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.976+0000 7ff2fcddc740 1 Processor -- start 2024-05-14T09:54:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.976+0000 7ff2fcddc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db292f0 0x55720defd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.976+0000 7ff2fcddc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db314d0 0x55720defdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.976+0000 7ff2fcddc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db521b0 0x55720df04ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.976+0000 7ff2fcddc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55720defb930 con 0x55720db521b0 2024-05-14T09:54:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.976+0000 7ff2fcddc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55720de15370 con 0x55720db314d0 2024-05-14T09:54:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.976+0000 7ff2fcddc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55720defbab0 con 0x55720db292f0 2024-05-14T09:54:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.976+0000 7ff2ef7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db292f0 0x55720defd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.976+0000 7ff2fcddc740 10 client.0 _ll_register_callbacks cb 0x55720db69a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:54:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.976+0000 7ff2ef7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db292f0 0x55720defd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:45400/0 (socket says 172.21.15.2:45400) 2024-05-14T09:54:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.976+0000 7ff2ef7fe640 1 -- 172.21.15.2:0/1907833252 learned_addr learned my addr 172.21.15.2:0/1907833252 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.976+0000 7ff2effff640 1 --2- 172.21.15.2:0/1907833252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db521b0 0x55720df04ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.976+0000 7ff2eeffd640 1 --2- 172.21.15.2:0/1907833252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db314d0 0x55720defdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.977+0000 7ff2ef7fe640 1 -- 172.21.15.2:0/1907833252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db314d0 msgr2=0x55720defdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.977+0000 7ff2ef7fe640 1 --2- 172.21.15.2:0/1907833252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db314d0 0x55720defdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.977+0000 7ff2ef7fe640 1 -- 172.21.15.2:0/1907833252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db521b0 msgr2=0x55720df04ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.977+0000 7ff2ef7fe640 1 --2- 172.21.15.2:0/1907833252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db521b0 0x55720df04ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.977+0000 7ff2ef7fe640 1 -- 172.21.15.2:0/1907833252 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55720dc36620 con 0x55720db292f0 2024-05-14T09:54:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.977+0000 7ff2ef7fe640 1 --2- 172.21.15.2:0/1907833252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db292f0 0x55720defd620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0005ab0 tx=0x7ff2e00c92b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8b084ff9717ea94 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.977+0000 7ff2ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:54:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.977+0000 7ff2ceffd640 1 -- 172.21.15.2:0/1907833252 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0048070 con 0x55720db292f0 2024-05-14T09:54:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.977+0000 7ff2fcddc740 1 -- 172.21.15.2:0/1907833252 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e40054d0 con 0x55720db292f0 2024-05-14T09:54:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.977+0000 7ff2ceffd640 1 -- 172.21.15.2:0/1907833252 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7ff2e00c5bf0 con 0x55720db292f0 2024-05-14T09:54:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.977+0000 7ff2ceffd640 1 -- 172.21.15.2:0/1907833252 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7ff2e00c9470 con 0x55720db292f0 2024-05-14T09:54:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.977+0000 7ff2fcddc740 10 client.26881 fetch_fsmap learned FSMap version 63 2024-05-14T09:54:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.978+0000 7ff2fcddc740 1 -- 172.21.15.2:0/1907833252 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55720defb9f0 con 0x55720db292f0 2024-05-14T09:54:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.978+0000 7ff2ceffd640 1 -- 172.21.15.2:0/1907833252 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e40054d0 con 0x55720db292f0 2024-05-14T09:54:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.978+0000 7ff2ceffd640 1 -- 172.21.15.2:0/1907833252 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 63) ==== 8346+0+0 (secure 0 0 0) 0x7ff2e0102020 con 0x55720db292f0 2024-05-14T09:54:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.978+0000 7ff2fcddc740 10 client.26881 fetch_fsmap finished waiting for FSMap version 63 2024-05-14T09:54:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.978+0000 7ff2fcddc740 10 client.26881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:54:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.978+0000 7ff2fcddc740 20 client.26881 populate_metadata read hostname 'smithi002' 2024-05-14T09:54:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.978+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/1907833252 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55720db10d80 0x55720df0b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.978+0000 7ff2fcddc740 4 client.26881 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:54:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.978+0000 7ff2fcddc740 1 -- 172.21.15.2:0/1907833252 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55720de661e0 con 0x55720db10d80 2024-05-14T09:54:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.978+0000 7ff2eeffd640 1 --2- 172.21.15.2:0/1907833252 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55720db10d80 0x55720df0b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.979+0000 7ff2ceffd640 10 client.26881 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.979+0000 7ff2eeffd640 1 --2- 172.21.15.2:0/1907833252 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55720db10d80 0x55720df0b060 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d83267f649c89f02 server_cookie=fff5ca9243b95e89 in_seq=0 out_seq=0 2024-05-14T09:54:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.983+0000 7ff2ceffd640 1 -- 172.21.15.2:0/1907833252 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55720de661e0 con 0x55720db10d80 2024-05-14T09:54:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.983+0000 7ff2ceffd640 10 client.26881 handle_command_reply: tid=0 2024-05-14T09:54:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.983+0000 7ff2fcddc740 1 client.26881 shutdown 2024-05-14T09:54:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.983+0000 7ff2fcddc740 1 -- 172.21.15.2:0/1907833252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db292f0 msgr2=0x55720defd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.983+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/1907833252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db292f0 0x55720defd620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0005ab0 tx=0x7ff2e00c92b0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.983+0000 7ff2fcddc740 1 -- 172.21.15.2:0/1907833252 shutdown_connections 2024-05-14T09:54:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.983+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/1907833252 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55720db10d80 0x55720df0b060 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.983+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/1907833252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db521b0 0x55720df04ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.983+0000 7ff2ceffd640 0 client.26881 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.983+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/1907833252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db314d0 0x55720defdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.983+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/1907833252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db292f0 0x55720defd620 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.983+0000 7ff2fcddc740 1 -- 172.21.15.2:0/1907833252 >> 172.21.15.2:0/1907833252 conn(0x55720db46300 msgr2=0x55720deee3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.984+0000 7ff2fcddc740 1 -- 172.21.15.2:0/1907833252 shutdown_connections 2024-05-14T09:54:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.984+0000 7ff2fcddc740 1 -- 172.21.15.2:0/1907833252 wait complete. 2024-05-14T09:54:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:16.984+0000 7ff2fcddc740 20 client.26881 trim_cache size 0 max 16384 2024-05-14T09:54:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.004+0000 7ff2fcddc740 1 Processor -- start 2024-05-14T09:54:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.004+0000 7ff2fcddc740 1 -- start start 2024-05-14T09:54:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.004+0000 7ff2fcddc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db10d80 0x55720defd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.004+0000 7ff2fcddc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db292f0 0x55720df04a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.004+0000 7ff2fcddc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db314d0 0x55720deed970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.004+0000 7ff2fcddc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f01b5b80 con 0x55720db292f0 2024-05-14T09:54:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.004+0000 7ff2fcddc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f01b3450 con 0x55720db314d0 2024-05-14T09:54:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.004+0000 7ff2fcddc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f01b12a0 con 0x55720db10d80 2024-05-14T09:54:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.004+0000 7ff2eeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db292f0 0x55720df04a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.004+0000 7ff2eeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db292f0 0x55720df04a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59018/0 (socket says 172.21.15.2:59018) 2024-05-14T09:54:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.004+0000 7ff2ef7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db10d80 0x55720defd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.004+0000 7ff2eeffd640 1 -- 172.21.15.2:0/256852602 learned_addr learned my addr 172.21.15.2:0/256852602 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.004+0000 7ff2effff640 1 --2- 172.21.15.2:0/256852602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db314d0 0x55720deed970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.005+0000 7ff2eeffd640 1 -- 172.21.15.2:0/256852602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db10d80 msgr2=0x55720defd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.005+0000 7ff2eeffd640 1 --2- 172.21.15.2:0/256852602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db10d80 0x55720defd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.005+0000 7ff2eeffd640 1 -- 172.21.15.2:0/256852602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db314d0 msgr2=0x55720deed970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.005+0000 7ff2eeffd640 1 --2- 172.21.15.2:0/256852602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db314d0 0x55720deed970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.005+0000 7ff2eeffd640 1 -- 172.21.15.2:0/256852602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55720defe690 con 0x55720db292f0 2024-05-14T09:54:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.005+0000 7ff2eeffd640 1 --2- 172.21.15.2:0/256852602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db292f0 0x55720df04a50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4013fd0 tx=0x7ff2e4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe394cd44ee4c9d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.005+0000 7ff2d57fa640 1 -- 172.21.15.2:0/256852602 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e402f020 con 0x55720db292f0 2024-05-14T09:54:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.005+0000 7ff2d57fa640 1 -- 172.21.15.2:0/256852602 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e4004400 con 0x55720db292f0 2024-05-14T09:54:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.005+0000 7ff2d57fa640 1 -- 172.21.15.2:0/256852602 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e401d040 con 0x55720db292f0 2024-05-14T09:54:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.006+0000 7ff2fcddc740 1 -- 172.21.15.2:0/256852602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db292f0 msgr2=0x55720df04a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.006+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/256852602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db292f0 0x55720df04a50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4013fd0 tx=0x7ff2e4011ab0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.006+0000 7ff2fcddc740 1 -- 172.21.15.2:0/256852602 shutdown_connections 2024-05-14T09:54:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.006+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/256852602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db314d0 0x55720deed970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.006+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/256852602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db292f0 0x55720df04a50 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.006+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/256852602 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db10d80 0x55720defd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.006+0000 7ff2fcddc740 1 -- 172.21.15.2:0/256852602 >> 172.21.15.2:0/256852602 conn(0x55720db46300 msgr2=0x55720defd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.006+0000 7ff2fcddc740 1 -- 172.21.15.2:0/256852602 shutdown_connections 2024-05-14T09:54:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.006+0000 7ff2fcddc740 1 -- 172.21.15.2:0/256852602 wait complete. 2024-05-14T09:54:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.007+0000 7ff2fcddc740 1 -- start start 2024-05-14T09:54:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.007+0000 7ff2fcddc740 1 Processor -- start 2024-05-14T09:54:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.007+0000 7ff2fcddc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db10d80 0x55720df01c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.007+0000 7ff2fcddc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db292f0 0x55720df021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.007+0000 7ff2fcddc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db314d0 0x55720df030a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.007+0000 7ff2fcddc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f01b1910 con 0x55720db10d80 2024-05-14T09:54:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.007+0000 7ff2fcddc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f01b2d80 con 0x55720db314d0 2024-05-14T09:54:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.007+0000 7ff2fcddc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f01b1fe0 con 0x55720db292f0 2024-05-14T09:54:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.007+0000 7ff2fcddc740 10 client.0 _ll_register_callbacks cb 0x7ff2e0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:54:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.007+0000 7ff2ef7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db10d80 0x55720df01c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.007+0000 7ff2ef7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db10d80 0x55720df01c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59032/0 (socket says 172.21.15.2:59032) 2024-05-14T09:54:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.007+0000 7ff2eeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db292f0 0x55720df021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.007+0000 7ff2ef7fe640 1 -- 172.21.15.2:0/575468598 learned_addr learned my addr 172.21.15.2:0/575468598 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.007+0000 7ff2effff640 1 --2- 172.21.15.2:0/575468598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db314d0 0x55720df030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.008+0000 7ff2ef7fe640 1 -- 172.21.15.2:0/575468598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db292f0 msgr2=0x55720df021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.008+0000 7ff2ef7fe640 1 --2- 172.21.15.2:0/575468598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db292f0 0x55720df021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.008+0000 7ff2ef7fe640 1 -- 172.21.15.2:0/575468598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db314d0 msgr2=0x55720df030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.008+0000 7ff2ef7fe640 1 --2- 172.21.15.2:0/575468598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db314d0 0x55720df030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.008+0000 7ff2ef7fe640 1 -- 172.21.15.2:0/575468598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55720df03660 con 0x55720db10d80 2024-05-14T09:54:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.008+0000 7ff2ef7fe640 1 --2- 172.21.15.2:0/575468598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db10d80 0x55720df01c80 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00d2850 tx=0x7ff2e0009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0880b28250c7a6c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.008+0000 7ff2ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:54:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.008+0000 7ff2ceffd640 1 -- 172.21.15.2:0/575468598 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0048070 con 0x55720db10d80 2024-05-14T09:54:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.008+0000 7ff2ceffd640 1 -- 172.21.15.2:0/575468598 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9282+0+0 (secure 0 0 0) 0x7ff2e00cadf0 con 0x55720db10d80 2024-05-14T09:54:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.008+0000 7ff2fcddc740 1 -- 172.21.15.2:0/575468598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e4004400 con 0x55720db10d80 2024-05-14T09:54:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.008+0000 7ff2ceffd640 1 -- 172.21.15.2:0/575468598 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7ff2e00c90d0 con 0x55720db10d80 2024-05-14T09:54:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.008+0000 7ff2fcddc740 10 client.17271 fetch_fsmap learned FSMap version 63 2024-05-14T09:54:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.008+0000 7ff2fcddc740 1 -- 172.21.15.2:0/575468598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55720df048f0 con 0x55720db10d80 2024-05-14T09:54:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.009+0000 7ff2ceffd640 1 -- 172.21.15.2:0/575468598 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e00cdc60 con 0x55720db10d80 2024-05-14T09:54:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.009+0000 7ff2ceffd640 1 -- 172.21.15.2:0/575468598 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 63) ==== 8346+0+0 (secure 0 0 0) 0x7ff2e0102040 con 0x55720db10d80 2024-05-14T09:54:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.009+0000 7ff2fcddc740 10 client.17271 fetch_fsmap finished waiting for FSMap version 63 2024-05-14T09:54:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.009+0000 7ff2fcddc740 10 client.17271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:54:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.009+0000 7ff2fcddc740 20 client.17271 populate_metadata read hostname 'smithi002' 2024-05-14T09:54:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.009+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/575468598 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55720db521b0 0x55720df75440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.009+0000 7ff2fcddc740 4 client.17271 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:54:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.009+0000 7ff2fcddc740 1 -- 172.21.15.2:0/575468598 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff2e4004400 con 0x55720db521b0 2024-05-14T09:54:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.009+0000 7ff2eeffd640 1 --2- 172.21.15.2:0/575468598 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55720db521b0 0x55720df75440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.010+0000 7ff2ceffd640 10 client.17271 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.010+0000 7ff2eeffd640 1 --2- 172.21.15.2:0/575468598 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55720db521b0 0x55720df75440 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8255044b2e0018f server_cookie=af867dd20f4595f0 in_seq=0 out_seq=0 2024-05-14T09:54:17.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:16 smithi002 ceph-mon[31557]: pgmap v1004: 129 pgs: 129 active+clean; 662 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 9.1 MiB/s wr, 654 op/s 2024-05-14T09:54:17.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:16 smithi029 ceph-mon[36249]: pgmap v1004: 129 pgs: 129 active+clean; 662 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 9.1 MiB/s wr, 654 op/s 2024-05-14T09:54:17.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 181460 27.79 MB/sec execute 223 sec latency 450.237 ms 2024-05-14T09:54:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.757+0000 7ff2ceffd640 1 -- 172.21.15.2:0/575468598 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff2e4004400 con 0x55720db521b0 2024-05-14T09:54:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.757+0000 7ff2ceffd640 10 client.17271 handle_command_reply: tid=0 2024-05-14T09:54:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.757+0000 7ff2fcddc740 1 client.17271 shutdown 2024-05-14T09:54:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.758+0000 7ff2fcddc740 1 -- 172.21.15.2:0/575468598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db10d80 msgr2=0x55720df01c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.758+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/575468598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db10d80 0x55720df01c80 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00d2850 tx=0x7ff2e0009300 comp rx=0 tx=0).stop 2024-05-14T09:54:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.758+0000 7ff2fcddc740 1 -- 172.21.15.2:0/575468598 shutdown_connections 2024-05-14T09:54:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.758+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/575468598 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55720db521b0 0x55720df75440 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.758+0000 7ff2ceffd640 0 client.17271 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.758+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/575468598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55720db314d0 0x55720df030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.758+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/575468598 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55720db292f0 0x55720df021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.758+0000 7ff2fcddc740 1 --2- 172.21.15.2:0/575468598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55720db10d80 0x55720df01c80 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.758+0000 7ff2fcddc740 1 -- 172.21.15.2:0/575468598 >> 172.21.15.2:0/575468598 conn(0x55720db46300 msgr2=0x55720deed970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.758+0000 7ff2fcddc740 1 -- 172.21.15.2:0/575468598 shutdown_connections 2024-05-14T09:54:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.758+0000 7ff2fcddc740 1 -- 172.21.15.2:0/575468598 wait complete. 2024-05-14T09:54:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.759+0000 7ff2fcddc740 20 client.17271 trim_cache size 0 max 16384 2024-05-14T09:54:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:54:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:54:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:54:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:54:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.759+0000 7ff2f5f2e640 1 -- 172.21.15.2:0/740460042 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff2d80b2920 msgr2=0x7ff2d80b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.759+0000 7ff2f5f2e640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff2d80b2920 0x7ff2d80b4e30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55720def4c30 tx=0x7ff2e8009500 comp rx=0 tx=0).stop 2024-05-14T09:54:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.760+0000 7ff2f5f2e640 1 -- 172.21.15.2:0/740460042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2f0104740 msgr2=0x7ff2f0103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.760+0000 7ff2f5f2e640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2f0104740 0x7ff2f0103b90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0006010 tx=0x7ff2e0004270 comp rx=0 tx=0).stop 2024-05-14T09:54:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.760+0000 7ff2f5f2e640 1 -- 172.21.15.2:0/740460042 shutdown_connections 2024-05-14T09:54:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.760+0000 7ff2f5f2e640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff2d80b2920 0x7ff2d80b4e30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.760+0000 7ff2f5f2e640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff2f0109a00 0x7ff2f0102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.760+0000 7ff2f5f2e640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2f01070a0 0x7ff2f0102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.760+0000 7ff2f5f2e640 1 --2- 172.21.15.2:0/740460042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2f0104740 0x7ff2f0103b90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.760+0000 7ff2f5f2e640 1 -- 172.21.15.2:0/740460042 >> 172.21.15.2:0/740460042 conn(0x7ff2f00fdd40 msgr2=0x7ff2f0105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.760+0000 7ff2f5f2e640 1 -- 172.21.15.2:0/740460042 shutdown_connections 2024-05-14T09:54:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:17.760+0000 7ff2f5f2e640 1 -- 172.21.15.2:0/740460042 wait complete. 2024-05-14T09:54:18.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 182076 27.78 MB/sec execute 224 sec latency 96.942 ms 2024-05-14T09:54:19.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:18 smithi002 ceph-mon[31557]: pgmap v1005: 129 pgs: 129 active+clean; 669 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.0 MiB/s wr, 661 op/s 2024-05-14T09:54:19.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:18 smithi029 ceph-mon[36249]: pgmap v1005: 129 pgs: 129 active+clean; 669 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.0 MiB/s wr, 661 op/s 2024-05-14T09:54:19.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:18 smithi026 ceph-mon[36056]: pgmap v1005: 129 pgs: 129 active+clean; 669 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.0 MiB/s wr, 661 op/s 2024-05-14T09:54:19.201 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:54:18 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e[66118]: ignoring --setuser ceph since I am not root 2024-05-14T09:54:19.201 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:54:18 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e[66118]: ignoring --setgroup ceph since I am not root 2024-05-14T09:54:19.201 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:54:18 smithi026 ceph-mds[66122]: -- 172.21.15.26:0/988986187 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555fd1f0ba40 con 0x555fd1f53180 2024-05-14T09:54:19.202 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:54:18 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e[66118]: starting mds.e at 2024-05-14T09:54:19.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 182735 27.84 MB/sec execute 225 sec latency 58.396 ms 2024-05-14T09:54:20.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:19 smithi002 ceph-mon[31557]: daemon mds.e restarted 2024-05-14T09:54:20.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:19 smithi029 ceph-mon[36249]: daemon mds.e restarted 2024-05-14T09:54:20.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:19 smithi026 ceph-mon[36056]: daemon mds.e restarted 2024-05-14T09:54:20.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 183371 27.89 MB/sec execute 226 sec latency 83.589 ms 2024-05-14T09:54:21.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:20 smithi002 ceph-mon[31557]: pgmap v1006: 129 pgs: 129 active+clean; 672 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 5.3 MiB/s wr, 664 op/s 2024-05-14T09:54:21.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:20 smithi002 ceph-mon[31557]: osdmap e92: 12 total, 12 up, 12 in 2024-05-14T09:54:21.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:20 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.26:6834/1924355535,v1:172.21.15.26:6835/1924355535] up:boot 2024-05-14T09:54:21.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:20 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T09:54:21.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:20 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T09:54:21.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:20 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:54:21.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:20 smithi029 ceph-mon[36249]: pgmap v1006: 129 pgs: 129 active+clean; 672 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 5.3 MiB/s wr, 664 op/s 2024-05-14T09:54:21.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:20 smithi029 ceph-mon[36249]: osdmap e92: 12 total, 12 up, 12 in 2024-05-14T09:54:21.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:20 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.26:6834/1924355535,v1:172.21.15.26:6835/1924355535] up:boot 2024-05-14T09:54:21.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:20 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T09:54:21.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:20 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T09:54:21.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:20 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:54:21.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:20 smithi026 ceph-mon[36056]: pgmap v1006: 129 pgs: 129 active+clean; 672 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 5.3 MiB/s wr, 664 op/s 2024-05-14T09:54:21.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:20 smithi026 ceph-mon[36056]: osdmap e92: 12 total, 12 up, 12 in 2024-05-14T09:54:21.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:20 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.26:6834/1924355535,v1:172.21.15.26:6835/1924355535] up:boot 2024-05-14T09:54:21.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:20 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T09:54:21.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:20 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T09:54:21.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:20 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:54:21.612 INFO:tasks.workunit.client.0.smithi002.stdout: 10 183851 27.88 MB/sec execute 227 sec latency 235.156 ms 2024-05-14T09:54:22.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 184299 27.85 MB/sec execute 228 sec latency 106.361 ms 2024-05-14T09:54:23.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:22 smithi002 ceph-mon[31557]: pgmap v1008: 129 pgs: 129 active+clean; 662 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.2 MiB/s wr, 574 op/s 2024-05-14T09:54:23.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:22 smithi029 ceph-mon[36249]: pgmap v1008: 129 pgs: 129 active+clean; 662 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.2 MiB/s wr, 574 op/s 2024-05-14T09:54:23.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:22 smithi026 ceph-mon[36056]: pgmap v1008: 129 pgs: 129 active+clean; 662 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.2 MiB/s wr, 574 op/s 2024-05-14T09:54:23.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 184814 27.86 MB/sec execute 229 sec latency 120.299 ms 2024-05-14T09:54:24.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 185354 27.85 MB/sec execute 230 sec latency 143.387 ms 2024-05-14T09:54:25.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:24 smithi002 ceph-mon[31557]: pgmap v1009: 129 pgs: 129 active+clean; 661 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.1 MiB/s wr, 522 op/s 2024-05-14T09:54:25.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:24 smithi029 ceph-mon[36249]: pgmap v1009: 129 pgs: 129 active+clean; 661 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.1 MiB/s wr, 522 op/s 2024-05-14T09:54:25.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:24 smithi026 ceph-mon[36056]: pgmap v1009: 129 pgs: 129 active+clean; 661 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.1 MiB/s wr, 522 op/s 2024-05-14T09:54:25.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 185969 27.87 MB/sec execute 231 sec latency 99.771 ms 2024-05-14T09:54:26.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 186546 27.90 MB/sec execute 232 sec latency 106.996 ms 2024-05-14T09:54:27.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:26 smithi002 ceph-mon[31557]: pgmap v1010: 129 pgs: 129 active+clean; 666 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.5 MiB/s wr, 574 op/s 2024-05-14T09:54:27.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:26 smithi029 ceph-mon[36249]: pgmap v1010: 129 pgs: 129 active+clean; 666 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.5 MiB/s wr, 574 op/s 2024-05-14T09:54:27.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:26 smithi026 ceph-mon[36056]: pgmap v1010: 129 pgs: 129 active+clean; 666 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.5 MiB/s wr, 574 op/s 2024-05-14T09:54:27.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 186964 27.88 MB/sec execute 233 sec latency 155.650 ms 2024-05-14T09:54:28.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 187515 27.88 MB/sec execute 234 sec latency 80.640 ms 2024-05-14T09:54:28.973 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:28 smithi002 ceph-mon[31557]: pgmap v1011: 129 pgs: 129 active+clean; 669 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.1 MiB/s wr, 503 op/s 2024-05-14T09:54:29.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:28 smithi029 ceph-mon[36249]: pgmap v1011: 129 pgs: 129 active+clean; 669 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.1 MiB/s wr, 503 op/s 2024-05-14T09:54:29.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:28 smithi026 ceph-mon[36056]: pgmap v1011: 129 pgs: 129 active+clean; 669 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.1 MiB/s wr, 503 op/s 2024-05-14T09:54:29.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 187969 27.85 MB/sec execute 235 sec latency 101.347 ms 2024-05-14T09:54:30.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 188674 27.90 MB/sec execute 236 sec latency 96.319 ms 2024-05-14T09:54:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.897+0000 7fe88b59d640 1 -- 172.21.15.2:0/1637554748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe884108b30 msgr2=0x7fe88410af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.897+0000 7fe88b59d640 1 --2- 172.21.15.2:0/1637554748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe884108b30 0x7fe88410af50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009be0 tx=0x7fe87c02c8d0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.897+0000 7fe88b59d640 1 -- 172.21.15.2:0/1637554748 shutdown_connections 2024-05-14T09:54:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.897+0000 7fe88b59d640 1 --2- 172.21.15.2:0/1637554748 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe884108b30 0x7fe88410af50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.897+0000 7fe88b59d640 1 --2- 172.21.15.2:0/1637554748 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe884103e30 0x7fe884104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.897+0000 7fe88b59d640 1 --2- 172.21.15.2:0/1637554748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe884102bf0 0x7fe8841038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.897+0000 7fe88b59d640 1 -- 172.21.15.2:0/1637554748 >> 172.21.15.2:0/1637554748 conn(0x7fe8840fddc0 msgr2=0x7fe884100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.897+0000 7fe88b59d640 1 -- 172.21.15.2:0/1637554748 shutdown_connections 2024-05-14T09:54:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.897+0000 7fe88b59d640 1 -- 172.21.15.2:0/1637554748 wait complete. 2024-05-14T09:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.898+0000 7fe88b59d640 1 Processor -- start 2024-05-14T09:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.898+0000 7fe88b59d640 1 -- start start 2024-05-14T09:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.899+0000 7fe88b59d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe884102bf0 0x7fe884111ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.899+0000 7fe88b59d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe884103e30 0x7fe884112400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.899+0000 7fe88b59d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe884108b30 0x7fe884113360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.899+0000 7fe88b59d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe88410c350 con 0x7fe884103e30 2024-05-14T09:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.899+0000 7fe88b59d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe88410c1d0 con 0x7fe884108b30 2024-05-14T09:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.899+0000 7fe88b59d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe88410c4d0 con 0x7fe884102bf0 2024-05-14T09:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.899+0000 7fe889339640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe884102bf0 0x7fe884111ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.899+0000 7fe888b38640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe884103e30 0x7fe884112400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.899+0000 7fe889339640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe884102bf0 0x7fe884111ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:33502/0 (socket says 172.21.15.2:33502) 2024-05-14T09:54:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.899+0000 7fe889339640 1 -- 172.21.15.2:0/946856702 learned_addr learned my addr 172.21.15.2:0/946856702 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.899+0000 7fe889b3a640 1 --2- 172.21.15.2:0/946856702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe884108b30 0x7fe884113360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.900+0000 7fe888b38640 1 -- 172.21.15.2:0/946856702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe884102bf0 msgr2=0x7fe884111ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.900+0000 7fe888b38640 1 --2- 172.21.15.2:0/946856702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe884102bf0 0x7fe884111ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.900+0000 7fe888b38640 1 -- 172.21.15.2:0/946856702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe884108b30 msgr2=0x7fe884113360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.900+0000 7fe888b38640 1 --2- 172.21.15.2:0/946856702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe884108b30 0x7fe884113360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.900+0000 7fe888b38640 1 -- 172.21.15.2:0/946856702 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8841139d0 con 0x7fe884103e30 2024-05-14T09:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.900+0000 7fe888b38640 1 --2- 172.21.15.2:0/946856702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe884103e30 0x7fe884112400 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe87400e790 tx=0x7fe87400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.901+0000 7fe87a7fc640 1 -- 172.21.15.2:0/946856702 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87400eea0 con 0x7fe884103e30 2024-05-14T09:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.901+0000 7fe88b59d640 1 -- 172.21.15.2:0/946856702 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe884113cc0 con 0x7fe884103e30 2024-05-14T09:54:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.901+0000 7fe88b59d640 1 -- 172.21.15.2:0/946856702 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe884114260 con 0x7fe884103e30 2024-05-14T09:54:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.901+0000 7fe87a7fc640 1 -- 172.21.15.2:0/946856702 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe87400ccc0 con 0x7fe884103e30 2024-05-14T09:54:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.901+0000 7fe87a7fc640 1 -- 172.21.15.2:0/946856702 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874018660 con 0x7fe884103e30 2024-05-14T09:54:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.904+0000 7fe892454740 1 Processor -- start 2024-05-14T09:54:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.905+0000 7fe892454740 1 -- start start 2024-05-14T09:54:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.905+0000 7fe892454740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651950601b0 0x565195401a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.905+0000 7fe892454740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950372f0 0x565195401fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.905+0000 7fe892454740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56519503f4d0 0x565195408ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.905+0000 7fe892454740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5651953665d0 con 0x56519503f4d0 2024-05-14T09:54:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.905+0000 7fe892454740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56519531f890 con 0x5651950372f0 2024-05-14T09:54:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.905+0000 7fe892454740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x565195362590 con 0x5651950601b0 2024-05-14T09:54:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.905+0000 7fe889b3a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651950601b0 0x565195401a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.905+0000 7fe889b3a640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651950601b0 0x565195401a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:33518/0 (socket says 172.21.15.2:33518) 2024-05-14T09:54:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.905+0000 7fe889b3a640 1 -- 172.21.15.2:0/3187901126 learned_addr learned my addr 172.21.15.2:0/3187901126 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.905+0000 7fe889339640 1 --2- 172.21.15.2:0/3187901126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950372f0 0x565195401fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.906+0000 7fe889339640 1 -- 172.21.15.2:0/3187901126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651950601b0 msgr2=0x565195401a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.906+0000 7fe889339640 1 --2- 172.21.15.2:0/3187901126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651950601b0 0x565195401a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.906+0000 7fe889339640 1 -- 172.21.15.2:0/3187901126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56519503f4d0 msgr2=0x565195408ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.906+0000 7fe87a7fc640 1 -- 172.21.15.2:0/946856702 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe874018800 con 0x7fe884103e30 2024-05-14T09:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.906+0000 7fe889339640 1 --2- 172.21.15.2:0/3187901126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56519503f4d0 0x565195408ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.906+0000 7fe889339640 1 -- 172.21.15.2:0/3187901126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565195078210 con 0x5651950372f0 2024-05-14T09:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.906+0000 7fe889339640 1 --2- 172.21.15.2:0/3187901126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950372f0 0x565195401fe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe880009c50 tx=0x7fe880011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=639edb6f1636e984 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.906+0000 7fe86f7fe640 1 -- 172.21.15.2:0/3187901126 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880004230 con 0x5651950372f0 2024-05-14T09:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.907+0000 7fe86f7fe640 1 -- 172.21.15.2:0/3187901126 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8800043d0 con 0x5651950372f0 2024-05-14T09:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.907+0000 7fe87a7fc640 1 --2- 172.21.15.2:0/946856702 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8700b29a0 0x7fe8700b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.907+0000 7fe87a7fc640 1 -- 172.21.15.2:0/946856702 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7fe8740e2700 con 0x7fe884103e30 2024-05-14T09:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.907+0000 7fe86f7fe640 1 -- 172.21.15.2:0/3187901126 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880019790 con 0x5651950372f0 2024-05-14T09:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.907+0000 7fe889b3a640 1 --2- 172.21.15.2:0/946856702 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8700b29a0 0x7fe8700b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.908+0000 7fe889b3a640 1 --2- 172.21.15.2:0/946856702 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8700b29a0 0x7fe8700b4eb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009420 tx=0x7fe87c005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.908+0000 7fe892454740 1 -- 172.21.15.2:0/3187901126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950372f0 msgr2=0x565195401fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.908+0000 7fe892454740 1 --2- 172.21.15.2:0/3187901126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950372f0 0x565195401fe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe880009c50 tx=0x7fe880011810 comp rx=0 tx=0).stop 2024-05-14T09:54:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.908+0000 7fe892454740 1 -- 172.21.15.2:0/3187901126 shutdown_connections 2024-05-14T09:54:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.908+0000 7fe892454740 1 --2- 172.21.15.2:0/3187901126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56519503f4d0 0x565195408ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.908+0000 7fe892454740 1 --2- 172.21.15.2:0/3187901126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950372f0 0x565195401fe0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.908+0000 7fe892454740 1 --2- 172.21.15.2:0/3187901126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651950601b0 0x565195401a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.908+0000 7fe892454740 1 -- 172.21.15.2:0/3187901126 >> 172.21.15.2:0/3187901126 conn(0x565195054300 msgr2=0x5651953fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.908+0000 7fe892454740 1 -- 172.21.15.2:0/3187901126 shutdown_connections 2024-05-14T09:54:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.908+0000 7fe892454740 1 -- 172.21.15.2:0/3187901126 wait complete. 2024-05-14T09:54:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.909+0000 7fe892454740 1 -- start start 2024-05-14T09:54:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.909+0000 7fe892454740 1 Processor -- start 2024-05-14T09:54:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe892454740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651950372f0 0x56519540b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe892454740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56519503f4d0 0x56519540bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe892454740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 0x565195412b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe892454740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5651954099d0 con 0x56519503f4d0 2024-05-14T09:54:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe892454740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565195323370 con 0x5651950601b0 2024-05-14T09:54:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe892454740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x565195409b50 con 0x5651950372f0 2024-05-14T09:54:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe892454740 10 client.0 _ll_register_callbacks cb 0x565195077a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:54:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe889b3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 0x565195412b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe889339640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651950372f0 0x56519540b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe889b3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 0x565195412b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:49080/0 (socket says 172.21.15.2:49080) 2024-05-14T09:54:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe889b3a640 1 -- 172.21.15.2:0/2416951353 learned_addr learned my addr 172.21.15.2:0/2416951353 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe888b38640 1 --2- 172.21.15.2:0/2416951353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56519503f4d0 0x56519540bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe889339640 1 -- 172.21.15.2:0/2416951353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 msgr2=0x565195412b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe889339640 1 --2- 172.21.15.2:0/2416951353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 0x565195412b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe889339640 1 -- 172.21.15.2:0/2416951353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56519503f4d0 msgr2=0x56519540bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe889339640 1 --2- 172.21.15.2:0/2416951353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56519503f4d0 0x56519540bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe889339640 1 -- 172.21.15.2:0/2416951353 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565195144620 con 0x5651950372f0 2024-05-14T09:54:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.910+0000 7fe889b3a640 1 --2- 172.21.15.2:0/2416951353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 0x565195412b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:54:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.911+0000 7fe889339640 1 --2- 172.21.15.2:0/2416951353 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651950372f0 0x56519540b6a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe880005890 tx=0x7fe880005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f732d63c7860f4a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.911+0000 7fe86cff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:54:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.911+0000 7fe86cff9640 1 -- 172.21.15.2:0/2416951353 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880028070 con 0x5651950372f0 2024-05-14T09:54:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.911+0000 7fe892454740 1 -- 172.21.15.2:0/2416951353 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8800043d0 con 0x5651950372f0 2024-05-14T09:54:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.911+0000 7fe86cff9640 1 -- 172.21.15.2:0/2416951353 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7fe8800035f0 con 0x5651950372f0 2024-05-14T09:54:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.911+0000 7fe86cff9640 1 -- 172.21.15.2:0/2416951353 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe880034050 con 0x5651950372f0 2024-05-14T09:54:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.911+0000 7fe892454740 10 client.26902 fetch_fsmap learned FSMap version 65 2024-05-14T09:54:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.911+0000 7fe892454740 1 -- 172.21.15.2:0/2416951353 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x565195415a70 con 0x5651950372f0 2024-05-14T09:54:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.911+0000 7fe86cff9640 1 -- 172.21.15.2:0/2416951353 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8800043d0 con 0x5651950372f0 2024-05-14T09:54:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.912+0000 7fe86cff9640 1 -- 172.21.15.2:0/2416951353 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 65) ==== 8346+0+0 (secure 0 0 0) 0x7fe880004bc0 con 0x5651950372f0 2024-05-14T09:54:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.912+0000 7fe892454740 10 client.26902 fetch_fsmap finished waiting for FSMap version 65 2024-05-14T09:54:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.912+0000 7fe892454740 10 client.26902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:54:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.912+0000 7fe892454740 20 client.26902 populate_metadata read hostname 'smithi002' 2024-05-14T09:54:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.912+0000 7fe892454740 1 --2- 172.21.15.2:0/2416951353 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56519501ed80 0x565195418fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.912+0000 7fe892454740 4 client.26902 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:54:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.912+0000 7fe892454740 1 -- 172.21.15.2:0/2416951353 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651953741e0 con 0x56519501ed80 2024-05-14T09:54:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.912+0000 7fe888b38640 1 --2- 172.21.15.2:0/2416951353 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56519501ed80 0x565195418fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.913+0000 7fe86cff9640 10 client.26902 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.913+0000 7fe888b38640 1 --2- 172.21.15.2:0/2416951353 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56519501ed80 0x565195418fa0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae9cc67432e179d4 server_cookie=9e273010c4ef05af in_seq=0 out_seq=0 2024-05-14T09:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.916+0000 7fe86cff9640 1 -- 172.21.15.2:0/2416951353 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5651953741e0 con 0x56519501ed80 2024-05-14T09:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.916+0000 7fe86cff9640 10 client.26902 handle_command_reply: tid=0 2024-05-14T09:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.916+0000 7fe892454740 1 client.26902 shutdown 2024-05-14T09:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.916+0000 7fe892454740 1 -- 172.21.15.2:0/2416951353 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651950372f0 msgr2=0x56519540b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.916+0000 7fe892454740 1 --2- 172.21.15.2:0/2416951353 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651950372f0 0x56519540b6a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe880005890 tx=0x7fe880005250 comp rx=0 tx=0).stop 2024-05-14T09:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.916+0000 7fe892454740 1 -- 172.21.15.2:0/2416951353 shutdown_connections 2024-05-14T09:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.916+0000 7fe892454740 1 --2- 172.21.15.2:0/2416951353 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x56519501ed80 0x565195418fa0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.916+0000 7fe892454740 1 --2- 172.21.15.2:0/2416951353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 0x565195412b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.916+0000 7fe892454740 1 --2- 172.21.15.2:0/2416951353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56519503f4d0 0x56519540bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.916+0000 7fe892454740 1 --2- 172.21.15.2:0/2416951353 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5651950372f0 0x56519540b6a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.916+0000 7fe86cff9640 0 client.26902 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.916+0000 7fe892454740 1 -- 172.21.15.2:0/2416951353 >> 172.21.15.2:0/2416951353 conn(0x565195054300 msgr2=0x565195400300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.916+0000 7fe892454740 1 -- 172.21.15.2:0/2416951353 shutdown_connections 2024-05-14T09:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.916+0000 7fe892454740 1 -- 172.21.15.2:0/2416951353 wait complete. 2024-05-14T09:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.917+0000 7fe892454740 20 client.26902 trim_cache size 0 max 16384 2024-05-14T09:54:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe892454740 1 Processor -- start 2024-05-14T09:54:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe892454740 1 -- start start 2024-05-14T09:54:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe892454740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651950372f0 0x5651954001f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe892454740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56519503f4d0 0x565195400730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe892454740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 0x565195412a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe892454740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe868012640 con 0x5651950372f0 2024-05-14T09:54:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe892454740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe86800ff10 con 0x5651950601b0 2024-05-14T09:54:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe892454740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe86800dd60 con 0x56519503f4d0 2024-05-14T09:54:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe889339640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651950372f0 0x5651954001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe888b38640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56519503f4d0 0x565195400730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe888b38640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56519503f4d0 0x565195400730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:33542/0 (socket says 172.21.15.2:33542) 2024-05-14T09:54:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe888b38640 1 -- 172.21.15.2:0/504254891 learned_addr learned my addr 172.21.15.2:0/504254891 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe889339640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651950372f0 0x5651954001f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48992/0 (socket says 172.21.15.2:48992) 2024-05-14T09:54:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe889b3a640 1 --2- 172.21.15.2:0/504254891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 0x565195412a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe888b38640 1 -- 172.21.15.2:0/504254891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 msgr2=0x565195412a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.939+0000 7fe888b38640 1 --2- 172.21.15.2:0/504254891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 0x565195412a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.940+0000 7fe888b38640 1 -- 172.21.15.2:0/504254891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651950372f0 msgr2=0x5651954001f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.940+0000 7fe888b38640 1 --2- 172.21.15.2:0/504254891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651950372f0 0x5651954001f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.940+0000 7fe888b38640 1 -- 172.21.15.2:0/504254891 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651953fdf30 con 0x56519503f4d0 2024-05-14T09:54:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.940+0000 7fe888b38640 1 --2- 172.21.15.2:0/504254891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56519503f4d0 0x565195400730 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8740a6bc0 tx=0x7fe8740b2040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1268ea8582566fa0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.940+0000 7fe86effd640 1 -- 172.21.15.2:0/504254891 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8740a9950 con 0x56519503f4d0 2024-05-14T09:54:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.940+0000 7fe86effd640 1 -- 172.21.15.2:0/504254891 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8740adc50 con 0x56519503f4d0 2024-05-14T09:54:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.940+0000 7fe86effd640 1 -- 172.21.15.2:0/504254891 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8740e2320 con 0x56519503f4d0 2024-05-14T09:54:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.941+0000 7fe892454740 1 -- 172.21.15.2:0/504254891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56519503f4d0 msgr2=0x565195400730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.941+0000 7fe892454740 1 --2- 172.21.15.2:0/504254891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56519503f4d0 0x565195400730 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8740a6bc0 tx=0x7fe8740b2040 comp rx=0 tx=0).stop 2024-05-14T09:54:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.941+0000 7fe892454740 1 -- 172.21.15.2:0/504254891 shutdown_connections 2024-05-14T09:54:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.941+0000 7fe892454740 1 --2- 172.21.15.2:0/504254891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 0x565195412a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.941+0000 7fe892454740 1 --2- 172.21.15.2:0/504254891 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56519503f4d0 0x565195400730 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.941+0000 7fe892454740 1 --2- 172.21.15.2:0/504254891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651950372f0 0x5651954001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.941+0000 7fe892454740 1 -- 172.21.15.2:0/504254891 >> 172.21.15.2:0/504254891 conn(0x565195054300 msgr2=0x565195405980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.941+0000 7fe892454740 1 -- 172.21.15.2:0/504254891 shutdown_connections 2024-05-14T09:54:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.941+0000 7fe892454740 1 -- 172.21.15.2:0/504254891 wait complete. 2024-05-14T09:54:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.942+0000 7fe892454740 1 -- start start 2024-05-14T09:54:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.942+0000 7fe892454740 1 Processor -- start 2024-05-14T09:54:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.942+0000 7fe892454740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651950372f0 0x565195412e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.942+0000 7fe892454740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56519503f4d0 0x56519540f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.942+0000 7fe892454740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 0x56519540ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.942+0000 7fe892454740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe86800e3d0 con 0x5651950372f0 2024-05-14T09:54:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.942+0000 7fe892454740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe86800f840 con 0x5651950601b0 2024-05-14T09:54:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.942+0000 7fe892454740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe86800eaa0 con 0x56519503f4d0 2024-05-14T09:54:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.942+0000 7fe892454740 10 client.0 _ll_register_callbacks cb 0x7fe880009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:54:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.942+0000 7fe889339640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651950372f0 0x565195412e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.942+0000 7fe888b38640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56519503f4d0 0x56519540f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.942+0000 7fe889339640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651950372f0 0x565195412e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49004/0 (socket says 172.21.15.2:49004) 2024-05-14T09:54:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.942+0000 7fe889339640 1 -- 172.21.15.2:0/1249896605 learned_addr learned my addr 172.21.15.2:0/1249896605 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.942+0000 7fe889b3a640 1 --2- 172.21.15.2:0/1249896605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 0x56519540ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.943+0000 7fe888b38640 1 -- 172.21.15.2:0/1249896605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 msgr2=0x56519540ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.943+0000 7fe888b38640 1 --2- 172.21.15.2:0/1249896605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 0x56519540ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.943+0000 7fe888b38640 1 -- 172.21.15.2:0/1249896605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651950372f0 msgr2=0x565195412e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.943+0000 7fe888b38640 1 --2- 172.21.15.2:0/1249896605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651950372f0 0x565195412e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.943+0000 7fe888b38640 1 -- 172.21.15.2:0/1249896605 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565195413500 con 0x56519503f4d0 2024-05-14T09:54:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.943+0000 7fe888b38640 1 --2- 172.21.15.2:0/1249896605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56519503f4d0 0x56519540f9e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe8740a59d0 tx=0x7fe87400c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4d9445ad4cac7f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.943+0000 7fe86cff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:54:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.943+0000 7fe86cff9640 1 -- 172.21.15.2:0/1249896605 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8740b2700 con 0x56519503f4d0 2024-05-14T09:54:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.943+0000 7fe892454740 1 -- 172.21.15.2:0/1249896605 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8740adc50 con 0x56519503f4d0 2024-05-14T09:54:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.943+0000 7fe86cff9640 1 -- 172.21.15.2:0/1249896605 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7fe8740e2320 con 0x56519503f4d0 2024-05-14T09:54:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.943+0000 7fe86cff9640 1 -- 172.21.15.2:0/1249896605 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe8740aabd0 con 0x56519503f4d0 2024-05-14T09:54:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.943+0000 7fe892454740 10 client.26908 fetch_fsmap learned FSMap version 65 2024-05-14T09:54:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.943+0000 7fe892454740 1 -- 172.21.15.2:0/1249896605 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x565195480220 con 0x56519503f4d0 2024-05-14T09:54:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.944+0000 7fe86cff9640 1 -- 172.21.15.2:0/1249896605 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8740adc50 con 0x56519503f4d0 2024-05-14T09:54:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.944+0000 7fe86cff9640 1 -- 172.21.15.2:0/1249896605 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 65) ==== 8346+0+0 (secure 0 0 0) 0x7fe874018af0 con 0x56519503f4d0 2024-05-14T09:54:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.944+0000 7fe892454740 10 client.26908 fetch_fsmap finished waiting for FSMap version 65 2024-05-14T09:54:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.944+0000 7fe892454740 10 client.26908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:54:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.944+0000 7fe892454740 20 client.26908 populate_metadata read hostname 'smithi002' 2024-05-14T09:54:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.944+0000 7fe892454740 1 --2- 172.21.15.2:0/1249896605 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x565194ff41c0 0x565195482c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.944+0000 7fe892454740 4 client.26908 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:54:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.944+0000 7fe892454740 1 -- 172.21.15.2:0/1249896605 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56519536e050 con 0x565194ff41c0 2024-05-14T09:54:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.944+0000 7fe889339640 1 --2- 172.21.15.2:0/1249896605 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x565194ff41c0 0x565195482c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.945+0000 7fe86cff9640 10 client.26908 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:30.945+0000 7fe889339640 1 --2- 172.21.15.2:0/1249896605 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x565194ff41c0 0x565195482c10 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a35a36c3e21c93c2 server_cookie=69ad5d0542b82716 in_seq=0 out_seq=0 2024-05-14T09:54:31.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:30 smithi002 ceph-mon[31557]: pgmap v1012: 129 pgs: 129 active+clean; 676 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.4 MiB/s wr, 482 op/s 2024-05-14T09:54:31.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:30 smithi029 ceph-mon[36249]: pgmap v1012: 129 pgs: 129 active+clean; 676 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.4 MiB/s wr, 482 op/s 2024-05-14T09:54:31.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:30 smithi026 ceph-mon[36056]: pgmap v1012: 129 pgs: 129 active+clean; 676 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.4 MiB/s wr, 482 op/s 2024-05-14T09:54:31.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 189014 27.84 MB/sec execute 237 sec latency 493.985 ms 2024-05-14T09:54:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.667+0000 7fe86cff9640 1 -- 172.21.15.2:0/1249896605 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56519536e050 con 0x565194ff41c0 2024-05-14T09:54:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.667+0000 7fe86cff9640 10 client.26908 handle_command_reply: tid=0 2024-05-14T09:54:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.667+0000 7fe892454740 1 client.26908 shutdown 2024-05-14T09:54:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.667+0000 7fe892454740 1 -- 172.21.15.2:0/1249896605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56519503f4d0 msgr2=0x56519540f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.667+0000 7fe892454740 1 --2- 172.21.15.2:0/1249896605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56519503f4d0 0x56519540f9e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe8740a59d0 tx=0x7fe87400c840 comp rx=0 tx=0).stop 2024-05-14T09:54:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.667+0000 7fe892454740 1 -- 172.21.15.2:0/1249896605 shutdown_connections 2024-05-14T09:54:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.667+0000 7fe892454740 1 --2- 172.21.15.2:0/1249896605 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x565194ff41c0 0x565195482c10 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.667+0000 7fe892454740 1 --2- 172.21.15.2:0/1249896605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651950601b0 0x56519540ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.667+0000 7fe892454740 1 --2- 172.21.15.2:0/1249896605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56519503f4d0 0x56519540f9e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.667+0000 7fe86cff9640 0 client.26908 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.667+0000 7fe892454740 1 --2- 172.21.15.2:0/1249896605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651950372f0 0x565195412e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.667+0000 7fe892454740 1 -- 172.21.15.2:0/1249896605 >> 172.21.15.2:0/1249896605 conn(0x565195054300 msgr2=0x5651954001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.667+0000 7fe892454740 1 -- 172.21.15.2:0/1249896605 shutdown_connections 2024-05-14T09:54:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.667+0000 7fe892454740 1 -- 172.21.15.2:0/1249896605 wait complete. 2024-05-14T09:54:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.667+0000 7fe892454740 20 client.26908 trim_cache size 0 max 16384 2024-05-14T09:54:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:54:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:54:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:54:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:54:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.668+0000 7fe88b59d640 1 -- 172.21.15.2:0/946856702 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8700b29a0 msgr2=0x7fe8700b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.668+0000 7fe88b59d640 1 --2- 172.21.15.2:0/946856702 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8700b29a0 0x7fe8700b4eb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009420 tx=0x7fe87c005bd0 comp rx=0 tx=0).stop 2024-05-14T09:54:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.668+0000 7fe88b59d640 1 -- 172.21.15.2:0/946856702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe884103e30 msgr2=0x7fe884112400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.668+0000 7fe88b59d640 1 --2- 172.21.15.2:0/946856702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe884103e30 0x7fe884112400 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe87400e790 tx=0x7fe87400ec50 comp rx=0 tx=0).stop 2024-05-14T09:54:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.668+0000 7fe88b59d640 1 -- 172.21.15.2:0/946856702 shutdown_connections 2024-05-14T09:54:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.668+0000 7fe88b59d640 1 --2- 172.21.15.2:0/946856702 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe8700b29a0 0x7fe8700b4eb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.668+0000 7fe88b59d640 1 --2- 172.21.15.2:0/946856702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe884108b30 0x7fe884113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.668+0000 7fe88b59d640 1 --2- 172.21.15.2:0/946856702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe884103e30 0x7fe884112400 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.668+0000 7fe88b59d640 1 --2- 172.21.15.2:0/946856702 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe884102bf0 0x7fe884111ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.668+0000 7fe88b59d640 1 -- 172.21.15.2:0/946856702 >> 172.21.15.2:0/946856702 conn(0x7fe8840fddc0 msgr2=0x7fe8841001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.668+0000 7fe88b59d640 1 -- 172.21.15.2:0/946856702 shutdown_connections 2024-05-14T09:54:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:31.668+0000 7fe88b59d640 1 -- 172.21.15.2:0/946856702 wait complete. 2024-05-14T09:54:32.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 189574 27.89 MB/sec execute 238 sec latency 91.412 ms 2024-05-14T09:54:33.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:32 smithi002 ceph-mon[31557]: pgmap v1013: 129 pgs: 129 active+clean; 678 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 440 op/s 2024-05-14T09:54:33.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:32 smithi029 ceph-mon[36249]: pgmap v1013: 129 pgs: 129 active+clean; 678 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 440 op/s 2024-05-14T09:54:33.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:32 smithi026 ceph-mon[36056]: pgmap v1013: 129 pgs: 129 active+clean; 678 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 440 op/s 2024-05-14T09:54:33.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 190047 27.89 MB/sec execute 239 sec latency 108.292 ms 2024-05-14T09:54:34.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 190537 27.89 MB/sec execute 240 sec latency 169.952 ms 2024-05-14T09:54:35.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:34 smithi002 ceph-mon[31557]: pgmap v1014: 129 pgs: 129 active+clean; 687 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.5 MiB/s wr, 526 op/s 2024-05-14T09:54:35.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:34 smithi029 ceph-mon[36249]: pgmap v1014: 129 pgs: 129 active+clean; 687 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.5 MiB/s wr, 526 op/s 2024-05-14T09:54:35.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:34 smithi026 ceph-mon[36056]: pgmap v1014: 129 pgs: 129 active+clean; 687 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.5 MiB/s wr, 526 op/s 2024-05-14T09:54:35.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 191032 27.84 MB/sec execute 241 sec latency 95.093 ms 2024-05-14T09:54:36.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 191919 27.91 MB/sec execute 242 sec latency 78.523 ms 2024-05-14T09:54:36.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:36 smithi002 ceph-mon[31557]: pgmap v1015: 129 pgs: 129 active+clean; 687 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.2 MiB/s wr, 519 op/s 2024-05-14T09:54:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:36 smithi029 ceph-mon[36249]: pgmap v1015: 129 pgs: 129 active+clean; 687 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.2 MiB/s wr, 519 op/s 2024-05-14T09:54:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:36 smithi026 ceph-mon[36056]: pgmap v1015: 129 pgs: 129 active+clean; 687 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.2 MiB/s wr, 519 op/s 2024-05-14T09:54:37.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 192705 27.99 MB/sec execute 243 sec latency 74.064 ms 2024-05-14T09:54:38.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 192787 27.89 MB/sec execute 244 sec latency 934.180 ms 2024-05-14T09:54:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:38 smithi002 ceph-mon[31557]: pgmap v1016: 129 pgs: 129 active+clean; 689 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.0 MiB/s wr, 505 op/s 2024-05-14T09:54:38.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:38 smithi029 ceph-mon[36249]: pgmap v1016: 129 pgs: 129 active+clean; 689 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.0 MiB/s wr, 505 op/s 2024-05-14T09:54:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:38 smithi026 ceph-mon[36056]: pgmap v1016: 129 pgs: 129 active+clean; 689 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.0 MiB/s wr, 505 op/s 2024-05-14T09:54:39.613 INFO:tasks.workunit.client.0.smithi002.stdout: 10 192814 27.79 MB/sec execute 245 sec latency 1911.532 ms 2024-05-14T09:54:40.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 192847 27.68 MB/sec execute 246 sec latency 2787.035 ms 2024-05-14T09:54:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:40 smithi002 ceph-mon[31557]: pgmap v1017: 129 pgs: 129 active+clean; 715 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 5.3 MiB/s wr, 556 op/s 2024-05-14T09:54:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:54:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:40 smithi029 ceph-mon[36249]: pgmap v1017: 129 pgs: 129 active+clean; 715 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 5.3 MiB/s wr, 556 op/s 2024-05-14T09:54:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:54:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:40 smithi026 ceph-mon[36056]: pgmap v1017: 129 pgs: 129 active+clean; 715 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 5.3 MiB/s wr, 556 op/s 2024-05-14T09:54:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:54:41.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 192979 27.61 MB/sec execute 247 sec latency 731.455 ms 2024-05-14T09:54:42.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 193160 27.53 MB/sec execute 248 sec latency 264.313 ms 2024-05-14T09:54:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:42 smithi002 ceph-mon[31557]: pgmap v1018: 129 pgs: 129 active+clean; 742 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.3 MiB/s wr, 555 op/s 2024-05-14T09:54:42.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:42 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:42.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:42 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:42 smithi029 ceph-mon[36249]: pgmap v1018: 129 pgs: 129 active+clean; 742 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.3 MiB/s wr, 555 op/s 2024-05-14T09:54:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:42 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:42 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:42 smithi026 ceph-mon[36056]: pgmap v1018: 129 pgs: 129 active+clean; 742 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.3 MiB/s wr, 555 op/s 2024-05-14T09:54:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:42 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:42 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:43.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 193301 27.46 MB/sec execute 249 sec latency 241.553 ms 2024-05-14T09:54:44.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 193439 27.40 MB/sec execute 250 sec latency 368.769 ms 2024-05-14T09:54:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.835+0000 7f1a2dee2640 1 -- 172.21.15.2:0/2382077318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a281101a0 msgr2=0x7f1a281105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.835+0000 7f1a2dee2640 1 --2- 172.21.15.2:0/2382077318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a281101a0 0x7f1a281105b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009ca0 tx=0x7f1a2002a8f0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.836+0000 7f1a2dee2640 1 -- 172.21.15.2:0/2382077318 shutdown_connections 2024-05-14T09:54:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.836+0000 7f1a2dee2640 1 --2- 172.21.15.2:0/2382077318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a281101a0 0x7f1a281105b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.836+0000 7f1a2dee2640 1 --2- 172.21.15.2:0/2382077318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a2810b480 0x7f1a2810b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.836+0000 7f1a2dee2640 1 --2- 172.21.15.2:0/2382077318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a2810a550 0x7f1a2810af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.836+0000 7f1a2dee2640 1 -- 172.21.15.2:0/2382077318 >> 172.21.15.2:0/2382077318 conn(0x7f1a28070640 msgr2=0x7f1a28072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.839+0000 7f1a2dee2640 1 -- 172.21.15.2:0/2382077318 shutdown_connections 2024-05-14T09:54:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.839+0000 7f1a2dee2640 1 -- 172.21.15.2:0/2382077318 wait complete. 2024-05-14T09:54:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.840+0000 7f1a2dee2640 1 Processor -- start 2024-05-14T09:54:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.842+0000 7f1a2dee2640 1 -- start start 2024-05-14T09:54:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.842+0000 7f1a2dee2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a2810a550 0x7f1a2811b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.843+0000 7f1a2dee2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a2810b480 0x7f1a2811bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.843+0000 7f1a2dee2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a281101a0 0x7f1a28119e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.843+0000 7f1a2dee2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1a281124b0 con 0x7f1a2810a550 2024-05-14T09:54:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.843+0000 7f1a2dee2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1a28112330 con 0x7f1a281101a0 2024-05-14T09:54:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.843+0000 7f1a27fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a281101a0 0x7f1a28119e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.843+0000 7f1a27fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a281101a0 0x7f1a28119e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:51944/0 (socket says 172.21.15.2:51944) 2024-05-14T09:54:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.843+0000 7f1a27fff640 1 -- 172.21.15.2:0/2301315148 learned_addr learned my addr 172.21.15.2:0/2301315148 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.843+0000 7f1a2dee2640 1 -- 172.21.15.2:0/2301315148 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1a28112630 con 0x7f1a2810b480 2024-05-14T09:54:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.843+0000 7f1a26ffd640 1 --2- 172.21.15.2:0/2301315148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a2810b480 0x7f1a2811bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.843+0000 7f1a277fe640 1 --2- 172.21.15.2:0/2301315148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a2810a550 0x7f1a2811b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.845+0000 7f1a27fff640 1 -- 172.21.15.2:0/2301315148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a2810b480 msgr2=0x7f1a2811bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.845+0000 7f1a27fff640 1 --2- 172.21.15.2:0/2301315148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a2810b480 0x7f1a2811bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.845+0000 7f1a27fff640 1 -- 172.21.15.2:0/2301315148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a2810a550 msgr2=0x7f1a2811b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.845+0000 7f1a27fff640 1 --2- 172.21.15.2:0/2301315148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a2810a550 0x7f1a2811b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.845+0000 7f1a27fff640 1 -- 172.21.15.2:0/2301315148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2811a540 con 0x7f1a281101a0 2024-05-14T09:54:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.846+0000 7f1a27fff640 1 --2- 172.21.15.2:0/2301315148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a281101a0 0x7f1a28119e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009bf0 tx=0x7f1a20030960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.846+0000 7f1a24ff9640 1 -- 172.21.15.2:0/2301315148 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20046480 con 0x7f1a281101a0 2024-05-14T09:54:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.846+0000 7f1a24ff9640 1 -- 172.21.15.2:0/2301315148 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a2002d070 con 0x7f1a281101a0 2024-05-14T09:54:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.846+0000 7f1a24ff9640 1 -- 172.21.15.2:0/2301315148 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2003b960 con 0x7f1a281101a0 2024-05-14T09:54:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.847+0000 7f1a2dee2640 1 -- 172.21.15.2:0/2301315148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a2811a7d0 con 0x7f1a281101a0 2024-05-14T09:54:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.847+0000 7f1a2dee2640 1 -- 172.21.15.2:0/2301315148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a2811ac10 con 0x7f1a281101a0 2024-05-14T09:54:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.851+0000 7f1a34d91740 1 Processor -- start 2024-05-14T09:54:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.854+0000 7f1a24ff9640 1 -- 172.21.15.2:0/2301315148 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1a2003bb00 con 0x7f1a281101a0 2024-05-14T09:54:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.855+0000 7f1a24ff9640 1 --2- 172.21.15.2:0/2301315148 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1a100b29a0 0x7f1a100b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.855+0000 7f1a24ff9640 1 -- 172.21.15.2:0/2301315148 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7f1a20103ea0 con 0x7f1a281101a0 2024-05-14T09:54:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.856+0000 7f1a26ffd640 1 --2- 172.21.15.2:0/2301315148 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1a100b29a0 0x7f1a100b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.856+0000 7f1a34d91740 1 -- start start 2024-05-14T09:54:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.856+0000 7f1a34d91740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330d071b0 0x5593310a8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.856+0000 7f1a34d91740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 0x5593310a8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.856+0000 7f1a34d91740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330ce64d0 0x5593310afee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.856+0000 7f1a34d91740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55933100d5d0 con 0x559330cde2f0 2024-05-14T09:54:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.856+0000 7f1a34d91740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559330fc6890 con 0x559330ce64d0 2024-05-14T09:54:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.856+0000 7f1a34d91740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x559331009590 con 0x559330d071b0 2024-05-14T09:54:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.856+0000 7f1a277fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330d071b0 0x5593310a8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.856+0000 7f1a277fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330d071b0 0x5593310a8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:53786/0 (socket says 172.21.15.2:53786) 2024-05-14T09:54:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.856+0000 7f1a277fe640 1 -- 172.21.15.2:0/1124270450 learned_addr learned my addr 172.21.15.2:0/1124270450 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.856+0000 7f1a27fff640 1 --2- 172.21.15.2:0/1124270450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330ce64d0 0x5593310afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.856+0000 7f1a26ffd640 1 --2- 172.21.15.2:0/2301315148 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1a100b29a0 0x7f1a100b4eb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c005fd0 tx=0x7f1a1c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.856+0000 7f1a26ffd640 1 --2- 172.21.15.2:0/1124270450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 0x5593310a8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.857+0000 7f1a277fe640 1 -- 172.21.15.2:0/1124270450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330ce64d0 msgr2=0x5593310afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.857+0000 7f1a277fe640 1 --2- 172.21.15.2:0/1124270450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330ce64d0 0x5593310afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.857+0000 7f1a277fe640 1 -- 172.21.15.2:0/1124270450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 msgr2=0x5593310a8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.857+0000 7f1a277fe640 1 --2- 172.21.15.2:0/1124270450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 0x5593310a8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.857+0000 7f1a277fe640 1 -- 172.21.15.2:0/1124270450 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559330d1f210 con 0x559330d071b0 2024-05-14T09:54:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.857+0000 7f1a26ffd640 1 --2- 172.21.15.2:0/1124270450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 0x5593310a8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:54:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.858+0000 7f1a277fe640 1 --2- 172.21.15.2:0/1124270450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330d071b0 0x5593310a8a60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800bc40 tx=0x7f1a1800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25c55c4f694a0786 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.859+0000 7f1a0dffb640 1 -- 172.21.15.2:0/1124270450 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18015070 con 0x559330d071b0 2024-05-14T09:54:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.859+0000 7f1a0dffb640 1 -- 172.21.15.2:0/1124270450 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a1801a910 con 0x559330d071b0 2024-05-14T09:54:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.860+0000 7f1a34d91740 1 -- 172.21.15.2:0/1124270450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330d071b0 msgr2=0x5593310a8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.860+0000 7f1a34d91740 1 --2- 172.21.15.2:0/1124270450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330d071b0 0x5593310a8a60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800bc40 tx=0x7f1a1800f730 comp rx=0 tx=0).stop 2024-05-14T09:54:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.861+0000 7f1a34d91740 1 -- 172.21.15.2:0/1124270450 shutdown_connections 2024-05-14T09:54:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.861+0000 7f1a34d91740 1 --2- 172.21.15.2:0/1124270450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330ce64d0 0x5593310afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.861+0000 7f1a34d91740 1 --2- 172.21.15.2:0/1124270450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 0x5593310a8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.861+0000 7f1a34d91740 1 --2- 172.21.15.2:0/1124270450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330d071b0 0x5593310a8a60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.861+0000 7f1a34d91740 1 -- 172.21.15.2:0/1124270450 >> 172.21.15.2:0/1124270450 conn(0x559330cfb300 msgr2=0x5593310a43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.862+0000 7f1a34d91740 1 -- 172.21.15.2:0/1124270450 shutdown_connections 2024-05-14T09:54:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.863+0000 7f1a34d91740 1 -- 172.21.15.2:0/1124270450 wait complete. 2024-05-14T09:54:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.863+0000 7f1a34d91740 1 -- start start 2024-05-14T09:54:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.863+0000 7f1a34d91740 1 Processor -- start 2024-05-14T09:54:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.864+0000 7f1a34d91740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 0x5593310b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.864+0000 7f1a34d91740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330ce64d0 0x5593310b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.864+0000 7f1a34d91740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330d071b0 0x5593310b9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.864+0000 7f1a34d91740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5593310b0980 con 0x559330cde2f0 2024-05-14T09:54:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.864+0000 7f1a34d91740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559330fca370 con 0x559330d071b0 2024-05-14T09:54:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.864+0000 7f1a34d91740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5593310b0b00 con 0x559330ce64d0 2024-05-14T09:54:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.864+0000 7f1a34d91740 10 client.0 _ll_register_callbacks cb 0x559330d1ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:54:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.865+0000 7f1a26ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330ce64d0 0x5593310b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.865+0000 7f1a26ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330ce64d0 0x5593310b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:53792/0 (socket says 172.21.15.2:53792) 2024-05-14T09:54:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.865+0000 7f1a26ffd640 1 -- 172.21.15.2:0/2352341455 learned_addr learned my addr 172.21.15.2:0/2352341455 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.865+0000 7f1a27fff640 1 --2- 172.21.15.2:0/2352341455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330d071b0 0x5593310b9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.865+0000 7f1a277fe640 1 --2- 172.21.15.2:0/2352341455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 0x5593310b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.865+0000 7f1a26ffd640 1 -- 172.21.15.2:0/2352341455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330d071b0 msgr2=0x5593310b9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.865+0000 7f1a26ffd640 1 --2- 172.21.15.2:0/2352341455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330d071b0 0x5593310b9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.865+0000 7f1a26ffd640 1 -- 172.21.15.2:0/2352341455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 msgr2=0x5593310b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.865+0000 7f1a26ffd640 1 --2- 172.21.15.2:0/2352341455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 0x5593310b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.865+0000 7f1a26ffd640 1 -- 172.21.15.2:0/2352341455 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559330deb620 con 0x559330ce64d0 2024-05-14T09:54:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.865+0000 7f1a26ffd640 1 --2- 172.21.15.2:0/2352341455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330ce64d0 0x5593310b2c00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0114c0 tx=0x7f1a1c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ba0e259f0562c11 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.866+0000 7f1a06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:54:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.866+0000 7f1a06ffd640 1 -- 172.21.15.2:0/2352341455 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c01f900 con 0x559330ce64d0 2024-05-14T09:54:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.866+0000 7f1a06ffd640 1 -- 172.21.15.2:0/2352341455 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7f1a1c01faa0 con 0x559330ce64d0 2024-05-14T09:54:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.866+0000 7f1a34d91740 1 -- 172.21.15.2:0/2352341455 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a1801a910 con 0x559330ce64d0 2024-05-14T09:54:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.866+0000 7f1a06ffd640 1 -- 172.21.15.2:0/2352341455 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f1a1c01fca0 con 0x559330ce64d0 2024-05-14T09:54:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.866+0000 7f1a34d91740 10 client.26920 fetch_fsmap learned FSMap version 65 2024-05-14T09:54:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.866+0000 7f1a34d91740 1 -- 172.21.15.2:0/2352341455 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5593310bcbb0 con 0x559330ce64d0 2024-05-14T09:54:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.867+0000 7f1a06ffd640 1 -- 172.21.15.2:0/2352341455 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a1801a910 con 0x559330ce64d0 2024-05-14T09:54:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.868+0000 7f1a06ffd640 1 -- 172.21.15.2:0/2352341455 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 65) ==== 8346+0+0 (secure 0 0 0) 0x7f1a1c01e3c0 con 0x559330ce64d0 2024-05-14T09:54:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.869+0000 7f1a34d91740 10 client.26920 fetch_fsmap finished waiting for FSMap version 65 2024-05-14T09:54:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.869+0000 7f1a34d91740 10 client.26920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:54:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.869+0000 7f1a34d91740 20 client.26920 populate_metadata read hostname 'smithi002' 2024-05-14T09:54:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.869+0000 7f1a34d91740 1 --2- 172.21.15.2:0/2352341455 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559330cc5d80 0x5593310c0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.869+0000 7f1a34d91740 4 client.26920 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:54:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.869+0000 7f1a34d91740 1 -- 172.21.15.2:0/2352341455 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55933101b1e0 con 0x559330cc5d80 2024-05-14T09:54:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.872+0000 7f1a277fe640 1 --2- 172.21.15.2:0/2352341455 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559330cc5d80 0x5593310c0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.886+0000 7f1a277fe640 1 --2- 172.21.15.2:0/2352341455 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559330cc5d80 0x5593310c0290 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24be711e92eff74e server_cookie=6eab0fad867a0a6a in_seq=0 out_seq=0 2024-05-14T09:54:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.886+0000 7f1a06ffd640 10 client.26920 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.889+0000 7f1a06ffd640 1 -- 172.21.15.2:0/2352341455 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55933101b1e0 con 0x559330cc5d80 2024-05-14T09:54:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.889+0000 7f1a06ffd640 10 client.26920 handle_command_reply: tid=0 2024-05-14T09:54:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.890+0000 7f1a34d91740 1 client.26920 shutdown 2024-05-14T09:54:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.892+0000 7f1a34d91740 1 -- 172.21.15.2:0/2352341455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330ce64d0 msgr2=0x5593310b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.892+0000 7f1a34d91740 1 --2- 172.21.15.2:0/2352341455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330ce64d0 0x5593310b2c00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0114c0 tx=0x7f1a1c0046c0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.894+0000 7f1a34d91740 1 -- 172.21.15.2:0/2352341455 shutdown_connections 2024-05-14T09:54:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.894+0000 7f1a34d91740 1 --2- 172.21.15.2:0/2352341455 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559330cc5d80 0x5593310c0290 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.894+0000 7f1a34d91740 1 --2- 172.21.15.2:0/2352341455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330d071b0 0x5593310b9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.894+0000 7f1a34d91740 1 --2- 172.21.15.2:0/2352341455 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330ce64d0 0x5593310b2c00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.894+0000 7f1a34d91740 1 --2- 172.21.15.2:0/2352341455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 0x5593310b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.894+0000 7f1a34d91740 1 -- 172.21.15.2:0/2352341455 >> 172.21.15.2:0/2352341455 conn(0x559330cfb300 msgr2=0x5593310a7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.894+0000 7f1a06ffd640 0 client.26920 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.894+0000 7f1a34d91740 1 -- 172.21.15.2:0/2352341455 shutdown_connections 2024-05-14T09:54:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.894+0000 7f1a34d91740 1 -- 172.21.15.2:0/2352341455 wait complete. 2024-05-14T09:54:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.895+0000 7f1a34d91740 20 client.26920 trim_cache size 0 max 16384 2024-05-14T09:54:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:44 smithi002 ceph-mon[31557]: pgmap v1019: 129 pgs: 129 active+clean; 779 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 12 MiB/s wr, 671 op/s 2024-05-14T09:54:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.928+0000 7f1a34d91740 1 Processor -- start 2024-05-14T09:54:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.928+0000 7f1a34d91740 1 -- start start 2024-05-14T09:54:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.929+0000 7f1a34d91740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 0x5593310a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.929+0000 7f1a34d91740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330ce64d0 0x5593310a7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.929+0000 7f1a34d91740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330d071b0 0x5593310b9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.929+0000 7f1a34d91740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1a281c6780 con 0x559330cde2f0 2024-05-14T09:54:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.929+0000 7f1a34d91740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1a281c4050 con 0x559330ce64d0 2024-05-14T09:54:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.929+0000 7f1a34d91740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1a281c1f50 con 0x559330d071b0 2024-05-14T09:54:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.929+0000 7f1a26ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330ce64d0 0x5593310a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.929+0000 7f1a26ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330ce64d0 0x5593310a7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:51986/0 (socket says 172.21.15.2:51986) 2024-05-14T09:54:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.929+0000 7f1a26ffd640 1 -- 172.21.15.2:0/1088339597 learned_addr learned my addr 172.21.15.2:0/1088339597 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.929+0000 7f1a277fe640 1 --2- 172.21.15.2:0/1088339597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 0x5593310a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.930+0000 7f1a277fe640 1 -- 172.21.15.2:0/1088339597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330d071b0 msgr2=0x5593310b9a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:54:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.930+0000 7f1a277fe640 1 --2- 172.21.15.2:0/1088339597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330d071b0 0x5593310b9a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.930+0000 7f1a277fe640 1 -- 172.21.15.2:0/1088339597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330ce64d0 msgr2=0x5593310a7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.930+0000 7f1a277fe640 1 --2- 172.21.15.2:0/1088339597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330ce64d0 0x5593310a7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.930+0000 7f1a277fe640 1 -- 172.21.15.2:0/1088339597 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593310a4d00 con 0x559330cde2f0 2024-05-14T09:54:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.930+0000 7f1a277fe640 1 --2- 172.21.15.2:0/1088339597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 0x5593310a71f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x5593310c0a10 tx=0x7f1a1800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e5d411a7f69258 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.931+0000 7f1a0d7fa640 1 -- 172.21.15.2:0/1088339597 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18015070 con 0x559330cde2f0 2024-05-14T09:54:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.931+0000 7f1a0d7fa640 1 -- 172.21.15.2:0/1088339597 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a18011c80 con 0x559330cde2f0 2024-05-14T09:54:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.931+0000 7f1a0d7fa640 1 -- 172.21.15.2:0/1088339597 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a180053c0 con 0x559330cde2f0 2024-05-14T09:54:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.933+0000 7f1a34d91740 1 -- 172.21.15.2:0/1088339597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 msgr2=0x5593310a71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.933+0000 7f1a34d91740 1 --2- 172.21.15.2:0/1088339597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 0x5593310a71f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x5593310c0a10 tx=0x7f1a1800b8f0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.937+0000 7f1a34d91740 1 -- 172.21.15.2:0/1088339597 shutdown_connections 2024-05-14T09:54:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.937+0000 7f1a34d91740 1 --2- 172.21.15.2:0/1088339597 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330d071b0 0x5593310b9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.937+0000 7f1a34d91740 1 --2- 172.21.15.2:0/1088339597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330ce64d0 0x5593310a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.937+0000 7f1a34d91740 1 --2- 172.21.15.2:0/1088339597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330cde2f0 0x5593310a71f0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.937+0000 7f1a34d91740 1 -- 172.21.15.2:0/1088339597 >> 172.21.15.2:0/1088339597 conn(0x559330cfb300 msgr2=0x5593310aca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:44 smithi029 ceph-mon[36249]: pgmap v1019: 129 pgs: 129 active+clean; 779 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 12 MiB/s wr, 671 op/s 2024-05-14T09:54:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.941+0000 7f1a34d91740 1 -- 172.21.15.2:0/1088339597 shutdown_connections 2024-05-14T09:54:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.943+0000 7f1a34d91740 1 -- 172.21.15.2:0/1088339597 wait complete. 2024-05-14T09:54:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.943+0000 7f1a34d91740 1 -- start start 2024-05-14T09:54:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.944+0000 7f1a34d91740 1 Processor -- start 2024-05-14T09:54:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.944+0000 7f1a34d91740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330cde2f0 0x5593310b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.945+0000 7f1a34d91740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330ce64d0 0x5593310b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.945+0000 7f1a277fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330cde2f0 0x5593310b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.945+0000 7f1a277fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330cde2f0 0x5593310b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:51990/0 (socket says 172.21.15.2:51990) 2024-05-14T09:54:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.945+0000 7f1a277fe640 1 -- 172.21.15.2:0/1105348040 learned_addr learned my addr 172.21.15.2:0/1105348040 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.945+0000 7f1a34d91740 1 --2- 172.21.15.2:0/1105348040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330d071b0 0x559331126610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.945+0000 7f1a34d91740 1 -- 172.21.15.2:0/1105348040 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1a281c2510 con 0x559330ce64d0 2024-05-14T09:54:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.945+0000 7f1a34d91740 1 -- 172.21.15.2:0/1105348040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1a281c3980 con 0x559330cde2f0 2024-05-14T09:54:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.945+0000 7f1a34d91740 1 -- 172.21.15.2:0/1105348040 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f1a281c2be0 con 0x559330d071b0 2024-05-14T09:54:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.946+0000 7f1a34d91740 10 client.0 _ll_register_callbacks cb 0x7f1a1c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:54:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.946+0000 7f1a26ffd640 1 --2- 172.21.15.2:0/1105348040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330ce64d0 0x5593310b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.946+0000 7f1a277fe640 1 -- 172.21.15.2:0/1105348040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330d071b0 msgr2=0x559331126610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.946+0000 7f1a277fe640 1 --2- 172.21.15.2:0/1105348040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330d071b0 0x559331126610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.946+0000 7f1a277fe640 1 -- 172.21.15.2:0/1105348040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330ce64d0 msgr2=0x5593310b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.946+0000 7f1a277fe640 1 --2- 172.21.15.2:0/1105348040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330ce64d0 0x5593310b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.946+0000 7f1a277fe640 1 -- 172.21.15.2:0/1105348040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593310b7d50 con 0x559330cde2f0 2024-05-14T09:54:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.951+0000 7f1a277fe640 1 --2- 172.21.15.2:0/1105348040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330cde2f0 0x5593310b71a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1a18004630 tx=0x7f1a1800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d316cd92afccccdc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:44 smithi026 ceph-mon[36056]: pgmap v1019: 129 pgs: 129 active+clean; 779 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 12 MiB/s wr, 671 op/s 2024-05-14T09:54:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.954+0000 7f1a06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:54:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.955+0000 7f1a06ffd640 1 -- 172.21.15.2:0/1105348040 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18015070 con 0x559330cde2f0 2024-05-14T09:54:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.955+0000 7f1a06ffd640 1 -- 172.21.15.2:0/1105348040 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7f1a18011a80 con 0x559330cde2f0 2024-05-14T09:54:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.956+0000 7f1a34d91740 1 -- 172.21.15.2:0/1105348040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a18011c80 con 0x559330cde2f0 2024-05-14T09:54:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.957+0000 7f1a06ffd640 1 -- 172.21.15.2:0/1105348040 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f1a180114e0 con 0x559330cde2f0 2024-05-14T09:54:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.957+0000 7f1a34d91740 10 client.26927 fetch_fsmap learned FSMap version 65 2024-05-14T09:54:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.957+0000 7f1a34d91740 1 -- 172.21.15.2:0/1105348040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x559331127bf0 con 0x559330cde2f0 2024-05-14T09:54:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.958+0000 7f1a06ffd640 1 -- 172.21.15.2:0/1105348040 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a18005600 con 0x559330cde2f0 2024-05-14T09:54:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.958+0000 7f1a06ffd640 1 -- 172.21.15.2:0/1105348040 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 65) ==== 8346+0+0 (secure 0 0 0) 0x7f1a180048a0 con 0x559330cde2f0 2024-05-14T09:54:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.958+0000 7f1a34d91740 10 client.26927 fetch_fsmap finished waiting for FSMap version 65 2024-05-14T09:54:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.959+0000 7f1a34d91740 10 client.26927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:54:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.959+0000 7f1a34d91740 20 client.26927 populate_metadata read hostname 'smithi002' 2024-05-14T09:54:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.959+0000 7f1a34d91740 1 --2- 172.21.15.2:0/1105348040 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559330c9b1c0 0x55933112a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.959+0000 7f1a34d91740 4 client.26927 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:54:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.959+0000 7f1a34d91740 1 -- 172.21.15.2:0/1105348040 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1a18011c80 con 0x559330c9b1c0 2024-05-14T09:54:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.967+0000 7f1a26ffd640 1 --2- 172.21.15.2:0/1105348040 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559330c9b1c0 0x55933112a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.968+0000 7f1a26ffd640 1 --2- 172.21.15.2:0/1105348040 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559330c9b1c0 0x55933112a2f0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c02b699343961319 server_cookie=87f9a5f20e1724f9 in_seq=0 out_seq=0 2024-05-14T09:54:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:44.968+0000 7f1a06ffd640 10 client.26927 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:45.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 193563 27.31 MB/sec execute 251 sec latency 270.353 ms 2024-05-14T09:54:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.835+0000 7f1a06ffd640 1 -- 172.21.15.2:0/1105348040 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1a18011c80 con 0x559330c9b1c0 2024-05-14T09:54:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.835+0000 7f1a06ffd640 10 client.26927 handle_command_reply: tid=0 2024-05-14T09:54:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.835+0000 7f1a34d91740 1 client.26927 shutdown 2024-05-14T09:54:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.836+0000 7f1a34d91740 1 -- 172.21.15.2:0/1105348040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330cde2f0 msgr2=0x5593310b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.836+0000 7f1a34d91740 1 --2- 172.21.15.2:0/1105348040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330cde2f0 0x5593310b71a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1a18004630 tx=0x7f1a1800f730 comp rx=0 tx=0).stop 2024-05-14T09:54:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.836+0000 7f1a34d91740 1 -- 172.21.15.2:0/1105348040 shutdown_connections 2024-05-14T09:54:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.836+0000 7f1a34d91740 1 --2- 172.21.15.2:0/1105348040 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x559330c9b1c0 0x55933112a2f0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.836+0000 7f1a06ffd640 0 client.26927 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.836+0000 7f1a34d91740 1 --2- 172.21.15.2:0/1105348040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559330d071b0 0x559331126610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.836+0000 7f1a34d91740 1 --2- 172.21.15.2:0/1105348040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559330ce64d0 0x5593310b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.836+0000 7f1a34d91740 1 --2- 172.21.15.2:0/1105348040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559330cde2f0 0x5593310b71a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.836+0000 7f1a34d91740 1 -- 172.21.15.2:0/1105348040 >> 172.21.15.2:0/1105348040 conn(0x559330cfb300 msgr2=0x5593310a71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.836+0000 7f1a34d91740 1 -- 172.21.15.2:0/1105348040 shutdown_connections 2024-05-14T09:54:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.836+0000 7f1a34d91740 1 -- 172.21.15.2:0/1105348040 wait complete. 2024-05-14T09:54:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.836+0000 7f1a34d91740 20 client.26927 trim_cache size 0 max 16384 2024-05-14T09:54:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:54:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:54:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:54:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:54:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.837+0000 7f1a2dee2640 1 -- 172.21.15.2:0/2301315148 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1a100b29a0 msgr2=0x7f1a100b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.837+0000 7f1a2dee2640 1 --2- 172.21.15.2:0/2301315148 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1a100b29a0 0x7f1a100b4eb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c005fd0 tx=0x7f1a1c005850 comp rx=0 tx=0).stop 2024-05-14T09:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.838+0000 7f1a2dee2640 1 -- 172.21.15.2:0/2301315148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a281101a0 msgr2=0x7f1a28119e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.838+0000 7f1a2dee2640 1 --2- 172.21.15.2:0/2301315148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a281101a0 0x7f1a28119e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009bf0 tx=0x7f1a20030960 comp rx=0 tx=0).stop 2024-05-14T09:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.838+0000 7f1a2dee2640 1 -- 172.21.15.2:0/2301315148 shutdown_connections 2024-05-14T09:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.838+0000 7f1a2dee2640 1 --2- 172.21.15.2:0/2301315148 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f1a100b29a0 0x7f1a100b4eb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.838+0000 7f1a2dee2640 1 --2- 172.21.15.2:0/2301315148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a281101a0 0x7f1a28119e10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.838+0000 7f1a2dee2640 1 --2- 172.21.15.2:0/2301315148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a2810b480 0x7f1a2811bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.838+0000 7f1a2dee2640 1 --2- 172.21.15.2:0/2301315148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a2810a550 0x7f1a2811b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.838+0000 7f1a2dee2640 1 -- 172.21.15.2:0/2301315148 >> 172.21.15.2:0/2301315148 conn(0x7f1a28070640 msgr2=0x7f1a28072960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.838+0000 7f1a2dee2640 1 -- 172.21.15.2:0/2301315148 shutdown_connections 2024-05-14T09:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:45.838+0000 7f1a2dee2640 1 -- 172.21.15.2:0/2301315148 wait complete. 2024-05-14T09:54:46.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 193753 27.26 MB/sec execute 252 sec latency 407.578 ms 2024-05-14T09:54:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:46 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:46.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:46 smithi002 ceph-mon[31557]: pgmap v1020: 129 pgs: 129 active+clean; 791 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 619 op/s 2024-05-14T09:54:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:46 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:46.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:46 smithi029 ceph-mon[36249]: pgmap v1020: 129 pgs: 129 active+clean; 791 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 619 op/s 2024-05-14T09:54:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:46 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:46.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:46 smithi026 ceph-mon[36056]: pgmap v1020: 129 pgs: 129 active+clean; 791 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 619 op/s 2024-05-14T09:54:47.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 193908 27.18 MB/sec execute 253 sec latency 411.762 ms 2024-05-14T09:54:48.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:48.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:48 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:48.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:48 smithi002 ceph-mon[31557]: pgmap v1021: 129 pgs: 129 active+clean; 771 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 589 op/s 2024-05-14T09:54:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:48 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:48.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:48 smithi029 ceph-mon[36249]: pgmap v1021: 129 pgs: 129 active+clean; 771 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 589 op/s 2024-05-14T09:54:48.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:48.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:48 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:54:48.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:48 smithi026 ceph-mon[36056]: pgmap v1021: 129 pgs: 129 active+clean; 771 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 589 op/s 2024-05-14T09:54:48.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 194068 27.10 MB/sec execute 254 sec latency 353.044 ms 2024-05-14T09:54:49.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 194534 27.07 MB/sec execute 255 sec latency 246.322 ms 2024-05-14T09:54:50.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 195137 27.05 MB/sec execute 256 sec latency 97.159 ms 2024-05-14T09:54:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:50 smithi002 ceph-mon[31557]: pgmap v1022: 129 pgs: 129 active+clean; 704 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 698 op/s 2024-05-14T09:54:50.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:50 smithi029 ceph-mon[36249]: pgmap v1022: 129 pgs: 129 active+clean; 704 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 698 op/s 2024-05-14T09:54:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:50 smithi026 ceph-mon[36056]: pgmap v1022: 129 pgs: 129 active+clean; 704 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 698 op/s 2024-05-14T09:54:51.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 195805 27.11 MB/sec execute 257 sec latency 334.799 ms 2024-05-14T09:54:52.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 196428 27.18 MB/sec execute 258 sec latency 177.465 ms 2024-05-14T09:54:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:52 smithi002 ceph-mon[31557]: pgmap v1023: 129 pgs: 129 active+clean; 682 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 12 MiB/s wr, 646 op/s 2024-05-14T09:54:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:52 smithi029 ceph-mon[36249]: pgmap v1023: 129 pgs: 129 active+clean; 682 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 12 MiB/s wr, 646 op/s 2024-05-14T09:54:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:52 smithi026 ceph-mon[36056]: pgmap v1023: 129 pgs: 129 active+clean; 682 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 12 MiB/s wr, 646 op/s 2024-05-14T09:54:53.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 196780 27.13 MB/sec execute 259 sec latency 411.967 ms 2024-05-14T09:54:54.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 197058 27.07 MB/sec execute 260 sec latency 255.122 ms 2024-05-14T09:54:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:54 smithi002 ceph-mon[31557]: pgmap v1024: 129 pgs: 129 active+clean; 682 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.6 MiB/s wr, 622 op/s 2024-05-14T09:54:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:54 smithi029 ceph-mon[36249]: pgmap v1024: 129 pgs: 129 active+clean; 682 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.6 MiB/s wr, 622 op/s 2024-05-14T09:54:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:54 smithi026 ceph-mon[36056]: pgmap v1024: 129 pgs: 129 active+clean; 682 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.6 MiB/s wr, 622 op/s 2024-05-14T09:54:55.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 197390 27.04 MB/sec execute 261 sec latency 615.064 ms 2024-05-14T09:54:56.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 197842 27.01 MB/sec execute 262 sec latency 214.426 ms 2024-05-14T09:54:56.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:56 smithi002 ceph-mon[31557]: pgmap v1025: 129 pgs: 129 active+clean; 686 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.4 MiB/s wr, 581 op/s 2024-05-14T09:54:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:56 smithi029 ceph-mon[36249]: pgmap v1025: 129 pgs: 129 active+clean; 686 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.4 MiB/s wr, 581 op/s 2024-05-14T09:54:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:56 smithi026 ceph-mon[36056]: pgmap v1025: 129 pgs: 129 active+clean; 686 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.4 MiB/s wr, 581 op/s 2024-05-14T09:54:57.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 198403 27.02 MB/sec execute 263 sec latency 367.624 ms 2024-05-14T09:54:58.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 199017 27.04 MB/sec execute 264 sec latency 195.775 ms 2024-05-14T09:54:58.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:54:58 smithi002 ceph-mon[31557]: pgmap v1026: 129 pgs: 129 active+clean; 688 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.8 MiB/s wr, 459 op/s 2024-05-14T09:54:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:54:58 smithi029 ceph-mon[36249]: pgmap v1026: 129 pgs: 129 active+clean; 688 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.8 MiB/s wr, 459 op/s 2024-05-14T09:54:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:54:58 smithi026 ceph-mon[36056]: pgmap v1026: 129 pgs: 129 active+clean; 688 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.8 MiB/s wr, 459 op/s 2024-05-14T09:54:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.963+0000 7fdfc0c49640 1 -- 172.21.15.2:0/560256388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfbc1045c0 msgr2=0x7fdfbc10ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.963+0000 7fdfc0c49640 1 --2- 172.21.15.2:0/560256388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfbc1045c0 0x7fdfbc10ad00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009ca0 tx=0x7fdfb402a8c0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.963+0000 7fdfc0c49640 1 -- 172.21.15.2:0/560256388 shutdown_connections 2024-05-14T09:54:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.963+0000 7fdfc0c49640 1 --2- 172.21.15.2:0/560256388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfbc1045c0 0x7fdfbc10ad00 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.963+0000 7fdfc0c49640 1 --2- 172.21.15.2:0/560256388 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdfbc103a80 0x7fdfbc103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.964+0000 7fdfc0c49640 1 --2- 172.21.15.2:0/560256388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc1030a0 0x7fdfbc1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.964+0000 7fdfc0c49640 1 -- 172.21.15.2:0/560256388 >> 172.21.15.2:0/560256388 conn(0x7fdfbc0fd4e0 msgr2=0x7fdfbc0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.964+0000 7fdfc0c49640 1 -- 172.21.15.2:0/560256388 shutdown_connections 2024-05-14T09:54:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.964+0000 7fdfc0c49640 1 -- 172.21.15.2:0/560256388 wait complete. 2024-05-14T09:54:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.964+0000 7fdfc0c49640 1 Processor -- start 2024-05-14T09:54:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.964+0000 7fdfc0c49640 1 -- start start 2024-05-14T09:54:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfc0c49640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfbc1030a0 0x7fdfbc102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfc0c49640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdfbc103a80 0x7fdfbc19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfc0c49640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc1045c0 0x7fdfbc1a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfc0c49640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc10e690 con 0x7fdfbc1030a0 2024-05-14T09:54:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfc0c49640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc10e510 con 0x7fdfbc1045c0 2024-05-14T09:54:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfc0c49640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc10e810 con 0x7fdfbc103a80 2024-05-14T09:54:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfba59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfbc1030a0 0x7fdfbc102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfba59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfbc1030a0 0x7fdfbc102360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54552/0 (socket says 172.21.15.2:54552) 2024-05-14T09:54:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfba59c640 1 -- 172.21.15.2:0/4038723045 learned_addr learned my addr 172.21.15.2:0/4038723045 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfb9d9b640 1 --2- 172.21.15.2:0/4038723045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdfbc103a80 0x7fdfbc19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/4038723045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc1045c0 0x7fdfbc1a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfba59c640 1 -- 172.21.15.2:0/4038723045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdfbc103a80 msgr2=0x7fdfbc19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfba59c640 1 --2- 172.21.15.2:0/4038723045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdfbc103a80 0x7fdfbc19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfba59c640 1 -- 172.21.15.2:0/4038723045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc1045c0 msgr2=0x7fdfbc1a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfba59c640 1 --2- 172.21.15.2:0/4038723045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc1045c0 0x7fdfbc1a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfba59c640 1 -- 172.21.15.2:0/4038723045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc1a1d90 con 0x7fdfbc1030a0 2024-05-14T09:54:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/4038723045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc1045c0 0x7fdfbc1a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:54:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfba59c640 1 --2- 172.21.15.2:0/4038723045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfbc1030a0 0x7fdfbc102360 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00d740 tx=0x7fdfac00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfab7fe640 1 -- 172.21.15.2:0/4038723045 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac017070 con 0x7fdfbc1030a0 2024-05-14T09:54:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.965+0000 7fdfc0c49640 1 -- 172.21.15.2:0/4038723045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfbc1a2080 con 0x7fdfbc1030a0 2024-05-14T09:54:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.966+0000 7fdfab7fe640 1 -- 172.21.15.2:0/4038723045 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfac00de50 con 0x7fdfbc1030a0 2024-05-14T09:54:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.966+0000 7fdfab7fe640 1 -- 172.21.15.2:0/4038723045 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00f620 con 0x7fdfbc1030a0 2024-05-14T09:54:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.966+0000 7fdfc0c49640 1 -- 172.21.15.2:0/4038723045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfbc1a2620 con 0x7fdfbc1030a0 2024-05-14T09:54:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.968+0000 7fdfc7bd3740 1 Processor -- start 2024-05-14T09:54:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.968+0000 7fdfc7bd3740 1 -- start start 2024-05-14T09:54:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.968+0000 7fdfab7fe640 1 -- 172.21.15.2:0/4038723045 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fdfac01a070 con 0x7fdfbc1030a0 2024-05-14T09:54:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.968+0000 7fdfc7bd3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692391b0 0x5591695dee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.968+0000 7fdfc7bd3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692102f0 0x5591695df3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.968+0000 7fdfc7bd3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692184d0 0x5591695e62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.968+0000 7fdfc7bd3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55916953f5d0 con 0x5591692391b0 2024-05-14T09:54:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.968+0000 7fdfc7bd3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5591694f8890 con 0x5591692102f0 2024-05-14T09:54:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.968+0000 7fdfc7bd3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55916953b590 con 0x5591692184d0 2024-05-14T09:54:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.969+0000 7fdfbad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692391b0 0x5591695dee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.969+0000 7fdfbad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692391b0 0x5591695dee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54556/0 (socket says 172.21.15.2:54556) 2024-05-14T09:54:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.969+0000 7fdfbad9d640 1 -- 172.21.15.2:0/890961975 learned_addr learned my addr 172.21.15.2:0/890961975 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.969+0000 7fdfba59c640 1 --2- 172.21.15.2:0/890961975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692102f0 0x5591695df3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.969+0000 7fdfb9d9b640 1 --2- 172.21.15.2:0/890961975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692184d0 0x5591695e62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.969+0000 7fdfbad9d640 1 -- 172.21.15.2:0/890961975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692184d0 msgr2=0x5591695e62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.969+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/890961975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692184d0 0x5591695e62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.969+0000 7fdfbad9d640 1 -- 172.21.15.2:0/890961975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692102f0 msgr2=0x5591695df3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.969+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/890961975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692102f0 0x5591695df3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.969+0000 7fdfbad9d640 1 -- 172.21.15.2:0/890961975 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559169251210 con 0x5591692391b0 2024-05-14T09:54:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.969+0000 7fdfba59c640 1 --2- 172.21.15.2:0/890961975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692102f0 0x5591695df3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:54:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.969+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/890961975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692391b0 0x5591695dee70 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4031a90 tx=0x7fdfb402f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6092ad1af8fc4476 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.969+0000 7fdfab7fe640 1 --2- 172.21.15.2:0/4038723045 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdfa40b28a0 0x7fdfa40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.969+0000 7fdfa8ff9640 1 -- 172.21.15.2:0/890961975 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4031650 con 0x5591692391b0 2024-05-14T09:54:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.970+0000 7fdfa8ff9640 1 -- 172.21.15.2:0/890961975 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfb4036070 con 0x5591692391b0 2024-05-14T09:54:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.970+0000 7fdfa8ff9640 1 -- 172.21.15.2:0/890961975 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4004610 con 0x5591692391b0 2024-05-14T09:54:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.970+0000 7fdfba59c640 1 --2- 172.21.15.2:0/4038723045 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdfa40b28a0 0x7fdfa40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.970+0000 7fdfab7fe640 1 -- 172.21.15.2:0/4038723045 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7fdfac0e2120 con 0x7fdfbc1030a0 2024-05-14T09:54:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/890961975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692391b0 msgr2=0x5591695dee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfba59c640 1 --2- 172.21.15.2:0/4038723045 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdfa40b28a0 0x7fdfa40b4db0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x5591695e0250 tx=0x7fdfac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/890961975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692391b0 0x5591695dee70 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4031a90 tx=0x7fdfb402f420 comp rx=0 tx=0).stop 2024-05-14T09:54:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/890961975 shutdown_connections 2024-05-14T09:54:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/890961975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692184d0 0x5591695e62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/890961975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692102f0 0x5591695df3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/890961975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692391b0 0x5591695dee70 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/890961975 >> 172.21.15.2:0/890961975 conn(0x55916922d300 msgr2=0x5591695d63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/890961975 shutdown_connections 2024-05-14T09:54:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/890961975 wait complete. 2024-05-14T09:54:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 -- start start 2024-05-14T09:54:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 Processor -- start 2024-05-14T09:54:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692102f0 0x5591695e8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692184d0 0x5591695e8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692391b0 0x5591695efef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5591695e6ce0 con 0x5591692102f0 2024-05-14T09:54:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5591694fc370 con 0x5591692184d0 2024-05-14T09:54:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5591695e6e60 con 0x5591692391b0 2024-05-14T09:54:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.971+0000 7fdfc7bd3740 10 client.0 _ll_register_callbacks cb 0x559169250a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:54:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfbad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692102f0 0x5591695e8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfbad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692102f0 0x5591695e8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54572/0 (socket says 172.21.15.2:54572) 2024-05-14T09:54:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfb9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692184d0 0x5591695e8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfbad9d640 1 -- 172.21.15.2:0/1347212961 learned_addr learned my addr 172.21.15.2:0/1347212961 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/1347212961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692391b0 0x5591695efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfbad9d640 1 -- 172.21.15.2:0/1347212961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692391b0 msgr2=0x5591695efef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/1347212961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692391b0 0x5591695efef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfbad9d640 1 -- 172.21.15.2:0/1347212961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692184d0 msgr2=0x5591695e8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/1347212961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692184d0 0x5591695e8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfbad9d640 1 -- 172.21.15.2:0/1347212961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55916931d620 con 0x5591692102f0 2024-05-14T09:54:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfb9d9b640 1 --2- 172.21.15.2:0/1347212961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692184d0 0x5591695e8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:54:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/1347212961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692102f0 0x5591695e8a50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdfb402a900 tx=0x7fdfb40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41e2202bc20c7f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:54:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfa1ffb640 1 -- 172.21.15.2:0/1347212961 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb40047a0 con 0x5591692102f0 2024-05-14T09:54:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfa1ffb640 1 -- 172.21.15.2:0/1347212961 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7fdfb4003970 con 0x5591692102f0 2024-05-14T09:54:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.972+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1347212961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfb4036070 con 0x5591692102f0 2024-05-14T09:54:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.973+0000 7fdfa1ffb640 1 -- 172.21.15.2:0/1347212961 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fdfb405d050 con 0x5591692102f0 2024-05-14T09:54:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.973+0000 7fdfc7bd3740 10 client.17322 fetch_fsmap learned FSMap version 65 2024-05-14T09:54:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.973+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1347212961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5591695e6ce0 con 0x5591692102f0 2024-05-14T09:54:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.973+0000 7fdfa1ffb640 1 -- 172.21.15.2:0/1347212961 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfb4036070 con 0x5591692102f0 2024-05-14T09:54:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.973+0000 7fdfa1ffb640 1 -- 172.21.15.2:0/1347212961 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 65) ==== 8346+0+0 (secure 0 0 0) 0x7fdfb402fbc0 con 0x5591692102f0 2024-05-14T09:54:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.973+0000 7fdfc7bd3740 10 client.17322 fetch_fsmap finished waiting for FSMap version 65 2024-05-14T09:54:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.973+0000 7fdfc7bd3740 10 client.17322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:54:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.973+0000 7fdfc7bd3740 20 client.17322 populate_metadata read hostname 'smithi002' 2024-05-14T09:54:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.973+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/1347212961 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5591691f7d80 0x5591695f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.973+0000 7fdfc7bd3740 4 client.17322 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:54:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.973+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1347212961 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55916954d1e0 con 0x5591691f7d80 2024-05-14T09:54:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.973+0000 7fdfb9d9b640 1 --2- 172.21.15.2:0/1347212961 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5591691f7d80 0x5591695f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.974+0000 7fdfa1ffb640 10 client.17322 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.974+0000 7fdfb9d9b640 1 --2- 172.21.15.2:0/1347212961 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5591691f7d80 0x5591695f60a0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b6354f87f338024 server_cookie=f9e34f630d586e13 in_seq=0 out_seq=0 2024-05-14T09:54:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.974+0000 7fdfa1ffb640 1 -- 172.21.15.2:0/1347212961 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55916954d1e0 con 0x5591691f7d80 2024-05-14T09:54:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.974+0000 7fdfa1ffb640 10 client.17322 handle_command_reply: tid=0 2024-05-14T09:54:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.974+0000 7fdfc7bd3740 1 client.17322 shutdown 2024-05-14T09:54:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.974+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1347212961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692102f0 msgr2=0x5591695e8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.974+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/1347212961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692102f0 0x5591695e8a50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdfb402a900 tx=0x7fdfb40095c0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.975+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1347212961 shutdown_connections 2024-05-14T09:54:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.975+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/1347212961 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5591691f7d80 0x5591695f60a0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.975+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/1347212961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692391b0 0x5591695efef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.975+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/1347212961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692184d0 0x5591695e8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.975+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/1347212961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692102f0 0x5591695e8a50 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.975+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1347212961 >> 172.21.15.2:0/1347212961 conn(0x55916922d300 msgr2=0x5591695d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.975+0000 7fdfa1ffb640 0 client.17322 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.975+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1347212961 shutdown_connections 2024-05-14T09:54:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.975+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1347212961 wait complete. 2024-05-14T09:54:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.975+0000 7fdfc7bd3740 20 client.17322 trim_cache size 0 max 16384 2024-05-14T09:54:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.990+0000 7fdfc7bd3740 1 Processor -- start 2024-05-14T09:54:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.990+0000 7fdfc7bd3740 1 -- start start 2024-05-14T09:54:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.990+0000 7fdfc7bd3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692102f0 0x5591695d6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.990+0000 7fdfc7bd3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692184d0 0x5591695efe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.990+0000 7fdfc7bd3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692391b0 0x5591695f03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.990+0000 7fdfc7bd3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1ad2d0 con 0x5591692391b0 2024-05-14T09:54:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.990+0000 7fdfc7bd3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1aaba0 con 0x5591692184d0 2024-05-14T09:54:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.990+0000 7fdfc7bd3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1a89f0 con 0x5591692102f0 2024-05-14T09:54:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.990+0000 7fdfbad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692391b0 0x5591695f03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.990+0000 7fdfbad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692391b0 0x5591695f03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54580/0 (socket says 172.21.15.2:54580) 2024-05-14T09:54:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.990+0000 7fdfbad9d640 1 -- 172.21.15.2:0/4092409149 learned_addr learned my addr 172.21.15.2:0/4092409149 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.991+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/4092409149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692102f0 0x5591695d6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.991+0000 7fdfb9d9b640 1 --2- 172.21.15.2:0/4092409149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692184d0 0x5591695efe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.991+0000 7fdfbad9d640 1 -- 172.21.15.2:0/4092409149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692102f0 msgr2=0x5591695d6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.991+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/4092409149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692102f0 0x5591695d6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.991+0000 7fdfbad9d640 1 -- 172.21.15.2:0/4092409149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692184d0 msgr2=0x5591695efe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.991+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/4092409149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692184d0 0x5591695efe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.991+0000 7fdfbad9d640 1 -- 172.21.15.2:0/4092409149 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591695d6940 con 0x5591692391b0 2024-05-14T09:54:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.991+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/4092409149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692391b0 0x5591695f03c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4030040 tx=0x7fdfb4004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2759a6bc4225d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.991+0000 7fdfa3fff640 1 -- 172.21.15.2:0/4092409149 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4031620 con 0x5591692391b0 2024-05-14T09:54:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.991+0000 7fdfa3fff640 1 -- 172.21.15.2:0/4092409149 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfb4036070 con 0x5591692391b0 2024-05-14T09:54:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.991+0000 7fdfa3fff640 1 -- 172.21.15.2:0/4092409149 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4003bd0 con 0x5591692391b0 2024-05-14T09:54:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.992+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/4092409149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692391b0 msgr2=0x5591695f03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.992+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/4092409149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692391b0 0x5591695f03c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4030040 tx=0x7fdfb4004320 comp rx=0 tx=0).stop 2024-05-14T09:54:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.992+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/4092409149 shutdown_connections 2024-05-14T09:54:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.992+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/4092409149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692391b0 0x5591695f03c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.992+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/4092409149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692184d0 0x5591695efe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.992+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/4092409149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692102f0 0x5591695d6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.992+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/4092409149 >> 172.21.15.2:0/4092409149 conn(0x55916922d300 msgr2=0x5591695d5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.992+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/4092409149 shutdown_connections 2024-05-14T09:54:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.992+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/4092409149 wait complete. 2024-05-14T09:54:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.992+0000 7fdfc7bd3740 1 -- start start 2024-05-14T09:54:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.992+0000 7fdfc7bd3740 1 Processor -- start 2024-05-14T09:54:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfc7bd3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692102f0 0x5591695f0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfc7bd3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692184d0 0x5591695ece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfc7bd3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692391b0 0x5591695ed480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfc7bd3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1a9060 con 0x5591692184d0 2024-05-14T09:54:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfc7bd3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1aa4d0 con 0x5591692391b0 2024-05-14T09:54:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfc7bd3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1a9730 con 0x5591692102f0 2024-05-14T09:54:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfc7bd3740 10 client.0 _ll_register_callbacks cb 0x7fdfb00095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:54:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfb9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692184d0 0x5591695ece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfb9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692184d0 0x5591695ece80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54588/0 (socket says 172.21.15.2:54588) 2024-05-14T09:54:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfb9d9b640 1 -- 172.21.15.2:0/1206081484 learned_addr learned my addr 172.21.15.2:0/1206081484 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:54:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/1206081484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692102f0 0x5591695f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/1206081484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692391b0 0x5591695ed480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfb9d9b640 1 -- 172.21.15.2:0/1206081484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692102f0 msgr2=0x5591695f0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfb9d9b640 1 --2- 172.21.15.2:0/1206081484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692102f0 0x5591695f0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfb9d9b640 1 -- 172.21.15.2:0/1206081484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692391b0 msgr2=0x5591695ed480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:54:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfb9d9b640 1 --2- 172.21.15.2:0/1206081484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692391b0 0x5591695ed480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfb9d9b640 1 -- 172.21.15.2:0/1206081484 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55916965c400 con 0x5591692184d0 2024-05-14T09:54:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/1206081484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692391b0 0x5591695ed480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:54:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfb9d9b640 1 --2- 172.21.15.2:0/1206081484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692184d0 0x5591695ece80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x5591695f6800 tx=0x7fdfb0012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d5aa7a5c004def server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:54:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:54:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.993+0000 7fdfa1ffb640 1 -- 172.21.15.2:0/1206081484 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb001fa70 con 0x5591692184d0 2024-05-14T09:54:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.994+0000 7fdfa1ffb640 1 -- 172.21.15.2:0/1206081484 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7fdfb001fc10 con 0x5591692184d0 2024-05-14T09:54:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.994+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1206081484 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfb4036070 con 0x5591692184d0 2024-05-14T09:54:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.994+0000 7fdfa1ffb640 1 -- 172.21.15.2:0/1206081484 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fdfb0033050 con 0x5591692184d0 2024-05-14T09:54:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.994+0000 7fdfc7bd3740 10 client.17328 fetch_fsmap learned FSMap version 65 2024-05-14T09:54:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.994+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1206081484 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55916965d740 con 0x5591692184d0 2024-05-14T09:54:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.994+0000 7fdfa1ffb640 1 -- 172.21.15.2:0/1206081484 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfb4036070 con 0x5591692184d0 2024-05-14T09:54:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.994+0000 7fdfa1ffb640 1 -- 172.21.15.2:0/1206081484 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 65) ==== 8346+0+0 (secure 0 0 0) 0x7fdfb00210b0 con 0x5591692184d0 2024-05-14T09:54:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.994+0000 7fdfc7bd3740 10 client.17328 fetch_fsmap finished waiting for FSMap version 65 2024-05-14T09:54:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.994+0000 7fdfc7bd3740 10 client.17328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:54:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.994+0000 7fdfc7bd3740 20 client.17328 populate_metadata read hostname 'smithi002' 2024-05-14T09:54:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.994+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/1206081484 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5591691cd1c0 0x559169660160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:54:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.994+0000 7fdfc7bd3740 4 client.17328 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:54:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.994+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1206081484 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559169547050 con 0x5591691cd1c0 2024-05-14T09:54:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.994+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/1206081484 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5591691cd1c0 0x559169660160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:54:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.995+0000 7fdfa1ffb640 10 client.17328 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:58.995+0000 7fdfbad9d640 1 --2- 172.21.15.2:0/1206081484 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5591691cd1c0 0x559169660160 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=566f01e34a0f5585 server_cookie=3b8d158f68d16a5d in_seq=0 out_seq=0 2024-05-14T09:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.604+0000 7fdfa1ffb640 1 -- 172.21.15.2:0/1206081484 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559169547050 con 0x5591691cd1c0 2024-05-14T09:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.604+0000 7fdfa1ffb640 10 client.17328 handle_command_reply: tid=0 2024-05-14T09:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.604+0000 7fdfc7bd3740 1 client.17328 shutdown 2024-05-14T09:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.604+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1206081484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692184d0 msgr2=0x5591695ece80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.604+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/1206081484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692184d0 0x5591695ece80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x5591695f6800 tx=0x7fdfb0012b00 comp rx=0 tx=0).stop 2024-05-14T09:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.605+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1206081484 shutdown_connections 2024-05-14T09:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.605+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/1206081484 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5591691cd1c0 0x559169660160 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.605+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/1206081484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5591692391b0 0x5591695ed480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.605+0000 7fdfa1ffb640 0 client.17328 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:54:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.605+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/1206081484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591692184d0 0x5591695ece80 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.605+0000 7fdfc7bd3740 1 --2- 172.21.15.2:0/1206081484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591692102f0 0x5591695f0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.605+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1206081484 >> 172.21.15.2:0/1206081484 conn(0x55916922d300 msgr2=0x5591695e8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.605+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1206081484 shutdown_connections 2024-05-14T09:54:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.605+0000 7fdfc7bd3740 1 -- 172.21.15.2:0/1206081484 wait complete. 2024-05-14T09:54:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.605+0000 7fdfc7bd3740 20 client.17328 trim_cache size 0 max 16384 2024-05-14T09:54:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:54:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:54:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:54:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:54:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.606+0000 7fdfc0c49640 1 -- 172.21.15.2:0/4038723045 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdfa40b28a0 msgr2=0x7fdfa40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.606+0000 7fdfc0c49640 1 --2- 172.21.15.2:0/4038723045 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdfa40b28a0 0x7fdfa40b4db0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x5591695e0250 tx=0x7fdfac000bc0 comp rx=0 tx=0).stop 2024-05-14T09:54:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.606+0000 7fdfc0c49640 1 -- 172.21.15.2:0/4038723045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfbc1030a0 msgr2=0x7fdfbc102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:54:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.606+0000 7fdfc0c49640 1 --2- 172.21.15.2:0/4038723045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfbc1030a0 0x7fdfbc102360 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00d740 tx=0x7fdfac00dc00 comp rx=0 tx=0).stop 2024-05-14T09:54:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.606+0000 7fdfc0c49640 1 -- 172.21.15.2:0/4038723045 shutdown_connections 2024-05-14T09:54:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.606+0000 7fdfc0c49640 1 --2- 172.21.15.2:0/4038723045 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fdfa40b28a0 0x7fdfa40b4db0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.606+0000 7fdfc0c49640 1 --2- 172.21.15.2:0/4038723045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc1045c0 0x7fdfbc1a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.606+0000 7fdfc0c49640 1 --2- 172.21.15.2:0/4038723045 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdfbc103a80 0x7fdfbc19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.606+0000 7fdfc0c49640 1 --2- 172.21.15.2:0/4038723045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfbc1030a0 0x7fdfbc102360 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:54:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.606+0000 7fdfc0c49640 1 -- 172.21.15.2:0/4038723045 >> 172.21.15.2:0/4038723045 conn(0x7fdfbc0fd4e0 msgr2=0x7fdfbc106c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:54:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.606+0000 7fdfc0c49640 1 -- 172.21.15.2:0/4038723045 shutdown_connections 2024-05-14T09:54:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:54:59.607+0000 7fdfc0c49640 1 -- 172.21.15.2:0/4038723045 wait complete. 2024-05-14T09:54:59.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 199363 27.03 MB/sec execute 265 sec latency 409.127 ms 2024-05-14T09:55:00.614 INFO:tasks.workunit.client.0.smithi002.stdout: 10 199905 27.04 MB/sec execute 266 sec latency 125.051 ms 2024-05-14T09:55:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:00 smithi002 ceph-mon[31557]: pgmap v1027: 129 pgs: 129 active+clean; 695 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.5 MiB/s wr, 482 op/s 2024-05-14T09:55:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:00 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:55:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:00 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:55:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:00 smithi029 ceph-mon[36249]: pgmap v1027: 129 pgs: 129 active+clean; 695 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.5 MiB/s wr, 482 op/s 2024-05-14T09:55:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:00 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:55:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:00 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:55:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:00 smithi026 ceph-mon[36056]: pgmap v1027: 129 pgs: 129 active+clean; 695 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.5 MiB/s wr, 482 op/s 2024-05-14T09:55:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:00 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:55:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:00 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:55:01.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 200370 27.05 MB/sec execute 267 sec latency 292.781 ms 2024-05-14T09:55:02.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 200828 27.02 MB/sec execute 268 sec latency 136.338 ms 2024-05-14T09:55:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:02 smithi002 ceph-mon[31557]: pgmap v1028: 129 pgs: 129 active+clean; 694 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.5 MiB/s wr, 381 op/s 2024-05-14T09:55:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:02 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:55:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:02 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:55:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:02 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:55:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:02 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:55:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:02 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:55:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:02 smithi029 ceph-mon[36249]: pgmap v1028: 129 pgs: 129 active+clean; 694 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.5 MiB/s wr, 381 op/s 2024-05-14T09:55:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:02 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:55:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:02 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:55:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:02 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:55:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:02 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:55:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:02 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:55:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:02 smithi026 ceph-mon[36056]: pgmap v1028: 129 pgs: 129 active+clean; 694 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.5 MiB/s wr, 381 op/s 2024-05-14T09:55:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:02 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:55:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:02 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:55:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:02 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:55:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:02 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:55:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:02 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:55:03.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 201416 27.01 MB/sec execute 269 sec latency 158.645 ms 2024-05-14T09:55:04.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 202243 27.07 MB/sec execute 270 sec latency 72.480 ms 2024-05-14T09:55:04.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:04 smithi002 ceph-mon[31557]: pgmap v1029: 129 pgs: 129 active+clean; 676 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.0 MiB/s wr, 465 op/s 2024-05-14T09:55:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:04 smithi029 ceph-mon[36249]: pgmap v1029: 129 pgs: 129 active+clean; 676 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.0 MiB/s wr, 465 op/s 2024-05-14T09:55:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:04 smithi026 ceph-mon[36056]: pgmap v1029: 129 pgs: 129 active+clean; 676 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.0 MiB/s wr, 465 op/s 2024-05-14T09:55:05.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 202953 27.15 MB/sec execute 271 sec latency 117.911 ms 2024-05-14T09:55:06.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 203497 27.17 MB/sec execute 272 sec latency 147.832 ms 2024-05-14T09:55:06.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:06 smithi002 ceph-mon[31557]: pgmap v1030: 129 pgs: 129 active+clean; 668 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.0 MiB/s wr, 487 op/s 2024-05-14T09:55:06.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:06 smithi029 ceph-mon[36249]: pgmap v1030: 129 pgs: 129 active+clean; 668 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.0 MiB/s wr, 487 op/s 2024-05-14T09:55:06.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:06 smithi026 ceph-mon[36056]: pgmap v1030: 129 pgs: 129 active+clean; 668 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.0 MiB/s wr, 487 op/s 2024-05-14T09:55:07.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 203913 27.14 MB/sec execute 273 sec latency 80.873 ms 2024-05-14T09:55:08.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 204361 27.13 MB/sec execute 274 sec latency 142.751 ms 2024-05-14T09:55:08.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:08 smithi002 ceph-mon[31557]: pgmap v1031: 129 pgs: 129 active+clean; 670 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 2.9 MiB/s wr, 474 op/s 2024-05-14T09:55:08.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:08 smithi029 ceph-mon[36249]: pgmap v1031: 129 pgs: 129 active+clean; 670 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 2.9 MiB/s wr, 474 op/s 2024-05-14T09:55:08.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:08 smithi026 ceph-mon[36056]: pgmap v1031: 129 pgs: 129 active+clean; 670 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 2.9 MiB/s wr, 474 op/s 2024-05-14T09:55:09.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 205024 27.15 MB/sec execute 275 sec latency 91.108 ms 2024-05-14T09:55:10.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 205792 27.17 MB/sec execute 276 sec latency 92.490 ms 2024-05-14T09:55:10.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:10 smithi002 ceph-mon[31557]: pgmap v1032: 129 pgs: 129 active+clean; 676 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.5 MiB/s wr, 532 op/s 2024-05-14T09:55:10.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:10 smithi029 ceph-mon[36249]: pgmap v1032: 129 pgs: 129 active+clean; 676 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.5 MiB/s wr, 532 op/s 2024-05-14T09:55:10.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:10 smithi026 ceph-mon[36056]: pgmap v1032: 129 pgs: 129 active+clean; 676 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.5 MiB/s wr, 532 op/s 2024-05-14T09:55:11.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 206504 27.24 MB/sec execute 277 sec latency 267.071 ms 2024-05-14T09:55:12.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 207059 27.28 MB/sec execute 278 sec latency 69.233 ms 2024-05-14T09:55:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.737+0000 7f9c4b2f9640 1 -- 172.21.15.2:0/2084211151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c44103d80 msgr2=0x7f9c44104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.737+0000 7f9c4b2f9640 1 --2- 172.21.15.2:0/2084211151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c44103d80 0x7f9c44104190 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009900 tx=0x7f9c3402a410 comp rx=0 tx=0).stop 2024-05-14T09:55:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.737+0000 7f9c4b2f9640 1 -- 172.21.15.2:0/2084211151 shutdown_connections 2024-05-14T09:55:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.737+0000 7f9c4b2f9640 1 --2- 172.21.15.2:0/2084211151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c44108a80 0x7f9c4410aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.737+0000 7f9c4b2f9640 1 --2- 172.21.15.2:0/2084211151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c44103d80 0x7f9c44104190 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.737+0000 7f9c4b2f9640 1 --2- 172.21.15.2:0/2084211151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102b40 0x7f9c44103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.737+0000 7f9c4b2f9640 1 -- 172.21.15.2:0/2084211151 >> 172.21.15.2:0/2084211151 conn(0x7f9c440fdd30 msgr2=0x7f9c441001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.737+0000 7f9c4b2f9640 1 -- 172.21.15.2:0/2084211151 shutdown_connections 2024-05-14T09:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.737+0000 7f9c4b2f9640 1 -- 172.21.15.2:0/2084211151 wait complete. 2024-05-14T09:55:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.738+0000 7f9c4b2f9640 1 Processor -- start 2024-05-14T09:55:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.738+0000 7f9c4b2f9640 1 -- start start 2024-05-14T09:55:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.738+0000 7f9c4b2f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102b40 0x7f9c4406bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.738+0000 7f9c4b2f9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c44103d80 0x7f9c4419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.738+0000 7f9c4b2f9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c44108a80 0x7f9c441a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.738+0000 7f9c4b2f9640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4410e690 con 0x7f9c44103d80 2024-05-14T09:55:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.738+0000 7f9c4b2f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4410e510 con 0x7f9c44102b40 2024-05-14T09:55:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.738+0000 7f9c4b2f9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4410e810 con 0x7f9c44108a80 2024-05-14T09:55:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.738+0000 7f9c48894640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c44103d80 0x7f9c4419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.738+0000 7f9c48894640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c44103d80 0x7f9c4419f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32816/0 (socket says 172.21.15.2:32816) 2024-05-14T09:55:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.739+0000 7f9c48894640 1 -- 172.21.15.2:0/4091609290 learned_addr learned my addr 172.21.15.2:0/4091609290 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.739+0000 7f9c49095640 1 --2- 172.21.15.2:0/4091609290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102b40 0x7f9c4406bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.739+0000 7f9c49896640 1 --2- 172.21.15.2:0/4091609290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c44108a80 0x7f9c441a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.739+0000 7f9c48894640 1 -- 172.21.15.2:0/4091609290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c44108a80 msgr2=0x7f9c441a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.739+0000 7f9c48894640 1 --2- 172.21.15.2:0/4091609290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c44108a80 0x7f9c441a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.739+0000 7f9c48894640 1 -- 172.21.15.2:0/4091609290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102b40 msgr2=0x7f9c4406bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.739+0000 7f9c48894640 1 --2- 172.21.15.2:0/4091609290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102b40 0x7f9c4406bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.739+0000 7f9c48894640 1 -- 172.21.15.2:0/4091609290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c441a1dc0 con 0x7f9c44103d80 2024-05-14T09:55:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.739+0000 7f9c49095640 1 --2- 172.21.15.2:0/4091609290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102b40 0x7f9c4406bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:55:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.739+0000 7f9c48894640 1 --2- 172.21.15.2:0/4091609290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c44103d80 0x7f9c4419f050 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9c34031440 tx=0x7f9c34003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.740+0000 7f9c3a7fc640 1 -- 172.21.15.2:0/4091609290 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3403f070 con 0x7f9c44103d80 2024-05-14T09:55:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.740+0000 7f9c3a7fc640 1 -- 172.21.15.2:0/4091609290 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c34031c50 con 0x7f9c44103d80 2024-05-14T09:55:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.740+0000 7f9c4b2f9640 1 -- 172.21.15.2:0/4091609290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c441a2050 con 0x7f9c44103d80 2024-05-14T09:55:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.740+0000 7f9c3a7fc640 1 -- 172.21.15.2:0/4091609290 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34035030 con 0x7f9c44103d80 2024-05-14T09:55:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.741+0000 7f9c4b2f9640 1 -- 172.21.15.2:0/4091609290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c441a2590 con 0x7f9c44103d80 2024-05-14T09:55:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.744+0000 7f9c521ab740 1 Processor -- start 2024-05-14T09:55:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.744+0000 7f9c521ab740 1 -- start start 2024-05-14T09:55:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.744+0000 7f9c521ab740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f6e1b0 0x55865230fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.744+0000 7f9c521ab740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x55865230ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.744+0000 7f9c521ab740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 0x558652316ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.744+0000 7f9c521ab740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5586522745d0 con 0x558651f452f0 2024-05-14T09:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.744+0000 7f9c521ab740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55865222d890 con 0x558651f6e1b0 2024-05-14T09:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.744+0000 7f9c521ab740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558652270590 con 0x558651f4d4d0 2024-05-14T09:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.744+0000 7f9c49095640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x55865230ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.744+0000 7f9c49095640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x55865230ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32826/0 (socket says 172.21.15.2:32826) 2024-05-14T09:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.744+0000 7f9c49095640 1 -- 172.21.15.2:0/3705988298 learned_addr learned my addr 172.21.15.2:0/3705988298 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.744+0000 7f9c49896640 1 --2- 172.21.15.2:0/3705988298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f6e1b0 0x55865230fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.745+0000 7f9c49095640 1 -- 172.21.15.2:0/3705988298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 msgr2=0x558652316ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.745+0000 7f9c49095640 1 --2- 172.21.15.2:0/3705988298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 0x558652316ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.745+0000 7f9c49095640 1 -- 172.21.15.2:0/3705988298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f6e1b0 msgr2=0x55865230fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.745+0000 7f9c49095640 1 --2- 172.21.15.2:0/3705988298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f6e1b0 0x55865230fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.745+0000 7f9c3a7fc640 1 -- 172.21.15.2:0/4091609290 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9c34042070 con 0x7f9c44103d80 2024-05-14T09:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.745+0000 7f9c49095640 1 -- 172.21.15.2:0/3705988298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558651f86210 con 0x558651f452f0 2024-05-14T09:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.745+0000 7f9c49095640 1 --2- 172.21.15.2:0/3705988298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x55865230ffe0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9c400114f0 tx=0x7f9c400119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6d63e1ddbc448f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.745+0000 7f9c2f7fe640 1 -- 172.21.15.2:0/3705988298 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4001e070 con 0x558651f452f0 2024-05-14T09:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.745+0000 7f9c2f7fe640 1 -- 172.21.15.2:0/3705988298 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c4000fad0 con 0x558651f452f0 2024-05-14T09:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.745+0000 7f9c2f7fe640 1 -- 172.21.15.2:0/3705988298 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40019b70 con 0x558651f452f0 2024-05-14T09:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.746+0000 7f9c3a7fc640 1 --2- 172.21.15.2:0/4091609290 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9c300b2920 0x7f9c300b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.746+0000 7f9c3a7fc640 1 -- 172.21.15.2:0/4091609290 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7f9c34102a40 con 0x7f9c44103d80 2024-05-14T09:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.746+0000 7f9c49896640 1 --2- 172.21.15.2:0/4091609290 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9c300b2920 0x7f9c300b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.747+0000 7f9c521ab740 1 -- 172.21.15.2:0/3705988298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 msgr2=0x55865230ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.747+0000 7f9c49896640 1 --2- 172.21.15.2:0/4091609290 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9c300b2920 0x7f9c300b4e30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x558652310c30 tx=0x7f9c3c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.747+0000 7f9c521ab740 1 --2- 172.21.15.2:0/3705988298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x55865230ffe0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9c400114f0 tx=0x7f9c400119b0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.747+0000 7f9c521ab740 1 -- 172.21.15.2:0/3705988298 shutdown_connections 2024-05-14T09:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.747+0000 7f9c521ab740 1 --2- 172.21.15.2:0/3705988298 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 0x558652316ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.747+0000 7f9c521ab740 1 --2- 172.21.15.2:0/3705988298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x55865230ffe0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.747+0000 7f9c521ab740 1 --2- 172.21.15.2:0/3705988298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f6e1b0 0x55865230fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.747+0000 7f9c521ab740 1 -- 172.21.15.2:0/3705988298 >> 172.21.15.2:0/3705988298 conn(0x558651f62300 msgr2=0x55865230b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.747+0000 7f9c521ab740 1 -- 172.21.15.2:0/3705988298 shutdown_connections 2024-05-14T09:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.747+0000 7f9c521ab740 1 -- 172.21.15.2:0/3705988298 wait complete. 2024-05-14T09:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.748+0000 7f9c521ab740 1 -- start start 2024-05-14T09:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.748+0000 7f9c521ab740 1 Processor -- start 2024-05-14T09:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.748+0000 7f9c521ab740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x55865231caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c521ab740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 0x55865231cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c521ab740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f6e1b0 0x558652319620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c49095640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x55865231caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c49095640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x55865231caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32830/0 (socket says 172.21.15.2:32830) 2024-05-14T09:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c521ab740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558652317930 con 0x558651f452f0 2024-05-14T09:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c521ab740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558652231370 con 0x558651f6e1b0 2024-05-14T09:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c521ab740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x558652317ab0 con 0x558651f4d4d0 2024-05-14T09:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c49095640 1 -- 172.21.15.2:0/1737282550 learned_addr learned my addr 172.21.15.2:0/1737282550 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c48894640 1 --2- 172.21.15.2:0/1737282550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 0x55865231cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c521ab740 10 client.0 _ll_register_callbacks cb 0x558651f85a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c49896640 1 --2- 172.21.15.2:0/1737282550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f6e1b0 0x558652319620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c48894640 1 -- 172.21.15.2:0/1737282550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f6e1b0 msgr2=0x558652319620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c48894640 1 --2- 172.21.15.2:0/1737282550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f6e1b0 0x558652319620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c48894640 1 -- 172.21.15.2:0/1737282550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 msgr2=0x55865231caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c48894640 1 --2- 172.21.15.2:0/1737282550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x55865231caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.749+0000 7f9c48894640 1 -- 172.21.15.2:0/1737282550 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558652052620 con 0x558651f4d4d0 2024-05-14T09:55:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.750+0000 7f9c48894640 1 --2- 172.21.15.2:0/1737282550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 0x55865231cfe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9c34100120 tx=0x7f9c34005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef726a3f4374fc9b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.750+0000 7f9c2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:55:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.750+0000 7f9c2cff9640 1 -- 172.21.15.2:0/1737282550 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34049040 con 0x558651f4d4d0 2024-05-14T09:55:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.750+0000 7f9c2cff9640 1 -- 172.21.15.2:0/1737282550 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7f9c340c9f50 con 0x558651f4d4d0 2024-05-14T09:55:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.750+0000 7f9c521ab740 1 -- 172.21.15.2:0/1737282550 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c4000fad0 con 0x558651f4d4d0 2024-05-14T09:55:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.750+0000 7f9c2cff9640 1 -- 172.21.15.2:0/1737282550 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f9c340c9800 con 0x558651f4d4d0 2024-05-14T09:55:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.750+0000 7f9c521ab740 10 client.26947 fetch_fsmap learned FSMap version 65 2024-05-14T09:55:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.750+0000 7f9c521ab740 1 -- 172.21.15.2:0/1737282550 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x558652317a50 con 0x558651f4d4d0 2024-05-14T09:55:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.751+0000 7f9c2cff9640 1 -- 172.21.15.2:0/1737282550 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c4000fad0 con 0x558651f4d4d0 2024-05-14T09:55:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.751+0000 7f9c2cff9640 1 -- 172.21.15.2:0/1737282550 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 65) ==== 8346+0+0 (secure 0 0 0) 0x7f9c340d2610 con 0x558651f4d4d0 2024-05-14T09:55:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.751+0000 7f9c521ab740 10 client.26947 fetch_fsmap finished waiting for FSMap version 65 2024-05-14T09:55:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.751+0000 7f9c521ab740 10 client.26947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:55:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.751+0000 7f9c521ab740 20 client.26947 populate_metadata read hostname 'smithi002' 2024-05-14T09:55:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.751+0000 7f9c521ab740 1 --2- 172.21.15.2:0/1737282550 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558651f2cd80 0x55865232fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.751+0000 7f9c521ab740 4 client.26947 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:55:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.751+0000 7f9c521ab740 1 -- 172.21.15.2:0/1737282550 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586522821e0 con 0x558651f2cd80 2024-05-14T09:55:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.751+0000 7f9c49095640 1 --2- 172.21.15.2:0/1737282550 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558651f2cd80 0x55865232fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.756+0000 7f9c49095640 1 --2- 172.21.15.2:0/1737282550 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558651f2cd80 0x55865232fa50 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4a26629713c1eaf server_cookie=8c5f21c441c96a2f in_seq=0 out_seq=0 2024-05-14T09:55:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.756+0000 7f9c2cff9640 10 client.26947 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.757+0000 7f9c2cff9640 1 -- 172.21.15.2:0/1737282550 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5586522821e0 con 0x558651f2cd80 2024-05-14T09:55:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.757+0000 7f9c2cff9640 10 client.26947 handle_command_reply: tid=0 2024-05-14T09:55:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.757+0000 7f9c521ab740 1 client.26947 shutdown 2024-05-14T09:55:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.757+0000 7f9c521ab740 1 -- 172.21.15.2:0/1737282550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 msgr2=0x55865231cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.757+0000 7f9c521ab740 1 --2- 172.21.15.2:0/1737282550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 0x55865231cfe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9c34100120 tx=0x7f9c34005e50 comp rx=0 tx=0).stop 2024-05-14T09:55:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.757+0000 7f9c521ab740 1 -- 172.21.15.2:0/1737282550 shutdown_connections 2024-05-14T09:55:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.757+0000 7f9c521ab740 1 --2- 172.21.15.2:0/1737282550 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558651f2cd80 0x55865232fa50 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.757+0000 7f9c521ab740 1 --2- 172.21.15.2:0/1737282550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f6e1b0 0x558652319620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.757+0000 7f9c521ab740 1 --2- 172.21.15.2:0/1737282550 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 0x55865231cfe0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.757+0000 7f9c521ab740 1 --2- 172.21.15.2:0/1737282550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x55865231caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.757+0000 7f9c521ab740 1 -- 172.21.15.2:0/1737282550 >> 172.21.15.2:0/1737282550 conn(0x558651f62300 msgr2=0x55865230e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.757+0000 7f9c2cff9640 0 client.26947 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.758+0000 7f9c521ab740 1 -- 172.21.15.2:0/1737282550 shutdown_connections 2024-05-14T09:55:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.758+0000 7f9c521ab740 1 -- 172.21.15.2:0/1737282550 wait complete. 2024-05-14T09:55:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.758+0000 7f9c521ab740 20 client.26947 trim_cache size 0 max 16384 2024-05-14T09:55:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.775+0000 7f9c521ab740 1 Processor -- start 2024-05-14T09:55:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.775+0000 7f9c521ab740 1 -- start start 2024-05-14T09:55:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c521ab740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x558652319040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c521ab740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 0x55865231c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c521ab740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f6e1b0 0x55865231cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c521ab740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441ad350 con 0x558651f452f0 2024-05-14T09:55:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c521ab740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441aac20 con 0x558651f6e1b0 2024-05-14T09:55:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c521ab740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441a8a70 con 0x558651f4d4d0 2024-05-14T09:55:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c49095640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x558652319040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c49095640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x558652319040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32840/0 (socket says 172.21.15.2:32840) 2024-05-14T09:55:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c48894640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 0x55865231c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c49095640 1 -- 172.21.15.2:0/247399916 learned_addr learned my addr 172.21.15.2:0/247399916 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c49896640 1 --2- 172.21.15.2:0/247399916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f6e1b0 0x55865231cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c48894640 1 -- 172.21.15.2:0/247399916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f6e1b0 msgr2=0x55865231cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c48894640 1 --2- 172.21.15.2:0/247399916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f6e1b0 0x55865231cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c48894640 1 -- 172.21.15.2:0/247399916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 msgr2=0x558652319040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c48894640 1 --2- 172.21.15.2:0/247399916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x558652319040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c48894640 1 -- 172.21.15.2:0/247399916 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55865230ab60 con 0x558651f4d4d0 2024-05-14T09:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c49095640 1 --2- 172.21.15.2:0/247399916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x558652319040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.776+0000 7f9c48894640 1 --2- 172.21.15.2:0/247399916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 0x55865231c9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9c34100490 tx=0x7f9c3402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7deda1c6007adea6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.777+0000 7f9c2effd640 1 -- 172.21.15.2:0/247399916 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34049070 con 0x558651f4d4d0 2024-05-14T09:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.777+0000 7f9c2effd640 1 -- 172.21.15.2:0/247399916 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c340e2060 con 0x558651f4d4d0 2024-05-14T09:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.777+0000 7f9c2effd640 1 -- 172.21.15.2:0/247399916 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c340d2aa0 con 0x558651f4d4d0 2024-05-14T09:55:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.777+0000 7f9c521ab740 1 -- 172.21.15.2:0/247399916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 msgr2=0x55865231c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.778+0000 7f9c521ab740 1 --2- 172.21.15.2:0/247399916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 0x55865231c9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9c34100490 tx=0x7f9c3402a410 comp rx=0 tx=0).stop 2024-05-14T09:55:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.778+0000 7f9c521ab740 1 -- 172.21.15.2:0/247399916 shutdown_connections 2024-05-14T09:55:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.778+0000 7f9c521ab740 1 --2- 172.21.15.2:0/247399916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f6e1b0 0x55865231cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.778+0000 7f9c521ab740 1 --2- 172.21.15.2:0/247399916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f4d4d0 0x55865231c9c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.778+0000 7f9c521ab740 1 --2- 172.21.15.2:0/247399916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f452f0 0x558652319040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.778+0000 7f9c521ab740 1 -- 172.21.15.2:0/247399916 >> 172.21.15.2:0/247399916 conn(0x558651f62300 msgr2=0x558652318930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.778+0000 7f9c521ab740 1 -- 172.21.15.2:0/247399916 shutdown_connections 2024-05-14T09:55:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.778+0000 7f9c521ab740 1 -- 172.21.15.2:0/247399916 wait complete. 2024-05-14T09:55:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.778+0000 7f9c521ab740 1 -- start start 2024-05-14T09:55:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.778+0000 7f9c521ab740 1 Processor -- start 2024-05-14T09:55:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c521ab740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f452f0 0x55865233e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c521ab740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f4d4d0 0x55865233e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c521ab740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f6e1b0 0x558652395a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c521ab740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441a90e0 con 0x558651f4d4d0 2024-05-14T09:55:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c521ab740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441aa550 con 0x558651f452f0 2024-05-14T09:55:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c521ab740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441a97b0 con 0x558651f6e1b0 2024-05-14T09:55:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c521ab740 10 client.0 _ll_register_callbacks cb 0x7f9c34005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:55:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c49896640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f6e1b0 0x558652395a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c49896640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f6e1b0 0x558652395a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:49720/0 (socket says 172.21.15.2:49720) 2024-05-14T09:55:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c49095640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f452f0 0x55865233e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c49896640 1 -- 172.21.15.2:0/1517352308 learned_addr learned my addr 172.21.15.2:0/1517352308 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c49896640 1 -- 172.21.15.2:0/1517352308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f452f0 msgr2=0x55865233e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c49896640 1 --2- 172.21.15.2:0/1517352308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f452f0 0x55865233e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c49896640 1 -- 172.21.15.2:0/1517352308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f4d4d0 msgr2=0x55865233e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c49896640 1 --2- 172.21.15.2:0/1517352308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f4d4d0 0x55865233e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.779+0000 7f9c49896640 1 -- 172.21.15.2:0/1517352308 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558652395fa0 con 0x558651f6e1b0 2024-05-14T09:55:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.780+0000 7f9c49896640 1 --2- 172.21.15.2:0/1517352308 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f6e1b0 0x558652395a60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c004620 tx=0x7f9c3c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd73782119dd51d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.780+0000 7f9c2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:55:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.780+0000 7f9c2cff9640 1 -- 172.21.15.2:0/1517352308 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c02f060 con 0x558651f6e1b0 2024-05-14T09:55:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.780+0000 7f9c2cff9640 1 -- 172.21.15.2:0/1517352308 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7f9c3c004650 con 0x558651f6e1b0 2024-05-14T09:55:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.780+0000 7f9c521ab740 1 -- 172.21.15.2:0/1517352308 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c340e2060 con 0x558651f6e1b0 2024-05-14T09:55:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.781+0000 7f9c2cff9640 1 -- 172.21.15.2:0/1517352308 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f9c3c036050 con 0x558651f6e1b0 2024-05-14T09:55:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.781+0000 7f9c521ab740 10 client.26953 fetch_fsmap learned FSMap version 65 2024-05-14T09:55:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.781+0000 7f9c521ab740 1 -- 172.21.15.2:0/1517352308 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x558652397230 con 0x558651f6e1b0 2024-05-14T09:55:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.781+0000 7f9c2cff9640 1 -- 172.21.15.2:0/1517352308 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c3c004850 con 0x558651f6e1b0 2024-05-14T09:55:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.781+0000 7f9c2cff9640 1 -- 172.21.15.2:0/1517352308 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 65) ==== 8346+0+0 (secure 0 0 0) 0x7f9c3c02a610 con 0x558651f6e1b0 2024-05-14T09:55:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.781+0000 7f9c521ab740 10 client.26953 fetch_fsmap finished waiting for FSMap version 65 2024-05-14T09:55:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.781+0000 7f9c521ab740 10 client.26953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:55:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.781+0000 7f9c521ab740 20 client.26953 populate_metadata read hostname 'smithi002' 2024-05-14T09:55:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.781+0000 7f9c521ab740 1 --2- 172.21.15.2:0/1517352308 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558651f021c0 0x558652399c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.781+0000 7f9c521ab740 4 client.26953 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:55:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.781+0000 7f9c521ab740 1 -- 172.21.15.2:0/1517352308 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9c340e2060 con 0x558651f021c0 2024-05-14T09:55:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.782+0000 7f9c49095640 1 --2- 172.21.15.2:0/1517352308 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558651f021c0 0x558652399c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.782+0000 7f9c49095640 1 --2- 172.21.15.2:0/1517352308 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558651f021c0 0x558652399c30 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9f979374b26ea70 server_cookie=496d5368ab49e79f in_seq=0 out_seq=0 2024-05-14T09:55:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:12.782+0000 7f9c2cff9640 10 client.26953 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:12.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:12 smithi002 ceph-mon[31557]: pgmap v1033: 129 pgs: 129 active+clean; 679 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.0 MiB/s wr, 480 op/s 2024-05-14T09:55:12.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:12 smithi029 ceph-mon[36249]: pgmap v1033: 129 pgs: 129 active+clean; 679 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.0 MiB/s wr, 480 op/s 2024-05-14T09:55:12.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:12 smithi026 ceph-mon[36056]: pgmap v1033: 129 pgs: 129 active+clean; 679 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.0 MiB/s wr, 480 op/s 2024-05-14T09:55:13.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 207298 27.23 MB/sec execute 279 sec latency 550.917 ms 2024-05-14T09:55:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.579+0000 7f9c2cff9640 1 -- 172.21.15.2:0/1517352308 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f9c340e2060 con 0x558651f021c0 2024-05-14T09:55:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.579+0000 7f9c2cff9640 10 client.26953 handle_command_reply: tid=0 2024-05-14T09:55:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.579+0000 7f9c521ab740 1 client.26953 shutdown 2024-05-14T09:55:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.579+0000 7f9c521ab740 1 -- 172.21.15.2:0/1517352308 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f6e1b0 msgr2=0x558652395a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.579+0000 7f9c521ab740 1 --2- 172.21.15.2:0/1517352308 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f6e1b0 0x558652395a60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c004620 tx=0x7f9c3c004a40 comp rx=0 tx=0).stop 2024-05-14T09:55:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.579+0000 7f9c521ab740 1 -- 172.21.15.2:0/1517352308 shutdown_connections 2024-05-14T09:55:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.579+0000 7f9c521ab740 1 --2- 172.21.15.2:0/1517352308 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558651f021c0 0x558652399c30 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.580+0000 7f9c2cff9640 0 client.26953 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.580+0000 7f9c521ab740 1 --2- 172.21.15.2:0/1517352308 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558651f6e1b0 0x558652395a60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.580+0000 7f9c521ab740 1 --2- 172.21.15.2:0/1517352308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558651f4d4d0 0x55865233e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.580+0000 7f9c521ab740 1 --2- 172.21.15.2:0/1517352308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558651f452f0 0x55865233e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.580+0000 7f9c521ab740 1 -- 172.21.15.2:0/1517352308 >> 172.21.15.2:0/1517352308 conn(0x558651f62300 msgr2=0x55865231c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.580+0000 7f9c521ab740 1 -- 172.21.15.2:0/1517352308 shutdown_connections 2024-05-14T09:55:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.580+0000 7f9c521ab740 1 -- 172.21.15.2:0/1517352308 wait complete. 2024-05-14T09:55:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.580+0000 7f9c521ab740 20 client.26953 trim_cache size 0 max 16384 2024-05-14T09:55:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:55:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:55:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:55:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:55:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.581+0000 7f9c4b2f9640 1 -- 172.21.15.2:0/4091609290 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9c300b2920 msgr2=0x7f9c300b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.581+0000 7f9c4b2f9640 1 --2- 172.21.15.2:0/4091609290 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9c300b2920 0x7f9c300b4e30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x558652310c30 tx=0x7f9c3c009500 comp rx=0 tx=0).stop 2024-05-14T09:55:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.581+0000 7f9c4b2f9640 1 -- 172.21.15.2:0/4091609290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c44103d80 msgr2=0x7f9c4419f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.581+0000 7f9c4b2f9640 1 --2- 172.21.15.2:0/4091609290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c44103d80 0x7f9c4419f050 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9c34031440 tx=0x7f9c34003c10 comp rx=0 tx=0).stop 2024-05-14T09:55:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.583+0000 7f9c4b2f9640 1 -- 172.21.15.2:0/4091609290 shutdown_connections 2024-05-14T09:55:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.583+0000 7f9c4b2f9640 1 --2- 172.21.15.2:0/4091609290 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f9c300b2920 0x7f9c300b4e30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.583+0000 7f9c4b2f9640 1 --2- 172.21.15.2:0/4091609290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9c44108a80 0x7f9c441a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.583+0000 7f9c4b2f9640 1 --2- 172.21.15.2:0/4091609290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c44103d80 0x7f9c4419f050 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.583+0000 7f9c4b2f9640 1 --2- 172.21.15.2:0/4091609290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c44102b40 0x7f9c4406bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.583+0000 7f9c4b2f9640 1 -- 172.21.15.2:0/4091609290 >> 172.21.15.2:0/4091609290 conn(0x7f9c440fdd30 msgr2=0x7f9c44104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.583+0000 7f9c4b2f9640 1 -- 172.21.15.2:0/4091609290 shutdown_connections 2024-05-14T09:55:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:14.583+0000 7f9c4b2f9640 1 -- 172.21.15.2:0/4091609290 wait complete. 2024-05-14T09:55:14.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 207318 27.13 MB/sec execute 280 sec latency 1105.596 ms 2024-05-14T09:55:14.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:14 smithi002 ceph-mon[31557]: pgmap v1034: 129 pgs: 129 active+clean; 686 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 528 op/s 2024-05-14T09:55:14.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:14 smithi029 ceph-mon[36249]: pgmap v1034: 129 pgs: 129 active+clean; 686 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 528 op/s 2024-05-14T09:55:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:14 smithi026 ceph-mon[36056]: pgmap v1034: 129 pgs: 129 active+clean; 686 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 528 op/s 2024-05-14T09:55:15.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 207345 27.04 MB/sec execute 281 sec latency 2054.852 ms 2024-05-14T09:55:16.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 207416 26.95 MB/sec execute 282 sec latency 2254.575 ms 2024-05-14T09:55:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:16 smithi002 ceph-mon[31557]: pgmap v1035: 129 pgs: 129 active+clean; 706 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 5.0 MiB/s wr, 541 op/s 2024-05-14T09:55:16.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:16 smithi029 ceph-mon[36249]: pgmap v1035: 129 pgs: 129 active+clean; 706 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 5.0 MiB/s wr, 541 op/s 2024-05-14T09:55:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:16 smithi026 ceph-mon[36056]: pgmap v1035: 129 pgs: 129 active+clean; 706 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 5.0 MiB/s wr, 541 op/s 2024-05-14T09:55:17.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 207608 26.89 MB/sec execute 283 sec latency 152.356 ms 2024-05-14T09:55:18.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 207788 26.82 MB/sec execute 284 sec latency 126.115 ms 2024-05-14T09:55:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:18 smithi002 ceph-mon[31557]: pgmap v1036: 129 pgs: 129 active+clean; 743 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.6 MiB/s wr, 604 op/s 2024-05-14T09:55:18.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:18 smithi029 ceph-mon[36249]: pgmap v1036: 129 pgs: 129 active+clean; 743 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.6 MiB/s wr, 604 op/s 2024-05-14T09:55:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:18 smithi026 ceph-mon[36056]: pgmap v1036: 129 pgs: 129 active+clean; 743 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.6 MiB/s wr, 604 op/s 2024-05-14T09:55:19.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 208023 26.77 MB/sec execute 285 sec latency 149.286 ms 2024-05-14T09:55:20.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 208412 26.75 MB/sec execute 286 sec latency 120.202 ms 2024-05-14T09:55:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:20 smithi002 ceph-mon[31557]: pgmap v1037: 129 pgs: 129 active+clean; 756 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 607 op/s 2024-05-14T09:55:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:20 smithi029 ceph-mon[36249]: pgmap v1037: 129 pgs: 129 active+clean; 756 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 607 op/s 2024-05-14T09:55:20.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:20 smithi026 ceph-mon[36056]: pgmap v1037: 129 pgs: 129 active+clean; 756 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 607 op/s 2024-05-14T09:55:21.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 208632 26.67 MB/sec execute 287 sec latency 321.281 ms 2024-05-14T09:55:22.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 209314 26.71 MB/sec execute 288 sec latency 267.999 ms 2024-05-14T09:55:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:22 smithi002 ceph-mon[31557]: pgmap v1038: 129 pgs: 129 active+clean; 749 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 706 op/s 2024-05-14T09:55:22.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:22 smithi029 ceph-mon[36249]: pgmap v1038: 129 pgs: 129 active+clean; 749 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 706 op/s 2024-05-14T09:55:22.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:22 smithi026 ceph-mon[36056]: pgmap v1038: 129 pgs: 129 active+clean; 749 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 706 op/s 2024-05-14T09:55:23.615 INFO:tasks.workunit.client.0.smithi002.stdout: 10 209913 26.75 MB/sec execute 289 sec latency 106.054 ms 2024-05-14T09:55:24.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 210611 26.81 MB/sec execute 290 sec latency 80.895 ms 2024-05-14T09:55:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:24 smithi002 ceph-mon[31557]: pgmap v1039: 129 pgs: 129 active+clean; 697 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 13 MiB/s wr, 807 op/s 2024-05-14T09:55:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:24 smithi029 ceph-mon[36249]: pgmap v1039: 129 pgs: 129 active+clean; 697 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 13 MiB/s wr, 807 op/s 2024-05-14T09:55:24.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:24 smithi026 ceph-mon[36056]: pgmap v1039: 129 pgs: 129 active+clean; 697 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 13 MiB/s wr, 807 op/s 2024-05-14T09:55:25.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 211015 26.78 MB/sec execute 291 sec latency 102.276 ms 2024-05-14T09:55:26.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 211489 26.77 MB/sec execute 292 sec latency 178.734 ms 2024-05-14T09:55:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:26 smithi002 ceph-mon[31557]: pgmap v1040: 129 pgs: 129 active+clean; 678 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 13 MiB/s wr, 753 op/s 2024-05-14T09:55:26.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:26 smithi029 ceph-mon[36249]: pgmap v1040: 129 pgs: 129 active+clean; 678 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 13 MiB/s wr, 753 op/s 2024-05-14T09:55:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:26 smithi026 ceph-mon[36056]: pgmap v1040: 129 pgs: 129 active+clean; 678 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 13 MiB/s wr, 753 op/s 2024-05-14T09:55:27.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 212211 26.79 MB/sec execute 293 sec latency 52.218 ms 2024-05-14T09:55:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.747+0000 7f83b6100640 1 -- 172.21.15.2:0/4108881141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b0102b50 msgr2=0x7f83b0103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.747+0000 7f83b6100640 1 --2- 172.21.15.2:0/4108881141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b0102b50 0x7f83b0103850 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009900 tx=0x7f83a002a410 comp rx=0 tx=0).stop 2024-05-14T09:55:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.747+0000 7f83b6100640 1 -- 172.21.15.2:0/4108881141 shutdown_connections 2024-05-14T09:55:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.747+0000 7f83b6100640 1 --2- 172.21.15.2:0/4108881141 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0108a90 0x7f83b010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.747+0000 7f83b6100640 1 --2- 172.21.15.2:0/4108881141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83b0103d90 0x7f83b01041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.747+0000 7f83b6100640 1 --2- 172.21.15.2:0/4108881141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b0102b50 0x7f83b0103850 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.747+0000 7f83b6100640 1 -- 172.21.15.2:0/4108881141 >> 172.21.15.2:0/4108881141 conn(0x7f83b00fdd40 msgr2=0x7f83b01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.747+0000 7f83b6100640 1 -- 172.21.15.2:0/4108881141 shutdown_connections 2024-05-14T09:55:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.748+0000 7f83b6100640 1 -- 172.21.15.2:0/4108881141 wait complete. 2024-05-14T09:55:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.748+0000 7f83b6100640 1 Processor -- start 2024-05-14T09:55:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.748+0000 7f83b6100640 1 -- start start 2024-05-14T09:55:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.748+0000 7f83b6100640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83b0102b50 0x7f83b019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.748+0000 7f83b6100640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b0103d90 0x7f83b019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.748+0000 7f83b6100640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0108a90 0x7f83b01a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.748+0000 7f83b6100640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83b010e650 con 0x7f83b0103d90 2024-05-14T09:55:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.749+0000 7f83b6100640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83b010e4d0 con 0x7f83b0102b50 2024-05-14T09:55:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.749+0000 7f83b6100640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83b010e7d0 con 0x7f83b0108a90 2024-05-14T09:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.749+0000 7f83af7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83b0102b50 0x7f83b019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.749+0000 7f83aeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b0103d90 0x7f83b019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.749+0000 7f83af7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83b0102b50 0x7f83b019a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:52928/0 (socket says 172.21.15.2:52928) 2024-05-14T09:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.749+0000 7f83af7fe640 1 -- 172.21.15.2:0/155308003 learned_addr learned my addr 172.21.15.2:0/155308003 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.749+0000 7f83affff640 1 --2- 172.21.15.2:0/155308003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0108a90 0x7f83b01a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.749+0000 7f83aeffd640 1 -- 172.21.15.2:0/155308003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0108a90 msgr2=0x7f83b01a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.749+0000 7f83aeffd640 1 --2- 172.21.15.2:0/155308003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0108a90 0x7f83b01a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.749+0000 7f83aeffd640 1 -- 172.21.15.2:0/155308003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83b0102b50 msgr2=0x7f83b019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.749+0000 7f83aeffd640 1 --2- 172.21.15.2:0/155308003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83b0102b50 0x7f83b019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.749+0000 7f83aeffd640 1 -- 172.21.15.2:0/155308003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b01a1df0 con 0x7f83b0103d90 2024-05-14T09:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.749+0000 7f83aeffd640 1 --2- 172.21.15.2:0/155308003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b0103d90 0x7f83b019a7f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f83a400e9f0 tx=0x7f83a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.749+0000 7f83acff9640 1 -- 172.21.15.2:0/155308003 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a400ccc0 con 0x7f83b0103d90 2024-05-14T09:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.750+0000 7f83b6100640 1 -- 172.21.15.2:0/155308003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b01a20e0 con 0x7f83b0103d90 2024-05-14T09:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.750+0000 7f83acff9640 1 -- 172.21.15.2:0/155308003 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a40108e0 con 0x7f83b0103d90 2024-05-14T09:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.750+0000 7f83acff9640 1 -- 172.21.15.2:0/155308003 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4018880 con 0x7f83b0103d90 2024-05-14T09:55:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.750+0000 7f83b6100640 1 -- 172.21.15.2:0/155308003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b00fe3e0 con 0x7f83b0103d90 2024-05-14T09:55:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.753+0000 7f83acff9640 1 -- 172.21.15.2:0/155308003 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f83a4010430 con 0x7f83b0103d90 2024-05-14T09:55:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.753+0000 7f83bcfb8740 1 Processor -- start 2024-05-14T09:55:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.754+0000 7f83bcfb8740 1 -- start start 2024-05-14T09:55:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.754+0000 7f83acff9640 1 --2- 172.21.15.2:0/155308003 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83980b28a0 0x7f83980b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.754+0000 7f83bcfb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a951b0 0x5585c7e36a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.754+0000 7f83bcfb8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a6c2f0 0x5585c7e36fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.754+0000 7f83bcfb8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a744d0 0x5585c7e3dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.754+0000 7f83bcfb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5585c7d9b5d0 con 0x5585c7a951b0 2024-05-14T09:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.754+0000 7f83bcfb8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5585c7d54890 con 0x5585c7a744d0 2024-05-14T09:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.754+0000 7f83bcfb8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5585c7d97590 con 0x5585c7a6c2f0 2024-05-14T09:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.754+0000 7f83acff9640 1 -- 172.21.15.2:0/155308003 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7f83a40e2970 con 0x7f83b0103d90 2024-05-14T09:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.754+0000 7f83affff640 1 --2- 172.21.15.2:0/155308003 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83980b28a0 0x7f83980b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.754+0000 7f83affff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a744d0 0x5585c7e3dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.754+0000 7f83aeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a6c2f0 0x5585c7e36fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.754+0000 7f83affff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a744d0 0x5585c7e3dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:52936/0 (socket says 172.21.15.2:52936) 2024-05-14T09:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.755+0000 7f83affff640 1 -- 172.21.15.2:0/354927569 learned_addr learned my addr 172.21.15.2:0/354927569 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.755+0000 7f83af7fe640 1 --2- 172.21.15.2:0/354927569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a951b0 0x5585c7e36a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.755+0000 7f83aeffd640 1 -- 172.21.15.2:0/354927569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a744d0 msgr2=0x5585c7e3dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.755+0000 7f83aeffd640 1 --2- 172.21.15.2:0/354927569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a744d0 0x5585c7e3dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.755+0000 7f83aeffd640 1 -- 172.21.15.2:0/354927569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a951b0 msgr2=0x5585c7e36a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.755+0000 7f83affff640 1 --2- 172.21.15.2:0/155308003 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83980b28a0 0x7f83980b4db0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f83a80139d0 tx=0x7f83a8009ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.755+0000 7f83aeffd640 1 --2- 172.21.15.2:0/354927569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a951b0 0x5585c7e36a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.755+0000 7f83aeffd640 1 -- 172.21.15.2:0/354927569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585c7aad210 con 0x5585c7a6c2f0 2024-05-14T09:55:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.756+0000 7f83af7fe640 1 --2- 172.21.15.2:0/354927569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a951b0 0x5585c7e36a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:55:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.756+0000 7f83aeffd640 1 --2- 172.21.15.2:0/354927569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a6c2f0 0x5585c7e36fe0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f83a400e9c0 tx=0x7f83a400ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24d0d2a28026b389 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.756+0000 7f8395ffb640 1 -- 172.21.15.2:0/354927569 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a40b29d0 con 0x5585c7a6c2f0 2024-05-14T09:55:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.756+0000 7f8395ffb640 1 -- 172.21.15.2:0/354927569 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a4010d90 con 0x5585c7a6c2f0 2024-05-14T09:55:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.756+0000 7f8395ffb640 1 -- 172.21.15.2:0/354927569 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a40a5590 con 0x5585c7a6c2f0 2024-05-14T09:55:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.757+0000 7f83bcfb8740 1 -- 172.21.15.2:0/354927569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a6c2f0 msgr2=0x5585c7e36fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.757+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/354927569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a6c2f0 0x5585c7e36fe0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f83a400e9c0 tx=0x7f83a400ce60 comp rx=0 tx=0).stop 2024-05-14T09:55:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.757+0000 7f83bcfb8740 1 -- 172.21.15.2:0/354927569 shutdown_connections 2024-05-14T09:55:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.757+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/354927569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a744d0 0x5585c7e3dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.757+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/354927569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a6c2f0 0x5585c7e36fe0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.757+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/354927569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a951b0 0x5585c7e36a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.757+0000 7f83bcfb8740 1 -- 172.21.15.2:0/354927569 >> 172.21.15.2:0/354927569 conn(0x5585c7a89300 msgr2=0x5585c7e323a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.757+0000 7f83bcfb8740 1 -- 172.21.15.2:0/354927569 shutdown_connections 2024-05-14T09:55:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.757+0000 7f83bcfb8740 1 -- 172.21.15.2:0/354927569 wait complete. 2024-05-14T09:55:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.758+0000 7f83bcfb8740 1 -- start start 2024-05-14T09:55:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.758+0000 7f83bcfb8740 1 Processor -- start 2024-05-14T09:55:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.759+0000 7f83bcfb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a6c2f0 0x5585c7e405c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.759+0000 7f83bcfb8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a744d0 0x5585c7e40b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.759+0000 7f83bcfb8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a951b0 0x5585c7e47a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.759+0000 7f83bcfb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5585c7e3e870 con 0x5585c7a6c2f0 2024-05-14T09:55:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.759+0000 7f83bcfb8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5585c7d58370 con 0x5585c7a951b0 2024-05-14T09:55:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.759+0000 7f83affff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a951b0 0x5585c7e47a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.759+0000 7f83affff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a951b0 0x5585c7e47a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:52940/0 (socket says 172.21.15.2:52940) 2024-05-14T09:55:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.759+0000 7f83affff640 1 -- 172.21.15.2:0/659869548 learned_addr learned my addr 172.21.15.2:0/659869548 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.759+0000 7f83af7fe640 1 --2- 172.21.15.2:0/659869548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a6c2f0 0x5585c7e405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.759+0000 7f83bcfb8740 1 -- 172.21.15.2:0/659869548 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5585c7e3e9f0 con 0x5585c7a744d0 2024-05-14T09:55:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.759+0000 7f83aeffd640 1 --2- 172.21.15.2:0/659869548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a744d0 0x5585c7e40b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.760+0000 7f83bcfb8740 10 client.0 _ll_register_callbacks cb 0x5585c7aaca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.760+0000 7f83affff640 1 -- 172.21.15.2:0/659869548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a744d0 msgr2=0x5585c7e40b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.760+0000 7f83affff640 1 --2- 172.21.15.2:0/659869548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a744d0 0x5585c7e40b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.760+0000 7f83affff640 1 -- 172.21.15.2:0/659869548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a6c2f0 msgr2=0x5585c7e405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.760+0000 7f83affff640 1 --2- 172.21.15.2:0/659869548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a6c2f0 0x5585c7e405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.760+0000 7f83affff640 1 -- 172.21.15.2:0/659869548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585c7b79620 con 0x5585c7a951b0 2024-05-14T09:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.760+0000 7f83aeffd640 1 --2- 172.21.15.2:0/659869548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a744d0 0x5585c7e40b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.760+0000 7f83affff640 1 --2- 172.21.15.2:0/659869548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a951b0 0x5585c7e47a60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f83a8005fd0 tx=0x7f83a801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f099e4e44cac2a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.760+0000 7f838effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.760+0000 7f838effd640 1 -- 172.21.15.2:0/659869548 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8003970 con 0x5585c7a951b0 2024-05-14T09:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.760+0000 7f83bcfb8740 1 -- 172.21.15.2:0/659869548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83a4010d90 con 0x5585c7a951b0 2024-05-14T09:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.760+0000 7f838effd640 1 -- 172.21.15.2:0/659869548 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7f83a8003b10 con 0x5585c7a951b0 2024-05-14T09:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.761+0000 7f838effd640 1 -- 172.21.15.2:0/659869548 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f83a802e050 con 0x5585c7a951b0 2024-05-14T09:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.761+0000 7f83bcfb8740 10 client.26972 fetch_fsmap learned FSMap version 65 2024-05-14T09:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.761+0000 7f83bcfb8740 1 -- 172.21.15.2:0/659869548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5585c7e4a9c0 con 0x5585c7a951b0 2024-05-14T09:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.761+0000 7f838effd640 1 -- 172.21.15.2:0/659869548 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a4010d90 con 0x5585c7a951b0 2024-05-14T09:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.762+0000 7f838effd640 1 -- 172.21.15.2:0/659869548 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 65) ==== 8346+0+0 (secure 0 0 0) 0x7f83a8027070 con 0x5585c7a951b0 2024-05-14T09:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.762+0000 7f83bcfb8740 10 client.26972 fetch_fsmap finished waiting for FSMap version 65 2024-05-14T09:55:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.762+0000 7f83bcfb8740 10 client.26972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:55:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.762+0000 7f83bcfb8740 20 client.26972 populate_metadata read hostname 'smithi002' 2024-05-14T09:55:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.762+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/659869548 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5585c7a53d80 0x5585c7e4e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.762+0000 7f83bcfb8740 4 client.26972 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:55:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.762+0000 7f83bcfb8740 1 -- 172.21.15.2:0/659869548 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585c7da91e0 con 0x5585c7a53d80 2024-05-14T09:55:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.762+0000 7f83af7fe640 1 --2- 172.21.15.2:0/659869548 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5585c7a53d80 0x5585c7e4e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.764+0000 7f838effd640 10 client.26972 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.764+0000 7f83af7fe640 1 --2- 172.21.15.2:0/659869548 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5585c7a53d80 0x5585c7e4e120 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d7b9df3c792335 server_cookie=7a541f47e854a889 in_seq=0 out_seq=0 2024-05-14T09:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.767+0000 7f838effd640 1 -- 172.21.15.2:0/659869548 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5585c7da91e0 con 0x5585c7a53d80 2024-05-14T09:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.767+0000 7f838effd640 10 client.26972 handle_command_reply: tid=0 2024-05-14T09:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.767+0000 7f83bcfb8740 1 client.26972 shutdown 2024-05-14T09:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.767+0000 7f83bcfb8740 1 -- 172.21.15.2:0/659869548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a951b0 msgr2=0x5585c7e47a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.767+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/659869548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a951b0 0x5585c7e47a60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f83a8005fd0 tx=0x7f83a801b000 comp rx=0 tx=0).stop 2024-05-14T09:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.767+0000 7f83bcfb8740 1 -- 172.21.15.2:0/659869548 shutdown_connections 2024-05-14T09:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.767+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/659869548 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5585c7a53d80 0x5585c7e4e120 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.767+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/659869548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a951b0 0x5585c7e47a60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.767+0000 7f838effd640 0 client.26972 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.767+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/659869548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a744d0 0x5585c7e40b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.767+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/659869548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a6c2f0 0x5585c7e405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.767+0000 7f83bcfb8740 1 -- 172.21.15.2:0/659869548 >> 172.21.15.2:0/659869548 conn(0x5585c7a89300 msgr2=0x5585c7e35300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.767+0000 7f83bcfb8740 1 -- 172.21.15.2:0/659869548 shutdown_connections 2024-05-14T09:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.767+0000 7f83bcfb8740 1 -- 172.21.15.2:0/659869548 wait complete. 2024-05-14T09:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.767+0000 7f83bcfb8740 20 client.26972 trim_cache size 0 max 16384 2024-05-14T09:55:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.789+0000 7f83bcfb8740 1 Processor -- start 2024-05-14T09:55:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.789+0000 7f83bcfb8740 1 -- start start 2024-05-14T09:55:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.790+0000 7f83bcfb8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a6c2f0 0x5585c7e31d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.790+0000 7f83bcfb8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a744d0 0x5585c7e322a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.790+0000 7f83bcfb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a951b0 0x5585c7e47980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.790+0000 7f83af7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a6c2f0 0x5585c7e31d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.790+0000 7f83af7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a6c2f0 0x5585c7e31d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:52950/0 (socket says 172.21.15.2:52950) 2024-05-14T09:55:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.790+0000 7f83aeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a744d0 0x5585c7e322a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.790+0000 7f83aeffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a744d0 0x5585c7e322a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:59512/0 (socket says 172.21.15.2:59512) 2024-05-14T09:55:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.790+0000 7f83bcfb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83b01b17c0 con 0x5585c7a951b0 2024-05-14T09:55:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.790+0000 7f83bcfb8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83b01af090 con 0x5585c7a6c2f0 2024-05-14T09:55:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.790+0000 7f83bcfb8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83b01acee0 con 0x5585c7a744d0 2024-05-14T09:55:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.790+0000 7f83af7fe640 1 -- 172.21.15.2:0/1745269461 learned_addr learned my addr 172.21.15.2:0/1745269461 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.790+0000 7f83affff640 1 --2- 172.21.15.2:0/1745269461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a951b0 0x5585c7e47980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.791+0000 7f83af7fe640 1 -- 172.21.15.2:0/1745269461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a744d0 msgr2=0x5585c7e322a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.791+0000 7f83af7fe640 1 --2- 172.21.15.2:0/1745269461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a744d0 0x5585c7e322a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.791+0000 7f83af7fe640 1 -- 172.21.15.2:0/1745269461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a951b0 msgr2=0x5585c7e47980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.791+0000 7f83af7fe640 1 --2- 172.21.15.2:0/1745269461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a951b0 0x5585c7e47980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.791+0000 7f83af7fe640 1 -- 172.21.15.2:0/1745269461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585c7e41630 con 0x5585c7a6c2f0 2024-05-14T09:55:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.791+0000 7f83af7fe640 1 --2- 172.21.15.2:0/1745269461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a6c2f0 0x5585c7e31d60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f83a0005ab0 tx=0x7f83a0031870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a2f54d70a1a50b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.791+0000 7f83957fa640 1 -- 172.21.15.2:0/1745269461 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0035040 con 0x5585c7a6c2f0 2024-05-14T09:55:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.791+0000 7f83957fa640 1 -- 172.21.15.2:0/1745269461 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a004a070 con 0x5585c7a6c2f0 2024-05-14T09:55:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.791+0000 7f83957fa640 1 -- 172.21.15.2:0/1745269461 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00035f0 con 0x5585c7a6c2f0 2024-05-14T09:55:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.793+0000 7f83bcfb8740 1 -- 172.21.15.2:0/1745269461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a6c2f0 msgr2=0x5585c7e31d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.793+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/1745269461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a6c2f0 0x5585c7e31d60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f83a0005ab0 tx=0x7f83a0031870 comp rx=0 tx=0).stop 2024-05-14T09:55:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.793+0000 7f83bcfb8740 1 -- 172.21.15.2:0/1745269461 shutdown_connections 2024-05-14T09:55:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.793+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/1745269461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a951b0 0x5585c7e47980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.793+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/1745269461 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a744d0 0x5585c7e322a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.793+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/1745269461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a6c2f0 0x5585c7e31d60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.793+0000 7f83bcfb8740 1 -- 172.21.15.2:0/1745269461 >> 172.21.15.2:0/1745269461 conn(0x5585c7a89300 msgr2=0x5585c7e3aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.793+0000 7f83bcfb8740 1 -- 172.21.15.2:0/1745269461 shutdown_connections 2024-05-14T09:55:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.793+0000 7f83bcfb8740 1 -- 172.21.15.2:0/1745269461 wait complete. 2024-05-14T09:55:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.793+0000 7f83bcfb8740 1 -- start start 2024-05-14T09:55:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.793+0000 7f83bcfb8740 1 Processor -- start 2024-05-14T09:55:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83bcfb8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a6c2f0 0x5585c7e453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83bcfb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a744d0 0x5585c7e45920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83bcfb8740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a951b0 0x5585c7e46850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83bcfb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83b01ad550 con 0x5585c7a744d0 2024-05-14T09:55:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83bcfb8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f83b01ae9c0 con 0x5585c7a6c2f0 2024-05-14T09:55:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83bcfb8740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f83b01adc20 con 0x5585c7a951b0 2024-05-14T09:55:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83bcfb8740 10 client.0 _ll_register_callbacks cb 0x7f83a8009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:55:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83aeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a744d0 0x5585c7e45920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83aeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a744d0 0x5585c7e45920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51952/0 (socket says 172.21.15.2:51952) 2024-05-14T09:55:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83aeffd640 1 -- 172.21.15.2:0/2328727912 learned_addr learned my addr 172.21.15.2:0/2328727912 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83af7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a6c2f0 0x5585c7e453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83affff640 1 --2- 172.21.15.2:0/2328727912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a951b0 0x5585c7e46850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83aeffd640 1 -- 172.21.15.2:0/2328727912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a951b0 msgr2=0x5585c7e46850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83aeffd640 1 --2- 172.21.15.2:0/2328727912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a951b0 0x5585c7e46850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83aeffd640 1 -- 172.21.15.2:0/2328727912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a6c2f0 msgr2=0x5585c7e453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83aeffd640 1 --2- 172.21.15.2:0/2328727912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a6c2f0 0x5585c7e453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83aeffd640 1 -- 172.21.15.2:0/2328727912 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585c7e46e10 con 0x5585c7a744d0 2024-05-14T09:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.794+0000 7f83af7fe640 1 --2- 172.21.15.2:0/2328727912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a6c2f0 0x5585c7e453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.795+0000 7f83aeffd640 1 --2- 172.21.15.2:0/2328727912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a744d0 0x5585c7e45920 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f83a40e00a0 tx=0x7f83a40ae0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7087e4d51162e03e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.795+0000 7f838effd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.795+0000 7f838effd640 1 -- 172.21.15.2:0/2328727912 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a40a99f0 con 0x5585c7a744d0 2024-05-14T09:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.795+0000 7f838effd640 1 -- 172.21.15.2:0/2328727912 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7f83a40a5590 con 0x5585c7a744d0 2024-05-14T09:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.795+0000 7f83bcfb8740 1 -- 172.21.15.2:0/2328727912 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83a004a070 con 0x5585c7a744d0 2024-05-14T09:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.795+0000 7f838effd640 1 -- 172.21.15.2:0/2328727912 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f83a4010d90 con 0x5585c7a744d0 2024-05-14T09:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.795+0000 7f83bcfb8740 10 client.17361 fetch_fsmap learned FSMap version 65 2024-05-14T09:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.795+0000 7f83bcfb8740 1 -- 172.21.15.2:0/2328727912 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5585c7e480a0 con 0x5585c7a744d0 2024-05-14T09:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.795+0000 7f838effd640 1 -- 172.21.15.2:0/2328727912 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a004a070 con 0x5585c7a744d0 2024-05-14T09:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.795+0000 7f838effd640 1 -- 172.21.15.2:0/2328727912 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 65) ==== 8346+0+0 (secure 0 0 0) 0x7f83a40c4020 con 0x5585c7a744d0 2024-05-14T09:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.796+0000 7f83bcfb8740 10 client.17361 fetch_fsmap finished waiting for FSMap version 65 2024-05-14T09:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.796+0000 7f83bcfb8740 10 client.17361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.796+0000 7f83bcfb8740 20 client.17361 populate_metadata read hostname 'smithi002' 2024-05-14T09:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.796+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/2328727912 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5585c7a291c0 0x5585c7eb84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.796+0000 7f83bcfb8740 4 client.17361 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.796+0000 7f83bcfb8740 1 -- 172.21.15.2:0/2328727912 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585c7da3050 con 0x5585c7a291c0 2024-05-14T09:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.796+0000 7f83af7fe640 1 --2- 172.21.15.2:0/2328727912 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5585c7a291c0 0x5585c7eb84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.796+0000 7f83af7fe640 1 --2- 172.21.15.2:0/2328727912 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5585c7a291c0 0x5585c7eb84e0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e51566842ba8eb46 server_cookie=6104b5117405b5ac in_seq=0 out_seq=0 2024-05-14T09:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:27.796+0000 7f838effd640 10 client.17361 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.369+0000 7f838effd640 1 -- 172.21.15.2:0/2328727912 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585c7da3050 con 0x5585c7a291c0 2024-05-14T09:55:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.369+0000 7f838effd640 10 client.17361 handle_command_reply: tid=0 2024-05-14T09:55:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.369+0000 7f83bcfb8740 1 client.17361 shutdown 2024-05-14T09:55:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.369+0000 7f83bcfb8740 1 -- 172.21.15.2:0/2328727912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a744d0 msgr2=0x5585c7e45920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.369+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/2328727912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a744d0 0x5585c7e45920 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f83a40e00a0 tx=0x7f83a40ae0a0 comp rx=0 tx=0).stop 2024-05-14T09:55:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.369+0000 7f83bcfb8740 1 -- 172.21.15.2:0/2328727912 shutdown_connections 2024-05-14T09:55:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.369+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/2328727912 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5585c7a291c0 0x5585c7eb84e0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.369+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/2328727912 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5585c7a951b0 0x5585c7e46850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.369+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/2328727912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5585c7a744d0 0x5585c7e45920 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.369+0000 7f83bcfb8740 1 --2- 172.21.15.2:0/2328727912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5585c7a6c2f0 0x5585c7e453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.369+0000 7f838effd640 0 client.17361 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.369+0000 7f83bcfb8740 1 -- 172.21.15.2:0/2328727912 >> 172.21.15.2:0/2328727912 conn(0x5585c7a89300 msgr2=0x5585c7e31d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.370+0000 7f83bcfb8740 1 -- 172.21.15.2:0/2328727912 shutdown_connections 2024-05-14T09:55:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.370+0000 7f83bcfb8740 1 -- 172.21.15.2:0/2328727912 wait complete. 2024-05-14T09:55:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.370+0000 7f83bcfb8740 20 client.17361 trim_cache size 0 max 16384 2024-05-14T09:55:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:55:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:55:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:55:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:55:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.370+0000 7f83b6100640 1 -- 172.21.15.2:0/155308003 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83980b28a0 msgr2=0x7f83980b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.370+0000 7f83b6100640 1 --2- 172.21.15.2:0/155308003 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83980b28a0 0x7f83980b4db0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f83a80139d0 tx=0x7f83a8009ea0 comp rx=0 tx=0).stop 2024-05-14T09:55:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.370+0000 7f83b6100640 1 -- 172.21.15.2:0/155308003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b0103d90 msgr2=0x7f83b019a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.370+0000 7f83b6100640 1 --2- 172.21.15.2:0/155308003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b0103d90 0x7f83b019a7f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f83a400e9f0 tx=0x7f83a400eeb0 comp rx=0 tx=0).stop 2024-05-14T09:55:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.370+0000 7f83b6100640 1 -- 172.21.15.2:0/155308003 shutdown_connections 2024-05-14T09:55:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.371+0000 7f83b6100640 1 --2- 172.21.15.2:0/155308003 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83980b28a0 0x7f83980b4db0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.371+0000 7f83b6100640 1 --2- 172.21.15.2:0/155308003 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f83b0108a90 0x7f83b01a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.371+0000 7f83b6100640 1 --2- 172.21.15.2:0/155308003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b0103d90 0x7f83b019a7f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.371+0000 7f83b6100640 1 --2- 172.21.15.2:0/155308003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83b0102b50 0x7f83b019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.371+0000 7f83b6100640 1 -- 172.21.15.2:0/155308003 >> 172.21.15.2:0/155308003 conn(0x7f83b00fdd40 msgr2=0x7f83b0109f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.371+0000 7f83b6100640 1 -- 172.21.15.2:0/155308003 shutdown_connections 2024-05-14T09:55:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:28.371+0000 7f83b6100640 1 -- 172.21.15.2:0/155308003 wait complete. 2024-05-14T09:55:28.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 212654 26.79 MB/sec execute 294 sec latency 401.296 ms 2024-05-14T09:55:28.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:28 smithi002 ceph-mon[31557]: pgmap v1041: 129 pgs: 129 active+clean; 686 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 11 MiB/s wr, 740 op/s 2024-05-14T09:55:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:28 smithi029 ceph-mon[36249]: pgmap v1041: 129 pgs: 129 active+clean; 686 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 11 MiB/s wr, 740 op/s 2024-05-14T09:55:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:28 smithi026 ceph-mon[36056]: pgmap v1041: 129 pgs: 129 active+clean; 686 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 11 MiB/s wr, 740 op/s 2024-05-14T09:55:29.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 213290 26.84 MB/sec execute 295 sec latency 152.347 ms 2024-05-14T09:55:30.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 213684 26.81 MB/sec execute 296 sec latency 464.619 ms 2024-05-14T09:55:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:30 smithi002 ceph-mon[31557]: pgmap v1042: 129 pgs: 129 active+clean; 689 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.6 MiB/s wr, 637 op/s 2024-05-14T09:55:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:30 smithi029 ceph-mon[36249]: pgmap v1042: 129 pgs: 129 active+clean; 689 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.6 MiB/s wr, 637 op/s 2024-05-14T09:55:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:30 smithi026 ceph-mon[36056]: pgmap v1042: 129 pgs: 129 active+clean; 689 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.6 MiB/s wr, 637 op/s 2024-05-14T09:55:31.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 214048 26.76 MB/sec execute 297 sec latency 156.227 ms 2024-05-14T09:55:32.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 214498 26.77 MB/sec execute 298 sec latency 234.569 ms 2024-05-14T09:55:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:32 smithi002 ceph-mon[31557]: pgmap v1043: 129 pgs: 129 active+clean; 689 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 649 op/s 2024-05-14T09:55:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:32 smithi029 ceph-mon[36249]: pgmap v1043: 129 pgs: 129 active+clean; 689 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 649 op/s 2024-05-14T09:55:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:32 smithi026 ceph-mon[36056]: pgmap v1043: 129 pgs: 129 active+clean; 689 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 649 op/s 2024-05-14T09:55:33.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 215078 26.79 MB/sec execute 299 sec latency 112.779 ms 2024-05-14T09:55:34.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 215678 26.77 MB/sec execute 300 sec latency 96.814 ms 2024-05-14T09:55:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:34 smithi002 ceph-mon[31557]: pgmap v1044: 129 pgs: 129 active+clean; 689 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.8 MiB/s wr, 564 op/s 2024-05-14T09:55:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:34 smithi029 ceph-mon[36249]: pgmap v1044: 129 pgs: 129 active+clean; 689 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.8 MiB/s wr, 564 op/s 2024-05-14T09:55:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:34 smithi026 ceph-mon[36056]: pgmap v1044: 129 pgs: 129 active+clean; 689 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.8 MiB/s wr, 564 op/s 2024-05-14T09:55:35.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 216388 26.83 MB/sec execute 301 sec latency 73.722 ms 2024-05-14T09:55:36.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 217005 26.89 MB/sec execute 302 sec latency 60.495 ms 2024-05-14T09:55:36.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:36 smithi002 ceph-mon[31557]: pgmap v1045: 129 pgs: 129 active+clean; 695 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.0 MiB/s wr, 509 op/s 2024-05-14T09:55:36.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:36 smithi029 ceph-mon[36249]: pgmap v1045: 129 pgs: 129 active+clean; 695 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.0 MiB/s wr, 509 op/s 2024-05-14T09:55:36.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:36 smithi026 ceph-mon[36056]: pgmap v1045: 129 pgs: 129 active+clean; 695 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.0 MiB/s wr, 509 op/s 2024-05-14T09:55:37.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 217324 26.86 MB/sec execute 303 sec latency 218.016 ms 2024-05-14T09:55:38.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 217669 26.80 MB/sec execute 304 sec latency 535.491 ms 2024-05-14T09:55:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:38 smithi002 ceph-mon[31557]: pgmap v1046: 129 pgs: 129 active+clean; 701 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.4 MiB/s wr, 530 op/s 2024-05-14T09:55:38.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:38 smithi029 ceph-mon[36249]: pgmap v1046: 129 pgs: 129 active+clean; 701 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.4 MiB/s wr, 530 op/s 2024-05-14T09:55:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:38 smithi026 ceph-mon[36056]: pgmap v1046: 129 pgs: 129 active+clean; 701 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.4 MiB/s wr, 530 op/s 2024-05-14T09:55:39.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 218193 26.79 MB/sec execute 305 sec latency 120.633 ms 2024-05-14T09:55:40.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 218978 26.83 MB/sec execute 306 sec latency 90.092 ms 2024-05-14T09:55:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:40 smithi002 ceph-mon[31557]: pgmap v1047: 129 pgs: 129 active+clean; 705 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.8 MiB/s wr, 451 op/s 2024-05-14T09:55:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:40 smithi029 ceph-mon[36249]: pgmap v1047: 129 pgs: 129 active+clean; 705 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.8 MiB/s wr, 451 op/s 2024-05-14T09:55:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:40 smithi026 ceph-mon[36056]: pgmap v1047: 129 pgs: 129 active+clean; 705 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.8 MiB/s wr, 451 op/s 2024-05-14T09:55:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.564+0000 7f94fcb8f640 1 -- 172.21.15.2:0/1991714024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94f8109b90 msgr2=0x7f94f810bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.564+0000 7f94fcb8f640 1 --2- 172.21.15.2:0/1991714024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94f8109b90 0x7f94f810bfb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009be0 tx=0x7f94f002a6c0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.564+0000 7f94fcb8f640 1 -- 172.21.15.2:0/1991714024 shutdown_connections 2024-05-14T09:55:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.564+0000 7f94fcb8f640 1 --2- 172.21.15.2:0/1991714024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94f8109b90 0x7f94f810bfb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.564+0000 7f94fcb8f640 1 --2- 172.21.15.2:0/1991714024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f8104e90 0x7f94f81052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.564+0000 7f94fcb8f640 1 --2- 172.21.15.2:0/1991714024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94f8103c50 0x7f94f8104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.564+0000 7f94fcb8f640 1 -- 172.21.15.2:0/1991714024 >> 172.21.15.2:0/1991714024 conn(0x7f94f80fee60 msgr2=0x7f94f81012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.564+0000 7f94fcb8f640 1 -- 172.21.15.2:0/1991714024 shutdown_connections 2024-05-14T09:55:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.564+0000 7f94fcb8f640 1 -- 172.21.15.2:0/1991714024 wait complete. 2024-05-14T09:55:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.565+0000 7f94fcb8f640 1 Processor -- start 2024-05-14T09:55:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.565+0000 7f94fcb8f640 1 -- start start 2024-05-14T09:55:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.565+0000 7f94fcb8f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f8103c50 0x7f94f819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.565+0000 7f94fcb8f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94f8104e90 0x7f94f819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.565+0000 7f94fcb8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94f8109b90 0x7f94f81a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.565+0000 7f94fcb8f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f94f810f760 con 0x7f94f8103c50 2024-05-14T09:55:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.565+0000 7f94fcb8f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94f810f5e0 con 0x7f94f8109b90 2024-05-14T09:55:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.565+0000 7f94fcb8f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f94f810f8e0 con 0x7f94f8104e90 2024-05-14T09:55:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.565+0000 7f94f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f8103c50 0x7f94f819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.565+0000 7f94effff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94f8104e90 0x7f94f819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.565+0000 7f94effff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94f8104e90 0x7f94f819a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:40732/0 (socket says 172.21.15.2:40732) 2024-05-14T09:55:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.565+0000 7f94effff640 1 -- 172.21.15.2:0/2611384002 learned_addr learned my addr 172.21.15.2:0/2611384002 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.565+0000 7f94f6d9d640 1 --2- 172.21.15.2:0/2611384002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94f8109b90 0x7f94f81a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.566+0000 7f94effff640 1 -- 172.21.15.2:0/2611384002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94f8109b90 msgr2=0x7f94f81a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.566+0000 7f94effff640 1 --2- 172.21.15.2:0/2611384002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94f8109b90 0x7f94f81a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.566+0000 7f94effff640 1 -- 172.21.15.2:0/2611384002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f8103c50 msgr2=0x7f94f819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.566+0000 7f94effff640 1 --2- 172.21.15.2:0/2611384002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f8103c50 0x7f94f819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.566+0000 7f94effff640 1 -- 172.21.15.2:0/2611384002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f81a1e70 con 0x7f94f8104e90 2024-05-14T09:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.566+0000 7f94effff640 1 --2- 172.21.15.2:0/2611384002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94f8104e90 0x7f94f819a870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f94e000e9c0 tx=0x7f94e000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.566+0000 7f94ef7fe640 1 -- 172.21.15.2:0/2611384002 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e000cde0 con 0x7f94f8104e90 2024-05-14T09:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.566+0000 7f94fcb8f640 1 -- 172.21.15.2:0/2611384002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f8074f60 con 0x7f94f8104e90 2024-05-14T09:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.567+0000 7f94ef7fe640 1 -- 172.21.15.2:0/2611384002 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94e00035f0 con 0x7f94f8104e90 2024-05-14T09:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.567+0000 7f94ef7fe640 1 -- 172.21.15.2:0/2611384002 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0010690 con 0x7f94f8104e90 2024-05-14T09:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.567+0000 7f94fcb8f640 1 -- 172.21.15.2:0/2611384002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f8075500 con 0x7f94f8104e90 2024-05-14T09:55:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.570+0000 7f94ef7fe640 1 -- 172.21.15.2:0/2611384002 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f94e0010830 con 0x7f94f8104e90 2024-05-14T09:55:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.572+0000 7f94ef7fe640 1 --2- 172.21.15.2:0/2611384002 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f94e40b2920 0x7f94e40b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.572+0000 7f94ef7fe640 1 -- 172.21.15.2:0/2611384002 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7f94e00e3780 con 0x7f94f8104e90 2024-05-14T09:55:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.572+0000 7f94f659c640 1 --2- 172.21.15.2:0/2611384002 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f94e40b2920 0x7f94e40b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.583+0000 7f9503a45740 1 Processor -- start 2024-05-14T09:55:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.583+0000 7f94f659c640 1 --2- 172.21.15.2:0/2611384002 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f94e40b2920 0x7f94e40b4e30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f94e80095f0 tx=0x7f94e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.583+0000 7f9503a45740 1 -- start start 2024-05-14T09:55:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.584+0000 7f9503a45740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acd61b0 0x562e1b080270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.584+0000 7f9503a45740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acad2f0 0x562e1b0807f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.584+0000 7f9503a45740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acb54d0 0x562e1b0876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.584+0000 7f9503a45740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562e1afdc5d0 con 0x562e1acb54d0 2024-05-14T09:55:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.584+0000 7f9503a45740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562e1af95890 con 0x562e1acd61b0 2024-05-14T09:55:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.584+0000 7f9503a45740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562e1afd8590 con 0x562e1acad2f0 2024-05-14T09:55:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.584+0000 7f94f6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acb54d0 0x562e1b0876f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.584+0000 7f94effff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acad2f0 0x562e1b0807f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.584+0000 7f94f659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acd61b0 0x562e1b080270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.584+0000 7f94f6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acb54d0 0x562e1b0876f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57290/0 (socket says 172.21.15.2:57290) 2024-05-14T09:55:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.584+0000 7f94f6d9d640 1 -- 172.21.15.2:0/1347312964 learned_addr learned my addr 172.21.15.2:0/1347312964 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.584+0000 7f94effff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acad2f0 0x562e1b0807f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:40744/0 (socket says 172.21.15.2:40744) 2024-05-14T09:55:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.585+0000 7f94f6d9d640 1 -- 172.21.15.2:0/1347312964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acad2f0 msgr2=0x562e1b0807f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.585+0000 7f94f6d9d640 1 --2- 172.21.15.2:0/1347312964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acad2f0 0x562e1b0807f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.585+0000 7f94f6d9d640 1 -- 172.21.15.2:0/1347312964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acd61b0 msgr2=0x562e1b080270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.585+0000 7f94f6d9d640 1 --2- 172.21.15.2:0/1347312964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acd61b0 0x562e1b080270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.585+0000 7f94f6d9d640 1 -- 172.21.15.2:0/1347312964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e1acee210 con 0x562e1acb54d0 2024-05-14T09:55:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.585+0000 7f94f6d9d640 1 --2- 172.21.15.2:0/1347312964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acb54d0 0x562e1b0876f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009be0 tx=0x7f94f00303a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47700f0a7e180869 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.586+0000 7f94ecff9640 1 -- 172.21.15.2:0/1347312964 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0036040 con 0x562e1acb54d0 2024-05-14T09:55:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.586+0000 7f94ecff9640 1 -- 172.21.15.2:0/1347312964 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94f0003970 con 0x562e1acb54d0 2024-05-14T09:55:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.586+0000 7f94ecff9640 1 -- 172.21.15.2:0/1347312964 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0004340 con 0x562e1acb54d0 2024-05-14T09:55:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.587+0000 7f9503a45740 1 -- 172.21.15.2:0/1347312964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acb54d0 msgr2=0x562e1b0876f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.587+0000 7f9503a45740 1 --2- 172.21.15.2:0/1347312964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acb54d0 0x562e1b0876f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009be0 tx=0x7f94f00303a0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.587+0000 7f9503a45740 1 -- 172.21.15.2:0/1347312964 shutdown_connections 2024-05-14T09:55:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.587+0000 7f9503a45740 1 --2- 172.21.15.2:0/1347312964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acb54d0 0x562e1b0876f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.587+0000 7f9503a45740 1 --2- 172.21.15.2:0/1347312964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acad2f0 0x562e1b0807f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.587+0000 7f9503a45740 1 --2- 172.21.15.2:0/1347312964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acd61b0 0x562e1b080270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.587+0000 7f9503a45740 1 -- 172.21.15.2:0/1347312964 >> 172.21.15.2:0/1347312964 conn(0x562e1acca300 msgr2=0x562e1b07bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.587+0000 7f9503a45740 1 -- 172.21.15.2:0/1347312964 shutdown_connections 2024-05-14T09:55:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.587+0000 7f9503a45740 1 -- 172.21.15.2:0/1347312964 wait complete. 2024-05-14T09:55:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.588+0000 7f9503a45740 1 -- start start 2024-05-14T09:55:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.588+0000 7f9503a45740 1 Processor -- start 2024-05-14T09:55:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.588+0000 7f9503a45740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 0x562e1b089d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.588+0000 7f9503a45740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acb54d0 0x562e1b08a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.588+0000 7f9503a45740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acd61b0 0x562e1b0911e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.588+0000 7f9503a45740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562e1b088080 con 0x562e1acad2f0 2024-05-14T09:55:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.588+0000 7f9503a45740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562e1af99370 con 0x562e1acd61b0 2024-05-14T09:55:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.588+0000 7f9503a45740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x562e1b088200 con 0x562e1acb54d0 2024-05-14T09:55:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.588+0000 7f94f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 0x562e1b089d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.588+0000 7f94f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 0x562e1b089d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57300/0 (socket says 172.21.15.2:57300) 2024-05-14T09:55:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.589+0000 7f94f659c640 1 -- 172.21.15.2:0/1561506809 learned_addr learned my addr 172.21.15.2:0/1561506809 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.589+0000 7f9503a45740 10 client.0 _ll_register_callbacks cb 0x562e1aceda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:55:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.589+0000 7f94f6d9d640 1 --2- 172.21.15.2:0/1561506809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acd61b0 0x562e1b0911e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.589+0000 7f94effff640 1 --2- 172.21.15.2:0/1561506809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acb54d0 0x562e1b08a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.589+0000 7f94f659c640 1 -- 172.21.15.2:0/1561506809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acb54d0 msgr2=0x562e1b08a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.589+0000 7f94f659c640 1 --2- 172.21.15.2:0/1561506809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acb54d0 0x562e1b08a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.589+0000 7f94f659c640 1 -- 172.21.15.2:0/1561506809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acd61b0 msgr2=0x562e1b0911e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.589+0000 7f94f659c640 1 --2- 172.21.15.2:0/1561506809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acd61b0 0x562e1b0911e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.589+0000 7f94f659c640 1 -- 172.21.15.2:0/1561506809 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e1adba620 con 0x562e1acad2f0 2024-05-14T09:55:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.589+0000 7f94f659c640 1 --2- 172.21.15.2:0/1561506809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 0x562e1b089d80 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f94e8003a40 tx=0x7f94e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a8c7964aeecf9f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.589+0000 7f94ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:55:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.590+0000 7f94ddffb640 1 -- 172.21.15.2:0/1561506809 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e801fdc0 con 0x562e1acad2f0 2024-05-14T09:55:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.590+0000 7f9503a45740 1 -- 172.21.15.2:0/1561506809 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94f0003970 con 0x562e1acad2f0 2024-05-14T09:55:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.590+0000 7f94ddffb640 1 -- 172.21.15.2:0/1561506809 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7f94e80273e0 con 0x562e1acad2f0 2024-05-14T09:55:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.590+0000 7f94ddffb640 1 -- 172.21.15.2:0/1561506809 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f94e80275e0 con 0x562e1acad2f0 2024-05-14T09:55:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.590+0000 7f9503a45740 10 client.17373 fetch_fsmap learned FSMap version 65 2024-05-14T09:55:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.590+0000 7f9503a45740 1 -- 172.21.15.2:0/1561506809 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x562e1b094240 con 0x562e1acad2f0 2024-05-14T09:55:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.591+0000 7f94ddffb640 1 -- 172.21.15.2:0/1561506809 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94f0003970 con 0x562e1acad2f0 2024-05-14T09:55:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.591+0000 7f94ddffb640 1 -- 172.21.15.2:0/1561506809 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 65) ==== 8346+0+0 (secure 0 0 0) 0x7f94e8026070 con 0x562e1acad2f0 2024-05-14T09:55:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.591+0000 7f9503a45740 10 client.17373 fetch_fsmap finished waiting for FSMap version 65 2024-05-14T09:55:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.591+0000 7f9503a45740 10 client.17373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:55:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.591+0000 7f9503a45740 20 client.17373 populate_metadata read hostname 'smithi002' 2024-05-14T09:55:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.591+0000 7f9503a45740 1 --2- 172.21.15.2:0/1561506809 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562e1ac94d80 0x562e1b097a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.591+0000 7f9503a45740 4 client.17373 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:55:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.591+0000 7f94effff640 1 --2- 172.21.15.2:0/1561506809 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562e1ac94d80 0x562e1b097a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.591+0000 7f9503a45740 1 -- 172.21.15.2:0/1561506809 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e1afea1e0 con 0x562e1ac94d80 2024-05-14T09:55:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.592+0000 7f94ddffb640 10 client.17373 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.592+0000 7f94effff640 1 --2- 172.21.15.2:0/1561506809 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562e1ac94d80 0x562e1b097a40 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cb73ffef2ba5898 server_cookie=30e5d712bf7f99f3 in_seq=0 out_seq=0 2024-05-14T09:55:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.594+0000 7f94ddffb640 1 -- 172.21.15.2:0/1561506809 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e1afea1e0 con 0x562e1ac94d80 2024-05-14T09:55:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.594+0000 7f94ddffb640 10 client.17373 handle_command_reply: tid=0 2024-05-14T09:55:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.594+0000 7f9503a45740 1 client.17373 shutdown 2024-05-14T09:55:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.594+0000 7f9503a45740 1 -- 172.21.15.2:0/1561506809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 msgr2=0x562e1b089d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.594+0000 7f9503a45740 1 --2- 172.21.15.2:0/1561506809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 0x562e1b089d80 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f94e8003a40 tx=0x7f94e8004730 comp rx=0 tx=0).stop 2024-05-14T09:55:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.594+0000 7f9503a45740 1 -- 172.21.15.2:0/1561506809 shutdown_connections 2024-05-14T09:55:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.594+0000 7f9503a45740 1 --2- 172.21.15.2:0/1561506809 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562e1ac94d80 0x562e1b097a40 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.594+0000 7f9503a45740 1 --2- 172.21.15.2:0/1561506809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acd61b0 0x562e1b0911e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.594+0000 7f9503a45740 1 --2- 172.21.15.2:0/1561506809 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acb54d0 0x562e1b08a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.594+0000 7f9503a45740 1 --2- 172.21.15.2:0/1561506809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 0x562e1b089d80 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.594+0000 7f9503a45740 1 -- 172.21.15.2:0/1561506809 >> 172.21.15.2:0/1561506809 conn(0x562e1acca300 msgr2=0x562e1b07eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.594+0000 7f94ddffb640 0 client.17373 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.594+0000 7f9503a45740 1 -- 172.21.15.2:0/1561506809 shutdown_connections 2024-05-14T09:55:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.594+0000 7f9503a45740 1 -- 172.21.15.2:0/1561506809 wait complete. 2024-05-14T09:55:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.594+0000 7f9503a45740 20 client.17373 trim_cache size 0 max 16384 2024-05-14T09:55:41.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 219773 26.93 MB/sec execute 307 sec latency 58.229 ms 2024-05-14T09:55:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.618+0000 7f9503a45740 1 Processor -- start 2024-05-14T09:55:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.618+0000 7f9503a45740 1 -- start start 2024-05-14T09:55:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.618+0000 7f9503a45740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 0x562e1b0839f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.618+0000 7f9503a45740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acb54d0 0x562e1b0885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.618+0000 7f9503a45740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acd61b0 0x562e1b088b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.618+0000 7f9503a45740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f94f81ad410 con 0x562e1acad2f0 2024-05-14T09:55:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.618+0000 7f9503a45740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94f81aace0 con 0x562e1acd61b0 2024-05-14T09:55:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.618+0000 7f94effff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acb54d0 0x562e1b0885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.618+0000 7f94effff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acb54d0 0x562e1b0885d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:40764/0 (socket says 172.21.15.2:40764) 2024-05-14T09:55:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.618+0000 7f94effff640 1 -- 172.21.15.2:0/1644300975 learned_addr learned my addr 172.21.15.2:0/1644300975 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.618+0000 7f9503a45740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f94f81a8b30 con 0x562e1acb54d0 2024-05-14T09:55:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.619+0000 7f94f659c640 1 --2- 172.21.15.2:0/1644300975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 0x562e1b0839f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.619+0000 7f94f6d9d640 1 --2- 172.21.15.2:0/1644300975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acd61b0 0x562e1b088b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.619+0000 7f94effff640 1 -- 172.21.15.2:0/1644300975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acd61b0 msgr2=0x562e1b088b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.619+0000 7f94effff640 1 --2- 172.21.15.2:0/1644300975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acd61b0 0x562e1b088b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.619+0000 7f94effff640 1 -- 172.21.15.2:0/1644300975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 msgr2=0x562e1b0839f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.619+0000 7f94effff640 1 --2- 172.21.15.2:0/1644300975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 0x562e1b0839f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.619+0000 7f94effff640 1 -- 172.21.15.2:0/1644300975 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e1b07c510 con 0x562e1acb54d0 2024-05-14T09:55:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.619+0000 7f94f659c640 1 --2- 172.21.15.2:0/1644300975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 0x562e1b0839f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:55:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.619+0000 7f94effff640 1 --2- 172.21.15.2:0/1644300975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acb54d0 0x562e1b0885d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f94e00e11d0 tx=0x7f94e00e0d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=debef7c11bff1768 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.620+0000 7f94dffff640 1 -- 172.21.15.2:0/1644300975 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e00ab080 con 0x562e1acb54d0 2024-05-14T09:55:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.620+0000 7f94dffff640 1 -- 172.21.15.2:0/1644300975 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94e00aa980 con 0x562e1acb54d0 2024-05-14T09:55:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.620+0000 7f94dffff640 1 -- 172.21.15.2:0/1644300975 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e00037a0 con 0x562e1acb54d0 2024-05-14T09:55:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.621+0000 7f9503a45740 1 -- 172.21.15.2:0/1644300975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acb54d0 msgr2=0x562e1b0885d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.621+0000 7f9503a45740 1 --2- 172.21.15.2:0/1644300975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acb54d0 0x562e1b0885d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f94e00e11d0 tx=0x7f94e00e0d30 comp rx=0 tx=0).stop 2024-05-14T09:55:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.621+0000 7f9503a45740 1 -- 172.21.15.2:0/1644300975 shutdown_connections 2024-05-14T09:55:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.621+0000 7f9503a45740 1 --2- 172.21.15.2:0/1644300975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acd61b0 0x562e1b088b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.621+0000 7f9503a45740 1 --2- 172.21.15.2:0/1644300975 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acb54d0 0x562e1b0885d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.621+0000 7f9503a45740 1 --2- 172.21.15.2:0/1644300975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 0x562e1b0839f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.621+0000 7f9503a45740 1 -- 172.21.15.2:0/1644300975 >> 172.21.15.2:0/1644300975 conn(0x562e1acca300 msgr2=0x562e1b0832e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.621+0000 7f9503a45740 1 -- 172.21.15.2:0/1644300975 shutdown_connections 2024-05-14T09:55:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.621+0000 7f9503a45740 1 -- 172.21.15.2:0/1644300975 wait complete. 2024-05-14T09:55:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.621+0000 7f9503a45740 1 -- start start 2024-05-14T09:55:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.622+0000 7f9503a45740 1 Processor -- start 2024-05-14T09:55:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.622+0000 7f9503a45740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 0x562e1b072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.622+0000 7f9503a45740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acb54d0 0x562e1b073390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.622+0000 7f9503a45740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acd61b0 0x562e1b079d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f9503a45740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f94f81a91a0 con 0x562e1acad2f0 2024-05-14T09:55:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f94effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acb54d0 0x562e1b073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f9503a45740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94f81aa610 con 0x562e1acb54d0 2024-05-14T09:55:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f9503a45740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f94f81a9870 con 0x562e1acd61b0 2024-05-14T09:55:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f94effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acb54d0 0x562e1b073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:44560/0 (socket says 172.21.15.2:44560) 2024-05-14T09:55:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f94effff640 1 -- 172.21.15.2:0/906254789 learned_addr learned my addr 172.21.15.2:0/906254789 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f94f659c640 1 --2- 172.21.15.2:0/906254789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 0x562e1b072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f9503a45740 10 client.0 _ll_register_callbacks cb 0x7f94e8004180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:55:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f94f6d9d640 1 --2- 172.21.15.2:0/906254789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acd61b0 0x562e1b079d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f94effff640 1 -- 172.21.15.2:0/906254789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acd61b0 msgr2=0x562e1b079d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f94effff640 1 --2- 172.21.15.2:0/906254789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acd61b0 0x562e1b079d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f94effff640 1 -- 172.21.15.2:0/906254789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 msgr2=0x562e1b072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f94effff640 1 --2- 172.21.15.2:0/906254789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 0x562e1b072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f94effff640 1 -- 172.21.15.2:0/906254789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e1b07a2c0 con 0x562e1acb54d0 2024-05-14T09:55:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f94effff640 1 --2- 172.21.15.2:0/906254789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acb54d0 0x562e1b073390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f94e00aa680 tx=0x7f94e00b3220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2f732f11a89c79c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.623+0000 7f94ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:55:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.624+0000 7f94ddffb640 1 -- 172.21.15.2:0/906254789 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e00ab080 con 0x562e1acb54d0 2024-05-14T09:55:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.624+0000 7f94ddffb640 1 -- 172.21.15.2:0/906254789 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9282+0+0 (secure 0 0 0) 0x7f94e00a6f70 con 0x562e1acb54d0 2024-05-14T09:55:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.624+0000 7f9503a45740 1 -- 172.21.15.2:0/906254789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94e00aa980 con 0x562e1acb54d0 2024-05-14T09:55:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.624+0000 7f94ddffb640 1 -- 172.21.15.2:0/906254789 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f94e00a6c10 con 0x562e1acb54d0 2024-05-14T09:55:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.624+0000 7f9503a45740 10 client.26996 fetch_fsmap learned FSMap version 65 2024-05-14T09:55:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.624+0000 7f9503a45740 1 -- 172.21.15.2:0/906254789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x562e1b07b5a0 con 0x562e1acb54d0 2024-05-14T09:55:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.624+0000 7f94ddffb640 1 -- 172.21.15.2:0/906254789 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94e00aa980 con 0x562e1acb54d0 2024-05-14T09:55:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.624+0000 7f94ddffb640 1 -- 172.21.15.2:0/906254789 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 65) ==== 8346+0+0 (secure 0 0 0) 0x7f94e00b3350 con 0x562e1acb54d0 2024-05-14T09:55:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.624+0000 7f9503a45740 10 client.26996 fetch_fsmap finished waiting for FSMap version 65 2024-05-14T09:55:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.624+0000 7f9503a45740 10 client.26996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:55:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.624+0000 7f9503a45740 20 client.26996 populate_metadata read hostname 'smithi002' 2024-05-14T09:55:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.624+0000 7f9503a45740 1 --2- 172.21.15.2:0/906254789 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562e1ac6a1c0 0x562e1b0f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.625+0000 7f9503a45740 4 client.26996 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:55:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.625+0000 7f9503a45740 1 -- 172.21.15.2:0/906254789 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e1afe4050 con 0x562e1ac6a1c0 2024-05-14T09:55:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.625+0000 7f94f659c640 1 --2- 172.21.15.2:0/906254789 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562e1ac6a1c0 0x562e1b0f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.625+0000 7f94ddffb640 10 client.26996 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:41.625+0000 7f94f659c640 1 --2- 172.21.15.2:0/906254789 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562e1ac6a1c0 0x562e1b0f9d30 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe5866c6425c280 server_cookie=c3934bde6869e145 in_seq=0 out_seq=0 2024-05-14T09:55:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.440+0000 7f94ddffb640 1 -- 172.21.15.2:0/906254789 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562e1afe4050 con 0x562e1ac6a1c0 2024-05-14T09:55:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.441+0000 7f94ddffb640 10 client.26996 handle_command_reply: tid=0 2024-05-14T09:55:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.441+0000 7f9503a45740 1 client.26996 shutdown 2024-05-14T09:55:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.441+0000 7f9503a45740 1 -- 172.21.15.2:0/906254789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acb54d0 msgr2=0x562e1b073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.441+0000 7f9503a45740 1 --2- 172.21.15.2:0/906254789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acb54d0 0x562e1b073390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f94e00aa680 tx=0x7f94e00b3220 comp rx=0 tx=0).stop 2024-05-14T09:55:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.441+0000 7f9503a45740 1 -- 172.21.15.2:0/906254789 shutdown_connections 2024-05-14T09:55:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.441+0000 7f9503a45740 1 --2- 172.21.15.2:0/906254789 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x562e1ac6a1c0 0x562e1b0f9d30 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.441+0000 7f94ddffb640 0 client.26996 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.442+0000 7f9503a45740 1 --2- 172.21.15.2:0/906254789 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562e1acd61b0 0x562e1b079d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.442+0000 7f9503a45740 1 --2- 172.21.15.2:0/906254789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e1acb54d0 0x562e1b073390 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.442+0000 7f9503a45740 1 --2- 172.21.15.2:0/906254789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562e1acad2f0 0x562e1b072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.442+0000 7f9503a45740 1 -- 172.21.15.2:0/906254789 >> 172.21.15.2:0/906254789 conn(0x562e1acca300 msgr2=0x562e1b0885d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.442+0000 7f9503a45740 1 -- 172.21.15.2:0/906254789 shutdown_connections 2024-05-14T09:55:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.442+0000 7f9503a45740 1 -- 172.21.15.2:0/906254789 wait complete. 2024-05-14T09:55:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.442+0000 7f9503a45740 20 client.26996 trim_cache size 0 max 16384 2024-05-14T09:55:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:55:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:55:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:55:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:55:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.443+0000 7f94fcb8f640 1 -- 172.21.15.2:0/2611384002 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f94e40b2920 msgr2=0x7f94e40b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.443+0000 7f94fcb8f640 1 --2- 172.21.15.2:0/2611384002 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f94e40b2920 0x7f94e40b4e30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f94e80095f0 tx=0x7f94e8005f90 comp rx=0 tx=0).stop 2024-05-14T09:55:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.443+0000 7f94fcb8f640 1 -- 172.21.15.2:0/2611384002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94f8104e90 msgr2=0x7f94f819a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.444+0000 7f94fcb8f640 1 --2- 172.21.15.2:0/2611384002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94f8104e90 0x7f94f819a870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f94e000e9c0 tx=0x7f94e000ee80 comp rx=0 tx=0).stop 2024-05-14T09:55:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.444+0000 7f94fcb8f640 1 -- 172.21.15.2:0/2611384002 shutdown_connections 2024-05-14T09:55:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.444+0000 7f94fcb8f640 1 --2- 172.21.15.2:0/2611384002 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f94e40b2920 0x7f94e40b4e30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.444+0000 7f94fcb8f640 1 --2- 172.21.15.2:0/2611384002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94f8109b90 0x7f94f81a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.444+0000 7f94fcb8f640 1 --2- 172.21.15.2:0/2611384002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f94f8104e90 0x7f94f819a870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.444+0000 7f94fcb8f640 1 --2- 172.21.15.2:0/2611384002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f8103c50 0x7f94f819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.444+0000 7f94fcb8f640 1 -- 172.21.15.2:0/2611384002 >> 172.21.15.2:0/2611384002 conn(0x7f94f80fee60 msgr2=0x7f94f8105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.444+0000 7f94fcb8f640 1 -- 172.21.15.2:0/2611384002 shutdown_connections 2024-05-14T09:55:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:42.445+0000 7f94fcb8f640 1 -- 172.21.15.2:0/2611384002 wait complete. 2024-05-14T09:55:42.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 220054 26.90 MB/sec execute 308 sec latency 358.898 ms 2024-05-14T09:55:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:42 smithi002 ceph-mon[31557]: pgmap v1048: 129 pgs: 129 active+clean; 708 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.0 MiB/s wr, 457 op/s 2024-05-14T09:55:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:42 smithi029 ceph-mon[36249]: pgmap v1048: 129 pgs: 129 active+clean; 708 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.0 MiB/s wr, 457 op/s 2024-05-14T09:55:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:42 smithi026 ceph-mon[36056]: pgmap v1048: 129 pgs: 129 active+clean; 708 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.0 MiB/s wr, 457 op/s 2024-05-14T09:55:42.951 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:55:42 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e[66118]: ignoring --setuser ceph since I am not root 2024-05-14T09:55:42.952 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:55:42 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e[66118]: ignoring --setgroup ceph since I am not root 2024-05-14T09:55:42.952 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:55:42 smithi026 ceph-mds[66122]: -- 172.21.15.26:0/3016735344 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555a1ee7fa40 con 0x555a1eec7180 2024-05-14T09:55:42.952 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 09:55:42 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e[66118]: starting mds.e at 2024-05-14T09:55:43.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 220524 26.89 MB/sec execute 309 sec latency 91.909 ms 2024-05-14T09:55:43.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:43 smithi002 ceph-mon[31557]: daemon mds.e restarted 2024-05-14T09:55:43.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:43 smithi029 ceph-mon[36249]: daemon mds.e restarted 2024-05-14T09:55:43.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:43 smithi026 ceph-mon[36056]: daemon mds.e restarted 2024-05-14T09:55:44.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 220936 26.86 MB/sec execute 310 sec latency 198.178 ms 2024-05-14T09:55:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:44 smithi002 ceph-mon[31557]: pgmap v1049: 129 pgs: 129 active+clean; 711 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.3 MiB/s wr, 505 op/s 2024-05-14T09:55:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:44 smithi002 ceph-mon[31557]: osdmap e93: 12 total, 12 up, 12 in 2024-05-14T09:55:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:44 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.26:6834/786173089,v1:172.21.15.26:6835/786173089] up:boot 2024-05-14T09:55:44.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:44 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T09:55:44.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:44 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T09:55:44.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:44 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:55:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:44 smithi029 ceph-mon[36249]: pgmap v1049: 129 pgs: 129 active+clean; 711 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.3 MiB/s wr, 505 op/s 2024-05-14T09:55:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:44 smithi029 ceph-mon[36249]: osdmap e93: 12 total, 12 up, 12 in 2024-05-14T09:55:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:44 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.26:6834/786173089,v1:172.21.15.26:6835/786173089] up:boot 2024-05-14T09:55:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:44 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T09:55:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:44 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T09:55:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:44 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:55:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:44 smithi026 ceph-mon[36056]: pgmap v1049: 129 pgs: 129 active+clean; 711 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.3 MiB/s wr, 505 op/s 2024-05-14T09:55:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:44 smithi026 ceph-mon[36056]: osdmap e93: 12 total, 12 up, 12 in 2024-05-14T09:55:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:44 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.26:6834/786173089,v1:172.21.15.26:6835/786173089] up:boot 2024-05-14T09:55:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:44 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T09:55:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:44 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T09:55:44.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:44 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T09:55:45.616 INFO:tasks.workunit.client.0.smithi002.stdout: 10 221002 26.79 MB/sec execute 311 sec latency 845.645 ms 2024-05-14T09:55:46.617 INFO:tasks.workunit.client.0.smithi002.stdout: 10 221017 26.71 MB/sec execute 312 sec latency 1845.689 ms 2024-05-14T09:55:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:46 smithi002 ceph-mon[31557]: pgmap v1051: 129 pgs: 129 active+clean; 715 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.2 MiB/s wr, 466 op/s 2024-05-14T09:55:46.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:46 smithi029 ceph-mon[36249]: pgmap v1051: 129 pgs: 129 active+clean; 715 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.2 MiB/s wr, 466 op/s 2024-05-14T09:55:46.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:46 smithi026 ceph-mon[36056]: pgmap v1051: 129 pgs: 129 active+clean; 715 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.2 MiB/s wr, 466 op/s 2024-05-14T09:55:47.617 INFO:tasks.workunit.client.0.smithi002.stdout: 10 221088 26.63 MB/sec execute 313 sec latency 2141.140 ms 2024-05-14T09:55:48.617 INFO:tasks.workunit.client.0.smithi002.stdout: 10 221312 26.58 MB/sec execute 314 sec latency 483.922 ms 2024-05-14T09:55:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:48 smithi002 ceph-mon[31557]: pgmap v1052: 129 pgs: 129 active+clean; 749 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.6 MiB/s wr, 573 op/s 2024-05-14T09:55:48.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:48 smithi029 ceph-mon[36249]: pgmap v1052: 129 pgs: 129 active+clean; 749 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.6 MiB/s wr, 573 op/s 2024-05-14T09:55:48.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:48 smithi026 ceph-mon[36056]: pgmap v1052: 129 pgs: 129 active+clean; 749 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.6 MiB/s wr, 573 op/s 2024-05-14T09:55:49.617 INFO:tasks.workunit.client.0.smithi002.stdout: 10 221555 26.53 MB/sec execute 315 sec latency 201.778 ms 2024-05-14T09:55:50.617 INFO:tasks.workunit.client.0.smithi002.stdout: 10 221764 26.46 MB/sec execute 316 sec latency 267.957 ms 2024-05-14T09:55:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:50 smithi002 ceph-mon[31557]: pgmap v1053: 129 pgs: 129 active+clean; 765 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 610 op/s 2024-05-14T09:55:50.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:50 smithi029 ceph-mon[36249]: pgmap v1053: 129 pgs: 129 active+clean; 765 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 610 op/s 2024-05-14T09:55:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:50 smithi026 ceph-mon[36056]: pgmap v1053: 129 pgs: 129 active+clean; 765 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 610 op/s 2024-05-14T09:55:51.617 INFO:tasks.workunit.client.0.smithi002.stdout: 10 222213 26.44 MB/sec execute 317 sec latency 153.992 ms 2024-05-14T09:55:52.617 INFO:tasks.workunit.client.0.smithi002.stdout: 10 222813 26.47 MB/sec execute 318 sec latency 219.297 ms 2024-05-14T09:55:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:52 smithi002 ceph-mon[31557]: pgmap v1054: 129 pgs: 129 active+clean; 742 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 683 op/s 2024-05-14T09:55:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:52 smithi029 ceph-mon[36249]: pgmap v1054: 129 pgs: 129 active+clean; 742 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 683 op/s 2024-05-14T09:55:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:52 smithi026 ceph-mon[36056]: pgmap v1054: 129 pgs: 129 active+clean; 742 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 683 op/s 2024-05-14T09:55:53.617 INFO:tasks.workunit.client.0.smithi002.stdout: 10 223561 26.56 MB/sec execute 319 sec latency 90.129 ms 2024-05-14T09:55:54.617 INFO:tasks.workunit.client.0.smithi002.stdout: 10 224009 26.55 MB/sec execute 320 sec latency 89.718 ms 2024-05-14T09:55:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:54 smithi002 ceph-mon[31557]: pgmap v1055: 129 pgs: 129 active+clean; 666 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 762 op/s 2024-05-14T09:55:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:54 smithi029 ceph-mon[36249]: pgmap v1055: 129 pgs: 129 active+clean; 666 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 762 op/s 2024-05-14T09:55:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:54 smithi026 ceph-mon[36056]: pgmap v1055: 129 pgs: 129 active+clean; 666 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 762 op/s 2024-05-14T09:55:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.608+0000 7f84c9829640 1 -- 172.21.15.2:0/873887206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84bc0ab760 msgr2=0x7f84bc0abb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.608+0000 7f84c9829640 1 --2- 172.21.15.2:0/873887206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84bc0ab760 0x7f84bc0abb70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009900 tx=0x7f84b802a400 comp rx=0 tx=0).stop 2024-05-14T09:55:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.609+0000 7f84c9829640 1 -- 172.21.15.2:0/873887206 shutdown_connections 2024-05-14T09:55:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.609+0000 7f84c9829640 1 --2- 172.21.15.2:0/873887206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84bc0ab760 0x7f84bc0abb70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.609+0000 7f84c9829640 1 --2- 172.21.15.2:0/873887206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84bc0a6a40 0x7f84bc0a6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.609+0000 7f84c9829640 1 --2- 172.21.15.2:0/873887206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc0a5940 0x7f84bc0a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.609+0000 7f84c9829640 1 -- 172.21.15.2:0/873887206 >> 172.21.15.2:0/873887206 conn(0x7f84bc0a09b0 msgr2=0x7f84bc0a2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.616+0000 7f84c9829640 1 -- 172.21.15.2:0/873887206 shutdown_connections 2024-05-14T09:55:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.616+0000 7f84c9829640 1 -- 172.21.15.2:0/873887206 wait complete. 2024-05-14T09:55:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.617+0000 7f84c9829640 1 Processor -- start 2024-05-14T09:55:55.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 224416 26.52 MB/sec execute 321 sec latency 157.811 ms 2024-05-14T09:55:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.619+0000 7f84c9829640 1 -- start start 2024-05-14T09:55:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.619+0000 7f84c9829640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84bc0a5940 0x7f84bc0b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.619+0000 7f84c9829640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84bc0a6a40 0x7f84bc0b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.619+0000 7f84c9829640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc0ab760 0x7f84bc0b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.619+0000 7f84c9829640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc0b9b70 con 0x7f84bc0a5940 2024-05-14T09:55:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.619+0000 7f84c9829640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc0b99f0 con 0x7f84bc0a6a40 2024-05-14T09:55:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.619+0000 7f84c9829640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc0b9cf0 con 0x7f84bc0ab760 2024-05-14T09:55:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.619+0000 7f84c3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84bc0a6a40 0x7f84bc0b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.619+0000 7f84c3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84bc0a6a40 0x7f84bc0b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:53402/0 (socket says 172.21.15.2:53402) 2024-05-14T09:55:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.619+0000 7f84c3fff640 1 -- 172.21.15.2:0/2353947138 learned_addr learned my addr 172.21.15.2:0/2353947138 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.620+0000 7f84c9028640 1 --2- 172.21.15.2:0/2353947138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc0ab760 0x7f84bc0b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.621+0000 7f84c3fff640 1 -- 172.21.15.2:0/2353947138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc0ab760 msgr2=0x7f84bc0b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.621+0000 7f84c3fff640 1 --2- 172.21.15.2:0/2353947138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc0ab760 0x7f84bc0b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.621+0000 7f84c3fff640 1 -- 172.21.15.2:0/2353947138 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84bc0a5940 msgr2=0x7f84bc0b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.621+0000 7f84c3fff640 1 --2- 172.21.15.2:0/2353947138 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84bc0a5940 0x7f84bc0b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.621+0000 7f84c3fff640 1 -- 172.21.15.2:0/2353947138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84bc0b40e0 con 0x7f84bc0a6a40 2024-05-14T09:55:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.622+0000 7f84c3fff640 1 --2- 172.21.15.2:0/2353947138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84bc0a6a40 0x7f84bc0b2b60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f84b000e9f0 tx=0x7f84b000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.626+0000 7f84c1ffb640 1 -- 172.21.15.2:0/2353947138 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b000cd80 con 0x7f84bc0a6a40 2024-05-14T09:55:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.626+0000 7f84c9829640 1 -- 172.21.15.2:0/2353947138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84bc0b43d0 con 0x7f84bc0a6a40 2024-05-14T09:55:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.626+0000 7f84c9829640 1 -- 172.21.15.2:0/2353947138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84bc0b49a0 con 0x7f84bc0a6a40 2024-05-14T09:55:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.627+0000 7f84c1ffb640 1 -- 172.21.15.2:0/2353947138 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b00108e0 con 0x7f84bc0a6a40 2024-05-14T09:55:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.627+0000 7f84c1ffb640 1 -- 172.21.15.2:0/2353947138 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0018880 con 0x7f84bc0a6a40 2024-05-14T09:55:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.631+0000 7f84d1a15740 1 Processor -- start 2024-05-14T09:55:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.631+0000 7f84d1a15740 1 -- start start 2024-05-14T09:55:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.636+0000 7f84c1ffb640 1 -- 172.21.15.2:0/2353947138 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f84b0018a20 con 0x7f84bc0a6a40 2024-05-14T09:55:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.636+0000 7f84d1a15740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284a21b0 0x556828843c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.636+0000 7f84d1a15740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284792f0 0x5568288441b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.636+0000 7f84d1a15740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284814d0 0x55682884b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.636+0000 7f84d1a15740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5568287a85d0 con 0x5568284792f0 2024-05-14T09:55:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.636+0000 7f84d1a15740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556828761890 con 0x5568284814d0 2024-05-14T09:55:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.636+0000 7f84d1a15740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5568287a4590 con 0x5568284a21b0 2024-05-14T09:55:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.637+0000 7f84c9028640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284a21b0 0x556828843c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.637+0000 7f84c9028640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284a21b0 0x556828843c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:53584/0 (socket says 172.21.15.2:53584) 2024-05-14T09:55:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.637+0000 7f84c9028640 1 -- 172.21.15.2:0/1371345140 learned_addr learned my addr 172.21.15.2:0/1371345140 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.637+0000 7f84c8827640 1 --2- 172.21.15.2:0/1371345140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284792f0 0x5568288441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.637+0000 7f84c1ffb640 1 --2- 172.21.15.2:0/2353947138 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f84b40b2a00 0x7f84b40b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.637+0000 7f84c3fff640 1 --2- 172.21.15.2:0/1371345140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284814d0 0x55682884b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.637+0000 7f84c9028640 1 -- 172.21.15.2:0/1371345140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284814d0 msgr2=0x55682884b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.637+0000 7f84c9028640 1 --2- 172.21.15.2:0/1371345140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284814d0 0x55682884b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.637+0000 7f84c9028640 1 -- 172.21.15.2:0/1371345140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284792f0 msgr2=0x5568288441b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.637+0000 7f84c9028640 1 --2- 172.21.15.2:0/1371345140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284792f0 0x5568288441b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.637+0000 7f84c9028640 1 -- 172.21.15.2:0/1371345140 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568284ba210 con 0x5568284a21b0 2024-05-14T09:55:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.638+0000 7f84c1ffb640 1 -- 172.21.15.2:0/2353947138 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f84b00e3a70 con 0x7f84bc0a6a40 2024-05-14T09:55:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.638+0000 7f84c9028640 1 --2- 172.21.15.2:0/2353947138 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f84b40b2a00 0x7f84b40b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.639+0000 7f84c9028640 1 --2- 172.21.15.2:0/2353947138 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f84b40b2a00 0x7f84b40b4f10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f84b8040000 tx=0x7f84b8041000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.639+0000 7f84c9028640 1 --2- 172.21.15.2:0/1371345140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284a21b0 0x556828843c30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f84b8005710 tx=0x7f84b8004470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33106cad2a5ca8af server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.639+0000 7f84aeffd640 1 -- 172.21.15.2:0/1371345140 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8035040 con 0x5568284a21b0 2024-05-14T09:55:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.639+0000 7f84aeffd640 1 -- 172.21.15.2:0/1371345140 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b8031c50 con 0x5568284a21b0 2024-05-14T09:55:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.639+0000 7f84aeffd640 1 -- 172.21.15.2:0/1371345140 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8046b70 con 0x5568284a21b0 2024-05-14T09:55:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.641+0000 7f84d1a15740 1 -- 172.21.15.2:0/1371345140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284a21b0 msgr2=0x556828843c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.641+0000 7f84d1a15740 1 --2- 172.21.15.2:0/1371345140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284a21b0 0x556828843c30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f84b8005710 tx=0x7f84b8004470 comp rx=0 tx=0).stop 2024-05-14T09:55:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.641+0000 7f84d1a15740 1 -- 172.21.15.2:0/1371345140 shutdown_connections 2024-05-14T09:55:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.641+0000 7f84d1a15740 1 --2- 172.21.15.2:0/1371345140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284814d0 0x55682884b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.641+0000 7f84d1a15740 1 --2- 172.21.15.2:0/1371345140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284792f0 0x5568288441b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.641+0000 7f84d1a15740 1 --2- 172.21.15.2:0/1371345140 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284a21b0 0x556828843c30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.641+0000 7f84d1a15740 1 -- 172.21.15.2:0/1371345140 >> 172.21.15.2:0/1371345140 conn(0x556828496300 msgr2=0x55682883f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.641+0000 7f84d1a15740 1 -- 172.21.15.2:0/1371345140 shutdown_connections 2024-05-14T09:55:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.641+0000 7f84d1a15740 1 -- 172.21.15.2:0/1371345140 wait complete. 2024-05-14T09:55:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.641+0000 7f84d1a15740 1 -- start start 2024-05-14T09:55:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.642+0000 7f84d1a15740 1 Processor -- start 2024-05-14T09:55:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.642+0000 7f84d1a15740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284792f0 0x55682884d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.642+0000 7f84d1a15740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284814d0 0x55682884dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.642+0000 7f84d1a15740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284a21b0 0x556828854c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.642+0000 7f84d1a15740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55682884baa0 con 0x5568284a21b0 2024-05-14T09:55:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.642+0000 7f84d1a15740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556828765370 con 0x5568284792f0 2024-05-14T09:55:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.642+0000 7f84d1a15740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55682884bc20 con 0x5568284814d0 2024-05-14T09:55:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.642+0000 7f84d1a15740 10 client.0 _ll_register_callbacks cb 0x5568284b9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:55:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.642+0000 7f84c3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284814d0 0x55682884dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.643+0000 7f84c8827640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284792f0 0x55682884d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.643+0000 7f84c3fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284814d0 0x55682884dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:53588/0 (socket says 172.21.15.2:53588) 2024-05-14T09:55:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.643+0000 7f84c3fff640 1 -- 172.21.15.2:0/179265454 learned_addr learned my addr 172.21.15.2:0/179265454 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.643+0000 7f84c9028640 1 --2- 172.21.15.2:0/179265454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284a21b0 0x556828854c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.643+0000 7f84c3fff640 1 -- 172.21.15.2:0/179265454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284792f0 msgr2=0x55682884d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.643+0000 7f84c3fff640 1 --2- 172.21.15.2:0/179265454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284792f0 0x55682884d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.643+0000 7f84c3fff640 1 -- 172.21.15.2:0/179265454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284a21b0 msgr2=0x556828854c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.643+0000 7f84c3fff640 1 --2- 172.21.15.2:0/179265454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284a21b0 0x556828854c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.643+0000 7f84c3fff640 1 -- 172.21.15.2:0/179265454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556828586620 con 0x5568284814d0 2024-05-14T09:55:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.643+0000 7f84c3fff640 1 --2- 172.21.15.2:0/179265454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284814d0 0x55682884dd90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f84b00aaba0 tx=0x7f84b0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5721679747e06a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.643+0000 7f84a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:55:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.644+0000 7f84a3fff640 1 -- 172.21.15.2:0/179265454 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b00ab310 con 0x5568284814d0 2024-05-14T09:55:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.644+0000 7f84d1a15740 1 -- 172.21.15.2:0/179265454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b8031c50 con 0x5568284814d0 2024-05-14T09:55:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.645+0000 7f84a3fff640 1 -- 172.21.15.2:0/179265454 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f84b00a6ea0 con 0x5568284814d0 2024-05-14T09:55:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.645+0000 7f84a3fff640 1 -- 172.21.15.2:0/179265454 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f84b00aac10 con 0x5568284814d0 2024-05-14T09:55:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.645+0000 7f84d1a15740 10 client.27004 fetch_fsmap learned FSMap version 67 2024-05-14T09:55:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.645+0000 7f84d1a15740 1 -- 172.21.15.2:0/179265454 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x556828857ca0 con 0x5568284814d0 2024-05-14T09:55:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.646+0000 7f84a3fff640 1 -- 172.21.15.2:0/179265454 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b8031c50 con 0x5568284814d0 2024-05-14T09:55:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.646+0000 7f84a3fff640 1 -- 172.21.15.2:0/179265454 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f84b00af220 con 0x5568284814d0 2024-05-14T09:55:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.646+0000 7f84d1a15740 10 client.27004 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:55:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.646+0000 7f84d1a15740 10 client.27004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:55:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.646+0000 7f84d1a15740 20 client.27004 populate_metadata read hostname 'smithi002' 2024-05-14T09:55:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.646+0000 7f84d1a15740 1 --2- 172.21.15.2:0/179265454 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556828460d80 0x55682885b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.646+0000 7f84d1a15740 4 client.27004 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:55:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.646+0000 7f84d1a15740 1 -- 172.21.15.2:0/179265454 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568287b61e0 con 0x556828460d80 2024-05-14T09:55:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.647+0000 7f84c8827640 1 --2- 172.21.15.2:0/179265454 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556828460d80 0x55682885b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.648+0000 7f84a3fff640 10 client.27004 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.648+0000 7f84c8827640 1 --2- 172.21.15.2:0/179265454 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556828460d80 0x55682885b9a0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb85f24cdfb8d283 server_cookie=d6b1055724ea7cdb in_seq=0 out_seq=0 2024-05-14T09:55:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.649+0000 7f84a3fff640 1 -- 172.21.15.2:0/179265454 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5568287b61e0 con 0x556828460d80 2024-05-14T09:55:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.649+0000 7f84a3fff640 10 client.27004 handle_command_reply: tid=0 2024-05-14T09:55:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.649+0000 7f84d1a15740 1 client.27004 shutdown 2024-05-14T09:55:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.650+0000 7f84d1a15740 1 -- 172.21.15.2:0/179265454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284814d0 msgr2=0x55682884dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.650+0000 7f84d1a15740 1 --2- 172.21.15.2:0/179265454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284814d0 0x55682884dd90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f84b00aaba0 tx=0x7f84b0014000 comp rx=0 tx=0).stop 2024-05-14T09:55:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.651+0000 7f84d1a15740 1 -- 172.21.15.2:0/179265454 shutdown_connections 2024-05-14T09:55:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.651+0000 7f84d1a15740 1 --2- 172.21.15.2:0/179265454 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556828460d80 0x55682885b9a0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.651+0000 7f84d1a15740 1 --2- 172.21.15.2:0/179265454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284a21b0 0x556828854c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.651+0000 7f84d1a15740 1 --2- 172.21.15.2:0/179265454 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284814d0 0x55682884dd90 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f84b00aaba0 tx=0x7f84b0014000 comp rx=0 tx=0).stop 2024-05-14T09:55:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.651+0000 7f84d1a15740 1 --2- 172.21.15.2:0/179265454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284792f0 0x55682884d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.651+0000 7f84d1a15740 1 -- 172.21.15.2:0/179265454 >> 172.21.15.2:0/179265454 conn(0x556828496300 msgr2=0x5568288424d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.651+0000 7f84a3fff640 0 client.27004 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.651+0000 7f84d1a15740 1 -- 172.21.15.2:0/179265454 shutdown_connections 2024-05-14T09:55:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.652+0000 7f84d1a15740 1 -- 172.21.15.2:0/179265454 wait complete. 2024-05-14T09:55:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.652+0000 7f84d1a15740 20 client.27004 trim_cache size 0 max 16384 2024-05-14T09:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.679+0000 7f84d1a15740 1 Processor -- start 2024-05-14T09:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.679+0000 7f84d1a15740 1 -- start start 2024-05-14T09:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84d1a15740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284792f0 0x5568288423c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84d1a15740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284a21b0 0x556828842900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84d1a15740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284361c0 0x556828854c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84d1a15740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f84a8012690 con 0x5568284792f0 2024-05-14T09:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84d1a15740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f84a800ff60 con 0x5568284a21b0 2024-05-14T09:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84d1a15740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f84a800ddb0 con 0x5568284361c0 2024-05-14T09:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84c3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284a21b0 0x556828842900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84c3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284a21b0 0x556828842900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:53432/0 (socket says 172.21.15.2:53432) 2024-05-14T09:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84c3fff640 1 -- 172.21.15.2:0/2947429021 learned_addr learned my addr 172.21.15.2:0/2947429021 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84c8827640 1 --2- 172.21.15.2:0/2947429021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284792f0 0x5568288423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84c9028640 1 --2- 172.21.15.2:0/2947429021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284361c0 0x556828854c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84c3fff640 1 -- 172.21.15.2:0/2947429021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284361c0 msgr2=0x556828854c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84c3fff640 1 --2- 172.21.15.2:0/2947429021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284361c0 0x556828854c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84c3fff640 1 -- 172.21.15.2:0/2947429021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284792f0 msgr2=0x5568288423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84c3fff640 1 --2- 172.21.15.2:0/2947429021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284792f0 0x5568288423c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.680+0000 7f84c3fff640 1 -- 172.21.15.2:0/2947429021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568288396e0 con 0x5568284a21b0 2024-05-14T09:55:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.681+0000 7f84c3fff640 1 --2- 172.21.15.2:0/2947429021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284a21b0 0x556828842900 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c880 tx=0x7f84b0010ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b07bd38e610fd4e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.681+0000 7f84ae7fc640 1 -- 172.21.15.2:0/2947429021 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b00aa560 con 0x5568284a21b0 2024-05-14T09:55:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.681+0000 7f84ae7fc640 1 -- 172.21.15.2:0/2947429021 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b00ab310 con 0x5568284a21b0 2024-05-14T09:55:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.681+0000 7f84ae7fc640 1 -- 172.21.15.2:0/2947429021 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b00a6ad0 con 0x5568284a21b0 2024-05-14T09:55:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.682+0000 7f84d1a15740 1 -- 172.21.15.2:0/2947429021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284a21b0 msgr2=0x556828842900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.682+0000 7f84d1a15740 1 --2- 172.21.15.2:0/2947429021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284a21b0 0x556828842900 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c880 tx=0x7f84b0010ca0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.682+0000 7f84d1a15740 1 -- 172.21.15.2:0/2947429021 shutdown_connections 2024-05-14T09:55:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.682+0000 7f84d1a15740 1 --2- 172.21.15.2:0/2947429021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284361c0 0x556828854c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.682+0000 7f84d1a15740 1 --2- 172.21.15.2:0/2947429021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284a21b0 0x556828842900 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.682+0000 7f84d1a15740 1 --2- 172.21.15.2:0/2947429021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284792f0 0x5568288423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.682+0000 7f84d1a15740 1 -- 172.21.15.2:0/2947429021 >> 172.21.15.2:0/2947429021 conn(0x556828496300 msgr2=0x556828847cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.682+0000 7f84d1a15740 1 -- 172.21.15.2:0/2947429021 shutdown_connections 2024-05-14T09:55:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.682+0000 7f84d1a15740 1 -- 172.21.15.2:0/2947429021 wait complete. 2024-05-14T09:55:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.682+0000 7f84d1a15740 1 -- start start 2024-05-14T09:55:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.682+0000 7f84d1a15740 1 Processor -- start 2024-05-14T09:55:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84d1a15740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284361c0 0x556828855410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84d1a15740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284792f0 0x556828851f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84d1a15740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284a21b0 0x5568288525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84d1a15740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f84a800e420 con 0x5568284a21b0 2024-05-14T09:55:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84d1a15740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f84a800f890 con 0x5568284792f0 2024-05-14T09:55:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84d1a15740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f84a800eaf0 con 0x5568284361c0 2024-05-14T09:55:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84d1a15740 10 client.0 _ll_register_callbacks cb 0x7f84b000c7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:55:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84c8827640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284361c0 0x556828855410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84c8827640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284361c0 0x556828855410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:53602/0 (socket says 172.21.15.2:53602) 2024-05-14T09:55:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84c3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284792f0 0x556828851f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84c8827640 1 -- 172.21.15.2:0/1558249619 learned_addr learned my addr 172.21.15.2:0/1558249619 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:55:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84c9028640 1 --2- 172.21.15.2:0/1558249619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284a21b0 0x5568288525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84c3fff640 1 -- 172.21.15.2:0/1558249619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284361c0 msgr2=0x556828855410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84c3fff640 1 --2- 172.21.15.2:0/1558249619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284361c0 0x556828855410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84c3fff640 1 -- 172.21.15.2:0/1558249619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284a21b0 msgr2=0x5568288525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:55:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84c3fff640 1 --2- 172.21.15.2:0/1558249619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284a21b0 0x5568288525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.683+0000 7f84c3fff640 1 -- 172.21.15.2:0/1558249619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568288c1d50 con 0x5568284792f0 2024-05-14T09:55:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.684+0000 7f84c8827640 1 --2- 172.21.15.2:0/1558249619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284361c0 0x556828855410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:55:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.684+0000 7f84c9028640 1 --2- 172.21.15.2:0/1558249619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284a21b0 0x5568288525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:55:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.684+0000 7f84c3fff640 1 --2- 172.21.15.2:0/1558249619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284792f0 0x556828851f90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c880 tx=0x7f84b00aa2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4fb95298e1127ec server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:55:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.684+0000 7f84a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:55:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.684+0000 7f84a3fff640 1 -- 172.21.15.2:0/1558249619 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b00c9020 con 0x5568284792f0 2024-05-14T09:55:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.684+0000 7f84d1a15740 1 -- 172.21.15.2:0/1558249619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b00ab310 con 0x5568284792f0 2024-05-14T09:55:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.684+0000 7f84a3fff640 1 -- 172.21.15.2:0/1558249619 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f84b00a7200 con 0x5568284792f0 2024-05-14T09:55:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.684+0000 7f84a3fff640 1 -- 172.21.15.2:0/1558249619 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f84b00a6ad0 con 0x5568284792f0 2024-05-14T09:55:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.684+0000 7f84d1a15740 10 client.27017 fetch_fsmap learned FSMap version 67 2024-05-14T09:55:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.684+0000 7f84d1a15740 1 -- 172.21.15.2:0/1558249619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x5568288c1d50 con 0x5568284792f0 2024-05-14T09:55:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.685+0000 7f84a3fff640 1 -- 172.21.15.2:0/1558249619 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b00ab310 con 0x5568284792f0 2024-05-14T09:55:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.685+0000 7f84a3fff640 1 -- 172.21.15.2:0/1558249619 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f84b00a9af0 con 0x5568284792f0 2024-05-14T09:55:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.685+0000 7f84d1a15740 10 client.27017 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:55:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.685+0000 7f84d1a15740 10 client.27017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:55:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.685+0000 7f84d1a15740 20 client.27017 populate_metadata read hostname 'smithi002' 2024-05-14T09:55:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.685+0000 7f84d1a15740 1 --2- 172.21.15.2:0/1558249619 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5568288c35d0 0x5568288c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:55:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.685+0000 7f84d1a15740 4 client.27017 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:55:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.685+0000 7f84d1a15740 1 -- 172.21.15.2:0/1558249619 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568287b0050 con 0x5568288c35d0 2024-05-14T09:55:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.685+0000 7f84c8827640 1 --2- 172.21.15.2:0/1558249619 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5568288c35d0 0x5568288c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:55:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.686+0000 7f84a3fff640 10 client.27017 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:55.686+0000 7f84c8827640 1 --2- 172.21.15.2:0/1558249619 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5568288c35d0 0x5568288c59f0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3513cbdff90d487e server_cookie=62a431fe890d5c7e in_seq=0 out_seq=0 2024-05-14T09:55:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.187+0000 7f84a3fff640 1 -- 172.21.15.2:0/1558249619 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568287b0050 con 0x5568288c35d0 2024-05-14T09:55:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.187+0000 7f84a3fff640 10 client.27017 handle_command_reply: tid=0 2024-05-14T09:55:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.187+0000 7f84d1a15740 1 client.27017 shutdown 2024-05-14T09:55:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.188+0000 7f84d1a15740 1 -- 172.21.15.2:0/1558249619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284792f0 msgr2=0x556828851f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.188+0000 7f84d1a15740 1 --2- 172.21.15.2:0/1558249619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284792f0 0x556828851f90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c880 tx=0x7f84b00aa2d0 comp rx=0 tx=0).stop 2024-05-14T09:55:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.188+0000 7f84d1a15740 1 -- 172.21.15.2:0/1558249619 shutdown_connections 2024-05-14T09:55:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.188+0000 7f84d1a15740 1 --2- 172.21.15.2:0/1558249619 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5568288c35d0 0x5568288c59f0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.188+0000 7f84a3fff640 0 client.27017 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:55:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.188+0000 7f84d1a15740 1 --2- 172.21.15.2:0/1558249619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568284a21b0 0x5568288525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.188+0000 7f84d1a15740 1 --2- 172.21.15.2:0/1558249619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568284792f0 0x556828851f90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.188+0000 7f84d1a15740 1 --2- 172.21.15.2:0/1558249619 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5568284361c0 0x556828855410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.188+0000 7f84d1a15740 1 -- 172.21.15.2:0/1558249619 >> 172.21.15.2:0/1558249619 conn(0x556828496300 msgr2=0x5568288423c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.189+0000 7f84d1a15740 1 -- 172.21.15.2:0/1558249619 shutdown_connections 2024-05-14T09:55:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.189+0000 7f84d1a15740 1 -- 172.21.15.2:0/1558249619 wait complete. 2024-05-14T09:55:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.189+0000 7f84d1a15740 20 client.27017 trim_cache size 0 max 16384 2024-05-14T09:55:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:55:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:55:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:55:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:55:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.190+0000 7f84c9829640 1 -- 172.21.15.2:0/2353947138 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f84b40b2a00 msgr2=0x7f84b40b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.190+0000 7f84c9829640 1 --2- 172.21.15.2:0/2353947138 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f84b40b2a00 0x7f84b40b4f10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f84b8040000 tx=0x7f84b8041000 comp rx=0 tx=0).stop 2024-05-14T09:55:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.190+0000 7f84c9829640 1 -- 172.21.15.2:0/2353947138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84bc0a6a40 msgr2=0x7f84bc0b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:55:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.190+0000 7f84c9829640 1 --2- 172.21.15.2:0/2353947138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84bc0a6a40 0x7f84bc0b2b60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f84b000e9f0 tx=0x7f84b000eeb0 comp rx=0 tx=0).stop 2024-05-14T09:55:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.190+0000 7f84c9829640 1 -- 172.21.15.2:0/2353947138 shutdown_connections 2024-05-14T09:55:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.190+0000 7f84c9829640 1 --2- 172.21.15.2:0/2353947138 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f84b40b2a00 0x7f84b40b4f10 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.190+0000 7f84c9829640 1 --2- 172.21.15.2:0/2353947138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f84bc0ab760 0x7f84bc0b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.190+0000 7f84c9829640 1 --2- 172.21.15.2:0/2353947138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84bc0a6a40 0x7f84bc0b2b60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.190+0000 7f84c9829640 1 --2- 172.21.15.2:0/2353947138 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84bc0a5940 0x7f84bc0b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:55:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.190+0000 7f84c9829640 1 -- 172.21.15.2:0/2353947138 >> 172.21.15.2:0/2353947138 conn(0x7f84bc0a09b0 msgr2=0x7f84bc0a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:55:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.190+0000 7f84c9829640 1 -- 172.21.15.2:0/2353947138 shutdown_connections 2024-05-14T09:55:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:55:56.190+0000 7f84c9829640 1 -- 172.21.15.2:0/2353947138 wait complete. 2024-05-14T09:55:56.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 224781 26.48 MB/sec execute 322 sec latency 298.733 ms 2024-05-14T09:55:56.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:56 smithi002 ceph-mon[31557]: pgmap v1056: 129 pgs: 129 active+clean; 660 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 12 MiB/s wr, 669 op/s 2024-05-14T09:55:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:56 smithi029 ceph-mon[36249]: pgmap v1056: 129 pgs: 129 active+clean; 660 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 12 MiB/s wr, 669 op/s 2024-05-14T09:55:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:56 smithi026 ceph-mon[36056]: pgmap v1056: 129 pgs: 129 active+clean; 660 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 12 MiB/s wr, 669 op/s 2024-05-14T09:55:57.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 225336 26.50 MB/sec execute 323 sec latency 327.807 ms 2024-05-14T09:55:58.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 226108 26.55 MB/sec execute 324 sec latency 88.339 ms 2024-05-14T09:55:58.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:55:58 smithi002 ceph-mon[31557]: pgmap v1057: 129 pgs: 129 active+clean; 662 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 12 MiB/s wr, 744 op/s 2024-05-14T09:55:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:55:58 smithi029 ceph-mon[36249]: pgmap v1057: 129 pgs: 129 active+clean; 662 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 12 MiB/s wr, 744 op/s 2024-05-14T09:55:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:55:58 smithi026 ceph-mon[36056]: pgmap v1057: 129 pgs: 129 active+clean; 662 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 12 MiB/s wr, 744 op/s 2024-05-14T09:55:59.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 226706 26.59 MB/sec execute 325 sec latency 88.281 ms 2024-05-14T09:56:00.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 227107 26.55 MB/sec execute 326 sec latency 157.902 ms 2024-05-14T09:56:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:00 smithi002 ceph-mon[31557]: pgmap v1058: 129 pgs: 129 active+clean; 662 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.1 MiB/s wr, 645 op/s 2024-05-14T09:56:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:00 smithi029 ceph-mon[36249]: pgmap v1058: 129 pgs: 129 active+clean; 662 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.1 MiB/s wr, 645 op/s 2024-05-14T09:56:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:00 smithi026 ceph-mon[36056]: pgmap v1058: 129 pgs: 129 active+clean; 662 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.1 MiB/s wr, 645 op/s 2024-05-14T09:56:01.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 227429 26.53 MB/sec execute 327 sec latency 205.279 ms 2024-05-14T09:56:02.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 227861 26.53 MB/sec execute 328 sec latency 250.257 ms 2024-05-14T09:56:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:02 smithi002 ceph-mon[31557]: pgmap v1059: 129 pgs: 129 active+clean; 665 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 5.6 MiB/s wr, 611 op/s 2024-05-14T09:56:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:02 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:56:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:02 smithi029 ceph-mon[36249]: pgmap v1059: 129 pgs: 129 active+clean; 665 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 5.6 MiB/s wr, 611 op/s 2024-05-14T09:56:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:02 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:56:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:02 smithi026 ceph-mon[36056]: pgmap v1059: 129 pgs: 129 active+clean; 665 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 5.6 MiB/s wr, 611 op/s 2024-05-14T09:56:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:02 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:56:03.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 228298 26.51 MB/sec execute 329 sec latency 116.922 ms 2024-05-14T09:56:04.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:04 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:04 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:04 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:04 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:04 smithi002 ceph-mon[31557]: pgmap v1060: 129 pgs: 129 active+clean; 670 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.1 MiB/s wr, 580 op/s 2024-05-14T09:56:04.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:04 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:04 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:04 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:04 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:04 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:04 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:04 smithi029 ceph-mon[36249]: pgmap v1060: 129 pgs: 129 active+clean; 670 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.1 MiB/s wr, 580 op/s 2024-05-14T09:56:04.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:04 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:04 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:04 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:04 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:04 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:04 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:04 smithi026 ceph-mon[36056]: pgmap v1060: 129 pgs: 129 active+clean; 670 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.1 MiB/s wr, 580 op/s 2024-05-14T09:56:04.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:04 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:04 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:04.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 228606 26.47 MB/sec execute 330 sec latency 190.723 ms 2024-05-14T09:56:05.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:56:05.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:56:05.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:05.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:56:05.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:56:05.437 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:05.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:56:05.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:56:05.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:56:05.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 229105 26.48 MB/sec execute 331 sec latency 89.277 ms 2024-05-14T09:56:06.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:06 smithi002 ceph-mon[31557]: pgmap v1061: 129 pgs: 129 active+clean; 674 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.7 MiB/s wr, 414 op/s 2024-05-14T09:56:06.436 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:06 smithi029 ceph-mon[36249]: pgmap v1061: 129 pgs: 129 active+clean; 674 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.7 MiB/s wr, 414 op/s 2024-05-14T09:56:06.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:06 smithi026 ceph-mon[36056]: pgmap v1061: 129 pgs: 129 active+clean; 674 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.7 MiB/s wr, 414 op/s 2024-05-14T09:56:06.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 229500 26.46 MB/sec execute 332 sec latency 163.331 ms 2024-05-14T09:56:07.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 229904 26.44 MB/sec execute 333 sec latency 133.067 ms 2024-05-14T09:56:08.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 230239 26.42 MB/sec execute 334 sec latency 315.758 ms 2024-05-14T09:56:08.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:08 smithi002 ceph-mon[31557]: pgmap v1062: 129 pgs: 129 active+clean; 680 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.2 MiB/s wr, 448 op/s 2024-05-14T09:56:08.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:08 smithi029 ceph-mon[36249]: pgmap v1062: 129 pgs: 129 active+clean; 680 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.2 MiB/s wr, 448 op/s 2024-05-14T09:56:08.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:08 smithi026 ceph-mon[36056]: pgmap v1062: 129 pgs: 129 active+clean; 680 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.2 MiB/s wr, 448 op/s 2024-05-14T09:56:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.354+0000 7fd42c3eb640 1 -- 172.21.15.2:0/563323980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd424103da0 msgr2=0x7fd4241041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.354+0000 7fd42c3eb640 1 --2- 172.21.15.2:0/563323980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd424103da0 0x7fd4241041b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd414009900 tx=0x7fd41402a410 comp rx=0 tx=0).stop 2024-05-14T09:56:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.354+0000 7fd42c3eb640 1 -- 172.21.15.2:0/563323980 shutdown_connections 2024-05-14T09:56:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.354+0000 7fd42c3eb640 1 --2- 172.21.15.2:0/563323980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd424108aa0 0x7fd42410aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.354+0000 7fd42c3eb640 1 --2- 172.21.15.2:0/563323980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd424103da0 0x7fd4241041b0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.354+0000 7fd42c3eb640 1 --2- 172.21.15.2:0/563323980 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd424102b60 0x7fd424103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.354+0000 7fd42c3eb640 1 -- 172.21.15.2:0/563323980 >> 172.21.15.2:0/563323980 conn(0x7fd4240fdd50 msgr2=0x7fd424100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.354+0000 7fd42c3eb640 1 -- 172.21.15.2:0/563323980 shutdown_connections 2024-05-14T09:56:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.354+0000 7fd42c3eb640 1 -- 172.21.15.2:0/563323980 wait complete. 2024-05-14T09:56:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42c3eb640 1 Processor -- start 2024-05-14T09:56:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42c3eb640 1 -- start start 2024-05-14T09:56:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42c3eb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd424102b60 0x7fd4241991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42c3eb640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd424103da0 0x7fd424199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42c3eb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd424108aa0 0x7fd4241a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42c3eb640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd42410e660 con 0x7fd424102b60 2024-05-14T09:56:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42c3eb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd42410e4e0 con 0x7fd424108aa0 2024-05-14T09:56:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42c3eb640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd42410e7e0 con 0x7fd424103da0 2024-05-14T09:56:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42a187640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd424102b60 0x7fd4241991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42a187640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd424102b60 0x7fd4241991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55978/0 (socket says 172.21.15.2:55978) 2024-05-14T09:56:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd429986640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd424103da0 0x7fd424199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42a187640 1 -- 172.21.15.2:0/1781582391 learned_addr learned my addr 172.21.15.2:0/1781582391 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42a988640 1 --2- 172.21.15.2:0/1781582391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd424108aa0 0x7fd4241a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42a187640 1 -- 172.21.15.2:0/1781582391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd424103da0 msgr2=0x7fd424199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42a187640 1 --2- 172.21.15.2:0/1781582391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd424103da0 0x7fd424199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42a187640 1 -- 172.21.15.2:0/1781582391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd424108aa0 msgr2=0x7fd4241a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42a187640 1 --2- 172.21.15.2:0/1781582391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd424108aa0 0x7fd4241a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd42a187640 1 -- 172.21.15.2:0/1781582391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4241a0d10 con 0x7fd424102b60 2024-05-14T09:56:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.355+0000 7fd429986640 1 --2- 172.21.15.2:0/1781582391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd424103da0 0x7fd424199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:56:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.356+0000 7fd42a187640 1 --2- 172.21.15.2:0/1781582391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd424102b60 0x7fd4241991d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd42000e9f0 tx=0x7fd42000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.356+0000 7fd41b7fe640 1 -- 172.21.15.2:0/1781582391 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42000ccc0 con 0x7fd424102b60 2024-05-14T09:56:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.356+0000 7fd41b7fe640 1 -- 172.21.15.2:0/1781582391 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4200035f0 con 0x7fd424102b60 2024-05-14T09:56:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.356+0000 7fd42c3eb640 1 -- 172.21.15.2:0/1781582391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4241a0fa0 con 0x7fd424102b60 2024-05-14T09:56:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.356+0000 7fd41b7fe640 1 -- 172.21.15.2:0/1781582391 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42000ce60 con 0x7fd424102b60 2024-05-14T09:56:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.356+0000 7fd42c3eb640 1 -- 172.21.15.2:0/1781582391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4241a1460 con 0x7fd424102b60 2024-05-14T09:56:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.359+0000 7fd433370740 1 Processor -- start 2024-05-14T09:56:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.359+0000 7fd433370740 1 -- start start 2024-05-14T09:56:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.359+0000 7fd433370740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170b141b0 0x55d170eb7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.359+0000 7fd433370740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170aeb2f0 0x55d170eb81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.359+0000 7fd433370740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ebf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.359+0000 7fd433370740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d170e1a5d0 con 0x55d170aeb2f0 2024-05-14T09:56:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.359+0000 7fd433370740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d170dd3890 con 0x55d170b141b0 2024-05-14T09:56:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.359+0000 7fd433370740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d170e16590 con 0x55d170af34d0 2024-05-14T09:56:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.359+0000 7fd429986640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ebf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.359+0000 7fd429986640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ebf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:37256/0 (socket says 172.21.15.2:37256) 2024-05-14T09:56:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.359+0000 7fd42a988640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170b141b0 0x55d170eb7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.359+0000 7fd429986640 1 -- 172.21.15.2:0/691050375 learned_addr learned my addr 172.21.15.2:0/691050375 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.360+0000 7fd41b7fe640 1 -- 172.21.15.2:0/1781582391 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd420004020 con 0x7fd424102b60 2024-05-14T09:56:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.360+0000 7fd42a988640 1 -- 172.21.15.2:0/691050375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 msgr2=0x55d170ebf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.360+0000 7fd42a988640 1 --2- 172.21.15.2:0/691050375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ebf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.360+0000 7fd42a187640 1 --2- 172.21.15.2:0/691050375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170aeb2f0 0x55d170eb81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.360+0000 7fd42a988640 1 -- 172.21.15.2:0/691050375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170aeb2f0 msgr2=0x55d170eb81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.360+0000 7fd42a988640 1 --2- 172.21.15.2:0/691050375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170aeb2f0 0x55d170eb81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.360+0000 7fd42a988640 1 -- 172.21.15.2:0/691050375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d170b2c210 con 0x55d170b141b0 2024-05-14T09:56:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.360+0000 7fd429986640 1 --2- 172.21.15.2:0/691050375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ebf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:56:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.360+0000 7fd42a187640 1 --2- 172.21.15.2:0/691050375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170aeb2f0 0x55d170eb81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:56:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.360+0000 7fd42a988640 1 --2- 172.21.15.2:0/691050375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170b141b0 0x55d170eb7c70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009f00 tx=0x7fd41c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=662fd953860c3db3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.360+0000 7fd418bfe640 1 -- 172.21.15.2:0/691050375 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c004d20 con 0x55d170b141b0 2024-05-14T09:56:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.360+0000 7fd41b7fe640 1 --2- 172.21.15.2:0/1781582391 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd4100b28a0 0x7fd4100b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.360+0000 7fd418bfe640 1 -- 172.21.15.2:0/691050375 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd41c004030 con 0x55d170b141b0 2024-05-14T09:56:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.360+0000 7fd418bfe640 1 -- 172.21.15.2:0/691050375 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c005260 con 0x55d170b141b0 2024-05-14T09:56:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.361+0000 7fd42a187640 1 --2- 172.21.15.2:0/1781582391 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd4100b28a0 0x7fd4100b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.361+0000 7fd41b7fe640 1 -- 172.21.15.2:0/1781582391 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fd4200e2940 con 0x7fd424102b60 2024-05-14T09:56:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.361+0000 7fd433370740 1 -- 172.21.15.2:0/691050375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170b141b0 msgr2=0x55d170eb7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.361+0000 7fd42a187640 1 --2- 172.21.15.2:0/1781582391 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd4100b28a0 0x7fd4100b4db0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55d170eb9050 tx=0x7fd420000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.361+0000 7fd433370740 1 --2- 172.21.15.2:0/691050375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170b141b0 0x55d170eb7c70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009f00 tx=0x7fd41c011810 comp rx=0 tx=0).stop 2024-05-14T09:56:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.361+0000 7fd433370740 1 -- 172.21.15.2:0/691050375 shutdown_connections 2024-05-14T09:56:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.361+0000 7fd433370740 1 --2- 172.21.15.2:0/691050375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ebf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.361+0000 7fd433370740 1 --2- 172.21.15.2:0/691050375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170aeb2f0 0x55d170eb81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.361+0000 7fd433370740 1 --2- 172.21.15.2:0/691050375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170b141b0 0x55d170eb7c70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.361+0000 7fd433370740 1 -- 172.21.15.2:0/691050375 >> 172.21.15.2:0/691050375 conn(0x55d170b08300 msgr2=0x55d170eb35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.361+0000 7fd433370740 1 -- 172.21.15.2:0/691050375 shutdown_connections 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.361+0000 7fd433370740 1 -- 172.21.15.2:0/691050375 wait complete. 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.362+0000 7fd433370740 1 -- start start 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.362+0000 7fd433370740 1 Processor -- start 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.362+0000 7fd433370740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170aeb2f0 0x55d170ec1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.362+0000 7fd433370740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ec1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.362+0000 7fd433370740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170b141b0 0x55d170ec8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.362+0000 7fd433370740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d170ebfb40 con 0x55d170b141b0 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.362+0000 7fd433370740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d170dd7370 con 0x55d170aeb2f0 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.362+0000 7fd433370740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55d170ebfcc0 con 0x55d170af34d0 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.362+0000 7fd433370740 10 client.0 _ll_register_callbacks cb 0x55d170b2ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.362+0000 7fd42a988640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170b141b0 0x55d170ec8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.362+0000 7fd429986640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ec1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.362+0000 7fd42a988640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170b141b0 0x55d170ec8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55992/0 (socket says 172.21.15.2:55992) 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.362+0000 7fd429986640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ec1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:37258/0 (socket says 172.21.15.2:37258) 2024-05-14T09:56:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.362+0000 7fd42a988640 1 -- 172.21.15.2:0/3744487554 learned_addr learned my addr 172.21.15.2:0/3744487554 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.362+0000 7fd42a988640 1 --2- 172.21.15.2:0/3744487554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170aeb2f0 0x55d170ec1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.363+0000 7fd429986640 1 -- 172.21.15.2:0/3744487554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170aeb2f0 msgr2=0x55d170ec1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.363+0000 7fd429986640 1 --2- 172.21.15.2:0/3744487554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170aeb2f0 0x55d170ec1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.363+0000 7fd429986640 1 -- 172.21.15.2:0/3744487554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170b141b0 msgr2=0x55d170ec8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.363+0000 7fd429986640 1 --2- 172.21.15.2:0/3744487554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170b141b0 0x55d170ec8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.363+0000 7fd429986640 1 -- 172.21.15.2:0/3744487554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d170bf8620 con 0x55d170af34d0 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.363+0000 7fd42a988640 1 --2- 172.21.15.2:0/3744487554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170b141b0 0x55d170ec8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.363+0000 7fd429986640 1 --2- 172.21.15.2:0/3744487554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ec1d90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd414032fd0 tx=0x7fd414031820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff07ee8b47602e8f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.363+0000 7fd40dffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.363+0000 7fd40dffb640 1 -- 172.21.15.2:0/3744487554 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414035040 con 0x55d170af34d0 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.363+0000 7fd433370740 1 -- 172.21.15.2:0/3744487554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd41c004030 con 0x55d170af34d0 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.363+0000 7fd40dffb640 1 -- 172.21.15.2:0/3744487554 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fd414004450 con 0x55d170af34d0 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.363+0000 7fd40dffb640 1 -- 172.21.15.2:0/3744487554 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd4140035f0 con 0x55d170af34d0 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.363+0000 7fd433370740 10 client.27022 fetch_fsmap learned FSMap version 67 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.363+0000 7fd433370740 1 -- 172.21.15.2:0/3744487554 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55d170ecbc50 con 0x55d170af34d0 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.364+0000 7fd40dffb640 1 -- 172.21.15.2:0/3744487554 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd41c004030 con 0x55d170af34d0 2024-05-14T09:56:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.364+0000 7fd40dffb640 1 -- 172.21.15.2:0/3744487554 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fd414044070 con 0x55d170af34d0 2024-05-14T09:56:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.364+0000 7fd433370740 10 client.27022 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:56:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.364+0000 7fd433370740 10 client.27022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:56:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.364+0000 7fd433370740 20 client.27022 populate_metadata read hostname 'smithi002' 2024-05-14T09:56:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.364+0000 7fd433370740 1 --2- 172.21.15.2:0/3744487554 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d170ad2d80 0x55d170ecf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.364+0000 7fd433370740 4 client.27022 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:56:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.364+0000 7fd433370740 1 -- 172.21.15.2:0/3744487554 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d170e281e0 con 0x55d170ad2d80 2024-05-14T09:56:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.364+0000 7fd42a988640 1 --2- 172.21.15.2:0/3744487554 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d170ad2d80 0x55d170ecf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.365+0000 7fd40dffb640 10 client.27022 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.365+0000 7fd42a988640 1 --2- 172.21.15.2:0/3744487554 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d170ad2d80 0x55d170ecf2d0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd78f6142ae66ef8 server_cookie=81dea654797ba2af in_seq=0 out_seq=0 2024-05-14T09:56:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.368+0000 7fd40dffb640 1 -- 172.21.15.2:0/3744487554 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d170e281e0 con 0x55d170ad2d80 2024-05-14T09:56:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.368+0000 7fd40dffb640 10 client.27022 handle_command_reply: tid=0 2024-05-14T09:56:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.368+0000 7fd433370740 1 client.27022 shutdown 2024-05-14T09:56:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.368+0000 7fd433370740 1 -- 172.21.15.2:0/3744487554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 msgr2=0x55d170ec1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.368+0000 7fd433370740 1 --2- 172.21.15.2:0/3744487554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ec1d90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd414032fd0 tx=0x7fd414031820 comp rx=0 tx=0).stop 2024-05-14T09:56:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.368+0000 7fd433370740 1 -- 172.21.15.2:0/3744487554 shutdown_connections 2024-05-14T09:56:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.368+0000 7fd433370740 1 --2- 172.21.15.2:0/3744487554 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d170ad2d80 0x55d170ecf2d0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.368+0000 7fd433370740 1 --2- 172.21.15.2:0/3744487554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170b141b0 0x55d170ec8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.368+0000 7fd40dffb640 0 client.27022 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.368+0000 7fd433370740 1 --2- 172.21.15.2:0/3744487554 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ec1d90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.368+0000 7fd433370740 1 --2- 172.21.15.2:0/3744487554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170aeb2f0 0x55d170ec1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.368+0000 7fd433370740 1 -- 172.21.15.2:0/3744487554 >> 172.21.15.2:0/3744487554 conn(0x55d170b08300 msgr2=0x55d170eb6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.368+0000 7fd433370740 1 -- 172.21.15.2:0/3744487554 shutdown_connections 2024-05-14T09:56:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.368+0000 7fd433370740 1 -- 172.21.15.2:0/3744487554 wait complete. 2024-05-14T09:56:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.368+0000 7fd433370740 20 client.27022 trim_cache size 0 max 16384 2024-05-14T09:56:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.385+0000 7fd433370740 1 Processor -- start 2024-05-14T09:56:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.385+0000 7fd433370740 1 -- start start 2024-05-14T09:56:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.385+0000 7fd433370740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170ad2d80 0x55d170ec07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.385+0000 7fd433370740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170aeb2f0 0x55d170ec8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.385+0000 7fd433370740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ec9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.385+0000 7fd433370740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241ad390 con 0x55d170ad2d80 2024-05-14T09:56:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.385+0000 7fd433370740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241aac60 con 0x55d170aeb2f0 2024-05-14T09:56:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.385+0000 7fd433370740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241a8ab0 con 0x55d170af34d0 2024-05-14T09:56:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.385+0000 7fd42a988640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170ad2d80 0x55d170ec07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.385+0000 7fd42a988640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170ad2d80 0x55d170ec07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56002/0 (socket says 172.21.15.2:56002) 2024-05-14T09:56:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.385+0000 7fd42a988640 1 -- 172.21.15.2:0/2971695049 learned_addr learned my addr 172.21.15.2:0/2971695049 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.385+0000 7fd429986640 1 --2- 172.21.15.2:0/2971695049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170aeb2f0 0x55d170ec8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.385+0000 7fd42a988640 1 --2- 172.21.15.2:0/2971695049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ec9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.386+0000 7fd42a988640 1 -- 172.21.15.2:0/2971695049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 msgr2=0x55d170ec9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.386+0000 7fd42a988640 1 --2- 172.21.15.2:0/2971695049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ec9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.386+0000 7fd42a988640 1 -- 172.21.15.2:0/2971695049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170aeb2f0 msgr2=0x55d170ec8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.386+0000 7fd42a988640 1 --2- 172.21.15.2:0/2971695049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170aeb2f0 0x55d170ec8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.386+0000 7fd42a988640 1 -- 172.21.15.2:0/2971695049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d170eb3f10 con 0x55d170ad2d80 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.386+0000 7fd42a988640 1 --2- 172.21.15.2:0/2971695049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170ad2d80 0x55d170ec07d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd41c011d10 tx=0x7fd41c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e420bb78316854d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.386+0000 7fd40ffff640 1 -- 172.21.15.2:0/2971695049 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c036070 con 0x55d170ad2d80 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.386+0000 7fd40ffff640 1 -- 172.21.15.2:0/2971695049 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd41c00f930 con 0x55d170ad2d80 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.386+0000 7fd40ffff640 1 -- 172.21.15.2:0/2971695049 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c02c070 con 0x55d170ad2d80 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.387+0000 7fd433370740 1 -- 172.21.15.2:0/2971695049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170ad2d80 msgr2=0x55d170ec07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.387+0000 7fd433370740 1 --2- 172.21.15.2:0/2971695049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170ad2d80 0x55d170ec07d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd41c011d10 tx=0x7fd41c009500 comp rx=0 tx=0).stop 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.387+0000 7fd433370740 1 -- 172.21.15.2:0/2971695049 shutdown_connections 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.387+0000 7fd433370740 1 --2- 172.21.15.2:0/2971695049 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170af34d0 0x55d170ec9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.387+0000 7fd433370740 1 --2- 172.21.15.2:0/2971695049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170aeb2f0 0x55d170ec8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.387+0000 7fd433370740 1 --2- 172.21.15.2:0/2971695049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170ad2d80 0x55d170ec07d0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.387+0000 7fd433370740 1 -- 172.21.15.2:0/2971695049 >> 172.21.15.2:0/2971695049 conn(0x55d170b08300 msgr2=0x55d170ec00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.387+0000 7fd433370740 1 -- 172.21.15.2:0/2971695049 shutdown_connections 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.387+0000 7fd433370740 1 -- 172.21.15.2:0/2971695049 wait complete. 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.387+0000 7fd433370740 1 -- start start 2024-05-14T09:56:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.387+0000 7fd433370740 1 Processor -- start 2024-05-14T09:56:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.387+0000 7fd433370740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170ad2d80 0x55d170ec6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.387+0000 7fd433370740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170aeb2f0 0x55d170ec6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.387+0000 7fd433370740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170af34d0 0x55d170ec7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.387+0000 7fd433370740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241a9120 con 0x55d170af34d0 2024-05-14T09:56:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd433370740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241aa590 con 0x55d170ad2d80 2024-05-14T09:56:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd433370740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241a97f0 con 0x55d170aeb2f0 2024-05-14T09:56:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd433370740 10 client.0 _ll_register_callbacks cb 0x7fd414009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:56:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd42a988640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170af34d0 0x55d170ec7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd42a988640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170af34d0 0x55d170ec7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56014/0 (socket says 172.21.15.2:56014) 2024-05-14T09:56:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd42a988640 1 -- 172.21.15.2:0/1354955291 learned_addr learned my addr 172.21.15.2:0/1354955291 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd429986640 1 --2- 172.21.15.2:0/1354955291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170aeb2f0 0x55d170ec6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd42a988640 1 --2- 172.21.15.2:0/1354955291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170ad2d80 0x55d170ec6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd42a988640 1 -- 172.21.15.2:0/1354955291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170aeb2f0 msgr2=0x55d170ec6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd42a988640 1 --2- 172.21.15.2:0/1354955291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170aeb2f0 0x55d170ec6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd42a988640 1 -- 172.21.15.2:0/1354955291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170ad2d80 msgr2=0x55d170ec6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd42a988640 1 --2- 172.21.15.2:0/1354955291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170ad2d80 0x55d170ec6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd42a988640 1 -- 172.21.15.2:0/1354955291 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d170ec84b0 con 0x55d170af34d0 2024-05-14T09:56:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd429986640 1 --2- 172.21.15.2:0/1354955291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170aeb2f0 0x55d170ec6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:56:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd42a988640 1 --2- 172.21.15.2:0/1354955291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170af34d0 0x55d170ec7ef0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009270 tx=0x7fd41c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc87a3d9a8115db7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd40dffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:56:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd40dffb640 1 -- 172.21.15.2:0/1354955291 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c02c070 con 0x55d170af34d0 2024-05-14T09:56:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd40dffb640 1 -- 172.21.15.2:0/1354955291 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fd41c00f680 con 0x55d170af34d0 2024-05-14T09:56:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.388+0000 7fd433370740 1 -- 172.21.15.2:0/1354955291 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd41c00f930 con 0x55d170af34d0 2024-05-14T09:56:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.389+0000 7fd40dffb640 1 -- 172.21.15.2:0/1354955291 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd41c036070 con 0x55d170af34d0 2024-05-14T09:56:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.389+0000 7fd433370740 10 client.17415 fetch_fsmap learned FSMap version 67 2024-05-14T09:56:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.389+0000 7fd433370740 1 -- 172.21.15.2:0/1354955291 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55d170ec9740 con 0x55d170af34d0 2024-05-14T09:56:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.389+0000 7fd40dffb640 1 -- 172.21.15.2:0/1354955291 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd41c00f930 con 0x55d170af34d0 2024-05-14T09:56:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.389+0000 7fd40dffb640 1 -- 172.21.15.2:0/1354955291 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fd41c024040 con 0x55d170af34d0 2024-05-14T09:56:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.389+0000 7fd433370740 10 client.17415 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:56:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.389+0000 7fd433370740 10 client.17415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:56:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.389+0000 7fd433370740 20 client.17415 populate_metadata read hostname 'smithi002' 2024-05-14T09:56:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.389+0000 7fd433370740 1 --2- 172.21.15.2:0/1354955291 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d170b141b0 0x55d170f39ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.389+0000 7fd433370740 4 client.17415 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:56:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.389+0000 7fd433370740 1 -- 172.21.15.2:0/1354955291 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d170e22050 con 0x55d170b141b0 2024-05-14T09:56:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.389+0000 7fd429986640 1 --2- 172.21.15.2:0/1354955291 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d170b141b0 0x55d170f39ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.390+0000 7fd429986640 1 --2- 172.21.15.2:0/1354955291 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d170b141b0 0x55d170f39ba0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ac0ea72d3beb677 server_cookie=788779082a1a19d6 in_seq=0 out_seq=0 2024-05-14T09:56:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:09.390+0000 7fd40dffb640 10 client.17415 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:09.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 230649 26.40 MB/sec execute 335 sec latency 251.112 ms 2024-05-14T09:56:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.110+0000 7fd40dffb640 1 -- 172.21.15.2:0/1354955291 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d170e22050 con 0x55d170b141b0 2024-05-14T09:56:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.110+0000 7fd40dffb640 10 client.17415 handle_command_reply: tid=0 2024-05-14T09:56:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.110+0000 7fd433370740 1 client.17415 shutdown 2024-05-14T09:56:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.110+0000 7fd433370740 1 -- 172.21.15.2:0/1354955291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170af34d0 msgr2=0x55d170ec7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.110+0000 7fd433370740 1 --2- 172.21.15.2:0/1354955291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170af34d0 0x55d170ec7ef0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009270 tx=0x7fd41c004340 comp rx=0 tx=0).stop 2024-05-14T09:56:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.110+0000 7fd433370740 1 -- 172.21.15.2:0/1354955291 shutdown_connections 2024-05-14T09:56:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.110+0000 7fd433370740 1 --2- 172.21.15.2:0/1354955291 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55d170b141b0 0x55d170f39ba0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.110+0000 7fd433370740 1 --2- 172.21.15.2:0/1354955291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d170af34d0 0x55d170ec7ef0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.110+0000 7fd433370740 1 --2- 172.21.15.2:0/1354955291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d170aeb2f0 0x55d170ec6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.110+0000 7fd40dffb640 0 client.17415 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.110+0000 7fd433370740 1 --2- 172.21.15.2:0/1354955291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d170ad2d80 0x55d170ec6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.111+0000 7fd433370740 1 -- 172.21.15.2:0/1354955291 >> 172.21.15.2:0/1354955291 conn(0x55d170b08300 msgr2=0x55d170eb2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.111+0000 7fd433370740 1 -- 172.21.15.2:0/1354955291 shutdown_connections 2024-05-14T09:56:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.111+0000 7fd433370740 1 -- 172.21.15.2:0/1354955291 wait complete. 2024-05-14T09:56:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.111+0000 7fd433370740 20 client.17415 trim_cache size 0 max 16384 2024-05-14T09:56:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:56:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:56:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:56:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:56:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.112+0000 7fd42c3eb640 1 -- 172.21.15.2:0/1781582391 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd4100b28a0 msgr2=0x7fd4100b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.112+0000 7fd42c3eb640 1 --2- 172.21.15.2:0/1781582391 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd4100b28a0 0x7fd4100b4db0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55d170eb9050 tx=0x7fd420000bc0 comp rx=0 tx=0).stop 2024-05-14T09:56:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.112+0000 7fd42c3eb640 1 -- 172.21.15.2:0/1781582391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd424102b60 msgr2=0x7fd4241991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.112+0000 7fd42c3eb640 1 --2- 172.21.15.2:0/1781582391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd424102b60 0x7fd4241991d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd42000e9f0 tx=0x7fd42000eeb0 comp rx=0 tx=0).stop 2024-05-14T09:56:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.112+0000 7fd42c3eb640 1 -- 172.21.15.2:0/1781582391 shutdown_connections 2024-05-14T09:56:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.112+0000 7fd42c3eb640 1 --2- 172.21.15.2:0/1781582391 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd4100b28a0 0x7fd4100b4db0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.112+0000 7fd42c3eb640 1 --2- 172.21.15.2:0/1781582391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd424108aa0 0x7fd4241a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.112+0000 7fd42c3eb640 1 --2- 172.21.15.2:0/1781582391 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd424103da0 0x7fd424199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.112+0000 7fd42c3eb640 1 --2- 172.21.15.2:0/1781582391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd424102b60 0x7fd4241991d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.112+0000 7fd42c3eb640 1 -- 172.21.15.2:0/1781582391 >> 172.21.15.2:0/1781582391 conn(0x7fd4240fdd50 msgr2=0x7fd424104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.112+0000 7fd42c3eb640 1 -- 172.21.15.2:0/1781582391 shutdown_connections 2024-05-14T09:56:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:10.112+0000 7fd42c3eb640 1 -- 172.21.15.2:0/1781582391 wait complete. 2024-05-14T09:56:10.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 231080 26.39 MB/sec execute 336 sec latency 474.310 ms 2024-05-14T09:56:10.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:10 smithi002 ceph-mon[31557]: pgmap v1063: 129 pgs: 129 active+clean; 684 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.6 MiB/s wr, 370 op/s 2024-05-14T09:56:10.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:10 smithi029 ceph-mon[36249]: pgmap v1063: 129 pgs: 129 active+clean; 684 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.6 MiB/s wr, 370 op/s 2024-05-14T09:56:10.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:10 smithi026 ceph-mon[36056]: pgmap v1063: 129 pgs: 129 active+clean; 684 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.6 MiB/s wr, 370 op/s 2024-05-14T09:56:11.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 231624 26.41 MB/sec execute 337 sec latency 201.921 ms 2024-05-14T09:56:12.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 232137 26.43 MB/sec execute 338 sec latency 70.698 ms 2024-05-14T09:56:12.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:12 smithi002 ceph-mon[31557]: pgmap v1064: 129 pgs: 129 active+clean; 685 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.4 MiB/s wr, 357 op/s 2024-05-14T09:56:12.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:12 smithi029 ceph-mon[36249]: pgmap v1064: 129 pgs: 129 active+clean; 685 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.4 MiB/s wr, 357 op/s 2024-05-14T09:56:12.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:12 smithi026 ceph-mon[36056]: pgmap v1064: 129 pgs: 129 active+clean; 685 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.4 MiB/s wr, 357 op/s 2024-05-14T09:56:13.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 232538 26.41 MB/sec execute 339 sec latency 212.136 ms 2024-05-14T09:56:14.619 INFO:tasks.workunit.client.0.smithi002.stdout: 10 232996 26.40 MB/sec execute 340 sec latency 162.629 ms 2024-05-14T09:56:14.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:14 smithi002 ceph-mon[31557]: pgmap v1065: 129 pgs: 129 active+clean; 687 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.8 MiB/s wr, 466 op/s 2024-05-14T09:56:14.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:14 smithi029 ceph-mon[36249]: pgmap v1065: 129 pgs: 129 active+clean; 687 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.8 MiB/s wr, 466 op/s 2024-05-14T09:56:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:14 smithi026 ceph-mon[36056]: pgmap v1065: 129 pgs: 129 active+clean; 687 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.8 MiB/s wr, 466 op/s 2024-05-14T09:56:15.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 233546 26.39 MB/sec execute 341 sec latency 188.238 ms 2024-05-14T09:56:16.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 233572 26.32 MB/sec execute 342 sec latency 1188.284 ms 2024-05-14T09:56:16.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:16 smithi002 ceph-mon[31557]: pgmap v1066: 129 pgs: 129 active+clean; 689 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.4 MiB/s wr, 419 op/s 2024-05-14T09:56:16.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:16 smithi029 ceph-mon[36249]: pgmap v1066: 129 pgs: 129 active+clean; 689 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.4 MiB/s wr, 419 op/s 2024-05-14T09:56:16.953 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:16 smithi026 ceph-mon[36056]: pgmap v1066: 129 pgs: 129 active+clean; 689 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.4 MiB/s wr, 419 op/s 2024-05-14T09:56:17.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 233645 26.25 MB/sec execute 343 sec latency 1916.911 ms 2024-05-14T09:56:18.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 233905 26.21 MB/sec execute 344 sec latency 1572.434 ms 2024-05-14T09:56:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:18 smithi002 ceph-mon[31557]: pgmap v1067: 129 pgs: 129 active+clean; 734 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.5 MiB/s wr, 501 op/s 2024-05-14T09:56:18.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:18 smithi029 ceph-mon[36249]: pgmap v1067: 129 pgs: 129 active+clean; 734 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.5 MiB/s wr, 501 op/s 2024-05-14T09:56:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:18 smithi026 ceph-mon[36056]: pgmap v1067: 129 pgs: 129 active+clean; 734 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.5 MiB/s wr, 501 op/s 2024-05-14T09:56:19.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 234180 26.17 MB/sec execute 345 sec latency 200.206 ms 2024-05-14T09:56:20.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 234469 26.15 MB/sec execute 346 sec latency 179.442 ms 2024-05-14T09:56:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:20 smithi002 ceph-mon[31557]: pgmap v1068: 129 pgs: 129 active+clean; 751 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.6 MiB/s wr, 515 op/s 2024-05-14T09:56:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:20 smithi029 ceph-mon[36249]: pgmap v1068: 129 pgs: 129 active+clean; 751 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.6 MiB/s wr, 515 op/s 2024-05-14T09:56:20.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:20 smithi026 ceph-mon[36056]: pgmap v1068: 129 pgs: 129 active+clean; 751 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.6 MiB/s wr, 515 op/s 2024-05-14T09:56:21.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 234778 26.15 MB/sec execute 347 sec latency 146.978 ms 2024-05-14T09:56:22.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 235414 26.17 MB/sec execute 348 sec latency 83.867 ms 2024-05-14T09:56:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:22 smithi002 ceph-mon[31557]: pgmap v1069: 129 pgs: 129 active+clean; 751 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.4 MiB/s wr, 548 op/s 2024-05-14T09:56:22.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:22 smithi029 ceph-mon[36249]: pgmap v1069: 129 pgs: 129 active+clean; 751 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.4 MiB/s wr, 548 op/s 2024-05-14T09:56:22.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:22 smithi026 ceph-mon[36056]: pgmap v1069: 129 pgs: 129 active+clean; 751 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.4 MiB/s wr, 548 op/s 2024-05-14T09:56:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.254+0000 7f2b18089640 1 -- 172.21.15.2:0/4108478292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b10103d90 msgr2=0x7f2b101041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.254+0000 7f2b18089640 1 --2- 172.21.15.2:0/4108478292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b10103d90 0x7f2b101041a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009900 tx=0x7f2b0002a410 comp rx=0 tx=0).stop 2024-05-14T09:56:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.254+0000 7f2b18089640 1 -- 172.21.15.2:0/4108478292 shutdown_connections 2024-05-14T09:56:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.254+0000 7f2b18089640 1 --2- 172.21.15.2:0/4108478292 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b10108a90 0x7f2b1010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.254+0000 7f2b18089640 1 --2- 172.21.15.2:0/4108478292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b10103d90 0x7f2b101041a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.254+0000 7f2b18089640 1 --2- 172.21.15.2:0/4108478292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b10102b50 0x7f2b10103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.254+0000 7f2b18089640 1 -- 172.21.15.2:0/4108478292 >> 172.21.15.2:0/4108478292 conn(0x7f2b100fdd40 msgr2=0x7f2b101001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.254+0000 7f2b18089640 1 -- 172.21.15.2:0/4108478292 shutdown_connections 2024-05-14T09:56:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.254+0000 7f2b18089640 1 -- 172.21.15.2:0/4108478292 wait complete. 2024-05-14T09:56:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.255+0000 7f2b18089640 1 Processor -- start 2024-05-14T09:56:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.255+0000 7f2b18089640 1 -- start start 2024-05-14T09:56:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.255+0000 7f2b18089640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b10102b50 0x7f2b1019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.255+0000 7f2b18089640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b10103d90 0x7f2b1019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.255+0000 7f2b18089640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b10108a90 0x7f2b101a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.255+0000 7f2b18089640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1010e6d0 con 0x7f2b10103d90 2024-05-14T09:56:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.255+0000 7f2b18089640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1010e550 con 0x7f2b10108a90 2024-05-14T09:56:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.255+0000 7f2b18089640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1010e850 con 0x7f2b10102b50 2024-05-14T09:56:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.255+0000 7f2b15624640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b10103d90 0x7f2b1019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.255+0000 7f2b15e25640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b10102b50 0x7f2b1019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.255+0000 7f2b15624640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b10103d90 0x7f2b1019a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37456/0 (socket says 172.21.15.2:37456) 2024-05-14T09:56:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.255+0000 7f2b15624640 1 -- 172.21.15.2:0/2676411586 learned_addr learned my addr 172.21.15.2:0/2676411586 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.256+0000 7f2b16626640 1 --2- 172.21.15.2:0/2676411586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b10108a90 0x7f2b101a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.256+0000 7f2b15e25640 1 -- 172.21.15.2:0/2676411586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b10108a90 msgr2=0x7f2b101a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.256+0000 7f2b15e25640 1 --2- 172.21.15.2:0/2676411586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b10108a90 0x7f2b101a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.256+0000 7f2b15e25640 1 -- 172.21.15.2:0/2676411586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b10103d90 msgr2=0x7f2b1019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.256+0000 7f2b15e25640 1 --2- 172.21.15.2:0/2676411586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b10103d90 0x7f2b1019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.256+0000 7f2b15e25640 1 -- 172.21.15.2:0/2676411586 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b101a1db0 con 0x7f2b10102b50 2024-05-14T09:56:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.256+0000 7f2b16626640 1 --2- 172.21.15.2:0/2676411586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b10108a90 0x7f2b101a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:56:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.256+0000 7f2b15e25640 1 --2- 172.21.15.2:0/2676411586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b10102b50 0x7f2b1019a270 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2b10102420 tx=0x7f2b0c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.256+0000 7f2b06ffd640 1 -- 172.21.15.2:0/2676411586 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c00cc40 con 0x7f2b10102b50 2024-05-14T09:56:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.256+0000 7f2b06ffd640 1 -- 172.21.15.2:0/2676411586 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b0c00cde0 con 0x7f2b10102b50 2024-05-14T09:56:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.256+0000 7f2b06ffd640 1 -- 172.21.15.2:0/2676411586 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c010430 con 0x7f2b10102b50 2024-05-14T09:56:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.256+0000 7f2b18089640 1 -- 172.21.15.2:0/2676411586 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b101a20a0 con 0x7f2b10102b50 2024-05-14T09:56:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.257+0000 7f2b18089640 1 -- 172.21.15.2:0/2676411586 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b101a25a0 con 0x7f2b10102b50 2024-05-14T09:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.260+0000 7f2b1f019740 1 Processor -- start 2024-05-14T09:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.260+0000 7f2b1f019740 1 -- start start 2024-05-14T09:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.260+0000 7f2b1f019740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18437a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.260+0000 7f2b1f019740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd1806d2f0 0x55cd18437fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.260+0000 7f2b1f019740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd180754d0 0x55cd1843eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.260+0000 7f2b1f019740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cd1839c5d0 con 0x55cd180961b0 2024-05-14T09:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.260+0000 7f2b1f019740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cd18355890 con 0x55cd1806d2f0 2024-05-14T09:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.260+0000 7f2b1f019740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cd18398590 con 0x55cd180754d0 2024-05-14T09:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.260+0000 7f2b16626640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18437a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.260+0000 7f2b16626640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18437a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37464/0 (socket says 172.21.15.2:37464) 2024-05-14T09:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.260+0000 7f2b16626640 1 -- 172.21.15.2:0/1749823504 learned_addr learned my addr 172.21.15.2:0/1749823504 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.260+0000 7f2b15624640 1 --2- 172.21.15.2:0/1749823504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd180754d0 0x55cd1843eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.261+0000 7f2b16626640 1 -- 172.21.15.2:0/1749823504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd180754d0 msgr2=0x55cd1843eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.261+0000 7f2b16626640 1 --2- 172.21.15.2:0/1749823504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd180754d0 0x55cd1843eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.261+0000 7f2b16626640 1 -- 172.21.15.2:0/1749823504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd1806d2f0 msgr2=0x55cd18437fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.261+0000 7f2b16626640 1 --2- 172.21.15.2:0/1749823504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd1806d2f0 0x55cd18437fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.261+0000 7f2b16626640 1 -- 172.21.15.2:0/1749823504 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd180ae210 con 0x55cd180961b0 2024-05-14T09:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.261+0000 7f2b16626640 1 --2- 172.21.15.2:0/1749823504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18437a60 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009fd0 tx=0x7f2b080108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f948caeccee5c159 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.261+0000 7f2b06ffd640 1 -- 172.21.15.2:0/2676411586 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2b0c004020 con 0x7f2b10102b50 2024-05-14T09:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.261+0000 7f2afbfff640 1 -- 172.21.15.2:0/1749823504 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b08011760 con 0x55cd180961b0 2024-05-14T09:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.261+0000 7f2afbfff640 1 -- 172.21.15.2:0/1749823504 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b08011db0 con 0x55cd180961b0 2024-05-14T09:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.261+0000 7f2afbfff640 1 -- 172.21.15.2:0/1749823504 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0800e3c0 con 0x55cd180961b0 2024-05-14T09:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.262+0000 7f2b1f019740 1 -- 172.21.15.2:0/1749823504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 msgr2=0x55cd18437a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.262+0000 7f2b1f019740 1 --2- 172.21.15.2:0/1749823504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18437a60 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009fd0 tx=0x7f2b080108e0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.262+0000 7f2b1f019740 1 -- 172.21.15.2:0/1749823504 shutdown_connections 2024-05-14T09:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.262+0000 7f2b1f019740 1 --2- 172.21.15.2:0/1749823504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd180754d0 0x55cd1843eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.262+0000 7f2b1f019740 1 --2- 172.21.15.2:0/1749823504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd1806d2f0 0x55cd18437fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.262+0000 7f2b1f019740 1 --2- 172.21.15.2:0/1749823504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18437a60 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.262+0000 7f2b1f019740 1 -- 172.21.15.2:0/1749823504 >> 172.21.15.2:0/1749823504 conn(0x55cd1808a300 msgr2=0x55cd184333a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.263+0000 7f2b1f019740 1 -- 172.21.15.2:0/1749823504 shutdown_connections 2024-05-14T09:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.263+0000 7f2b1f019740 1 -- 172.21.15.2:0/1749823504 wait complete. 2024-05-14T09:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.263+0000 7f2b06ffd640 1 --2- 172.21.15.2:0/2676411586 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2afc0b2830 0x7f2afc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.263+0000 7f2b1f019740 1 -- start start 2024-05-14T09:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.263+0000 7f2b15624640 1 --2- 172.21.15.2:0/2676411586 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2afc0b2830 0x7f2afc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.263+0000 7f2b1f019740 1 Processor -- start 2024-05-14T09:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b1f019740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd1806d2f0 0x55cd184416b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b1f019740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd180754d0 0x55cd18441bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b1f019740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18448b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b1f019740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cd1843f930 con 0x55cd180961b0 2024-05-14T09:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b1f019740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cd18359370 con 0x55cd1806d2f0 2024-05-14T09:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b1f019740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cd1843fab0 con 0x55cd180754d0 2024-05-14T09:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b15e25640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd1806d2f0 0x55cd184416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b15e25640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd1806d2f0 0x55cd184416b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:44950/0 (socket says 172.21.15.2:44950) 2024-05-14T09:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b15e25640 1 -- 172.21.15.2:0/408627885 learned_addr learned my addr 172.21.15.2:0/408627885 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b16626640 1 --2- 172.21.15.2:0/408627885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18448b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b1f019740 10 client.0 _ll_register_callbacks cb 0x55cd180ada00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b15624640 1 --2- 172.21.15.2:0/2676411586 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2afc0b2830 0x7f2afc0b4d40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2b00000c00 tx=0x7f2b000314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b15e25640 1 -- 172.21.15.2:0/408627885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd180754d0 msgr2=0x55cd18441bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b15e25640 1 --2- 172.21.15.2:0/408627885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd180754d0 0x55cd18441bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b15e25640 1 -- 172.21.15.2:0/408627885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 msgr2=0x55cd18448b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b15e25640 1 --2- 172.21.15.2:0/408627885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18448b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b15e25640 1 -- 172.21.15.2:0/408627885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd1817a620 con 0x55cd1806d2f0 2024-05-14T09:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b16626640 1 --2- 172.21.15.2:0/408627885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18448b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.264+0000 7f2b06ffd640 1 -- 172.21.15.2:0/2676411586 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f2b0c0e2bb0 con 0x7f2b10102b50 2024-05-14T09:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.265+0000 7f2b15e25640 1 --2- 172.21.15.2:0/408627885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd1806d2f0 0x55cd184416b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0a62f0 tx=0x7f2b0c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2d3e2779e266842 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.265+0000 7f2af97fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.266+0000 7f2af97fa640 1 -- 172.21.15.2:0/408627885 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c0a9860 con 0x55cd1806d2f0 2024-05-14T09:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.266+0000 7f2af97fa640 1 -- 172.21.15.2:0/408627885 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f2b0c004310 con 0x55cd1806d2f0 2024-05-14T09:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.266+0000 7f2b1f019740 1 -- 172.21.15.2:0/408627885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b08011db0 con 0x55cd1806d2f0 2024-05-14T09:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.266+0000 7f2af97fa640 1 -- 172.21.15.2:0/408627885 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f2b0c0aa450 con 0x55cd1806d2f0 2024-05-14T09:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.266+0000 7f2b1f019740 10 client.27044 fetch_fsmap learned FSMap version 67 2024-05-14T09:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.266+0000 7f2b1f019740 1 -- 172.21.15.2:0/408627885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55cd1844bb10 con 0x55cd1806d2f0 2024-05-14T09:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.266+0000 7f2af97fa640 1 -- 172.21.15.2:0/408627885 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b08011db0 con 0x55cd1806d2f0 2024-05-14T09:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.266+0000 7f2af97fa640 1 -- 172.21.15.2:0/408627885 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f2b0c0a9fb0 con 0x55cd1806d2f0 2024-05-14T09:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.266+0000 7f2b1f019740 10 client.27044 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.266+0000 7f2b1f019740 10 client.27044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.266+0000 7f2b1f019740 20 client.27044 populate_metadata read hostname 'smithi002' 2024-05-14T09:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.266+0000 7f2b1f019740 1 --2- 172.21.15.2:0/408627885 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cd18054d80 0x55cd1844f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.266+0000 7f2b1f019740 4 client.27044 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.267+0000 7f2b1f019740 1 -- 172.21.15.2:0/408627885 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd183aa1e0 con 0x55cd18054d80 2024-05-14T09:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.267+0000 7f2b15624640 1 --2- 172.21.15.2:0/408627885 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cd18054d80 0x55cd1844f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.267+0000 7f2b15624640 1 --2- 172.21.15.2:0/408627885 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cd18054d80 0x55cd1844f230 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f80946e871fb4944 server_cookie=72188d8a0b88452b in_seq=0 out_seq=0 2024-05-14T09:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.267+0000 7f2af97fa640 10 client.27044 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.270+0000 7f2af97fa640 1 -- 172.21.15.2:0/408627885 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cd183aa1e0 con 0x55cd18054d80 2024-05-14T09:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.270+0000 7f2af97fa640 10 client.27044 handle_command_reply: tid=0 2024-05-14T09:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.271+0000 7f2b1f019740 1 client.27044 shutdown 2024-05-14T09:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.271+0000 7f2b1f019740 1 -- 172.21.15.2:0/408627885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd1806d2f0 msgr2=0x55cd184416b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.271+0000 7f2b1f019740 1 --2- 172.21.15.2:0/408627885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd1806d2f0 0x55cd184416b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0a62f0 tx=0x7f2b0c020000 comp rx=0 tx=0).stop 2024-05-14T09:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.271+0000 7f2b1f019740 1 -- 172.21.15.2:0/408627885 shutdown_connections 2024-05-14T09:56:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.271+0000 7f2b1f019740 1 --2- 172.21.15.2:0/408627885 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cd18054d80 0x55cd1844f230 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.271+0000 7f2b1f019740 1 --2- 172.21.15.2:0/408627885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18448b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.271+0000 7f2b1f019740 1 --2- 172.21.15.2:0/408627885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd180754d0 0x55cd18441bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.271+0000 7f2b1f019740 1 --2- 172.21.15.2:0/408627885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd1806d2f0 0x55cd184416b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.271+0000 7f2b1f019740 1 -- 172.21.15.2:0/408627885 >> 172.21.15.2:0/408627885 conn(0x55cd1808a300 msgr2=0x55cd184323a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.271+0000 7f2af97fa640 0 client.27044 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.271+0000 7f2b1f019740 1 -- 172.21.15.2:0/408627885 shutdown_connections 2024-05-14T09:56:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.271+0000 7f2b1f019740 1 -- 172.21.15.2:0/408627885 wait complete. 2024-05-14T09:56:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.272+0000 7f2b1f019740 20 client.27044 trim_cache size 0 max 16384 2024-05-14T09:56:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.292+0000 7f2b1f019740 1 Processor -- start 2024-05-14T09:56:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.292+0000 7f2b1f019740 1 -- start start 2024-05-14T09:56:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.293+0000 7f2b1f019740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd1806d2f0 0x55cd1843bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.293+0000 7f2b1f019740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd180754d0 0x55cd1843c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.293+0000 7f2b1f019740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18432cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.293+0000 7f2b1f019740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2b101af670 con 0x55cd180961b0 2024-05-14T09:56:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.293+0000 7f2b1f019740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b101acf40 con 0x55cd1806d2f0 2024-05-14T09:56:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.293+0000 7f2b1f019740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2b101aad90 con 0x55cd180754d0 2024-05-14T09:56:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.293+0000 7f2b16626640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18432cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.293+0000 7f2b16626640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18432cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37488/0 (socket says 172.21.15.2:37488) 2024-05-14T09:56:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.293+0000 7f2b16626640 1 -- 172.21.15.2:0/2940377894 learned_addr learned my addr 172.21.15.2:0/2940377894 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.293+0000 7f2b15624640 1 --2- 172.21.15.2:0/2940377894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd180754d0 0x55cd1843c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.293+0000 7f2b16626640 1 -- 172.21.15.2:0/2940377894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd180754d0 msgr2=0x55cd1843c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.293+0000 7f2b16626640 1 --2- 172.21.15.2:0/2940377894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd180754d0 0x55cd1843c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.293+0000 7f2b16626640 1 -- 172.21.15.2:0/2940377894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd1806d2f0 msgr2=0x55cd1843bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.293+0000 7f2b16626640 1 --2- 172.21.15.2:0/2940377894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd1806d2f0 0x55cd1843bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.293+0000 7f2b16626640 1 -- 172.21.15.2:0/2940377894 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd18442720 con 0x55cd180961b0 2024-05-14T09:56:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.294+0000 7f2b16626640 1 --2- 172.21.15.2:0/2940377894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18432cf0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2b0801dfd0 tx=0x7f2b08004c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd5a78d0881404e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.294+0000 7f2afb7fe640 1 -- 172.21.15.2:0/2940377894 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0801b430 con 0x55cd180961b0 2024-05-14T09:56:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.294+0000 7f2afb7fe640 1 -- 172.21.15.2:0/2940377894 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b08011560 con 0x55cd180961b0 2024-05-14T09:56:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.294+0000 7f2afb7fe640 1 -- 172.21.15.2:0/2940377894 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b080256d0 con 0x55cd180961b0 2024-05-14T09:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.295+0000 7f2b1f019740 1 -- 172.21.15.2:0/2940377894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 msgr2=0x55cd18432cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.295+0000 7f2b1f019740 1 --2- 172.21.15.2:0/2940377894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18432cf0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2b0801dfd0 tx=0x7f2b08004c20 comp rx=0 tx=0).stop 2024-05-14T09:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.295+0000 7f2b1f019740 1 -- 172.21.15.2:0/2940377894 shutdown_connections 2024-05-14T09:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.295+0000 7f2b1f019740 1 --2- 172.21.15.2:0/2940377894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180961b0 0x55cd18432cf0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.295+0000 7f2b1f019740 1 --2- 172.21.15.2:0/2940377894 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd180754d0 0x55cd1843c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.295+0000 7f2b1f019740 1 --2- 172.21.15.2:0/2940377894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd1806d2f0 0x55cd1843bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.295+0000 7f2b1f019740 1 -- 172.21.15.2:0/2940377894 >> 172.21.15.2:0/2940377894 conn(0x55cd1808a300 msgr2=0x55cd1843ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.295+0000 7f2b1f019740 1 -- 172.21.15.2:0/2940377894 shutdown_connections 2024-05-14T09:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.295+0000 7f2b1f019740 1 -- 172.21.15.2:0/2940377894 wait complete. 2024-05-14T09:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.295+0000 7f2b1f019740 1 -- start start 2024-05-14T09:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b1f019740 1 Processor -- start 2024-05-14T09:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b1f019740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd1806d2f0 0x55cd184452f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b1f019740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180754d0 0x55cd18445830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b1f019740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd180961b0 0x55cd18446760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b1f019740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2b101ab400 con 0x55cd180754d0 2024-05-14T09:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b1f019740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2b101ac870 con 0x55cd180961b0 2024-05-14T09:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b1f019740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2b101abad0 con 0x55cd1806d2f0 2024-05-14T09:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b1f019740 10 client.0 _ll_register_callbacks cb 0x7f2b0c0a42a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b15624640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180754d0 0x55cd18445830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b15624640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180754d0 0x55cd18445830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37500/0 (socket says 172.21.15.2:37500) 2024-05-14T09:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b15624640 1 -- 172.21.15.2:0/584232053 learned_addr learned my addr 172.21.15.2:0/584232053 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b15e25640 1 --2- 172.21.15.2:0/584232053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd1806d2f0 0x55cd184452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b16626640 1 --2- 172.21.15.2:0/584232053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd180961b0 0x55cd18446760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b15624640 1 -- 172.21.15.2:0/584232053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd1806d2f0 msgr2=0x55cd184452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b15624640 1 --2- 172.21.15.2:0/584232053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd1806d2f0 0x55cd184452f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b15624640 1 -- 172.21.15.2:0/584232053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd180961b0 msgr2=0x55cd18446760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b15624640 1 --2- 172.21.15.2:0/584232053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd180961b0 0x55cd18446760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b15624640 1 -- 172.21.15.2:0/584232053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd18446d20 con 0x55cd180754d0 2024-05-14T09:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.296+0000 7f2b16626640 1 --2- 172.21.15.2:0/584232053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd180961b0 0x55cd18446760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.297+0000 7f2b15624640 1 --2- 172.21.15.2:0/584232053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180754d0 0x55cd18445830 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2b0002aa00 tx=0x7f2b0003f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505e5b7a0417384d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.297+0000 7f2af97fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.297+0000 7f2af97fa640 1 -- 172.21.15.2:0/584232053 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b00035040 con 0x55cd180754d0 2024-05-14T09:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.297+0000 7f2af97fa640 1 -- 172.21.15.2:0/584232053 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f2b00004dd0 con 0x55cd180754d0 2024-05-14T09:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.297+0000 7f2b1f019740 1 -- 172.21.15.2:0/584232053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b08011560 con 0x55cd180754d0 2024-05-14T09:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.297+0000 7f2af97fa640 1 -- 172.21.15.2:0/584232053 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f2b00062050 con 0x55cd180754d0 2024-05-14T09:56:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.297+0000 7f2b1f019740 10 client.17433 fetch_fsmap learned FSMap version 67 2024-05-14T09:56:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.297+0000 7f2b1f019740 1 -- 172.21.15.2:0/584232053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55cd184480b0 con 0x55cd180754d0 2024-05-14T09:56:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.297+0000 7f2af97fa640 1 -- 172.21.15.2:0/584232053 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b08011560 con 0x55cd180754d0 2024-05-14T09:56:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.298+0000 7f2af97fa640 1 -- 172.21.15.2:0/584232053 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f2b00051ce0 con 0x55cd180754d0 2024-05-14T09:56:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.298+0000 7f2b1f019740 10 client.17433 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:56:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.298+0000 7f2b1f019740 10 client.17433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:56:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.298+0000 7f2b1f019740 20 client.17433 populate_metadata read hostname 'smithi002' 2024-05-14T09:56:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.298+0000 7f2b1f019740 1 --2- 172.21.15.2:0/584232053 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cd1802a1c0 0x55cd184b9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.298+0000 7f2b1f019740 4 client.17433 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:56:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.298+0000 7f2b1f019740 1 -- 172.21.15.2:0/584232053 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd183a4050 con 0x55cd1802a1c0 2024-05-14T09:56:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.298+0000 7f2b15e25640 1 --2- 172.21.15.2:0/584232053 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cd1802a1c0 0x55cd184b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.299+0000 7f2b15e25640 1 --2- 172.21.15.2:0/584232053 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cd1802a1c0 0x55cd184b9960 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5480be2bce88d9ff server_cookie=ab90056a4a698bcf in_seq=0 out_seq=0 2024-05-14T09:56:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.299+0000 7f2af97fa640 10 client.17433 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:23.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 235697 26.14 MB/sec execute 349 sec latency 235.637 ms 2024-05-14T09:56:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.853+0000 7f2af97fa640 1 -- 172.21.15.2:0/584232053 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cd183a4050 con 0x55cd1802a1c0 2024-05-14T09:56:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.853+0000 7f2af97fa640 10 client.17433 handle_command_reply: tid=0 2024-05-14T09:56:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.853+0000 7f2b1f019740 1 client.17433 shutdown 2024-05-14T09:56:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.853+0000 7f2b1f019740 1 -- 172.21.15.2:0/584232053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180754d0 msgr2=0x55cd18445830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.853+0000 7f2b1f019740 1 --2- 172.21.15.2:0/584232053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180754d0 0x55cd18445830 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2b0002aa00 tx=0x7f2b0003f070 comp rx=0 tx=0).stop 2024-05-14T09:56:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.854+0000 7f2b1f019740 1 -- 172.21.15.2:0/584232053 shutdown_connections 2024-05-14T09:56:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.854+0000 7f2b1f019740 1 --2- 172.21.15.2:0/584232053 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cd1802a1c0 0x55cd184b9960 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.854+0000 7f2af97fa640 0 client.17433 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.854+0000 7f2b1f019740 1 --2- 172.21.15.2:0/584232053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd180961b0 0x55cd18446760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.854+0000 7f2b1f019740 1 --2- 172.21.15.2:0/584232053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd180754d0 0x55cd18445830 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.854+0000 7f2b1f019740 1 --2- 172.21.15.2:0/584232053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cd1806d2f0 0x55cd184452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.854+0000 7f2b1f019740 1 -- 172.21.15.2:0/584232053 >> 172.21.15.2:0/584232053 conn(0x55cd1808a300 msgr2=0x55cd1843ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.854+0000 7f2b1f019740 1 -- 172.21.15.2:0/584232053 shutdown_connections 2024-05-14T09:56:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.854+0000 7f2b1f019740 1 -- 172.21.15.2:0/584232053 wait complete. 2024-05-14T09:56:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.855+0000 7f2b1f019740 20 client.17433 trim_cache size 0 max 16384 2024-05-14T09:56:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:56:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:56:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:56:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:56:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.855+0000 7f2b18089640 1 -- 172.21.15.2:0/2676411586 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2afc0b2830 msgr2=0x7f2afc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.855+0000 7f2b18089640 1 --2- 172.21.15.2:0/2676411586 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2afc0b2830 0x7f2afc0b4d40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2b00000c00 tx=0x7f2b000314a0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.855+0000 7f2b18089640 1 -- 172.21.15.2:0/2676411586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b10102b50 msgr2=0x7f2b1019a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.855+0000 7f2b18089640 1 --2- 172.21.15.2:0/2676411586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b10102b50 0x7f2b1019a270 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2b10102420 tx=0x7f2b0c00ee50 comp rx=0 tx=0).stop 2024-05-14T09:56:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.856+0000 7f2b18089640 1 -- 172.21.15.2:0/2676411586 shutdown_connections 2024-05-14T09:56:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.856+0000 7f2b18089640 1 --2- 172.21.15.2:0/2676411586 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2afc0b2830 0x7f2afc0b4d40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.856+0000 7f2b18089640 1 --2- 172.21.15.2:0/2676411586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b10108a90 0x7f2b101a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.856+0000 7f2b18089640 1 --2- 172.21.15.2:0/2676411586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b10103d90 0x7f2b1019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.856+0000 7f2b18089640 1 --2- 172.21.15.2:0/2676411586 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b10102b50 0x7f2b1019a270 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.856+0000 7f2b18089640 1 -- 172.21.15.2:0/2676411586 >> 172.21.15.2:0/2676411586 conn(0x7f2b100fdd40 msgr2=0x7f2b100ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.856+0000 7f2b18089640 1 -- 172.21.15.2:0/2676411586 shutdown_connections 2024-05-14T09:56:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:23.856+0000 7f2b18089640 1 -- 172.21.15.2:0/2676411586 wait complete. 2024-05-14T09:56:24.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 236119 26.13 MB/sec execute 350 sec latency 412.628 ms 2024-05-14T09:56:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:24 smithi002 ceph-mon[31557]: pgmap v1070: 129 pgs: 129 active+clean; 696 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 720 op/s 2024-05-14T09:56:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:24 smithi029 ceph-mon[36249]: pgmap v1070: 129 pgs: 129 active+clean; 696 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 720 op/s 2024-05-14T09:56:24.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:24 smithi026 ceph-mon[36056]: pgmap v1070: 129 pgs: 129 active+clean; 696 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 720 op/s 2024-05-14T09:56:25.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 236652 26.13 MB/sec execute 351 sec latency 77.858 ms 2024-05-14T09:56:26.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 237223 26.13 MB/sec execute 352 sec latency 88.294 ms 2024-05-14T09:56:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:26 smithi002 ceph-mon[31557]: pgmap v1071: 129 pgs: 129 active+clean; 692 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 616 op/s 2024-05-14T09:56:26.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:26 smithi029 ceph-mon[36249]: pgmap v1071: 129 pgs: 129 active+clean; 692 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 616 op/s 2024-05-14T09:56:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:26 smithi026 ceph-mon[36056]: pgmap v1071: 129 pgs: 129 active+clean; 692 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 616 op/s 2024-05-14T09:56:27.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 237887 26.17 MB/sec execute 353 sec latency 79.295 ms 2024-05-14T09:56:28.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 238504 26.20 MB/sec execute 354 sec latency 299.722 ms 2024-05-14T09:56:28.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:28 smithi002 ceph-mon[31557]: pgmap v1072: 129 pgs: 129 active+clean; 697 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 727 op/s 2024-05-14T09:56:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:28 smithi029 ceph-mon[36249]: pgmap v1072: 129 pgs: 129 active+clean; 697 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 727 op/s 2024-05-14T09:56:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:28 smithi026 ceph-mon[36056]: pgmap v1072: 129 pgs: 129 active+clean; 697 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 727 op/s 2024-05-14T09:56:29.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 238886 26.17 MB/sec execute 355 sec latency 275.976 ms 2024-05-14T09:56:30.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 239374 26.18 MB/sec execute 356 sec latency 144.743 ms 2024-05-14T09:56:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:30 smithi002 ceph-mon[31557]: pgmap v1073: 129 pgs: 129 active+clean; 693 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.5 MiB/s wr, 651 op/s 2024-05-14T09:56:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:30 smithi029 ceph-mon[36249]: pgmap v1073: 129 pgs: 129 active+clean; 693 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.5 MiB/s wr, 651 op/s 2024-05-14T09:56:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:30 smithi026 ceph-mon[36056]: pgmap v1073: 129 pgs: 129 active+clean; 693 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.5 MiB/s wr, 651 op/s 2024-05-14T09:56:31.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 239850 26.17 MB/sec execute 357 sec latency 91.197 ms 2024-05-14T09:56:32.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 240599 26.21 MB/sec execute 358 sec latency 77.529 ms 2024-05-14T09:56:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:32 smithi002 ceph-mon[31557]: pgmap v1074: 129 pgs: 129 active+clean; 694 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.9 MiB/s wr, 585 op/s 2024-05-14T09:56:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:32 smithi029 ceph-mon[36249]: pgmap v1074: 129 pgs: 129 active+clean; 694 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.9 MiB/s wr, 585 op/s 2024-05-14T09:56:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:32 smithi026 ceph-mon[36056]: pgmap v1074: 129 pgs: 129 active+clean; 694 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.9 MiB/s wr, 585 op/s 2024-05-14T09:56:33.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 241080 26.21 MB/sec execute 359 sec latency 249.101 ms 2024-05-14T09:56:34.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 241769 26.26 MB/sec execute 360 sec latency 72.997 ms 2024-05-14T09:56:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:34 smithi002 ceph-mon[31557]: pgmap v1075: 129 pgs: 129 active+clean; 704 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.7 MiB/s wr, 625 op/s 2024-05-14T09:56:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:34 smithi029 ceph-mon[36249]: pgmap v1075: 129 pgs: 129 active+clean; 704 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.7 MiB/s wr, 625 op/s 2024-05-14T09:56:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:34 smithi026 ceph-mon[36056]: pgmap v1075: 129 pgs: 129 active+clean; 704 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.7 MiB/s wr, 625 op/s 2024-05-14T09:56:35.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 242277 26.26 MB/sec execute 361 sec latency 96.743 ms 2024-05-14T09:56:36.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 242686 26.24 MB/sec execute 362 sec latency 114.121 ms 2024-05-14T09:56:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:36 smithi002 ceph-mon[31557]: pgmap v1076: 129 pgs: 129 active+clean; 693 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.8 MiB/s wr, 416 op/s 2024-05-14T09:56:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:36 smithi029 ceph-mon[36249]: pgmap v1076: 129 pgs: 129 active+clean; 693 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.8 MiB/s wr, 416 op/s 2024-05-14T09:56:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:36 smithi026 ceph-mon[36056]: pgmap v1076: 129 pgs: 129 active+clean; 693 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.8 MiB/s wr, 416 op/s 2024-05-14T09:56:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.048+0000 7fac25bc5640 1 -- 172.21.15.2:0/4255641778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2010c180 msgr2=0x7fac2010c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.048+0000 7fac25bc5640 1 --2- 172.21.15.2:0/4255641778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2010c180 0x7fac2010c590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fac14009860 tx=0x7fac1402a340 comp rx=0 tx=0).stop 2024-05-14T09:56:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.049+0000 7fac25bc5640 1 -- 172.21.15.2:0/4255641778 shutdown_connections 2024-05-14T09:56:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.049+0000 7fac25bc5640 1 --2- 172.21.15.2:0/4255641778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2010c180 0x7fac2010c590 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.049+0000 7fac25bc5640 1 --2- 172.21.15.2:0/4255641778 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac2010b480 0x7fac2010b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.049+0000 7fac25bc5640 1 --2- 172.21.15.2:0/4255641778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac2010a550 0x7fac2010af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.049+0000 7fac25bc5640 1 -- 172.21.15.2:0/4255641778 >> 172.21.15.2:0/4255641778 conn(0x7fac20070640 msgr2=0x7fac20072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.050+0000 7fac25bc5640 1 -- 172.21.15.2:0/4255641778 shutdown_connections 2024-05-14T09:56:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.051+0000 7fac25bc5640 1 -- 172.21.15.2:0/4255641778 wait complete. 2024-05-14T09:56:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.052+0000 7fac25bc5640 1 Processor -- start 2024-05-14T09:56:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.052+0000 7fac25bc5640 1 -- start start 2024-05-14T09:56:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.052+0000 7fac25bc5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac2010a550 0x7fac201a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.052+0000 7fac25bc5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2010b480 0x7fac201a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.052+0000 7fac25bc5640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac2010c180 0x7fac2007af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.052+0000 7fac25bc5640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac2011a2c0 con 0x7fac2010a550 2024-05-14T09:56:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.052+0000 7fac25bc5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fac2011a140 con 0x7fac2010b480 2024-05-14T09:56:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.052+0000 7fac25bc5640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac2011a440 con 0x7fac2010c180 2024-05-14T09:56:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.053+0000 7fac1ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac2010c180 0x7fac2007af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.053+0000 7fac1ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac2010c180 0x7fac2007af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:47156/0 (socket says 172.21.15.2:47156) 2024-05-14T09:56:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.053+0000 7fac1ffff640 1 -- 172.21.15.2:0/1648175426 learned_addr learned my addr 172.21.15.2:0/1648175426 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.053+0000 7fac1effd640 1 --2- 172.21.15.2:0/1648175426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2010b480 0x7fac201a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.054+0000 7fac1f7fe640 1 --2- 172.21.15.2:0/1648175426 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac2010a550 0x7fac201a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.054+0000 7fac1effd640 1 -- 172.21.15.2:0/1648175426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac2010c180 msgr2=0x7fac2007af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.054+0000 7fac1effd640 1 --2- 172.21.15.2:0/1648175426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac2010c180 0x7fac2007af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.054+0000 7fac1effd640 1 -- 172.21.15.2:0/1648175426 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac2010a550 msgr2=0x7fac201a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.054+0000 7fac1effd640 1 --2- 172.21.15.2:0/1648175426 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac2010a550 0x7fac201a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.054+0000 7fac1effd640 1 -- 172.21.15.2:0/1648175426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2007b5a0 con 0x7fac2010b480 2024-05-14T09:56:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.054+0000 7fac1f7fe640 1 --2- 172.21.15.2:0/1648175426 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac2010a550 0x7fac201a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:56:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.055+0000 7fac1effd640 1 --2- 172.21.15.2:0/1648175426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2010b480 0x7fac201a5250 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fac1800bd70 tx=0x7fac1800be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.055+0000 7fac1cff9640 1 -- 172.21.15.2:0/1648175426 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18019070 con 0x7fac2010b480 2024-05-14T09:56:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.055+0000 7fac1cff9640 1 -- 172.21.15.2:0/1648175426 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac180148f0 con 0x7fac2010b480 2024-05-14T09:56:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.055+0000 7fac25bc5640 1 -- 172.21.15.2:0/1648175426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2010d8e0 con 0x7fac2010b480 2024-05-14T09:56:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.055+0000 7fac1cff9640 1 -- 172.21.15.2:0/1648175426 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18012860 con 0x7fac2010b480 2024-05-14T09:56:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.055+0000 7fac25bc5640 1 -- 172.21.15.2:0/1648175426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2010de30 con 0x7fac2010b480 2024-05-14T09:56:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.059+0000 7fac2ca77740 1 Processor -- start 2024-05-14T09:56:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.060+0000 7fac2ca77740 1 -- start start 2024-05-14T09:56:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.060+0000 7fac2ca77740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853ad11b0 0x555853e72a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.060+0000 7fac2ca77740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853aa82f0 0x555853e72fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.060+0000 7fac2ca77740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853ab04d0 0x555853e79ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.060+0000 7fac2ca77740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555853dd75d0 con 0x555853ab04d0 2024-05-14T09:56:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.060+0000 7fac2ca77740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555853d90890 con 0x555853ad11b0 2024-05-14T09:56:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.060+0000 7fac2ca77740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555853dd3590 con 0x555853aa82f0 2024-05-14T09:56:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.061+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853aa82f0 0x555853e72fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.062+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853aa82f0 0x555853e72fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:47160/0 (socket says 172.21.15.2:47160) 2024-05-14T09:56:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.062+0000 7fac1f7fe640 1 -- 172.21.15.2:0/3646941294 learned_addr learned my addr 172.21.15.2:0/3646941294 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.062+0000 7fac1f7fe640 1 -- 172.21.15.2:0/3646941294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853ad11b0 msgr2=0x555853e72a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.062+0000 7fac1f7fe640 1 --2- 172.21.15.2:0/3646941294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853ad11b0 0x555853e72a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.062+0000 7fac1f7fe640 1 -- 172.21.15.2:0/3646941294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853ab04d0 msgr2=0x555853e79ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:56:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.062+0000 7fac1f7fe640 1 --2- 172.21.15.2:0/3646941294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853ab04d0 0x555853e79ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.062+0000 7fac1f7fe640 1 -- 172.21.15.2:0/3646941294 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555853ae9210 con 0x555853aa82f0 2024-05-14T09:56:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.063+0000 7fac1f7fe640 1 --2- 172.21.15.2:0/3646941294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853aa82f0 0x555853e72fe0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fac10009bd0 tx=0x7fac10010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab102c29b948bd27 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.063+0000 7fac1cff9640 1 -- 172.21.15.2:0/1648175426 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fac1801c070 con 0x7fac2010b480 2024-05-14T09:56:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.063+0000 7fac05ffb640 1 -- 172.21.15.2:0/3646941294 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10005020 con 0x555853aa82f0 2024-05-14T09:56:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.064+0000 7fac05ffb640 1 -- 172.21.15.2:0/3646941294 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac100047a0 con 0x555853aa82f0 2024-05-14T09:56:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.064+0000 7fac05ffb640 1 -- 172.21.15.2:0/3646941294 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac100126b0 con 0x555853aa82f0 2024-05-14T09:56:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.064+0000 7fac1cff9640 1 --2- 172.21.15.2:0/1648175426 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fac080b2ab0 0x7fac080b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.065+0000 7fac1cff9640 1 -- 172.21.15.2:0/1648175426 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fac180e1e00 con 0x7fac2010b480 2024-05-14T09:56:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.065+0000 7fac2ca77740 1 -- 172.21.15.2:0/3646941294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853aa82f0 msgr2=0x555853e72fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.065+0000 7fac2ca77740 1 --2- 172.21.15.2:0/3646941294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853aa82f0 0x555853e72fe0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fac10009bd0 tx=0x7fac10010740 comp rx=0 tx=0).stop 2024-05-14T09:56:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.065+0000 7fac1ffff640 1 --2- 172.21.15.2:0/1648175426 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fac080b2ab0 0x7fac080b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.066+0000 7fac2ca77740 1 -- 172.21.15.2:0/3646941294 shutdown_connections 2024-05-14T09:56:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.066+0000 7fac2ca77740 1 --2- 172.21.15.2:0/3646941294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853ab04d0 0x555853e79ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.066+0000 7fac2ca77740 1 --2- 172.21.15.2:0/3646941294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853aa82f0 0x555853e72fe0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.066+0000 7fac2ca77740 1 --2- 172.21.15.2:0/3646941294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853ad11b0 0x555853e72a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.066+0000 7fac2ca77740 1 -- 172.21.15.2:0/3646941294 >> 172.21.15.2:0/3646941294 conn(0x555853ac5300 msgr2=0x555853e6e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.066+0000 7fac1ffff640 1 --2- 172.21.15.2:0/1648175426 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fac080b2ab0 0x7fac080b4fc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x555853e73c30 tx=0x7fac14009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.066+0000 7fac2ca77740 1 -- 172.21.15.2:0/3646941294 shutdown_connections 2024-05-14T09:56:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.066+0000 7fac2ca77740 1 -- 172.21.15.2:0/3646941294 wait complete. 2024-05-14T09:56:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.067+0000 7fac2ca77740 1 -- start start 2024-05-14T09:56:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.067+0000 7fac2ca77740 1 Processor -- start 2024-05-14T09:56:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.067+0000 7fac2ca77740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853aa82f0 0x555853e7c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.067+0000 7fac2ca77740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853ab04d0 0x555853e7cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.067+0000 7fac2ca77740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853ad11b0 0x555853e83be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.067+0000 7fac2ca77740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555853e7a930 con 0x555853ab04d0 2024-05-14T09:56:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.067+0000 7fac2ca77740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555853d94370 con 0x555853aa82f0 2024-05-14T09:56:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.067+0000 7fac2ca77740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555853e7aab0 con 0x555853ad11b0 2024-05-14T09:56:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.067+0000 7fac2ca77740 10 client.0 _ll_register_callbacks cb 0x555853ae8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:56:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.068+0000 7fac1ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853ad11b0 0x555853e83be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.068+0000 7fac1ffff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853ad11b0 0x555853e83be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:47166/0 (socket says 172.21.15.2:47166) 2024-05-14T09:56:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.068+0000 7fac1ffff640 1 -- 172.21.15.2:0/1982531513 learned_addr learned my addr 172.21.15.2:0/1982531513 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.068+0000 7fac1effd640 1 --2- 172.21.15.2:0/1982531513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853ab04d0 0x555853e7cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.068+0000 7fac1f7fe640 1 --2- 172.21.15.2:0/1982531513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853aa82f0 0x555853e7c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.068+0000 7fac1ffff640 1 -- 172.21.15.2:0/1982531513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853aa82f0 msgr2=0x555853e7c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.068+0000 7fac1ffff640 1 --2- 172.21.15.2:0/1982531513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853aa82f0 0x555853e7c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.068+0000 7fac1ffff640 1 -- 172.21.15.2:0/1982531513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853ab04d0 msgr2=0x555853e7cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.068+0000 7fac1ffff640 1 --2- 172.21.15.2:0/1982531513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853ab04d0 0x555853e7cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.068+0000 7fac1ffff640 1 -- 172.21.15.2:0/1982531513 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555853bb5620 con 0x555853ad11b0 2024-05-14T09:56:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.069+0000 7fac1ffff640 1 --2- 172.21.15.2:0/1982531513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853ad11b0 0x555853e83be0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fac14030fa0 tx=0x7fac1402a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3117f77b74c058f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.069+0000 7fabfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:56:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.069+0000 7fabfeffd640 1 -- 172.21.15.2:0/1982531513 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac14042070 con 0x555853ad11b0 2024-05-14T09:56:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.069+0000 7fabfeffd640 1 -- 172.21.15.2:0/1982531513 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fac14003ac0 con 0x555853ad11b0 2024-05-14T09:56:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.069+0000 7fac2ca77740 1 -- 172.21.15.2:0/1982531513 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac100047a0 con 0x555853ad11b0 2024-05-14T09:56:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.070+0000 7fabfeffd640 1 -- 172.21.15.2:0/1982531513 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fac14057050 con 0x555853ad11b0 2024-05-14T09:56:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.070+0000 7fac2ca77740 10 client.27052 fetch_fsmap learned FSMap version 67 2024-05-14T09:56:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.070+0000 7fac2ca77740 1 -- 172.21.15.2:0/1982531513 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x555853e86bd0 con 0x555853ad11b0 2024-05-14T09:56:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.070+0000 7fabfeffd640 1 -- 172.21.15.2:0/1982531513 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac100047a0 con 0x555853ad11b0 2024-05-14T09:56:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.071+0000 7fabfeffd640 1 -- 172.21.15.2:0/1982531513 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fac14049470 con 0x555853ad11b0 2024-05-14T09:56:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.071+0000 7fac2ca77740 10 client.27052 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:56:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.071+0000 7fac2ca77740 10 client.27052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:56:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.071+0000 7fac2ca77740 20 client.27052 populate_metadata read hostname 'smithi002' 2024-05-14T09:56:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.071+0000 7fac2ca77740 1 --2- 172.21.15.2:0/1982531513 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555853a8fd80 0x555853e8a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.071+0000 7fac2ca77740 4 client.27052 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:56:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.071+0000 7fac2ca77740 1 -- 172.21.15.2:0/1982531513 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555853de51e0 con 0x555853a8fd80 2024-05-14T09:56:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.072+0000 7fac1f7fe640 1 --2- 172.21.15.2:0/1982531513 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555853a8fd80 0x555853e8a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.073+0000 7fabfeffd640 10 client.27052 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.073+0000 7fac1f7fe640 1 --2- 172.21.15.2:0/1982531513 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555853a8fd80 0x555853e8a2d0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=442c1dd14803ddf0 server_cookie=93feb5e11eb51963 in_seq=0 out_seq=0 2024-05-14T09:56:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.078+0000 7fabfeffd640 1 -- 172.21.15.2:0/1982531513 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555853de51e0 con 0x555853a8fd80 2024-05-14T09:56:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.078+0000 7fabfeffd640 10 client.27052 handle_command_reply: tid=0 2024-05-14T09:56:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.078+0000 7fac2ca77740 1 client.27052 shutdown 2024-05-14T09:56:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.078+0000 7fac2ca77740 1 -- 172.21.15.2:0/1982531513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853ad11b0 msgr2=0x555853e83be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.078+0000 7fac2ca77740 1 --2- 172.21.15.2:0/1982531513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853ad11b0 0x555853e83be0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fac14030fa0 tx=0x7fac1402a340 comp rx=0 tx=0).stop 2024-05-14T09:56:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.079+0000 7fac2ca77740 1 -- 172.21.15.2:0/1982531513 shutdown_connections 2024-05-14T09:56:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.079+0000 7fac2ca77740 1 --2- 172.21.15.2:0/1982531513 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555853a8fd80 0x555853e8a2d0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.079+0000 7fac2ca77740 1 --2- 172.21.15.2:0/1982531513 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853ad11b0 0x555853e83be0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.079+0000 7fabfeffd640 0 client.27052 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.079+0000 7fac2ca77740 1 --2- 172.21.15.2:0/1982531513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853ab04d0 0x555853e7cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.079+0000 7fac2ca77740 1 --2- 172.21.15.2:0/1982531513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853aa82f0 0x555853e7c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.079+0000 7fac2ca77740 1 -- 172.21.15.2:0/1982531513 >> 172.21.15.2:0/1982531513 conn(0x555853ac5300 msgr2=0x555853e71300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.079+0000 7fac2ca77740 1 -- 172.21.15.2:0/1982531513 shutdown_connections 2024-05-14T09:56:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.079+0000 7fac2ca77740 1 -- 172.21.15.2:0/1982531513 wait complete. 2024-05-14T09:56:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.079+0000 7fac2ca77740 20 client.27052 trim_cache size 0 max 16384 2024-05-14T09:56:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.097+0000 7fac2ca77740 1 Processor -- start 2024-05-14T09:56:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.097+0000 7fac2ca77740 1 -- start start 2024-05-14T09:56:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.097+0000 7fac2ca77740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853a8fd80 0x555853e6e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.097+0000 7fac2ca77740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853aa82f0 0x555853e83b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.097+0000 7fac2ca77740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853ab04d0 0x555853e84040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.097+0000 7fac2ca77740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac200846d0 con 0x555853aa82f0 2024-05-14T09:56:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.097+0000 7fac2ca77740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fac20081fa0 con 0x555853ab04d0 2024-05-14T09:56:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.097+0000 7fac2ca77740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac2007fdf0 con 0x555853a8fd80 2024-05-14T09:56:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.097+0000 7fac1effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853aa82f0 0x555853e83b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.097+0000 7fac1effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853aa82f0 0x555853e83b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57040/0 (socket says 172.21.15.2:57040) 2024-05-14T09:56:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.097+0000 7fac1effd640 1 -- 172.21.15.2:0/2428720393 learned_addr learned my addr 172.21.15.2:0/2428720393 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.097+0000 7fac1ffff640 1 --2- 172.21.15.2:0/2428720393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853ab04d0 0x555853e84040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.097+0000 7fac1f7fe640 1 --2- 172.21.15.2:0/2428720393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853a8fd80 0x555853e6e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.098+0000 7fac1effd640 1 -- 172.21.15.2:0/2428720393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853a8fd80 msgr2=0x555853e6e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.098+0000 7fac1effd640 1 --2- 172.21.15.2:0/2428720393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853a8fd80 0x555853e6e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.098+0000 7fac1effd640 1 -- 172.21.15.2:0/2428720393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853ab04d0 msgr2=0x555853e84040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.098+0000 7fac1effd640 1 --2- 172.21.15.2:0/2428720393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853ab04d0 0x555853e84040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.098+0000 7fac1effd640 1 -- 172.21.15.2:0/2428720393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555853e7d7b0 con 0x555853aa82f0 2024-05-14T09:56:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.098+0000 7fac1effd640 1 --2- 172.21.15.2:0/2428720393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853aa82f0 0x555853e83b00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fac180a8d50 tx=0x7fac18009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee0fc254e10d723 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.098+0000 7fac057fa640 1 -- 172.21.15.2:0/2428720393 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac180c3070 con 0x555853aa82f0 2024-05-14T09:56:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.098+0000 7fac057fa640 1 -- 172.21.15.2:0/2428720393 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac180b1e00 con 0x555853aa82f0 2024-05-14T09:56:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.098+0000 7fac057fa640 1 -- 172.21.15.2:0/2428720393 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac180a82b0 con 0x555853aa82f0 2024-05-14T09:56:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.099+0000 7fac2ca77740 1 -- 172.21.15.2:0/2428720393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853aa82f0 msgr2=0x555853e83b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.099+0000 7fac2ca77740 1 --2- 172.21.15.2:0/2428720393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853aa82f0 0x555853e83b00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fac180a8d50 tx=0x7fac18009960 comp rx=0 tx=0).stop 2024-05-14T09:56:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.099+0000 7fac2ca77740 1 -- 172.21.15.2:0/2428720393 shutdown_connections 2024-05-14T09:56:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.099+0000 7fac2ca77740 1 --2- 172.21.15.2:0/2428720393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853ab04d0 0x555853e84040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.099+0000 7fac2ca77740 1 --2- 172.21.15.2:0/2428720393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853aa82f0 0x555853e83b00 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.099+0000 7fac2ca77740 1 --2- 172.21.15.2:0/2428720393 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853a8fd80 0x555853e6e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.099+0000 7fac2ca77740 1 -- 172.21.15.2:0/2428720393 >> 172.21.15.2:0/2428720393 conn(0x555853ac5300 msgr2=0x555853e6dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.099+0000 7fac2ca77740 1 -- 172.21.15.2:0/2428720393 shutdown_connections 2024-05-14T09:56:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.099+0000 7fac2ca77740 1 -- 172.21.15.2:0/2428720393 wait complete. 2024-05-14T09:56:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.100+0000 7fac2ca77740 1 -- start start 2024-05-14T09:56:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.100+0000 7fac2ca77740 1 Processor -- start 2024-05-14T09:56:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.100+0000 7fac2ca77740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853a8fd80 0x555853e81cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.100+0000 7fac2ca77740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853aa82f0 0x555853e82200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.100+0000 7fac2ca77740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853ab04d0 0x555853e830c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.100+0000 7fac2ca77740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac20080460 con 0x555853a8fd80 2024-05-14T09:56:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.100+0000 7fac2ca77740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fac200818d0 con 0x555853ab04d0 2024-05-14T09:56:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.100+0000 7fac2ca77740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fac20080b30 con 0x555853aa82f0 2024-05-14T09:56:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.100+0000 7fac2ca77740 10 client.0 _ll_register_callbacks cb 0x7fac10005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:56:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.100+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853a8fd80 0x555853e81cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.100+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853a8fd80 0x555853e81cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57052/0 (socket says 172.21.15.2:57052) 2024-05-14T09:56:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.100+0000 7fac1f7fe640 1 -- 172.21.15.2:0/1113168559 learned_addr learned my addr 172.21.15.2:0/1113168559 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.100+0000 7fac1ffff640 1 --2- 172.21.15.2:0/1113168559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853ab04d0 0x555853e830c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.101+0000 7fac1f7fe640 1 -- 172.21.15.2:0/1113168559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853aa82f0 msgr2=0x555853e82200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.101+0000 7fac1f7fe640 1 --2- 172.21.15.2:0/1113168559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853aa82f0 0x555853e82200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.101+0000 7fac1f7fe640 1 -- 172.21.15.2:0/1113168559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853ab04d0 msgr2=0x555853e830c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.101+0000 7fac1f7fe640 1 --2- 172.21.15.2:0/1113168559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853ab04d0 0x555853e830c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.101+0000 7fac1f7fe640 1 -- 172.21.15.2:0/1113168559 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555853e83680 con 0x555853a8fd80 2024-05-14T09:56:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.101+0000 7fac1f7fe640 1 --2- 172.21.15.2:0/1113168559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853a8fd80 0x555853e81cc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fac10005940 tx=0x7fac10009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09db42900ca7554 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.101+0000 7fabfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:56:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.101+0000 7fabfeffd640 1 -- 172.21.15.2:0/1113168559 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10004220 con 0x555853a8fd80 2024-05-14T09:56:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.101+0000 7fabfeffd640 1 -- 172.21.15.2:0/1113168559 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fac100043c0 con 0x555853a8fd80 2024-05-14T09:56:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.101+0000 7fac2ca77740 1 -- 172.21.15.2:0/1113168559 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac180b1e00 con 0x555853a8fd80 2024-05-14T09:56:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.101+0000 7fabfeffd640 1 -- 172.21.15.2:0/1113168559 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fac10005020 con 0x555853a8fd80 2024-05-14T09:56:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.101+0000 7fac2ca77740 10 client.17445 fetch_fsmap learned FSMap version 67 2024-05-14T09:56:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.101+0000 7fac2ca77740 1 -- 172.21.15.2:0/1113168559 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x555853ef2740 con 0x555853a8fd80 2024-05-14T09:56:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.102+0000 7fabfeffd640 1 -- 172.21.15.2:0/1113168559 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac10005670 con 0x555853a8fd80 2024-05-14T09:56:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.102+0000 7fabfeffd640 1 -- 172.21.15.2:0/1113168559 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fac1000f040 con 0x555853a8fd80 2024-05-14T09:56:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.102+0000 7fac2ca77740 10 client.17445 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:56:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.102+0000 7fac2ca77740 10 client.17445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:56:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.102+0000 7fac2ca77740 20 client.17445 populate_metadata read hostname 'smithi002' 2024-05-14T09:56:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.102+0000 7fac2ca77740 1 --2- 172.21.15.2:0/1113168559 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555853ad11b0 0x555853ef5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.102+0000 7fac2ca77740 4 client.17445 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:56:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.102+0000 7fac2ca77740 1 -- 172.21.15.2:0/1113168559 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fac180b1e00 con 0x555853ad11b0 2024-05-14T09:56:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.102+0000 7fac1effd640 1 --2- 172.21.15.2:0/1113168559 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555853ad11b0 0x555853ef5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.103+0000 7fabfeffd640 10 client.17445 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.103+0000 7fac1effd640 1 --2- 172.21.15.2:0/1113168559 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555853ad11b0 0x555853ef5020 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9528124c9ef4d427 server_cookie=fb489b164a5d2be5 in_seq=0 out_seq=0 2024-05-14T09:56:37.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 242928 26.18 MB/sec execute 363 sec latency 422.535 ms 2024-05-14T09:56:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.921+0000 7fabfeffd640 1 -- 172.21.15.2:0/1113168559 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fac180b1e00 con 0x555853ad11b0 2024-05-14T09:56:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.922+0000 7fabfeffd640 10 client.17445 handle_command_reply: tid=0 2024-05-14T09:56:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.922+0000 7fac2ca77740 1 client.17445 shutdown 2024-05-14T09:56:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.922+0000 7fac2ca77740 1 -- 172.21.15.2:0/1113168559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853a8fd80 msgr2=0x555853e81cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.922+0000 7fac2ca77740 1 --2- 172.21.15.2:0/1113168559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853a8fd80 0x555853e81cc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fac10005940 tx=0x7fac10009b60 comp rx=0 tx=0).stop 2024-05-14T09:56:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.922+0000 7fac2ca77740 1 -- 172.21.15.2:0/1113168559 shutdown_connections 2024-05-14T09:56:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.922+0000 7fac2ca77740 1 --2- 172.21.15.2:0/1113168559 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555853ad11b0 0x555853ef5020 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.922+0000 7fac2ca77740 1 --2- 172.21.15.2:0/1113168559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555853ab04d0 0x555853e830c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.922+0000 7fac2ca77740 1 --2- 172.21.15.2:0/1113168559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555853aa82f0 0x555853e82200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.922+0000 7fabfeffd640 0 client.17445 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.922+0000 7fac2ca77740 1 --2- 172.21.15.2:0/1113168559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555853a8fd80 0x555853e81cc0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.922+0000 7fac2ca77740 1 -- 172.21.15.2:0/1113168559 >> 172.21.15.2:0/1113168559 conn(0x555853ac5300 msgr2=0x555853e8eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.922+0000 7fac2ca77740 1 -- 172.21.15.2:0/1113168559 shutdown_connections 2024-05-14T09:56:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.922+0000 7fac2ca77740 1 -- 172.21.15.2:0/1113168559 wait complete. 2024-05-14T09:56:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.922+0000 7fac2ca77740 20 client.17445 trim_cache size 0 max 16384 2024-05-14T09:56:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:56:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:56:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:56:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:56:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.923+0000 7fac25bc5640 1 -- 172.21.15.2:0/1648175426 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fac080b2ab0 msgr2=0x7fac080b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.923+0000 7fac25bc5640 1 --2- 172.21.15.2:0/1648175426 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fac080b2ab0 0x7fac080b4fc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x555853e73c30 tx=0x7fac14009520 comp rx=0 tx=0).stop 2024-05-14T09:56:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.923+0000 7fac25bc5640 1 -- 172.21.15.2:0/1648175426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2010b480 msgr2=0x7fac201a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.923+0000 7fac25bc5640 1 --2- 172.21.15.2:0/1648175426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2010b480 0x7fac201a5250 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fac1800bd70 tx=0x7fac1800be60 comp rx=0 tx=0).stop 2024-05-14T09:56:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.923+0000 7fac25bc5640 1 -- 172.21.15.2:0/1648175426 shutdown_connections 2024-05-14T09:56:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.923+0000 7fac25bc5640 1 --2- 172.21.15.2:0/1648175426 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fac080b2ab0 0x7fac080b4fc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.923+0000 7fac25bc5640 1 --2- 172.21.15.2:0/1648175426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac2010c180 0x7fac2007af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.923+0000 7fac25bc5640 1 --2- 172.21.15.2:0/1648175426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2010b480 0x7fac201a5250 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.923+0000 7fac25bc5640 1 --2- 172.21.15.2:0/1648175426 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac2010a550 0x7fac201a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.923+0000 7fac25bc5640 1 -- 172.21.15.2:0/1648175426 >> 172.21.15.2:0/1648175426 conn(0x7fac20070640 msgr2=0x7fac20072a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.923+0000 7fac25bc5640 1 -- 172.21.15.2:0/1648175426 shutdown_connections 2024-05-14T09:56:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:37.923+0000 7fac25bc5640 1 -- 172.21.15.2:0/1648175426 wait complete. 2024-05-14T09:56:38.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 243441 26.19 MB/sec execute 364 sec latency 561.712 ms 2024-05-14T09:56:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:38 smithi002 ceph-mon[31557]: pgmap v1077: 129 pgs: 129 active+clean; 697 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.3 MiB/s wr, 454 op/s 2024-05-14T09:56:38.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:38 smithi029 ceph-mon[36249]: pgmap v1077: 129 pgs: 129 active+clean; 697 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.3 MiB/s wr, 454 op/s 2024-05-14T09:56:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:38 smithi026 ceph-mon[36056]: pgmap v1077: 129 pgs: 129 active+clean; 697 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.3 MiB/s wr, 454 op/s 2024-05-14T09:56:39.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 244151 26.21 MB/sec execute 365 sec latency 241.761 ms 2024-05-14T09:56:40.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 244890 26.27 MB/sec execute 366 sec latency 85.974 ms 2024-05-14T09:56:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:40 smithi002 ceph-mon[31557]: pgmap v1078: 129 pgs: 129 active+clean; 693 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.9 MiB/s wr, 407 op/s 2024-05-14T09:56:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:40 smithi029 ceph-mon[36249]: pgmap v1078: 129 pgs: 129 active+clean; 693 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.9 MiB/s wr, 407 op/s 2024-05-14T09:56:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:40 smithi026 ceph-mon[36056]: pgmap v1078: 129 pgs: 129 active+clean; 693 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.9 MiB/s wr, 407 op/s 2024-05-14T09:56:41.620 INFO:tasks.workunit.client.0.smithi002.stdout: 10 245482 26.31 MB/sec execute 367 sec latency 210.679 ms 2024-05-14T09:56:42.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 245960 26.30 MB/sec execute 368 sec latency 83.333 ms 2024-05-14T09:56:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:42 smithi002 ceph-mon[31557]: pgmap v1079: 129 pgs: 129 active+clean; 690 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.7 MiB/s wr, 387 op/s 2024-05-14T09:56:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:42 smithi029 ceph-mon[36249]: pgmap v1079: 129 pgs: 129 active+clean; 690 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.7 MiB/s wr, 387 op/s 2024-05-14T09:56:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:42 smithi026 ceph-mon[36056]: pgmap v1079: 129 pgs: 129 active+clean; 690 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.7 MiB/s wr, 387 op/s 2024-05-14T09:56:43.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 246366 26.27 MB/sec execute 369 sec latency 100.820 ms 2024-05-14T09:56:44.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 246849 26.26 MB/sec execute 370 sec latency 274.669 ms 2024-05-14T09:56:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:44 smithi002 ceph-mon[31557]: pgmap v1080: 129 pgs: 129 active+clean; 671 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.5 MiB/s wr, 534 op/s 2024-05-14T09:56:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:44 smithi029 ceph-mon[36249]: pgmap v1080: 129 pgs: 129 active+clean; 671 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.5 MiB/s wr, 534 op/s 2024-05-14T09:56:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:44 smithi026 ceph-mon[36056]: pgmap v1080: 129 pgs: 129 active+clean; 671 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.5 MiB/s wr, 534 op/s 2024-05-14T09:56:45.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 247596 26.28 MB/sec execute 371 sec latency 79.314 ms 2024-05-14T09:56:46.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 248311 26.35 MB/sec execute 372 sec latency 127.336 ms 2024-05-14T09:56:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:46 smithi002 ceph-mon[31557]: pgmap v1081: 129 pgs: 129 active+clean; 675 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.8 MiB/s wr, 445 op/s 2024-05-14T09:56:46.936 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:46 smithi029 ceph-mon[36249]: pgmap v1081: 129 pgs: 129 active+clean; 675 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.8 MiB/s wr, 445 op/s 2024-05-14T09:56:46.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:46 smithi026 ceph-mon[36056]: pgmap v1081: 129 pgs: 129 active+clean; 675 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.8 MiB/s wr, 445 op/s 2024-05-14T09:56:47.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 248918 26.37 MB/sec execute 373 sec latency 125.041 ms 2024-05-14T09:56:48.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 249392 26.37 MB/sec execute 374 sec latency 67.398 ms 2024-05-14T09:56:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:48 smithi002 ceph-mon[31557]: pgmap v1082: 129 pgs: 129 active+clean; 680 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.3 MiB/s wr, 503 op/s 2024-05-14T09:56:48.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:48 smithi029 ceph-mon[36249]: pgmap v1082: 129 pgs: 129 active+clean; 680 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.3 MiB/s wr, 503 op/s 2024-05-14T09:56:48.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:48 smithi026 ceph-mon[36056]: pgmap v1082: 129 pgs: 129 active+clean; 680 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.3 MiB/s wr, 503 op/s 2024-05-14T09:56:49.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 249800 26.35 MB/sec execute 375 sec latency 84.755 ms 2024-05-14T09:56:50.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 250387 26.37 MB/sec execute 376 sec latency 81.671 ms 2024-05-14T09:56:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:50 smithi002 ceph-mon[31557]: pgmap v1083: 129 pgs: 129 active+clean; 686 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.1 MiB/s wr, 495 op/s 2024-05-14T09:56:50.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:50 smithi029 ceph-mon[36249]: pgmap v1083: 129 pgs: 129 active+clean; 686 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.1 MiB/s wr, 495 op/s 2024-05-14T09:56:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:50 smithi026 ceph-mon[36056]: pgmap v1083: 129 pgs: 129 active+clean; 686 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.1 MiB/s wr, 495 op/s 2024-05-14T09:56:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.093+0000 7fcf4f59e640 1 -- 172.21.15.2:0/2932220573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf50103c00 msgr2=0x7fcf50104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.093+0000 7fcf4f59e640 1 --2- 172.21.15.2:0/2932220573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf50103c00 0x7fcf50104010 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009900 tx=0x7fcf3802a410 comp rx=0 tx=0).stop 2024-05-14T09:56:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.093+0000 7fcf4f59e640 1 -- 172.21.15.2:0/2932220573 shutdown_connections 2024-05-14T09:56:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.093+0000 7fcf4f59e640 1 --2- 172.21.15.2:0/2932220573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf50108900 0x7fcf5010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.093+0000 7fcf4f59e640 1 --2- 172.21.15.2:0/2932220573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf50103c00 0x7fcf50104010 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.093+0000 7fcf4f59e640 1 --2- 172.21.15.2:0/2932220573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf501029c0 0x7fcf501036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.093+0000 7fcf4f59e640 1 -- 172.21.15.2:0/2932220573 >> 172.21.15.2:0/2932220573 conn(0x7fcf500fdbb0 msgr2=0x7fcf50100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.093+0000 7fcf4f59e640 1 -- 172.21.15.2:0/2932220573 shutdown_connections 2024-05-14T09:56:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.093+0000 7fcf4f59e640 1 -- 172.21.15.2:0/2932220573 wait complete. 2024-05-14T09:56:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4f59e640 1 Processor -- start 2024-05-14T09:56:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4f59e640 1 -- start start 2024-05-14T09:56:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4f59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf501029c0 0x7fcf5019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf50103c00 0x7fcf5019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4f59e640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf50108900 0x7fcf501a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4f59e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5010f680 con 0x7fcf501029c0 2024-05-14T09:56:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4f59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5010f500 con 0x7fcf50103c00 2024-05-14T09:56:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4f59e640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5010f800 con 0x7fcf50108900 2024-05-14T09:56:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf501029c0 0x7fcf5019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf501029c0 0x7fcf5019a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55400/0 (socket says 172.21.15.2:55400) 2024-05-14T09:56:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4e59c640 1 -- 172.21.15.2:0/362558191 learned_addr learned my addr 172.21.15.2:0/362558191 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4dd9b640 1 --2- 172.21.15.2:0/362558191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf50103c00 0x7fcf5019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4ed9d640 1 --2- 172.21.15.2:0/362558191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf50108900 0x7fcf501a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4e59c640 1 -- 172.21.15.2:0/362558191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf50108900 msgr2=0x7fcf501a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4e59c640 1 --2- 172.21.15.2:0/362558191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf50108900 0x7fcf501a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4e59c640 1 -- 172.21.15.2:0/362558191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf50103c00 msgr2=0x7fcf5019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4e59c640 1 --2- 172.21.15.2:0/362558191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf50103c00 0x7fcf5019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.094+0000 7fcf4e59c640 1 -- 172.21.15.2:0/362558191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf501a1ce0 con 0x7fcf501029c0 2024-05-14T09:56:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.095+0000 7fcf4e59c640 1 --2- 172.21.15.2:0/362558191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf501029c0 0x7fcf5019a1a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800eb50 tx=0x7fcf4800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.095+0000 7fcf477fe640 1 -- 172.21.15.2:0/362558191 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48003b70 con 0x7fcf501029c0 2024-05-14T09:56:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.095+0000 7fcf477fe640 1 -- 172.21.15.2:0/362558191 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf4800ee50 con 0x7fcf501029c0 2024-05-14T09:56:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.095+0000 7fcf477fe640 1 -- 172.21.15.2:0/362558191 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48003b70 con 0x7fcf501029c0 2024-05-14T09:56:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.095+0000 7fcf4f59e640 1 -- 172.21.15.2:0/362558191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf501a1fd0 con 0x7fcf501029c0 2024-05-14T09:56:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.095+0000 7fcf4f59e640 1 -- 172.21.15.2:0/362558191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf501a23e0 con 0x7fcf501029c0 2024-05-14T09:56:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.099+0000 7fcf477fe640 1 -- 172.21.15.2:0/362558191 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fcf48004020 con 0x7fcf501029c0 2024-05-14T09:56:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.100+0000 7fcf5bbd0740 1 Processor -- start 2024-05-14T09:56:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.100+0000 7fcf5bbd0740 1 -- start start 2024-05-14T09:56:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.101+0000 7fcf5bbd0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97a81b0 0x55e6d9b4d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.101+0000 7fcf5bbd0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b4d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.101+0000 7fcf5bbd0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97874d0 0x55e6d9b49cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.101+0000 7fcf5bbd0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e6d9aae5d0 con 0x55e6d977f2f0 2024-05-14T09:56:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.101+0000 7fcf477fe640 1 --2- 172.21.15.2:0/362558191 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcf3c0b2ca0 0x7fcf3c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.101+0000 7fcf5bbd0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e6d9a67890 con 0x55e6d97874d0 2024-05-14T09:56:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.101+0000 7fcf5bbd0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e6d9aaa590 con 0x55e6d97a81b0 2024-05-14T09:56:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.101+0000 7fcf4ed9d640 1 --2- 172.21.15.2:0/362558191 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcf3c0b2ca0 0x7fcf3c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.101+0000 7fcf4dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97874d0 0x55e6d9b49cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.101+0000 7fcf4dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97874d0 0x55e6d9b49cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:34168/0 (socket says 172.21.15.2:34168) 2024-05-14T09:56:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.101+0000 7fcf477fe640 1 -- 172.21.15.2:0/362558191 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fcf48003970 con 0x7fcf501029c0 2024-05-14T09:56:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.101+0000 7fcf4ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97a81b0 0x55e6d9b4d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.101+0000 7fcf4e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b4d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.101+0000 7fcf4dd9b640 1 -- 172.21.15.2:0/3642516474 learned_addr learned my addr 172.21.15.2:0/3642516474 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.102+0000 7fcf4dd9b640 1 -- 172.21.15.2:0/3642516474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97a81b0 msgr2=0x55e6d9b4d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.102+0000 7fcf4dd9b640 1 --2- 172.21.15.2:0/3642516474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97a81b0 0x55e6d9b4d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.102+0000 7fcf4dd9b640 1 -- 172.21.15.2:0/3642516474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 msgr2=0x55e6d9b4d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.102+0000 7fcf4dd9b640 1 --2- 172.21.15.2:0/3642516474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b4d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.102+0000 7fcf4dd9b640 1 -- 172.21.15.2:0/3642516474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6d97c0210 con 0x55e6d97874d0 2024-05-14T09:56:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.102+0000 7fcf4e59c640 1 --2- 172.21.15.2:0/3642516474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b4d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:56:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.102+0000 7fcf4dd9b640 1 --2- 172.21.15.2:0/3642516474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97874d0 0x55e6d9b49cc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcf38031500 tx=0x7fcf38009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d58860a502e3ca37 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.102+0000 7fcf4ed9d640 1 --2- 172.21.15.2:0/362558191 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcf3c0b2ca0 0x7fcf3c0b51b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x55e6d9b4ad80 tx=0x7fcf40009910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.102+0000 7fcf44ff9640 1 -- 172.21.15.2:0/3642516474 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38035030 con 0x55e6d97874d0 2024-05-14T09:56:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.102+0000 7fcf44ff9640 1 -- 172.21.15.2:0/3642516474 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf38040070 con 0x55e6d97874d0 2024-05-14T09:56:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.102+0000 7fcf44ff9640 1 -- 172.21.15.2:0/3642516474 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38003970 con 0x55e6d97874d0 2024-05-14T09:56:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.103+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3642516474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97874d0 msgr2=0x55e6d9b49cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.103+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/3642516474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97874d0 0x55e6d9b49cc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcf38031500 tx=0x7fcf38009640 comp rx=0 tx=0).stop 2024-05-14T09:56:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.103+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3642516474 shutdown_connections 2024-05-14T09:56:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.103+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/3642516474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97874d0 0x55e6d9b49cc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.103+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/3642516474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b4d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.103+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/3642516474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97a81b0 0x55e6d9b4d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.103+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3642516474 >> 172.21.15.2:0/3642516474 conn(0x55e6d979c300 msgr2=0x55e6d9b455c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.103+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3642516474 shutdown_connections 2024-05-14T09:56:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.103+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3642516474 wait complete. 2024-05-14T09:56:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.104+0000 7fcf5bbd0740 1 -- start start 2024-05-14T09:56:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.104+0000 7fcf5bbd0740 1 Processor -- start 2024-05-14T09:56:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.104+0000 7fcf5bbd0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d977f2f0 0x55e6d9b5c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.104+0000 7fcf5bbd0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97874d0 0x55e6d9b5c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.104+0000 7fcf5bbd0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d97a81b0 0x55e6d9b634e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.104+0000 7fcf5bbd0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e6d9b4a760 con 0x55e6d97a81b0 2024-05-14T09:56:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.104+0000 7fcf5bbd0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e6d9a6b370 con 0x55e6d977f2f0 2024-05-14T09:56:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.104+0000 7fcf5bbd0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55e6d9b5a4c0 con 0x55e6d97874d0 2024-05-14T09:56:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.104+0000 7fcf5bbd0740 10 client.0 _ll_register_callbacks cb 0x55e6d97bfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:56:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.104+0000 7fcf4e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d977f2f0 0x55e6d9b5c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.104+0000 7fcf4e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d977f2f0 0x55e6d9b5c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:34174/0 (socket says 172.21.15.2:34174) 2024-05-14T09:56:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.104+0000 7fcf4e59c640 1 -- 172.21.15.2:0/1965087303 learned_addr learned my addr 172.21.15.2:0/1965087303 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.104+0000 7fcf4dd9b640 1 --2- 172.21.15.2:0/1965087303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97874d0 0x55e6d9b5c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.105+0000 7fcf4ed9d640 1 --2- 172.21.15.2:0/1965087303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d97a81b0 0x55e6d9b634e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.105+0000 7fcf4e59c640 1 -- 172.21.15.2:0/1965087303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97874d0 msgr2=0x55e6d9b5c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.105+0000 7fcf4e59c640 1 --2- 172.21.15.2:0/1965087303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97874d0 0x55e6d9b5c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.105+0000 7fcf4e59c640 1 -- 172.21.15.2:0/1965087303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d97a81b0 msgr2=0x55e6d9b634e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.105+0000 7fcf4e59c640 1 --2- 172.21.15.2:0/1965087303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d97a81b0 0x55e6d9b634e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.105+0000 7fcf4e59c640 1 -- 172.21.15.2:0/1965087303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6d988c620 con 0x55e6d977f2f0 2024-05-14T09:56:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.105+0000 7fcf4e59c640 1 --2- 172.21.15.2:0/1965087303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d977f2f0 0x55e6d9b5c040 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcf480a5a80 tx=0x7fcf480a9510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6100c5286f1e214e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.105+0000 7fcf35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:56:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.105+0000 7fcf35ffb640 1 -- 172.21.15.2:0/1965087303 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf480c3060 con 0x55e6d977f2f0 2024-05-14T09:56:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.105+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/1965087303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf38040070 con 0x55e6d977f2f0 2024-05-14T09:56:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.105+0000 7fcf35ffb640 1 -- 172.21.15.2:0/1965087303 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fcf480cc9a0 con 0x55e6d977f2f0 2024-05-14T09:56:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.105+0000 7fcf35ffb640 1 -- 172.21.15.2:0/1965087303 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fcf480addb0 con 0x55e6d977f2f0 2024-05-14T09:56:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.105+0000 7fcf5bbd0740 10 client.27074 fetch_fsmap learned FSMap version 67 2024-05-14T09:56:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.106+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/1965087303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55e6d9b66480 con 0x55e6d977f2f0 2024-05-14T09:56:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.106+0000 7fcf35ffb640 1 -- 172.21.15.2:0/1965087303 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf38040070 con 0x55e6d977f2f0 2024-05-14T09:56:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.106+0000 7fcf35ffb640 1 -- 172.21.15.2:0/1965087303 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fcf480c5020 con 0x55e6d977f2f0 2024-05-14T09:56:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.107+0000 7fcf5bbd0740 10 client.27074 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:56:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.107+0000 7fcf5bbd0740 10 client.27074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:56:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.107+0000 7fcf5bbd0740 20 client.27074 populate_metadata read hostname 'smithi002' 2024-05-14T09:56:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.107+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/1965087303 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e6d9766d80 0x55e6d9b69b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.107+0000 7fcf5bbd0740 4 client.27074 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:56:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.107+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/1965087303 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6d9abc1e0 con 0x55e6d9766d80 2024-05-14T09:56:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.107+0000 7fcf4dd9b640 1 --2- 172.21.15.2:0/1965087303 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e6d9766d80 0x55e6d9b69b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.107+0000 7fcf4dd9b640 1 --2- 172.21.15.2:0/1965087303 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e6d9766d80 0x55e6d9b69b80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=960622e581a3ec45 server_cookie=67a554308fe0da14 in_seq=0 out_seq=0 2024-05-14T09:56:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.107+0000 7fcf35ffb640 10 client.27074 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.111+0000 7fcf35ffb640 1 -- 172.21.15.2:0/1965087303 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e6d9abc1e0 con 0x55e6d9766d80 2024-05-14T09:56:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.111+0000 7fcf35ffb640 10 client.27074 handle_command_reply: tid=0 2024-05-14T09:56:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.111+0000 7fcf5bbd0740 1 client.27074 shutdown 2024-05-14T09:56:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.111+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/1965087303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d977f2f0 msgr2=0x55e6d9b5c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.111+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/1965087303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d977f2f0 0x55e6d9b5c040 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcf480a5a80 tx=0x7fcf480a9510 comp rx=0 tx=0).stop 2024-05-14T09:56:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.111+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/1965087303 shutdown_connections 2024-05-14T09:56:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.111+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/1965087303 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e6d9766d80 0x55e6d9b69b80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.112+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/1965087303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d97a81b0 0x55e6d9b634e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.112+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/1965087303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97874d0 0x55e6d9b5c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.112+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/1965087303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d977f2f0 0x55e6d9b5c040 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.112+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/1965087303 >> 172.21.15.2:0/1965087303 conn(0x55e6d979c300 msgr2=0x55e6d9b44580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.112+0000 7fcf35ffb640 0 client.27074 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.112+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/1965087303 shutdown_connections 2024-05-14T09:56:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.112+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/1965087303 wait complete. 2024-05-14T09:56:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.112+0000 7fcf5bbd0740 20 client.27074 trim_cache size 0 max 16384 2024-05-14T09:56:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.133+0000 7fcf5bbd0740 1 Processor -- start 2024-05-14T09:56:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.133+0000 7fcf5bbd0740 1 -- start start 2024-05-14T09:56:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.133+0000 7fcf5bbd0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b4d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.133+0000 7fcf5bbd0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97874d0 0x55e6d9b44ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.133+0000 7fcf5bbd0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97a81b0 0x55e6d9b45410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.133+0000 7fcf5bbd0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcf501ad160 con 0x55e6d977f2f0 2024-05-14T09:56:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.133+0000 7fcf5bbd0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcf501aaa30 con 0x55e6d97874d0 2024-05-14T09:56:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.133+0000 7fcf5bbd0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcf501a88a0 con 0x55e6d97a81b0 2024-05-14T09:56:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.133+0000 7fcf4e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b4d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.133+0000 7fcf4ed9d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97a81b0 0x55e6d9b45410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.133+0000 7fcf4e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b4d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55436/0 (socket says 172.21.15.2:55436) 2024-05-14T09:56:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.133+0000 7fcf4e59c640 1 -- 172.21.15.2:0/3130747527 learned_addr learned my addr 172.21.15.2:0/3130747527 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.134+0000 7fcf4dd9b640 1 --2- 172.21.15.2:0/3130747527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97874d0 0x55e6d9b44ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.134+0000 7fcf4ed9d640 1 -- 172.21.15.2:0/3130747527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97874d0 msgr2=0x55e6d9b44ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.134+0000 7fcf4ed9d640 1 --2- 172.21.15.2:0/3130747527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97874d0 0x55e6d9b44ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.134+0000 7fcf4ed9d640 1 -- 172.21.15.2:0/3130747527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 msgr2=0x55e6d9b4d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.134+0000 7fcf4ed9d640 1 --2- 172.21.15.2:0/3130747527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b4d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.134+0000 7fcf4ed9d640 1 -- 172.21.15.2:0/3130747527 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6d9b5cc40 con 0x55e6d97a81b0 2024-05-14T09:56:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.134+0000 7fcf4e59c640 1 --2- 172.21.15.2:0/3130747527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b4d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:56:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.134+0000 7fcf4ed9d640 1 --2- 172.21.15.2:0/3130747527 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97a81b0 0x55e6d9b45410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009700 tx=0x7fcf40005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2540308a9d991c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.135+0000 7fcf37fff640 1 -- 172.21.15.2:0/3130747527 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4002d070 con 0x55e6d97a81b0 2024-05-14T09:56:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.135+0000 7fcf37fff640 1 -- 172.21.15.2:0/3130747527 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf4000fe70 con 0x55e6d97a81b0 2024-05-14T09:56:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.135+0000 7fcf37fff640 1 -- 172.21.15.2:0/3130747527 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40028d60 con 0x55e6d97a81b0 2024-05-14T09:56:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.136+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3130747527 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97a81b0 msgr2=0x55e6d9b45410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.136+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/3130747527 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97a81b0 0x55e6d9b45410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009700 tx=0x7fcf40005f90 comp rx=0 tx=0).stop 2024-05-14T09:56:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.136+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3130747527 shutdown_connections 2024-05-14T09:56:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.136+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/3130747527 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97a81b0 0x55e6d9b45410 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.136+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/3130747527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97874d0 0x55e6d9b44ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.136+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/3130747527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b4d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.136+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3130747527 >> 172.21.15.2:0/3130747527 conn(0x55e6d979c300 msgr2=0x55e6d9b4cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.136+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3130747527 shutdown_connections 2024-05-14T09:56:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.136+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3130747527 wait complete. 2024-05-14T09:56:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.137+0000 7fcf5bbd0740 1 -- start start 2024-05-14T09:56:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.137+0000 7fcf5bbd0740 1 Processor -- start 2024-05-14T09:56:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.137+0000 7fcf5bbd0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b5fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.137+0000 7fcf5bbd0740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97874d0 0x55e6d9b60300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.137+0000 7fcf5bbd0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97a81b0 0x55e6d9b61230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf5bbd0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcf501a8ef0 con 0x55e6d977f2f0 2024-05-14T09:56:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf5bbd0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcf501aa360 con 0x55e6d97a81b0 2024-05-14T09:56:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf5bbd0740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcf501a95c0 con 0x55e6d97874d0 2024-05-14T09:56:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf5bbd0740 10 client.0 _ll_register_callbacks cb 0x7fcf480a9170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:56:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf4e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b5fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf4e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b5fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55446/0 (socket says 172.21.15.2:55446) 2024-05-14T09:56:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf4e59c640 1 -- 172.21.15.2:0/3298597898 learned_addr learned my addr 172.21.15.2:0/3298597898 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:56:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf4dd9b640 1 --2- 172.21.15.2:0/3298597898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97874d0 0x55e6d9b60300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf4ed9d640 1 --2- 172.21.15.2:0/3298597898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97a81b0 0x55e6d9b61230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf4e59c640 1 -- 172.21.15.2:0/3298597898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97874d0 msgr2=0x55e6d9b60300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf4e59c640 1 --2- 172.21.15.2:0/3298597898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97874d0 0x55e6d9b60300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf4e59c640 1 -- 172.21.15.2:0/3298597898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97a81b0 msgr2=0x55e6d9b61230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:56:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf4e59c640 1 --2- 172.21.15.2:0/3298597898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97a81b0 0x55e6d9b61230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf4e59c640 1 -- 172.21.15.2:0/3298597898 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6d9b617f0 con 0x55e6d977f2f0 2024-05-14T09:56:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf4e59c640 1 --2- 172.21.15.2:0/3298597898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b5fdc0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800eb20 tx=0x7fcf480045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a2b75858df193b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:56:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.138+0000 7fcf35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:56:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.139+0000 7fcf35ffb640 1 -- 172.21.15.2:0/3298597898 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48089060 con 0x55e6d977f2f0 2024-05-14T09:56:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.139+0000 7fcf35ffb640 1 -- 172.21.15.2:0/3298597898 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fcf480a8e70 con 0x55e6d977f2f0 2024-05-14T09:56:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.139+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3298597898 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf4000fe70 con 0x55e6d977f2f0 2024-05-14T09:56:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.139+0000 7fcf35ffb640 1 -- 172.21.15.2:0/3298597898 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fcf480c3070 con 0x55e6d977f2f0 2024-05-14T09:56:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.139+0000 7fcf5bbd0740 10 client.17463 fetch_fsmap learned FSMap version 67 2024-05-14T09:56:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.139+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3298597898 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55e6d9b62ae0 con 0x55e6d977f2f0 2024-05-14T09:56:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.139+0000 7fcf35ffb640 1 -- 172.21.15.2:0/3298597898 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf4000fe70 con 0x55e6d977f2f0 2024-05-14T09:56:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.140+0000 7fcf35ffb640 1 -- 172.21.15.2:0/3298597898 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fcf480c5030 con 0x55e6d977f2f0 2024-05-14T09:56:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.140+0000 7fcf5bbd0740 10 client.17463 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:56:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.140+0000 7fcf5bbd0740 10 client.17463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:56:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.140+0000 7fcf5bbd0740 20 client.17463 populate_metadata read hostname 'smithi002' 2024-05-14T09:56:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.140+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/3298597898 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e6d973c1c0 0x55e6d9bd42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:56:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.140+0000 7fcf5bbd0740 4 client.17463 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:56:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.140+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3298597898 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6d9ab6050 con 0x55e6d973c1c0 2024-05-14T09:56:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.140+0000 7fcf4dd9b640 1 --2- 172.21.15.2:0/3298597898 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e6d973c1c0 0x55e6d9bd42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:56:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.140+0000 7fcf35ffb640 10 client.17463 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:51.141+0000 7fcf4dd9b640 1 --2- 172.21.15.2:0/3298597898 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e6d973c1c0 0x55e6d9bd42e0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caede796664a55a6 server_cookie=ce7c5caf5363fd86 in_seq=0 out_seq=0 2024-05-14T09:56:51.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 250755 26.32 MB/sec execute 377 sec latency 435.713 ms 2024-05-14T09:56:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.243+0000 7fcf35ffb640 1 -- 172.21.15.2:0/3298597898 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6d9ab6050 con 0x55e6d973c1c0 2024-05-14T09:56:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.243+0000 7fcf35ffb640 10 client.17463 handle_command_reply: tid=0 2024-05-14T09:56:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.243+0000 7fcf5bbd0740 1 client.17463 shutdown 2024-05-14T09:56:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.243+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3298597898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 msgr2=0x55e6d9b5fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.243+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/3298597898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b5fdc0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800eb20 tx=0x7fcf480045c0 comp rx=0 tx=0).stop 2024-05-14T09:56:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.243+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3298597898 shutdown_connections 2024-05-14T09:56:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.243+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/3298597898 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55e6d973c1c0 0x55e6d9bd42e0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.243+0000 7fcf35ffb640 0 client.17463 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:56:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.243+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/3298597898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e6d97a81b0 0x55e6d9b61230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.243+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/3298597898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e6d97874d0 0x55e6d9b60300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.243+0000 7fcf5bbd0740 1 --2- 172.21.15.2:0/3298597898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d977f2f0 0x55e6d9b5fdc0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.243+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3298597898 >> 172.21.15.2:0/3298597898 conn(0x55e6d979c300 msgr2=0x55e6d9b59e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.243+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3298597898 shutdown_connections 2024-05-14T09:56:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.243+0000 7fcf5bbd0740 1 -- 172.21.15.2:0/3298597898 wait complete. 2024-05-14T09:56:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.243+0000 7fcf5bbd0740 20 client.17463 trim_cache size 0 max 16384 2024-05-14T09:56:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:56:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:56:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:56:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:56:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.244+0000 7fcf4f59e640 1 -- 172.21.15.2:0/362558191 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcf3c0b2ca0 msgr2=0x7fcf3c0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.244+0000 7fcf4f59e640 1 --2- 172.21.15.2:0/362558191 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcf3c0b2ca0 0x7fcf3c0b51b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x55e6d9b4ad80 tx=0x7fcf40009910 comp rx=0 tx=0).stop 2024-05-14T09:56:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.244+0000 7fcf4f59e640 1 -- 172.21.15.2:0/362558191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf501029c0 msgr2=0x7fcf5019a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:56:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.244+0000 7fcf4f59e640 1 --2- 172.21.15.2:0/362558191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf501029c0 0x7fcf5019a1a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800eb50 tx=0x7fcf4800c6a0 comp rx=0 tx=0).stop 2024-05-14T09:56:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.244+0000 7fcf4f59e640 1 -- 172.21.15.2:0/362558191 shutdown_connections 2024-05-14T09:56:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.244+0000 7fcf4f59e640 1 --2- 172.21.15.2:0/362558191 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcf3c0b2ca0 0x7fcf3c0b51b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.244+0000 7fcf4f59e640 1 --2- 172.21.15.2:0/362558191 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcf50108900 0x7fcf501a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.244+0000 7fcf4f59e640 1 --2- 172.21.15.2:0/362558191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf50103c00 0x7fcf5019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.244+0000 7fcf4f59e640 1 --2- 172.21.15.2:0/362558191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf501029c0 0x7fcf5019a1a0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:56:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.244+0000 7fcf4f59e640 1 -- 172.21.15.2:0/362558191 >> 172.21.15.2:0/362558191 conn(0x7fcf500fdbb0 msgr2=0x7fcf501048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:56:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.244+0000 7fcf4f59e640 1 -- 172.21.15.2:0/362558191 shutdown_connections 2024-05-14T09:56:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:56:52.244+0000 7fcf4f59e640 1 -- 172.21.15.2:0/362558191 wait complete. 2024-05-14T09:56:52.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 251051 26.30 MB/sec execute 378 sec latency 579.716 ms 2024-05-14T09:56:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:52 smithi029 ceph-mon[36249]: pgmap v1084: 129 pgs: 129 active+clean; 687 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.8 MiB/s wr, 415 op/s 2024-05-14T09:56:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:52 smithi026 ceph-mon[36056]: pgmap v1084: 129 pgs: 129 active+clean; 687 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.8 MiB/s wr, 415 op/s 2024-05-14T09:56:53.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:52 smithi002 ceph-mon[31557]: pgmap v1084: 129 pgs: 129 active+clean; 687 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.8 MiB/s wr, 415 op/s 2024-05-14T09:56:53.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 251594 26.32 MB/sec execute 379 sec latency 80.451 ms 2024-05-14T09:56:54.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 252342 26.38 MB/sec execute 380 sec latency 82.557 ms 2024-05-14T09:56:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:54 smithi029 ceph-mon[36249]: pgmap v1085: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 5.0 MiB/s wr, 599 op/s 2024-05-14T09:56:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:54 smithi026 ceph-mon[36056]: pgmap v1085: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 5.0 MiB/s wr, 599 op/s 2024-05-14T09:56:55.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:54 smithi002 ceph-mon[31557]: pgmap v1085: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 5.0 MiB/s wr, 599 op/s 2024-05-14T09:56:55.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 252827 26.38 MB/sec execute 381 sec latency 101.336 ms 2024-05-14T09:56:56.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 253254 26.34 MB/sec execute 382 sec latency 220.206 ms 2024-05-14T09:56:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:56 smithi029 ceph-mon[36249]: pgmap v1086: 129 pgs: 129 active+clean; 709 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.6 MiB/s wr, 527 op/s 2024-05-14T09:56:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:56 smithi026 ceph-mon[36056]: pgmap v1086: 129 pgs: 129 active+clean; 709 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.6 MiB/s wr, 527 op/s 2024-05-14T09:56:57.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:56 smithi002 ceph-mon[31557]: pgmap v1086: 129 pgs: 129 active+clean; 709 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.6 MiB/s wr, 527 op/s 2024-05-14T09:56:57.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 253662 26.33 MB/sec execute 383 sec latency 565.785 ms 2024-05-14T09:56:58.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 253994 26.31 MB/sec execute 384 sec latency 135.831 ms 2024-05-14T09:56:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:56:58 smithi029 ceph-mon[36249]: pgmap v1087: 129 pgs: 129 active+clean; 711 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.7 MiB/s wr, 583 op/s 2024-05-14T09:56:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:56:58 smithi026 ceph-mon[36056]: pgmap v1087: 129 pgs: 129 active+clean; 711 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.7 MiB/s wr, 583 op/s 2024-05-14T09:56:58.973 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:56:58 smithi002 ceph-mon[31557]: pgmap v1087: 129 pgs: 129 active+clean; 711 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.7 MiB/s wr, 583 op/s 2024-05-14T09:56:59.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 254689 26.35 MB/sec execute 385 sec latency 211.714 ms 2024-05-14T09:57:00.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 255333 26.37 MB/sec execute 386 sec latency 73.471 ms 2024-05-14T09:57:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:00 smithi026 ceph-mon[36056]: pgmap v1088: 129 pgs: 129 active+clean; 713 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.7 MiB/s wr, 550 op/s 2024-05-14T09:57:01.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:00 smithi002 ceph-mon[31557]: pgmap v1088: 129 pgs: 129 active+clean; 713 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.7 MiB/s wr, 550 op/s 2024-05-14T09:57:01.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:00 smithi029 ceph-mon[36249]: pgmap v1088: 129 pgs: 129 active+clean; 713 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.7 MiB/s wr, 550 op/s 2024-05-14T09:57:01.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 255789 26.38 MB/sec execute 387 sec latency 97.524 ms 2024-05-14T09:57:02.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 255802 26.31 MB/sec execute 388 sec latency 1078.205 ms 2024-05-14T09:57:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:02 smithi026 ceph-mon[36056]: pgmap v1089: 129 pgs: 129 active+clean; 720 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.2 MiB/s wr, 586 op/s 2024-05-14T09:57:03.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:02 smithi002 ceph-mon[31557]: pgmap v1089: 129 pgs: 129 active+clean; 720 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.2 MiB/s wr, 586 op/s 2024-05-14T09:57:03.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:02 smithi029 ceph-mon[36249]: pgmap v1089: 129 pgs: 129 active+clean; 720 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.2 MiB/s wr, 586 op/s 2024-05-14T09:57:03.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 255807 26.24 MB/sec execute 389 sec latency 2065.266 ms 2024-05-14T09:57:04.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 255825 26.18 MB/sec execute 390 sec latency 2547.463 ms 2024-05-14T09:57:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:04 smithi026 ceph-mon[36056]: pgmap v1090: 129 pgs: 129 active+clean; 755 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 711 op/s 2024-05-14T09:57:05.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:04 smithi002 ceph-mon[31557]: pgmap v1090: 129 pgs: 129 active+clean; 755 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 711 op/s 2024-05-14T09:57:05.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:04 smithi029 ceph-mon[36249]: pgmap v1090: 129 pgs: 129 active+clean; 755 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 711 op/s 2024-05-14T09:57:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.452+0000 7f38acccc640 1 -- 172.21.15.2:0/1826339365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8110020 msgr2=0x7f38a8110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.452+0000 7f38acccc640 1 --2- 172.21.15.2:0/1826339365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8110020 0x7f38a8110430 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3890009900 tx=0x7f389002a440 comp rx=0 tx=0).stop 2024-05-14T09:57:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.458+0000 7f38acccc640 1 -- 172.21.15.2:0/1826339365 shutdown_connections 2024-05-14T09:57:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.458+0000 7f38acccc640 1 --2- 172.21.15.2:0/1826339365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8110020 0x7f38a8110430 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.458+0000 7f38acccc640 1 --2- 172.21.15.2:0/1826339365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38a810b300 0x7f38a810b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.458+0000 7f38acccc640 1 --2- 172.21.15.2:0/1826339365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38a810a480 0x7f38a810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.458+0000 7f38acccc640 1 -- 172.21.15.2:0/1826339365 >> 172.21.15.2:0/1826339365 conn(0x7f38a80704a0 msgr2=0x7f38a8072950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.465+0000 7f38acccc640 1 -- 172.21.15.2:0/1826339365 shutdown_connections 2024-05-14T09:57:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.468+0000 7f38acccc640 1 -- 172.21.15.2:0/1826339365 wait complete. 2024-05-14T09:57:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.469+0000 7f38acccc640 1 Processor -- start 2024-05-14T09:57:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.469+0000 7f38acccc640 1 -- start start 2024-05-14T09:57:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.469+0000 7f38acccc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38a810a480 0x7f38a8119c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.469+0000 7f38acccc640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38a810b300 0x7f38a811a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.469+0000 7f38acccc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8110020 0x7f38a811b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.469+0000 7f38acccc640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8112160 con 0x7f38a810a480 2024-05-14T09:57:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.469+0000 7f38acccc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8111fe0 con 0x7f38a8110020 2024-05-14T09:57:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.469+0000 7f38acccc640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f38a81122e0 con 0x7f38a810b300 2024-05-14T09:57:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.470+0000 7f38a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8110020 0x7f38a811b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.470+0000 7f38a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8110020 0x7f38a811b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:45230/0 (socket says 172.21.15.2:45230) 2024-05-14T09:57:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.470+0000 7f38a7fff640 1 -- 172.21.15.2:0/2279514001 learned_addr learned my addr 172.21.15.2:0/2279514001 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.470+0000 7f38a7fff640 1 -- 172.21.15.2:0/2279514001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38a810b300 msgr2=0x7f38a811a160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T09:57:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.470+0000 7f38a7fff640 1 --2- 172.21.15.2:0/2279514001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38a810b300 0x7f38a811a160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.470+0000 7f38a7fff640 1 -- 172.21.15.2:0/2279514001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38a810a480 msgr2=0x7f38a8119c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T09:57:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.470+0000 7f38a7fff640 1 --2- 172.21.15.2:0/2279514001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38a810a480 0x7f38a8119c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.470+0000 7f38a7fff640 1 -- 172.21.15.2:0/2279514001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a811b730 con 0x7f38a8110020 2024-05-14T09:57:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.471+0000 7f38a7fff640 1 --2- 172.21.15.2:0/2279514001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8110020 0x7f38a811b0c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3890031610 tx=0x7f3890030dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.473+0000 7f38a4ff9640 1 -- 172.21.15.2:0/2279514001 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890040020 con 0x7f38a8110020 2024-05-14T09:57:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.473+0000 7f38acccc640 1 -- 172.21.15.2:0/2279514001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a811b9c0 con 0x7f38a8110020 2024-05-14T09:57:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.474+0000 7f38acccc640 1 -- 172.21.15.2:0/2279514001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a811bf30 con 0x7f38a8110020 2024-05-14T09:57:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.477+0000 7f38b4ddc740 1 Processor -- start 2024-05-14T09:57:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.477+0000 7f38b4ddc740 1 -- start start 2024-05-14T09:57:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.478+0000 7f38b4ddc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d7f1b0 0x5589d6120c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.478+0000 7f38b4ddc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d562f0 0x5589d61211b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.478+0000 7f38b4ddc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d5e4d0 0x5589d61280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.478+0000 7f38b4ddc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5589d60855d0 con 0x5589d5d5e4d0 2024-05-14T09:57:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.478+0000 7f38b4ddc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5589d603e890 con 0x5589d5d562f0 2024-05-14T09:57:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.478+0000 7f38b4ddc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5589d6081590 con 0x5589d5d7f1b0 2024-05-14T09:57:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.478+0000 7f38a4ff9640 1 -- 172.21.15.2:0/2279514001 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3890004460 con 0x7f38a8110020 2024-05-14T09:57:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.478+0000 7f38a4ff9640 1 -- 172.21.15.2:0/2279514001 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890035070 con 0x7f38a8110020 2024-05-14T09:57:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.483+0000 7f38a4ff9640 1 -- 172.21.15.2:0/2279514001 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3890043020 con 0x7f38a8110020 2024-05-14T09:57:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.484+0000 7f38a4ff9640 1 --2- 172.21.15.2:0/2279514001 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f38940b2a30 0x7f38940b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.484+0000 7f38a4ff9640 1 -- 172.21.15.2:0/2279514001 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f38901032d0 con 0x7f38a8110020 2024-05-14T09:57:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.487+0000 7f38a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d562f0 0x5589d61211b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:05.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.488+0000 7f38a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d562f0 0x5589d61211b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:45242/0 (socket says 172.21.15.2:45242) 2024-05-14T09:57:05.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.488+0000 7f38a77fe640 1 -- 172.21.15.2:0/3137648066 learned_addr learned my addr 172.21.15.2:0/3137648066 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.491+0000 7f38a77fe640 1 -- 172.21.15.2:0/3137648066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d7f1b0 msgr2=0x5589d6120c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:57:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.491+0000 7f38a77fe640 1 --2- 172.21.15.2:0/3137648066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d7f1b0 0x5589d6120c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.491+0000 7f38a77fe640 1 -- 172.21.15.2:0/3137648066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d5e4d0 msgr2=0x5589d61280b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.491+0000 7f38a77fe640 1 --2- 172.21.15.2:0/3137648066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d5e4d0 0x5589d61280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.491+0000 7f38a77fe640 1 -- 172.21.15.2:0/3137648066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589d5d97210 con 0x5589d5d562f0 2024-05-14T09:57:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.491+0000 7f38a77fe640 1 --2- 172.21.15.2:0/3137648066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d562f0 0x5589d61211b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009780 tx=0x7f38a0009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb09f307c840776c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.492+0000 7f389dffb640 1 -- 172.21.15.2:0/3137648066 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0015020 con 0x5589d5d562f0 2024-05-14T09:57:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.492+0000 7f389dffb640 1 -- 172.21.15.2:0/3137648066 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a0011420 con 0x5589d5d562f0 2024-05-14T09:57:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.493+0000 7f38b4ddc740 1 -- 172.21.15.2:0/3137648066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d562f0 msgr2=0x5589d61211b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.493+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/3137648066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d562f0 0x5589d61211b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009780 tx=0x7f38a0009c40 comp rx=0 tx=0).stop 2024-05-14T09:57:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.494+0000 7f38b4ddc740 1 -- 172.21.15.2:0/3137648066 shutdown_connections 2024-05-14T09:57:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.494+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/3137648066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d5e4d0 0x5589d61280b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.494+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/3137648066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d562f0 0x5589d61211b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.494+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/3137648066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d7f1b0 0x5589d6120c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.494+0000 7f38b4ddc740 1 -- 172.21.15.2:0/3137648066 >> 172.21.15.2:0/3137648066 conn(0x5589d5d73300 msgr2=0x5589d611c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.497+0000 7f38b4ddc740 1 -- 172.21.15.2:0/3137648066 shutdown_connections 2024-05-14T09:57:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.497+0000 7f38a7fff640 1 --2- 172.21.15.2:0/2279514001 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f38940b2a30 0x7f38940b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.500+0000 7f38b4ddc740 1 -- 172.21.15.2:0/3137648066 wait complete. 2024-05-14T09:57:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.500+0000 7f38b4ddc740 1 -- start start 2024-05-14T09:57:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.500+0000 7f38b4ddc740 1 Processor -- start 2024-05-14T09:57:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.501+0000 7f38a7fff640 1 --2- 172.21.15.2:0/2279514001 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f38940b2a30 0x7f38940b4f40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f38900c7570 tx=0x7f3890004610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.504+0000 7f38b4ddc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d562f0 0x5589d614c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.504+0000 7f38b4ddc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d5e4d0 0x5589d614cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.504+0000 7f38b4ddc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d7f1b0 0x5589d6153d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.504+0000 7f38b4ddc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5589d614ab60 con 0x5589d5d562f0 2024-05-14T09:57:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.504+0000 7f38b4ddc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5589d6042370 con 0x5589d5d7f1b0 2024-05-14T09:57:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.504+0000 7f38b4ddc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5589d614ace0 con 0x5589d5d5e4d0 2024-05-14T09:57:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.504+0000 7f38b4ddc740 10 client.0 _ll_register_callbacks cb 0x5589d5d96a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:57:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.504+0000 7f38a77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d5e4d0 0x5589d614cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.504+0000 7f38a77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d5e4d0 0x5589d614cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:55404/0 (socket says 172.21.15.2:55404) 2024-05-14T09:57:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.504+0000 7f38a77fe640 1 -- 172.21.15.2:0/1096232531 learned_addr learned my addr 172.21.15.2:0/1096232531 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.505+0000 7f38a77fe640 1 -- 172.21.15.2:0/1096232531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d7f1b0 msgr2=0x5589d6153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.505+0000 7f38a77fe640 1 --2- 172.21.15.2:0/1096232531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d7f1b0 0x5589d6153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.505+0000 7f38a77fe640 1 -- 172.21.15.2:0/1096232531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d562f0 msgr2=0x5589d614c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.505+0000 7f38a77fe640 1 --2- 172.21.15.2:0/1096232531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d562f0 0x5589d614c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.505+0000 7f38a77fe640 1 -- 172.21.15.2:0/1096232531 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589d5e63620 con 0x5589d5d5e4d0 2024-05-14T09:57:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.505+0000 7f38a77fe640 1 --2- 172.21.15.2:0/1096232531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d5e4d0 0x5589d614cdb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f38a0006fd0 tx=0x7f38a0009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f46f15d3804b5680 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.505+0000 7f3886ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:57:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.509+0000 7f3886ffd640 1 -- 172.21.15.2:0/1096232531 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0015050 con 0x5589d5d5e4d0 2024-05-14T09:57:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.509+0000 7f38b4ddc740 1 -- 172.21.15.2:0/1096232531 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a0011420 con 0x5589d5d5e4d0 2024-05-14T09:57:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.509+0000 7f3886ffd640 1 -- 172.21.15.2:0/1096232531 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f38a000bce0 con 0x5589d5d5e4d0 2024-05-14T09:57:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.509+0000 7f3886ffd640 1 -- 172.21.15.2:0/1096232531 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f38a002d050 con 0x5589d5d5e4d0 2024-05-14T09:57:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.509+0000 7f38b4ddc740 10 client.27076 fetch_fsmap learned FSMap version 67 2024-05-14T09:57:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.509+0000 7f38b4ddc740 1 -- 172.21.15.2:0/1096232531 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x5589d6156c70 con 0x5589d5d5e4d0 2024-05-14T09:57:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.510+0000 7f3886ffd640 1 -- 172.21.15.2:0/1096232531 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a0011420 con 0x5589d5d5e4d0 2024-05-14T09:57:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.511+0000 7f3886ffd640 1 -- 172.21.15.2:0/1096232531 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f38a001a5b0 con 0x5589d5d5e4d0 2024-05-14T09:57:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.512+0000 7f38b4ddc740 10 client.27076 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:57:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.512+0000 7f38b4ddc740 10 client.27076 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:57:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.512+0000 7f38b4ddc740 20 client.27076 populate_metadata read hostname 'smithi002' 2024-05-14T09:57:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.512+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/1096232531 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5589d5d3dd80 0x5589d615a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.512+0000 7f38b4ddc740 4 client.27076 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:57:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.512+0000 7f38b4ddc740 1 -- 172.21.15.2:0/1096232531 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589d60931e0 con 0x5589d5d3dd80 2024-05-14T09:57:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.512+0000 7f38a6ffd640 1 --2- 172.21.15.2:0/1096232531 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5589d5d3dd80 0x5589d615a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.522+0000 7f38a6ffd640 1 --2- 172.21.15.2:0/1096232531 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5589d5d3dd80 0x5589d615a800 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c863a314e96827f server_cookie=e6c0ed236d217598 in_seq=0 out_seq=0 2024-05-14T09:57:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.522+0000 7f3886ffd640 10 client.27076 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.528+0000 7f3886ffd640 1 -- 172.21.15.2:0/1096232531 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5589d60931e0 con 0x5589d5d3dd80 2024-05-14T09:57:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.528+0000 7f3886ffd640 10 client.27076 handle_command_reply: tid=0 2024-05-14T09:57:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.528+0000 7f38b4ddc740 1 client.27076 shutdown 2024-05-14T09:57:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.532+0000 7f38b4ddc740 1 -- 172.21.15.2:0/1096232531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d5e4d0 msgr2=0x5589d614cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.532+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/1096232531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d5e4d0 0x5589d614cdb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f38a0006fd0 tx=0x7f38a0009500 comp rx=0 tx=0).stop 2024-05-14T09:57:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.532+0000 7f38b4ddc740 1 -- 172.21.15.2:0/1096232531 shutdown_connections 2024-05-14T09:57:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.532+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/1096232531 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5589d5d3dd80 0x5589d615a800 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.532+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/1096232531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d7f1b0 0x5589d6153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.532+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/1096232531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d5e4d0 0x5589d614cdb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.532+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/1096232531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d562f0 0x5589d614c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.532+0000 7f38b4ddc740 1 -- 172.21.15.2:0/1096232531 >> 172.21.15.2:0/1096232531 conn(0x5589d5d73300 msgr2=0x5589d611f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.534+0000 7f3886ffd640 0 client.27076 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.534+0000 7f38b4ddc740 1 -- 172.21.15.2:0/1096232531 shutdown_connections 2024-05-14T09:57:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.539+0000 7f38b4ddc740 1 -- 172.21.15.2:0/1096232531 wait complete. 2024-05-14T09:57:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.541+0000 7f38b4ddc740 20 client.27076 trim_cache size 0 max 16384 2024-05-14T09:57:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.591+0000 7f38b4ddc740 1 Processor -- start 2024-05-14T09:57:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.596+0000 7f38b4ddc740 1 -- start start 2024-05-14T09:57:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.596+0000 7f38b4ddc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d562f0 0x5589d6124b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.596+0000 7f38b4ddc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d5e4d0 0x5589d61250d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.596+0000 7f38b4ddc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d7f1b0 0x5589d6153c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.597+0000 7f38b4ddc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3888012610 con 0x5589d5d7f1b0 2024-05-14T09:57:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.597+0000 7f38b4ddc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f388800fee0 con 0x5589d5d5e4d0 2024-05-14T09:57:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.597+0000 7f38b4ddc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f388800dde0 con 0x5589d5d562f0 2024-05-14T09:57:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.599+0000 7f38a6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d562f0 0x5589d6124b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.599+0000 7f38a6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d562f0 0x5589d6124b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:55406/0 (socket says 172.21.15.2:55406) 2024-05-14T09:57:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.599+0000 7f38a6ffd640 1 -- 172.21.15.2:0/3810812348 learned_addr learned my addr 172.21.15.2:0/3810812348 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.599+0000 7f38a6ffd640 1 --2- 172.21.15.2:0/3810812348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d7f1b0 0x5589d6153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.600+0000 7f38a6ffd640 1 -- 172.21.15.2:0/3810812348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d5e4d0 msgr2=0x5589d61250d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:57:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.600+0000 7f38a6ffd640 1 --2- 172.21.15.2:0/3810812348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d5e4d0 0x5589d61250d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.600+0000 7f38a6ffd640 1 -- 172.21.15.2:0/3810812348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d7f1b0 msgr2=0x5589d6153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.600+0000 7f38a6ffd640 1 --2- 172.21.15.2:0/3810812348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d7f1b0 0x5589d6153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.600+0000 7f38a6ffd640 1 -- 172.21.15.2:0/3810812348 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589d6127f70 con 0x5589d5d562f0 2024-05-14T09:57:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.601+0000 7f38a6ffd640 1 --2- 172.21.15.2:0/3810812348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d562f0 0x5589d6124b90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3898000f80 tx=0x7f38980016b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4083fc7a3094235 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.602+0000 7f389d7fa640 1 -- 172.21.15.2:0/3810812348 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898013a70 con 0x5589d5d562f0 2024-05-14T09:57:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.602+0000 7f389d7fa640 1 -- 172.21.15.2:0/3810812348 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5589d60931e0 con 0x5589d5d562f0 2024-05-14T09:57:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.602+0000 7f389d7fa640 1 -- 172.21.15.2:0/3810812348 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38980143a0 con 0x5589d5d562f0 2024-05-14T09:57:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.604+0000 7f38b4ddc740 1 -- 172.21.15.2:0/3810812348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d562f0 msgr2=0x5589d6124b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.604+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/3810812348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d562f0 0x5589d6124b90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3898000f80 tx=0x7f38980016b0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.604+0000 7f38b4ddc740 1 -- 172.21.15.2:0/3810812348 shutdown_connections 2024-05-14T09:57:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.604+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/3810812348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d7f1b0 0x5589d6153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.604+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/3810812348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d5e4d0 0x5589d61250d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.604+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/3810812348 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d562f0 0x5589d6124b90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.604+0000 7f38b4ddc740 1 -- 172.21.15.2:0/3810812348 >> 172.21.15.2:0/3810812348 conn(0x5589d5d73300 msgr2=0x5589d614c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.607+0000 7f38b4ddc740 1 -- 172.21.15.2:0/3810812348 shutdown_connections 2024-05-14T09:57:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.609+0000 7f38b4ddc740 1 -- 172.21.15.2:0/3810812348 wait complete. 2024-05-14T09:57:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.609+0000 7f38b4ddc740 1 -- start start 2024-05-14T09:57:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.609+0000 7f38b4ddc740 1 Processor -- start 2024-05-14T09:57:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.610+0000 7f38b4ddc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d562f0 0x5589d6154200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.610+0000 7f38b4ddc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d5e4d0 0x5589d6150d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.610+0000 7f38b4ddc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d7f1b0 0x5589d61512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.610+0000 7f38b4ddc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f388800e3a0 con 0x5589d5d562f0 2024-05-14T09:57:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.610+0000 7f38b4ddc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f388800f810 con 0x5589d5d5e4d0 2024-05-14T09:57:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.610+0000 7f38b4ddc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f388800ea70 con 0x5589d5d7f1b0 2024-05-14T09:57:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.610+0000 7f38b4ddc740 10 client.0 _ll_register_callbacks cb 0x7f38a0009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:57:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.610+0000 7f38a6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d562f0 0x5589d6154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.610+0000 7f38a6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d562f0 0x5589d6154200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49084/0 (socket says 172.21.15.2:49084) 2024-05-14T09:57:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.610+0000 7f38a6ffd640 1 -- 172.21.15.2:0/4274584480 learned_addr learned my addr 172.21.15.2:0/4274584480 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.614+0000 7f38a6ffd640 1 --2- 172.21.15.2:0/4274584480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d7f1b0 0x5589d61512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.614+0000 7f38a77fe640 1 --2- 172.21.15.2:0/4274584480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d5e4d0 0x5589d6150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.615+0000 7f38a6ffd640 1 -- 172.21.15.2:0/4274584480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d7f1b0 msgr2=0x5589d61512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.615+0000 7f38a6ffd640 1 --2- 172.21.15.2:0/4274584480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d7f1b0 0x5589d61512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.615+0000 7f38a6ffd640 1 -- 172.21.15.2:0/4274584480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d5e4d0 msgr2=0x5589d6150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.615+0000 7f38a6ffd640 1 --2- 172.21.15.2:0/4274584480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d5e4d0 0x5589d6150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.615+0000 7f38a6ffd640 1 -- 172.21.15.2:0/4274584480 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589d61518b0 con 0x5589d5d562f0 2024-05-14T09:57:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.618+0000 7f38a6ffd640 1 --2- 172.21.15.2:0/4274584480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d562f0 0x5589d6154200 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f3898001400 tx=0x7f38980017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5d1c5e047012ee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.618+0000 7f3886ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:57:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.618+0000 7f3886ffd640 1 -- 172.21.15.2:0/4274584480 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389800b030 con 0x5589d5d562f0 2024-05-14T09:57:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.619+0000 7f3886ffd640 1 -- 172.21.15.2:0/4274584480 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f3898013a70 con 0x5589d5d562f0 2024-05-14T09:57:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.619+0000 7f38b4ddc740 1 -- 172.21.15.2:0/4274584480 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5589d60931e0 con 0x5589d5d562f0 2024-05-14T09:57:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.619+0000 7f3886ffd640 1 -- 172.21.15.2:0/4274584480 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f3898036050 con 0x5589d5d562f0 2024-05-14T09:57:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.619+0000 7f38b4ddc740 10 client.17469 fetch_fsmap learned FSMap version 67 2024-05-14T09:57:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.619+0000 7f38b4ddc740 1 -- 172.21.15.2:0/4274584480 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x5589d61c1c60 con 0x5589d5d562f0 2024-05-14T09:57:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.619+0000 7f3886ffd640 1 -- 172.21.15.2:0/4274584480 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5589d60931e0 con 0x5589d5d562f0 2024-05-14T09:57:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.620+0000 7f3886ffd640 1 -- 172.21.15.2:0/4274584480 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f3898022b30 con 0x5589d5d562f0 2024-05-14T09:57:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.620+0000 7f38b4ddc740 10 client.17469 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:57:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.620+0000 7f38b4ddc740 10 client.17469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:57:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.620+0000 7f38b4ddc740 20 client.17469 populate_metadata read hostname 'smithi002' 2024-05-14T09:57:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.620+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/4274584480 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5589d5d131c0 0x5589d61c4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.620+0000 7f38b4ddc740 4 client.17469 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:57:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.620+0000 7f38b4ddc740 1 -- 172.21.15.2:0/4274584480 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589d608d050 con 0x5589d5d131c0 2024-05-14T09:57:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.620+0000 7f38a77fe640 1 --2- 172.21.15.2:0/4274584480 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5589d5d131c0 0x5589d61c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.621+0000 7f38a77fe640 1 --2- 172.21.15.2:0/4274584480 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5589d5d131c0 0x5589d61c4600 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50952c8ba11e20e server_cookie=8f8b6509d301b1ad in_seq=0 out_seq=0 2024-05-14T09:57:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:05.621+0000 7f3886ffd640 10 client.17469 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:05.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 255959 26.13 MB/sec execute 391 sec latency 2672.647 ms 2024-05-14T09:57:06.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:05 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:57:06.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:05 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:57:06.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:05 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:57:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.509+0000 7f3886ffd640 1 -- 172.21.15.2:0/4274584480 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589d608d050 con 0x5589d5d131c0 2024-05-14T09:57:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.509+0000 7f3886ffd640 10 client.17469 handle_command_reply: tid=0 2024-05-14T09:57:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.509+0000 7f38b4ddc740 1 client.17469 shutdown 2024-05-14T09:57:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.509+0000 7f38b4ddc740 1 -- 172.21.15.2:0/4274584480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d562f0 msgr2=0x5589d6154200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.509+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/4274584480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d562f0 0x5589d6154200 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f3898001400 tx=0x7f38980017e0 comp rx=0 tx=0).stop 2024-05-14T09:57:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.509+0000 7f38b4ddc740 1 -- 172.21.15.2:0/4274584480 shutdown_connections 2024-05-14T09:57:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.509+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/4274584480 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5589d5d131c0 0x5589d61c4600 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.509+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/4274584480 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5589d5d7f1b0 0x5589d61512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.509+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/4274584480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589d5d5e4d0 0x5589d6150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.509+0000 7f38b4ddc740 1 --2- 172.21.15.2:0/4274584480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589d5d562f0 0x5589d6154200 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.509+0000 7f38b4ddc740 1 -- 172.21.15.2:0/4274584480 >> 172.21.15.2:0/4274584480 conn(0x5589d5d73300 msgr2=0x5589d6124b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.509+0000 7f3886ffd640 0 client.17469 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.510+0000 7f38b4ddc740 1 -- 172.21.15.2:0/4274584480 shutdown_connections 2024-05-14T09:57:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.510+0000 7f38b4ddc740 1 -- 172.21.15.2:0/4274584480 wait complete. 2024-05-14T09:57:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.510+0000 7f38b4ddc740 20 client.17469 trim_cache size 0 max 16384 2024-05-14T09:57:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:57:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:57:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:57:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:57:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.511+0000 7f38acccc640 1 -- 172.21.15.2:0/2279514001 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f38940b2a30 msgr2=0x7f38940b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.511+0000 7f38acccc640 1 --2- 172.21.15.2:0/2279514001 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f38940b2a30 0x7f38940b4f40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f38900c7570 tx=0x7f3890004610 comp rx=0 tx=0).stop 2024-05-14T09:57:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.511+0000 7f38acccc640 1 -- 172.21.15.2:0/2279514001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8110020 msgr2=0x7f38a811b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.511+0000 7f38acccc640 1 --2- 172.21.15.2:0/2279514001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8110020 0x7f38a811b0c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3890031610 tx=0x7f3890030dd0 comp rx=0 tx=0).stop 2024-05-14T09:57:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.511+0000 7f38acccc640 1 -- 172.21.15.2:0/2279514001 shutdown_connections 2024-05-14T09:57:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.511+0000 7f38acccc640 1 --2- 172.21.15.2:0/2279514001 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f38940b2a30 0x7f38940b4f40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.511+0000 7f38acccc640 1 --2- 172.21.15.2:0/2279514001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a8110020 0x7f38a811b0c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.511+0000 7f38acccc640 1 --2- 172.21.15.2:0/2279514001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38a810b300 0x7f38a811a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.511+0000 7f38acccc640 1 --2- 172.21.15.2:0/2279514001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38a810a480 0x7f38a8119c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.512+0000 7f38acccc640 1 -- 172.21.15.2:0/2279514001 >> 172.21.15.2:0/2279514001 conn(0x7f38a80704a0 msgr2=0x7f38a80726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.512+0000 7f38acccc640 1 -- 172.21.15.2:0/2279514001 shutdown_connections 2024-05-14T09:57:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:06.512+0000 7f38acccc640 1 -- 172.21.15.2:0/2279514001 wait complete. 2024-05-14T09:57:06.621 INFO:tasks.workunit.client.0.smithi002.stdout: 10 256110 26.07 MB/sec execute 392 sec latency 546.422 ms 2024-05-14T09:57:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:06 smithi002 ceph-mon[31557]: pgmap v1091: 129 pgs: 129 active+clean; 786 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 565 op/s 2024-05-14T09:57:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:06 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:57:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:06 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:57:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:06 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:57:07.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:06 smithi029 ceph-mon[36249]: pgmap v1091: 129 pgs: 129 active+clean; 786 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 565 op/s 2024-05-14T09:57:07.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:06 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:57:07.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:06 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:57:07.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:06 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:57:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:06 smithi026 ceph-mon[36056]: pgmap v1091: 129 pgs: 129 active+clean; 786 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 565 op/s 2024-05-14T09:57:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:06 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:57:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:06 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:57:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:06 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:57:07.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 256330 26.04 MB/sec execute 393 sec latency 196.660 ms 2024-05-14T09:57:08.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 256538 25.99 MB/sec execute 394 sec latency 119.818 ms 2024-05-14T09:57:08.989 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:08 smithi029 ceph-mon[36249]: pgmap v1092: 129 pgs: 129 active+clean; 815 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 569 op/s 2024-05-14T09:57:09.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:08 smithi002 ceph-mon[31557]: pgmap v1092: 129 pgs: 129 active+clean; 815 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 569 op/s 2024-05-14T09:57:09.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:08 smithi026 ceph-mon[36056]: pgmap v1092: 129 pgs: 129 active+clean; 815 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 569 op/s 2024-05-14T09:57:09.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 256728 25.96 MB/sec execute 395 sec latency 181.770 ms 2024-05-14T09:57:10.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 256969 25.93 MB/sec execute 396 sec latency 336.062 ms 2024-05-14T09:57:11.003 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:10 smithi026 ceph-mon[36056]: pgmap v1093: 129 pgs: 129 active+clean; 811 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 16 MiB/s wr, 662 op/s 2024-05-14T09:57:11.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:10 smithi002 ceph-mon[31557]: pgmap v1093: 129 pgs: 129 active+clean; 811 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 16 MiB/s wr, 662 op/s 2024-05-14T09:57:11.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:10 smithi029 ceph-mon[36249]: pgmap v1093: 129 pgs: 129 active+clean; 811 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 16 MiB/s wr, 662 op/s 2024-05-14T09:57:11.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 257147 25.88 MB/sec execute 397 sec latency 425.774 ms 2024-05-14T09:57:12.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 257561 25.87 MB/sec execute 398 sec latency 122.160 ms 2024-05-14T09:57:13.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:12 smithi002 ceph-mon[31557]: pgmap v1094: 129 pgs: 129 active+clean; 772 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 716 op/s 2024-05-14T09:57:13.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:12 smithi029 ceph-mon[36249]: pgmap v1094: 129 pgs: 129 active+clean; 772 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 716 op/s 2024-05-14T09:57:13.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:12 smithi026 ceph-mon[36056]: pgmap v1094: 129 pgs: 129 active+clean; 772 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 716 op/s 2024-05-14T09:57:13.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 258291 25.91 MB/sec execute 399 sec latency 56.833 ms 2024-05-14T09:57:14.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 258898 25.94 MB/sec execute 400 sec latency 72.386 ms 2024-05-14T09:57:15.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:14 smithi002 ceph-mon[31557]: pgmap v1095: 129 pgs: 129 active+clean; 707 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 15 MiB/s wr, 762 op/s 2024-05-14T09:57:15.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:14 smithi029 ceph-mon[36249]: pgmap v1095: 129 pgs: 129 active+clean; 707 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 15 MiB/s wr, 762 op/s 2024-05-14T09:57:15.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:14 smithi026 ceph-mon[36056]: pgmap v1095: 129 pgs: 129 active+clean; 707 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 15 MiB/s wr, 762 op/s 2024-05-14T09:57:15.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 259384 25.94 MB/sec execute 401 sec latency 143.228 ms 2024-05-14T09:57:16.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 259707 25.91 MB/sec execute 402 sec latency 349.175 ms 2024-05-14T09:57:17.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:16 smithi002 ceph-mon[31557]: pgmap v1096: 129 pgs: 129 active+clean; 699 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 646 op/s 2024-05-14T09:57:17.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:16 smithi029 ceph-mon[36249]: pgmap v1096: 129 pgs: 129 active+clean; 699 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 646 op/s 2024-05-14T09:57:17.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:16 smithi026 ceph-mon[36056]: pgmap v1096: 129 pgs: 129 active+clean; 699 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 646 op/s 2024-05-14T09:57:17.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 260287 25.91 MB/sec execute 403 sec latency 68.545 ms 2024-05-14T09:57:18.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 260907 25.94 MB/sec execute 404 sec latency 70.112 ms 2024-05-14T09:57:19.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:18 smithi002 ceph-mon[31557]: pgmap v1097: 129 pgs: 129 active+clean; 696 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.6 MiB/s wr, 666 op/s 2024-05-14T09:57:19.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:18 smithi029 ceph-mon[36249]: pgmap v1097: 129 pgs: 129 active+clean; 696 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.6 MiB/s wr, 666 op/s 2024-05-14T09:57:19.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:18 smithi026 ceph-mon[36056]: pgmap v1097: 129 pgs: 129 active+clean; 696 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.6 MiB/s wr, 666 op/s 2024-05-14T09:57:19.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 261608 25.98 MB/sec execute 405 sec latency 91.015 ms 2024-05-14T09:57:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.693+0000 7ff6bc54d640 1 -- 172.21.15.2:0/4274155355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b4109bd0 msgr2=0x7ff6b410bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.693+0000 7ff6bc54d640 1 --2- 172.21.15.2:0/4274155355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b4109bd0 0x7ff6b410bff0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009ca0 tx=0x7ff6ac02a8e0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.694+0000 7ff6bc54d640 1 -- 172.21.15.2:0/4274155355 shutdown_connections 2024-05-14T09:57:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.694+0000 7ff6bc54d640 1 --2- 172.21.15.2:0/4274155355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b4109bd0 0x7ff6b410bff0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.694+0000 7ff6bc54d640 1 --2- 172.21.15.2:0/4274155355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6b4104ed0 0x7ff6b41052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.694+0000 7ff6bc54d640 1 --2- 172.21.15.2:0/4274155355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b4103c90 0x7ff6b4104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.694+0000 7ff6bc54d640 1 -- 172.21.15.2:0/4274155355 >> 172.21.15.2:0/4274155355 conn(0x7ff6b40feea0 msgr2=0x7ff6b4101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.694+0000 7ff6bc54d640 1 -- 172.21.15.2:0/4274155355 shutdown_connections 2024-05-14T09:57:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.694+0000 7ff6bc54d640 1 -- 172.21.15.2:0/4274155355 wait complete. 2024-05-14T09:57:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.695+0000 7ff6bc54d640 1 Processor -- start 2024-05-14T09:57:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.696+0000 7ff6bc54d640 1 -- start start 2024-05-14T09:57:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.696+0000 7ff6bc54d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6b4103c90 0x7ff6b40769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.696+0000 7ff6bc54d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b4104ed0 0x7ff6b4075040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.696+0000 7ff6bc54d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b4109bd0 0x7ff6b4075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.696+0000 7ff6bc54d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b410f880 con 0x7ff6b4104ed0 2024-05-14T09:57:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.696+0000 7ff6bc54d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b410f700 con 0x7ff6b4109bd0 2024-05-14T09:57:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.696+0000 7ff6bc54d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b410fa00 con 0x7ff6b4103c90 2024-05-14T09:57:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.697+0000 7ff6b9ae8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b4104ed0 0x7ff6b4075040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.697+0000 7ff6b9ae8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b4104ed0 0x7ff6b4075040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60756/0 (socket says 172.21.15.2:60756) 2024-05-14T09:57:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.697+0000 7ff6b9ae8640 1 -- 172.21.15.2:0/126997365 learned_addr learned my addr 172.21.15.2:0/126997365 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.697+0000 7ff6b9ae8640 1 -- 172.21.15.2:0/126997365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6b4103c90 msgr2=0x7ff6b40769a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.697+0000 7ff6b9ae8640 1 --2- 172.21.15.2:0/126997365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6b4103c90 0x7ff6b40769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.697+0000 7ff6b9ae8640 1 -- 172.21.15.2:0/126997365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b4109bd0 msgr2=0x7ff6b4075580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T09:57:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.697+0000 7ff6b9ae8640 1 --2- 172.21.15.2:0/126997365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b4109bd0 0x7ff6b4075580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.697+0000 7ff6b9ae8640 1 -- 172.21.15.2:0/126997365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b419e260 con 0x7ff6b4104ed0 2024-05-14T09:57:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.698+0000 7ff6b9ae8640 1 --2- 172.21.15.2:0/126997365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b4104ed0 0x7ff6b4075040 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400e9f0 tx=0x7ff6a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.698+0000 7ff6ab7fe640 1 -- 172.21.15.2:0/126997365 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a400cd80 con 0x7ff6b4104ed0 2024-05-14T09:57:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.698+0000 7ff6ab7fe640 1 -- 172.21.15.2:0/126997365 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6a40054d0 con 0x7ff6b4104ed0 2024-05-14T09:57:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.698+0000 7ff6ab7fe640 1 -- 172.21.15.2:0/126997365 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4010430 con 0x7ff6b4104ed0 2024-05-14T09:57:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.698+0000 7ff6bc54d640 1 -- 172.21.15.2:0/126997365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b419e550 con 0x7ff6b4104ed0 2024-05-14T09:57:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.698+0000 7ff6bc54d640 1 -- 172.21.15.2:0/126997365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b419e9f0 con 0x7ff6b4104ed0 2024-05-14T09:57:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.706+0000 7ff6c3403740 1 Processor -- start 2024-05-14T09:57:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.707+0000 7ff6c3403740 1 -- start start 2024-05-14T09:57:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.707+0000 7ff6c3403740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 0x55a8b1bc5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.707+0000 7ff6c3403740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bc5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.707+0000 7ff6c3403740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 0x55a8b1bccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.707+0000 7ff6c3403740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a8b1b2a5d0 con 0x55a8b18034d0 2024-05-14T09:57:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.707+0000 7ff6c3403740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a8b1ae3890 con 0x55a8b17fb2f0 2024-05-14T09:57:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.707+0000 7ff6c3403740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a8b1b26590 con 0x55a8b18241b0 2024-05-14T09:57:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.707+0000 7ff6ba2e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bc5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.707+0000 7ff6baaea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 0x55a8b1bc5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.707+0000 7ff6ba2e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bc5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:35936/0 (socket says 172.21.15.2:35936) 2024-05-14T09:57:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.707+0000 7ff6ba2e9640 1 -- 172.21.15.2:0/1013241151 learned_addr learned my addr 172.21.15.2:0/1013241151 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.708+0000 7ff6baaea640 1 -- 172.21.15.2:0/1013241151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b17fb2f0 msgr2=0x55a8b1bc5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.708+0000 7ff6baaea640 1 --2- 172.21.15.2:0/1013241151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bc5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.708+0000 7ff6baaea640 1 -- 172.21.15.2:0/1013241151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 msgr2=0x55a8b1bccee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:57:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.708+0000 7ff6baaea640 1 --2- 172.21.15.2:0/1013241151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 0x55a8b1bccee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.708+0000 7ff6baaea640 1 -- 172.21.15.2:0/1013241151 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8b183c210 con 0x55a8b18241b0 2024-05-14T09:57:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.708+0000 7ff6ab7fe640 1 -- 172.21.15.2:0/126997365 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff6a4010610 con 0x7ff6b4104ed0 2024-05-14T09:57:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.708+0000 7ff6baaea640 1 --2- 172.21.15.2:0/1013241151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 0x55a8b1bc5a60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac02ae00 tx=0x7ff6ac009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b76ba18918da896 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.709+0000 7ff6a8bfe640 1 -- 172.21.15.2:0/1013241151 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac03f070 con 0x55a8b18241b0 2024-05-14T09:57:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.709+0000 7ff6a8bfe640 1 -- 172.21.15.2:0/1013241151 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac036040 con 0x55a8b18241b0 2024-05-14T09:57:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.709+0000 7ff6a8bfe640 1 -- 172.21.15.2:0/1013241151 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0042a0 con 0x55a8b18241b0 2024-05-14T09:57:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.709+0000 7ff6ab7fe640 1 --2- 172.21.15.2:0/126997365 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff6a00b2a70 0x7ff6a00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.710+0000 7ff6ab7fe640 1 -- 172.21.15.2:0/126997365 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7ff6a40e2a90 con 0x7ff6b4104ed0 2024-05-14T09:57:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.710+0000 7ff6ba2e9640 1 --2- 172.21.15.2:0/126997365 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff6a00b2a70 0x7ff6a00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.710+0000 7ff6c3403740 1 -- 172.21.15.2:0/1013241151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 msgr2=0x55a8b1bc5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.710+0000 7ff6c3403740 1 --2- 172.21.15.2:0/1013241151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 0x55a8b1bc5a60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac02ae00 tx=0x7ff6ac009800 comp rx=0 tx=0).stop 2024-05-14T09:57:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.710+0000 7ff6c3403740 1 -- 172.21.15.2:0/1013241151 shutdown_connections 2024-05-14T09:57:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.710+0000 7ff6c3403740 1 --2- 172.21.15.2:0/1013241151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 0x55a8b1bccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.710+0000 7ff6c3403740 1 --2- 172.21.15.2:0/1013241151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bc5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.710+0000 7ff6c3403740 1 --2- 172.21.15.2:0/1013241151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 0x55a8b1bc5a60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.710+0000 7ff6c3403740 1 -- 172.21.15.2:0/1013241151 >> 172.21.15.2:0/1013241151 conn(0x55a8b1818300 msgr2=0x55a8b1bc13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.711+0000 7ff6c3403740 1 -- 172.21.15.2:0/1013241151 shutdown_connections 2024-05-14T09:57:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.711+0000 7ff6ba2e9640 1 --2- 172.21.15.2:0/126997365 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff6a00b2a70 0x7ff6a00b4f80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55a8b1bc6e40 tx=0x7ff6b0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.711+0000 7ff6c3403740 1 -- 172.21.15.2:0/1013241151 wait complete. 2024-05-14T09:57:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.711+0000 7ff6c3403740 1 -- start start 2024-05-14T09:57:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.711+0000 7ff6c3403740 1 Processor -- start 2024-05-14T09:57:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.711+0000 7ff6c3403740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bcf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.712+0000 7ff6c3403740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 0x55a8b1bcfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.712+0000 7ff6c3403740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 0x55a8b1bd6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.712+0000 7ff6c3403740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a8b1bcd8d0 con 0x55a8b18034d0 2024-05-14T09:57:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.712+0000 7ff6c3403740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a8b1ae7370 con 0x55a8b17fb2f0 2024-05-14T09:57:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.712+0000 7ff6c3403740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a8b1bcda50 con 0x55a8b18241b0 2024-05-14T09:57:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.712+0000 7ff6c3403740 10 client.0 _ll_register_callbacks cb 0x55a8b183ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:57:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.712+0000 7ff6ba2e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bcf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.712+0000 7ff6baaea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 0x55a8b1bd6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.712+0000 7ff6b9ae8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 0x55a8b1bcfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.712+0000 7ff6b9ae8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 0x55a8b1bcfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60770/0 (socket says 172.21.15.2:60770) 2024-05-14T09:57:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.712+0000 7ff6b9ae8640 1 -- 172.21.15.2:0/682533365 learned_addr learned my addr 172.21.15.2:0/682533365 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.713+0000 7ff6b9ae8640 1 -- 172.21.15.2:0/682533365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 msgr2=0x55a8b1bd6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.713+0000 7ff6b9ae8640 1 --2- 172.21.15.2:0/682533365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 0x55a8b1bd6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.713+0000 7ff6b9ae8640 1 -- 172.21.15.2:0/682533365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b17fb2f0 msgr2=0x55a8b1bcf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.713+0000 7ff6b9ae8640 1 --2- 172.21.15.2:0/682533365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bcf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.713+0000 7ff6b9ae8640 1 -- 172.21.15.2:0/682533365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8b1908620 con 0x55a8b18034d0 2024-05-14T09:57:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.713+0000 7ff6b9ae8640 1 --2- 172.21.15.2:0/682533365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 0x55a8b1bcfc70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff6a40a5e70 tx=0x7ff6a40a9900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258ca81fd4b1616b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.713+0000 7ff69dffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:57:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.714+0000 7ff69dffb640 1 -- 172.21.15.2:0/682533365 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a40c42d0 con 0x55a8b18034d0 2024-05-14T09:57:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.714+0000 7ff69dffb640 1 -- 172.21.15.2:0/682533365 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7ff6a40b2600 con 0x55a8b18034d0 2024-05-14T09:57:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.714+0000 7ff6c3403740 1 -- 172.21.15.2:0/682533365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ac036040 con 0x55a8b18034d0 2024-05-14T09:57:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.716+0000 7ff69dffb640 1 -- 172.21.15.2:0/682533365 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7ff6a40b2840 con 0x55a8b18034d0 2024-05-14T09:57:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.716+0000 7ff6c3403740 10 client.17478 fetch_fsmap learned FSMap version 67 2024-05-14T09:57:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.716+0000 7ff6c3403740 1 -- 172.21.15.2:0/682533365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55a8b1bd9bd0 con 0x55a8b18034d0 2024-05-14T09:57:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.717+0000 7ff69dffb640 1 -- 172.21.15.2:0/682533365 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac036040 con 0x55a8b18034d0 2024-05-14T09:57:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.718+0000 7ff69dffb640 1 -- 172.21.15.2:0/682533365 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7ff6a40b2ce0 con 0x55a8b18034d0 2024-05-14T09:57:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.718+0000 7ff6c3403740 10 client.17478 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:57:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.718+0000 7ff6c3403740 10 client.17478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:57:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.718+0000 7ff6c3403740 20 client.17478 populate_metadata read hostname 'smithi002' 2024-05-14T09:57:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.718+0000 7ff6c3403740 1 --2- 172.21.15.2:0/682533365 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a8b17e2d80 0x55a8b1bdd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.718+0000 7ff6c3403740 4 client.17478 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:57:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.718+0000 7ff6c3403740 1 -- 172.21.15.2:0/682533365 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8b1b381e0 con 0x55a8b17e2d80 2024-05-14T09:57:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.719+0000 7ff6ba2e9640 1 --2- 172.21.15.2:0/682533365 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a8b17e2d80 0x55a8b1bdd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.720+0000 7ff6ba2e9640 1 --2- 172.21.15.2:0/682533365 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a8b17e2d80 0x55a8b1bdd2c0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa38627a0eb3a10f server_cookie=8e2ac158f94ecc19 in_seq=0 out_seq=0 2024-05-14T09:57:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.720+0000 7ff69dffb640 10 client.17478 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.721+0000 7ff69dffb640 1 -- 172.21.15.2:0/682533365 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a8b1b381e0 con 0x55a8b17e2d80 2024-05-14T09:57:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.721+0000 7ff69dffb640 10 client.17478 handle_command_reply: tid=0 2024-05-14T09:57:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.722+0000 7ff6c3403740 1 client.17478 shutdown 2024-05-14T09:57:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.722+0000 7ff6c3403740 1 -- 172.21.15.2:0/682533365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 msgr2=0x55a8b1bcfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.722+0000 7ff6c3403740 1 --2- 172.21.15.2:0/682533365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 0x55a8b1bcfc70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff6a40a5e70 tx=0x7ff6a40a9900 comp rx=0 tx=0).stop 2024-05-14T09:57:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.722+0000 7ff6c3403740 1 -- 172.21.15.2:0/682533365 shutdown_connections 2024-05-14T09:57:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.722+0000 7ff6c3403740 1 --2- 172.21.15.2:0/682533365 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a8b17e2d80 0x55a8b1bdd2c0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.722+0000 7ff6c3403740 1 --2- 172.21.15.2:0/682533365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 0x55a8b1bd6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.722+0000 7ff6c3403740 1 --2- 172.21.15.2:0/682533365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 0x55a8b1bcfc70 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.722+0000 7ff6c3403740 1 --2- 172.21.15.2:0/682533365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bcf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.722+0000 7ff6c3403740 1 -- 172.21.15.2:0/682533365 >> 172.21.15.2:0/682533365 conn(0x55a8b1818300 msgr2=0x55a8b1bc03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.722+0000 7ff69dffb640 0 client.17478 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.722+0000 7ff6c3403740 1 -- 172.21.15.2:0/682533365 shutdown_connections 2024-05-14T09:57:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.722+0000 7ff6c3403740 1 -- 172.21.15.2:0/682533365 wait complete. 2024-05-14T09:57:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.722+0000 7ff6c3403740 20 client.17478 trim_cache size 0 max 16384 2024-05-14T09:57:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.758+0000 7ff6c3403740 1 Processor -- start 2024-05-14T09:57:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.758+0000 7ff6c3403740 1 -- start start 2024-05-14T09:57:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.758+0000 7ff6c3403740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.758+0000 7ff6c3403740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 0x55a8b1bc0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.759+0000 7ff6c3403740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 0x55a8b1bcde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.759+0000 7ff6b9ae8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 0x55a8b1bc0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.759+0000 7ff6c3403740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff698012570 con 0x55a8b18034d0 2024-05-14T09:57:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.759+0000 7ff6c3403740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff69800fe40 con 0x55a8b17fb2f0 2024-05-14T09:57:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.759+0000 7ff6c3403740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff69800dd40 con 0x55a8b18241b0 2024-05-14T09:57:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.759+0000 7ff6b9ae8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 0x55a8b1bc0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60780/0 (socket says 172.21.15.2:60780) 2024-05-14T09:57:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.759+0000 7ff6b9ae8640 1 -- 172.21.15.2:0/3036713448 learned_addr learned my addr 172.21.15.2:0/3036713448 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.759+0000 7ff6baaea640 1 --2- 172.21.15.2:0/3036713448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 0x55a8b1bcde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.760+0000 7ff6baaea640 1 -- 172.21.15.2:0/3036713448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b17fb2f0 msgr2=0x55a8b1bca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.760+0000 7ff6baaea640 1 --2- 172.21.15.2:0/3036713448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.760+0000 7ff6baaea640 1 -- 172.21.15.2:0/3036713448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 msgr2=0x55a8b1bc0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.760+0000 7ff6baaea640 1 --2- 172.21.15.2:0/3036713448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 0x55a8b1bc0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.760+0000 7ff6baaea640 1 -- 172.21.15.2:0/3036713448 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8b1bc1d00 con 0x55a8b18241b0 2024-05-14T09:57:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.760+0000 7ff6b9ae8640 1 --2- 172.21.15.2:0/3036713448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 0x55a8b1bc0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:57:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.760+0000 7ff6baaea640 1 --2- 172.21.15.2:0/3036713448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 0x55a8b1bcde80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac02c010 tx=0x7ff6ac005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2046e2211a34e4f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.761+0000 7ff69ffff640 1 -- 172.21.15.2:0/3036713448 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac03f040 con 0x55a8b18241b0 2024-05-14T09:57:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.761+0000 7ff69ffff640 1 -- 172.21.15.2:0/3036713448 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac036040 con 0x55a8b18241b0 2024-05-14T09:57:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.761+0000 7ff69ffff640 1 -- 172.21.15.2:0/3036713448 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac004380 con 0x55a8b18241b0 2024-05-14T09:57:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.762+0000 7ff6c3403740 1 -- 172.21.15.2:0/3036713448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 msgr2=0x55a8b1bcde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.762+0000 7ff6c3403740 1 --2- 172.21.15.2:0/3036713448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 0x55a8b1bcde80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac02c010 tx=0x7ff6ac005fd0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.763+0000 7ff6c3403740 1 -- 172.21.15.2:0/3036713448 shutdown_connections 2024-05-14T09:57:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.763+0000 7ff6c3403740 1 --2- 172.21.15.2:0/3036713448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18241b0 0x55a8b1bcde80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.763+0000 7ff6c3403740 1 --2- 172.21.15.2:0/3036713448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b18034d0 0x55a8b1bc0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.763+0000 7ff6c3403740 1 --2- 172.21.15.2:0/3036713448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.763+0000 7ff6c3403740 1 -- 172.21.15.2:0/3036713448 >> 172.21.15.2:0/3036713448 conn(0x55a8b1818300 msgr2=0x55a8b1bbf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.763+0000 7ff6c3403740 1 -- 172.21.15.2:0/3036713448 shutdown_connections 2024-05-14T09:57:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.763+0000 7ff6c3403740 1 -- 172.21.15.2:0/3036713448 wait complete. 2024-05-14T09:57:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.764+0000 7ff6c3403740 1 -- start start 2024-05-14T09:57:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.764+0000 7ff6c3403740 1 Processor -- start 2024-05-14T09:57:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.764+0000 7ff6c3403740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bd1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6c3403740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18034d0 0x55a8b1bd1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6c3403740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b18241b0 0x55a8b1bd4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6c3403740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff69800e300 con 0x55a8b17fb2f0 2024-05-14T09:57:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6c3403740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff69800f770 con 0x55a8b18241b0 2024-05-14T09:57:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6ba2e9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bd1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6c3403740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff69800e9d0 con 0x55a8b18034d0 2024-05-14T09:57:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6ba2e9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bd1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60784/0 (socket says 172.21.15.2:60784) 2024-05-14T09:57:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6ba2e9640 1 -- 172.21.15.2:0/2197336019 learned_addr learned my addr 172.21.15.2:0/2197336019 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6baaea640 1 --2- 172.21.15.2:0/2197336019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b18241b0 0x55a8b1bd4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6c3403740 10 client.0 _ll_register_callbacks cb 0x7ff6a400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6ba2e9640 1 -- 172.21.15.2:0/2197336019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18034d0 msgr2=0x55a8b1bd1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6ba2e9640 1 --2- 172.21.15.2:0/2197336019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18034d0 0x55a8b1bd1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6ba2e9640 1 -- 172.21.15.2:0/2197336019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b18241b0 msgr2=0x55a8b1bd4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6ba2e9640 1 --2- 172.21.15.2:0/2197336019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b18241b0 0x55a8b1bd4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6ba2e9640 1 -- 172.21.15.2:0/2197336019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8b1bd5380 con 0x55a8b17fb2f0 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff6ba2e9640 1 --2- 172.21.15.2:0/2197336019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bd1990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x55a8b1bdda30 tx=0x7ff6b0004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc25b18aacc347f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.765+0000 7ff69dffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.766+0000 7ff69dffb640 1 -- 172.21.15.2:0/2197336019 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b000ee60 con 0x55a8b17fb2f0 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.766+0000 7ff6c3403740 1 -- 172.21.15.2:0/2197336019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ac036040 con 0x55a8b17fb2f0 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.766+0000 7ff69dffb640 1 -- 172.21.15.2:0/2197336019 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7ff6b000f040 con 0x55a8b17fb2f0 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.766+0000 7ff69dffb640 1 -- 172.21.15.2:0/2197336019 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7ff6b0037050 con 0x55a8b17fb2f0 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.766+0000 7ff6c3403740 10 client.17484 fetch_fsmap learned FSMap version 67 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.766+0000 7ff6c3403740 1 -- 172.21.15.2:0/2197336019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55a8b1c45670 con 0x55a8b17fb2f0 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.766+0000 7ff69dffb640 1 -- 172.21.15.2:0/2197336019 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac036040 con 0x55a8b17fb2f0 2024-05-14T09:57:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.766+0000 7ff69dffb640 1 -- 172.21.15.2:0/2197336019 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7ff6b002c020 con 0x55a8b17fb2f0 2024-05-14T09:57:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.766+0000 7ff6c3403740 10 client.17484 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:57:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.766+0000 7ff6c3403740 10 client.17484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:57:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.767+0000 7ff6c3403740 20 client.17484 populate_metadata read hostname 'smithi002' 2024-05-14T09:57:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.767+0000 7ff6c3403740 1 --2- 172.21.15.2:0/2197336019 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a8b17b81c0 0x55a8b1c48040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.767+0000 7ff6c3403740 4 client.17484 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:57:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.767+0000 7ff6c3403740 1 -- 172.21.15.2:0/2197336019 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a8b1b32050 con 0x55a8b17b81c0 2024-05-14T09:57:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.767+0000 7ff6b9ae8640 1 --2- 172.21.15.2:0/2197336019 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a8b17b81c0 0x55a8b1c48040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.769+0000 7ff6b9ae8640 1 --2- 172.21.15.2:0/2197336019 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a8b17b81c0 0x55a8b1c48040 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=240acdeff281ffa5 server_cookie=10b39f88be8f8b63 in_seq=0 out_seq=0 2024-05-14T09:57:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:19.769+0000 7ff69dffb640 10 client.17484 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.457+0000 7ff69dffb640 1 -- 172.21.15.2:0/2197336019 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a8b1b32050 con 0x55a8b17b81c0 2024-05-14T09:57:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.457+0000 7ff69dffb640 10 client.17484 handle_command_reply: tid=0 2024-05-14T09:57:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.457+0000 7ff6c3403740 1 client.17484 shutdown 2024-05-14T09:57:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.457+0000 7ff6c3403740 1 -- 172.21.15.2:0/2197336019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b17fb2f0 msgr2=0x55a8b1bd1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.457+0000 7ff6c3403740 1 --2- 172.21.15.2:0/2197336019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bd1990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x55a8b1bdda30 tx=0x7ff6b0004840 comp rx=0 tx=0).stop 2024-05-14T09:57:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.458+0000 7ff6c3403740 1 -- 172.21.15.2:0/2197336019 shutdown_connections 2024-05-14T09:57:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.458+0000 7ff6c3403740 1 --2- 172.21.15.2:0/2197336019 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a8b17b81c0 0x55a8b1c48040 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.458+0000 7ff6c3403740 1 --2- 172.21.15.2:0/2197336019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8b18241b0 0x55a8b1bd4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.458+0000 7ff6c3403740 1 --2- 172.21.15.2:0/2197336019 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a8b18034d0 0x55a8b1bd1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.458+0000 7ff6c3403740 1 --2- 172.21.15.2:0/2197336019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a8b17fb2f0 0x55a8b1bd1990 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.458+0000 7ff69dffb640 0 client.17484 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.458+0000 7ff6c3403740 1 -- 172.21.15.2:0/2197336019 >> 172.21.15.2:0/2197336019 conn(0x55a8b1818300 msgr2=0x55a8b1bbf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.458+0000 7ff6c3403740 1 -- 172.21.15.2:0/2197336019 shutdown_connections 2024-05-14T09:57:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.458+0000 7ff6c3403740 1 -- 172.21.15.2:0/2197336019 wait complete. 2024-05-14T09:57:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.458+0000 7ff6c3403740 20 client.17484 trim_cache size 0 max 16384 2024-05-14T09:57:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:57:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:57:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:57:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:57:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.459+0000 7ff6bc54d640 1 -- 172.21.15.2:0/126997365 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff6a00b2a70 msgr2=0x7ff6a00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.459+0000 7ff6bc54d640 1 --2- 172.21.15.2:0/126997365 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff6a00b2a70 0x7ff6a00b4f80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55a8b1bc6e40 tx=0x7ff6b0009190 comp rx=0 tx=0).stop 2024-05-14T09:57:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.459+0000 7ff6bc54d640 1 -- 172.21.15.2:0/126997365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b4104ed0 msgr2=0x7ff6b4075040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.459+0000 7ff6bc54d640 1 --2- 172.21.15.2:0/126997365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b4104ed0 0x7ff6b4075040 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400e9f0 tx=0x7ff6a400edf0 comp rx=0 tx=0).stop 2024-05-14T09:57:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.459+0000 7ff6bc54d640 1 -- 172.21.15.2:0/126997365 shutdown_connections 2024-05-14T09:57:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.459+0000 7ff6bc54d640 1 --2- 172.21.15.2:0/126997365 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff6a00b2a70 0x7ff6a00b4f80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.459+0000 7ff6bc54d640 1 --2- 172.21.15.2:0/126997365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b4109bd0 0x7ff6b4075580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.459+0000 7ff6bc54d640 1 --2- 172.21.15.2:0/126997365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b4104ed0 0x7ff6b4075040 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.459+0000 7ff6bc54d640 1 --2- 172.21.15.2:0/126997365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff6b4103c90 0x7ff6b40769a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.459+0000 7ff6bc54d640 1 -- 172.21.15.2:0/126997365 >> 172.21.15.2:0/126997365 conn(0x7ff6b40feea0 msgr2=0x7ff6b4105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.459+0000 7ff6bc54d640 1 -- 172.21.15.2:0/126997365 shutdown_connections 2024-05-14T09:57:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:20.459+0000 7ff6bc54d640 1 -- 172.21.15.2:0/126997365 wait complete. 2024-05-14T09:57:20.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 261832 25.95 MB/sec execute 406 sec latency 575.276 ms 2024-05-14T09:57:21.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:20 smithi002 ceph-mon[31557]: pgmap v1098: 129 pgs: 129 active+clean; 701 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.7 MiB/s wr, 640 op/s 2024-05-14T09:57:21.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:20 smithi029 ceph-mon[36249]: pgmap v1098: 129 pgs: 129 active+clean; 701 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.7 MiB/s wr, 640 op/s 2024-05-14T09:57:21.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:20 smithi026 ceph-mon[36056]: pgmap v1098: 129 pgs: 129 active+clean; 701 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.7 MiB/s wr, 640 op/s 2024-05-14T09:57:21.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 262352 25.94 MB/sec execute 407 sec latency 79.744 ms 2024-05-14T09:57:22.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 262921 25.95 MB/sec execute 408 sec latency 79.476 ms 2024-05-14T09:57:23.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:22 smithi002 ceph-mon[31557]: pgmap v1099: 129 pgs: 129 active+clean; 700 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.1 MiB/s wr, 510 op/s 2024-05-14T09:57:23.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:22 smithi029 ceph-mon[36249]: pgmap v1099: 129 pgs: 129 active+clean; 700 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.1 MiB/s wr, 510 op/s 2024-05-14T09:57:23.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:22 smithi026 ceph-mon[36056]: pgmap v1099: 129 pgs: 129 active+clean; 700 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.1 MiB/s wr, 510 op/s 2024-05-14T09:57:23.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 263557 25.97 MB/sec execute 409 sec latency 78.144 ms 2024-05-14T09:57:24.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 264195 25.98 MB/sec execute 410 sec latency 81.238 ms 2024-05-14T09:57:25.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:24 smithi002 ceph-mon[31557]: pgmap v1100: 129 pgs: 129 active+clean; 703 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.1 MiB/s wr, 627 op/s 2024-05-14T09:57:25.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:24 smithi029 ceph-mon[36249]: pgmap v1100: 129 pgs: 129 active+clean; 703 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.1 MiB/s wr, 627 op/s 2024-05-14T09:57:25.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:24 smithi026 ceph-mon[36056]: pgmap v1100: 129 pgs: 129 active+clean; 703 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.1 MiB/s wr, 627 op/s 2024-05-14T09:57:25.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 264775 26.02 MB/sec execute 411 sec latency 80.908 ms 2024-05-14T09:57:26.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 265282 26.02 MB/sec execute 412 sec latency 234.413 ms 2024-05-14T09:57:26.701 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:26 smithi026 ceph-mon[36056]: pgmap v1101: 129 pgs: 129 active+clean; 688 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.8 MiB/s wr, 490 op/s 2024-05-14T09:57:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:26 smithi002 ceph-mon[31557]: pgmap v1101: 129 pgs: 129 active+clean; 688 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.8 MiB/s wr, 490 op/s 2024-05-14T09:57:26.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:26 smithi029 ceph-mon[36249]: pgmap v1101: 129 pgs: 129 active+clean; 688 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.8 MiB/s wr, 490 op/s 2024-05-14T09:57:27.622 INFO:tasks.workunit.client.0.smithi002.stdout: 10 265772 26.01 MB/sec execute 413 sec latency 82.174 ms 2024-05-14T09:57:28.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 266370 26.02 MB/sec execute 414 sec latency 92.381 ms 2024-05-14T09:57:28.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:28 smithi002 ceph-mon[31557]: pgmap v1102: 129 pgs: 129 active+clean; 677 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.9 MiB/s wr, 485 op/s 2024-05-14T09:57:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:28 smithi029 ceph-mon[36249]: pgmap v1102: 129 pgs: 129 active+clean; 677 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.9 MiB/s wr, 485 op/s 2024-05-14T09:57:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:28 smithi026 ceph-mon[36056]: pgmap v1102: 129 pgs: 129 active+clean; 677 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.9 MiB/s wr, 485 op/s 2024-05-14T09:57:29.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 267010 26.05 MB/sec execute 415 sec latency 123.546 ms 2024-05-14T09:57:30.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 267705 26.07 MB/sec execute 416 sec latency 80.459 ms 2024-05-14T09:57:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:30 smithi002 ceph-mon[31557]: pgmap v1103: 129 pgs: 129 active+clean; 676 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.3 MiB/s wr, 522 op/s 2024-05-14T09:57:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:30 smithi029 ceph-mon[36249]: pgmap v1103: 129 pgs: 129 active+clean; 676 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.3 MiB/s wr, 522 op/s 2024-05-14T09:57:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:30 smithi026 ceph-mon[36056]: pgmap v1103: 129 pgs: 129 active+clean; 676 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.3 MiB/s wr, 522 op/s 2024-05-14T09:57:31.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 268253 26.07 MB/sec execute 417 sec latency 103.001 ms 2024-05-14T09:57:32.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 268650 26.07 MB/sec execute 418 sec latency 374.173 ms 2024-05-14T09:57:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:32 smithi002 ceph-mon[31557]: pgmap v1104: 129 pgs: 129 active+clean; 676 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 518 op/s 2024-05-14T09:57:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:32 smithi029 ceph-mon[36249]: pgmap v1104: 129 pgs: 129 active+clean; 676 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 518 op/s 2024-05-14T09:57:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:32 smithi026 ceph-mon[36056]: pgmap v1104: 129 pgs: 129 active+clean; 676 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 518 op/s 2024-05-14T09:57:33.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 268654 26.01 MB/sec execute 419 sec latency 1371.155 ms 2024-05-14T09:57:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.655+0000 7fe16b3b7640 1 -- 172.21.15.2:0/2107015167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1641101a0 msgr2=0x7fe1641105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.655+0000 7fe16b3b7640 1 --2- 172.21.15.2:0/2107015167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1641101a0 0x7fe1641105b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009ca0 tx=0x7fe15c02a8f0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.662+0000 7fe16b3b7640 1 -- 172.21.15.2:0/2107015167 shutdown_connections 2024-05-14T09:57:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.662+0000 7fe16b3b7640 1 --2- 172.21.15.2:0/2107015167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1641101a0 0x7fe1641105b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.662+0000 7fe16b3b7640 1 --2- 172.21.15.2:0/2107015167 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe16410b480 0x7fe16410b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.662+0000 7fe16b3b7640 1 --2- 172.21.15.2:0/2107015167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe16410a550 0x7fe16410af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.662+0000 7fe16b3b7640 1 -- 172.21.15.2:0/2107015167 >> 172.21.15.2:0/2107015167 conn(0x7fe164070640 msgr2=0x7fe164072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.669+0000 7fe16b3b7640 1 -- 172.21.15.2:0/2107015167 shutdown_connections 2024-05-14T09:57:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.672+0000 7fe16b3b7640 1 -- 172.21.15.2:0/2107015167 wait complete. 2024-05-14T09:57:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.674+0000 7fe16b3b7640 1 Processor -- start 2024-05-14T09:57:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.678+0000 7fe16b3b7640 1 -- start start 2024-05-14T09:57:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.678+0000 7fe16b3b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe16410a550 0x7fe1641a9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.678+0000 7fe16b3b7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe16410b480 0x7fe1641a95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.679+0000 7fe16b3b7640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1641101a0 0x7fe1641b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.679+0000 7fe16b3b7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe16411e700 con 0x7fe16410b480 2024-05-14T09:57:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.679+0000 7fe16b3b7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe16411e580 con 0x7fe16410a550 2024-05-14T09:57:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.679+0000 7fe16b3b7640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe16411e880 con 0x7fe1641101a0 2024-05-14T09:57:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.680+0000 7fe169153640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe16410a550 0x7fe1641a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.680+0000 7fe169153640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe16410a550 0x7fe1641a9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:45570/0 (socket says 172.21.15.2:45570) 2024-05-14T09:57:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.680+0000 7fe168952640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe16410b480 0x7fe1641a95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.680+0000 7fe168952640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe16410b480 0x7fe1641a95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56934/0 (socket says 172.21.15.2:56934) 2024-05-14T09:57:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.680+0000 7fe169153640 1 -- 172.21.15.2:0/1855513437 learned_addr learned my addr 172.21.15.2:0/1855513437 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.680+0000 7fe169954640 1 --2- 172.21.15.2:0/1855513437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1641101a0 0x7fe1641b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.682+0000 7fe169153640 1 -- 172.21.15.2:0/1855513437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1641101a0 msgr2=0x7fe1641b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.682+0000 7fe169153640 1 --2- 172.21.15.2:0/1855513437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1641101a0 0x7fe1641b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.682+0000 7fe169153640 1 -- 172.21.15.2:0/1855513437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe16410b480 msgr2=0x7fe1641a95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.682+0000 7fe169153640 1 --2- 172.21.15.2:0/1855513437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe16410b480 0x7fe1641a95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.683+0000 7fe169153640 1 -- 172.21.15.2:0/1855513437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1641b0bd0 con 0x7fe16410a550 2024-05-14T09:57:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.685+0000 7fe169153640 1 --2- 172.21.15.2:0/1855513437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe16410a550 0x7fe1641a9090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe160006fd0 tx=0x7fe16000b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.689+0000 7fe15a7fc640 1 -- 172.21.15.2:0/1855513437 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160011830 con 0x7fe16410a550 2024-05-14T09:57:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.689+0000 7fe15a7fc640 1 -- 172.21.15.2:0/1855513437 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe160016070 con 0x7fe16410a550 2024-05-14T09:57:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.689+0000 7fe15a7fc640 1 -- 172.21.15.2:0/1855513437 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160010e00 con 0x7fe16410a550 2024-05-14T09:57:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.692+0000 7fe16b3b7640 1 -- 172.21.15.2:0/1855513437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1641b0ec0 con 0x7fe16410a550 2024-05-14T09:57:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.692+0000 7fe16b3b7640 1 -- 172.21.15.2:0/1855513437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1641b1350 con 0x7fe16410a550 2024-05-14T09:57:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.700+0000 7fe17226d740 1 Processor -- start 2024-05-14T09:57:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.701+0000 7fe17226d740 1 -- start start 2024-05-14T09:57:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.701+0000 7fe17226d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf897a1b0 0x555cf8d1ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.701+0000 7fe17226d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf89512f0 0x555cf8d1bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.701+0000 7fe17226d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89594d0 0x555cf8d22ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.701+0000 7fe17226d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555cf8c805d0 con 0x555cf897a1b0 2024-05-14T09:57:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.701+0000 7fe17226d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555cf8c39890 con 0x555cf89512f0 2024-05-14T09:57:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.701+0000 7fe17226d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555cf8c7c590 con 0x555cf89594d0 2024-05-14T09:57:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.701+0000 7fe15a7fc640 1 -- 172.21.15.2:0/1855513437 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe16001b460 con 0x7fe16410a550 2024-05-14T09:57:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.703+0000 7fe15a7fc640 1 --2- 172.21.15.2:0/1855513437 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe1500b2ac0 0x7fe1500b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.703+0000 7fe15a7fc640 1 -- 172.21.15.2:0/1855513437 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fe1600e2b80 con 0x7fe16410a550 2024-05-14T09:57:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.703+0000 7fe168952640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89594d0 0x555cf8d22ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.703+0000 7fe168952640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89594d0 0x555cf8d22ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:54550/0 (socket says 172.21.15.2:54550) 2024-05-14T09:57:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.703+0000 7fe168952640 1 -- 172.21.15.2:0/44143795 learned_addr learned my addr 172.21.15.2:0/44143795 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.703+0000 7fe169153640 1 --2- 172.21.15.2:0/44143795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf89512f0 0x555cf8d1bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.708+0000 7fe168952640 1 -- 172.21.15.2:0/44143795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf89512f0 msgr2=0x555cf8d1bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.708+0000 7fe168952640 1 --2- 172.21.15.2:0/44143795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf89512f0 0x555cf8d1bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.708+0000 7fe168952640 1 -- 172.21.15.2:0/44143795 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf897a1b0 msgr2=0x555cf8d1ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.708+0000 7fe168952640 1 --2- 172.21.15.2:0/44143795 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf897a1b0 0x555cf8d1ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.708+0000 7fe168952640 1 -- 172.21.15.2:0/44143795 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf8992210 con 0x555cf89594d0 2024-05-14T09:57:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.709+0000 7fe168952640 1 --2- 172.21.15.2:0/44143795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89594d0 0x555cf8d22ee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe154009a10 tx=0x7fe154009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32828d0fe7ecadbc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.711+0000 7fe169954640 1 --2- 172.21.15.2:0/1855513437 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe1500b2ac0 0x7fe1500b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.719+0000 7fe14f7fe640 1 -- 172.21.15.2:0/44143795 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154017070 con 0x555cf89594d0 2024-05-14T09:57:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.719+0000 7fe14f7fe640 1 -- 172.21.15.2:0/44143795 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe154013470 con 0x555cf89594d0 2024-05-14T09:57:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.719+0000 7fe14f7fe640 1 -- 172.21.15.2:0/44143795 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154012630 con 0x555cf89594d0 2024-05-14T09:57:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.721+0000 7fe17226d740 1 -- 172.21.15.2:0/44143795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89594d0 msgr2=0x555cf8d22ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.721+0000 7fe17226d740 1 --2- 172.21.15.2:0/44143795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89594d0 0x555cf8d22ee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe154009a10 tx=0x7fe154009ed0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.724+0000 7fe169954640 1 --2- 172.21.15.2:0/1855513437 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe1500b2ac0 0x7fe1500b4fd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0094e0 tx=0x7fe15c02adf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.724+0000 7fe17226d740 1 -- 172.21.15.2:0/44143795 shutdown_connections 2024-05-14T09:57:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.724+0000 7fe17226d740 1 --2- 172.21.15.2:0/44143795 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89594d0 0x555cf8d22ee0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.724+0000 7fe17226d740 1 --2- 172.21.15.2:0/44143795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf89512f0 0x555cf8d1bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.724+0000 7fe17226d740 1 --2- 172.21.15.2:0/44143795 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf897a1b0 0x555cf8d1ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.724+0000 7fe17226d740 1 -- 172.21.15.2:0/44143795 >> 172.21.15.2:0/44143795 conn(0x555cf896e300 msgr2=0x555cf8d173a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.732+0000 7fe17226d740 1 -- 172.21.15.2:0/44143795 shutdown_connections 2024-05-14T09:57:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.736+0000 7fe17226d740 1 -- 172.21.15.2:0/44143795 wait complete. 2024-05-14T09:57:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.736+0000 7fe17226d740 1 -- start start 2024-05-14T09:57:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.736+0000 7fe17226d740 1 Processor -- start 2024-05-14T09:57:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.737+0000 7fe17226d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89512f0 0x555cf8d256d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.737+0000 7fe17226d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf89594d0 0x555cf8d25c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.737+0000 7fe17226d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf897a1b0 0x555cf8d2cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.737+0000 7fe17226d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555cf8d23930 con 0x555cf89512f0 2024-05-14T09:57:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.737+0000 7fe17226d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555cf8c3d370 con 0x555cf89594d0 2024-05-14T09:57:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.737+0000 7fe17226d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555cf8d23ab0 con 0x555cf897a1b0 2024-05-14T09:57:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.737+0000 7fe17226d740 10 client.0 _ll_register_callbacks cb 0x555cf8991a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:57:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.739+0000 7fe169954640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf897a1b0 0x555cf8d2cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.740+0000 7fe169954640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf897a1b0 0x555cf8d2cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:54558/0 (socket says 172.21.15.2:54558) 2024-05-14T09:57:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.740+0000 7fe169954640 1 -- 172.21.15.2:0/1478072574 learned_addr learned my addr 172.21.15.2:0/1478072574 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.743+0000 7fe168952640 1 --2- 172.21.15.2:0/1478072574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf89594d0 0x555cf8d25c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.743+0000 7fe169153640 1 --2- 172.21.15.2:0/1478072574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89512f0 0x555cf8d256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:33.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.747+0000 7fe168952640 1 -- 172.21.15.2:0/1478072574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf897a1b0 msgr2=0x555cf8d2cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:33.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.747+0000 7fe168952640 1 --2- 172.21.15.2:0/1478072574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf897a1b0 0x555cf8d2cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.747+0000 7fe168952640 1 -- 172.21.15.2:0/1478072574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89512f0 msgr2=0x555cf8d256d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:33.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.747+0000 7fe168952640 1 --2- 172.21.15.2:0/1478072574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89512f0 0x555cf8d256d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.747+0000 7fe168952640 1 -- 172.21.15.2:0/1478072574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf8a5e620 con 0x555cf89594d0 2024-05-14T09:57:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.751+0000 7fe168952640 1 --2- 172.21.15.2:0/1478072574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf89594d0 0x555cf8d25c10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe15400e5a0 tx=0x7fe154013c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5793f29505e7be80 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.753+0000 7fe14cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:57:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.753+0000 7fe14cff9640 1 -- 172.21.15.2:0/1478072574 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154017050 con 0x555cf89594d0 2024-05-14T09:57:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.753+0000 7fe17226d740 1 -- 172.21.15.2:0/1478072574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe154013470 con 0x555cf89594d0 2024-05-14T09:57:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.753+0000 7fe14cff9640 1 -- 172.21.15.2:0/1478072574 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fe15401c450 con 0x555cf89594d0 2024-05-14T09:57:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.758+0000 7fe14cff9640 1 -- 172.21.15.2:0/1478072574 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fe15401c650 con 0x555cf89594d0 2024-05-14T09:57:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.762+0000 7fe17226d740 10 client.27107 fetch_fsmap learned FSMap version 67 2024-05-14T09:57:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.762+0000 7fe17226d740 1 -- 172.21.15.2:0/1478072574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x555cf8d40ab0 con 0x555cf89594d0 2024-05-14T09:57:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.767+0000 7fe14cff9640 1 -- 172.21.15.2:0/1478072574 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe154013470 con 0x555cf89594d0 2024-05-14T09:57:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.768+0000 7fe14cff9640 1 -- 172.21.15.2:0/1478072574 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fe1540309d0 con 0x555cf89594d0 2024-05-14T09:57:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.768+0000 7fe17226d740 10 client.27107 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:57:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.768+0000 7fe17226d740 10 client.27107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:57:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.768+0000 7fe17226d740 20 client.27107 populate_metadata read hostname 'smithi002' 2024-05-14T09:57:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.768+0000 7fe17226d740 1 --2- 172.21.15.2:0/1478072574 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cf8938d80 0x555cf8d44500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.768+0000 7fe17226d740 4 client.27107 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:57:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.768+0000 7fe17226d740 1 -- 172.21.15.2:0/1478072574 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cf8c8e1e0 con 0x555cf8938d80 2024-05-14T09:57:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.774+0000 7fe169153640 1 --2- 172.21.15.2:0/1478072574 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cf8938d80 0x555cf8d44500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.786+0000 7fe169153640 1 --2- 172.21.15.2:0/1478072574 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cf8938d80 0x555cf8d44500 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b7720f0664ecda server_cookie=bf5bcc8aca3712a8 in_seq=0 out_seq=0 2024-05-14T09:57:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.789+0000 7fe14cff9640 10 client.27107 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.793+0000 7fe14cff9640 1 -- 172.21.15.2:0/1478072574 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555cf8c8e1e0 con 0x555cf8938d80 2024-05-14T09:57:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.793+0000 7fe14cff9640 10 client.27107 handle_command_reply: tid=0 2024-05-14T09:57:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.794+0000 7fe17226d740 1 client.27107 shutdown 2024-05-14T09:57:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.799+0000 7fe17226d740 1 -- 172.21.15.2:0/1478072574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf89594d0 msgr2=0x555cf8d25c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.799+0000 7fe17226d740 1 --2- 172.21.15.2:0/1478072574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf89594d0 0x555cf8d25c10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe15400e5a0 tx=0x7fe154013c60 comp rx=0 tx=0).stop 2024-05-14T09:57:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.802+0000 7fe17226d740 1 -- 172.21.15.2:0/1478072574 shutdown_connections 2024-05-14T09:57:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.802+0000 7fe17226d740 1 --2- 172.21.15.2:0/1478072574 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cf8938d80 0x555cf8d44500 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.802+0000 7fe17226d740 1 --2- 172.21.15.2:0/1478072574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf897a1b0 0x555cf8d2cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.802+0000 7fe17226d740 1 --2- 172.21.15.2:0/1478072574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf89594d0 0x555cf8d25c10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.802+0000 7fe17226d740 1 --2- 172.21.15.2:0/1478072574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89512f0 0x555cf8d256d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.802+0000 7fe17226d740 1 -- 172.21.15.2:0/1478072574 >> 172.21.15.2:0/1478072574 conn(0x555cf896e300 msgr2=0x555cf8d163a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.802+0000 7fe14cff9640 0 client.27107 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.806+0000 7fe17226d740 1 -- 172.21.15.2:0/1478072574 shutdown_connections 2024-05-14T09:57:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.808+0000 7fe17226d740 1 -- 172.21.15.2:0/1478072574 wait complete. 2024-05-14T09:57:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.810+0000 7fe17226d740 20 client.27107 trim_cache size 0 max 16384 2024-05-14T09:57:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.842+0000 7fe17226d740 1 Processor -- start 2024-05-14T09:57:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.846+0000 7fe17226d740 1 -- start start 2024-05-14T09:57:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.846+0000 7fe17226d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89512f0 0x555cf8d23eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.846+0000 7fe17226d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89594d0 0x555cf8d2ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.846+0000 7fe17226d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf897a1b0 0x555cf8d243f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.846+0000 7fe169153640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89512f0 0x555cf8d23eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.846+0000 7fe169153640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89512f0 0x555cf8d23eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:54564/0 (socket says 172.21.15.2:54564) 2024-05-14T09:57:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.846+0000 7fe169153640 1 -- 172.21.15.2:0/2610641835 learned_addr learned my addr 172.21.15.2:0/2610641835 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.847+0000 7fe17226d740 1 -- 172.21.15.2:0/2610641835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641b5b50 con 0x555cf89594d0 2024-05-14T09:57:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.847+0000 7fe17226d740 1 -- 172.21.15.2:0/2610641835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641b3420 con 0x555cf897a1b0 2024-05-14T09:57:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.847+0000 7fe17226d740 1 -- 172.21.15.2:0/2610641835 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe16411a130 con 0x555cf89512f0 2024-05-14T09:57:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.847+0000 7fe168952640 1 --2- 172.21.15.2:0/2610641835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89594d0 0x555cf8d2ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.848+0000 7fe169153640 1 -- 172.21.15.2:0/2610641835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf897a1b0 msgr2=0x555cf8d243f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:57:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.848+0000 7fe169153640 1 --2- 172.21.15.2:0/2610641835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf897a1b0 0x555cf8d243f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.848+0000 7fe169153640 1 -- 172.21.15.2:0/2610641835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89594d0 msgr2=0x555cf8d2ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.848+0000 7fe169153640 1 --2- 172.21.15.2:0/2610641835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89594d0 0x555cf8d2ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.848+0000 7fe169153640 1 -- 172.21.15.2:0/2610641835 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf8d17d00 con 0x555cf89512f0 2024-05-14T09:57:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.849+0000 7fe169153640 1 --2- 172.21.15.2:0/2610641835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89512f0 0x555cf8d23eb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe1600a6f90 tx=0x7fe160010680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfc4b220d605d68f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.850+0000 7fe14effd640 1 -- 172.21.15.2:0/2610641835 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1600c0070 con 0x555cf89512f0 2024-05-14T09:57:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.850+0000 7fe14effd640 1 -- 172.21.15.2:0/2610641835 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1600b2b80 con 0x555cf89512f0 2024-05-14T09:57:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.851+0000 7fe17226d740 1 -- 172.21.15.2:0/2610641835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89512f0 msgr2=0x555cf8d23eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.851+0000 7fe17226d740 1 --2- 172.21.15.2:0/2610641835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89512f0 0x555cf8d23eb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe1600a6f90 tx=0x7fe160010680 comp rx=0 tx=0).stop 2024-05-14T09:57:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.855+0000 7fe17226d740 1 -- 172.21.15.2:0/2610641835 shutdown_connections 2024-05-14T09:57:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.855+0000 7fe17226d740 1 --2- 172.21.15.2:0/2610641835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf897a1b0 0x555cf8d243f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.855+0000 7fe17226d740 1 --2- 172.21.15.2:0/2610641835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89594d0 0x555cf8d2ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.855+0000 7fe17226d740 1 --2- 172.21.15.2:0/2610641835 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89512f0 0x555cf8d23eb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.855+0000 7fe17226d740 1 -- 172.21.15.2:0/2610641835 >> 172.21.15.2:0/2610641835 conn(0x555cf896e300 msgr2=0x555cf8d1fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.859+0000 7fe17226d740 1 -- 172.21.15.2:0/2610641835 shutdown_connections 2024-05-14T09:57:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.862+0000 7fe17226d740 1 -- 172.21.15.2:0/2610641835 wait complete. 2024-05-14T09:57:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.862+0000 7fe17226d740 1 -- start start 2024-05-14T09:57:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.863+0000 7fe17226d740 1 Processor -- start 2024-05-14T09:57:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.863+0000 7fe17226d740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89512f0 0x555cf8d2c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.863+0000 7fe17226d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89594d0 0x555cf8d29510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.863+0000 7fe17226d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf897a1b0 0x555cf8d29a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.863+0000 7fe17226d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641b18e0 con 0x555cf89594d0 2024-05-14T09:57:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.863+0000 7fe17226d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641b2d50 con 0x555cf897a1b0 2024-05-14T09:57:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.863+0000 7fe17226d740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641b1fb0 con 0x555cf89512f0 2024-05-14T09:57:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.863+0000 7fe17226d740 10 client.0 _ll_register_callbacks cb 0x7fe154009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:57:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.863+0000 7fe168952640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89594d0 0x555cf8d29510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.863+0000 7fe168952640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89594d0 0x555cf8d29510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56976/0 (socket says 172.21.15.2:56976) 2024-05-14T09:57:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.863+0000 7fe168952640 1 -- 172.21.15.2:0/1297492031 learned_addr learned my addr 172.21.15.2:0/1297492031 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.864+0000 7fe169954640 1 --2- 172.21.15.2:0/1297492031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf897a1b0 0x555cf8d29a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.865+0000 7fe168952640 1 -- 172.21.15.2:0/1297492031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89512f0 msgr2=0x555cf8d2c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.865+0000 7fe168952640 1 --2- 172.21.15.2:0/1297492031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89512f0 0x555cf8d2c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.865+0000 7fe168952640 1 -- 172.21.15.2:0/1297492031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf897a1b0 msgr2=0x555cf8d29a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.865+0000 7fe168952640 1 --2- 172.21.15.2:0/1297492031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf897a1b0 0x555cf8d29a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.865+0000 7fe168952640 1 -- 172.21.15.2:0/1297492031 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf8daa8a0 con 0x555cf89594d0 2024-05-14T09:57:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.871+0000 7fe168952640 1 --2- 172.21.15.2:0/1297492031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89594d0 0x555cf8d29510 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe154006fd0 tx=0x7fe1540266f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7971ca57d5714a41 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.872+0000 7fe14cff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:57:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.878+0000 7fe14cff9640 1 -- 172.21.15.2:0/1297492031 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154017040 con 0x555cf89594d0 2024-05-14T09:57:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.878+0000 7fe14cff9640 1 -- 172.21.15.2:0/1297492031 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fe154004020 con 0x555cf89594d0 2024-05-14T09:57:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.879+0000 7fe17226d740 1 -- 172.21.15.2:0/1297492031 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555cf8c88050 con 0x555cf89594d0 2024-05-14T09:57:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.882+0000 7fe14cff9640 1 -- 172.21.15.2:0/1297492031 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fe15400e5d0 con 0x555cf89594d0 2024-05-14T09:57:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.888+0000 7fe17226d740 10 client.17496 fetch_fsmap learned FSMap version 67 2024-05-14T09:57:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.888+0000 7fe17226d740 1 -- 172.21.15.2:0/1297492031 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x555cf8dabb10 con 0x555cf89594d0 2024-05-14T09:57:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.891+0000 7fe14cff9640 1 -- 172.21.15.2:0/1297492031 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555cf8c88050 con 0x555cf89594d0 2024-05-14T09:57:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.891+0000 7fe14cff9640 1 -- 172.21.15.2:0/1297492031 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fe154026c40 con 0x555cf89594d0 2024-05-14T09:57:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.892+0000 7fe17226d740 10 client.17496 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:57:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.892+0000 7fe17226d740 10 client.17496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:57:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.892+0000 7fe17226d740 20 client.17496 populate_metadata read hostname 'smithi002' 2024-05-14T09:57:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.892+0000 7fe17226d740 1 --2- 172.21.15.2:0/1297492031 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cf890e1c0 0x555cf8dae530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.892+0000 7fe17226d740 4 client.17496 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:57:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.892+0000 7fe17226d740 1 -- 172.21.15.2:0/1297492031 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555cf8c035f0 con 0x555cf890e1c0 2024-05-14T09:57:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.899+0000 7fe169153640 1 --2- 172.21.15.2:0/1297492031 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cf890e1c0 0x555cf8dae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.905+0000 7fe169153640 1 --2- 172.21.15.2:0/1297492031 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cf890e1c0 0x555cf8dae530 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ab29316d90a3ed server_cookie=552a31c3b5ab0c9d in_seq=0 out_seq=0 2024-05-14T09:57:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:33.906+0000 7fe14cff9640 10 client.17496 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:34.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 268690 25.95 MB/sec execute 420 sec latency 2371.198 ms 2024-05-14T09:57:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:34 smithi002 ceph-mon[31557]: pgmap v1105: 129 pgs: 129 active+clean; 693 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.7 MiB/s wr, 526 op/s 2024-05-14T09:57:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:34 smithi029 ceph-mon[36249]: pgmap v1105: 129 pgs: 129 active+clean; 693 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.7 MiB/s wr, 526 op/s 2024-05-14T09:57:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:34 smithi026 ceph-mon[36056]: pgmap v1105: 129 pgs: 129 active+clean; 693 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.7 MiB/s wr, 526 op/s 2024-05-14T09:57:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.611+0000 7fe14cff9640 1 -- 172.21.15.2:0/1297492031 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555cf8c035f0 con 0x555cf890e1c0 2024-05-14T09:57:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.611+0000 7fe14cff9640 10 client.17496 handle_command_reply: tid=0 2024-05-14T09:57:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.611+0000 7fe17226d740 1 client.17496 shutdown 2024-05-14T09:57:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.611+0000 7fe17226d740 1 -- 172.21.15.2:0/1297492031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89594d0 msgr2=0x555cf8d29510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.611+0000 7fe17226d740 1 --2- 172.21.15.2:0/1297492031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89594d0 0x555cf8d29510 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe154006fd0 tx=0x7fe1540266f0 comp rx=0 tx=0).stop 2024-05-14T09:57:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.611+0000 7fe17226d740 1 -- 172.21.15.2:0/1297492031 shutdown_connections 2024-05-14T09:57:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.611+0000 7fe17226d740 1 --2- 172.21.15.2:0/1297492031 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555cf890e1c0 0x555cf8dae530 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.611+0000 7fe17226d740 1 --2- 172.21.15.2:0/1297492031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555cf897a1b0 0x555cf8d29a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.611+0000 7fe17226d740 1 --2- 172.21.15.2:0/1297492031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555cf89594d0 0x555cf8d29510 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.611+0000 7fe17226d740 1 --2- 172.21.15.2:0/1297492031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555cf89512f0 0x555cf8d2c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.611+0000 7fe17226d740 1 -- 172.21.15.2:0/1297492031 >> 172.21.15.2:0/1297492031 conn(0x555cf896e300 msgr2=0x555cf8d23eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.612+0000 7fe14cff9640 0 client.17496 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.612+0000 7fe17226d740 1 -- 172.21.15.2:0/1297492031 shutdown_connections 2024-05-14T09:57:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.612+0000 7fe17226d740 1 -- 172.21.15.2:0/1297492031 wait complete. 2024-05-14T09:57:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.612+0000 7fe17226d740 20 client.17496 trim_cache size 0 max 16384 2024-05-14T09:57:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:57:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:57:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:57:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:57:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.613+0000 7fe16b3b7640 1 -- 172.21.15.2:0/1855513437 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe1500b2ac0 msgr2=0x7fe1500b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.613+0000 7fe16b3b7640 1 --2- 172.21.15.2:0/1855513437 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe1500b2ac0 0x7fe1500b4fd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0094e0 tx=0x7fe15c02adf0 comp rx=0 tx=0).stop 2024-05-14T09:57:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.613+0000 7fe16b3b7640 1 -- 172.21.15.2:0/1855513437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe16410a550 msgr2=0x7fe1641a9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.613+0000 7fe16b3b7640 1 --2- 172.21.15.2:0/1855513437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe16410a550 0x7fe1641a9090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe160006fd0 tx=0x7fe16000b8e0 comp rx=0 tx=0).stop 2024-05-14T09:57:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.613+0000 7fe16b3b7640 1 -- 172.21.15.2:0/1855513437 shutdown_connections 2024-05-14T09:57:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.613+0000 7fe16b3b7640 1 --2- 172.21.15.2:0/1855513437 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe1500b2ac0 0x7fe1500b4fd0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.613+0000 7fe16b3b7640 1 --2- 172.21.15.2:0/1855513437 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe1641101a0 0x7fe1641b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.613+0000 7fe16b3b7640 1 --2- 172.21.15.2:0/1855513437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe16410b480 0x7fe1641a95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.613+0000 7fe16b3b7640 1 --2- 172.21.15.2:0/1855513437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe16410a550 0x7fe1641a9090 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.613+0000 7fe16b3b7640 1 -- 172.21.15.2:0/1855513437 >> 172.21.15.2:0/1855513437 conn(0x7fe164070640 msgr2=0x7fe164072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.616+0000 7fe16b3b7640 1 -- 172.21.15.2:0/1855513437 shutdown_connections 2024-05-14T09:57:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:35.616+0000 7fe16b3b7640 1 -- 172.21.15.2:0/1855513437 wait complete. 2024-05-14T09:57:35.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 268795 25.90 MB/sec execute 421 sec latency 2549.046 ms 2024-05-14T09:57:36.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 269026 25.86 MB/sec execute 422 sec latency 303.024 ms 2024-05-14T09:57:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:36 smithi002 ceph-mon[31557]: pgmap v1106: 129 pgs: 129 active+clean; 718 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.2 MiB/s wr, 400 op/s 2024-05-14T09:57:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:36 smithi029 ceph-mon[36249]: pgmap v1106: 129 pgs: 129 active+clean; 718 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.2 MiB/s wr, 400 op/s 2024-05-14T09:57:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:36 smithi026 ceph-mon[36056]: pgmap v1106: 129 pgs: 129 active+clean; 718 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.2 MiB/s wr, 400 op/s 2024-05-14T09:57:37.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 269288 25.85 MB/sec execute 423 sec latency 171.028 ms 2024-05-14T09:57:38.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 269546 25.82 MB/sec execute 424 sec latency 188.051 ms 2024-05-14T09:57:38.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:38 smithi002 ceph-mon[31557]: pgmap v1107: 129 pgs: 129 active+clean; 750 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.8 MiB/s wr, 486 op/s 2024-05-14T09:57:38.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:38 smithi029 ceph-mon[36249]: pgmap v1107: 129 pgs: 129 active+clean; 750 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.8 MiB/s wr, 486 op/s 2024-05-14T09:57:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:38 smithi026 ceph-mon[36056]: pgmap v1107: 129 pgs: 129 active+clean; 750 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.8 MiB/s wr, 486 op/s 2024-05-14T09:57:39.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 269880 25.79 MB/sec execute 425 sec latency 272.682 ms 2024-05-14T09:57:40.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 270575 25.83 MB/sec execute 426 sec latency 130.868 ms 2024-05-14T09:57:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:40 smithi002 ceph-mon[31557]: pgmap v1108: 129 pgs: 129 active+clean; 728 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 12 MiB/s wr, 656 op/s 2024-05-14T09:57:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:40 smithi029 ceph-mon[36249]: pgmap v1108: 129 pgs: 129 active+clean; 728 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 12 MiB/s wr, 656 op/s 2024-05-14T09:57:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:40 smithi026 ceph-mon[36056]: pgmap v1108: 129 pgs: 129 active+clean; 728 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 12 MiB/s wr, 656 op/s 2024-05-14T09:57:41.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 270862 25.81 MB/sec execute 427 sec latency 114.759 ms 2024-05-14T09:57:42.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 271324 25.81 MB/sec execute 428 sec latency 314.518 ms 2024-05-14T09:57:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:42 smithi002 ceph-mon[31557]: pgmap v1109: 129 pgs: 129 active+clean; 699 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 12 MiB/s wr, 611 op/s 2024-05-14T09:57:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:42 smithi029 ceph-mon[36249]: pgmap v1109: 129 pgs: 129 active+clean; 699 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 12 MiB/s wr, 611 op/s 2024-05-14T09:57:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:42 smithi026 ceph-mon[36056]: pgmap v1109: 129 pgs: 129 active+clean; 699 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 12 MiB/s wr, 611 op/s 2024-05-14T09:57:43.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 271814 25.79 MB/sec execute 429 sec latency 125.832 ms 2024-05-14T09:57:44.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 272586 25.82 MB/sec execute 430 sec latency 81.329 ms 2024-05-14T09:57:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:44 smithi002 ceph-mon[31557]: pgmap v1110: 129 pgs: 129 active+clean; 680 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 12 MiB/s wr, 629 op/s 2024-05-14T09:57:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:44 smithi029 ceph-mon[36249]: pgmap v1110: 129 pgs: 129 active+clean; 680 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 12 MiB/s wr, 629 op/s 2024-05-14T09:57:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:44 smithi026 ceph-mon[36056]: pgmap v1110: 129 pgs: 129 active+clean; 680 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 12 MiB/s wr, 629 op/s 2024-05-14T09:57:45.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 273250 25.86 MB/sec execute 431 sec latency 50.308 ms 2024-05-14T09:57:46.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 273932 25.90 MB/sec execute 432 sec latency 78.668 ms 2024-05-14T09:57:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:46 smithi002 ceph-mon[31557]: pgmap v1111: 129 pgs: 129 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 594 op/s 2024-05-14T09:57:46.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:46 smithi029 ceph-mon[36249]: pgmap v1111: 129 pgs: 129 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 594 op/s 2024-05-14T09:57:46.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:46 smithi026 ceph-mon[36056]: pgmap v1111: 129 pgs: 129 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 594 op/s 2024-05-14T09:57:47.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 274220 25.88 MB/sec execute 433 sec latency 354.669 ms 2024-05-14T09:57:48.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 274674 25.87 MB/sec execute 434 sec latency 216.850 ms 2024-05-14T09:57:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.791+0000 7f738c334640 1 -- 172.21.15.2:0/2269153893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73841105b0 msgr2=0x7f73841129d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.791+0000 7f738c334640 1 --2- 172.21.15.2:0/2269153893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73841105b0 0x7f73841129d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f737c009ca0 tx=0x7f737c02a8f0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.791+0000 7f738c334640 1 -- 172.21.15.2:0/2269153893 shutdown_connections 2024-05-14T09:57:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.791+0000 7f738c334640 1 --2- 172.21.15.2:0/2269153893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73841105b0 0x7f73841129d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.791+0000 7f738c334640 1 --2- 172.21.15.2:0/2269153893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f738410b8b0 0x7f738410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.791+0000 7f738c334640 1 --2- 172.21.15.2:0/2269153893 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f738410a670 0x7f738410b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.791+0000 7f738c334640 1 -- 172.21.15.2:0/2269153893 >> 172.21.15.2:0/2269153893 conn(0x7f7384070a80 msgr2=0x7f7384072f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.792+0000 7f738c334640 1 -- 172.21.15.2:0/2269153893 shutdown_connections 2024-05-14T09:57:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.792+0000 7f738c334640 1 -- 172.21.15.2:0/2269153893 wait complete. 2024-05-14T09:57:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.793+0000 7f738c334640 1 Processor -- start 2024-05-14T09:57:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.793+0000 7f738c334640 1 -- start start 2024-05-14T09:57:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.793+0000 7f738c334640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f738410a670 0x7f738411b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.793+0000 7f738c334640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f738410b8b0 0x7f738411bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.793+0000 7f738c334640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73841105b0 0x7f73841b8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.793+0000 7f738c334640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7384115160 con 0x7f73841105b0 2024-05-14T09:57:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.793+0000 7f738c334640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7384114fe0 con 0x7f738410b8b0 2024-05-14T09:57:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.793+0000 7f738c334640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f73841152e0 con 0x7f738410a670 2024-05-14T09:57:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.793+0000 7f738a0d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f738410a670 0x7f738411b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.793+0000 7f738a8d1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73841105b0 0x7f73841b8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.793+0000 7f738a0d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f738410a670 0x7f738411b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:59572/0 (socket says 172.21.15.2:59572) 2024-05-14T09:57:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.793+0000 7f738a0d0640 1 -- 172.21.15.2:0/1093083488 learned_addr learned my addr 172.21.15.2:0/1093083488 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.793+0000 7f73898cf640 1 --2- 172.21.15.2:0/1093083488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f738410b8b0 0x7f738411bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.794+0000 7f738a0d0640 1 -- 172.21.15.2:0/1093083488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f738410b8b0 msgr2=0x7f738411bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.794+0000 7f738a0d0640 1 --2- 172.21.15.2:0/1093083488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f738410b8b0 0x7f738411bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.794+0000 7f738a0d0640 1 -- 172.21.15.2:0/1093083488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73841105b0 msgr2=0x7f73841b8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.794+0000 7f738a0d0640 1 --2- 172.21.15.2:0/1093083488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73841105b0 0x7f73841b8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.794+0000 7f738a0d0640 1 -- 172.21.15.2:0/1093083488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73841b9230 con 0x7f738410a670 2024-05-14T09:57:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.794+0000 7f738a8d1640 1 --2- 172.21.15.2:0/1093083488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73841105b0 0x7f73841b8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:57:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.794+0000 7f738a0d0640 1 --2- 172.21.15.2:0/1093083488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f738410a670 0x7f738411b9c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f73800077b0 tx=0x7f7380007bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.795+0000 7f737b7fe640 1 -- 172.21.15.2:0/1093083488 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380017030 con 0x7f738410a670 2024-05-14T09:57:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.795+0000 7f738c334640 1 -- 172.21.15.2:0/1093083488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73841b9520 con 0x7f738410a670 2024-05-14T09:57:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.795+0000 7f738c334640 1 -- 172.21.15.2:0/1093083488 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73841b9b10 con 0x7f738410a670 2024-05-14T09:57:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.795+0000 7f737b7fe640 1 -- 172.21.15.2:0/1093083488 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7380007e00 con 0x7f738410a670 2024-05-14T09:57:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.795+0000 7f737b7fe640 1 -- 172.21.15.2:0/1093083488 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380013c60 con 0x7f738410a670 2024-05-14T09:57:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.799+0000 7f73931e1740 1 Processor -- start 2024-05-14T09:57:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.799+0000 7f73931e1740 1 -- start start 2024-05-14T09:57:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.800+0000 7f73931e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c901b0 0x55cfc5031a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.800+0000 7f73931e1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c672f0 0x55cfc5031fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.800+0000 7f73931e1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 0x55cfc5038ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.800+0000 7f73931e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cfc4f965d0 con 0x55cfc4c901b0 2024-05-14T09:57:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.800+0000 7f73931e1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cfc4f4f890 con 0x55cfc4c6f4d0 2024-05-14T09:57:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.800+0000 7f73931e1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cfc4f92590 con 0x55cfc4c672f0 2024-05-14T09:57:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.800+0000 7f738a8d1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c901b0 0x55cfc5031a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.800+0000 7f738a8d1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c901b0 0x55cfc5031a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34558/0 (socket says 172.21.15.2:34558) 2024-05-14T09:57:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.800+0000 7f738a8d1640 1 -- 172.21.15.2:0/2968732159 learned_addr learned my addr 172.21.15.2:0/2968732159 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.800+0000 7f737b7fe640 1 -- 172.21.15.2:0/1093083488 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f738001a070 con 0x7f738410a670 2024-05-14T09:57:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.801+0000 7f738a8d1640 1 -- 172.21.15.2:0/2968732159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c672f0 msgr2=0x55cfc5031fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:57:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.801+0000 7f738a8d1640 1 --2- 172.21.15.2:0/2968732159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c672f0 0x55cfc5031fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.801+0000 7f738a8d1640 1 -- 172.21.15.2:0/2968732159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 msgr2=0x55cfc5038ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:57:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.801+0000 7f738a8d1640 1 --2- 172.21.15.2:0/2968732159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 0x55cfc5038ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.801+0000 7f738a8d1640 1 -- 172.21.15.2:0/2968732159 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfc4ca8210 con 0x55cfc4c901b0 2024-05-14T09:57:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.801+0000 7f738a8d1640 1 --2- 172.21.15.2:0/2968732159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c901b0 0x55cfc5031a60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f737c009830 tx=0x7f737c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95155d4d06edc3b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.802+0000 7f7378bfe640 1 -- 172.21.15.2:0/2968732159 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c003970 con 0x55cfc4c901b0 2024-05-14T09:57:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.807+0000 7f7378bfe640 1 -- 172.21.15.2:0/2968732159 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f737c036040 con 0x55cfc4c901b0 2024-05-14T09:57:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.807+0000 7f7378bfe640 1 -- 172.21.15.2:0/2968732159 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c0305d0 con 0x55cfc4c901b0 2024-05-14T09:57:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.808+0000 7f737b7fe640 1 --2- 172.21.15.2:0/1093083488 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f73700b29a0 0x7f73700b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.808+0000 7f73931e1740 1 -- 172.21.15.2:0/2968732159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c901b0 msgr2=0x55cfc5031a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.808+0000 7f73931e1740 1 --2- 172.21.15.2:0/2968732159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c901b0 0x55cfc5031a60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f737c009830 tx=0x7f737c009860 comp rx=0 tx=0).stop 2024-05-14T09:57:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.808+0000 7f737b7fe640 1 -- 172.21.15.2:0/1093083488 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f73800e1ce0 con 0x7f738410a670 2024-05-14T09:57:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.808+0000 7f738a0d0640 1 --2- 172.21.15.2:0/1093083488 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f73700b29a0 0x7f73700b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.809+0000 7f73931e1740 1 -- 172.21.15.2:0/2968732159 shutdown_connections 2024-05-14T09:57:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.809+0000 7f73931e1740 1 --2- 172.21.15.2:0/2968732159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 0x55cfc5038ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.809+0000 7f73931e1740 1 --2- 172.21.15.2:0/2968732159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c672f0 0x55cfc5031fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.809+0000 7f73931e1740 1 --2- 172.21.15.2:0/2968732159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c901b0 0x55cfc5031a60 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.809+0000 7f73931e1740 1 -- 172.21.15.2:0/2968732159 >> 172.21.15.2:0/2968732159 conn(0x55cfc4c84300 msgr2=0x55cfc502d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.809+0000 7f738a0d0640 1 --2- 172.21.15.2:0/1093083488 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f73700b29a0 0x7f73700b4eb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f73800a8b80 tx=0x7f73800a8c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.809+0000 7f73931e1740 1 -- 172.21.15.2:0/2968732159 shutdown_connections 2024-05-14T09:57:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.809+0000 7f73931e1740 1 -- 172.21.15.2:0/2968732159 wait complete. 2024-05-14T09:57:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.810+0000 7f73931e1740 1 -- start start 2024-05-14T09:57:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.810+0000 7f73931e1740 1 Processor -- start 2024-05-14T09:57:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.810+0000 7f73931e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c672f0 0x55cfc503b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.810+0000 7f73931e1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 0x55cfc503baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f73931e1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 0x55cfc5042a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f73931e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cfc50398d0 con 0x55cfc4c672f0 2024-05-14T09:57:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f73931e1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cfc4f53370 con 0x55cfc4c6f4d0 2024-05-14T09:57:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f73931e1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55cfc5039a50 con 0x55cfc4c901b0 2024-05-14T09:57:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f73931e1740 10 client.0 _ll_register_callbacks cb 0x55cfc4ca7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:57:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f738a8d1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 0x55cfc5042a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f73898cf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 0x55cfc503baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f738a8d1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c672f0 0x55cfc503b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f73898cf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 0x55cfc503baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:42096/0 (socket says 172.21.15.2:42096) 2024-05-14T09:57:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f738a8d1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c672f0 0x55cfc503b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34560/0 (socket says 172.21.15.2:34560) 2024-05-14T09:57:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f73898cf640 1 -- 172.21.15.2:0/2625759099 learned_addr learned my addr 172.21.15.2:0/2625759099 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f738a8d1640 1 -- 172.21.15.2:0/2625759099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 msgr2=0x55cfc503baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f738a8d1640 1 --2- 172.21.15.2:0/2625759099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 0x55cfc503baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f738a8d1640 1 -- 172.21.15.2:0/2625759099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c672f0 msgr2=0x55cfc503b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f738a8d1640 1 --2- 172.21.15.2:0/2625759099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c672f0 0x55cfc503b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.811+0000 7f738a8d1640 1 -- 172.21.15.2:0/2625759099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfc4d74620 con 0x55cfc4c901b0 2024-05-14T09:57:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.812+0000 7f73898cf640 1 --2- 172.21.15.2:0/2625759099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 0x55cfc503baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:57:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.812+0000 7f738a8d1640 1 --2- 172.21.15.2:0/2625759099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 0x55cfc5042a50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f737c009ca0 tx=0x7f737c005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74684bdc04f9413e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.812+0000 7f736dffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:57:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.812+0000 7f736dffb640 1 -- 172.21.15.2:0/2625759099 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c06c040 con 0x55cfc4c901b0 2024-05-14T09:57:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.812+0000 7f736dffb640 1 -- 172.21.15.2:0/2625759099 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f737c003b70 con 0x55cfc4c901b0 2024-05-14T09:57:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.812+0000 7f73931e1740 1 -- 172.21.15.2:0/2625759099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f737c036040 con 0x55cfc4c901b0 2024-05-14T09:57:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.813+0000 7f736dffb640 1 -- 172.21.15.2:0/2625759099 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f737c004b40 con 0x55cfc4c901b0 2024-05-14T09:57:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.813+0000 7f73931e1740 10 client.27109 fetch_fsmap learned FSMap version 67 2024-05-14T09:57:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.813+0000 7f73931e1740 1 -- 172.21.15.2:0/2625759099 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55cfc50398d0 con 0x55cfc4c901b0 2024-05-14T09:57:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.813+0000 7f736dffb640 1 -- 172.21.15.2:0/2625759099 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f737c036040 con 0x55cfc4c901b0 2024-05-14T09:57:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.813+0000 7f736dffb640 1 -- 172.21.15.2:0/2625759099 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f737c072040 con 0x55cfc4c901b0 2024-05-14T09:57:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.813+0000 7f73931e1740 10 client.27109 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:57:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.813+0000 7f73931e1740 10 client.27109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:57:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.813+0000 7f73931e1740 20 client.27109 populate_metadata read hostname 'smithi002' 2024-05-14T09:57:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.813+0000 7f73931e1740 1 --2- 172.21.15.2:0/2625759099 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cfc4c4ed80 0x55cfc5048bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.813+0000 7f73931e1740 4 client.27109 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:57:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.813+0000 7f73931e1740 1 -- 172.21.15.2:0/2625759099 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfc4fa41e0 con 0x55cfc4c4ed80 2024-05-14T09:57:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.814+0000 7f73898cf640 1 --2- 172.21.15.2:0/2625759099 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cfc4c4ed80 0x55cfc5048bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.815+0000 7f73898cf640 1 --2- 172.21.15.2:0/2625759099 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cfc4c4ed80 0x55cfc5048bf0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4244b4955045bca4 server_cookie=1f19137e5239bf9e in_seq=0 out_seq=0 2024-05-14T09:57:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.815+0000 7f736dffb640 10 client.27109 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.818+0000 7f736dffb640 1 -- 172.21.15.2:0/2625759099 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cfc4fa41e0 con 0x55cfc4c4ed80 2024-05-14T09:57:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.818+0000 7f736dffb640 10 client.27109 handle_command_reply: tid=0 2024-05-14T09:57:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.818+0000 7f73931e1740 1 client.27109 shutdown 2024-05-14T09:57:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.821+0000 7f73931e1740 1 -- 172.21.15.2:0/2625759099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 msgr2=0x55cfc5042a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.821+0000 7f73931e1740 1 --2- 172.21.15.2:0/2625759099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 0x55cfc5042a50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f737c009ca0 tx=0x7f737c005bd0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.821+0000 7f73931e1740 1 -- 172.21.15.2:0/2625759099 shutdown_connections 2024-05-14T09:57:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.822+0000 7f73931e1740 1 --2- 172.21.15.2:0/2625759099 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cfc4c4ed80 0x55cfc5048bf0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.822+0000 7f73931e1740 1 --2- 172.21.15.2:0/2625759099 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 0x55cfc5042a50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.822+0000 7f73931e1740 1 --2- 172.21.15.2:0/2625759099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 0x55cfc503baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.822+0000 7f73931e1740 1 --2- 172.21.15.2:0/2625759099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c672f0 0x55cfc503b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.822+0000 7f73931e1740 1 -- 172.21.15.2:0/2625759099 >> 172.21.15.2:0/2625759099 conn(0x55cfc4c84300 msgr2=0x55cfc502c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.822+0000 7f736dffb640 0 client.27109 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.822+0000 7f73931e1740 1 -- 172.21.15.2:0/2625759099 shutdown_connections 2024-05-14T09:57:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.822+0000 7f73931e1740 1 -- 172.21.15.2:0/2625759099 wait complete. 2024-05-14T09:57:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.822+0000 7f73931e1740 20 client.27109 trim_cache size 0 max 16384 2024-05-14T09:57:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.848+0000 7f73931e1740 1 Processor -- start 2024-05-14T09:57:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.848+0000 7f73931e1740 1 -- start start 2024-05-14T09:57:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.848+0000 7f73931e1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c672f0 0x55cfc502d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.848+0000 7f73931e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c6f4d0 0x55cfc50429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.848+0000 7f73931e1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 0x55cfc5042f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.848+0000 7f73931e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7368012760 con 0x55cfc4c6f4d0 2024-05-14T09:57:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.848+0000 7f73931e1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7368010030 con 0x55cfc4c672f0 2024-05-14T09:57:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.848+0000 7f73931e1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f736800de80 con 0x55cfc4c901b0 2024-05-14T09:57:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.848+0000 7f738a8d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c672f0 0x55cfc502d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.848+0000 7f738a8d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c672f0 0x55cfc502d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:42098/0 (socket says 172.21.15.2:42098) 2024-05-14T09:57:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.848+0000 7f738a8d1640 1 -- 172.21.15.2:0/919038482 learned_addr learned my addr 172.21.15.2:0/919038482 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.848+0000 7f738a8d1640 1 --2- 172.21.15.2:0/919038482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 0x55cfc5042f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.849+0000 7f73898cf640 1 --2- 172.21.15.2:0/919038482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c6f4d0 0x55cfc50429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.849+0000 7f738a8d1640 1 -- 172.21.15.2:0/919038482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 msgr2=0x55cfc5042f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.849+0000 7f738a8d1640 1 --2- 172.21.15.2:0/919038482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 0x55cfc5042f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.849+0000 7f738a8d1640 1 -- 172.21.15.2:0/919038482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c6f4d0 msgr2=0x55cfc50429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.849+0000 7f738a8d1640 1 --2- 172.21.15.2:0/919038482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c6f4d0 0x55cfc50429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.849+0000 7f738a8d1640 1 -- 172.21.15.2:0/919038482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfc503b360 con 0x55cfc4c672f0 2024-05-14T09:57:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.849+0000 7f738a8d1640 1 --2- 172.21.15.2:0/919038482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c672f0 0x55cfc502d1e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f737c009bf0 tx=0x7f737c031000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38cacb4630a4936f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.849+0000 7f736ffff640 1 -- 172.21.15.2:0/919038482 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c0047d0 con 0x55cfc4c672f0 2024-05-14T09:57:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.849+0000 7f736ffff640 1 -- 172.21.15.2:0/919038482 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f737c036070 con 0x55cfc4c672f0 2024-05-14T09:57:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.849+0000 7f736ffff640 1 -- 172.21.15.2:0/919038482 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c0054e0 con 0x55cfc4c672f0 2024-05-14T09:57:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.850+0000 7f73931e1740 1 -- 172.21.15.2:0/919038482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c672f0 msgr2=0x55cfc502d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.850+0000 7f73931e1740 1 --2- 172.21.15.2:0/919038482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c672f0 0x55cfc502d1e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f737c009bf0 tx=0x7f737c031000 comp rx=0 tx=0).stop 2024-05-14T09:57:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.850+0000 7f73931e1740 1 -- 172.21.15.2:0/919038482 shutdown_connections 2024-05-14T09:57:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.850+0000 7f73931e1740 1 --2- 172.21.15.2:0/919038482 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 0x55cfc5042f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.850+0000 7f73931e1740 1 --2- 172.21.15.2:0/919038482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c6f4d0 0x55cfc50429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.850+0000 7f73931e1740 1 --2- 172.21.15.2:0/919038482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c672f0 0x55cfc502d1e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.851+0000 7f73931e1740 1 -- 172.21.15.2:0/919038482 >> 172.21.15.2:0/919038482 conn(0x55cfc4c84300 msgr2=0x55cfc502cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.851+0000 7f73931e1740 1 -- 172.21.15.2:0/919038482 shutdown_connections 2024-05-14T09:57:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.851+0000 7f73931e1740 1 -- 172.21.15.2:0/919038482 wait complete. 2024-05-14T09:57:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.851+0000 7f73931e1740 1 -- start start 2024-05-14T09:57:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.851+0000 7f73931e1740 1 Processor -- start 2024-05-14T09:57:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f73931e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c672f0 0x55cfc503e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f73931e1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 0x55cfc503e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f73931e1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 0x55cfc5041690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f73931e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f736800e4f0 con 0x55cfc4c672f0 2024-05-14T09:57:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f73931e1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f736800f960 con 0x55cfc4c6f4d0 2024-05-14T09:57:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f73931e1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f736800ebc0 con 0x55cfc4c901b0 2024-05-14T09:57:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f73931e1740 10 client.0 _ll_register_callbacks cb 0x7f737c030a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:57:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f73898cf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 0x55cfc503e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f73898cf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 0x55cfc503e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:42110/0 (socket says 172.21.15.2:42110) 2024-05-14T09:57:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f73898cf640 1 -- 172.21.15.2:0/1724303186 learned_addr learned my addr 172.21.15.2:0/1724303186 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:57:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f738a8d1640 1 --2- 172.21.15.2:0/1724303186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c672f0 0x55cfc503e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f738a8d1640 1 --2- 172.21.15.2:0/1724303186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 0x55cfc5041690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f73898cf640 1 -- 172.21.15.2:0/1724303186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 msgr2=0x55cfc5041690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f73898cf640 1 --2- 172.21.15.2:0/1724303186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 0x55cfc5041690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f73898cf640 1 -- 172.21.15.2:0/1724303186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c672f0 msgr2=0x55cfc503e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f73898cf640 1 --2- 172.21.15.2:0/1724303186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c672f0 0x55cfc503e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f73898cf640 1 -- 172.21.15.2:0/1724303186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfc5041c50 con 0x55cfc4c6f4d0 2024-05-14T09:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.852+0000 7f738a8d1640 1 --2- 172.21.15.2:0/1724303186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 0x55cfc5041690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.853+0000 7f73898cf640 1 --2- 172.21.15.2:0/1724303186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 0x55cfc503e750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55cfc5049320 tx=0x7f7374016710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55823ff35e617532 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.853+0000 7f736dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.853+0000 7f736dffb640 1 -- 172.21.15.2:0/1724303186 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737401c450 con 0x55cfc4c6f4d0 2024-05-14T09:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.853+0000 7f736dffb640 1 -- 172.21.15.2:0/1724303186 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f737401c5f0 con 0x55cfc4c6f4d0 2024-05-14T09:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.853+0000 7f73931e1740 1 -- 172.21.15.2:0/1724303186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f737c036070 con 0x55cfc4c6f4d0 2024-05-14T09:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.854+0000 7f736dffb640 1 -- 172.21.15.2:0/1724303186 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f737402d050 con 0x55cfc4c6f4d0 2024-05-14T09:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.854+0000 7f73931e1740 10 client.27125 fetch_fsmap learned FSMap version 67 2024-05-14T09:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.854+0000 7f73931e1740 1 -- 172.21.15.2:0/1724303186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55cfc5042fe0 con 0x55cfc4c6f4d0 2024-05-14T09:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.854+0000 7f736dffb640 1 -- 172.21.15.2:0/1724303186 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f737c036070 con 0x55cfc4c6f4d0 2024-05-14T09:57:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.854+0000 7f736dffb640 1 -- 172.21.15.2:0/1724303186 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f7374024c30 con 0x55cfc4c6f4d0 2024-05-14T09:57:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.854+0000 7f73931e1740 10 client.27125 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:57:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.854+0000 7f73931e1740 10 client.27125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:57:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.854+0000 7f73931e1740 20 client.27125 populate_metadata read hostname 'smithi002' 2024-05-14T09:57:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.855+0000 7f73931e1740 1 --2- 172.21.15.2:0/1724303186 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cfc4c241c0 0x55cfc50b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:57:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.855+0000 7f73931e1740 4 client.27125 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:57:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.855+0000 7f738a8d1640 1 --2- 172.21.15.2:0/1724303186 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cfc4c241c0 0x55cfc50b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:57:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.855+0000 7f73931e1740 1 -- 172.21.15.2:0/1724303186 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cfc4f9e050 con 0x55cfc4c241c0 2024-05-14T09:57:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.855+0000 7f738a8d1640 1 --2- 172.21.15.2:0/1724303186 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cfc4c241c0 0x55cfc50b3480 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71a619e3d944e4ba server_cookie=bf8c16028dc4b78a in_seq=0 out_seq=0 2024-05-14T09:57:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:48.856+0000 7f736dffb640 10 client.27125 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:48 smithi002 ceph-mon[31557]: pgmap v1112: 129 pgs: 129 active+clean; 680 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 8.2 MiB/s wr, 604 op/s 2024-05-14T09:57:48.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:48 smithi029 ceph-mon[36249]: pgmap v1112: 129 pgs: 129 active+clean; 680 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 8.2 MiB/s wr, 604 op/s 2024-05-14T09:57:48.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:48 smithi026 ceph-mon[36056]: pgmap v1112: 129 pgs: 129 active+clean; 680 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 8.2 MiB/s wr, 604 op/s 2024-05-14T09:57:49.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 274863 25.83 MB/sec execute 435 sec latency 585.781 ms 2024-05-14T09:57:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.705+0000 7f736dffb640 1 -- 172.21.15.2:0/1724303186 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cfc4f9e050 con 0x55cfc4c241c0 2024-05-14T09:57:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.705+0000 7f736dffb640 10 client.27125 handle_command_reply: tid=0 2024-05-14T09:57:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.705+0000 7f73931e1740 1 client.27125 shutdown 2024-05-14T09:57:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.706+0000 7f73931e1740 1 -- 172.21.15.2:0/1724303186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 msgr2=0x55cfc503e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.706+0000 7f73931e1740 1 --2- 172.21.15.2:0/1724303186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 0x55cfc503e750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55cfc5049320 tx=0x7f7374016710 comp rx=0 tx=0).stop 2024-05-14T09:57:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.706+0000 7f73931e1740 1 -- 172.21.15.2:0/1724303186 shutdown_connections 2024-05-14T09:57:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.706+0000 7f73931e1740 1 --2- 172.21.15.2:0/1724303186 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55cfc4c241c0 0x55cfc50b3480 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.706+0000 7f73931e1740 1 --2- 172.21.15.2:0/1724303186 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55cfc4c901b0 0x55cfc5041690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.706+0000 7f736dffb640 0 client.27125 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:57:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.706+0000 7f73931e1740 1 --2- 172.21.15.2:0/1724303186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfc4c6f4d0 0x55cfc503e750 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.706+0000 7f73931e1740 1 --2- 172.21.15.2:0/1724303186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfc4c672f0 0x55cfc503e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.706+0000 7f73931e1740 1 -- 172.21.15.2:0/1724303186 >> 172.21.15.2:0/1724303186 conn(0x55cfc4c84300 msgr2=0x55cfc504d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.707+0000 7f73931e1740 1 -- 172.21.15.2:0/1724303186 shutdown_connections 2024-05-14T09:57:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.707+0000 7f73931e1740 1 -- 172.21.15.2:0/1724303186 wait complete. 2024-05-14T09:57:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.707+0000 7f73931e1740 20 client.27125 trim_cache size 0 max 16384 2024-05-14T09:57:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:57:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:57:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:57:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:57:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.708+0000 7f738c334640 1 -- 172.21.15.2:0/1093083488 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f73700b29a0 msgr2=0x7f73700b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.708+0000 7f738c334640 1 --2- 172.21.15.2:0/1093083488 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f73700b29a0 0x7f73700b4eb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f73800a8b80 tx=0x7f73800a8c50 comp rx=0 tx=0).stop 2024-05-14T09:57:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.708+0000 7f738c334640 1 -- 172.21.15.2:0/1093083488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f738410a670 msgr2=0x7f738411b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:57:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.708+0000 7f738c334640 1 --2- 172.21.15.2:0/1093083488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f738410a670 0x7f738411b9c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f73800077b0 tx=0x7f7380007bb0 comp rx=0 tx=0).stop 2024-05-14T09:57:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.708+0000 7f738c334640 1 -- 172.21.15.2:0/1093083488 shutdown_connections 2024-05-14T09:57:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.708+0000 7f738c334640 1 --2- 172.21.15.2:0/1093083488 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f73700b29a0 0x7f73700b4eb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.708+0000 7f738c334640 1 --2- 172.21.15.2:0/1093083488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73841105b0 0x7f73841b8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.708+0000 7f738c334640 1 --2- 172.21.15.2:0/1093083488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f738410b8b0 0x7f738411bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.708+0000 7f738c334640 1 --2- 172.21.15.2:0/1093083488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f738410a670 0x7f738411b9c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:57:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.708+0000 7f738c334640 1 -- 172.21.15.2:0/1093083488 >> 172.21.15.2:0/1093083488 conn(0x7f7384070a80 msgr2=0x7f7384070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:57:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.708+0000 7f738c334640 1 -- 172.21.15.2:0/1093083488 shutdown_connections 2024-05-14T09:57:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:57:49.708+0000 7f738c334640 1 -- 172.21.15.2:0/1093083488 wait complete. 2024-05-14T09:57:50.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 275455 25.83 MB/sec execute 436 sec latency 643.732 ms 2024-05-14T09:57:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:50 smithi002 ceph-mon[31557]: pgmap v1113: 129 pgs: 129 active+clean; 687 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.2 MiB/s wr, 571 op/s 2024-05-14T09:57:50.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:50 smithi029 ceph-mon[36249]: pgmap v1113: 129 pgs: 129 active+clean; 687 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.2 MiB/s wr, 571 op/s 2024-05-14T09:57:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:50 smithi026 ceph-mon[36056]: pgmap v1113: 129 pgs: 129 active+clean; 687 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.2 MiB/s wr, 571 op/s 2024-05-14T09:57:51.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 276160 25.85 MB/sec execute 437 sec latency 124.797 ms 2024-05-14T09:57:52.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 276887 25.90 MB/sec execute 438 sec latency 65.077 ms 2024-05-14T09:57:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:52 smithi002 ceph-mon[31557]: pgmap v1114: 129 pgs: 129 active+clean; 686 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.7 MiB/s wr, 438 op/s 2024-05-14T09:57:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:52 smithi029 ceph-mon[36249]: pgmap v1114: 129 pgs: 129 active+clean; 686 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.7 MiB/s wr, 438 op/s 2024-05-14T09:57:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:52 smithi026 ceph-mon[36056]: pgmap v1114: 129 pgs: 129 active+clean; 686 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.7 MiB/s wr, 438 op/s 2024-05-14T09:57:53.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 277519 25.93 MB/sec execute 439 sec latency 79.755 ms 2024-05-14T09:57:54.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 277915 25.92 MB/sec execute 440 sec latency 208.488 ms 2024-05-14T09:57:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:54 smithi002 ceph-mon[31557]: pgmap v1115: 129 pgs: 129 active+clean; 694 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.3 MiB/s wr, 519 op/s 2024-05-14T09:57:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:54 smithi029 ceph-mon[36249]: pgmap v1115: 129 pgs: 129 active+clean; 694 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.3 MiB/s wr, 519 op/s 2024-05-14T09:57:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:54 smithi026 ceph-mon[36056]: pgmap v1115: 129 pgs: 129 active+clean; 694 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.3 MiB/s wr, 519 op/s 2024-05-14T09:57:55.623 INFO:tasks.workunit.client.0.smithi002.stdout: 10 278367 25.91 MB/sec execute 441 sec latency 88.811 ms 2024-05-14T09:57:56.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 278799 25.88 MB/sec execute 442 sec latency 90.705 ms 2024-05-14T09:57:56.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:56 smithi002 ceph-mon[31557]: pgmap v1116: 129 pgs: 129 active+clean; 696 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.1 MiB/s wr, 496 op/s 2024-05-14T09:57:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:56 smithi029 ceph-mon[36249]: pgmap v1116: 129 pgs: 129 active+clean; 696 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.1 MiB/s wr, 496 op/s 2024-05-14T09:57:56.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:56 smithi026 ceph-mon[36056]: pgmap v1116: 129 pgs: 129 active+clean; 696 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.1 MiB/s wr, 496 op/s 2024-05-14T09:57:57.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 279257 25.89 MB/sec execute 443 sec latency 240.238 ms 2024-05-14T09:57:58.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 279891 25.91 MB/sec execute 444 sec latency 140.271 ms 2024-05-14T09:57:58.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:57:58 smithi002 ceph-mon[31557]: pgmap v1117: 129 pgs: 129 active+clean; 700 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.2 MiB/s wr, 506 op/s 2024-05-14T09:57:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:57:58 smithi029 ceph-mon[36249]: pgmap v1117: 129 pgs: 129 active+clean; 700 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.2 MiB/s wr, 506 op/s 2024-05-14T09:57:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:57:58 smithi026 ceph-mon[36056]: pgmap v1117: 129 pgs: 129 active+clean; 700 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.2 MiB/s wr, 506 op/s 2024-05-14T09:57:59.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 280312 25.89 MB/sec execute 445 sec latency 306.302 ms 2024-05-14T09:58:00.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 280978 25.92 MB/sec execute 446 sec latency 67.385 ms 2024-05-14T09:58:00.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:00 smithi002 ceph-mon[31557]: pgmap v1118: 129 pgs: 129 active+clean; 709 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.5 MiB/s wr, 533 op/s 2024-05-14T09:58:00.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:00 smithi029 ceph-mon[36249]: pgmap v1118: 129 pgs: 129 active+clean; 709 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.5 MiB/s wr, 533 op/s 2024-05-14T09:58:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:00 smithi026 ceph-mon[36056]: pgmap v1118: 129 pgs: 129 active+clean; 709 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.5 MiB/s wr, 533 op/s 2024-05-14T09:58:01.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 281572 25.95 MB/sec execute 447 sec latency 65.262 ms 2024-05-14T09:58:02.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 282016 25.93 MB/sec execute 448 sec latency 246.282 ms 2024-05-14T09:58:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.852+0000 7fe08e7d6640 1 -- 172.21.15.2:0/2124027631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe088103d70 msgr2=0x7fe088104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.852+0000 7fe08e7d6640 1 --2- 172.21.15.2:0/2124027631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe088103d70 0x7fe088104180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009900 tx=0x7fe07c02a410 comp rx=0 tx=0).stop 2024-05-14T09:58:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.852+0000 7fe08e7d6640 1 -- 172.21.15.2:0/2124027631 shutdown_connections 2024-05-14T09:58:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.852+0000 7fe08e7d6640 1 --2- 172.21.15.2:0/2124027631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe088108a70 0x7fe08810ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.852+0000 7fe08e7d6640 1 --2- 172.21.15.2:0/2124027631 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe088103d70 0x7fe088104180 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.852+0000 7fe08e7d6640 1 --2- 172.21.15.2:0/2124027631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe088102b30 0x7fe088103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.852+0000 7fe08e7d6640 1 -- 172.21.15.2:0/2124027631 >> 172.21.15.2:0/2124027631 conn(0x7fe0880fdd60 msgr2=0x7fe0881001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.852+0000 7fe08e7d6640 1 -- 172.21.15.2:0/2124027631 shutdown_connections 2024-05-14T09:58:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.852+0000 7fe08e7d6640 1 -- 172.21.15.2:0/2124027631 wait complete. 2024-05-14T09:58:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.853+0000 7fe08e7d6640 1 Processor -- start 2024-05-14T09:58:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.853+0000 7fe08e7d6640 1 -- start start 2024-05-14T09:58:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.853+0000 7fe08e7d6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe088102b30 0x7fe08819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.853+0000 7fe08e7d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe088103d70 0x7fe08819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.853+0000 7fe08e7d6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe088108a70 0x7fe0881a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.853+0000 7fe087fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe088102b30 0x7fe08819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.853+0000 7fe08e7d6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe08810e660 con 0x7fe088102b30 2024-05-14T09:58:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.853+0000 7fe087fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe088102b30 0x7fe08819c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47638/0 (socket says 172.21.15.2:47638) 2024-05-14T09:58:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.853+0000 7fe08e7d6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe08810e4e0 con 0x7fe088103d70 2024-05-14T09:58:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.853+0000 7fe087fff640 1 -- 172.21.15.2:0/394430243 learned_addr learned my addr 172.21.15.2:0/394430243 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.853+0000 7fe08e7d6640 1 -- 172.21.15.2:0/394430243 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe08810e7e0 con 0x7fe088108a70 2024-05-14T09:58:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.853+0000 7fe0877fe640 1 --2- 172.21.15.2:0/394430243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe088103d70 0x7fe08819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.853+0000 7fe08cd73640 1 --2- 172.21.15.2:0/394430243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe088108a70 0x7fe0881a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.854+0000 7fe087fff640 1 -- 172.21.15.2:0/394430243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe088108a70 msgr2=0x7fe0881a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.854+0000 7fe087fff640 1 --2- 172.21.15.2:0/394430243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe088108a70 0x7fe0881a3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.854+0000 7fe087fff640 1 -- 172.21.15.2:0/394430243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe088103d70 msgr2=0x7fe08819c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.854+0000 7fe087fff640 1 --2- 172.21.15.2:0/394430243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe088103d70 0x7fe08819c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.854+0000 7fe087fff640 1 -- 172.21.15.2:0/394430243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0881a3fd0 con 0x7fe088102b30 2024-05-14T09:58:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.854+0000 7fe087fff640 1 --2- 172.21.15.2:0/394430243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe088102b30 0x7fe08819c490 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe07800e9f0 tx=0x7fe07800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.854+0000 7fe0857fa640 1 -- 172.21.15.2:0/394430243 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07800ccc0 con 0x7fe088102b30 2024-05-14T09:58:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.854+0000 7fe08e7d6640 1 -- 172.21.15.2:0/394430243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0881a42c0 con 0x7fe088102b30 2024-05-14T09:58:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.854+0000 7fe0857fa640 1 -- 172.21.15.2:0/394430243 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0780035f0 con 0x7fe088102b30 2024-05-14T09:58:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.854+0000 7fe0857fa640 1 -- 172.21.15.2:0/394430243 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07800ce60 con 0x7fe088102b30 2024-05-14T09:58:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.854+0000 7fe08e7d6640 1 -- 172.21.15.2:0/394430243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0881a4860 con 0x7fe088102b30 2024-05-14T09:58:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.857+0000 7fe095685740 1 Processor -- start 2024-05-14T09:58:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.857+0000 7fe0857fa640 1 -- 172.21.15.2:0/394430243 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe078010690 con 0x7fe088102b30 2024-05-14T09:58:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.857+0000 7fe095685740 1 -- start start 2024-05-14T09:58:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.857+0000 7fe095685740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12c51b0 0x55a6a1666a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.857+0000 7fe095685740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a129c2f0 0x55a6a1666fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.858+0000 7fe095685740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a12a44d0 0x55a6a166dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.858+0000 7fe095685740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a6a15cb5d0 con 0x55a6a129c2f0 2024-05-14T09:58:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.858+0000 7fe095685740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a6a1584890 con 0x55a6a12a44d0 2024-05-14T09:58:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.858+0000 7fe095685740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a6a15c7590 con 0x55a6a12c51b0 2024-05-14T09:58:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.858+0000 7fe08cd73640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12c51b0 0x55a6a1666a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.858+0000 7fe087fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a129c2f0 0x55a6a1666fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.858+0000 7fe08cd73640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12c51b0 0x55a6a1666a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:48804/0 (socket says 172.21.15.2:48804) 2024-05-14T09:58:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.858+0000 7fe08cd73640 1 -- 172.21.15.2:0/3794247984 learned_addr learned my addr 172.21.15.2:0/3794247984 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.858+0000 7fe08cd73640 1 -- 172.21.15.2:0/3794247984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a12a44d0 msgr2=0x55a6a166dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.858+0000 7fe08cd73640 1 --2- 172.21.15.2:0/3794247984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a12a44d0 0x55a6a166dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.858+0000 7fe08cd73640 1 -- 172.21.15.2:0/3794247984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a129c2f0 msgr2=0x55a6a1666fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.858+0000 7fe08cd73640 1 --2- 172.21.15.2:0/3794247984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a129c2f0 0x55a6a1666fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.858+0000 7fe08cd73640 1 -- 172.21.15.2:0/3794247984 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6a12dd210 con 0x55a6a12c51b0 2024-05-14T09:58:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.858+0000 7fe0857fa640 1 --2- 172.21.15.2:0/394430243 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0700b28a0 0x7fe0700b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.858+0000 7fe08cd73640 1 --2- 172.21.15.2:0/3794247984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12c51b0 0x55a6a1666a60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe080009fd0 tx=0x7fe0800118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdf831ac6b2d7802 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.859+0000 7fe087fff640 1 --2- 172.21.15.2:0/394430243 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0700b28a0 0x7fe0700b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.859+0000 7fe06e7fc640 1 -- 172.21.15.2:0/3794247984 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe080004d20 con 0x55a6a12c51b0 2024-05-14T09:58:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.859+0000 7fe0857fa640 1 -- 172.21.15.2:0/394430243 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fe0780e2940 con 0x7fe088102b30 2024-05-14T09:58:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.859+0000 7fe06e7fc640 1 -- 172.21.15.2:0/3794247984 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe080004050 con 0x55a6a12c51b0 2024-05-14T09:58:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.859+0000 7fe06e7fc640 1 -- 172.21.15.2:0/3794247984 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe080005280 con 0x55a6a12c51b0 2024-05-14T09:58:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.860+0000 7fe095685740 1 -- 172.21.15.2:0/3794247984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12c51b0 msgr2=0x55a6a1666a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.860+0000 7fe095685740 1 --2- 172.21.15.2:0/3794247984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12c51b0 0x55a6a1666a60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe080009fd0 tx=0x7fe0800118d0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.860+0000 7fe087fff640 1 --2- 172.21.15.2:0/394430243 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0700b28a0 0x7fe0700b4db0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x55a6a1667e40 tx=0x7fe0780e8550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.860+0000 7fe095685740 1 -- 172.21.15.2:0/3794247984 shutdown_connections 2024-05-14T09:58:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.860+0000 7fe095685740 1 --2- 172.21.15.2:0/3794247984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a12a44d0 0x55a6a166dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.860+0000 7fe095685740 1 --2- 172.21.15.2:0/3794247984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a129c2f0 0x55a6a1666fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.860+0000 7fe095685740 1 --2- 172.21.15.2:0/3794247984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12c51b0 0x55a6a1666a60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.860+0000 7fe095685740 1 -- 172.21.15.2:0/3794247984 >> 172.21.15.2:0/3794247984 conn(0x55a6a12b9300 msgr2=0x55a6a16623a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.860+0000 7fe095685740 1 -- 172.21.15.2:0/3794247984 shutdown_connections 2024-05-14T09:58:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.860+0000 7fe095685740 1 -- 172.21.15.2:0/3794247984 wait complete. 2024-05-14T09:58:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.861+0000 7fe095685740 1 -- start start 2024-05-14T09:58:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.861+0000 7fe095685740 1 Processor -- start 2024-05-14T09:58:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.861+0000 7fe095685740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a129c2f0 0x55a6a1670700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.861+0000 7fe095685740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 0x55a6a1670c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe095685740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a12c51b0 0x55a6a1677b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe095685740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a6a166e930 con 0x55a6a12c51b0 2024-05-14T09:58:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe095685740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a6a1588370 con 0x55a6a129c2f0 2024-05-14T09:58:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe095685740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a6a166eab0 con 0x55a6a12a44d0 2024-05-14T09:58:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe095685740 10 client.0 _ll_register_callbacks cb 0x55a6a12dca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:58:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe0877fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 0x55a6a1670c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe08cd73640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a12c51b0 0x55a6a1677b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe0877fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 0x55a6a1670c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:48818/0 (socket says 172.21.15.2:48818) 2024-05-14T09:58:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe0877fe640 1 -- 172.21.15.2:0/2944009838 learned_addr learned my addr 172.21.15.2:0/2944009838 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe08cd73640 1 --2- 172.21.15.2:0/2944009838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a129c2f0 0x55a6a1670700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe0877fe640 1 -- 172.21.15.2:0/2944009838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a129c2f0 msgr2=0x55a6a1670700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe0877fe640 1 --2- 172.21.15.2:0/2944009838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a129c2f0 0x55a6a1670700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe0877fe640 1 -- 172.21.15.2:0/2944009838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a12c51b0 msgr2=0x55a6a1677b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe0877fe640 1 --2- 172.21.15.2:0/2944009838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a12c51b0 0x55a6a1677b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe0877fe640 1 -- 172.21.15.2:0/2944009838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6a13a9620 con 0x55a6a12a44d0 2024-05-14T09:58:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe0877fe640 1 --2- 172.21.15.2:0/2944009838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 0x55a6a1670c40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe07c031500 tx=0x7fe07c031d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5197b93839a4ce20 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.862+0000 7fe0677fe640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:58:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.863+0000 7fe0677fe640 1 -- 172.21.15.2:0/2944009838 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c035030 con 0x55a6a12a44d0 2024-05-14T09:58:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.863+0000 7fe0677fe640 1 -- 172.21.15.2:0/2944009838 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fe07c004030 con 0x55a6a12a44d0 2024-05-14T09:58:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.863+0000 7fe095685740 1 -- 172.21.15.2:0/2944009838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe080004050 con 0x55a6a12a44d0 2024-05-14T09:58:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.863+0000 7fe0677fe640 1 -- 172.21.15.2:0/2944009838 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fe07c004230 con 0x55a6a12a44d0 2024-05-14T09:58:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.863+0000 7fe095685740 10 client.27127 fetch_fsmap learned FSMap version 67 2024-05-14T09:58:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.863+0000 7fe095685740 1 -- 172.21.15.2:0/2944009838 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55a6a167ab00 con 0x55a6a12a44d0 2024-05-14T09:58:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.863+0000 7fe0677fe640 1 -- 172.21.15.2:0/2944009838 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe080004050 con 0x55a6a12a44d0 2024-05-14T09:58:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.863+0000 7fe0677fe640 1 -- 172.21.15.2:0/2944009838 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fe07c03c440 con 0x55a6a12a44d0 2024-05-14T09:58:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.863+0000 7fe095685740 10 client.27127 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:58:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.863+0000 7fe095685740 10 client.27127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:58:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.863+0000 7fe095685740 20 client.27127 populate_metadata read hostname 'smithi002' 2024-05-14T09:58:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.864+0000 7fe095685740 1 --2- 172.21.15.2:0/2944009838 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a6a1283d80 0x55a6a167e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.864+0000 7fe095685740 4 client.27127 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:58:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.864+0000 7fe095685740 1 -- 172.21.15.2:0/2944009838 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6a15d91e0 con 0x55a6a1283d80 2024-05-14T09:58:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.864+0000 7fe08cd73640 1 --2- 172.21.15.2:0/2944009838 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a6a1283d80 0x55a6a167e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.864+0000 7fe0677fe640 10 client.27127 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.864+0000 7fe08cd73640 1 --2- 172.21.15.2:0/2944009838 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a6a1283d80 0x55a6a167e220 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21910a22ec1d3b33 server_cookie=2c484d64cc817d45 in_seq=0 out_seq=0 2024-05-14T09:58:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.869+0000 7fe0677fe640 1 -- 172.21.15.2:0/2944009838 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a6a15d91e0 con 0x55a6a1283d80 2024-05-14T09:58:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.869+0000 7fe0677fe640 10 client.27127 handle_command_reply: tid=0 2024-05-14T09:58:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.869+0000 7fe095685740 1 client.27127 shutdown 2024-05-14T09:58:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.869+0000 7fe095685740 1 -- 172.21.15.2:0/2944009838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 msgr2=0x55a6a1670c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.869+0000 7fe095685740 1 --2- 172.21.15.2:0/2944009838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 0x55a6a1670c40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe07c031500 tx=0x7fe07c031d40 comp rx=0 tx=0).stop 2024-05-14T09:58:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.869+0000 7fe095685740 1 -- 172.21.15.2:0/2944009838 shutdown_connections 2024-05-14T09:58:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.869+0000 7fe095685740 1 --2- 172.21.15.2:0/2944009838 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a6a1283d80 0x55a6a167e220 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.869+0000 7fe0677fe640 0 client.27127 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.869+0000 7fe095685740 1 --2- 172.21.15.2:0/2944009838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a12c51b0 0x55a6a1677b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.869+0000 7fe095685740 1 --2- 172.21.15.2:0/2944009838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 0x55a6a1670c40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.869+0000 7fe095685740 1 --2- 172.21.15.2:0/2944009838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a129c2f0 0x55a6a1670700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.869+0000 7fe095685740 1 -- 172.21.15.2:0/2944009838 >> 172.21.15.2:0/2944009838 conn(0x55a6a12b9300 msgr2=0x55a6a16613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.869+0000 7fe095685740 1 -- 172.21.15.2:0/2944009838 shutdown_connections 2024-05-14T09:58:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.869+0000 7fe095685740 1 -- 172.21.15.2:0/2944009838 wait complete. 2024-05-14T09:58:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.870+0000 7fe095685740 20 client.27127 trim_cache size 0 max 16384 2024-05-14T09:58:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.888+0000 7fe095685740 1 Processor -- start 2024-05-14T09:58:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.888+0000 7fe095685740 1 -- start start 2024-05-14T09:58:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.888+0000 7fe095685740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a1283d80 0x55a6a1661080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.888+0000 7fe095685740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a129c2f0 0x55a6a1677a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.888+0000 7fe095685740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 0x55a6a1677f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.888+0000 7fe095685740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881b1730 con 0x55a6a1283d80 2024-05-14T09:58:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.888+0000 7fe095685740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881af000 con 0x55a6a129c2f0 2024-05-14T09:58:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.888+0000 7fe095685740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881ace50 con 0x55a6a12a44d0 2024-05-14T09:58:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.889+0000 7fe08cd73640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a1283d80 0x55a6a1661080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.889+0000 7fe08cd73640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a1283d80 0x55a6a1661080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47664/0 (socket says 172.21.15.2:47664) 2024-05-14T09:58:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.889+0000 7fe08cd73640 1 -- 172.21.15.2:0/1451971534 learned_addr learned my addr 172.21.15.2:0/1451971534 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.889+0000 7fe0877fe640 1 --2- 172.21.15.2:0/1451971534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a129c2f0 0x55a6a1677a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.889+0000 7fe08cd73640 1 --2- 172.21.15.2:0/1451971534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 0x55a6a1677f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.889+0000 7fe08cd73640 1 -- 172.21.15.2:0/1451971534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 msgr2=0x55a6a1677f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.889+0000 7fe08cd73640 1 --2- 172.21.15.2:0/1451971534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 0x55a6a1677f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.889+0000 7fe08cd73640 1 -- 172.21.15.2:0/1451971534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a129c2f0 msgr2=0x55a6a1677a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.889+0000 7fe08cd73640 1 --2- 172.21.15.2:0/1451971534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a129c2f0 0x55a6a1677a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.889+0000 7fe08cd73640 1 -- 172.21.15.2:0/1451971534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6a1662d00 con 0x55a6a1283d80 2024-05-14T09:58:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.889+0000 7fe08cd73640 1 --2- 172.21.15.2:0/1451971534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a1283d80 0x55a6a1661080 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe080014040 tx=0x7fe080009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e220ad1896094cab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.889+0000 7fe06dffb640 1 -- 172.21.15.2:0/1451971534 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe080003fc0 con 0x55a6a1283d80 2024-05-14T09:58:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.889+0000 7fe06dffb640 1 -- 172.21.15.2:0/1451971534 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe080004610 con 0x55a6a1283d80 2024-05-14T09:58:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.889+0000 7fe06dffb640 1 -- 172.21.15.2:0/1451971534 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08000fd40 con 0x55a6a1283d80 2024-05-14T09:58:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.890+0000 7fe095685740 1 -- 172.21.15.2:0/1451971534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a1283d80 msgr2=0x55a6a1661080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.890+0000 7fe095685740 1 --2- 172.21.15.2:0/1451971534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a1283d80 0x55a6a1661080 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe080014040 tx=0x7fe080009cf0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.890+0000 7fe095685740 1 -- 172.21.15.2:0/1451971534 shutdown_connections 2024-05-14T09:58:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.890+0000 7fe095685740 1 --2- 172.21.15.2:0/1451971534 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 0x55a6a1677f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.890+0000 7fe095685740 1 --2- 172.21.15.2:0/1451971534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a129c2f0 0x55a6a1677a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.890+0000 7fe095685740 1 --2- 172.21.15.2:0/1451971534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a1283d80 0x55a6a1661080 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.890+0000 7fe095685740 1 -- 172.21.15.2:0/1451971534 >> 172.21.15.2:0/1451971534 conn(0x55a6a12b9300 msgr2=0x55a6a1660970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.890+0000 7fe095685740 1 -- 172.21.15.2:0/1451971534 shutdown_connections 2024-05-14T09:58:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.890+0000 7fe095685740 1 -- 172.21.15.2:0/1451971534 wait complete. 2024-05-14T09:58:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.891+0000 7fe095685740 1 -- start start 2024-05-14T09:58:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.891+0000 7fe095685740 1 Processor -- start 2024-05-14T09:58:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.891+0000 7fe095685740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a1283d80 0x55a6a16752c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.891+0000 7fe095685740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a129c2f0 0x55a6a1675800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.891+0000 7fe095685740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 0x55a6a16e45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.891+0000 7fe095685740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881ad4c0 con 0x55a6a129c2f0 2024-05-14T09:58:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.891+0000 7fe095685740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881ae930 con 0x55a6a1283d80 2024-05-14T09:58:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.891+0000 7fe095685740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881adb90 con 0x55a6a12a44d0 2024-05-14T09:58:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe095685740 10 client.0 _ll_register_callbacks cb 0x7fe0800093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:58:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe0877fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a129c2f0 0x55a6a1675800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe0877fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a129c2f0 0x55a6a1675800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47668/0 (socket says 172.21.15.2:47668) 2024-05-14T09:58:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe0877fe640 1 -- 172.21.15.2:0/499192687 learned_addr learned my addr 172.21.15.2:0/499192687 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe08cd73640 1 --2- 172.21.15.2:0/499192687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 0x55a6a16e45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe08cd73640 1 --2- 172.21.15.2:0/499192687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a1283d80 0x55a6a16752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe0877fe640 1 -- 172.21.15.2:0/499192687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 msgr2=0x55a6a16e45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe0877fe640 1 --2- 172.21.15.2:0/499192687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 0x55a6a16e45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe0877fe640 1 -- 172.21.15.2:0/499192687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a1283d80 msgr2=0x55a6a16752c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe0877fe640 1 --2- 172.21.15.2:0/499192687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a1283d80 0x55a6a16752c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe0877fe640 1 -- 172.21.15.2:0/499192687 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6a16e4b00 con 0x55a6a129c2f0 2024-05-14T09:58:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe0877fe640 1 --2- 172.21.15.2:0/499192687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a129c2f0 0x55a6a1675800 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe07c02a930 tx=0x7fe07c0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e232fb297a7acf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe0677fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:58:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe0677fe640 1 -- 172.21.15.2:0/499192687 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c035050 con 0x55a6a129c2f0 2024-05-14T09:58:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe0677fe640 1 -- 172.21.15.2:0/499192687 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fe07c03cc80 con 0x55a6a129c2f0 2024-05-14T09:58:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe095685740 1 -- 172.21.15.2:0/499192687 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a6a15d3050 con 0x55a6a129c2f0 2024-05-14T09:58:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe0677fe640 1 -- 172.21.15.2:0/499192687 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fe07c05c7a0 con 0x55a6a129c2f0 2024-05-14T09:58:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe095685740 10 client.17529 fetch_fsmap learned FSMap version 67 2024-05-14T09:58:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.892+0000 7fe095685740 1 -- 172.21.15.2:0/499192687 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55a6a16e5df0 con 0x55a6a129c2f0 2024-05-14T09:58:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.893+0000 7fe0677fe640 1 -- 172.21.15.2:0/499192687 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a6a15d3050 con 0x55a6a129c2f0 2024-05-14T09:58:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.893+0000 7fe0677fe640 1 -- 172.21.15.2:0/499192687 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fe07c0456c0 con 0x55a6a129c2f0 2024-05-14T09:58:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.893+0000 7fe095685740 10 client.17529 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:58:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.893+0000 7fe095685740 10 client.17529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:58:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.893+0000 7fe095685740 20 client.17529 populate_metadata read hostname 'smithi002' 2024-05-14T09:58:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.893+0000 7fe095685740 1 --2- 172.21.15.2:0/499192687 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a6a12c51b0 0x55a6a16e8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.893+0000 7fe095685740 4 client.17529 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:58:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.893+0000 7fe095685740 1 -- 172.21.15.2:0/499192687 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6a154e5f0 con 0x55a6a12c51b0 2024-05-14T09:58:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.893+0000 7fe08cd73640 1 --2- 172.21.15.2:0/499192687 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a6a12c51b0 0x55a6a16e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.894+0000 7fe08cd73640 1 --2- 172.21.15.2:0/499192687 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a6a12c51b0 0x55a6a16e8860 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f31cac68f2875cdc server_cookie=a24e123bdde86b5f in_seq=0 out_seq=0 2024-05-14T09:58:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:02.894+0000 7fe0677fe640 10 client.17529 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:02.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:02 smithi002 ceph-mon[31557]: pgmap v1119: 129 pgs: 129 active+clean; 710 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.9 MiB/s wr, 463 op/s 2024-05-14T09:58:02.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:02 smithi029 ceph-mon[36249]: pgmap v1119: 129 pgs: 129 active+clean; 710 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.9 MiB/s wr, 463 op/s 2024-05-14T09:58:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:02 smithi026 ceph-mon[36056]: pgmap v1119: 129 pgs: 129 active+clean; 710 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.9 MiB/s wr, 463 op/s 2024-05-14T09:58:03.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 282271 25.90 MB/sec execute 449 sec latency 550.114 ms 2024-05-14T09:58:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.727+0000 7fe0677fe640 1 -- 172.21.15.2:0/499192687 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a6a154e5f0 con 0x55a6a12c51b0 2024-05-14T09:58:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.727+0000 7fe0677fe640 10 client.17529 handle_command_reply: tid=0 2024-05-14T09:58:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.727+0000 7fe095685740 1 client.17529 shutdown 2024-05-14T09:58:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.727+0000 7fe095685740 1 -- 172.21.15.2:0/499192687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a129c2f0 msgr2=0x55a6a1675800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.727+0000 7fe095685740 1 --2- 172.21.15.2:0/499192687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a129c2f0 0x55a6a1675800 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe07c02a930 tx=0x7fe07c0092c0 comp rx=0 tx=0).stop 2024-05-14T09:58:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.727+0000 7fe095685740 1 -- 172.21.15.2:0/499192687 shutdown_connections 2024-05-14T09:58:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.727+0000 7fe095685740 1 --2- 172.21.15.2:0/499192687 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a6a12c51b0 0x55a6a16e8860 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.727+0000 7fe0677fe640 0 client.17529 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.727+0000 7fe095685740 1 --2- 172.21.15.2:0/499192687 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a6a12a44d0 0x55a6a16e45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.728+0000 7fe095685740 1 --2- 172.21.15.2:0/499192687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6a129c2f0 0x55a6a1675800 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.728+0000 7fe095685740 1 --2- 172.21.15.2:0/499192687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6a1283d80 0x55a6a16752c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.728+0000 7fe095685740 1 -- 172.21.15.2:0/499192687 >> 172.21.15.2:0/499192687 conn(0x55a6a12b9300 msgr2=0x55a6a1660970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.728+0000 7fe095685740 1 -- 172.21.15.2:0/499192687 shutdown_connections 2024-05-14T09:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.728+0000 7fe095685740 1 -- 172.21.15.2:0/499192687 wait complete. 2024-05-14T09:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.728+0000 7fe095685740 20 client.17529 trim_cache size 0 max 16384 2024-05-14T09:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:58:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:58:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.729+0000 7fe08e7d6640 1 -- 172.21.15.2:0/394430243 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0700b28a0 msgr2=0x7fe0700b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.729+0000 7fe08e7d6640 1 --2- 172.21.15.2:0/394430243 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0700b28a0 0x7fe0700b4db0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x55a6a1667e40 tx=0x7fe0780e8550 comp rx=0 tx=0).stop 2024-05-14T09:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.729+0000 7fe08e7d6640 1 -- 172.21.15.2:0/394430243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe088102b30 msgr2=0x7fe08819c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.729+0000 7fe08e7d6640 1 --2- 172.21.15.2:0/394430243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe088102b30 0x7fe08819c490 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe07800e9f0 tx=0x7fe07800eeb0 comp rx=0 tx=0).stop 2024-05-14T09:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.729+0000 7fe08e7d6640 1 -- 172.21.15.2:0/394430243 shutdown_connections 2024-05-14T09:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.730+0000 7fe08e7d6640 1 --2- 172.21.15.2:0/394430243 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fe0700b28a0 0x7fe0700b4db0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.730+0000 7fe08e7d6640 1 --2- 172.21.15.2:0/394430243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe088108a70 0x7fe0881a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.730+0000 7fe08e7d6640 1 --2- 172.21.15.2:0/394430243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe088103d70 0x7fe08819c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.730+0000 7fe08e7d6640 1 --2- 172.21.15.2:0/394430243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe088102b30 0x7fe08819c490 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.730+0000 7fe08e7d6640 1 -- 172.21.15.2:0/394430243 >> 172.21.15.2:0/394430243 conn(0x7fe0880fdd60 msgr2=0x7fe088100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.730+0000 7fe08e7d6640 1 -- 172.21.15.2:0/394430243 shutdown_connections 2024-05-14T09:58:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:03.730+0000 7fe08e7d6640 1 -- 172.21.15.2:0/394430243 wait complete. 2024-05-14T09:58:04.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 282708 25.90 MB/sec execute 450 sec latency 76.130 ms 2024-05-14T09:58:04.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:04 smithi002 ceph-mon[31557]: pgmap v1120: 129 pgs: 129 active+clean; 716 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.3 MiB/s wr, 505 op/s 2024-05-14T09:58:04.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:04 smithi029 ceph-mon[36249]: pgmap v1120: 129 pgs: 129 active+clean; 716 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.3 MiB/s wr, 505 op/s 2024-05-14T09:58:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:04 smithi026 ceph-mon[36056]: pgmap v1120: 129 pgs: 129 active+clean; 716 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.3 MiB/s wr, 505 op/s 2024-05-14T09:58:05.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 283408 25.92 MB/sec execute 451 sec latency 89.869 ms 2024-05-14T09:58:06.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 283982 25.94 MB/sec execute 452 sec latency 87.534 ms 2024-05-14T09:58:06.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:06 smithi002 ceph-mon[31557]: pgmap v1121: 129 pgs: 129 active+clean; 717 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.8 MiB/s wr, 415 op/s 2024-05-14T09:58:06.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:06 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:58:06.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:06 smithi029 ceph-mon[36249]: pgmap v1121: 129 pgs: 129 active+clean; 717 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.8 MiB/s wr, 415 op/s 2024-05-14T09:58:06.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:06 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:58:06.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:06 smithi026 ceph-mon[36056]: pgmap v1121: 129 pgs: 129 active+clean; 717 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.8 MiB/s wr, 415 op/s 2024-05-14T09:58:06.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:06 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:58:07.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 284486 25.94 MB/sec execute 453 sec latency 142.644 ms 2024-05-14T09:58:07.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:07 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:58:07.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:07 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:58:07.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:07 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:58:07.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:07 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:58:07.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:07 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:58:07.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:07 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:58:07.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:07 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:58:07.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:07 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:58:07.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:07 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:58:08.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 284837 25.92 MB/sec execute 454 sec latency 460.658 ms 2024-05-14T09:58:08.838 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:08 smithi029 ceph-mon[36249]: pgmap v1122: 129 pgs: 129 active+clean; 715 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.9 MiB/s wr, 426 op/s 2024-05-14T09:58:08.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:08 smithi002 ceph-mon[31557]: pgmap v1122: 129 pgs: 129 active+clean; 715 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.9 MiB/s wr, 426 op/s 2024-05-14T09:58:08.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:08 smithi026 ceph-mon[36056]: pgmap v1122: 129 pgs: 129 active+clean; 715 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.9 MiB/s wr, 426 op/s 2024-05-14T09:58:09.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 285343 25.92 MB/sec execute 455 sec latency 129.743 ms 2024-05-14T09:58:10.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 285954 25.94 MB/sec execute 456 sec latency 84.754 ms 2024-05-14T09:58:10.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:10 smithi002 ceph-mon[31557]: pgmap v1123: 129 pgs: 129 active+clean; 724 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.7 MiB/s wr, 542 op/s 2024-05-14T09:58:10.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:10 smithi029 ceph-mon[36249]: pgmap v1123: 129 pgs: 129 active+clean; 724 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.7 MiB/s wr, 542 op/s 2024-05-14T09:58:10.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:10 smithi026 ceph-mon[36056]: pgmap v1123: 129 pgs: 129 active+clean; 724 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.7 MiB/s wr, 542 op/s 2024-05-14T09:58:11.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 286590 25.97 MB/sec execute 457 sec latency 65.296 ms 2024-05-14T09:58:12.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 287025 25.96 MB/sec execute 458 sec latency 174.948 ms 2024-05-14T09:58:12.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:12 smithi002 ceph-mon[31557]: pgmap v1124: 129 pgs: 129 active+clean; 728 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.4 MiB/s wr, 479 op/s 2024-05-14T09:58:12.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:12 smithi029 ceph-mon[36249]: pgmap v1124: 129 pgs: 129 active+clean; 728 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.4 MiB/s wr, 479 op/s 2024-05-14T09:58:12.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:12 smithi026 ceph-mon[36056]: pgmap v1124: 129 pgs: 129 active+clean; 728 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.4 MiB/s wr, 479 op/s 2024-05-14T09:58:13.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 287355 25.94 MB/sec execute 459 sec latency 382.688 ms 2024-05-14T09:58:14.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 287563 25.90 MB/sec execute 460 sec latency 576.283 ms 2024-05-14T09:58:14.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:14 smithi002 ceph-mon[31557]: pgmap v1125: 129 pgs: 129 active+clean; 730 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.1 MiB/s wr, 527 op/s 2024-05-14T09:58:14.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:14 smithi029 ceph-mon[36249]: pgmap v1125: 129 pgs: 129 active+clean; 730 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.1 MiB/s wr, 527 op/s 2024-05-14T09:58:14.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:14 smithi026 ceph-mon[36056]: pgmap v1125: 129 pgs: 129 active+clean; 730 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.1 MiB/s wr, 527 op/s 2024-05-14T09:58:15.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 287962 25.89 MB/sec execute 461 sec latency 172.625 ms 2024-05-14T09:58:16.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 288579 25.91 MB/sec execute 462 sec latency 205.407 ms 2024-05-14T09:58:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.905+0000 7fc120215640 1 -- 172.21.15.2:0/1444975920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc118103db0 msgr2=0x7fc1181041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.905+0000 7fc120215640 1 --2- 172.21.15.2:0/1444975920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc118103db0 0x7fc1181041c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc108009900 tx=0x7fc10802a410 comp rx=0 tx=0).stop 2024-05-14T09:58:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.905+0000 7fc120215640 1 -- 172.21.15.2:0/1444975920 shutdown_connections 2024-05-14T09:58:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.905+0000 7fc120215640 1 --2- 172.21.15.2:0/1444975920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc118108ab0 0x7fc11810aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.905+0000 7fc120215640 1 --2- 172.21.15.2:0/1444975920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc118103db0 0x7fc1181041c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.905+0000 7fc120215640 1 --2- 172.21.15.2:0/1444975920 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc118102b70 0x7fc118103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.905+0000 7fc120215640 1 -- 172.21.15.2:0/1444975920 >> 172.21.15.2:0/1444975920 conn(0x7fc1180fdd60 msgr2=0x7fc118100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.905+0000 7fc120215640 1 -- 172.21.15.2:0/1444975920 shutdown_connections 2024-05-14T09:58:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.905+0000 7fc120215640 1 -- 172.21.15.2:0/1444975920 wait complete. 2024-05-14T09:58:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.906+0000 7fc120215640 1 Processor -- start 2024-05-14T09:58:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.906+0000 7fc120215640 1 -- start start 2024-05-14T09:58:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.906+0000 7fc120215640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc118102b70 0x7fc11806bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.906+0000 7fc120215640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc118103db0 0x7fc11819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.906+0000 7fc120215640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc118108ab0 0x7fc1181a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.906+0000 7fc120215640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc11810e710 con 0x7fc118102b70 2024-05-14T09:58:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.906+0000 7fc120215640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc11810e590 con 0x7fc118108ab0 2024-05-14T09:58:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.906+0000 7fc120215640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc11810e890 con 0x7fc118103db0 2024-05-14T09:58:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.906+0000 7fc11dfb1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc118102b70 0x7fc11806bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.906+0000 7fc11dfb1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc118102b70 0x7fc11806bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33116/0 (socket says 172.21.15.2:33116) 2024-05-14T09:58:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.906+0000 7fc11dfb1640 1 -- 172.21.15.2:0/1001376719 learned_addr learned my addr 172.21.15.2:0/1001376719 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.906+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/1001376719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc118108ab0 0x7fc1181a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.906+0000 7fc11d7b0640 1 --2- 172.21.15.2:0/1001376719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc118103db0 0x7fc11819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.907+0000 7fc11dfb1640 1 -- 172.21.15.2:0/1001376719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc118103db0 msgr2=0x7fc11819f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.907+0000 7fc11dfb1640 1 --2- 172.21.15.2:0/1001376719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc118103db0 0x7fc11819f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.907+0000 7fc11dfb1640 1 -- 172.21.15.2:0/1001376719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc118108ab0 msgr2=0x7fc1181a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.907+0000 7fc11dfb1640 1 --2- 172.21.15.2:0/1001376719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc118108ab0 0x7fc1181a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.907+0000 7fc11dfb1640 1 -- 172.21.15.2:0/1001376719 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1181a1e70 con 0x7fc118102b70 2024-05-14T09:58:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.907+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/1001376719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc118108ab0 0x7fc1181a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:58:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.907+0000 7fc11dfb1640 1 --2- 172.21.15.2:0/1001376719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc118102b70 0x7fc11806bda0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc11400e9f0 tx=0x7fc11400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.907+0000 7fc10effd640 1 -- 172.21.15.2:0/1001376719 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11400ccc0 con 0x7fc118102b70 2024-05-14T09:58:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.907+0000 7fc120215640 1 -- 172.21.15.2:0/1001376719 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1181a2160 con 0x7fc118102b70 2024-05-14T09:58:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.907+0000 7fc120215640 1 -- 172.21.15.2:0/1001376719 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1181a2730 con 0x7fc118102b70 2024-05-14T09:58:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.908+0000 7fc10effd640 1 -- 172.21.15.2:0/1001376719 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1140035f0 con 0x7fc118102b70 2024-05-14T09:58:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.908+0000 7fc10effd640 1 -- 172.21.15.2:0/1001376719 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11400ce60 con 0x7fc118102b70 2024-05-14T09:58:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.912+0000 7fc10effd640 1 -- 172.21.15.2:0/1001376719 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc114010690 con 0x7fc118102b70 2024-05-14T09:58:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.913+0000 7fc1271a3740 1 Processor -- start 2024-05-14T09:58:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.913+0000 7fc1271a3740 1 -- start start 2024-05-14T09:58:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.913+0000 7fc1271a3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f192a1b0 0x55f4f1cd6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.913+0000 7fc1271a3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f19012f0 0x55f4f1cd8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.913+0000 7fc1271a3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19094d0 0x55f4f1cdb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.913+0000 7fc1271a3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f4f1c305d0 con 0x55f4f19012f0 2024-05-14T09:58:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.913+0000 7fc1271a3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f4f1be9890 con 0x55f4f192a1b0 2024-05-14T09:58:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.913+0000 7fc1271a3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f4f1c2c590 con 0x55f4f19094d0 2024-05-14T09:58:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.913+0000 7fc11e7b2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f192a1b0 0x55f4f1cd6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.913+0000 7fc11e7b2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f192a1b0 0x55f4f1cd6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:43446/0 (socket says 172.21.15.2:43446) 2024-05-14T09:58:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.913+0000 7fc11e7b2640 1 -- 172.21.15.2:0/1320623878 learned_addr learned my addr 172.21.15.2:0/1320623878 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.913+0000 7fc11d7b0640 1 --2- 172.21.15.2:0/1320623878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19094d0 0x55f4f1cdb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.914+0000 7fc11dfb1640 1 --2- 172.21.15.2:0/1320623878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f19012f0 0x55f4f1cd8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.914+0000 7fc11e7b2640 1 -- 172.21.15.2:0/1320623878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19094d0 msgr2=0x55f4f1cdb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.914+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/1320623878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19094d0 0x55f4f1cdb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.914+0000 7fc11e7b2640 1 -- 172.21.15.2:0/1320623878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f19012f0 msgr2=0x55f4f1cd8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.914+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/1320623878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f19012f0 0x55f4f1cd8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.914+0000 7fc11e7b2640 1 -- 172.21.15.2:0/1320623878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f1942210 con 0x55f4f192a1b0 2024-05-14T09:58:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.914+0000 7fc11dfb1640 1 --2- 172.21.15.2:0/1320623878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f19012f0 0x55f4f1cd8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:58:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.914+0000 7fc10effd640 1 --2- 172.21.15.2:0/1001376719 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc1040b2920 0x7fc1040b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.914+0000 7fc11dfb1640 1 --2- 172.21.15.2:0/1001376719 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc1040b2920 0x7fc1040b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.914+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/1320623878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f192a1b0 0x55f4f1cd6870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc11000a870 tx=0x7fc110009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9771f2ab1620d25 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.915+0000 7fc103fff640 1 -- 172.21.15.2:0/1320623878 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110018020 con 0x55f4f192a1b0 2024-05-14T09:58:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.915+0000 7fc103fff640 1 -- 172.21.15.2:0/1320623878 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc110014400 con 0x55f4f192a1b0 2024-05-14T09:58:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.915+0000 7fc103fff640 1 -- 172.21.15.2:0/1320623878 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1100135c0 con 0x55f4f192a1b0 2024-05-14T09:58:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.915+0000 7fc10effd640 1 -- 172.21.15.2:0/1001376719 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fc1140e2820 con 0x7fc118102b70 2024-05-14T09:58:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.916+0000 7fc1271a3740 1 -- 172.21.15.2:0/1320623878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f192a1b0 msgr2=0x55f4f1cd6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.916+0000 7fc11dfb1640 1 --2- 172.21.15.2:0/1001376719 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc1040b2920 0x7fc1040b4e30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc114013040 tx=0x7fc1140e8550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.916+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1320623878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f192a1b0 0x55f4f1cd6870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc11000a870 tx=0x7fc110009650 comp rx=0 tx=0).stop 2024-05-14T09:58:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.916+0000 7fc1271a3740 1 -- 172.21.15.2:0/1320623878 shutdown_connections 2024-05-14T09:58:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.916+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1320623878 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19094d0 0x55f4f1cdb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.916+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1320623878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f19012f0 0x55f4f1cd8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.916+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1320623878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f192a1b0 0x55f4f1cd6870 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.916+0000 7fc1271a3740 1 -- 172.21.15.2:0/1320623878 >> 172.21.15.2:0/1320623878 conn(0x55f4f191e300 msgr2=0x55f4f1cc7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.916+0000 7fc1271a3740 1 -- 172.21.15.2:0/1320623878 shutdown_connections 2024-05-14T09:58:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.916+0000 7fc1271a3740 1 -- 172.21.15.2:0/1320623878 wait complete. 2024-05-14T09:58:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.916+0000 7fc1271a3740 1 -- start start 2024-05-14T09:58:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.917+0000 7fc1271a3740 1 Processor -- start 2024-05-14T09:58:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.917+0000 7fc1271a3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 0x55f4f1cdded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.917+0000 7fc1271a3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 0x55f4f1cde410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.917+0000 7fc1271a3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f192a1b0 0x55f4f1ce5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.917+0000 7fc1271a3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f4f1cdc130 con 0x55f4f192a1b0 2024-05-14T09:58:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.917+0000 7fc1271a3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f4f1bed370 con 0x55f4f19094d0 2024-05-14T09:58:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.917+0000 7fc1271a3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55f4f1cdc2b0 con 0x55f4f19012f0 2024-05-14T09:58:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.917+0000 7fc1271a3740 10 client.0 _ll_register_callbacks cb 0x55f4f1941a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:58:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.917+0000 7fc11e7b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 0x55f4f1cdded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.917+0000 7fc11e7b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 0x55f4f1cdded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:59114/0 (socket says 172.21.15.2:59114) 2024-05-14T09:58:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.917+0000 7fc11e7b2640 1 -- 172.21.15.2:0/1219401964 learned_addr learned my addr 172.21.15.2:0/1219401964 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.917+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/1219401964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f192a1b0 0x55f4f1ce5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.917+0000 7fc11d7b0640 1 --2- 172.21.15.2:0/1219401964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 0x55f4f1cde410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.918+0000 7fc11e7b2640 1 -- 172.21.15.2:0/1219401964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 msgr2=0x55f4f1cde410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.918+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/1219401964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 0x55f4f1cde410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.918+0000 7fc11e7b2640 1 -- 172.21.15.2:0/1219401964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f192a1b0 msgr2=0x55f4f1ce5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.918+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/1219401964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f192a1b0 0x55f4f1ce5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.918+0000 7fc11e7b2640 1 -- 172.21.15.2:0/1219401964 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f1a0e620 con 0x55f4f19012f0 2024-05-14T09:58:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.918+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/1219401964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 0x55f4f1cdded0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc110019000 tx=0x7fc110004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e72511cec8d7906c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.918+0000 7fc1017fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:58:16.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.918+0000 7fc1017fa640 1 -- 172.21.15.2:0/1219401964 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110018030 con 0x55f4f19012f0 2024-05-14T09:58:16.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.918+0000 7fc1271a3740 1 -- 172.21.15.2:0/1219401964 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc110014400 con 0x55f4f19012f0 2024-05-14T09:58:16.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.919+0000 7fc1017fa640 1 -- 172.21.15.2:0/1219401964 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fc11000ee30 con 0x55f4f19012f0 2024-05-14T09:58:16.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.919+0000 7fc1017fa640 1 -- 172.21.15.2:0/1219401964 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fc11001e430 con 0x55f4f19012f0 2024-05-14T09:58:16.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.919+0000 7fc1271a3740 10 client.27145 fetch_fsmap learned FSMap version 67 2024-05-14T09:58:16.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.919+0000 7fc1271a3740 1 -- 172.21.15.2:0/1219401964 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55f4f1ce8310 con 0x55f4f19012f0 2024-05-14T09:58:16.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.919+0000 7fc1017fa640 1 -- 172.21.15.2:0/1219401964 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc110014400 con 0x55f4f19012f0 2024-05-14T09:58:16.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.919+0000 7fc1017fa640 1 -- 172.21.15.2:0/1219401964 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fc11001d450 con 0x55f4f19012f0 2024-05-14T09:58:16.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.919+0000 7fc1271a3740 10 client.27145 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:58:16.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.919+0000 7fc1271a3740 10 client.27145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:58:16.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.919+0000 7fc1271a3740 20 client.27145 populate_metadata read hostname 'smithi002' 2024-05-14T09:58:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.919+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1219401964 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4f18e8d80 0x55f4f1cebe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.919+0000 7fc1271a3740 4 client.27145 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:58:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.919+0000 7fc1271a3740 1 -- 172.21.15.2:0/1219401964 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4f1c3e1e0 con 0x55f4f18e8d80 2024-05-14T09:58:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.920+0000 7fc11d7b0640 1 --2- 172.21.15.2:0/1219401964 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4f18e8d80 0x55f4f1cebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.921+0000 7fc1017fa640 10 client.27145 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:16.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:16 smithi002 ceph-mon[31557]: pgmap v1126: 129 pgs: 129 active+clean; 707 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.1 MiB/s wr, 486 op/s 2024-05-14T09:58:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.921+0000 7fc11d7b0640 1 --2- 172.21.15.2:0/1219401964 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4f18e8d80 0x55f4f1cebe30 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8989b701ba77038e server_cookie=1e78164d42207bcf in_seq=0 out_seq=0 2024-05-14T09:58:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.922+0000 7fc1017fa640 1 -- 172.21.15.2:0/1219401964 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4f1c3e1e0 con 0x55f4f18e8d80 2024-05-14T09:58:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.922+0000 7fc1017fa640 10 client.27145 handle_command_reply: tid=0 2024-05-14T09:58:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.923+0000 7fc1271a3740 1 client.27145 shutdown 2024-05-14T09:58:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.923+0000 7fc1271a3740 1 -- 172.21.15.2:0/1219401964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 msgr2=0x55f4f1cdded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.923+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1219401964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 0x55f4f1cdded0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc110019000 tx=0x7fc110004b50 comp rx=0 tx=0).stop 2024-05-14T09:58:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.923+0000 7fc1271a3740 1 -- 172.21.15.2:0/1219401964 shutdown_connections 2024-05-14T09:58:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.923+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1219401964 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4f18e8d80 0x55f4f1cebe30 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.923+0000 7fc1017fa640 0 client.27145 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.923+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1219401964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f192a1b0 0x55f4f1ce5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.923+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1219401964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 0x55f4f1cde410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.923+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1219401964 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 0x55f4f1cdded0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.923+0000 7fc1271a3740 1 -- 172.21.15.2:0/1219401964 >> 172.21.15.2:0/1219401964 conn(0x55f4f191e300 msgr2=0x55f4f1cc63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.923+0000 7fc1271a3740 1 -- 172.21.15.2:0/1219401964 shutdown_connections 2024-05-14T09:58:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.923+0000 7fc1271a3740 1 -- 172.21.15.2:0/1219401964 wait complete. 2024-05-14T09:58:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.923+0000 7fc1271a3740 20 client.27145 trim_cache size 0 max 16384 2024-05-14T09:58:16.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:16 smithi029 ceph-mon[36249]: pgmap v1126: 129 pgs: 129 active+clean; 707 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.1 MiB/s wr, 486 op/s 2024-05-14T09:58:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.943+0000 7fc1271a3740 1 Processor -- start 2024-05-14T09:58:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.943+0000 7fc1271a3740 1 -- start start 2024-05-14T09:58:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.943+0000 7fc1271a3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f191e300 0x55f4f1cc5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.943+0000 7fc1271a3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 0x55f4f1ce5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.944+0000 7fc1271a3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 0x55f4f1ce5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.944+0000 7fc11e7b2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f191e300 0x55f4f1cc5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.944+0000 7fc1271a3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc012660 con 0x55f4f191e300 2024-05-14T09:58:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.944+0000 7fc1271a3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc00ff30 con 0x55f4f19094d0 2024-05-14T09:58:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.944+0000 7fc1271a3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc00dd80 con 0x55f4f19012f0 2024-05-14T09:58:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.944+0000 7fc11e7b2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f191e300 0x55f4f1cc5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33146/0 (socket says 172.21.15.2:33146) 2024-05-14T09:58:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.944+0000 7fc11e7b2640 1 -- 172.21.15.2:0/2517597195 learned_addr learned my addr 172.21.15.2:0/2517597195 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.944+0000 7fc11d7b0640 1 --2- 172.21.15.2:0/2517597195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 0x55f4f1ce5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.944+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/2517597195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 0x55f4f1ce5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.944+0000 7fc11e7b2640 1 -- 172.21.15.2:0/2517597195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 msgr2=0x55f4f1ce5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.944+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/2517597195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 0x55f4f1ce5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.944+0000 7fc11e7b2640 1 -- 172.21.15.2:0/2517597195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 msgr2=0x55f4f1ce5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.944+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/2517597195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 0x55f4f1ce5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.944+0000 7fc11e7b2640 1 -- 172.21.15.2:0/2517597195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f1cc7c80 con 0x55f4f191e300 2024-05-14T09:58:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.945+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/2517597195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f191e300 0x55f4f1cc5970 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc110009710 tx=0x7fc11000e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce39a4299bae1485 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.945+0000 7fc1037fe640 1 -- 172.21.15.2:0/2517597195 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110018050 con 0x55f4f191e300 2024-05-14T09:58:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.945+0000 7fc1037fe640 1 -- 172.21.15.2:0/2517597195 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc110023030 con 0x55f4f191e300 2024-05-14T09:58:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.945+0000 7fc1037fe640 1 -- 172.21.15.2:0/2517597195 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11001da90 con 0x55f4f191e300 2024-05-14T09:58:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.946+0000 7fc1271a3740 1 -- 172.21.15.2:0/2517597195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f191e300 msgr2=0x55f4f1cc5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.946+0000 7fc1271a3740 1 --2- 172.21.15.2:0/2517597195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f191e300 0x55f4f1cc5970 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc110009710 tx=0x7fc11000e8b0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.946+0000 7fc1271a3740 1 -- 172.21.15.2:0/2517597195 shutdown_connections 2024-05-14T09:58:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.946+0000 7fc1271a3740 1 --2- 172.21.15.2:0/2517597195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 0x55f4f1ce5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.946+0000 7fc1271a3740 1 --2- 172.21.15.2:0/2517597195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 0x55f4f1ce5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.946+0000 7fc1271a3740 1 --2- 172.21.15.2:0/2517597195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f191e300 0x55f4f1cc5970 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.946+0000 7fc1271a3740 1 -- 172.21.15.2:0/2517597195 >> 172.21.15.2:0/2517597195 conn(0x55f4f1cde5e0 msgr2=0x55f4f1cdd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.946+0000 7fc1271a3740 1 -- 172.21.15.2:0/2517597195 shutdown_connections 2024-05-14T09:58:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.946+0000 7fc1271a3740 1 -- 172.21.15.2:0/2517597195 wait complete. 2024-05-14T09:58:16.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.947+0000 7fc1271a3740 1 -- start start 2024-05-14T09:58:16.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.947+0000 7fc1271a3740 1 Processor -- start 2024-05-14T09:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.947+0000 7fc1271a3740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 0x55f4f1ce5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.947+0000 7fc1271a3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 0x55f4f1ce2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.947+0000 7fc1271a3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f191e300 0x55f4f1ce2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.947+0000 7fc1271a3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc00e3f0 con 0x55f4f191e300 2024-05-14T09:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.947+0000 7fc1271a3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc00f860 con 0x55f4f19094d0 2024-05-14T09:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.947+0000 7fc1271a3740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc00eac0 con 0x55f4f19012f0 2024-05-14T09:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.948+0000 7fc1271a3740 10 client.0 _ll_register_callbacks cb 0x7fc1140e8420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.948+0000 7fc11e7b2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 0x55f4f1ce5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.948+0000 7fc11d7b0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 0x55f4f1ce2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.948+0000 7fc11e7b2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f191e300 0x55f4f1ce2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.948+0000 7fc11d7b0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 0x55f4f1ce2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:43468/0 (socket says 172.21.15.2:43468) 2024-05-14T09:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.948+0000 7fc11d7b0640 1 -- 172.21.15.2:0/1366522610 learned_addr learned my addr 172.21.15.2:0/1366522610 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.948+0000 7fc11e7b2640 1 -- 172.21.15.2:0/1366522610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 msgr2=0x55f4f1ce2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.948+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/1366522610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 0x55f4f1ce2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.948+0000 7fc11e7b2640 1 -- 172.21.15.2:0/1366522610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f191e300 msgr2=0x55f4f1ce2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.948+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/1366522610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f191e300 0x55f4f1ce2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.948+0000 7fc11e7b2640 1 -- 172.21.15.2:0/1366522610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f1d52250 con 0x55f4f19012f0 2024-05-14T09:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.948+0000 7fc11e7b2640 1 --2- 172.21.15.2:0/1366522610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 0x55f4f1ce5b10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc110002230 tx=0x7fc11000e8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6533e21acded893 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.948+0000 7fc1017fa640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.949+0000 7fc1017fa640 1 -- 172.21.15.2:0/1366522610 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110018070 con 0x55f4f19012f0 2024-05-14T09:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.949+0000 7fc1017fa640 1 -- 172.21.15.2:0/1366522610 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fc11001d450 con 0x55f4f19012f0 2024-05-14T09:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.949+0000 7fc1271a3740 1 -- 172.21.15.2:0/1366522610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc110023030 con 0x55f4f19012f0 2024-05-14T09:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.949+0000 7fc1017fa640 1 -- 172.21.15.2:0/1366522610 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fc11004c040 con 0x55f4f19012f0 2024-05-14T09:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.949+0000 7fc1271a3740 10 client.27151 fetch_fsmap learned FSMap version 67 2024-05-14T09:58:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.949+0000 7fc1271a3740 1 -- 172.21.15.2:0/1366522610 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55f4f1ccb040 con 0x55f4f19012f0 2024-05-14T09:58:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.949+0000 7fc1017fa640 1 -- 172.21.15.2:0/1366522610 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc110023030 con 0x55f4f19012f0 2024-05-14T09:58:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.949+0000 7fc1017fa640 1 -- 172.21.15.2:0/1366522610 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fc110052070 con 0x55f4f19012f0 2024-05-14T09:58:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.949+0000 7fc1271a3740 10 client.27151 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:58:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.949+0000 7fc1271a3740 10 client.27151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:58:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.949+0000 7fc1271a3740 20 client.27151 populate_metadata read hostname 'smithi002' 2024-05-14T09:58:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.949+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1366522610 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4f192a1b0 0x55f4f1ccda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.949+0000 7fc1271a3740 4 client.27151 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:58:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.949+0000 7fc1271a3740 1 -- 172.21.15.2:0/1366522610 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4f1c38050 con 0x55f4f192a1b0 2024-05-14T09:58:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.950+0000 7fc11d7b0640 1 --2- 172.21.15.2:0/1366522610 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4f192a1b0 0x55f4f1ccda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.950+0000 7fc1017fa640 10 client.27151 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:16.950+0000 7fc11d7b0640 1 --2- 172.21.15.2:0/1366522610 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4f192a1b0 0x55f4f1ccda00 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cd1a67e7cce3176 server_cookie=cb8f19a711814090 in_seq=0 out_seq=0 2024-05-14T09:58:16.954 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:16 smithi026 ceph-mon[36056]: pgmap v1126: 129 pgs: 129 active+clean; 707 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.1 MiB/s wr, 486 op/s 2024-05-14T09:58:17.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 288906 25.89 MB/sec execute 463 sec latency 542.359 ms 2024-05-14T09:58:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.676+0000 7fc1017fa640 1 -- 172.21.15.2:0/1366522610 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f4f1c38050 con 0x55f4f192a1b0 2024-05-14T09:58:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.676+0000 7fc1017fa640 10 client.27151 handle_command_reply: tid=0 2024-05-14T09:58:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.676+0000 7fc1271a3740 1 client.27151 shutdown 2024-05-14T09:58:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.676+0000 7fc1271a3740 1 -- 172.21.15.2:0/1366522610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 msgr2=0x55f4f1ce5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.677+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1366522610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 0x55f4f1ce5b10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc110002230 tx=0x7fc11000e8b0 comp rx=0 tx=0).stop 2024-05-14T09:58:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.677+0000 7fc1271a3740 1 -- 172.21.15.2:0/1366522610 shutdown_connections 2024-05-14T09:58:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.677+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1366522610 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55f4f192a1b0 0x55f4f1ccda00 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.677+0000 7fc1017fa640 0 client.27151 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.677+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1366522610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4f191e300 0x55f4f1ce2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.677+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1366522610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4f19094d0 0x55f4f1ce2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.677+0000 7fc1271a3740 1 --2- 172.21.15.2:0/1366522610 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f4f19012f0 0x55f4f1ce5b10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.677+0000 7fc1271a3740 1 -- 172.21.15.2:0/1366522610 >> 172.21.15.2:0/1366522610 conn(0x55f4f1cde5e0 msgr2=0x55f4f1cd3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.677+0000 7fc1271a3740 1 -- 172.21.15.2:0/1366522610 shutdown_connections 2024-05-14T09:58:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.677+0000 7fc1271a3740 1 -- 172.21.15.2:0/1366522610 wait complete. 2024-05-14T09:58:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.677+0000 7fc1271a3740 20 client.27151 trim_cache size 0 max 16384 2024-05-14T09:58:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:58:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:58:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:58:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:58:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.678+0000 7fc120215640 1 -- 172.21.15.2:0/1001376719 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc1040b2920 msgr2=0x7fc1040b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.678+0000 7fc120215640 1 --2- 172.21.15.2:0/1001376719 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc1040b2920 0x7fc1040b4e30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc114013040 tx=0x7fc1140e8550 comp rx=0 tx=0).stop 2024-05-14T09:58:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.678+0000 7fc120215640 1 -- 172.21.15.2:0/1001376719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc118102b70 msgr2=0x7fc11806bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.678+0000 7fc120215640 1 --2- 172.21.15.2:0/1001376719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc118102b70 0x7fc11806bda0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc11400e9f0 tx=0x7fc11400eeb0 comp rx=0 tx=0).stop 2024-05-14T09:58:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.678+0000 7fc120215640 1 -- 172.21.15.2:0/1001376719 shutdown_connections 2024-05-14T09:58:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.678+0000 7fc120215640 1 --2- 172.21.15.2:0/1001376719 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc1040b2920 0x7fc1040b4e30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.678+0000 7fc120215640 1 --2- 172.21.15.2:0/1001376719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc118108ab0 0x7fc1181a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.678+0000 7fc120215640 1 --2- 172.21.15.2:0/1001376719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc118103db0 0x7fc11819f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.678+0000 7fc120215640 1 --2- 172.21.15.2:0/1001376719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc118102b70 0x7fc11806bda0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.678+0000 7fc120215640 1 -- 172.21.15.2:0/1001376719 >> 172.21.15.2:0/1001376719 conn(0x7fc1180fdd60 msgr2=0x7fc118104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.678+0000 7fc120215640 1 -- 172.21.15.2:0/1001376719 shutdown_connections 2024-05-14T09:58:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:17.678+0000 7fc120215640 1 -- 172.21.15.2:0/1001376719 wait complete. 2024-05-14T09:58:18.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 289402 25.89 MB/sec execute 464 sec latency 553.223 ms 2024-05-14T09:58:18.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:18 smithi002 ceph-mon[31557]: pgmap v1127: 129 pgs: 129 active+clean; 700 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.8 MiB/s wr, 495 op/s 2024-05-14T09:58:18.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:18 smithi029 ceph-mon[36249]: pgmap v1127: 129 pgs: 129 active+clean; 700 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.8 MiB/s wr, 495 op/s 2024-05-14T09:58:18.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:18 smithi026 ceph-mon[36056]: pgmap v1127: 129 pgs: 129 active+clean; 700 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.8 MiB/s wr, 495 op/s 2024-05-14T09:58:19.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 289424 25.83 MB/sec execute 465 sec latency 1299.578 ms 2024-05-14T09:58:20.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 289430 25.78 MB/sec execute 466 sec latency 2299.625 ms 2024-05-14T09:58:20.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:20 smithi002 ceph-mon[31557]: pgmap v1128: 129 pgs: 129 active+clean; 685 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.8 MiB/s wr, 585 op/s 2024-05-14T09:58:20.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:20 smithi029 ceph-mon[36249]: pgmap v1128: 129 pgs: 129 active+clean; 685 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.8 MiB/s wr, 585 op/s 2024-05-14T09:58:20.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:20 smithi026 ceph-mon[36056]: pgmap v1128: 129 pgs: 129 active+clean; 685 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.8 MiB/s wr, 585 op/s 2024-05-14T09:58:21.624 INFO:tasks.workunit.client.0.smithi002.stdout: 10 289470 25.72 MB/sec execute 467 sec latency 2969.922 ms 2024-05-14T09:58:22.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 289651 25.69 MB/sec execute 468 sec latency 708.181 ms 2024-05-14T09:58:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:22 smithi002 ceph-mon[31557]: pgmap v1129: 129 pgs: 129 active+clean; 717 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.7 MiB/s wr, 557 op/s 2024-05-14T09:58:22.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:22 smithi029 ceph-mon[36249]: pgmap v1129: 129 pgs: 129 active+clean; 717 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.7 MiB/s wr, 557 op/s 2024-05-14T09:58:22.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:22 smithi026 ceph-mon[36056]: pgmap v1129: 129 pgs: 129 active+clean; 717 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.7 MiB/s wr, 557 op/s 2024-05-14T09:58:23.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 289911 25.68 MB/sec execute 469 sec latency 157.776 ms 2024-05-14T09:58:24.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 290153 25.67 MB/sec execute 470 sec latency 170.170 ms 2024-05-14T09:58:24.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:24 smithi002 ceph-mon[31557]: pgmap v1130: 129 pgs: 129 active+clean; 784 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 14 MiB/s wr, 657 op/s 2024-05-14T09:58:24.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:24 smithi029 ceph-mon[36249]: pgmap v1130: 129 pgs: 129 active+clean; 784 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 14 MiB/s wr, 657 op/s 2024-05-14T09:58:24.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:24 smithi026 ceph-mon[36056]: pgmap v1130: 129 pgs: 129 active+clean; 784 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 14 MiB/s wr, 657 op/s 2024-05-14T09:58:25.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 290395 25.63 MB/sec execute 471 sec latency 330.209 ms 2024-05-14T09:58:26.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 290649 25.61 MB/sec execute 472 sec latency 348.097 ms 2024-05-14T09:58:26.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:26 smithi002 ceph-mon[31557]: pgmap v1131: 129 pgs: 129 active+clean; 787 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 14 MiB/s wr, 646 op/s 2024-05-14T09:58:26.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:26 smithi029 ceph-mon[36249]: pgmap v1131: 129 pgs: 129 active+clean; 787 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 14 MiB/s wr, 646 op/s 2024-05-14T09:58:26.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:26 smithi026 ceph-mon[36056]: pgmap v1131: 129 pgs: 129 active+clean; 787 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 14 MiB/s wr, 646 op/s 2024-05-14T09:58:27.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 290904 25.58 MB/sec execute 473 sec latency 185.953 ms 2024-05-14T09:58:28.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 291113 25.55 MB/sec execute 474 sec latency 141.263 ms 2024-05-14T09:58:28.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:28 smithi002 ceph-mon[31557]: pgmap v1132: 129 pgs: 129 active+clean; 760 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 16 MiB/s wr, 741 op/s 2024-05-14T09:58:28.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:28 smithi029 ceph-mon[36249]: pgmap v1132: 129 pgs: 129 active+clean; 760 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 16 MiB/s wr, 741 op/s 2024-05-14T09:58:28.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:28 smithi026 ceph-mon[36056]: pgmap v1132: 129 pgs: 129 active+clean; 760 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 16 MiB/s wr, 741 op/s 2024-05-14T09:58:29.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 291565 25.54 MB/sec execute 475 sec latency 122.925 ms 2024-05-14T09:58:30.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 292198 25.57 MB/sec execute 476 sec latency 66.773 ms 2024-05-14T09:58:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.816+0000 7f811d7a8640 1 -- 172.21.15.2:0/1289211268 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8118102b50 msgr2=0x7f8118103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.816+0000 7f811d7a8640 1 --2- 172.21.15.2:0/1289211268 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8118102b50 0x7f8118103850 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8108009900 tx=0x7f810802a410 comp rx=0 tx=0).stop 2024-05-14T09:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.816+0000 7f811d7a8640 1 -- 172.21.15.2:0/1289211268 shutdown_connections 2024-05-14T09:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.816+0000 7f811d7a8640 1 --2- 172.21.15.2:0/1289211268 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8118108a90 0x7f811810aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.816+0000 7f811d7a8640 1 --2- 172.21.15.2:0/1289211268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8118103d90 0x7f81181041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.816+0000 7f811d7a8640 1 --2- 172.21.15.2:0/1289211268 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8118102b50 0x7f8118103850 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.816+0000 7f811d7a8640 1 -- 172.21.15.2:0/1289211268 >> 172.21.15.2:0/1289211268 conn(0x7f81180fdd40 msgr2=0x7f81181001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.816+0000 7f811d7a8640 1 -- 172.21.15.2:0/1289211268 shutdown_connections 2024-05-14T09:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.816+0000 7f811d7a8640 1 -- 172.21.15.2:0/1289211268 wait complete. 2024-05-14T09:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.817+0000 7f811d7a8640 1 Processor -- start 2024-05-14T09:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.817+0000 7f811d7a8640 1 -- start start 2024-05-14T09:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.817+0000 7f811d7a8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8118102b50 0x7f811806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.817+0000 7f811d7a8640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8118103d90 0x7f811819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.817+0000 7f811d7a8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8118108a90 0x7f81181a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.817+0000 7f811d7a8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f811810e6b0 con 0x7f8118108a90 2024-05-14T09:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.817+0000 7f811d7a8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f811810e530 con 0x7f8118102b50 2024-05-14T09:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.817+0000 7f811d7a8640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f811810e830 con 0x7f8118103d90 2024-05-14T09:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.817+0000 7f81177fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8118108a90 0x7f81181a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.817+0000 7f81177fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8118108a90 0x7f81181a1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33840/0 (socket says 172.21.15.2:33840) 2024-05-14T09:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.817+0000 7f81177fe640 1 -- 172.21.15.2:0/2053279605 learned_addr learned my addr 172.21.15.2:0/2053279605 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.817+0000 7f8116ffd640 1 --2- 172.21.15.2:0/2053279605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8118102b50 0x7f811806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.817+0000 7f81167fc640 1 --2- 172.21.15.2:0/2053279605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8118103d90 0x7f811819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.818+0000 7f81177fe640 1 -- 172.21.15.2:0/2053279605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8118103d90 msgr2=0x7f811819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.818+0000 7f81177fe640 1 --2- 172.21.15.2:0/2053279605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8118103d90 0x7f811819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.818+0000 7f81177fe640 1 -- 172.21.15.2:0/2053279605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8118102b50 msgr2=0x7f811806b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.818+0000 7f81177fe640 1 --2- 172.21.15.2:0/2053279605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8118102b50 0x7f811806b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.818+0000 7f81177fe640 1 -- 172.21.15.2:0/2053279605 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81181a1e00 con 0x7f8118108a90 2024-05-14T09:58:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.818+0000 7f81177fe640 1 --2- 172.21.15.2:0/2053279605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8118108a90 0x7f81181a1670 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f811000cce0 tx=0x7f811000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.818+0000 7f80fffff640 1 -- 172.21.15.2:0/2053279605 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811000f450 con 0x7f8118108a90 2024-05-14T09:58:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.818+0000 7f811d7a8640 1 -- 172.21.15.2:0/2053279605 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81181a20f0 con 0x7f8118108a90 2024-05-14T09:58:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.818+0000 7f80fffff640 1 -- 172.21.15.2:0/2053279605 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f811000faa0 con 0x7f8118108a90 2024-05-14T09:58:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.818+0000 7f80fffff640 1 -- 172.21.15.2:0/2053279605 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110018a80 con 0x7f8118108a90 2024-05-14T09:58:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.818+0000 7f811d7a8640 1 -- 172.21.15.2:0/2053279605 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81181a26c0 con 0x7f8118108a90 2024-05-14T09:58:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.821+0000 7f812465b740 1 Processor -- start 2024-05-14T09:58:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.821+0000 7f80fffff640 1 -- 172.21.15.2:0/2053279605 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f811000fc50 con 0x7f8118108a90 2024-05-14T09:58:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.821+0000 7f812465b740 1 -- start start 2024-05-14T09:58:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f812465b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bff11b0 0x55a62c392a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f812465b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c392fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f812465b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c399ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f812465b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a62c2f75d0 con 0x55a62bfd04d0 2024-05-14T09:58:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f812465b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a62c2b0890 con 0x55a62bff11b0 2024-05-14T09:58:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f812465b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a62c2f3590 con 0x55a62bfc82f0 2024-05-14T09:58:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f81167fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c399ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f8116ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c392fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f8116ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c392fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:50596/0 (socket says 172.21.15.2:50596) 2024-05-14T09:58:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f81167fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c399ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33850/0 (socket says 172.21.15.2:33850) 2024-05-14T09:58:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f8116ffd640 1 -- 172.21.15.2:0/3728434569 learned_addr learned my addr 172.21.15.2:0/3728434569 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f81167fc640 1 --2- 172.21.15.2:0/3728434569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bff11b0 0x55a62c392a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f80fffff640 1 --2- 172.21.15.2:0/2053279605 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81000b28a0 0x7f81000b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f8116ffd640 1 -- 172.21.15.2:0/3728434569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bff11b0 msgr2=0x55a62c392a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f8116ffd640 1 --2- 172.21.15.2:0/3728434569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bff11b0 0x55a62c392a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f8116ffd640 1 -- 172.21.15.2:0/3728434569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 msgr2=0x55a62c399ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f8116ffd640 1 --2- 172.21.15.2:0/3728434569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c399ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f8116ffd640 1 -- 172.21.15.2:0/3728434569 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a62c009210 con 0x55a62bfc82f0 2024-05-14T09:58:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f81177fe640 1 --2- 172.21.15.2:0/2053279605 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81000b28a0 0x7f81000b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f80fffff640 1 -- 172.21.15.2:0/2053279605 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f81100e29f0 con 0x7f8118108a90 2024-05-14T09:58:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.822+0000 7f81167fc640 1 --2- 172.21.15.2:0/3728434569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c399ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:58:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.823+0000 7f81177fe640 1 --2- 172.21.15.2:0/2053279605 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81000b28a0 0x7f81000b4db0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8110020000 tx=0x7f811000ff70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.825+0000 7f8116ffd640 1 --2- 172.21.15.2:0/3728434569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c392fe0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8108009230 tx=0x7f8108031de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a044022ba0c794a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.825+0000 7f80fd7fa640 1 -- 172.21.15.2:0/3728434569 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108035030 con 0x55a62bfc82f0 2024-05-14T09:58:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.825+0000 7f80fd7fa640 1 -- 172.21.15.2:0/3728434569 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8108040070 con 0x55a62bfc82f0 2024-05-14T09:58:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.825+0000 7f80fd7fa640 1 -- 172.21.15.2:0/3728434569 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108003b90 con 0x55a62bfc82f0 2024-05-14T09:58:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.826+0000 7f812465b740 1 -- 172.21.15.2:0/3728434569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 msgr2=0x55a62c392fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.826+0000 7f812465b740 1 --2- 172.21.15.2:0/3728434569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c392fe0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8108009230 tx=0x7f8108031de0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.826+0000 7f812465b740 1 -- 172.21.15.2:0/3728434569 shutdown_connections 2024-05-14T09:58:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.826+0000 7f812465b740 1 --2- 172.21.15.2:0/3728434569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c399ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.826+0000 7f812465b740 1 --2- 172.21.15.2:0/3728434569 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c392fe0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.826+0000 7f812465b740 1 --2- 172.21.15.2:0/3728434569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bff11b0 0x55a62c392a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.826+0000 7f812465b740 1 -- 172.21.15.2:0/3728434569 >> 172.21.15.2:0/3728434569 conn(0x55a62bfe5300 msgr2=0x55a62c38e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.827+0000 7f812465b740 1 -- 172.21.15.2:0/3728434569 shutdown_connections 2024-05-14T09:58:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.827+0000 7f812465b740 1 -- 172.21.15.2:0/3728434569 wait complete. 2024-05-14T09:58:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.827+0000 7f812465b740 1 -- start start 2024-05-14T09:58:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.827+0000 7f812465b740 1 Processor -- start 2024-05-14T09:58:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.827+0000 7f812465b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c39c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.827+0000 7f812465b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c39cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.827+0000 7f812465b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bff11b0 0x55a62c3a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.827+0000 7f812465b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a62c39a8d0 con 0x55a62bfd04d0 2024-05-14T09:58:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.827+0000 7f812465b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a62c2b4370 con 0x55a62bff11b0 2024-05-14T09:58:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.827+0000 7f812465b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55a62c39aa50 con 0x55a62bfc82f0 2024-05-14T09:58:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.827+0000 7f812465b740 10 client.0 _ll_register_callbacks cb 0x55a62c008a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:58:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.828+0000 7f8116ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c39cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.828+0000 7f8116ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c39cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33860/0 (socket says 172.21.15.2:33860) 2024-05-14T09:58:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.828+0000 7f8116ffd640 1 -- 172.21.15.2:0/3276048326 learned_addr learned my addr 172.21.15.2:0/3276048326 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.828+0000 7f81167fc640 1 --2- 172.21.15.2:0/3276048326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c39c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.828+0000 7f8116ffd640 1 -- 172.21.15.2:0/3276048326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 msgr2=0x55a62c39c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.828+0000 7f8116ffd640 1 --2- 172.21.15.2:0/3276048326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c39c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.828+0000 7f8116ffd640 1 -- 172.21.15.2:0/3276048326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bff11b0 msgr2=0x55a62c3a3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.828+0000 7f8116ffd640 1 --2- 172.21.15.2:0/3276048326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bff11b0 0x55a62c3a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.828+0000 7f8116ffd640 1 -- 172.21.15.2:0/3276048326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a62c0d5620 con 0x55a62bfd04d0 2024-05-14T09:58:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.828+0000 7f8116ffd640 1 --2- 172.21.15.2:0/3276048326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c39cb50 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8108006010 tx=0x7f81080317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81329db0bf553696 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.828+0000 7f80faff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:58:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.828+0000 7f80faff5640 1 -- 172.21.15.2:0/3276048326 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108035040 con 0x55a62bfd04d0 2024-05-14T09:58:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.828+0000 7f80faff5640 1 -- 172.21.15.2:0/3276048326 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f8108004250 con 0x55a62bfd04d0 2024-05-14T09:58:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.829+0000 7f812465b740 1 -- 172.21.15.2:0/3276048326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8108040070 con 0x55a62bfd04d0 2024-05-14T09:58:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.829+0000 7f80faff5640 1 -- 172.21.15.2:0/3276048326 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f8108004450 con 0x55a62bfd04d0 2024-05-14T09:58:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.829+0000 7f812465b740 10 client.17559 fetch_fsmap learned FSMap version 67 2024-05-14T09:58:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.829+0000 7f812465b740 1 -- 172.21.15.2:0/3276048326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55a62c3a69e0 con 0x55a62bfd04d0 2024-05-14T09:58:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.829+0000 7f80faff5640 1 -- 172.21.15.2:0/3276048326 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8108040070 con 0x55a62bfd04d0 2024-05-14T09:58:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.829+0000 7f80faff5640 1 -- 172.21.15.2:0/3276048326 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f8108045710 con 0x55a62bfd04d0 2024-05-14T09:58:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.829+0000 7f812465b740 10 client.17559 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:58:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.829+0000 7f812465b740 10 client.17559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:58:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.829+0000 7f812465b740 20 client.17559 populate_metadata read hostname 'smithi002' 2024-05-14T09:58:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.829+0000 7f812465b740 1 --2- 172.21.15.2:0/3276048326 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a62bfafd80 0x55a62c3a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.829+0000 7f812465b740 4 client.17559 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:58:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.829+0000 7f812465b740 1 -- 172.21.15.2:0/3276048326 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a62c3051e0 con 0x55a62bfafd80 2024-05-14T09:58:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.830+0000 7f81167fc640 1 --2- 172.21.15.2:0/3276048326 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a62bfafd80 0x55a62c3a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.830+0000 7f80faff5640 10 client.17559 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.830+0000 7f81167fc640 1 --2- 172.21.15.2:0/3276048326 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a62bfafd80 0x55a62c3a9d40 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde024e415ae6782 server_cookie=a54fbf81180f150c in_seq=0 out_seq=0 2024-05-14T09:58:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.831+0000 7f80faff5640 1 -- 172.21.15.2:0/3276048326 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a62c3051e0 con 0x55a62bfafd80 2024-05-14T09:58:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.831+0000 7f80faff5640 10 client.17559 handle_command_reply: tid=0 2024-05-14T09:58:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.831+0000 7f812465b740 1 client.17559 shutdown 2024-05-14T09:58:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.831+0000 7f812465b740 1 -- 172.21.15.2:0/3276048326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 msgr2=0x55a62c39cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.831+0000 7f812465b740 1 --2- 172.21.15.2:0/3276048326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c39cb50 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8108006010 tx=0x7f81080317d0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.831+0000 7f812465b740 1 -- 172.21.15.2:0/3276048326 shutdown_connections 2024-05-14T09:58:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.831+0000 7f812465b740 1 --2- 172.21.15.2:0/3276048326 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a62bfafd80 0x55a62c3a9d40 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.831+0000 7f812465b740 1 --2- 172.21.15.2:0/3276048326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bff11b0 0x55a62c3a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.831+0000 7f80faff5640 0 client.17559 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.831+0000 7f812465b740 1 --2- 172.21.15.2:0/3276048326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c39cb50 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.831+0000 7f812465b740 1 --2- 172.21.15.2:0/3276048326 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c39c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.831+0000 7f812465b740 1 -- 172.21.15.2:0/3276048326 >> 172.21.15.2:0/3276048326 conn(0x55a62bfe5300 msgr2=0x55a62c391300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.832+0000 7f812465b740 1 -- 172.21.15.2:0/3276048326 shutdown_connections 2024-05-14T09:58:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.832+0000 7f812465b740 1 -- 172.21.15.2:0/3276048326 wait complete. 2024-05-14T09:58:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.832+0000 7f812465b740 20 client.17559 trim_cache size 0 max 16384 2024-05-14T09:58:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.851+0000 7f812465b740 1 Processor -- start 2024-05-14T09:58:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.851+0000 7f812465b740 1 -- start start 2024-05-14T09:58:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.851+0000 7f812465b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c38dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.851+0000 7f812465b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c38e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.851+0000 7f812465b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bff11b0 0x55a62c3a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.851+0000 7f812465b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f81181af420 con 0x55a62bfd04d0 2024-05-14T09:58:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.851+0000 7f812465b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f81181accf0 con 0x55a62bff11b0 2024-05-14T09:58:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.851+0000 7f812465b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f81181aab40 con 0x55a62bfc82f0 2024-05-14T09:58:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.851+0000 7f8116ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c38e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.852+0000 7f8116ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c38e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33868/0 (socket says 172.21.15.2:33868) 2024-05-14T09:58:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.852+0000 7f8116ffd640 1 -- 172.21.15.2:0/3970453909 learned_addr learned my addr 172.21.15.2:0/3970453909 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.852+0000 7f81167fc640 1 --2- 172.21.15.2:0/3970453909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c38dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.852+0000 7f81167fc640 1 --2- 172.21.15.2:0/3970453909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bff11b0 0x55a62c3a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.852+0000 7f8116ffd640 1 -- 172.21.15.2:0/3970453909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 msgr2=0x55a62c38dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.852+0000 7f8116ffd640 1 --2- 172.21.15.2:0/3970453909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c38dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.852+0000 7f8116ffd640 1 -- 172.21.15.2:0/3970453909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bff11b0 msgr2=0x55a62c3a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.852+0000 7f8116ffd640 1 --2- 172.21.15.2:0/3970453909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bff11b0 0x55a62c3a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.852+0000 7f8116ffd640 1 -- 172.21.15.2:0/3970453909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a62c38d980 con 0x55a62bfd04d0 2024-05-14T09:58:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.852+0000 7f8116ffd640 1 --2- 172.21.15.2:0/3970453909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c38e100 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8108045d20 tx=0x7f8108000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97cfd2ad3b19f02 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.852+0000 7f80fcff9640 1 -- 172.21.15.2:0/3970453909 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108035040 con 0x55a62bfd04d0 2024-05-14T09:58:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.852+0000 7f80fcff9640 1 -- 172.21.15.2:0/3970453909 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8108040070 con 0x55a62bfd04d0 2024-05-14T09:58:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.852+0000 7f80fcff9640 1 -- 172.21.15.2:0/3970453909 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81080039b0 con 0x55a62bfd04d0 2024-05-14T09:58:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.853+0000 7f812465b740 1 -- 172.21.15.2:0/3970453909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 msgr2=0x55a62c38e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.853+0000 7f812465b740 1 --2- 172.21.15.2:0/3970453909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c38e100 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8108045d20 tx=0x7f8108000bc0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.853+0000 7f812465b740 1 -- 172.21.15.2:0/3970453909 shutdown_connections 2024-05-14T09:58:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.853+0000 7f812465b740 1 --2- 172.21.15.2:0/3970453909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bff11b0 0x55a62c3a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.853+0000 7f812465b740 1 --2- 172.21.15.2:0/3970453909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bfd04d0 0x55a62c38e100 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.853+0000 7f812465b740 1 --2- 172.21.15.2:0/3970453909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c38dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.853+0000 7f812465b740 1 -- 172.21.15.2:0/3970453909 >> 172.21.15.2:0/3970453909 conn(0x55a62bfe5300 msgr2=0x55a62c396aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.853+0000 7f812465b740 1 -- 172.21.15.2:0/3970453909 shutdown_connections 2024-05-14T09:58:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.853+0000 7f812465b740 1 -- 172.21.15.2:0/3970453909 wait complete. 2024-05-14T09:58:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.854+0000 7f812465b740 1 -- start start 2024-05-14T09:58:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.854+0000 7f812465b740 1 Processor -- start 2024-05-14T09:58:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.854+0000 7f812465b740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c3a1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.854+0000 7f812465b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bfd04d0 0x55a62c3a1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.854+0000 7f812465b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bff11b0 0x55a62c410050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.854+0000 7f812465b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f81181ab1b0 con 0x55a62bff11b0 2024-05-14T09:58:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.854+0000 7f812465b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f81181ac620 con 0x55a62bfd04d0 2024-05-14T09:58:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.854+0000 7f812465b740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f81181ab880 con 0x55a62bfc82f0 2024-05-14T09:58:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.854+0000 7f812465b740 10 client.0 _ll_register_callbacks cb 0x7f810c0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:58:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.854+0000 7f81167fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bff11b0 0x55a62c410050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.854+0000 7f81167fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bff11b0 0x55a62c410050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33874/0 (socket says 172.21.15.2:33874) 2024-05-14T09:58:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.854+0000 7f81167fc640 1 -- 172.21.15.2:0/173493027 learned_addr learned my addr 172.21.15.2:0/173493027 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.854+0000 7f8116ffd640 1 --2- 172.21.15.2:0/173493027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bfd04d0 0x55a62c3a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.855+0000 7f81167fc640 1 -- 172.21.15.2:0/173493027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 msgr2=0x55a62c3a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.855+0000 7f81167fc640 1 --2- 172.21.15.2:0/173493027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c3a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.855+0000 7f81167fc640 1 -- 172.21.15.2:0/173493027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bfd04d0 msgr2=0x55a62c3a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.855+0000 7f81167fc640 1 --2- 172.21.15.2:0/173493027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bfd04d0 0x55a62c3a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.855+0000 7f81167fc640 1 -- 172.21.15.2:0/173493027 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a62c410590 con 0x55a62bff11b0 2024-05-14T09:58:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.855+0000 7f81167fc640 1 --2- 172.21.15.2:0/173493027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bff11b0 0x55a62c410050 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f810c00a010 tx=0x7f810c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1d2ec299601c8b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.855+0000 7f80faff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:58:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.855+0000 7f80faff5640 1 -- 172.21.15.2:0/173493027 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c0208d0 con 0x55a62bff11b0 2024-05-14T09:58:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.856+0000 7f80faff5640 1 -- 172.21.15.2:0/173493027 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f810c020a70 con 0x55a62bff11b0 2024-05-14T09:58:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.856+0000 7f812465b740 1 -- 172.21.15.2:0/173493027 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8108040070 con 0x55a62bff11b0 2024-05-14T09:58:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.856+0000 7f80faff5640 1 -- 172.21.15.2:0/173493027 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f810c033050 con 0x55a62bff11b0 2024-05-14T09:58:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.856+0000 7f812465b740 10 client.17565 fetch_fsmap learned FSMap version 67 2024-05-14T09:58:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.856+0000 7f812465b740 1 -- 172.21.15.2:0/173493027 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55a62c411750 con 0x55a62bff11b0 2024-05-14T09:58:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.856+0000 7f80faff5640 1 -- 172.21.15.2:0/173493027 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8108040070 con 0x55a62bff11b0 2024-05-14T09:58:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.856+0000 7f80faff5640 1 -- 172.21.15.2:0/173493027 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f810c0174d0 con 0x55a62bff11b0 2024-05-14T09:58:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.856+0000 7f812465b740 10 client.17565 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:58:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.856+0000 7f812465b740 10 client.17565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:58:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.856+0000 7f812465b740 20 client.17565 populate_metadata read hostname 'smithi002' 2024-05-14T09:58:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.856+0000 7f812465b740 1 --2- 172.21.15.2:0/173493027 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a62bf851c0 0x55a62c414120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.856+0000 7f812465b740 4 client.17565 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:58:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.856+0000 7f812465b740 1 -- 172.21.15.2:0/173493027 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a62c2ff050 con 0x55a62bf851c0 2024-05-14T09:58:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.857+0000 7f8116ffd640 1 --2- 172.21.15.2:0/173493027 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a62bf851c0 0x55a62c414120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.862+0000 7f80faff5640 10 client.17565 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:30.862+0000 7f8116ffd640 1 --2- 172.21.15.2:0/173493027 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a62bf851c0 0x55a62c414120 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de7f0a4dd3025943 server_cookie=fa42ba997cca4fb6 in_seq=0 out_seq=0 2024-05-14T09:58:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:30 smithi029 ceph-mon[36249]: pgmap v1133: 129 pgs: 129 active+clean; 705 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 15 MiB/s wr, 814 op/s 2024-05-14T09:58:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:30 smithi026 ceph-mon[36056]: pgmap v1133: 129 pgs: 129 active+clean; 705 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 15 MiB/s wr, 814 op/s 2024-05-14T09:58:31.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:30 smithi002 ceph-mon[31557]: pgmap v1133: 129 pgs: 129 active+clean; 705 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 15 MiB/s wr, 814 op/s 2024-05-14T09:58:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.283+0000 7f80faff5640 1 -- 172.21.15.2:0/173493027 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a62c2ff050 con 0x55a62bf851c0 2024-05-14T09:58:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.283+0000 7f80faff5640 10 client.17565 handle_command_reply: tid=0 2024-05-14T09:58:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.283+0000 7f812465b740 1 client.17565 shutdown 2024-05-14T09:58:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.283+0000 7f812465b740 1 -- 172.21.15.2:0/173493027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bff11b0 msgr2=0x55a62c410050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.283+0000 7f812465b740 1 --2- 172.21.15.2:0/173493027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bff11b0 0x55a62c410050 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f810c00a010 tx=0x7f810c000bc0 comp rx=0 tx=0).stop 2024-05-14T09:58:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.283+0000 7f812465b740 1 -- 172.21.15.2:0/173493027 shutdown_connections 2024-05-14T09:58:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.283+0000 7f812465b740 1 --2- 172.21.15.2:0/173493027 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55a62bf851c0 0x55a62c414120 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.283+0000 7f812465b740 1 --2- 172.21.15.2:0/173493027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a62bff11b0 0x55a62c410050 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.283+0000 7f812465b740 1 --2- 172.21.15.2:0/173493027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a62bfd04d0 0x55a62c3a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.283+0000 7f812465b740 1 --2- 172.21.15.2:0/173493027 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55a62bfc82f0 0x55a62c3a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.283+0000 7f812465b740 1 -- 172.21.15.2:0/173493027 >> 172.21.15.2:0/173493027 conn(0x55a62bfe5300 msgr2=0x55a62c3911f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.283+0000 7f80faff5640 0 client.17565 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.283+0000 7f812465b740 1 -- 172.21.15.2:0/173493027 shutdown_connections 2024-05-14T09:58:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.283+0000 7f812465b740 1 -- 172.21.15.2:0/173493027 wait complete. 2024-05-14T09:58:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.283+0000 7f812465b740 20 client.17565 trim_cache size 0 max 16384 2024-05-14T09:58:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:58:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:58:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:58:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:58:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.284+0000 7f811d7a8640 1 -- 172.21.15.2:0/2053279605 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81000b28a0 msgr2=0x7f81000b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.284+0000 7f811d7a8640 1 --2- 172.21.15.2:0/2053279605 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81000b28a0 0x7f81000b4db0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8110020000 tx=0x7f811000ff70 comp rx=0 tx=0).stop 2024-05-14T09:58:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.284+0000 7f811d7a8640 1 -- 172.21.15.2:0/2053279605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8118108a90 msgr2=0x7f81181a1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.284+0000 7f811d7a8640 1 --2- 172.21.15.2:0/2053279605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8118108a90 0x7f81181a1670 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f811000cce0 tx=0x7f811000a620 comp rx=0 tx=0).stop 2024-05-14T09:58:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.284+0000 7f811d7a8640 1 -- 172.21.15.2:0/2053279605 shutdown_connections 2024-05-14T09:58:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.284+0000 7f811d7a8640 1 --2- 172.21.15.2:0/2053279605 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f81000b28a0 0x7f81000b4db0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.284+0000 7f811d7a8640 1 --2- 172.21.15.2:0/2053279605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8118108a90 0x7f81181a1670 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.284+0000 7f811d7a8640 1 --2- 172.21.15.2:0/2053279605 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8118103d90 0x7f811819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.284+0000 7f811d7a8640 1 --2- 172.21.15.2:0/2053279605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8118102b50 0x7f811806b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.284+0000 7f811d7a8640 1 -- 172.21.15.2:0/2053279605 >> 172.21.15.2:0/2053279605 conn(0x7f81180fdd40 msgr2=0x7f8118104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.284+0000 7f811d7a8640 1 -- 172.21.15.2:0/2053279605 shutdown_connections 2024-05-14T09:58:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:31.284+0000 7f811d7a8640 1 -- 172.21.15.2:0/2053279605 wait complete. 2024-05-14T09:58:31.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 292631 25.56 MB/sec execute 477 sec latency 315.541 ms 2024-05-14T09:58:32.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 292991 25.54 MB/sec execute 478 sec latency 300.228 ms 2024-05-14T09:58:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:32 smithi029 ceph-mon[36249]: pgmap v1134: 129 pgs: 129 active+clean; 677 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 14 MiB/s wr, 744 op/s 2024-05-14T09:58:32.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:32 smithi026 ceph-mon[36056]: pgmap v1134: 129 pgs: 129 active+clean; 677 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 14 MiB/s wr, 744 op/s 2024-05-14T09:58:33.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:32 smithi002 ceph-mon[31557]: pgmap v1134: 129 pgs: 129 active+clean; 677 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 14 MiB/s wr, 744 op/s 2024-05-14T09:58:33.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 293549 25.55 MB/sec execute 479 sec latency 75.680 ms 2024-05-14T09:58:34.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 294230 25.57 MB/sec execute 480 sec latency 71.050 ms 2024-05-14T09:58:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:34 smithi029 ceph-mon[36249]: pgmap v1135: 129 pgs: 129 active+clean; 666 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 693 op/s 2024-05-14T09:58:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:34 smithi026 ceph-mon[36056]: pgmap v1135: 129 pgs: 129 active+clean; 666 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 693 op/s 2024-05-14T09:58:35.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:34 smithi002 ceph-mon[31557]: pgmap v1135: 129 pgs: 129 active+clean; 666 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 693 op/s 2024-05-14T09:58:35.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 294817 25.59 MB/sec execute 481 sec latency 169.565 ms 2024-05-14T09:58:36.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 295456 25.62 MB/sec execute 482 sec latency 118.426 ms 2024-05-14T09:58:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:36 smithi002 ceph-mon[31557]: pgmap v1136: 129 pgs: 129 active+clean; 672 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.8 MiB/s wr, 611 op/s 2024-05-14T09:58:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:36 smithi029 ceph-mon[36249]: pgmap v1136: 129 pgs: 129 active+clean; 672 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.8 MiB/s wr, 611 op/s 2024-05-14T09:58:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:36 smithi026 ceph-mon[36056]: pgmap v1136: 129 pgs: 129 active+clean; 672 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.8 MiB/s wr, 611 op/s 2024-05-14T09:58:37.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 296031 25.62 MB/sec execute 483 sec latency 95.370 ms 2024-05-14T09:58:38.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 296512 25.63 MB/sec execute 484 sec latency 130.116 ms 2024-05-14T09:58:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:38 smithi002 ceph-mon[31557]: pgmap v1137: 129 pgs: 129 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.2 MiB/s wr, 623 op/s 2024-05-14T09:58:38.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:38 smithi029 ceph-mon[36249]: pgmap v1137: 129 pgs: 129 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.2 MiB/s wr, 623 op/s 2024-05-14T09:58:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:38 smithi026 ceph-mon[36056]: pgmap v1137: 129 pgs: 129 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.2 MiB/s wr, 623 op/s 2024-05-14T09:58:39.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 297089 25.62 MB/sec execute 485 sec latency 152.516 ms 2024-05-14T09:58:40.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 297719 25.64 MB/sec execute 486 sec latency 88.258 ms 2024-05-14T09:58:40.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:40 smithi002 ceph-mon[31557]: pgmap v1138: 129 pgs: 129 active+clean; 682 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.8 MiB/s wr, 563 op/s 2024-05-14T09:58:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:40 smithi029 ceph-mon[36249]: pgmap v1138: 129 pgs: 129 active+clean; 682 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.8 MiB/s wr, 563 op/s 2024-05-14T09:58:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:40 smithi026 ceph-mon[36056]: pgmap v1138: 129 pgs: 129 active+clean; 682 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.8 MiB/s wr, 563 op/s 2024-05-14T09:58:41.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 298365 25.67 MB/sec execute 487 sec latency 83.645 ms 2024-05-14T09:58:42.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 298976 25.68 MB/sec execute 488 sec latency 121.655 ms 2024-05-14T09:58:42.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:42 smithi002 ceph-mon[31557]: pgmap v1139: 129 pgs: 129 active+clean; 689 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.1 MiB/s wr, 486 op/s 2024-05-14T09:58:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:42 smithi029 ceph-mon[36249]: pgmap v1139: 129 pgs: 129 active+clean; 689 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.1 MiB/s wr, 486 op/s 2024-05-14T09:58:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:42 smithi026 ceph-mon[36056]: pgmap v1139: 129 pgs: 129 active+clean; 689 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.1 MiB/s wr, 486 op/s 2024-05-14T09:58:43.625 INFO:tasks.workunit.client.0.smithi002.stdout: 10 299462 25.67 MB/sec execute 489 sec latency 90.284 ms 2024-05-14T09:58:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.498+0000 7fc103534640 1 -- 172.21.15.2:0/2301043017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc1101a0 msgr2=0x7fc0fc1105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.498+0000 7fc103534640 1 --2- 172.21.15.2:0/2301043017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc1101a0 0x7fc0fc1105b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009900 tx=0x7fc0f802a420 comp rx=0 tx=0).stop 2024-05-14T09:58:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.501+0000 7fc103534640 1 -- 172.21.15.2:0/2301043017 shutdown_connections 2024-05-14T09:58:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.501+0000 7fc103534640 1 --2- 172.21.15.2:0/2301043017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc1101a0 0x7fc0fc1105b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.501+0000 7fc103534640 1 --2- 172.21.15.2:0/2301043017 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0fc10b480 0x7fc0fc10b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.501+0000 7fc103534640 1 --2- 172.21.15.2:0/2301043017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0fc10a550 0x7fc0fc10af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.501+0000 7fc103534640 1 -- 172.21.15.2:0/2301043017 >> 172.21.15.2:0/2301043017 conn(0x7fc0fc070640 msgr2=0x7fc0fc072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.507+0000 7fc103534640 1 -- 172.21.15.2:0/2301043017 shutdown_connections 2024-05-14T09:58:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.510+0000 7fc103534640 1 -- 172.21.15.2:0/2301043017 wait complete. 2024-05-14T09:58:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.511+0000 7fc103534640 1 Processor -- start 2024-05-14T09:58:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.511+0000 7fc103534640 1 -- start start 2024-05-14T09:58:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.511+0000 7fc103534640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0fc10a550 0x7fc0fc1170e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.511+0000 7fc103534640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0fc10b480 0x7fc0fc117620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.511+0000 7fc103534640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc1101a0 0x7fc0fc118580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.511+0000 7fc103534640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc11e660 con 0x7fc0fc10b480 2024-05-14T09:58:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.511+0000 7fc103534640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc11e4e0 con 0x7fc0fc1101a0 2024-05-14T09:58:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.511+0000 7fc103534640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc11e7e0 con 0x7fc0fc10a550 2024-05-14T09:58:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.515+0000 7fc101ad1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc1101a0 0x7fc0fc118580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.515+0000 7fc101ad1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc1101a0 0x7fc0fc118580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:46298/0 (socket says 172.21.15.2:46298) 2024-05-14T09:58:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.515+0000 7fc101ad1640 1 -- 172.21.15.2:0/4103490114 learned_addr learned my addr 172.21.15.2:0/4103490114 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.516+0000 7fc1012d0640 1 --2- 172.21.15.2:0/4103490114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0fc10a550 0x7fc0fc1170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.516+0000 7fc100acf640 1 --2- 172.21.15.2:0/4103490114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0fc10b480 0x7fc0fc117620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.519+0000 7fc101ad1640 1 -- 172.21.15.2:0/4103490114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0fc10a550 msgr2=0x7fc0fc1170e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.519+0000 7fc101ad1640 1 --2- 172.21.15.2:0/4103490114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0fc10a550 0x7fc0fc1170e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.519+0000 7fc101ad1640 1 -- 172.21.15.2:0/4103490114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0fc10b480 msgr2=0x7fc0fc117620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.519+0000 7fc101ad1640 1 --2- 172.21.15.2:0/4103490114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0fc10b480 0x7fc0fc117620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.519+0000 7fc101ad1640 1 -- 172.21.15.2:0/4103490114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0fc118bf0 con 0x7fc0fc1101a0 2024-05-14T09:58:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.520+0000 7fc101ad1640 1 --2- 172.21.15.2:0/4103490114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc1101a0 0x7fc0fc118580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8003cd0 tx=0x7fc0f8003d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.525+0000 7fc0f27fc640 1 -- 172.21.15.2:0/4103490114 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f803f070 con 0x7fc0fc1101a0 2024-05-14T09:58:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.525+0000 7fc0f27fc640 1 -- 172.21.15.2:0/4103490114 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f8031c50 con 0x7fc0fc1101a0 2024-05-14T09:58:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.525+0000 7fc0f27fc640 1 -- 172.21.15.2:0/4103490114 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8035030 con 0x7fc0fc1101a0 2024-05-14T09:58:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.526+0000 7fc103534640 1 -- 172.21.15.2:0/4103490114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0fc118e80 con 0x7fc0fc1101a0 2024-05-14T09:58:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.531+0000 7fc103534640 1 -- 172.21.15.2:0/4103490114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0fc1193a0 con 0x7fc0fc1101a0 2024-05-14T09:58:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.543+0000 7fc10a3e1740 1 Processor -- start 2024-05-14T09:58:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.552+0000 7fc10a3e1740 1 -- start start 2024-05-14T09:58:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.552+0000 7fc10a3e1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877664b1b0 0x5587769eca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.552+0000 7fc10a3e1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5587766222f0 0x5587769ed000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.552+0000 7fc10a3e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55877662a4d0 0x5587769f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.552+0000 7fc10a3e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5587769515d0 con 0x55877662a4d0 2024-05-14T09:58:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.552+0000 7fc10a3e1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55877690a890 con 0x55877664b1b0 2024-05-14T09:58:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.552+0000 7fc10a3e1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55877694d590 con 0x5587766222f0 2024-05-14T09:58:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.555+0000 7fc1012d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5587766222f0 0x5587769ed000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.556+0000 7fc1012d0640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5587766222f0 0x5587769ed000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:43562/0 (socket says 172.21.15.2:43562) 2024-05-14T09:58:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.556+0000 7fc1012d0640 1 -- 172.21.15.2:0/4119162760 learned_addr learned my addr 172.21.15.2:0/4119162760 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.560+0000 7fc0f27fc640 1 -- 172.21.15.2:0/4103490114 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc0f8042070 con 0x7fc0fc1101a0 2024-05-14T09:58:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.560+0000 7fc100acf640 1 --2- 172.21.15.2:0/4119162760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877664b1b0 0x5587769eca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.560+0000 7fc100acf640 1 --2- 172.21.15.2:0/4119162760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55877662a4d0 0x5587769f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.561+0000 7fc100acf640 1 -- 172.21.15.2:0/4119162760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5587766222f0 msgr2=0x5587769ed000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.561+0000 7fc100acf640 1 --2- 172.21.15.2:0/4119162760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5587766222f0 0x5587769ed000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.561+0000 7fc100acf640 1 -- 172.21.15.2:0/4119162760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55877662a4d0 msgr2=0x5587769f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.561+0000 7fc100acf640 1 --2- 172.21.15.2:0/4119162760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55877662a4d0 0x5587769f3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.562+0000 7fc100acf640 1 -- 172.21.15.2:0/4119162760 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558776663210 con 0x55877664b1b0 2024-05-14T09:58:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.562+0000 7fc0f27fc640 1 --2- 172.21.15.2:0/4103490114 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0e80b2ac0 0x7fc0e80b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.562+0000 7fc100acf640 1 --2- 172.21.15.2:0/4119162760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877664b1b0 0x5587769eca80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc118180 tx=0x7fc0f400a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eef9aa12d636a9c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.562+0000 7fc0f27fc640 1 -- 172.21.15.2:0/4103490114 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fc0f8103290 con 0x7fc0fc1101a0 2024-05-14T09:58:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.564+0000 7fc0e77fe640 1 -- 172.21.15.2:0/4119162760 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40132f0 con 0x55877664b1b0 2024-05-14T09:58:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.564+0000 7fc0e77fe640 1 -- 172.21.15.2:0/4119162760 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f400e7d0 con 0x55877664b1b0 2024-05-14T09:58:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.564+0000 7fc0e77fe640 1 -- 172.21.15.2:0/4119162760 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4005490 con 0x55877664b1b0 2024-05-14T09:58:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.566+0000 7fc10a3e1740 1 -- 172.21.15.2:0/4119162760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877664b1b0 msgr2=0x5587769eca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.566+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/4119162760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877664b1b0 0x5587769eca80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc118180 tx=0x7fc0f400a640 comp rx=0 tx=0).stop 2024-05-14T09:58:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.568+0000 7fc1012d0640 1 --2- 172.21.15.2:0/4103490114 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0e80b2ac0 0x7fc0e80b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.570+0000 7fc10a3e1740 1 -- 172.21.15.2:0/4119162760 shutdown_connections 2024-05-14T09:58:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.570+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/4119162760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55877662a4d0 0x5587769f3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.570+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/4119162760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5587766222f0 0x5587769ed000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.570+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/4119162760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877664b1b0 0x5587769eca80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.570+0000 7fc10a3e1740 1 -- 172.21.15.2:0/4119162760 >> 172.21.15.2:0/4119162760 conn(0x55877663f300 msgr2=0x5587769e8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.577+0000 7fc1012d0640 1 --2- 172.21.15.2:0/4103490114 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0e80b2ac0 0x7fc0e80b4fd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x5587769e91a0 tx=0x7fc0ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.577+0000 7fc10a3e1740 1 -- 172.21.15.2:0/4119162760 shutdown_connections 2024-05-14T09:58:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.579+0000 7fc10a3e1740 1 -- 172.21.15.2:0/4119162760 wait complete. 2024-05-14T09:58:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.580+0000 7fc10a3e1740 1 -- start start 2024-05-14T09:58:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.580+0000 7fc10a3e1740 1 Processor -- start 2024-05-14T09:58:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.581+0000 7fc10a3e1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5587766222f0 0x5587769f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.581+0000 7fc10a3e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55877662a4d0 0x5587769f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.581+0000 7fc10a3e1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877664b1b0 0x5587769fdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.581+0000 7fc10a3e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5587769f4950 con 0x55877662a4d0 2024-05-14T09:58:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.581+0000 7fc10a3e1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55877690e370 con 0x55877664b1b0 2024-05-14T09:58:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.581+0000 7fc101ad1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877664b1b0 0x5587769fdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.581+0000 7fc101ad1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877664b1b0 0x5587769fdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:46316/0 (socket says 172.21.15.2:46316) 2024-05-14T09:58:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.582+0000 7fc101ad1640 1 -- 172.21.15.2:0/759988397 learned_addr learned my addr 172.21.15.2:0/759988397 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.582+0000 7fc10a3e1740 1 -- 172.21.15.2:0/759988397 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5587769f4ad0 con 0x5587766222f0 2024-05-14T09:58:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.582+0000 7fc10a3e1740 10 client.0 _ll_register_callbacks cb 0x558776662a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:58:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.583+0000 7fc1012d0640 1 --2- 172.21.15.2:0/759988397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5587766222f0 0x5587769f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.584+0000 7fc100acf640 1 --2- 172.21.15.2:0/759988397 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55877662a4d0 0x5587769f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.587+0000 7fc1012d0640 1 -- 172.21.15.2:0/759988397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877664b1b0 msgr2=0x5587769fdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.587+0000 7fc1012d0640 1 --2- 172.21.15.2:0/759988397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877664b1b0 0x5587769fdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.587+0000 7fc1012d0640 1 -- 172.21.15.2:0/759988397 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55877662a4d0 msgr2=0x5587769f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.587+0000 7fc1012d0640 1 --2- 172.21.15.2:0/759988397 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55877662a4d0 0x5587769f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.587+0000 7fc1012d0640 1 -- 172.21.15.2:0/759988397 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55877672f620 con 0x5587766222f0 2024-05-14T09:58:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.587+0000 7fc1012d0640 1 --2- 172.21.15.2:0/759988397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5587766222f0 0x5587769f67e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec0156b0 tx=0x7fc0ec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b44e3b20271488 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.587+0000 7fc101ad1640 1 --2- 172.21.15.2:0/759988397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877664b1b0 0x5587769fdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:58:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.587+0000 7fc0e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:58:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.587+0000 7fc0e4ff9640 1 -- 172.21.15.2:0/759988397 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec023070 con 0x5587766222f0 2024-05-14T09:58:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.588+0000 7fc0e4ff9640 1 -- 172.21.15.2:0/759988397 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fc0ec0139b0 con 0x5587766222f0 2024-05-14T09:58:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.588+0000 7fc10a3e1740 1 -- 172.21.15.2:0/759988397 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0f400e7d0 con 0x5587766222f0 2024-05-14T09:58:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.590+0000 7fc0e4ff9640 1 -- 172.21.15.2:0/759988397 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fc0ec02d050 con 0x5587766222f0 2024-05-14T09:58:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.593+0000 7fc10a3e1740 10 client.27169 fetch_fsmap learned FSMap version 67 2024-05-14T09:58:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.593+0000 7fc10a3e1740 1 -- 172.21.15.2:0/759988397 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x558776a00c80 con 0x5587766222f0 2024-05-14T09:58:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.599+0000 7fc0e4ff9640 1 -- 172.21.15.2:0/759988397 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f400e7d0 con 0x5587766222f0 2024-05-14T09:58:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.599+0000 7fc0e4ff9640 1 -- 172.21.15.2:0/759988397 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fc0ec01e070 con 0x5587766222f0 2024-05-14T09:58:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.602+0000 7fc10a3e1740 10 client.27169 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:58:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.602+0000 7fc10a3e1740 10 client.27169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:58:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.602+0000 7fc10a3e1740 20 client.27169 populate_metadata read hostname 'smithi002' 2024-05-14T09:58:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.602+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/759988397 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558776609d80 0x558776a043f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.602+0000 7fc10a3e1740 4 client.27169 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:58:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.602+0000 7fc10a3e1740 1 -- 172.21.15.2:0/759988397 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55877695f1e0 con 0x558776609d80 2024-05-14T09:58:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.602+0000 7fc100acf640 1 --2- 172.21.15.2:0/759988397 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558776609d80 0x558776a043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.613+0000 7fc100acf640 1 --2- 172.21.15.2:0/759988397 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558776609d80 0x558776a043f0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=accc297eec42ba26 server_cookie=3cb52e16c2f5d938 in_seq=0 out_seq=0 2024-05-14T09:58:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.615+0000 7fc0e4ff9640 10 client.27169 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.617+0000 7fc0e4ff9640 1 -- 172.21.15.2:0/759988397 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55877695f1e0 con 0x558776609d80 2024-05-14T09:58:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.617+0000 7fc0e4ff9640 10 client.27169 handle_command_reply: tid=0 2024-05-14T09:58:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.618+0000 7fc10a3e1740 1 client.27169 shutdown 2024-05-14T09:58:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.625+0000 7fc10a3e1740 1 -- 172.21.15.2:0/759988397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5587766222f0 msgr2=0x5587769f67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.625+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/759988397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5587766222f0 0x5587769f67e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec0156b0 tx=0x7fc0ec0046c0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.628+0000 7fc10a3e1740 1 -- 172.21.15.2:0/759988397 shutdown_connections 2024-05-14T09:58:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.628+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/759988397 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x558776609d80 0x558776a043f0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.628+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/759988397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877664b1b0 0x5587769fdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.628+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/759988397 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55877662a4d0 0x5587769f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.628+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/759988397 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5587766222f0 0x5587769f67e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.628+0000 7fc10a3e1740 1 -- 172.21.15.2:0/759988397 >> 172.21.15.2:0/759988397 conn(0x55877663f300 msgr2=0x5587769e7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:44.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.628+0000 7fc0e4ff9640 0 client.27169 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:44.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 299706 25.66 MB/sec execute 490 sec latency 691.813 ms 2024-05-14T09:58:44.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.631+0000 7fc10a3e1740 1 -- 172.21.15.2:0/759988397 shutdown_connections 2024-05-14T09:58:44.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.632+0000 7fc10a3e1740 1 -- 172.21.15.2:0/759988397 wait complete. 2024-05-14T09:58:44.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.633+0000 7fc10a3e1740 20 client.27169 trim_cache size 0 max 16384 2024-05-14T09:58:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.674+0000 7fc10a3e1740 1 Processor -- start 2024-05-14T09:58:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.678+0000 7fc10a3e1740 1 -- start start 2024-05-14T09:58:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.678+0000 7fc10a3e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587766222f0 0x5587769e84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.678+0000 7fc10a3e1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877662a4d0 0x5587769fdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.678+0000 7fc10a3e1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55877664b1b0 0x5587769fe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.678+0000 7fc10a3e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc1c6cd0 con 0x5587766222f0 2024-05-14T09:58:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.678+0000 7fc10a3e1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc1c45a0 con 0x55877662a4d0 2024-05-14T09:58:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.678+0000 7fc10a3e1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc1c23f0 con 0x55877664b1b0 2024-05-14T09:58:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.680+0000 7fc101ad1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55877664b1b0 0x5587769fe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.680+0000 7fc101ad1640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55877664b1b0 0x5587769fe150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:43590/0 (socket says 172.21.15.2:43590) 2024-05-14T09:58:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.680+0000 7fc101ad1640 1 -- 172.21.15.2:0/1673370640 learned_addr learned my addr 172.21.15.2:0/1673370640 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.680+0000 7fc100acf640 1 --2- 172.21.15.2:0/1673370640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877662a4d0 0x5587769fdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.682+0000 7fc1012d0640 1 --2- 172.21.15.2:0/1673370640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587766222f0 0x5587769e84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.683+0000 7fc100acf640 1 -- 172.21.15.2:0/1673370640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55877664b1b0 msgr2=0x5587769fe150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.683+0000 7fc100acf640 1 --2- 172.21.15.2:0/1673370640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55877664b1b0 0x5587769fe150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.683+0000 7fc100acf640 1 -- 172.21.15.2:0/1673370640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587766222f0 msgr2=0x5587769e84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.683+0000 7fc100acf640 1 --2- 172.21.15.2:0/1673370640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587766222f0 0x5587769e84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.683+0000 7fc100acf640 1 -- 172.21.15.2:0/1673370640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587769f3fb0 con 0x55877662a4d0 2024-05-14T09:58:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.687+0000 7fc100acf640 1 --2- 172.21.15.2:0/1673370640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877662a4d0 0x5587769fdc10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5587769f7850 tx=0x7fc0f400a3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc1e65832ebed33 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.697+0000 7fc0e6ffd640 1 -- 172.21.15.2:0/1673370640 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f402d030 con 0x55877662a4d0 2024-05-14T09:58:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.697+0000 7fc0e6ffd640 1 -- 172.21.15.2:0/1673370640 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f4013470 con 0x55877662a4d0 2024-05-14T09:58:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.697+0000 7fc0e6ffd640 1 -- 172.21.15.2:0/1673370640 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f400ee80 con 0x55877662a4d0 2024-05-14T09:58:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.699+0000 7fc10a3e1740 1 -- 172.21.15.2:0/1673370640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877662a4d0 msgr2=0x5587769fdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.699+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/1673370640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877662a4d0 0x5587769fdc10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5587769f7850 tx=0x7fc0f400a3b0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.700+0000 7fc10a3e1740 1 -- 172.21.15.2:0/1673370640 shutdown_connections 2024-05-14T09:58:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.700+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/1673370640 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55877664b1b0 0x5587769fe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.700+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/1673370640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55877662a4d0 0x5587769fdc10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.700+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/1673370640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587766222f0 0x5587769e84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.700+0000 7fc10a3e1740 1 -- 172.21.15.2:0/1673370640 >> 172.21.15.2:0/1673370640 conn(0x55877663f300 msgr2=0x5587769e7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.704+0000 7fc10a3e1740 1 -- 172.21.15.2:0/1673370640 shutdown_connections 2024-05-14T09:58:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.704+0000 7fc10a3e1740 1 -- 172.21.15.2:0/1673370640 wait complete. 2024-05-14T09:58:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.704+0000 7fc10a3e1740 1 -- start start 2024-05-14T09:58:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.704+0000 7fc10a3e1740 1 Processor -- start 2024-05-14T09:58:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.707+0000 7fc10a3e1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587766222f0 0x5587769f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.707+0000 7fc10a3e1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55877662a4d0 0x5587769fe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.707+0000 7fc10a3e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55877664b1b0 0x5587769f9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.707+0000 7fc10a3e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc1c2a60 con 0x55877664b1b0 2024-05-14T09:58:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.707+0000 7fc10a3e1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc1c3ed0 con 0x5587766222f0 2024-05-14T09:58:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.707+0000 7fc10a3e1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc1c3130 con 0x55877662a4d0 2024-05-14T09:58:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.707+0000 7fc10a3e1740 10 client.0 _ll_register_callbacks cb 0x7fc0ec00ff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:58:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.708+0000 7fc100acf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55877662a4d0 0x5587769fe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.708+0000 7fc100acf640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55877662a4d0 0x5587769fe450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:43592/0 (socket says 172.21.15.2:43592) 2024-05-14T09:58:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.708+0000 7fc100acf640 1 -- 172.21.15.2:0/118706509 learned_addr learned my addr 172.21.15.2:0/118706509 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:44.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.709+0000 7fc100acf640 1 -- 172.21.15.2:0/118706509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587766222f0 msgr2=0x5587769f8fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:58:44.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.709+0000 7fc100acf640 1 --2- 172.21.15.2:0/118706509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587766222f0 0x5587769f8fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.709+0000 7fc100acf640 1 -- 172.21.15.2:0/118706509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55877664b1b0 msgr2=0x5587769f9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:44.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.709+0000 7fc100acf640 1 --2- 172.21.15.2:0/118706509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55877664b1b0 0x5587769f9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:44.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.709+0000 7fc100acf640 1 -- 172.21.15.2:0/118706509 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558776a6a810 con 0x55877662a4d0 2024-05-14T09:58:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.710+0000 7fc100acf640 1 --2- 172.21.15.2:0/118706509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55877662a4d0 0x5587769fe450 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400ad80 tx=0x7fc0f4005a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abf98fa4eaa7b24a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.710+0000 7fc0e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:58:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.710+0000 7fc0e4ff9640 1 -- 172.21.15.2:0/118706509 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f402d070 con 0x55877662a4d0 2024-05-14T09:58:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.710+0000 7fc10a3e1740 1 -- 172.21.15.2:0/118706509 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0f4013470 con 0x55877662a4d0 2024-05-14T09:58:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.710+0000 7fc0e4ff9640 1 -- 172.21.15.2:0/118706509 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fc0f4003970 con 0x55877662a4d0 2024-05-14T09:58:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.710+0000 7fc0e4ff9640 1 -- 172.21.15.2:0/118706509 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fc0f4003b90 con 0x55877662a4d0 2024-05-14T09:58:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.713+0000 7fc10a3e1740 10 client.27175 fetch_fsmap learned FSMap version 67 2024-05-14T09:58:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.713+0000 7fc10a3e1740 1 -- 172.21.15.2:0/118706509 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x558776a6baa0 con 0x55877662a4d0 2024-05-14T09:58:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.714+0000 7fc0e4ff9640 1 -- 172.21.15.2:0/118706509 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f4013470 con 0x55877662a4d0 2024-05-14T09:58:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.714+0000 7fc0e4ff9640 1 -- 172.21.15.2:0/118706509 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fc0f4027070 con 0x55877662a4d0 2024-05-14T09:58:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.714+0000 7fc10a3e1740 10 client.27175 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:58:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.714+0000 7fc10a3e1740 10 client.27175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:58:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.714+0000 7fc10a3e1740 20 client.27175 populate_metadata read hostname 'smithi002' 2024-05-14T09:58:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.714+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/118706509 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5587765df1c0 0x558776a6e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.714+0000 7fc10a3e1740 4 client.27175 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:58:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.714+0000 7fc10a3e1740 1 -- 172.21.15.2:0/118706509 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558776959050 con 0x5587765df1c0 2024-05-14T09:58:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.719+0000 7fc1012d0640 1 --2- 172.21.15.2:0/118706509 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5587765df1c0 0x558776a6e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.722+0000 7fc1012d0640 1 --2- 172.21.15.2:0/118706509 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5587765df1c0 0x558776a6e360 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a17cc53906ffe050 server_cookie=119331ebe71885e6 in_seq=0 out_seq=0 2024-05-14T09:58:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:44.722+0000 7fc0e4ff9640 10 client.27175 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:44.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:44 smithi002 ceph-mon[31557]: pgmap v1140: 129 pgs: 129 active+clean; 694 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.3 MiB/s wr, 463 op/s 2024-05-14T09:58:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:44 smithi029 ceph-mon[36249]: pgmap v1140: 129 pgs: 129 active+clean; 694 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.3 MiB/s wr, 463 op/s 2024-05-14T09:58:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:44 smithi026 ceph-mon[36056]: pgmap v1140: 129 pgs: 129 active+clean; 694 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.3 MiB/s wr, 463 op/s 2024-05-14T09:58:45.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 299718 25.61 MB/sec execute 491 sec latency 1657.337 ms 2024-05-14T09:58:46.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 299836 25.57 MB/sec execute 492 sec latency 2539.526 ms 2024-05-14T09:58:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.815+0000 7fc0e4ff9640 1 -- 172.21.15.2:0/118706509 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558776959050 con 0x5587765df1c0 2024-05-14T09:58:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.815+0000 7fc0e4ff9640 10 client.27175 handle_command_reply: tid=0 2024-05-14T09:58:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.815+0000 7fc10a3e1740 1 client.27175 shutdown 2024-05-14T09:58:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.815+0000 7fc10a3e1740 1 -- 172.21.15.2:0/118706509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55877662a4d0 msgr2=0x5587769fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.815+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/118706509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55877662a4d0 0x5587769fe450 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400ad80 tx=0x7fc0f4005a10 comp rx=0 tx=0).stop 2024-05-14T09:58:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.815+0000 7fc10a3e1740 1 -- 172.21.15.2:0/118706509 shutdown_connections 2024-05-14T09:58:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.815+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/118706509 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5587765df1c0 0x558776a6e360 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.815+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/118706509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55877664b1b0 0x5587769f9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.815+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/118706509 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55877662a4d0 0x5587769fe450 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.815+0000 7fc10a3e1740 1 --2- 172.21.15.2:0/118706509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587766222f0 0x5587769f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.815+0000 7fc10a3e1740 1 -- 172.21.15.2:0/118706509 >> 172.21.15.2:0/118706509 conn(0x55877663f300 msgr2=0x5587769f4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.815+0000 7fc0e4ff9640 0 client.27175 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.815+0000 7fc10a3e1740 1 -- 172.21.15.2:0/118706509 shutdown_connections 2024-05-14T09:58:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.815+0000 7fc10a3e1740 1 -- 172.21.15.2:0/118706509 wait complete. 2024-05-14T09:58:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.816+0000 7fc10a3e1740 20 client.27175 trim_cache size 0 max 16384 2024-05-14T09:58:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:58:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:58:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:58:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:58:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.816+0000 7fc103534640 1 -- 172.21.15.2:0/4103490114 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0e80b2ac0 msgr2=0x7fc0e80b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.816+0000 7fc103534640 1 --2- 172.21.15.2:0/4103490114 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0e80b2ac0 0x7fc0e80b4fd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x5587769e91a0 tx=0x7fc0ec008040 comp rx=0 tx=0).stop 2024-05-14T09:58:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.816+0000 7fc103534640 1 -- 172.21.15.2:0/4103490114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc1101a0 msgr2=0x7fc0fc118580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.816+0000 7fc103534640 1 --2- 172.21.15.2:0/4103490114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc1101a0 0x7fc0fc118580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8003cd0 tx=0x7fc0f8003d00 comp rx=0 tx=0).stop 2024-05-14T09:58:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.817+0000 7fc103534640 1 -- 172.21.15.2:0/4103490114 shutdown_connections 2024-05-14T09:58:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.817+0000 7fc103534640 1 --2- 172.21.15.2:0/4103490114 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0e80b2ac0 0x7fc0e80b4fd0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.817+0000 7fc103534640 1 --2- 172.21.15.2:0/4103490114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc1101a0 0x7fc0fc118580 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.817+0000 7fc103534640 1 --2- 172.21.15.2:0/4103490114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0fc10b480 0x7fc0fc117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.817+0000 7fc103534640 1 --2- 172.21.15.2:0/4103490114 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc0fc10a550 0x7fc0fc1170e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.817+0000 7fc103534640 1 -- 172.21.15.2:0/4103490114 >> 172.21.15.2:0/4103490114 conn(0x7fc0fc070640 msgr2=0x7fc0fc10c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.817+0000 7fc103534640 1 -- 172.21.15.2:0/4103490114 shutdown_connections 2024-05-14T09:58:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:46.817+0000 7fc103534640 1 -- 172.21.15.2:0/4103490114 wait complete. 2024-05-14T09:58:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:46 smithi002 ceph-mon[31557]: pgmap v1141: 129 pgs: 129 active+clean; 701 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.6 MiB/s wr, 396 op/s 2024-05-14T09:58:46.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:46 smithi029 ceph-mon[36249]: pgmap v1141: 129 pgs: 129 active+clean; 701 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.6 MiB/s wr, 396 op/s 2024-05-14T09:58:46.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:46 smithi026 ceph-mon[36056]: pgmap v1141: 129 pgs: 129 active+clean; 701 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.6 MiB/s wr, 396 op/s 2024-05-14T09:58:47.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 299997 25.54 MB/sec execute 493 sec latency 690.219 ms 2024-05-14T09:58:48.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 300247 25.51 MB/sec execute 494 sec latency 218.846 ms 2024-05-14T09:58:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:48 smithi002 ceph-mon[31557]: pgmap v1142: 129 pgs: 129 active+clean; 741 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 487 op/s 2024-05-14T09:58:48.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:48 smithi029 ceph-mon[36249]: pgmap v1142: 129 pgs: 129 active+clean; 741 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 487 op/s 2024-05-14T09:58:48.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:48 smithi026 ceph-mon[36056]: pgmap v1142: 129 pgs: 129 active+clean; 741 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 487 op/s 2024-05-14T09:58:49.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 300401 25.48 MB/sec execute 495 sec latency 228.332 ms 2024-05-14T09:58:50.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 300761 25.46 MB/sec execute 496 sec latency 134.176 ms 2024-05-14T09:58:50.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:50 smithi002 ceph-mon[31557]: pgmap v1143: 129 pgs: 129 active+clean; 761 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.8 MiB/s wr, 555 op/s 2024-05-14T09:58:50.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:50 smithi029 ceph-mon[36249]: pgmap v1143: 129 pgs: 129 active+clean; 761 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.8 MiB/s wr, 555 op/s 2024-05-14T09:58:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:50 smithi026 ceph-mon[36056]: pgmap v1143: 129 pgs: 129 active+clean; 761 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.8 MiB/s wr, 555 op/s 2024-05-14T09:58:51.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 301334 25.46 MB/sec execute 497 sec latency 88.504 ms 2024-05-14T09:58:52.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 301855 25.48 MB/sec execute 498 sec latency 88.859 ms 2024-05-14T09:58:52.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:52 smithi029 ceph-mon[36249]: pgmap v1144: 129 pgs: 129 active+clean; 724 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 10 MiB/s wr, 625 op/s 2024-05-14T09:58:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:52 smithi026 ceph-mon[36056]: pgmap v1144: 129 pgs: 129 active+clean; 724 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 10 MiB/s wr, 625 op/s 2024-05-14T09:58:53.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:52 smithi002 ceph-mon[31557]: pgmap v1144: 129 pgs: 129 active+clean; 724 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 10 MiB/s wr, 625 op/s 2024-05-14T09:58:53.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 302524 25.50 MB/sec execute 499 sec latency 110.183 ms 2024-05-14T09:58:54.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 303048 25.50 MB/sec execute 500 sec latency 116.385 ms 2024-05-14T09:58:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:54 smithi029 ceph-mon[36249]: pgmap v1145: 129 pgs: 129 active+clean; 691 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 647 op/s 2024-05-14T09:58:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:54 smithi026 ceph-mon[36056]: pgmap v1145: 129 pgs: 129 active+clean; 691 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 647 op/s 2024-05-14T09:58:55.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:54 smithi002 ceph-mon[31557]: pgmap v1145: 129 pgs: 129 active+clean; 691 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 647 op/s 2024-05-14T09:58:55.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 303677 25.52 MB/sec execute 501 sec latency 60.137 ms 2024-05-14T09:58:56.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 304251 25.54 MB/sec execute 502 sec latency 79.582 ms 2024-05-14T09:58:56.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:56 smithi029 ceph-mon[36249]: pgmap v1146: 129 pgs: 129 active+clean; 691 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.6 MiB/s wr, 610 op/s 2024-05-14T09:58:56.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:56 smithi026 ceph-mon[36056]: pgmap v1146: 129 pgs: 129 active+clean; 691 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.6 MiB/s wr, 610 op/s 2024-05-14T09:58:57.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:56 smithi002 ceph-mon[31557]: pgmap v1146: 129 pgs: 129 active+clean; 691 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.6 MiB/s wr, 610 op/s 2024-05-14T09:58:57.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 304786 25.53 MB/sec execute 503 sec latency 227.894 ms 2024-05-14T09:58:58.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 305329 25.55 MB/sec execute 504 sec latency 294.534 ms 2024-05-14T09:58:58.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:58:58 smithi029 ceph-mon[36249]: pgmap v1147: 129 pgs: 129 active+clean; 696 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 9.8 MiB/s wr, 702 op/s 2024-05-14T09:58:58.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:58:58 smithi026 ceph-mon[36056]: pgmap v1147: 129 pgs: 129 active+clean; 696 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 9.8 MiB/s wr, 702 op/s 2024-05-14T09:58:58.972 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:58:58 smithi002 ceph-mon[31557]: pgmap v1147: 129 pgs: 129 active+clean; 696 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 9.8 MiB/s wr, 702 op/s 2024-05-14T09:58:59.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 305980 25.57 MB/sec execute 505 sec latency 75.735 ms 2024-05-14T09:58:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.959+0000 7fd6d7261640 1 -- 172.21.15.2:0/358022916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6d0109a00 msgr2=0x7fd6d010be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.959+0000 7fd6d7261640 1 --2- 172.21.15.2:0/358022916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6d0109a00 0x7fd6d010be20 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009ca0 tx=0x7fd6c802a8c0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.959+0000 7fd6d7261640 1 -- 172.21.15.2:0/358022916 shutdown_connections 2024-05-14T09:58:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.959+0000 7fd6d7261640 1 --2- 172.21.15.2:0/358022916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6d0109a00 0x7fd6d010be20 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.959+0000 7fd6d7261640 1 --2- 172.21.15.2:0/358022916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6d01070a0 0x7fd6d01094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.959+0000 7fd6d7261640 1 --2- 172.21.15.2:0/358022916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d0104740 0x7fd6d0106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.959+0000 7fd6d7261640 1 -- 172.21.15.2:0/358022916 >> 172.21.15.2:0/358022916 conn(0x7fd6d00fdd40 msgr2=0x7fd6d01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.959+0000 7fd6d7261640 1 -- 172.21.15.2:0/358022916 shutdown_connections 2024-05-14T09:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.959+0000 7fd6d7261640 1 -- 172.21.15.2:0/358022916 wait complete. 2024-05-14T09:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.960+0000 7fd6d7261640 1 Processor -- start 2024-05-14T09:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.960+0000 7fd6d7261640 1 -- start start 2024-05-14T09:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.960+0000 7fd6d7261640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6d0104740 0x7fd6d019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.960+0000 7fd6d7261640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6d01070a0 0x7fd6d019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.960+0000 7fd6d7261640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d0109a00 0x7fd6d01a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.960+0000 7fd6d7261640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d010f780 con 0x7fd6d0104740 2024-05-14T09:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.960+0000 7fd6d7261640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d010f600 con 0x7fd6d0109a00 2024-05-14T09:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.960+0000 7fd6d7261640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d010f900 con 0x7fd6d01070a0 2024-05-14T09:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.960+0000 7fd6d4ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6d0104740 0x7fd6d019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.960+0000 7fd6d4ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6d0104740 0x7fd6d019a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45166/0 (socket says 172.21.15.2:45166) 2024-05-14T09:58:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.960+0000 7fd6d4ffd640 1 -- 172.21.15.2:0/4240241111 learned_addr learned my addr 172.21.15.2:0/4240241111 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.960+0000 7fd6cffff640 1 --2- 172.21.15.2:0/4240241111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6d01070a0 0x7fd6d019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.960+0000 7fd6d57fe640 1 --2- 172.21.15.2:0/4240241111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d0109a00 0x7fd6d01a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.961+0000 7fd6d4ffd640 1 -- 172.21.15.2:0/4240241111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6d01070a0 msgr2=0x7fd6d019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.961+0000 7fd6d4ffd640 1 --2- 172.21.15.2:0/4240241111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6d01070a0 0x7fd6d019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.961+0000 7fd6d4ffd640 1 -- 172.21.15.2:0/4240241111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d0109a00 msgr2=0x7fd6d01a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.961+0000 7fd6d4ffd640 1 --2- 172.21.15.2:0/4240241111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d0109a00 0x7fd6d01a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.961+0000 7fd6d4ffd640 1 -- 172.21.15.2:0/4240241111 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d01a1db0 con 0x7fd6d0104740 2024-05-14T09:58:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.961+0000 7fd6d4ffd640 1 --2- 172.21.15.2:0/4240241111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6d0104740 0x7fd6d019a270 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400e9c0 tx=0x7fd6c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.961+0000 7fd6cdffb640 1 -- 172.21.15.2:0/4240241111 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c400cd80 con 0x7fd6d0104740 2024-05-14T09:58:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.961+0000 7fd6cdffb640 1 -- 172.21.15.2:0/4240241111 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6c40054d0 con 0x7fd6d0104740 2024-05-14T09:58:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.961+0000 7fd6d7261640 1 -- 172.21.15.2:0/4240241111 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d01a20a0 con 0x7fd6d0104740 2024-05-14T09:58:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.961+0000 7fd6cdffb640 1 -- 172.21.15.2:0/4240241111 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4010430 con 0x7fd6d0104740 2024-05-14T09:58:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.961+0000 7fd6d7261640 1 -- 172.21.15.2:0/4240241111 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d01a25f0 con 0x7fd6d0104740 2024-05-14T09:58:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.964+0000 7fd6de1fc740 1 Processor -- start 2024-05-14T09:58:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.964+0000 7fd6cdffb640 1 -- 172.21.15.2:0/4240241111 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd6c4004fd0 con 0x7fd6d0104740 2024-05-14T09:58:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.964+0000 7fd6de1fc740 1 -- start start 2024-05-14T09:58:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.964+0000 7fd6de1fc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d431b0 0x55bb390e8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.965+0000 7fd6de1fc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d1a2f0 0x55bb390e93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.965+0000 7fd6de1fc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d224d0 0x55bb390f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.965+0000 7fd6de1fc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55bb390495d0 con 0x55bb38d224d0 2024-05-14T09:58:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.965+0000 7fd6de1fc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bb39002890 con 0x55bb38d431b0 2024-05-14T09:58:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.965+0000 7fd6de1fc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bb39045590 con 0x55bb38d1a2f0 2024-05-14T09:58:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.965+0000 7fd6d57fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d431b0 0x55bb390e8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.965+0000 7fd6d57fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d431b0 0x55bb390e8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:38952/0 (socket says 172.21.15.2:38952) 2024-05-14T09:58:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.965+0000 7fd6d57fe640 1 -- 172.21.15.2:0/2851377777 learned_addr learned my addr 172.21.15.2:0/2851377777 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.965+0000 7fd6d4ffd640 1 --2- 172.21.15.2:0/2851377777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d1a2f0 0x55bb390e93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.965+0000 7fd6d57fe640 1 -- 172.21.15.2:0/2851377777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d1a2f0 msgr2=0x55bb390e93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.965+0000 7fd6d57fe640 1 --2- 172.21.15.2:0/2851377777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d1a2f0 0x55bb390e93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.965+0000 7fd6d57fe640 1 -- 172.21.15.2:0/2851377777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d224d0 msgr2=0x55bb390f02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.965+0000 7fd6d57fe640 1 --2- 172.21.15.2:0/2851377777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d224d0 0x55bb390f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.965+0000 7fd6d57fe640 1 -- 172.21.15.2:0/2851377777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb38d5b210 con 0x55bb38d431b0 2024-05-14T09:58:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.966+0000 7fd6cdffb640 1 --2- 172.21.15.2:0/4240241111 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd6c00b28a0 0x7fd6c00b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.966+0000 7fd6d57fe640 1 --2- 172.21.15.2:0/2851377777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d431b0 0x55bb390e8e70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd6d006b740 tx=0x7fd6c802f410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f50aaf85ccec1c39 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.966+0000 7fd6cdffb640 1 -- 172.21.15.2:0/4240241111 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fd6c40e2a90 con 0x7fd6d0104740 2024-05-14T09:58:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.966+0000 7fd6baffd640 1 -- 172.21.15.2:0/2851377777 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8031650 con 0x55bb38d431b0 2024-05-14T09:58:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.966+0000 7fd6baffd640 1 -- 172.21.15.2:0/2851377777 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6c8036030 con 0x55bb38d431b0 2024-05-14T09:58:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.966+0000 7fd6baffd640 1 -- 172.21.15.2:0/2851377777 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8004730 con 0x55bb38d431b0 2024-05-14T09:58:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.967+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2851377777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d431b0 msgr2=0x55bb390e8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.967+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/2851377777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d431b0 0x55bb390e8e70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd6d006b740 tx=0x7fd6c802f410 comp rx=0 tx=0).stop 2024-05-14T09:58:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.968+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2851377777 shutdown_connections 2024-05-14T09:58:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.968+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/2851377777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d224d0 0x55bb390f02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.968+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/2851377777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d1a2f0 0x55bb390e93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.968+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/2851377777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d431b0 0x55bb390e8e70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.968+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2851377777 >> 172.21.15.2:0/2851377777 conn(0x55bb38d37300 msgr2=0x55bb390e03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.968+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2851377777 shutdown_connections 2024-05-14T09:58:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.968+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2851377777 wait complete. 2024-05-14T09:58:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.968+0000 7fd6d4ffd640 1 --2- 172.21.15.2:0/4240241111 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd6c00b28a0 0x7fd6c00b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.968+0000 7fd6de1fc740 1 -- start start 2024-05-14T09:58:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.969+0000 7fd6de1fc740 1 Processor -- start 2024-05-14T09:58:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.969+0000 7fd6d4ffd640 1 --2- 172.21.15.2:0/4240241111 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd6c00b28a0 0x7fd6c00b4db0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55bb390ea250 tx=0x7fd6c40049b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.969+0000 7fd6de1fc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d1a2f0 0x55bb390f2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.969+0000 7fd6de1fc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d224d0 0x55bb390f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.969+0000 7fd6de1fc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d431b0 0x55bb390f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.969+0000 7fd6de1fc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55bb390f0b60 con 0x55bb38d1a2f0 2024-05-14T09:58:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.969+0000 7fd6de1fc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bb39006370 con 0x55bb38d224d0 2024-05-14T09:58:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.969+0000 7fd6de1fc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x55bb390f0ce0 con 0x55bb38d431b0 2024-05-14T09:58:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.969+0000 7fd6de1fc740 10 client.0 _ll_register_callbacks cb 0x55bb38d5aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:58:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.969+0000 7fd6d57fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d1a2f0 0x55bb390f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.969+0000 7fd6d57fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d1a2f0 0x55bb390f2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45172/0 (socket says 172.21.15.2:45172) 2024-05-14T09:58:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.969+0000 7fd6d57fe640 1 -- 172.21.15.2:0/3389124471 learned_addr learned my addr 172.21.15.2:0/3389124471 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:58:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.969+0000 7fd6d57fe640 1 --2- 172.21.15.2:0/3389124471 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d431b0 0x55bb390f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.969+0000 7fd6cffff640 1 --2- 172.21.15.2:0/3389124471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d224d0 0x55bb390f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.970+0000 7fd6d57fe640 1 -- 172.21.15.2:0/3389124471 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d431b0 msgr2=0x55bb390f9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.970+0000 7fd6d57fe640 1 --2- 172.21.15.2:0/3389124471 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d431b0 0x55bb390f9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.970+0000 7fd6d57fe640 1 -- 172.21.15.2:0/3389124471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d224d0 msgr2=0x55bb390f2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:58:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.970+0000 7fd6d57fe640 1 --2- 172.21.15.2:0/3389124471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d224d0 0x55bb390f2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.970+0000 7fd6d57fe640 1 -- 172.21.15.2:0/3389124471 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb38e27620 con 0x55bb38d1a2f0 2024-05-14T09:58:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.971+0000 7fd6d57fe640 1 --2- 172.21.15.2:0/3389124471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d1a2f0 0x55bb390f2930 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009620 tx=0x7fd6c802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0967fa3eef0770 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:58:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.971+0000 7fd6affff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:58:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.971+0000 7fd6affff640 1 -- 172.21.15.2:0/3389124471 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8059060 con 0x55bb38d1a2f0 2024-05-14T09:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.971+0000 7fd6affff640 1 -- 172.21.15.2:0/3389124471 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fd6c8003fb0 con 0x55bb38d1a2f0 2024-05-14T09:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.972+0000 7fd6de1fc740 1 -- 172.21.15.2:0/3389124471 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6c8036030 con 0x55bb38d1a2f0 2024-05-14T09:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.972+0000 7fd6affff640 1 -- 172.21.15.2:0/3389124471 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd6c8009360 con 0x55bb38d1a2f0 2024-05-14T09:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.972+0000 7fd6de1fc740 10 client.17586 fetch_fsmap learned FSMap version 67 2024-05-14T09:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.972+0000 7fd6de1fc740 1 -- 172.21.15.2:0/3389124471 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55bb390f0b60 con 0x55bb38d1a2f0 2024-05-14T09:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.973+0000 7fd6affff640 1 -- 172.21.15.2:0/3389124471 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6c8036030 con 0x55bb38d1a2f0 2024-05-14T09:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.973+0000 7fd6affff640 1 -- 172.21.15.2:0/3389124471 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fd6c8043070 con 0x55bb38d1a2f0 2024-05-14T09:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.973+0000 7fd6de1fc740 10 client.17586 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.973+0000 7fd6de1fc740 10 client.17586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.973+0000 7fd6de1fc740 20 client.17586 populate_metadata read hostname 'smithi002' 2024-05-14T09:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.973+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/3389124471 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb38d01d80 0x55bb390fff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.973+0000 7fd6de1fc740 4 client.17586 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.973+0000 7fd6de1fc740 1 -- 172.21.15.2:0/3389124471 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb390571e0 con 0x55bb38d01d80 2024-05-14T09:58:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.977+0000 7fd6cffff640 1 --2- 172.21.15.2:0/3389124471 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb38d01d80 0x55bb390fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:58:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.978+0000 7fd6cffff640 1 --2- 172.21.15.2:0/3389124471 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb38d01d80 0x55bb390fff00 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a56a33b72950b8e server_cookie=f8a7550f80e9562c in_seq=0 out_seq=0 2024-05-14T09:58:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.978+0000 7fd6affff640 10 client.17586 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.979+0000 7fd6affff640 1 -- 172.21.15.2:0/3389124471 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bb390571e0 con 0x55bb38d01d80 2024-05-14T09:58:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.979+0000 7fd6affff640 10 client.17586 handle_command_reply: tid=0 2024-05-14T09:58:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.979+0000 7fd6de1fc740 1 client.17586 shutdown 2024-05-14T09:58:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.979+0000 7fd6de1fc740 1 -- 172.21.15.2:0/3389124471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d1a2f0 msgr2=0x55bb390f2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:58:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.979+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/3389124471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d1a2f0 0x55bb390f2930 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009620 tx=0x7fd6c802a8c0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.980+0000 7fd6de1fc740 1 -- 172.21.15.2:0/3389124471 shutdown_connections 2024-05-14T09:58:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.980+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/3389124471 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb38d01d80 0x55bb390fff00 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.980+0000 7fd6affff640 0 client.17586 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:58:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.980+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/3389124471 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d431b0 0x55bb390f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.980+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/3389124471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d224d0 0x55bb390f2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.980+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/3389124471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d1a2f0 0x55bb390f2930 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:58:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.980+0000 7fd6de1fc740 1 -- 172.21.15.2:0/3389124471 >> 172.21.15.2:0/3389124471 conn(0x55bb38d37300 msgr2=0x55bb390df3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:58:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.980+0000 7fd6de1fc740 1 -- 172.21.15.2:0/3389124471 shutdown_connections 2024-05-14T09:58:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.980+0000 7fd6de1fc740 1 -- 172.21.15.2:0/3389124471 wait complete. 2024-05-14T09:58:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:58:59.980+0000 7fd6de1fc740 20 client.17586 trim_cache size 0 max 16384 2024-05-14T09:59:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.002+0000 7fd6de1fc740 1 Processor -- start 2024-05-14T09:59:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.002+0000 7fd6de1fc740 1 -- start start 2024-05-14T09:59:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.002+0000 7fd6de1fc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d01d80 0x55bb390df080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.003+0000 7fd6de1fc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d1a2f0 0x55bb390f9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.003+0000 7fd6de1fc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d224d0 0x55bb390fa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.003+0000 7fd6de1fc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01ad2c0 con 0x55bb38d01d80 2024-05-14T09:59:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.003+0000 7fd6de1fc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01aab90 con 0x55bb38d1a2f0 2024-05-14T09:59:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.003+0000 7fd6de1fc740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01a89e0 con 0x55bb38d224d0 2024-05-14T09:59:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.003+0000 7fd6cffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d1a2f0 0x55bb390f9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.003+0000 7fd6d57fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d224d0 0x55bb390fa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.003+0000 7fd6cffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d1a2f0 0x55bb390f9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:38970/0 (socket says 172.21.15.2:38970) 2024-05-14T09:59:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.003+0000 7fd6cffff640 1 -- 172.21.15.2:0/1718437486 learned_addr learned my addr 172.21.15.2:0/1718437486 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.003+0000 7fd6d57fe640 1 --2- 172.21.15.2:0/1718437486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d01d80 0x55bb390df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.003+0000 7fd6cffff640 1 -- 172.21.15.2:0/1718437486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d224d0 msgr2=0x55bb390fa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.003+0000 7fd6cffff640 1 --2- 172.21.15.2:0/1718437486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d224d0 0x55bb390fa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.004+0000 7fd6cffff640 1 -- 172.21.15.2:0/1718437486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d01d80 msgr2=0x55bb390df080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.004+0000 7fd6cffff640 1 --2- 172.21.15.2:0/1718437486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d01d80 0x55bb390df080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.004+0000 7fd6cffff640 1 -- 172.21.15.2:0/1718437486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb390e0d00 con 0x55bb38d1a2f0 2024-05-14T09:59:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.004+0000 7fd6d57fe640 1 --2- 172.21.15.2:0/1718437486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d01d80 0x55bb390df080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:59:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.004+0000 7fd6cffff640 1 --2- 172.21.15.2:0/1718437486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d1a2f0 0x55bb390f9d60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55bb390e1180 tx=0x7fd6bc010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3f866c5226749bd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.004+0000 7fd6ba7fc640 1 -- 172.21.15.2:0/1718437486 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc01d670 con 0x55bb38d1a2f0 2024-05-14T09:59:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.004+0000 7fd6ba7fc640 1 -- 172.21.15.2:0/1718437486 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6bc023070 con 0x55bb38d1a2f0 2024-05-14T09:59:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.004+0000 7fd6ba7fc640 1 -- 172.21.15.2:0/1718437486 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc01ce50 con 0x55bb38d1a2f0 2024-05-14T09:59:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.005+0000 7fd6de1fc740 1 -- 172.21.15.2:0/1718437486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d1a2f0 msgr2=0x55bb390f9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.005+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/1718437486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d1a2f0 0x55bb390f9d60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55bb390e1180 tx=0x7fd6bc010800 comp rx=0 tx=0).stop 2024-05-14T09:59:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.005+0000 7fd6de1fc740 1 -- 172.21.15.2:0/1718437486 shutdown_connections 2024-05-14T09:59:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.005+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/1718437486 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d224d0 0x55bb390fa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.005+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/1718437486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d1a2f0 0x55bb390f9d60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.005+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/1718437486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d01d80 0x55bb390df080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.006+0000 7fd6de1fc740 1 -- 172.21.15.2:0/1718437486 >> 172.21.15.2:0/1718437486 conn(0x55bb38d37300 msgr2=0x55bb390de970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.006+0000 7fd6de1fc740 1 -- 172.21.15.2:0/1718437486 shutdown_connections 2024-05-14T09:59:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.006+0000 7fd6de1fc740 1 -- 172.21.15.2:0/1718437486 wait complete. 2024-05-14T09:59:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.006+0000 7fd6de1fc740 1 -- start start 2024-05-14T09:59:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.007+0000 7fd6de1fc740 1 Processor -- start 2024-05-14T09:59:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.007+0000 7fd6de1fc740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d01d80 0x55bb390f7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.007+0000 7fd6de1fc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d1a2f0 0x55bb390f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6d57fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d01d80 0x55bb390f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6d57fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d01d80 0x55bb390f7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:54754/0 (socket says 172.21.15.2:54754) 2024-05-14T09:59:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6d57fe640 1 -- 172.21.15.2:0/2081758538 learned_addr learned my addr 172.21.15.2:0/2081758538 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/2081758538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d224d0 0x55bb390f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2081758538 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01a9050 con 0x55bb38d224d0 2024-05-14T09:59:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2081758538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01aa4c0 con 0x55bb38d1a2f0 2024-05-14T09:59:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2081758538 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01a9720 con 0x55bb38d01d80 2024-05-14T09:59:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6cffff640 1 --2- 172.21.15.2:0/2081758538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d1a2f0 0x55bb390f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6de1fc740 10 client.0 _ll_register_callbacks cb 0x7fd6c8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:59:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6d57fe640 1 --2- 172.21.15.2:0/2081758538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d224d0 0x55bb390f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6cffff640 1 -- 172.21.15.2:0/2081758538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d01d80 msgr2=0x55bb390f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6cffff640 1 --2- 172.21.15.2:0/2081758538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d01d80 0x55bb390f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6cffff640 1 -- 172.21.15.2:0/2081758538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d224d0 msgr2=0x55bb390f8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6cffff640 1 --2- 172.21.15.2:0/2081758538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d224d0 0x55bb390f8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6cffff640 1 -- 172.21.15.2:0/2081758538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb390f9230 con 0x55bb38d1a2f0 2024-05-14T09:59:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.008+0000 7fd6d57fe640 1 --2- 172.21.15.2:0/2081758538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d224d0 0x55bb390f8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:59:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.009+0000 7fd6cffff640 1 --2- 172.21.15.2:0/2081758538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d1a2f0 0x55bb390f7db0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55bb390456b0 tx=0x7fd6bc01cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adb570c00f2796df server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.009+0000 7fd6affff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:59:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.009+0000 7fd6affff640 1 -- 172.21.15.2:0/2081758538 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc01c7e0 con 0x55bb38d1a2f0 2024-05-14T09:59:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.009+0000 7fd6affff640 1 -- 172.21.15.2:0/2081758538 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fd6bc005050 con 0x55bb38d1a2f0 2024-05-14T09:59:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.009+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2081758538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6bc023070 con 0x55bb38d1a2f0 2024-05-14T09:59:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.009+0000 7fd6affff640 1 -- 172.21.15.2:0/2081758538 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd6bc037050 con 0x55bb38d1a2f0 2024-05-14T09:59:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.009+0000 7fd6de1fc740 10 client.27197 fetch_fsmap learned FSMap version 67 2024-05-14T09:59:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.009+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2081758538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55bb390fa480 con 0x55bb38d1a2f0 2024-05-14T09:59:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.010+0000 7fd6affff640 1 -- 172.21.15.2:0/2081758538 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6bc01d470 con 0x55bb38d1a2f0 2024-05-14T09:59:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.010+0000 7fd6affff640 1 -- 172.21.15.2:0/2081758538 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fd6bc004990 con 0x55bb38d1a2f0 2024-05-14T09:59:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.010+0000 7fd6de1fc740 10 client.27197 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:59:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.010+0000 7fd6de1fc740 10 client.27197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:59:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.010+0000 7fd6de1fc740 20 client.27197 populate_metadata read hostname 'smithi002' 2024-05-14T09:59:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.010+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/2081758538 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb38d431b0 0x55bb3916a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.010+0000 7fd6de1fc740 4 client.27197 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:59:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.010+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2081758538 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fd6bc023070 con 0x55bb38d431b0 2024-05-14T09:59:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.010+0000 7fd6d57fe640 1 --2- 172.21.15.2:0/2081758538 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb38d431b0 0x55bb3916a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.012+0000 7fd6affff640 10 client.27197 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.012+0000 7fd6d57fe640 1 --2- 172.21.15.2:0/2081758538 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb38d431b0 0x55bb3916a630 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c819178cae5111da server_cookie=60e6899d7b10ca05 in_seq=0 out_seq=0 2024-05-14T09:59:00.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 306137 25.53 MB/sec execute 506 sec latency 505.530 ms 2024-05-14T09:59:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.889+0000 7fd6affff640 1 -- 172.21.15.2:0/2081758538 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fd6bc023070 con 0x55bb38d431b0 2024-05-14T09:59:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.890+0000 7fd6affff640 10 client.27197 handle_command_reply: tid=0 2024-05-14T09:59:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.890+0000 7fd6de1fc740 1 client.27197 shutdown 2024-05-14T09:59:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.890+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2081758538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d1a2f0 msgr2=0x55bb390f7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.890+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/2081758538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d1a2f0 0x55bb390f7db0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55bb390456b0 tx=0x7fd6bc01cdb0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.890+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2081758538 shutdown_connections 2024-05-14T09:59:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.890+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/2081758538 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x55bb38d431b0 0x55bb3916a630 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.890+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/2081758538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb38d224d0 0x55bb390f8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.890+0000 7fd6affff640 0 client.27197 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.891+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/2081758538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb38d1a2f0 0x55bb390f7db0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.891+0000 7fd6de1fc740 1 --2- 172.21.15.2:0/2081758538 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55bb38d01d80 0x55bb390f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.891+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2081758538 >> 172.21.15.2:0/2081758538 conn(0x55bb38d37300 msgr2=0x55bb390df3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.891+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2081758538 shutdown_connections 2024-05-14T09:59:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.891+0000 7fd6de1fc740 1 -- 172.21.15.2:0/2081758538 wait complete. 2024-05-14T09:59:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.891+0000 7fd6de1fc740 20 client.27197 trim_cache size 0 max 16384 2024-05-14T09:59:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:59:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:59:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:59:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:59:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.892+0000 7fd6d7261640 1 -- 172.21.15.2:0/4240241111 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd6c00b28a0 msgr2=0x7fd6c00b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.892+0000 7fd6d7261640 1 --2- 172.21.15.2:0/4240241111 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd6c00b28a0 0x7fd6c00b4db0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55bb390ea250 tx=0x7fd6c40049b0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.892+0000 7fd6d7261640 1 -- 172.21.15.2:0/4240241111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6d0104740 msgr2=0x7fd6d019a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.892+0000 7fd6d7261640 1 --2- 172.21.15.2:0/4240241111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6d0104740 0x7fd6d019a270 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400e9c0 tx=0x7fd6c400ee80 comp rx=0 tx=0).stop 2024-05-14T09:59:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.892+0000 7fd6d7261640 1 -- 172.21.15.2:0/4240241111 shutdown_connections 2024-05-14T09:59:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.892+0000 7fd6d7261640 1 --2- 172.21.15.2:0/4240241111 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fd6c00b28a0 0x7fd6c00b4db0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.892+0000 7fd6d7261640 1 --2- 172.21.15.2:0/4240241111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d0109a00 0x7fd6d01a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.892+0000 7fd6d7261640 1 --2- 172.21.15.2:0/4240241111 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd6d01070a0 0x7fd6d019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.892+0000 7fd6d7261640 1 --2- 172.21.15.2:0/4240241111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6d0104740 0x7fd6d019a270 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.892+0000 7fd6d7261640 1 -- 172.21.15.2:0/4240241111 >> 172.21.15.2:0/4240241111 conn(0x7fd6d00fdd40 msgr2=0x7fd6d0105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.892+0000 7fd6d7261640 1 -- 172.21.15.2:0/4240241111 shutdown_connections 2024-05-14T09:59:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:00.892+0000 7fd6d7261640 1 -- 172.21.15.2:0/4240241111 wait complete. 2024-05-14T09:59:00.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:00 smithi026 ceph-mon[36056]: pgmap v1148: 129 pgs: 129 active+clean; 701 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.0 MiB/s wr, 586 op/s 2024-05-14T09:59:01.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:00 smithi002 ceph-mon[31557]: pgmap v1148: 129 pgs: 129 active+clean; 701 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.0 MiB/s wr, 586 op/s 2024-05-14T09:59:01.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:00 smithi029 ceph-mon[36249]: pgmap v1148: 129 pgs: 129 active+clean; 701 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.0 MiB/s wr, 586 op/s 2024-05-14T09:59:01.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 306700 25.53 MB/sec execute 507 sec latency 518.170 ms 2024-05-14T09:59:02.626 INFO:tasks.workunit.client.0.smithi002.stdout: 10 307182 25.54 MB/sec execute 508 sec latency 120.766 ms 2024-05-14T09:59:02.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:02 smithi026 ceph-mon[36056]: pgmap v1149: 129 pgs: 129 active+clean; 696 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.9 MiB/s wr, 586 op/s 2024-05-14T09:59:03.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:02 smithi002 ceph-mon[31557]: pgmap v1149: 129 pgs: 129 active+clean; 696 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.9 MiB/s wr, 586 op/s 2024-05-14T09:59:03.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:02 smithi029 ceph-mon[36249]: pgmap v1149: 129 pgs: 129 active+clean; 696 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.9 MiB/s wr, 586 op/s 2024-05-14T09:59:03.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 307694 25.54 MB/sec execute 509 sec latency 394.100 ms 2024-05-14T09:59:04.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 308257 25.54 MB/sec execute 510 sec latency 104.371 ms 2024-05-14T09:59:04.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:04 smithi026 ceph-mon[36056]: pgmap v1150: 129 pgs: 129 active+clean; 662 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.0 MiB/s wr, 503 op/s 2024-05-14T09:59:05.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:04 smithi002 ceph-mon[31557]: pgmap v1150: 129 pgs: 129 active+clean; 662 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.0 MiB/s wr, 503 op/s 2024-05-14T09:59:05.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:04 smithi029 ceph-mon[36249]: pgmap v1150: 129 pgs: 129 active+clean; 662 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.0 MiB/s wr, 503 op/s 2024-05-14T09:59:05.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 308863 25.57 MB/sec execute 511 sec latency 67.155 ms 2024-05-14T09:59:06.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 309443 25.58 MB/sec execute 512 sec latency 73.519 ms 2024-05-14T09:59:06.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:06 smithi026 ceph-mon[36056]: pgmap v1151: 129 pgs: 129 active+clean; 659 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.7 MiB/s wr, 425 op/s 2024-05-14T09:59:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:06 smithi002 ceph-mon[31557]: pgmap v1151: 129 pgs: 129 active+clean; 659 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.7 MiB/s wr, 425 op/s 2024-05-14T09:59:07.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:06 smithi029 ceph-mon[36249]: pgmap v1151: 129 pgs: 129 active+clean; 659 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.7 MiB/s wr, 425 op/s 2024-05-14T09:59:07.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 309905 25.58 MB/sec execute 513 sec latency 137.838 ms 2024-05-14T09:59:08.088 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:07 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:59:08.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:07 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:59:08.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:07 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T09:59:08.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 310227 25.56 MB/sec execute 514 sec latency 460.499 ms 2024-05-14T09:59:09.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:08 smithi002 ceph-mon[31557]: pgmap v1152: 129 pgs: 129 active+clean; 664 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.4 MiB/s wr, 509 op/s 2024-05-14T09:59:09.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:08 smithi029 ceph-mon[36249]: pgmap v1152: 129 pgs: 129 active+clean; 664 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.4 MiB/s wr, 509 op/s 2024-05-14T09:59:09.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:08 smithi026 ceph-mon[36056]: pgmap v1152: 129 pgs: 129 active+clean; 664 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.4 MiB/s wr, 509 op/s 2024-05-14T09:59:09.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 310726 25.57 MB/sec execute 515 sec latency 74.769 ms 2024-05-14T09:59:10.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 311329 25.58 MB/sec execute 516 sec latency 189.011 ms 2024-05-14T09:59:11.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:10 smithi002 ceph-mon[31557]: pgmap v1153: 129 pgs: 129 active+clean; 668 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.9 MiB/s wr, 458 op/s 2024-05-14T09:59:11.195 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:10 smithi029 ceph-mon[36249]: pgmap v1153: 129 pgs: 129 active+clean; 668 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.9 MiB/s wr, 458 op/s 2024-05-14T09:59:11.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:10 smithi026 ceph-mon[36056]: pgmap v1153: 129 pgs: 129 active+clean; 668 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.9 MiB/s wr, 458 op/s 2024-05-14T09:59:11.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 311917 25.59 MB/sec execute 517 sec latency 231.225 ms 2024-05-14T09:59:12.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 312509 25.60 MB/sec execute 518 sec latency 88.535 ms 2024-05-14T09:59:13.043 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:12 smithi002 ceph-mon[31557]: pgmap v1154: 129 pgs: 129 active+clean; 674 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.1 MiB/s wr, 480 op/s 2024-05-14T09:59:13.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:12 smithi029 ceph-mon[36249]: pgmap v1154: 129 pgs: 129 active+clean; 674 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.1 MiB/s wr, 480 op/s 2024-05-14T09:59:13.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:12 smithi026 ceph-mon[36056]: pgmap v1154: 129 pgs: 129 active+clean; 674 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.1 MiB/s wr, 480 op/s 2024-05-14T09:59:13.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 313052 25.61 MB/sec execute 519 sec latency 112.273 ms 2024-05-14T09:59:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.073+0000 7f7f039ce640 1 -- 172.21.15.2:0/824447838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7efc109b90 msgr2=0x7f7efc10bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.073+0000 7f7f039ce640 1 --2- 172.21.15.2:0/824447838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7efc109b90 0x7f7efc10bfb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800fb10 tx=0x7f7ef802a160 comp rx=0 tx=0).stop 2024-05-14T09:59:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.073+0000 7f7f039ce640 1 -- 172.21.15.2:0/824447838 shutdown_connections 2024-05-14T09:59:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.073+0000 7f7f039ce640 1 --2- 172.21.15.2:0/824447838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7efc109b90 0x7f7efc10bfb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.073+0000 7f7f039ce640 1 --2- 172.21.15.2:0/824447838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7efc104e90 0x7f7efc1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.073+0000 7f7f039ce640 1 --2- 172.21.15.2:0/824447838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7efc103c50 0x7f7efc104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.073+0000 7f7f039ce640 1 -- 172.21.15.2:0/824447838 >> 172.21.15.2:0/824447838 conn(0x7f7efc0fee60 msgr2=0x7f7efc1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.074+0000 7f7f039ce640 1 -- 172.21.15.2:0/824447838 shutdown_connections 2024-05-14T09:59:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.074+0000 7f7f039ce640 1 -- 172.21.15.2:0/824447838 wait complete. 2024-05-14T09:59:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.075+0000 7f7f039ce640 1 Processor -- start 2024-05-14T09:59:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.075+0000 7f7f039ce640 1 -- start start 2024-05-14T09:59:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.078+0000 7f7f039ce640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7efc103c50 0x7f7efc06ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.078+0000 7f7f039ce640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7efc104e90 0x7f7efc06cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.078+0000 7f7f039ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7efc109b90 0x7f7efc073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.078+0000 7f7f039ce640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc10d5d0 con 0x7f7efc103c50 2024-05-14T09:59:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.078+0000 7f7f039ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc10d450 con 0x7f7efc109b90 2024-05-14T09:59:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.078+0000 7f7f039ce640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc10d750 con 0x7f7efc104e90 2024-05-14T09:59:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.078+0000 7f7f00f69640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7efc104e90 0x7f7efc06cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.079+0000 7f7f00f69640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7efc104e90 0x7f7efc06cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:54026/0 (socket says 172.21.15.2:54026) 2024-05-14T09:59:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.079+0000 7f7f00f69640 1 -- 172.21.15.2:0/2047260627 learned_addr learned my addr 172.21.15.2:0/2047260627 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.079+0000 7f7f0176a640 1 --2- 172.21.15.2:0/2047260627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7efc103c50 0x7f7efc06ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.079+0000 7f7f00f69640 1 -- 172.21.15.2:0/2047260627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7efc109b90 msgr2=0x7f7efc073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.079+0000 7f7f00f69640 1 --2- 172.21.15.2:0/2047260627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7efc109b90 0x7f7efc073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.079+0000 7f7f00f69640 1 -- 172.21.15.2:0/2047260627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7efc103c50 msgr2=0x7f7efc06ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.079+0000 7f7f00f69640 1 --2- 172.21.15.2:0/2047260627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7efc103c50 0x7f7efc06ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.079+0000 7f7f00f69640 1 -- 172.21.15.2:0/2047260627 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc0745b0 con 0x7f7efc104e90 2024-05-14T09:59:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.080+0000 7f7f00f69640 1 --2- 172.21.15.2:0/2047260627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7efc104e90 0x7f7efc06cfb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400ebc0 tx=0x7f7ef400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.085+0000 7f7ef27fc640 1 -- 172.21.15.2:0/2047260627 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4010630 con 0x7f7efc104e90 2024-05-14T09:59:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.085+0000 7f7f039ce640 1 -- 172.21.15.2:0/2047260627 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7efc0748a0 con 0x7f7efc104e90 2024-05-14T09:59:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.085+0000 7f7f039ce640 1 -- 172.21.15.2:0/2047260627 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7efc1a8d00 con 0x7f7efc104e90 2024-05-14T09:59:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.085+0000 7f7ef27fc640 1 -- 172.21.15.2:0/2047260627 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef4010c80 con 0x7f7efc104e90 2024-05-14T09:59:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.085+0000 7f7ef27fc640 1 -- 172.21.15.2:0/2047260627 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4018b60 con 0x7f7efc104e90 2024-05-14T09:59:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.094+0000 7f7ef27fc640 1 -- 172.21.15.2:0/2047260627 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f7ef4018d00 con 0x7f7efc104e90 2024-05-14T09:59:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.095+0000 7f7f0a882740 1 Processor -- start 2024-05-14T09:59:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.095+0000 7f7f0a882740 1 -- start start 2024-05-14T09:59:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.095+0000 7f7f0a882740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 0x5630039b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.095+0000 7f7f0a882740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035e62f0 0x5630039b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.095+0000 7f7f0a882740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5630035ee4d0 0x5630039b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.095+0000 7f7f0a882740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5630039155d0 con 0x56300360f1b0 2024-05-14T09:59:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.095+0000 7f7f0a882740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5630038ce890 con 0x5630035e62f0 2024-05-14T09:59:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.095+0000 7f7f0a882740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563003911590 con 0x5630035ee4d0 2024-05-14T09:59:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.096+0000 7f7ef27fc640 1 --2- 172.21.15.2:0/2047260627 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ee80b2a70 0x7f7ee80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.096+0000 7f7ef27fc640 1 -- 172.21.15.2:0/2047260627 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f7ef40e2c20 con 0x7f7efc104e90 2024-05-14T09:59:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.096+0000 7f7f0176a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035e62f0 0x5630039b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.096+0000 7f7f0176a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035e62f0 0x5630039b0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:55746/0 (socket says 172.21.15.2:55746) 2024-05-14T09:59:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.096+0000 7f7f0176a640 1 -- 172.21.15.2:0/1839056627 learned_addr learned my addr 172.21.15.2:0/1839056627 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.097+0000 7f7f01f6b640 1 --2- 172.21.15.2:0/2047260627 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ee80b2a70 0x7f7ee80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.097+0000 7f7f0176a640 1 -- 172.21.15.2:0/1839056627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5630035ee4d0 msgr2=0x5630039b7ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T09:59:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.097+0000 7f7f0176a640 1 --2- 172.21.15.2:0/1839056627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5630035ee4d0 0x5630039b7ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.097+0000 7f7f01f6b640 1 --2- 172.21.15.2:0/1839056627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 0x5630039b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.097+0000 7f7f0176a640 1 -- 172.21.15.2:0/1839056627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 msgr2=0x5630039b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.097+0000 7f7f0176a640 1 --2- 172.21.15.2:0/1839056627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 0x5630039b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.097+0000 7f7f0176a640 1 -- 172.21.15.2:0/1839056627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563003627210 con 0x5630035e62f0 2024-05-14T09:59:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.098+0000 7f7f01f6b640 1 --2- 172.21.15.2:0/1839056627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 0x5630039b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T09:59:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.099+0000 7f7f0176a640 1 --2- 172.21.15.2:0/1839056627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035e62f0 0x5630039b0fe0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0174f0 tx=0x7f7eec0179b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13b28b820f002f6f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.101+0000 7f7f01f6b640 1 --2- 172.21.15.2:0/2047260627 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ee80b2a70 0x7f7ee80b4f80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x5630039b1c30 tx=0x7f7ef8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.101+0000 7f7ee77fe640 1 -- 172.21.15.2:0/1839056627 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec004020 con 0x5630035e62f0 2024-05-14T09:59:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.102+0000 7f7ee77fe640 1 -- 172.21.15.2:0/1839056627 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eec0041c0 con 0x5630035e62f0 2024-05-14T09:59:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.103+0000 7f7f0a882740 1 -- 172.21.15.2:0/1839056627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035e62f0 msgr2=0x5630039b0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.103+0000 7f7f0a882740 1 --2- 172.21.15.2:0/1839056627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035e62f0 0x5630039b0fe0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0174f0 tx=0x7f7eec0179b0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.104+0000 7f7f0a882740 1 -- 172.21.15.2:0/1839056627 shutdown_connections 2024-05-14T09:59:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.104+0000 7f7f0a882740 1 --2- 172.21.15.2:0/1839056627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5630035ee4d0 0x5630039b7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.104+0000 7f7f0a882740 1 --2- 172.21.15.2:0/1839056627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035e62f0 0x5630039b0fe0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.104+0000 7f7f0a882740 1 --2- 172.21.15.2:0/1839056627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 0x5630039b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.104+0000 7f7f0a882740 1 -- 172.21.15.2:0/1839056627 >> 172.21.15.2:0/1839056627 conn(0x563003603300 msgr2=0x5630039ac3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.105+0000 7f7f0a882740 1 -- 172.21.15.2:0/1839056627 shutdown_connections 2024-05-14T09:59:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.105+0000 7f7f0a882740 1 -- 172.21.15.2:0/1839056627 wait complete. 2024-05-14T09:59:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.106+0000 7f7f0a882740 1 -- start start 2024-05-14T09:59:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.106+0000 7f7f0a882740 1 Processor -- start 2024-05-14T09:59:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.106+0000 7f7f0a882740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5630035e62f0 0x5630039ba6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.106+0000 7f7f0a882740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035ee4d0 0x5630039bac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.106+0000 7f7f0a882740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 0x5630039c1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.107+0000 7f7f0a882740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5630039b8980 con 0x56300360f1b0 2024-05-14T09:59:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.107+0000 7f7f0a882740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5630038d2370 con 0x5630035ee4d0 2024-05-14T09:59:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.107+0000 7f7f0a882740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x5630039b8b00 con 0x5630035e62f0 2024-05-14T09:59:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.107+0000 7f7f01f6b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 0x5630039c1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.107+0000 7f7f01f6b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 0x5630039c1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60950/0 (socket says 172.21.15.2:60950) 2024-05-14T09:59:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.107+0000 7f7f01f6b640 1 -- 172.21.15.2:0/3533929400 learned_addr learned my addr 172.21.15.2:0/3533929400 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.107+0000 7f7f01f6b640 1 -- 172.21.15.2:0/3533929400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5630035e62f0 msgr2=0x5630039ba6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:59:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.107+0000 7f7f01f6b640 1 --2- 172.21.15.2:0/3533929400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5630035e62f0 0x5630039ba6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.107+0000 7f7f01f6b640 1 -- 172.21.15.2:0/3533929400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035ee4d0 msgr2=0x5630039bac00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:59:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.107+0000 7f7f01f6b640 1 --2- 172.21.15.2:0/3533929400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035ee4d0 0x5630039bac00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.107+0000 7f7f01f6b640 1 -- 172.21.15.2:0/3533929400 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630036f3620 con 0x56300360f1b0 2024-05-14T09:59:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.108+0000 7f7f01f6b640 1 --2- 172.21.15.2:0/3533929400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 0x5630039c1b60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8032000 tx=0x7f7ef8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebc6349e94651d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.108+0000 7f7ee4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:59:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.108+0000 7f7f0a882740 10 client.0 _ll_register_callbacks cb 0x563003626a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:59:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.108+0000 7f7f0a882740 1 -- 172.21.15.2:0/3533929400 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eec0041c0 con 0x56300360f1b0 2024-05-14T09:59:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.109+0000 7f7ee4ff9640 1 -- 172.21.15.2:0/3533929400 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef804b020 con 0x56300360f1b0 2024-05-14T09:59:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.109+0000 7f7ee4ff9640 1 -- 172.21.15.2:0/3533929400 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f7ef803d470 con 0x56300360f1b0 2024-05-14T09:59:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.111+0000 7f7ee4ff9640 1 -- 172.21.15.2:0/3533929400 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f7ef8056050 con 0x56300360f1b0 2024-05-14T09:59:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.114+0000 7f7f0a882740 10 client.17598 fetch_fsmap learned FSMap version 67 2024-05-14T09:59:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.114+0000 7f7f0a882740 1 -- 172.21.15.2:0/3533929400 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({fsmap=67}) -- 0x5630039c4970 con 0x56300360f1b0 2024-05-14T09:59:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.114+0000 7f7ee4ff9640 1 -- 172.21.15.2:0/3533929400 <== mon.0 v2:172.21.15.2:3300/0 4 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f7ef8046030 con 0x56300360f1b0 2024-05-14T09:59:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.114+0000 7f7f0a882740 10 client.17598 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:59:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.115+0000 7f7f0a882740 10 client.17598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:59:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.115+0000 7f7f0a882740 20 client.17598 populate_metadata read hostname 'smithi002' 2024-05-14T09:59:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.115+0000 7f7f0a882740 1 --2- 172.21.15.2:0/3533929400 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5630035cdd80 0x5630039c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.115+0000 7f7f0a882740 4 client.17598 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:59:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.115+0000 7f7f0a882740 1 -- 172.21.15.2:0/3533929400 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7eec0041c0 con 0x5630035cdd80 2024-05-14T09:59:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.116+0000 7f7f0176a640 1 --2- 172.21.15.2:0/3533929400 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5630035cdd80 0x5630039c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.119+0000 7f7f0176a640 1 --2- 172.21.15.2:0/3533929400 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5630035cdd80 0x5630039c80a0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=213f9c1c2946197f server_cookie=21e45de1a810fff in_seq=0 out_seq=0 2024-05-14T09:59:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.119+0000 7f7ee4ff9640 10 client.17598 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.123+0000 7f7ee4ff9640 1 -- 172.21.15.2:0/3533929400 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f7eec0041c0 con 0x5630035cdd80 2024-05-14T09:59:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.123+0000 7f7ee4ff9640 10 client.17598 handle_command_reply: tid=0 2024-05-14T09:59:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.123+0000 7f7f0a882740 1 client.17598 shutdown 2024-05-14T09:59:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.123+0000 7f7f0a882740 1 -- 172.21.15.2:0/3533929400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 msgr2=0x5630039c1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.123+0000 7f7f0a882740 1 --2- 172.21.15.2:0/3533929400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 0x5630039c1b60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8032000 tx=0x7f7ef8005fd0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.124+0000 7f7f0a882740 1 -- 172.21.15.2:0/3533929400 shutdown_connections 2024-05-14T09:59:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.124+0000 7f7f0a882740 1 --2- 172.21.15.2:0/3533929400 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5630035cdd80 0x5630039c80a0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.124+0000 7f7f0a882740 1 --2- 172.21.15.2:0/3533929400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 0x5630039c1b60 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.124+0000 7f7f0a882740 1 --2- 172.21.15.2:0/3533929400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035ee4d0 0x5630039bac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.124+0000 7f7f0a882740 1 --2- 172.21.15.2:0/3533929400 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5630035e62f0 0x5630039ba6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.124+0000 7f7f0a882740 1 -- 172.21.15.2:0/3533929400 >> 172.21.15.2:0/3533929400 conn(0x563003603300 msgr2=0x5630039af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.125+0000 7f7ee4ff9640 0 client.17598 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.125+0000 7f7f0a882740 1 -- 172.21.15.2:0/3533929400 shutdown_connections 2024-05-14T09:59:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.125+0000 7f7f0a882740 1 -- 172.21.15.2:0/3533929400 wait complete. 2024-05-14T09:59:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.125+0000 7f7f0a882740 20 client.17598 trim_cache size 0 max 16384 2024-05-14T09:59:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.156+0000 7f7f0a882740 1 Processor -- start 2024-05-14T09:59:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.157+0000 7f7f0a882740 1 -- start start 2024-05-14T09:59:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.157+0000 7f7f0a882740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035e62f0 0x5630039ac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.157+0000 7f7f0a882740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5630035ee4d0 0x5630039c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.157+0000 7f7f0a882740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 0x5630039c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.157+0000 7f7f00f69640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5630035ee4d0 0x5630039c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.157+0000 7f7f00f69640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5630035ee4d0 0x5630039c1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:54042/0 (socket says 172.21.15.2:54042) 2024-05-14T09:59:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.157+0000 7f7f00f69640 1 -- 172.21.15.2:0/741842299 learned_addr learned my addr 172.21.15.2:0/741842299 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.157+0000 7f7f0a882740 1 -- 172.21.15.2:0/741842299 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee0012710 con 0x56300360f1b0 2024-05-14T09:59:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.157+0000 7f7f0a882740 1 -- 172.21.15.2:0/741842299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee000ffe0 con 0x5630035e62f0 2024-05-14T09:59:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.157+0000 7f7f0a882740 1 -- 172.21.15.2:0/741842299 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee000de30 con 0x5630035ee4d0 2024-05-14T09:59:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.158+0000 7f7f01f6b640 1 --2- 172.21.15.2:0/741842299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 0x5630039c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.158+0000 7f7f00f69640 1 -- 172.21.15.2:0/741842299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035e62f0 msgr2=0x5630039ac1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:59:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.158+0000 7f7f00f69640 1 --2- 172.21.15.2:0/741842299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035e62f0 0x5630039ac1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.158+0000 7f7f00f69640 1 -- 172.21.15.2:0/741842299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 msgr2=0x5630039c1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.158+0000 7f7f00f69640 1 --2- 172.21.15.2:0/741842299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 0x5630039c1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.158+0000 7f7f00f69640 1 -- 172.21.15.2:0/741842299 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630039b4510 con 0x5630035ee4d0 2024-05-14T09:59:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.159+0000 7f7f00f69640 1 --2- 172.21.15.2:0/741842299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5630035ee4d0 0x5630039c1a80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40a7030 tx=0x7f7ef40a9710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9ed13380f600217 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.161+0000 7f7ee6ffd640 1 -- 172.21.15.2:0/741842299 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40aa4c0 con 0x5630035ee4d0 2024-05-14T09:59:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.161+0000 7f7ee6ffd640 1 -- 172.21.15.2:0/741842299 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef40a9dc0 con 0x5630035ee4d0 2024-05-14T09:59:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.161+0000 7f7ee6ffd640 1 -- 172.21.15.2:0/741842299 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40a9020 con 0x5630035ee4d0 2024-05-14T09:59:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.163+0000 7f7f0a882740 1 -- 172.21.15.2:0/741842299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5630035ee4d0 msgr2=0x5630039c1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.163+0000 7f7f0a882740 1 --2- 172.21.15.2:0/741842299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5630035ee4d0 0x5630039c1a80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40a7030 tx=0x7f7ef40a9710 comp rx=0 tx=0).stop 2024-05-14T09:59:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.163+0000 7f7f0a882740 1 -- 172.21.15.2:0/741842299 shutdown_connections 2024-05-14T09:59:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.163+0000 7f7f0a882740 1 --2- 172.21.15.2:0/741842299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56300360f1b0 0x5630039c1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.163+0000 7f7f0a882740 1 --2- 172.21.15.2:0/741842299 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5630035ee4d0 0x5630039c1a80 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40a7030 tx=0x7f7ef40a9710 comp rx=0 tx=0).stop 2024-05-14T09:59:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.163+0000 7f7f0a882740 1 --2- 172.21.15.2:0/741842299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035e62f0 0x5630039ac1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.163+0000 7f7f0a882740 1 -- 172.21.15.2:0/741842299 >> 172.21.15.2:0/741842299 conn(0x563003603300 msgr2=0x5630039abac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.165+0000 7f7f0a882740 1 -- 172.21.15.2:0/741842299 shutdown_connections 2024-05-14T09:59:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.165+0000 7f7f0a882740 1 -- 172.21.15.2:0/741842299 wait complete. 2024-05-14T09:59:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.166+0000 7f7f0a882740 1 -- start start 2024-05-14T09:59:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.166+0000 7f7f0a882740 1 Processor -- start 2024-05-14T09:59:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.167+0000 7f7f0a882740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630035e62f0 0x5630039bf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.167+0000 7f7f0a882740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56300360f1b0 0x5630039bfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.167+0000 7f7f0a882740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035a31c0 0x5630039c0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.167+0000 7f7f0a882740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee000e4a0 con 0x5630035e62f0 2024-05-14T09:59:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.167+0000 7f7f0a882740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee000f910 con 0x5630035a31c0 2024-05-14T09:59:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.167+0000 7f7f0a882740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee000eb70 con 0x56300360f1b0 2024-05-14T09:59:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.168+0000 7f7f0a882740 10 client.0 _ll_register_callbacks cb 0x7f7ef40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:59:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.169+0000 7f7f01f6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035a31c0 0x5630039c0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.169+0000 7f7f01f6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035a31c0 0x5630039c0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:55768/0 (socket says 172.21.15.2:55768) 2024-05-14T09:59:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.169+0000 7f7f01f6b640 1 -- 172.21.15.2:0/1664997044 learned_addr learned my addr 172.21.15.2:0/1664997044 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.169+0000 7f7f01f6b640 1 -- 172.21.15.2:0/1664997044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56300360f1b0 msgr2=0x5630039bfde0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:59:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.169+0000 7f7f01f6b640 1 --2- 172.21.15.2:0/1664997044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56300360f1b0 0x5630039bfde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.169+0000 7f7f01f6b640 1 -- 172.21.15.2:0/1664997044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630035e62f0 msgr2=0x5630039bf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.169+0000 7f7f01f6b640 1 --2- 172.21.15.2:0/1664997044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630035e62f0 0x5630039bf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.169+0000 7f7f01f6b640 1 -- 172.21.15.2:0/1664997044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630039c1280 con 0x5630035a31c0 2024-05-14T09:59:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.170+0000 7f7f01f6b640 1 --2- 172.21.15.2:0/1664997044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035a31c0 0x5630039c0cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8005ab0 tx=0x7f7ef803e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c5dfcf40b9dc258 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.170+0000 7f7ee4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:59:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.170+0000 7f7ee4ff9640 1 -- 172.21.15.2:0/1664997044 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8043040 con 0x5630035a31c0 2024-05-14T09:59:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.171+0000 7f7ee4ff9640 1 -- 172.21.15.2:0/1664997044 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f7ef805b440 con 0x5630035a31c0 2024-05-14T09:59:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.171+0000 7f7f0a882740 1 -- 172.21.15.2:0/1664997044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ef40a9dc0 con 0x5630035a31c0 2024-05-14T09:59:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.171+0000 7f7ee4ff9640 1 -- 172.21.15.2:0/1664997044 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f7ef805b640 con 0x5630035a31c0 2024-05-14T09:59:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.171+0000 7f7f0a882740 10 client.27206 fetch_fsmap learned FSMap version 67 2024-05-14T09:59:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.171+0000 7f7f0a882740 1 -- 172.21.15.2:0/1664997044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x5630039c2570 con 0x5630035a31c0 2024-05-14T09:59:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.176+0000 7f7ee4ff9640 1 -- 172.21.15.2:0/1664997044 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef40a9dc0 con 0x5630035a31c0 2024-05-14T09:59:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.176+0000 7f7ee4ff9640 1 -- 172.21.15.2:0/1664997044 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f7ef8046040 con 0x5630035a31c0 2024-05-14T09:59:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.176+0000 7f7f0a882740 10 client.27206 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:59:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.176+0000 7f7f0a882740 10 client.27206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:59:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.176+0000 7f7f0a882740 20 client.27206 populate_metadata read hostname 'smithi002' 2024-05-14T09:59:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.176+0000 7f7f0a882740 1 --2- 172.21.15.2:0/1664997044 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563003a30a80 0x563003a32f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.176+0000 7f7f0a882740 4 client.27206 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:59:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.176+0000 7f7f0a882740 1 -- 172.21.15.2:0/1664997044 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630039231e0 con 0x563003a30a80 2024-05-14T09:59:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.177+0000 7f7f0176a640 1 --2- 172.21.15.2:0/1664997044 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563003a30a80 0x563003a32f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.181+0000 7f7f0176a640 1 --2- 172.21.15.2:0/1664997044 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563003a30a80 0x563003a32f00 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3da93b806c5eb731 server_cookie=c641f61f10065dd0 in_seq=0 out_seq=0 2024-05-14T09:59:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:14.182+0000 7f7ee4ff9640 10 client.27206 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:14.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 313412 25.60 MB/sec execute 520 sec latency 343.310 ms 2024-05-14T09:59:15.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:14 smithi002 ceph-mon[31557]: pgmap v1155: 129 pgs: 129 active+clean; 681 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.1 MiB/s wr, 456 op/s 2024-05-14T09:59:15.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:14 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:59:15.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:14 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:59:15.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:14 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:59:15.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:14 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:59:15.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:14 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:59:15.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:14 smithi029 ceph-mon[36249]: pgmap v1155: 129 pgs: 129 active+clean; 681 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.1 MiB/s wr, 456 op/s 2024-05-14T09:59:15.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:59:15.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:59:15.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:59:15.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:59:15.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:59:15.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:14 smithi026 ceph-mon[36056]: pgmap v1155: 129 pgs: 129 active+clean; 681 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.1 MiB/s wr, 456 op/s 2024-05-14T09:59:15.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:14 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:59:15.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:14 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:59:15.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:14 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T09:59:15.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:14 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T09:59:15.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:14 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T09:59:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.569+0000 7f7ee4ff9640 1 -- 172.21.15.2:0/1664997044 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630039231e0 con 0x563003a30a80 2024-05-14T09:59:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.569+0000 7f7ee4ff9640 10 client.27206 handle_command_reply: tid=0 2024-05-14T09:59:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.569+0000 7f7f0a882740 1 client.27206 shutdown 2024-05-14T09:59:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.570+0000 7f7f0a882740 1 -- 172.21.15.2:0/1664997044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035a31c0 msgr2=0x5630039c0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.570+0000 7f7f0a882740 1 --2- 172.21.15.2:0/1664997044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035a31c0 0x5630039c0cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8005ab0 tx=0x7f7ef803e000 comp rx=0 tx=0).stop 2024-05-14T09:59:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.570+0000 7f7f0a882740 1 -- 172.21.15.2:0/1664997044 shutdown_connections 2024-05-14T09:59:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.570+0000 7f7f0a882740 1 --2- 172.21.15.2:0/1664997044 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563003a30a80 0x563003a32f00 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.570+0000 7f7f0a882740 1 --2- 172.21.15.2:0/1664997044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5630035a31c0 0x5630039c0cc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.570+0000 7f7f0a882740 1 --2- 172.21.15.2:0/1664997044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56300360f1b0 0x5630039bfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.570+0000 7f7f0a882740 1 --2- 172.21.15.2:0/1664997044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630035e62f0 0x5630039bf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.570+0000 7f7f0a882740 1 -- 172.21.15.2:0/1664997044 >> 172.21.15.2:0/1664997044 conn(0x563003603300 msgr2=0x5630039afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.570+0000 7f7ee4ff9640 0 client.27206 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.570+0000 7f7f0a882740 1 -- 172.21.15.2:0/1664997044 shutdown_connections 2024-05-14T09:59:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.570+0000 7f7f0a882740 1 -- 172.21.15.2:0/1664997044 wait complete. 2024-05-14T09:59:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.570+0000 7f7f0a882740 20 client.27206 trim_cache size 0 max 16384 2024-05-14T09:59:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:59:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:59:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:59:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:59:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.571+0000 7f7f039ce640 1 -- 172.21.15.2:0/2047260627 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ee80b2a70 msgr2=0x7f7ee80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.571+0000 7f7f039ce640 1 --2- 172.21.15.2:0/2047260627 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ee80b2a70 0x7f7ee80b4f80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x5630039b1c30 tx=0x7f7ef8005a20 comp rx=0 tx=0).stop 2024-05-14T09:59:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.571+0000 7f7f039ce640 1 -- 172.21.15.2:0/2047260627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7efc104e90 msgr2=0x7f7efc06cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.571+0000 7f7f039ce640 1 --2- 172.21.15.2:0/2047260627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7efc104e90 0x7f7efc06cfb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400ebc0 tx=0x7f7ef400c6a0 comp rx=0 tx=0).stop 2024-05-14T09:59:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.572+0000 7f7f039ce640 1 -- 172.21.15.2:0/2047260627 shutdown_connections 2024-05-14T09:59:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.572+0000 7f7f039ce640 1 --2- 172.21.15.2:0/2047260627 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ee80b2a70 0x7f7ee80b4f80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.572+0000 7f7f039ce640 1 --2- 172.21.15.2:0/2047260627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7efc109b90 0x7f7efc073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.572+0000 7f7f039ce640 1 --2- 172.21.15.2:0/2047260627 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7efc104e90 0x7f7efc06cfb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.572+0000 7f7f039ce640 1 --2- 172.21.15.2:0/2047260627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7efc103c50 0x7f7efc06ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.572+0000 7f7f039ce640 1 -- 172.21.15.2:0/2047260627 >> 172.21.15.2:0/2047260627 conn(0x7f7efc0fee60 msgr2=0x7f7efc105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.575+0000 7f7f039ce640 1 -- 172.21.15.2:0/2047260627 shutdown_connections 2024-05-14T09:59:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:15.575+0000 7f7f039ce640 1 -- 172.21.15.2:0/2047260627 wait complete. 2024-05-14T09:59:15.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 313630 25.58 MB/sec execute 521 sec latency 769.793 ms 2024-05-14T09:59:16.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 314159 25.58 MB/sec execute 522 sec latency 129.151 ms 2024-05-14T09:59:17.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:16 smithi002 ceph-mon[31557]: pgmap v1156: 129 pgs: 129 active+clean; 682 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.9 MiB/s wr, 402 op/s 2024-05-14T09:59:17.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:16 smithi029 ceph-mon[36249]: pgmap v1156: 129 pgs: 129 active+clean; 682 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.9 MiB/s wr, 402 op/s 2024-05-14T09:59:17.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:16 smithi026 ceph-mon[36056]: pgmap v1156: 129 pgs: 129 active+clean; 682 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.9 MiB/s wr, 402 op/s 2024-05-14T09:59:17.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 314692 25.57 MB/sec execute 523 sec latency 80.384 ms 2024-05-14T09:59:18.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 315301 25.60 MB/sec execute 524 sec latency 101.413 ms 2024-05-14T09:59:19.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:18 smithi002 ceph-mon[31557]: pgmap v1157: 129 pgs: 129 active+clean; 683 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.4 MiB/s wr, 494 op/s 2024-05-14T09:59:19.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:18 smithi029 ceph-mon[36249]: pgmap v1157: 129 pgs: 129 active+clean; 683 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.4 MiB/s wr, 494 op/s 2024-05-14T09:59:19.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:18 smithi026 ceph-mon[36056]: pgmap v1157: 129 pgs: 129 active+clean; 683 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.4 MiB/s wr, 494 op/s 2024-05-14T09:59:19.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 315959 25.62 MB/sec execute 525 sec latency 91.302 ms 2024-05-14T09:59:20.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 316304 25.60 MB/sec execute 526 sec latency 215.337 ms 2024-05-14T09:59:21.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:20 smithi002 ceph-mon[31557]: pgmap v1158: 129 pgs: 129 active+clean; 687 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.9 MiB/s wr, 506 op/s 2024-05-14T09:59:21.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:20 smithi029 ceph-mon[36249]: pgmap v1158: 129 pgs: 129 active+clean; 687 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.9 MiB/s wr, 506 op/s 2024-05-14T09:59:21.204 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:20 smithi026 ceph-mon[36056]: pgmap v1158: 129 pgs: 129 active+clean; 687 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.9 MiB/s wr, 506 op/s 2024-05-14T09:59:21.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 316825 25.60 MB/sec execute 527 sec latency 99.295 ms 2024-05-14T09:59:22.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 317392 25.62 MB/sec execute 528 sec latency 86.799 ms 2024-05-14T09:59:23.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:22 smithi002 ceph-mon[31557]: pgmap v1159: 129 pgs: 129 active+clean; 695 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.6 MiB/s wr, 534 op/s 2024-05-14T09:59:23.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:22 smithi029 ceph-mon[36249]: pgmap v1159: 129 pgs: 129 active+clean; 695 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.6 MiB/s wr, 534 op/s 2024-05-14T09:59:23.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:22 smithi026 ceph-mon[36056]: pgmap v1159: 129 pgs: 129 active+clean; 695 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.6 MiB/s wr, 534 op/s 2024-05-14T09:59:23.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 318034 25.63 MB/sec execute 529 sec latency 53.304 ms 2024-05-14T09:59:24.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 318574 25.64 MB/sec execute 530 sec latency 80.461 ms 2024-05-14T09:59:25.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:24 smithi002 ceph-mon[31557]: pgmap v1160: 129 pgs: 129 active+clean; 698 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.7 MiB/s wr, 520 op/s 2024-05-14T09:59:25.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:24 smithi029 ceph-mon[36249]: pgmap v1160: 129 pgs: 129 active+clean; 698 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.7 MiB/s wr, 520 op/s 2024-05-14T09:59:25.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:24 smithi026 ceph-mon[36056]: pgmap v1160: 129 pgs: 129 active+clean; 698 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.7 MiB/s wr, 520 op/s 2024-05-14T09:59:25.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 318711 25.60 MB/sec execute 531 sec latency 744.778 ms 2024-05-14T09:59:26.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 319129 25.60 MB/sec execute 532 sec latency 754.804 ms 2024-05-14T09:59:27.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:26 smithi002 ceph-mon[31557]: pgmap v1161: 129 pgs: 129 active+clean; 701 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.4 MiB/s wr, 464 op/s 2024-05-14T09:59:27.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:26 smithi029 ceph-mon[36249]: pgmap v1161: 129 pgs: 129 active+clean; 701 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.4 MiB/s wr, 464 op/s 2024-05-14T09:59:27.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:26 smithi026 ceph-mon[36056]: pgmap v1161: 129 pgs: 129 active+clean; 701 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.4 MiB/s wr, 464 op/s 2024-05-14T09:59:27.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 319385 25.57 MB/sec execute 533 sec latency 155.474 ms 2024-05-14T09:59:28.627 INFO:tasks.workunit.client.0.smithi002.stdout: 10 320047 25.61 MB/sec execute 534 sec latency 64.454 ms 2024-05-14T09:59:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.752+0000 7fb2bafb2640 1 -- 172.21.15.2:0/2732274151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2b4102b50 msgr2=0x7fb2b4103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.752+0000 7fb2bafb2640 1 --2- 172.21.15.2:0/2732274151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2b4102b50 0x7fb2b4103850 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009900 tx=0x7fb2a802a410 comp rx=0 tx=0).stop 2024-05-14T09:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.752+0000 7fb2bafb2640 1 -- 172.21.15.2:0/2732274151 shutdown_connections 2024-05-14T09:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.752+0000 7fb2bafb2640 1 --2- 172.21.15.2:0/2732274151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4108a90 0x7fb2b410aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.752+0000 7fb2bafb2640 1 --2- 172.21.15.2:0/2732274151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b4103d90 0x7fb2b41041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.752+0000 7fb2bafb2640 1 --2- 172.21.15.2:0/2732274151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2b4102b50 0x7fb2b4103850 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.752+0000 7fb2bafb2640 1 -- 172.21.15.2:0/2732274151 >> 172.21.15.2:0/2732274151 conn(0x7fb2b40fdd40 msgr2=0x7fb2b41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.752+0000 7fb2bafb2640 1 -- 172.21.15.2:0/2732274151 shutdown_connections 2024-05-14T09:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.752+0000 7fb2bafb2640 1 -- 172.21.15.2:0/2732274151 wait complete. 2024-05-14T09:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.753+0000 7fb2bafb2640 1 Processor -- start 2024-05-14T09:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.753+0000 7fb2bafb2640 1 -- start start 2024-05-14T09:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.753+0000 7fb2bafb2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2b4102b50 0x7fb2b41a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.753+0000 7fb2bafb2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4103d90 0x7fb2b41a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.753+0000 7fb2bafb2640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b4108a90 0x7fb2b41a7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.753+0000 7fb2bafb2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b410d5d0 con 0x7fb2b4102b50 2024-05-14T09:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.753+0000 7fb2bafb2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b410d450 con 0x7fb2b4103d90 2024-05-14T09:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.753+0000 7fb2bafb2640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b410d750 con 0x7fb2b4108a90 2024-05-14T09:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.754+0000 7fb2b954f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b4108a90 0x7fb2b41a7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.754+0000 7fb2b3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4103d90 0x7fb2b41a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.754+0000 7fb2b954f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b4108a90 0x7fb2b41a7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:57750/0 (socket says 172.21.15.2:57750) 2024-05-14T09:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.754+0000 7fb2b954f640 1 -- 172.21.15.2:0/1412794995 learned_addr learned my addr 172.21.15.2:0/1412794995 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.754+0000 7fb2b8d4e640 1 --2- 172.21.15.2:0/1412794995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2b4102b50 0x7fb2b41a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.754+0000 7fb2b954f640 1 -- 172.21.15.2:0/1412794995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4103d90 msgr2=0x7fb2b41a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.754+0000 7fb2b954f640 1 --2- 172.21.15.2:0/1412794995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4103d90 0x7fb2b41a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.754+0000 7fb2b954f640 1 -- 172.21.15.2:0/1412794995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2b4102b50 msgr2=0x7fb2b41a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.754+0000 7fb2b954f640 1 --2- 172.21.15.2:0/1412794995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2b4102b50 0x7fb2b41a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.754+0000 7fb2b954f640 1 -- 172.21.15.2:0/1412794995 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b41a8400 con 0x7fb2b4108a90 2024-05-14T09:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.754+0000 7fb2b954f640 1 --2- 172.21.15.2:0/1412794995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b4108a90 0x7fb2b41a7d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00eb10 tx=0x7fb2ac00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.754+0000 7fb2b1ffb640 1 -- 172.21.15.2:0/1412794995 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac004b20 con 0x7fb2b4108a90 2024-05-14T09:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.755+0000 7fb2b1ffb640 1 -- 172.21.15.2:0/1412794995 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2ac004cc0 con 0x7fb2b4108a90 2024-05-14T09:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.755+0000 7fb2b1ffb640 1 -- 172.21.15.2:0/1412794995 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac010670 con 0x7fb2b4108a90 2024-05-14T09:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.755+0000 7fb2bafb2640 1 -- 172.21.15.2:0/1412794995 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b41a86f0 con 0x7fb2b4108a90 2024-05-14T09:59:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.755+0000 7fb2bafb2640 1 -- 172.21.15.2:0/1412794995 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b41a8bf0 con 0x7fb2b4108a90 2024-05-14T09:59:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.759+0000 7fb2c1e60740 1 Processor -- start 2024-05-14T09:59:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.759+0000 7fb2b1ffb640 1 -- 172.21.15.2:0/1412794995 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fb2ac005020 con 0x7fb2b4108a90 2024-05-14T09:59:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.759+0000 7fb2c1e60740 1 -- start start 2024-05-14T09:59:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.759+0000 7fb2c1e60740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68fc21b0 0x555b69363a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.759+0000 7fb2c1e60740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68f992f0 0x555b69363fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.759+0000 7fb2c1e60740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fa14d0 0x555b6936aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.759+0000 7fb2c1e60740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555b692c85d0 con 0x555b68fc21b0 2024-05-14T09:59:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.759+0000 7fb2c1e60740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555b69281890 con 0x555b68f992f0 2024-05-14T09:59:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.759+0000 7fb2c1e60740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555b692c4590 con 0x555b68fa14d0 2024-05-14T09:59:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.760+0000 7fb2b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68fc21b0 0x555b69363a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.760+0000 7fb2b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68fc21b0 0x555b69363a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57638/0 (socket says 172.21.15.2:57638) 2024-05-14T09:59:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.760+0000 7fb2b3fff640 1 -- 172.21.15.2:0/2014778613 learned_addr learned my addr 172.21.15.2:0/2014778613 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.760+0000 7fb2b3fff640 1 --2- 172.21.15.2:0/2014778613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fa14d0 0x555b6936aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.760+0000 7fb2b8d4e640 1 --2- 172.21.15.2:0/2014778613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68f992f0 0x555b69363fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.760+0000 7fb2b3fff640 1 -- 172.21.15.2:0/2014778613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fa14d0 msgr2=0x555b6936aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.760+0000 7fb2b3fff640 1 --2- 172.21.15.2:0/2014778613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fa14d0 0x555b6936aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.760+0000 7fb2b3fff640 1 -- 172.21.15.2:0/2014778613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68f992f0 msgr2=0x555b69363fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.760+0000 7fb2b3fff640 1 --2- 172.21.15.2:0/2014778613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68f992f0 0x555b69363fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.760+0000 7fb2b3fff640 1 -- 172.21.15.2:0/2014778613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b68fda210 con 0x555b68fc21b0 2024-05-14T09:59:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.761+0000 7fb2b1ffb640 1 --2- 172.21.15.2:0/1412794995 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2a40b2920 0x7fb2a40b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.761+0000 7fb2b8d4e640 1 --2- 172.21.15.2:0/1412794995 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2a40b2920 0x7fb2a40b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.761+0000 7fb2b3fff640 1 --2- 172.21.15.2:0/2014778613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68fc21b0 0x555b69363a60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0013820 tx=0x7fb2a00097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75baaa0c8d22cc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.761+0000 7fb2b1ffb640 1 -- 172.21.15.2:0/1412794995 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fb2ac0e2b90 con 0x7fb2b4108a90 2024-05-14T09:59:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.761+0000 7fb29effd640 1 -- 172.21.15.2:0/2014778613 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0013da0 con 0x555b68fc21b0 2024-05-14T09:59:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.762+0000 7fb29effd640 1 -- 172.21.15.2:0/2014778613 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2a0013170 con 0x555b68fc21b0 2024-05-14T09:59:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.762+0000 7fb29effd640 1 -- 172.21.15.2:0/2014778613 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0004d70 con 0x555b68fc21b0 2024-05-14T09:59:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.762+0000 7fb2c1e60740 1 -- 172.21.15.2:0/2014778613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68fc21b0 msgr2=0x555b69363a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.762+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/2014778613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68fc21b0 0x555b69363a60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0013820 tx=0x7fb2a00097a0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.763+0000 7fb2b8d4e640 1 --2- 172.21.15.2:0/1412794995 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2a40b2920 0x7fb2a40b4e30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb2a802aa00 tx=0x7fb2a80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.763+0000 7fb2c1e60740 1 -- 172.21.15.2:0/2014778613 shutdown_connections 2024-05-14T09:59:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.763+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/2014778613 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fa14d0 0x555b6936aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.763+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/2014778613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68f992f0 0x555b69363fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.763+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/2014778613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68fc21b0 0x555b69363a60 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.763+0000 7fb2c1e60740 1 -- 172.21.15.2:0/2014778613 >> 172.21.15.2:0/2014778613 conn(0x555b68fb6300 msgr2=0x555b6935f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.763+0000 7fb2c1e60740 1 -- 172.21.15.2:0/2014778613 shutdown_connections 2024-05-14T09:59:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.763+0000 7fb2c1e60740 1 -- 172.21.15.2:0/2014778613 wait complete. 2024-05-14T09:59:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.763+0000 7fb2c1e60740 1 -- start start 2024-05-14T09:59:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.763+0000 7fb2c1e60740 1 Processor -- start 2024-05-14T09:59:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.764+0000 7fb2c1e60740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68f992f0 0x555b6936d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.764+0000 7fb2c1e60740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fa14d0 0x555b6936dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.764+0000 7fb2c1e60740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68fc21b0 0x555b69374b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.764+0000 7fb2c1e60740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555b6936b8d0 con 0x555b68f992f0 2024-05-14T09:59:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.764+0000 7fb2c1e60740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555b69285370 con 0x555b68fc21b0 2024-05-14T09:59:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.764+0000 7fb2c1e60740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x555b6936ba50 con 0x555b68fa14d0 2024-05-14T09:59:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.764+0000 7fb2c1e60740 10 client.0 _ll_register_callbacks cb 0x555b68fd9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:59:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.764+0000 7fb2b8d4e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68f992f0 0x555b6936d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.764+0000 7fb2b8d4e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68f992f0 0x555b6936d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57650/0 (socket says 172.21.15.2:57650) 2024-05-14T09:59:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.764+0000 7fb2b8d4e640 1 -- 172.21.15.2:0/1108781290 learned_addr learned my addr 172.21.15.2:0/1108781290 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.764+0000 7fb2b954f640 1 --2- 172.21.15.2:0/1108781290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68fc21b0 0x555b69374b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.764+0000 7fb2b3fff640 1 --2- 172.21.15.2:0/1108781290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fa14d0 0x555b6936dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.765+0000 7fb2b8d4e640 1 -- 172.21.15.2:0/1108781290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fa14d0 msgr2=0x555b6936dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.765+0000 7fb2b8d4e640 1 --2- 172.21.15.2:0/1108781290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fa14d0 0x555b6936dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.765+0000 7fb2b8d4e640 1 -- 172.21.15.2:0/1108781290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68fc21b0 msgr2=0x555b69374b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.765+0000 7fb2b8d4e640 1 --2- 172.21.15.2:0/1108781290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68fc21b0 0x555b69374b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.765+0000 7fb2b8d4e640 1 -- 172.21.15.2:0/1108781290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b690a6620 con 0x555b68f992f0 2024-05-14T09:59:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.765+0000 7fb2b954f640 1 --2- 172.21.15.2:0/1108781290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68fc21b0 0x555b69374b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:59:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.765+0000 7fb2b8d4e640 1 --2- 172.21.15.2:0/1108781290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68f992f0 0x555b6936d6a0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8006010 tx=0x7fb2a8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4846f1273495a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.765+0000 7fb293fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:59:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.765+0000 7fb293fff640 1 -- 172.21.15.2:0/1108781290 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8035070 con 0x555b68f992f0 2024-05-14T09:59:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.765+0000 7fb293fff640 1 -- 172.21.15.2:0/1108781290 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fb2a8031d70 con 0x555b68f992f0 2024-05-14T09:59:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.765+0000 7fb2c1e60740 1 -- 172.21.15.2:0/1108781290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2a0013170 con 0x555b68f992f0 2024-05-14T09:59:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.766+0000 7fb293fff640 1 -- 172.21.15.2:0/1108781290 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fb2a80035f0 con 0x555b68f992f0 2024-05-14T09:59:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.766+0000 7fb2c1e60740 10 client.17613 fetch_fsmap learned FSMap version 67 2024-05-14T09:59:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.766+0000 7fb2c1e60740 1 -- 172.21.15.2:0/1108781290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x555b69377ae0 con 0x555b68f992f0 2024-05-14T09:59:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.766+0000 7fb293fff640 1 -- 172.21.15.2:0/1108781290 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2a0013170 con 0x555b68f992f0 2024-05-14T09:59:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.766+0000 7fb293fff640 1 -- 172.21.15.2:0/1108781290 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fb2a8043030 con 0x555b68f992f0 2024-05-14T09:59:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.766+0000 7fb2c1e60740 10 client.17613 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:59:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.766+0000 7fb2c1e60740 10 client.17613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:59:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.766+0000 7fb2c1e60740 20 client.17613 populate_metadata read hostname 'smithi002' 2024-05-14T09:59:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.767+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/1108781290 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555b68f80d80 0x555b6937b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.767+0000 7fb2c1e60740 4 client.17613 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:59:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.767+0000 7fb2c1e60740 1 -- 172.21.15.2:0/1108781290 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b692d61e0 con 0x555b68f80d80 2024-05-14T09:59:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.767+0000 7fb2b3fff640 1 --2- 172.21.15.2:0/1108781290 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555b68f80d80 0x555b6937b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.767+0000 7fb293fff640 10 client.17613 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.767+0000 7fb2b3fff640 1 --2- 172.21.15.2:0/1108781290 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555b68f80d80 0x555b6937b4e0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb6a56d21733c56 server_cookie=e8d3cb1fae2d9690 in_seq=0 out_seq=0 2024-05-14T09:59:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.768+0000 7fb293fff640 1 -- 172.21.15.2:0/1108781290 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555b692d61e0 con 0x555b68f80d80 2024-05-14T09:59:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.768+0000 7fb293fff640 10 client.17613 handle_command_reply: tid=0 2024-05-14T09:59:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.769+0000 7fb2c1e60740 1 client.17613 shutdown 2024-05-14T09:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.769+0000 7fb2c1e60740 1 -- 172.21.15.2:0/1108781290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68f992f0 msgr2=0x555b6936d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.769+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/1108781290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68f992f0 0x555b6936d6a0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8006010 tx=0x7fb2a8009260 comp rx=0 tx=0).stop 2024-05-14T09:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.770+0000 7fb2c1e60740 1 -- 172.21.15.2:0/1108781290 shutdown_connections 2024-05-14T09:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.770+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/1108781290 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555b68f80d80 0x555b6937b4e0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.770+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/1108781290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68fc21b0 0x555b69374b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.770+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/1108781290 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fa14d0 0x555b6936dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.770+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/1108781290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68f992f0 0x555b6936d6a0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.770+0000 7fb2c1e60740 1 -- 172.21.15.2:0/1108781290 >> 172.21.15.2:0/1108781290 conn(0x555b68fb6300 msgr2=0x555b69362300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.770+0000 7fb293fff640 0 client.17613 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.770+0000 7fb2c1e60740 1 -- 172.21.15.2:0/1108781290 shutdown_connections 2024-05-14T09:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.770+0000 7fb2c1e60740 1 -- 172.21.15.2:0/1108781290 wait complete. 2024-05-14T09:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.770+0000 7fb2c1e60740 20 client.17613 trim_cache size 0 max 16384 2024-05-14T09:59:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.791+0000 7fb2c1e60740 1 Processor -- start 2024-05-14T09:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.791+0000 7fb2c1e60740 1 -- start start 2024-05-14T09:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.791+0000 7fb2c1e60740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68f992f0 0x555b69362900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.791+0000 7fb2c1e60740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68fa14d0 0x555b69374ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.791+0000 7fb2c1e60740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fc21b0 0x555b69375010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.791+0000 7fb2c1e60740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b41b5b30 con 0x555b68f992f0 2024-05-14T09:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.791+0000 7fb2c1e60740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b41b3400 con 0x555b68fa14d0 2024-05-14T09:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.791+0000 7fb2c1e60740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b41b1250 con 0x555b68fc21b0 2024-05-14T09:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.791+0000 7fb2b8d4e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68f992f0 0x555b69362900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.791+0000 7fb2b954f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fc21b0 0x555b69375010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.791+0000 7fb2b8d4e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68f992f0 0x555b69362900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57658/0 (socket says 172.21.15.2:57658) 2024-05-14T09:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.791+0000 7fb2b954f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fc21b0 0x555b69375010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:57784/0 (socket says 172.21.15.2:57784) 2024-05-14T09:59:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.791+0000 7fb2b954f640 1 -- 172.21.15.2:0/635225802 learned_addr learned my addr 172.21.15.2:0/635225802 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.792+0000 7fb2b3fff640 1 --2- 172.21.15.2:0/635225802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68fa14d0 0x555b69374ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.792+0000 7fb2b954f640 1 -- 172.21.15.2:0/635225802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68fa14d0 msgr2=0x555b69374ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.792+0000 7fb2b954f640 1 --2- 172.21.15.2:0/635225802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68fa14d0 0x555b69374ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.792+0000 7fb2b954f640 1 -- 172.21.15.2:0/635225802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68f992f0 msgr2=0x555b69362900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.792+0000 7fb2b954f640 1 --2- 172.21.15.2:0/635225802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68f992f0 0x555b69362900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.792+0000 7fb2b954f640 1 -- 172.21.15.2:0/635225802 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b6936e710 con 0x555b68fc21b0 2024-05-14T09:59:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.792+0000 7fb2b3fff640 1 --2- 172.21.15.2:0/635225802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68fa14d0 0x555b69374ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T09:59:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.792+0000 7fb2b954f640 1 --2- 172.21.15.2:0/635225802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fc21b0 0x555b69375010 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0e0210 tx=0x7fb2ac0a9740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15bbd213e4cb4dba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.792+0000 7fb29e7fc640 1 -- 172.21.15.2:0/635225802 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0a9d30 con 0x555b68fc21b0 2024-05-14T09:59:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.792+0000 7fb29e7fc640 1 -- 172.21.15.2:0/635225802 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2ac0c0030 con 0x555b68fc21b0 2024-05-14T09:59:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.792+0000 7fb29e7fc640 1 -- 172.21.15.2:0/635225802 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0b2700 con 0x555b68fc21b0 2024-05-14T09:59:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.793+0000 7fb2c1e60740 1 -- 172.21.15.2:0/635225802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fc21b0 msgr2=0x555b69375010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.793+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/635225802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fc21b0 0x555b69375010 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0e0210 tx=0x7fb2ac0a9740 comp rx=0 tx=0).stop 2024-05-14T09:59:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.794+0000 7fb2c1e60740 1 -- 172.21.15.2:0/635225802 shutdown_connections 2024-05-14T09:59:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.794+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/635225802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fc21b0 0x555b69375010 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.794+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/635225802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68fa14d0 0x555b69374ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.794+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/635225802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68f992f0 0x555b69362900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.794+0000 7fb2c1e60740 1 -- 172.21.15.2:0/635225802 >> 172.21.15.2:0/635225802 conn(0x555b68fb6300 msgr2=0x555b693621f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.794+0000 7fb2c1e60740 1 -- 172.21.15.2:0/635225802 shutdown_connections 2024-05-14T09:59:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.794+0000 7fb2c1e60740 1 -- 172.21.15.2:0/635225802 wait complete. 2024-05-14T09:59:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.794+0000 7fb2c1e60740 1 -- start start 2024-05-14T09:59:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.794+0000 7fb2c1e60740 1 Processor -- start 2024-05-14T09:59:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.795+0000 7fb2c1e60740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68f992f0 0x555b693737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.795+0000 7fb2c1e60740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fa14d0 0x555b69373d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.795+0000 7fb2c1e60740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68fc21b0 0x555b69374250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.795+0000 7fb2c1e60740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b41b18c0 con 0x555b68fc21b0 2024-05-14T09:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.795+0000 7fb2c1e60740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b41b2d30 con 0x555b68f992f0 2024-05-14T09:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.795+0000 7fb2c1e60740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b41b1f90 con 0x555b68fa14d0 2024-05-14T09:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.795+0000 7fb2c1e60740 10 client.0 _ll_register_callbacks cb 0x7fb2a0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.795+0000 7fb2b954f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68fc21b0 0x555b69374250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.795+0000 7fb2b954f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68fc21b0 0x555b69374250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57666/0 (socket says 172.21.15.2:57666) 2024-05-14T09:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.795+0000 7fb2b954f640 1 -- 172.21.15.2:0/18858711 learned_addr learned my addr 172.21.15.2:0/18858711 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.796+0000 7fb2b8d4e640 1 --2- 172.21.15.2:0/18858711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68f992f0 0x555b693737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.796+0000 7fb2b8d4e640 1 -- 172.21.15.2:0/18858711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fa14d0 msgr2=0x555b69373d10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T09:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.796+0000 7fb2b8d4e640 1 --2- 172.21.15.2:0/18858711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fa14d0 0x555b69373d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.796+0000 7fb2b8d4e640 1 -- 172.21.15.2:0/18858711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68fc21b0 msgr2=0x555b69374250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.796+0000 7fb2b8d4e640 1 --2- 172.21.15.2:0/18858711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68fc21b0 0x555b69374250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.796+0000 7fb2b8d4e640 1 -- 172.21.15.2:0/18858711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b693e1880 con 0x555b68f992f0 2024-05-14T09:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.796+0000 7fb2b8d4e640 1 --2- 172.21.15.2:0/18858711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68f992f0 0x555b693737d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb2a803c7a0 tx=0x7fb2a803cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e5236009567e91c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.797+0000 7fb293fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:59:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.798+0000 7fb293fff640 1 -- 172.21.15.2:0/18858711 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8003790 con 0x555b68f992f0 2024-05-14T09:59:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.798+0000 7fb293fff640 1 -- 172.21.15.2:0/18858711 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fb2a8003930 con 0x555b68f992f0 2024-05-14T09:59:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.798+0000 7fb2c1e60740 1 -- 172.21.15.2:0/18858711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2ac0c0030 con 0x555b68f992f0 2024-05-14T09:59:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.798+0000 7fb293fff640 1 -- 172.21.15.2:0/18858711 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fb2a8003b30 con 0x555b68f992f0 2024-05-14T09:59:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.799+0000 7fb2c1e60740 10 client.27224 fetch_fsmap learned FSMap version 67 2024-05-14T09:59:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.799+0000 7fb2c1e60740 1 -- 172.21.15.2:0/18858711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x555b693e2ac0 con 0x555b68f992f0 2024-05-14T09:59:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.799+0000 7fb293fff640 1 -- 172.21.15.2:0/18858711 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2ac0c0030 con 0x555b68f992f0 2024-05-14T09:59:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.800+0000 7fb293fff640 1 -- 172.21.15.2:0/18858711 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fb2a805b070 con 0x555b68f992f0 2024-05-14T09:59:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.800+0000 7fb2c1e60740 10 client.27224 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:59:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.800+0000 7fb2c1e60740 10 client.27224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:59:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.800+0000 7fb2c1e60740 20 client.27224 populate_metadata read hostname 'smithi002' 2024-05-14T09:59:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.800+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/18858711 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555b68f561c0 0x555b693e5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.800+0000 7fb2c1e60740 4 client.27224 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:59:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.800+0000 7fb2c1e60740 1 -- 172.21.15.2:0/18858711 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b692d0050 con 0x555b68f561c0 2024-05-14T09:59:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.800+0000 7fb2b3fff640 1 --2- 172.21.15.2:0/18858711 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555b68f561c0 0x555b693e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.801+0000 7fb293fff640 10 client.27224 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:28.801+0000 7fb2b3fff640 1 --2- 172.21.15.2:0/18858711 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555b68f561c0 0x555b693e5460 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f290a03d1fd15394 server_cookie=ef63ffe65edeb3d2 in_seq=0 out_seq=0 2024-05-14T09:59:29.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:28 smithi002 ceph-mon[31557]: pgmap v1162: 129 pgs: 129 active+clean; 715 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.5 MiB/s wr, 574 op/s 2024-05-14T09:59:29.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:28 smithi029 ceph-mon[36249]: pgmap v1162: 129 pgs: 129 active+clean; 715 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.5 MiB/s wr, 574 op/s 2024-05-14T09:59:29.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:28 smithi026 ceph-mon[36056]: pgmap v1162: 129 pgs: 129 active+clean; 715 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.5 MiB/s wr, 574 op/s 2024-05-14T09:59:29.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 320237 25.57 MB/sec execute 535 sec latency 687.089 ms 2024-05-14T09:59:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.707+0000 7fb293fff640 1 -- 172.21.15.2:0/18858711 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b692d0050 con 0x555b68f561c0 2024-05-14T09:59:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.707+0000 7fb293fff640 10 client.27224 handle_command_reply: tid=0 2024-05-14T09:59:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.707+0000 7fb2c1e60740 1 client.27224 shutdown 2024-05-14T09:59:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.708+0000 7fb2c1e60740 1 -- 172.21.15.2:0/18858711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68f992f0 msgr2=0x555b693737d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.708+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/18858711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68f992f0 0x555b693737d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb2a803c7a0 tx=0x7fb2a803cc20 comp rx=0 tx=0).stop 2024-05-14T09:59:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.708+0000 7fb2c1e60740 1 -- 172.21.15.2:0/18858711 shutdown_connections 2024-05-14T09:59:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.708+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/18858711 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x555b68f561c0 0x555b693e5460 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.708+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/18858711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b68fc21b0 0x555b69374250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.708+0000 7fb293fff640 0 client.27224 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.708+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/18858711 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x555b68fa14d0 0x555b69373d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.708+0000 7fb2c1e60740 1 --2- 172.21.15.2:0/18858711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b68f992f0 0x555b693737d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.708+0000 7fb2c1e60740 1 -- 172.21.15.2:0/18858711 >> 172.21.15.2:0/18858711 conn(0x555b68fb6300 msgr2=0x555b6936d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.708+0000 7fb2c1e60740 1 -- 172.21.15.2:0/18858711 shutdown_connections 2024-05-14T09:59:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.709+0000 7fb2c1e60740 1 -- 172.21.15.2:0/18858711 wait complete. 2024-05-14T09:59:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.709+0000 7fb2c1e60740 20 client.27224 trim_cache size 0 max 16384 2024-05-14T09:59:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:59:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:59:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:59:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:59:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.710+0000 7fb2bafb2640 1 -- 172.21.15.2:0/1412794995 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2a40b2920 msgr2=0x7fb2a40b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.710+0000 7fb2bafb2640 1 --2- 172.21.15.2:0/1412794995 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2a40b2920 0x7fb2a40b4e30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb2a802aa00 tx=0x7fb2a80314a0 comp rx=0 tx=0).stop 2024-05-14T09:59:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.710+0000 7fb2bafb2640 1 -- 172.21.15.2:0/1412794995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b4108a90 msgr2=0x7fb2b41a7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.710+0000 7fb2bafb2640 1 --2- 172.21.15.2:0/1412794995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b4108a90 0x7fb2b41a7d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00eb10 tx=0x7fb2ac00c2d0 comp rx=0 tx=0).stop 2024-05-14T09:59:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.710+0000 7fb2bafb2640 1 -- 172.21.15.2:0/1412794995 shutdown_connections 2024-05-14T09:59:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.710+0000 7fb2bafb2640 1 --2- 172.21.15.2:0/1412794995 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2a40b2920 0x7fb2a40b4e30 secure :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb2a802aa00 tx=0x7fb2a80314a0 comp rx=0 tx=0).stop 2024-05-14T09:59:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.710+0000 7fb2bafb2640 1 --2- 172.21.15.2:0/1412794995 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2b4108a90 0x7fb2b41a7d90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.710+0000 7fb2bafb2640 1 --2- 172.21.15.2:0/1412794995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b4103d90 0x7fb2b41a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.710+0000 7fb2bafb2640 1 --2- 172.21.15.2:0/1412794995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2b4102b50 0x7fb2b41a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.710+0000 7fb2bafb2640 1 -- 172.21.15.2:0/1412794995 >> 172.21.15.2:0/1412794995 conn(0x7fb2b40fdd40 msgr2=0x7fb2b4104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.710+0000 7fb2bafb2640 1 -- 172.21.15.2:0/1412794995 shutdown_connections 2024-05-14T09:59:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:29.710+0000 7fb2bafb2640 1 -- 172.21.15.2:0/1412794995 wait complete. 2024-05-14T09:59:30.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 320402 25.54 MB/sec execute 536 sec latency 754.022 ms 2024-05-14T09:59:31.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:30 smithi002 ceph-mon[31557]: pgmap v1163: 129 pgs: 129 active+clean; 715 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 5.8 MiB/s wr, 508 op/s 2024-05-14T09:59:31.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:30 smithi029 ceph-mon[36249]: pgmap v1163: 129 pgs: 129 active+clean; 715 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 5.8 MiB/s wr, 508 op/s 2024-05-14T09:59:31.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:30 smithi026 ceph-mon[36056]: pgmap v1163: 129 pgs: 129 active+clean; 715 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 5.8 MiB/s wr, 508 op/s 2024-05-14T09:59:31.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 320783 25.52 MB/sec execute 537 sec latency 781.793 ms 2024-05-14T09:59:32.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:31 smithi002 ceph-mon[31557]: pgmap v1164: 129 pgs: 129 active+clean; 713 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.4 MiB/s wr, 516 op/s 2024-05-14T09:59:32.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:31 smithi029 ceph-mon[36249]: pgmap v1164: 129 pgs: 129 active+clean; 713 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.4 MiB/s wr, 516 op/s 2024-05-14T09:59:32.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:31 smithi026 ceph-mon[36056]: pgmap v1164: 129 pgs: 129 active+clean; 713 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.4 MiB/s wr, 516 op/s 2024-05-14T09:59:32.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 321100 25.50 MB/sec execute 538 sec latency 323.629 ms 2024-05-14T09:59:33.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 321744 25.53 MB/sec execute 539 sec latency 95.434 ms 2024-05-14T09:59:34.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 322353 25.55 MB/sec execute 540 sec latency 74.929 ms 2024-05-14T09:59:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:34 smithi002 ceph-mon[31557]: pgmap v1165: 129 pgs: 129 active+clean; 713 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.0 MiB/s wr, 598 op/s 2024-05-14T09:59:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:34 smithi029 ceph-mon[36249]: pgmap v1165: 129 pgs: 129 active+clean; 713 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.0 MiB/s wr, 598 op/s 2024-05-14T09:59:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:34 smithi026 ceph-mon[36056]: pgmap v1165: 129 pgs: 129 active+clean; 713 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.0 MiB/s wr, 598 op/s 2024-05-14T09:59:35.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 322638 25.53 MB/sec execute 541 sec latency 501.353 ms 2024-05-14T09:59:36.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 322910 25.51 MB/sec execute 542 sec latency 659.473 ms 2024-05-14T09:59:36.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:36 smithi002 ceph-mon[31557]: pgmap v1166: 129 pgs: 129 active+clean; 714 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.8 MiB/s wr, 582 op/s 2024-05-14T09:59:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:36 smithi029 ceph-mon[36249]: pgmap v1166: 129 pgs: 129 active+clean; 714 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.8 MiB/s wr, 582 op/s 2024-05-14T09:59:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:36 smithi026 ceph-mon[36056]: pgmap v1166: 129 pgs: 129 active+clean; 714 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.8 MiB/s wr, 582 op/s 2024-05-14T09:59:37.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 323252 25.49 MB/sec execute 543 sec latency 169.214 ms 2024-05-14T09:59:38.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 323841 25.50 MB/sec execute 544 sec latency 111.570 ms 2024-05-14T09:59:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:38 smithi002 ceph-mon[31557]: pgmap v1167: 129 pgs: 129 active+clean; 717 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.8 MiB/s wr, 625 op/s 2024-05-14T09:59:38.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:38 smithi029 ceph-mon[36249]: pgmap v1167: 129 pgs: 129 active+clean; 717 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.8 MiB/s wr, 625 op/s 2024-05-14T09:59:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:38 smithi026 ceph-mon[36056]: pgmap v1167: 129 pgs: 129 active+clean; 717 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.8 MiB/s wr, 625 op/s 2024-05-14T09:59:39.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 324391 25.49 MB/sec execute 545 sec latency 103.668 ms 2024-05-14T09:59:40.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 324629 25.48 MB/sec execute 546 sec latency 366.728 ms 2024-05-14T09:59:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:40 smithi029 ceph-mon[36249]: pgmap v1168: 129 pgs: 129 active+clean; 720 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.5 MiB/s wr, 573 op/s 2024-05-14T09:59:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:40 smithi026 ceph-mon[36056]: pgmap v1168: 129 pgs: 129 active+clean; 720 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.5 MiB/s wr, 573 op/s 2024-05-14T09:59:41.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:40 smithi002 ceph-mon[31557]: pgmap v1168: 129 pgs: 129 active+clean; 720 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.5 MiB/s wr, 573 op/s 2024-05-14T09:59:41.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 325029 25.48 MB/sec execute 547 sec latency 535.291 ms 2024-05-14T09:59:42.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 325424 25.48 MB/sec execute 548 sec latency 185.325 ms 2024-05-14T09:59:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.863+0000 7f06f1af9640 1 -- 172.21.15.2:0/1086022885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec102b50 msgr2=0x7f06ec103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.863+0000 7f06f1af9640 1 --2- 172.21.15.2:0/1086022885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec102b50 0x7f06ec103850 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009900 tx=0x7f06dc02a410 comp rx=0 tx=0).stop 2024-05-14T09:59:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.863+0000 7f06f1af9640 1 -- 172.21.15.2:0/1086022885 shutdown_connections 2024-05-14T09:59:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.863+0000 7f06f1af9640 1 --2- 172.21.15.2:0/1086022885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06ec108a90 0x7f06ec10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.863+0000 7f06f1af9640 1 --2- 172.21.15.2:0/1086022885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06ec103d90 0x7f06ec1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.863+0000 7f06f1af9640 1 --2- 172.21.15.2:0/1086022885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec102b50 0x7f06ec103850 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.863+0000 7f06f1af9640 1 -- 172.21.15.2:0/1086022885 >> 172.21.15.2:0/1086022885 conn(0x7f06ec0fdd40 msgr2=0x7f06ec1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.863+0000 7f06f1af9640 1 -- 172.21.15.2:0/1086022885 shutdown_connections 2024-05-14T09:59:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.864+0000 7f06f1af9640 1 -- 172.21.15.2:0/1086022885 wait complete. 2024-05-14T09:59:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.864+0000 7f06f1af9640 1 Processor -- start 2024-05-14T09:59:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.864+0000 7f06f1af9640 1 -- start start 2024-05-14T09:59:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.865+0000 7f06f1af9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec102b50 0x7f06ec1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.865+0000 7f06f1af9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06ec103d90 0x7f06ec199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.865+0000 7f06f1af9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06ec108a90 0x7f06ec06a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.865+0000 7f06f1af9640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec10e650 con 0x7f06ec102b50 2024-05-14T09:59:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.865+0000 7f06f1af9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec10e4d0 con 0x7f06ec103d90 2024-05-14T09:59:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.865+0000 7f06f1af9640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec10e7d0 con 0x7f06ec108a90 2024-05-14T09:59:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.865+0000 7f06eb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec102b50 0x7f06ec1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.865+0000 7f06eb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec102b50 0x7f06ec1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60286/0 (socket says 172.21.15.2:60286) 2024-05-14T09:59:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.865+0000 7f06eb7fe640 1 -- 172.21.15.2:0/758340930 learned_addr learned my addr 172.21.15.2:0/758340930 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.865+0000 7f06ebfff640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06ec108a90 0x7f06ec06a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.865+0000 7f06eaffd640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06ec103d90 0x7f06ec199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.866+0000 7f06eaffd640 1 -- 172.21.15.2:0/758340930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06ec108a90 msgr2=0x7f06ec06a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.866+0000 7f06eaffd640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06ec108a90 0x7f06ec06a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.866+0000 7f06eaffd640 1 -- 172.21.15.2:0/758340930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec102b50 msgr2=0x7f06ec1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.866+0000 7f06eaffd640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec102b50 0x7f06ec1991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.866+0000 7f06eaffd640 1 -- 172.21.15.2:0/758340930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ec06acd0 con 0x7f06ec103d90 2024-05-14T09:59:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.866+0000 7f06ebfff640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06ec108a90 0x7f06ec06a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:59:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.866+0000 7f06eb7fe640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec102b50 0x7f06ec1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T09:59:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.866+0000 7f06eaffd640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06ec103d90 0x7f06ec199730 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f06e000e9f0 tx=0x7f06e000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.866+0000 7f06e8ff9640 1 -- 172.21.15.2:0/758340930 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e000ccc0 con 0x7f06ec103d90 2024-05-14T09:59:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.867+0000 7f06e8ff9640 1 -- 172.21.15.2:0/758340930 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06e0005020 con 0x7f06ec103d90 2024-05-14T09:59:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.867+0000 7f06f1af9640 1 -- 172.21.15.2:0/758340930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ec06afc0 con 0x7f06ec103d90 2024-05-14T09:59:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.867+0000 7f06e8ff9640 1 -- 172.21.15.2:0/758340930 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0004a00 con 0x7f06ec103d90 2024-05-14T09:59:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.867+0000 7f06f1af9640 1 -- 172.21.15.2:0/758340930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ec06b510 con 0x7f06ec103d90 2024-05-14T09:59:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.870+0000 7f06f89a9740 1 Processor -- start 2024-05-14T09:59:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.871+0000 7f06f89a9740 1 -- start start 2024-05-14T09:59:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.872+0000 7f06f89a9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb386c1b0 0x557bb3c16270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.872+0000 7f06f89a9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb38432f0 0x557bb3c167f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.872+0000 7f06f89a9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb384b4d0 0x557bb3c1d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.872+0000 7f06f89a9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557bb3b725d0 con 0x557bb386c1b0 2024-05-14T09:59:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.872+0000 7f06f89a9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557bb3b2b890 con 0x557bb38432f0 2024-05-14T09:59:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.872+0000 7f06f89a9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557bb3b6e590 con 0x557bb384b4d0 2024-05-14T09:59:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.872+0000 7f06ebfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb386c1b0 0x557bb3c16270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.872+0000 7f06ebfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb386c1b0 0x557bb3c16270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60294/0 (socket says 172.21.15.2:60294) 2024-05-14T09:59:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.872+0000 7f06ebfff640 1 -- 172.21.15.2:0/3907530758 learned_addr learned my addr 172.21.15.2:0/3907530758 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.872+0000 7f06e8ff9640 1 -- 172.21.15.2:0/758340930 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f06e0004ba0 con 0x7f06ec103d90 2024-05-14T09:59:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.872+0000 7f06eb7fe640 1 --2- 172.21.15.2:0/3907530758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb38432f0 0x557bb3c167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.873+0000 7f06eaffd640 1 --2- 172.21.15.2:0/3907530758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb384b4d0 0x557bb3c1d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.873+0000 7f06ebfff640 1 -- 172.21.15.2:0/3907530758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb384b4d0 msgr2=0x557bb3c1d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.873+0000 7f06ebfff640 1 --2- 172.21.15.2:0/3907530758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb384b4d0 0x557bb3c1d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.873+0000 7f06ebfff640 1 -- 172.21.15.2:0/3907530758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb38432f0 msgr2=0x557bb3c167f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.873+0000 7f06ebfff640 1 --2- 172.21.15.2:0/3907530758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb38432f0 0x557bb3c167f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.873+0000 7f06ebfff640 1 -- 172.21.15.2:0/3907530758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bb3884210 con 0x557bb386c1b0 2024-05-14T09:59:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.873+0000 7f06ebfff640 1 --2- 172.21.15.2:0/3907530758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb386c1b0 0x557bb3c16270 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f06e40117e0 tx=0x7f06e4011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ea966740c77d16 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.874+0000 7f06d1ffb640 1 -- 172.21.15.2:0/3907530758 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4011310 con 0x557bb386c1b0 2024-05-14T09:59:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.874+0000 7f06d1ffb640 1 -- 172.21.15.2:0/3907530758 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06e4003e20 con 0x557bb386c1b0 2024-05-14T09:59:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.874+0000 7f06e8ff9640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f06d40b2920 0x7f06d40b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.874+0000 7f06e8ff9640 1 -- 172.21.15.2:0/758340930 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f06e00e2530 con 0x7f06ec103d90 2024-05-14T09:59:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.874+0000 7f06eb7fe640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f06d40b2920 0x7f06d40b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.875+0000 7f06d1ffb640 1 -- 172.21.15.2:0/3907530758 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e40044f0 con 0x557bb386c1b0 2024-05-14T09:59:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.875+0000 7f06f89a9740 1 -- 172.21.15.2:0/3907530758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb386c1b0 msgr2=0x557bb3c16270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.875+0000 7f06f89a9740 1 --2- 172.21.15.2:0/3907530758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb386c1b0 0x557bb3c16270 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f06e40117e0 tx=0x7f06e4011ca0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.875+0000 7f06f89a9740 1 -- 172.21.15.2:0/3907530758 shutdown_connections 2024-05-14T09:59:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.875+0000 7f06f89a9740 1 --2- 172.21.15.2:0/3907530758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb384b4d0 0x557bb3c1d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.875+0000 7f06f89a9740 1 --2- 172.21.15.2:0/3907530758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb38432f0 0x557bb3c167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.875+0000 7f06f89a9740 1 --2- 172.21.15.2:0/3907530758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb386c1b0 0x557bb3c16270 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.875+0000 7f06f89a9740 1 -- 172.21.15.2:0/3907530758 >> 172.21.15.2:0/3907530758 conn(0x557bb3860300 msgr2=0x557bb3c093a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.875+0000 7f06eb7fe640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f06d40b2920 0x7f06d40b4e30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f06dc031500 tx=0x7f06dc0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.875+0000 7f06f89a9740 1 -- 172.21.15.2:0/3907530758 shutdown_connections 2024-05-14T09:59:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.875+0000 7f06f89a9740 1 -- 172.21.15.2:0/3907530758 wait complete. 2024-05-14T09:59:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.876+0000 7f06f89a9740 1 -- start start 2024-05-14T09:59:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.876+0000 7f06f89a9740 1 Processor -- start 2024-05-14T09:59:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.876+0000 7f06f89a9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb38432f0 0x557bb3c1fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.876+0000 7f06f89a9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb384b4d0 0x557bb3c20430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.876+0000 7f06f89a9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb386c1b0 0x557bb3c27390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.876+0000 7f06eaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb384b4d0 0x557bb3c20430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.876+0000 7f06f89a9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557bb3c1e190 con 0x557bb384b4d0 2024-05-14T09:59:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.876+0000 7f06f89a9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557bb3b2f370 con 0x557bb38432f0 2024-05-14T09:59:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.876+0000 7f06f89a9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x557bb3c1e310 con 0x557bb386c1b0 2024-05-14T09:59:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.876+0000 7f06ebfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb386c1b0 0x557bb3c27390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.876+0000 7f06eb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb38432f0 0x557bb3c1fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.876+0000 7f06eaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb384b4d0 0x557bb3c20430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60310/0 (socket says 172.21.15.2:60310) 2024-05-14T09:59:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.876+0000 7f06eaffd640 1 -- 172.21.15.2:0/2831088999 learned_addr learned my addr 172.21.15.2:0/2831088999 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.876+0000 7f06eb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb38432f0 0x557bb3c1fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:35780/0 (socket says 172.21.15.2:35780) 2024-05-14T09:59:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.876+0000 7f06ebfff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb386c1b0 0x557bb3c27390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:52992/0 (socket says 172.21.15.2:52992) 2024-05-14T09:59:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.877+0000 7f06f89a9740 10 client.0 _ll_register_callbacks cb 0x557bb3883a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:59:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.877+0000 7f06eaffd640 1 -- 172.21.15.2:0/2831088999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb386c1b0 msgr2=0x557bb3c27390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.877+0000 7f06eaffd640 1 --2- 172.21.15.2:0/2831088999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb386c1b0 0x557bb3c27390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.877+0000 7f06eaffd640 1 -- 172.21.15.2:0/2831088999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb38432f0 msgr2=0x557bb3c1fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.877+0000 7f06eaffd640 1 --2- 172.21.15.2:0/2831088999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb38432f0 0x557bb3c1fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.877+0000 7f06eaffd640 1 -- 172.21.15.2:0/2831088999 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bb3950620 con 0x557bb384b4d0 2024-05-14T09:59:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.877+0000 7f06eaffd640 1 --2- 172.21.15.2:0/2831088999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb384b4d0 0x557bb3c20430 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009230 tx=0x7f06e00a55f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8e0cca4f9dc13e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.877+0000 7f06caffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:59:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.877+0000 7f06caffd640 1 -- 172.21.15.2:0/2831088999 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e00e2030 con 0x557bb384b4d0 2024-05-14T09:59:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.877+0000 7f06caffd640 1 -- 172.21.15.2:0/2831088999 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f06e00e21d0 con 0x557bb384b4d0 2024-05-14T09:59:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.877+0000 7f06f89a9740 1 -- 172.21.15.2:0/2831088999 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e4003e20 con 0x557bb384b4d0 2024-05-14T09:59:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.878+0000 7f06caffd640 1 -- 172.21.15.2:0/2831088999 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f06e001c070 con 0x557bb384b4d0 2024-05-14T09:59:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.878+0000 7f06f89a9740 10 client.17631 fetch_fsmap learned FSMap version 67 2024-05-14T09:59:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.878+0000 7f06f89a9740 1 -- 172.21.15.2:0/2831088999 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x557bb3c2a370 con 0x557bb384b4d0 2024-05-14T09:59:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.878+0000 7f06caffd640 1 -- 172.21.15.2:0/2831088999 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06e4003e20 con 0x557bb384b4d0 2024-05-14T09:59:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.878+0000 7f06caffd640 1 -- 172.21.15.2:0/2831088999 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f06e00910f0 con 0x557bb384b4d0 2024-05-14T09:59:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.878+0000 7f06f89a9740 10 client.17631 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:59:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.878+0000 7f06f89a9740 10 client.17631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:59:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.878+0000 7f06f89a9740 20 client.17631 populate_metadata read hostname 'smithi002' 2024-05-14T09:59:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.878+0000 7f06f89a9740 1 --2- 172.21.15.2:0/2831088999 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557bb382ad80 0x557bb3c2da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.878+0000 7f06f89a9740 4 client.17631 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:59:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.878+0000 7f06f89a9740 1 -- 172.21.15.2:0/2831088999 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bb3b801e0 con 0x557bb382ad80 2024-05-14T09:59:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.878+0000 7f06eb7fe640 1 --2- 172.21.15.2:0/2831088999 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557bb382ad80 0x557bb3c2da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.879+0000 7f06eb7fe640 1 --2- 172.21.15.2:0/2831088999 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557bb382ad80 0x557bb3c2da60 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b66ba4f7d0af46d server_cookie=424d033812e9e0cb in_seq=0 out_seq=0 2024-05-14T09:59:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.879+0000 7f06caffd640 10 client.17631 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.882+0000 7f06caffd640 1 -- 172.21.15.2:0/2831088999 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557bb3b801e0 con 0x557bb382ad80 2024-05-14T09:59:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.882+0000 7f06caffd640 10 client.17631 handle_command_reply: tid=0 2024-05-14T09:59:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.882+0000 7f06f89a9740 1 client.17631 shutdown 2024-05-14T09:59:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.882+0000 7f06f89a9740 1 -- 172.21.15.2:0/2831088999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb384b4d0 msgr2=0x557bb3c20430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.882+0000 7f06f89a9740 1 --2- 172.21.15.2:0/2831088999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb384b4d0 0x557bb3c20430 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009230 tx=0x7f06e00a55f0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.882+0000 7f06f89a9740 1 -- 172.21.15.2:0/2831088999 shutdown_connections 2024-05-14T09:59:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.882+0000 7f06f89a9740 1 --2- 172.21.15.2:0/2831088999 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557bb382ad80 0x557bb3c2da60 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.882+0000 7f06f89a9740 1 --2- 172.21.15.2:0/2831088999 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb386c1b0 0x557bb3c27390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.882+0000 7f06f89a9740 1 --2- 172.21.15.2:0/2831088999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb384b4d0 0x557bb3c20430 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.882+0000 7f06f89a9740 1 --2- 172.21.15.2:0/2831088999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb38432f0 0x557bb3c1fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.882+0000 7f06f89a9740 1 -- 172.21.15.2:0/2831088999 >> 172.21.15.2:0/2831088999 conn(0x557bb3860300 msgr2=0x557bb3c083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.882+0000 7f06caffd640 0 client.17631 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.882+0000 7f06f89a9740 1 -- 172.21.15.2:0/2831088999 shutdown_connections 2024-05-14T09:59:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.882+0000 7f06f89a9740 1 -- 172.21.15.2:0/2831088999 wait complete. 2024-05-14T09:59:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.882+0000 7f06f89a9740 20 client.17631 trim_cache size 0 max 16384 2024-05-14T09:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.899+0000 7f06f89a9740 1 Processor -- start 2024-05-14T09:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.899+0000 7f06f89a9740 1 -- start start 2024-05-14T09:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.899+0000 7f06f89a9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb38432f0 0x557bb3c08cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.899+0000 7f06f89a9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb384b4d0 0x557bb3c09230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.899+0000 7f06f89a9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb386c1b0 0x557bb3c0c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.899+0000 7f06f89a9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1ad360 con 0x557bb386c1b0 2024-05-14T09:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.899+0000 7f06f89a9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1aac30 con 0x557bb384b4d0 2024-05-14T09:59:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.899+0000 7f06f89a9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1a8a80 con 0x557bb38432f0 2024-05-14T09:59:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.899+0000 7f06ebfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb386c1b0 0x557bb3c0c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.899+0000 7f06eaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb384b4d0 0x557bb3c09230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.899+0000 7f06ebfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb386c1b0 0x557bb3c0c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60318/0 (socket says 172.21.15.2:60318) 2024-05-14T09:59:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.899+0000 7f06ebfff640 1 -- 172.21.15.2:0/3301844452 learned_addr learned my addr 172.21.15.2:0/3301844452 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.900+0000 7f06eb7fe640 1 --2- 172.21.15.2:0/3301844452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb38432f0 0x557bb3c08cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.900+0000 7f06ebfff640 1 -- 172.21.15.2:0/3301844452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb38432f0 msgr2=0x557bb3c08cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.900+0000 7f06ebfff640 1 --2- 172.21.15.2:0/3301844452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb38432f0 0x557bb3c08cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.900+0000 7f06ebfff640 1 -- 172.21.15.2:0/3301844452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb384b4d0 msgr2=0x557bb3c09230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.900+0000 7f06ebfff640 1 --2- 172.21.15.2:0/3301844452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb384b4d0 0x557bb3c09230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.900+0000 7f06ebfff640 1 -- 172.21.15.2:0/3301844452 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bb3c09d00 con 0x557bb386c1b0 2024-05-14T09:59:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.900+0000 7f06ebfff640 1 --2- 172.21.15.2:0/3301844452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb386c1b0 0x557bb3c0c1f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f06e400f790 tx=0x7f06e4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992c16af565cc5e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.900+0000 7f06d17fa640 1 -- 172.21.15.2:0/3301844452 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e402b070 con 0x557bb386c1b0 2024-05-14T09:59:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.900+0000 7f06d17fa640 1 -- 172.21.15.2:0/3301844452 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06e400fe50 con 0x557bb386c1b0 2024-05-14T09:59:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.900+0000 7f06d17fa640 1 -- 172.21.15.2:0/3301844452 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e40055e0 con 0x557bb386c1b0 2024-05-14T09:59:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.901+0000 7f06f89a9740 1 -- 172.21.15.2:0/3301844452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb386c1b0 msgr2=0x557bb3c0c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.901+0000 7f06f89a9740 1 --2- 172.21.15.2:0/3301844452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb386c1b0 0x557bb3c0c1f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f06e400f790 tx=0x7f06e4005bd0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.901+0000 7f06f89a9740 1 -- 172.21.15.2:0/3301844452 shutdown_connections 2024-05-14T09:59:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.901+0000 7f06f89a9740 1 --2- 172.21.15.2:0/3301844452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb386c1b0 0x557bb3c0c1f0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.901+0000 7f06f89a9740 1 --2- 172.21.15.2:0/3301844452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb384b4d0 0x557bb3c09230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.901+0000 7f06f89a9740 1 --2- 172.21.15.2:0/3301844452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb38432f0 0x557bb3c08cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.901+0000 7f06f89a9740 1 -- 172.21.15.2:0/3301844452 >> 172.21.15.2:0/3301844452 conn(0x557bb3860300 msgr2=0x557bb3c1a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.902+0000 7f06f89a9740 1 -- 172.21.15.2:0/3301844452 shutdown_connections 2024-05-14T09:59:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.902+0000 7f06f89a9740 1 -- 172.21.15.2:0/3301844452 wait complete. 2024-05-14T09:59:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.902+0000 7f06f89a9740 1 -- start start 2024-05-14T09:59:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.902+0000 7f06f89a9740 1 Processor -- start 2024-05-14T09:59:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.902+0000 7f06f89a9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb38432f0 0x557bb3c23ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.902+0000 7f06f89a9740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb384b4d0 0x557bb3c243e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.902+0000 7f06f89a9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb386c1b0 0x557bb3c25310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.902+0000 7f06f89a9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1a90f0 con 0x557bb38432f0 2024-05-14T09:59:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.902+0000 7f06f89a9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1aa560 con 0x557bb386c1b0 2024-05-14T09:59:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.902+0000 7f06f89a9740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1a97c0 con 0x557bb384b4d0 2024-05-14T09:59:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.902+0000 7f06f89a9740 10 client.0 _ll_register_callbacks cb 0x7f06e00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:59:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.902+0000 7f06eaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb384b4d0 0x557bb3c243e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.902+0000 7f06eaffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb384b4d0 0x557bb3c243e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:53014/0 (socket says 172.21.15.2:53014) 2024-05-14T09:59:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06eaffd640 1 -- 172.21.15.2:0/2032594024 learned_addr learned my addr 172.21.15.2:0/2032594024 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06eb7fe640 1 --2- 172.21.15.2:0/2032594024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb38432f0 0x557bb3c23ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06ebfff640 1 --2- 172.21.15.2:0/2032594024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb386c1b0 0x557bb3c25310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06eaffd640 1 -- 172.21.15.2:0/2032594024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb386c1b0 msgr2=0x557bb3c25310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06eaffd640 1 --2- 172.21.15.2:0/2032594024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb386c1b0 0x557bb3c25310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06eaffd640 1 -- 172.21.15.2:0/2032594024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb38432f0 msgr2=0x557bb3c23ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06eaffd640 1 --2- 172.21.15.2:0/2032594024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb38432f0 0x557bb3c23ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06eaffd640 1 -- 172.21.15.2:0/2032594024 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bb3c258d0 con 0x557bb384b4d0 2024-05-14T09:59:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06eaffd640 1 --2- 172.21.15.2:0/2032594024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb384b4d0 0x557bb3c243e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f06e000e9c0 tx=0x7f06e00e2410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e73bf1608402e15 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06caffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T09:59:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06caffd640 1 -- 172.21.15.2:0/2032594024 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e00e2040 con 0x557bb384b4d0 2024-05-14T09:59:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06f89a9740 1 -- 172.21.15.2:0/2032594024 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e400fe50 con 0x557bb384b4d0 2024-05-14T09:59:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06caffd640 1 -- 172.21.15.2:0/2032594024 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f06e00b2310 con 0x557bb384b4d0 2024-05-14T09:59:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06caffd640 1 -- 172.21.15.2:0/2032594024 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f06e00b0e60 con 0x557bb384b4d0 2024-05-14T09:59:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06f89a9740 10 client.27235 fetch_fsmap learned FSMap version 67 2024-05-14T09:59:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.903+0000 7f06f89a9740 1 -- 172.21.15.2:0/2032594024 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x557bb3c26bd0 con 0x557bb384b4d0 2024-05-14T09:59:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.904+0000 7f06caffd640 1 -- 172.21.15.2:0/2032594024 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06e400fe50 con 0x557bb384b4d0 2024-05-14T09:59:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.904+0000 7f06caffd640 1 -- 172.21.15.2:0/2032594024 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f06e00beb80 con 0x557bb384b4d0 2024-05-14T09:59:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.904+0000 7f06f89a9740 10 client.27235 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:59:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.904+0000 7f06f89a9740 10 client.27235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:59:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.904+0000 7f06f89a9740 20 client.27235 populate_metadata read hostname 'smithi002' 2024-05-14T09:59:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.904+0000 7f06f89a9740 1 --2- 172.21.15.2:0/2032594024 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557bb38001c0 0x557bb3c98380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.904+0000 7f06f89a9740 4 client.27235 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:59:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.904+0000 7f06f89a9740 1 -- 172.21.15.2:0/2032594024 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557bb3b7a050 con 0x557bb38001c0 2024-05-14T09:59:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.904+0000 7f06eb7fe640 1 --2- 172.21.15.2:0/2032594024 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557bb38001c0 0x557bb3c98380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.905+0000 7f06eb7fe640 1 --2- 172.21.15.2:0/2032594024 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557bb38001c0 0x557bb3c98380 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3071444ef6b2e0d3 server_cookie=21d6c3b2ae398f1b in_seq=0 out_seq=0 2024-05-14T09:59:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:42.905+0000 7f06caffd640 10 client.27235 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:42.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:42 smithi029 ceph-mon[36249]: pgmap v1169: 129 pgs: 129 active+clean; 720 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.7 MiB/s wr, 598 op/s 2024-05-14T09:59:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:42 smithi026 ceph-mon[36056]: pgmap v1169: 129 pgs: 129 active+clean; 720 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.7 MiB/s wr, 598 op/s 2024-05-14T09:59:43.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:42 smithi002 ceph-mon[31557]: pgmap v1169: 129 pgs: 129 active+clean; 720 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.7 MiB/s wr, 598 op/s 2024-05-14T09:59:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.582+0000 7f06caffd640 1 -- 172.21.15.2:0/2032594024 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557bb3b7a050 con 0x557bb38001c0 2024-05-14T09:59:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.582+0000 7f06caffd640 10 client.27235 handle_command_reply: tid=0 2024-05-14T09:59:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.582+0000 7f06f89a9740 1 client.27235 shutdown 2024-05-14T09:59:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.582+0000 7f06f89a9740 1 -- 172.21.15.2:0/2032594024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb384b4d0 msgr2=0x557bb3c243e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.582+0000 7f06f89a9740 1 --2- 172.21.15.2:0/2032594024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb384b4d0 0x557bb3c243e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f06e000e9c0 tx=0x7f06e00e2410 comp rx=0 tx=0).stop 2024-05-14T09:59:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.582+0000 7f06f89a9740 1 -- 172.21.15.2:0/2032594024 shutdown_connections 2024-05-14T09:59:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.582+0000 7f06f89a9740 1 --2- 172.21.15.2:0/2032594024 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x557bb38001c0 0x557bb3c98380 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.582+0000 7f06f89a9740 1 --2- 172.21.15.2:0/2032594024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557bb386c1b0 0x557bb3c25310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.582+0000 7f06f89a9740 1 --2- 172.21.15.2:0/2032594024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x557bb384b4d0 0x557bb3c243e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.582+0000 7f06caffd640 0 client.27235 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.582+0000 7f06f89a9740 1 --2- 172.21.15.2:0/2032594024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557bb38432f0 0x557bb3c23ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.582+0000 7f06f89a9740 1 -- 172.21.15.2:0/2032594024 >> 172.21.15.2:0/2032594024 conn(0x557bb3860300 msgr2=0x557bb3c08cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.582+0000 7f06f89a9740 1 -- 172.21.15.2:0/2032594024 shutdown_connections 2024-05-14T09:59:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.582+0000 7f06f89a9740 1 -- 172.21.15.2:0/2032594024 wait complete. 2024-05-14T09:59:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.582+0000 7f06f89a9740 20 client.27235 trim_cache size 0 max 16384 2024-05-14T09:59:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:59:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:59:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:59:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:59:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.583+0000 7f06f1af9640 1 -- 172.21.15.2:0/758340930 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f06d40b2920 msgr2=0x7f06d40b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.583+0000 7f06f1af9640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f06d40b2920 0x7f06d40b4e30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f06dc031500 tx=0x7f06dc0046c0 comp rx=0 tx=0).stop 2024-05-14T09:59:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.583+0000 7f06f1af9640 1 -- 172.21.15.2:0/758340930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06ec103d90 msgr2=0x7f06ec199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.583+0000 7f06f1af9640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06ec103d90 0x7f06ec199730 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f06e000e9f0 tx=0x7f06e000eeb0 comp rx=0 tx=0).stop 2024-05-14T09:59:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.583+0000 7f06f1af9640 1 -- 172.21.15.2:0/758340930 shutdown_connections 2024-05-14T09:59:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.583+0000 7f06f1af9640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f06d40b2920 0x7f06d40b4e30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.583+0000 7f06f1af9640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f06ec108a90 0x7f06ec06a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.583+0000 7f06f1af9640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06ec103d90 0x7f06ec199730 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.583+0000 7f06f1af9640 1 --2- 172.21.15.2:0/758340930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec102b50 0x7f06ec1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.583+0000 7f06f1af9640 1 -- 172.21.15.2:0/758340930 >> 172.21.15.2:0/758340930 conn(0x7f06ec0fdd40 msgr2=0x7f06ec104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.583+0000 7f06f1af9640 1 -- 172.21.15.2:0/758340930 shutdown_connections 2024-05-14T09:59:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:43.583+0000 7f06f1af9640 1 -- 172.21.15.2:0/758340930 wait complete. 2024-05-14T09:59:43.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 325759 25.45 MB/sec execute 549 sec latency 466.200 ms 2024-05-14T09:59:44.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 326238 25.46 MB/sec execute 550 sec latency 106.095 ms 2024-05-14T09:59:44.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:44 smithi029 ceph-mon[36249]: pgmap v1170: 129 pgs: 129 active+clean; 724 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.1 MiB/s wr, 584 op/s 2024-05-14T09:59:44.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:44 smithi026 ceph-mon[36056]: pgmap v1170: 129 pgs: 129 active+clean; 724 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.1 MiB/s wr, 584 op/s 2024-05-14T09:59:45.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:44 smithi002 ceph-mon[31557]: pgmap v1170: 129 pgs: 129 active+clean; 724 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.1 MiB/s wr, 584 op/s 2024-05-14T09:59:45.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 326493 25.44 MB/sec execute 551 sec latency 264.594 ms 2024-05-14T09:59:46.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 326795 25.43 MB/sec execute 552 sec latency 432.472 ms 2024-05-14T09:59:46.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:46 smithi026 ceph-mon[36056]: pgmap v1171: 129 pgs: 129 active+clean; 724 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.4 MiB/s wr, 426 op/s 2024-05-14T09:59:47.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:46 smithi002 ceph-mon[31557]: pgmap v1171: 129 pgs: 129 active+clean; 724 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.4 MiB/s wr, 426 op/s 2024-05-14T09:59:47.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:46 smithi029 ceph-mon[36249]: pgmap v1171: 129 pgs: 129 active+clean; 724 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.4 MiB/s wr, 426 op/s 2024-05-14T09:59:47.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 327133 25.42 MB/sec execute 553 sec latency 101.194 ms 2024-05-14T09:59:48.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 327708 25.41 MB/sec execute 554 sec latency 104.157 ms 2024-05-14T09:59:48.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:48 smithi026 ceph-mon[36056]: pgmap v1172: 129 pgs: 129 active+clean; 710 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.2 MiB/s wr, 576 op/s 2024-05-14T09:59:49.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:48 smithi002 ceph-mon[31557]: pgmap v1172: 129 pgs: 129 active+clean; 710 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.2 MiB/s wr, 576 op/s 2024-05-14T09:59:49.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:48 smithi029 ceph-mon[36249]: pgmap v1172: 129 pgs: 129 active+clean; 710 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.2 MiB/s wr, 576 op/s 2024-05-14T09:59:49.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 328170 25.42 MB/sec execute 555 sec latency 126.158 ms 2024-05-14T09:59:50.628 INFO:tasks.workunit.client.0.smithi002.stdout: 10 328617 25.42 MB/sec execute 556 sec latency 140.527 ms 2024-05-14T09:59:50.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:50 smithi026 ceph-mon[36056]: pgmap v1173: 129 pgs: 129 active+clean; 692 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-05-14T09:59:51.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:50 smithi002 ceph-mon[31557]: pgmap v1173: 129 pgs: 129 active+clean; 692 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-05-14T09:59:51.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:50 smithi029 ceph-mon[36249]: pgmap v1173: 129 pgs: 129 active+clean; 692 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-05-14T09:59:51.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 328885 25.39 MB/sec execute 557 sec latency 415.524 ms 2024-05-14T09:59:52.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 329098 25.36 MB/sec execute 558 sec latency 319.320 ms 2024-05-14T09:59:52.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:52 smithi026 ceph-mon[36056]: pgmap v1174: 129 pgs: 129 active+clean; 682 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 595 op/s 2024-05-14T09:59:53.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:52 smithi002 ceph-mon[31557]: pgmap v1174: 129 pgs: 129 active+clean; 682 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 595 op/s 2024-05-14T09:59:53.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:52 smithi029 ceph-mon[36249]: pgmap v1174: 129 pgs: 129 active+clean; 682 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 595 op/s 2024-05-14T09:59:53.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 329565 25.36 MB/sec execute 559 sec latency 107.584 ms 2024-05-14T09:59:54.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 330177 25.38 MB/sec execute 560 sec latency 103.650 ms 2024-05-14T09:59:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:54 smithi026 ceph-mon[36056]: pgmap v1175: 129 pgs: 129 active+clean; 683 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 580 op/s 2024-05-14T09:59:55.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:54 smithi002 ceph-mon[31557]: pgmap v1175: 129 pgs: 129 active+clean; 683 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 580 op/s 2024-05-14T09:59:55.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:54 smithi029 ceph-mon[36249]: pgmap v1175: 129 pgs: 129 active+clean; 683 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 580 op/s 2024-05-14T09:59:55.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 330633 25.38 MB/sec execute 561 sec latency 333.301 ms 2024-05-14T09:59:56.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 330836 25.35 MB/sec execute 562 sec latency 638.372 ms 2024-05-14T09:59:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.782+0000 7fb75754d640 1 -- 172.21.15.2:0/894181742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb750103d90 msgr2=0x7fb7501041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.782+0000 7fb75754d640 1 --2- 172.21.15.2:0/894181742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb750103d90 0x7fb7501041a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb748009ca0 tx=0x7fb74802a8d0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.785+0000 7fb75754d640 1 -- 172.21.15.2:0/894181742 shutdown_connections 2024-05-14T09:59:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.785+0000 7fb75754d640 1 --2- 172.21.15.2:0/894181742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb750108a90 0x7fb75010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.785+0000 7fb75754d640 1 --2- 172.21.15.2:0/894181742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb750103d90 0x7fb7501041a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.785+0000 7fb75754d640 1 --2- 172.21.15.2:0/894181742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb750102b50 0x7fb750103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.785+0000 7fb75754d640 1 -- 172.21.15.2:0/894181742 >> 172.21.15.2:0/894181742 conn(0x7fb7500fdd40 msgr2=0x7fb7501001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.788+0000 7fb75754d640 1 -- 172.21.15.2:0/894181742 shutdown_connections 2024-05-14T09:59:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.788+0000 7fb75754d640 1 -- 172.21.15.2:0/894181742 wait complete. 2024-05-14T09:59:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.789+0000 7fb75754d640 1 Processor -- start 2024-05-14T09:59:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.789+0000 7fb75754d640 1 -- start start 2024-05-14T09:59:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.789+0000 7fb75754d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb750102b50 0x7fb750113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.789+0000 7fb75754d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb750103d90 0x7fb750113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.789+0000 7fb75754d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb750108a90 0x7fb750112120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.789+0000 7fb75754d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb75010c490 con 0x7fb750103d90 2024-05-14T09:59:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.789+0000 7fb75754d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb75010c310 con 0x7fb750108a90 2024-05-14T09:59:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.789+0000 7fb75754d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb75010c610 con 0x7fb750102b50 2024-05-14T09:59:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.790+0000 7fb7552e9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb750102b50 0x7fb750113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.790+0000 7fb7552e9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb750102b50 0x7fb750113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:55962/0 (socket says 172.21.15.2:55962) 2024-05-14T09:59:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.790+0000 7fb7552e9640 1 -- 172.21.15.2:0/1618376815 learned_addr learned my addr 172.21.15.2:0/1618376815 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.790+0000 7fb7552e9640 1 -- 172.21.15.2:0/1618376815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb750108a90 msgr2=0x7fb750112120 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T09:59:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.790+0000 7fb7552e9640 1 --2- 172.21.15.2:0/1618376815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb750108a90 0x7fb750112120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.790+0000 7fb7552e9640 1 -- 172.21.15.2:0/1618376815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb750103d90 msgr2=0x7fb750113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.790+0000 7fb7552e9640 1 --2- 172.21.15.2:0/1618376815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb750103d90 0x7fb750113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.790+0000 7fb7552e9640 1 -- 172.21.15.2:0/1618376815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7501128b0 con 0x7fb750102b50 2024-05-14T09:59:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.791+0000 7fb7552e9640 1 --2- 172.21.15.2:0/1618376815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb750102b50 0x7fb750113970 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00c9c0 tx=0x7fb74c00ce80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.794+0000 7fb7467fc640 1 -- 172.21.15.2:0/1618376815 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c007cd0 con 0x7fb750102b50 2024-05-14T09:59:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.795+0000 7fb75754d640 1 -- 172.21.15.2:0/1618376815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb750112ba0 con 0x7fb750102b50 2024-05-14T09:59:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.795+0000 7fb75754d640 1 -- 172.21.15.2:0/1618376815 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7501b1510 con 0x7fb750102b50 2024-05-14T09:59:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.795+0000 7fb7467fc640 1 -- 172.21.15.2:0/1618376815 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb74c00f900 con 0x7fb750102b50 2024-05-14T09:59:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.795+0000 7fb7467fc640 1 -- 172.21.15.2:0/1618376815 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c018840 con 0x7fb750102b50 2024-05-14T09:59:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.799+0000 7fb75e403740 1 Processor -- start 2024-05-14T09:59:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.803+0000 7fb7467fc640 1 -- 172.21.15.2:0/1618376815 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fb74c00f450 con 0x7fb750102b50 2024-05-14T09:59:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.808+0000 7fb7467fc640 1 --2- 172.21.15.2:0/1618376815 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb73c0b29f0 0x7fb73c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.808+0000 7fb7467fc640 1 -- 172.21.15.2:0/1618376815 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fb74c0e2850 con 0x7fb750102b50 2024-05-14T09:59:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.809+0000 7fb75e403740 1 -- start start 2024-05-14T09:59:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.809+0000 7fb75e403740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34b4ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.809+0000 7fb75e403740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f347822f0 0x556f34b4cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.809+0000 7fb75e403740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f3478a4d0 0x556f34b53ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.809+0000 7fb75e403740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556f34ab15d0 con 0x556f347ab1b0 2024-05-14T09:59:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.809+0000 7fb75e403740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556f34a6a890 con 0x556f347822f0 2024-05-14T09:59:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.809+0000 7fb75e403740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556f34aad590 con 0x556f3478a4d0 2024-05-14T09:59:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.811+0000 7fb755aea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f3478a4d0 0x556f34b53ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.811+0000 7fb755aea640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f3478a4d0 0x556f34b53ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:55966/0 (socket says 172.21.15.2:55966) 2024-05-14T09:59:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.811+0000 7fb755aea640 1 -- 172.21.15.2:0/982217092 learned_addr learned my addr 172.21.15.2:0/982217092 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.811+0000 7fb755aea640 1 --2- 172.21.15.2:0/1618376815 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb73c0b29f0 0x7fb73c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:56.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.814+0000 7fb754ae8640 1 --2- 172.21.15.2:0/982217092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f347822f0 0x556f34b4cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:56.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.814+0000 7fb7552e9640 1 --2- 172.21.15.2:0/982217092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34b4ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.816+0000 7fb754ae8640 1 -- 172.21.15.2:0/982217092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f3478a4d0 msgr2=0x556f34b53ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.816+0000 7fb754ae8640 1 --2- 172.21.15.2:0/982217092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f3478a4d0 0x556f34b53ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.816+0000 7fb754ae8640 1 -- 172.21.15.2:0/982217092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 msgr2=0x556f34b4ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.816+0000 7fb754ae8640 1 --2- 172.21.15.2:0/982217092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34b4ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.817+0000 7fb754ae8640 1 -- 172.21.15.2:0/982217092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f347c3210 con 0x556f347822f0 2024-05-14T09:59:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.818+0000 7fb754ae8640 1 --2- 172.21.15.2:0/982217092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f347822f0 0x556f34b4cfe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb748009bf0 tx=0x7fb74802f3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc17da399394977d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:56.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.822+0000 7fb755aea640 1 --2- 172.21.15.2:0/1618376815 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb73c0b29f0 0x7fb73c0b4f00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb740008fd0 tx=0x7fb7400068d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:56.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.823+0000 7fb73b7fe640 1 -- 172.21.15.2:0/982217092 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748031460 con 0x556f347822f0 2024-05-14T09:59:56.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.823+0000 7fb73b7fe640 1 -- 172.21.15.2:0/982217092 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb74803f070 con 0x556f347822f0 2024-05-14T09:59:56.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.823+0000 7fb73b7fe640 1 -- 172.21.15.2:0/982217092 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7480044c0 con 0x556f347822f0 2024-05-14T09:59:56.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.824+0000 7fb75e403740 1 -- 172.21.15.2:0/982217092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f347822f0 msgr2=0x556f34b4cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:56.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.824+0000 7fb75e403740 1 --2- 172.21.15.2:0/982217092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f347822f0 0x556f34b4cfe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb748009bf0 tx=0x7fb74802f3d0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.826+0000 7fb75e403740 1 -- 172.21.15.2:0/982217092 shutdown_connections 2024-05-14T09:59:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.826+0000 7fb75e403740 1 --2- 172.21.15.2:0/982217092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f3478a4d0 0x556f34b53ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.826+0000 7fb75e403740 1 --2- 172.21.15.2:0/982217092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f347822f0 0x556f34b4cfe0 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb748009bf0 tx=0x7fb74802f3d0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.826+0000 7fb75e403740 1 --2- 172.21.15.2:0/982217092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34b4ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.826+0000 7fb75e403740 1 -- 172.21.15.2:0/982217092 >> 172.21.15.2:0/982217092 conn(0x556f3479f300 msgr2=0x556f34b483a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.833+0000 7fb75e403740 1 -- 172.21.15.2:0/982217092 shutdown_connections 2024-05-14T09:59:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.834+0000 7fb75e403740 1 -- 172.21.15.2:0/982217092 wait complete. 2024-05-14T09:59:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.835+0000 7fb75e403740 1 -- start start 2024-05-14T09:59:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.835+0000 7fb75e403740 1 Processor -- start 2024-05-14T09:59:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.838+0000 7fb75e403740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.838+0000 7fb75e403740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34b67d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.838+0000 7fb75e403740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f34769d80 0x556f34b6ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.838+0000 7fb75e403740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556f34b658e0 con 0x556f347ab1b0 2024-05-14T09:59:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.838+0000 7fb75e403740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556f34a6e370 con 0x556f3478a4d0 2024-05-14T09:59:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.838+0000 7fb75e403740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556f34b65a60 con 0x556f34769d80 2024-05-14T09:59:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.838+0000 7fb75e403740 10 client.0 _ll_register_callbacks cb 0x556f347c2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:59:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.839+0000 7fb754ae8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34b67d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.839+0000 7fb754ae8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34b67d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42180/0 (socket says 172.21.15.2:42180) 2024-05-14T09:59:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.839+0000 7fb754ae8640 1 -- 172.21.15.2:0/2245307428 learned_addr learned my addr 172.21.15.2:0/2245307428 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.839+0000 7fb7552e9640 1 --2- 172.21.15.2:0/2245307428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.841+0000 7fb755aea640 1 --2- 172.21.15.2:0/2245307428 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f34769d80 0x556f34b6ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.843+0000 7fb754ae8640 1 -- 172.21.15.2:0/2245307428 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f34769d80 msgr2=0x556f34b6ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.843+0000 7fb754ae8640 1 --2- 172.21.15.2:0/2245307428 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f34769d80 0x556f34b6ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.843+0000 7fb754ae8640 1 -- 172.21.15.2:0/2245307428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 msgr2=0x556f34b677d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.843+0000 7fb754ae8640 1 --2- 172.21.15.2:0/2245307428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b677d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.843+0000 7fb754ae8640 1 -- 172.21.15.2:0/2245307428 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f3488f620 con 0x556f347ab1b0 2024-05-14T09:59:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.846+0000 7fb754ae8640 1 --2- 172.21.15.2:0/2245307428 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34b67d10 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x556f34b4de40 tx=0x7fb748031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3364d708f5307b9e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.847+0000 7fb738ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T09:59:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.850+0000 7fb738ff9640 1 -- 172.21.15.2:0/2245307428 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748003c00 con 0x556f347ab1b0 2024-05-14T09:59:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.850+0000 7fb738ff9640 1 -- 172.21.15.2:0/2245307428 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fb748003da0 con 0x556f347ab1b0 2024-05-14T09:59:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.851+0000 7fb75e403740 1 -- 172.21.15.2:0/2245307428 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb74803f070 con 0x556f347ab1b0 2024-05-14T09:59:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.855+0000 7fb738ff9640 1 -- 172.21.15.2:0/2245307428 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fb7480041c0 con 0x556f347ab1b0 2024-05-14T09:59:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.860+0000 7fb75e403740 10 client.17643 fetch_fsmap learned FSMap version 67 2024-05-14T09:59:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.860+0000 7fb75e403740 1 -- 172.21.15.2:0/2245307428 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x556f34b71ba0 con 0x556f347ab1b0 2024-05-14T09:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.863+0000 7fb738ff9640 1 -- 172.21.15.2:0/2245307428 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb74803f070 con 0x556f347ab1b0 2024-05-14T09:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.863+0000 7fb738ff9640 1 -- 172.21.15.2:0/2245307428 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fb74802fd10 con 0x556f347ab1b0 2024-05-14T09:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.865+0000 7fb75e403740 10 client.17643 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.865+0000 7fb75e403740 10 client.17643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.865+0000 7fb75e403740 20 client.17643 populate_metadata read hostname 'smithi002' 2024-05-14T09:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.865+0000 7fb75e403740 1 --2- 172.21.15.2:0/2245307428 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556f3473f1c0 0x556f34b74f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.865+0000 7fb75e403740 4 client.17643 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T09:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.865+0000 7fb75e403740 1 -- 172.21.15.2:0/2245307428 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f34abf1e0 con 0x556f3473f1c0 2024-05-14T09:59:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.867+0000 7fb7552e9640 1 --2- 172.21.15.2:0/2245307428 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556f3473f1c0 0x556f34b74f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.876+0000 7fb7552e9640 1 --2- 172.21.15.2:0/2245307428 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556f3473f1c0 0x556f34b74f00 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1705a0ba8909c9c server_cookie=496a9ea9c803eb66 in_seq=0 out_seq=0 2024-05-14T09:59:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.876+0000 7fb738ff9640 10 client.17643 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.881+0000 7fb738ff9640 1 -- 172.21.15.2:0/2245307428 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f34abf1e0 con 0x556f3473f1c0 2024-05-14T09:59:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.881+0000 7fb738ff9640 10 client.17643 handle_command_reply: tid=0 2024-05-14T09:59:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.882+0000 7fb75e403740 1 client.17643 shutdown 2024-05-14T09:59:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.887+0000 7fb75e403740 1 -- 172.21.15.2:0/2245307428 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 msgr2=0x556f34b67d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.887+0000 7fb75e403740 1 --2- 172.21.15.2:0/2245307428 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34b67d10 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x556f34b4de40 tx=0x7fb748031890 comp rx=0 tx=0).stop 2024-05-14T09:59:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.892+0000 7fb75e403740 1 -- 172.21.15.2:0/2245307428 shutdown_connections 2024-05-14T09:59:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.892+0000 7fb75e403740 1 --2- 172.21.15.2:0/2245307428 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556f3473f1c0 0x556f34b74f00 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.892+0000 7fb75e403740 1 --2- 172.21.15.2:0/2245307428 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f34769d80 0x556f34b6ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.892+0000 7fb75e403740 1 --2- 172.21.15.2:0/2245307428 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34b67d10 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.892+0000 7fb75e403740 1 --2- 172.21.15.2:0/2245307428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b677d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.892+0000 7fb75e403740 1 -- 172.21.15.2:0/2245307428 >> 172.21.15.2:0/2245307428 conn(0x556f3479f300 msgr2=0x556f34b4b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.893+0000 7fb738ff9640 0 client.17643 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.895+0000 7fb75e403740 1 -- 172.21.15.2:0/2245307428 shutdown_connections 2024-05-14T09:59:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.896+0000 7fb75e403740 1 -- 172.21.15.2:0/2245307428 wait complete. 2024-05-14T09:59:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.896+0000 7fb75e403740 20 client.17643 trim_cache size 0 max 16384 2024-05-14T09:59:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.927+0000 7fb75e403740 1 Processor -- start 2024-05-14T09:59:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.927+0000 7fb75e403740 1 -- start start 2024-05-14T09:59:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.928+0000 7fb75e403740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f34769d80 0x556f34b66a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.928+0000 7fb75e403740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b66fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.928+0000 7fb75e403740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34b47d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.928+0000 7fb75e403740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb7340127b0 con 0x556f347ab1b0 2024-05-14T09:59:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.928+0000 7fb75e403740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb734010080 con 0x556f3478a4d0 2024-05-14T09:59:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.928+0000 7fb75e403740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb73400ded0 con 0x556f34769d80 2024-05-14T09:59:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.928+0000 7fb754ae8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b66fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.928+0000 7fb754ae8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b66fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:48260/0 (socket says 172.21.15.2:48260) 2024-05-14T09:59:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.928+0000 7fb754ae8640 1 -- 172.21.15.2:0/58431833 learned_addr learned my addr 172.21.15.2:0/58431833 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.928+0000 7fb7552e9640 1 --2- 172.21.15.2:0/58431833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f34769d80 0x556f34b66a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.929+0000 7fb754ae8640 1 -- 172.21.15.2:0/58431833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f34769d80 msgr2=0x556f34b66a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.929+0000 7fb754ae8640 1 --2- 172.21.15.2:0/58431833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f34769d80 0x556f34b66a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.929+0000 7fb754ae8640 1 -- 172.21.15.2:0/58431833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 msgr2=0x556f34b47d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.929+0000 7fb754ae8640 1 --2- 172.21.15.2:0/58431833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34b47d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.929+0000 7fb754ae8640 1 -- 172.21.15.2:0/58431833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f34b48d00 con 0x556f3478a4d0 2024-05-14T09:59:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.932+0000 7fb754ae8640 1 --2- 172.21.15.2:0/58431833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b66fa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb748030740 tx=0x7fb7480095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36353672e6695f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.933+0000 7fb73affd640 1 -- 172.21.15.2:0/58431833 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74805d070 con 0x556f3478a4d0 2024-05-14T09:59:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.933+0000 7fb73affd640 1 -- 172.21.15.2:0/58431833 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb74803f070 con 0x556f3478a4d0 2024-05-14T09:59:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.934+0000 7fb75e403740 1 -- 172.21.15.2:0/58431833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 msgr2=0x556f34b66fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.934+0000 7fb75e403740 1 --2- 172.21.15.2:0/58431833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b66fa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb748030740 tx=0x7fb7480095c0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.937+0000 7fb75e403740 1 -- 172.21.15.2:0/58431833 shutdown_connections 2024-05-14T09:59:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.937+0000 7fb75e403740 1 --2- 172.21.15.2:0/58431833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34b47d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.937+0000 7fb75e403740 1 --2- 172.21.15.2:0/58431833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b66fa0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.937+0000 7fb75e403740 1 --2- 172.21.15.2:0/58431833 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f34769d80 0x556f34b66a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.937+0000 7fb75e403740 1 -- 172.21.15.2:0/58431833 >> 172.21.15.2:0/58431833 conn(0x556f3479f300 msgr2=0x556f34b50bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.941+0000 7fb75e403740 1 -- 172.21.15.2:0/58431833 shutdown_connections 2024-05-14T09:59:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.943+0000 7fb75e403740 1 -- 172.21.15.2:0/58431833 wait complete. 2024-05-14T09:59:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.943+0000 7fb75e403740 1 -- start start 2024-05-14T09:59:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.944+0000 7fb75e403740 1 Processor -- start 2024-05-14T09:59:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.944+0000 7fb75e403740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f34769d80 0x556f34b6b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.944+0000 7fb75e403740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b6b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.944+0000 7fb75e403740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34bdb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.944+0000 7fb75e403740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb73400e540 con 0x556f347ab1b0 2024-05-14T09:59:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.944+0000 7fb75e403740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb73400f9b0 con 0x556f3478a4d0 2024-05-14T09:59:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.944+0000 7fb75e403740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb73400ec10 con 0x556f34769d80 2024-05-14T09:59:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.944+0000 7fb75e403740 10 client.0 _ll_register_callbacks cb 0x7fb74c0a97c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T09:59:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.949+0000 7fb754ae8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b6b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.949+0000 7fb7552e9640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f34769d80 0x556f34b6b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.949+0000 7fb754ae8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b6b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:48276/0 (socket says 172.21.15.2:48276) 2024-05-14T09:59:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.949+0000 7fb754ae8640 1 -- 172.21.15.2:0/3321328145 learned_addr learned my addr 172.21.15.2:0/3321328145 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T09:59:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.949+0000 7fb755aea640 1 --2- 172.21.15.2:0/3321328145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34bdb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.950+0000 7fb754ae8640 1 -- 172.21.15.2:0/3321328145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f34769d80 msgr2=0x556f34b6b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.950+0000 7fb754ae8640 1 --2- 172.21.15.2:0/3321328145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f34769d80 0x556f34b6b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.950+0000 7fb754ae8640 1 -- 172.21.15.2:0/3321328145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 msgr2=0x556f34bdb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T09:59:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.950+0000 7fb754ae8640 1 --2- 172.21.15.2:0/3321328145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34bdb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.950+0000 7fb754ae8640 1 -- 172.21.15.2:0/3321328145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f34bdb720 con 0x556f3478a4d0 2024-05-14T09:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.952+0000 7fb754ae8640 1 --2- 172.21.15.2:0/3321328145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b6b870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb7480305d0 tx=0x7fb74802f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a6b9f189587e01c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T09:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.953+0000 7fb738ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T09:59:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.953+0000 7fb738ff9640 1 -- 172.21.15.2:0/3321328145 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74805d070 con 0x556f3478a4d0 2024-05-14T09:59:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.953+0000 7fb738ff9640 1 -- 172.21.15.2:0/3321328145 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fb74802fbd0 con 0x556f3478a4d0 2024-05-14T09:59:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.954+0000 7fb75e403740 1 -- 172.21.15.2:0/3321328145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb74803f070 con 0x556f3478a4d0 2024-05-14T09:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.960+0000 7fb738ff9640 1 -- 172.21.15.2:0/3321328145 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fb748004820 con 0x556f3478a4d0 2024-05-14T09:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.964+0000 7fb75e403740 10 client.27257 fetch_fsmap learned FSMap version 67 2024-05-14T09:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.964+0000 7fb75e403740 1 -- 172.21.15.2:0/3321328145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x556f34bdc950 con 0x556f3478a4d0 2024-05-14T09:59:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.968+0000 7fb738ff9640 1 -- 172.21.15.2:0/3321328145 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb74803f070 con 0x556f3478a4d0 2024-05-14T09:59:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.968+0000 7fb738ff9640 1 -- 172.21.15.2:0/3321328145 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fb748044920 con 0x556f3478a4d0 2024-05-14T09:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.970+0000 7fb75e403740 10 client.27257 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T09:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.970+0000 7fb75e403740 10 client.27257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T09:59:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.970+0000 7fb75e403740 20 client.27257 populate_metadata read hostname 'smithi002' 2024-05-14T09:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.970+0000 7fb75e403740 1 --2- 172.21.15.2:0/3321328145 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556f34bdd280 0x556f34bdf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T09:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.970+0000 7fb75e403740 4 client.27257 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T09:59:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.970+0000 7fb75e403740 1 -- 172.21.15.2:0/3321328145 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f34ab9050 con 0x556f34bdd280 2024-05-14T09:59:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.974+0000 7fb7552e9640 1 --2- 172.21.15.2:0/3321328145 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556f34bdd280 0x556f34bdf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T09:59:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.982+0000 7fb7552e9640 1 --2- 172.21.15.2:0/3321328145 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556f34bdd280 0x556f34bdf6a0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fcf2d4ff613cde0 server_cookie=d20d89eca5814bd0 in_seq=0 out_seq=0 2024-05-14T09:59:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:56.985+0000 7fb738ff9640 10 client.27257 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:57.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:56 smithi002 ceph-mon[31557]: pgmap v1176: 129 pgs: 129 active+clean; 686 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.7 MiB/s wr, 513 op/s 2024-05-14T09:59:57.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:56 smithi029 ceph-mon[36249]: pgmap v1176: 129 pgs: 129 active+clean; 686 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.7 MiB/s wr, 513 op/s 2024-05-14T09:59:57.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:56 smithi026 ceph-mon[36056]: pgmap v1176: 129 pgs: 129 active+clean; 686 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.7 MiB/s wr, 513 op/s 2024-05-14T09:59:57.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 330961 25.31 MB/sec execute 563 sec latency 466.694 ms 2024-05-14T09:59:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.934+0000 7fb738ff9640 1 -- 172.21.15.2:0/3321328145 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f34ab9050 con 0x556f34bdd280 2024-05-14T09:59:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.934+0000 7fb738ff9640 10 client.27257 handle_command_reply: tid=0 2024-05-14T09:59:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.934+0000 7fb75e403740 1 client.27257 shutdown 2024-05-14T09:59:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.935+0000 7fb75e403740 1 -- 172.21.15.2:0/3321328145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 msgr2=0x556f34b6b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.935+0000 7fb75e403740 1 --2- 172.21.15.2:0/3321328145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b6b870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb7480305d0 tx=0x7fb74802f850 comp rx=0 tx=0).stop 2024-05-14T09:59:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.935+0000 7fb75e403740 1 -- 172.21.15.2:0/3321328145 shutdown_connections 2024-05-14T09:59:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.935+0000 7fb75e403740 1 --2- 172.21.15.2:0/3321328145 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556f34bdd280 0x556f34bdf6a0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.935+0000 7fb75e403740 1 --2- 172.21.15.2:0/3321328145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f347ab1b0 0x556f34bdb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.935+0000 7fb75e403740 1 --2- 172.21.15.2:0/3321328145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f3478a4d0 0x556f34b6b870 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.935+0000 7fb75e403740 1 --2- 172.21.15.2:0/3321328145 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556f34769d80 0x556f34b6b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.935+0000 7fb75e403740 1 -- 172.21.15.2:0/3321328145 >> 172.21.15.2:0/3321328145 conn(0x556f3479f300 msgr2=0x556f34b66a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.935+0000 7fb738ff9640 0 client.27257 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T09:59:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.935+0000 7fb75e403740 1 -- 172.21.15.2:0/3321328145 shutdown_connections 2024-05-14T09:59:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.935+0000 7fb75e403740 1 -- 172.21.15.2:0/3321328145 wait complete. 2024-05-14T09:59:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.935+0000 7fb75e403740 20 client.27257 trim_cache size 0 max 16384 2024-05-14T09:59:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T09:59:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T09:59:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T09:59:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T09:59:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.936+0000 7fb75754d640 1 -- 172.21.15.2:0/1618376815 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb73c0b29f0 msgr2=0x7fb73c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.936+0000 7fb75754d640 1 --2- 172.21.15.2:0/1618376815 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb73c0b29f0 0x7fb73c0b4f00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb740008fd0 tx=0x7fb7400068d0 comp rx=0 tx=0).stop 2024-05-14T09:59:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.936+0000 7fb75754d640 1 -- 172.21.15.2:0/1618376815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb750102b50 msgr2=0x7fb750113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T09:59:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.936+0000 7fb75754d640 1 --2- 172.21.15.2:0/1618376815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb750102b50 0x7fb750113970 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00c9c0 tx=0x7fb74c00ce80 comp rx=0 tx=0).stop 2024-05-14T09:59:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.936+0000 7fb75754d640 1 -- 172.21.15.2:0/1618376815 shutdown_connections 2024-05-14T09:59:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.936+0000 7fb75754d640 1 --2- 172.21.15.2:0/1618376815 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb73c0b29f0 0x7fb73c0b4f00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.936+0000 7fb75754d640 1 --2- 172.21.15.2:0/1618376815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb750108a90 0x7fb750112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.936+0000 7fb75754d640 1 --2- 172.21.15.2:0/1618376815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb750103d90 0x7fb750113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.936+0000 7fb75754d640 1 --2- 172.21.15.2:0/1618376815 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb750102b50 0x7fb750113970 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T09:59:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.936+0000 7fb75754d640 1 -- 172.21.15.2:0/1618376815 >> 172.21.15.2:0/1618376815 conn(0x7fb7500fdd40 msgr2=0x7fb750104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T09:59:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.936+0000 7fb75754d640 1 -- 172.21.15.2:0/1618376815 shutdown_connections 2024-05-14T09:59:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T09:59:57.936+0000 7fb75754d640 1 -- 172.21.15.2:0/1618376815 wait complete. 2024-05-14T09:59:58.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 331410 25.32 MB/sec execute 564 sec latency 581.509 ms 2024-05-14T09:59:59.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 09:59:58 smithi002 ceph-mon[31557]: pgmap v1177: 129 pgs: 129 active+clean; 690 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.3 MiB/s wr, 598 op/s 2024-05-14T09:59:59.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 09:59:58 smithi029 ceph-mon[36249]: pgmap v1177: 129 pgs: 129 active+clean; 690 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.3 MiB/s wr, 598 op/s 2024-05-14T09:59:59.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 09:59:58 smithi026 ceph-mon[36056]: pgmap v1177: 129 pgs: 129 active+clean; 690 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.3 MiB/s wr, 598 op/s 2024-05-14T09:59:59.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 332021 25.35 MB/sec execute 565 sec latency 104.159 ms 2024-05-14T10:00:00.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a[31553]: 2024-05-14T09:59:59.999+0000 7f82afa15700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-14T10:00:00.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 332520 25.34 MB/sec execute 566 sec latency 88.432 ms 2024-05-14T10:00:01.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:00 smithi002 ceph-mon[31557]: pgmap v1178: 129 pgs: 129 active+clean; 690 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.5 MiB/s wr, 539 op/s 2024-05-14T10:00:01.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:00 smithi002 ceph-mon[31557]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-14T10:00:01.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:00 smithi029 ceph-mon[36249]: pgmap v1178: 129 pgs: 129 active+clean; 690 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.5 MiB/s wr, 539 op/s 2024-05-14T10:00:01.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:00 smithi029 ceph-mon[36249]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-14T10:00:01.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:00 smithi026 ceph-mon[36056]: pgmap v1178: 129 pgs: 129 active+clean; 690 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.5 MiB/s wr, 539 op/s 2024-05-14T10:00:01.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:00 smithi026 ceph-mon[36056]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-14T10:00:01.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 332695 25.31 MB/sec execute 567 sec latency 437.210 ms 2024-05-14T10:00:02.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 332963 25.28 MB/sec execute 568 sec latency 160.592 ms 2024-05-14T10:00:03.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:02 smithi002 ceph-mon[31557]: pgmap v1179: 129 pgs: 129 active+clean; 682 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 559 op/s 2024-05-14T10:00:03.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:02 smithi029 ceph-mon[36249]: pgmap v1179: 129 pgs: 129 active+clean; 682 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 559 op/s 2024-05-14T10:00:03.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:02 smithi026 ceph-mon[36056]: pgmap v1179: 129 pgs: 129 active+clean; 682 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 559 op/s 2024-05-14T10:00:03.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 333653 25.31 MB/sec execute 569 sec latency 56.187 ms 2024-05-14T10:00:04.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 334345 25.32 MB/sec execute 570 sec latency 128.690 ms 2024-05-14T10:00:05.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:04 smithi002 ceph-mon[31557]: pgmap v1180: 129 pgs: 129 active+clean; 686 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.8 MiB/s wr, 561 op/s 2024-05-14T10:00:05.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:04 smithi029 ceph-mon[36249]: pgmap v1180: 129 pgs: 129 active+clean; 686 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.8 MiB/s wr, 561 op/s 2024-05-14T10:00:05.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:04 smithi026 ceph-mon[36056]: pgmap v1180: 129 pgs: 129 active+clean; 686 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.8 MiB/s wr, 561 op/s 2024-05-14T10:00:05.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 334886 25.34 MB/sec execute 571 sec latency 136.649 ms 2024-05-14T10:00:06.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 335185 25.32 MB/sec execute 572 sec latency 629.544 ms 2024-05-14T10:00:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:06 smithi002 ceph-mon[31557]: pgmap v1181: 129 pgs: 129 active+clean; 686 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.7 MiB/s wr, 498 op/s 2024-05-14T10:00:07.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:06 smithi029 ceph-mon[36249]: pgmap v1181: 129 pgs: 129 active+clean; 686 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.7 MiB/s wr, 498 op/s 2024-05-14T10:00:07.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:06 smithi026 ceph-mon[36056]: pgmap v1181: 129 pgs: 129 active+clean; 686 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.7 MiB/s wr, 498 op/s 2024-05-14T10:00:07.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 335449 25.31 MB/sec execute 573 sec latency 131.523 ms 2024-05-14T10:00:08.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 335978 25.31 MB/sec execute 574 sec latency 76.587 ms 2024-05-14T10:00:09.063 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:08 smithi026 ceph-mon[36056]: pgmap v1182: 129 pgs: 129 active+clean; 696 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 595 op/s 2024-05-14T10:00:09.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:08 smithi002 ceph-mon[31557]: pgmap v1182: 129 pgs: 129 active+clean; 696 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 595 op/s 2024-05-14T10:00:09.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:08 smithi029 ceph-mon[36249]: pgmap v1182: 129 pgs: 129 active+clean; 696 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 595 op/s 2024-05-14T10:00:09.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 336339 25.30 MB/sec execute 575 sec latency 116.955 ms 2024-05-14T10:00:10.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 336885 25.30 MB/sec execute 576 sec latency 163.646 ms 2024-05-14T10:00:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.148+0000 7fa284d1f640 1 -- 172.21.15.2:0/3097044083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa280107800 msgr2=0x7fa280107c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.148+0000 7fa284d1f640 1 --2- 172.21.15.2:0/3097044083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa280107800 0x7fa280107c10 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa278009ca0 tx=0x7fa27802a8f0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.151+0000 7fa284d1f640 1 -- 172.21.15.2:0/3097044083 shutdown_connections 2024-05-14T10:00:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.151+0000 7fa284d1f640 1 --2- 172.21.15.2:0/3097044083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa280107800 0x7fa280107c10 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.151+0000 7fa284d1f640 1 --2- 172.21.15.2:0/3097044083 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa280104ea0 0x7fa2801072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.151+0000 7fa284d1f640 1 --2- 172.21.15.2:0/3097044083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa280102540 0x7fa280104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.151+0000 7fa284d1f640 1 -- 172.21.15.2:0/3097044083 >> 172.21.15.2:0/3097044083 conn(0x7fa2800fdd50 msgr2=0x7fa280100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.155+0000 7fa284d1f640 1 -- 172.21.15.2:0/3097044083 shutdown_connections 2024-05-14T10:00:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.156+0000 7fa284d1f640 1 -- 172.21.15.2:0/3097044083 wait complete. 2024-05-14T10:00:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.157+0000 7fa284d1f640 1 Processor -- start 2024-05-14T10:00:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.163+0000 7fa284d1f640 1 -- start start 2024-05-14T10:00:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.163+0000 7fa284d1f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa280102540 0x7fa2800743e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.163+0000 7fa284d1f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa280104ea0 0x7fa280074920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.163+0000 7fa284d1f640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa280107800 0x7fa280072b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.163+0000 7fa284d1f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa28006cf50 con 0x7fa280102540 2024-05-14T10:00:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.163+0000 7fa284d1f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa28006cdd0 con 0x7fa280104ea0 2024-05-14T10:00:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.163+0000 7fa284d1f640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa28006d0d0 con 0x7fa280107800 2024-05-14T10:00:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.167+0000 7fa27e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa280102540 0x7fa2800743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.167+0000 7fa27e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa280102540 0x7fa2800743e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45218/0 (socket says 172.21.15.2:45218) 2024-05-14T10:00:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.167+0000 7fa27e59c640 1 -- 172.21.15.2:0/2453701195 learned_addr learned my addr 172.21.15.2:0/2453701195 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.168+0000 7fa27e59c640 1 -- 172.21.15.2:0/2453701195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa280107800 msgr2=0x7fa280072b40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T10:00:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.168+0000 7fa27e59c640 1 --2- 172.21.15.2:0/2453701195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa280107800 0x7fa280072b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.168+0000 7fa27e59c640 1 -- 172.21.15.2:0/2453701195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa280104ea0 msgr2=0x7fa280074920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T10:00:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.168+0000 7fa27e59c640 1 --2- 172.21.15.2:0/2453701195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa280104ea0 0x7fa280074920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.168+0000 7fa27e59c640 1 -- 172.21.15.2:0/2453701195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa280073360 con 0x7fa280102540 2024-05-14T10:00:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.168+0000 7fa27e59c640 1 --2- 172.21.15.2:0/2453701195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa280102540 0x7fa2800743e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa27400b470 tx=0x7fa27400b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.169+0000 7fa26f7fe640 1 -- 172.21.15.2:0/2453701195 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274015020 con 0x7fa280102540 2024-05-14T10:00:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.169+0000 7fa284d1f640 1 -- 172.21.15.2:0/2453701195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa280073650 con 0x7fa280102540 2024-05-14T10:00:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.169+0000 7fa284d1f640 1 -- 172.21.15.2:0/2453701195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2801b14b0 con 0x7fa280102540 2024-05-14T10:00:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.169+0000 7fa26f7fe640 1 -- 172.21.15.2:0/2453701195 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa274011d40 con 0x7fa280102540 2024-05-14T10:00:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.169+0000 7fa26f7fe640 1 -- 172.21.15.2:0/2453701195 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27400f340 con 0x7fa280102540 2024-05-14T10:00:11.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:10 smithi002 ceph-mon[31557]: pgmap v1183: 129 pgs: 129 active+clean; 697 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.5 MiB/s wr, 549 op/s 2024-05-14T10:00:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.177+0000 7fa28bbd1740 1 Processor -- start 2024-05-14T10:00:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.177+0000 7fa26f7fe640 1 -- 172.21.15.2:0/2453701195 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa274018020 con 0x7fa280102540 2024-05-14T10:00:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.178+0000 7fa26f7fe640 1 --2- 172.21.15.2:0/2453701195 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa2680b2ac0 0x7fa2680b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.178+0000 7fa26f7fe640 1 -- 172.21.15.2:0/2453701195 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fa2740e2070 con 0x7fa280102540 2024-05-14T10:00:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.181+0000 7fa28bbd1740 1 -- start start 2024-05-14T10:00:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.181+0000 7fa28bbd1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a301b0 0x563e18dd1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.181+0000 7fa28bbd1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a072f0 0x563e18dd1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.181+0000 7fa28bbd1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a0f4d0 0x563e18dd8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.181+0000 7fa28bbd1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563e18d365d0 con 0x563e18a0f4d0 2024-05-14T10:00:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.181+0000 7fa28bbd1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563e18cef890 con 0x563e18a072f0 2024-05-14T10:00:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.181+0000 7fa28bbd1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563e18d32590 con 0x563e18a301b0 2024-05-14T10:00:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.181+0000 7fa27ed9d640 1 --2- 172.21.15.2:0/2453701195 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa2680b2ac0 0x7fa2680b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.183+0000 7fa27dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a072f0 0x563e18dd1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:11.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:10 smithi029 ceph-mon[36249]: pgmap v1183: 129 pgs: 129 active+clean; 697 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.5 MiB/s wr, 549 op/s 2024-05-14T10:00:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.186+0000 7fa27e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a301b0 0x563e18dd1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.187+0000 7fa27ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a0f4d0 0x563e18dd8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.187+0000 7fa27ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a0f4d0 0x563e18dd8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45220/0 (socket says 172.21.15.2:45220) 2024-05-14T10:00:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.187+0000 7fa27ed9d640 1 -- 172.21.15.2:0/4010469885 learned_addr learned my addr 172.21.15.2:0/4010469885 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.192+0000 7fa27e59c640 1 -- 172.21.15.2:0/4010469885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a072f0 msgr2=0x563e18dd1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.192+0000 7fa27e59c640 1 --2- 172.21.15.2:0/4010469885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a072f0 0x563e18dd1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.192+0000 7fa27e59c640 1 -- 172.21.15.2:0/4010469885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a0f4d0 msgr2=0x563e18dd8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.192+0000 7fa27e59c640 1 --2- 172.21.15.2:0/4010469885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a0f4d0 0x563e18dd8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.192+0000 7fa27e59c640 1 -- 172.21.15.2:0/4010469885 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e18a48210 con 0x563e18a301b0 2024-05-14T10:00:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.195+0000 7fa27ed9d640 1 --2- 172.21.15.2:0/4010469885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a0f4d0 0x563e18dd8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T10:00:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.195+0000 7fa27e59c640 1 --2- 172.21.15.2:0/4010469885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a301b0 0x563e18dd1a60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa2740df6f0 tx=0x7fa2740a9110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5887cbf1949ee9b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.198+0000 7fa27ed9d640 1 --2- 172.21.15.2:0/2453701195 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa2680b2ac0 0x7fa2680b4fd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa278009bf0 tx=0x7fa27802f450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.199+0000 7fa26cff9640 1 -- 172.21.15.2:0/4010469885 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2740ad5c0 con 0x563e18a301b0 2024-05-14T10:00:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.199+0000 7fa26cff9640 1 -- 172.21.15.2:0/4010469885 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2740b2070 con 0x563e18a301b0 2024-05-14T10:00:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.199+0000 7fa26cff9640 1 -- 172.21.15.2:0/4010469885 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2740dfbb0 con 0x563e18a301b0 2024-05-14T10:00:11.203 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:10 smithi026 ceph-mon[36056]: pgmap v1183: 129 pgs: 129 active+clean; 697 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.5 MiB/s wr, 549 op/s 2024-05-14T10:00:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.203+0000 7fa28bbd1740 1 -- 172.21.15.2:0/4010469885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a301b0 msgr2=0x563e18dd1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.203+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/4010469885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a301b0 0x563e18dd1a60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa2740df6f0 tx=0x7fa2740a9110 comp rx=0 tx=0).stop 2024-05-14T10:00:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.207+0000 7fa28bbd1740 1 -- 172.21.15.2:0/4010469885 shutdown_connections 2024-05-14T10:00:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.207+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/4010469885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a0f4d0 0x563e18dd8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.207+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/4010469885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a072f0 0x563e18dd1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.207+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/4010469885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a301b0 0x563e18dd1a60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.207+0000 7fa28bbd1740 1 -- 172.21.15.2:0/4010469885 >> 172.21.15.2:0/4010469885 conn(0x563e18a24300 msgr2=0x563e18dcd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.211+0000 7fa28bbd1740 1 -- 172.21.15.2:0/4010469885 shutdown_connections 2024-05-14T10:00:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.215+0000 7fa28bbd1740 1 -- 172.21.15.2:0/4010469885 wait complete. 2024-05-14T10:00:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.215+0000 7fa28bbd1740 1 -- start start 2024-05-14T10:00:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.215+0000 7fa28bbd1740 1 Processor -- start 2024-05-14T10:00:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.219+0000 7fa28bbd1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a072f0 0x563e18dec770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.219+0000 7fa28bbd1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a0f4d0 0x563e18deccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.219+0000 7fa28bbd1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a301b0 0x563e18df3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.219+0000 7fa28bbd1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563e18dea940 con 0x563e18a0f4d0 2024-05-14T10:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.219+0000 7fa28bbd1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563e18cf3370 con 0x563e18a301b0 2024-05-14T10:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.219+0000 7fa28bbd1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x563e18deaac0 con 0x563e18a072f0 2024-05-14T10:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.219+0000 7fa28bbd1740 10 client.0 _ll_register_callbacks cb 0x563e18a47a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T10:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.220+0000 7fa27dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a0f4d0 0x563e18deccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.220+0000 7fa27dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a0f4d0 0x563e18deccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45230/0 (socket says 172.21.15.2:45230) 2024-05-14T10:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.220+0000 7fa27dd9b640 1 -- 172.21.15.2:0/1115011529 learned_addr learned my addr 172.21.15.2:0/1115011529 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.222+0000 7fa27e59c640 1 --2- 172.21.15.2:0/1115011529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a072f0 0x563e18dec770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.222+0000 7fa27ed9d640 1 --2- 172.21.15.2:0/1115011529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a301b0 0x563e18df3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.225+0000 7fa27e59c640 1 -- 172.21.15.2:0/1115011529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a301b0 msgr2=0x563e18df3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.225+0000 7fa27e59c640 1 --2- 172.21.15.2:0/1115011529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a301b0 0x563e18df3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.225+0000 7fa27e59c640 1 -- 172.21.15.2:0/1115011529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a0f4d0 msgr2=0x563e18deccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.225+0000 7fa27e59c640 1 --2- 172.21.15.2:0/1115011529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a0f4d0 0x563e18deccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.225+0000 7fa27e59c640 1 -- 172.21.15.2:0/1115011529 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e18b14620 con 0x563e18a072f0 2024-05-14T10:00:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.229+0000 7fa27dd9b640 1 --2- 172.21.15.2:0/1115011529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a0f4d0 0x563e18deccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T10:00:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.229+0000 7fa27e59c640 1 --2- 172.21.15.2:0/1115011529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a072f0 0x563e18dec770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa2740a91a0 tx=0x7fa2740a5200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cadc238249d12a26 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.230+0000 7fa265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T10:00:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.233+0000 7fa265ffb640 1 -- 172.21.15.2:0/1115011529 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2740c7020 con 0x563e18a072f0 2024-05-14T10:00:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.233+0000 7fa265ffb640 1 -- 172.21.15.2:0/1115011529 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fa274011aa0 con 0x563e18a072f0 2024-05-14T10:00:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.234+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1115011529 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2740b2070 con 0x563e18a072f0 2024-05-14T10:00:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.239+0000 7fa265ffb640 1 -- 172.21.15.2:0/1115011529 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fa2740ba050 con 0x563e18a072f0 2024-05-14T10:00:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.241+0000 7fa28bbd1740 10 client.27256 fetch_fsmap learned FSMap version 67 2024-05-14T10:00:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.241+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1115011529 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x563e18dea940 con 0x563e18a072f0 2024-05-14T10:00:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.247+0000 7fa265ffb640 1 -- 172.21.15.2:0/1115011529 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2740b2070 con 0x563e18a072f0 2024-05-14T10:00:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.248+0000 7fa265ffb640 1 -- 172.21.15.2:0/1115011529 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fa2740a4b80 con 0x563e18a072f0 2024-05-14T10:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.250+0000 7fa28bbd1740 10 client.27256 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T10:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.250+0000 7fa28bbd1740 10 client.27256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T10:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.250+0000 7fa28bbd1740 20 client.27256 populate_metadata read hostname 'smithi002' 2024-05-14T10:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.250+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/1115011529 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e189eed80 0x563e18df9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.250+0000 7fa28bbd1740 4 client.27256 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T10:00:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.250+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1115011529 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e18d441e0 con 0x563e189eed80 2024-05-14T10:00:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.254+0000 7fa27dd9b640 1 --2- 172.21.15.2:0/1115011529 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e189eed80 0x563e18df9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.279+0000 7fa27dd9b640 1 --2- 172.21.15.2:0/1115011529 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e189eed80 0x563e18df9b10 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c12f68fb1a702034 server_cookie=7082422a2d8459ca in_seq=0 out_seq=0 2024-05-14T10:00:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.282+0000 7fa265ffb640 10 client.27256 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T10:00:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.288+0000 7fa265ffb640 1 -- 172.21.15.2:0/1115011529 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563e18d441e0 con 0x563e189eed80 2024-05-14T10:00:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.288+0000 7fa265ffb640 10 client.27256 handle_command_reply: tid=0 2024-05-14T10:00:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.289+0000 7fa28bbd1740 1 client.27256 shutdown 2024-05-14T10:00:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.295+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1115011529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a072f0 msgr2=0x563e18dec770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.295+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/1115011529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a072f0 0x563e18dec770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa2740a91a0 tx=0x7fa2740a5200 comp rx=0 tx=0).stop 2024-05-14T10:00:11.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.301+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1115011529 shutdown_connections 2024-05-14T10:00:11.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.301+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/1115011529 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e189eed80 0x563e18df9b10 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.301+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/1115011529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a301b0 0x563e18df3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.301+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/1115011529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a0f4d0 0x563e18deccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.301+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/1115011529 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a072f0 0x563e18dec770 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.301+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1115011529 >> 172.21.15.2:0/1115011529 conn(0x563e18a24300 msgr2=0x563e18dcc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.301+0000 7fa265ffb640 0 client.27256 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T10:00:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.304+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1115011529 shutdown_connections 2024-05-14T10:00:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.304+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1115011529 wait complete. 2024-05-14T10:00:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.306+0000 7fa28bbd1740 20 client.27256 trim_cache size 0 max 16384 2024-05-14T10:00:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.348+0000 7fa28bbd1740 1 Processor -- start 2024-05-14T10:00:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.349+0000 7fa28bbd1740 1 -- start start 2024-05-14T10:00:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.349+0000 7fa28bbd1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a072f0 0x563e18dcd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.349+0000 7fa28bbd1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a0f4d0 0x563e18dd01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.349+0000 7fa28bbd1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a301b0 0x563e18dd0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.349+0000 7fa28bbd1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa2600125d0 con 0x563e18a072f0 2024-05-14T10:00:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.349+0000 7fa28bbd1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa26000fea0 con 0x563e18a0f4d0 2024-05-14T10:00:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.349+0000 7fa28bbd1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa26000dda0 con 0x563e18a301b0 2024-05-14T10:00:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.351+0000 7fa27dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a0f4d0 0x563e18dd01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.351+0000 7fa27dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a0f4d0 0x563e18dd01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:55378/0 (socket says 172.21.15.2:55378) 2024-05-14T10:00:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.351+0000 7fa27dd9b640 1 -- 172.21.15.2:0/274038671 learned_addr learned my addr 172.21.15.2:0/274038671 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.355+0000 7fa27ed9d640 1 --2- 172.21.15.2:0/274038671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a301b0 0x563e18dd0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.355+0000 7fa27dd9b640 1 -- 172.21.15.2:0/274038671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a301b0 msgr2=0x563e18dd0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.355+0000 7fa27dd9b640 1 --2- 172.21.15.2:0/274038671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a301b0 0x563e18dd0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.355+0000 7fa27dd9b640 1 -- 172.21.15.2:0/274038671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a072f0 msgr2=0x563e18dcd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.355+0000 7fa27dd9b640 1 --2- 172.21.15.2:0/274038671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a072f0 0x563e18dcd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.355+0000 7fa27dd9b640 1 -- 172.21.15.2:0/274038671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e18dd8f30 con 0x563e18a0f4d0 2024-05-14T10:00:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.356+0000 7fa27dd9b640 1 --2- 172.21.15.2:0/274038671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a0f4d0 0x563e18dd01f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa270000c00 tx=0x7fa270003d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87e0b66f4451e001 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.359+0000 7fa267fff640 1 -- 172.21.15.2:0/274038671 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27000bc10 con 0x563e18a0f4d0 2024-05-14T10:00:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.359+0000 7fa267fff640 1 -- 172.21.15.2:0/274038671 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa27000bdb0 con 0x563e18a0f4d0 2024-05-14T10:00:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.359+0000 7fa267fff640 1 -- 172.21.15.2:0/274038671 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27001c790 con 0x563e18a0f4d0 2024-05-14T10:00:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.360+0000 7fa28bbd1740 1 -- 172.21.15.2:0/274038671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a0f4d0 msgr2=0x563e18dd01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.360+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/274038671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a0f4d0 0x563e18dd01f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa270000c00 tx=0x7fa270003d90 comp rx=0 tx=0).stop 2024-05-14T10:00:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.360+0000 7fa28bbd1740 1 -- 172.21.15.2:0/274038671 shutdown_connections 2024-05-14T10:00:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.360+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/274038671 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a301b0 0x563e18dd0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.360+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/274038671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a0f4d0 0x563e18dd01f0 secure :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa270000c00 tx=0x7fa270003d90 comp rx=0 tx=0).stop 2024-05-14T10:00:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.360+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/274038671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e18a072f0 0x563e18dcd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.360+0000 7fa28bbd1740 1 -- 172.21.15.2:0/274038671 >> 172.21.15.2:0/274038671 conn(0x563e18a24300 msgr2=0x563e18dccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.364+0000 7fa28bbd1740 1 -- 172.21.15.2:0/274038671 shutdown_connections 2024-05-14T10:00:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.364+0000 7fa28bbd1740 1 -- 172.21.15.2:0/274038671 wait complete. 2024-05-14T10:00:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.364+0000 7fa28bbd1740 1 -- start start 2024-05-14T10:00:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.365+0000 7fa28bbd1740 1 Processor -- start 2024-05-14T10:00:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.365+0000 7fa28bbd1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a072f0 0x563e18dee9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.365+0000 7fa28bbd1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a301b0 0x563e18deef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.366+0000 7fa27e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a072f0 0x563e18dee9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.366+0000 7fa27e59c640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a072f0 0x563e18dee9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:44678/0 (socket says 172.21.15.2:44678) 2024-05-14T10:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.366+0000 7fa27e59c640 1 -- 172.21.15.2:0/1013579052 learned_addr learned my addr 172.21.15.2:0/1013579052 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.366+0000 7fa27dd9b640 1 --2- 172.21.15.2:0/1013579052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a301b0 0x563e18deef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.366+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/1013579052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e189c41c0 0x563e18df1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.366+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1013579052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa26000e360 con 0x563e189c41c0 2024-05-14T10:00:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.366+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1013579052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa26000f7d0 con 0x563e18a301b0 2024-05-14T10:00:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.366+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1013579052 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fa26000ea30 con 0x563e18a072f0 2024-05-14T10:00:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.367+0000 7fa28bbd1740 10 client.0 _ll_register_callbacks cb 0x7fa270005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T10:00:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.367+0000 7fa27ed9d640 1 --2- 172.21.15.2:0/1013579052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e189c41c0 0x563e18df1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.367+0000 7fa27dd9b640 1 -- 172.21.15.2:0/1013579052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a072f0 msgr2=0x563e18dee9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.367+0000 7fa27dd9b640 1 --2- 172.21.15.2:0/1013579052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a072f0 0x563e18dee9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.367+0000 7fa27dd9b640 1 -- 172.21.15.2:0/1013579052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e189c41c0 msgr2=0x563e18df1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.367+0000 7fa27dd9b640 1 --2- 172.21.15.2:0/1013579052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e189c41c0 0x563e18df1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.367+0000 7fa27dd9b640 1 -- 172.21.15.2:0/1013579052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e18df23e0 con 0x563e18a301b0 2024-05-14T10:00:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.368+0000 7fa27dd9b640 1 --2- 172.21.15.2:0/1013579052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a301b0 0x563e18deef30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x563e18dce150 tx=0x7fa27000bd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d5b79236b6ee096 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.368+0000 7fa265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-05-14T10:00:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.368+0000 7fa265ffb640 1 -- 172.21.15.2:0/1013579052 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270034060 con 0x563e18a301b0 2024-05-14T10:00:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.368+0000 7fa265ffb640 1 -- 172.21.15.2:0/1013579052 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fa27001d830 con 0x563e18a301b0 2024-05-14T10:00:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.369+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1013579052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa27000bdb0 con 0x563e18a301b0 2024-05-14T10:00:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.369+0000 7fa265ffb640 1 -- 172.21.15.2:0/1013579052 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fa27001da30 con 0x563e18a301b0 2024-05-14T10:00:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.373+0000 7fa28bbd1740 10 client.27269 fetch_fsmap learned FSMap version 67 2024-05-14T10:00:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.373+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1013579052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x563e18e61e80 con 0x563e18a301b0 2024-05-14T10:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.379+0000 7fa265ffb640 1 -- 172.21.15.2:0/1013579052 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2700118f0 con 0x563e18a301b0 2024-05-14T10:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.379+0000 7fa265ffb640 1 -- 172.21.15.2:0/1013579052 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7fa2700397f0 con 0x563e18a301b0 2024-05-14T10:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.381+0000 7fa28bbd1740 10 client.27269 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T10:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.381+0000 7fa28bbd1740 10 client.27269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T10:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.381+0000 7fa28bbd1740 20 client.27269 populate_metadata read hostname 'smithi002' 2024-05-14T10:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.381+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/1013579052 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e18e62480 0x563e18e648a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.381+0000 7fa28bbd1740 4 client.27269 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T10:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.381+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1013579052 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa27000bdb0 con 0x563e18e62480 2024-05-14T10:00:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.383+0000 7fa27e59c640 1 --2- 172.21.15.2:0/1013579052 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e18e62480 0x563e18e648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.399+0000 7fa27e59c640 1 --2- 172.21.15.2:0/1013579052 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e18e62480 0x563e18e648a0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef24e47801973a20 server_cookie=75b2a17f6e48ae54 in_seq=0 out_seq=0 2024-05-14T10:00:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:11.399+0000 7fa265ffb640 10 client.27269 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T10:00:11.629 INFO:tasks.workunit.client.0.smithi002.stdout: 10 337225 25.29 MB/sec execute 577 sec latency 353.158 ms 2024-05-14T10:00:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.554+0000 7fa265ffb640 1 -- 172.21.15.2:0/1013579052 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fa27000bdb0 con 0x563e18e62480 2024-05-14T10:00:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.554+0000 7fa265ffb640 10 client.27269 handle_command_reply: tid=0 2024-05-14T10:00:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.554+0000 7fa28bbd1740 1 client.27269 shutdown 2024-05-14T10:00:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.554+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1013579052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a301b0 msgr2=0x563e18deef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.554+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/1013579052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a301b0 0x563e18deef30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x563e18dce150 tx=0x7fa27000bd10 comp rx=0 tx=0).stop 2024-05-14T10:00:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.554+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1013579052 shutdown_connections 2024-05-14T10:00:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.554+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/1013579052 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x563e18e62480 0x563e18e648a0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.555+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/1013579052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e189c41c0 0x563e18df1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.555+0000 7fa265ffb640 0 client.27269 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T10:00:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.555+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/1013579052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e18a301b0 0x563e18deef30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.555+0000 7fa28bbd1740 1 --2- 172.21.15.2:0/1013579052 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563e18a072f0 0x563e18dee9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.555+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1013579052 >> 172.21.15.2:0/1013579052 conn(0x563e18a24300 msgr2=0x563e18dcb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.555+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1013579052 shutdown_connections 2024-05-14T10:00:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.555+0000 7fa28bbd1740 1 -- 172.21.15.2:0/1013579052 wait complete. 2024-05-14T10:00:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.555+0000 7fa28bbd1740 20 client.27269 trim_cache size 0 max 16384 2024-05-14T10:00:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T10:00:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T10:00:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T10:00:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T10:00:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.555+0000 7fa284d1f640 1 -- 172.21.15.2:0/2453701195 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa2680b2ac0 msgr2=0x7fa2680b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.555+0000 7fa284d1f640 1 --2- 172.21.15.2:0/2453701195 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa2680b2ac0 0x7fa2680b4fd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa278009bf0 tx=0x7fa27802f450 comp rx=0 tx=0).stop 2024-05-14T10:00:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.555+0000 7fa284d1f640 1 -- 172.21.15.2:0/2453701195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa280102540 msgr2=0x7fa2800743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.555+0000 7fa284d1f640 1 --2- 172.21.15.2:0/2453701195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa280102540 0x7fa2800743e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa27400b470 tx=0x7fa27400b930 comp rx=0 tx=0).stop 2024-05-14T10:00:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.555+0000 7fa284d1f640 1 -- 172.21.15.2:0/2453701195 shutdown_connections 2024-05-14T10:00:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.556+0000 7fa284d1f640 1 --2- 172.21.15.2:0/2453701195 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fa2680b2ac0 0x7fa2680b4fd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.556+0000 7fa284d1f640 1 --2- 172.21.15.2:0/2453701195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa280107800 0x7fa280072b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.556+0000 7fa284d1f640 1 --2- 172.21.15.2:0/2453701195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa280104ea0 0x7fa280074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.556+0000 7fa284d1f640 1 --2- 172.21.15.2:0/2453701195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa280102540 0x7fa2800743e0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.556+0000 7fa284d1f640 1 -- 172.21.15.2:0/2453701195 >> 172.21.15.2:0/2453701195 conn(0x7fa2800fdd50 msgr2=0x7fa280102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.556+0000 7fa284d1f640 1 -- 172.21.15.2:0/2453701195 shutdown_connections 2024-05-14T10:00:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:12.556+0000 7fa284d1f640 1 -- 172.21.15.2:0/2453701195 wait complete. 2024-05-14T10:00:12.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 337346 25.26 MB/sec execute 578 sec latency 647.154 ms 2024-05-14T10:00:13.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:12 smithi002 ceph-mon[31557]: pgmap v1184: 129 pgs: 129 active+clean; 702 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.5 MiB/s wr, 549 op/s 2024-05-14T10:00:13.459 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:12 smithi026 ceph-mon[36056]: pgmap v1184: 129 pgs: 129 active+clean; 702 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.5 MiB/s wr, 549 op/s 2024-05-14T10:00:13.459 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:12 smithi029 ceph-mon[36249]: pgmap v1184: 129 pgs: 129 active+clean; 702 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.5 MiB/s wr, 549 op/s 2024-05-14T10:00:13.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 337847 25.26 MB/sec execute 579 sec latency 120.651 ms 2024-05-14T10:00:14.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 338384 25.28 MB/sec execute 580 sec latency 91.205 ms 2024-05-14T10:00:15.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:14 smithi002 ceph-mon[31557]: pgmap v1185: 129 pgs: 129 active+clean; 698 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.5 MiB/s wr, 563 op/s 2024-05-14T10:00:15.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:14 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T10:00:15.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:14 smithi029 ceph-mon[36249]: pgmap v1185: 129 pgs: 129 active+clean; 698 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.5 MiB/s wr, 563 op/s 2024-05-14T10:00:15.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:14 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T10:00:15.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:14 smithi026 ceph-mon[36056]: pgmap v1185: 129 pgs: 129 active+clean; 698 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.5 MiB/s wr, 563 op/s 2024-05-14T10:00:15.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:14 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T10:00:15.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 338898 25.29 MB/sec execute 581 sec latency 88.244 ms 2024-05-14T10:00:16.167 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:15 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:16.167 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:15 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:16.167 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:15 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:16.167 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:15 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:16.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:15 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:16.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:15 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:16.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:15 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:16.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:15 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:16.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:15 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:16.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:15 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:16.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:15 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:16.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:15 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:16.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 339085 25.26 MB/sec execute 582 sec latency 495.679 ms 2024-05-14T10:00:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:16 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T10:00:16.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:16 smithi026 ceph-mon[36056]: Unable to set osd_memory_target on smithi002 to 185877708: error parsing value: Value '185877708' is below minimum 939524096 2024-05-14T10:00:16.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:16 smithi026 ceph-mon[36056]: pgmap v1186: 129 pgs: 129 active+clean; 698 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 471 op/s 2024-05-14T10:00:17.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:16 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T10:00:17.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:16 smithi002 ceph-mon[31557]: Unable to set osd_memory_target on smithi002 to 185877708: error parsing value: Value '185877708' is below minimum 939524096 2024-05-14T10:00:17.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:16 smithi002 ceph-mon[31557]: pgmap v1186: 129 pgs: 129 active+clean; 698 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 471 op/s 2024-05-14T10:00:17.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:16 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T10:00:17.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:16 smithi029 ceph-mon[36249]: Unable to set osd_memory_target on smithi002 to 185877708: error parsing value: Value '185877708' is below minimum 939524096 2024-05-14T10:00:17.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:16 smithi029 ceph-mon[36249]: pgmap v1186: 129 pgs: 129 active+clean; 698 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 471 op/s 2024-05-14T10:00:17.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 339327 25.23 MB/sec execute 583 sec latency 250.164 ms 2024-05-14T10:00:18.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 339486 25.19 MB/sec execute 584 sec latency 437.051 ms 2024-05-14T10:00:19.058 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:18 smithi029 ceph-mon[36249]: pgmap v1187: 129 pgs: 129 active+clean; 701 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.6 MiB/s wr, 629 op/s 2024-05-14T10:00:19.173 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:18 smithi002 ceph-mon[31557]: pgmap v1187: 129 pgs: 129 active+clean; 701 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.6 MiB/s wr, 629 op/s 2024-05-14T10:00:19.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:18 smithi026 ceph-mon[36056]: pgmap v1187: 129 pgs: 129 active+clean; 701 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.6 MiB/s wr, 629 op/s 2024-05-14T10:00:19.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 339779 25.18 MB/sec execute 585 sec latency 318.292 ms 2024-05-14T10:00:20.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 340135 25.18 MB/sec execute 586 sec latency 265.551 ms 2024-05-14T10:00:21.059 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:20 smithi029 ceph-mon[36249]: pgmap v1188: 129 pgs: 129 active+clean; 704 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.8 MiB/s wr, 547 op/s 2024-05-14T10:00:21.088 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:20 smithi026 ceph-mon[36056]: pgmap v1188: 129 pgs: 129 active+clean; 704 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.8 MiB/s wr, 547 op/s 2024-05-14T10:00:21.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:20 smithi002 ceph-mon[31557]: pgmap v1188: 129 pgs: 129 active+clean; 704 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.8 MiB/s wr, 547 op/s 2024-05-14T10:00:21.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 340340 25.15 MB/sec execute 587 sec latency 498.207 ms 2024-05-14T10:00:22.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 340570 25.12 MB/sec execute 588 sec latency 677.564 ms 2024-05-14T10:00:23.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:22 smithi002 ceph-mon[31557]: pgmap v1189: 129 pgs: 129 active+clean; 704 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.4 MiB/s wr, 557 op/s 2024-05-14T10:00:23.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:22 smithi029 ceph-mon[36249]: pgmap v1189: 129 pgs: 129 active+clean; 704 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.4 MiB/s wr, 557 op/s 2024-05-14T10:00:23.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:22 smithi026 ceph-mon[36056]: pgmap v1189: 129 pgs: 129 active+clean; 704 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.4 MiB/s wr, 557 op/s 2024-05-14T10:00:23.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 340801 25.09 MB/sec execute 589 sec latency 360.795 ms 2024-05-14T10:00:24.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 341282 25.10 MB/sec execute 590 sec latency 229.730 ms 2024-05-14T10:00:24.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:24 smithi026 ceph-mon[36056]: pgmap v1190: 129 pgs: 129 active+clean; 708 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.2 MiB/s wr, 505 op/s 2024-05-14T10:00:25.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:24 smithi002 ceph-mon[31557]: pgmap v1190: 129 pgs: 129 active+clean; 708 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.2 MiB/s wr, 505 op/s 2024-05-14T10:00:25.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:24 smithi029 ceph-mon[36249]: pgmap v1190: 129 pgs: 129 active+clean; 708 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.2 MiB/s wr, 505 op/s 2024-05-14T10:00:25.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 341815 25.11 MB/sec execute 591 sec latency 244.423 ms 2024-05-14T10:00:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.695+0000 7f04fdc69640 1 -- 172.21.15.2:0/921704560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04f8108ab0 msgr2=0x7f04f810aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.695+0000 7f04fdc69640 1 --2- 172.21.15.2:0/921704560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04f8108ab0 0x7f04f810aed0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009900 tx=0x7f04f002a410 comp rx=0 tx=0).stop 2024-05-14T10:00:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.696+0000 7f04fdc69640 1 -- 172.21.15.2:0/921704560 shutdown_connections 2024-05-14T10:00:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.696+0000 7f04fdc69640 1 --2- 172.21.15.2:0/921704560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04f8108ab0 0x7f04f810aed0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.696+0000 7f04fdc69640 1 --2- 172.21.15.2:0/921704560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8103db0 0x7f04f81041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.696+0000 7f04fdc69640 1 --2- 172.21.15.2:0/921704560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04f8102b70 0x7f04f8103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.696+0000 7f04fdc69640 1 -- 172.21.15.2:0/921704560 >> 172.21.15.2:0/921704560 conn(0x7f04f80fdd60 msgr2=0x7f04f8100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.696+0000 7f04fdc69640 1 -- 172.21.15.2:0/921704560 shutdown_connections 2024-05-14T10:00:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.696+0000 7f04fdc69640 1 -- 172.21.15.2:0/921704560 wait complete. 2024-05-14T10:00:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.696+0000 7f04fdc69640 1 Processor -- start 2024-05-14T10:00:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.697+0000 7f04fdc69640 1 -- start start 2024-05-14T10:00:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.697+0000 7f04fdc69640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04f8102b70 0x7f04f806bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.697+0000 7f04fdc69640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8103db0 0x7f04f819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.697+0000 7f04fdc69640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04f8108ab0 0x7f04f81a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.697+0000 7f04fdc69640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f04f810e6a0 con 0x7f04f8108ab0 2024-05-14T10:00:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.697+0000 7f04fdc69640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04f810e520 con 0x7f04f8102b70 2024-05-14T10:00:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.697+0000 7f04fdc69640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f04f810e820 con 0x7f04f8103db0 2024-05-14T10:00:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.697+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8103db0 0x7f04f819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.697+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8103db0 0x7f04f819f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:46844/0 (socket says 172.21.15.2:46844) 2024-05-14T10:00:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.697+0000 7f04f6ffd640 1 -- 172.21.15.2:0/1108575505 learned_addr learned my addr 172.21.15.2:0/1108575505 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.697+0000 7f04f7fff640 1 --2- 172.21.15.2:0/1108575505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04f8108ab0 0x7f04f81a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.697+0000 7f04f77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04f8102b70 0x7f04f806bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.698+0000 7f04f6ffd640 1 -- 172.21.15.2:0/1108575505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04f8102b70 msgr2=0x7f04f806bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.698+0000 7f04f6ffd640 1 --2- 172.21.15.2:0/1108575505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04f8102b70 0x7f04f806bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.698+0000 7f04f6ffd640 1 -- 172.21.15.2:0/1108575505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04f8108ab0 msgr2=0x7f04f81a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.698+0000 7f04f6ffd640 1 --2- 172.21.15.2:0/1108575505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04f8108ab0 0x7f04f81a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.698+0000 7f04f6ffd640 1 -- 172.21.15.2:0/1108575505 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f81a1db0 con 0x7f04f8103db0 2024-05-14T10:00:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.698+0000 7f04f7fff640 1 --2- 172.21.15.2:0/1108575505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04f8108ab0 0x7f04f81a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T10:00:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.698+0000 7f04f6ffd640 1 --2- 172.21.15.2:0/1108575505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8103db0 0x7f04f819f020 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f04e000d770 tx=0x7f04e000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.698+0000 7f04f4ff9640 1 -- 172.21.15.2:0/1108575505 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0017070 con 0x7f04f8103db0 2024-05-14T10:00:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.698+0000 7f04fdc69640 1 -- 172.21.15.2:0/1108575505 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f81a20a0 con 0x7f04f8103db0 2024-05-14T10:00:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.698+0000 7f04f4ff9640 1 -- 172.21.15.2:0/1108575505 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04e00054d0 con 0x7f04f8103db0 2024-05-14T10:00:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.698+0000 7f04fdc69640 1 -- 172.21.15.2:0/1108575505 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f81a24e0 con 0x7f04f8103db0 2024-05-14T10:00:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.698+0000 7f04f4ff9640 1 -- 172.21.15.2:0/1108575505 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0004c30 con 0x7f04f8103db0 2024-05-14T10:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.702+0000 7f04f4ff9640 1 -- 172.21.15.2:0/1108575505 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f04e0010460 con 0x7f04f8103db0 2024-05-14T10:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.702+0000 7f0504bf1740 1 Processor -- start 2024-05-14T10:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.702+0000 7f0504bf1740 1 -- start start 2024-05-14T10:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.702+0000 7f0504bf1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565193931b0 0x556519738e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.703+0000 7f0504bf1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55651936a2f0 0x5565197393f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.703+0000 7f0504bf1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193724d0 0x5565197402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.703+0000 7f0504bf1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5565196995d0 con 0x5565193724d0 2024-05-14T10:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.703+0000 7f0504bf1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556519652890 con 0x5565193931b0 2024-05-14T10:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.703+0000 7f0504bf1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556519695590 con 0x55651936a2f0 2024-05-14T10:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.703+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193724d0 0x5565197402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.703+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193724d0 0x5565197402f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54798/0 (socket says 172.21.15.2:54798) 2024-05-14T10:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.703+0000 7f04f6ffd640 1 -- 172.21.15.2:0/1088513668 learned_addr learned my addr 172.21.15.2:0/1088513668 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.703+0000 7f04f7fff640 1 --2- 172.21.15.2:0/1088513668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565193931b0 0x556519738e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.703+0000 7f04f77fe640 1 --2- 172.21.15.2:0/1088513668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55651936a2f0 0x5565197393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.703+0000 7f04f6ffd640 1 -- 172.21.15.2:0/1088513668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55651936a2f0 msgr2=0x5565197393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.703+0000 7f04f6ffd640 1 --2- 172.21.15.2:0/1088513668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55651936a2f0 0x5565197393f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.703+0000 7f04f6ffd640 1 -- 172.21.15.2:0/1088513668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565193931b0 msgr2=0x556519738e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.703+0000 7f04f6ffd640 1 --2- 172.21.15.2:0/1088513668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565193931b0 0x556519738e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.703+0000 7f04f6ffd640 1 -- 172.21.15.2:0/1088513668 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565193ab210 con 0x5565193724d0 2024-05-14T10:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.704+0000 7f04f4ff9640 1 --2- 172.21.15.2:0/1108575505 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f04e40b28a0 0x7f04e40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.704+0000 7f04f7fff640 1 --2- 172.21.15.2:0/1108575505 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f04e40b28a0 0x7f04e40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.704+0000 7f04f6ffd640 1 --2- 172.21.15.2:0/1088513668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193724d0 0x5565197402f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f04e00ea210 tx=0x7f04e00ea6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324fd926e734712e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.704+0000 7f04f4ff9640 1 -- 172.21.15.2:0/1108575505 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f04e00e2020 con 0x7f04f8103db0 2024-05-14T10:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.704+0000 7f04edffb640 1 -- 172.21.15.2:0/1088513668 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e00a5190 con 0x5565193724d0 2024-05-14T10:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.704+0000 7f04edffb640 1 -- 172.21.15.2:0/1088513668 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04e00a8de0 con 0x5565193724d0 2024-05-14T10:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.704+0000 7f04edffb640 1 -- 172.21.15.2:0/1088513668 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e00a8a40 con 0x5565193724d0 2024-05-14T10:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.706+0000 7f0504bf1740 1 -- 172.21.15.2:0/1088513668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193724d0 msgr2=0x5565197402f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.706+0000 7f04f7fff640 1 --2- 172.21.15.2:0/1108575505 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f04e40b28a0 0x7f04e40b4db0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f04f0031440 tx=0x7f04f00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.706+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1088513668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193724d0 0x5565197402f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f04e00ea210 tx=0x7f04e00ea6d0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.706+0000 7f0504bf1740 1 -- 172.21.15.2:0/1088513668 shutdown_connections 2024-05-14T10:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.706+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1088513668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193724d0 0x5565197402f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.706+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1088513668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55651936a2f0 0x5565197393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.706+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1088513668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565193931b0 0x556519738e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.706+0000 7f0504bf1740 1 -- 172.21.15.2:0/1088513668 >> 172.21.15.2:0/1088513668 conn(0x556519387300 msgr2=0x5565197303a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.706+0000 7f0504bf1740 1 -- 172.21.15.2:0/1088513668 shutdown_connections 2024-05-14T10:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.706+0000 7f0504bf1740 1 -- 172.21.15.2:0/1088513668 wait complete. 2024-05-14T10:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.706+0000 7f0504bf1740 1 -- start start 2024-05-14T10:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.707+0000 7f0504bf1740 1 Processor -- start 2024-05-14T10:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.707+0000 7f0504bf1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55651936a2f0 0x556519742a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.707+0000 7f0504bf1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565193724d0 0x556519742f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.707+0000 7f0504bf1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193931b0 0x556519749ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.707+0000 7f0504bf1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556519740ce0 con 0x5565193931b0 2024-05-14T10:00:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.707+0000 7f0504bf1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556519656370 con 0x5565193724d0 2024-05-14T10:00:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.707+0000 7f0504bf1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x556519740e60 con 0x55651936a2f0 2024-05-14T10:00:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.707+0000 7f0504bf1740 10 client.0 _ll_register_callbacks cb 0x5565193aaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T10:00:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.707+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565193724d0 0x556519742f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.707+0000 7f04f77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55651936a2f0 0x556519742a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.707+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565193724d0 0x556519742f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:56854/0 (socket says 172.21.15.2:56854) 2024-05-14T10:00:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.707+0000 7f04f77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55651936a2f0 0x556519742a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:46854/0 (socket says 172.21.15.2:46854) 2024-05-14T10:00:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.707+0000 7f04f6ffd640 1 -- 172.21.15.2:0/1097748303 learned_addr learned my addr 172.21.15.2:0/1097748303 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.707+0000 7f04f7fff640 1 --2- 172.21.15.2:0/1097748303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193931b0 0x556519749ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.707+0000 7f04f77fe640 1 -- 172.21.15.2:0/1097748303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565193724d0 msgr2=0x556519742f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.708+0000 7f04f77fe640 1 --2- 172.21.15.2:0/1097748303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565193724d0 0x556519742f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.708+0000 7f04f77fe640 1 -- 172.21.15.2:0/1097748303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193931b0 msgr2=0x556519749ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.708+0000 7f04f77fe640 1 --2- 172.21.15.2:0/1097748303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193931b0 0x556519749ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.708+0000 7f04f77fe640 1 -- 172.21.15.2:0/1097748303 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556519477620 con 0x55651936a2f0 2024-05-14T10:00:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.708+0000 7f04f6ffd640 1 --2- 172.21.15.2:0/1097748303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565193724d0 0x556519742f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T10:00:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.708+0000 7f04f77fe640 1 --2- 172.21.15.2:0/1097748303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55651936a2f0 0x556519742a20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55651973a250 tx=0x7f04e800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1016e2be5cd1f79 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.708+0000 7f04d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T10:00:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.708+0000 7f04d6ffd640 1 -- 172.21.15.2:0/1097748303 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8012110 con 0x55651936a2f0 2024-05-14T10:00:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.708+0000 7f0504bf1740 1 -- 172.21.15.2:0/1097748303 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04e00a8de0 con 0x55651936a2f0 2024-05-14T10:00:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.708+0000 7f04d6ffd640 1 -- 172.21.15.2:0/1097748303 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f04e8003970 con 0x55651936a2f0 2024-05-14T10:00:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.708+0000 7f04d6ffd640 1 -- 172.21.15.2:0/1097748303 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f04e8025050 con 0x55651936a2f0 2024-05-14T10:00:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.708+0000 7f0504bf1740 10 client.27271 fetch_fsmap learned FSMap version 67 2024-05-14T10:00:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.708+0000 7f0504bf1740 1 -- 172.21.15.2:0/1097748303 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55651974cef0 con 0x55651936a2f0 2024-05-14T10:00:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.709+0000 7f04d6ffd640 1 -- 172.21.15.2:0/1097748303 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04e00a8de0 con 0x55651936a2f0 2024-05-14T10:00:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.709+0000 7f04d6ffd640 1 -- 172.21.15.2:0/1097748303 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f04e8010430 con 0x55651936a2f0 2024-05-14T10:00:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.709+0000 7f0504bf1740 10 client.27271 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T10:00:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.709+0000 7f0504bf1740 10 client.27271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T10:00:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.709+0000 7f0504bf1740 20 client.27271 populate_metadata read hostname 'smithi002' 2024-05-14T10:00:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.709+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1097748303 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556519351d80 0x556519750650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.709+0000 7f0504bf1740 4 client.27271 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T10:00:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.709+0000 7f0504bf1740 1 -- 172.21.15.2:0/1097748303 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565196a71e0 con 0x556519351d80 2024-05-14T10:00:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.710+0000 7f04f6ffd640 1 --2- 172.21.15.2:0/1097748303 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556519351d80 0x556519750650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.714+0000 7f04d6ffd640 10 client.27271 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T10:00:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.714+0000 7f04f6ffd640 1 --2- 172.21.15.2:0/1097748303 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556519351d80 0x556519750650 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf41e0771eeb2c7b server_cookie=26c08eef3cce4922 in_seq=0 out_seq=0 2024-05-14T10:00:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.716+0000 7f04d6ffd640 1 -- 172.21.15.2:0/1097748303 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5565196a71e0 con 0x556519351d80 2024-05-14T10:00:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.716+0000 7f04d6ffd640 10 client.27271 handle_command_reply: tid=0 2024-05-14T10:00:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.716+0000 7f0504bf1740 1 client.27271 shutdown 2024-05-14T10:00:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.716+0000 7f0504bf1740 1 -- 172.21.15.2:0/1097748303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55651936a2f0 msgr2=0x556519742a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.716+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1097748303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55651936a2f0 0x556519742a20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55651973a250 tx=0x7f04e800f790 comp rx=0 tx=0).stop 2024-05-14T10:00:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.716+0000 7f0504bf1740 1 -- 172.21.15.2:0/1097748303 shutdown_connections 2024-05-14T10:00:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.716+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1097748303 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x556519351d80 0x556519750650 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.716+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1097748303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193931b0 0x556519749ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.716+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1097748303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565193724d0 0x556519742f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.716+0000 7f04d6ffd640 0 client.27271 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T10:00:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.716+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1097748303 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55651936a2f0 0x556519742a20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.716+0000 7f0504bf1740 1 -- 172.21.15.2:0/1097748303 >> 172.21.15.2:0/1097748303 conn(0x556519387300 msgr2=0x556519733300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.716+0000 7f0504bf1740 1 -- 172.21.15.2:0/1097748303 shutdown_connections 2024-05-14T10:00:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.717+0000 7f0504bf1740 1 -- 172.21.15.2:0/1097748303 wait complete. 2024-05-14T10:00:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.717+0000 7f0504bf1740 20 client.27271 trim_cache size 0 max 16384 2024-05-14T10:00:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.738+0000 7f0504bf1740 1 Processor -- start 2024-05-14T10:00:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.738+0000 7f0504bf1740 1 -- start start 2024-05-14T10:00:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.738+0000 7f0504bf1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556519351d80 0x5565197304e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.738+0000 7f0504bf1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55651936a2f0 0x556519749de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.738+0000 7f0504bf1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5565193724d0 0x55651974a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.738+0000 7f0504bf1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8012640 con 0x55651936a2f0 2024-05-14T10:00:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.738+0000 7f0504bf1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04d800ff10 con 0x556519351d80 2024-05-14T10:00:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.738+0000 7f0504bf1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f04d800dd60 con 0x5565193724d0 2024-05-14T10:00:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.738+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55651936a2f0 0x556519749de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.738+0000 7f04f6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55651936a2f0 0x556519749de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54828/0 (socket says 172.21.15.2:54828) 2024-05-14T10:00:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.738+0000 7f04f7fff640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5565193724d0 0x55651974a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.738+0000 7f04f6ffd640 1 -- 172.21.15.2:0/3364255291 learned_addr learned my addr 172.21.15.2:0/3364255291 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.738+0000 7f04f77fe640 1 --2- 172.21.15.2:0/3364255291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556519351d80 0x5565197304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.739+0000 7f04f6ffd640 1 -- 172.21.15.2:0/3364255291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5565193724d0 msgr2=0x55651974a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.739+0000 7f04f6ffd640 1 --2- 172.21.15.2:0/3364255291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5565193724d0 0x55651974a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.739+0000 7f04f6ffd640 1 -- 172.21.15.2:0/3364255291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556519351d80 msgr2=0x5565197304e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.739+0000 7f04f6ffd640 1 --2- 172.21.15.2:0/3364255291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556519351d80 0x5565197304e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.739+0000 7f04f6ffd640 1 -- 172.21.15.2:0/3364255291 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55651972a4d0 con 0x55651936a2f0 2024-05-14T10:00:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.739+0000 7f04f7fff640 1 --2- 172.21.15.2:0/3364255291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5565193724d0 0x55651974a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T10:00:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.739+0000 7f04f6ffd640 1 --2- 172.21.15.2:0/3364255291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55651936a2f0 0x556519749de0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f04e00dfa70 tx=0x7f04e00e2320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ea7b4be8e2a0cc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.739+0000 7f04ed7fa640 1 -- 172.21.15.2:0/3364255291 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0020040 con 0x55651936a2f0 2024-05-14T10:00:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.739+0000 7f04ed7fa640 1 -- 172.21.15.2:0/3364255291 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04e00e58d0 con 0x55651936a2f0 2024-05-14T10:00:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.739+0000 7f04ed7fa640 1 -- 172.21.15.2:0/3364255291 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e00a8c20 con 0x55651936a2f0 2024-05-14T10:00:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.740+0000 7f0504bf1740 1 -- 172.21.15.2:0/3364255291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55651936a2f0 msgr2=0x556519749de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.740+0000 7f0504bf1740 1 --2- 172.21.15.2:0/3364255291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55651936a2f0 0x556519749de0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f04e00dfa70 tx=0x7f04e00e2320 comp rx=0 tx=0).stop 2024-05-14T10:00:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.740+0000 7f0504bf1740 1 -- 172.21.15.2:0/3364255291 shutdown_connections 2024-05-14T10:00:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.740+0000 7f0504bf1740 1 --2- 172.21.15.2:0/3364255291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5565193724d0 0x55651974a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.740+0000 7f0504bf1740 1 --2- 172.21.15.2:0/3364255291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55651936a2f0 0x556519749de0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.740+0000 7f0504bf1740 1 --2- 172.21.15.2:0/3364255291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556519351d80 0x5565197304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.740+0000 7f0504bf1740 1 -- 172.21.15.2:0/3364255291 >> 172.21.15.2:0/3364255291 conn(0x556519387300 msgr2=0x55651972fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.740+0000 7f0504bf1740 1 -- 172.21.15.2:0/3364255291 shutdown_connections 2024-05-14T10:00:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.740+0000 7f0504bf1740 1 -- 172.21.15.2:0/3364255291 wait complete. 2024-05-14T10:00:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.740+0000 7f0504bf1740 1 -- start start 2024-05-14T10:00:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.740+0000 7f0504bf1740 1 Processor -- start 2024-05-14T10:00:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f0504bf1740 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556519351d80 0x556519747c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f0504bf1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55651936a2f0 0x556519748150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f0504bf1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193724d0 0x556519749010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f0504bf1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f04d800e3d0 con 0x5565193724d0 2024-05-14T10:00:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f0504bf1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04d800f840 con 0x55651936a2f0 2024-05-14T10:00:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f0504bf1740 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f04d800eaa0 con 0x556519351d80 2024-05-14T10:00:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f0504bf1740 10 client.0 _ll_register_callbacks cb 0x7f04e8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T10:00:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f04f77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556519351d80 0x556519747c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f04f77fe640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556519351d80 0x556519747c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:46874/0 (socket says 172.21.15.2:46874) 2024-05-14T10:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f04f7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193724d0 0x556519749010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f04f77fe640 1 -- 172.21.15.2:0/1794403634 learned_addr learned my addr 172.21.15.2:0/1794403634 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f04f6ffd640 1 --2- 172.21.15.2:0/1794403634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55651936a2f0 0x556519748150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f04f77fe640 1 -- 172.21.15.2:0/1794403634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55651936a2f0 msgr2=0x556519748150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f04f77fe640 1 --2- 172.21.15.2:0/1794403634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55651936a2f0 0x556519748150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f04f77fe640 1 -- 172.21.15.2:0/1794403634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193724d0 msgr2=0x556519749010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T10:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f04f77fe640 1 --2- 172.21.15.2:0/1794403634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193724d0 0x556519749010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.741+0000 7f04f77fe640 1 -- 172.21.15.2:0/1794403634 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565197495d0 con 0x556519351d80 2024-05-14T10:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.742+0000 7f04f77fe640 1 --2- 172.21.15.2:0/1794403634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556519351d80 0x556519747c10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x556519739a50 tx=0x7f04e800fd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4667e6a9051276ab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.742+0000 7f04d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2024-05-14T10:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.742+0000 7f04d6ffd640 1 -- 172.21.15.2:0/1794403634 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8022040 con 0x556519351d80 2024-05-14T10:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.742+0000 7f0504bf1740 1 -- 172.21.15.2:0/1794403634 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04e00e58d0 con 0x556519351d80 2024-05-14T10:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.742+0000 7f04d6ffd640 1 -- 172.21.15.2:0/1794403634 <== mon.1 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f04e8010a70 con 0x556519351d80 2024-05-14T10:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.742+0000 7f04d6ffd640 1 -- 172.21.15.2:0/1794403634 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f04e8036050 con 0x556519351d80 2024-05-14T10:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.742+0000 7f0504bf1740 10 client.27277 fetch_fsmap learned FSMap version 67 2024-05-14T10:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.742+0000 7f0504bf1740 1 -- 172.21.15.2:0/1794403634 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55651974a8d0 con 0x556519351d80 2024-05-14T10:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.742+0000 7f04d6ffd640 1 -- 172.21.15.2:0/1794403634 <== mon.1 v2:172.21.15.29:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04e00e58d0 con 0x556519351d80 2024-05-14T10:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.742+0000 7f04d6ffd640 1 -- 172.21.15.2:0/1794403634 <== mon.1 v2:172.21.15.29:3300/0 5 ==== fsmap(e 67) ==== 8346+0+0 (secure 0 0 0) 0x7f04e8031030 con 0x556519351d80 2024-05-14T10:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.742+0000 7f0504bf1740 10 client.27277 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T10:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.742+0000 7f0504bf1740 10 client.27277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T10:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.742+0000 7f0504bf1740 20 client.27277 populate_metadata read hostname 'smithi002' 2024-05-14T10:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.743+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1794403634 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5565193931b0 0x5565197bb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.743+0000 7f0504bf1740 4 client.27277 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T10:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.743+0000 7f0504bf1740 1 -- 172.21.15.2:0/1794403634 --> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565196a1050 con 0x5565193931b0 2024-05-14T10:00:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.743+0000 7f04f6ffd640 1 --2- 172.21.15.2:0/1794403634 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5565193931b0 0x5565197bb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.743+0000 7f04d6ffd640 10 client.27277 ms_handle_connect on v2:172.21.15.26:6838/903985766 2024-05-14T10:00:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:25.743+0000 7f04f6ffd640 1 --2- 172.21.15.2:0/1794403634 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5565193931b0 0x5565197bb170 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eac94186dac1c2b server_cookie=841a7040b717c731 in_seq=0 out_seq=0 2024-05-14T10:00:26.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 341985 25.10 MB/sec execute 592 sec latency 855.953 ms 2024-05-14T10:00:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.018+0000 7f04d6ffd640 1 -- 172.21.15.2:0/1794403634 <== mds.0 v2:172.21.15.26:6838/903985766 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565196a1050 con 0x5565193931b0 2024-05-14T10:00:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.018+0000 7f04d6ffd640 10 client.27277 handle_command_reply: tid=0 2024-05-14T10:00:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.018+0000 7f0504bf1740 1 client.27277 shutdown 2024-05-14T10:00:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.019+0000 7f0504bf1740 1 -- 172.21.15.2:0/1794403634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556519351d80 msgr2=0x556519747c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.019+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1794403634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556519351d80 0x556519747c10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x556519739a50 tx=0x7f04e800fd00 comp rx=0 tx=0).stop 2024-05-14T10:00:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.019+0000 7f0504bf1740 1 -- 172.21.15.2:0/1794403634 shutdown_connections 2024-05-14T10:00:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.019+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1794403634 >> [v2:172.21.15.26:6838/903985766,v1:172.21.15.26:6839/903985766] conn(0x5565193931b0 0x5565197bb170 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.019+0000 7f04d6ffd640 0 client.27277 ms_handle_reset on v2:172.21.15.26:6838/903985766 2024-05-14T10:00:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.019+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1794403634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565193724d0 0x556519749010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.019+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1794403634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55651936a2f0 0x556519748150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.019+0000 7f0504bf1740 1 --2- 172.21.15.2:0/1794403634 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556519351d80 0x556519747c10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.019+0000 7f0504bf1740 1 -- 172.21.15.2:0/1794403634 >> 172.21.15.2:0/1794403634 conn(0x556519387300 msgr2=0x5565197331f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.019+0000 7f0504bf1740 1 -- 172.21.15.2:0/1794403634 shutdown_connections 2024-05-14T10:00:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.019+0000 7f0504bf1740 1 -- 172.21.15.2:0/1794403634 wait complete. 2024-05-14T10:00:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.019+0000 7f0504bf1740 20 client.27277 trim_cache size 0 max 16384 2024-05-14T10:00:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T10:00:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-05-14T10:00:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T10:00:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T10:00:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.020+0000 7f04fdc69640 1 -- 172.21.15.2:0/1108575505 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f04e40b28a0 msgr2=0x7f04e40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.020+0000 7f04fdc69640 1 --2- 172.21.15.2:0/1108575505 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f04e40b28a0 0x7f04e40b4db0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f04f0031440 tx=0x7f04f00314a0 comp rx=0 tx=0).stop 2024-05-14T10:00:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.020+0000 7f04fdc69640 1 -- 172.21.15.2:0/1108575505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8103db0 msgr2=0x7f04f819f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.020+0000 7f04fdc69640 1 --2- 172.21.15.2:0/1108575505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8103db0 0x7f04f819f020 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f04e000d770 tx=0x7f04e000dc30 comp rx=0 tx=0).stop 2024-05-14T10:00:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.020+0000 7f04fdc69640 1 -- 172.21.15.2:0/1108575505 shutdown_connections 2024-05-14T10:00:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.020+0000 7f04fdc69640 1 --2- 172.21.15.2:0/1108575505 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f04e40b28a0 0x7f04e40b4db0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.020+0000 7f04fdc69640 1 --2- 172.21.15.2:0/1108575505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04f8108ab0 0x7f04f81a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.020+0000 7f04fdc69640 1 --2- 172.21.15.2:0/1108575505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f04f8103db0 0x7f04f819f020 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.020+0000 7f04fdc69640 1 --2- 172.21.15.2:0/1108575505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04f8102b70 0x7f04f806bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.020+0000 7f04fdc69640 1 -- 172.21.15.2:0/1108575505 >> 172.21.15.2:0/1108575505 conn(0x7f04f80fdd60 msgr2=0x7f04f8104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.021+0000 7f04fdc69640 1 -- 172.21.15.2:0/1108575505 shutdown_connections 2024-05-14T10:00:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T10:00:27.021+0000 7f04fdc69640 1 -- 172.21.15.2:0/1108575505 wait complete. 2024-05-14T10:00:27.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:26 smithi002 ceph-mon[31557]: pgmap v1191: 129 pgs: 129 active+clean; 708 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 388 op/s 2024-05-14T10:00:27.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:26 smithi029 ceph-mon[36249]: pgmap v1191: 129 pgs: 129 active+clean; 708 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 388 op/s 2024-05-14T10:00:27.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:26 smithi026 ceph-mon[36056]: pgmap v1191: 129 pgs: 129 active+clean; 708 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 388 op/s 2024-05-14T10:00:27.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 342156 25.07 MB/sec execute 593 sec latency 1044.592 ms 2024-05-14T10:00:28.632 INFO:tasks.workunit.client.0.smithi002.stdout: 10 342382 25.05 MB/sec execute 594 sec latency 184.185 ms 2024-05-14T10:00:28.974 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:28 smithi002 ceph-mon[31557]: pgmap v1192: 129 pgs: 129 active+clean; 709 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.1 MiB/s wr, 463 op/s 2024-05-14T10:00:29.190 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:28 smithi029 ceph-mon[36249]: pgmap v1192: 129 pgs: 129 active+clean; 709 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.1 MiB/s wr, 463 op/s 2024-05-14T10:00:29.198 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:28 smithi026 ceph-mon[36056]: pgmap v1192: 129 pgs: 129 active+clean; 709 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.1 MiB/s wr, 463 op/s 2024-05-14T10:00:29.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 342751 25.04 MB/sec execute 595 sec latency 177.741 ms 2024-05-14T10:00:30.630 INFO:tasks.workunit.client.0.smithi002.stdout: 10 343140 25.03 MB/sec execute 596 sec latency 190.973 ms 2024-05-14T10:00:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:00:30.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:00:30.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:30.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:30.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:30.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:30 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:30.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:30 smithi002 ceph-mon[31557]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-05-14T10:00:30.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:30 smithi002 ceph-mon[31557]: Unable to set osd_memory_target on smithi026 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-14T10:00:30.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:30 smithi002 ceph-mon[31557]: pgmap v1193: 129 pgs: 129 active+clean; 711 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 471 op/s 2024-05-14T10:00:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:00:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:00:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:30 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:30 smithi029 ceph-mon[36249]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-05-14T10:00:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:30 smithi029 ceph-mon[36249]: Unable to set osd_memory_target on smithi026 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-14T10:00:30.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:30 smithi029 ceph-mon[36249]: pgmap v1193: 129 pgs: 129 active+clean; 711 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 471 op/s 2024-05-14T10:00:30.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:00:30.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:00:30.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:30.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:30.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:30.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:30 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T10:00:30.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:30 smithi026 ceph-mon[36056]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-05-14T10:00:30.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:30 smithi026 ceph-mon[36056]: Unable to set osd_memory_target on smithi026 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-14T10:00:30.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:30 smithi026 ceph-mon[36056]: pgmap v1193: 129 pgs: 129 active+clean; 711 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 471 op/s 2024-05-14T10:00:31.631 INFO:tasks.workunit.client.0.smithi002.stdout: 10 343588 25.01 MB/sec execute 597 sec latency 215.406 ms 2024-05-14T10:00:31.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:31 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:00:31.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:31 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:00:31.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:31 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T10:00:31.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:31 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T10:00:31.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:31 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:00:31.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:31 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:00:31.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:31 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:00:31.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:31 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T10:00:31.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:31 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T10:00:31.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:31 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:00:31.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:31 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:00:31.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:31 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:00:31.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:31 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T10:00:31.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:31 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T10:00:31.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:31 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:00:32.631 INFO:tasks.workunit.client.0.smithi002.stdout: 10 343827 24.99 MB/sec execute 598 sec latency 416.977 ms 2024-05-14T10:00:32.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:32 smithi002 ceph-mon[31557]: pgmap v1194: 129 pgs: 129 active+clean; 707 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 414 op/s 2024-05-14T10:00:32.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:32 smithi029 ceph-mon[36249]: pgmap v1194: 129 pgs: 129 active+clean; 707 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 414 op/s 2024-05-14T10:00:32.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:32 smithi026 ceph-mon[36056]: pgmap v1194: 129 pgs: 129 active+clean; 707 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 414 op/s 2024-05-14T10:00:33.631 INFO:tasks.workunit.client.0.smithi002.stdout: 10 344281 24.99 MB/sec execute 599 sec latency 146.360 ms 2024-05-14T10:00:34.631 INFO:tasks.workunit.client.0.smithi002.stdout: 10 cleanup 600 sec 2024-05-14T10:00:34.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:34 smithi002 ceph-mon[31557]: pgmap v1195: 129 pgs: 129 active+clean; 715 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.3 MiB/s wr, 528 op/s 2024-05-14T10:00:34.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:34 smithi029 ceph-mon[36249]: pgmap v1195: 129 pgs: 129 active+clean; 715 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.3 MiB/s wr, 528 op/s 2024-05-14T10:00:34.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:34 smithi026 ceph-mon[36056]: pgmap v1195: 129 pgs: 129 active+clean; 715 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.3 MiB/s wr, 528 op/s 2024-05-14T10:00:35.566 INFO:tasks.workunit.client.0.smithi002.stdout: 0 cleanup 601 sec 2024-05-14T10:00:35.566 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-05-14T10:00:35.566 INFO:tasks.workunit.client.0.smithi002.stdout: Operation Count AvgLat MaxLat 2024-05-14T10:00:35.566 INFO:tasks.workunit.client.0.smithi002.stdout: ---------------------------------------- 2024-05-14T10:00:35.566 INFO:tasks.workunit.client.0.smithi002.stdout: NTCreateX 481817 4.679 2657.147 2024-05-14T10:00:35.566 INFO:tasks.workunit.client.0.smithi002.stdout: Close 353876 0.003 0.697 2024-05-14T10:00:35.566 INFO:tasks.workunit.client.0.smithi002.stdout: Rename 20427 20.980 2756.451 2024-05-14T10:00:35.566 INFO:tasks.workunit.client.0.smithi002.stdout: Unlink 97293 5.393 937.907 2024-05-14T10:00:35.566 INFO:tasks.workunit.client.0.smithi002.stdout: Qpathinfo 437038 3.870 2787.025 2024-05-14T10:00:35.566 INFO:tasks.workunit.client.0.smithi002.stdout: Qfileinfo 76238 0.001 0.053 2024-05-14T10:00:35.566 INFO:tasks.workunit.client.0.smithi002.stdout: Qfsinfo 80068 0.371 161.231 2024-05-14T10:00:35.566 INFO:tasks.workunit.client.0.smithi002.stdout: Sfileinfo 39312 0.479 436.666 2024-05-14T10:00:35.567 INFO:tasks.workunit.client.0.smithi002.stdout: Find 168862 0.319 621.519 2024-05-14T10:00:35.567 INFO:tasks.workunit.client.0.smithi002.stdout: WriteX 238124 0.232 2969.912 2024-05-14T10:00:35.567 INFO:tasks.workunit.client.0.smithi002.stdout: ReadX 756007 0.130 2254.570 2024-05-14T10:00:35.567 INFO:tasks.workunit.client.0.smithi002.stdout: LockX 1572 3.654 96.932 2024-05-14T10:00:35.567 INFO:tasks.workunit.client.0.smithi002.stdout: UnlockX 1572 3.367 157.798 2024-05-14T10:00:35.567 INFO:tasks.workunit.client.0.smithi002.stdout: Flush 33804 24.437 2672.638 2024-05-14T10:00:35.567 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-05-14T10:00:35.567 INFO:tasks.workunit.client.0.smithi002.stdout:Throughput 24.9909 MB/sec 10 clients 10 procs max_latency=2969.922 ms 2024-05-14T10:00:35.568 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-14T10:00:35.568 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-14T10:00:36.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:36 smithi029 ceph-mon[36249]: pgmap v1196: 129 pgs: 129 active+clean; 716 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 451 op/s 2024-05-14T10:00:36.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:36 smithi026 ceph-mon[36056]: pgmap v1196: 129 pgs: 129 active+clean; 716 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 451 op/s 2024-05-14T10:00:37.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:36 smithi002 ceph-mon[31557]: pgmap v1196: 129 pgs: 129 active+clean; 716 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 451 op/s 2024-05-14T10:00:37.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:37 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T10:00:37.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:37 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T10:00:38.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:37 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T10:00:38.611 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-05-14T10:00:38.611 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-14T10:00:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:38 smithi002 ceph-mon[31557]: pgmap v1197: 129 pgs: 129 active+clean; 669 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 514 op/s 2024-05-14T10:00:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:38 smithi026 ceph-mon[36056]: pgmap v1197: 129 pgs: 129 active+clean; 669 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 514 op/s 2024-05-14T10:00:39.000 DEBUG:teuthology.parallel:result is None 2024-05-14T10:00:39.001 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-14T10:00:39.032 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-14T10:00:39.033 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-14T10:00:39.046 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T10:00:39.082 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-14T10:00:39.094 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-14T10:00:39.095 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-14T10:00:39.095 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-14T10:00:39.109 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T10:00:39.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:38 smithi029 ceph-mon[36249]: pgmap v1197: 129 pgs: 129 active+clean; 669 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 514 op/s 2024-05-14T10:00:39.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.238+0000 7fccab18d640 1 -- 172.21.15.2:0/1353683273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca4104390 msgr2=0x7fcca41067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.238+0000 7fccab18d640 1 --2- 172.21.15.2:0/1353683273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca4104390 0x7fcca41067b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009900 tx=0x7fcc9c02a410 comp rx=0 tx=0).stop 2024-05-14T10:00:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.238+0000 7fccab18d640 1 -- 172.21.15.2:0/1353683273 shutdown_connections 2024-05-14T10:00:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.238+0000 7fccab18d640 1 --2- 172.21.15.2:0/1353683273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca41098f0 0x7fcca410be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.238+0000 7fccab18d640 1 --2- 172.21.15.2:0/1353683273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcca4106d80 0x7fcca4109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.238+0000 7fccab18d640 1 --2- 172.21.15.2:0/1353683273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca4104390 0x7fcca41067b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.238+0000 7fccab18d640 1 -- 172.21.15.2:0/1353683273 >> 172.21.15.2:0/1353683273 conn(0x7fcca40fdd70 msgr2=0x7fcca4100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.238+0000 7fccab18d640 1 -- 172.21.15.2:0/1353683273 shutdown_connections 2024-05-14T10:00:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.238+0000 7fccab18d640 1 -- 172.21.15.2:0/1353683273 wait complete. 2024-05-14T10:00:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.239+0000 7fccab18d640 1 Processor -- start 2024-05-14T10:00:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.239+0000 7fccab18d640 1 -- start start 2024-05-14T10:00:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.239+0000 7fccab18d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca4104390 0x7fcca419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.239+0000 7fccab18d640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcca4106d80 0x7fcca419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.239+0000 7fccab18d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca41098f0 0x7fcca41a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.239+0000 7fccab18d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcca410f730 con 0x7fcca4104390 2024-05-14T10:00:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.239+0000 7fccab18d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcca410f5b0 con 0x7fcca41098f0 2024-05-14T10:00:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.239+0000 7fccab18d640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fcca410f8b0 con 0x7fcca4106d80 2024-05-14T10:00:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.239+0000 7fcca8f29640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca4104390 0x7fcca419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.239+0000 7fcca8f29640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca4104390 0x7fcca419a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34790/0 (socket says 172.21.15.2:34790) 2024-05-14T10:00:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.239+0000 7fcca8f29640 1 -- 172.21.15.2:0/4009707411 learned_addr learned my addr 172.21.15.2:0/4009707411 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.239+0000 7fcca3fff640 1 --2- 172.21.15.2:0/4009707411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcca4106d80 0x7fcca419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.240+0000 7fcca972a640 1 --2- 172.21.15.2:0/4009707411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca41098f0 0x7fcca41a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.240+0000 7fcca8f29640 1 -- 172.21.15.2:0/4009707411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcca4106d80 msgr2=0x7fcca419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.240+0000 7fcca8f29640 1 --2- 172.21.15.2:0/4009707411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcca4106d80 0x7fcca419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.240+0000 7fcca8f29640 1 -- 172.21.15.2:0/4009707411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca41098f0 msgr2=0x7fcca41a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.240+0000 7fcca8f29640 1 --2- 172.21.15.2:0/4009707411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca41098f0 0x7fcca41a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.240+0000 7fcca8f29640 1 -- 172.21.15.2:0/4009707411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca41a1d70 con 0x7fcca4104390 2024-05-14T10:00:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.240+0000 7fcca972a640 1 --2- 172.21.15.2:0/4009707411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca41098f0 0x7fcca41a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T10:00:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.240+0000 7fcca8f29640 1 --2- 172.21.15.2:0/4009707411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca4104390 0x7fcca419a230 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c031440 tx=0x7fcc9c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:39.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.240+0000 7fcca1ffb640 1 -- 172.21.15.2:0/4009707411 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c03f070 con 0x7fcca4104390 2024-05-14T10:00:39.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.240+0000 7fcca1ffb640 1 -- 172.21.15.2:0/4009707411 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc9c031c50 con 0x7fcca4104390 2024-05-14T10:00:39.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.240+0000 7fcca1ffb640 1 -- 172.21.15.2:0/4009707411 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c035030 con 0x7fcca4104390 2024-05-14T10:00:39.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.240+0000 7fccab18d640 1 -- 172.21.15.2:0/4009707411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca41a2000 con 0x7fcca4104390 2024-05-14T10:00:39.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.240+0000 7fccab18d640 1 -- 172.21.15.2:0/4009707411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca41a24f0 con 0x7fcca4104390 2024-05-14T10:00:39.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.241+0000 7fccab18d640 1 -- 172.21.15.2:0/4009707411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca40ff890 con 0x7fcca4104390 2024-05-14T10:00:39.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.243+0000 7fcca1ffb640 1 -- 172.21.15.2:0/4009707411 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fcc9c042070 con 0x7fcca4104390 2024-05-14T10:00:39.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.244+0000 7fcca1ffb640 1 --2- 172.21.15.2:0/4009707411 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcc900b28a0 0x7fcc900b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:39.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.244+0000 7fcca3fff640 1 --2- 172.21.15.2:0/4009707411 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcc900b28a0 0x7fcc900b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:39.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.244+0000 7fcca1ffb640 1 -- 172.21.15.2:0/4009707411 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fcc9c102e70 con 0x7fcca4104390 2024-05-14T10:00:39.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.244+0000 7fcca3fff640 1 --2- 172.21.15.2:0/4009707411 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcc900b28a0 0x7fcc900b4db0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcca419b2d0 tx=0x7fcc94009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:39.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.249+0000 7fcca1ffb640 1 -- 172.21.15.2:0/4009707411 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc9c0ca710 con 0x7fcca4104390 2024-05-14T10:00:39.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.444+0000 7fccab18d640 1 -- 172.21.15.2:0/4009707411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcca40638d0 con 0x7fcca4104390 2024-05-14T10:00:39.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.446+0000 7fcca1ffb640 1 -- 172.21.15.2:0/4009707411 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24740 (secure 0 0 0) 0x7fcc9c0ca010 con 0x7fcca4104390 2024-05-14T10:00:39.447 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T10:00:39.447 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":93,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","created":"2024-05-14T09:22:34.937047+0000","modified":"2024-05-14T09:55:42.615872+0000","last_up_change":"2024-05-14T09:29:55.157569+0000","last_in_change":"2024-05-14T09:29:34.922562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T09:26:03.046454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T09:30:51.814350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T09:30:55.500000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T09:31:06.099087+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10d34289-414e-4b1b-bc7c-ba1dd24c7297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6803","nonce":2622184634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6805","nonce":2622184634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6809","nonce":2622184634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6807","nonce":2622184634}]},"public_addr":"172.21.15.2:6803/2622184634","cluster_addr":"172.21.15.2:6805/2622184634","heartbeat_back_addr":"172.21.15.2:6809/2622184634","heartbeat_front_addr":"172.21.15.2:6807/2622184634","state":["exists","up"]},{"osd":1,"uuid":"bc4aefc2-706e-4faf-9042-64b87ad097b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6801","nonce":3084747226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6803","nonce":3084747226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6807","nonce":3084747226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6805","nonce":3084747226}]},"public_addr":"172.21.15.26:6801/3084747226","cluster_addr":"172.21.15.26:6803/3084747226","heartbeat_back_addr":"172.21.15.26:6807/3084747226","heartbeat_front_addr":"172.21.15.26:6805/3084747226","state":["exists","up"]},{"osd":2,"uuid":"8c5bb589-56e2-469f-9a6f-6035b3ddf4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6801","nonce":814048086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6803","nonce":814048086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6807","nonce":814048086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6805","nonce":814048086}]},"public_addr":"172.21.15.29:6801/814048086","cluster_addr":"172.21.15.29:6803/814048086","heartbeat_back_addr":"172.21.15.29:6807/814048086","heartbeat_front_addr":"172.21.15.29:6805/814048086","state":["exists","up"]},{"osd":3,"uuid":"8fbfc078-85e7-4dc2-acda-5e7b0ab927e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6811","nonce":879639451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6813","nonce":879639451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6817","nonce":879639451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6815","nonce":879639451}]},"public_addr":"172.21.15.2:6811/879639451","cluster_addr":"172.21.15.2:6813/879639451","heartbeat_back_addr":"172.21.15.2:6817/879639451","heartbeat_front_addr":"172.21.15.2:6815/879639451","state":["exists","up"]},{"osd":4,"uuid":"f2515448-e9c1-48a2-b270-1e5e781a2fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6809","nonce":457100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6811","nonce":457100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6815","nonce":457100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6813","nonce":457100752}]},"public_addr":"172.21.15.26:6809/457100752","cluster_addr":"172.21.15.26:6811/457100752","heartbeat_back_addr":"172.21.15.26:6815/457100752","heartbeat_front_addr":"172.21.15.26:6813/457100752","state":["exists","up"]},{"osd":5,"uuid":"20030857-a1e6-4b0a-b186-1cfa44f2e4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6809","nonce":1554045093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6811","nonce":1554045093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6815","nonce":1554045093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6813","nonce":1554045093}]},"public_addr":"172.21.15.29:6809/1554045093","cluster_addr":"172.21.15.29:6811/1554045093","heartbeat_back_addr":"172.21.15.29:6815/1554045093","heartbeat_front_addr":"172.21.15.29:6813/1554045093","state":["exists","up"]},{"osd":6,"uuid":"d757c72a-29f0-4f14-9f13-a2e61eb5e6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6819","nonce":2749897595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6821","nonce":2749897595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6825","nonce":2749897595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6823","nonce":2749897595}]},"public_addr":"172.21.15.2:6819/2749897595","cluster_addr":"172.21.15.2:6821/2749897595","heartbeat_back_addr":"172.21.15.2:6825/2749897595","heartbeat_front_addr":"172.21.15.2:6823/2749897595","state":["exists","up"]},{"osd":7,"uuid":"22ddef80-3633-44c5-bc0e-edc75b546ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6817","nonce":1180401611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6819","nonce":1180401611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6823","nonce":1180401611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6821","nonce":1180401611}]},"public_addr":"172.21.15.26:6817/1180401611","cluster_addr":"172.21.15.26:6819/1180401611","heartbeat_back_addr":"172.21.15.26:6823/1180401611","heartbeat_front_addr":"172.21.15.26:6821/1180401611","state":["exists","up"]},{"osd":8,"uuid":"466e7ac4-3ff7-4254-a3fd-8096420b1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6817","nonce":2858408446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6819","nonce":2858408446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6823","nonce":2858408446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6821","nonce":2858408446}]},"public_addr":"172.21.15.29:6817/2858408446","cluster_addr":"172.21.15.29:6819/2858408446","heartbeat_back_addr":"172.21.15.29:6823/2858408446","heartbeat_front_addr":"172.21.15.29:6821/2858408446","state":["exists","up"]},{"osd":9,"uuid":"119eeac9-29f7-4086-a1ad-04a0bc154b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6827","nonce":176898226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6829","nonce":176898226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6833","nonce":176898226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6831","nonce":176898226}]},"public_addr":"172.21.15.2:6827/176898226","cluster_addr":"172.21.15.2:6829/176898226","heartbeat_back_addr":"172.21.15.2:6833/176898226","heartbeat_front_addr":"172.21.15.2:6831/176898226","state":["exists","up"]},{"osd":10,"uuid":"55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6825","nonce":4065159515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6827","nonce":4065159515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6831","nonce":4065159515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6829","nonce":4065159515}]},"public_addr":"172.21.15.26:6825/4065159515","cluster_addr":"172.21.15.26:6827/4065159515","heartbeat_back_addr":"172.21.15.26:6831/4065159515","heartbeat_front_addr":"172.21.15.26:6829/4065159515","state":["exists","up"]},{"osd":11,"uuid":"c5246763-c4f6-4ae3-8a93-5929c541ac5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6825","nonce":378800417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6827","nonce":378800417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6831","nonce":378800417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6829","nonce":378800417}]},"public_addr":"172.21.15.29:6825/378800417","cluster_addr":"172.21.15.29:6827/378800417","heartbeat_back_addr":"172.21.15.29:6831/378800417","heartbeat_front_addr":"172.21.15.29:6829/378800417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:09.342387+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:34.156359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:58.860653+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:24.009323+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:49.057146+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:15.007172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:41.085748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:06.674923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:33.409128+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:00.106186+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:26.803487+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:53.668283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11},{"from":8,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9},{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4051922409":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/2353790181":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/4268171474":"2024-05-15T09:23:28.940670+0000","172.21.15.2:6801/1423742578":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/969850441":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3252991268":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6801/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3789067476":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/1423742578":"2024-05-15T09:23:28.940670+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T10:00:39.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.448+0000 7fccab18d640 1 -- 172.21.15.2:0/4009707411 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcc900b28a0 msgr2=0x7fcc900b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:39.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.448+0000 7fccab18d640 1 --2- 172.21.15.2:0/4009707411 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcc900b28a0 0x7fcc900b4db0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcca419b2d0 tx=0x7fcc94009500 comp rx=0 tx=0).stop 2024-05-14T10:00:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.448+0000 7fccab18d640 1 -- 172.21.15.2:0/4009707411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca4104390 msgr2=0x7fcca419a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.448+0000 7fccab18d640 1 --2- 172.21.15.2:0/4009707411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca4104390 0x7fcca419a230 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c031440 tx=0x7fcc9c003c10 comp rx=0 tx=0).stop 2024-05-14T10:00:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.448+0000 7fccab18d640 1 -- 172.21.15.2:0/4009707411 shutdown_connections 2024-05-14T10:00:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.448+0000 7fccab18d640 1 --2- 172.21.15.2:0/4009707411 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fcc900b28a0 0x7fcc900b4db0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.448+0000 7fccab18d640 1 --2- 172.21.15.2:0/4009707411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcca41098f0 0x7fcca41a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.448+0000 7fccab18d640 1 --2- 172.21.15.2:0/4009707411 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcca4106d80 0x7fcca419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.448+0000 7fccab18d640 1 --2- 172.21.15.2:0/4009707411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca4104390 0x7fcca419a230 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.448+0000 7fccab18d640 1 -- 172.21.15.2:0/4009707411 >> 172.21.15.2:0/4009707411 conn(0x7fcca40fdd70 msgr2=0x7fcca4104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.448+0000 7fccab18d640 1 -- 172.21.15.2:0/4009707411 shutdown_connections 2024-05-14T10:00:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.448+0000 7fccab18d640 1 -- 172.21.15.2:0/4009707411 wait complete. 2024-05-14T10:00:39.461 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T09:26:03.046454+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-14T09:30:51.814350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-14T09:30:55.500000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-14T09:31:06.099087+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-14T10:00:39.462 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-14T10:00:39.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.580+0000 7f8356a51640 1 -- 172.21.15.2:0/439691593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8350103d40 msgr2=0x7f8350104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.580+0000 7f8356a51640 1 --2- 172.21.15.2:0/439691593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8350103d40 0x7f8350104150 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8344009900 tx=0x7f834402a410 comp rx=0 tx=0).stop 2024-05-14T10:00:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.580+0000 7f8356a51640 1 -- 172.21.15.2:0/439691593 shutdown_connections 2024-05-14T10:00:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.580+0000 7f8356a51640 1 --2- 172.21.15.2:0/439691593 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8350108a40 0x7f835010ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.580+0000 7f8356a51640 1 --2- 172.21.15.2:0/439691593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8350103d40 0x7f8350104150 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.580+0000 7f8356a51640 1 --2- 172.21.15.2:0/439691593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8350102b00 0x7f8350103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.580+0000 7f8356a51640 1 -- 172.21.15.2:0/439691593 >> 172.21.15.2:0/439691593 conn(0x7f83500fdcf0 msgr2=0x7f83501001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.580+0000 7f8356a51640 1 -- 172.21.15.2:0/439691593 shutdown_connections 2024-05-14T10:00:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.580+0000 7f8356a51640 1 -- 172.21.15.2:0/439691593 wait complete. 2024-05-14T10:00:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.581+0000 7f8356a51640 1 Processor -- start 2024-05-14T10:00:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.581+0000 7f8356a51640 1 -- start start 2024-05-14T10:00:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.581+0000 7f8356a51640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8350102b00 0x7f835019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.581+0000 7f8356a51640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8350103d40 0x7f835019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.581+0000 7f8356a51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8350108a40 0x7f83501a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.581+0000 7f8356a51640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f835010e600 con 0x7f8350103d40 2024-05-14T10:00:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.581+0000 7f8356a51640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f835010e480 con 0x7f8350108a40 2024-05-14T10:00:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.581+0000 7f8356a51640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f835010e780 con 0x7f8350102b00 2024-05-14T10:00:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.581+0000 7f834f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8350103d40 0x7f835019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.581+0000 7f834f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8350103d40 0x7f835019a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34816/0 (socket says 172.21.15.2:34816) 2024-05-14T10:00:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.581+0000 7f834f7fe640 1 -- 172.21.15.2:0/2661074109 learned_addr learned my addr 172.21.15.2:0/2661074109 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.581+0000 7f834ffff640 1 --2- 172.21.15.2:0/2661074109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8350102b00 0x7f835019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.581+0000 7f8354fee640 1 --2- 172.21.15.2:0/2661074109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8350108a40 0x7f83501a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.582+0000 7f834f7fe640 1 -- 172.21.15.2:0/2661074109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8350102b00 msgr2=0x7f835019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.582+0000 7f834f7fe640 1 --2- 172.21.15.2:0/2661074109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8350102b00 0x7f835019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.582+0000 7f834f7fe640 1 -- 172.21.15.2:0/2661074109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8350108a40 msgr2=0x7f83501a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.582+0000 7f834f7fe640 1 --2- 172.21.15.2:0/2661074109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8350108a40 0x7f83501a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.582+0000 7f834f7fe640 1 -- 172.21.15.2:0/2661074109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83501a1e60 con 0x7f8350103d40 2024-05-14T10:00:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.582+0000 7f8354fee640 1 --2- 172.21.15.2:0/2661074109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8350108a40 0x7f83501a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T10:00:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.582+0000 7f834f7fe640 1 --2- 172.21.15.2:0/2661074109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8350103d40 0x7f835019a860 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f8344006010 tx=0x7f8344004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.582+0000 7f834d7fa640 1 -- 172.21.15.2:0/2661074109 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834403f030 con 0x7f8350103d40 2024-05-14T10:00:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.582+0000 7f834d7fa640 1 -- 172.21.15.2:0/2661074109 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8344003cb0 con 0x7f8350103d40 2024-05-14T10:00:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.582+0000 7f8356a51640 1 -- 172.21.15.2:0/2661074109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83501a20f0 con 0x7f8350103d40 2024-05-14T10:00:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.582+0000 7f834d7fa640 1 -- 172.21.15.2:0/2661074109 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344035070 con 0x7f8350103d40 2024-05-14T10:00:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.582+0000 7f8356a51640 1 -- 172.21.15.2:0/2661074109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83501a25e0 con 0x7f8350103d40 2024-05-14T10:00:39.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.583+0000 7f8356a51640 1 -- 172.21.15.2:0/2661074109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83501a2840 con 0x7f8350103d40 2024-05-14T10:00:39.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.585+0000 7f834d7fa640 1 -- 172.21.15.2:0/2661074109 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8344042030 con 0x7f8350103d40 2024-05-14T10:00:39.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.585+0000 7f834d7fa640 1 --2- 172.21.15.2:0/2661074109 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83380b27f0 0x7f83380b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:39.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.586+0000 7f834ffff640 1 --2- 172.21.15.2:0/2661074109 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83380b27f0 0x7f83380b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:39.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.586+0000 7f834d7fa640 1 -- 172.21.15.2:0/2661074109 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f8344103340 con 0x7f8350103d40 2024-05-14T10:00:39.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.586+0000 7f834ffff640 1 --2- 172.21.15.2:0/2661074109 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83380b27f0 0x7f83380b4d00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8350103700 tx=0x7f834000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:39.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.589+0000 7f834d7fa640 1 -- 172.21.15.2:0/2661074109 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f83440cac40 con 0x7f8350103d40 2024-05-14T10:00:39.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.787+0000 7f8356a51640 1 -- 172.21.15.2:0/2661074109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f83501a2a10 con 0x7f8350103d40 2024-05-14T10:00:39.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.788+0000 7f834d7fa640 1 -- 172.21.15.2:0/2661074109 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f8344003e60 con 0x7f8350103d40 2024-05-14T10:00:39.789 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-05-14T10:00:39.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.790+0000 7f8356a51640 1 -- 172.21.15.2:0/2661074109 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83380b27f0 msgr2=0x7f83380b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:39.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.790+0000 7f8356a51640 1 --2- 172.21.15.2:0/2661074109 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83380b27f0 0x7f83380b4d00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8350103700 tx=0x7f834000b390 comp rx=0 tx=0).stop 2024-05-14T10:00:39.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.790+0000 7f8356a51640 1 -- 172.21.15.2:0/2661074109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8350103d40 msgr2=0x7f835019a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:39.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.790+0000 7f8356a51640 1 --2- 172.21.15.2:0/2661074109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8350103d40 0x7f835019a860 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f8344006010 tx=0x7f8344004340 comp rx=0 tx=0).stop 2024-05-14T10:00:39.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.790+0000 7f8356a51640 1 -- 172.21.15.2:0/2661074109 shutdown_connections 2024-05-14T10:00:39.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.790+0000 7f8356a51640 1 --2- 172.21.15.2:0/2661074109 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f83380b27f0 0x7f83380b4d00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.790+0000 7f8356a51640 1 --2- 172.21.15.2:0/2661074109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8350108a40 0x7f83501a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.790+0000 7f8356a51640 1 --2- 172.21.15.2:0/2661074109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8350103d40 0x7f835019a860 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.790+0000 7f8356a51640 1 --2- 172.21.15.2:0/2661074109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8350102b00 0x7f835019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.790+0000 7f8356a51640 1 -- 172.21.15.2:0/2661074109 >> 172.21.15.2:0/2661074109 conn(0x7f83500fdcf0 msgr2=0x7f8350109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:39.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.790+0000 7f8356a51640 1 -- 172.21.15.2:0/2661074109 shutdown_connections 2024-05-14T10:00:39.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.790+0000 7f8356a51640 1 -- 172.21.15.2:0/2661074109 wait complete. 2024-05-14T10:00:39.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:39 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/4009707411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T10:00:39.803 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-14T10:00:39.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.920+0000 7f6385f34640 1 -- 172.21.15.2:0/3985898535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6380108a40 msgr2=0x7f638010ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:39.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.920+0000 7f6385f34640 1 --2- 172.21.15.2:0/3985898535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6380108a40 0x7f638010ae60 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6378009ca0 tx=0x7f637802a8c0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.920+0000 7f6385f34640 1 -- 172.21.15.2:0/3985898535 shutdown_connections 2024-05-14T10:00:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.920+0000 7f6385f34640 1 --2- 172.21.15.2:0/3985898535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6380108a40 0x7f638010ae60 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.920+0000 7f6385f34640 1 --2- 172.21.15.2:0/3985898535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6380103d40 0x7f6380104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.920+0000 7f6385f34640 1 --2- 172.21.15.2:0/3985898535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6380102b00 0x7f6380103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.920+0000 7f6385f34640 1 -- 172.21.15.2:0/3985898535 >> 172.21.15.2:0/3985898535 conn(0x7f63800fdcf0 msgr2=0x7f63801001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.920+0000 7f6385f34640 1 -- 172.21.15.2:0/3985898535 shutdown_connections 2024-05-14T10:00:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.920+0000 7f6385f34640 1 -- 172.21.15.2:0/3985898535 wait complete. 2024-05-14T10:00:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.921+0000 7f6385f34640 1 Processor -- start 2024-05-14T10:00:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.921+0000 7f6385f34640 1 -- start start 2024-05-14T10:00:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.921+0000 7f6385f34640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6380102b00 0x7f638019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.921+0000 7f6385f34640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6380103d40 0x7f638019a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.921+0000 7f6385f34640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6380108a40 0x7f63801a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.921+0000 7f6385f34640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f638010f730 con 0x7f6380103d40 2024-05-14T10:00:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.921+0000 7f6385f34640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f638010f5b0 con 0x7f6380108a40 2024-05-14T10:00:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.921+0000 7f6385f34640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f638010f8b0 con 0x7f6380102b00 2024-05-14T10:00:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.921+0000 7f6376ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6380103d40 0x7f638019a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.921+0000 7f6376ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6380103d40 0x7f638019a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34832/0 (socket says 172.21.15.2:34832) 2024-05-14T10:00:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.921+0000 7f6376ffd640 1 -- 172.21.15.2:0/1092145673 learned_addr learned my addr 172.21.15.2:0/1092145673 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.921+0000 7f637f7fe640 1 --2- 172.21.15.2:0/1092145673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6380102b00 0x7f638019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.921+0000 7f637ffff640 1 --2- 172.21.15.2:0/1092145673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6380108a40 0x7f63801a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.922+0000 7f6376ffd640 1 -- 172.21.15.2:0/1092145673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6380102b00 msgr2=0x7f638019a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.922+0000 7f6376ffd640 1 --2- 172.21.15.2:0/1092145673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6380102b00 0x7f638019a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.922+0000 7f6376ffd640 1 -- 172.21.15.2:0/1092145673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6380108a40 msgr2=0x7f63801a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.922+0000 7f6376ffd640 1 --2- 172.21.15.2:0/1092145673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6380108a40 0x7f63801a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.922+0000 7f6376ffd640 1 -- 172.21.15.2:0/1092145673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63801a1e20 con 0x7f6380103d40 2024-05-14T10:00:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.922+0000 7f637ffff640 1 --2- 172.21.15.2:0/1092145673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6380108a40 0x7f63801a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T10:00:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.922+0000 7f6376ffd640 1 --2- 172.21.15.2:0/1092145673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6380103d40 0x7f638019a820 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f636800e9f0 tx=0x7f636800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.922+0000 7f637d7fa640 1 -- 172.21.15.2:0/1092145673 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636800cd80 con 0x7f6380103d40 2024-05-14T10:00:39.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.922+0000 7f637d7fa640 1 -- 172.21.15.2:0/1092145673 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63680108e0 con 0x7f6380103d40 2024-05-14T10:00:39.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.922+0000 7f6385f34640 1 -- 172.21.15.2:0/1092145673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63801a2110 con 0x7f6380103d40 2024-05-14T10:00:39.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.922+0000 7f637d7fa640 1 -- 172.21.15.2:0/1092145673 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368018980 con 0x7f6380103d40 2024-05-14T10:00:39.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.922+0000 7f6385f34640 1 -- 172.21.15.2:0/1092145673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63801a2660 con 0x7f6380103d40 2024-05-14T10:00:39.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.923+0000 7f6385f34640 1 -- 172.21.15.2:0/1092145673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6380107850 con 0x7f6380103d40 2024-05-14T10:00:39.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.925+0000 7f637d7fa640 1 -- 172.21.15.2:0/1092145673 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6368010430 con 0x7f6380103d40 2024-05-14T10:00:39.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.926+0000 7f637d7fa640 1 --2- 172.21.15.2:0/1092145673 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f636c0b27f0 0x7f636c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:39.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.926+0000 7f637f7fe640 1 --2- 172.21.15.2:0/1092145673 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f636c0b27f0 0x7f636c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:39.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.926+0000 7f637d7fa640 1 -- 172.21.15.2:0/1092145673 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f63680e2e80 con 0x7f6380103d40 2024-05-14T10:00:39.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.926+0000 7f637f7fe640 1 --2- 172.21.15.2:0/1092145673 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f636c0b27f0 0x7f636c0b4d00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63700058c0 tx=0x7f637000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:39.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:39.930+0000 7f637d7fa640 1 -- 172.21.15.2:0/1092145673 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f63680aa780 con 0x7f6380103d40 2024-05-14T10:00:40.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.122+0000 7f6385f34640 1 -- 172.21.15.2:0/1092145673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f638002ddc0 con 0x7f6380103d40 2024-05-14T10:00:40.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.123+0000 7f637d7fa640 1 -- 172.21.15.2:0/1092145673 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f638002ddc0 con 0x7f6380103d40 2024-05-14T10:00:40.124 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-05-14T10:00:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.125+0000 7f6385f34640 1 -- 172.21.15.2:0/1092145673 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f636c0b27f0 msgr2=0x7f636c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.125+0000 7f6385f34640 1 --2- 172.21.15.2:0/1092145673 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f636c0b27f0 0x7f636c0b4d00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63700058c0 tx=0x7f637000a2d0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.125+0000 7f6385f34640 1 -- 172.21.15.2:0/1092145673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6380103d40 msgr2=0x7f638019a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.125+0000 7f6385f34640 1 --2- 172.21.15.2:0/1092145673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6380103d40 0x7f638019a820 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f636800e9f0 tx=0x7f636800eeb0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.125+0000 7f6385f34640 1 -- 172.21.15.2:0/1092145673 shutdown_connections 2024-05-14T10:00:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.125+0000 7f6385f34640 1 --2- 172.21.15.2:0/1092145673 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f636c0b27f0 0x7f636c0b4d00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.126+0000 7f6385f34640 1 --2- 172.21.15.2:0/1092145673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6380108a40 0x7f63801a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.126+0000 7f6385f34640 1 --2- 172.21.15.2:0/1092145673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6380103d40 0x7f638019a820 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.126+0000 7f6385f34640 1 --2- 172.21.15.2:0/1092145673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6380102b00 0x7f638019a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.126+0000 7f6385f34640 1 -- 172.21.15.2:0/1092145673 >> 172.21.15.2:0/1092145673 conn(0x7f63800fdcf0 msgr2=0x7f6380104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.126+0000 7f6385f34640 1 -- 172.21.15.2:0/1092145673 shutdown_connections 2024-05-14T10:00:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.126+0000 7f6385f34640 1 -- 172.21.15.2:0/1092145673 wait complete. 2024-05-14T10:00:40.139 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-14T10:00:40.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:39 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/4009707411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T10:00:40.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:39 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/4009707411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T10:00:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.257+0000 7f7f10496640 1 -- 172.21.15.2:0/1924179053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f08102b00 msgr2=0x7f7f08103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.257+0000 7f7f10496640 1 --2- 172.21.15.2:0/1924179053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f08102b00 0x7f7f08103800 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009900 tx=0x7f7f0402a410 comp rx=0 tx=0).stop 2024-05-14T10:00:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.257+0000 7f7f10496640 1 -- 172.21.15.2:0/1924179053 shutdown_connections 2024-05-14T10:00:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.257+0000 7f7f10496640 1 --2- 172.21.15.2:0/1924179053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08108a40 0x7f7f0810ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.257+0000 7f7f10496640 1 --2- 172.21.15.2:0/1924179053 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f08103d40 0x7f7f08104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.257+0000 7f7f10496640 1 --2- 172.21.15.2:0/1924179053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f08102b00 0x7f7f08103800 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.257+0000 7f7f10496640 1 -- 172.21.15.2:0/1924179053 >> 172.21.15.2:0/1924179053 conn(0x7f7f080fdcf0 msgr2=0x7f7f081001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.257+0000 7f7f10496640 1 -- 172.21.15.2:0/1924179053 shutdown_connections 2024-05-14T10:00:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.257+0000 7f7f10496640 1 -- 172.21.15.2:0/1924179053 wait complete. 2024-05-14T10:00:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.258+0000 7f7f10496640 1 Processor -- start 2024-05-14T10:00:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.258+0000 7f7f10496640 1 -- start start 2024-05-14T10:00:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.258+0000 7f7f10496640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08102b00 0x7f7f0819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.258+0000 7f7f10496640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f08103d40 0x7f7f0819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.258+0000 7f7f10496640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f08108a40 0x7f7f081a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.258+0000 7f7f10496640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0810f6f0 con 0x7f7f08108a40 2024-05-14T10:00:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.258+0000 7f7f10496640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0810f570 con 0x7f7f08102b00 2024-05-14T10:00:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.258+0000 7f7f10496640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0810f870 con 0x7f7f08103d40 2024-05-14T10:00:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.258+0000 7f7f0ea33640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f08108a40 0x7f7f081a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.258+0000 7f7f0ea33640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f08108a40 0x7f7f081a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34850/0 (socket says 172.21.15.2:34850) 2024-05-14T10:00:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.258+0000 7f7f0ea33640 1 -- 172.21.15.2:0/2128887248 learned_addr learned my addr 172.21.15.2:0/2128887248 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.258+0000 7f7f0e232640 1 --2- 172.21.15.2:0/2128887248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08102b00 0x7f7f0819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.258+0000 7f7f0da31640 1 --2- 172.21.15.2:0/2128887248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f08103d40 0x7f7f0819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.259+0000 7f7f0ea33640 1 -- 172.21.15.2:0/2128887248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f08103d40 msgr2=0x7f7f0819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.259+0000 7f7f0ea33640 1 --2- 172.21.15.2:0/2128887248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f08103d40 0x7f7f0819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.259+0000 7f7f0ea33640 1 -- 172.21.15.2:0/2128887248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08102b00 msgr2=0x7f7f0819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.259+0000 7f7f0ea33640 1 --2- 172.21.15.2:0/2128887248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08102b00 0x7f7f0819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.259+0000 7f7f0ea33640 1 -- 172.21.15.2:0/2128887248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f081a1db0 con 0x7f7f08108a40 2024-05-14T10:00:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.259+0000 7f7f0ea33640 1 --2- 172.21.15.2:0/2128887248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f08108a40 0x7f7f081a17f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000eaa0 tx=0x7f7f0000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.259+0000 7f7efb7fe640 1 -- 172.21.15.2:0/2128887248 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0000cdb0 con 0x7f7f08108a40 2024-05-14T10:00:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.259+0000 7f7f10496640 1 -- 172.21.15.2:0/2128887248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f081a20a0 con 0x7f7f08108a40 2024-05-14T10:00:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.259+0000 7f7efb7fe640 1 -- 172.21.15.2:0/2128887248 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f00005020 con 0x7f7f08108a40 2024-05-14T10:00:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.259+0000 7f7efb7fe640 1 -- 172.21.15.2:0/2128887248 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f000116d0 con 0x7f7f08108a40 2024-05-14T10:00:40.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.259+0000 7f7f10496640 1 -- 172.21.15.2:0/2128887248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f081a25f0 con 0x7f7f08108a40 2024-05-14T10:00:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.260+0000 7f7f10496640 1 -- 172.21.15.2:0/2128887248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f08107850 con 0x7f7f08108a40 2024-05-14T10:00:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.262+0000 7f7efb7fe640 1 -- 172.21.15.2:0/2128887248 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f7f000051d0 con 0x7f7f08108a40 2024-05-14T10:00:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.262+0000 7f7efb7fe640 1 --2- 172.21.15.2:0/2128887248 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ef40b27f0 0x7f7ef40b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.263+0000 7f7f0e232640 1 --2- 172.21.15.2:0/2128887248 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ef40b27f0 0x7f7ef40b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.263+0000 7f7efb7fe640 1 -- 172.21.15.2:0/2128887248 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f7f000e2c00 con 0x7f7f08108a40 2024-05-14T10:00:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.263+0000 7f7f0e232640 1 --2- 172.21.15.2:0/2128887248 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ef40b27f0 0x7f7ef40b4d00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7f04005ab0 tx=0x7f7f04031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:40.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.266+0000 7f7efb7fe640 1 -- 172.21.15.2:0/2128887248 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f00016030 con 0x7f7f08108a40 2024-05-14T10:00:40.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.457+0000 7f7f10496640 1 -- 172.21.15.2:0/2128887248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7f0802dc30 con 0x7f7f08108a40 2024-05-14T10:00:40.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.457+0000 7f7efb7fe640 1 -- 172.21.15.2:0/2128887248 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f7f000aa500 con 0x7f7f08108a40 2024-05-14T10:00:40.458 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-05-14T10:00:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.459+0000 7f7f10496640 1 -- 172.21.15.2:0/2128887248 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ef40b27f0 msgr2=0x7f7ef40b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.459+0000 7f7f10496640 1 --2- 172.21.15.2:0/2128887248 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ef40b27f0 0x7f7ef40b4d00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7f04005ab0 tx=0x7f7f04031400 comp rx=0 tx=0).stop 2024-05-14T10:00:40.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.459+0000 7f7f10496640 1 -- 172.21.15.2:0/2128887248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f08108a40 msgr2=0x7f7f081a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:40.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.459+0000 7f7f10496640 1 --2- 172.21.15.2:0/2128887248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f08108a40 0x7f7f081a17f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000eaa0 tx=0x7f7f0000ef60 comp rx=0 tx=0).stop 2024-05-14T10:00:40.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.459+0000 7f7f10496640 1 -- 172.21.15.2:0/2128887248 shutdown_connections 2024-05-14T10:00:40.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.459+0000 7f7f10496640 1 --2- 172.21.15.2:0/2128887248 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f7ef40b27f0 0x7f7ef40b4d00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.459+0000 7f7f10496640 1 --2- 172.21.15.2:0/2128887248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f08108a40 0x7f7f081a17f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.459+0000 7f7f10496640 1 --2- 172.21.15.2:0/2128887248 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f08103d40 0x7f7f0819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.459+0000 7f7f10496640 1 --2- 172.21.15.2:0/2128887248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08102b00 0x7f7f0819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.459+0000 7f7f10496640 1 -- 172.21.15.2:0/2128887248 >> 172.21.15.2:0/2128887248 conn(0x7f7f080fdcf0 msgr2=0x7f7f08104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:40.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.459+0000 7f7f10496640 1 -- 172.21.15.2:0/2128887248 shutdown_connections 2024-05-14T10:00:40.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.459+0000 7f7f10496640 1 -- 172.21.15.2:0/2128887248 wait complete. 2024-05-14T10:00:40.472 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-14T10:00:40.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.589+0000 7f67fcd86640 1 -- 172.21.15.2:0/1323231522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67f8102b00 msgr2=0x7f67f8103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:40.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.589+0000 7f67fcd86640 1 --2- 172.21.15.2:0/1323231522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67f8102b00 0x7f67f8103800 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009ca0 tx=0x7f67e802a8c0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.589+0000 7f67fcd86640 1 -- 172.21.15.2:0/1323231522 shutdown_connections 2024-05-14T10:00:40.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.589+0000 7f67fcd86640 1 --2- 172.21.15.2:0/1323231522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67f8108a40 0x7f67f810ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.589+0000 7f67fcd86640 1 --2- 172.21.15.2:0/1323231522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67f8103d40 0x7f67f8104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.589+0000 7f67fcd86640 1 --2- 172.21.15.2:0/1323231522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67f8102b00 0x7f67f8103800 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.589+0000 7f67fcd86640 1 -- 172.21.15.2:0/1323231522 >> 172.21.15.2:0/1323231522 conn(0x7f67f80fdcf0 msgr2=0x7f67f81001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:40.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.589+0000 7f67fcd86640 1 -- 172.21.15.2:0/1323231522 shutdown_connections 2024-05-14T10:00:40.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.589+0000 7f67fcd86640 1 -- 172.21.15.2:0/1323231522 wait complete. 2024-05-14T10:00:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.590+0000 7f67fcd86640 1 Processor -- start 2024-05-14T10:00:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.590+0000 7f67fcd86640 1 -- start start 2024-05-14T10:00:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.590+0000 7f67fcd86640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67f8102b00 0x7f67f819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.590+0000 7f67fcd86640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67f8103d40 0x7f67f819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.590+0000 7f67fcd86640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67f8108a40 0x7f67f81a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.590+0000 7f67fcd86640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f67f810f730 con 0x7f67f8103d40 2024-05-14T10:00:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.590+0000 7f67fcd86640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f67f810f5b0 con 0x7f67f8102b00 2024-05-14T10:00:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.590+0000 7f67fcd86640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f67f810f8b0 con 0x7f67f8108a40 2024-05-14T10:00:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.590+0000 7f67f5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67f8103d40 0x7f67f819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.590+0000 7f67f5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67f8103d40 0x7f67f819a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34864/0 (socket says 172.21.15.2:34864) 2024-05-14T10:00:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.590+0000 7f67f5d9b640 1 -- 172.21.15.2:0/446194826 learned_addr learned my addr 172.21.15.2:0/446194826 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.591+0000 7f67f659c640 1 --2- 172.21.15.2:0/446194826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67f8102b00 0x7f67f819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.591+0000 7f67f6d9d640 1 --2- 172.21.15.2:0/446194826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67f8108a40 0x7f67f81a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.591+0000 7f67f5d9b640 1 -- 172.21.15.2:0/446194826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67f8108a40 msgr2=0x7f67f81a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.591+0000 7f67f5d9b640 1 --2- 172.21.15.2:0/446194826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67f8108a40 0x7f67f81a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.591+0000 7f67f5d9b640 1 -- 172.21.15.2:0/446194826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67f8102b00 msgr2=0x7f67f819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.591+0000 7f67f5d9b640 1 --2- 172.21.15.2:0/446194826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67f8102b00 0x7f67f819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.591+0000 7f67f5d9b640 1 -- 172.21.15.2:0/446194826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f81a1dd0 con 0x7f67f8103d40 2024-05-14T10:00:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.591+0000 7f67f659c640 1 --2- 172.21.15.2:0/446194826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67f8102b00 0x7f67f819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T10:00:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.591+0000 7f67f5d9b640 1 --2- 172.21.15.2:0/446194826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67f8103d40 0x7f67f819a880 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00d950 tx=0x7f67ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.591+0000 7f67e77fe640 1 -- 172.21.15.2:0/446194826 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec017070 con 0x7f67f8103d40 2024-05-14T10:00:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.591+0000 7f67e77fe640 1 -- 172.21.15.2:0/446194826 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67ec009c30 con 0x7f67f8103d40 2024-05-14T10:00:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.591+0000 7f67fcd86640 1 -- 172.21.15.2:0/446194826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f81a20c0 con 0x7f67f8103d40 2024-05-14T10:00:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.591+0000 7f67e77fe640 1 -- 172.21.15.2:0/446194826 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec00f620 con 0x7f67f8103d40 2024-05-14T10:00:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.591+0000 7f67fcd86640 1 -- 172.21.15.2:0/446194826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f81a2610 con 0x7f67f8103d40 2024-05-14T10:00:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.592+0000 7f67fcd86640 1 -- 172.21.15.2:0/446194826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f8107850 con 0x7f67f8103d40 2024-05-14T10:00:40.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.594+0000 7f67e77fe640 1 -- 172.21.15.2:0/446194826 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f67ec010460 con 0x7f67f8103d40 2024-05-14T10:00:40.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.594+0000 7f67e77fe640 1 --2- 172.21.15.2:0/446194826 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67e00b27f0 0x7f67e00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:40.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.595+0000 7f67f659c640 1 --2- 172.21.15.2:0/446194826 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67e00b27f0 0x7f67e00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:40.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.595+0000 7f67e77fe640 1 -- 172.21.15.2:0/446194826 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f67ec0e22f0 con 0x7f67f8103d40 2024-05-14T10:00:40.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.595+0000 7f67f659c640 1 --2- 172.21.15.2:0/446194826 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67e00b27f0 0x7f67e00b4d00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f67e8000f80 tx=0x7f67e8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:40.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.598+0000 7f67e77fe640 1 -- 172.21.15.2:0/446194826 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f67ec0a9bf0 con 0x7f67f8103d40 2024-05-14T10:00:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.805+0000 7f67fcd86640 1 -- 172.21.15.2:0/446194826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f67f802ddc0 con 0x7f67f8103d40 2024-05-14T10:00:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.805+0000 7f67e77fe640 1 -- 172.21.15.2:0/446194826 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f67ec009de0 con 0x7f67f8103d40 2024-05-14T10:00:40.807 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-05-14T10:00:40.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.807+0000 7f67fcd86640 1 -- 172.21.15.2:0/446194826 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67e00b27f0 msgr2=0x7f67e00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:40.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.807+0000 7f67fcd86640 1 --2- 172.21.15.2:0/446194826 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67e00b27f0 0x7f67e00b4d00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f67e8000f80 tx=0x7f67e8004a40 comp rx=0 tx=0).stop 2024-05-14T10:00:40.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.807+0000 7f67fcd86640 1 -- 172.21.15.2:0/446194826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67f8103d40 msgr2=0x7f67f819a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:40.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.807+0000 7f67fcd86640 1 --2- 172.21.15.2:0/446194826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67f8103d40 0x7f67f819a880 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00d950 tx=0x7f67ec00de10 comp rx=0 tx=0).stop 2024-05-14T10:00:40.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.807+0000 7f67fcd86640 1 -- 172.21.15.2:0/446194826 shutdown_connections 2024-05-14T10:00:40.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.807+0000 7f67fcd86640 1 --2- 172.21.15.2:0/446194826 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f67e00b27f0 0x7f67e00b4d00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.807+0000 7f67fcd86640 1 --2- 172.21.15.2:0/446194826 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f67f8108a40 0x7f67f81a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.807+0000 7f67fcd86640 1 --2- 172.21.15.2:0/446194826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67f8103d40 0x7f67f819a880 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.807+0000 7f67fcd86640 1 --2- 172.21.15.2:0/446194826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67f8102b00 0x7f67f819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.807+0000 7f67fcd86640 1 -- 172.21.15.2:0/446194826 >> 172.21.15.2:0/446194826 conn(0x7f67f80fdcf0 msgr2=0x7f67f8104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:40.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.808+0000 7f67fcd86640 1 -- 172.21.15.2:0/446194826 shutdown_connections 2024-05-14T10:00:40.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.808+0000 7f67fcd86640 1 -- 172.21.15.2:0/446194826 wait complete. 2024-05-14T10:00:40.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:40 smithi002 ceph-mon[31557]: pgmap v1198: 129 pgs: 129 active+clean; 591 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.8 MiB/s wr, 564 op/s 2024-05-14T10:00:40.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:40 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T10:00:40.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:40 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2661074109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T10:00:40.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:40 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1092145673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T10:00:40.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:40 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2128887248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T10:00:40.820 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-14T10:00:40.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.936+0000 7f2c5c2b6640 1 -- 172.21.15.2:0/3746497144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c54102b30 msgr2=0x7f2c54103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:40.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.936+0000 7f2c5c2b6640 1 --2- 172.21.15.2:0/3746497144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c54102b30 0x7f2c54103830 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009900 tx=0x7f2c5002a410 comp rx=0 tx=0).stop 2024-05-14T10:00:40.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.936+0000 7f2c5c2b6640 1 -- 172.21.15.2:0/3746497144 shutdown_connections 2024-05-14T10:00:40.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.936+0000 7f2c5c2b6640 1 --2- 172.21.15.2:0/3746497144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c54108a70 0x7f2c5410ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.936+0000 7f2c5c2b6640 1 --2- 172.21.15.2:0/3746497144 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c54103d70 0x7f2c54104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.936+0000 7f2c5c2b6640 1 --2- 172.21.15.2:0/3746497144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c54102b30 0x7f2c54103830 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.936+0000 7f2c5c2b6640 1 -- 172.21.15.2:0/3746497144 >> 172.21.15.2:0/3746497144 conn(0x7f2c540fdd20 msgr2=0x7f2c541001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:40.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.936+0000 7f2c5c2b6640 1 -- 172.21.15.2:0/3746497144 shutdown_connections 2024-05-14T10:00:40.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.936+0000 7f2c5c2b6640 1 -- 172.21.15.2:0/3746497144 wait complete. 2024-05-14T10:00:40.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.937+0000 7f2c5c2b6640 1 Processor -- start 2024-05-14T10:00:40.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.937+0000 7f2c5c2b6640 1 -- start start 2024-05-14T10:00:40.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.937+0000 7f2c5c2b6640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c54102b30 0x7f2c541991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:40.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.937+0000 7f2c5c2b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c54103d70 0x7f2c54199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:40.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.937+0000 7f2c5c2b6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c54108a70 0x7f2c541a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:40.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.937+0000 7f2c5c2b6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5410e610 con 0x7f2c54108a70 2024-05-14T10:00:40.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.937+0000 7f2c5c2b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5410e490 con 0x7f2c54103d70 2024-05-14T10:00:40.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.937+0000 7f2c5c2b6640 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5410e790 con 0x7f2c54102b30 2024-05-14T10:00:40.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.937+0000 7f2c5a052640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c54102b30 0x7f2c541991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:40.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.937+0000 7f2c5a052640 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c54102b30 0x7f2c541991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:41586/0 (socket says 172.21.15.2:41586) 2024-05-14T10:00:40.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.937+0000 7f2c5a052640 1 -- 172.21.15.2:0/3280556014 learned_addr learned my addr 172.21.15.2:0/3280556014 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:00:40.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.937+0000 7f2c59851640 1 --2- 172.21.15.2:0/3280556014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c54103d70 0x7f2c54199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:40.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.937+0000 7f2c5a853640 1 --2- 172.21.15.2:0/3280556014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c54108a70 0x7f2c541a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:40.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.938+0000 7f2c5a853640 1 -- 172.21.15.2:0/3280556014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c54102b30 msgr2=0x7f2c541991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:40.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.938+0000 7f2c5a853640 1 --2- 172.21.15.2:0/3280556014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c54102b30 0x7f2c541991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.938+0000 7f2c5a853640 1 -- 172.21.15.2:0/3280556014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c54103d70 msgr2=0x7f2c54199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:40.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.938+0000 7f2c5a853640 1 --2- 172.21.15.2:0/3280556014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c54103d70 0x7f2c54199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:40.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.938+0000 7f2c5a853640 1 -- 172.21.15.2:0/3280556014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c541a0d20 con 0x7f2c54108a70 2024-05-14T10:00:40.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.938+0000 7f2c5a052640 1 --2- 172.21.15.2:0/3280556014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c54102b30 0x7f2c541991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T10:00:40.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.938+0000 7f2c5a853640 1 --2- 172.21.15.2:0/3280556014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c54108a70 0x7f2c541a06b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00da10 tx=0x7f2c4c00ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:40.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.938+0000 7f2c4b7fe640 1 -- 172.21.15.2:0/3280556014 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c00bd70 con 0x7f2c54108a70 2024-05-14T10:00:40.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.938+0000 7f2c4b7fe640 1 -- 172.21.15.2:0/3280556014 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c4c003e20 con 0x7f2c54108a70 2024-05-14T10:00:40.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.938+0000 7f2c5c2b6640 1 -- 172.21.15.2:0/3280556014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c541a0fb0 con 0x7f2c54108a70 2024-05-14T10:00:40.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.938+0000 7f2c4b7fe640 1 -- 172.21.15.2:0/3280556014 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0106c0 con 0x7f2c54108a70 2024-05-14T10:00:40.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.938+0000 7f2c5c2b6640 1 -- 172.21.15.2:0/3280556014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c541a1420 con 0x7f2c54108a70 2024-05-14T10:00:40.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.939+0000 7f2c5c2b6640 1 -- 172.21.15.2:0/3280556014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c54107880 con 0x7f2c54108a70 2024-05-14T10:00:40.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.941+0000 7f2c4b7fe640 1 -- 172.21.15.2:0/3280556014 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2c4c004270 con 0x7f2c54108a70 2024-05-14T10:00:40.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.942+0000 7f2c4b7fe640 1 --2- 172.21.15.2:0/3280556014 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2c400b2920 0x7f2c400b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:00:40.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.943+0000 7f2c5a052640 1 --2- 172.21.15.2:0/3280556014 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2c400b2920 0x7f2c400b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:00:40.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.943+0000 7f2c4b7fe640 1 -- 172.21.15.2:0/3280556014 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f2c4c0e2d40 con 0x7f2c54108a70 2024-05-14T10:00:40.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.943+0000 7f2c5a052640 1 --2- 172.21.15.2:0/3280556014 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2c400b2920 0x7f2c400b4e30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2c50031500 tx=0x7f2c500046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:00:40.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:40.948+0000 7f2c4b7fe640 1 -- 172.21.15.2:0/3280556014 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c4c0aa5e0 con 0x7f2c54108a70 2024-05-14T10:00:41.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:40 smithi029 ceph-mon[36249]: pgmap v1198: 129 pgs: 129 active+clean; 591 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.8 MiB/s wr, 564 op/s 2024-05-14T10:00:41.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:40 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T10:00:41.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:40 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2661074109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T10:00:41.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:40 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1092145673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T10:00:41.188 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:40 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2128887248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T10:00:41.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:41.194+0000 7f2c5c2b6640 1 -- 172.21.15.2:0/3280556014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f2c5402de00 con 0x7f2c54108a70 2024-05-14T10:00:41.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:40 smithi026 ceph-mon[36056]: pgmap v1198: 129 pgs: 129 active+clean; 591 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.8 MiB/s wr, 564 op/s 2024-05-14T10:00:41.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:40 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T10:00:41.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:40 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2661074109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T10:00:41.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:40 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1092145673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T10:00:41.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:40 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2128887248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T10:00:41.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:41.203+0000 7f2c4b7fe640 1 -- 172.21.15.2:0/3280556014 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+815633 (secure 0 0 0) 0x7f2c4c0a9ee0 con 0x7f2c54108a70 2024-05-14T10:00:41.216 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T10:00:41.216 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T10:00:41.216 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 18, 2024-05-14T10:00:41.216 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.216 INFO:teuthology.orchestra.run.smithi002.stdout: "active_gid": 14152, 2024-05-14T10:00:41.217 INFO:teuthology.orchestra.run.smithi002.stdout: "active_name": "x", 2024-05-14T10:00:41.217 INFO:teuthology.orchestra.run.smithi002.stdout: "active_addrs": { 2024-05-14T10:00:41.217 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-05-14T10:00:41.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-05-14T10:00:41.217 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-05-14T10:00:41.217 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3050449557 2024-05-14T10:00:41.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-05-14T10:00:41.217 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-05-14T10:00:41.217 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3050449557 2024-05-14T10:00:41.217 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.218 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T10:00:41.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.218 INFO:teuthology.orchestra.run.smithi002.stdout: "active_addr": "172.21.15.2:6801/3050449557", 2024-05-14T10:00:41.218 INFO:teuthology.orchestra.run.smithi002.stdout: "active_change": "2024-05-14T09:23:28.940862+0000", 2024-05-14T10:00:41.218 INFO:teuthology.orchestra.run.smithi002.stdout: "active_mgr_features": 4540701547738038271, 2024-05-14T10:00:41.218 INFO:teuthology.orchestra.run.smithi002.stdout: "available": true, 2024-05-14T10:00:41.218 INFO:teuthology.orchestra.run.smithi002.stdout: "standbys": [ 2024-05-14T10:00:41.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.218 INFO:teuthology.orchestra.run.smithi002.stdout: "gid": 24109, 2024-05-14T10:00:41.218 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "z", 2024-05-14T10:00:41.218 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_features": 4540701547738038271, 2024-05-14T10:00:41.218 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2024-05-14T10:00:41.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.219 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2024-05-14T10:00:41.219 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.219 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.219 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.219 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T10:00:41.219 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T10:00:41.219 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.219 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.219 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.219 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T10:00:41.219 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.220 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.220 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.220 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2024-05-14T10:00:41.220 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.220 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.220 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.220 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.220 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.220 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.220 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.220 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.221 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.221 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.221 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.221 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.221 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.221 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.221 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.221 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.221 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.221 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.221 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.221 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.222 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.222 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.222 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.222 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.222 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.222 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.222 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.222 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.222 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.222 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.222 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.223 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.223 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.223 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.223 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.223 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.223 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.223 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.223 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.223 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.223 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.224 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.224 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.224 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.224 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.224 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.224 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.224 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.224 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.224 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.224 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.224 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.224 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.225 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.225 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.225 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.225 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.225 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.225 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.225 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.225 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.225 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.225 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.225 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.225 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2024-05-14T10:00:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2024-05-14T10:00:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2024-05-14T10:00:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2024-05-14T10:00:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2024-05-14T10:00:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2024-05-14T10:00:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2024-05-14T10:00:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2024-05-14T10:00:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2024-05-14T10:00:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2024-05-14T10:00:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2024-05-14T10:00:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2024-05-14T10:00:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2024-05-14T10:00:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2024-05-14T10:00:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2024-05-14T10:00:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2024-05-14T10:00:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2024-05-14T10:00:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2024-05-14T10:00:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2024-05-14T10:00:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2024-05-14T10:00:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2024-05-14T10:00:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2024-05-14T10:00:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-14T10:00:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2024-05-14T10:00:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2024-05-14T10:00:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2024-05-14T10:00:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2024-05-14T10:00:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2024-05-14T10:00:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2024-05-14T10:00:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2024-05-14T10:00:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2024-05-14T10:00:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2024-05-14T10:00:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2024-05-14T10:00:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2024-05-14T10:00:41.239 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T10:00:41.239 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.239 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.242 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2024-05-14T10:00:41.242 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2024-05-14T10:00:41.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T10:00:41.242 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.242 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.242 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.242 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-05-14T10:00:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-05-14T10:00:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-14T10:00:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T10:00:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2024-05-14T10:00:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2024-05-14T10:00:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T10:00:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.244 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2024-05-14T10:00:41.244 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T10:00:41.244 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2024-05-14T10:00:41.244 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.244 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-14T10:00:41.244 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.244 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.244 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.244 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2024-05-14T10:00:41.244 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2024-05-14T10:00:41.244 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.245 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.245 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.245 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2024-05-14T10:00:41.245 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.245 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.245 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.245 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-14T10:00:41.245 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-14T10:00:41.245 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.245 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.245 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2024-05-14T10:00:41.246 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2024-05-14T10:00:41.246 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:41.246 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.246 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.246 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2024-05-14T10:00:41.246 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2024-05-14T10:00:41.246 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2024-05-14T10:00:41.246 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.246 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2024-05-14T10:00:41.246 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-14T10:00:41.247 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.247 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.247 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2024-05-14T10:00:41.247 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2024-05-14T10:00:41.247 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.247 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.247 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.247 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2024-05-14T10:00:41.247 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.247 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.247 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.247 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2024-05-14T10:00:41.247 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T10:00:41.248 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.248 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.248 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2024-05-14T10:00:41.248 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2024-05-14T10:00:41.248 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T10:00:41.248 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.248 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.248 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.248 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-05-14T10:00:41.248 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-05-14T10:00:41.248 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.248 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-14T10:00:41.249 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T10:00:41.249 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.249 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.249 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.249 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.249 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.249 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.249 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.249 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.249 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.249 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.249 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.250 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.250 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.250 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.250 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.250 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.250 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.250 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.250 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.250 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.250 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.251 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.251 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.251 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.251 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.251 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.251 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.251 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.251 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.251 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.251 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.251 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.252 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.252 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.252 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.252 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.252 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.252 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.252 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.252 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.252 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.252 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.252 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.253 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.253 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.253 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.253 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.253 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.253 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.253 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.253 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.254 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.254 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.254 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.254 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.254 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.254 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.254 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.254 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.254 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.255 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.255 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.255 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.255 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.255 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.255 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.255 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.255 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2024-05-14T10:00:41.255 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2024-05-14T10:00:41.255 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:41.256 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.256 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.256 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.256 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.256 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.256 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.256 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-14T10:00:41.256 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.256 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.256 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.256 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-05-14T10:00:41.257 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-05-14T10:00:41.257 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.257 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.257 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.257 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2024-05-14T10:00:41.257 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.257 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.257 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.257 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2024-05-14T10:00:41.257 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2024-05-14T10:00:41.257 INFO:teuthology.orchestra.run.smithi002.stdout: "read", 2024-05-14T10:00:41.257 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap", 2024-05-14T10:00:41.258 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap-read" 2024-05-14T10:00:41.258 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.258 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2024-05-14T10:00:41.258 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.258 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.258 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.258 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2024-05-14T10:00:41.258 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2024-05-14T10:00:41.258 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.258 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.258 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.259 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.259 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.259 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.259 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.259 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-14T10:00:41.259 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.259 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.259 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.259 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T10:00:41.259 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T10:00:41.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.260 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.260 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.260 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T10:00:41.260 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.260 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.260 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.260 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-14T10:00:41.260 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.260 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.260 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.260 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.260 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.261 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.261 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.261 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.261 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.261 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.261 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.261 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.261 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.261 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.261 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.261 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.261 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2024-05-14T10:00:41.262 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2024-05-14T10:00:41.262 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.262 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.262 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.262 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T10:00:41.262 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T10:00:41.262 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.262 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.262 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-14T10:00:41.262 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-14T10:00:41.262 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.262 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.263 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2024-05-14T10:00:41.263 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2024-05-14T10:00:41.263 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T10:00:41.263 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.263 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.263 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:41.263 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.263 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.263 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.263 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-14T10:00:41.263 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.263 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.264 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.264 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.264 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.264 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2024-05-14T10:00:41.264 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.264 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.264 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.264 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2024-05-14T10:00:41.264 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2024-05-14T10:00:41.264 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:41.264 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.265 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-05-14T10:00:41.265 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.265 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.265 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.265 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-14T10:00:41.265 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.265 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.265 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.265 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2024-05-14T10:00:41.265 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2024-05-14T10:00:41.266 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.266 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.266 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.266 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2024-05-14T10:00:41.266 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.266 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.266 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.266 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-14T10:00:41.266 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.266 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.267 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2024-05-14T10:00:41.267 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2024-05-14T10:00:41.267 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.267 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.267 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.267 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2024-05-14T10:00:41.267 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.267 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.267 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.267 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-14T10:00:41.267 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.267 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.268 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.268 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2024-05-14T10:00:41.268 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2024-05-14T10:00:41.268 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.268 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.268 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.268 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.268 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.268 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.268 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.268 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-14T10:00:41.268 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-14T10:00:41.269 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.269 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.269 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2024-05-14T10:00:41.269 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2024-05-14T10:00:41.269 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.269 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.269 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.269 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T10:00:41.269 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.269 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.269 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.270 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-14T10:00:41.270 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.270 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.270 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.270 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2024-05-14T10:00:41.270 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2024-05-14T10:00:41.270 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:41.270 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.270 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.270 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2024-05-14T10:00:41.270 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.271 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.271 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.271 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-14T10:00:41.271 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.271 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.271 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.271 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm_log_destination": { 2024-05-14T10:00:41.271 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm_log_destination", 2024-05-14T10:00:41.271 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.271 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.271 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.271 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.271 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.272 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.272 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.272 INFO:teuthology.orchestra.run.smithi002.stdout: "file", 2024-05-14T10:00:41.272 INFO:teuthology.orchestra.run.smithi002.stdout: "file,syslog", 2024-05-14T10:00:41.272 INFO:teuthology.orchestra.run.smithi002.stdout: "syslog" 2024-05-14T10:00:41.272 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.272 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-14T10:00:41.272 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.272 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.272 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.272 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2024-05-14T10:00:41.272 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2024-05-14T10:00:41.272 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.273 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.273 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.273 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.273 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.273 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.273 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.273 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-14T10:00:41.273 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.273 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.273 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.273 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2024-05-14T10:00:41.273 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2024-05-14T10:00:41.273 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.274 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.274 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.274 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.274 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.274 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.274 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.274 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-14T10:00:41.274 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.274 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.274 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.275 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2024-05-14T10:00:41.275 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2024-05-14T10:00:41.275 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.275 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.275 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.275 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.275 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.275 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.275 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.275 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-14T10:00:41.275 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.275 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.275 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.276 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2024-05-14T10:00:41.276 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2024-05-14T10:00:41.276 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.276 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.276 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.276 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-14T10:00:41.276 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.276 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.276 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.276 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T10:00:41.276 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.276 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.276 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.277 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2024-05-14T10:00:41.277 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2024-05-14T10:00:41.277 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.277 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.277 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.277 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-14T10:00:41.277 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.277 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.277 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.277 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2024-05-14T10:00:41.277 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.277 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.277 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.278 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2024-05-14T10:00:41.278 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2024-05-14T10:00:41.278 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.278 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.278 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.278 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-14T10:00:41.278 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.278 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.278 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.278 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2024-05-14T10:00:41.279 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.279 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.279 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2024-05-14T10:00:41.279 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2024-05-14T10:00:41.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.279 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.279 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.279 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-14T10:00:41.279 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.279 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.279 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.280 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T10:00:41.280 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.280 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.280 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.280 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2024-05-14T10:00:41.280 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2024-05-14T10:00:41.280 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.280 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.280 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.280 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-14T10:00:41.280 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.281 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.281 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.281 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2024-05-14T10:00:41.281 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.281 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.281 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.281 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2024-05-14T10:00:41.281 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2024-05-14T10:00:41.281 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.281 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.281 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.281 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-14T10:00:41.282 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.282 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.282 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.282 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2024-05-14T10:00:41.282 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.282 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.282 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.282 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2024-05-14T10:00:41.282 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2024-05-14T10:00:41.282 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.282 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.282 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.283 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-14T10:00:41.283 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.283 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.283 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.283 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2024-05-14T10:00:41.283 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.283 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.283 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.283 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2024-05-14T10:00:41.283 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2024-05-14T10:00:41.283 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.283 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.284 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.284 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-14T10:00:41.284 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.284 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.284 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.284 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2024-05-14T10:00:41.284 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.284 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.284 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.284 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2024-05-14T10:00:41.284 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2024-05-14T10:00:41.284 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.285 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.285 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-14T10:00:41.285 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.285 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.285 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.285 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2024-05-14T10:00:41.285 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.285 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.285 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.285 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2024-05-14T10:00:41.285 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2024-05-14T10:00:41.285 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.286 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.286 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.286 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-14T10:00:41.286 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.286 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.286 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.286 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2024-05-14T10:00:41.286 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.286 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.286 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.286 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2024-05-14T10:00:41.286 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2024-05-14T10:00:41.287 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.287 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.287 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.287 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-14T10:00:41.287 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.287 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.287 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.287 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T10:00:41.287 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.287 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.287 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.287 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_nvmeof": { 2024-05-14T10:00:41.287 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_nvmeof", 2024-05-14T10:00:41.288 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.288 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.288 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.288 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-14T10:00:41.288 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.288 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.288 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.288 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Nvme-of container image", 2024-05-14T10:00:41.288 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.288 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.288 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.288 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2024-05-14T10:00:41.288 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2024-05-14T10:00:41.289 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.289 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.289 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.289 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-14T10:00:41.289 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.289 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.289 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.289 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T10:00:41.289 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.289 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.289 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.290 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2024-05-14T10:00:41.290 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2024-05-14T10:00:41.290 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.290 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.290 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.290 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-14T10:00:41.290 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.290 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.290 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.290 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2024-05-14T10:00:41.290 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.291 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.291 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_samba": { 2024-05-14T10:00:41.291 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_samba", 2024-05-14T10:00:41.291 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.291 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.291 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.291 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-14T10:00:41.291 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.291 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.291 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.292 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Samba/SMB container image", 2024-05-14T10:00:41.292 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.292 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.292 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.292 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2024-05-14T10:00:41.292 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2024-05-14T10:00:41.292 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.292 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.292 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.292 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-14T10:00:41.292 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.293 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.293 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.293 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2024-05-14T10:00:41.293 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.293 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.293 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.293 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2024-05-14T10:00:41.293 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2024-05-14T10:00:41.293 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.293 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.293 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.294 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.294 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.294 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.294 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.294 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2024-05-14T10:00:41.294 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.294 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.294 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2024-05-14T10:00:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2024-05-14T10:00:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T10:00:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-14T10:00:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2024-05-14T10:00:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2024-05-14T10:00:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2024-05-14T10:00:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-14T10:00:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2024-05-14T10:00:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2024-05-14T10:00:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2024-05-14T10:00:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-14T10:00:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2024-05-14T10:00:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2024-05-14T10:00:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2024-05-14T10:00:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2024-05-14T10:00:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2024-05-14T10:00:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2024-05-14T10:00:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-14T10:00:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2024-05-14T10:00:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2024-05-14T10:00:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T10:00:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2024-05-14T10:00:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: "grafana_dashboards_path": { 2024-05-14T10:00:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "grafana_dashboards_path", 2024-05-14T10:00:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.303 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-14T10:00:41.303 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.303 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.303 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.303 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-14T10:00:41.303 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.303 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.303 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.303 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2024-05-14T10:00:41.303 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2024-05-14T10:00:41.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.303 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.303 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.303 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T10:00:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2024-05-14T10:00:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "hw_monitoring": { 2024-05-14T10:00:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hw_monitoring", 2024-05-14T10:00:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-14T10:00:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "inventory_list_all": { 2024-05-14T10:00:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "inventory_list_all", 2024-05-14T10:00:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-14T10:00:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2024-05-14T10:00:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2024-05-14T10:00:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-14T10:00:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-14T10:00:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-14T10:00:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-14T10:00:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-14T10:00:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-14T10:00:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-14T10:00:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2024-05-14T10:00:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-14T10:00:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2024-05-14T10:00:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2024-05-14T10:00:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2024-05-14T10:00:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2024-05-14T10:00:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2024-05-14T10:00:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-14T10:00:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2024-05-14T10:00:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2024-05-14T10:00:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2024-05-14T10:00:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-05-14T10:00:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-05-14T10:00:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2024-05-14T10:00:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2024-05-14T10:00:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2024-05-14T10:00:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2024-05-14T10:00:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "oob_default_addr": { 2024-05-14T10:00:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "oob_default_addr", 2024-05-14T10:00:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "169.254.1.1", 2024-05-14T10:00:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-14T10:00:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2024-05-14T10:00:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2024-05-14T10:00:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-14T10:00:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-14T10:00:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2024-05-14T10:00:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2024-05-14T10:00:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-14T10:00:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2024-05-14T10:00:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2024-05-14T10:00:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-14T10:00:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2024-05-14T10:00:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2024-05-14T10:00:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-14T10:00:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2024-05-14T10:00:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2024-05-14T10:00:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-14T10:00:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2024-05-14T10:00:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2024-05-14T10:00:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-14T10:00:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2024-05-14T10:00:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2024-05-14T10:00:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2024-05-14T10:00:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2024-05-14T10:00:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2024-05-14T10:00:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2024-05-14T10:00:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-14T10:00:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2024-05-14T10:00:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2024-05-14T10:00:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-14T10:00:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2024-05-14T10:00:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2024-05-14T10:00:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-14T10:00:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2024-05-14T10:00:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2024-05-14T10:00:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2024-05-14T10:00:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2024-05-14T10:00:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2024-05-14T10:00:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-14T10:00:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2024-05-14T10:00:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2024-05-14T10:00:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-14T10:00:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2024-05-14T10:00:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2024-05-14T10:00:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2024-05-14T10:00:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2024-05-14T10:00:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-14T10:00:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2024-05-14T10:00:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2024-05-14T10:00:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2024-05-14T10:00:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-14T10:00:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2024-05-14T10:00:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-14T10:00:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-14T10:00:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2024-05-14T10:00:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-14T10:00:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-14T10:00:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-14T10:00:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2024-05-14T10:00:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2024-05-14T10:00:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-14T10:00:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-14T10:00:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2024-05-14T10:00:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-14T10:00:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-14T10:00:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-14T10:00:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-14T10:00:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-14T10:00:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-14T10:00:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-14T10:00:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-14T10:00:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-14T10:00:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-14T10:00:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-14T10:00:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-14T10:00:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-14T10:00:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-14T10:00:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-14T10:00:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-14T10:00:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-14T10:00:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2024-05-14T10:00:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-14T10:00:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T10:00:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-14T10:00:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-14T10:00:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2024-05-14T10:00:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2024-05-14T10:00:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2024-05-14T10:00:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-14T10:00:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T10:00:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-14T10:00:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-14T10:00:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-14T10:00:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-14T10:00:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-14T10:00:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-14T10:00:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-14T10:00:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-14T10:00:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-14T10:00:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-14T10:00:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "[]", 2024-05-14T10:00:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "MULTICLUSTER_CONFIG": { 2024-05-14T10:00:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-14T10:00:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "{}", 2024-05-14T10:00:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2024-05-14T10:00:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-14T10:00:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-14T10:00:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-14T10:00:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-14T10:00:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-14T10:00:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-14T10:00:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-14T10:00:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-14T10:00:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-14T10:00:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-14T10:00:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-14T10:00:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-14T10:00:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-14T10:00:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-14T10:00:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-14T10:00:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-14T10:00:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-14T10:00:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2024-05-14T10:00:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-14T10:00:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-14T10:00:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-14T10:00:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-14T10:00:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-14T10:00:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-14T10:00:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-14T10:00:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-14T10:00:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2024-05-14T10:00:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-14T10:00:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-14T10:00:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2024-05-14T10:00:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2024-05-14T10:00:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-14T10:00:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.392 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.392 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-14T10:00:41.392 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-14T10:00:41.392 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.392 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.392 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.392 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T10:00:41.392 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.392 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.392 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.392 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.393 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.393 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.393 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.393 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2024-05-14T10:00:41.393 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-14T10:00:41.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.393 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.393 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.393 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.393 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.393 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.394 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.394 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.394 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.394 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.394 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2024-05-14T10:00:41.394 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-14T10:00:41.394 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.394 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.394 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.394 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.394 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.395 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.395 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.395 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.395 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.395 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.395 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.395 INFO:teuthology.orchestra.run.smithi002.stdout: "UNSAFE_TLS_v1_2": { 2024-05-14T10:00:41.395 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-14T10:00:41.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.395 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.395 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.395 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.395 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.396 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.396 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.396 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.396 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.396 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.396 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.396 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-14T10:00:41.396 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-14T10:00:41.396 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.396 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.396 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.396 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.397 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.397 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.397 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.397 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.397 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.397 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.397 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.397 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-14T10:00:41.397 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-14T10:00:41.397 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.397 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.397 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.397 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:41.398 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.398 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.398 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.398 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.398 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.398 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.398 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.398 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-14T10:00:41.398 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-14T10:00:41.398 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.398 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.399 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.399 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T10:00:41.399 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.399 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.399 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.399 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.399 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.399 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.399 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.399 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2024-05-14T10:00:41.399 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2024-05-14T10:00:41.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.400 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.400 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.400 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.400 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.400 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.400 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.400 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.400 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.401 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2024-05-14T10:00:41.401 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2024-05-14T10:00:41.401 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.401 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.401 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.401 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.401 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.401 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.401 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.401 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.401 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.402 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.402 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.402 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2024-05-14T10:00:41.402 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2024-05-14T10:00:41.402 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.402 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.402 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.402 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.402 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.402 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.402 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.402 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2024-05-14T10:00:41.403 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.403 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.403 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.403 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2024-05-14T10:00:41.403 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2024-05-14T10:00:41.403 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.403 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.403 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.403 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2024-05-14T10:00:41.403 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.403 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.403 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.403 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.404 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.404 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.404 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.404 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-05-14T10:00:41.404 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-05-14T10:00:41.404 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.404 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.404 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.404 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.404 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.404 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.405 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.405 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.405 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.405 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.405 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.405 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.405 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.405 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.405 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.405 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.405 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.405 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.405 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.406 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.406 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.406 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.406 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.406 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.406 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.406 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.406 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.406 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.406 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.406 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.406 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.407 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.407 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.407 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.407 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.407 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.407 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.407 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.407 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.407 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.407 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.407 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.407 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.408 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.408 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.408 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.408 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.408 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.408 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.408 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.408 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.408 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.408 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.408 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.408 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.408 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.408 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.409 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.409 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.409 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.409 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.409 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.409 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.409 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.409 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.409 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.409 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.410 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.410 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.410 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.410 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.410 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.410 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.410 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.410 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.410 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.410 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2024-05-14T10:00:41.410 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2024-05-14T10:00:41.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.411 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.411 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.411 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.411 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.411 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.411 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2024-05-14T10:00:41.411 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.411 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.412 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2024-05-14T10:00:41.412 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2024-05-14T10:00:41.412 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.412 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.412 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.412 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.412 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.412 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.412 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.412 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.412 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.412 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.412 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.413 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T10:00:41.413 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T10:00:41.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.413 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.413 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.413 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-05-14T10:00:41.413 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.413 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.413 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.413 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.413 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.413 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.413 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.414 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T10:00:41.414 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T10:00:41.414 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.414 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.414 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.414 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2024-05-14T10:00:41.414 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.414 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.414 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.414 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.414 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-05-14T10:00:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-05-14T10:00:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.417 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.417 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.417 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.417 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.417 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.417 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.417 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2024-05-14T10:00:41.417 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2024-05-14T10:00:41.417 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.417 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.418 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.418 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2024-05-14T10:00:41.418 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.418 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.418 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.418 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.418 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.418 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.418 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.418 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-05-14T10:00:41.418 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-05-14T10:00:41.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.419 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.419 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.419 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2024-05-14T10:00:41.419 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.419 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.419 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.419 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.419 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2024-05-14T10:00:41.419 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.419 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.419 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.420 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.420 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.420 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-05-14T10:00:41.420 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-05-14T10:00:41.420 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.420 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.420 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.420 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-05-14T10:00:41.420 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-05-14T10:00:41.420 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-05-14T10:00:41.421 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.421 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.421 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.421 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.421 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.421 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2024-05-14T10:00:41.421 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2024-05-14T10:00:41.421 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.421 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.421 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.422 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.422 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.422 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.422 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.422 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.422 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.422 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.422 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.422 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.422 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.423 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-05-14T10:00:41.423 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.423 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.423 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.423 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2024-05-14T10:00:41.423 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2024-05-14T10:00:41.423 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.423 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.423 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.423 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.423 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.424 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.424 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.424 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2024-05-14T10:00:41.424 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.424 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.424 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.424 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.424 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.424 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.424 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.424 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.425 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.425 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.425 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.425 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.425 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.425 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.425 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.425 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.425 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.425 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.425 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.425 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.426 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.426 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.426 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.426 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.426 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.426 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.426 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.426 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.426 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.426 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.426 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.426 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.427 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.427 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.427 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.427 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.427 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.427 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.427 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.427 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.427 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.427 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.427 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.428 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.428 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.428 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.428 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.428 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.428 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.428 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.428 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.428 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.428 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.428 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.428 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.429 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.429 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.429 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.429 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.429 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.429 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.429 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.429 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.429 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.429 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.430 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.430 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.430 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2024-05-14T10:00:41.430 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2024-05-14T10:00:41.430 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.430 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.430 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2024-05-14T10:00:41.430 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.430 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.430 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.431 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-14T10:00:41.431 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.431 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.431 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2024-05-14T10:00:41.431 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2024-05-14T10:00:41.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.431 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.431 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.431 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2024-05-14T10:00:41.432 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.432 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.432 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.432 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-14T10:00:41.432 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.432 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.432 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.432 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2024-05-14T10:00:41.432 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2024-05-14T10:00:41.432 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.432 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.433 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.433 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2024-05-14T10:00:41.433 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.433 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.433 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.433 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2024-05-14T10:00:41.433 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.433 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.433 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.433 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2024-05-14T10:00:41.433 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2024-05-14T10:00:41.433 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.433 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.434 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.434 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-05-14T10:00:41.434 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.434 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.434 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.434 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-14T10:00:41.434 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.434 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.434 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.434 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2024-05-14T10:00:41.434 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2024-05-14T10:00:41.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.435 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.435 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.435 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.435 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.435 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.435 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.435 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-14T10:00:41.435 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.435 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.435 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.435 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T10:00:41.435 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T10:00:41.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.436 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.436 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.436 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T10:00:41.436 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.436 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.436 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.436 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2024-05-14T10:00:41.436 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.436 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.436 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.436 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.436 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.437 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.437 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.437 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.437 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.437 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.437 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.437 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.437 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.437 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.437 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.437 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2024-05-14T10:00:41.437 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2024-05-14T10:00:41.438 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.438 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.438 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.438 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2024-05-14T10:00:41.438 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.438 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.438 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.438 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-14T10:00:41.438 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.438 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.438 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.439 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2024-05-14T10:00:41.439 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.439 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.439 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.439 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.439 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.439 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.439 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.440 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.440 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.440 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.440 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.440 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.440 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.440 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.440 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.440 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.440 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.440 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.440 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.440 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.441 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.441 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.441 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.441 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.441 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.441 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.441 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.441 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.441 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.441 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.441 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.441 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.442 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.442 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.442 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.442 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.442 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.442 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.442 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.442 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.442 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.442 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.442 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.443 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.443 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.443 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.443 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.443 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.443 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.446 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.446 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.446 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.446 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.446 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.446 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.447 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.447 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.447 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.447 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.447 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.447 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.447 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.447 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.447 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.447 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.447 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2024-05-14T10:00:41.448 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2024-05-14T10:00:41.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.448 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.448 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-05-14T10:00:41.448 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.448 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.448 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.448 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.448 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.448 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.448 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.449 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2024-05-14T10:00:41.449 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2024-05-14T10:00:41.449 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.449 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.449 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.449 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2024-05-14T10:00:41.449 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.449 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.449 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.449 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.449 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.449 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.450 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.450 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T10:00:41.450 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T10:00:41.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.450 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.450 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.450 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T10:00:41.450 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.450 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.450 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.451 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.451 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.451 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.451 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.451 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.451 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.451 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.451 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.451 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.451 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.452 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.452 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.452 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.452 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.452 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.452 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.452 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.452 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.452 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2024-05-14T10:00:41.452 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2024-05-14T10:00:41.453 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2024-05-14T10:00:41.453 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.453 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2024-05-14T10:00:41.453 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2024-05-14T10:00:41.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.453 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.453 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.453 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2024-05-14T10:00:41.453 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.453 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.453 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.453 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-14T10:00:41.454 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.454 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.454 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.454 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2024-05-14T10:00:41.454 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2024-05-14T10:00:41.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.454 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.454 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.454 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2024-05-14T10:00:41.454 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.454 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.454 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.455 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-14T10:00:41.455 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.455 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.455 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.455 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2024-05-14T10:00:41.455 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2024-05-14T10:00:41.455 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.455 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.455 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.455 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.455 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.455 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.456 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.456 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2024-05-14T10:00:41.456 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.456 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.456 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.456 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T10:00:41.456 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T10:00:41.456 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.456 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.456 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.456 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2024-05-14T10:00:41.456 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2024-05-14T10:00:41.457 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.457 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.457 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-14T10:00:41.457 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.457 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.457 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.457 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.457 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.457 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.458 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.458 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.458 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.458 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.458 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.458 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.458 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.458 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.458 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.458 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.458 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.458 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.459 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.459 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.459 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.459 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.459 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.459 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.459 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.459 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.459 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.459 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.460 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.460 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.460 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.460 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.460 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.460 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.460 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.460 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.460 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.460 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.460 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.460 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.461 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.461 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.461 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.461 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.461 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.461 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.461 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.461 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.461 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.461 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.461 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.461 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.461 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.462 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.462 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.462 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.462 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.462 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.462 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.462 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.462 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.462 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.462 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.462 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.462 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.463 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.463 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.463 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2024-05-14T10:00:41.463 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2024-05-14T10:00:41.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.463 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.463 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.463 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.463 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.463 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.463 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.463 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2024-05-14T10:00:41.464 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.464 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.464 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.464 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2024-05-14T10:00:41.464 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2024-05-14T10:00:41.464 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.464 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.464 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.464 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2024-05-14T10:00:41.464 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.464 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.464 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.465 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2024-05-14T10:00:41.465 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.465 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.465 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.465 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.465 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.465 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.465 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.465 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.465 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.465 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.465 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.465 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.465 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.466 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.466 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.466 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.466 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-05-14T10:00:41.466 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-05-14T10:00:41.466 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.466 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.466 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.466 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2024-05-14T10:00:41.466 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.466 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.466 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.467 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T10:00:41.467 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.467 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.467 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.467 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2024-05-14T10:00:41.467 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2024-05-14T10:00:41.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.467 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.467 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T10:00:41.467 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T10:00:41.468 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2024-05-14T10:00:41.468 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.468 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-14T10:00:41.468 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.468 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.468 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2024-05-14T10:00:41.468 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2024-05-14T10:00:41.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.468 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.468 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.469 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.469 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.469 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.469 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2024-05-14T10:00:41.469 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.469 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.469 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.469 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2024-05-14T10:00:41.469 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2024-05-14T10:00:41.469 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.469 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.469 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.470 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2024-05-14T10:00:41.470 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.470 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.470 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.470 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T10:00:41.470 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.470 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.470 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.470 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.470 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.471 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2024-05-14T10:00:41.471 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.471 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.471 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.471 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.471 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.471 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.471 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.471 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.471 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.471 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.471 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.471 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.472 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.472 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.472 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.472 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.472 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.472 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.472 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.472 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.472 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.472 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.472 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.473 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.473 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.473 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.473 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.473 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.473 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.473 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.473 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.473 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.473 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.473 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.474 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.474 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.474 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.474 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.474 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.474 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.474 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.474 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.474 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.474 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.475 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.475 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.475 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.475 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.475 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.475 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.475 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.475 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.475 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.475 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.476 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.476 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.476 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.476 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.476 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.476 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.476 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.476 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.476 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.476 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.476 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.477 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.477 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.477 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.477 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.477 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.477 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.477 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.477 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.477 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.477 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.478 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.478 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.478 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2024-05-14T10:00:41.478 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.478 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.478 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.479 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.479 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.479 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.479 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.479 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.479 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.479 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.479 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.479 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.479 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.480 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.480 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.480 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.480 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.480 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.480 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.480 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.480 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.480 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.480 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.481 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.481 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.481 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.481 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.481 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.481 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.481 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.481 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.481 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.481 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2024-05-14T10:00:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2024-05-14T10:00:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2024-05-14T10:00:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2024-05-14T10:00:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-14T10:00:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2024-05-14T10:00:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2024-05-14T10:00:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2024-05-14T10:00:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-14T10:00:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2024-05-14T10:00:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2024-05-14T10:00:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2024-05-14T10:00:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2024-05-14T10:00:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2024-05-14T10:00:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-14T10:00:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2024-05-14T10:00:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2024-05-14T10:00:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2024-05-14T10:00:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-14T10:00:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2024-05-14T10:00:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2024-05-14T10:00:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2024-05-14T10:00:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2024-05-14T10:00:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2024-05-14T10:00:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2024-05-14T10:00:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2024-05-14T10:00:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2024-05-14T10:00:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2024-05-14T10:00:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2024-05-14T10:00:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2024-05-14T10:00:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2024-05-14T10:00:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2024-05-14T10:00:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2024-05-14T10:00:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-14T10:00:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-14T10:00:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2024-05-14T10:00:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2024-05-14T10:00:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2024-05-14T10:00:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-05-14T10:00:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2024-05-14T10:00:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2024-05-14T10:00:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-14T10:00:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2024-05-14T10:00:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-05-14T10:00:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-05-14T10:00:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2024-05-14T10:00:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2024-05-14T10:00:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2024-05-14T10:00:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2024-05-14T10:00:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2024-05-14T10:00:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2024-05-14T10:00:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T10:00:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T10:00:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T10:00:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2024-05-14T10:00:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2024-05-14T10:00:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-05-14T10:00:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2024-05-14T10:00:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2024-05-14T10:00:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-14T10:00:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2024-05-14T10:00:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2024-05-14T10:00:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2024-05-14T10:00:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-14T10:00:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-05-14T10:00:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-05-14T10:00:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.572 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.572 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.572 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.572 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.572 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.572 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.572 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.572 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.572 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.572 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.572 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.572 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.573 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.573 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.573 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.573 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.573 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.573 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.573 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.573 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.573 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.573 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.573 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.573 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.574 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.574 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.574 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.574 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.574 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.574 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2024-05-14T10:00:41.574 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2024-05-14T10:00:41.574 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.574 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.574 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.574 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2024-05-14T10:00:41.575 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.575 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.575 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.575 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2024-05-14T10:00:41.575 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.575 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.575 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.575 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T10:00:41.575 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T10:00:41.575 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.575 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.575 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.575 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T10:00:41.576 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.576 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.576 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.576 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2024-05-14T10:00:41.576 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.576 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.576 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.576 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.576 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.576 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.577 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.577 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.577 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.577 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.577 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.577 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.577 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.577 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.578 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2024-05-14T10:00:41.578 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.578 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.578 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.578 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2024-05-14T10:00:41.578 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2024-05-14T10:00:41.578 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.578 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.578 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.578 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.578 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.578 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.579 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.579 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.579 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.579 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.579 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.579 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2024-05-14T10:00:41.579 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2024-05-14T10:00:41.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.579 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.580 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.580 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.580 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.580 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.580 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-14T10:00:41.580 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-14T10:00:41.580 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.580 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.580 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.580 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.580 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.581 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.581 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.581 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.581 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.581 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.581 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.581 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.581 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.581 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.581 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.582 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.582 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.582 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.582 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.582 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.582 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.582 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.582 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.582 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.583 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.583 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.583 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.583 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.583 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.583 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.583 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.583 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.583 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.583 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.583 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.584 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.584 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.584 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.584 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.584 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.584 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.584 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.584 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.584 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.584 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.584 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.585 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.585 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.585 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.585 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.585 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.585 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.585 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.585 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.585 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.585 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.586 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.586 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.586 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.586 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.586 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.586 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.586 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.586 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.586 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.586 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.586 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.587 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2024-05-14T10:00:41.587 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2024-05-14T10:00:41.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.587 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.587 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.587 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.587 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.587 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.587 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.587 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.587 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.587 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.587 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.588 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-14T10:00:41.588 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-14T10:00:41.588 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.588 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.588 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.588 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2024-05-14T10:00:41.588 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.588 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.588 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.588 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.588 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.588 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.589 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.589 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2024-05-14T10:00:41.589 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2024-05-14T10:00:41.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:41.589 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.589 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.589 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2024-05-14T10:00:41.589 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.589 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.589 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.589 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.590 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.590 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.590 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.590 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T10:00:41.590 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T10:00:41.590 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.590 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.590 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.590 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-05-14T10:00:41.591 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.591 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.591 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.591 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-14T10:00:41.591 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.591 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.591 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.591 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T10:00:41.591 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T10:00:41.591 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.591 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.591 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.592 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2024-05-14T10:00:41.592 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.592 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.592 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.592 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-14T10:00:41.592 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.592 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.592 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.592 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.592 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.592 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.593 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.593 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.593 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.593 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.593 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.593 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.593 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.593 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.593 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.593 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2024-05-14T10:00:41.593 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2024-05-14T10:00:41.594 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.594 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.594 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.594 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2024-05-14T10:00:41.594 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.594 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.594 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.594 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.594 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.594 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.594 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.595 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-05-14T10:00:41.595 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-05-14T10:00:41.595 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.595 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.595 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.595 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2024-05-14T10:00:41.595 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.595 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.595 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.595 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2024-05-14T10:00:41.595 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2024-05-14T10:00:41.595 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.595 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.595 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.596 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.596 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.596 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-05-14T10:00:41.596 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-05-14T10:00:41.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.596 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.596 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.596 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-05-14T10:00:41.596 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-05-14T10:00:41.596 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-05-14T10:00:41.596 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.597 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.597 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.597 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.597 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-05-14T10:00:41.597 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.597 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.598 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.598 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.598 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.598 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.598 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.598 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.598 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.598 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.598 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.598 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.598 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.598 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.598 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.599 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.599 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.599 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.599 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.599 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.599 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.599 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.599 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.599 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.599 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.599 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.600 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.600 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.600 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.600 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.600 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.600 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.600 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.600 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.600 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.600 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.600 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.601 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.601 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.601 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.601 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.601 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.601 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.601 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.601 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.601 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.601 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.601 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.601 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.602 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.602 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.602 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.602 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.602 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.602 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.602 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.602 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.602 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.602 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.603 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.603 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.603 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.603 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.603 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.603 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.603 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.603 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.603 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2024-05-14T10:00:41.603 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2024-05-14T10:00:41.603 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.604 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.604 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.604 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:41.604 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.604 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.604 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.604 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.604 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.604 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.604 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.604 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2024-05-14T10:00:41.605 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2024-05-14T10:00:41.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.605 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.605 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.605 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.605 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.605 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.605 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.605 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.605 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.605 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.605 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.606 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.606 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.606 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.606 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.606 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.606 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.606 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.606 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.607 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.607 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.607 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2024-05-14T10:00:41.607 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2024-05-14T10:00:41.607 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.607 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.607 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.607 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.607 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.607 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.608 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.608 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.608 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.608 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.608 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.608 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.608 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2024-05-14T10:00:41.608 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.608 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.609 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.609 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2024-05-14T10:00:41.609 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2024-05-14T10:00:41.609 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.609 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.609 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.609 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.609 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.609 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.609 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.609 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.609 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.610 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.610 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.610 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-05-14T10:00:41.610 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-05-14T10:00:41.610 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.610 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.610 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.610 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.610 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.610 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.610 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.611 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.611 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.611 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.611 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.611 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.611 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.611 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.611 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.611 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.611 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.611 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.612 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.612 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.612 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.612 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.612 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.612 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.612 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.612 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.612 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.612 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.612 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.612 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.613 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.613 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.613 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.613 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.613 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.613 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.613 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.613 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.613 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.613 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.613 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.614 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.614 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.614 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.614 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.614 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.614 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.614 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.614 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.614 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.614 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.614 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.615 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.615 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.615 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.615 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.615 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.615 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.615 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.615 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.615 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.615 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.615 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.616 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.616 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.616 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.616 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.616 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.616 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.616 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.616 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.616 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.616 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.616 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.617 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.617 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T10:00:41.617 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T10:00:41.617 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.617 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.617 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.617 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.617 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.617 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.617 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.617 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.618 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.618 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.618 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.618 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T10:00:41.618 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T10:00:41.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.618 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.618 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.618 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.618 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.619 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.619 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.619 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.619 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.619 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.619 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.619 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.619 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.619 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.619 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.619 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.620 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.620 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.620 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.620 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.620 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.620 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.620 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.620 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.620 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.620 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2024-05-14T10:00:41.621 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.621 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.621 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.621 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.621 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.621 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.621 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.621 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.621 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.621 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.621 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.621 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.622 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.622 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.622 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.622 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.622 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.622 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.622 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.622 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.622 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.622 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.622 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.622 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.623 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.623 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.623 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.623 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.623 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.623 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.623 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.623 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.623 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.623 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.624 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.624 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.624 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.624 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.624 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.624 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.624 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.624 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.624 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.624 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.624 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.624 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.624 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.624 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.625 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.625 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.625 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.625 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.625 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.625 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.625 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.625 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.625 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.625 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.625 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.625 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.625 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.626 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.626 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.626 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.626 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.626 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.626 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.626 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.626 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.626 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.627 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.627 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.627 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.627 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.627 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.627 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.627 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2024-05-14T10:00:41.627 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.628 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.628 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.628 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.628 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.628 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.628 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.628 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.628 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.628 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.628 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.628 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.628 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.629 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.629 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.629 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.629 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.629 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.629 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.629 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.629 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.629 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.630 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.630 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.630 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.630 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.630 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.630 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.630 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.630 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.630 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.631 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.631 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.631 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.631 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.631 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.631 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.631 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.631 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.631 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.631 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.632 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.632 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.632 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.632 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.632 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.632 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.632 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.632 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.632 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.632 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.632 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.633 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.633 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.633 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.633 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.633 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.633 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.633 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.633 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.634 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.634 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.634 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.634 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.634 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.634 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.634 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.634 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.634 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.635 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.635 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.635 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.635 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.635 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.635 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.635 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.635 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.635 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.636 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2024-05-14T10:00:41.636 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2024-05-14T10:00:41.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.636 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.636 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2024-05-14T10:00:41.636 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.636 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.636 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.637 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-14T10:00:41.637 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.637 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.637 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.637 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2024-05-14T10:00:41.637 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.637 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.637 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.638 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.638 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.638 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.638 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.638 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.638 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.638 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.638 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.638 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.639 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.639 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.639 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.639 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.639 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.639 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.639 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.639 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.639 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.639 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.639 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.640 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.640 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.640 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.640 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.640 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.640 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.640 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.640 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.640 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.640 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.641 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.641 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.641 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.641 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.641 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.641 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.641 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.641 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.641 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.641 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.641 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.642 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.642 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.642 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.642 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.642 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.642 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.642 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.642 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.642 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.642 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.643 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.643 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.643 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.643 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.643 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.643 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.643 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.643 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.643 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.643 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.644 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2024-05-14T10:00:41.644 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2024-05-14T10:00:41.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.644 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.644 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.644 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.644 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.644 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.644 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.644 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.645 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.645 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.645 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2024-05-14T10:00:41.645 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2024-05-14T10:00:41.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.645 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.645 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.645 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2024-05-14T10:00:41.645 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.645 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.645 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.646 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.646 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.646 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.646 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.646 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2024-05-14T10:00:41.646 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2024-05-14T10:00:41.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.646 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.646 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.646 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.646 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.647 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.647 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.647 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.647 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.647 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.647 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.647 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2024-05-14T10:00:41.647 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2024-05-14T10:00:41.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.648 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.648 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.648 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.648 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.648 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.648 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.648 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.648 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.649 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2024-05-14T10:00:41.649 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2024-05-14T10:00:41.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:41.649 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.649 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.649 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.649 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.649 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.649 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.649 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.650 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.650 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.650 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.650 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2024-05-14T10:00:41.650 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2024-05-14T10:00:41.650 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.650 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.650 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.650 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.650 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.650 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.651 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.651 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.651 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.651 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.651 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2024-05-14T10:00:41.651 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2024-05-14T10:00:41.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.651 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.651 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.652 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.652 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.652 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.652 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.652 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.652 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.652 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.652 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2024-05-14T10:00:41.652 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2024-05-14T10:00:41.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.652 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.653 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.653 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.653 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.653 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.653 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.653 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.653 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.653 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2024-05-14T10:00:41.653 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2024-05-14T10:00:41.654 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.654 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.654 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.654 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.654 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T10:00:41.654 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2024-05-14T10:00:41.654 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.654 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.654 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.654 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.654 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.655 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.655 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.655 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.655 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.655 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.655 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.655 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.655 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.655 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.655 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.656 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.656 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2024-05-14T10:00:41.656 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2024-05-14T10:00:41.656 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.656 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.656 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.656 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.656 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.656 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.656 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.656 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.657 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.657 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2024-05-14T10:00:41.657 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2024-05-14T10:00:41.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.657 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.657 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.657 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.657 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.657 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.657 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.657 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.657 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.658 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.658 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.658 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2024-05-14T10:00:41.658 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2024-05-14T10:00:41.658 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.658 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.658 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.658 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.658 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.658 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.659 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.659 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.659 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.659 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.659 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.659 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smb", 2024-05-14T10:00:41.659 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.659 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.659 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.660 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.660 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.660 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.660 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.660 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.660 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.660 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.660 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.660 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.660 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.660 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.661 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.661 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.661 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.661 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.661 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.661 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.661 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.661 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.661 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.661 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.661 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.662 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.662 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.662 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.662 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.662 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.662 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.662 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.662 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.662 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.662 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.662 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.662 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.663 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.663 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.663 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.663 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.663 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.663 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.663 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.663 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.663 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.663 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.663 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.663 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.664 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.664 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.664 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.664 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.664 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.664 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.664 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.664 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.664 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.664 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.665 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.665 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.665 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.665 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.665 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.665 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.665 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.665 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.665 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.666 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.666 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.666 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.666 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.666 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.666 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.666 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.666 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.666 INFO:teuthology.orchestra.run.smithi002.stdout: "update_orchestration": { 2024-05-14T10:00:41.666 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "update_orchestration", 2024-05-14T10:00:41.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.666 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.667 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.667 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.667 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.667 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.667 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.667 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-14T10:00:41.667 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.667 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.667 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.668 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-05-14T10:00:41.668 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.668 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.668 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.668 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2024-05-14T10:00:41.668 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2024-05-14T10:00:41.668 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.668 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.668 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.668 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.668 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.669 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.669 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.669 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2024-05-14T10:00:41.669 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.669 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.669 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.669 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2024-05-14T10:00:41.669 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2024-05-14T10:00:41.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.669 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.670 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.670 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.670 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.670 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.670 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.670 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2024-05-14T10:00:41.670 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.670 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.670 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.670 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.671 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.671 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.671 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.671 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.671 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.671 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.671 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.671 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.671 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.671 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.671 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.671 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.672 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.672 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.672 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.672 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.672 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.672 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.672 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.672 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.672 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.672 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.672 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.672 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.673 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.673 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.673 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.673 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.673 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.673 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.673 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.673 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.673 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.673 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.674 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.674 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.674 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.674 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.674 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.674 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.674 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.674 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.674 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.674 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.674 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.674 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.674 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.675 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.675 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.675 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.675 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.675 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.675 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.675 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.675 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.675 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.675 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.675 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.676 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.676 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.676 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.676 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.676 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.676 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.676 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.676 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.676 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.676 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.676 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.676 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.676 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.677 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.677 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.677 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.677 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2024-05-14T10:00:41.677 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.677 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.677 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.677 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.678 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.679 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.679 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.679 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.679 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.679 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.679 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.679 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.680 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.680 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.680 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.680 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.680 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.680 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.680 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.680 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.680 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.680 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.680 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.681 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.681 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.681 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.681 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.681 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.681 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.681 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.681 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.681 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.681 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.681 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.682 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.682 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.682 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.682 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.682 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.682 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.682 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.682 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.682 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.682 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.683 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.683 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.683 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.683 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.683 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.683 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.683 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.683 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.683 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.683 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.683 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.684 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.684 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.684 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.684 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.684 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.684 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.684 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.684 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.684 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.684 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.684 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.685 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.685 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.685 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.685 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.685 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.685 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.685 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.685 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.685 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.685 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.685 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.686 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.686 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2024-05-14T10:00:41.686 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.686 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.686 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.686 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.686 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.686 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.686 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.687 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.687 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.687 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.687 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.687 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.687 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.687 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.687 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.687 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.687 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.687 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.688 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.688 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.688 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.688 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.688 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.688 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.688 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.689 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.689 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.689 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.689 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.689 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.689 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.689 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.689 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.689 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.690 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.690 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.690 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.690 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.690 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.690 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.690 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.690 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.690 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.690 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.691 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.691 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.691 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.691 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.691 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.691 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.691 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.691 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.691 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.691 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.692 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.692 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.692 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.692 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.692 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.692 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.692 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.692 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.692 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.692 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.692 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.693 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.693 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.693 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.693 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.693 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.693 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.693 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.693 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.693 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.693 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.693 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.693 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.694 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.694 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.694 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2024-05-14T10:00:41.694 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.694 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.694 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.695 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2024-05-14T10:00:41.695 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2024-05-14T10:00:41.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.695 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.695 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-14T10:00:41.695 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.695 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.695 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.695 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.695 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.696 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.696 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.696 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T10:00:41.696 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T10:00:41.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.696 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.696 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2024-05-14T10:00:41.696 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.696 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.696 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.697 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.697 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.697 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.697 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.697 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.697 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.697 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.697 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.698 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.698 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.698 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.698 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.698 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.698 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.698 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.698 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.698 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.698 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.698 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.699 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.699 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.699 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.699 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.699 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.699 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.699 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.700 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.700 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.700 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.700 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.700 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.700 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.700 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.700 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.700 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.700 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.701 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.701 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.701 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.701 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.701 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.701 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.701 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.701 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.701 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.701 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.702 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.702 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.702 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.702 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.702 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.702 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.702 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.702 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.702 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.703 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.703 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.703 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.703 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.703 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.703 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.703 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.703 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.703 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.703 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.703 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.703 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.704 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.704 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.704 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.704 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.704 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.704 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.704 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.704 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.704 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.704 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.705 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.705 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.705 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2024-05-14T10:00:41.705 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.705 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.705 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.705 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2024-05-14T10:00:41.705 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2024-05-14T10:00:41.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.705 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.705 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.706 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.706 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.706 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.706 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.706 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-14T10:00:41.706 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.706 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.706 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.706 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2024-05-14T10:00:41.706 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2024-05-14T10:00:41.706 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.706 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.707 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.707 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.707 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.707 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.707 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.707 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-14T10:00:41.707 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.707 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.707 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.707 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2024-05-14T10:00:41.707 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2024-05-14T10:00:41.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.708 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.708 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.708 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.708 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.708 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.708 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.708 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-14T10:00:41.708 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.708 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.708 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.708 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2024-05-14T10:00:41.709 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2024-05-14T10:00:41.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.709 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.709 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.709 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.709 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.709 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.709 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-14T10:00:41.709 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.709 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.710 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2024-05-14T10:00:41.710 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2024-05-14T10:00:41.710 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.710 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.710 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.710 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.710 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.710 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.710 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.710 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-14T10:00:41.710 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.710 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.711 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.711 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2024-05-14T10:00:41.711 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2024-05-14T10:00:41.711 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.711 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.711 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.711 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.711 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.711 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.711 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.711 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.711 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.712 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.712 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.712 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2024-05-14T10:00:41.712 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2024-05-14T10:00:41.712 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.712 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.712 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.712 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.712 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.712 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.713 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.713 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.713 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.713 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.713 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2024-05-14T10:00:41.713 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2024-05-14T10:00:41.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.713 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.713 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.713 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-14T10:00:41.714 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.714 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.714 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.714 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.714 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.714 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.714 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.714 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-05-14T10:00:41.714 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-05-14T10:00:41.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.715 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.715 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.715 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.715 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.715 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.715 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.715 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.715 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.715 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.715 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.715 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T10:00:41.716 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T10:00:41.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.716 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.716 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.716 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2024-05-14T10:00:41.716 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2024-05-14T10:00:41.716 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.716 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.716 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.716 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.716 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.716 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.717 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2024-05-14T10:00:41.717 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2024-05-14T10:00:41.717 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.717 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.717 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.717 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2024-05-14T10:00:41.717 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.717 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.717 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.718 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.718 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.718 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.718 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.718 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2024-05-14T10:00:41.718 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2024-05-14T10:00:41.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.718 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.718 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.718 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.719 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.719 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.719 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.719 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.719 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.719 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.719 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.719 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2024-05-14T10:00:41.719 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2024-05-14T10:00:41.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.720 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.720 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.720 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.720 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.720 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.720 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.720 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.720 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.720 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.720 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.720 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.721 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.721 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.721 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.721 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.721 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.721 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.721 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.721 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.721 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.721 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.721 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.722 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.722 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.722 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.722 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.722 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.722 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.722 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.722 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.722 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.722 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.723 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.723 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.723 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.723 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.723 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.723 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.723 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.723 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.723 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.723 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.723 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.724 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.724 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.724 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.724 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.724 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.724 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.724 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.724 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.724 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.724 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.724 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.724 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.725 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.725 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.725 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.725 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.725 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.725 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.725 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.725 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.725 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.725 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.726 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.726 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.726 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.726 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.726 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.726 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.726 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2024-05-14T10:00:41.726 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2024-05-14T10:00:41.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.726 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.726 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.727 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.727 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.727 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.727 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.727 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.727 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.727 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.727 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.727 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2024-05-14T10:00:41.727 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2024-05-14T10:00:41.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.727 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.728 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.728 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.728 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.728 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.728 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.728 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.728 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.728 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.728 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.728 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.729 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.729 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.729 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.729 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.729 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.729 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.729 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.729 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.729 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.730 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2024-05-14T10:00:41.730 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2024-05-14T10:00:41.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.730 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.730 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.730 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-14T10:00:41.730 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.730 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.730 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.730 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.730 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.731 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.731 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.731 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.731 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.731 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2024-05-14T10:00:41.731 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.731 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.731 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.731 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.731 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.732 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.732 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.732 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.732 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.732 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.732 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.732 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.732 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.732 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.732 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.732 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.732 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.732 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.733 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.733 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.733 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.733 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.733 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.733 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.733 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.733 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.734 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.734 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.734 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.734 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.734 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.734 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.734 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.734 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.734 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.734 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.734 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.735 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.735 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.735 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.735 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.735 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.735 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.735 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.735 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.735 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.735 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.735 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.735 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.735 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.736 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.736 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.736 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.736 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.736 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.736 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.736 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.736 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.736 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.736 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.736 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.737 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.737 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.737 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.737 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.737 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.737 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.737 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.737 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.737 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.737 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.738 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.738 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.738 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.738 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.738 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.738 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.738 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.738 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-05-14T10:00:41.739 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.739 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.739 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.739 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.739 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.739 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.739 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.739 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.739 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.739 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.739 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.739 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.740 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.740 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.740 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.740 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.740 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.740 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.740 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.740 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.740 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.740 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.740 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.741 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.741 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.741 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.741 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.741 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.741 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.741 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.741 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.741 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.741 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.741 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.742 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.742 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.742 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.742 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.742 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.742 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.742 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.742 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.743 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.743 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.743 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.743 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.743 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.743 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.743 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.743 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.743 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.743 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.743 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.743 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.744 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.744 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.744 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.744 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.744 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.744 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.744 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.744 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.744 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.744 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.744 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.744 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.745 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.745 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.745 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2024-05-14T10:00:41.745 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2024-05-14T10:00:41.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.745 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.745 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2024-05-14T10:00:41.745 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.745 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.746 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.746 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-14T10:00:41.746 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.746 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.746 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.746 INFO:teuthology.orchestra.run.smithi002.stdout: "periodic_async_work": { 2024-05-14T10:00:41.746 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "periodic_async_work", 2024-05-14T10:00:41.746 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.746 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.746 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.746 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.747 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.747 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.747 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.747 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Periodically check for async work", 2024-05-14T10:00:41.747 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.747 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.747 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.747 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2024-05-14T10:00:41.747 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2024-05-14T10:00:41.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.747 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.748 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.748 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.748 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.748 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.748 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.748 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-14T10:00:41.748 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.748 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.748 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.748 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_no_wait": { 2024-05-14T10:00:41.748 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_no_wait", 2024-05-14T10:00:41.749 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.749 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.749 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.749 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.749 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.749 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.749 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.749 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-14T10:00:41.749 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.749 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.749 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.750 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.750 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.750 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.750 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.750 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.750 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.750 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.750 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.750 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.750 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.750 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.751 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.751 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.751 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.751 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.751 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2024-05-14T10:00:41.751 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.751 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.751 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.751 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2024-05-14T10:00:41.751 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2024-05-14T10:00:41.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T10:00:41.752 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.752 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.752 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2024-05-14T10:00:41.752 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.752 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.752 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.752 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.752 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.752 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.752 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.753 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2024-05-14T10:00:41.753 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2024-05-14T10:00:41.753 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.753 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.753 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.753 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.753 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.753 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.753 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.753 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.753 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.753 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.754 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.754 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T10:00:41.754 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T10:00:41.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.754 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.754 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.754 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T10:00:41.754 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.754 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.754 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.755 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.755 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.755 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.755 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.755 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.755 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.755 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.755 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.755 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.755 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.756 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.756 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.756 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.756 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.756 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.756 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.756 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.756 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.756 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.756 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.757 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.757 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.757 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.757 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.757 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.757 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.757 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.757 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.757 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.757 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.758 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.758 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.758 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.758 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.758 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.758 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.758 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.758 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.758 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.759 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.760 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.761 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.762 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2024-05-14T10:00:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2024-05-14T10:00:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.763 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2024-05-14T10:00:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2024-05-14T10:00:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2024-05-14T10:00:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.764 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2024-05-14T10:00:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2024-05-14T10:00:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-14T10:00:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.765 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T10:00:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "gid": 24116, 2024-05-14T10:00:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "y", 2024-05-14T10:00:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_features": 4540701547738038271, 2024-05-14T10:00:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2024-05-14T10:00:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2024-05-14T10:00:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T10:00:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T10:00:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.767 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T10:00:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2024-05-14T10:00:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.768 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.769 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.773 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2024-05-14T10:00:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2024-05-14T10:00:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.774 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2024-05-14T10:00:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2024-05-14T10:00:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2024-05-14T10:00:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2024-05-14T10:00:41.775 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2024-05-14T10:00:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2024-05-14T10:00:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2024-05-14T10:00:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.776 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2024-05-14T10:00:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2024-05-14T10:00:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2024-05-14T10:00:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.777 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2024-05-14T10:00:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2024-05-14T10:00:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2024-05-14T10:00:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.778 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2024-05-14T10:00:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2024-05-14T10:00:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2024-05-14T10:00:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2024-05-14T10:00:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2024-05-14T10:00:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2024-05-14T10:00:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2024-05-14T10:00:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-14T10:00:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.781 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2024-05-14T10:00:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2024-05-14T10:00:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2024-05-14T10:00:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.782 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.783 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2024-05-14T10:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2024-05-14T10:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2024-05-14T10:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.784 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2024-05-14T10:00:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2024-05-14T10:00:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2024-05-14T10:00:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.785 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2024-05-14T10:00:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2024-05-14T10:00:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T10:00:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2024-05-14T10:00:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2024-05-14T10:00:41.786 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T10:00:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-05-14T10:00:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-05-14T10:00:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-14T10:00:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T10:00:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2024-05-14T10:00:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2024-05-14T10:00:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T10:00:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2024-05-14T10:00:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T10:00:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2024-05-14T10:00:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-14T10:00:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2024-05-14T10:00:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2024-05-14T10:00:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.788 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2024-05-14T10:00:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-14T10:00:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-14T10:00:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2024-05-14T10:00:41.789 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2024-05-14T10:00:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2024-05-14T10:00:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2024-05-14T10:00:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2024-05-14T10:00:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2024-05-14T10:00:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-14T10:00:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2024-05-14T10:00:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2024-05-14T10:00:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2024-05-14T10:00:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2024-05-14T10:00:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T10:00:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.791 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2024-05-14T10:00:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2024-05-14T10:00:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T10:00:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-05-14T10:00:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-05-14T10:00:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-14T10:00:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T10:00:41.792 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.793 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.795 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.796 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.797 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2024-05-14T10:00:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2024-05-14T10:00:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-14T10:00:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-05-14T10:00:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-05-14T10:00:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2024-05-14T10:00:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.800 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2024-05-14T10:00:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2024-05-14T10:00:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "read", 2024-05-14T10:00:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap", 2024-05-14T10:00:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap-read" 2024-05-14T10:00:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2024-05-14T10:00:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2024-05-14T10:00:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2024-05-14T10:00:41.801 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-14T10:00:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.802 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T10:00:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T10:00:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T10:00:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-14T10:00:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.803 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.804 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2024-05-14T10:00:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2024-05-14T10:00:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T10:00:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T10:00:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-14T10:00:41.805 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-14T10:00:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2024-05-14T10:00:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2024-05-14T10:00:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T10:00:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.806 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-14T10:00:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2024-05-14T10:00:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.807 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2024-05-14T10:00:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2024-05-14T10:00:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-05-14T10:00:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-14T10:00:41.808 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2024-05-14T10:00:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2024-05-14T10:00:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2024-05-14T10:00:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.809 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-14T10:00:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2024-05-14T10:00:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2024-05-14T10:00:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.810 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2024-05-14T10:00:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-14T10:00:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2024-05-14T10:00:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2024-05-14T10:00:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.811 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-14T10:00:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-14T10:00:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2024-05-14T10:00:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2024-05-14T10:00:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.812 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T10:00:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-14T10:00:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2024-05-14T10:00:41.813 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2024-05-14T10:00:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2024-05-14T10:00:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-14T10:00:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.814 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm_log_destination": { 2024-05-14T10:00:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm_log_destination", 2024-05-14T10:00:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.815 INFO:teuthology.orchestra.run.smithi002.stdout: "file", 2024-05-14T10:00:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: "file,syslog", 2024-05-14T10:00:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: "syslog" 2024-05-14T10:00:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-14T10:00:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2024-05-14T10:00:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2024-05-14T10:00:41.816 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-14T10:00:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.817 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2024-05-14T10:00:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2024-05-14T10:00:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-14T10:00:41.818 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2024-05-14T10:00:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2024-05-14T10:00:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.819 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-14T10:00:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2024-05-14T10:00:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2024-05-14T10:00:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.820 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-14T10:00:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T10:00:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2024-05-14T10:00:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2024-05-14T10:00:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-14T10:00:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2024-05-14T10:00:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.822 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2024-05-14T10:00:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2024-05-14T10:00:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-14T10:00:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2024-05-14T10:00:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.823 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2024-05-14T10:00:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2024-05-14T10:00:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-14T10:00:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T10:00:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.824 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2024-05-14T10:00:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2024-05-14T10:00:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-14T10:00:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2024-05-14T10:00:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.825 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2024-05-14T10:00:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2024-05-14T10:00:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-14T10:00:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.826 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2024-05-14T10:00:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2024-05-14T10:00:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2024-05-14T10:00:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-14T10:00:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.827 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2024-05-14T10:00:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2024-05-14T10:00:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2024-05-14T10:00:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.828 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-14T10:00:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2024-05-14T10:00:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2024-05-14T10:00:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2024-05-14T10:00:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.829 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-14T10:00:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2024-05-14T10:00:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.830 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2024-05-14T10:00:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2024-05-14T10:00:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-14T10:00:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2024-05-14T10:00:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.831 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2024-05-14T10:00:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2024-05-14T10:00:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-14T10:00:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T10:00:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.832 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_nvmeof": { 2024-05-14T10:00:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_nvmeof", 2024-05-14T10:00:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-14T10:00:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Nvme-of container image", 2024-05-14T10:00:41.833 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2024-05-14T10:00:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2024-05-14T10:00:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-14T10:00:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.834 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T10:00:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2024-05-14T10:00:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2024-05-14T10:00:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-14T10:00:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.835 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2024-05-14T10:00:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_samba": { 2024-05-14T10:00:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_samba", 2024-05-14T10:00:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.836 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-14T10:00:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Samba/SMB container image", 2024-05-14T10:00:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2024-05-14T10:00:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2024-05-14T10:00:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.837 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-14T10:00:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2024-05-14T10:00:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2024-05-14T10:00:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2024-05-14T10:00:41.838 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2024-05-14T10:00:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2024-05-14T10:00:41.839 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2024-05-14T10:00:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T10:00:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-14T10:00:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.840 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2024-05-14T10:00:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2024-05-14T10:00:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2024-05-14T10:00:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.841 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-14T10:00:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2024-05-14T10:00:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2024-05-14T10:00:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.842 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2024-05-14T10:00:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-14T10:00:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2024-05-14T10:00:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2024-05-14T10:00:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.843 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2024-05-14T10:00:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2024-05-14T10:00:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.844 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2024-05-14T10:00:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2024-05-14T10:00:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-14T10:00:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2024-05-14T10:00:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2024-05-14T10:00:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T10:00:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.846 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2024-05-14T10:00:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "grafana_dashboards_path": { 2024-05-14T10:00:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "grafana_dashboards_path", 2024-05-14T10:00:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.847 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-14T10:00:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-14T10:00:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2024-05-14T10:00:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2024-05-14T10:00:41.848 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T10:00:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2024-05-14T10:00:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.849 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "hw_monitoring": { 2024-05-14T10:00:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hw_monitoring", 2024-05-14T10:00:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-14T10:00:41.850 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "inventory_list_all": { 2024-05-14T10:00:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "inventory_list_all", 2024-05-14T10:00:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.851 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-14T10:00:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.854 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.854 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.854 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.854 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.854 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.854 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.854 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.854 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.854 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2024-05-14T10:00:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2024-05-14T10:00:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-14T10:00:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-14T10:00:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-14T10:00:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-14T10:00:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-14T10:00:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-14T10:00:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-14T10:00:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2024-05-14T10:00:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-14T10:00:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2024-05-14T10:00:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2024-05-14T10:00:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2024-05-14T10:00:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2024-05-14T10:00:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2024-05-14T10:00:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-14T10:00:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2024-05-14T10:00:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2024-05-14T10:00:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2024-05-14T10:00:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-05-14T10:00:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-05-14T10:00:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2024-05-14T10:00:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2024-05-14T10:00:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2024-05-14T10:00:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2024-05-14T10:00:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "oob_default_addr": { 2024-05-14T10:00:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "oob_default_addr", 2024-05-14T10:00:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "169.254.1.1", 2024-05-14T10:00:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-14T10:00:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2024-05-14T10:00:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2024-05-14T10:00:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-14T10:00:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-14T10:00:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2024-05-14T10:00:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2024-05-14T10:00:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-14T10:00:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2024-05-14T10:00:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2024-05-14T10:00:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-14T10:00:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2024-05-14T10:00:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2024-05-14T10:00:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-14T10:00:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2024-05-14T10:00:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2024-05-14T10:00:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-14T10:00:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2024-05-14T10:00:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2024-05-14T10:00:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-14T10:00:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2024-05-14T10:00:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2024-05-14T10:00:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2024-05-14T10:00:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2024-05-14T10:00:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2024-05-14T10:00:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2024-05-14T10:00:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-14T10:00:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2024-05-14T10:00:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2024-05-14T10:00:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-14T10:00:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2024-05-14T10:00:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2024-05-14T10:00:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-14T10:00:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2024-05-14T10:00:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2024-05-14T10:00:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2024-05-14T10:00:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2024-05-14T10:00:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2024-05-14T10:00:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-14T10:00:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2024-05-14T10:00:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2024-05-14T10:00:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-14T10:00:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2024-05-14T10:00:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2024-05-14T10:00:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2024-05-14T10:00:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2024-05-14T10:00:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-14T10:00:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2024-05-14T10:00:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2024-05-14T10:00:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2024-05-14T10:00:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-14T10:00:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2024-05-14T10:00:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-14T10:00:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-14T10:00:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2024-05-14T10:00:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-14T10:00:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-14T10:00:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-14T10:00:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2024-05-14T10:00:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2024-05-14T10:00:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-14T10:00:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-14T10:00:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2024-05-14T10:00:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-14T10:00:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-14T10:00:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-14T10:00:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-14T10:00:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-14T10:00:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-14T10:00:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-14T10:00:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-14T10:00:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-14T10:00:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-14T10:00:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-14T10:00:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-14T10:00:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-14T10:00:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-14T10:00:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-14T10:00:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-14T10:00:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-14T10:00:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2024-05-14T10:00:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-14T10:00:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T10:00:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-14T10:00:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-14T10:00:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2024-05-14T10:00:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2024-05-14T10:00:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2024-05-14T10:00:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-14T10:00:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T10:00:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-14T10:00:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-14T10:00:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-14T10:00:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-14T10:00:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-14T10:00:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-14T10:00:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-14T10:00:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-14T10:00:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-14T10:00:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-14T10:00:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "[]", 2024-05-14T10:00:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "MULTICLUSTER_CONFIG": { 2024-05-14T10:00:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-14T10:00:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "{}", 2024-05-14T10:00:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2024-05-14T10:00:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-14T10:00:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-14T10:00:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-14T10:00:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-14T10:00:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-14T10:00:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-14T10:00:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-14T10:00:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-14T10:00:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-14T10:00:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-14T10:00:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-14T10:00:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-14T10:00:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-14T10:00:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-14T10:00:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-14T10:00:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-14T10:00:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-14T10:00:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2024-05-14T10:00:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-14T10:00:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-14T10:00:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-14T10:00:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-14T10:00:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-14T10:00:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-14T10:00:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-14T10:00:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-14T10:00:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2024-05-14T10:00:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-14T10:00:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-14T10:00:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2024-05-14T10:00:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2024-05-14T10:00:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-14T10:00:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-14T10:00:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-14T10:00:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T10:00:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2024-05-14T10:00:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-14T10:00:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2024-05-14T10:00:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-14T10:00:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "UNSAFE_TLS_v1_2": { 2024-05-14T10:00:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-14T10:00:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-14T10:00:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-14T10:00:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-14T10:00:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-14T10:00:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-14T10:00:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-14T10:00:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T10:00:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2024-05-14T10:00:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2024-05-14T10:00:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2024-05-14T10:00:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2024-05-14T10:00:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2024-05-14T10:00:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2024-05-14T10:00:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2024-05-14T10:00:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2024-05-14T10:00:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2024-05-14T10:00:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2024-05-14T10:00:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-05-14T10:00:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-05-14T10:00:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.957 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.957 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.957 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.957 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.957 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.957 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.957 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.957 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.957 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.957 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.957 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.958 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.958 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.958 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.958 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.958 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.958 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.958 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.958 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.958 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.958 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.958 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.959 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.959 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.959 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.959 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.959 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.959 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.959 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.959 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.959 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.959 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.960 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.960 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.960 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.960 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.960 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.960 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.960 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.960 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.960 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.960 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2024-05-14T10:00:41.960 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2024-05-14T10:00:41.960 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.960 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.961 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.961 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.961 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.961 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.961 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.961 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2024-05-14T10:00:41.961 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.961 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.961 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.961 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2024-05-14T10:00:41.961 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2024-05-14T10:00:41.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.962 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.962 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.962 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.962 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.962 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.962 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.962 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.962 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.962 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.962 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.962 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T10:00:41.963 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T10:00:41.963 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.963 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.963 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.963 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-05-14T10:00:41.963 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.963 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.963 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.963 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.963 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.963 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.963 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.964 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T10:00:41.964 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T10:00:41.964 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.964 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.964 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.964 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2024-05-14T10:00:41.964 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.964 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.964 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.964 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.964 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.964 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.964 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.965 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.965 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.965 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.965 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.965 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.965 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.965 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.965 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.965 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.965 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.965 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.965 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.966 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.966 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-05-14T10:00:41.966 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-05-14T10:00:41.966 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.966 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.966 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.966 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.966 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.966 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.966 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.966 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.967 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.967 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.967 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.967 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2024-05-14T10:00:41.967 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2024-05-14T10:00:41.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.967 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.967 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.967 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2024-05-14T10:00:41.967 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.967 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.967 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.968 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.968 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.968 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.968 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.968 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-05-14T10:00:41.968 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-05-14T10:00:41.968 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.968 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.968 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.968 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2024-05-14T10:00:41.968 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.968 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.969 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.969 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.969 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2024-05-14T10:00:41.969 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.969 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.969 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.969 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.969 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.969 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-05-14T10:00:41.969 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-05-14T10:00:41.969 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.969 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.970 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.970 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-05-14T10:00:41.970 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-05-14T10:00:41.970 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-05-14T10:00:41.970 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.970 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.970 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.970 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.970 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.970 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2024-05-14T10:00:41.970 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2024-05-14T10:00:41.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.971 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.971 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.971 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.971 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.971 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.971 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.971 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.971 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.971 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.971 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.972 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-05-14T10:00:41.972 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.972 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.972 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.972 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2024-05-14T10:00:41.972 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2024-05-14T10:00:41.972 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.972 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.972 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.973 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.973 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.973 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.973 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.973 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2024-05-14T10:00:41.973 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.973 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.973 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.973 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.973 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.973 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.974 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.974 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.974 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.974 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.974 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.974 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.974 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.974 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.974 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.974 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.974 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.974 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.975 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.975 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.975 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.975 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.975 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.975 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.975 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.975 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.975 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.975 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.976 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.976 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.976 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.976 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.976 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.976 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.976 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.976 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.976 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.976 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.976 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.977 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.977 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.977 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.977 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.977 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.977 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.977 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.977 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.977 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.977 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.977 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.977 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.978 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.978 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.978 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.978 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.978 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.978 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.978 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.978 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.978 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.979 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.979 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.979 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.979 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.979 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.979 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.979 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.979 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.979 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2024-05-14T10:00:41.979 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2024-05-14T10:00:41.979 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.980 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.980 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2024-05-14T10:00:41.980 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.980 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.980 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.980 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-14T10:00:41.980 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.980 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.981 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2024-05-14T10:00:41.981 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2024-05-14T10:00:41.981 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.981 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.981 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.981 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2024-05-14T10:00:41.981 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.981 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.981 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.981 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-14T10:00:41.981 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.982 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.982 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.982 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2024-05-14T10:00:41.982 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2024-05-14T10:00:41.982 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.982 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.982 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.982 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2024-05-14T10:00:41.982 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.982 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.982 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.983 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2024-05-14T10:00:41.983 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.983 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.983 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.983 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2024-05-14T10:00:41.983 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2024-05-14T10:00:41.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.983 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.983 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.983 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-05-14T10:00:41.983 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.984 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.984 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.984 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-14T10:00:41.984 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.984 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.984 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.984 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2024-05-14T10:00:41.984 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2024-05-14T10:00:41.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.984 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.985 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:41.985 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.985 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.985 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.985 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-14T10:00:41.985 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.985 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.985 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.985 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T10:00:41.985 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T10:00:41.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.985 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.986 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.986 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T10:00:41.986 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.986 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.986 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.986 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2024-05-14T10:00:41.986 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.986 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.986 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.986 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.986 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.987 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.987 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.987 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.987 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.987 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.987 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.987 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.987 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.988 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.988 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.988 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.988 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2024-05-14T10:00:41.988 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2024-05-14T10:00:41.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:41.988 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.988 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2024-05-14T10:00:41.988 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.988 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.988 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.989 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-14T10:00:41.989 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.989 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.989 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:41.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:41.989 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2024-05-14T10:00:41.989 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:41.989 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:41.989 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:41.989 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:41.990 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:41.990 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.990 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.990 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.990 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:41.990 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.990 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.990 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.990 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.990 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.990 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.991 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.991 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.991 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.991 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.991 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.991 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.991 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.991 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.991 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:41.991 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:41.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.991 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.992 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.992 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.992 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.992 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.992 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.992 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.992 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.992 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.992 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.992 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:41.994 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:41.994 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:41.994 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.994 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.994 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.994 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.994 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:41.994 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:41.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:41.994 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.994 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.994 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:41.995 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.995 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.995 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.995 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.995 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.995 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.995 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.995 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2024-05-14T10:00:41.995 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2024-05-14T10:00:41.995 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.995 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.995 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.995 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-05-14T10:00:41.996 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.996 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.996 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.996 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.996 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.996 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.996 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.996 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2024-05-14T10:00:41.996 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2024-05-14T10:00:41.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.996 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.997 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2024-05-14T10:00:41.997 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.997 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.997 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.997 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.997 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.997 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.997 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.997 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T10:00:41.997 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T10:00:41.998 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:41.998 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:41.998 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:41.998 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T10:00:41.998 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.998 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.998 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.998 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.998 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:41.998 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:41.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:41.999 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:41.999 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:41.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:41.999 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:41.999 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:41.999 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:41.999 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:41.999 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:41.999 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:41.999 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:41.999 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:41.999 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.000 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.000 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2024-05-14T10:00:42.000 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2024-05-14T10:00:42.000 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2024-05-14T10:00:42.000 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.000 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2024-05-14T10:00:42.000 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2024-05-14T10:00:42.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.001 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.001 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.001 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2024-05-14T10:00:42.001 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.001 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.001 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.001 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-14T10:00:42.001 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.001 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.001 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2024-05-14T10:00:42.001 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2024-05-14T10:00:42.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.002 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.002 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.002 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2024-05-14T10:00:42.002 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.002 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.002 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.002 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-14T10:00:42.002 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.002 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.002 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.003 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2024-05-14T10:00:42.003 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2024-05-14T10:00:42.003 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.003 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.003 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.003 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.003 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.003 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.003 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.003 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2024-05-14T10:00:42.004 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.004 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.004 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.004 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T10:00:42.004 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T10:00:42.004 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.004 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.004 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.004 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2024-05-14T10:00:42.004 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2024-05-14T10:00:42.004 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.005 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.005 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-14T10:00:42.005 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.005 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.005 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.005 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.005 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.005 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.005 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.005 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.005 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.006 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.006 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.006 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.006 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.006 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.006 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.006 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.006 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.006 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.006 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.006 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.007 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.007 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.007 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.007 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.007 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.007 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.007 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.007 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.007 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.007 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.007 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.007 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.007 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.008 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.008 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.008 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.008 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.008 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.008 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.008 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.008 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.008 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.008 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.008 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.008 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.009 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.009 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.009 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.009 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.009 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.009 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.009 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.009 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.009 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.009 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.009 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.009 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.009 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.010 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.010 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.010 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.010 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.010 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.010 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.010 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.010 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.010 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2024-05-14T10:00:42.010 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2024-05-14T10:00:42.010 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.011 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.011 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.011 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.011 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.011 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.011 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.011 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2024-05-14T10:00:42.011 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.011 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.011 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.011 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2024-05-14T10:00:42.011 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2024-05-14T10:00:42.012 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.012 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.012 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.012 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2024-05-14T10:00:42.012 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.012 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.012 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.012 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2024-05-14T10:00:42.012 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.012 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.012 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.012 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.013 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.013 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.013 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.013 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.013 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.013 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.013 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.013 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.013 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.013 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.013 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.013 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-05-14T10:00:42.013 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-05-14T10:00:42.014 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.014 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.014 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.014 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2024-05-14T10:00:42.014 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.014 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.014 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.014 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T10:00:42.014 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.014 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.014 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.014 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2024-05-14T10:00:42.015 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2024-05-14T10:00:42.015 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.015 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.015 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.015 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T10:00:42.015 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T10:00:42.015 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2024-05-14T10:00:42.015 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.015 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-14T10:00:42.015 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.015 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.015 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.016 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2024-05-14T10:00:42.016 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2024-05-14T10:00:42.016 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.016 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.016 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.016 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.016 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.016 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.016 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.016 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2024-05-14T10:00:42.016 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.016 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.016 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.017 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2024-05-14T10:00:42.017 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2024-05-14T10:00:42.017 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.017 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.017 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.017 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2024-05-14T10:00:42.017 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.017 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.017 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.017 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T10:00:42.017 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.017 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.017 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.018 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2024-05-14T10:00:42.018 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.018 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.018 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.018 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.018 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.018 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.018 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.019 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.019 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.019 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.019 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.019 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.019 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.019 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.019 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.019 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.019 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.019 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.019 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.019 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.020 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.020 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.020 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.020 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.020 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.020 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.020 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.020 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.020 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.020 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.020 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.021 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.021 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.021 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.021 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.021 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.021 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.021 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.021 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.021 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.021 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.021 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.021 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.022 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.022 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.022 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.022 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.022 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.022 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.022 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.022 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.022 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.022 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.022 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.023 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.023 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.023 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.023 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.023 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.023 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.023 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.023 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.023 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.023 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2024-05-14T10:00:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.026 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.026 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.026 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.026 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.026 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.026 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.026 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.026 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.026 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.026 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.026 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.026 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.027 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.027 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.027 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.027 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.027 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.027 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.027 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.027 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.027 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.027 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.027 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.027 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.028 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.028 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.028 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.028 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.028 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.028 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.028 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.028 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.028 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.028 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.029 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.029 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.029 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.029 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.029 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.029 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.029 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.029 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.029 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.029 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.029 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.029 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.030 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.030 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.030 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.030 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.030 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.030 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.030 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.030 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.030 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.030 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.030 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.030 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.031 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.031 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.031 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.031 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.031 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.031 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.031 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.031 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.031 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.031 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.031 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.031 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.032 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.032 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.032 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.032 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.032 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.032 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.032 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.032 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.032 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.032 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.032 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.032 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2024-05-14T10:00:42.033 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.033 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.033 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.033 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2024-05-14T10:00:42.033 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2024-05-14T10:00:42.033 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.033 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.033 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.033 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2024-05-14T10:00:42.033 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.033 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.033 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.034 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-14T10:00:42.034 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.034 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.034 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.034 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2024-05-14T10:00:42.034 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2024-05-14T10:00:42.034 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.034 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.034 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.034 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:42.034 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2024-05-14T10:00:42.034 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.035 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.035 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-14T10:00:42.035 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.035 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.035 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.035 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.035 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.035 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.035 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.035 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.035 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.035 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.036 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.036 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.036 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.036 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.036 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.036 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.036 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.036 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.036 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.036 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.036 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.036 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.036 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.037 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.037 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.037 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.037 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.037 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.037 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.037 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.037 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.037 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.037 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.037 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.037 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.038 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.038 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.038 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.038 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.038 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.038 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.038 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.038 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.038 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.038 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.038 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.039 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.039 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.039 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.039 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.039 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.039 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.039 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.039 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.039 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.039 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.039 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.040 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.040 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.040 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.040 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.040 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.040 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.040 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.040 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.040 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.040 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.040 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.040 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.041 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.041 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.041 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.041 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.041 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.041 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.041 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.041 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.041 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.041 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.041 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.041 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.042 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2024-05-14T10:00:42.042 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.042 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.042 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.042 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2024-05-14T10:00:42.042 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2024-05-14T10:00:42.042 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.042 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.043 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.043 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2024-05-14T10:00:42.043 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.043 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.043 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.043 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2024-05-14T10:00:42.043 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-14T10:00:42.043 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.043 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.043 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.043 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.044 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.044 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.044 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.044 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.044 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.044 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.044 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.044 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.044 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.044 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.044 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.044 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.044 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.045 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.045 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.045 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.045 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.045 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.045 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.045 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.045 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.045 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.045 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.045 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.045 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.046 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.046 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.046 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.046 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.046 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.046 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.046 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.046 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.046 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.046 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.046 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.047 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.047 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.047 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.047 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.047 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.047 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.047 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.047 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.047 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.047 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.047 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.047 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.048 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.048 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.048 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.048 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.048 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.048 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.048 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.048 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.048 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.048 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.048 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.049 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.049 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.049 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.049 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.049 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.049 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2024-05-14T10:00:42.049 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2024-05-14T10:00:42.049 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.049 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.049 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.049 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.050 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.050 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.050 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.050 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2024-05-14T10:00:42.050 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-14T10:00:42.050 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.050 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.050 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2024-05-14T10:00:42.050 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2024-05-14T10:00:42.050 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.051 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.051 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.051 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2024-05-14T10:00:42.051 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.051 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.051 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.051 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2024-05-14T10:00:42.051 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.051 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.051 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.052 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2024-05-14T10:00:42.052 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2024-05-14T10:00:42.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.052 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.052 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2024-05-14T10:00:42.052 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.052 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.052 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.052 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2024-05-14T10:00:42.052 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.052 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.053 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.053 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2024-05-14T10:00:42.053 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2024-05-14T10:00:42.053 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.053 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.053 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.053 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.053 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.053 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.053 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.053 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2024-05-14T10:00:42.054 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.054 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.054 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.054 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.054 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.054 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.054 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.054 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.054 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.054 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.054 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.054 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.055 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.055 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.055 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.055 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.058 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2024-05-14T10:00:42.058 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2024-05-14T10:00:42.058 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.058 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.058 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.058 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2024-05-14T10:00:42.059 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.059 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.059 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.059 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-14T10:00:42.059 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-14T10:00:42.059 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.059 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.060 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2024-05-14T10:00:42.060 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.060 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.060 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.060 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.060 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.060 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.060 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.060 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.060 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.060 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.060 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.060 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.061 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.061 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.061 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.061 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.061 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.061 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.061 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.061 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.061 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.061 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.061 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.061 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.062 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.062 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.062 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.062 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.062 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.062 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.062 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.062 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.062 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.062 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.062 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.062 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.063 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.063 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.063 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.063 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.063 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.063 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.063 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.063 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.063 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.063 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.063 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.064 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.064 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.064 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.064 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.064 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.064 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.064 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.064 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.064 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.064 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.064 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.065 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.065 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.065 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.065 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.065 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.065 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.065 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.065 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.065 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.065 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.065 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.065 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.066 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.066 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.066 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.066 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.066 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.066 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.066 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.066 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.066 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.066 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.066 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.067 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.067 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.067 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2024-05-14T10:00:42.067 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.067 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.067 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.067 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.067 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.067 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.067 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.068 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.068 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.068 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.068 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.068 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.068 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.068 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.068 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.068 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.068 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.068 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.068 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.069 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.069 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.069 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.069 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.069 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.069 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.069 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.069 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.069 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.069 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.070 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.070 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.070 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.070 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.070 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.070 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.070 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.070 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.070 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.070 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.070 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.070 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.071 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.071 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.071 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.071 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.071 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.071 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.071 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.071 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.071 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.071 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.071 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.071 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.071 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.072 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.072 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.072 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.072 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.072 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.072 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.072 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.072 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.072 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.072 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.072 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.072 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.072 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.073 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.073 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.073 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.073 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.073 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.073 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.073 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.073 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.073 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.073 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.073 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.073 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.073 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.074 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.074 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.074 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.074 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2024-05-14T10:00:42.074 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.074 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.074 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.074 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.074 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.074 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.075 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.075 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.075 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.075 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.075 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.075 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.075 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.075 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.075 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.075 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.075 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.075 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.075 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.076 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.076 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.076 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.076 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.076 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.076 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.076 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.076 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.076 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.076 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.076 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.076 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.076 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.077 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.077 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.077 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.077 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.077 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.077 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.077 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.077 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.077 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.077 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.077 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.077 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.078 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.078 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.078 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.078 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.078 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.078 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.078 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.078 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.078 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.078 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.078 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.078 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.079 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.079 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.079 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.079 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.079 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.079 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.079 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.079 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.079 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.079 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.079 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.079 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.080 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.080 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.080 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.080 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.080 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.080 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.080 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.080 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.080 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.080 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.080 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.080 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.080 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.081 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.081 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.081 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.081 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-05-14T10:00:42.081 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.081 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.081 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.081 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2024-05-14T10:00:42.081 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2024-05-14T10:00:42.082 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.082 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.082 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.082 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.082 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.082 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.082 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.082 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-14T10:00:42.082 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.082 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.082 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.083 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.083 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.083 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.083 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.083 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.083 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.083 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.083 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.083 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.083 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.083 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.083 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.083 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.084 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.084 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.084 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.084 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.084 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.084 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.084 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.084 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.084 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.084 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.084 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.085 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.085 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.085 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.085 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.085 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.085 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.085 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.085 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.085 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.085 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.085 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.086 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.086 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.086 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.086 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.086 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.086 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.086 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.086 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.086 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.086 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.086 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.087 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.087 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.087 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.087 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.087 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.087 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.087 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.087 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.087 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.087 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.087 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.088 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.088 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.088 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.088 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.088 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.088 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.088 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.088 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.088 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2024-05-14T10:00:42.088 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-05-14T10:00:42.089 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.089 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.089 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.089 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.089 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.089 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.089 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.089 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-05-14T10:00:42.089 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2024-05-14T10:00:42.089 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2024-05-14T10:00:42.089 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.089 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2024-05-14T10:00:42.090 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.090 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.090 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.090 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.090 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.090 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.090 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.090 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.090 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.090 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.090 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.091 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.091 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.091 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.091 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.091 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.091 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.091 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.091 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2024-05-14T10:00:42.091 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.091 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.092 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.092 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.092 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.092 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.092 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.092 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.092 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.092 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.092 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.092 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.092 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.092 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.092 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.093 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.093 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.093 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.093 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.093 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.093 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.093 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.093 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.093 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.093 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.093 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.094 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.094 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.094 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.094 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.094 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.094 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.094 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.094 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.094 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.094 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.095 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.095 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.095 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.095 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.095 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.095 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.095 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.095 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.095 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.095 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.095 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.096 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.096 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.096 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.096 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.096 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.096 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.096 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.096 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.096 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.096 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.096 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.097 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.097 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.097 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.097 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.097 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.097 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.097 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.097 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.097 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.097 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.097 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.098 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.098 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.098 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.098 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.098 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.098 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.098 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.098 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.098 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.099 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.099 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.099 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.099 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.099 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.099 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.099 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2024-05-14T10:00:42.099 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.099 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.099 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.100 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.100 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.100 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.100 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.100 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.100 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.100 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.100 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.100 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.100 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.100 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.100 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.101 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.101 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.101 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.101 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.101 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.101 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.101 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.101 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.101 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.101 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.101 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.101 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.101 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.102 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.102 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.102 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.102 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.102 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.102 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.102 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.102 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.102 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.102 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.102 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.103 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.103 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.103 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.103 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.103 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.103 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.103 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.103 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.103 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.103 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.103 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.103 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.103 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.104 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.104 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.104 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.104 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.104 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.104 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.104 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.104 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.104 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.104 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.105 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.105 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.105 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.105 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.105 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.105 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.105 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.105 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.105 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.105 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.105 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.105 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.106 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.106 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.106 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.106 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.106 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.106 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.106 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.106 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.106 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2024-05-14T10:00:42.107 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.107 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.107 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.107 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.107 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.107 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.107 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.107 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.107 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.107 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.107 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.107 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.107 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.108 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.108 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.108 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.108 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.108 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.108 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.108 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.108 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.108 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.108 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.108 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.108 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.108 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.109 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.109 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.109 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.109 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.109 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.109 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.109 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.109 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.109 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.109 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.109 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.110 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.110 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.110 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.110 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.110 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.110 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.110 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.110 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.110 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.110 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.110 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.110 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.110 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.111 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.111 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.111 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.111 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.111 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.111 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.111 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.111 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.111 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.111 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.111 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.111 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.112 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.112 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.112 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.112 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.112 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.112 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.112 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.112 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T10:00:42.112 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T10:00:42.112 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.112 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.112 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.112 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T10:00:42.113 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.113 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.113 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.113 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.113 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.113 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.113 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.113 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.113 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.113 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.113 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.113 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.114 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.114 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.114 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.114 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.114 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.114 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.114 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.114 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.114 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2024-05-14T10:00:42.114 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2024-05-14T10:00:42.114 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:42.115 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.115 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-05-14T10:00:42.115 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2024-05-14T10:00:42.115 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.115 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.115 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2024-05-14T10:00:42.115 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-14T10:00:42.115 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.115 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.115 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.115 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.116 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2024-05-14T10:00:42.116 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.116 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.116 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.116 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2024-05-14T10:00:42.116 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2024-05-14T10:00:42.116 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.116 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.116 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.116 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.116 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.117 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.117 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.117 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-14T10:00:42.117 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.117 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.117 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.117 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-05-14T10:00:42.117 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-05-14T10:00:42.117 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.117 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.117 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.117 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.117 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.117 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.118 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.118 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.118 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.118 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.118 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.118 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.118 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.118 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.118 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.118 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.118 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.119 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.119 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.119 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.119 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.119 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.119 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.119 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.119 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.119 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.119 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.119 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.119 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.119 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.120 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.120 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.120 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.120 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.120 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.120 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.120 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.120 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.120 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.120 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.121 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.121 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.121 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.121 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.121 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.121 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.121 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.121 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.121 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.121 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.121 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.121 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.121 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.121 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.122 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.122 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.122 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.122 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.122 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.122 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.122 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.122 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.122 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.122 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.122 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.123 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.123 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.123 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.123 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.123 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.123 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.123 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.123 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.123 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.123 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.123 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2024-05-14T10:00:42.124 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2024-05-14T10:00:42.124 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.124 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.124 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.124 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2024-05-14T10:00:42.124 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.124 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.124 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.124 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2024-05-14T10:00:42.124 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.124 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.124 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.125 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T10:00:42.125 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T10:00:42.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.125 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.125 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.125 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T10:00:42.125 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.125 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.125 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.125 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2024-05-14T10:00:42.125 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.125 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.125 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.126 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.126 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.126 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.126 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.126 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.126 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.126 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.126 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.126 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.126 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.126 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.127 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.127 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2024-05-14T10:00:42.127 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.127 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.127 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.127 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2024-05-14T10:00:42.127 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2024-05-14T10:00:42.127 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.127 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.127 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.127 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.128 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.128 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.128 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.128 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.128 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.128 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.128 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.128 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2024-05-14T10:00:42.128 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2024-05-14T10:00:42.128 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.128 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.128 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.128 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.129 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.129 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.129 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.129 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-14T10:00:42.129 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-14T10:00:42.129 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.129 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.129 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.129 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.129 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.129 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.130 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.130 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.130 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.130 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.130 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.130 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.130 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.130 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.130 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.130 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.130 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.130 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.131 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.131 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.131 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.131 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.131 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.131 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.131 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.131 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.131 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.131 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.131 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.132 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.132 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.132 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.132 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.132 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.132 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.132 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.132 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.132 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.132 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.133 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.133 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.133 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.133 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.133 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.133 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.133 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.133 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.133 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.133 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.133 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.133 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.134 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.134 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.134 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.134 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.134 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.134 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.134 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.134 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.134 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.134 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.135 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.135 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.135 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.135 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.135 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2024-05-14T10:00:42.135 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2024-05-14T10:00:42.135 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.135 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.135 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.135 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.135 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.135 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.136 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.136 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.136 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.136 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.136 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.136 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-14T10:00:42.136 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-14T10:00:42.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.136 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.136 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.136 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2024-05-14T10:00:42.136 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.137 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.137 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.137 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.137 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.137 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.137 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.137 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2024-05-14T10:00:42.137 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2024-05-14T10:00:42.137 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:42.137 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.137 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.137 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2024-05-14T10:00:42.137 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.138 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.138 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.138 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.138 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.138 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.138 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.138 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T10:00:42.138 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T10:00:42.138 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.138 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.138 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.139 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-05-14T10:00:42.139 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.139 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.139 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.139 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-14T10:00:42.139 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.139 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.139 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.139 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T10:00:42.139 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T10:00:42.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.140 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.140 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.140 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2024-05-14T10:00:42.140 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.140 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.140 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.140 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-14T10:00:42.140 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.140 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.140 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.141 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.141 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.141 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.141 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.141 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.141 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.141 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.141 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.141 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.141 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.141 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.141 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.142 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2024-05-14T10:00:42.142 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2024-05-14T10:00:42.142 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.142 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.142 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.142 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2024-05-14T10:00:42.142 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.142 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.142 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.142 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.143 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.143 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.143 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.143 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-05-14T10:00:42.143 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-05-14T10:00:42.143 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.143 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.143 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.143 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2024-05-14T10:00:42.143 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.143 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.144 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.144 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2024-05-14T10:00:42.144 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2024-05-14T10:00:42.144 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.144 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.144 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.144 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.144 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.144 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-05-14T10:00:42.144 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-05-14T10:00:42.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.145 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.145 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.145 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-05-14T10:00:42.145 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-05-14T10:00:42.145 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-05-14T10:00:42.145 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.145 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.145 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.145 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.145 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.146 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.146 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-05-14T10:00:42.146 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.146 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.146 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.146 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.146 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.146 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.146 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.146 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.147 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.147 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.147 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.147 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.147 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.147 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.147 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.147 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.147 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.147 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.147 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.147 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.148 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.148 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.148 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.148 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.148 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.148 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.148 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.148 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.148 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.148 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.148 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.149 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.149 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.149 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.149 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.149 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.149 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.149 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.149 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.149 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.149 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.150 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.150 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.150 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.150 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.150 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.150 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.150 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.150 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.150 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.150 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.150 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.150 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.151 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.151 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.151 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.151 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.151 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.151 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.151 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.151 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.151 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.151 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.151 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.152 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.152 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.152 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.152 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2024-05-14T10:00:42.152 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2024-05-14T10:00:42.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.152 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.152 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.152 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:42.152 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.152 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.152 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2024-05-14T10:00:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2024-05-14T10:00:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.153 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.155 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.155 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.155 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.155 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.155 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.155 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.155 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.155 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2024-05-14T10:00:42.155 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2024-05-14T10:00:42.155 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.155 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.156 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.156 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.156 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.156 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.156 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.156 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.156 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.156 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.157 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2024-05-14T10:00:42.157 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.157 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.157 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.157 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2024-05-14T10:00:42.157 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2024-05-14T10:00:42.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.157 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.157 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.157 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.157 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.157 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.158 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.158 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.158 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.158 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.158 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.158 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-05-14T10:00:42.158 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-05-14T10:00:42.158 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.158 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.158 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.159 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.159 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.159 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.159 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.159 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.159 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.159 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.159 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.159 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.159 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.159 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.159 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.160 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.160 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.160 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.160 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.160 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.160 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.160 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.160 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.160 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.160 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.160 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.160 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.161 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.161 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.161 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.161 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.161 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.161 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.161 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.161 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.161 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.161 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.162 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.162 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.162 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.162 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.162 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.162 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.162 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.162 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.162 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.163 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.163 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.163 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.163 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.163 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.163 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.163 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.163 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.163 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.163 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.163 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.163 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.163 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.164 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.164 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.164 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.164 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.164 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.164 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.164 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.164 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.164 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.164 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.165 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.165 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.165 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.165 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.165 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.165 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.165 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T10:00:42.165 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T10:00:42.165 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.165 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.165 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.166 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.166 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.166 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.166 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.166 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.166 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.166 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.166 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.166 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T10:00:42.166 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T10:00:42.166 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.167 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.167 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.167 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.167 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.167 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.167 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.167 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.167 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.167 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.167 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.167 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.168 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.168 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.168 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.168 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.168 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.168 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.168 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.168 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.168 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.168 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.168 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.168 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.169 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.169 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.169 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2024-05-14T10:00:42.169 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.169 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.169 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.169 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.169 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.169 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.169 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.170 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.170 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.170 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.170 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.170 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.170 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.170 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.170 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.170 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.170 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.170 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.171 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.171 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.171 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.171 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.171 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.171 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.171 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.171 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.171 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.171 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.171 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.171 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.172 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.172 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.172 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.172 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.172 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.172 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.172 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.172 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.172 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.172 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.172 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.173 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.173 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.173 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.173 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.173 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.173 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.173 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.173 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.173 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.173 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.173 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.173 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.173 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.174 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.174 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.174 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.174 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.174 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.174 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.174 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.174 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.174 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.174 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.174 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.175 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.175 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.175 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.175 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.175 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.175 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.175 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.175 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.175 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.175 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.175 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.175 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.176 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.176 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.176 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.176 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.176 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.176 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2024-05-14T10:00:42.176 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.176 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.177 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.177 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.177 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.177 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.177 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.177 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.177 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.177 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.177 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.177 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.177 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.177 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.178 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.178 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.178 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.178 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.178 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.178 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.178 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.178 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.178 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.178 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.178 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2024-05-14T10:00:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2024-05-14T10:00:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2024-05-14T10:00:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-14T10:00:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2024-05-14T10:00:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2024-05-14T10:00:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2024-05-14T10:00:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2024-05-14T10:00:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2024-05-14T10:00:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2024-05-14T10:00:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2024-05-14T10:00:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2024-05-14T10:00:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2024-05-14T10:00:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2024-05-14T10:00:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2024-05-14T10:00:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2024-05-14T10:00:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2024-05-14T10:00:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2024-05-14T10:00:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2024-05-14T10:00:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2024-05-14T10:00:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2024-05-14T10:00:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2024-05-14T10:00:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2024-05-14T10:00:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2024-05-14T10:00:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T10:00:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2024-05-14T10:00:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2024-05-14T10:00:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2024-05-14T10:00:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2024-05-14T10:00:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2024-05-14T10:00:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2024-05-14T10:00:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2024-05-14T10:00:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smb", 2024-05-14T10:00:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "update_orchestration": { 2024-05-14T10:00:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "update_orchestration", 2024-05-14T10:00:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-14T10:00:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-05-14T10:00:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2024-05-14T10:00:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2024-05-14T10:00:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2024-05-14T10:00:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2024-05-14T10:00:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2024-05-14T10:00:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2024-05-14T10:00:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2024-05-14T10:00:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2024-05-14T10:00:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2024-05-14T10:00:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2024-05-14T10:00:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2024-05-14T10:00:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-14T10:00:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T10:00:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T10:00:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2024-05-14T10:00:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2024-05-14T10:00:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2024-05-14T10:00:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2024-05-14T10:00:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-14T10:00:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2024-05-14T10:00:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2024-05-14T10:00:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-14T10:00:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2024-05-14T10:00:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2024-05-14T10:00:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-14T10:00:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2024-05-14T10:00:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2024-05-14T10:00:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-14T10:00:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2024-05-14T10:00:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2024-05-14T10:00:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-14T10:00:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2024-05-14T10:00:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2024-05-14T10:00:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2024-05-14T10:00:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2024-05-14T10:00:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2024-05-14T10:00:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2024-05-14T10:00:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-14T10:00:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-05-14T10:00:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-05-14T10:00:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T10:00:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T10:00:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2024-05-14T10:00:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2024-05-14T10:00:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2024-05-14T10:00:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2024-05-14T10:00:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2024-05-14T10:00:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2024-05-14T10:00:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2024-05-14T10:00:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2024-05-14T10:00:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2024-05-14T10:00:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2024-05-14T10:00:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2024-05-14T10:00:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2024-05-14T10:00:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2024-05-14T10:00:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2024-05-14T10:00:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2024-05-14T10:00:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-14T10:00:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2024-05-14T10:00:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-05-14T10:00:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2024-05-14T10:00:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2024-05-14T10:00:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2024-05-14T10:00:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-14T10:00:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "periodic_async_work": { 2024-05-14T10:00:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "periodic_async_work", 2024-05-14T10:00:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Periodically check for async work", 2024-05-14T10:00:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2024-05-14T10:00:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2024-05-14T10:00:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-14T10:00:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_no_wait": { 2024-05-14T10:00:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_no_wait", 2024-05-14T10:00:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-14T10:00:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2024-05-14T10:00:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2024-05-14T10:00:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2024-05-14T10:00:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T10:00:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2024-05-14T10:00:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2024-05-14T10:00:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2024-05-14T10:00:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T10:00:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T10:00:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T10:00:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2024-05-14T10:00:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2024-05-14T10:00:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2024-05-14T10:00:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2024-05-14T10:00:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2024-05-14T10:00:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.311 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.311 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.311 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2024-05-14T10:00:42.311 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2024-05-14T10:00:42.311 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.311 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.311 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.311 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-14T10:00:42.311 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.311 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.311 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.312 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.312 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.312 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.312 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.312 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T10:00:42.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.312 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.312 INFO:teuthology.orchestra.run.smithi002.stdout: "modules": [ 2024-05-14T10:00:42.313 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-05-14T10:00:42.313 INFO:teuthology.orchestra.run.smithi002.stdout: "dashboard", 2024-05-14T10:00:42.313 INFO:teuthology.orchestra.run.smithi002.stdout: "iostat", 2024-05-14T10:00:42.313 INFO:teuthology.orchestra.run.smithi002.stdout: "nfs", 2024-05-14T10:00:42.313 INFO:teuthology.orchestra.run.smithi002.stdout: "restful" 2024-05-14T10:00:42.313 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.313 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2024-05-14T10:00:42.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.314 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2024-05-14T10:00:42.314 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.314 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.314 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.314 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T10:00:42.314 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T10:00:42.314 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.314 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.314 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.314 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T10:00:42.314 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.314 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.315 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.315 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2024-05-14T10:00:42.315 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.315 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.315 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.315 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.315 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.315 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.315 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.315 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.316 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.316 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.316 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.316 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.316 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.316 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.316 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.316 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.316 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.316 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.316 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.316 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.317 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.317 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.317 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.317 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.317 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.317 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.317 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.317 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.317 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.317 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.317 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.318 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.318 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.318 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.318 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.318 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.318 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.318 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.318 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.318 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.318 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.319 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.319 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.319 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.319 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.319 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.319 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.319 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.319 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.319 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.319 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.319 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.319 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.319 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.320 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.320 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.320 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.320 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.320 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.320 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.320 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.320 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.320 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.320 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.320 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.321 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.321 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2024-05-14T10:00:42.321 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2024-05-14T10:00:42.321 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.321 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.321 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.321 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.321 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.321 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.321 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2024-05-14T10:00:42.322 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.322 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.322 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.322 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2024-05-14T10:00:42.322 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2024-05-14T10:00:42.322 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.322 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.322 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.322 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2024-05-14T10:00:42.322 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.322 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.323 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.323 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2024-05-14T10:00:42.323 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.323 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.323 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.323 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2024-05-14T10:00:42.323 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2024-05-14T10:00:42.323 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.323 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.323 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.323 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.323 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.324 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.324 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.324 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2024-05-14T10:00:42.324 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.324 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.324 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.324 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2024-05-14T10:00:42.324 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2024-05-14T10:00:42.324 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.324 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.324 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.324 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.325 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.325 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.325 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.325 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2024-05-14T10:00:42.325 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.325 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.325 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.325 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2024-05-14T10:00:42.325 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2024-05-14T10:00:42.325 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.325 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.326 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.326 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2024-05-14T10:00:42.326 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.326 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.326 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.326 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2024-05-14T10:00:42.326 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.326 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.326 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.326 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2024-05-14T10:00:42.327 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2024-05-14T10:00:42.327 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.327 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.327 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.327 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.327 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.327 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.327 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.327 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2024-05-14T10:00:42.327 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.327 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.327 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.328 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2024-05-14T10:00:42.328 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2024-05-14T10:00:42.328 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.328 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.328 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.328 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.328 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.328 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.328 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.328 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-14T10:00:42.328 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.328 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.328 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.329 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2024-05-14T10:00:42.329 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2024-05-14T10:00:42.329 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.329 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.329 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.329 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.329 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.329 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.329 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.329 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2024-05-14T10:00:42.329 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.329 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.330 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.330 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.330 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.330 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.330 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.330 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.330 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.330 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.330 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.330 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.330 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.331 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.331 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.331 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.331 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2024-05-14T10:00:42.331 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.331 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.331 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.331 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2024-05-14T10:00:42.331 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2024-05-14T10:00:42.332 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.332 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.332 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.332 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.332 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.332 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.332 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.332 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2024-05-14T10:00:42.332 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.332 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.332 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.333 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2024-05-14T10:00:42.333 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2024-05-14T10:00:42.333 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.333 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.333 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.333 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2024-05-14T10:00:42.333 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.333 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.333 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.333 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2024-05-14T10:00:42.333 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T10:00:42.333 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.333 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.334 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2024-05-14T10:00:42.334 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2024-05-14T10:00:42.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T10:00:42.334 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.334 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.334 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-05-14T10:00:42.334 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-05-14T10:00:42.334 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.334 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-14T10:00:42.334 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T10:00:42.334 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2024-05-14T10:00:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2024-05-14T10:00:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T10:00:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2024-05-14T10:00:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T10:00:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2024-05-14T10:00:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-14T10:00:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2024-05-14T10:00:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2024-05-14T10:00:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2024-05-14T10:00:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.336 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-14T10:00:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-14T10:00:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2024-05-14T10:00:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2024-05-14T10:00:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2024-05-14T10:00:42.337 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2024-05-14T10:00:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2024-05-14T10:00:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2024-05-14T10:00:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-14T10:00:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2024-05-14T10:00:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2024-05-14T10:00:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.338 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2024-05-14T10:00:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2024-05-14T10:00:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T10:00:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2024-05-14T10:00:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2024-05-14T10:00:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T10:00:42.339 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-05-14T10:00:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-05-14T10:00:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-14T10:00:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T10:00:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.340 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.341 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.342 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.343 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.344 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.345 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2024-05-14T10:00:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2024-05-14T10:00:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.346 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-14T10:00:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-05-14T10:00:42.347 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-05-14T10:00:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2024-05-14T10:00:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2024-05-14T10:00:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2024-05-14T10:00:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "read", 2024-05-14T10:00:42.348 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap", 2024-05-14T10:00:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap-read" 2024-05-14T10:00:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2024-05-14T10:00:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2024-05-14T10:00:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2024-05-14T10:00:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.349 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-14T10:00:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T10:00:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T10:00:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T10:00:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-14T10:00:42.351 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.352 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2024-05-14T10:00:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2024-05-14T10:00:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T10:00:42.353 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T10:00:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-14T10:00:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-14T10:00:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2024-05-14T10:00:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2024-05-14T10:00:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T10:00:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-14T10:00:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2024-05-14T10:00:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2024-05-14T10:00:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2024-05-14T10:00:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-05-14T10:00:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.356 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-14T10:00:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2024-05-14T10:00:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2024-05-14T10:00:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2024-05-14T10:00:42.357 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-14T10:00:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2024-05-14T10:00:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2024-05-14T10:00:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.358 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2024-05-14T10:00:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-14T10:00:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2024-05-14T10:00:42.359 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2024-05-14T10:00:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-14T10:00:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-14T10:00:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.360 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2024-05-14T10:00:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2024-05-14T10:00:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T10:00:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-14T10:00:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.361 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2024-05-14T10:00:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2024-05-14T10:00:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2024-05-14T10:00:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-14T10:00:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm_log_destination": { 2024-05-14T10:00:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm_log_destination", 2024-05-14T10:00:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.363 INFO:teuthology.orchestra.run.smithi002.stdout: "file", 2024-05-14T10:00:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "file,syslog", 2024-05-14T10:00:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "syslog" 2024-05-14T10:00:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-14T10:00:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2024-05-14T10:00:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2024-05-14T10:00:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.364 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-14T10:00:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2024-05-14T10:00:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2024-05-14T10:00:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.365 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-14T10:00:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2024-05-14T10:00:42.366 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2024-05-14T10:00:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-14T10:00:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2024-05-14T10:00:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2024-05-14T10:00:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-14T10:00:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T10:00:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.368 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2024-05-14T10:00:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2024-05-14T10:00:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-14T10:00:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.369 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2024-05-14T10:00:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2024-05-14T10:00:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2024-05-14T10:00:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-14T10:00:42.370 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2024-05-14T10:00:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2024-05-14T10:00:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2024-05-14T10:00:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.371 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-14T10:00:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T10:00:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2024-05-14T10:00:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2024-05-14T10:00:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.372 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-14T10:00:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2024-05-14T10:00:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2024-05-14T10:00:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2024-05-14T10:00:42.373 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-14T10:00:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2024-05-14T10:00:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2024-05-14T10:00:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2024-05-14T10:00:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-14T10:00:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2024-05-14T10:00:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.375 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2024-05-14T10:00:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2024-05-14T10:00:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-14T10:00:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2024-05-14T10:00:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.376 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2024-05-14T10:00:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2024-05-14T10:00:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-14T10:00:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.377 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2024-05-14T10:00:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2024-05-14T10:00:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2024-05-14T10:00:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-14T10:00:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.378 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2024-05-14T10:00:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2024-05-14T10:00:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2024-05-14T10:00:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-14T10:00:42.379 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T10:00:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_nvmeof": { 2024-05-14T10:00:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_nvmeof", 2024-05-14T10:00:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.380 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-14T10:00:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Nvme-of container image", 2024-05-14T10:00:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2024-05-14T10:00:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2024-05-14T10:00:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.381 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-14T10:00:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T10:00:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.382 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2024-05-14T10:00:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2024-05-14T10:00:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-14T10:00:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2024-05-14T10:00:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.383 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_samba": { 2024-05-14T10:00:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_samba", 2024-05-14T10:00:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-14T10:00:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Samba/SMB container image", 2024-05-14T10:00:42.384 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2024-05-14T10:00:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2024-05-14T10:00:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-14T10:00:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.385 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2024-05-14T10:00:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2024-05-14T10:00:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2024-05-14T10:00:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.386 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2024-05-14T10:00:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2024-05-14T10:00:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2024-05-14T10:00:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T10:00:42.387 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-14T10:00:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2024-05-14T10:00:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2024-05-14T10:00:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.388 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2024-05-14T10:00:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-14T10:00:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2024-05-14T10:00:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2024-05-14T10:00:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2024-05-14T10:00:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-14T10:00:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2024-05-14T10:00:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2024-05-14T10:00:42.390 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2024-05-14T10:00:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2024-05-14T10:00:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2024-05-14T10:00:42.391 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2024-05-14T10:00:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-14T10:00:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2024-05-14T10:00:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2024-05-14T10:00:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T10:00:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2024-05-14T10:00:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.393 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "grafana_dashboards_path": { 2024-05-14T10:00:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "grafana_dashboards_path", 2024-05-14T10:00:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-14T10:00:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-14T10:00:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.394 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2024-05-14T10:00:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2024-05-14T10:00:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T10:00:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2024-05-14T10:00:42.395 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "hw_monitoring": { 2024-05-14T10:00:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hw_monitoring", 2024-05-14T10:00:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-14T10:00:42.396 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "inventory_list_all": { 2024-05-14T10:00:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "inventory_list_all", 2024-05-14T10:00:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-14T10:00:42.397 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.398 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2024-05-14T10:00:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2024-05-14T10:00:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-14T10:00:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.401 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-14T10:00:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.402 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-14T10:00:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-14T10:00:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-14T10:00:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.405 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-14T10:00:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-14T10:00:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2024-05-14T10:00:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-14T10:00:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.406 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2024-05-14T10:00:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2024-05-14T10:00:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2024-05-14T10:00:42.407 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2024-05-14T10:00:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2024-05-14T10:00:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.408 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-14T10:00:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2024-05-14T10:00:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2024-05-14T10:00:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.409 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2024-05-14T10:00:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-05-14T10:00:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-05-14T10:00:42.410 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2024-05-14T10:00:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2024-05-14T10:00:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2024-05-14T10:00:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2024-05-14T10:00:42.411 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "oob_default_addr": { 2024-05-14T10:00:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "oob_default_addr", 2024-05-14T10:00:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "169.254.1.1", 2024-05-14T10:00:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.412 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-14T10:00:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2024-05-14T10:00:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2024-05-14T10:00:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-14T10:00:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-14T10:00:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2024-05-14T10:00:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2024-05-14T10:00:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.414 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-14T10:00:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2024-05-14T10:00:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2024-05-14T10:00:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.415 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-14T10:00:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2024-05-14T10:00:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2024-05-14T10:00:42.416 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-14T10:00:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2024-05-14T10:00:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2024-05-14T10:00:42.417 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-14T10:00:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2024-05-14T10:00:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2024-05-14T10:00:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-14T10:00:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.419 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2024-05-14T10:00:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2024-05-14T10:00:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2024-05-14T10:00:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2024-05-14T10:00:42.420 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.421 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2024-05-14T10:00:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2024-05-14T10:00:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.422 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-14T10:00:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2024-05-14T10:00:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2024-05-14T10:00:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.423 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-14T10:00:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2024-05-14T10:00:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2024-05-14T10:00:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.424 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.425 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.425 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.425 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.425 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.425 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-14T10:00:42.425 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.425 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.425 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.425 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2024-05-14T10:00:42.425 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2024-05-14T10:00:42.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.426 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.426 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.426 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.426 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.426 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.426 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.426 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2024-05-14T10:00:42.426 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.426 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.426 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.426 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2024-05-14T10:00:42.427 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2024-05-14T10:00:42.427 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.427 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.427 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.427 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.427 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.427 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.427 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.427 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-14T10:00:42.427 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.427 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.427 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.428 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2024-05-14T10:00:42.428 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2024-05-14T10:00:42.428 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.428 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.428 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.428 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.428 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.428 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.428 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.428 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-14T10:00:42.428 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.429 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.429 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.429 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.429 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.429 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2024-05-14T10:00:42.429 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.429 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.429 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.429 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.429 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.430 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.430 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.430 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.430 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.430 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.430 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.430 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.430 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.430 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.430 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.430 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.430 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.431 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.431 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.431 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.431 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.431 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.431 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.431 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.431 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.431 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.431 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.432 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.432 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.432 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.432 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.432 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.432 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.432 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.432 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.432 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.432 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.432 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.432 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.433 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.433 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.433 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.433 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.433 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.433 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.433 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.433 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.433 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.433 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.433 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.433 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.434 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.434 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.434 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.434 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.434 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.434 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.434 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.434 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.434 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.434 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.435 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.435 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.435 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.435 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.435 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.435 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2024-05-14T10:00:42.435 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2024-05-14T10:00:42.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.435 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.435 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.436 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2024-05-14T10:00:42.436 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.436 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.436 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.436 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-14T10:00:42.436 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.436 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.436 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.436 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.436 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.436 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.437 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.437 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.437 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.437 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.437 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.437 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.437 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.437 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.437 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.437 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2024-05-14T10:00:42.437 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2024-05-14T10:00:42.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.437 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.438 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.438 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2024-05-14T10:00:42.438 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.438 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.438 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.438 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-14T10:00:42.438 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.438 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.438 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.439 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2024-05-14T10:00:42.439 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.439 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.439 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.439 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-14T10:00:42.439 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-14T10:00:42.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.439 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.439 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.439 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:42.439 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.439 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.440 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.440 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.440 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.440 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.440 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.440 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2024-05-14T10:00:42.440 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-14T10:00:42.440 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.440 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.440 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.440 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.441 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.441 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.441 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.441 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.441 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.441 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.441 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.441 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-14T10:00:42.441 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-14T10:00:42.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.442 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.442 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.442 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.442 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.442 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.442 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.442 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.442 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.442 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.442 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2024-05-14T10:00:42.443 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2024-05-14T10:00:42.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.443 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.443 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.443 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.443 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.443 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.443 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.443 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.443 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.443 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.443 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.443 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-14T10:00:42.444 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-14T10:00:42.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.444 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.444 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.444 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.444 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.444 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.444 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.444 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.444 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.444 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.445 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.445 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2024-05-14T10:00:42.445 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-14T10:00:42.445 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.445 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.445 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.445 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.445 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.445 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.445 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.446 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.446 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.446 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.446 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.446 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-14T10:00:42.446 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-14T10:00:42.446 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.446 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.446 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.446 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.447 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.447 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.447 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.447 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.447 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.447 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.447 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.447 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-14T10:00:42.447 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-14T10:00:42.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.447 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.448 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.448 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.448 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.448 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.448 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.448 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.448 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.448 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.448 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-14T10:00:42.448 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-14T10:00:42.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.448 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.449 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.449 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.449 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.449 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.449 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.449 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.449 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.449 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.449 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.449 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-14T10:00:42.449 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-14T10:00:42.449 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.449 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.450 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.450 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.450 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.450 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.450 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.450 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.450 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.450 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.450 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.450 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-14T10:00:42.450 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-14T10:00:42.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.451 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.451 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.451 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.451 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.451 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.451 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.451 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.451 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.451 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.451 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.452 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-14T10:00:42.452 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-14T10:00:42.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.452 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.452 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.452 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.452 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.452 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.452 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.452 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.452 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.452 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.452 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.453 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-14T10:00:42.453 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-14T10:00:42.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.453 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.453 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.453 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.453 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.453 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.453 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.453 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.453 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.454 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.454 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.454 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-14T10:00:42.454 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-14T10:00:42.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.454 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.454 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.454 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.454 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.454 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.454 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.455 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.455 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.455 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.455 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.455 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2024-05-14T10:00:42.455 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-14T10:00:42.455 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.455 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.455 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.455 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T10:00:42.455 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.455 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.456 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.456 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.456 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.456 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.456 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.456 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-14T10:00:42.456 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-14T10:00:42.456 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.456 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.456 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.456 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.456 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.457 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.457 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.457 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.457 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.457 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.457 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.457 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2024-05-14T10:00:42.457 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2024-05-14T10:00:42.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.457 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.457 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.458 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.458 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.458 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.458 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.458 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.458 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.458 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.458 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2024-05-14T10:00:42.458 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-14T10:00:42.458 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.458 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.459 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.459 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T10:00:42.459 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.459 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.459 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.459 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.459 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.459 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.459 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.459 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-14T10:00:42.459 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-14T10:00:42.460 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.460 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.460 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.460 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.460 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.460 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.460 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.460 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.460 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.460 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.460 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.461 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-14T10:00:42.461 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-14T10:00:42.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.461 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.461 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.461 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.461 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.461 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.461 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.461 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.461 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.461 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.462 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-14T10:00:42.462 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-14T10:00:42.462 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.462 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.462 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.462 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.462 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.462 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.462 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.462 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.463 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.466 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.466 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.466 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-14T10:00:42.466 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-14T10:00:42.466 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.466 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.467 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.467 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.467 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.467 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.467 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.467 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.467 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.467 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.467 INFO:teuthology.orchestra.run.smithi002.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-14T10:00:42.467 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-14T10:00:42.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.468 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.468 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "[]", 2024-05-14T10:00:42.468 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.468 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.468 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.468 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.468 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.468 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.468 INFO:teuthology.orchestra.run.smithi002.stdout: "MULTICLUSTER_CONFIG": { 2024-05-14T10:00:42.469 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-14T10:00:42.469 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.469 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.469 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.469 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "{}", 2024-05-14T10:00:42.469 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.469 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.469 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.469 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.469 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.469 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.469 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.470 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2024-05-14T10:00:42.470 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-14T10:00:42.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.470 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.470 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.470 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.470 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.470 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.470 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.470 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.470 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.471 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.471 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.471 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-14T10:00:42.471 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-14T10:00:42.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.471 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.471 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.471 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.471 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.471 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.471 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.472 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.472 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.472 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.472 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.472 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-14T10:00:42.472 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-14T10:00:42.472 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.472 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.472 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.472 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.473 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.473 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.473 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.473 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.473 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.473 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.473 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.473 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-14T10:00:42.473 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-14T10:00:42.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.473 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.474 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.474 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.474 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.474 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.474 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.474 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.474 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.474 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.474 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.474 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-14T10:00:42.474 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-14T10:00:42.475 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.475 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.475 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.475 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.475 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.475 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.475 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.475 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.475 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.475 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.475 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.475 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-14T10:00:42.476 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-14T10:00:42.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.476 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.476 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.476 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.476 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.476 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.476 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.476 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.476 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.476 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.476 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.477 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-14T10:00:42.477 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-14T10:00:42.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.477 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.477 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.477 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.477 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.477 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.477 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.477 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.477 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.478 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.478 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.478 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-14T10:00:42.478 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-14T10:00:42.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.478 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.478 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.478 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.478 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.478 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.478 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.478 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.479 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.479 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.479 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.479 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-14T10:00:42.479 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-14T10:00:42.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.479 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.479 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.479 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.479 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.479 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.480 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.480 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.480 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.480 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.480 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.480 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2024-05-14T10:00:42.480 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-14T10:00:42.480 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.480 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.480 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.480 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.480 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.481 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.481 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.481 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.481 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.481 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.481 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.481 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-14T10:00:42.481 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-14T10:00:42.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.481 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.481 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-14T10:00:42.482 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.482 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.482 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.482 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.482 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.482 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.482 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.482 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-14T10:00:42.482 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-14T10:00:42.482 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.482 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.483 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:42.483 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.483 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.483 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.483 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.483 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.483 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.483 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-14T10:00:42.483 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-14T10:00:42.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.484 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.484 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.484 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2024-05-14T10:00:42.484 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.484 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.484 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.484 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.484 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.484 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.484 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.484 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-14T10:00:42.485 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-14T10:00:42.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.485 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.485 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2024-05-14T10:00:42.485 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.485 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.485 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.485 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.485 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.485 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.485 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.486 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2024-05-14T10:00:42.486 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-14T10:00:42.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.486 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.486 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.486 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.486 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.486 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.486 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.486 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.486 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.486 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.487 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.487 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-14T10:00:42.487 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-14T10:00:42.487 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.487 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.487 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.487 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T10:00:42.487 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.487 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.487 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.487 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.487 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.488 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.488 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.488 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2024-05-14T10:00:42.488 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-14T10:00:42.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.488 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.488 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.488 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.488 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.488 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.489 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.489 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.489 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.489 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.489 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2024-05-14T10:00:42.489 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-14T10:00:42.489 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.489 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.489 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.489 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.489 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.489 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.490 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.490 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.490 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.490 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.490 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.490 INFO:teuthology.orchestra.run.smithi002.stdout: "UNSAFE_TLS_v1_2": { 2024-05-14T10:00:42.490 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-14T10:00:42.490 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.490 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.490 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.490 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.491 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.491 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.491 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.491 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.491 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.491 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.491 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.491 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-14T10:00:42.491 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-14T10:00:42.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.491 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.491 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.492 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.492 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.492 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.492 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.492 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.492 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.492 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.492 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-14T10:00:42.492 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-14T10:00:42.492 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.492 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.493 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.493 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:42.493 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.493 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.493 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.493 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.493 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.493 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.493 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.493 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-14T10:00:42.493 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-14T10:00:42.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.494 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.494 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.494 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T10:00:42.494 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.494 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.494 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.494 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.494 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.494 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.494 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.495 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2024-05-14T10:00:42.495 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2024-05-14T10:00:42.495 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.495 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.495 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.495 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.495 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.495 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.495 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.495 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.495 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.495 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.496 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.496 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2024-05-14T10:00:42.496 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2024-05-14T10:00:42.496 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.496 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.496 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.496 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.496 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.496 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.496 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.497 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.497 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.497 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.497 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.497 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2024-05-14T10:00:42.497 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2024-05-14T10:00:42.497 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.497 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.497 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.497 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.497 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.497 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.498 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.498 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2024-05-14T10:00:42.498 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.498 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.498 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.498 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2024-05-14T10:00:42.498 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2024-05-14T10:00:42.498 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.498 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.498 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.498 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2024-05-14T10:00:42.499 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.499 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.499 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.499 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.499 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.499 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.499 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.499 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-05-14T10:00:42.499 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-05-14T10:00:42.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.499 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.499 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.500 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.500 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.500 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.500 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.500 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.500 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.500 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.500 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.500 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.500 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.500 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.500 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.501 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.501 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.501 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.501 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.501 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.501 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.501 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.501 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.501 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.501 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.501 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.501 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.502 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.502 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.502 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.502 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.502 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.502 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.502 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.502 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.502 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.502 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.503 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.503 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.503 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.503 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.503 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.503 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.503 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.503 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.503 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.503 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.504 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.504 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.504 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.504 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.504 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.504 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.504 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.504 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.504 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.504 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.504 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.504 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.505 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.505 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.505 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.505 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.505 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.505 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.505 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.505 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.505 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.505 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.505 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.506 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.506 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.506 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.506 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.506 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.506 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.506 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2024-05-14T10:00:42.506 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2024-05-14T10:00:42.506 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.506 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.506 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.507 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.507 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.507 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.507 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.507 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2024-05-14T10:00:42.507 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.507 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.507 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.507 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2024-05-14T10:00:42.507 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2024-05-14T10:00:42.507 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.508 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.508 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.508 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.508 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.508 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.508 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.508 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.508 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.508 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.508 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T10:00:42.508 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T10:00:42.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.509 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.509 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-05-14T10:00:42.509 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.509 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.509 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.509 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.509 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.509 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.509 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.510 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T10:00:42.510 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T10:00:42.510 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.510 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.510 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.510 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2024-05-14T10:00:42.510 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.510 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.510 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.510 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.510 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.510 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.511 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.511 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.511 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.511 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.511 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.511 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.511 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.511 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.511 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.511 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.511 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.511 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.512 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.512 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-05-14T10:00:42.512 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-05-14T10:00:42.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.512 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.512 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.512 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.512 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.512 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.512 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.512 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.513 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.513 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.513 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.513 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2024-05-14T10:00:42.513 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2024-05-14T10:00:42.513 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.513 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.513 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.513 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2024-05-14T10:00:42.513 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.513 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.513 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.514 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.514 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.514 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.514 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.514 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-05-14T10:00:42.514 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-05-14T10:00:42.514 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.514 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.514 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.514 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2024-05-14T10:00:42.514 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.515 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.515 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.515 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.515 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2024-05-14T10:00:42.515 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.515 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.515 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.515 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.515 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.515 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-05-14T10:00:42.515 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-05-14T10:00:42.515 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.515 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.516 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.516 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-05-14T10:00:42.516 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-05-14T10:00:42.516 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-05-14T10:00:42.516 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.516 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.516 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.516 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.516 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.516 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2024-05-14T10:00:42.517 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2024-05-14T10:00:42.517 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.517 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.517 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.517 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.517 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.517 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.517 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.517 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.518 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.518 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.518 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.518 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.518 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-05-14T10:00:42.518 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.518 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.519 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.519 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2024-05-14T10:00:42.519 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2024-05-14T10:00:42.519 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.519 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.519 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.519 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.519 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.519 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.519 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.519 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2024-05-14T10:00:42.519 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.520 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.520 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.520 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.520 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.520 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.520 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.520 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.520 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.520 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.520 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.520 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.520 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.521 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.521 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.521 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.521 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.521 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.521 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.521 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.521 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.521 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.521 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.521 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.522 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.522 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.522 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.522 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.522 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.522 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.522 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.522 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.522 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.522 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.522 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.522 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.523 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.523 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.523 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.523 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.523 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.523 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.523 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.523 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.523 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.523 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.523 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.523 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.524 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.524 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.524 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.524 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.524 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.524 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.524 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.524 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.524 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.524 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.524 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.525 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.525 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.525 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.525 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.525 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.525 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.525 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.525 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.525 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.525 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.525 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2024-05-14T10:00:42.525 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2024-05-14T10:00:42.526 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.526 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.526 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.526 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2024-05-14T10:00:42.526 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.526 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.526 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.526 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-14T10:00:42.526 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.526 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.526 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.527 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2024-05-14T10:00:42.527 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2024-05-14T10:00:42.527 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.527 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.527 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2024-05-14T10:00:42.527 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.527 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.527 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.527 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-14T10:00:42.527 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.527 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.528 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.528 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2024-05-14T10:00:42.528 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2024-05-14T10:00:42.528 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.528 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.528 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.528 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2024-05-14T10:00:42.528 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.528 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.528 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.528 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2024-05-14T10:00:42.528 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.529 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.529 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.529 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2024-05-14T10:00:42.529 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2024-05-14T10:00:42.529 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.529 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.529 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-05-14T10:00:42.529 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.529 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.530 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.530 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-14T10:00:42.530 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.530 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.530 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.530 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2024-05-14T10:00:42.530 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2024-05-14T10:00:42.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.530 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.530 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.530 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.531 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.531 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.531 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.531 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-14T10:00:42.531 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.531 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.531 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.531 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T10:00:42.531 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T10:00:42.531 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.531 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.531 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.532 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T10:00:42.532 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.532 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.532 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.532 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2024-05-14T10:00:42.532 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.532 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.532 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.532 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.532 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.532 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.533 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.533 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.533 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.533 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.533 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.533 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.533 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.533 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.533 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.533 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.533 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2024-05-14T10:00:42.533 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2024-05-14T10:00:42.534 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.534 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.534 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.534 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2024-05-14T10:00:42.534 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.534 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.534 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.534 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-14T10:00:42.534 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.534 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.534 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.534 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.535 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.535 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2024-05-14T10:00:42.535 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.535 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.535 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.535 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.535 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.535 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.535 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.536 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.536 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.536 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.536 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.536 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.536 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.536 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.536 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.536 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.536 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.536 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.536 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.536 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.537 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.537 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.537 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.537 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.537 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.537 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.537 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.537 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.537 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.537 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.537 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.538 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.538 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.538 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.538 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.538 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.538 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.538 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.538 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.538 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.538 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.538 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.540 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.540 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.540 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.540 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.540 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.540 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.540 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.541 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.541 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.541 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.541 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.541 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.541 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.542 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.542 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.542 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.542 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.542 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.542 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.542 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.543 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2024-05-14T10:00:42.543 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2024-05-14T10:00:42.543 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.543 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.543 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.543 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-05-14T10:00:42.543 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.544 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.544 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.544 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.544 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.544 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.544 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.544 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2024-05-14T10:00:42.545 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2024-05-14T10:00:42.545 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.545 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.545 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.545 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2024-05-14T10:00:42.545 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.545 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.545 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.545 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.546 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.546 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.546 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.546 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T10:00:42.546 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T10:00:42.546 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.546 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.547 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.547 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T10:00:42.547 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.547 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.547 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.547 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.547 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.547 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.547 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.547 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.548 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.548 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.548 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.548 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.548 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.548 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.548 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.548 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.548 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.548 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.548 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.548 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.549 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2024-05-14T10:00:42.549 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2024-05-14T10:00:42.549 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2024-05-14T10:00:42.549 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.549 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2024-05-14T10:00:42.549 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2024-05-14T10:00:42.549 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.549 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.550 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.550 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2024-05-14T10:00:42.550 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.550 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.550 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.550 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-14T10:00:42.550 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.550 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.550 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.551 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2024-05-14T10:00:42.551 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2024-05-14T10:00:42.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.551 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.551 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2024-05-14T10:00:42.551 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.551 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.551 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.551 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-14T10:00:42.551 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.552 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.552 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.552 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2024-05-14T10:00:42.552 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2024-05-14T10:00:42.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.552 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.552 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.552 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.553 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.553 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.553 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2024-05-14T10:00:42.553 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.553 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.553 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.553 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T10:00:42.553 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T10:00:42.553 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.553 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.553 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.553 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2024-05-14T10:00:42.554 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2024-05-14T10:00:42.554 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.554 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.554 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-14T10:00:42.554 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.554 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.554 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.554 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.554 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.554 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.555 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.555 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.555 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.555 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.555 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.555 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.555 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.555 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.555 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.555 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.555 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.555 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.555 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.555 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.556 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.556 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.556 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.556 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.556 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.556 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.556 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.556 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.556 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.556 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.556 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.556 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.556 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.557 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.557 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.557 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.557 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.557 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.557 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.557 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.557 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.557 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.557 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.557 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.557 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.557 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.558 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.558 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.558 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.558 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.558 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.558 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.558 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.558 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.558 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.558 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.559 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.559 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.559 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.559 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.559 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.559 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.559 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.559 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.559 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.559 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.559 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2024-05-14T10:00:42.559 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2024-05-14T10:00:42.560 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.560 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.560 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.560 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.560 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.560 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.560 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.560 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2024-05-14T10:00:42.560 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.560 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.560 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.561 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2024-05-14T10:00:42.561 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2024-05-14T10:00:42.561 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.561 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.561 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.561 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2024-05-14T10:00:42.561 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.561 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.561 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.561 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2024-05-14T10:00:42.561 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.561 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.562 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.562 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.562 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.562 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.562 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.562 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.562 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.562 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.562 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.563 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.563 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.563 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.563 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-05-14T10:00:42.563 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-05-14T10:00:42.563 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.563 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.563 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2024-05-14T10:00:42.564 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.564 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.564 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.564 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T10:00:42.564 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.564 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.564 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.564 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2024-05-14T10:00:42.564 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2024-05-14T10:00:42.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.564 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.564 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.565 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T10:00:42.565 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T10:00:42.565 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2024-05-14T10:00:42.565 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.565 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-14T10:00:42.565 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.565 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.565 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.565 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2024-05-14T10:00:42.565 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2024-05-14T10:00:42.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.565 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.566 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.566 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.566 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.566 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.566 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.566 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2024-05-14T10:00:42.566 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.566 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.566 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.566 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2024-05-14T10:00:42.567 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2024-05-14T10:00:42.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.567 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.567 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2024-05-14T10:00:42.567 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.567 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.567 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.567 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T10:00:42.567 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.567 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.567 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.568 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2024-05-14T10:00:42.568 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.568 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.568 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.568 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.568 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.568 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.568 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.568 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.569 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.569 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.569 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.569 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.569 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.569 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.569 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.569 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.569 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.569 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.569 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.569 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.569 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.570 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.570 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.570 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.570 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.570 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.570 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.570 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.570 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.570 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.571 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.571 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.571 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.571 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.571 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.571 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.571 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.571 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.571 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.571 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.572 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.572 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.572 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.572 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.572 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.572 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.572 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.572 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.572 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.572 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.572 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.572 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.572 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.573 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.573 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.573 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.573 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.573 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.573 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.573 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.573 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.573 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.573 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.573 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.574 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.574 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.574 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.574 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.574 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.574 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.574 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.574 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.574 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.574 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.574 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.575 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.575 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.575 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.575 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.575 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2024-05-14T10:00:42.575 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.575 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.575 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.575 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.576 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.576 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.576 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.576 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.576 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.576 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.576 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.576 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.576 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.576 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.576 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.576 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.576 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.577 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.577 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.577 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.577 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.577 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.577 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.577 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.577 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.577 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.577 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.577 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.578 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.578 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.578 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.578 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.578 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.578 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.578 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.578 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.578 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.578 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.578 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.579 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.579 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.579 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.579 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.579 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.579 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.579 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.579 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.579 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.579 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.579 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.580 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.580 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.580 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.580 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.580 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.580 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.580 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.580 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.580 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.580 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.580 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.580 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.580 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.581 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.581 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.581 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.581 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.581 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.581 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.581 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.581 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.581 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.581 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.582 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.582 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.582 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.582 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.582 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.582 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.582 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.582 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2024-05-14T10:00:42.583 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.583 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.583 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.583 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2024-05-14T10:00:42.583 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2024-05-14T10:00:42.583 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.583 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.583 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.583 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2024-05-14T10:00:42.583 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.583 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.584 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.584 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-14T10:00:42.584 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.584 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.584 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.584 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2024-05-14T10:00:42.584 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2024-05-14T10:00:42.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.584 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.584 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.584 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:42.584 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2024-05-14T10:00:42.585 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.585 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.585 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-14T10:00:42.585 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.585 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.585 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.585 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.585 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.585 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.585 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.585 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.585 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.586 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.586 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.586 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.586 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.586 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.586 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.586 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.586 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.586 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.586 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.586 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.586 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.587 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.587 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.587 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.587 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.587 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.587 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.587 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.587 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.587 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.587 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.587 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.588 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.588 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.588 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.588 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.588 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.588 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.588 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.588 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.588 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.588 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.588 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.590 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.590 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.590 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.590 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.590 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.590 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.590 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.590 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.590 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.590 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.590 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.591 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.591 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.591 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.591 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.591 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.591 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.591 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.591 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.591 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.591 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.591 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.591 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.592 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.592 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.592 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.592 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.592 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.592 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2024-05-14T10:00:42.592 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.592 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.592 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.592 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2024-05-14T10:00:42.592 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2024-05-14T10:00:42.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.593 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.593 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.593 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2024-05-14T10:00:42.593 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.593 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.593 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.593 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2024-05-14T10:00:42.593 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-14T10:00:42.593 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.593 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.594 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.594 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.594 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.594 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.594 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.594 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.594 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.594 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.594 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.594 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.594 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.594 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.595 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.595 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.595 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.595 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.595 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.595 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.595 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.595 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.595 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.595 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.595 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.595 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.596 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.596 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.596 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.596 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.596 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.596 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.596 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.596 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.596 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.596 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.596 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.596 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.597 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.597 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.597 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.597 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.597 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.597 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.597 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.597 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.597 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.597 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.597 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.597 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.597 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.598 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.598 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.598 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.598 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.598 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.598 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.598 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.598 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.598 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.598 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.598 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.599 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.599 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.599 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.599 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.599 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.599 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2024-05-14T10:00:42.599 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2024-05-14T10:00:42.599 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.599 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.599 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.599 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.599 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.600 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.600 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.600 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2024-05-14T10:00:42.600 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-14T10:00:42.600 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.600 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.600 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2024-05-14T10:00:42.600 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2024-05-14T10:00:42.600 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.600 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.600 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.601 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2024-05-14T10:00:42.601 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.601 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.601 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.601 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2024-05-14T10:00:42.601 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.601 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.601 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.601 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2024-05-14T10:00:42.601 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2024-05-14T10:00:42.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.601 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.602 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2024-05-14T10:00:42.602 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.602 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.602 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.602 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2024-05-14T10:00:42.602 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.602 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.602 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.602 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2024-05-14T10:00:42.602 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2024-05-14T10:00:42.603 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.603 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.603 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.603 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.603 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.603 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.603 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.603 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2024-05-14T10:00:42.603 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.603 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.603 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.603 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.604 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.604 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.604 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.604 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.604 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.604 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.604 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.604 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.604 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.604 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.604 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.604 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.605 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2024-05-14T10:00:42.605 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2024-05-14T10:00:42.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.605 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.605 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.605 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2024-05-14T10:00:42.605 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.605 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.605 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.605 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-14T10:00:42.605 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-14T10:00:42.606 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.606 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.606 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2024-05-14T10:00:42.606 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.606 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.606 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.606 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.606 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.607 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.607 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.607 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.607 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.607 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.607 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.607 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.607 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.607 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.607 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.607 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.607 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.607 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.608 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.608 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.608 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.608 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.608 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.608 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.608 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.608 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.608 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.608 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.609 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.609 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.609 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.609 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.609 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.609 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.609 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.609 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.609 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.609 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.609 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.610 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.610 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.610 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.610 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.610 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.610 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.610 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.610 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.610 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.610 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.610 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.610 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.611 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.611 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.611 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.611 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.611 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.611 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.611 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.611 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.611 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.611 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.611 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.611 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.612 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.612 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.612 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.612 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.612 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.612 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.612 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.612 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.612 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.612 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.613 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.613 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.613 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.613 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.613 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.613 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.613 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.613 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.613 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2024-05-14T10:00:42.613 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.614 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.614 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.614 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.614 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.614 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.614 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.614 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.614 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.614 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.614 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.614 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.615 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.615 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.615 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.615 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.615 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.615 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.615 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.615 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.615 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.615 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.615 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.615 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.615 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.616 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.616 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.616 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.616 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.616 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.616 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.616 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.616 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.616 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.616 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.616 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.616 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.617 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.617 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.617 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.617 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.617 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.617 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.617 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.617 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.617 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.617 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.617 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.617 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.617 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.617 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.618 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.618 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.618 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.618 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.618 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.618 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.618 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.619 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.619 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.619 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.619 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.619 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.619 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.619 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.619 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.619 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.619 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.619 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.620 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.620 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.620 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.620 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.620 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.620 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.620 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.620 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.620 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.620 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.620 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.621 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2024-05-14T10:00:42.621 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.621 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.621 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.621 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.621 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.621 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.621 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.621 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.621 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.621 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.621 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.622 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.622 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.622 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.622 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.622 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.622 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.622 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.622 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.622 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.622 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.622 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.623 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.623 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.623 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.623 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.623 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.623 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.623 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.623 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.624 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.624 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.624 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.624 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.624 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.624 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.624 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.624 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.624 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.624 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.624 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.625 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.625 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.625 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.625 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.625 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.625 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.625 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.625 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.625 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.625 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.625 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.625 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.626 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.626 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.626 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.626 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.626 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.626 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.626 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.626 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.626 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.626 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.627 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.627 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.627 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.627 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.627 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.627 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.627 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.627 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.627 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.628 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.628 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.628 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.628 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.628 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.628 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-05-14T10:00:42.628 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.628 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.628 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.628 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2024-05-14T10:00:42.629 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2024-05-14T10:00:42.629 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.629 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.629 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.629 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.629 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.629 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.629 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.629 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-14T10:00:42.629 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.629 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.630 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.630 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.630 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.630 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.630 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.630 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.630 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.630 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.631 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.631 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.631 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.631 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.631 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.631 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.631 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.631 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.631 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.631 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.632 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.632 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.632 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.632 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.632 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.632 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.632 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.632 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2024-05-14T10:00:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-05-14T10:00:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-05-14T10:00:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2024-05-14T10:00:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2024-05-14T10:00:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2024-05-14T10:00:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2024-05-14T10:00:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2024-05-14T10:00:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2024-05-14T10:00:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T10:00:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T10:00:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T10:00:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2024-05-14T10:00:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2024-05-14T10:00:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-05-14T10:00:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2024-05-14T10:00:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2024-05-14T10:00:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-14T10:00:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2024-05-14T10:00:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2024-05-14T10:00:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2024-05-14T10:00:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-14T10:00:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-05-14T10:00:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-05-14T10:00:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2024-05-14T10:00:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2024-05-14T10:00:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.674 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2024-05-14T10:00:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2024-05-14T10:00:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T10:00:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T10:00:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T10:00:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2024-05-14T10:00:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2024-05-14T10:00:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2024-05-14T10:00:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2024-05-14T10:00:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2024-05-14T10:00:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2024-05-14T10:00:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-14T10:00:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-14T10:00:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2024-05-14T10:00:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2024-05-14T10:00:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-14T10:00:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-14T10:00:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2024-05-14T10:00:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2024-05-14T10:00:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2024-05-14T10:00:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2024-05-14T10:00:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T10:00:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T10:00:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-05-14T10:00:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-14T10:00:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T10:00:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T10:00:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2024-05-14T10:00:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-14T10:00:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2024-05-14T10:00:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2024-05-14T10:00:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2024-05-14T10:00:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-05-14T10:00:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-05-14T10:00:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2024-05-14T10:00:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2024-05-14T10:00:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2024-05-14T10:00:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-05-14T10:00:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-05-14T10:00:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-05-14T10:00:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-05-14T10:00:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-05-14T10:00:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-05-14T10:00:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2024-05-14T10:00:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2024-05-14T10:00:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T10:00:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2024-05-14T10:00:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2024-05-14T10:00:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2024-05-14T10:00:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2024-05-14T10:00:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2024-05-14T10:00:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2024-05-14T10:00:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2024-05-14T10:00:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-05-14T10:00:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-05-14T10:00:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T10:00:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T10:00:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T10:00:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T10:00:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2024-05-14T10:00:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2024-05-14T10:00:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2024-05-14T10:00:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2024-05-14T10:00:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2024-05-14T10:00:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-14T10:00:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2024-05-14T10:00:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2024-05-14T10:00:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2024-05-14T10:00:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.742 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.742 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.742 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.742 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.742 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.742 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2024-05-14T10:00:42.742 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2024-05-14T10:00:42.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.742 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.742 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2024-05-14T10:00:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2024-05-14T10:00:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2024-05-14T10:00:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2024-05-14T10:00:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2024-05-14T10:00:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2024-05-14T10:00:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2024-05-14T10:00:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T10:00:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.747 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.747 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.747 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2024-05-14T10:00:42.747 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2024-05-14T10:00:42.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.747 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.747 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.747 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.747 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.747 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.747 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.747 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2024-05-14T10:00:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2024-05-14T10:00:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2024-05-14T10:00:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2024-05-14T10:00:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2024-05-14T10:00:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2024-05-14T10:00:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T10:00:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2024-05-14T10:00:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2024-05-14T10:00:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2024-05-14T10:00:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2024-05-14T10:00:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2024-05-14T10:00:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2024-05-14T10:00:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2024-05-14T10:00:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smb", 2024-05-14T10:00:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "update_orchestration": { 2024-05-14T10:00:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "update_orchestration", 2024-05-14T10:00:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-14T10:00:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-05-14T10:00:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2024-05-14T10:00:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2024-05-14T10:00:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2024-05-14T10:00:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2024-05-14T10:00:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2024-05-14T10:00:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2024-05-14T10:00:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2024-05-14T10:00:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2024-05-14T10:00:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.784 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.784 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.784 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.784 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.784 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.784 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.784 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.784 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.784 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.784 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.784 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.784 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.784 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.785 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.785 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.785 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.785 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.785 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.785 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.785 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.785 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.785 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.785 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.785 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.785 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.785 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.786 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.786 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.786 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.786 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.786 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.786 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.786 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.786 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.786 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.786 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.786 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.786 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.787 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.787 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.787 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.787 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.787 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.787 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.787 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.787 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.787 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.787 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.787 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.788 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.788 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.788 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.788 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.788 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.788 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.788 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.788 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.788 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.788 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.788 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.788 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.789 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2024-05-14T10:00:42.789 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.789 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.789 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.789 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2024-05-14T10:00:42.789 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2024-05-14T10:00:42.789 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.789 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.789 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.789 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-14T10:00:42.789 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.789 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.790 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.790 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.790 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.790 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.790 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.790 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T10:00:42.790 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T10:00:42.790 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.790 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.790 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2024-05-14T10:00:42.791 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.791 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.791 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.791 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.791 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.791 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.791 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.791 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.791 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.791 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.791 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.791 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.791 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.792 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.792 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.792 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.792 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.792 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.792 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.792 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.792 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.792 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.792 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.792 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.792 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.792 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.793 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.793 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.793 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.793 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.793 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.793 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.793 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.793 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.793 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.793 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.793 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.793 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.793 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.794 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.794 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.794 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.794 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.794 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.794 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.794 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.794 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.794 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.794 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.794 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.795 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.795 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.795 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.795 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.795 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.795 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.795 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.795 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.795 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.795 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.795 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.796 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.796 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.796 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.796 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.796 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.796 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.796 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.796 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.796 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.796 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.796 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.796 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.797 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.797 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.797 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.797 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.797 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.797 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.797 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.797 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.797 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.797 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.797 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.798 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2024-05-14T10:00:42.798 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.798 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.798 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.798 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2024-05-14T10:00:42.798 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2024-05-14T10:00:42.798 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.798 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.798 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.798 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.799 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.799 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.799 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-14T10:00:42.799 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.799 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.799 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2024-05-14T10:00:42.799 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2024-05-14T10:00:42.799 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.799 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.799 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.799 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.800 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.800 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.800 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.800 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-14T10:00:42.800 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.800 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.800 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.800 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2024-05-14T10:00:42.800 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2024-05-14T10:00:42.800 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.800 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.801 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.801 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.801 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.801 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.801 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.801 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-14T10:00:42.801 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.801 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.801 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.801 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2024-05-14T10:00:42.801 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2024-05-14T10:00:42.801 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.802 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.802 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.802 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.802 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.802 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.802 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.802 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-14T10:00:42.802 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.802 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.802 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.802 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2024-05-14T10:00:42.803 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2024-05-14T10:00:42.803 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.803 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.803 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.803 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.803 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.803 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.803 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.803 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-14T10:00:42.803 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.803 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.803 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.804 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2024-05-14T10:00:42.804 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2024-05-14T10:00:42.804 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.804 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.804 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.804 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.804 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.804 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.804 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.805 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.805 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.805 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.805 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.805 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2024-05-14T10:00:42.805 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2024-05-14T10:00:42.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.805 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.805 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.805 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.806 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.806 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.806 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.806 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.806 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.806 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.806 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2024-05-14T10:00:42.806 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2024-05-14T10:00:42.806 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.806 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.807 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.807 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-14T10:00:42.807 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.807 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.807 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.807 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.807 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.807 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.807 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.807 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-05-14T10:00:42.808 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-05-14T10:00:42.808 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.808 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.808 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.808 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.808 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.808 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.808 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.808 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.808 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.808 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.808 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.809 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T10:00:42.809 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T10:00:42.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.809 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.809 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2024-05-14T10:00:42.809 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2024-05-14T10:00:42.809 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.809 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.809 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.809 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.810 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.810 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.810 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2024-05-14T10:00:42.810 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2024-05-14T10:00:42.810 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.810 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.810 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.810 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2024-05-14T10:00:42.810 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.810 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.810 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.811 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.811 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.811 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.811 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.811 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2024-05-14T10:00:42.811 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2024-05-14T10:00:42.811 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.811 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.811 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.811 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.811 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.812 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.812 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.812 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.812 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.812 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.812 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.812 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2024-05-14T10:00:42.812 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2024-05-14T10:00:42.812 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.812 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.812 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.812 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.813 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.813 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.813 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.813 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.813 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.813 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.813 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.813 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.813 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.813 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.813 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.814 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.814 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.814 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.814 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.814 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.814 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.814 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.814 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.814 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.814 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.814 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.815 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.815 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.815 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.815 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.815 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.815 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.815 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.815 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.815 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.815 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.816 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.816 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.816 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.816 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.816 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.816 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.816 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.816 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.816 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.816 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.816 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.817 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.817 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.817 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.817 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.817 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.817 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.817 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.817 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.817 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.817 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.818 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.818 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.818 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.818 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.818 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.818 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.818 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.818 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.818 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.819 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.819 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.819 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.819 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.819 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.819 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.819 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.819 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.819 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.820 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2024-05-14T10:00:42.820 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2024-05-14T10:00:42.820 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.820 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.820 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.820 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.820 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.820 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.820 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.820 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.820 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.821 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.821 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.821 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2024-05-14T10:00:42.821 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2024-05-14T10:00:42.821 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.821 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.821 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.821 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.821 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.821 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.822 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.822 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.822 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.822 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.822 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.822 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.822 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.822 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.822 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.822 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.823 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.823 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.823 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.823 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.823 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.823 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.823 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.823 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2024-05-14T10:00:42.823 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2024-05-14T10:00:42.823 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.823 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.824 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.824 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-14T10:00:42.824 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.824 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.824 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.824 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.824 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.824 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.824 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.824 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.824 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.825 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2024-05-14T10:00:42.825 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.825 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.825 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.825 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.825 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.825 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.825 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.825 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.825 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.825 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.825 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.826 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.826 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.826 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.826 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.826 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.826 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.826 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.826 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.826 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.826 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.826 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.826 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.827 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.827 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.827 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.827 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.827 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.827 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.827 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.827 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.827 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.827 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.828 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.828 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.828 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.828 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.828 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.828 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.828 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.828 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.828 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.828 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.828 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.829 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.829 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.829 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.829 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.829 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.829 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.829 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.829 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.829 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.829 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.829 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.830 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.830 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.830 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.830 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.830 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.830 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.830 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.830 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.830 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.830 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.830 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.831 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.831 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.831 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.831 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.831 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.831 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.831 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.831 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.831 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.832 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.832 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.832 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.832 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.832 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.832 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-05-14T10:00:42.832 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.832 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.833 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.833 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.833 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.833 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.833 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.833 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.833 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.833 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.833 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.833 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.833 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.833 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.834 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.834 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.834 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.834 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.834 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.834 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.834 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.834 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.834 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.834 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.835 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.835 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.835 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.835 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.835 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.835 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.835 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.835 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.835 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.835 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.836 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.836 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.836 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.836 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.836 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.836 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.836 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.836 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.837 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.837 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.837 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.837 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.837 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.837 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.837 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.837 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.837 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.837 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.838 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.838 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.838 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.838 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.838 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.838 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.838 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.838 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.838 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.838 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.839 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.839 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.839 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.839 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.839 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2024-05-14T10:00:42.839 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2024-05-14T10:00:42.839 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.839 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.839 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.839 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2024-05-14T10:00:42.839 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.839 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.840 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.840 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-14T10:00:42.840 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.840 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.840 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.840 INFO:teuthology.orchestra.run.smithi002.stdout: "periodic_async_work": { 2024-05-14T10:00:42.840 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "periodic_async_work", 2024-05-14T10:00:42.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.840 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.841 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.841 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.841 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.841 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.841 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Periodically check for async work", 2024-05-14T10:00:42.841 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.841 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.841 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.841 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2024-05-14T10:00:42.841 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2024-05-14T10:00:42.841 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.842 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.842 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.842 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.842 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.842 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.842 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.842 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-14T10:00:42.842 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.842 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.842 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_no_wait": { 2024-05-14T10:00:42.843 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_no_wait", 2024-05-14T10:00:42.843 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.843 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.843 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.843 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T10:00:42.843 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.843 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.843 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.843 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-14T10:00:42.843 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.843 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.843 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.844 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.844 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.844 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.844 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.844 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.844 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.844 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.844 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.844 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.844 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.844 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.845 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.845 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.845 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2024-05-14T10:00:42.845 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T10:00:42.845 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T10:00:42.845 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T10:00:42.845 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2024-05-14T10:00:42.845 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2024-05-14T10:00:42.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T10:00:42.846 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.846 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2024-05-14T10:00:42.846 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.846 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.846 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.846 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.846 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.846 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.846 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.846 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2024-05-14T10:00:42.846 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2024-05-14T10:00:42.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.847 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.847 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.847 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.847 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.847 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.847 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.847 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.847 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.847 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.847 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.847 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T10:00:42.847 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T10:00:42.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T10:00:42.848 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.848 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.848 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T10:00:42.848 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.848 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.848 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.848 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.848 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.848 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.848 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.848 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T10:00:42.848 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T10:00:42.848 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.849 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.849 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.849 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.849 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.849 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.849 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.849 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.849 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.849 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.849 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.849 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.849 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.849 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T10:00:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T10:00:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T10:00:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T10:00:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T10:00:42.852 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.852 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.852 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T10:00:42.852 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T10:00:42.852 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T10:00:42.852 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T10:00:42.852 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T10:00:42.852 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T10:00:42.852 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T10:00:42.852 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.852 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.852 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.852 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.853 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.853 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T10:00:42.853 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T10:00:42.853 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T10:00:42.853 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.853 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.853 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T10:00:42.853 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.853 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.853 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.853 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.853 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.854 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.854 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.854 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T10:00:42.854 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T10:00:42.854 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.854 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T10:00:42.854 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T10:00:42.854 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T10:00:42.854 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.854 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.854 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.854 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.855 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.855 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.855 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.855 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2024-05-14T10:00:42.855 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2024-05-14T10:00:42.855 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.855 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.855 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.855 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T10:00:42.855 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.856 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.856 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.856 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.856 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.856 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.856 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.856 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2024-05-14T10:00:42.856 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2024-05-14T10:00:42.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T10:00:42.856 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.857 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.857 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2024-05-14T10:00:42.857 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.857 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.857 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.857 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.857 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.857 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.857 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.857 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2024-05-14T10:00:42.857 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2024-05-14T10:00:42.857 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T10:00:42.858 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T10:00:42.858 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T10:00:42.858 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-14T10:00:42.858 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T10:00:42.858 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T10:00:42.858 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T10:00:42.858 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T10:00:42.858 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T10:00:42.858 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T10:00:42.858 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T10:00:42.858 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.858 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.859 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.859 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.859 INFO:teuthology.orchestra.run.smithi002.stdout: "services": { 2024-05-14T10:00:42.859 INFO:teuthology.orchestra.run.smithi002.stdout: "dashboard": "https://172.21.15.2:8443/" 2024-05-14T10:00:42.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.859 INFO:teuthology.orchestra.run.smithi002.stdout: "always_on_modules": { 2024-05-14T10:00:42.859 INFO:teuthology.orchestra.run.smithi002.stdout: "octopus": [ 2024-05-14T10:00:42.859 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-05-14T10:00:42.859 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-05-14T10:00:42.859 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-05-14T10:00:42.859 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-05-14T10:00:42.859 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-05-14T10:00:42.859 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-05-14T10:00:42.860 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-05-14T10:00:42.860 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-05-14T10:00:42.860 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-05-14T10:00:42.860 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-05-14T10:00:42.860 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.860 INFO:teuthology.orchestra.run.smithi002.stdout: "pacific": [ 2024-05-14T10:00:42.860 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-05-14T10:00:42.860 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-05-14T10:00:42.860 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-05-14T10:00:42.860 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-05-14T10:00:42.860 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-05-14T10:00:42.860 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-05-14T10:00:42.861 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-05-14T10:00:42.861 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-05-14T10:00:42.861 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-05-14T10:00:42.861 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-05-14T10:00:42.861 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.861 INFO:teuthology.orchestra.run.smithi002.stdout: "quincy": [ 2024-05-14T10:00:42.861 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-05-14T10:00:42.861 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-05-14T10:00:42.861 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-05-14T10:00:42.861 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-05-14T10:00:42.861 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-05-14T10:00:42.861 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-05-14T10:00:42.861 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-05-14T10:00:42.862 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-05-14T10:00:42.862 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-05-14T10:00:42.862 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-05-14T10:00:42.862 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.862 INFO:teuthology.orchestra.run.smithi002.stdout: "reef": [ 2024-05-14T10:00:42.862 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-05-14T10:00:42.862 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-05-14T10:00:42.862 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-05-14T10:00:42.862 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-05-14T10:00:42.862 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-05-14T10:00:42.862 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-05-14T10:00:42.862 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-05-14T10:00:42.863 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-05-14T10:00:42.863 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-05-14T10:00:42.863 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-05-14T10:00:42.863 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T10:00:42.863 INFO:teuthology.orchestra.run.smithi002.stdout: "squid": [ 2024-05-14T10:00:42.863 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-05-14T10:00:42.863 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-05-14T10:00:42.863 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-05-14T10:00:42.863 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-05-14T10:00:42.863 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-05-14T10:00:42.863 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-05-14T10:00:42.863 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-05-14T10:00:42.864 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-05-14T10:00:42.864 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-05-14T10:00:42.864 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-05-14T10:00:42.864 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T10:00:42.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.864 INFO:teuthology.orchestra.run.smithi002.stdout: "last_failure_osd_epoch": 3, 2024-05-14T10:00:42.864 INFO:teuthology.orchestra.run.smithi002.stdout: "active_clients": [ 2024-05-14T10:00:42.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.864 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-05-14T10:00:42.864 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-05-14T10:00:42.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.864 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-05-14T10:00:42.865 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-05-14T10:00:42.865 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1733118536 2024-05-14T10:00:42.865 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.865 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T10:00:42.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.865 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "libcephsqlite", 2024-05-14T10:00:42.865 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-05-14T10:00:42.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.865 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-05-14T10:00:42.865 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-05-14T10:00:42.865 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 865198306 2024-05-14T10:00:42.865 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.866 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T10:00:42.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.866 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-05-14T10:00:42.866 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-05-14T10:00:42.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.866 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-05-14T10:00:42.866 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-05-14T10:00:42.866 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 939147374 2024-05-14T10:00:42.866 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.866 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T10:00:42.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:42.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.867 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-05-14T10:00:42.867 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-05-14T10:00:42.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T10:00:42.867 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-05-14T10:00:42.867 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-05-14T10:00:42.867 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3572577192 2024-05-14T10:00:42.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.867 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T10:00:42.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:42.867 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T10:00:42.867 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T10:00:42.869 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:41 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T10:00:42.869 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:41 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/446194826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-14T10:00:42.869 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:41 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3280556014' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-14T10:00:42.869 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:41 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T10:00:42.869 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:41 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/446194826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-14T10:00:42.869 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:41 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3280556014' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-14T10:00:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:41.209+0000 7f2c5c2b6640 1 -- 172.21.15.2:0/3280556014 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2c400b2920 msgr2=0x7f2c400b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:41.209+0000 7f2c5c2b6640 1 --2- 172.21.15.2:0/3280556014 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2c400b2920 0x7f2c400b4e30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2c50031500 tx=0x7f2c500046c0 comp rx=0 tx=0).stop 2024-05-14T10:00:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:41.209+0000 7f2c5c2b6640 1 -- 172.21.15.2:0/3280556014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c54108a70 msgr2=0x7f2c541a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:00:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:41.209+0000 7f2c5c2b6640 1 --2- 172.21.15.2:0/3280556014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c54108a70 0x7f2c541a06b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00da10 tx=0x7f2c4c00ded0 comp rx=0 tx=0).stop 2024-05-14T10:00:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:41.209+0000 7f2c5c2b6640 1 -- 172.21.15.2:0/3280556014 shutdown_connections 2024-05-14T10:00:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:41.209+0000 7f2c5c2b6640 1 --2- 172.21.15.2:0/3280556014 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f2c400b2920 0x7f2c400b4e30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:41.209+0000 7f2c5c2b6640 1 --2- 172.21.15.2:0/3280556014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c54108a70 0x7f2c541a06b0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:41.209+0000 7f2c5c2b6640 1 --2- 172.21.15.2:0/3280556014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c54103d70 0x7f2c54199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:41.209+0000 7f2c5c2b6640 1 --2- 172.21.15.2:0/3280556014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2c54102b30 0x7f2c541991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:00:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:41.209+0000 7f2c5c2b6640 1 -- 172.21.15.2:0/3280556014 >> 172.21.15.2:0/3280556014 conn(0x7f2c540fdd20 msgr2=0x7f2c54104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:00:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:41.209+0000 7f2c5c2b6640 1 -- 172.21.15.2:0/3280556014 shutdown_connections 2024-05-14T10:00:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:00:41.209+0000 7f2c5c2b6640 1 -- 172.21.15.2:0/3280556014 wait complete. 2024-05-14T10:00:42.883 DEBUG:tasks.check_counter:Getting stats from a 2024-05-14T10:00:42.883 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-14T10:00:42.884 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:41 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T10:00:42.884 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:41 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/446194826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-14T10:00:42.884 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:41 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3280556014' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-14T10:00:43.148 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T10:00:43.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:42 smithi002 ceph-mon[31557]: pgmap v1199: 129 pgs: 129 active+clean; 582 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.6 MiB/s wr, 408 op/s 2024-05-14T10:00:43.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:42 smithi002 ceph-mon[31557]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T10:00:43.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:42 smithi029 ceph-mon[36249]: pgmap v1199: 129 pgs: 129 active+clean; 582 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.6 MiB/s wr, 408 op/s 2024-05-14T10:00:43.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:42 smithi029 ceph-mon[36249]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T10:00:43.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:42 smithi026 ceph-mon[36056]: pgmap v1199: 129 pgs: 129 active+clean; 582 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.6 MiB/s wr, 408 op/s 2024-05-14T10:00:43.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:42 smithi026 ceph-mon[36056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T10:00:44.915 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:44 smithi002 ceph-mon[31557]: pgmap v1200: 129 pgs: 129 active+clean; 535 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.7 MiB/s wr, 485 op/s 2024-05-14T10:00:44.915 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T10:00:44.916 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T10:00:44.916 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2024-05-14T10:00:44.916 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2024-05-14T10:00:44.916 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2024-05-14T10:00:44.916 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2024-05-14T10:00:44.916 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-05-14T10:00:44.916 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2024-05-14T10:00:44.916 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.001605778, 2024-05-14T10:00:44.916 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T10:00:44.916 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000360939, 2024-05-14T10:00:44.916 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T10:00:44.916 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:44.917 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:44.917 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:44.917 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:44.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:44.917 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:44.917 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:44.917 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:44.917 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:44.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:44.917 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T10:00:44.917 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T10:00:44.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:44.917 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T10:00:44.917 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 485, 2024-05-14T10:00:44.918 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 480, 2024-05-14T10:00:44.918 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 592000, 2024-05-14T10:00:44.918 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 296736, 2024-05-14T10:00:44.918 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2024-05-14T10:00:44.918 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2024-05-14T10:00:44.918 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.064979841, 2024-05-14T10:00:44.918 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.027668649, 2024-05-14T10:00:44.918 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.023856972, 2024-05-14T10:00:44.918 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.009997254, 2024-05-14T10:00:44.918 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:44.918 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 480, 2024-05-14T10:00:44.918 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.034762551, 2024-05-14T10:00:44.918 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000072421 2024-05-14T10:00:44.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:44.919 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:44.919 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:44.919 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:44.919 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:44.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:44.919 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 592000, 2024-05-14T10:00:44.919 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 296736 2024-05-14T10:00:44.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:44.919 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T10:00:44.919 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 3, 2024-05-14T10:00:44.919 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 366, 2024-05-14T10:00:44.919 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 480, 2024-05-14T10:00:44.919 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 300704, 2024-05-14T10:00:44.919 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 5, 2024-05-14T10:00:44.919 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 2, 2024-05-14T10:00:44.920 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.026152489, 2024-05-14T10:00:44.920 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.020599142, 2024-05-14T10:00:44.920 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.001986950, 2024-05-14T10:00:44.920 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T10:00:44.920 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:44.920 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 364, 2024-05-14T10:00:44.920 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.022191366, 2024-05-14T10:00:44.920 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000060965 2024-05-14T10:00:44.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:44.920 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:44.920 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:44.920 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:44.920 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:44.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:44.921 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T10:00:44.921 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 300704 2024-05-14T10:00:44.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:44.921 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-05-14T10:00:44.921 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2024-05-14T10:00:44.921 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-05-14T10:00:44.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:44.921 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-05-14T10:00:44.921 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-05-14T10:00:44.921 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-05-14T10:00:44.921 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T10:00:44.921 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-05-14T10:00:44.921 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T10:00:44.922 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-05-14T10:00:44.922 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T10:00:44.922 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T10:00:44.922 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T10:00:44.922 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T10:00:44.922 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T10:00:44.922 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-05-14T10:00:44.922 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T10:00:44.922 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T10:00:44.922 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-05-14T10:00:44.922 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-05-14T10:00:44.922 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-05-14T10:00:44.922 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-05-14T10:00:44.922 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T10:00:44.923 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T10:00:44.923 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T10:00:44.923 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T10:00:44.923 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T10:00:44.923 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-05-14T10:00:44.923 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-05-14T10:00:44.923 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-05-14T10:00:44.923 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T10:00:44.923 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T10:00:44.923 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-05-14T10:00:44.923 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-05-14T10:00:44.923 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-05-14T10:00:44.923 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-05-14T10:00:44.923 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T10:00:44.924 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-05-14T10:00:44.924 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T10:00:44.924 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-05-14T10:00:44.924 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148761, 2024-05-14T10:00:44.924 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 31, 2024-05-14T10:00:44.924 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-05-14T10:00:44.924 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-05-14T10:00:44.924 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-05-14T10:00:44.924 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-05-14T10:00:44.924 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-05-14T10:00:44.924 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-05-14T10:00:44.924 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-05-14T10:00:44.924 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-05-14T10:00:44.924 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2024-05-14T10:00:44.925 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2024-05-14T10:00:44.925 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T10:00:44.925 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-05-14T10:00:44.925 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-05-14T10:00:44.925 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-05-14T10:00:44.925 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2024-05-14T10:00:44.925 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2024-05-14T10:00:44.925 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-05-14T10:00:44.925 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-05-14T10:00:44.925 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-05-14T10:00:44.925 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-05-14T10:00:44.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:44.925 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T10:00:44.925 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-05-14T10:00:44.926 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-05-14T10:00:44.926 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-05-14T10:00:44.926 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 484, 2024-05-14T10:00:44.926 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 521037, 2024-05-14T10:00:44.926 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-05-14T10:00:44.926 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 484, 2024-05-14T10:00:44.926 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-05-14T10:00:44.926 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-05-14T10:00:44.926 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 484, 2024-05-14T10:00:44.926 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 521037, 2024-05-14T10:00:44.926 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-05-14T10:00:44.926 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:44.926 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:44.926 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:44.927 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:44.927 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:44.927 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T10:00:45.004 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:44 smithi026 ceph-mon[36056]: pgmap v1200: 129 pgs: 129 active+clean; 535 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.7 MiB/s wr, 485 op/s 2024-05-14T10:00:45.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:44 smithi029 ceph-mon[36249]: pgmap v1200: 129 pgs: 129 active+clean; 535 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.7 MiB/s wr, 485 op/s 2024-05-14T10:00:45.771 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-14T10:00:45.771 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-14T10:00:45.772 DEBUG:tasks.check_counter:Getting stats from d 2024-05-14T10:00:45.772 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-14T10:00:46.032 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T10:00:47.165 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:46 smithi002 ceph-mon[31557]: pgmap v1201: 129 pgs: 129 active+clean; 535 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 311 op/s 2024-05-14T10:00:47.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:46 smithi029 ceph-mon[36249]: pgmap v1201: 129 pgs: 129 active+clean; 535 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 311 op/s 2024-05-14T10:00:47.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:46 smithi026 ceph-mon[36056]: pgmap v1201: 129 pgs: 129 active+clean; 535 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 311 op/s 2024-05-14T10:00:48.002 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T10:00:48.002 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T10:00:48.002 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2024-05-14T10:00:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2024-05-14T10:00:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2024-05-14T10:00:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2024-05-14T10:00:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-05-14T10:00:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2024-05-14T10:00:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.001438934, 2024-05-14T10:00:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T10:00:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000339519, 2024-05-14T10:00:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T10:00:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T10:00:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T10:00:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T10:00:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 488, 2024-05-14T10:00:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 485, 2024-05-14T10:00:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 591024, 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 300288, 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.065962071, 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.028557851, 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.023284301, 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.010045496, 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 484, 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.022583590, 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000046660 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 591024, 2024-05-14T10:00:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 300288 2024-05-14T10:00:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T10:00:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 6, 2024-05-14T10:00:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 372, 2024-05-14T10:00:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 3264, 2024-05-14T10:00:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 222976, 2024-05-14T10:00:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-05-14T10:00:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 2, 2024-05-14T10:00:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.027739286, 2024-05-14T10:00:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.022166103, 2024-05-14T10:00:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.002238167, 2024-05-14T10:00:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T10:00:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 371, 2024-05-14T10:00:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.018531920, 2024-05-14T10:00:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000049951 2024-05-14T10:00:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-14T10:00:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 222976 2024-05-14T10:00:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-05-14T10:00:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2024-05-14T10:00:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-05-14T10:00:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-05-14T10:00:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-05-14T10:00:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-05-14T10:00:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T10:00:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-05-14T10:00:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T10:00:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-05-14T10:00:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T10:00:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T10:00:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T10:00:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T10:00:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T10:00:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-05-14T10:00:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T10:00:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T10:00:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-05-14T10:00:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-05-14T10:00:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-05-14T10:00:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-05-14T10:00:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T10:00:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T10:00:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T10:00:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T10:00:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T10:00:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-05-14T10:00:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-05-14T10:00:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-05-14T10:00:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T10:00:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T10:00:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-05-14T10:00:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-05-14T10:00:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-05-14T10:00:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-05-14T10:00:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T10:00:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-05-14T10:00:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T10:00:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-05-14T10:00:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 147954, 2024-05-14T10:00:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 29, 2024-05-14T10:00:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-05-14T10:00:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-05-14T10:00:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-05-14T10:00:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-05-14T10:00:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-05-14T10:00:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-05-14T10:00:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-05-14T10:00:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-05-14T10:00:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2024-05-14T10:00:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2024-05-14T10:00:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T10:00:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-05-14T10:00:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-05-14T10:00:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-05-14T10:00:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2024-05-14T10:00:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2024-05-14T10:00:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 490, 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 521984, 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 490, 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 490, 2024-05-14T10:00:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 521984, 2024-05-14T10:00:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-05-14T10:00:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:48.013 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T10:00:48.739 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-14T10:00:48.740 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-14T10:00:48.740 DEBUG:tasks.check_counter:Getting stats from g 2024-05-14T10:00:48.740 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-14T10:00:48.996 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T10:00:49.120 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:48 smithi002 ceph-mon[31557]: pgmap v1202: 129 pgs: 129 active+clean; 533 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 330 op/s 2024-05-14T10:00:49.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:48 smithi029 ceph-mon[36249]: pgmap v1202: 129 pgs: 129 active+clean; 533 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 330 op/s 2024-05-14T10:00:49.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:48 smithi026 ceph-mon[36056]: pgmap v1202: 129 pgs: 129 active+clean; 533 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 330 op/s 2024-05-14T10:00:50.773 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:50 smithi002 ceph-mon[31557]: pgmap v1203: 129 pgs: 129 active+clean; 528 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 306 op/s 2024-05-14T10:00:50.773 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T10:00:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T10:00:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 4, 2024-05-14T10:00:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 4, 2024-05-14T10:00:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 2944, 2024-05-14T10:00:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 448, 2024-05-14T10:00:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-05-14T10:00:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2024-05-14T10:00:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.001893586, 2024-05-14T10:00:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000040781, 2024-05-14T10:00:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000744212, 2024-05-14T10:00:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T10:00:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 3, 2024-05-14T10:00:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000830139, 2024-05-14T10:00:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000276713 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 448 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 3, 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 368, 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 480, 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 301056, 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 5, 2024-05-14T10:00:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 2, 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.026342917, 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.020385073, 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.002345733, 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 366, 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.016524731, 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000045149 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T10:00:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 301056 2024-05-14T10:00:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T10:00:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 483, 2024-05-14T10:00:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 480, 2024-05-14T10:00:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 589600, 2024-05-14T10:00:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 297568, 2024-05-14T10:00:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2024-05-14T10:00:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2024-05-14T10:00:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.065674822, 2024-05-14T10:00:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.028657004, 2024-05-14T10:00:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.023481904, 2024-05-14T10:00:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.009953306, 2024-05-14T10:00:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 479, 2024-05-14T10:00:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.022967694, 2024-05-14T10:00:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000047949 2024-05-14T10:00:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 589600, 2024-05-14T10:00:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 297568 2024-05-14T10:00:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-05-14T10:00:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2024-05-14T10:00:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-05-14T10:00:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-05-14T10:00:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-05-14T10:00:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-05-14T10:00:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T10:00:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-05-14T10:00:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T10:00:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-05-14T10:00:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T10:00:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T10:00:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T10:00:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T10:00:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T10:00:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-05-14T10:00:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T10:00:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T10:00:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-05-14T10:00:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 147937, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 28, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2024-05-14T10:00:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2024-05-14T10:00:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T10:00:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-05-14T10:00:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-05-14T10:00:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-05-14T10:00:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2024-05-14T10:00:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2024-05-14T10:00:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-05-14T10:00:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-05-14T10:00:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-05-14T10:00:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-05-14T10:00:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T10:00:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-05-14T10:00:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-05-14T10:00:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-05-14T10:00:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 486, 2024-05-14T10:00:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 521297, 2024-05-14T10:00:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-05-14T10:00:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 486, 2024-05-14T10:00:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-05-14T10:00:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-05-14T10:00:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 486, 2024-05-14T10:00:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 521297, 2024-05-14T10:00:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-05-14T10:00:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:50.784 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T10:00:51.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:50 smithi029 ceph-mon[36249]: pgmap v1203: 129 pgs: 129 active+clean; 528 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 306 op/s 2024-05-14T10:00:51.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:50 smithi026 ceph-mon[36056]: pgmap v1203: 129 pgs: 129 active+clean; 528 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 306 op/s 2024-05-14T10:00:51.520 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-14T10:00:51.520 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-14T10:00:51.520 DEBUG:tasks.check_counter:Getting stats from j 2024-05-14T10:00:51.520 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-14T10:00:51.773 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T10:00:53.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:52 smithi002 ceph-mon[31557]: pgmap v1204: 129 pgs: 129 active+clean; 527 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 2.8 MiB/s wr, 171 op/s 2024-05-14T10:00:53.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:52 smithi029 ceph-mon[36249]: pgmap v1204: 129 pgs: 129 active+clean; 527 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 2.8 MiB/s wr, 171 op/s 2024-05-14T10:00:53.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:52 smithi026 ceph-mon[36056]: pgmap v1204: 129 pgs: 129 active+clean; 527 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 2.8 MiB/s wr, 171 op/s 2024-05-14T10:00:53.782 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T10:00:53.782 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T10:00:53.782 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2024-05-14T10:00:53.782 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2024-05-14T10:00:53.782 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2024-05-14T10:00:53.783 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2024-05-14T10:00:53.783 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-05-14T10:00:53.783 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2024-05-14T10:00:53.783 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.001382540, 2024-05-14T10:00:53.783 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T10:00:53.783 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000199790, 2024-05-14T10:00:53.783 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T10:00:53.783 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:53.783 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:53.783 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:53.783 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:53.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:53.783 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:53.783 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T10:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T10:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T10:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 3, 2024-05-14T10:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 373, 2024-05-14T10:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 480, 2024-05-14T10:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 301936, 2024-05-14T10:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 5, 2024-05-14T10:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 2, 2024-05-14T10:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.025936376, 2024-05-14T10:00:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.020365630, 2024-05-14T10:00:53.785 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.002170542, 2024-05-14T10:00:53.785 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T10:00:53.785 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:53.785 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 371, 2024-05-14T10:00:53.785 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.020799174, 2024-05-14T10:00:53.785 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000056062 2024-05-14T10:00:53.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:53.785 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:53.785 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:53.785 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:53.785 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:53.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:53.785 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T10:00:53.785 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 301936 2024-05-14T10:00:53.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:53.786 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T10:00:53.786 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 494, 2024-05-14T10:00:53.786 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 489, 2024-05-14T10:00:53.786 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 594512, 2024-05-14T10:00:53.786 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 302016, 2024-05-14T10:00:53.786 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2024-05-14T10:00:53.786 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2024-05-14T10:00:53.786 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.065845929, 2024-05-14T10:00:53.786 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.027676813, 2024-05-14T10:00:53.786 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.024159937, 2024-05-14T10:00:53.786 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.010102450, 2024-05-14T10:00:53.786 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:53.786 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 489, 2024-05-14T10:00:53.786 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.022419942, 2024-05-14T10:00:53.787 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000045848 2024-05-14T10:00:53.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:53.787 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:53.787 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:53.787 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:53.787 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:53.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:53.787 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 594512, 2024-05-14T10:00:53.787 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 302016 2024-05-14T10:00:53.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:53.787 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-05-14T10:00:53.787 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2024-05-14T10:00:53.787 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-05-14T10:00:53.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:53.787 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-05-14T10:00:53.788 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-05-14T10:00:53.788 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-05-14T10:00:53.788 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T10:00:53.788 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-05-14T10:00:53.788 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T10:00:53.788 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-05-14T10:00:53.788 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T10:00:53.788 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T10:00:53.788 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T10:00:53.788 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T10:00:53.788 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T10:00:53.788 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-05-14T10:00:53.788 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T10:00:53.788 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T10:00:53.788 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-05-14T10:00:53.789 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-05-14T10:00:53.789 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-05-14T10:00:53.789 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-05-14T10:00:53.789 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T10:00:53.789 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T10:00:53.789 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T10:00:53.789 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T10:00:53.789 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T10:00:53.789 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-05-14T10:00:53.789 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-05-14T10:00:53.789 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-05-14T10:00:53.789 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T10:00:53.789 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T10:00:53.789 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-05-14T10:00:53.789 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-05-14T10:00:53.790 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-05-14T10:00:53.790 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-05-14T10:00:53.790 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T10:00:53.790 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-05-14T10:00:53.790 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T10:00:53.790 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-05-14T10:00:53.790 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148699, 2024-05-14T10:00:53.790 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 30, 2024-05-14T10:00:53.790 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-05-14T10:00:53.790 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-05-14T10:00:53.790 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-05-14T10:00:53.790 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-05-14T10:00:53.790 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-05-14T10:00:53.790 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-05-14T10:00:53.790 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T10:00:53.791 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-05-14T10:00:53.792 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-05-14T10:00:53.792 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-05-14T10:00:53.792 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 493, 2024-05-14T10:00:53.792 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 522271, 2024-05-14T10:00:53.792 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-05-14T10:00:53.792 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 493, 2024-05-14T10:00:53.792 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-05-14T10:00:53.792 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-05-14T10:00:53.792 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 493, 2024-05-14T10:00:53.792 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 522271, 2024-05-14T10:00:53.792 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-05-14T10:00:53.793 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T10:00:53.793 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T10:00:53.793 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T10:00:53.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:53.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T10:00:53.793 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T10:00:54.464 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-14T10:00:54.465 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-14T10:00:54.465 DEBUG:tasks.check_counter:Getting stats from b 2024-05-14T10:00:54.465 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-14T10:00:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T10:00:54.899 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:54 smithi026 ceph-mon[36056]: pgmap v1205: 129 pgs: 129 active+clean; 512 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-05-14T10:00:55.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:54 smithi002 ceph-mon[31557]: pgmap v1205: 129 pgs: 129 active+clean; 512 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-05-14T10:00:55.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:54 smithi029 ceph-mon[36249]: pgmap v1205: 129 pgs: 129 active+clean; 512 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-05-14T10:00:56.812 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-14T10:00:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T10:00:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 345073, 2024-05-14T10:00:56.812 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 355480, 2024-05-14T10:00:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 85986806, 2024-05-14T10:00:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 396330634, 2024-05-14T10:00:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 263, 2024-05-14T10:00:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 6, 2024-05-14T10:00:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 27.414420115, 2024-05-14T10:00:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 9.103064413, 2024-05-14T10:00:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 13.825415429, 2024-05-14T10:00:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 5.852898743, 2024-05-14T10:00:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 355476, 2024-05-14T10:00:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 64.682775329, 2024-05-14T10:00:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000181961 2024-05-14T10:00:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:56.813 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 489245, 2024-05-14T10:00:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.106521022, 2024-05-14T10:00:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000217 2024-05-14T10:00:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-14T10:00:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 448 2024-05-14T10:00:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T10:00:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 632704, 2024-05-14T10:00:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 903821, 2024-05-14T10:00:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 189573788, 2024-05-14T10:00:56.814 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 863628048, 2024-05-14T10:00:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 16, 2024-05-14T10:00:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 5, 2024-05-14T10:00:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 43.369137235, 2024-05-14T10:00:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 17.521608502, 2024-05-14T10:00:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 24.000587532, 2024-05-14T10:00:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 5.687764657, 2024-05-14T10:00:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 903819, 2024-05-14T10:00:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 162.971194393, 2024-05-14T10:00:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000180313 2024-05-14T10:00:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 732379, 2024-05-14T10:00:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.248570203, 2024-05-14T10:00:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000339 2024-05-14T10:00:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 56048, 2024-05-14T10:00:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 858512 2024-05-14T10:00:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T10:00:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 215285, 2024-05-14T10:00:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 246637, 2024-05-14T10:00:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 53197725, 2024-05-14T10:00:56.816 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 211869109, 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 10, 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 5, 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 21.894291509, 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 8.435881937, 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 8.556689150, 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 5.237313011, 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 246639, 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 60.966950954, 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000247191 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 295881, 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.070528038, 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000238 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.817 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 836320, 2024-05-14T10:00:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 313488 2024-05-14T10:00:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-05-14T10:00:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2024-05-14T10:00:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-05-14T10:00:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2024-05-14T10:00:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-05-14T10:00:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-05-14T10:00:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 202822, 2024-05-14T10:00:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 311.460175375, 2024-05-14T10:00:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.001535633 2024-05-14T10:00:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:56.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2024-05-14T10:00:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-05-14T10:00:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-05-14T10:00:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 91194, 2024-05-14T10:00:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 20.226588942, 2024-05-14T10:00:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000221797 2024-05-14T10:00:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2024-05-14T10:00:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 370516, 2024-05-14T10:00:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 344698, 2024-05-14T10:00:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2024-05-14T10:00:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 344698, 2024-05-14T10:00:56.819 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 2416.382117245, 2024-05-14T10:00:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.007010142 2024-05-14T10:00:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2024-05-14T10:00:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 25818, 2024-05-14T10:00:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 9, 2024-05-14T10:00:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 2, 2024-05-14T10:00:56.820 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 4647, 2024-05-14T10:00:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 23, 2024-05-14T10:00:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 28, 2024-05-14T10:00:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 1317, 2024-05-14T10:00:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 267, 2024-05-14T10:00:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 326, 2024-05-14T10:00:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 724, 2024-05-14T10:00:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 934, 2024-05-14T10:00:56.821 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 1908, 2024-05-14T10:00:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 241, 2024-05-14T10:00:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 241, 2024-05-14T10:00:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 4, 2024-05-14T10:00:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 575368, 2024-05-14T10:00:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 346303, 2024-05-14T10:00:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 25812, 2024-05-14T10:00:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 69, 2024-05-14T10:00:56.822 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2024-05-14T10:00:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2024-05-14T10:00:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 5737, 2024-05-14T10:00:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 7914, 2024-05-14T10:00:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2024-05-14T10:00:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 21, 2024-05-14T10:00:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 1166, 2024-05-14T10:00:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 21, 2024-05-14T10:00:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 328, 2024-05-14T10:00:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2024-05-14T10:00:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 162, 2024-05-14T10:00:56.823 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 104, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 23855958, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 0, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 162, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 58, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 4, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 54, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 104, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 37, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 47726, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 215492, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T10:00:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T10:00:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T10:00:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 130578, 2024-05-14T10:00:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 47364, 2024-05-14T10:00:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 18813, 2024-05-14T10:00:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 213228 2024-05-14T10:00:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2024-05-14T10:00:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 39, 2024-05-14T10:00:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 21, 2024-05-14T10:00:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 14, 2024-05-14T10:00:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 76, 2024-05-14T10:00:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 309, 2024-05-14T10:00:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 252, 2024-05-14T10:00:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2024-05-14T10:00:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2024-05-14T10:00:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 52551, 2024-05-14T10:00:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 52299, 2024-05-14T10:00:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2024-05-14T10:00:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2024-05-14T10:00:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2024-05-14T10:00:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2024-05-14T10:00:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2024-05-14T10:00:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2024-05-14T10:00:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2024-05-14T10:00:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_path": 0, 2024-05-14T10:00:56.826 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_inode": 0, 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 4, 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 21, 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 51, 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "evlrg": 0, 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 293815, 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 290500, 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 290500, 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 3315, 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 2589, 2024-05-14T10:00:56.827 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 418, 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 414, 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 414, 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 4, 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "segmjr": 1, 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 3, 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 605555177, 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 611681545, 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 0, 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 109458, 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 1411.158897518, 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.012892240 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 0 2024-05-14T10:00:56.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2024-05-14T10:00:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 338, 2024-05-14T10:00:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 53382, 2024-05-14T10:00:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 53044, 2024-05-14T10:00:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 57, 2024-05-14T10:00:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 376, 2024-05-14T10:00:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 319, 2024-05-14T10:00:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 1300, 2024-05-14T10:00:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 112836, 2024-05-14T10:00:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 111536, 2024-05-14T10:00:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 241, 2024-05-14T10:00:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 58259, 2024-05-14T10:00:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 58018, 2024-05-14T10:00:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 162640, 2024-05-14T10:00:56.829 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 231824 2024-05-14T10:00:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2024-05-14T10:00:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 511193, 2024-05-14T10:00:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 52294, 2024-05-14T10:00:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 370516, 2024-05-14T10:00:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 8209, 2024-05-14T10:00:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 112384, 2024-05-14T10:00:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2024-05-14T10:00:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 52920, 2024-05-14T10:00:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 312.606202046, 2024-05-14T10:00:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.005907146 2024-05-14T10:00:56.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2024-05-14T10:00:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 38626, 2024-05-14T10:00:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 450.229273109, 2024-05-14T10:00:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.011656119 2024-05-14T10:00:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2024-05-14T10:00:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2024-05-14T10:00:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 105098, 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 460.881963798, 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.004385259 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1, 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000222141, 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000222141 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.832 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2024-05-14T10:00:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2024-05-14T10:00:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2024-05-14T10:00:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.834 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2024-05-14T10:00:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2024-05-14T10:00:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 119, 2024-05-14T10:00:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.720023556, 2024-05-14T10:00:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.006050618 2024-05-14T10:00:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2024-05-14T10:00:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2024-05-14T10:00:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2024-05-14T10:00:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 10, 2024-05-14T10:00:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.025426330, 2024-05-14T10:00:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.002542633 2024-05-14T10:00:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2024-05-14T10:00:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 80279, 2024-05-14T10:00:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 825.821263195, 2024-05-14T10:00:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.010286890 2024-05-14T10:00:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2024-05-14T10:00:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 12711, 2024-05-14T10:00:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 66.840192114, 2024-05-14T10:00:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.005258452 2024-05-14T10:00:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2024-05-14T10:00:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2024-05-14T10:00:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2024-05-14T10:00:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 54, 2024-05-14T10:00:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 1.505548571, 2024-05-14T10:00:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.027880529 2024-05-14T10:00:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2024-05-14T10:00:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2024-05-14T10:00:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2024-05-14T10:00:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 108, 2024-05-14T10:00:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 5.768652853, 2024-05-14T10:00:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.053413452 2024-05-14T10:00:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2024-05-14T10:00:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2024-05-14T10:00:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 2256, 2024-05-14T10:00:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 4.689345774, 2024-05-14T10:00:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.002078610 2024-05-14T10:00:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2024-05-14T10:00:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2024-05-14T10:00:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 8, 2024-05-14T10:00:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.023023639, 2024-05-14T10:00:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.002877954 2024-05-14T10:00:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2024-05-14T10:00:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2024-05-14T10:00:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 52508, 2024-05-14T10:00:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 287.270980119, 2024-05-14T10:00:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.005470994 2024-05-14T10:00:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2024-05-14T10:00:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2024-05-14T10:00:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2024-05-14T10:00:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.849 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2024-05-14T10:00:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 1, 2024-05-14T10:00:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 3, 2024-05-14T10:00:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 2, 2024-05-14T10:00:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 1, 2024-05-14T10:00:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2024-05-14T10:00:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 22590, 2024-05-14T10:00:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 22590, 2024-05-14T10:00:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 1565, 2024-05-14T10:00:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "mdthresh_evicted": 0 2024-05-14T10:00:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-05-14T10:00:56.851 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 360, 2024-05-14T10:00:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 360, 2024-05-14T10:00:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T10:00:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-05-14T10:00:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T10:00:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-05-14T10:00:56.852 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T10:00:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T10:00:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T10:00:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T10:00:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T10:00:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-05-14T10:00:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T10:00:56.853 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T10:00:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-05-14T10:00:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-05-14T10:00:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-05-14T10:00:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-05-14T10:00:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T10:00:56.854 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T10:00:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T10:00:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T10:00:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T10:00:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-05-14T10:00:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-05-14T10:00:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-05-14T10:00:56.855 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T10:00:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T10:00:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-05-14T10:00:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-05-14T10:00:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-05-14T10:00:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-05-14T10:00:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T10:00:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-05-14T10:00:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T10:00:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-05-14T10:00:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 1803881, 2024-05-14T10:00:56.856 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 892, 2024-05-14T10:00:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 352, 2024-05-14T10:00:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 4, 2024-05-14T10:00:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-05-14T10:00:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-05-14T10:00:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-05-14T10:00:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-05-14T10:00:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-05-14T10:00:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-05-14T10:00:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 9288, 2024-05-14T10:00:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 180, 2024-05-14T10:00:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T10:00:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-05-14T10:00:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-05-14T10:00:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-05-14T10:00:56.857 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 2406859, 2024-05-14T10:00:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 77561, 2024-05-14T10:00:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-05-14T10:00:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-05-14T10:00:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-05-14T10:00:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-05-14T10:00:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2024-05-14T10:00:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2024-05-14T10:00:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2024-05-14T10:00:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 276075, 2024-05-14T10:00:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 656233677, 2024-05-14T10:00:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 37, 2024-05-14T10:00:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 276075, 2024-05-14T10:00:56.858 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2024-05-14T10:00:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 276075, 2024-05-14T10:00:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 15957.024407356, 2024-05-14T10:00:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.057799599 2024-05-14T10:00:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2024-05-14T10:00:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2024-05-14T10:00:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 276075, 2024-05-14T10:00:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 364717 2024-05-14T10:00:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 276075, 2024-05-14T10:00:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 1999, 2024-05-14T10:00:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 274076, 2024-05-14T10:00:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2024-05-14T10:00:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2024-05-14T10:00:56.859 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 4304, 2024-05-14T10:00:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 49721, 2024-05-14T10:00:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 1822, 2024-05-14T10:00:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 111276, 2024-05-14T10:00:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 1333, 2024-05-14T10:00:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2024-05-14T10:00:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2024-05-14T10:00:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2024-05-14T10:00:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 23, 2024-05-14T10:00:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 105535, 2024-05-14T10:00:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2024-05-14T10:00:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2024-05-14T10:00:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2024-05-14T10:00:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 177, 2024-05-14T10:00:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 75313, 2024-05-14T10:00:56.860 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2024-05-14T10:00:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2024-05-14T10:00:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2024-05-14T10:00:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2024-05-14T10:00:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2024-05-14T10:00:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2024-05-14T10:00:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T10:00:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2024-05-14T10:00:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2024-05-14T10:00:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2024-05-14T10:00:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2024-05-14T10:00:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2024-05-14T10:00:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2024-05-14T10:00:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2024-05-14T10:00:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2024-05-14T10:00:56.861 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2024-05-14T10:00:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2024-05-14T10:00:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2024-05-14T10:00:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2024-05-14T10:00:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2024-05-14T10:00:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2024-05-14T10:00:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2024-05-14T10:00:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2024-05-14T10:00:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2024-05-14T10:00:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2024-05-14T10:00:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2024-05-14T10:00:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 93, 2024-05-14T10:00:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2024-05-14T10:00:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 14, 2024-05-14T10:00:56.862 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 12, 2024-05-14T10:00:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 12, 2024-05-14T10:00:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2024-05-14T10:00:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2024-05-14T10:00:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 11477, 2024-05-14T10:00:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 22, 2024-05-14T10:00:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 3714 2024-05-14T10:00:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2024-05-14T10:00:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 1, 2024-05-14T10:00:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2024-05-14T10:00:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2024-05-14T10:00:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2024-05-14T10:00:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2024-05-14T10:00:56.863 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 0 2024-05-14T10:00:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2024-05-14T10:00:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2024-05-14T10:00:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 16, 2024-05-14T10:00:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2024-05-14T10:00:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 15, 2024-05-14T10:00:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed_ops": 104568, 2024-05-14T10:00:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 52299, 2024-05-14T10:00:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2024-05-14T10:00:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T10:00:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-14T10:00:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-05-14T10:00:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:00:56.864 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 1193058, 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 229966799, 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 1193058, 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1193058, 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 229966799, 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-14T10:00:56.865 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-05-14T10:00:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:00:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-05-14T10:00:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-05-14T10:00:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:00:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-05-14T10:00:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 276075, 2024-05-14T10:00:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 661933876, 2024-05-14T10:00:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 276075, 2024-05-14T10:00:56.866 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 661933876, 2024-05-14T10:00:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:00:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2024-05-14T10:00:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-14T10:00:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2024-05-14T10:00:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:00:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-05-14T10:00:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-05-14T10:00:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:00:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-05-14T10:00:56.867 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 276075, 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 276075, 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 276075, 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 276075, 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 52299, 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 5700351, 2024-05-14T10:00:56.868 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:00:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 52299, 2024-05-14T10:00:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-05-14T10:00:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-05-14T10:00:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1818, 2024-05-14T10:00:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 5700351, 2024-05-14T10:00:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:00:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x5608481da4e0": { 2024-05-14T10:00:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 38336, 2024-05-14T10:00:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-05-14T10:00:56.869 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:00:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 293815, 2024-05-14T10:00:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 607487241, 2024-05-14T10:00:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:00:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 293815, 2024-05-14T10:00:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-05-14T10:00:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-05-14T10:00:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 109458, 2024-05-14T10:00:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 607448905, 2024-05-14T10:00:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:00:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:56.870 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:56.871 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:56.871 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-14T10:00:57.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:56 smithi002 ceph-mon[31557]: pgmap v1206: 129 pgs: 129 active+clean; 511 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 2.2 MiB/s wr, 119 op/s 2024-05-14T10:00:57.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:56 smithi029 ceph-mon[36249]: pgmap v1206: 129 pgs: 129 active+clean; 511 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 2.2 MiB/s wr, 119 op/s 2024-05-14T10:00:57.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:56 smithi026 ceph-mon[36056]: pgmap v1206: 129 pgs: 129 active+clean; 511 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 2.2 MiB/s wr, 119 op/s 2024-05-14T10:00:57.769 INFO:tasks.check_counter:Daemon mds.b mds.exported=21 2024-05-14T10:00:57.769 INFO:tasks.check_counter:Daemon mds.b mds.imported=21 2024-05-14T10:00:57.769 DEBUG:tasks.check_counter:Getting stats from e 2024-05-14T10:00:57.769 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-14T10:00:58.022 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T10:00:58.966 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:00:58 smithi002 ceph-mon[31557]: pgmap v1207: 129 pgs: 129 active+clean; 508 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 2.6 MiB/s wr, 145 op/s 2024-05-14T10:00:59.158 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:00:58 smithi026 ceph-mon[36056]: pgmap v1207: 129 pgs: 129 active+clean; 508 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 2.6 MiB/s wr, 145 op/s 2024-05-14T10:00:59.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:00:58 smithi029 ceph-mon[36249]: pgmap v1207: 129 pgs: 129 active+clean; 508 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 2.6 MiB/s wr, 145 op/s 2024-05-14T10:00:59.838 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-14T10:00:59.838 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T10:00:59.838 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 728, 2024-05-14T10:00:59.838 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 723, 2024-05-14T10:00:59.838 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 50218688, 2024-05-14T10:00:59.838 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 254330, 2024-05-14T10:00:59.838 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 7, 2024-05-14T10:00:59.838 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2024-05-14T10:00:59.838 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.095073243, 2024-05-14T10:00:59.838 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.021195332, 2024-05-14T10:00:59.838 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.049601554, 2024-05-14T10:00:59.839 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.022542689, 2024-05-14T10:00:59.839 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:59.839 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 722, 2024-05-14T10:00:59.839 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.051840936, 2024-05-14T10:00:59.839 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000071801 2024-05-14T10:00:59.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.839 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:59.839 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.839 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.839 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.839 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 208928, 2024-05-14T10:00:59.839 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 54304 2024-05-14T10:00:59.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T10:00:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 368, 2024-05-14T10:00:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 432, 2024-05-14T10:00:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 58818423, 2024-05-14T10:00:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 318963, 2024-05-14T10:00:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 7, 2024-05-14T10:00:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 6, 2024-05-14T10:00:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.077137751, 2024-05-14T10:00:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.010381382, 2024-05-14T10:00:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.051062739, 2024-05-14T10:00:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.012785543, 2024-05-14T10:00:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 431, 2024-05-14T10:00:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.035909953, 2024-05-14T10:00:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000083317 2024-05-14T10:00:59.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.841 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:59.841 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.841 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.841 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.841 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-14T10:00:59.841 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 203472 2024-05-14T10:00:59.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.841 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T10:00:59.841 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 223, 2024-05-14T10:00:59.841 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 225, 2024-05-14T10:00:59.841 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 33586719, 2024-05-14T10:00:59.841 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 70930, 2024-05-14T10:00:59.841 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 6, 2024-05-14T10:00:59.842 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 5, 2024-05-14T10:00:59.842 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.043903555, 2024-05-14T10:00:59.842 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.004842333, 2024-05-14T10:00:59.842 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.028998268, 2024-05-14T10:00:59.842 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.008164349, 2024-05-14T10:00:59.842 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:00:59.842 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 223, 2024-05-14T10:00:59.842 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.013332745, 2024-05-14T10:00:59.842 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000059788 2024-05-14T10:00:59.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.842 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:00:59.842 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.842 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.842 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 2045, 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.209689836, 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000102537 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:59.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.844 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2024-05-14T10:00:59.844 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-05-14T10:00:59.844 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-05-14T10:00:59.844 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.844 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.844 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.844 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:59.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.844 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2024-05-14T10:00:59.844 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 0, 2024-05-14T10:00:59.844 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 0, 2024-05-14T10:00:59.844 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2024-05-14T10:00:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2024-05-14T10:00:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 0, 2024-05-14T10:00:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 0, 2024-05-14T10:00:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 0, 2024-05-14T10:00:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 0, 2024-05-14T10:00:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 0, 2024-05-14T10:00:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 0, 2024-05-14T10:00:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 14709, 2024-05-14T10:00:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 9447, 2024-05-14T10:00:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 5262, 2024-05-14T10:00:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 0, 2024-05-14T10:00:59.846 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 1363, 2024-05-14T10:00:59.846 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 0, 2024-05-14T10:00:59.846 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 0, 2024-05-14T10:00:59.846 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 0, 2024-05-14T10:00:59.846 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 2, 2024-05-14T10:00:59.846 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 0, 2024-05-14T10:00:59.846 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 0, 2024-05-14T10:00:59.846 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 0, 2024-05-14T10:00:59.846 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 0, 2024-05-14T10:00:59.846 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2024-05-14T10:00:59.846 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2024-05-14T10:00:59.846 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 0, 2024-05-14T10:00:59.846 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 0, 2024-05-14T10:00:59.846 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2024-05-14T10:00:59.846 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 0, 2024-05-14T10:00:59.847 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 0, 2024-05-14T10:00:59.847 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 0, 2024-05-14T10:00:59.847 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 0, 2024-05-14T10:00:59.847 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2024-05-14T10:00:59.847 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 0, 2024-05-14T10:00:59.847 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2024-05-14T10:00:59.847 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 4, 2024-05-14T10:00:59.847 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 234, 2024-05-14T10:00:59.847 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 0, 2024-05-14T10:00:59.847 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T10:00:59.847 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2024-05-14T10:00:59.847 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T10:00:59.847 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2024-05-14T10:00:59.847 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 0, 2024-05-14T10:00:59.847 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T10:00:59.848 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T10:00:59.848 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 0, 2024-05-14T10:00:59.848 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 0, 2024-05-14T10:00:59.848 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 0, 2024-05-14T10:00:59.848 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 0, 2024-05-14T10:00:59.848 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T10:00:59.848 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T10:00:59.848 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T10:00:59.848 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 0, 2024-05-14T10:00:59.848 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 0, 2024-05-14T10:00:59.848 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 0, 2024-05-14T10:00:59.848 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 0 2024-05-14T10:00:59.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.848 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2024-05-14T10:00:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 0, 2024-05-14T10:00:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 0, 2024-05-14T10:00:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 0, 2024-05-14T10:00:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 0, 2024-05-14T10:00:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 0, 2024-05-14T10:00:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 0, 2024-05-14T10:00:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2024-05-14T10:00:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2024-05-14T10:00:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 0, 2024-05-14T10:00:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 0, 2024-05-14T10:00:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2024-05-14T10:00:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2024-05-14T10:00:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2024-05-14T10:00:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2024-05-14T10:00:59.849 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2024-05-14T10:00:59.850 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2024-05-14T10:00:59.850 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2024-05-14T10:00:59.850 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_path": 0, 2024-05-14T10:00:59.850 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_inode": 0, 2024-05-14T10:00:59.850 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T10:00:59.850 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 0, 2024-05-14T10:00:59.850 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2024-05-14T10:00:59.850 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 0, 2024-05-14T10:00:59.850 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2024-05-14T10:00:59.850 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2024-05-14T10:00:59.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.850 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2024-05-14T10:00:59.850 INFO:teuthology.orchestra.run.smithi026.stdout: "evlrg": 0, 2024-05-14T10:00:59.850 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 0, 2024-05-14T10:00:59.851 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 0, 2024-05-14T10:00:59.851 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 0, 2024-05-14T10:00:59.851 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 70615, 2024-05-14T10:00:59.851 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2024-05-14T10:00:59.851 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 0, 2024-05-14T10:00:59.851 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 0, 2024-05-14T10:00:59.851 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 0, 2024-05-14T10:00:59.851 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 0, 2024-05-14T10:00:59.851 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 5, 2024-05-14T10:00:59.851 INFO:teuthology.orchestra.run.smithi026.stdout: "segmjr": 21, 2024-05-14T10:00:59.851 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2024-05-14T10:00:59.851 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 0, 2024-05-14T10:00:59.851 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 605555177, 2024-05-14T10:00:59.851 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 611861838, 2024-05-14T10:00:59.851 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 611860700, 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 70615 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 269, 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 14554, 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 14285, 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 52, 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 155, 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 103, 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 14709, 2024-05-14T10:00:59.852 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 15129, 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 420, 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 0, 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 0, 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 0, 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 136172, 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 215416 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 0, 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 0, 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 0, 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 0, 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 0, 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.853 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.854 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.854 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2024-05-14T10:00:59.854 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.854 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.854 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.854 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2024-05-14T10:00:59.854 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.854 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.854 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.854 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2024-05-14T10:00:59.854 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.854 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.855 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.856 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2024-05-14T10:00:59.857 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.857 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.857 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.857 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2024-05-14T10:00:59.857 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.857 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.857 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.858 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2024-05-14T10:00:59.858 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.858 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.858 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.858 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2024-05-14T10:00:59.858 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.858 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.858 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.858 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2024-05-14T10:00:59.858 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.858 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.859 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.859 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2024-05-14T10:00:59.859 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.859 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.859 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.859 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2024-05-14T10:00:59.859 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.859 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.859 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.859 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2024-05-14T10:00:59.859 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.859 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.860 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.860 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2024-05-14T10:00:59.860 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.860 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.860 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.860 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2024-05-14T10:00:59.860 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.860 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.860 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.860 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2024-05-14T10:00:59.861 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.861 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.861 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.861 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2024-05-14T10:00:59.861 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.861 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.861 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.861 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2024-05-14T10:00:59.861 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.861 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.861 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.862 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2024-05-14T10:00:59.862 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.862 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.862 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.862 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2024-05-14T10:00:59.862 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.862 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.862 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.862 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2024-05-14T10:00:59.862 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.862 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.862 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2024-05-14T10:00:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2024-05-14T10:00:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2024-05-14T10:00:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.863 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.864 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2024-05-14T10:00:59.864 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2024-05-14T10:00:59.864 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2024-05-14T10:00:59.864 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.864 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.864 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.864 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:59.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.864 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2024-05-14T10:00:59.864 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 1, 2024-05-14T10:00:59.864 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 1, 2024-05-14T10:00:59.864 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 0, 2024-05-14T10:00:59.864 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 0, 2024-05-14T10:00:59.865 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2024-05-14T10:00:59.865 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 0, 2024-05-14T10:00:59.865 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 0, 2024-05-14T10:00:59.865 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 3118, 2024-05-14T10:00:59.865 INFO:teuthology.orchestra.run.smithi026.stdout: "mdthresh_evicted": 0 2024-05-14T10:00:59.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.865 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-05-14T10:00:59.865 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-05-14T10:00:59.865 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-05-14T10:00:59.865 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T10:00:59.865 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-05-14T10:00:59.865 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T10:00:59.865 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-05-14T10:00:59.865 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T10:00:59.866 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T10:00:59.866 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T10:00:59.866 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T10:00:59.866 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T10:00:59.866 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-05-14T10:00:59.866 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T10:00:59.866 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T10:00:59.866 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-05-14T10:00:59.866 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-05-14T10:00:59.866 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-05-14T10:00:59.866 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-05-14T10:00:59.866 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T10:00:59.866 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T10:00:59.866 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T10:00:59.866 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T10:00:59.867 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T10:00:59.867 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-05-14T10:00:59.867 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-05-14T10:00:59.867 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-05-14T10:00:59.867 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T10:00:59.867 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T10:00:59.867 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-05-14T10:00:59.867 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-05-14T10:00:59.867 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-05-14T10:00:59.867 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-05-14T10:00:59.867 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T10:00:59.867 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-05-14T10:00:59.867 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T10:00:59.867 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-05-14T10:00:59.867 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 250442, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 389, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 9288, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 180, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-05-14T10:00:59.868 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 11246568, 2024-05-14T10:00:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 100895, 2024-05-14T10:00:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-05-14T10:00:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-05-14T10:00:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-05-14T10:00:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-05-14T10:00:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2024-05-14T10:00:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2024-05-14T10:00:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2024-05-14T10:00:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 1215, 2024-05-14T10:00:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 26, 2024-05-14T10:00:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 4, 2024-05-14T10:00:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 1215, 2024-05-14T10:00:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2024-05-14T10:00:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1215, 2024-05-14T10:00:59.870 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 16.135787747, 2024-05-14T10:00:59.870 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.013280483 2024-05-14T10:00:59.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.870 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2024-05-14T10:00:59.870 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2024-05-14T10:00:59.870 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1215, 2024-05-14T10:00:59.870 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 1216 2024-05-14T10:00:59.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.870 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 1215, 2024-05-14T10:00:59.870 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 1215, 2024-05-14T10:00:59.870 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 0, 2024-05-14T10:00:59.870 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2024-05-14T10:00:59.870 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2024-05-14T10:00:59.870 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 612, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 0, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 600, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 0, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 0, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 0, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 0, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 2, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 0, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2024-05-14T10:00:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2024-05-14T10:00:59.872 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2024-05-14T10:00:59.872 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2024-05-14T10:00:59.872 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2024-05-14T10:00:59.872 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2024-05-14T10:00:59.872 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T10:00:59.872 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2024-05-14T10:00:59.872 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2024-05-14T10:00:59.872 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2024-05-14T10:00:59.872 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2024-05-14T10:00:59.872 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2024-05-14T10:00:59.872 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2024-05-14T10:00:59.872 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2024-05-14T10:00:59.872 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2024-05-14T10:00:59.872 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 0, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 0, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 11, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 11, 2024-05-14T10:00:59.873 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2024-05-14T10:00:59.874 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2024-05-14T10:00:59.874 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 0, 2024-05-14T10:00:59.874 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 2, 2024-05-14T10:00:59.874 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 0 2024-05-14T10:00:59.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.874 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2024-05-14T10:00:59.874 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 0, 2024-05-14T10:00:59.874 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2024-05-14T10:00:59.874 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2024-05-14T10:00:59.874 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2024-05-14T10:00:59.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.874 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2024-05-14T10:00:59.874 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 0 2024-05-14T10:00:59.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.874 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2024-05-14T10:00:59.875 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2024-05-14T10:00:59.875 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 0, 2024-05-14T10:00:59.875 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2024-05-14T10:00:59.875 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 0, 2024-05-14T10:00:59.875 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed_ops": 0, 2024-05-14T10:00:59.875 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 0, 2024-05-14T10:00:59.875 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2024-05-14T10:00:59.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.875 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T10:00:59.875 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-14T10:00:59.875 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-05-14T10:00:59.875 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:00:59.875 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 1315, 2024-05-14T10:00:59.875 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 142497455, 2024-05-14T10:00:59.876 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:00:59.876 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 1315, 2024-05-14T10:00:59.876 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-05-14T10:00:59.876 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-05-14T10:00:59.876 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1315, 2024-05-14T10:00:59.876 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 142497455, 2024-05-14T10:00:59.876 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:00:59.876 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.876 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.876 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.876 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:59.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.876 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2024-05-14T10:00:59.876 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-14T10:00:59.876 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-05-14T10:00:59.877 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:00:59.877 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-05-14T10:00:59.877 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-05-14T10:00:59.877 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:00:59.877 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-05-14T10:00:59.877 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 1215, 2024-05-14T10:00:59.877 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 142076988, 2024-05-14T10:00:59.877 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1215, 2024-05-14T10:00:59.877 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 142076988, 2024-05-14T10:00:59.877 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:00:59.877 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.877 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.877 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.877 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:59.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.878 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2024-05-14T10:00:59.878 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-14T10:00:59.878 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2024-05-14T10:00:59.878 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:00:59.878 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-05-14T10:00:59.878 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-05-14T10:00:59.878 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:00:59.878 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-05-14T10:00:59.878 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 1215, 2024-05-14T10:00:59.878 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 1215, 2024-05-14T10:00:59.878 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1215, 2024-05-14T10:00:59.878 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1215, 2024-05-14T10:00:59.878 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:00:59.878 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.878 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.879 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.879 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:59.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.879 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2024-05-14T10:00:59.879 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-14T10:00:59.879 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-05-14T10:00:59.879 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:00:59.879 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-05-14T10:00:59.879 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-05-14T10:00:59.879 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:00:59.879 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-05-14T10:00:59.879 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-05-14T10:00:59.879 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-05-14T10:00:59.879 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x555a23e14b60": { 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-05-14T10:00:59.880 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-05-14T10:00:59.881 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-05-14T10:00:59.881 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2024-05-14T10:00:59.881 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2024-05-14T10:00:59.881 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:00:59.881 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:00:59.881 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:00:59.881 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:00:59.881 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:59.881 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:00:59.881 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-14T10:01:00.891 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:00 smithi026 ceph-mon[36056]: pgmap v1208: 129 pgs: 129 active+clean; 515 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 3.0 MiB/s wr, 146 op/s 2024-05-14T10:01:00.921 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-14T10:01:00.921 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-14T10:01:00.921 DEBUG:tasks.check_counter:Getting stats from h 2024-05-14T10:01:00.921 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-14T10:01:01.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:00 smithi002 ceph-mon[31557]: pgmap v1208: 129 pgs: 129 active+clean; 515 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 3.0 MiB/s wr, 146 op/s 2024-05-14T10:01:01.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:00 smithi029 ceph-mon[36249]: pgmap v1208: 129 pgs: 129 active+clean; 515 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 3.0 MiB/s wr, 146 op/s 2024-05-14T10:01:01.187 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T10:01:03.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:02 smithi002 ceph-mon[31557]: pgmap v1209: 129 pgs: 129 active+clean; 516 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 2.2 MiB/s wr, 106 op/s 2024-05-14T10:01:03.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:02 smithi029 ceph-mon[36249]: pgmap v1209: 129 pgs: 129 active+clean; 516 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 2.2 MiB/s wr, 106 op/s 2024-05-14T10:01:03.196 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:02 smithi026 ceph-mon[36056]: pgmap v1209: 129 pgs: 129 active+clean; 516 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 2.2 MiB/s wr, 106 op/s 2024-05-14T10:01:03.374 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-14T10:01:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T10:01:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 204433, 2024-05-14T10:01:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 172986, 2024-05-14T10:01:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 54888741, 2024-05-14T10:01:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 122966478, 2024-05-14T10:01:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 17, 2024-05-14T10:01:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 5, 2024-05-14T10:01:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 15.822056545, 2024-05-14T10:01:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 6.096973484, 2024-05-14T10:01:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 8.862856646, 2024-05-14T10:01:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 2.443735333, 2024-05-14T10:01:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 172985, 2024-05-14T10:01:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 17.922537892, 2024-05-14T10:01:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000103607 2024-05-14T10:01:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 296096, 2024-05-14T10:01:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.071789414, 2024-05-14T10:01:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000242 2024-05-14T10:01:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 823440, 2024-05-14T10:01:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 333296 2024-05-14T10:01:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T10:01:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 446531, 2024-05-14T10:01:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 676453, 2024-05-14T10:01:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 131000535, 2024-05-14T10:01:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 770741138, 2024-05-14T10:01:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 15, 2024-05-14T10:01:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 5, 2024-05-14T10:01:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 33.708913487, 2024-05-14T10:01:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 13.983548931, 2024-05-14T10:01:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 18.217371392, 2024-05-14T10:01:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 3.468911634, 2024-05-14T10:01:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 676452, 2024-05-14T10:01:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 119.745245122, 2024-05-14T10:01:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000177019 2024-05-14T10:01:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 552717, 2024-05-14T10:01:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.199384768, 2024-05-14T10:01:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000360 2024-05-14T10:01:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 4192, 2024-05-14T10:01:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 808080 2024-05-14T10:01:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T10:01:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 116212, 2024-05-14T10:01:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 111288, 2024-05-14T10:01:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 33813241, 2024-05-14T10:01:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 222344250, 2024-05-14T10:01:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 8, 2024-05-14T10:01:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 6, 2024-05-14T10:01:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 11.627995408, 2024-05-14T10:01:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 3.807893376, 2024-05-14T10:01:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 3.878184445, 2024-05-14T10:01:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 3.853583231, 2024-05-14T10:01:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 111286, 2024-05-14T10:01:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 19.836247021, 2024-05-14T10:01:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000178245 2024-05-14T10:01:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 88029, 2024-05-14T10:01:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.022510390, 2024-05-14T10:01:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000255 2024-05-14T10:01:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T10:01:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T10:01:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-05-14T10:01:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2024-05-14T10:01:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-05-14T10:01:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2024-05-14T10:01:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-05-14T10:01:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-05-14T10:01:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 103711, 2024-05-14T10:01:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 246.357133834, 2024-05-14T10:01:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.002375419 2024-05-14T10:01:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:01:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2024-05-14T10:01:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-05-14T10:01:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-05-14T10:01:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 79149, 2024-05-14T10:01:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 19.330789118, 2024-05-14T10:01:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000244232 2024-05-14T10:01:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:01:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2024-05-14T10:01:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 274511, 2024-05-14T10:01:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 265045, 2024-05-14T10:01:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2024-05-14T10:01:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 265045, 2024-05-14T10:01:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 1685.802453031, 2024-05-14T10:01:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.006360438 2024-05-14T10:01:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2024-05-14T10:01:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 9466, 2024-05-14T10:01:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 0, 2024-05-14T10:01:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 0, 2024-05-14T10:01:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 136, 2024-05-14T10:01:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 13, 2024-05-14T10:01:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 11, 2024-05-14T10:01:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 1749, 2024-05-14T10:01:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 861, 2024-05-14T10:01:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 494, 2024-05-14T10:01:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 394, 2024-05-14T10:01:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 517, 2024-05-14T10:01:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 259, 2024-05-14T10:01:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 177, 2024-05-14T10:01:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 177, 2024-05-14T10:01:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 4, 2024-05-14T10:01:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 428903, 2024-05-14T10:01:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 225006, 2024-05-14T10:01:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 9458, 2024-05-14T10:01:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 49, 2024-05-14T10:01:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2024-05-14T10:01:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2024-05-14T10:01:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 3271, 2024-05-14T10:01:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 0, 2024-05-14T10:01:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2024-05-14T10:01:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 10, 2024-05-14T10:01:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 138, 2024-05-14T10:01:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 10, 2024-05-14T10:01:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 602, 2024-05-14T10:01:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2024-05-14T10:01:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 25, 2024-05-14T10:01:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2024-05-14T10:01:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 4, 2024-05-14T10:01:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 234, 2024-05-14T10:01:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 0, 2024-05-14T10:01:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 25, 2024-05-14T10:01:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2024-05-14T10:01:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T10:01:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2024-05-14T10:01:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 0, 2024-05-14T10:01:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T10:01:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T10:01:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 25, 2024-05-14T10:01:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 6, 2024-05-14T10:01:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 14032, 2024-05-14T10:01:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 192173, 2024-05-14T10:01:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T10:01:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T10:01:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T10:01:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 90405, 2024-05-14T10:01:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 39477, 2024-05-14T10:01:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 18887, 2024-05-14T10:01:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 175975 2024-05-14T10:01:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2024-05-14T10:01:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 111, 2024-05-14T10:01:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 231, 2024-05-14T10:01:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 123, 2024-05-14T10:01:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 282, 2024-05-14T10:01:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 129, 2024-05-14T10:01:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 178, 2024-05-14T10:01:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2024-05-14T10:01:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2024-05-14T10:01:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 45148, 2024-05-14T10:01:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 44970, 2024-05-14T10:01:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2024-05-14T10:01:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2024-05-14T10:01:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2024-05-14T10:01:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2024-05-14T10:01:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2024-05-14T10:01:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2024-05-14T10:01:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2024-05-14T10:01:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_path": 0, 2024-05-14T10:01:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_inode": 0, 2024-05-14T10:01:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T10:01:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 11, 2024-05-14T10:01:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2024-05-14T10:01:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 24, 2024-05-14T10:01:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2024-05-14T10:01:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2024-05-14T10:01:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2024-05-14T10:01:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "evlrg": 0, 2024-05-14T10:01:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 231354, 2024-05-14T10:01:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 124253, 2024-05-14T10:01:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 122901, 2024-05-14T10:01:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 108453, 2024-05-14T10:01:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2024-05-14T10:01:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 1352, 2024-05-14T10:01:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 275, 2024-05-14T10:01:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 147, 2024-05-14T10:01:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 145, 2024-05-14T10:01:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 130, 2024-05-14T10:01:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "segmjr": 9, 2024-05-14T10:01:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2024-05-14T10:01:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 2, 2024-05-14T10:01:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 230073525, 2024-05-14T10:01:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 453580162, 2024-05-14T10:01:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 0, 2024-05-14T10:01:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2024-05-14T10:01:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 60575, 2024-05-14T10:01:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 864.492828596, 2024-05-14T10:01:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.014271445 2024-05-14T10:01:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 0 2024-05-14T10:01:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2024-05-14T10:01:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 283, 2024-05-14T10:01:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 45262, 2024-05-14T10:01:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 44979, 2024-05-14T10:01:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 54, 2024-05-14T10:01:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 424, 2024-05-14T10:01:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 370, 2024-05-14T10:01:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 1755, 2024-05-14T10:01:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 93119, 2024-05-14T10:01:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 91364, 2024-05-14T10:01:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 66, 2024-05-14T10:01:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 51262, 2024-05-14T10:01:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 51196, 2024-05-14T10:01:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 151128, 2024-05-14T10:01:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 223632 2024-05-14T10:01:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2024-05-14T10:01:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 370159, 2024-05-14T10:01:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 15, 2024-05-14T10:01:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 274511, 2024-05-14T10:01:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 6003, 2024-05-14T10:01:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 14247, 2024-05-14T10:01:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2024-05-14T10:01:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 44915, 2024-05-14T10:01:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 207.321987985, 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.004615874 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 2070, 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 48.413521451, 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.023388174 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2024-05-14T10:01:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 92307, 2024-05-14T10:01:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 260.313747490, 2024-05-14T10:01:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.002820086 2024-05-14T10:01:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2024-05-14T10:01:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2024-05-14T10:01:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1, 2024-05-14T10:01:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000182452, 2024-05-14T10:01:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000182452 2024-05-14T10:01:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2024-05-14T10:01:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2024-05-14T10:01:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2024-05-14T10:01:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2024-05-14T10:01:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2024-05-14T10:01:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2024-05-14T10:01:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2024-05-14T10:01:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2024-05-14T10:01:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2024-05-14T10:01:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 70727, 2024-05-14T10:01:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 923.127966724, 2024-05-14T10:01:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.013051988 2024-05-14T10:01:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2024-05-14T10:01:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 8473, 2024-05-14T10:01:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 40.080207375, 2024-05-14T10:01:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.004730344 2024-05-14T10:01:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2024-05-14T10:01:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2024-05-14T10:01:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2024-05-14T10:01:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 30, 2024-05-14T10:01:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.745377665, 2024-05-14T10:01:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.024845922 2024-05-14T10:01:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2024-05-14T10:01:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2024-05-14T10:01:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2024-05-14T10:01:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 72, 2024-05-14T10:01:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 4.345111782, 2024-05-14T10:01:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.060348774 2024-05-14T10:01:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2024-05-14T10:01:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2024-05-14T10:01:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1332, 2024-05-14T10:01:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 3.783155778, 2024-05-14T10:01:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.002840207 2024-05-14T10:01:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2024-05-14T10:01:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2024-05-14T10:01:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2024-05-14T10:01:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2024-05-14T10:01:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 45118, 2024-05-14T10:01:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 197.671194329, 2024-05-14T10:01:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.004381204 2024-05-14T10:01:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2024-05-14T10:01:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2024-05-14T10:01:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2024-05-14T10:01:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:01:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2024-05-14T10:01:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 1, 2024-05-14T10:01:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 2, 2024-05-14T10:01:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 1, 2024-05-14T10:01:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 1, 2024-05-14T10:01:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2024-05-14T10:01:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 24121, 2024-05-14T10:01:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 24121, 2024-05-14T10:01:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 1572, 2024-05-14T10:01:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "mdthresh_evicted": 0 2024-05-14T10:01:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-05-14T10:01:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-05-14T10:01:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-05-14T10:01:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T10:01:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-05-14T10:01:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T10:01:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-05-14T10:01:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T10:01:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T10:01:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T10:01:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T10:01:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T10:01:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-05-14T10:01:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T10:01:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T10:01:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-05-14T10:01:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-05-14T10:01:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-05-14T10:01:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-05-14T10:01:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-05-14T10:01:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T10:01:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-05-14T10:01:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 1434486, 2024-05-14T10:01:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 686, 2024-05-14T10:01:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 352, 2024-05-14T10:01:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 4, 2024-05-14T10:01:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-05-14T10:01:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-05-14T10:01:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-05-14T10:01:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-05-14T10:01:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-05-14T10:01:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-05-14T10:01:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 9288, 2024-05-14T10:01:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 180, 2024-05-14T10:01:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T10:01:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-05-14T10:01:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-05-14T10:01:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-05-14T10:01:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 2431825, 2024-05-14T10:01:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 67829, 2024-05-14T10:01:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-05-14T10:01:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-05-14T10:01:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-05-14T10:01:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-05-14T10:01:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2024-05-14T10:01:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2024-05-14T10:01:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2024-05-14T10:01:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 156128, 2024-05-14T10:01:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 459955860, 2024-05-14T10:01:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 33, 2024-05-14T10:01:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 156128, 2024-05-14T10:01:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2024-05-14T10:01:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 156128, 2024-05-14T10:01:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 3154.429231026, 2024-05-14T10:01:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.020204122 2024-05-14T10:01:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2024-05-14T10:01:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2024-05-14T10:01:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 156128, 2024-05-14T10:01:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 158459 2024-05-14T10:01:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 156128, 2024-05-14T10:01:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 1577, 2024-05-14T10:01:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 154551, 2024-05-14T10:01:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2024-05-14T10:01:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2024-05-14T10:01:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 10, 2024-05-14T10:01:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 61, 2024-05-14T10:01:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 1552, 2024-05-14T10:01:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 62126, 2024-05-14T10:01:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 763, 2024-05-14T10:01:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2024-05-14T10:01:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2024-05-14T10:01:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2024-05-14T10:01:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 13, 2024-05-14T10:01:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 90175, 2024-05-14T10:01:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2024-05-14T10:01:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2024-05-14T10:01:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2024-05-14T10:01:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 25, 2024-05-14T10:01:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 102, 2024-05-14T10:01:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2024-05-14T10:01:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2024-05-14T10:01:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2024-05-14T10:01:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2024-05-14T10:01:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2024-05-14T10:01:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2024-05-14T10:01:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T10:01:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2024-05-14T10:01:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2024-05-14T10:01:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2024-05-14T10:01:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2024-05-14T10:01:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2024-05-14T10:01:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2024-05-14T10:01:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2024-05-14T10:01:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2024-05-14T10:01:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2024-05-14T10:01:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2024-05-14T10:01:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2024-05-14T10:01:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2024-05-14T10:01:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2024-05-14T10:01:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2024-05-14T10:01:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2024-05-14T10:01:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2024-05-14T10:01:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2024-05-14T10:01:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2024-05-14T10:01:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2024-05-14T10:01:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 93, 2024-05-14T10:01:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2024-05-14T10:01:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 5, 2024-05-14T10:01:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 12, 2024-05-14T10:01:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 12, 2024-05-14T10:01:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2024-05-14T10:01:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2024-05-14T10:01:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 2746, 2024-05-14T10:01:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 0, 2024-05-14T10:01:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 886 2024-05-14T10:01:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2024-05-14T10:01:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 1, 2024-05-14T10:01:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2024-05-14T10:01:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2024-05-14T10:01:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2024-05-14T10:01:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2024-05-14T10:01:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 0 2024-05-14T10:01:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2024-05-14T10:01:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2024-05-14T10:01:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 17, 2024-05-14T10:01:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2024-05-14T10:01:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 13, 2024-05-14T10:01:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed_ops": 89913, 2024-05-14T10:01:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 44970, 2024-05-14T10:01:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2024-05-14T10:01:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T10:01:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-14T10:01:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-05-14T10:01:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:01:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 767172, 2024-05-14T10:01:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 156982269, 2024-05-14T10:01:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 767172, 2024-05-14T10:01:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-05-14T10:01:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-05-14T10:01:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 767172, 2024-05-14T10:01:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 156982269, 2024-05-14T10:01:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:01:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:01:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2024-05-14T10:01:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-14T10:01:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-05-14T10:01:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:01:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-05-14T10:01:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-05-14T10:01:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-05-14T10:01:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 156128, 2024-05-14T10:01:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 464857374, 2024-05-14T10:01:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 156128, 2024-05-14T10:01:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 464857374, 2024-05-14T10:01:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:01:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:01:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2024-05-14T10:01:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-14T10:01:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2024-05-14T10:01:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 156128, 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 156128, 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 156128, 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 156128, 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2024-05-14T10:01:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-14T10:01:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-05-14T10:01:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:01:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 44970, 2024-05-14T10:01:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 4901514, 2024-05-14T10:01:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 44970, 2024-05-14T10:01:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-05-14T10:01:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-05-14T10:01:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1551, 2024-05-14T10:01:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 4901514, 2024-05-14T10:01:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:01:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:01:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x55ecea696410": { 2024-05-14T10:01:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 279742, 2024-05-14T10:01:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-05-14T10:01:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:01:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 231354, 2024-05-14T10:01:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 449385858, 2024-05-14T10:01:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 231354, 2024-05-14T10:01:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-05-14T10:01:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-05-14T10:01:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 60575, 2024-05-14T10:01:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 449106116, 2024-05-14T10:01:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:01:03.419 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:03.419 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:03.419 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:03.419 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:01:03.419 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:01:03.419 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-14T10:01:04.209 INFO:tasks.check_counter:Daemon mds.h mds.exported=10 2024-05-14T10:01:04.209 INFO:tasks.check_counter:Daemon mds.h mds.imported=10 2024-05-14T10:01:04.210 DEBUG:tasks.check_counter:Getting stats from k 2024-05-14T10:01:04.210 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-14T10:01:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T10:01:04.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:04 smithi026 ceph-mon[36056]: pgmap v1210: 129 pgs: 129 active+clean; 510 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 2.9 MiB/s wr, 145 op/s 2024-05-14T10:01:05.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:04 smithi002 ceph-mon[31557]: pgmap v1210: 129 pgs: 129 active+clean; 510 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 2.9 MiB/s wr, 145 op/s 2024-05-14T10:01:05.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:04 smithi029 ceph-mon[36249]: pgmap v1210: 129 pgs: 129 active+clean; 510 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 2.9 MiB/s wr, 145 op/s 2024-05-14T10:01:06.299 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-05-14T10:01:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T10:01:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 0, 2024-05-14T10:01:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 2, 2024-05-14T10:01:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 0, 2024-05-14T10:01:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 0, 2024-05-14T10:01:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-05-14T10:01:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2024-05-14T10:01:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.001302557, 2024-05-14T10:01:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T10:01:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000334914, 2024-05-14T10:01:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T10:01:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T10:01:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T10:01:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T10:01:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 3, 2024-05-14T10:01:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 376, 2024-05-14T10:01:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 480, 2024-05-14T10:01:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 302464, 2024-05-14T10:01:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 5, 2024-05-14T10:01:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 2, 2024-05-14T10:01:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.020628908, 2024-05-14T10:01:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.014856424, 2024-05-14T10:01:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.001892559, 2024-05-14T10:01:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T10:01:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 374, 2024-05-14T10:01:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.030028185, 2024-05-14T10:01:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000080289 2024-05-14T10:01:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T10:01:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 302464 2024-05-14T10:01:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T10:01:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 498, 2024-05-14T10:01:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 493, 2024-05-14T10:01:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 595600, 2024-05-14T10:01:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 304560, 2024-05-14T10:01:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2024-05-14T10:01:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2024-05-14T10:01:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.066665041, 2024-05-14T10:01:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.025368519, 2024-05-14T10:01:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.026407695, 2024-05-14T10:01:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.011838142, 2024-05-14T10:01:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 493, 2024-05-14T10:01:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.029723109, 2024-05-14T10:01:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000060290 2024-05-14T10:01:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 595600, 2024-05-14T10:01:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 304560 2024-05-14T10:01:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-05-14T10:01:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2024-05-14T10:01:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-05-14T10:01:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-05-14T10:01:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-05-14T10:01:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-05-14T10:01:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T10:01:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-05-14T10:01:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T10:01:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-05-14T10:01:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T10:01:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T10:01:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T10:01:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T10:01:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T10:01:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-05-14T10:01:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T10:01:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T10:01:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-05-14T10:01:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-05-14T10:01:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-05-14T10:01:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-05-14T10:01:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T10:01:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T10:01:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T10:01:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T10:01:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T10:01:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-05-14T10:01:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-05-14T10:01:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-05-14T10:01:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T10:01:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T10:01:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-05-14T10:01:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-05-14T10:01:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-05-14T10:01:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-05-14T10:01:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T10:01:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-05-14T10:01:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T10:01:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-05-14T10:01:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 147937, 2024-05-14T10:01:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 28, 2024-05-14T10:01:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-05-14T10:01:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-05-14T10:01:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-05-14T10:01:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-05-14T10:01:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-05-14T10:01:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-05-14T10:01:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-05-14T10:01:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-05-14T10:01:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2024-05-14T10:01:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2024-05-14T10:01:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T10:01:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-05-14T10:01:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-05-14T10:01:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-05-14T10:01:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2024-05-14T10:01:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2024-05-14T10:01:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-05-14T10:01:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-05-14T10:01:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-05-14T10:01:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-05-14T10:01:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-05-14T10:01:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T10:01:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-05-14T10:01:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-05-14T10:01:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-05-14T10:01:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 497, 2024-05-14T10:01:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 522791, 2024-05-14T10:01:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 497, 2024-05-14T10:01:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-05-14T10:01:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-05-14T10:01:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 497, 2024-05-14T10:01:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 522791, 2024-05-14T10:01:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-05-14T10:01:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-05-14T10:01:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-05-14T10:01:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-05-14T10:01:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:01:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-05-14T10:01:06.310 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-05-14T10:01:07.033 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:06 smithi026 ceph-mon[36056]: pgmap v1211: 129 pgs: 129 active+clean; 509 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.3 MiB/s wr, 101 op/s 2024-05-14T10:01:07.034 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-14T10:01:07.034 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-14T10:01:07.034 DEBUG:tasks.check_counter:Getting stats from c 2024-05-14T10:01:07.034 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-14T10:01:07.105 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:06 smithi029 ceph-mon[36249]: pgmap v1211: 129 pgs: 129 active+clean; 509 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.3 MiB/s wr, 101 op/s 2024-05-14T10:01:07.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:06 smithi002 ceph-mon[31557]: pgmap v1211: 129 pgs: 129 active+clean; 509 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.3 MiB/s wr, 101 op/s 2024-05-14T10:01:07.559 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T10:01:09.101 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:08 smithi029 ceph-mon[36249]: pgmap v1212: 129 pgs: 129 active+clean; 502 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 3.1 MiB/s wr, 118 op/s 2024-05-14T10:01:09.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:08 smithi002 ceph-mon[31557]: pgmap v1212: 129 pgs: 129 active+clean; 502 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 3.1 MiB/s wr, 118 op/s 2024-05-14T10:01:09.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:08 smithi026 ceph-mon[36056]: pgmap v1212: 129 pgs: 129 active+clean; 502 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 3.1 MiB/s wr, 118 op/s 2024-05-14T10:01:09.426 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-05-14T10:01:09.426 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T10:01:09.426 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 3715, 2024-05-14T10:01:09.426 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 3715, 2024-05-14T10:01:09.426 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 147886492, 2024-05-14T10:01:09.426 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 1180553, 2024-05-14T10:01:09.426 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 8, 2024-05-14T10:01:09.426 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 4, 2024-05-14T10:01:09.426 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.401096421, 2024-05-14T10:01:09.426 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.092395793, 2024-05-14T10:01:09.427 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.171314526, 2024-05-14T10:01:09.427 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.110604995, 2024-05-14T10:01:09.427 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:09.427 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 3714, 2024-05-14T10:01:09.427 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.171749484, 2024-05-14T10:01:09.427 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000046243 2024-05-14T10:01:09.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.427 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:09.427 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.427 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.427 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.427 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-14T10:01:09.427 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 448 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 1420, 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 1791, 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 134582408, 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 1259818, 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 8, 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 6, 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.225007844, 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.043834750, 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.126578679, 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.041176369, 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 1789, 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.090989761, 2024-05-14T10:01:09.428 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000050860 2024-05-14T10:01:09.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.429 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:09.429 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.429 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.429 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.429 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-14T10:01:09.429 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 808832 2024-05-14T10:01:09.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.429 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T10:01:09.429 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 1807, 2024-05-14T10:01:09.429 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 1758, 2024-05-14T10:01:09.429 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 169498329, 2024-05-14T10:01:09.429 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 704600, 2024-05-14T10:01:09.430 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 6, 2024-05-14T10:01:09.430 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 4, 2024-05-14T10:01:09.430 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.277730812, 2024-05-14T10:01:09.430 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.054333462, 2024-05-14T10:01:09.430 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.162249577, 2024-05-14T10:01:09.430 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.049967788, 2024-05-14T10:01:09.430 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:09.430 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 1757, 2024-05-14T10:01:09.430 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.085637010, 2024-05-14T10:01:09.430 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000048740 2024-05-14T10:01:09.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.430 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:09.430 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.430 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 807056, 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 303920 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: "cct": { 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: "total_workers": 1, 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: "unhealthy_workers": 0 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: "finisher-MDSRank": { 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: "queue_len": 0, 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: "complete_latency": { 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 10799, 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.330649638, 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000030618 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:09.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.432 INFO:teuthology.orchestra.run.smithi029.stdout: "finisher-PurgeQueue": { 2024-05-14T10:01:09.432 INFO:teuthology.orchestra.run.smithi029.stdout: "queue_len": 0, 2024-05-14T10:01:09.432 INFO:teuthology.orchestra.run.smithi029.stdout: "complete_latency": { 2024-05-14T10:01:09.432 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.432 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.432 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.432 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:09.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.432 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": { 2024-05-14T10:01:09.432 INFO:teuthology.orchestra.run.smithi029.stdout: "request": 0, 2024-05-14T10:01:09.432 INFO:teuthology.orchestra.run.smithi029.stdout: "reply": 0, 2024-05-14T10:01:09.432 INFO:teuthology.orchestra.run.smithi029.stdout: "reply_latency": { 2024-05-14T10:01:09.432 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.432 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.432 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.433 INFO:teuthology.orchestra.run.smithi029.stdout: "slow_reply": 0, 2024-05-14T10:01:09.433 INFO:teuthology.orchestra.run.smithi029.stdout: "forward": 0, 2024-05-14T10:01:09.433 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_fetch_complete": 0, 2024-05-14T10:01:09.433 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_fetch_keys": 0, 2024-05-14T10:01:09.433 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_commit": 0, 2024-05-14T10:01:09.433 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_split": 0, 2024-05-14T10:01:09.433 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_merge": 0, 2024-05-14T10:01:09.433 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes": 45215, 2024-05-14T10:01:09.433 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_top": 15575, 2024-05-14T10:01:09.433 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_bottom": 29640, 2024-05-14T10:01:09.433 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_pin_tail": 0, 2024-05-14T10:01:09.433 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_pinned": 22964, 2024-05-14T10:01:09.433 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_expired": 0, 2024-05-14T10:01:09.434 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_with_caps": 0, 2024-05-14T10:01:09.434 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": 0, 2024-05-14T10:01:09.434 INFO:teuthology.orchestra.run.smithi029.stdout: "subtrees": 1, 2024-05-14T10:01:09.434 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse": 0, 2024-05-14T10:01:09.434 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_hit": 0, 2024-05-14T10:01:09.434 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_forward": 0, 2024-05-14T10:01:09.434 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_discover": 0, 2024-05-14T10:01:09.434 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_dir_fetch": 0, 2024-05-14T10:01:09.434 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_remote_ino": 0, 2024-05-14T10:01:09.434 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_lock": 0, 2024-05-14T10:01:09.434 INFO:teuthology.orchestra.run.smithi029.stdout: "load_cent": 0, 2024-05-14T10:01:09.434 INFO:teuthology.orchestra.run.smithi029.stdout: "q": 0, 2024-05-14T10:01:09.434 INFO:teuthology.orchestra.run.smithi029.stdout: "exported": 0, 2024-05-14T10:01:09.434 INFO:teuthology.orchestra.run.smithi029.stdout: "exported_inodes": 0, 2024-05-14T10:01:09.435 INFO:teuthology.orchestra.run.smithi029.stdout: "imported": 0, 2024-05-14T10:01:09.435 INFO:teuthology.orchestra.run.smithi029.stdout: "imported_inodes": 0, 2024-05-14T10:01:09.435 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_dir_fetch": 0, 2024-05-14T10:01:09.435 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_backtrace_fetch": 0, 2024-05-14T10:01:09.435 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_peer_discover": 0, 2024-05-14T10:01:09.435 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rfiles": 0, 2024-05-14T10:01:09.435 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rbytes": 0, 2024-05-14T10:01:09.435 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rsnaps": 0, 2024-05-14T10:01:09.435 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T10:01:09.435 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_set_tag": 0, 2024-05-14T10:01:09.435 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T10:01:09.435 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_inotable_repaired": 0, 2024-05-14T10:01:09.435 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dir_inodes": 0, 2024-05-14T10:01:09.435 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T10:01:09.436 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T10:01:09.436 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_file_inodes": 0, 2024-05-14T10:01:09.436 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_inode_file_caps": 0, 2024-05-14T10:01:09.436 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_revoke": 0, 2024-05-14T10:01:09.436 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_grant": 0, 2024-05-14T10:01:09.436 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T10:01:09.436 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T10:01:09.436 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T10:01:09.436 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_caps": 0, 2024-05-14T10:01:09.436 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_caps_dirty": 0, 2024-05-14T10:01:09.436 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_cap_release": 0, 2024-05-14T10:01:09.436 INFO:teuthology.orchestra.run.smithi029.stdout: "process_request_cap_release": 0 2024-05-14T10:01:09.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.436 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache": { 2024-05-14T10:01:09.436 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_update": 0, 2024-05-14T10:01:09.437 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_update_receipt": 0, 2024-05-14T10:01:09.437 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_try_discover": 0, 2024-05-14T10:01:09.437 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_send_discover": 0, 2024-05-14T10:01:09.437 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_handle_discover": 0, 2024-05-14T10:01:09.437 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays": 0, 2024-05-14T10:01:09.437 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays_delayed": 0, 2024-05-14T10:01:09.437 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays_enqueuing": 0, 2024-05-14T10:01:09.437 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_created": 0, 2024-05-14T10:01:09.437 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_enqueued": 0, 2024-05-14T10:01:09.437 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_reintegrated": 0, 2024-05-14T10:01:09.437 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_migrated": 0, 2024-05-14T10:01:09.437 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_processing": 0, 2024-05-14T10:01:09.437 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_enqueued": 0, 2024-05-14T10:01:09.437 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_prioritized": 0, 2024-05-14T10:01:09.437 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_started": 0, 2024-05-14T10:01:09.438 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_completed": 0, 2024-05-14T10:01:09.438 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_quiesce_path": 0, 2024-05-14T10:01:09.438 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_quiesce_inode": 0, 2024-05-14T10:01:09.438 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T10:01:09.438 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_exportdir": 0, 2024-05-14T10:01:09.438 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_flush": 0, 2024-05-14T10:01:09.438 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_fragmentdir": 0, 2024-05-14T10:01:09.438 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_fragstats": 0, 2024-05-14T10:01:09.438 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_inodestats": 0 2024-05-14T10:01:09.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.438 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log": { 2024-05-14T10:01:09.438 INFO:teuthology.orchestra.run.smithi029.stdout: "evlrg": 0, 2024-05-14T10:01:09.438 INFO:teuthology.orchestra.run.smithi029.stdout: "evadd": 0, 2024-05-14T10:01:09.438 INFO:teuthology.orchestra.run.smithi029.stdout: "evex": 0, 2024-05-14T10:01:09.439 INFO:teuthology.orchestra.run.smithi029.stdout: "evtrm": 0, 2024-05-14T10:01:09.439 INFO:teuthology.orchestra.run.smithi029.stdout: "ev": 231355, 2024-05-14T10:01:09.439 INFO:teuthology.orchestra.run.smithi029.stdout: "evexg": 0, 2024-05-14T10:01:09.439 INFO:teuthology.orchestra.run.smithi029.stdout: "evexd": 0, 2024-05-14T10:01:09.439 INFO:teuthology.orchestra.run.smithi029.stdout: "segadd": 0, 2024-05-14T10:01:09.439 INFO:teuthology.orchestra.run.smithi029.stdout: "segex": 0, 2024-05-14T10:01:09.439 INFO:teuthology.orchestra.run.smithi029.stdout: "segtrm": 0, 2024-05-14T10:01:09.439 INFO:teuthology.orchestra.run.smithi029.stdout: "seg": 130, 2024-05-14T10:01:09.439 INFO:teuthology.orchestra.run.smithi029.stdout: "segmjr": 20, 2024-05-14T10:01:09.439 INFO:teuthology.orchestra.run.smithi029.stdout: "segexg": 0, 2024-05-14T10:01:09.439 INFO:teuthology.orchestra.run.smithi029.stdout: "segexd": 0, 2024-05-14T10:01:09.439 INFO:teuthology.orchestra.run.smithi029.stdout: "expos": 230073525, 2024-05-14T10:01:09.439 INFO:teuthology.orchestra.run.smithi029.stdout: "wrpos": 453581300, 2024-05-14T10:01:09.439 INFO:teuthology.orchestra.run.smithi029.stdout: "rdpos": 453580162, 2024-05-14T10:01:09.439 INFO:teuthology.orchestra.run.smithi029.stdout: "jlat": { 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: "replayed": 231355 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_mem": { 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: "ino": 191, 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: "ino+": 45233, 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: "ino-": 45042, 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: "dir": 28, 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: "dir+": 192, 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: "dir-": 164, 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: "dn": 45215, 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: "dn+": 45794, 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: "dn-": 579, 2024-05-14T10:01:09.440 INFO:teuthology.orchestra.run.smithi029.stdout: "cap": 0, 2024-05-14T10:01:09.441 INFO:teuthology.orchestra.run.smithi029.stdout: "cap+": 0, 2024-05-14T10:01:09.441 INFO:teuthology.orchestra.run.smithi029.stdout: "cap-": 0, 2024-05-14T10:01:09.441 INFO:teuthology.orchestra.run.smithi029.stdout: "rss": 167180, 2024-05-14T10:01:09.441 INFO:teuthology.orchestra.run.smithi029.stdout: "heap": 215440 2024-05-14T10:01:09.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.441 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_server": { 2024-05-14T10:01:09.441 INFO:teuthology.orchestra.run.smithi029.stdout: "dispatch_client_request": 0, 2024-05-14T10:01:09.441 INFO:teuthology.orchestra.run.smithi029.stdout: "dispatch_server_request": 0, 2024-05-14T10:01:09.441 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_request": 0, 2024-05-14T10:01:09.441 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_session": 0, 2024-05-14T10:01:09.441 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_peer_request": 0, 2024-05-14T10:01:09.441 INFO:teuthology.orchestra.run.smithi029.stdout: "req_create_latency": { 2024-05-14T10:01:09.441 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.441 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.441 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.442 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getattr_latency": { 2024-05-14T10:01:09.442 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.442 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.442 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.442 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getfilelock_latency": { 2024-05-14T10:01:09.442 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.442 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.442 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.442 INFO:teuthology.orchestra.run.smithi029.stdout: "req_link_latency": { 2024-05-14T10:01:09.442 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.442 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.442 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.443 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookup_latency": { 2024-05-14T10:01:09.443 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.443 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.443 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.443 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookuphash_latency": { 2024-05-14T10:01:09.443 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.443 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.443 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.443 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupino_latency": { 2024-05-14T10:01:09.443 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.443 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupname_latency": { 2024-05-14T10:01:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupparent_latency": { 2024-05-14T10:01:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupsnap_latency": { 2024-05-14T10:01:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.444 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.445 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.445 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lssnap_latency": { 2024-05-14T10:01:09.445 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.445 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.445 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.445 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mkdir_latency": { 2024-05-14T10:01:09.446 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.446 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.446 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.446 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mknod_latency": { 2024-05-14T10:01:09.446 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.446 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.446 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.446 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mksnap_latency": { 2024-05-14T10:01:09.446 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.446 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.446 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.447 INFO:teuthology.orchestra.run.smithi029.stdout: "req_open_latency": { 2024-05-14T10:01:09.447 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.447 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.447 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.447 INFO:teuthology.orchestra.run.smithi029.stdout: "req_readdir_latency": { 2024-05-14T10:01:09.447 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.447 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.447 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.447 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rename_latency": { 2024-05-14T10:01:09.447 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.447 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.448 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.448 INFO:teuthology.orchestra.run.smithi029.stdout: "req_renamesnap_latency": { 2024-05-14T10:01:09.448 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.448 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.448 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.448 INFO:teuthology.orchestra.run.smithi029.stdout: "req_snapdiff_latency": { 2024-05-14T10:01:09.448 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.448 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.448 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.448 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmdir_latency": { 2024-05-14T10:01:09.448 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.448 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.449 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.449 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmsnap_latency": { 2024-05-14T10:01:09.449 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.449 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.449 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.449 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmxattr_latency": { 2024-05-14T10:01:09.449 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.449 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.449 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.449 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setattr_latency": { 2024-05-14T10:01:09.449 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.450 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.450 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.450 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setdirlayout_latency": { 2024-05-14T10:01:09.450 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.450 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.450 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.450 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setfilelock_latency": { 2024-05-14T10:01:09.450 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.450 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.450 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.450 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setlayout_latency": { 2024-05-14T10:01:09.451 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.451 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.451 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.451 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setxattr_latency": { 2024-05-14T10:01:09.451 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.451 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.451 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.451 INFO:teuthology.orchestra.run.smithi029.stdout: "req_symlink_latency": { 2024-05-14T10:01:09.451 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.451 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.451 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.452 INFO:teuthology.orchestra.run.smithi029.stdout: "req_unlink_latency": { 2024-05-14T10:01:09.452 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.452 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.452 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.452 INFO:teuthology.orchestra.run.smithi029.stdout: "cap_revoke_eviction": 0, 2024-05-14T10:01:09.452 INFO:teuthology.orchestra.run.smithi029.stdout: "cap_acquisition_throttle": 0, 2024-05-14T10:01:09.452 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getvxattr_latency": { 2024-05-14T10:01:09.452 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.452 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.452 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.452 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:09.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.453 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_sessions": { 2024-05-14T10:01:09.453 INFO:teuthology.orchestra.run.smithi029.stdout: "session_count": 1, 2024-05-14T10:01:09.453 INFO:teuthology.orchestra.run.smithi029.stdout: "session_add": 2, 2024-05-14T10:01:09.453 INFO:teuthology.orchestra.run.smithi029.stdout: "session_remove": 1, 2024-05-14T10:01:09.453 INFO:teuthology.orchestra.run.smithi029.stdout: "sessions_open": 1, 2024-05-14T10:01:09.453 INFO:teuthology.orchestra.run.smithi029.stdout: "sessions_stale": 0, 2024-05-14T10:01:09.453 INFO:teuthology.orchestra.run.smithi029.stdout: "total_load": 0, 2024-05-14T10:01:09.453 INFO:teuthology.orchestra.run.smithi029.stdout: "average_load": 0, 2024-05-14T10:01:09.453 INFO:teuthology.orchestra.run.smithi029.stdout: "avg_session_uptime": 3081, 2024-05-14T10:01:09.453 INFO:teuthology.orchestra.run.smithi029.stdout: "mdthresh_evicted": 0 2024-05-14T10:01:09.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.453 INFO:teuthology.orchestra.run.smithi029.stdout: "mempool": { 2024-05-14T10:01:09.453 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_bytes": 0, 2024-05-14T10:01:09.453 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_items": 0, 2024-05-14T10:01:09.454 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T10:01:09.454 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_items": 0, 2024-05-14T10:01:09.454 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T10:01:09.454 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_items": 0, 2024-05-14T10:01:09.454 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T10:01:09.454 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T10:01:09.454 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T10:01:09.454 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T10:01:09.454 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T10:01:09.454 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_items": 0, 2024-05-14T10:01:09.454 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T10:01:09.454 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T10:01:09.454 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_bytes": 0, 2024-05-14T10:01:09.454 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_items": 0, 2024-05-14T10:01:09.455 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_bytes": 0, 2024-05-14T10:01:09.455 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_items": 0, 2024-05-14T10:01:09.455 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T10:01:09.455 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T10:01:09.455 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T10:01:09.455 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T10:01:09.455 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T10:01:09.455 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_items": 0, 2024-05-14T10:01:09.455 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_bytes": 0, 2024-05-14T10:01:09.455 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_items": 0, 2024-05-14T10:01:09.455 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T10:01:09.455 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T10:01:09.455 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_bytes": 0, 2024-05-14T10:01:09.455 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_items": 0, 2024-05-14T10:01:09.455 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_bytes": 0, 2024-05-14T10:01:09.456 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_items": 0, 2024-05-14T10:01:09.456 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T10:01:09.456 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_items": 0, 2024-05-14T10:01:09.456 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T10:01:09.456 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_items": 0, 2024-05-14T10:01:09.456 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_bytes": 249177, 2024-05-14T10:01:09.456 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_items": 264, 2024-05-14T10:01:09.456 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_bytes": 0, 2024-05-14T10:01:09.456 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_items": 0, 2024-05-14T10:01:09.456 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bytes": 0, 2024-05-14T10:01:09.456 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_items": 0, 2024-05-14T10:01:09.456 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_bytes": 0, 2024-05-14T10:01:09.456 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_items": 0, 2024-05-14T10:01:09.456 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_bytes": 0, 2024-05-14T10:01:09.456 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_items": 0, 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_bytes": 9288, 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_items": 180, 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_items": 0, 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_bytes": 0, 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_items": 0, 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_bytes": 32335380, 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_items": 204627, 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_bytes": 0, 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_items": 0, 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_bytes": 0, 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_items": 0 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter": { 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: "op_active": 0, 2024-05-14T10:01:09.457 INFO:teuthology.orchestra.run.smithi029.stdout: "op_laggy": 0, 2024-05-14T10:01:09.458 INFO:teuthology.orchestra.run.smithi029.stdout: "op_send": 6389, 2024-05-14T10:01:09.458 INFO:teuthology.orchestra.run.smithi029.stdout: "op_send_bytes": 21, 2024-05-14T10:01:09.458 INFO:teuthology.orchestra.run.smithi029.stdout: "op_resend": 3, 2024-05-14T10:01:09.458 INFO:teuthology.orchestra.run.smithi029.stdout: "op_reply": 6389, 2024-05-14T10:01:09.458 INFO:teuthology.orchestra.run.smithi029.stdout: "op_latency": { 2024-05-14T10:01:09.458 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 6389, 2024-05-14T10:01:09.458 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 51.007176864, 2024-05-14T10:01:09.458 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.007983593 2024-05-14T10:01:09.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.458 INFO:teuthology.orchestra.run.smithi029.stdout: "op_inflight": 0, 2024-05-14T10:01:09.458 INFO:teuthology.orchestra.run.smithi029.stdout: "oplen_avg": { 2024-05-14T10:01:09.458 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 6389, 2024-05-14T10:01:09.458 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 6390 2024-05-14T10:01:09.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.458 INFO:teuthology.orchestra.run.smithi029.stdout: "op": 6389, 2024-05-14T10:01:09.459 INFO:teuthology.orchestra.run.smithi029.stdout: "op_r": 6389, 2024-05-14T10:01:09.459 INFO:teuthology.orchestra.run.smithi029.stdout: "op_w": 0, 2024-05-14T10:01:09.459 INFO:teuthology.orchestra.run.smithi029.stdout: "op_rmw": 0, 2024-05-14T10:01:09.459 INFO:teuthology.orchestra.run.smithi029.stdout: "op_pg": 0, 2024-05-14T10:01:09.459 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_stat": 3418, 2024-05-14T10:01:09.459 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_create": 0, 2024-05-14T10:01:09.459 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_read": 2969, 2024-05-14T10:01:09.459 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_write": 0, 2024-05-14T10:01:09.459 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_writefull": 0, 2024-05-14T10:01:09.459 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_writesame": 0, 2024-05-14T10:01:09.459 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_append": 0, 2024-05-14T10:01:09.459 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_zero": 0, 2024-05-14T10:01:09.459 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_truncate": 0, 2024-05-14T10:01:09.459 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_delete": 0, 2024-05-14T10:01:09.460 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_mapext": 0, 2024-05-14T10:01:09.460 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_sparse_read": 0, 2024-05-14T10:01:09.460 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_clonerange": 0, 2024-05-14T10:01:09.460 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_getxattr": 1, 2024-05-14T10:01:09.460 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_setxattr": 0, 2024-05-14T10:01:09.460 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_cmpxattr": 0, 2024-05-14T10:01:09.460 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_rmxattr": 0, 2024-05-14T10:01:09.460 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_resetxattrs": 0, 2024-05-14T10:01:09.460 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_call": 0, 2024-05-14T10:01:09.460 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_watch": 0, 2024-05-14T10:01:09.460 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_notify": 0, 2024-05-14T10:01:09.460 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T10:01:09.460 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_pgls": 0, 2024-05-14T10:01:09.460 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_pgls_filter": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_other": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_active": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_send": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_resend": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_ping": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_active": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_send": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_resend": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_active": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_send": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_resend": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_active": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_send": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_resend": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "command_active": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "command_send": 0, 2024-05-14T10:01:09.461 INFO:teuthology.orchestra.run.smithi029.stdout: "command_resend": 0, 2024-05-14T10:01:09.462 INFO:teuthology.orchestra.run.smithi029.stdout: "map_epoch": 93, 2024-05-14T10:01:09.462 INFO:teuthology.orchestra.run.smithi029.stdout: "map_full": 0, 2024-05-14T10:01:09.462 INFO:teuthology.orchestra.run.smithi029.stdout: "map_inc": 5, 2024-05-14T10:01:09.462 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_sessions": 11, 2024-05-14T10:01:09.462 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_session_open": 12, 2024-05-14T10:01:09.462 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_session_close": 1, 2024-05-14T10:01:09.462 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_laggy": 0, 2024-05-14T10:01:09.462 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_wr": 0, 2024-05-14T10:01:09.462 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_rd": 2, 2024-05-14T10:01:09.462 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_del": 0 2024-05-14T10:01:09.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.462 INFO:teuthology.orchestra.run.smithi029.stdout: "oft": { 2024-05-14T10:01:09.462 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_objs": 0, 2024-05-14T10:01:09.462 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_kv_pairs": 0, 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_updates": 0, 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_removes": 0 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: "osd-slow-ops": { 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: "slow_ops_count": 0 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: "purge_queue": { 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_ops": 0, 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_ops_high_water": 0, 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing": 0, 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_high_water": 0, 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executed_ops": 0, 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executed": 0, 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_item_in_journal": 0 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.463 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 6938, 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 451311883, 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 6938, 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 6938, 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 451311883, 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:09.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-objecter_bytes": { 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 6389, 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 449395481, 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 6389, 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 449395481, 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.465 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.466 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:09.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.466 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-objecter_ops": { 2024-05-14T10:01:09.466 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:09.466 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 1024, 2024-05-14T10:01:09.466 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:09.466 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-05-14T10:01:09.466 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-05-14T10:01:09.466 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:09.466 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-05-14T10:01:09.466 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 6389, 2024-05-14T10:01:09.466 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 6389, 2024-05-14T10:01:09.466 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 6389, 2024-05-14T10:01:09.466 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 6389, 2024-05-14T10:01:09.466 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-write_buf_throttle": { 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 3758096384, 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 0, 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 0, 2024-05-14T10:01:09.467 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-write_buf_throttle-0x55a8173d88f0": { 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 3758096384, 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 0, 2024-05-14T10:01:09.468 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 0, 2024-05-14T10:01:09.469 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:09.469 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:09.469 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:09.469 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:09.469 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:09.469 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:09.469 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-05-14T10:01:10.247 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-14T10:01:10.247 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-14T10:01:10.247 DEBUG:tasks.check_counter:Getting stats from f 2024-05-14T10:01:10.247 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-14T10:01:10.504 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T10:01:10.888 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:10 smithi029 ceph-mon[36249]: pgmap v1213: 129 pgs: 129 active+clean; 501 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 3.4 MiB/s wr, 116 op/s 2024-05-14T10:01:11.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:10 smithi002 ceph-mon[31557]: pgmap v1213: 129 pgs: 129 active+clean; 501 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 3.4 MiB/s wr, 116 op/s 2024-05-14T10:01:11.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:10 smithi026 ceph-mon[36056]: pgmap v1213: 129 pgs: 129 active+clean; 501 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 3.4 MiB/s wr, 116 op/s 2024-05-14T10:01:13.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:12 smithi002 ceph-mon[31557]: pgmap v1214: 129 pgs: 129 active+clean; 500 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.6 MiB/s wr, 93 op/s 2024-05-14T10:01:13.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:12 smithi029 ceph-mon[36249]: pgmap v1214: 129 pgs: 129 active+clean; 500 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.6 MiB/s wr, 93 op/s 2024-05-14T10:01:13.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:12 smithi026 ceph-mon[36056]: pgmap v1214: 129 pgs: 129 active+clean; 500 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.6 MiB/s wr, 93 op/s 2024-05-14T10:01:14.032 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-05-14T10:01:14.032 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T10:01:14.032 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 1798, 2024-05-14T10:01:14.032 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 1747, 2024-05-14T10:01:14.033 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 155193930, 2024-05-14T10:01:14.033 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 701564, 2024-05-14T10:01:14.033 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 7, 2024-05-14T10:01:14.033 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 3, 2024-05-14T10:01:14.033 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.262507610, 2024-05-14T10:01:14.033 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.054109111, 2024-05-14T10:01:14.033 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.150519463, 2024-05-14T10:01:14.033 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.045354054, 2024-05-14T10:01:14.033 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:14.033 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 1747, 2024-05-14T10:01:14.033 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.078792880, 2024-05-14T10:01:14.033 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000045101 2024-05-14T10:01:14.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.033 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:14.034 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.034 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.034 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.034 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 810544, 2024-05-14T10:01:14.034 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 305648 2024-05-14T10:01:14.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.034 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T10:01:14.034 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 1934, 2024-05-14T10:01:14.034 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 2306, 2024-05-14T10:01:14.034 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 218596288, 2024-05-14T10:01:14.034 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 1423389, 2024-05-14T10:01:14.034 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 8, 2024-05-14T10:01:14.034 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 6, 2024-05-14T10:01:14.035 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.280899871, 2024-05-14T10:01:14.035 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.059150313, 2024-05-14T10:01:14.035 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.153621081, 2024-05-14T10:01:14.035 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.053013406, 2024-05-14T10:01:14.035 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:14.035 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 2304, 2024-05-14T10:01:14.035 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.106069532, 2024-05-14T10:01:14.035 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000046037 2024-05-14T10:01:14.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.035 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:14.035 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.035 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.035 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.035 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-14T10:01:14.036 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 809008 2024-05-14T10:01:14.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.036 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T10:01:14.036 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 3086, 2024-05-14T10:01:14.036 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 3088, 2024-05-14T10:01:14.036 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 168695717, 2024-05-14T10:01:14.036 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 981406, 2024-05-14T10:01:14.036 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 6, 2024-05-14T10:01:14.036 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 5, 2024-05-14T10:01:14.036 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.392442711, 2024-05-14T10:01:14.036 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.077136776, 2024-05-14T10:01:14.036 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.192504914, 2024-05-14T10:01:14.036 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.097811670, 2024-05-14T10:01:14.036 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:14.036 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 3086, 2024-05-14T10:01:14.037 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.147405505, 2024-05-14T10:01:14.037 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000047765 2024-05-14T10:01:14.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.037 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:14.037 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.037 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.037 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.037 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T10:01:14.037 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T10:01:14.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.037 INFO:teuthology.orchestra.run.smithi029.stdout: "cct": { 2024-05-14T10:01:14.037 INFO:teuthology.orchestra.run.smithi029.stdout: "total_workers": 1, 2024-05-14T10:01:14.037 INFO:teuthology.orchestra.run.smithi029.stdout: "unhealthy_workers": 0 2024-05-14T10:01:14.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.038 INFO:teuthology.orchestra.run.smithi029.stdout: "finisher-MDSRank": { 2024-05-14T10:01:14.038 INFO:teuthology.orchestra.run.smithi029.stdout: "queue_len": 0, 2024-05-14T10:01:14.038 INFO:teuthology.orchestra.run.smithi029.stdout: "complete_latency": { 2024-05-14T10:01:14.038 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 10750, 2024-05-14T10:01:14.038 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.494696244, 2024-05-14T10:01:14.038 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000046018 2024-05-14T10:01:14.038 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:14.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.038 INFO:teuthology.orchestra.run.smithi029.stdout: "finisher-PurgeQueue": { 2024-05-14T10:01:14.038 INFO:teuthology.orchestra.run.smithi029.stdout: "queue_len": 0, 2024-05-14T10:01:14.038 INFO:teuthology.orchestra.run.smithi029.stdout: "complete_latency": { 2024-05-14T10:01:14.038 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.038 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.038 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.038 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:14.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.039 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": { 2024-05-14T10:01:14.039 INFO:teuthology.orchestra.run.smithi029.stdout: "request": 0, 2024-05-14T10:01:14.039 INFO:teuthology.orchestra.run.smithi029.stdout: "reply": 0, 2024-05-14T10:01:14.039 INFO:teuthology.orchestra.run.smithi029.stdout: "reply_latency": { 2024-05-14T10:01:14.039 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.039 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.039 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.039 INFO:teuthology.orchestra.run.smithi029.stdout: "slow_reply": 0, 2024-05-14T10:01:14.039 INFO:teuthology.orchestra.run.smithi029.stdout: "forward": 0, 2024-05-14T10:01:14.039 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_fetch_complete": 0, 2024-05-14T10:01:14.039 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_fetch_keys": 0, 2024-05-14T10:01:14.039 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_commit": 0, 2024-05-14T10:01:14.039 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_split": 0, 2024-05-14T10:01:14.040 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_merge": 0, 2024-05-14T10:01:14.040 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes": 60296, 2024-05-14T10:01:14.040 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_top": 24552, 2024-05-14T10:01:14.040 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_bottom": 35744, 2024-05-14T10:01:14.040 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_pin_tail": 0, 2024-05-14T10:01:14.040 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_pinned": 25221, 2024-05-14T10:01:14.040 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_expired": 0, 2024-05-14T10:01:14.040 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_with_caps": 0, 2024-05-14T10:01:14.040 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": 0, 2024-05-14T10:01:14.040 INFO:teuthology.orchestra.run.smithi029.stdout: "subtrees": 1, 2024-05-14T10:01:14.040 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse": 0, 2024-05-14T10:01:14.040 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_hit": 0, 2024-05-14T10:01:14.040 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_forward": 0, 2024-05-14T10:01:14.040 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_discover": 0, 2024-05-14T10:01:14.040 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_dir_fetch": 0, 2024-05-14T10:01:14.041 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_remote_ino": 0, 2024-05-14T10:01:14.041 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_lock": 0, 2024-05-14T10:01:14.041 INFO:teuthology.orchestra.run.smithi029.stdout: "load_cent": 0, 2024-05-14T10:01:14.041 INFO:teuthology.orchestra.run.smithi029.stdout: "q": 0, 2024-05-14T10:01:14.041 INFO:teuthology.orchestra.run.smithi029.stdout: "exported": 0, 2024-05-14T10:01:14.041 INFO:teuthology.orchestra.run.smithi029.stdout: "exported_inodes": 0, 2024-05-14T10:01:14.041 INFO:teuthology.orchestra.run.smithi029.stdout: "imported": 0, 2024-05-14T10:01:14.041 INFO:teuthology.orchestra.run.smithi029.stdout: "imported_inodes": 0, 2024-05-14T10:01:14.041 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_dir_fetch": 0, 2024-05-14T10:01:14.041 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_backtrace_fetch": 0, 2024-05-14T10:01:14.041 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_peer_discover": 0, 2024-05-14T10:01:14.041 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rfiles": 0, 2024-05-14T10:01:14.041 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rbytes": 0, 2024-05-14T10:01:14.041 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rsnaps": 0, 2024-05-14T10:01:14.041 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T10:01:14.042 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_set_tag": 0, 2024-05-14T10:01:14.042 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T10:01:14.042 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_inotable_repaired": 0, 2024-05-14T10:01:14.042 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dir_inodes": 0, 2024-05-14T10:01:14.042 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T10:01:14.042 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T10:01:14.042 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_file_inodes": 0, 2024-05-14T10:01:14.042 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_inode_file_caps": 0, 2024-05-14T10:01:14.042 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_revoke": 0, 2024-05-14T10:01:14.042 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_grant": 0, 2024-05-14T10:01:14.042 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T10:01:14.042 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T10:01:14.042 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T10:01:14.042 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_caps": 0, 2024-05-14T10:01:14.042 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_caps_dirty": 0, 2024-05-14T10:01:14.043 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_cap_release": 0, 2024-05-14T10:01:14.043 INFO:teuthology.orchestra.run.smithi029.stdout: "process_request_cap_release": 0 2024-05-14T10:01:14.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.043 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache": { 2024-05-14T10:01:14.043 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_update": 0, 2024-05-14T10:01:14.043 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_update_receipt": 0, 2024-05-14T10:01:14.043 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_try_discover": 0, 2024-05-14T10:01:14.043 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_send_discover": 0, 2024-05-14T10:01:14.043 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_handle_discover": 0, 2024-05-14T10:01:14.043 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays": 0, 2024-05-14T10:01:14.043 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays_delayed": 0, 2024-05-14T10:01:14.043 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays_enqueuing": 0, 2024-05-14T10:01:14.043 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_created": 0, 2024-05-14T10:01:14.043 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_enqueued": 0, 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_reintegrated": 0, 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_migrated": 0, 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_processing": 0, 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_enqueued": 0, 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_prioritized": 0, 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_started": 0, 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_completed": 0, 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_quiesce_path": 0, 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_quiesce_inode": 0, 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_exportdir": 0, 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_flush": 0, 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_fragmentdir": 0, 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_fragstats": 0, 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_inodestats": 0 2024-05-14T10:01:14.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.045 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log": { 2024-05-14T10:01:14.045 INFO:teuthology.orchestra.run.smithi029.stdout: "evlrg": 0, 2024-05-14T10:01:14.045 INFO:teuthology.orchestra.run.smithi029.stdout: "evadd": 0, 2024-05-14T10:01:14.045 INFO:teuthology.orchestra.run.smithi029.stdout: "evex": 0, 2024-05-14T10:01:14.045 INFO:teuthology.orchestra.run.smithi029.stdout: "evtrm": 0, 2024-05-14T10:01:14.045 INFO:teuthology.orchestra.run.smithi029.stdout: "ev": 292609, 2024-05-14T10:01:14.045 INFO:teuthology.orchestra.run.smithi029.stdout: "evexg": 0, 2024-05-14T10:01:14.045 INFO:teuthology.orchestra.run.smithi029.stdout: "evexd": 0, 2024-05-14T10:01:14.045 INFO:teuthology.orchestra.run.smithi029.stdout: "segadd": 0, 2024-05-14T10:01:14.045 INFO:teuthology.orchestra.run.smithi029.stdout: "segex": 0, 2024-05-14T10:01:14.045 INFO:teuthology.orchestra.run.smithi029.stdout: "segtrm": 0, 2024-05-14T10:01:14.045 INFO:teuthology.orchestra.run.smithi029.stdout: "seg": 141, 2024-05-14T10:01:14.045 INFO:teuthology.orchestra.run.smithi029.stdout: "segmjr": 25, 2024-05-14T10:01:14.045 INFO:teuthology.orchestra.run.smithi029.stdout: "segexg": 0, 2024-05-14T10:01:14.045 INFO:teuthology.orchestra.run.smithi029.stdout: "segexd": 0, 2024-05-14T10:01:14.046 INFO:teuthology.orchestra.run.smithi029.stdout: "expos": 290794851, 2024-05-14T10:01:14.046 INFO:teuthology.orchestra.run.smithi029.stdout: "wrpos": 544134031, 2024-05-14T10:01:14.046 INFO:teuthology.orchestra.run.smithi029.stdout: "rdpos": 544131947, 2024-05-14T10:01:14.046 INFO:teuthology.orchestra.run.smithi029.stdout: "jlat": { 2024-05-14T10:01:14.046 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.046 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.046 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.046 INFO:teuthology.orchestra.run.smithi029.stdout: "replayed": 292609 2024-05-14T10:01:14.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.046 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_mem": { 2024-05-14T10:01:14.046 INFO:teuthology.orchestra.run.smithi029.stdout: "ino": 93, 2024-05-14T10:01:14.046 INFO:teuthology.orchestra.run.smithi029.stdout: "ino+": 60352, 2024-05-14T10:01:14.046 INFO:teuthology.orchestra.run.smithi029.stdout: "ino-": 60259, 2024-05-14T10:01:14.046 INFO:teuthology.orchestra.run.smithi029.stdout: "dir": 14, 2024-05-14T10:01:14.047 INFO:teuthology.orchestra.run.smithi029.stdout: "dir+": 162, 2024-05-14T10:01:14.047 INFO:teuthology.orchestra.run.smithi029.stdout: "dir-": 148, 2024-05-14T10:01:14.047 INFO:teuthology.orchestra.run.smithi029.stdout: "dn": 60296, 2024-05-14T10:01:14.047 INFO:teuthology.orchestra.run.smithi029.stdout: "dn+": 60936, 2024-05-14T10:01:14.047 INFO:teuthology.orchestra.run.smithi029.stdout: "dn-": 640, 2024-05-14T10:01:14.047 INFO:teuthology.orchestra.run.smithi029.stdout: "cap": 0, 2024-05-14T10:01:14.047 INFO:teuthology.orchestra.run.smithi029.stdout: "cap+": 0, 2024-05-14T10:01:14.047 INFO:teuthology.orchestra.run.smithi029.stdout: "cap-": 0, 2024-05-14T10:01:14.047 INFO:teuthology.orchestra.run.smithi029.stdout: "rss": 175620, 2024-05-14T10:01:14.047 INFO:teuthology.orchestra.run.smithi029.stdout: "heap": 215440 2024-05-14T10:01:14.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.047 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_server": { 2024-05-14T10:01:14.047 INFO:teuthology.orchestra.run.smithi029.stdout: "dispatch_client_request": 0, 2024-05-14T10:01:14.047 INFO:teuthology.orchestra.run.smithi029.stdout: "dispatch_server_request": 0, 2024-05-14T10:01:14.047 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_request": 0, 2024-05-14T10:01:14.048 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_session": 0, 2024-05-14T10:01:14.048 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_peer_request": 0, 2024-05-14T10:01:14.048 INFO:teuthology.orchestra.run.smithi029.stdout: "req_create_latency": { 2024-05-14T10:01:14.048 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.048 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.048 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.048 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getattr_latency": { 2024-05-14T10:01:14.048 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.048 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.048 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.048 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getfilelock_latency": { 2024-05-14T10:01:14.048 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.048 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: "req_link_latency": { 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookup_latency": { 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookuphash_latency": { 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.049 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.050 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupino_latency": { 2024-05-14T10:01:14.050 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.050 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.050 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.050 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupname_latency": { 2024-05-14T10:01:14.050 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.050 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.050 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.050 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupparent_latency": { 2024-05-14T10:01:14.050 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.050 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.050 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.051 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupsnap_latency": { 2024-05-14T10:01:14.051 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.051 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.051 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.051 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lssnap_latency": { 2024-05-14T10:01:14.051 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.051 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.051 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.052 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mkdir_latency": { 2024-05-14T10:01:14.052 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.052 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.052 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.052 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mknod_latency": { 2024-05-14T10:01:14.052 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.052 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.052 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.053 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mksnap_latency": { 2024-05-14T10:01:14.053 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.053 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.053 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.053 INFO:teuthology.orchestra.run.smithi029.stdout: "req_open_latency": { 2024-05-14T10:01:14.053 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.053 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.053 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.053 INFO:teuthology.orchestra.run.smithi029.stdout: "req_readdir_latency": { 2024-05-14T10:01:14.053 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.053 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.053 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.054 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rename_latency": { 2024-05-14T10:01:14.054 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.054 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.054 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.054 INFO:teuthology.orchestra.run.smithi029.stdout: "req_renamesnap_latency": { 2024-05-14T10:01:14.054 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.054 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.054 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.054 INFO:teuthology.orchestra.run.smithi029.stdout: "req_snapdiff_latency": { 2024-05-14T10:01:14.054 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.054 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.054 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmdir_latency": { 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmsnap_latency": { 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmxattr_latency": { 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.056 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setattr_latency": { 2024-05-14T10:01:14.056 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.056 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.056 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.056 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setdirlayout_latency": { 2024-05-14T10:01:14.056 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.056 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.056 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.056 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setfilelock_latency": { 2024-05-14T10:01:14.056 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.056 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.056 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.057 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setlayout_latency": { 2024-05-14T10:01:14.057 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.057 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.057 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.057 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setxattr_latency": { 2024-05-14T10:01:14.057 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.057 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.057 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.057 INFO:teuthology.orchestra.run.smithi029.stdout: "req_symlink_latency": { 2024-05-14T10:01:14.057 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.057 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.057 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.058 INFO:teuthology.orchestra.run.smithi029.stdout: "req_unlink_latency": { 2024-05-14T10:01:14.058 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.058 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.058 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.058 INFO:teuthology.orchestra.run.smithi029.stdout: "cap_revoke_eviction": 0, 2024-05-14T10:01:14.058 INFO:teuthology.orchestra.run.smithi029.stdout: "cap_acquisition_throttle": 0, 2024-05-14T10:01:14.058 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getvxattr_latency": { 2024-05-14T10:01:14.058 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.058 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.058 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.058 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:14.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.059 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_sessions": { 2024-05-14T10:01:14.059 INFO:teuthology.orchestra.run.smithi029.stdout: "session_count": 1, 2024-05-14T10:01:14.059 INFO:teuthology.orchestra.run.smithi029.stdout: "session_add": 1, 2024-05-14T10:01:14.059 INFO:teuthology.orchestra.run.smithi029.stdout: "session_remove": 0, 2024-05-14T10:01:14.059 INFO:teuthology.orchestra.run.smithi029.stdout: "sessions_open": 1, 2024-05-14T10:01:14.059 INFO:teuthology.orchestra.run.smithi029.stdout: "sessions_stale": 0, 2024-05-14T10:01:14.059 INFO:teuthology.orchestra.run.smithi029.stdout: "total_load": 0, 2024-05-14T10:01:14.059 INFO:teuthology.orchestra.run.smithi029.stdout: "average_load": 0, 2024-05-14T10:01:14.059 INFO:teuthology.orchestra.run.smithi029.stdout: "avg_session_uptime": 3083, 2024-05-14T10:01:14.059 INFO:teuthology.orchestra.run.smithi029.stdout: "mdthresh_evicted": 0 2024-05-14T10:01:14.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.059 INFO:teuthology.orchestra.run.smithi029.stdout: "mempool": { 2024-05-14T10:01:14.059 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_bytes": 0, 2024-05-14T10:01:14.059 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_items": 0, 2024-05-14T10:01:14.059 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T10:01:14.060 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_items": 0, 2024-05-14T10:01:14.060 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T10:01:14.060 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_items": 0, 2024-05-14T10:01:14.060 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T10:01:14.060 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T10:01:14.060 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T10:01:14.060 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T10:01:14.060 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T10:01:14.060 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_items": 0, 2024-05-14T10:01:14.060 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T10:01:14.060 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T10:01:14.060 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_bytes": 0, 2024-05-14T10:01:14.060 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_items": 0, 2024-05-14T10:01:14.060 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_bytes": 0, 2024-05-14T10:01:14.060 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_items": 0, 2024-05-14T10:01:14.061 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T10:01:14.061 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T10:01:14.061 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T10:01:14.061 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T10:01:14.061 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T10:01:14.061 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_items": 0, 2024-05-14T10:01:14.061 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_bytes": 0, 2024-05-14T10:01:14.061 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_items": 0, 2024-05-14T10:01:14.061 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T10:01:14.061 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T10:01:14.061 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_bytes": 0, 2024-05-14T10:01:14.061 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_items": 0, 2024-05-14T10:01:14.061 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_bytes": 0, 2024-05-14T10:01:14.061 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_items": 0, 2024-05-14T10:01:14.061 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T10:01:14.062 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_items": 0, 2024-05-14T10:01:14.062 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T10:01:14.062 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_items": 0, 2024-05-14T10:01:14.062 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_bytes": 246503, 2024-05-14T10:01:14.062 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_items": 211, 2024-05-14T10:01:14.062 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_bytes": 0, 2024-05-14T10:01:14.062 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_items": 0, 2024-05-14T10:01:14.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bytes": 0, 2024-05-14T10:01:14.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_items": 0, 2024-05-14T10:01:14.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_bytes": 0, 2024-05-14T10:01:14.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_items": 0, 2024-05-14T10:01:14.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_bytes": 0, 2024-05-14T10:01:14.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_items": 0, 2024-05-14T10:01:14.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_bytes": 9288, 2024-05-14T10:01:14.062 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_items": 180, 2024-05-14T10:01:14.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T10:01:14.063 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_items": 0, 2024-05-14T10:01:14.063 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_bytes": 0, 2024-05-14T10:01:14.063 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_items": 0, 2024-05-14T10:01:14.063 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_bytes": 42811912, 2024-05-14T10:01:14.063 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_items": 258244, 2024-05-14T10:01:14.063 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_bytes": 0, 2024-05-14T10:01:14.063 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_items": 0, 2024-05-14T10:01:14.063 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_bytes": 0, 2024-05-14T10:01:14.063 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_items": 0 2024-05-14T10:01:14.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.063 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter": { 2024-05-14T10:01:14.063 INFO:teuthology.orchestra.run.smithi029.stdout: "op_active": 0, 2024-05-14T10:01:14.063 INFO:teuthology.orchestra.run.smithi029.stdout: "op_laggy": 0, 2024-05-14T10:01:14.063 INFO:teuthology.orchestra.run.smithi029.stdout: "op_send": 6263, 2024-05-14T10:01:14.064 INFO:teuthology.orchestra.run.smithi029.stdout: "op_send_bytes": 21, 2024-05-14T10:01:14.064 INFO:teuthology.orchestra.run.smithi029.stdout: "op_resend": 3, 2024-05-14T10:01:14.064 INFO:teuthology.orchestra.run.smithi029.stdout: "op_reply": 6263, 2024-05-14T10:01:14.064 INFO:teuthology.orchestra.run.smithi029.stdout: "op_latency": { 2024-05-14T10:01:14.064 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 6263, 2024-05-14T10:01:14.064 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 51.454170136, 2024-05-14T10:01:14.064 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.008215578 2024-05-14T10:01:14.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.064 INFO:teuthology.orchestra.run.smithi029.stdout: "op_inflight": 0, 2024-05-14T10:01:14.064 INFO:teuthology.orchestra.run.smithi029.stdout: "oplen_avg": { 2024-05-14T10:01:14.064 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 6263, 2024-05-14T10:01:14.064 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 6264 2024-05-14T10:01:14.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.064 INFO:teuthology.orchestra.run.smithi029.stdout: "op": 6263, 2024-05-14T10:01:14.064 INFO:teuthology.orchestra.run.smithi029.stdout: "op_r": 6263, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "op_w": 0, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "op_rmw": 0, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "op_pg": 0, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_stat": 3246, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_create": 0, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_read": 3015, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_write": 0, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_writefull": 0, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_writesame": 0, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_append": 0, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_zero": 0, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_truncate": 0, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_delete": 0, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_mapext": 0, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_sparse_read": 0, 2024-05-14T10:01:14.065 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_clonerange": 0, 2024-05-14T10:01:14.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_getxattr": 1, 2024-05-14T10:01:14.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_setxattr": 0, 2024-05-14T10:01:14.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_cmpxattr": 0, 2024-05-14T10:01:14.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_rmxattr": 0, 2024-05-14T10:01:14.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_resetxattrs": 0, 2024-05-14T10:01:14.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_call": 0, 2024-05-14T10:01:14.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_watch": 0, 2024-05-14T10:01:14.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_notify": 0, 2024-05-14T10:01:14.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T10:01:14.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_pgls": 0, 2024-05-14T10:01:14.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_pgls_filter": 0, 2024-05-14T10:01:14.066 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_other": 0, 2024-05-14T10:01:14.066 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_active": 0, 2024-05-14T10:01:14.066 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_send": 0, 2024-05-14T10:01:14.066 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_resend": 0, 2024-05-14T10:01:14.067 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_ping": 0, 2024-05-14T10:01:14.067 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_active": 0, 2024-05-14T10:01:14.067 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_send": 0, 2024-05-14T10:01:14.067 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_resend": 0, 2024-05-14T10:01:14.067 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_active": 0, 2024-05-14T10:01:14.067 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_send": 0, 2024-05-14T10:01:14.067 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_resend": 0, 2024-05-14T10:01:14.067 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_active": 0, 2024-05-14T10:01:14.067 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_send": 0, 2024-05-14T10:01:14.067 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_resend": 0, 2024-05-14T10:01:14.067 INFO:teuthology.orchestra.run.smithi029.stdout: "command_active": 0, 2024-05-14T10:01:14.067 INFO:teuthology.orchestra.run.smithi029.stdout: "command_send": 0, 2024-05-14T10:01:14.067 INFO:teuthology.orchestra.run.smithi029.stdout: "command_resend": 0, 2024-05-14T10:01:14.067 INFO:teuthology.orchestra.run.smithi029.stdout: "map_epoch": 93, 2024-05-14T10:01:14.067 INFO:teuthology.orchestra.run.smithi029.stdout: "map_full": 0, 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: "map_inc": 5, 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_sessions": 11, 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_session_open": 11, 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_session_close": 0, 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_laggy": 0, 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_wr": 0, 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_rd": 2, 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_del": 0 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: "oft": { 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_objs": 0, 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_kv_pairs": 0, 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_updates": 0, 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_removes": 0 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.068 INFO:teuthology.orchestra.run.smithi029.stdout: "osd-slow-ops": { 2024-05-14T10:01:14.069 INFO:teuthology.orchestra.run.smithi029.stdout: "slow_ops_count": 0 2024-05-14T10:01:14.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.069 INFO:teuthology.orchestra.run.smithi029.stdout: "purge_queue": { 2024-05-14T10:01:14.069 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_ops": 0, 2024-05-14T10:01:14.069 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_ops_high_water": 0, 2024-05-14T10:01:14.069 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing": 0, 2024-05-14T10:01:14.069 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_high_water": 0, 2024-05-14T10:01:14.069 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executed_ops": 0, 2024-05-14T10:01:14.069 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executed": 0, 2024-05-14T10:01:14.069 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_item_in_journal": 0 2024-05-14T10:01:14.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.069 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T10:01:14.069 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:14.069 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2024-05-14T10:01:14.069 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:14.070 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 6814, 2024-05-14T10:01:14.070 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 541841645, 2024-05-14T10:01:14.070 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:14.070 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 6814, 2024-05-14T10:01:14.070 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-05-14T10:01:14.070 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-05-14T10:01:14.070 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 6814, 2024-05-14T10:01:14.070 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 541841645, 2024-05-14T10:01:14.070 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:14.070 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.070 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.070 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.070 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:14.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.070 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-objecter_bytes": { 2024-05-14T10:01:14.071 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:14.071 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2024-05-14T10:01:14.071 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:14.071 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-05-14T10:01:14.071 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-05-14T10:01:14.071 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:14.071 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-05-14T10:01:14.071 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 6263, 2024-05-14T10:01:14.071 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 539943527, 2024-05-14T10:01:14.071 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 6263, 2024-05-14T10:01:14.071 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 539943527, 2024-05-14T10:01:14.071 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:14.071 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.071 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.071 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.072 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:14.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.072 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-objecter_ops": { 2024-05-14T10:01:14.072 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:14.072 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 1024, 2024-05-14T10:01:14.072 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:14.072 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-05-14T10:01:14.072 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-05-14T10:01:14.072 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:14.072 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-05-14T10:01:14.072 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 6263, 2024-05-14T10:01:14.072 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 6263, 2024-05-14T10:01:14.072 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 6263, 2024-05-14T10:01:14.072 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 6263, 2024-05-14T10:01:14.072 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-write_buf_throttle": { 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 3758096384, 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-05-14T10:01:14.073 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 0, 2024-05-14T10:01:14.074 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 0, 2024-05-14T10:01:14.074 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:14.074 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.074 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.074 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.074 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:14.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:14.074 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-write_buf_throttle-0x55d7caeda410": { 2024-05-14T10:01:14.074 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:14.074 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 3758096384, 2024-05-14T10:01:14.074 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:14.074 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-05-14T10:01:14.074 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-05-14T10:01:14.074 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:14.074 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-05-14T10:01:14.075 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-05-14T10:01:14.075 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-05-14T10:01:14.075 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 0, 2024-05-14T10:01:14.075 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 0, 2024-05-14T10:01:14.075 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:14.075 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:14.075 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:14.075 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:14.075 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:14.075 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:14.075 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-05-14T10:01:15.091 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:14 smithi029 ceph-mon[36249]: pgmap v1215: 129 pgs: 129 active+clean; 477 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 2.5 MiB/s wr, 119 op/s 2024-05-14T10:01:15.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:14 smithi002 ceph-mon[31557]: pgmap v1215: 129 pgs: 129 active+clean; 477 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 2.5 MiB/s wr, 119 op/s 2024-05-14T10:01:15.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:14 smithi026 ceph-mon[36056]: pgmap v1215: 129 pgs: 129 active+clean; 477 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 2.5 MiB/s wr, 119 op/s 2024-05-14T10:01:15.626 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-14T10:01:15.626 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-14T10:01:15.626 DEBUG:tasks.check_counter:Getting stats from i 2024-05-14T10:01:15.626 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-14T10:01:15.883 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T10:01:17.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:16 smithi002 ceph-mon[31557]: pgmap v1216: 129 pgs: 129 active+clean; 477 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.7 MiB/s wr, 75 op/s 2024-05-14T10:01:17.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:16 smithi029 ceph-mon[36249]: pgmap v1216: 129 pgs: 129 active+clean; 477 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.7 MiB/s wr, 75 op/s 2024-05-14T10:01:17.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:16 smithi026 ceph-mon[36056]: pgmap v1216: 129 pgs: 129 active+clean; 477 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.7 MiB/s wr, 75 op/s 2024-05-14T10:01:18.571 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-05-14T10:01:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T10:01:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 550826, 2024-05-14T10:01:18.571 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 861936, 2024-05-14T10:01:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 163393401, 2024-05-14T10:01:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 722771127, 2024-05-14T10:01:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 19, 2024-05-14T10:01:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 6, 2024-05-14T10:01:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 41.037536222, 2024-05-14T10:01:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 15.714706193, 2024-05-14T10:01:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 20.104586000, 2024-05-14T10:01:18.572 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 8.701925068, 2024-05-14T10:01:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 861933, 2024-05-14T10:01:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 86.052371814, 2024-05-14T10:01:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000099836 2024-05-14T10:01:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 727179, 2024-05-14T10:01:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.199966020, 2024-05-14T10:01:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000274 2024-05-14T10:01:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 819472, 2024-05-14T10:01:18.573 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 311632 2024-05-14T10:01:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T10:01:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 110182, 2024-05-14T10:01:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 115481, 2024-05-14T10:01:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 27163352, 2024-05-14T10:01:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 207232370, 2024-05-14T10:01:18.574 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 10, 2024-05-14T10:01:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 6, 2024-05-14T10:01:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 15.679441601, 2024-05-14T10:01:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 3.609685578, 2024-05-14T10:01:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 3.567397109, 2024-05-14T10:01:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 8.364585128, 2024-05-14T10:01:18.575 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 115477, 2024-05-14T10:01:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 13.421764255, 2024-05-14T10:01:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000116228 2024-05-14T10:01:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:18.576 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 87698, 2024-05-14T10:01:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.022388212, 2024-05-14T10:01:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000255 2024-05-14T10:01:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-14T10:01:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 810512 2024-05-14T10:01:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T10:01:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 338013, 2024-05-14T10:01:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 327612, 2024-05-14T10:01:18.577 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 94057157, 2024-05-14T10:01:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 343171788, 2024-05-14T10:01:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 10, 2024-05-14T10:01:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 5, 2024-05-14T10:01:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 31.162099625, 2024-05-14T10:01:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 7.810465106, 2024-05-14T10:01:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 12.375469304, 2024-05-14T10:01:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 12.629914689, 2024-05-14T10:01:18.578 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 327615, 2024-05-14T10:01:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 21.512802244, 2024-05-14T10:01:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000065664 2024-05-14T10:01:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 487706, 2024-05-14T10:01:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.097451796, 2024-05-14T10:01:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000199 2024-05-14T10:01:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.579 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-14T10:01:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 448 2024-05-14T10:01:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "cct": { 2024-05-14T10:01:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "total_workers": 1, 2024-05-14T10:01:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "unhealthy_workers": 0 2024-05-14T10:01:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "finisher-MDSRank": { 2024-05-14T10:01:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "queue_len": 0, 2024-05-14T10:01:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "complete_latency": { 2024-05-14T10:01:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 140199, 2024-05-14T10:01:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 265.813622449, 2024-05-14T10:01:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.001895973 2024-05-14T10:01:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:18.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "finisher-PurgeQueue": { 2024-05-14T10:01:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "queue_len": 0, 2024-05-14T10:01:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "complete_latency": { 2024-05-14T10:01:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 108070, 2024-05-14T10:01:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 21.298917484, 2024-05-14T10:01:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000197084 2024-05-14T10:01:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": { 2024-05-14T10:01:18.581 INFO:teuthology.orchestra.run.smithi029.stdout: "request": 360370, 2024-05-14T10:01:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "reply": 344610, 2024-05-14T10:01:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "reply_latency": { 2024-05-14T10:01:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 344610, 2024-05-14T10:01:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 2120.389286786, 2024-05-14T10:01:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.006153011 2024-05-14T10:01:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "slow_reply": 0, 2024-05-14T10:01:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "forward": 15760, 2024-05-14T10:01:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_fetch_complete": 5, 2024-05-14T10:01:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_fetch_keys": 0, 2024-05-14T10:01:18.582 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_commit": 95, 2024-05-14T10:01:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_split": 9, 2024-05-14T10:01:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_merge": 10, 2024-05-14T10:01:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes": 916, 2024-05-14T10:01:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_top": 576, 2024-05-14T10:01:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_bottom": 250, 2024-05-14T10:01:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_pin_tail": 90, 2024-05-14T10:01:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_pinned": 93, 2024-05-14T10:01:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_expired": 427, 2024-05-14T10:01:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_with_caps": 0, 2024-05-14T10:01:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": 0, 2024-05-14T10:01:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "subtrees": 4, 2024-05-14T10:01:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse": 606155, 2024-05-14T10:01:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_hit": 314695, 2024-05-14T10:01:18.583 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_forward": 15757, 2024-05-14T10:01:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_discover": 38, 2024-05-14T10:01:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_dir_fetch": 1, 2024-05-14T10:01:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_remote_ino": 0, 2024-05-14T10:01:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_lock": 6213, 2024-05-14T10:01:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "load_cent": 1000, 2024-05-14T10:01:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "q": 1, 2024-05-14T10:01:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "exported": 11, 2024-05-14T10:01:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "exported_inodes": 190, 2024-05-14T10:01:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "imported": 11, 2024-05-14T10:01:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "imported_inodes": 564, 2024-05-14T10:01:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_dir_fetch": 0, 2024-05-14T10:01:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_backtrace_fetch": 16, 2024-05-14T10:01:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_peer_discover": 0, 2024-05-14T10:01:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rfiles": 4, 2024-05-14T10:01:18.584 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rbytes": 234, 2024-05-14T10:01:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rsnaps": 0, 2024-05-14T10:01:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_backtrace_fetch": 16, 2024-05-14T10:01:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_set_tag": 0, 2024-05-14T10:01:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T10:01:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_inotable_repaired": 0, 2024-05-14T10:01:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dir_inodes": 0, 2024-05-14T10:01:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T10:01:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T10:01:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_file_inodes": 16, 2024-05-14T10:01:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_inode_file_caps": 24, 2024-05-14T10:01:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_revoke": 23258, 2024-05-14T10:01:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_grant": 238621, 2024-05-14T10:01:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T10:01:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T10:01:18.585 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T10:01:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_caps": 97335, 2024-05-14T10:01:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_caps_dirty": 38792, 2024-05-14T10:01:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_cap_release": 18108, 2024-05-14T10:01:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "process_request_cap_release": 246039 2024-05-14T10:01:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache": { 2024-05-14T10:01:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_update": 87, 2024-05-14T10:01:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_update_receipt": 267, 2024-05-14T10:01:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_try_discover": 145, 2024-05-14T10:01:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_send_discover": 271, 2024-05-14T10:01:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_handle_discover": 191, 2024-05-14T10:01:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays": 3, 2024-05-14T10:01:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays_delayed": 0, 2024-05-14T10:01:18.586 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays_enqueuing": 0, 2024-05-14T10:01:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_created": 60229, 2024-05-14T10:01:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_enqueued": 60226, 2024-05-14T10:01:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_reintegrated": 0, 2024-05-14T10:01:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_migrated": 0, 2024-05-14T10:01:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_processing": 0, 2024-05-14T10:01:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_enqueued": 0, 2024-05-14T10:01:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_prioritized": 0, 2024-05-14T10:01:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_started": 0, 2024-05-14T10:01:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_completed": 0, 2024-05-14T10:01:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_quiesce_path": 0, 2024-05-14T10:01:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_quiesce_inode": 0, 2024-05-14T10:01:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T10:01:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_exportdir": 12, 2024-05-14T10:01:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_flush": 0, 2024-05-14T10:01:18.587 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_fragmentdir": 19, 2024-05-14T10:01:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_fragstats": 0, 2024-05-14T10:01:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_inodestats": 0 2024-05-14T10:01:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log": { 2024-05-14T10:01:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "evlrg": 0, 2024-05-14T10:01:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "evadd": 292687, 2024-05-14T10:01:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "evex": 183416, 2024-05-14T10:01:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "evtrm": 172061, 2024-05-14T10:01:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "ev": 120626, 2024-05-14T10:01:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "evexg": 0, 2024-05-14T10:01:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "evexd": 11355, 2024-05-14T10:01:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "segadd": 348, 2024-05-14T10:01:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "segex": 220, 2024-05-14T10:01:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "segtrm": 207, 2024-05-14T10:01:18.588 INFO:teuthology.orchestra.run.smithi029.stdout: "seg": 141, 2024-05-14T10:01:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "segmjr": 10, 2024-05-14T10:01:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "segexg": 0, 2024-05-14T10:01:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "segexd": 13, 2024-05-14T10:01:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "expos": 290794851, 2024-05-14T10:01:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "wrpos": 544181292, 2024-05-14T10:01:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "rdpos": 0, 2024-05-14T10:01:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "jlat": { 2024-05-14T10:01:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 80598, 2024-05-14T10:01:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 1161.521946339, 2024-05-14T10:01:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.014411299 2024-05-14T10:01:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "replayed": 0 2024-05-14T10:01:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.589 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_mem": { 2024-05-14T10:01:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "ino": 28, 2024-05-14T10:01:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "ino+": 60471, 2024-05-14T10:01:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "ino-": 60443, 2024-05-14T10:01:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "dir": 36, 2024-05-14T10:01:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "dir+": 334, 2024-05-14T10:01:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "dir-": 298, 2024-05-14T10:01:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "dn": 839, 2024-05-14T10:01:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "dn+": 111853, 2024-05-14T10:01:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "dn-": 111014, 2024-05-14T10:01:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "cap": 0, 2024-05-14T10:01:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "cap+": 65503, 2024-05-14T10:01:18.590 INFO:teuthology.orchestra.run.smithi029.stdout: "cap-": 65503, 2024-05-14T10:01:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "rss": 156208, 2024-05-14T10:01:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "heap": 223632 2024-05-14T10:01:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_server": { 2024-05-14T10:01:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "dispatch_client_request": 521549, 2024-05-14T10:01:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "dispatch_server_request": 7927, 2024-05-14T10:01:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_request": 360370, 2024-05-14T10:01:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_session": 7031, 2024-05-14T10:01:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_peer_request": 53922, 2024-05-14T10:01:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "req_create_latency": { 2024-05-14T10:01:18.591 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 59984, 2024-05-14T10:01:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 228.568340853, 2024-05-14T10:01:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.003810488 2024-05-14T10:01:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getattr_latency": { 2024-05-14T10:01:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 8627, 2024-05-14T10:01:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 327.883589644, 2024-05-14T10:01:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.038006675 2024-05-14T10:01:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getfilelock_latency": { 2024-05-14T10:01:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "req_link_latency": { 2024-05-14T10:01:18.592 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookup_latency": { 2024-05-14T10:01:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 111839, 2024-05-14T10:01:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 284.381946282, 2024-05-14T10:01:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.002542779 2024-05-14T10:01:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookuphash_latency": { 2024-05-14T10:01:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.593 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupino_latency": { 2024-05-14T10:01:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupname_latency": { 2024-05-14T10:01:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupparent_latency": { 2024-05-14T10:01:18.594 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupsnap_latency": { 2024-05-14T10:01:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lssnap_latency": { 2024-05-14T10:01:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.595 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mkdir_latency": { 2024-05-14T10:01:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mknod_latency": { 2024-05-14T10:01:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mksnap_latency": { 2024-05-14T10:01:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.596 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "req_open_latency": { 2024-05-14T10:01:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 1, 2024-05-14T10:01:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000852463, 2024-05-14T10:01:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000852463 2024-05-14T10:01:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "req_readdir_latency": { 2024-05-14T10:01:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 90359, 2024-05-14T10:01:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 1006.229415702, 2024-05-14T10:01:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.011135906 2024-05-14T10:01:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rename_latency": { 2024-05-14T10:01:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 11808, 2024-05-14T10:01:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 52.966389575, 2024-05-14T10:01:18.597 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.004485635 2024-05-14T10:01:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "req_renamesnap_latency": { 2024-05-14T10:01:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "req_snapdiff_latency": { 2024-05-14T10:01:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmdir_latency": { 2024-05-14T10:01:18.598 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 40, 2024-05-14T10:01:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.406483983, 2024-05-14T10:01:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.010162099 2024-05-14T10:01:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmsnap_latency": { 2024-05-14T10:01:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmxattr_latency": { 2024-05-14T10:01:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setattr_latency": { 2024-05-14T10:01:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 279, 2024-05-14T10:01:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 8.366709321, 2024-05-14T10:01:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.029988205 2024-05-14T10:01:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setdirlayout_latency": { 2024-05-14T10:01:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setfilelock_latency": { 2024-05-14T10:01:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 1484, 2024-05-14T10:01:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 4.499056226, 2024-05-14T10:01:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.003031709 2024-05-14T10:01:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setlayout_latency": { 2024-05-14T10:01:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setxattr_latency": { 2024-05-14T10:01:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.601 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "req_symlink_latency": { 2024-05-14T10:01:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "req_unlink_latency": { 2024-05-14T10:01:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 60189, 2024-05-14T10:01:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 207.086502737, 2024-05-14T10:01:18.602 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.003440603 2024-05-14T10:01:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "cap_revoke_eviction": 0, 2024-05-14T10:01:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "cap_acquisition_throttle": 0, 2024-05-14T10:01:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getvxattr_latency": { 2024-05-14T10:01:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_sessions": { 2024-05-14T10:01:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "session_count": 1, 2024-05-14T10:01:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "session_add": 1, 2024-05-14T10:01:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "session_remove": 0, 2024-05-14T10:01:18.603 INFO:teuthology.orchestra.run.smithi029.stdout: "sessions_open": 1, 2024-05-14T10:01:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "sessions_stale": 0, 2024-05-14T10:01:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "total_load": 25627, 2024-05-14T10:01:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "average_load": 25627, 2024-05-14T10:01:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "avg_session_uptime": 1475, 2024-05-14T10:01:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "mdthresh_evicted": 0 2024-05-14T10:01:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "mempool": { 2024-05-14T10:01:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_bytes": 0, 2024-05-14T10:01:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_items": 0, 2024-05-14T10:01:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T10:01:18.604 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_items": 0, 2024-05-14T10:01:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T10:01:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_items": 0, 2024-05-14T10:01:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T10:01:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T10:01:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T10:01:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T10:01:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T10:01:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_items": 0, 2024-05-14T10:01:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T10:01:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T10:01:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_bytes": 0, 2024-05-14T10:01:18.605 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_items": 0, 2024-05-14T10:01:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_bytes": 0, 2024-05-14T10:01:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_items": 0, 2024-05-14T10:01:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T10:01:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T10:01:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T10:01:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T10:01:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T10:01:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_items": 0, 2024-05-14T10:01:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_bytes": 0, 2024-05-14T10:01:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_items": 0, 2024-05-14T10:01:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T10:01:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T10:01:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_bytes": 0, 2024-05-14T10:01:18.606 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_items": 0, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_bytes": 0, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_items": 0, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_items": 0, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_items": 0, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_bytes": 908326, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_items": 352, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_bytes": 616, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_items": 7, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bytes": 0, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_items": 0, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_bytes": 0, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_items": 0, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_bytes": 0, 2024-05-14T10:01:18.607 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_items": 0, 2024-05-14T10:01:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_bytes": 9288, 2024-05-14T10:01:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_items": 180, 2024-05-14T10:01:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T10:01:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_items": 0, 2024-05-14T10:01:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_bytes": 0, 2024-05-14T10:01:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_items": 0, 2024-05-14T10:01:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_bytes": 1022948, 2024-05-14T10:01:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_items": 21486, 2024-05-14T10:01:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_bytes": 0, 2024-05-14T10:01:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_items": 0, 2024-05-14T10:01:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_bytes": 0, 2024-05-14T10:01:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_items": 0 2024-05-14T10:01:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter": { 2024-05-14T10:01:18.608 INFO:teuthology.orchestra.run.smithi029.stdout: "op_active": 0, 2024-05-14T10:01:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "op_laggy": 0, 2024-05-14T10:01:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "op_send": 207397, 2024-05-14T10:01:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "op_send_bytes": 553249976, 2024-05-14T10:01:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "op_resend": 28, 2024-05-14T10:01:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "op_reply": 207397, 2024-05-14T10:01:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "op_latency": { 2024-05-14T10:01:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 207397, 2024-05-14T10:01:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 3513.501338398, 2024-05-14T10:01:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.016940945 2024-05-14T10:01:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "op_inflight": 0, 2024-05-14T10:01:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "oplen_avg": { 2024-05-14T10:01:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 207397, 2024-05-14T10:01:18.609 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 209845 2024-05-14T10:01:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "op": 207397, 2024-05-14T10:01:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "op_r": 1864, 2024-05-14T10:01:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "op_w": 205533, 2024-05-14T10:01:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "op_rmw": 0, 2024-05-14T10:01:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "op_pg": 0, 2024-05-14T10:01:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_stat": 2, 2024-05-14T10:01:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_create": 21, 2024-05-14T10:01:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_read": 1843, 2024-05-14T10:01:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_write": 82446, 2024-05-14T10:01:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_writefull": 935, 2024-05-14T10:01:18.610 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_writesame": 0, 2024-05-14T10:01:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_append": 0, 2024-05-14T10:01:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_zero": 0, 2024-05-14T10:01:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_truncate": 9, 2024-05-14T10:01:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_delete": 120698, 2024-05-14T10:01:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_mapext": 0, 2024-05-14T10:01:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_sparse_read": 0, 2024-05-14T10:01:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_clonerange": 0, 2024-05-14T10:01:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_getxattr": 21, 2024-05-14T10:01:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_setxattr": 42, 2024-05-14T10:01:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_cmpxattr": 0, 2024-05-14T10:01:18.611 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_rmxattr": 0, 2024-05-14T10:01:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_resetxattrs": 0, 2024-05-14T10:01:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_call": 0, 2024-05-14T10:01:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_watch": 0, 2024-05-14T10:01:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_notify": 0, 2024-05-14T10:01:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T10:01:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_pgls": 0, 2024-05-14T10:01:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_pgls_filter": 0, 2024-05-14T10:01:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_other": 0, 2024-05-14T10:01:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_active": 0, 2024-05-14T10:01:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_send": 0, 2024-05-14T10:01:18.612 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_resend": 0, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_ping": 0, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_active": 0, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_send": 0, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_resend": 0, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_active": 0, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_send": 0, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_resend": 0, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_active": 0, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_send": 0, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_resend": 0, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "command_active": 0, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "command_send": 0, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "command_resend": 0, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "map_epoch": 93, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "map_full": 0, 2024-05-14T10:01:18.613 INFO:teuthology.orchestra.run.smithi029.stdout: "map_inc": 5, 2024-05-14T10:01:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_sessions": 12, 2024-05-14T10:01:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_session_open": 12, 2024-05-14T10:01:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_session_close": 0, 2024-05-14T10:01:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_laggy": 0, 2024-05-14T10:01:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_wr": 2796, 2024-05-14T10:01:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_rd": 10, 2024-05-14T10:01:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_del": 1022 2024-05-14T10:01:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "oft": { 2024-05-14T10:01:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_objs": 1, 2024-05-14T10:01:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_kv_pairs": 0, 2024-05-14T10:01:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_updates": 0, 2024-05-14T10:01:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_removes": 0 2024-05-14T10:01:18.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "osd-slow-ops": { 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "slow_ops_count": 0 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "purge_queue": { 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_ops": 0, 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_ops_high_water": 17, 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing": 0, 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_high_water": 16, 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executed_ops": 120415, 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executed": 60226, 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_item_in_journal": 0 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2024-05-14T10:01:18.615 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 999017, 2024-05-14T10:01:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 202597532, 2024-05-14T10:01:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 999017, 2024-05-14T10:01:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-05-14T10:01:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-05-14T10:01:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 999017, 2024-05-14T10:01:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 202597532, 2024-05-14T10:01:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.616 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-objecter_bytes": { 2024-05-14T10:01:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2024-05-14T10:01:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-05-14T10:01:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-05-14T10:01:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-05-14T10:01:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 207397, 2024-05-14T10:01:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 559814234, 2024-05-14T10:01:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 207397, 2024-05-14T10:01:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 559814234, 2024-05-14T10:01:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.617 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-objecter_ops": { 2024-05-14T10:01:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 1024, 2024-05-14T10:01:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2024-05-14T10:01:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2024-05-14T10:01:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2024-05-14T10:01:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 207397, 2024-05-14T10:01:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 207397, 2024-05-14T10:01:18.618 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 207397, 2024-05-14T10:01:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 207397, 2024-05-14T10:01:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-write_buf_throttle": { 2024-05-14T10:01:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 3758096384, 2024-05-14T10:01:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 60226, 2024-05-14T10:01:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 6564338, 2024-05-14T10:01:18.619 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 60226, 2024-05-14T10:01:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-05-14T10:01:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-05-14T10:01:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 1848, 2024-05-14T10:01:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 6564338, 2024-05-14T10:01:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-write_buf_throttle-0x55703b38c410": { 2024-05-14T10:01:18.620 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 3758096384, 2024-05-14T10:01:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 292687, 2024-05-14T10:01:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 539986988, 2024-05-14T10:01:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 292687, 2024-05-14T10:01:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-05-14T10:01:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-05-14T10:01:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 80598, 2024-05-14T10:01:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 539986988, 2024-05-14T10:01:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:18.621 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:18.622 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:18.622 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-05-14T10:01:19.065 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:18 smithi029 ceph-mon[36249]: pgmap v1217: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.7 MiB/s wr, 76 op/s 2024-05-14T10:01:19.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:18 smithi002 ceph-mon[31557]: pgmap v1217: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.7 MiB/s wr, 76 op/s 2024-05-14T10:01:19.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:18 smithi026 ceph-mon[36056]: pgmap v1217: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.7 MiB/s wr, 76 op/s 2024-05-14T10:01:19.514 INFO:tasks.check_counter:Daemon mds.i mds.exported=11 2024-05-14T10:01:19.515 INFO:tasks.check_counter:Daemon mds.i mds.imported=11 2024-05-14T10:01:19.515 DEBUG:tasks.check_counter:Getting stats from l 2024-05-14T10:01:19.515 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-14T10:01:19.769 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T10:01:21.115 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:20 smithi029 ceph-mon[36249]: pgmap v1218: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 934 KiB/s wr, 60 op/s 2024-05-14T10:01:21.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:20 smithi002 ceph-mon[31557]: pgmap v1218: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 934 KiB/s wr, 60 op/s 2024-05-14T10:01:21.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:20 smithi026 ceph-mon[36056]: pgmap v1218: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 934 KiB/s wr, 60 op/s 2024-05-14T10:01:21.785 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-05-14T10:01:21.785 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T10:01:21.786 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 499, 2024-05-14T10:01:21.786 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 496, 2024-05-14T10:01:21.786 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 594016, 2024-05-14T10:01:21.786 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 307328, 2024-05-14T10:01:21.786 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 4, 2024-05-14T10:01:21.786 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 0, 2024-05-14T10:01:21.786 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.068136678, 2024-05-14T10:01:21.786 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.025951327, 2024-05-14T10:01:21.786 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.025749822, 2024-05-14T10:01:21.786 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.011645535, 2024-05-14T10:01:21.786 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:21.786 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 495, 2024-05-14T10:01:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.025767172, 2024-05-14T10:01:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000052054 2024-05-14T10:01:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 594016, 2024-05-14T10:01:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 307328 2024-05-14T10:01:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T10:01:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 3, 2024-05-14T10:01:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 380, 2024-05-14T10:01:21.787 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 480, 2024-05-14T10:01:21.788 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 303168, 2024-05-14T10:01:21.788 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 5, 2024-05-14T10:01:21.788 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 2, 2024-05-14T10:01:21.788 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.020465774, 2024-05-14T10:01:21.788 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.015206529, 2024-05-14T10:01:21.788 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.001742932, 2024-05-14T10:01:21.789 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T10:01:21.789 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:21.789 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 378, 2024-05-14T10:01:21.789 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.017874755, 2024-05-14T10:01:21.789 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000047287 2024-05-14T10:01:21.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:21.789 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:21.789 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:21.789 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:21.789 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:21.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:21.789 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T10:01:21.789 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 303168 2024-05-14T10:01:21.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:21.790 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T10:01:21.790 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 4, 2024-05-14T10:01:21.790 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 4, 2024-05-14T10:01:21.790 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 2944, 2024-05-14T10:01:21.790 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 448, 2024-05-14T10:01:21.790 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 2, 2024-05-14T10:01:21.790 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 1, 2024-05-14T10:01:21.790 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.000693260, 2024-05-14T10:01:21.790 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.000044438, 2024-05-14T10:01:21.791 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.000623036, 2024-05-14T10:01:21.791 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T10:01:21.791 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T10:01:21.791 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 3, 2024-05-14T10:01:21.791 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000830724, 2024-05-14T10:01:21.791 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000276908 2024-05-14T10:01:21.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:21.792 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2024-05-14T10:01:21.792 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:21.792 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:21.792 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:21.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:21.792 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-14T10:01:21.792 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 448 2024-05-14T10:01:21.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:21.793 INFO:teuthology.orchestra.run.smithi029.stdout: "cct": { 2024-05-14T10:01:21.793 INFO:teuthology.orchestra.run.smithi029.stdout: "total_workers": 0, 2024-05-14T10:01:21.793 INFO:teuthology.orchestra.run.smithi029.stdout: "unhealthy_workers": 0 2024-05-14T10:01:21.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:21.793 INFO:teuthology.orchestra.run.smithi029.stdout: "mempool": { 2024-05-14T10:01:21.793 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_bytes": 0, 2024-05-14T10:01:21.793 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_items": 0, 2024-05-14T10:01:21.793 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T10:01:21.794 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_items": 0, 2024-05-14T10:01:21.794 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T10:01:21.794 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_items": 0, 2024-05-14T10:01:21.794 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T10:01:21.794 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T10:01:21.794 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T10:01:21.794 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T10:01:21.794 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T10:01:21.794 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_items": 0, 2024-05-14T10:01:21.794 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T10:01:21.794 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T10:01:21.795 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_bytes": 0, 2024-05-14T10:01:21.795 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_items": 0, 2024-05-14T10:01:21.795 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_bytes": 0, 2024-05-14T10:01:21.795 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_items": 0, 2024-05-14T10:01:21.795 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T10:01:21.795 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T10:01:21.796 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T10:01:21.796 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T10:01:21.796 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T10:01:21.796 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_items": 0, 2024-05-14T10:01:21.796 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_bytes": 0, 2024-05-14T10:01:21.796 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_items": 0, 2024-05-14T10:01:21.796 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T10:01:21.796 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T10:01:21.796 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_bytes": 0, 2024-05-14T10:01:21.797 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_items": 0, 2024-05-14T10:01:21.797 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_bytes": 0, 2024-05-14T10:01:21.797 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_items": 0, 2024-05-14T10:01:21.797 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T10:01:21.797 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_items": 0, 2024-05-14T10:01:21.797 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T10:01:21.797 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_items": 0, 2024-05-14T10:01:21.797 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_bytes": 148704, 2024-05-14T10:01:21.797 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_items": 29, 2024-05-14T10:01:21.797 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_bytes": 0, 2024-05-14T10:01:21.797 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_items": 0, 2024-05-14T10:01:21.797 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bytes": 0, 2024-05-14T10:01:21.797 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_items": 0, 2024-05-14T10:01:21.797 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_bytes": 0, 2024-05-14T10:01:21.797 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_items": 0, 2024-05-14T10:01:21.798 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_bytes": 0, 2024-05-14T10:01:21.798 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_items": 0, 2024-05-14T10:01:21.798 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_bytes": 0, 2024-05-14T10:01:21.798 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_items": 0, 2024-05-14T10:01:21.798 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T10:01:21.798 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_items": 0, 2024-05-14T10:01:21.798 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_bytes": 0, 2024-05-14T10:01:21.798 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_items": 0, 2024-05-14T10:01:21.798 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_bytes": 928, 2024-05-14T10:01:21.798 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_items": 2, 2024-05-14T10:01:21.798 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_bytes": 0, 2024-05-14T10:01:21.798 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_items": 0, 2024-05-14T10:01:21.798 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_bytes": 0, 2024-05-14T10:01:21.798 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_items": 0 2024-05-14T10:01:21.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 502, 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 523441, 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 502, 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 502, 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 523441, 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2024-05-14T10:01:21.799 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:21.800 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-05-14T10:01:21.800 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-05-14T10:01:22.464 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-14T10:01:22.464 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-14T10:01:22.464 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-14T10:01:22.476 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-14T10:01:22.476 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T10:01:22.476 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T10:01:22.501 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T10:01:22.501 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T10:01:22.561 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-14T10:01:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T10:01:22.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:22 smithi002 ceph-mon[31557]: pgmap v1219: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 148 KiB/s wr, 36 op/s 2024-05-14T10:01:23.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:22 smithi029 ceph-mon[36249]: pgmap v1219: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 148 KiB/s wr, 36 op/s 2024-05-14T10:01:23.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:22 smithi026 ceph-mon[36056]: pgmap v1219: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 148 KiB/s wr, 36 op/s 2024-05-14T10:01:24.912 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:24 smithi002 ceph-mon[31557]: pgmap v1220: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 82 KiB/s wr, 35 op/s 2024-05-14T10:01:25.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.034+0000 7f0073c17700 1 -- 172.21.15.2:0/4225643561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f006c1034c0 msgr2=0x7f006c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.034+0000 7f0073c17700 1 --2- 172.21.15.2:0/4225643561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f006c1034c0 0x7f006c1038d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f00680099c0 tx=0x7f0068009cd0 comp rx=0 tx=0).stop 2024-05-14T10:01:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.034+0000 7f0073c17700 1 -- 172.21.15.2:0/4225643561 shutdown_connections 2024-05-14T10:01:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.034+0000 7f0073c17700 1 --2- 172.21.15.2:0/4225643561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c108740 0x7f006c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.034+0000 7f0073c17700 1 --2- 172.21.15.2:0/4225643561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f006c1034c0 0x7f006c1038d0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.034+0000 7f0073c17700 1 --2- 172.21.15.2:0/4225643561 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f006c107df0 0x7f006c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.034+0000 7f0073c17700 1 -- 172.21.15.2:0/4225643561 >> 172.21.15.2:0/4225643561 conn(0x7f006c0fedf0 msgr2=0x7f006c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.034+0000 7f0073c17700 1 -- 172.21.15.2:0/4225643561 shutdown_connections 2024-05-14T10:01:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.034+0000 7f0073c17700 1 -- 172.21.15.2:0/4225643561 wait complete. 2024-05-14T10:01:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.035+0000 7f0073c17700 1 Processor -- start 2024-05-14T10:01:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.035+0000 7f0073c17700 1 -- start start 2024-05-14T10:01:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.035+0000 7f0073c17700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1034c0 0x7f006c19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.035+0000 7f0073c17700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f006c107df0 0x7f006c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.035+0000 7f0073c17700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f006c108740 0x7f006c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.035+0000 7f0073c17700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f006c113370 con 0x7f006c108740 2024-05-14T10:01:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.035+0000 7f0073c17700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1131f0 con 0x7f006c1034c0 2024-05-14T10:01:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.035+0000 7f0073c17700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1134f0 con 0x7f006c107df0 2024-05-14T10:01:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.035+0000 7f00721b4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f006c108740 0x7f006c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.035+0000 7f00721b4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f006c108740 0x7f006c19f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40786/0 (socket says 172.21.15.2:40786) 2024-05-14T10:01:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.035+0000 7f00721b4700 1 -- 172.21.15.2:0/2794063899 learned_addr learned my addr 172.21.15.2:0/2794063899 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:01:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.035+0000 7f00711b2700 1 --2- 172.21.15.2:0/2794063899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f006c107df0 0x7f006c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.035+0000 7f00719b3700 1 --2- 172.21.15.2:0/2794063899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1034c0 0x7f006c19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.036+0000 7f00721b4700 1 -- 172.21.15.2:0/2794063899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f006c107df0 msgr2=0x7f006c19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.036+0000 7f00721b4700 1 --2- 172.21.15.2:0/2794063899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f006c107df0 0x7f006c19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.036+0000 7f00721b4700 1 -- 172.21.15.2:0/2794063899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1034c0 msgr2=0x7f006c19ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:25.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.036+0000 7f00721b4700 1 --2- 172.21.15.2:0/2794063899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1034c0 0x7f006c19ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:25.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.036+0000 7f00721b4700 1 -- 172.21.15.2:0/2794063899 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c19fcf0 con 0x7f006c108740 2024-05-14T10:01:25.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.036+0000 7f00721b4700 1 --2- 172.21.15.2:0/2794063899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f006c108740 0x7f006c19f650 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0064007ee0 tx=0x7f006400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:25.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.036+0000 7f0062ffd700 1 -- 172.21.15.2:0/2794063899 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064004bb0 con 0x7f006c108740 2024-05-14T10:01:25.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.036+0000 7f0062ffd700 1 -- 172.21.15.2:0/2794063899 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0064004d50 con 0x7f006c108740 2024-05-14T10:01:25.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.036+0000 7f0073c17700 1 -- 172.21.15.2:0/2794063899 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006c19ffe0 con 0x7f006c108740 2024-05-14T10:01:25.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.036+0000 7f0062ffd700 1 -- 172.21.15.2:0/2794063899 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064005920 con 0x7f006c108740 2024-05-14T10:01:25.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.036+0000 7f0073c17700 1 -- 172.21.15.2:0/2794063899 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006c1a7900 con 0x7f006c108740 2024-05-14T10:01:25.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.037+0000 7f0073c17700 1 -- 172.21.15.2:0/2794063899 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f006c0512c0 con 0x7f006c108740 2024-05-14T10:01:25.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.039+0000 7f0062ffd700 1 -- 172.21.15.2:0/2794063899 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f006401a020 con 0x7f006c108740 2024-05-14T10:01:25.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.040+0000 7f0062ffd700 1 --2- 172.21.15.2:0/2794063899 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f00580b2900 0x7f00580b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:25.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.040+0000 7f00719b3700 1 --2- 172.21.15.2:0/2794063899 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f00580b2900 0x7f00580b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:25.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.040+0000 7f0062ffd700 1 -- 172.21.15.2:0/2794063899 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f00640e2270 con 0x7f006c108740 2024-05-14T10:01:25.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.040+0000 7f00719b3700 1 --2- 172.21.15.2:0/2794063899 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f00580b2900 0x7f00580b4e10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0068009990 tx=0x7f0068000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:25.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.044+0000 7f0062ffd700 1 -- 172.21.15.2:0/2794063899 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0064010070 con 0x7f006c108740 2024-05-14T10:01:25.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:24 smithi029 ceph-mon[36249]: pgmap v1220: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 82 KiB/s wr, 35 op/s 2024-05-14T10:01:25.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:24 smithi026 ceph-mon[36056]: pgmap v1220: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 82 KiB/s wr, 35 op/s 2024-05-14T10:01:25.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.276+0000 7f0073c17700 1 -- 172.21.15.2:0/2794063899 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f006c1a7b30 con 0x7f006c108740 2024-05-14T10:01:25.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.277+0000 7f0062ffd700 1 -- 172.21.15.2:0/2794063899 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) ==== 81+0+581 (secure 0 0 0) 0x7f00640a9b70 con 0x7f006c108740 2024-05-14T10:01:25.277 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/4051922409 2024-05-15T09:23:28.940670+0000 2024-05-14T10:01:25.277 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2353790181 2024-05-15T09:23:28.940670+0000 2024-05-14T10:01:25.277 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/4268171474 2024-05-15T09:23:28.940670+0000 2024-05-14T10:01:25.277 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/1423742578 2024-05-15T09:23:28.940670+0000 2024-05-14T10:01:25.277 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/969850441 2024-05-15T09:23:02.498446+0000 2024-05-14T10:01:25.277 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3252991268 2024-05-15T09:23:02.498446+0000 2024-05-14T10:01:25.278 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3593583257 2024-05-15T09:23:02.498446+0000 2024-05-14T10:01:25.278 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3593583257 2024-05-15T09:23:02.498446+0000 2024-05-14T10:01:25.278 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3789067476 2024-05-15T09:23:02.498446+0000 2024-05-14T10:01:25.278 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/1423742578 2024-05-15T09:23:28.940670+0000 2024-05-14T10:01:25.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.279+0000 7f0073c17700 1 -- 172.21.15.2:0/2794063899 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f00580b2900 msgr2=0x7f00580b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:25.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.279+0000 7f0073c17700 1 --2- 172.21.15.2:0/2794063899 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f00580b2900 0x7f00580b4e10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0068009990 tx=0x7f0068000bc0 comp rx=0 tx=0).stop 2024-05-14T10:01:25.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.279+0000 7f0073c17700 1 -- 172.21.15.2:0/2794063899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f006c108740 msgr2=0x7f006c19f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:25.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.279+0000 7f0073c17700 1 --2- 172.21.15.2:0/2794063899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f006c108740 0x7f006c19f650 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0064007ee0 tx=0x7f006400d3b0 comp rx=0 tx=0).stop 2024-05-14T10:01:25.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.279+0000 7f0073c17700 1 -- 172.21.15.2:0/2794063899 shutdown_connections 2024-05-14T10:01:25.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.279+0000 7f0073c17700 1 --2- 172.21.15.2:0/2794063899 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f00580b2900 0x7f00580b4e10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:25.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.279+0000 7f0073c17700 1 --2- 172.21.15.2:0/2794063899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c1034c0 0x7f006c19ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:25.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.279+0000 7f0073c17700 1 --2- 172.21.15.2:0/2794063899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f006c108740 0x7f006c19f650 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:25.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.279+0000 7f0073c17700 1 --2- 172.21.15.2:0/2794063899 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f006c107df0 0x7f006c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:25.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.279+0000 7f0073c17700 1 -- 172.21.15.2:0/2794063899 >> 172.21.15.2:0/2794063899 conn(0x7f006c0fedf0 msgr2=0x7f006c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:25.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.279+0000 7f0073c17700 1 -- 172.21.15.2:0/2794063899 shutdown_connections 2024-05-14T10:01:25.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:25.280+0000 7f0073c17700 1 -- 172.21.15.2:0/2794063899 wait complete. 2024-05-14T10:01:25.281 INFO:teuthology.orchestra.run.smithi002.stderr:listed 10 entries 2024-05-14T10:01:25.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:25 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2794063899' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T10:01:26.079 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-14T10:01:26.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T10:01:26.080 DEBUG:teuthology.orchestra.run.smithi002:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-14T10:01:26.171 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T10:01:26.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T10:01:26.172 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T10:01:26.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:25 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2794063899' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T10:01:26.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:25 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2794063899' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T10:01:26.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T10:01:26.214 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-05-14T10:01:26.298 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T10:01:26.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T10:01:26.298 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T10:01:26.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T10:01:26.408 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2024-05-14T10:01:26.515 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-14T10:01:26.515 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T10:01:26.515 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T10:01:26.553 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19896 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/d3845f32443d93d840c9e7c075cbc2e63890eeeeb826535b6b7a1b59f11a8af2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/FD2JATCBUFIMFF4WIXZAGWWUJR:/var/lib/containers/storage/overlay/l/62D6AQM4HOF2HZMDR3MQCENXPQ,upperdir=/var/lib/containers/storage/overlay/d3845f32443d93d840c9e7c075cbc2e63890eeeeb826535b6b7a1b59f11a8af2/diff,workdir=/var/lib/containers/storage/overlay/d3845f32443d93d840c9e7c075cbc2e63890eeeeb826535b6b7a1b59f11a8af2/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f3564f39eb0ccb4334754840b560fc084ad97e3f0c7182efb8832e04738ba2d2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/FD2JATCBUFIMFF4WIXZAGWWUJR:/var/lib/containers/storage/overlay/l/62D6AQM4HOF2HZMDR3MQCENXPQ,upperdir=/var/lib/containers/storage/overlay/f3564f39eb0ccb4334754840b560fc084ad97e3f0c7182efb8832e04738ba2d2/diff,workdir=/var/lib/containers/storage/overlay/f3564f39eb0ccb4334754840b560fc084ad97e3f0c7182efb8832e04738ba2d2/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/26ddc5c72de0dfdfe0f6afff12d2986882ee8cdbc6e9ce04d13ef047b9ba0af2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/FD2JATCBUFIMFF4WIXZAGWWUJR:/var/lib/containers/storage/overlay/l/62D6AQM4HOF2HZMDR3MQCENXPQ,upperdir=/var/lib/containers/storage/overlay/26ddc5c72de0dfdfe0f6afff12d2986882ee8cdbc6e9ce04d13ef047b9ba0af2/diff,workdir=/var/lib/containers/storage/overlay/26ddc5c72de0dfdfe0f6afff12d2986882ee8cdbc6e9ce04d13ef047b9ba0af2/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8d9f499420fc82e8c9c434b1804b50a6f7caedc103ac2963de8ee990936dd096/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/FD2JATCBUFIMFF4WIXZAGWWUJR:/var/lib/containers/storage/overlay/l/62D6AQM4HOF2HZMDR3MQCENXPQ,upperdir=/var/lib/containers/storage/overlay/8d9f499420fc82e8c9c434b1804b50a6f7caedc103ac2963de8ee990936dd096/diff,workdir=/var/lib/containers/storage/overlay/8d9f499420fc82e8c9c434b1804b50a6f7caedc103ac2963de8ee990936dd096/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/865d157079ded1f0608cfd392a71f6a394c9eb4c66fed870bf299ee0f6e7d305/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/FD2JATCBUFIMFF4WIXZAGWWUJR:/var/lib/containers/storage/overlay/l/62D6AQM4HOF2HZMDR3MQCENXPQ,upperdir=/var/lib/containers/storage/overlay/865d157079ded1f0608cfd392a71f6a394c9eb4c66fed870bf299ee0f6e7d305/diff,workdir=/var/lib/containers/storage/overlay/865d157079ded1f0608cfd392a71f6a394c9eb4c66fed870bf299ee0f6e7d305/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/658bd59d389641a37b5b84b36e0c1fd8b0d1ac7548f97dd369ce0a41d05852d5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/FD2JATCBUFIMFF4WIXZAGWWUJR:/var/lib/containers/storage/overlay/l/62D6AQM4HOF2HZMDR3MQCENXPQ,upperdir=/var/lib/containers/storage/overlay/658bd59d389641a37b5b84b36e0c1fd8b0d1ac7548f97dd369ce0a41d05852d5/diff,workdir=/var/lib/containers/storage/overlay/658bd59d389641a37b5b84b36e0c1fd8b0d1ac7548f97dd369ce0a41d05852d5/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/16eb213df9a547367cc4be26f2718248d777e59fe9354ce45ac9a0db1ba8b26d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/FD2JATCBUFIMFF4WIXZAGWWUJR:/var/lib/containers/storage/overlay/l/62D6AQM4HOF2HZMDR3MQCENXPQ,upperdir=/var/lib/containers/storage/overlay/16eb213df9a547367cc4be26f2718248d777e59fe9354ce45ac9a0db1ba8b26d/diff,workdir=/var/lib/containers/storage/overlay/16eb213df9a547367cc4be26f2718248d777e59fe9354ce45ac9a0db1ba8b26d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0e5f1c3b30c22a7789aa02867831843eccf489fdf642c3ef838106c1f4f279f2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/FD2JATCBUFIMFF4WIXZAGWWUJR:/var/lib/containers/storage/overlay/l/62D6AQM4HOF2HZMDR3MQCENXPQ,upperdir=/var/lib/containers/storage/overlay/0e5f1c3b30c22a7789aa02867831843eccf489fdf642c3ef838106c1f4f279f2/diff,workdir=/var/lib/containers/storage/overlay/0e5f1c3b30c22a7789aa02867831843eccf489fdf642c3ef838106c1f4f279f2/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/494885cfbc4bd3a8d050c7e5ebeaea673b35440fb700ab8bc6e8cdd428f89ad2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/FD2JATCBUFIMFF4WIXZAGWWUJR:/var/lib/containers/storage/overlay/l/62D6AQM4HOF2HZMDR3MQCENXPQ,upperdir=/var/lib/containers/storage/overlay/494885cfbc4bd3a8d050c7e5ebeaea673b35440fb700ab8bc6e8cdd428f89ad2/diff,workdir=/var/lib/containers/storage/overlay/494885cfbc4bd3a8d050c7e5ebeaea673b35440fb700ab8bc6e8cdd428f89ad2/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3c39ff43924d545676eccd20bf8eade336b0bbcb5c8e578ce3a9ac99d4278084/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/FD2JATCBUFIMFF4WIXZAGWWUJR:/var/lib/containers/storage/overlay/l/62D6AQM4HOF2HZMDR3MQCENXPQ,upperdir=/var/lib/containers/storage/overlay/3c39ff43924d545676eccd20bf8eade336b0bbcb5c8e578ce3a9ac99d4278084/diff,workdir=/var/lib/containers/storage/overlay/3c39ff43924d545676eccd20bf8eade336b0bbcb5c8e578ce3a9ac99d4278084/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 2024-05-14T10:01:26.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T10:01:26.553 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-05-14T10:01:26.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T10:01:26.634 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2024-05-14T10:01:26.751 INFO:teuthology.orchestra.run.smithi002.stderr:Cannot find device "ceph-brx" 2024-05-14T10:01:26.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T10:01:26.753 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-14T10:01:26.766 INFO:tasks.cephadm:Teardown begin 2024-05-14T10:01:26.767 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-14T10:01:26.767 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T10:01:27.020 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T10:01:27.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:26 smithi002 ceph-mon[31557]: pgmap v1221: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-05-14T10:01:27.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:26 smithi029 ceph-mon[36249]: pgmap v1221: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-05-14T10:01:27.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:26 smithi026 ceph-mon[36056]: pgmap v1221: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-05-14T10:01:28.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.961+0000 7f392eb9d700 1 -- 172.21.15.2:0/1075551503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3928108740 msgr2=0x7f39281100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:28.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.961+0000 7f392eb9d700 1 --2- 172.21.15.2:0/1075551503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3928108740 0x7f39281100a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f39200099c0 tx=0x7f3920009cd0 comp rx=0 tx=0).stop 2024-05-14T10:01:28.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.962+0000 7f392eb9d700 1 -- 172.21.15.2:0/1075551503 shutdown_connections 2024-05-14T10:01:28.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.962+0000 7f392eb9d700 1 --2- 172.21.15.2:0/1075551503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39281034c0 0x7f39281038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:28.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.962+0000 7f392eb9d700 1 --2- 172.21.15.2:0/1075551503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3928108740 0x7f39281100a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:28.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.962+0000 7f392eb9d700 1 --2- 172.21.15.2:0/1075551503 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3928107df0 0x7f3928108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:28.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.962+0000 7f392eb9d700 1 -- 172.21.15.2:0/1075551503 >> 172.21.15.2:0/1075551503 conn(0x7f39280fedf0 msgr2=0x7f3928101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:28.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.962+0000 7f392eb9d700 1 -- 172.21.15.2:0/1075551503 shutdown_connections 2024-05-14T10:01:28.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.962+0000 7f392eb9d700 1 -- 172.21.15.2:0/1075551503 wait complete. 2024-05-14T10:01:28.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.962+0000 7f392eb9d700 1 Processor -- start 2024-05-14T10:01:28.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.962+0000 7f392eb9d700 1 -- start start 2024-05-14T10:01:28.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.963+0000 7f392eb9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39281034c0 0x7f392819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:28.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.963+0000 7f392eb9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3928107df0 0x7f392819c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:28.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.963+0000 7f392eb9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3928108740 0x7f392819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:28.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.963+0000 7f392eb9d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f39281133c0 con 0x7f3928108740 2024-05-14T10:01:28.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.963+0000 7f392eb9d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3928113240 con 0x7f3928107df0 2024-05-14T10:01:28.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.963+0000 7f392eb9d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f3928113540 con 0x7f39281034c0 2024-05-14T10:01:28.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.963+0000 7f392d13a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3928108740 0x7f392819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:28.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.963+0000 7f392d13a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3928108740 0x7f392819c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51138/0 (socket says 172.21.15.2:51138) 2024-05-14T10:01:28.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.963+0000 7f392d13a700 1 -- 172.21.15.2:0/1395883061 learned_addr learned my addr 172.21.15.2:0/1395883061 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:01:28.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.963+0000 7f3927fff700 1 --2- 172.21.15.2:0/1395883061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3928107df0 0x7f392819c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:28.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.963+0000 7f392c939700 1 --2- 172.21.15.2:0/1395883061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39281034c0 0x7f392819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:28.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.963+0000 7f3927fff700 1 -- 172.21.15.2:0/1395883061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39281034c0 msgr2=0x7f392819bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:28.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.963+0000 7f3927fff700 1 --2- 172.21.15.2:0/1395883061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39281034c0 0x7f392819bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:28.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.964+0000 7f3927fff700 1 -- 172.21.15.2:0/1395883061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3928108740 msgr2=0x7f392819c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:28.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.964+0000 7f3927fff700 1 --2- 172.21.15.2:0/1395883061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3928108740 0x7f392819c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:28.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.964+0000 7f3927fff700 1 -- 172.21.15.2:0/1395883061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39281a6fd0 con 0x7f3928107df0 2024-05-14T10:01:28.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.964+0000 7f392d13a700 1 --2- 172.21.15.2:0/1395883061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3928108740 0x7f392819c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T10:01:28.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.964+0000 7f3927fff700 1 --2- 172.21.15.2:0/1395883061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3928107df0 0x7f392819c1c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3910007e90 tx=0x7f391000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:28.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.964+0000 7f3925ffb700 1 -- 172.21.15.2:0/1395883061 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910004db0 con 0x7f3928107df0 2024-05-14T10:01:28.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.964+0000 7f392eb9d700 1 -- 172.21.15.2:0/1395883061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39281a72c0 con 0x7f3928107df0 2024-05-14T10:01:28.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.964+0000 7f3925ffb700 1 -- 172.21.15.2:0/1395883061 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3910003a00 con 0x7f3928107df0 2024-05-14T10:01:28.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.964+0000 7f3925ffb700 1 -- 172.21.15.2:0/1395883061 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910004590 con 0x7f3928107df0 2024-05-14T10:01:28.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.964+0000 7f392eb9d700 1 -- 172.21.15.2:0/1395883061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39281a7850 con 0x7f3928107df0 2024-05-14T10:01:28.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.965+0000 7f392eb9d700 1 -- 172.21.15.2:0/1395883061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39280512c0 con 0x7f3928107df0 2024-05-14T10:01:28.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.967+0000 7f3925ffb700 1 -- 172.21.15.2:0/1395883061 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f391001c060 con 0x7f3928107df0 2024-05-14T10:01:28.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.967+0000 7f3925ffb700 1 --2- 172.21.15.2:0/1395883061 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f39140b2880 0x7f39140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:28.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.968+0000 7f392c939700 1 --2- 172.21.15.2:0/1395883061 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f39140b2880 0x7f39140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:28.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.968+0000 7f3925ffb700 1 -- 172.21.15.2:0/1395883061 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f391001a070 con 0x7f3928107df0 2024-05-14T10:01:28.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.968+0000 7f392c939700 1 --2- 172.21.15.2:0/1395883061 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f39140b2880 0x7f39140b4d90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3918009c60 tx=0x7f3918009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:28.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:28.972+0000 7f3925ffb700 1 -- 172.21.15.2:0/1395883061 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3910023070 con 0x7f3928107df0 2024-05-14T10:01:29.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:28 smithi002 ceph-mon[31557]: pgmap v1222: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-05-14T10:01:29.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:28 smithi029 ceph-mon[36249]: pgmap v1222: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-05-14T10:01:29.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:28 smithi026 ceph-mon[36056]: pgmap v1222: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-05-14T10:01:29.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:29.244+0000 7f392eb9d700 1 -- 172.21.15.2:0/1395883061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3928075fe0 con 0x7f3928107df0 2024-05-14T10:01:29.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:29.245+0000 7f3925ffb700 1 -- 172.21.15.2:0/1395883061 <== mon.2 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+11846 (secure 0 0 0) 0x7f3910010070 con 0x7f3928107df0 2024-05-14T10:01:29.246 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T10:01:29.246 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T10:00:41.489255+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":103,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26995":{"gid":26995,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/786173089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":786173089},{"type":"v1","addr":"172.21.15.26:6835","nonce":786173089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24497,24524,24487]},"id":1}]} 2024-05-14T10:01:29.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:29.247+0000 7f392eb9d700 1 -- 172.21.15.2:0/1395883061 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f39140b2880 msgr2=0x7f39140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:29.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:29.247+0000 7f392eb9d700 1 --2- 172.21.15.2:0/1395883061 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f39140b2880 0x7f39140b4d90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3918009c60 tx=0x7f3918009260 comp rx=0 tx=0).stop 2024-05-14T10:01:29.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:29.247+0000 7f392eb9d700 1 -- 172.21.15.2:0/1395883061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3928107df0 msgr2=0x7f392819c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:29.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:29.247+0000 7f392eb9d700 1 --2- 172.21.15.2:0/1395883061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3928107df0 0x7f392819c1c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3910007e90 tx=0x7f391000d3b0 comp rx=0 tx=0).stop 2024-05-14T10:01:29.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:29.247+0000 7f392eb9d700 1 -- 172.21.15.2:0/1395883061 shutdown_connections 2024-05-14T10:01:29.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:29.247+0000 7f392eb9d700 1 --2- 172.21.15.2:0/1395883061 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f39140b2880 0x7f39140b4d90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:29.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:29.247+0000 7f392eb9d700 1 --2- 172.21.15.2:0/1395883061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3928108740 0x7f392819c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:29.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:29.247+0000 7f392eb9d700 1 --2- 172.21.15.2:0/1395883061 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f39281034c0 0x7f392819bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:29.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:29.247+0000 7f392eb9d700 1 --2- 172.21.15.2:0/1395883061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3928107df0 0x7f392819c1c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:29.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:29.247+0000 7f392eb9d700 1 -- 172.21.15.2:0/1395883061 >> 172.21.15.2:0/1395883061 conn(0x7f39280fedf0 msgr2=0x7f3928107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:29.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:29.247+0000 7f392eb9d700 1 -- 172.21.15.2:0/1395883061 shutdown_connections 2024-05-14T10:01:29.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:29.247+0000 7f392eb9d700 1 -- 172.21.15.2:0/1395883061 wait complete. 2024-05-14T10:01:29.249 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 71 2024-05-14T10:01:30.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:29 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1395883061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T10:01:30.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:29 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1395883061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T10:01:30.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:29 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1395883061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T10:01:30.217 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T10:01:30.470 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T10:01:30.843 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:30 smithi002 ceph-mon[31557]: pgmap v1223: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-14T10:01:31.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:30 smithi029 ceph-mon[36249]: pgmap v1223: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-14T10:01:31.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:30 smithi026 ceph-mon[36056]: pgmap v1223: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-14T10:01:31.915 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:31 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T10:01:32.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.171+0000 7f54a0afa700 1 -- 172.21.15.2:0/2703953653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f549c108760 msgr2=0x7f549c1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:32.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.171+0000 7f54a0afa700 1 --2- 172.21.15.2:0/2703953653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f549c108760 0x7f549c1100c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f54940099c0 tx=0x7f5494009cd0 comp rx=0 tx=0).stop 2024-05-14T10:01:32.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.171+0000 7f54a0afa700 1 -- 172.21.15.2:0/2703953653 shutdown_connections 2024-05-14T10:01:32.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.171+0000 7f54a0afa700 1 --2- 172.21.15.2:0/2703953653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f549c108760 0x7f549c1100c0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:32.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.171+0000 7f54a0afa700 1 --2- 172.21.15.2:0/2703953653 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f549c1034e0 0x7f549c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:32.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.171+0000 7f54a0afa700 1 --2- 172.21.15.2:0/2703953653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549c107e10 0x7f549c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:32.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.171+0000 7f54a0afa700 1 -- 172.21.15.2:0/2703953653 >> 172.21.15.2:0/2703953653 conn(0x7f549c0fef30 msgr2=0x7f549c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:32.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.171+0000 7f54a0afa700 1 -- 172.21.15.2:0/2703953653 shutdown_connections 2024-05-14T10:01:32.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.171+0000 7f54a0afa700 1 -- 172.21.15.2:0/2703953653 wait complete. 2024-05-14T10:01:32.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.172+0000 7f54a0afa700 1 Processor -- start 2024-05-14T10:01:32.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.172+0000 7f54a0afa700 1 -- start start 2024-05-14T10:01:32.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.172+0000 7f54a0afa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549c1034e0 0x7f549c19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:32.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.172+0000 7f54a0afa700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f549c107e10 0x7f549c19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:32.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.172+0000 7f54a0afa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f549c108760 0x7f549c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:32.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.172+0000 7f54a0afa700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f549c113390 con 0x7f549c108760 2024-05-14T10:01:32.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.172+0000 7f54a0afa700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f549c113210 con 0x7f549c1034e0 2024-05-14T10:01:32.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.172+0000 7f54a0afa700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7f549c113510 con 0x7f549c107e10 2024-05-14T10:01:32.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.172+0000 7f549ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f549c108760 0x7f549c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:32.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.172+0000 7f549ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f549c108760 0x7f549c19f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51152/0 (socket says 172.21.15.2:51152) 2024-05-14T10:01:32.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.172+0000 7f549ad9d700 1 -- 172.21.15.2:0/4050722852 learned_addr learned my addr 172.21.15.2:0/4050722852 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:01:32.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.172+0000 7f5499d9b700 1 --2- 172.21.15.2:0/4050722852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f549c107e10 0x7f549c19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:32.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.172+0000 7f549a59c700 1 --2- 172.21.15.2:0/4050722852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549c1034e0 0x7f549c19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:32.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.173+0000 7f549ad9d700 1 -- 172.21.15.2:0/4050722852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f549c107e10 msgr2=0x7f549c19b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:32.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.173+0000 7f549ad9d700 1 --2- 172.21.15.2:0/4050722852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f549c107e10 0x7f549c19b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:32.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.173+0000 7f549ad9d700 1 -- 172.21.15.2:0/4050722852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549c1034e0 msgr2=0x7f549c19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:32.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.173+0000 7f549ad9d700 1 --2- 172.21.15.2:0/4050722852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549c1034e0 0x7f549c19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:32.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.173+0000 7f549ad9d700 1 -- 172.21.15.2:0/4050722852 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549c19fcd0 con 0x7f549c108760 2024-05-14T10:01:32.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.173+0000 7f5499d9b700 1 --2- 172.21.15.2:0/4050722852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f549c107e10 0x7f549c19b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T10:01:32.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.173+0000 7f549ad9d700 1 --2- 172.21.15.2:0/4050722852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f549c108760 0x7f549c19f6e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5494009990 tx=0x7f54940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:32.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.173+0000 7f548b7fe700 1 -- 172.21.15.2:0/4050722852 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494027070 con 0x7f549c108760 2024-05-14T10:01:32.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.173+0000 7f548b7fe700 1 -- 172.21.15.2:0/4050722852 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5494003fa0 con 0x7f549c108760 2024-05-14T10:01:32.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.173+0000 7f54a0afa700 1 -- 172.21.15.2:0/4050722852 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549c19ff60 con 0x7f549c108760 2024-05-14T10:01:32.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.173+0000 7f548b7fe700 1 -- 172.21.15.2:0/4050722852 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494022d10 con 0x7f549c108760 2024-05-14T10:01:32.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.173+0000 7f54a0afa700 1 -- 172.21.15.2:0/4050722852 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549c1a0440 con 0x7f549c108760 2024-05-14T10:01:32.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.174+0000 7f54a0afa700 1 -- 172.21.15.2:0/4050722852 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f549c0512c0 con 0x7f549c108760 2024-05-14T10:01:32.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.176+0000 7f548b7fe700 1 -- 172.21.15.2:0/4050722852 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f549402a070 con 0x7f549c108760 2024-05-14T10:01:32.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.177+0000 7f548b7fe700 1 --2- 172.21.15.2:0/4050722852 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f54840b6c90 0x7f54840b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:32.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.177+0000 7f549a59c700 1 --2- 172.21.15.2:0/4050722852 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f54840b6c90 0x7f54840b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:32.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.177+0000 7f548b7fe700 1 -- 172.21.15.2:0/4050722852 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7f54940e3910 con 0x7f549c108760 2024-05-14T10:01:32.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.178+0000 7f549a59c700 1 --2- 172.21.15.2:0/4050722852 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f54840b6c90 0x7f54840b91a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f548c0060b0 tx=0x7f548c00c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:32.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.181+0000 7f548b7fe700 1 -- 172.21.15.2:0/4050722852 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5494031080 con 0x7f549c108760 2024-05-14T10:01:32.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:31 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T10:01:32.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:31 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T10:01:32.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.450+0000 7f54a0afa700 1 -- 172.21.15.2:0/4050722852 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f549c02e5f0 con 0x7f549c108760 2024-05-14T10:01:32.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.451+0000 7f548b7fe700 1 -- 172.21.15.2:0/4050722852 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+11846 (secure 0 0 0) 0x7f5494013070 con 0x7f549c108760 2024-05-14T10:01:32.453 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T10:01:32.453 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/3590687009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3590687009},{"type":"v1","addr":"172.21.15.2:6835","nonce":3590687009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2302731082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2302731082},{"type":"v1","addr":"172.21.15.2:6837","nonce":2302731082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2887812543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2887812543},{"type":"v1","addr":"172.21.15.2:6839","nonce":2887812543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/4114057120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":4114057120},{"type":"v1","addr":"172.21.15.2:6841","nonce":4114057120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/2153165231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":2153165231},{"type":"v1","addr":"172.21.15.29:6833","nonce":2153165231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/39434912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":39434912},{"type":"v1","addr":"172.21.15.26:6833","nonce":39434912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T09:30:58.644918+0000","modified":"2024-05-14T10:00:41.489255+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6835/3647517494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":3647517494},{"type":"v1","addr":"172.21.15.29:6835","nonce":3647517494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.29:6837/1333834114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1333834114},{"type":"v1","addr":"172.21.15.29:6837","nonce":1333834114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.29:6839/3625063120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":3625063120},{"type":"v1","addr":"172.21.15.29:6839","nonce":3625063120}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":103,"addr":"172.21.15.26:6837/3301773578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3301773578},{"type":"v1","addr":"172.21.15.26:6837","nonce":3301773578}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6839/903985766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":903985766},{"type":"v1","addr":"172.21.15.26:6839","nonce":903985766}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26995":{"gid":26995,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/786173089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":786173089},{"type":"v1","addr":"172.21.15.26:6835","nonce":786173089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24497,24524,24487]},"id":1}]} 2024-05-14T10:01:32.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.454+0000 7f54a0afa700 1 -- 172.21.15.2:0/4050722852 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f54840b6c90 msgr2=0x7f54840b91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:32.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.454+0000 7f54a0afa700 1 --2- 172.21.15.2:0/4050722852 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f54840b6c90 0x7f54840b91a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f548c0060b0 tx=0x7f548c00c040 comp rx=0 tx=0).stop 2024-05-14T10:01:32.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.454+0000 7f54a0afa700 1 -- 172.21.15.2:0/4050722852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f549c108760 msgr2=0x7f549c19f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:32.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.454+0000 7f54a0afa700 1 --2- 172.21.15.2:0/4050722852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f549c108760 0x7f549c19f6e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5494009990 tx=0x7f54940047b0 comp rx=0 tx=0).stop 2024-05-14T10:01:32.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.454+0000 7f54a0afa700 1 -- 172.21.15.2:0/4050722852 shutdown_connections 2024-05-14T10:01:32.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.454+0000 7f54a0afa700 1 --2- 172.21.15.2:0/4050722852 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7f54840b6c90 0x7f54840b91a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:32.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.454+0000 7f54a0afa700 1 --2- 172.21.15.2:0/4050722852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549c1034e0 0x7f549c19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:32.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.454+0000 7f54a0afa700 1 --2- 172.21.15.2:0/4050722852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f549c108760 0x7f549c19f6e0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:32.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.454+0000 7f54a0afa700 1 --2- 172.21.15.2:0/4050722852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f549c107e10 0x7f549c19b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:32.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.454+0000 7f54a0afa700 1 -- 172.21.15.2:0/4050722852 >> 172.21.15.2:0/4050722852 conn(0x7f549c0fef30 msgr2=0x7f549c06c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:32.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.454+0000 7f54a0afa700 1 -- 172.21.15.2:0/4050722852 shutdown_connections 2024-05-14T10:01:32.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:32.454+0000 7f54a0afa700 1 -- 172.21.15.2:0/4050722852 wait complete. 2024-05-14T10:01:32.457 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 71 2024-05-14T10:01:33.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:32 smithi002 ceph-mon[31557]: pgmap v1224: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T10:01:33.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T10:01:33.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T10:01:33.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:32 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:01:33.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:32 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/4050722852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T10:01:33.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:32 smithi029 ceph-mon[36249]: pgmap v1224: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T10:01:33.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T10:01:33.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T10:01:33.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:32 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:01:33.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:32 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/4050722852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T10:01:33.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:32 smithi026 ceph-mon[36056]: pgmap v1224: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T10:01:33.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T10:01:33.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T10:01:33.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:32 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' 2024-05-14T10:01:33.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:32 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/4050722852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T10:01:33.294 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T10:01:33.549 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T10:01:34.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:34 smithi002 ceph-mon[31557]: pgmap v1225: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T10:01:35.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:34 smithi029 ceph-mon[36249]: pgmap v1225: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T10:01:35.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:34 smithi026 ceph-mon[36056]: pgmap v1225: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T10:01:35.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.209+0000 7ff8a38dc700 1 -- 172.21.15.2:0/3365412575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff89c108f60 msgr2=0x7ff89c109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:35.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.209+0000 7ff8a38dc700 1 --2- 172.21.15.2:0/3365412575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff89c108f60 0x7ff89c109370 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009990 tx=0x7ff88c009ca0 comp rx=0 tx=0).stop 2024-05-14T10:01:35.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.209+0000 7ff8a38dc700 1 -- 172.21.15.2:0/3365412575 shutdown_connections 2024-05-14T10:01:35.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.209+0000 7ff8a38dc700 1 --2- 172.21.15.2:0/3365412575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89c1098b0 0x7ff89c1111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:35.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.209+0000 7ff8a38dc700 1 --2- 172.21.15.2:0/3365412575 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff89c102b40 0x7ff89c102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:35.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.209+0000 7ff8a38dc700 1 --2- 172.21.15.2:0/3365412575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff89c108f60 0x7ff89c109370 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:35.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.209+0000 7ff8a38dc700 1 -- 172.21.15.2:0/3365412575 >> 172.21.15.2:0/3365412575 conn(0x7ff89c0fe520 msgr2=0x7ff89c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:35.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.209+0000 7ff8a38dc700 1 -- 172.21.15.2:0/3365412575 shutdown_connections 2024-05-14T10:01:35.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.209+0000 7ff8a38dc700 1 -- 172.21.15.2:0/3365412575 wait complete. 2024-05-14T10:01:35.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.210+0000 7ff8a38dc700 1 Processor -- start 2024-05-14T10:01:35.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.210+0000 7ff8a38dc700 1 -- start start 2024-05-14T10:01:35.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.210+0000 7ff8a38dc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89c102b40 0x7ff89c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:35.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.210+0000 7ff8a38dc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff89c108f60 0x7ff89c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:35.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.210+0000 7ff8a38dc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff89c1098b0 0x7ff89c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:35.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.210+0000 7ff8a38dc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c114490 con 0x7ff89c1098b0 2024-05-14T10:01:35.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.210+0000 7ff8a38dc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c114310 con 0x7ff89c102b40 2024-05-14T10:01:35.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.210+0000 7ff8a38dc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c114610 con 0x7ff89c108f60 2024-05-14T10:01:35.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.210+0000 7ff8a0e77700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff89c108f60 0x7ff89c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:35.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.210+0000 7ff8a0e77700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff89c108f60 0x7ff89c19b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.2:47314/0 (socket says 172.21.15.2:47314) 2024-05-14T10:01:35.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.210+0000 7ff8a0e77700 1 -- 172.21.15.2:0/3689401808 learned_addr learned my addr 172.21.15.2:0/3689401808 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:01:35.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.210+0000 7ff8a1e79700 1 --2- 172.21.15.2:0/3689401808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff89c1098b0 0x7ff89c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:35.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.210+0000 7ff8a1678700 1 --2- 172.21.15.2:0/3689401808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89c102b40 0x7ff89c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:35.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.211+0000 7ff8a0e77700 1 -- 172.21.15.2:0/3689401808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89c102b40 msgr2=0x7ff89c19ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:35.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.211+0000 7ff8a0e77700 1 --2- 172.21.15.2:0/3689401808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89c102b40 0x7ff89c19ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:35.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.211+0000 7ff8a0e77700 1 -- 172.21.15.2:0/3689401808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff89c1098b0 msgr2=0x7ff89c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:35.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.211+0000 7ff8a0e77700 1 --2- 172.21.15.2:0/3689401808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff89c1098b0 0x7ff89c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:35.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.211+0000 7ff8a0e77700 1 -- 172.21.15.2:0/3689401808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89c19fd10 con 0x7ff89c108f60 2024-05-14T10:01:35.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.211+0000 7ff8a0e77700 1 --2- 172.21.15.2:0/3689401808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff89c108f60 0x7ff89c19b3e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff88c006010 tx=0x7ff88c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:35.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.211+0000 7ff8927fc700 1 -- 172.21.15.2:0/3689401808 <== mon.1 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c027070 con 0x7ff89c108f60 2024-05-14T10:01:35.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.211+0000 7ff8a38dc700 1 -- 172.21.15.2:0/3689401808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89c19ffa0 con 0x7ff89c108f60 2024-05-14T10:01:35.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.211+0000 7ff8927fc700 1 -- 172.21.15.2:0/3689401808 <== mon.1 v2:172.21.15.29:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff88c022470 con 0x7ff89c108f60 2024-05-14T10:01:35.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.211+0000 7ff8927fc700 1 -- 172.21.15.2:0/3689401808 <== mon.1 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c017650 con 0x7ff89c108f60 2024-05-14T10:01:35.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.211+0000 7ff8a38dc700 1 -- 172.21.15.2:0/3689401808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89c1a7920 con 0x7ff89c108f60 2024-05-14T10:01:35.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.212+0000 7ff8a38dc700 1 -- 172.21.15.2:0/3689401808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff89c0512c0 con 0x7ff89c108f60 2024-05-14T10:01:35.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.214+0000 7ff8927fc700 1 -- 172.21.15.2:0/3689401808 <== mon.1 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff88c02a070 con 0x7ff89c108f60 2024-05-14T10:01:35.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.215+0000 7ff8927fc700 1 --2- 172.21.15.2:0/3689401808 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff8880b2880 0x7ff8880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:35.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.215+0000 7ff8927fc700 1 -- 172.21.15.2:0/3689401808 <== mon.1 v2:172.21.15.29:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7ff88c0e3e50 con 0x7ff89c108f60 2024-05-14T10:01:35.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.215+0000 7ff8a1678700 1 --2- 172.21.15.2:0/3689401808 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff8880b2880 0x7ff8880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:35.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.215+0000 7ff8a1678700 1 --2- 172.21.15.2:0/3689401808 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff8880b2880 0x7ff8880b4d90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff89c06bc10 tx=0x7ff898009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:35.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.219+0000 7ff8927fc700 1 -- 172.21.15.2:0/3689401808 <== mon.1 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff88c031070 con 0x7ff89c108f60 2024-05-14T10:01:35.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.435+0000 7ff8a38dc700 1 -- 172.21.15.2:0/3689401808 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff89c02e5f0 con 0x7ff89c108f60 2024-05-14T10:01:35.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.437+0000 7ff8927fc700 1 -- 172.21.15.2:0/3689401808 <== mon.1 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24740 (secure 0 0 0) 0x7ff88c013070 con 0x7ff89c108f60 2024-05-14T10:01:35.438 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T10:01:35.438 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":93,"fsid":"50d5aebe-11d3-11ef-bc98-c7b262605968","created":"2024-05-14T09:22:34.937047+0000","modified":"2024-05-14T09:55:42.615872+0000","last_up_change":"2024-05-14T09:29:55.157569+0000","last_in_change":"2024-05-14T09:29:34.922562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T09:26:03.046454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T09:30:51.814350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T09:30:55.500000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T09:31:06.099087+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"10d34289-414e-4b1b-bc7c-ba1dd24c7297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6803","nonce":2622184634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6805","nonce":2622184634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6809","nonce":2622184634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2622184634},{"type":"v1","addr":"172.21.15.2:6807","nonce":2622184634}]},"public_addr":"172.21.15.2:6803/2622184634","cluster_addr":"172.21.15.2:6805/2622184634","heartbeat_back_addr":"172.21.15.2:6809/2622184634","heartbeat_front_addr":"172.21.15.2:6807/2622184634","state":["exists","up"]},{"osd":1,"uuid":"bc4aefc2-706e-4faf-9042-64b87ad097b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6801","nonce":3084747226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6803","nonce":3084747226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6807","nonce":3084747226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3084747226},{"type":"v1","addr":"172.21.15.26:6805","nonce":3084747226}]},"public_addr":"172.21.15.26:6801/3084747226","cluster_addr":"172.21.15.26:6803/3084747226","heartbeat_back_addr":"172.21.15.26:6807/3084747226","heartbeat_front_addr":"172.21.15.26:6805/3084747226","state":["exists","up"]},{"osd":2,"uuid":"8c5bb589-56e2-469f-9a6f-6035b3ddf4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6801","nonce":814048086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6803","nonce":814048086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6807","nonce":814048086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":814048086},{"type":"v1","addr":"172.21.15.29:6805","nonce":814048086}]},"public_addr":"172.21.15.29:6801/814048086","cluster_addr":"172.21.15.29:6803/814048086","heartbeat_back_addr":"172.21.15.29:6807/814048086","heartbeat_front_addr":"172.21.15.29:6805/814048086","state":["exists","up"]},{"osd":3,"uuid":"8fbfc078-85e7-4dc2-acda-5e7b0ab927e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6811","nonce":879639451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6813","nonce":879639451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6817","nonce":879639451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":879639451},{"type":"v1","addr":"172.21.15.2:6815","nonce":879639451}]},"public_addr":"172.21.15.2:6811/879639451","cluster_addr":"172.21.15.2:6813/879639451","heartbeat_back_addr":"172.21.15.2:6817/879639451","heartbeat_front_addr":"172.21.15.2:6815/879639451","state":["exists","up"]},{"osd":4,"uuid":"f2515448-e9c1-48a2-b270-1e5e781a2fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6809","nonce":457100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6811","nonce":457100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6815","nonce":457100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":457100752},{"type":"v1","addr":"172.21.15.26:6813","nonce":457100752}]},"public_addr":"172.21.15.26:6809/457100752","cluster_addr":"172.21.15.26:6811/457100752","heartbeat_back_addr":"172.21.15.26:6815/457100752","heartbeat_front_addr":"172.21.15.26:6813/457100752","state":["exists","up"]},{"osd":5,"uuid":"20030857-a1e6-4b0a-b186-1cfa44f2e4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6809","nonce":1554045093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6811","nonce":1554045093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6815","nonce":1554045093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":1554045093},{"type":"v1","addr":"172.21.15.29:6813","nonce":1554045093}]},"public_addr":"172.21.15.29:6809/1554045093","cluster_addr":"172.21.15.29:6811/1554045093","heartbeat_back_addr":"172.21.15.29:6815/1554045093","heartbeat_front_addr":"172.21.15.29:6813/1554045093","state":["exists","up"]},{"osd":6,"uuid":"d757c72a-29f0-4f14-9f13-a2e61eb5e6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6819","nonce":2749897595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6821","nonce":2749897595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6825","nonce":2749897595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2749897595},{"type":"v1","addr":"172.21.15.2:6823","nonce":2749897595}]},"public_addr":"172.21.15.2:6819/2749897595","cluster_addr":"172.21.15.2:6821/2749897595","heartbeat_back_addr":"172.21.15.2:6825/2749897595","heartbeat_front_addr":"172.21.15.2:6823/2749897595","state":["exists","up"]},{"osd":7,"uuid":"22ddef80-3633-44c5-bc0e-edc75b546ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6817","nonce":1180401611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6819","nonce":1180401611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6823","nonce":1180401611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1180401611},{"type":"v1","addr":"172.21.15.26:6821","nonce":1180401611}]},"public_addr":"172.21.15.26:6817/1180401611","cluster_addr":"172.21.15.26:6819/1180401611","heartbeat_back_addr":"172.21.15.26:6823/1180401611","heartbeat_front_addr":"172.21.15.26:6821/1180401611","state":["exists","up"]},{"osd":8,"uuid":"466e7ac4-3ff7-4254-a3fd-8096420b1397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6817","nonce":2858408446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6819","nonce":2858408446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6823","nonce":2858408446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2858408446},{"type":"v1","addr":"172.21.15.29:6821","nonce":2858408446}]},"public_addr":"172.21.15.29:6817/2858408446","cluster_addr":"172.21.15.29:6819/2858408446","heartbeat_back_addr":"172.21.15.29:6823/2858408446","heartbeat_front_addr":"172.21.15.29:6821/2858408446","state":["exists","up"]},{"osd":9,"uuid":"119eeac9-29f7-4086-a1ad-04a0bc154b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6827","nonce":176898226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6829","nonce":176898226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6833","nonce":176898226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":176898226},{"type":"v1","addr":"172.21.15.2:6831","nonce":176898226}]},"public_addr":"172.21.15.2:6827/176898226","cluster_addr":"172.21.15.2:6829/176898226","heartbeat_back_addr":"172.21.15.2:6833/176898226","heartbeat_front_addr":"172.21.15.2:6831/176898226","state":["exists","up"]},{"osd":10,"uuid":"55cc0dbc-a79d-4abb-9ee0-18a3b0b785ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6825","nonce":4065159515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6827","nonce":4065159515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6831","nonce":4065159515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":4065159515},{"type":"v1","addr":"172.21.15.26:6829","nonce":4065159515}]},"public_addr":"172.21.15.26:6825/4065159515","cluster_addr":"172.21.15.26:6827/4065159515","heartbeat_back_addr":"172.21.15.26:6831/4065159515","heartbeat_front_addr":"172.21.15.26:6829/4065159515","state":["exists","up"]},{"osd":11,"uuid":"c5246763-c4f6-4ae3-8a93-5929c541ac5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6825","nonce":378800417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6827","nonce":378800417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6831","nonce":378800417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":378800417},{"type":"v1","addr":"172.21.15.29:6829","nonce":378800417}]},"public_addr":"172.21.15.29:6825/378800417","cluster_addr":"172.21.15.29:6827/378800417","heartbeat_back_addr":"172.21.15.29:6831/378800417","heartbeat_front_addr":"172.21.15.29:6829/378800417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:09.342387+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:34.156359+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:25:58.860653+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:24.009323+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:26:49.057146+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:15.007172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:27:41.085748+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:06.674923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:28:33.409128+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:00.106186+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:26.803487+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T09:29:53.668283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.d","mappings":[{"from":5,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11},{"from":8,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9},{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4051922409":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/2353790181":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/4268171474":"2024-05-15T09:23:28.940670+0000","172.21.15.2:6801/1423742578":"2024-05-15T09:23:28.940670+0000","172.21.15.2:0/969850441":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3252991268":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6801/3593583257":"2024-05-15T09:23:02.498446+0000","172.21.15.2:0/3789067476":"2024-05-15T09:23:02.498446+0000","172.21.15.2:6800/1423742578":"2024-05-15T09:23:28.940670+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T10:01:35.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.440+0000 7ff8a38dc700 1 -- 172.21.15.2:0/3689401808 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff8880b2880 msgr2=0x7ff8880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:35.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.440+0000 7ff8a38dc700 1 --2- 172.21.15.2:0/3689401808 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff8880b2880 0x7ff8880b4d90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff89c06bc10 tx=0x7ff898009260 comp rx=0 tx=0).stop 2024-05-14T10:01:35.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.440+0000 7ff8a38dc700 1 -- 172.21.15.2:0/3689401808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff89c108f60 msgr2=0x7ff89c19b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:35.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.440+0000 7ff8a38dc700 1 --2- 172.21.15.2:0/3689401808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff89c108f60 0x7ff89c19b3e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff88c006010 tx=0x7ff88c0047b0 comp rx=0 tx=0).stop 2024-05-14T10:01:35.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.440+0000 7ff8a38dc700 1 -- 172.21.15.2:0/3689401808 shutdown_connections 2024-05-14T10:01:35.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.440+0000 7ff8a38dc700 1 --2- 172.21.15.2:0/3689401808 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff8880b2880 0x7ff8880b4d90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:35.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.440+0000 7ff8a38dc700 1 --2- 172.21.15.2:0/3689401808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff89c102b40 0x7ff89c19ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:35.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.440+0000 7ff8a38dc700 1 --2- 172.21.15.2:0/3689401808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff89c1098b0 0x7ff89c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:35.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.440+0000 7ff8a38dc700 1 --2- 172.21.15.2:0/3689401808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff89c108f60 0x7ff89c19b3e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:35.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.440+0000 7ff8a38dc700 1 -- 172.21.15.2:0/3689401808 >> 172.21.15.2:0/3689401808 conn(0x7ff89c0fe520 msgr2=0x7ff89c100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:35.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.440+0000 7ff8a38dc700 1 -- 172.21.15.2:0/3689401808 shutdown_connections 2024-05-14T10:01:35.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:35.440+0000 7ff8a38dc700 1 -- 172.21.15.2:0/3689401808 wait complete. 2024-05-14T10:01:36.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:35 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/3689401808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T10:01:36.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:35 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/3689401808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T10:01:36.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:35 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/3689401808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T10:01:36.257 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs fail cephfs 2024-05-14T10:01:36.515 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T10:01:36.888 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:36 smithi002 ceph-mon[31557]: pgmap v1226: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T10:01:37.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:36 smithi029 ceph-mon[36249]: pgmap v1226: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T10:01:37.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:36 smithi026 ceph-mon[36056]: pgmap v1226: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T10:01:38.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.309+0000 7fb978714700 1 -- 172.21.15.2:0/62686244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9701034c0 msgr2=0x7fb9701038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:38.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.309+0000 7fb978714700 1 --2- 172.21.15.2:0/62686244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9701034c0 0x7fb9701038d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0099c0 tx=0x7fb96c009cd0 comp rx=0 tx=0).stop 2024-05-14T10:01:38.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.309+0000 7fb978714700 1 -- 172.21.15.2:0/62686244 shutdown_connections 2024-05-14T10:01:38.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.309+0000 7fb978714700 1 --2- 172.21.15.2:0/62686244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb970108740 0x7fb9701100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:38.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.309+0000 7fb978714700 1 --2- 172.21.15.2:0/62686244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9701034c0 0x7fb9701038d0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:38.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.309+0000 7fb978714700 1 --2- 172.21.15.2:0/62686244 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb970107df0 0x7fb970108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:38.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.309+0000 7fb978714700 1 -- 172.21.15.2:0/62686244 >> 172.21.15.2:0/62686244 conn(0x7fb9700fedf0 msgr2=0x7fb970101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:38.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.309+0000 7fb978714700 1 -- 172.21.15.2:0/62686244 shutdown_connections 2024-05-14T10:01:38.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.309+0000 7fb978714700 1 -- 172.21.15.2:0/62686244 wait complete. 2024-05-14T10:01:38.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.310+0000 7fb978714700 1 Processor -- start 2024-05-14T10:01:38.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.310+0000 7fb978714700 1 -- start start 2024-05-14T10:01:38.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.310+0000 7fb978714700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9701034c0 0x7fb9700ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:38.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.310+0000 7fb978714700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb970107df0 0x7fb9700ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:38.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.310+0000 7fb978714700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb970108740 0x7fb9700ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:38.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.310+0000 7fb978714700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb970113370 con 0x7fb970108740 2024-05-14T10:01:38.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.310+0000 7fb978714700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb9701131f0 con 0x7fb970107df0 2024-05-14T10:01:38.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.310+0000 7fb978714700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb9701134f0 con 0x7fb9701034c0 2024-05-14T10:01:38.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.310+0000 7fb976cb1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb970108740 0x7fb9700ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:38.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.310+0000 7fb976cb1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb970108740 0x7fb9700ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59140/0 (socket says 172.21.15.2:59140) 2024-05-14T10:01:38.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.310+0000 7fb976cb1700 1 -- 172.21.15.2:0/807000705 learned_addr learned my addr 172.21.15.2:0/807000705 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:01:38.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.310+0000 7fb9764b0700 1 --2- 172.21.15.2:0/807000705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9701034c0 0x7fb9700ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:38.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.311+0000 7fb975caf700 1 --2- 172.21.15.2:0/807000705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb970107df0 0x7fb9700ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:38.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.311+0000 7fb976cb1700 1 -- 172.21.15.2:0/807000705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9701034c0 msgr2=0x7fb9700ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:38.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.311+0000 7fb976cb1700 1 --2- 172.21.15.2:0/807000705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9701034c0 0x7fb9700ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:38.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.311+0000 7fb976cb1700 1 -- 172.21.15.2:0/807000705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb970107df0 msgr2=0x7fb9700ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:38.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.311+0000 7fb976cb1700 1 --2- 172.21.15.2:0/807000705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb970107df0 0x7fb9700ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:38.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.311+0000 7fb976cb1700 1 -- 172.21.15.2:0/807000705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9701a9150 con 0x7fb970108740 2024-05-14T10:01:38.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.311+0000 7fb976cb1700 1 --2- 172.21.15.2:0/807000705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb970108740 0x7fb9700ffd50 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb968007ee0 tx=0x7fb96800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:38.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.311+0000 7fb9677fe700 1 -- 172.21.15.2:0/807000705 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968004bb0 con 0x7fb970108740 2024-05-14T10:01:38.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.311+0000 7fb978714700 1 -- 172.21.15.2:0/807000705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9701a9440 con 0x7fb970108740 2024-05-14T10:01:38.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.311+0000 7fb9677fe700 1 -- 172.21.15.2:0/807000705 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb968004d50 con 0x7fb970108740 2024-05-14T10:01:38.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.311+0000 7fb9677fe700 1 -- 172.21.15.2:0/807000705 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968005920 con 0x7fb970108740 2024-05-14T10:01:38.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.311+0000 7fb978714700 1 -- 172.21.15.2:0/807000705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9701a99d0 con 0x7fb970108740 2024-05-14T10:01:38.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.312+0000 7fb978714700 1 -- 172.21.15.2:0/807000705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9700512c0 con 0x7fb970108740 2024-05-14T10:01:38.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.314+0000 7fb9677fe700 1 -- 172.21.15.2:0/807000705 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fb96801a020 con 0x7fb970108740 2024-05-14T10:01:38.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.315+0000 7fb9677fe700 1 --2- 172.21.15.2:0/807000705 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb95c0b2880 0x7fb95c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:38.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.315+0000 7fb9677fe700 1 -- 172.21.15.2:0/807000705 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9282+0+0 (secure 0 0 0) 0x7fb9680e2290 con 0x7fb970108740 2024-05-14T10:01:38.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.315+0000 7fb9764b0700 1 --2- 172.21.15.2:0/807000705 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb95c0b2880 0x7fb95c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:38.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.315+0000 7fb9764b0700 1 --2- 172.21.15.2:0/807000705 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb95c0b2880 0x7fb95c0b4d90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb96c000c00 tx=0x7fb96c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:38.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.318+0000 7fb9677fe700 1 -- 172.21.15.2:0/807000705 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb968010070 con 0x7fb970108740 2024-05-14T10:01:38.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.582+0000 7fb978714700 1 -- 172.21.15.2:0/807000705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb97002e5f0 con 0x7fb970108740 2024-05-14T10:01:38.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.584+0000 7fb9677fe700 1 -- 172.21.15.2:0/807000705 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) ==== 161+0+0 (secure 0 0 0) 0x7fb9680a9b90 con 0x7fb970108740 2024-05-14T10:01:38.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.588+0000 7fb978714700 1 -- 172.21.15.2:0/807000705 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb95c0b2880 msgr2=0x7fb95c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:38.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.588+0000 7fb978714700 1 --2- 172.21.15.2:0/807000705 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb95c0b2880 0x7fb95c0b4d90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb96c000c00 tx=0x7fb96c01a040 comp rx=0 tx=0).stop 2024-05-14T10:01:38.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.588+0000 7fb978714700 1 -- 172.21.15.2:0/807000705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb970108740 msgr2=0x7fb9700ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:38.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.588+0000 7fb978714700 1 --2- 172.21.15.2:0/807000705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb970108740 0x7fb9700ffd50 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb968007ee0 tx=0x7fb96800d3b0 comp rx=0 tx=0).stop 2024-05-14T10:01:38.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.588+0000 7fb978714700 1 -- 172.21.15.2:0/807000705 shutdown_connections 2024-05-14T10:01:38.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.588+0000 7fb978714700 1 --2- 172.21.15.2:0/807000705 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb95c0b2880 0x7fb95c0b4d90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:38.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.588+0000 7fb978714700 1 --2- 172.21.15.2:0/807000705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb970108740 0x7fb9700ffd50 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:38.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.588+0000 7fb978714700 1 --2- 172.21.15.2:0/807000705 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb9701034c0 0x7fb9700ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:38.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.588+0000 7fb978714700 1 --2- 172.21.15.2:0/807000705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb970107df0 0x7fb9700ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:38.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.588+0000 7fb978714700 1 -- 172.21.15.2:0/807000705 >> 172.21.15.2:0/807000705 conn(0x7fb9700fedf0 msgr2=0x7fb9701063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:38.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.588+0000 7fb978714700 1 -- 172.21.15.2:0/807000705 shutdown_connections 2024-05-14T10:01:38.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:38.588+0000 7fb978714700 1 -- 172.21.15.2:0/807000705 wait complete. 2024-05-14T10:01:38.589 INFO:teuthology.orchestra.run.smithi002.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T10:01:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:38 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a[31553]: 2024-05-14T10:01:38.583+0000 7f82ad210700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T10:01:38.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:38 smithi002 ceph-mon[31557]: pgmap v1227: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T10:01:38.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:38 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/807000705' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-14T10:01:38.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:38 smithi002 ceph-mon[31557]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-14T10:01:38.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:38 smithi002 ceph-mon[31557]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T10:01:38.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:38 smithi002 ceph-mon[31557]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-05-14T10:01:38.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:38 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/807000705' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-14T10:01:38.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:38 smithi002 ceph-mon[31557]: osdmap e94: 12 total, 12 up, 12 in 2024-05-14T10:01:38.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:38 smithi002 ceph-mon[31557]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-14T10:01:38.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:38 smithi026 ceph-mon[36056]: pgmap v1227: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T10:01:38.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:38 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/807000705' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-14T10:01:38.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:38 smithi026 ceph-mon[36056]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-14T10:01:38.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:38 smithi026 ceph-mon[36056]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T10:01:38.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:38 smithi026 ceph-mon[36056]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-05-14T10:01:38.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:38 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/807000705' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-14T10:01:38.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:38 smithi026 ceph-mon[36056]: osdmap e94: 12 total, 12 up, 12 in 2024-05-14T10:01:38.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:38 smithi026 ceph-mon[36056]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-14T10:01:38.952 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 10:01:38 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b[66741]: ignoring --setuser ceph since I am not root 2024-05-14T10:01:38.952 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 10:01:38 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b[66741]: ignoring --setgroup ceph since I am not root 2024-05-14T10:01:38.952 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 10:01:38 smithi026 ceph-mds[66745]: -- 172.21.15.26:0/2729293533 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b078dafa40 con 0x55b078df7600 2024-05-14T10:01:38.953 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 10:01:38 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b[66741]: starting mds.b at 2024-05-14T10:01:38.953 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 10:01:38 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e[66118]: ignoring --setuser ceph since I am not root 2024-05-14T10:01:38.953 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 10:01:38 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e[66118]: ignoring --setgroup ceph since I am not root 2024-05-14T10:01:38.953 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 10:01:38 smithi026 ceph-mds[66122]: -- 172.21.15.26:0/1021808194 <== mon.1 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b0c0a7fa40 con 0x55b0c0ac7180 2024-05-14T10:01:38.953 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 10:01:38 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e[66118]: starting mds.e at 2024-05-14T10:01:38.953 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 10:01:38 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: ignoring --setuser ceph since I am not root 2024-05-14T10:01:38.953 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 10:01:38 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: ignoring --setgroup ceph since I am not root 2024-05-14T10:01:38.953 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 10:01:38 smithi026 ceph-mds[66426]: -- 172.21.15.26:0/1074719867 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd993aba40 con 0x55dd993f3180 2024-05-14T10:01:38.953 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 10:01:38 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: starting mds.h at 2024-05-14T10:01:38.990 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:38 smithi029 ceph-mon[36249]: pgmap v1227: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T10:01:38.990 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:38 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/807000705' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-14T10:01:38.990 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:38 smithi029 ceph-mon[36249]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-14T10:01:38.990 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:38 smithi029 ceph-mon[36249]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T10:01:38.990 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:38 smithi029 ceph-mon[36249]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-05-14T10:01:38.990 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:38 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/807000705' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-14T10:01:38.990 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:38 smithi029 ceph-mon[36249]: osdmap e94: 12 total, 12 up, 12 in 2024-05-14T10:01:38.990 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:38 smithi029 ceph-mon[36249]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-14T10:01:38.991 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 10:01:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-c[66381]: ignoring --setuser ceph since I am not root 2024-05-14T10:01:38.991 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 10:01:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-c[66381]: ignoring --setgroup ceph since I am not root 2024-05-14T10:01:38.991 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 10:01:38 smithi029 ceph-mds[66385]: -- 172.21.15.29:0/1525541489 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55866ddd9a40 con 0x55866de21600 2024-05-14T10:01:38.991 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 10:01:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-c[66381]: starting mds.c at 2024-05-14T10:01:38.991 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 10:01:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-f[66076]: ignoring --setuser ceph since I am not root 2024-05-14T10:01:38.991 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 10:01:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-f[66076]: ignoring --setgroup ceph since I am not root 2024-05-14T10:01:38.991 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 10:01:38 smithi029 ceph-mds[66080]: -- 172.21.15.29:0/845608936 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563618217a40 con 0x56361825f600 2024-05-14T10:01:38.991 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 10:01:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-f[66076]: starting mds.f at 2024-05-14T10:01:38.992 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 10:01:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-i[66685]: ignoring --setuser ceph since I am not root 2024-05-14T10:01:38.992 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 10:01:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-i[66685]: ignoring --setgroup ceph since I am not root 2024-05-14T10:01:38.992 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 10:01:38 smithi029 ceph-mds[66689]: -- 172.21.15.29:0/140598349 <== mon.1 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5597d65e3a40 con 0x5597d662b180 2024-05-14T10:01:38.992 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 10:01:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-i[66685]: starting mds.i at 2024-05-14T10:01:39.399 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-14T10:01:39.656 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T10:01:40.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:40 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.26:6834/3903878399,v1:172.21.15.26:6836/3903878399] up:boot 2024-05-14T10:01:40.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:40 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.26:6837/151226900,v1:172.21.15.26:6839/151226900] up:boot 2024-05-14T10:01:40.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:40 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.26:6835/71308740,v1:172.21.15.26:6838/71308740] up:boot 2024-05-14T10:01:40.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:40 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.29:6834/1635700758,v1:172.21.15.29:6835/1635700758] up:boot 2024-05-14T10:01:40.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:40 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.29:6837/563935213,v1:172.21.15.29:6839/563935213] up:boot 2024-05-14T10:01:40.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:40 smithi002 ceph-mon[31557]: mds.? [v2:172.21.15.29:6836/3729816890,v1:172.21.15.29:6838/3729816890] up:boot 2024-05-14T10:01:40.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:40 smithi002 ceph-mon[31557]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-14T10:01:40.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T10:01:40.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T10:01:40.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T10:01:40.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T10:01:40.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T10:01:40.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:40 smithi002 ceph-mon[31557]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T10:01:40.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:40 smithi002 ceph-mon[31557]: pgmap v1229: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T10:01:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:40 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.26:6834/3903878399,v1:172.21.15.26:6836/3903878399] up:boot 2024-05-14T10:01:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:40 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.26:6837/151226900,v1:172.21.15.26:6839/151226900] up:boot 2024-05-14T10:01:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:40 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.26:6835/71308740,v1:172.21.15.26:6838/71308740] up:boot 2024-05-14T10:01:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:40 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.29:6834/1635700758,v1:172.21.15.29:6835/1635700758] up:boot 2024-05-14T10:01:40.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:40 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.29:6837/563935213,v1:172.21.15.29:6839/563935213] up:boot 2024-05-14T10:01:40.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:40 smithi029 ceph-mon[36249]: mds.? [v2:172.21.15.29:6836/3729816890,v1:172.21.15.29:6838/3729816890] up:boot 2024-05-14T10:01:40.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:40 smithi029 ceph-mon[36249]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-14T10:01:40.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T10:01:40.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T10:01:40.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T10:01:40.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T10:01:40.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T10:01:40.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:40 smithi029 ceph-mon[36249]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T10:01:40.938 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:40 smithi029 ceph-mon[36249]: pgmap v1229: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T10:01:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:40 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.26:6834/3903878399,v1:172.21.15.26:6836/3903878399] up:boot 2024-05-14T10:01:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:40 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.26:6837/151226900,v1:172.21.15.26:6839/151226900] up:boot 2024-05-14T10:01:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:40 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.26:6835/71308740,v1:172.21.15.26:6838/71308740] up:boot 2024-05-14T10:01:40.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:40 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.29:6834/1635700758,v1:172.21.15.29:6835/1635700758] up:boot 2024-05-14T10:01:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:40 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.29:6837/563935213,v1:172.21.15.29:6839/563935213] up:boot 2024-05-14T10:01:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:40 smithi026 ceph-mon[36056]: mds.? [v2:172.21.15.29:6836/3729816890,v1:172.21.15.29:6838/3729816890] up:boot 2024-05-14T10:01:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:40 smithi026 ceph-mon[36056]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-14T10:01:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T10:01:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T10:01:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T10:01:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T10:01:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T10:01:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:40 smithi026 ceph-mon[36056]: from='mgr.14152 172.21.15.2:0/2066646370' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T10:01:40.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:40 smithi026 ceph-mon[36056]: pgmap v1229: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T10:01:41.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.530+0000 7fb23d992700 1 -- 172.21.15.2:0/4027206502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2381045e0 msgr2=0x7fb2381049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:41.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.530+0000 7fb23d992700 1 --2- 172.21.15.2:0/4027206502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2381045e0 0x7fb2381049f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fb2280099c0 tx=0x7fb228009cd0 comp rx=0 tx=0).stop 2024-05-14T10:01:41.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.530+0000 7fb23d992700 1 -- 172.21.15.2:0/4027206502 shutdown_connections 2024-05-14T10:01:41.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.530+0000 7fb23d992700 1 --2- 172.21.15.2:0/4027206502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb238109860 0x7fb2381111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:41.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.530+0000 7fb23d992700 1 --2- 172.21.15.2:0/4027206502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2381045e0 0x7fb2381049f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:41.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.530+0000 7fb23d992700 1 --2- 172.21.15.2:0/4027206502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb238108f10 0x7fb238109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:41.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.530+0000 7fb23d992700 1 -- 172.21.15.2:0/4027206502 >> 172.21.15.2:0/4027206502 conn(0x7fb2380ffed0 msgr2=0x7fb238102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:41.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.530+0000 7fb23d992700 1 -- 172.21.15.2:0/4027206502 shutdown_connections 2024-05-14T10:01:41.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.530+0000 7fb23d992700 1 -- 172.21.15.2:0/4027206502 wait complete. 2024-05-14T10:01:41.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.531+0000 7fb23d992700 1 Processor -- start 2024-05-14T10:01:41.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.531+0000 7fb23d992700 1 -- start start 2024-05-14T10:01:41.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.531+0000 7fb23d992700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2381045e0 0x7fb23819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:41.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.531+0000 7fb23d992700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb238108f10 0x7fb23819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:41.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.531+0000 7fb23d992700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb238109860 0x7fb23819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:41.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.531+0000 7fb23d992700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381144e0 con 0x7fb238109860 2024-05-14T10:01:41.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.531+0000 7fb23d992700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb238114360 con 0x7fb238108f10 2024-05-14T10:01:41.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.531+0000 7fb23d992700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fb238114660 con 0x7fb2381045e0 2024-05-14T10:01:41.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.531+0000 7fb2377fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb238109860 0x7fb23819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:41.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.531+0000 7fb236ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2381045e0 0x7fb23819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:41.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.531+0000 7fb2377fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb238109860 0x7fb23819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59156/0 (socket says 172.21.15.2:59156) 2024-05-14T10:01:41.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.531+0000 7fb2377fe700 1 -- 172.21.15.2:0/2577851118 learned_addr learned my addr 172.21.15.2:0/2577851118 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:01:41.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.531+0000 7fb2367fc700 1 --2- 172.21.15.2:0/2577851118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb238108f10 0x7fb23819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:41.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.532+0000 7fb2377fe700 1 -- 172.21.15.2:0/2577851118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2381045e0 msgr2=0x7fb23819ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:41.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.532+0000 7fb2377fe700 1 --2- 172.21.15.2:0/2577851118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2381045e0 0x7fb23819ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:41.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.532+0000 7fb2377fe700 1 -- 172.21.15.2:0/2577851118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb238108f10 msgr2=0x7fb23819b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:41.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.532+0000 7fb2377fe700 1 --2- 172.21.15.2:0/2577851118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb238108f10 0x7fb23819b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:41.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.532+0000 7fb2377fe700 1 -- 172.21.15.2:0/2577851118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23819fc90 con 0x7fb238109860 2024-05-14T10:01:41.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.532+0000 7fb236ffd700 1 --2- 172.21.15.2:0/2577851118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2381045e0 0x7fb23819ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T10:01:41.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.532+0000 7fb2377fe700 1 --2- 172.21.15.2:0/2577851118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb238109860 0x7fb23819f6a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb23000ada0 tx=0x7fb23000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:41.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.532+0000 7fb23c990700 1 -- 172.21.15.2:0/2577851118 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230017020 con 0x7fb238109860 2024-05-14T10:01:41.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.532+0000 7fb23d992700 1 -- 172.21.15.2:0/2577851118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23819ff80 con 0x7fb238109860 2024-05-14T10:01:41.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.532+0000 7fb23c990700 1 -- 172.21.15.2:0/2577851118 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb23000cd80 con 0x7fb238109860 2024-05-14T10:01:41.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.532+0000 7fb23c990700 1 -- 172.21.15.2:0/2577851118 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230005260 con 0x7fb238109860 2024-05-14T10:01:41.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.532+0000 7fb23d992700 1 -- 172.21.15.2:0/2577851118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2381a04c0 con 0x7fb238109860 2024-05-14T10:01:41.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.533+0000 7fb23d992700 1 -- 172.21.15.2:0/2577851118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2380512c0 con 0x7fb238109860 2024-05-14T10:01:41.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.535+0000 7fb23c990700 1 -- 172.21.15.2:0/2577851118 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fb23000a090 con 0x7fb238109860 2024-05-14T10:01:41.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.536+0000 7fb23c990700 1 --2- 172.21.15.2:0/2577851118 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2200b2900 0x7fb2200b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:41.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.536+0000 7fb236ffd700 1 --2- 172.21.15.2:0/2577851118 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2200b2900 0x7fb2200b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:41.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.536+0000 7fb23c990700 1 -- 172.21.15.2:0/2577851118 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9540+0+0 (secure 0 0 0) 0x7fb2300e2370 con 0x7fb238109860 2024-05-14T10:01:41.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.536+0000 7fb236ffd700 1 --2- 172.21.15.2:0/2577851118 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2200b2900 0x7fb2200b4e10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb22800b580 tx=0x7fb22801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:41.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.539+0000 7fb23c990700 1 -- 172.21.15.2:0/2577851118 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb230021020 con 0x7fb238109860 2024-05-14T10:01:41.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:41.803+0000 7fb23d992700 1 -- 172.21.15.2:0/2577851118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb23819c1d0 con 0x7fb238109860 2024-05-14T10:01:42.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:42.706+0000 7fb23c990700 1 -- 172.21.15.2:0/2577851118 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v74) ==== 104+0+0 (secure 0 0 0) 0x7fb2300a9ba0 con 0x7fb238109860 2024-05-14T10:01:42.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:42.710+0000 7fb23d992700 1 -- 172.21.15.2:0/2577851118 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2200b2900 msgr2=0x7fb2200b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:42.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:42.710+0000 7fb23d992700 1 --2- 172.21.15.2:0/2577851118 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2200b2900 0x7fb2200b4e10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb22800b580 tx=0x7fb22801a040 comp rx=0 tx=0).stop 2024-05-14T10:01:42.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:42.710+0000 7fb23d992700 1 -- 172.21.15.2:0/2577851118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb238109860 msgr2=0x7fb23819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:42.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:42.710+0000 7fb23d992700 1 --2- 172.21.15.2:0/2577851118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb238109860 0x7fb23819f6a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb23000ada0 tx=0x7fb23000c330 comp rx=0 tx=0).stop 2024-05-14T10:01:42.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:42.711+0000 7fb23d992700 1 -- 172.21.15.2:0/2577851118 shutdown_connections 2024-05-14T10:01:42.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:42.711+0000 7fb23d992700 1 --2- 172.21.15.2:0/2577851118 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fb2200b2900 0x7fb2200b4e10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:42.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:42.711+0000 7fb23d992700 1 --2- 172.21.15.2:0/2577851118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb238109860 0x7fb23819f6a0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:42.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:42.711+0000 7fb23d992700 1 --2- 172.21.15.2:0/2577851118 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2381045e0 0x7fb23819ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:42.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:42.711+0000 7fb23d992700 1 --2- 172.21.15.2:0/2577851118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb238108f10 0x7fb23819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:42.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:42.711+0000 7fb23d992700 1 -- 172.21.15.2:0/2577851118 >> 172.21.15.2:0/2577851118 conn(0x7fb2380ffed0 msgr2=0x7fb23810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:42.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:42.711+0000 7fb23d992700 1 -- 172.21.15.2:0/2577851118 shutdown_connections 2024-05-14T10:01:42.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:42.711+0000 7fb23d992700 1 -- 172.21.15.2:0/2577851118 wait complete. 2024-05-14T10:01:42.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:42 smithi026 ceph-mon[36056]: pgmap v1230: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T10:01:42.952 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:42 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2577851118' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T10:01:43.046 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:42 smithi002 ceph-mon[31557]: pgmap v1230: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T10:01:43.046 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:42 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2577851118' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T10:01:43.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:42 smithi029 ceph-mon[36249]: pgmap v1230: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T10:01:43.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:42 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2577851118' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T10:01:43.390 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-14T10:01:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T10:01:44.033 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:43 smithi002 ceph-mon[31557]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T10:01:44.033 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:43 smithi002 ceph-mon[31557]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-14T10:01:44.033 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:43 smithi002 ceph-mon[31557]: Cluster is now healthy 2024-05-14T10:01:44.033 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:43 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/2577851118' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-14T10:01:44.033 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:43 smithi002 ceph-mon[31557]: fsmap 12 up:standby 2024-05-14T10:01:44.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:43 smithi029 ceph-mon[36249]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T10:01:44.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:43 smithi029 ceph-mon[36249]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-14T10:01:44.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:43 smithi029 ceph-mon[36249]: Cluster is now healthy 2024-05-14T10:01:44.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:43 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/2577851118' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-14T10:01:44.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:43 smithi029 ceph-mon[36249]: fsmap 12 up:standby 2024-05-14T10:01:44.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:43 smithi026 ceph-mon[36056]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T10:01:44.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:43 smithi026 ceph-mon[36056]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-14T10:01:44.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:43 smithi026 ceph-mon[36056]: Cluster is now healthy 2024-05-14T10:01:44.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:43 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/2577851118' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-14T10:01:44.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:43 smithi026 ceph-mon[36056]: fsmap 12 up:standby 2024-05-14T10:01:44.830 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:44 smithi002 ceph-mon[31557]: pgmap v1231: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-14T10:01:45.103 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:44 smithi026 ceph-mon[36056]: pgmap v1231: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-14T10:01:45.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:44 smithi029 ceph-mon[36249]: pgmap v1231: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-14T10:01:45.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.420+0000 7fc0209f6700 1 -- 172.21.15.2:0/4054249189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc01c1045d0 msgr2=0x7fc01c1049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:45.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.420+0000 7fc0209f6700 1 --2- 172.21.15.2:0/4054249189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc01c1045d0 0x7fc01c1049e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc0140099c0 tx=0x7fc014009cd0 comp rx=0 tx=0).stop 2024-05-14T10:01:45.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.421+0000 7fc0209f6700 1 -- 172.21.15.2:0/4054249189 shutdown_connections 2024-05-14T10:01:45.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.421+0000 7fc0209f6700 1 --2- 172.21.15.2:0/4054249189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc01c109850 0x7fc01c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:45.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.421+0000 7fc0209f6700 1 --2- 172.21.15.2:0/4054249189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc01c1045d0 0x7fc01c1049e0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:45.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.421+0000 7fc0209f6700 1 --2- 172.21.15.2:0/4054249189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c108f00 0x7fc01c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:45.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.421+0000 7fc0209f6700 1 -- 172.21.15.2:0/4054249189 >> 172.21.15.2:0/4054249189 conn(0x7fc01c0ffea0 msgr2=0x7fc01c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:45.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.421+0000 7fc0209f6700 1 -- 172.21.15.2:0/4054249189 shutdown_connections 2024-05-14T10:01:45.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.421+0000 7fc0209f6700 1 -- 172.21.15.2:0/4054249189 wait complete. 2024-05-14T10:01:45.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc0209f6700 1 Processor -- start 2024-05-14T10:01:45.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc0209f6700 1 -- start start 2024-05-14T10:01:45.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc0209f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c1045d0 0x7fc01c076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:45.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc0209f6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc01c108f00 0x7fc01c077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:45.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc0209f6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc01c109850 0x7fc01c077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:45.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc0209f6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1144d0 con 0x7fc01c108f00 2024-05-14T10:01:45.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc0209f6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c114350 con 0x7fc01c1045d0 2024-05-14T10:01:45.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc0209f6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c114650 con 0x7fc01c109850 2024-05-14T10:01:45.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc019d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc01c108f00 0x7fc01c077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:45.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc019d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc01c108f00 0x7fc01c077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59176/0 (socket says 172.21.15.2:59176) 2024-05-14T10:01:45.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc01a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c1045d0 0x7fc01c076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:45.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc019d9b700 1 -- 172.21.15.2:0/4170240648 learned_addr learned my addr 172.21.15.2:0/4170240648 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:01:45.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc01ad9d700 1 --2- 172.21.15.2:0/4170240648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc01c109850 0x7fc01c077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:45.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc019d9b700 1 -- 172.21.15.2:0/4170240648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc01c109850 msgr2=0x7fc01c077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:45.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc019d9b700 1 --2- 172.21.15.2:0/4170240648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc01c109850 0x7fc01c077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:45.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc019d9b700 1 -- 172.21.15.2:0/4170240648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c1045d0 msgr2=0x7fc01c076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:45.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc019d9b700 1 --2- 172.21.15.2:0/4170240648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c1045d0 0x7fc01c076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:45.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.422+0000 7fc019d9b700 1 -- 172.21.15.2:0/4170240648 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c1a9280 con 0x7fc01c108f00 2024-05-14T10:01:45.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.423+0000 7fc01a59c700 1 --2- 172.21.15.2:0/4170240648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c1045d0 0x7fc01c076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T10:01:45.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.423+0000 7fc019d9b700 1 --2- 172.21.15.2:0/4170240648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc01c108f00 0x7fc01c077110 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc00400e970 tx=0x7fc00400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:45.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.423+0000 7fc0137fe700 1 -- 172.21.15.2:0/4170240648 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004004a30 con 0x7fc01c108f00 2024-05-14T10:01:45.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.423+0000 7fc0209f6700 1 -- 172.21.15.2:0/4170240648 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c1a9570 con 0x7fc01c108f00 2024-05-14T10:01:45.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.423+0000 7fc0137fe700 1 -- 172.21.15.2:0/4170240648 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc00400cc90 con 0x7fc01c108f00 2024-05-14T10:01:45.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.423+0000 7fc0137fe700 1 -- 172.21.15.2:0/4170240648 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004005610 con 0x7fc01c108f00 2024-05-14T10:01:45.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.423+0000 7fc0209f6700 1 -- 172.21.15.2:0/4170240648 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c1a9ab0 con 0x7fc01c108f00 2024-05-14T10:01:45.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.424+0000 7fc0209f6700 1 -- 172.21.15.2:0/4170240648 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01c0512c0 con 0x7fc01c108f00 2024-05-14T10:01:45.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.425+0000 7fc0137fe700 1 -- 172.21.15.2:0/4170240648 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc00400c7e0 con 0x7fc01c108f00 2024-05-14T10:01:45.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.426+0000 7fc0137fe700 1 --2- 172.21.15.2:0/4170240648 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0080b2880 0x7fc0080b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:45.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.426+0000 7fc01a59c700 1 --2- 172.21.15.2:0/4170240648 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0080b2880 0x7fc0080b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:45.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.426+0000 7fc0137fe700 1 -- 172.21.15.2:0/4170240648 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9540+0+0 (secure 0 0 0) 0x7fc0040e28f0 con 0x7fc01c108f00 2024-05-14T10:01:45.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.427+0000 7fc01a59c700 1 --2- 172.21.15.2:0/4170240648 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0080b2880 0x7fc0080b4d90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc014000c00 tx=0x7fc01401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:45.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.431+0000 7fc0137fe700 1 -- 172.21.15.2:0/4170240648 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc0040aa090 con 0x7fc01c108f00 2024-05-14T10:01:45.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.643+0000 7fc0209f6700 1 -- 172.21.15.2:0/4170240648 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc01c078100 con 0x7fc01c108f00 2024-05-14T10:01:45.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.714+0000 7fc0137fe700 1 -- 172.21.15.2:0/4170240648 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) ==== 181+0+0 (secure 0 0 0) 0x7fc0040a9990 con 0x7fc01c108f00 2024-05-14T10:01:45.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.717+0000 7fc0209f6700 1 -- 172.21.15.2:0/4170240648 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0080b2880 msgr2=0x7fc0080b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:45.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.718+0000 7fc0209f6700 1 --2- 172.21.15.2:0/4170240648 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0080b2880 0x7fc0080b4d90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc014000c00 tx=0x7fc01401a040 comp rx=0 tx=0).stop 2024-05-14T10:01:45.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.718+0000 7fc0209f6700 1 -- 172.21.15.2:0/4170240648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc01c108f00 msgr2=0x7fc01c077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:45.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.718+0000 7fc0209f6700 1 --2- 172.21.15.2:0/4170240648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc01c108f00 0x7fc01c077110 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc00400e970 tx=0x7fc00400ec80 comp rx=0 tx=0).stop 2024-05-14T10:01:45.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.719+0000 7fc0209f6700 1 -- 172.21.15.2:0/4170240648 shutdown_connections 2024-05-14T10:01:45.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.719+0000 7fc0209f6700 1 --2- 172.21.15.2:0/4170240648 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fc0080b2880 0x7fc0080b4d90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:45.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.719+0000 7fc0209f6700 1 --2- 172.21.15.2:0/4170240648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c1045d0 0x7fc01c076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:45.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.719+0000 7fc0209f6700 1 --2- 172.21.15.2:0/4170240648 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc01c109850 0x7fc01c077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:45.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.719+0000 7fc0209f6700 1 --2- 172.21.15.2:0/4170240648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc01c108f00 0x7fc01c077110 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:45.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.719+0000 7fc0209f6700 1 -- 172.21.15.2:0/4170240648 >> 172.21.15.2:0/4170240648 conn(0x7fc01c0ffea0 msgr2=0x7fc01c10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:45.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.719+0000 7fc0209f6700 1 -- 172.21.15.2:0/4170240648 shutdown_connections 2024-05-14T10:01:45.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:45.719+0000 7fc0209f6700 1 -- 172.21.15.2:0/4170240648 wait complete. 2024-05-14T10:01:45.725 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' removed 2024-05-14T10:01:45.962 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:45 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/4170240648' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T10:01:46.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:45 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/4170240648' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T10:01:46.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:45 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/4170240648' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T10:01:46.568 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-14T10:01:46.820 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T10:01:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:46 smithi002 ceph-mon[31557]: pgmap v1232: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-14T10:01:46.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:46 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/4170240648' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T10:01:46.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:46 smithi002 ceph-mon[31557]: osdmap e95: 12 total, 12 up, 12 in 2024-05-14T10:01:47.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:46 smithi029 ceph-mon[36249]: pgmap v1232: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-14T10:01:47.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:46 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/4170240648' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T10:01:47.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:46 smithi029 ceph-mon[36249]: osdmap e95: 12 total, 12 up, 12 in 2024-05-14T10:01:47.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:46 smithi026 ceph-mon[36056]: pgmap v1232: 129 pgs: 129 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s 2024-05-14T10:01:47.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:46 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/4170240648' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T10:01:47.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:46 smithi026 ceph-mon[36056]: osdmap e95: 12 total, 12 up, 12 in 2024-05-14T10:01:48.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.524+0000 7fbde78b4700 1 -- 172.21.15.2:0/209660553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbde01034c0 msgr2=0x7fbde01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:48.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.524+0000 7fbde78b4700 1 --2- 172.21.15.2:0/209660553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbde01034c0 0x7fbde01038d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0099c0 tx=0x7fbddc009cd0 comp rx=0 tx=0).stop 2024-05-14T10:01:48.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.524+0000 7fbde78b4700 1 -- 172.21.15.2:0/209660553 shutdown_connections 2024-05-14T10:01:48.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.524+0000 7fbde78b4700 1 --2- 172.21.15.2:0/209660553 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbde0108740 0x7fbde01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:48.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.524+0000 7fbde78b4700 1 --2- 172.21.15.2:0/209660553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbde01034c0 0x7fbde01038d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:48.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.524+0000 7fbde78b4700 1 --2- 172.21.15.2:0/209660553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbde0107df0 0x7fbde0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:48.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.524+0000 7fbde78b4700 1 -- 172.21.15.2:0/209660553 >> 172.21.15.2:0/209660553 conn(0x7fbde00fedf0 msgr2=0x7fbde0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:48.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.524+0000 7fbde78b4700 1 -- 172.21.15.2:0/209660553 shutdown_connections 2024-05-14T10:01:48.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.524+0000 7fbde78b4700 1 -- 172.21.15.2:0/209660553 wait complete. 2024-05-14T10:01:48.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.525+0000 7fbde78b4700 1 Processor -- start 2024-05-14T10:01:48.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.525+0000 7fbde78b4700 1 -- start start 2024-05-14T10:01:48.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde78b4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbde01034c0 0x7fbde019bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:48.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde78b4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbde0107df0 0x7fbde019c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:48.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde78b4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbde0108740 0x7fbde019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:48.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde78b4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbde01133f0 con 0x7fbde0108740 2024-05-14T10:01:48.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde78b4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0113270 con 0x7fbde0107df0 2024-05-14T10:01:48.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde78b4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0113570 con 0x7fbde01034c0 2024-05-14T10:01:48.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde5e51700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbde0108740 0x7fbde019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:48.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde5e51700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbde0108740 0x7fbde019c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60168/0 (socket says 172.21.15.2:60168) 2024-05-14T10:01:48.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde5e51700 1 -- 172.21.15.2:0/1934251293 learned_addr learned my addr 172.21.15.2:0/1934251293 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:01:48.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde4e4f700 1 --2- 172.21.15.2:0/1934251293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbde0107df0 0x7fbde019c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:48.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde5650700 1 --2- 172.21.15.2:0/1934251293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbde01034c0 0x7fbde019bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:48.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde5e51700 1 -- 172.21.15.2:0/1934251293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbde01034c0 msgr2=0x7fbde019bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:48.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde5e51700 1 --2- 172.21.15.2:0/1934251293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbde01034c0 0x7fbde019bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:48.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde5e51700 1 -- 172.21.15.2:0/1934251293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbde0107df0 msgr2=0x7fbde019c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:48.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde5e51700 1 --2- 172.21.15.2:0/1934251293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbde0107df0 0x7fbde019c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:48.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde5e51700 1 -- 172.21.15.2:0/1934251293 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde01a7060 con 0x7fbde0108740 2024-05-14T10:01:48.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde5650700 1 --2- 172.21.15.2:0/1934251293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbde01034c0 0x7fbde019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T10:01:48.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.526+0000 7fbde5e51700 1 --2- 172.21.15.2:0/1934251293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbde0108740 0x7fbde019c7b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800b970 tx=0x7fbdd800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:48.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.527+0000 7fbdd67fc700 1 -- 172.21.15.2:0/1934251293 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8019070 con 0x7fbde0108740 2024-05-14T10:01:48.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.527+0000 7fbde78b4700 1 -- 172.21.15.2:0/1934251293 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde01a7350 con 0x7fbde0108740 2024-05-14T10:01:48.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.527+0000 7fbdd67fc700 1 -- 172.21.15.2:0/1934251293 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd800cde0 con 0x7fbde0108740 2024-05-14T10:01:48.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.527+0000 7fbdd67fc700 1 -- 172.21.15.2:0/1934251293 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8007370 con 0x7fbde0108740 2024-05-14T10:01:48.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.527+0000 7fbde78b4700 1 -- 172.21.15.2:0/1934251293 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde01a7890 con 0x7fbde0108740 2024-05-14T10:01:48.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.528+0000 7fbde78b4700 1 -- 172.21.15.2:0/1934251293 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde00512c0 con 0x7fbde0108740 2024-05-14T10:01:48.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.529+0000 7fbdd67fc700 1 -- 172.21.15.2:0/1934251293 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fbdd801c070 con 0x7fbde0108740 2024-05-14T10:01:48.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.530+0000 7fbdd67fc700 1 --2- 172.21.15.2:0/1934251293 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbdcc0b2880 0x7fbdcc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:48.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.530+0000 7fbdd67fc700 1 -- 172.21.15.2:0/1934251293 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8891+0+0 (secure 0 0 0) 0x7fbdd80e19b0 con 0x7fbde0108740 2024-05-14T10:01:48.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.530+0000 7fbde5650700 1 --2- 172.21.15.2:0/1934251293 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbdcc0b2880 0x7fbdcc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:48.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.531+0000 7fbde5650700 1 --2- 172.21.15.2:0/1934251293 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbdcc0b2880 0x7fbdcc0b4d90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbddc003800 tx=0x7fbddc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:48.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.535+0000 7fbdd67fc700 1 -- 172.21.15.2:0/1934251293 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbdd8051df0 con 0x7fbde0108740 2024-05-14T10:01:48.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:48.751+0000 7fbde78b4700 1 -- 172.21.15.2:0/1934251293 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbde0076000 con 0x7fbde0108740 2024-05-14T10:01:48.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:48 smithi002 ceph-mon[31557]: pgmap v1234: 97 pgs: 97 active+clean; 577 KiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T10:01:49.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:48 smithi029 ceph-mon[36249]: pgmap v1234: 97 pgs: 97 active+clean; 577 KiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T10:01:49.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:48 smithi026 ceph-mon[36056]: pgmap v1234: 97 pgs: 97 active+clean; 577 KiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T10:01:49.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:49.724+0000 7fbdd67fc700 1 -- 172.21.15.2:0/1934251293 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) ==== 169+0+0 (secure 0 0 0) 0x7fbdd80a9440 con 0x7fbde0108740 2024-05-14T10:01:49.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:49.728+0000 7fbde78b4700 1 -- 172.21.15.2:0/1934251293 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbdcc0b2880 msgr2=0x7fbdcc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:49.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:49.728+0000 7fbde78b4700 1 --2- 172.21.15.2:0/1934251293 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbdcc0b2880 0x7fbdcc0b4d90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbddc003800 tx=0x7fbddc01a040 comp rx=0 tx=0).stop 2024-05-14T10:01:49.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:49.728+0000 7fbde78b4700 1 -- 172.21.15.2:0/1934251293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbde0108740 msgr2=0x7fbde019c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:49.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:49.728+0000 7fbde78b4700 1 --2- 172.21.15.2:0/1934251293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbde0108740 0x7fbde019c7b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800b970 tx=0x7fbdd800bc80 comp rx=0 tx=0).stop 2024-05-14T10:01:49.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:49.728+0000 7fbde78b4700 1 -- 172.21.15.2:0/1934251293 shutdown_connections 2024-05-14T10:01:49.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:49.728+0000 7fbde78b4700 1 --2- 172.21.15.2:0/1934251293 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7fbdcc0b2880 0x7fbdcc0b4d90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:49.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:49.728+0000 7fbde78b4700 1 --2- 172.21.15.2:0/1934251293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbde0108740 0x7fbde019c7b0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:49.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:49.728+0000 7fbde78b4700 1 --2- 172.21.15.2:0/1934251293 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbde01034c0 0x7fbde019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:49.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:49.728+0000 7fbde78b4700 1 --2- 172.21.15.2:0/1934251293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbde0107df0 0x7fbde019c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:49.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:49.728+0000 7fbde78b4700 1 -- 172.21.15.2:0/1934251293 >> 172.21.15.2:0/1934251293 conn(0x7fbde00fedf0 msgr2=0x7fbde01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:49.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:49.728+0000 7fbde78b4700 1 -- 172.21.15.2:0/1934251293 shutdown_connections 2024-05-14T10:01:49.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:49.728+0000 7fbde78b4700 1 -- 172.21.15.2:0/1934251293 wait complete. 2024-05-14T10:01:49.729 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' removed 2024-05-14T10:01:50.104 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:49 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1934251293' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T10:01:50.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:49 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1934251293' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T10:01:50.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:49 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1934251293' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T10:01:50.426 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-14T10:01:50.676 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T10:01:51.125 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:50 smithi002 ceph-mon[31557]: pgmap v1235: 97 pgs: 97 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T10:01:51.125 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:50 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/1934251293' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T10:01:51.125 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:50 smithi002 ceph-mon[31557]: osdmap e96: 12 total, 12 up, 12 in 2024-05-14T10:01:51.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:50 smithi029 ceph-mon[36249]: pgmap v1235: 97 pgs: 97 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T10:01:51.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:50 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/1934251293' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T10:01:51.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:50 smithi029 ceph-mon[36249]: osdmap e96: 12 total, 12 up, 12 in 2024-05-14T10:01:51.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:50 smithi026 ceph-mon[36056]: pgmap v1235: 97 pgs: 97 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T10:01:51.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:50 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/1934251293' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T10:01:51.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:50 smithi026 ceph-mon[36056]: osdmap e96: 12 total, 12 up, 12 in 2024-05-14T10:01:52.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.557+0000 7ff3a4535700 1 -- 172.21.15.2:0/70500018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff39c103550 msgr2=0x7ff39c103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:52.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.557+0000 7ff3a4535700 1 --2- 172.21.15.2:0/70500018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff39c103550 0x7ff39c103960 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff3980099c0 tx=0x7ff398009cd0 comp rx=0 tx=0).stop 2024-05-14T10:01:52.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.557+0000 7ff3a4535700 1 -- 172.21.15.2:0/70500018 shutdown_connections 2024-05-14T10:01:52.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.557+0000 7ff3a4535700 1 --2- 172.21.15.2:0/70500018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff39c108780 0x7ff39c1100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:52.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.557+0000 7ff3a4535700 1 --2- 172.21.15.2:0/70500018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff39c103550 0x7ff39c103960 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:52.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.557+0000 7ff3a4535700 1 --2- 172.21.15.2:0/70500018 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff39c107e30 0x7ff39c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:52.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.557+0000 7ff3a4535700 1 -- 172.21.15.2:0/70500018 >> 172.21.15.2:0/70500018 conn(0x7ff39c078e00 msgr2=0x7ff39c079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:52.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.557+0000 7ff3a4535700 1 -- 172.21.15.2:0/70500018 shutdown_connections 2024-05-14T10:01:52.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.557+0000 7ff3a4535700 1 -- 172.21.15.2:0/70500018 wait complete. 2024-05-14T10:01:52.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.558+0000 7ff3a4535700 1 Processor -- start 2024-05-14T10:01:52.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.558+0000 7ff3a4535700 1 -- start start 2024-05-14T10:01:52.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.558+0000 7ff3a4535700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff39c103550 0x7ff39c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:52.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.558+0000 7ff3a4535700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff39c107e30 0x7ff39c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:52.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.558+0000 7ff3a4535700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff39c108780 0x7ff39c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:52.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.558+0000 7ff3a4535700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c114500 con 0x7ff39c107e30 2024-05-14T10:01:52.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3a4535700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c114380 con 0x7ff39c108780 2024-05-14T10:01:52.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3a4535700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c114680 con 0x7ff39c103550 2024-05-14T10:01:52.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3a1ad0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff39c107e30 0x7ff39c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:52.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3a1ad0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff39c107e30 0x7ff39c19b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60180/0 (socket says 172.21.15.2:60180) 2024-05-14T10:01:52.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3a1ad0700 1 -- 172.21.15.2:0/605619838 learned_addr learned my addr 172.21.15.2:0/605619838 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T10:01:52.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3a22d1700 1 --2- 172.21.15.2:0/605619838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff39c103550 0x7ff39c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:52.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3a2ad2700 1 --2- 172.21.15.2:0/605619838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff39c108780 0x7ff39c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:52.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3a1ad0700 1 -- 172.21.15.2:0/605619838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff39c103550 msgr2=0x7ff39c19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:52.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3a1ad0700 1 --2- 172.21.15.2:0/605619838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff39c103550 0x7ff39c19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3a1ad0700 1 -- 172.21.15.2:0/605619838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff39c108780 msgr2=0x7ff39c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3a1ad0700 1 --2- 172.21.15.2:0/605619838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff39c108780 0x7ff39c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3a1ad0700 1 -- 172.21.15.2:0/605619838 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39c19fd20 con 0x7ff39c107e30 2024-05-14T10:01:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3a2ad2700 1 --2- 172.21.15.2:0/605619838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff39c108780 0x7ff39c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T10:01:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3a1ad0700 1 --2- 172.21.15.2:0/605619838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff39c107e30 0x7ff39c19b3f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00d980 tx=0x7ff38c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3937fe700 1 -- 172.21.15.2:0/605619838 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c017070 con 0x7ff39c107e30 2024-05-14T10:01:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.559+0000 7ff3a4535700 1 -- 172.21.15.2:0/605619838 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39c1a0010 con 0x7ff39c107e30 2024-05-14T10:01:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.560+0000 7ff3937fe700 1 -- 172.21.15.2:0/605619838 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff38c009de0 con 0x7ff39c107e30 2024-05-14T10:01:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.560+0000 7ff3937fe700 1 -- 172.21.15.2:0/605619838 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00f3c0 con 0x7ff39c107e30 2024-05-14T10:01:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.560+0000 7ff3a4535700 1 -- 172.21.15.2:0/605619838 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff39c1a7930 con 0x7ff39c107e30 2024-05-14T10:01:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.561+0000 7ff3a4535700 1 -- 172.21.15.2:0/605619838 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff39c0512c0 con 0x7ff39c107e30 2024-05-14T10:01:52.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.562+0000 7ff3937fe700 1 -- 172.21.15.2:0/605619838 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff38c009930 con 0x7ff39c107e30 2024-05-14T10:01:52.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.563+0000 7ff3937fe700 1 --2- 172.21.15.2:0/605619838 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3880b6c90 0x7ff3880b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T10:01:52.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.563+0000 7ff3a22d1700 1 --2- 172.21.15.2:0/605619838 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3880b6c90 0x7ff3880b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T10:01:52.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.563+0000 7ff3937fe700 1 -- 172.21.15.2:0/605619838 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8324+0+0 (secure 0 0 0) 0x7ff38c0e1dc0 con 0x7ff39c107e30 2024-05-14T10:01:52.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.563+0000 7ff3a22d1700 1 --2- 172.21.15.2:0/605619838 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3880b6c90 0x7ff3880b91a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff398000c00 tx=0x7ff39801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T10:01:52.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.568+0000 7ff3937fe700 1 -- 172.21.15.2:0/605619838 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff38c0a9a80 con 0x7ff39c107e30 2024-05-14T10:01:52.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:52.780+0000 7ff3a4535700 1 -- 172.21.15.2:0/605619838 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff39c19c1b0 con 0x7ff39c107e30 2024-05-14T10:01:52.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:52 smithi002 ceph-mon[31557]: pgmap v1237: 33 pgs: 33 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T10:01:53.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:52 smithi029 ceph-mon[36249]: pgmap v1237: 33 pgs: 33 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T10:01:53.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:52 smithi026 ceph-mon[36056]: pgmap v1237: 33 pgs: 33 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T10:01:53.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:53.735+0000 7ff3937fe700 1 -- 172.21.15.2:0/605619838 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v97) ==== 178+0+0 (secure 0 0 0) 0x7ff38c0a9380 con 0x7ff39c107e30 2024-05-14T10:01:53.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:53.740+0000 7ff3a4535700 1 -- 172.21.15.2:0/605619838 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3880b6c90 msgr2=0x7ff3880b91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:53.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:53.740+0000 7ff3a4535700 1 --2- 172.21.15.2:0/605619838 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3880b6c90 0x7ff3880b91a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff398000c00 tx=0x7ff39801a040 comp rx=0 tx=0).stop 2024-05-14T10:01:53.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:53.740+0000 7ff3a4535700 1 -- 172.21.15.2:0/605619838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff39c107e30 msgr2=0x7ff39c19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T10:01:53.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:53.740+0000 7ff3a4535700 1 --2- 172.21.15.2:0/605619838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff39c107e30 0x7ff39c19b3f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00d980 tx=0x7ff38c00dc90 comp rx=0 tx=0).stop 2024-05-14T10:01:53.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:53.740+0000 7ff3a4535700 1 -- 172.21.15.2:0/605619838 shutdown_connections 2024-05-14T10:01:53.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:53.740+0000 7ff3a4535700 1 --2- 172.21.15.2:0/605619838 >> [v2:172.21.15.2:6800/3050449557,v1:172.21.15.2:6801/3050449557] conn(0x7ff3880b6c90 0x7ff3880b91a0 secure :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff398000c00 tx=0x7ff39801a040 comp rx=0 tx=0).stop 2024-05-14T10:01:53.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:53.740+0000 7ff3a4535700 1 --2- 172.21.15.2:0/605619838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff39c108780 0x7ff39c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:53.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:53.740+0000 7ff3a4535700 1 --2- 172.21.15.2:0/605619838 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff39c103550 0x7ff39c19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:53.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:53.740+0000 7ff3a4535700 1 --2- 172.21.15.2:0/605619838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff39c107e30 0x7ff39c19b3f0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T10:01:53.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:53.740+0000 7ff3a4535700 1 -- 172.21.15.2:0/605619838 >> 172.21.15.2:0/605619838 conn(0x7ff39c078e00 msgr2=0x7ff39c1019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T10:01:53.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:53.740+0000 7ff3a4535700 1 -- 172.21.15.2:0/605619838 shutdown_connections 2024-05-14T10:01:53.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T10:01:53.740+0000 7ff3a4535700 1 -- 172.21.15.2:0/605619838 wait complete. 2024-05-14T10:01:53.742 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data_ec' removed 2024-05-14T10:01:54.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:53 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/605619838' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T10:01:54.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:53 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/605619838' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T10:01:54.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:53 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/605619838' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T10:01:54.565 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T10:01:54.593 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T10:01:54.640 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T10:01:54.681 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-14T10:01:54.681 DEBUG:teuthology.orchestra.run.smithi002:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-14T10:01:54.697 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-14T10:01:54.713 DEBUG:teuthology.orchestra.run.smithi029:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-14T10:01:54.737 INFO:tasks.cephadm:Stopping all daemons... 2024-05-14T10:01:54.737 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-14T10:01:54.737 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.a 2024-05-14T10:01:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:54 smithi002 ceph-mon[31557]: pgmap v1238: 33 pgs: 33 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T10:01:54.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:54 smithi002 ceph-mon[31557]: from='client.? 172.21.15.2:0/605619838' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T10:01:54.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:54 smithi002 ceph-mon[31557]: osdmap e97: 12 total, 12 up, 12 in 2024-05-14T10:01:54.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:54 smithi002 systemd[1]: Stopping Ceph mon.a for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:01:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:54 smithi029 ceph-mon[36249]: pgmap v1238: 33 pgs: 33 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T10:01:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:54 smithi029 ceph-mon[36249]: from='client.? 172.21.15.2:0/605619838' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T10:01:54.937 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:54 smithi029 ceph-mon[36249]: osdmap e97: 12 total, 12 up, 12 in 2024-05-14T10:01:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:54 smithi026 ceph-mon[36056]: pgmap v1238: 33 pgs: 33 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T10:01:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:54 smithi026 ceph-mon[36056]: from='client.? 172.21.15.2:0/605619838' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T10:01:54.951 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:54 smithi026 ceph-mon[36056]: osdmap e97: 12 total, 12 up, 12 in 2024-05-14T10:01:55.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:55 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a[31553]: 2024-05-14T10:01:55.285+0000 7f82b2a1b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:01:55.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:55 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a[31553]: 2024-05-14T10:01:55.285+0000 7f82b2a1b700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-14T10:01:55.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:55 smithi002 podman[108045]: 2024-05-14 10:01:55.367906027 +0000 UTC m=+0.435835118 container died 8434b6d71ae2c7dbb71fc1bfc2e5e96eb681516c5a81898881b8009d71a7e18d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-14T10:01:56.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:55 smithi002 podman[108045]: 2024-05-14 10:01:55.968786359 +0000 UTC m=+1.036715444 container cleanup 8434b6d71ae2c7dbb71fc1bfc2e5e96eb681516c5a81898881b8009d71a7e18d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T10:01:56.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 10:01:55 smithi002 bash[108045]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-a 2024-05-14T10:01:56.399 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.a.service' 2024-05-14T10:01:56.452 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:01:56.452 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-14T10:01:56.452 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-14T10:01:56.453 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.b 2024-05-14T10:01:56.833 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:56 smithi026 systemd[1]: Stopping Ceph mon.b for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:01:57.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:56 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-b[36052]: 2024-05-14T10:01:56.832+0000 7fb2001dc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:01:57.201 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:56 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-b[36052]: 2024-05-14T10:01:56.832+0000 7fb2001dc700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-14T10:01:57.202 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:56 smithi026 podman[77945]: 2024-05-14 10:01:56.966442209 +0000 UTC m=+0.258451762 container died a44b30303c5ae2dca3a01ff65ee13b96cff9ae70e6c885e19a827f6434bbcf6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-14T10:01:57.634 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:57 smithi026 podman[77945]: 2024-05-14 10:01:57.375497834 +0000 UTC m=+0.667507383 container cleanup a44b30303c5ae2dca3a01ff65ee13b96cff9ae70e6c885e19a827f6434bbcf6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-14T10:01:57.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:57 smithi026 bash[77945]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-b 2024-05-14T10:01:57.868 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.b.service' 2024-05-14T10:01:57.890 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:57 smithi026 podman[77958]: 2024-05-14 10:01:57.635588125 +0000 UTC m=+0.673215318 container remove a44b30303c5ae2dca3a01ff65ee13b96cff9ae70e6c885e19a827f6434bbcf6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-b, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T10:01:57.890 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:57 smithi026 systemd[1]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.b.service: Deactivated successfully. 2024-05-14T10:01:57.890 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:57 smithi026 systemd[1]: Stopped Ceph mon.b for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T10:01:57.890 INFO:journalctl@ceph.mon.b.smithi026.stdout:May 14 10:01:57 smithi026 systemd[1]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.b.service: Consumed 44.271s CPU time. 2024-05-14T10:01:57.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:01:57.911 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-14T10:01:57.912 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-14T10:01:57.912 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.c 2024-05-14T10:01:58.285 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:57 smithi029 systemd[1]: Stopping Ceph mon.c for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:01:58.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:58 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-c[36234]: 2024-05-14T10:01:58.283+0000 7f71e2dcc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:01:58.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:58 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-c[36234]: 2024-05-14T10:01:58.283+0000 7f71e2dcc700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-14T10:01:58.687 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:58 smithi029 podman[78575]: 2024-05-14 10:01:58.519812462 +0000 UTC m=+0.357533987 container died d42860fea015e86df8dd2f51b1382b69ad8d6dea7bad53b44ace96d225e993c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-14T10:01:59.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:58 smithi029 podman[78575]: 2024-05-14 10:01:58.941912084 +0000 UTC m=+0.779633601 container cleanup d42860fea015e86df8dd2f51b1382b69ad8d6dea7bad53b44ace96d225e993c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-c, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T10:01:59.187 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:58 smithi029 bash[78575]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-c 2024-05-14T10:01:59.496 INFO:journalctl@ceph.mon.c.smithi029.stdout:May 14 10:01:59 smithi029 podman[78587]: 2024-05-14 10:01:59.23750708 +0000 UTC m=+0.712671080 container remove d42860fea015e86df8dd2f51b1382b69ad8d6dea7bad53b44ace96d225e993c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-14T10:01:59.501 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mon.c.service' 2024-05-14T10:01:59.555 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:01:59.555 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-14T10:01:59.555 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-14T10:01:59.555 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mgr.x 2024-05-14T10:01:59.873 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 10:01:59 smithi002 systemd[1]: Stopping Ceph mgr.x for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:02:00.172 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 10:01:59 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T10:01:59.873+0000 7f1a36ed7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:02:00.172 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 10:01:59 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x[31782]: 2024-05-14T10:01:59.873+0000 7f1a36ed7700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-14T10:02:00.172 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 10:01:59 smithi002 podman[108167]: 2024-05-14 10:01:59.904855301 +0000 UTC m=+0.147952319 container died 50319b871636f7327a4ac2b6fd4bf6f5ad7e3aa1d41a439c7c676e6332d83904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T10:02:00.672 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 10:02:00 smithi002 podman[108167]: 2024-05-14 10:02:00.366103496 +0000 UTC m=+0.609200508 container cleanup 50319b871636f7327a4ac2b6fd4bf6f5ad7e3aa1d41a439c7c676e6332d83904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2) 2024-05-14T10:02:00.672 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 10:02:00 smithi002 bash[108167]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-x 2024-05-14T10:02:00.842 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-14T10:02:00.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:02:00.882 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-14T10:02:00.882 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-14T10:02:00.882 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mgr.y 2024-05-14T10:02:01.202 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 10:02:00 smithi026 systemd[1]: Stopping Ceph mgr.y for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:02:01.490 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 10:02:01 smithi026 podman[78065]: 2024-05-14 10:02:01.219862714 +0000 UTC m=+0.138379310 container died 3a307e8b7787fbb512be24aade2bd243edefb6965d0847a2a4ae09903bf0791f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T10:02:01.865 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 10:02:01 smithi026 podman[78065]: 2024-05-14 10:02:01.597620239 +0000 UTC m=+0.516136841 container cleanup 3a307e8b7787fbb512be24aade2bd243edefb6965d0847a2a4ae09903bf0791f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-05-14T10:02:01.865 INFO:journalctl@ceph.mgr.y.smithi026.stdout:May 14 10:02:01 smithi026 bash[78065]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-y 2024-05-14T10:02:02.030 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-14T10:02:02.065 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:02:02.065 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-14T10:02:02.066 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-14T10:02:02.066 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mgr.z 2024-05-14T10:02:02.412 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 10:02:02 smithi029 systemd[1]: Stopping Ceph mgr.z for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:02:02.687 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 10:02:02 smithi029 podman[78699]: 2024-05-14 10:02:02.413617828 +0000 UTC m=+0.143222734 container died aeefc0356c8a7ded73ee6430df419b01caec3ad5381f757639dafdddcfde8428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-14T10:02:03.091 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 10:02:02 smithi029 podman[78699]: 2024-05-14 10:02:02.831315929 +0000 UTC m=+0.560920837 container cleanup aeefc0356c8a7ded73ee6430df419b01caec3ad5381f757639dafdddcfde8428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-05-14T10:02:03.091 INFO:journalctl@ceph.mgr.z.smithi029.stdout:May 14 10:02:02 smithi029 bash[78699]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mgr-z 2024-05-14T10:02:03.259 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-14T10:02:03.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:02:03.294 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-14T10:02:03.294 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-14T10:02:03.294 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.0 2024-05-14T10:02:03.613 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 10:02:03 smithi002 systemd[1]: Stopping Ceph osd.0 for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:02:03.921 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 10:02:03 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-0[43457]: 2024-05-14T10:02:03.612+0000 7f79bd1c5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:02:03.921 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 10:02:03 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-0[43457]: 2024-05-14T10:02:03.612+0000 7f79bd1c5700 -1 osd.0 97 *** Got signal Terminated *** 2024-05-14T10:02:03.921 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 10:02:03 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-0[43457]: 2024-05-14T10:02:03.612+0000 7f79bd1c5700 -1 osd.0 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T10:02:09.421 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 10:02:09 smithi002 podman[108300]: 2024-05-14 10:02:09.017602347 +0000 UTC m=+5.519540506 container died 907f0fb0714e66a3db48fc0cbdef71f267e4fa212780a91185d64ad0dc9204bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-14T10:02:09.797 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 10:02:09 smithi002 podman[108300]: 2024-05-14 10:02:09.538284474 +0000 UTC m=+6.040222653 container cleanup 907f0fb0714e66a3db48fc0cbdef71f267e4fa212780a91185d64ad0dc9204bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-14T10:02:09.798 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 10:02:09 smithi002 bash[108300]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-0 2024-05-14T10:02:10.125 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 10:02:09 smithi002 podman[108318]: 2024-05-14 10:02:09.798048883 +0000 UTC m=+0.766989523 container remove 907f0fb0714e66a3db48fc0cbdef71f267e4fa212780a91185d64ad0dc9204bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507) 2024-05-14T10:02:10.384 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 10:02:10 smithi002 podman[108411]: 2024-05-14 10:02:10.029089231 +0000 UTC m=+0.018119139 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T10:02:10.384 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 10:02:10 smithi002 podman[108411]: 2024-05-14 10:02:10.151405832 +0000 UTC m=+0.140435769 container create 958e03a6f13122cfc03818a09096166c415182b68be077bef7ef6d62d7effaa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-14T10:02:10.664 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 10:02:10 smithi002 podman[108411]: 2024-05-14 10:02:10.38492737 +0000 UTC m=+0.373957257 container init 958e03a6f13122cfc03818a09096166c415182b68be077bef7ef6d62d7effaa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=) 2024-05-14T10:02:10.664 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 10:02:10 smithi002 podman[108411]: 2024-05-14 10:02:10.388379957 +0000 UTC m=+0.377409856 container start 958e03a6f13122cfc03818a09096166c415182b68be077bef7ef6d62d7effaa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T10:02:10.664 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 10:02:10 smithi002 podman[108411]: 2024-05-14 10:02:10.476735473 +0000 UTC m=+0.465765384 container attach 958e03a6f13122cfc03818a09096166c415182b68be077bef7ef6d62d7effaa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-14T10:02:10.921 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 10:02:10 smithi002 podman[108411]: 2024-05-14 10:02:10.710749502 +0000 UTC m=+0.699779404 container died 958e03a6f13122cfc03818a09096166c415182b68be077bef7ef6d62d7effaa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:02:11.414 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.0.service' 2024-05-14T10:02:11.456 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:02:11.457 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-14T10:02:11.457 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-14T10:02:11.457 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.1 2024-05-14T10:02:11.805 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 10:02:11 smithi026 systemd[1]: Stopping Ceph osd.1 for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:02:12.201 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 10:02:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-1[40621]: 2024-05-14T10:02:11.805+0000 7fdacf790700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:02:12.201 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 10:02:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-1[40621]: 2024-05-14T10:02:11.805+0000 7fdacf790700 -1 osd.1 97 *** Got signal Terminated *** 2024-05-14T10:02:12.202 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 10:02:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-1[40621]: 2024-05-14T10:02:11.805+0000 7fdacf790700 -1 osd.1 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T10:02:17.686 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 10:02:17 smithi026 podman[78189]: 2024-05-14 10:02:17.408339664 +0000 UTC m=+5.747719704 container died 38b8eef45a4c9e93ab38c071e914e5ded5dcafb47abcc5ad93107fa1ba7ddf38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-1, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-14T10:02:17.951 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 10:02:17 smithi026 podman[78189]: 2024-05-14 10:02:17.808212461 +0000 UTC m=+6.147592456 container cleanup 38b8eef45a4c9e93ab38c071e914e5ded5dcafb47abcc5ad93107fa1ba7ddf38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-1, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-14T10:02:17.951 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 10:02:17 smithi026 bash[78189]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-1 2024-05-14T10:02:18.409 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 10:02:18 smithi026 podman[78204]: 2024-05-14 10:02:18.087641172 +0000 UTC m=+0.673000919 container remove 38b8eef45a4c9e93ab38c071e914e5ded5dcafb47abcc5ad93107fa1ba7ddf38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-14T10:02:18.666 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 10:02:18 smithi026 podman[78294]: 2024-05-14 10:02:18.312636689 +0000 UTC m=+0.017808685 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T10:02:18.667 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 10:02:18 smithi026 podman[78294]: 2024-05-14 10:02:18.438519153 +0000 UTC m=+0.143691142 container create b9f45f42bfe02d68bfd4cee3150629ad100fddc7252c8b6867c1a97f7e16190e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True) 2024-05-14T10:02:18.667 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 10:02:18 smithi026 podman[78294]: 2024-05-14 10:02:18.667362041 +0000 UTC m=+0.372534029 container init b9f45f42bfe02d68bfd4cee3150629ad100fddc7252c8b6867c1a97f7e16190e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:02:18.952 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 10:02:18 smithi026 podman[78294]: 2024-05-14 10:02:18.671099373 +0000 UTC m=+0.376271361 container start b9f45f42bfe02d68bfd4cee3150629ad100fddc7252c8b6867c1a97f7e16190e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T10:02:18.952 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 10:02:18 smithi026 podman[78294]: 2024-05-14 10:02:18.76659195 +0000 UTC m=+0.471763947 container attach b9f45f42bfe02d68bfd4cee3150629ad100fddc7252c8b6867c1a97f7e16190e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T10:02:19.328 INFO:journalctl@ceph.osd.1.smithi026.stdout:May 14 10:02:19 smithi026 podman[78294]: 2024-05-14 10:02:19.04286219 +0000 UTC m=+0.748034186 container died b9f45f42bfe02d68bfd4cee3150629ad100fddc7252c8b6867c1a97f7e16190e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:02:19.689 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.1.service' 2024-05-14T10:02:19.729 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:02:19.729 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-14T10:02:19.729 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-14T10:02:19.729 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.2 2024-05-14T10:02:20.058 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 10:02:19 smithi029 systemd[1]: Stopping Ceph osd.2 for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:02:20.437 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 10:02:20 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-2[41339]: 2024-05-14T10:02:20.056+0000 7f65dbb4a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:02:20.437 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 10:02:20 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-2[41339]: 2024-05-14T10:02:20.056+0000 7f65dbb4a700 -1 osd.2 97 *** Got signal Terminated *** 2024-05-14T10:02:20.437 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 10:02:20 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-2[41339]: 2024-05-14T10:02:20.056+0000 7f65dbb4a700 -1 osd.2 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T10:02:25.911 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 10:02:25 smithi029 podman[78835]: 2024-05-14 10:02:25.61455302 +0000 UTC m=+5.682410897 container died a13f7cc56956be907f84cca05ccc309fffa722eb34427e621f09daed2558fa89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-14T10:02:26.187 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 10:02:26 smithi029 podman[78835]: 2024-05-14 10:02:26.011373594 +0000 UTC m=+6.079231467 container cleanup a13f7cc56956be907f84cca05ccc309fffa722eb34427e621f09daed2558fa89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-14T10:02:26.187 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 10:02:26 smithi029 bash[78835]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-2 2024-05-14T10:02:26.687 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 10:02:26 smithi029 podman[78852]: 2024-05-14 10:02:26.266323243 +0000 UTC m=+0.650500116 container remove a13f7cc56956be907f84cca05ccc309fffa722eb34427e621f09daed2558fa89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-14T10:02:26.944 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 10:02:26 smithi029 podman[78941]: 2024-05-14 10:02:26.596872318 +0000 UTC m=+0.017910834 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T10:02:26.944 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 10:02:26 smithi029 podman[78941]: 2024-05-14 10:02:26.709849263 +0000 UTC m=+0.130887760 container create 76251f281189c746969e11a2c9bb2f8135e86f908326a0946578c9a882c3ad71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-2-deactivate, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:02:26.944 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 10:02:26 smithi029 podman[78941]: 2024-05-14 10:02:26.941703525 +0000 UTC m=+0.362742033 container init 76251f281189c746969e11a2c9bb2f8135e86f908326a0946578c9a882c3ad71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-14T10:02:26.944 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 10:02:26 smithi029 podman[78941]: 2024-05-14 10:02:26.945101289 +0000 UTC m=+0.366139801 container start 76251f281189c746969e11a2c9bb2f8135e86f908326a0946578c9a882c3ad71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-14T10:02:27.322 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 10:02:27 smithi029 podman[78941]: 2024-05-14 10:02:27.050465956 +0000 UTC m=+0.471504473 container attach 76251f281189c746969e11a2c9bb2f8135e86f908326a0946578c9a882c3ad71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-14T10:02:27.610 INFO:journalctl@ceph.osd.2.smithi029.stdout:May 14 10:02:27 smithi029 podman[78941]: 2024-05-14 10:02:27.323045616 +0000 UTC m=+0.744084126 container died 76251f281189c746969e11a2c9bb2f8135e86f908326a0946578c9a882c3ad71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-14T10:02:27.931 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.2.service' 2024-05-14T10:02:27.971 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:02:27.971 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-14T10:02:27.971 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-14T10:02:27.971 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.3 2024-05-14T10:02:28.294 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 10:02:28 smithi002 systemd[1]: Stopping Ceph osd.3 for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:02:28.671 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 10:02:28 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-3[50388]: 2024-05-14T10:02:28.293+0000 7f49926fd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:02:28.671 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 10:02:28 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-3[50388]: 2024-05-14T10:02:28.293+0000 7f49926fd700 -1 osd.3 97 *** Got signal Terminated *** 2024-05-14T10:02:28.671 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 10:02:28 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-3[50388]: 2024-05-14T10:02:28.293+0000 7f49926fd700 -1 osd.3 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T10:02:29.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:29 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:29.251+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:29.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:29 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:29.536+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:30.474 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:30 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:30.187+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:30.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:30 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:30.270+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:30.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:30 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:30.550+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:31.490 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:31 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:31.195+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:31.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:31 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:31.290+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:31.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:31 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:31.599+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:31.938 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:31 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:31.470+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:32.421 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:32 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:32.169+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:32.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:32 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:32.246+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:32.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:32 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:32.556+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:32.937 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:32 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:32.469+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:33.421 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:33 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:33.167+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:33.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:33 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:33.202+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:33.937 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:33 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:33.477+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:33.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:33 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:33.585+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:34.278 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:34 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:34.200+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:34.278 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 10:02:33 smithi002 podman[108528]: 2024-05-14 10:02:33.968750725 +0000 UTC m=+5.793883184 container died e545f915774a4b73dafe3a15f998c981848dcfc0ed2b5a0428d411ee1d35670d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-05-14T10:02:34.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:34 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:34.193+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:34.645 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 10:02:34 smithi002 podman[108528]: 2024-05-14 10:02:34.394320976 +0000 UTC m=+6.219453428 container cleanup e545f915774a4b73dafe3a15f998c981848dcfc0ed2b5a0428d411ee1d35670d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:02:34.645 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 10:02:34 smithi002 bash[108528]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-3 2024-05-14T10:02:34.687 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:34 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:34.429+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:34.687 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:34 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:34.593+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:34.921 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 10:02:34 smithi002 podman[108555]: 2024-05-14 10:02:34.645973687 +0000 UTC m=+0.668370597 container remove e545f915774a4b73dafe3a15f998c981848dcfc0ed2b5a0428d411ee1d35670d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-14T10:02:35.398 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:35 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:35.207+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:35.399 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 10:02:35 smithi002 podman[108646]: 2024-05-14 10:02:34.969100816 +0000 UTC m=+0.018367061 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T10:02:35.399 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 10:02:35 smithi002 podman[108646]: 2024-05-14 10:02:35.071886202 +0000 UTC m=+0.121152451 container create ea236cdf78d71f9f2110796ae4acedfdea88f2f83460dce08aeabd35a96b8f25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-14T10:02:35.399 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 10:02:35 smithi002 podman[108646]: 2024-05-14 10:02:35.299349506 +0000 UTC m=+0.348615737 container init ea236cdf78d71f9f2110796ae4acedfdea88f2f83460dce08aeabd35a96b8f25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-14T10:02:35.399 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 10:02:35 smithi002 podman[108646]: 2024-05-14 10:02:35.30272642 +0000 UTC m=+0.351992654 container start ea236cdf78d71f9f2110796ae4acedfdea88f2f83460dce08aeabd35a96b8f25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=) 2024-05-14T10:02:35.399 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 10:02:35 smithi002 podman[108646]: 2024-05-14 10:02:35.399273966 +0000 UTC m=+0.448540204 container attach ea236cdf78d71f9f2110796ae4acedfdea88f2f83460dce08aeabd35a96b8f25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-14T10:02:35.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:35 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:35.163+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:35.671 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 10:02:35 smithi002 podman[108646]: 2024-05-14 10:02:35.620885327 +0000 UTC m=+0.670151570 container died ea236cdf78d71f9f2110796ae4acedfdea88f2f83460dce08aeabd35a96b8f25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-14T10:02:35.937 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:35 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:35.469+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:35.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:35 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:35.597+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:36.231 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.3.service' 2024-05-14T10:02:36.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:02:36.277 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-14T10:02:36.277 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-14T10:02:36.277 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.4 2024-05-14T10:02:36.451 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:36 smithi026 systemd[1]: Stopping Ceph osd.4 for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:02:36.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:36 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:36.135+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:36.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:36 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:36.225+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:36.937 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:36 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:36.443+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:36.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:36 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:36.615+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:36.951 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:36 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4[47536]: 2024-05-14T10:02:36.606+0000 7f0a9b875700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:02:36.951 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:36 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4[47536]: 2024-05-14T10:02:36.606+0000 7f0a9b875700 -1 osd.4 97 *** Got signal Terminated *** 2024-05-14T10:02:36.951 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:36 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4[47536]: 2024-05-14T10:02:36.606+0000 7f0a9b875700 -1 osd.4 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T10:02:37.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:37 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:37.160+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:37.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:37 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:37.216+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:37.687 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:37 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:37.398+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:37.687 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:37 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:37.601+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:38.421 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:38 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:38.262+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:38.422 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:38 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:38.126+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:38.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:38 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:38.117+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:38.452 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:38 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:38.296+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:38.687 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:38.424+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:38.687 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:38.583+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:39.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:39 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:39.118+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:39.421 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:39 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:39.230+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:39.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:39 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:39.230+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:39.451 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:39 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:39.263+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:39.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:39 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:39.115+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:39.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:39 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:39.115+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:39.687 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:39 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:39.384+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:39.687 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:39 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:39.598+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:39.951 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:39 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4[47536]: 2024-05-14T10:02:39.529+0000 7f0a97e2b700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.017441+0000 front 2024-05-14T10:02:16.017470+0000 (oldest deadline 2024-05-14T10:02:38.917567+0000) 2024-05-14T10:02:40.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:40 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:40.141+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:40.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:40 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:40.266+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:40.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:40 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:40.266+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:40.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:40 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:40.099+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:40.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:40 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:40.099+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:40.452 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:40 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:40.232+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:40.687 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:40 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:40.608+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:40.687 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:40 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:40.433+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:40.688 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:40 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:40.433+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.505082+0000 front 2024-05-14T10:02:16.505053+0000 (oldest deadline 2024-05-14T10:02:39.405104+0000) 2024-05-14T10:02:40.688 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:40 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:40.636+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:40.951 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:40 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4[47536]: 2024-05-14T10:02:40.488+0000 7f0a97e2b700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.017441+0000 front 2024-05-14T10:02:16.017470+0000 (oldest deadline 2024-05-14T10:02:38.917567+0000) 2024-05-14T10:02:41.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:41 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:41.154+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:41.421 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:41 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:41.306+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:41.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:41 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:41.306+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:41.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:41 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:41.140+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:41.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:41 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:41.140+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:41.451 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:41 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:41.270+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:41.687 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:41 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:41.630+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:41.687 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:41 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:41.392+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:41.687 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:41 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:41.392+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.505082+0000 front 2024-05-14T10:02:16.505053+0000 (oldest deadline 2024-05-14T10:02:39.405104+0000) 2024-05-14T10:02:41.688 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:41 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:41.624+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:41.951 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:41 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4[47536]: 2024-05-14T10:02:41.501+0000 7f0a97e2b700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.017441+0000 front 2024-05-14T10:02:16.017470+0000 (oldest deadline 2024-05-14T10:02:38.917567+0000) 2024-05-14T10:02:42.345 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:42 smithi026 podman[78411]: 2024-05-14 10:02:42.085802985 +0000 UTC m=+5.605530194 container died 7847e042ca071e9b0bdad91757199cd3dcbb946b804c5a931b7bc610ab9c3573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-05-14T10:02:42.345 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:42 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:42.295+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:42.346 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:42 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:42.181+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:42.346 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:42 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:42.181+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:42.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:42 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:42.151+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:42.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:42 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:42.300+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:42.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:42 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:42.300+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:42.687 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:42 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:42.644+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:42.687 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:42 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:42.407+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:42.687 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:42 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:42.407+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.505082+0000 front 2024-05-14T10:02:16.505053+0000 (oldest deadline 2024-05-14T10:02:39.405104+0000) 2024-05-14T10:02:42.688 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:42 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:42.584+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:42.701 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:42 smithi026 podman[78411]: 2024-05-14 10:02:42.4563447 +0000 UTC m=+5.976071885 container cleanup 7847e042ca071e9b0bdad91757199cd3dcbb946b804c5a931b7bc610ab9c3573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-14T10:02:42.702 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:42 smithi026 bash[78411]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4 2024-05-14T10:02:43.062 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:42 smithi026 podman[78429]: 2024-05-14 10:02:42.7327869 +0000 UTC m=+0.649382960 container remove 7847e042ca071e9b0bdad91757199cd3dcbb946b804c5a931b7bc610ab9c3573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-14T10:02:43.331 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:43 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:43.213+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:43.332 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:43 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:43.213+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:43.332 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:43 smithi026 podman[78519]: 2024-05-14 10:02:42.966187514 +0000 UTC m=+0.018110566 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T10:02:43.332 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:43 smithi026 podman[78519]: 2024-05-14 10:02:43.096984168 +0000 UTC m=+0.148907210 container create 0c03e571a2efa8792197f7352ef19321e23f323c1771470ea631dad4a4aaebec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4-deactivate, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-14T10:02:43.421 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:43 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:43.275+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:43.421 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:43 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:43.275+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:43.422 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:43 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:43.159+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:43.659 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:43 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:43.332+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:43.660 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:43 smithi026 podman[78519]: 2024-05-14 10:02:43.353602043 +0000 UTC m=+0.405525104 container init 0c03e571a2efa8792197f7352ef19321e23f323c1771470ea631dad4a4aaebec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-14T10:02:43.660 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:43 smithi026 podman[78519]: 2024-05-14 10:02:43.357506811 +0000 UTC m=+0.409429854 container start 0c03e571a2efa8792197f7352ef19321e23f323c1771470ea631dad4a4aaebec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T10:02:43.660 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:43 smithi026 podman[78519]: 2024-05-14 10:02:43.428347254 +0000 UTC m=+0.480270317 container attach 0c03e571a2efa8792197f7352ef19321e23f323c1771470ea631dad4a4aaebec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-14T10:02:43.680 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:43 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:43.535+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:43.681 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:43 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:43.411+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:43.681 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:43 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:43.411+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.505082+0000 front 2024-05-14T10:02:16.505053+0000 (oldest deadline 2024-05-14T10:02:39.405104+0000) 2024-05-14T10:02:43.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:43 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:43.681+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:43.951 INFO:journalctl@ceph.osd.4.smithi026.stdout:May 14 10:02:43 smithi026 podman[78519]: 2024-05-14 10:02:43.660642238 +0000 UTC m=+0.712565286 container died 0c03e571a2efa8792197f7352ef19321e23f323c1771470ea631dad4a4aaebec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:02:44.281 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:44 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:44.176+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:44.282 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:44 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:44.176+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:44.303 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.4.service' 2024-05-14T10:02:44.345 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:02:44.345 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-14T10:02:44.345 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-14T10:02:44.345 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.5 2024-05-14T10:02:44.636 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:44 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:44.542+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:44.637 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:44 smithi029 systemd[1]: Stopping Ceph osd.5 for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:02:44.637 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:44 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:44.451+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:44.637 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:44 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:44.451+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.505082+0000 front 2024-05-14T10:02:16.505053+0000 (oldest deadline 2024-05-14T10:02:39.405104+0000) 2024-05-14T10:02:44.671 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:44 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:44.198+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:44.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:44 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:44.313+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:44.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:44 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:44.313+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:44.701 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:44 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:44.329+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:44.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:44 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:44.637+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:44.937 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:44 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:44.701+0000 7f2c9c079700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:02:44.937 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:44 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:44.701+0000 7f2c9c079700 -1 osd.5 97 *** Got signal Terminated *** 2024-05-14T10:02:44.937 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:44 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:44.701+0000 7f2c9c079700 -1 osd.5 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T10:02:45.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:45 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:45.154+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:45.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:45 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:45.295+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:45.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:45 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:45.295+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:45.451 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:45 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:45.341+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:45.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:45 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:45.187+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:45.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:45 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:45.187+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:45.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:45 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:45.650+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:45.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:45 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:45.505+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:45.938 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:45 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:45.450+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:45.938 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:45 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:45.450+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.505082+0000 front 2024-05-14T10:02:16.505053+0000 (oldest deadline 2024-05-14T10:02:39.405104+0000) 2024-05-14T10:02:46.451 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:46 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:46.332+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:46.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:46 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:46.180+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:46.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:46 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:46.180+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:46.671 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:46 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:46.187+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:46.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:46 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:46.335+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:46.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:46 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:46.335+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:46.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:46 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:46.636+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:46.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:46 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:46.499+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:46.938 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:46 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:46.475+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:46.938 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:46 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:46.475+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.505082+0000 front 2024-05-14T10:02:16.505053+0000 (oldest deadline 2024-05-14T10:02:39.405104+0000) 2024-05-14T10:02:47.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:47 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:47.140+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:47.422 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:47 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:47.140+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:02:47.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:47 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:47.317+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:47.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:47 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:47.317+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:47.646 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:47 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:47.205+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:47.646 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:47 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:47.205+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:47.646 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:47 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:47.310+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:47.646 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:47 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:47.310+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:02:47.687 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:47 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:47.635+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:47.687 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:47 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:47.540+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:47.688 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:47 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:47.433+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:47.688 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:47 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:47.433+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.505082+0000 front 2024-05-14T10:02:16.505053+0000 (oldest deadline 2024-05-14T10:02:39.405104+0000) 2024-05-14T10:02:48.421 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:48 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:48.288+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:48.421 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:48 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:48.288+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:48.422 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:48 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:48.103+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:48.422 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:48 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:48.103+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:02:48.451 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:48 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:48.291+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:48.451 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:48 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:48.291+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:02:48.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:48 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:48.184+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:48.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:48 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:48.184+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:48.687 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:48 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:48.626+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:48.687 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:48 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:48.546+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:48.688 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:48 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:48.405+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:48.688 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:48 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:48.405+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.505082+0000 front 2024-05-14T10:02:16.505053+0000 (oldest deadline 2024-05-14T10:02:39.405104+0000) 2024-05-14T10:02:48.688 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:48 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:48.405+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.106249+0000 front 2024-05-14T10:02:24.106283+0000 (oldest deadline 2024-05-14T10:02:48.206464+0000) 2024-05-14T10:02:49.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:49 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:49.089+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:49.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:49 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:49.089+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:02:49.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:49 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:49.297+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:49.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:49 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:49.297+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:49.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:49 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:49.297+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:02:49.451 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:49 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:49.266+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:49.452 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:49 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:49.266+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:02:49.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:49 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:49.193+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:49.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:49 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:49.193+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:49.687 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:49 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:49.579+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:49.687 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:49 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:49.539+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:49.688 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:49 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:49.539+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:02:49.688 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:49 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:49.421+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:08.403639+0000 front 2024-05-14T10:02:08.403541+0000 (oldest deadline 2024-05-14T10:02:31.303353+0000) 2024-05-14T10:02:49.688 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:49 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:49.421+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.505082+0000 front 2024-05-14T10:02:16.505053+0000 (oldest deadline 2024-05-14T10:02:39.405104+0000) 2024-05-14T10:02:49.688 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:49 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5[47804]: 2024-05-14T10:02:49.421+0000 7f2c9862f700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.106249+0000 front 2024-05-14T10:02:24.106283+0000 (oldest deadline 2024-05-14T10:02:48.206464+0000) 2024-05-14T10:02:50.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:50 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:50.092+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:50.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:50 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:50.092+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:02:50.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:50 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:50.300+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:50.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:50 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:50.300+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:50.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:50 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:50.300+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:02:50.451 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:50 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:50.300+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:50.451 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:50 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:50.300+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:02:50.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:50 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:50.184+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:50.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:50 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:50.184+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:50.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:50 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:50.184+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:02:50.687 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:50 smithi029 podman[79071]: 2024-05-14 10:02:50.334769743 +0000 UTC m=+5.777530197 container died ab73c1f396bc4e267a1aee1046d9ad57908b7b8042378f2b01a5c1560ff3db42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T10:02:50.688 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:50 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:50.556+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:50.688 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:50 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:50.556+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:02:50.688 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:50 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:50.548+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:50.688 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:50 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:50.548+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:02:51.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:51 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:51.115+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:51.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:51 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:51.115+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:02:51.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:51 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:51.276+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:51.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:51 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:51.276+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:51.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:51 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:51.276+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:02:51.451 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:51 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:51.319+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:51.452 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:51 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:51.319+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:02:51.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:51 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:51.150+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:51.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:51 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:51.150+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:51.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:51 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:51.150+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:02:51.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:51 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:51.564+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:51.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:51 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:51.564+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:02:51.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:51 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:51.585+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:51.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:51 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:51.585+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:02:52.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:52 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:52.120+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:52.422 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:52 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:52.120+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:02:52.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:52 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:52.289+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:52.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:52 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:52.289+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:52.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:52 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:52.289+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:02:52.452 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:52 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:52.326+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:52.452 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:52 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:52.326+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:02:52.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:52 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:52.195+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:52.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:52 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:52.195+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:52.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:52 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:52.195+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:02:52.924 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:52 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:52.592+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:52.924 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:52 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:52.592+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:02:52.924 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:52 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:52.607+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:52.924 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:52 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:52.607+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:02:53.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:53 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:53.157+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:53.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:53 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:53.157+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:02:53.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:53 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:53.325+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:53.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:53 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:53.325+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:53.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:53 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:53.325+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:02:53.701 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:53 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:53.368+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:53.702 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:53 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:53.368+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:02:53.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:53 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:53.212+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:53.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:53 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:53.212+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:53.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:53 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:53.212+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:02:53.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:53 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:53.554+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:53.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:53 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:53.554+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:02:53.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:53 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:53.588+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:53.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:53 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:53.588+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:02:54.369 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:54 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:54.115+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:54.369 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:54 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:54.115+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:02:54.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:54 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:54.369+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:54.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:54 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:54.369+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:54.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:54 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:54.369+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:02:54.701 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:54 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:54.400+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:54.701 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:54 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:54.400+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:02:54.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:54 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:54.213+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:54.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:54 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:54.213+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:54.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:54 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:54.213+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:02:54.874 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:54 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:54.566+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:54.874 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:54 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:54.566+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:02:54.874 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:54 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:54.588+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:54.874 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:54 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:54.588+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:02:54.874 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:54 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:54.588+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:02:55.161 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:54 smithi029 podman[79071]: 2024-05-14 10:02:54.87501576 +0000 UTC m=+10.317776196 container cleanup ab73c1f396bc4e267a1aee1046d9ad57908b7b8042378f2b01a5c1560ff3db42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:02:55.161 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:54 smithi029 bash[79071]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5 2024-05-14T10:02:55.400 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:55 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:55.137+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:55.400 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:55 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:55.137+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:02:55.437 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:55 smithi029 podman[79085]: 2024-05-14 10:02:55.161640976 +0000 UTC m=+4.821196606 container remove ab73c1f396bc4e267a1aee1046d9ad57908b7b8042378f2b01a5c1560ff3db42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-14T10:02:55.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:55 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:55.400+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:55.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:55 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:55.400+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:55.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:55 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:55.400+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:02:55.701 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:55 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:55.433+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:55.701 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:55 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:55.433+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:02:55.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:55 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:55.254+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:55.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:55 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:55.254+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:55.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:55 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:55.254+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:02:55.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:55 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:55.254+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:02:55.861 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:55 smithi029 podman[79177]: 2024-05-14 10:02:55.477373726 +0000 UTC m=+0.018098815 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T10:02:55.861 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:55 smithi029 podman[79177]: 2024-05-14 10:02:55.58881144 +0000 UTC m=+0.129536528 container create a61303c6b457048203310b1872e1c563f8d68da0a1a6fd1fcb3118fea375e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5-deactivate, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-14T10:02:55.861 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:55 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:55.561+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:55.862 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:55 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:55.561+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:02:55.862 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:55 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:55.573+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:55.862 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:55 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:55.573+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:02:55.862 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:55 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:55.573+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:02:56.164 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:55 smithi029 podman[79177]: 2024-05-14 10:02:55.86182581 +0000 UTC m=+0.402550901 container init a61303c6b457048203310b1872e1c563f8d68da0a1a6fd1fcb3118fea375e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-14T10:02:56.165 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:55 smithi029 podman[79177]: 2024-05-14 10:02:55.86526073 +0000 UTC m=+0.405985825 container start a61303c6b457048203310b1872e1c563f8d68da0a1a6fd1fcb3118fea375e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:02:56.165 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:55 smithi029 podman[79177]: 2024-05-14 10:02:55.970800245 +0000 UTC m=+0.511525346 container attach a61303c6b457048203310b1872e1c563f8d68da0a1a6fd1fcb3118fea375e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-14T10:02:56.433 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:56 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:56.181+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:56.434 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:56 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:56.181+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:02:56.434 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:56 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:56.181+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.954558+0000 front 2024-05-14T10:02:31.954634+0000 (oldest deadline 2024-05-14T10:02:56.054767+0000) 2024-05-14T10:02:56.437 INFO:journalctl@ceph.osd.5.smithi029.stdout:May 14 10:02:56 smithi029 podman[79177]: 2024-05-14 10:02:56.165410558 +0000 UTC m=+0.706135652 container died a61303c6b457048203310b1872e1c563f8d68da0a1a6fd1fcb3118fea375e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T10:02:56.701 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:56 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:56.417+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:56.702 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:56 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:56.417+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:02:56.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:56 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:56.248+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:56.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:56 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:56.248+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:56.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:56 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:56.248+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:02:56.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:56 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:56.248+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:02:56.812 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:56 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:56.560+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:56.813 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:56 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:56.560+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:02:56.813 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:56 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:56.567+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:56.813 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:56 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:56.567+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:02:56.813 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:56 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:56.567+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:02:56.835 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.5.service' 2024-05-14T10:02:56.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:02:56.875 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-14T10:02:56.875 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-14T10:02:56.875 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.6 2024-05-14T10:02:56.904 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:56 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:56.434+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:56.904 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:56 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:56.434+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:56.904 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:56 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:56.434+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:02:57.172 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:56 smithi002 systemd[1]: Stopping Ceph osd.6 for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:02:57.172 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:57.146+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:57.172 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:57.146+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:02:57.172 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:57.146+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.954558+0000 front 2024-05-14T10:02:31.954634+0000 (oldest deadline 2024-05-14T10:02:56.054767+0000) 2024-05-14T10:02:57.671 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:57.197+0000 7fb3be3c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:02:57.671 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:57.197+0000 7fb3be3c8700 -1 osd.6 97 *** Got signal Terminated *** 2024-05-14T10:02:57.671 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:57.197+0000 7fb3be3c8700 -1 osd.6 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T10:02:57.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:57.407+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:57.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:57.407+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:57.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:57.407+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:02:57.701 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:57 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:57.458+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:57.719 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:57 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:57.458+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:02:57.719 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:57 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:57.216+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:57.719 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:57 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:57.216+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:57.719 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:57 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:57.216+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:02:57.719 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:57 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:57.216+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:02:57.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:57 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:57.580+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:57.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:57 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:57.580+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:02:57.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:57 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:57.609+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:57.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:57 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:57.609+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:02:57.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:57 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:57.609+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:02:58.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:58 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:58.165+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:58.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:58 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:58.165+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:02:58.421 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:58 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:58.165+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.954558+0000 front 2024-05-14T10:02:31.954634+0000 (oldest deadline 2024-05-14T10:02:56.054767+0000) 2024-05-14T10:02:58.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:58 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:58.361+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:58.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:58 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:58.361+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:58.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:58 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:58.361+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:02:58.422 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:58 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:58.361+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:02:58.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:58 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:58.227+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:58.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:58 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:58.227+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:58.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:58 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:58.227+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:02:58.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:58 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:58.227+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:02:58.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:58 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:58.552+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:58.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:58 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:58.552+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:02:58.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:58 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:58.552+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:02:58.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:58 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:58.607+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:58.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:58 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:58.607+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:02:58.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:58 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:58.607+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:02:58.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:58 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:58.504+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:58.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:58 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:58.504+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:02:58.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:58 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:58.504+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.804562+0000 front 2024-05-14T10:02:32.804814+0000 (oldest deadline 2024-05-14T10:02:58.104552+0000) 2024-05-14T10:02:59.526 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:59 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:59.202+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:02:59.526 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:59 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:59.202+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:02:59.526 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:59 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:59.202+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:02:59.526 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:02:59 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:02:59.202+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:02:59.671 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:59 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:59.198+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:02:59.671 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:59 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:59.198+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:02:59.671 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:02:59 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:02:59.198+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.954558+0000 front 2024-05-14T10:02:31.954634+0000 (oldest deadline 2024-05-14T10:02:56.054767+0000) 2024-05-14T10:02:59.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:59 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:59.402+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:02:59.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:59 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:59.402+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:02:59.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:59 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:59.402+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:02:59.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:02:59 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:02:59.402+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:02:59.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:59 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:59.596+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:02:59.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:59 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:59.596+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:02:59.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:02:59 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:02:59.596+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:02:59.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:59 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:59.636+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:02:59.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:59 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:59.636+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:02:59.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:02:59 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:02:59.636+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:02:59.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:59 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:59.525+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:02:59.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:59 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:59.525+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:02:59.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:02:59 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:02:59.525+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.804562+0000 front 2024-05-14T10:02:32.804814+0000 (oldest deadline 2024-05-14T10:02:58.104552+0000) 2024-05-14T10:03:00.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:00 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:00.159+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:00.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:00 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:00.159+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:00.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:00 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:00.159+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:00.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:00 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:00.159+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:00.573 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:03:00.228+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:03:00.573 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:03:00.228+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:03:00.573 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:03:00.228+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.954558+0000 front 2024-05-14T10:02:31.954634+0000 (oldest deadline 2024-05-14T10:02:56.054767+0000) 2024-05-14T10:03:00.573 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:00.386+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:00.574 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:00.386+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:00.574 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:00.386+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:00.574 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:00 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:00.386+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:00.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:00 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:00.567+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:00.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:00 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:00.567+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:00.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:00 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:00.567+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:00.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:00 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:00.681+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:00.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:00 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:00.681+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:00.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:00 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:00.681+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:00.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:00 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:00.574+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:03:00.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:00 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:00.574+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:03:00.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:00 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:00.574+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.804562+0000 front 2024-05-14T10:02:32.804814+0000 (oldest deadline 2024-05-14T10:02:58.104552+0000) 2024-05-14T10:03:01.580 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:01 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:01.204+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:01.580 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:01 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:01.204+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:01.580 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:01 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:01.204+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:01.580 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:01 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:01.204+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:01.589 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:01 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:03:01.188+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:03:01.589 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:01 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:03:01.188+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:03:01.589 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:01 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:03:01.188+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.954558+0000 front 2024-05-14T10:02:31.954634+0000 (oldest deadline 2024-05-14T10:02:56.054767+0000) 2024-05-14T10:03:01.589 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:01 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:01.405+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:01.589 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:01 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:01.405+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:01.590 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:01 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:01.405+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:01.590 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:01 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:01.405+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:01.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:01 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:01.601+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:01.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:01 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:01.601+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:01.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:01 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:01.601+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:01.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:01 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:01.653+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:01.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:01 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:01.653+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:01.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:01 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:01.653+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:01.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:01 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:01.579+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:03:01.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:01 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:01.579+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:03:01.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:01 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:01.579+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.804562+0000 front 2024-05-14T10:02:32.804814+0000 (oldest deadline 2024-05-14T10:02:58.104552+0000) 2024-05-14T10:03:02.444 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:02 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:03:02.171+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.750814+0000 front 2024-05-14T10:02:15.750919+0000 (oldest deadline 2024-05-14T10:02:38.050856+0000) 2024-05-14T10:03:02.444 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:02 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:03:02.171+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:25.053233+0000 front 2024-05-14T10:02:25.053153+0000 (oldest deadline 2024-05-14T10:02:46.753162+0000) 2024-05-14T10:03:02.444 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:02 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6[57184]: 2024-05-14T10:03:02.171+0000 7fb3ba97e700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.954558+0000 front 2024-05-14T10:02:31.954634+0000 (oldest deadline 2024-05-14T10:02:56.054767+0000) 2024-05-14T10:03:02.601 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:02 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:02.247+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:02.601 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:02 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:02.247+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:02.602 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:02 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:02.247+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:02.602 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:02 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:02.247+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:02.786 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:02 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:02.442+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:02.786 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:02 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:02.442+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:02.786 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:02 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:02.442+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:02.786 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:02 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:02.442+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:02.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:02 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:02.640+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:02.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:02 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:02.640+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:02.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:02 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:02.640+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:02.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:02 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:02.620+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:02.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:02 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:02.620+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:02.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:02 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:02.620+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:02.952 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:02 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:02.601+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:03:02.952 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:02 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:02.601+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:03:02.952 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:02 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:02.601+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.804562+0000 front 2024-05-14T10:02:32.804814+0000 (oldest deadline 2024-05-14T10:02:58.104552+0000) 2024-05-14T10:03:03.171 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:02 smithi002 podman[108752]: 2024-05-14 10:03:02.786709689 +0000 UTC m=+5.709986201 container died c65e6f5092ab81594e5eeb4120f7c75d34d2b54260cb32b0b3bfb43571f25f33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-14T10:03:03.556 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:03 smithi002 podman[108752]: 2024-05-14 10:03:03.305931841 +0000 UTC m=+6.229208344 container cleanup c65e6f5092ab81594e5eeb4120f7c75d34d2b54260cb32b0b3bfb43571f25f33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T10:03:03.556 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:03 smithi002 bash[108752]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6 2024-05-14T10:03:03.557 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:03 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:03.447+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:03.557 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:03 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:03.447+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:03.557 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:03 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:03.447+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:03.557 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:03 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:03.447+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:03.557 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:03 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:03.447+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:03.606 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:03 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:03.255+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:03.606 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:03 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:03.255+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:03.606 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:03 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:03.255+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:03.606 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:03 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:03.255+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:03.607 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:03 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:03.255+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:03.874 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:03 smithi002 podman[108781]: 2024-05-14 10:03:03.557191872 +0000 UTC m=+0.760427079 container remove c65e6f5092ab81594e5eeb4120f7c75d34d2b54260cb32b0b3bfb43571f25f33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T10:03:03.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:03 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:03.639+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:03.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:03 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:03.639+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:03.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:03 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:03.639+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:03.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:03 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:03.639+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.496559+0000 front 2024-05-14T10:02:40.496632+0000 (oldest deadline 2024-05-14T10:03:03.396558+0000) 2024-05-14T10:03:03.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:03 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:03.617+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:03.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:03 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:03.617+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:03.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:03 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:03.617+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:03.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:03 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:03.617+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:03.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:03 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:03.606+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:03:03.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:03 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:03.606+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:03:03.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:03 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:03.606+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.804562+0000 front 2024-05-14T10:02:32.804814+0000 (oldest deadline 2024-05-14T10:02:58.104552+0000) 2024-05-14T10:03:03.952 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:03 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:03.606+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.105029+0000 front 2024-05-14T10:02:38.104999+0000 (oldest deadline 2024-05-14T10:03:03.405132+0000) 2024-05-14T10:03:04.126 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:03 smithi002 podman[108870]: 2024-05-14 10:03:03.874571737 +0000 UTC m=+0.108529488 container create f18d0e4c328eb8e9f299b4e2a83dea76f870f7968d2c26700f701d1f7fb28195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T10:03:04.126 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:03 smithi002 podman[108870]: 2024-05-14 10:03:03.783573035 +0000 UTC m=+0.017530800 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T10:03:04.126 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:04 smithi002 podman[108870]: 2024-05-14 10:03:04.051825644 +0000 UTC m=+0.285783396 container init f18d0e4c328eb8e9f299b4e2a83dea76f870f7968d2c26700f701d1f7fb28195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-14T10:03:04.126 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:04 smithi002 podman[108870]: 2024-05-14 10:03:04.055761958 +0000 UTC m=+0.289719717 container start f18d0e4c328eb8e9f299b4e2a83dea76f870f7968d2c26700f701d1f7fb28195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:03:04.126 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:04 smithi002 podman[108870]: 2024-05-14 10:03:04.126882537 +0000 UTC m=+0.360840312 container attach f18d0e4c328eb8e9f299b4e2a83dea76f870f7968d2c26700f701d1f7fb28195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-14T10:03:04.412 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 10:03:04 smithi002 podman[108870]: 2024-05-14 10:03:04.368458382 +0000 UTC m=+0.602416145 container died f18d0e4c328eb8e9f299b4e2a83dea76f870f7968d2c26700f701d1f7fb28195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-6-deactivate, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-14T10:03:04.626 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:04 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:04.304+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:04.626 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:04 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:04.304+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:04.626 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:04 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:04.304+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:04.626 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:04 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:04.304+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:04.626 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:04 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:04.304+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:04.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:04 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:04.411+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:04.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:04 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:04.411+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:04.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:04 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:04.411+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:04.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:04 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:04.411+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:04.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:04 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:04.411+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:04.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:04 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:04.674+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:04.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:04 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:04.674+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:04.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:04 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:04.674+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:04.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:04 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:04.674+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.496559+0000 front 2024-05-14T10:02:40.496632+0000 (oldest deadline 2024-05-14T10:03:03.396558+0000) 2024-05-14T10:03:04.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:04 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:04.600+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:04.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:04 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:04.600+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:04.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:04 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:04.600+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:04.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:04 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:04.600+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:04.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:04 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:04.625+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:03:04.952 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:04 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:04.625+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:03:04.952 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:04 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:04.625+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.804562+0000 front 2024-05-14T10:02:32.804814+0000 (oldest deadline 2024-05-14T10:02:58.104552+0000) 2024-05-14T10:03:04.952 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:04 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:04.625+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.105029+0000 front 2024-05-14T10:02:38.104999+0000 (oldest deadline 2024-05-14T10:03:03.405132+0000) 2024-05-14T10:03:05.059 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.6.service' 2024-05-14T10:03:05.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:03:05.148 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-14T10:03:05.148 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-14T10:03:05.148 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.7 2024-05-14T10:03:05.451 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:05 smithi026 systemd[1]: Stopping Ceph osd.7 for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:03:05.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:05 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:05.301+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:05.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:05 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:05.301+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:05.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:05 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:05.301+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:05.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:05 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:05.301+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:05.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:05 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:05.301+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:05.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:05 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:05.380+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:05.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:05 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:05.380+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:05.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:05 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:05.380+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:05.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:05 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:05.380+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:05.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:05 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:05.380+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:05.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:05 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:05.705+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:05.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:05 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:05.705+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:05.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:05 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:05.705+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:05.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:05 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:05.705+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.496559+0000 front 2024-05-14T10:02:40.496632+0000 (oldest deadline 2024-05-14T10:03:03.396558+0000) 2024-05-14T10:03:05.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:05 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:05.624+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:05.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:05 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:05.624+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:05.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:05 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:05.624+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:05.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:05 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:05.624+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:05.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:05 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:05.483+0000 7f12c1398700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:03:05.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:05 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:05.483+0000 7f12c1398700 -1 osd.7 97 *** Got signal Terminated *** 2024-05-14T10:03:05.952 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:05 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:05.483+0000 7f12c1398700 -1 osd.7 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T10:03:05.952 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:05 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:05.597+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:03:05.952 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:05 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:05.597+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:03:05.952 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:05 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:05.597+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.804562+0000 front 2024-05-14T10:02:32.804814+0000 (oldest deadline 2024-05-14T10:02:58.104552+0000) 2024-05-14T10:03:05.952 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:05 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:05.597+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.105029+0000 front 2024-05-14T10:02:38.104999+0000 (oldest deadline 2024-05-14T10:03:03.405132+0000) 2024-05-14T10:03:06.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:06 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:06.420+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:06.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:06 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:06.420+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:06.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:06 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:06.420+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:06.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:06 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:06.420+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:06.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:06 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:06.420+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:06.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:06 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:06.340+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:06.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:06 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:06.340+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:06.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:06 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:06.340+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:06.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:06 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:06.340+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:06.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:06 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:06.340+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:06.702 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:06 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:06.559+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:03:06.702 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:06 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:06.559+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:03:06.702 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:06 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:06.559+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.804562+0000 front 2024-05-14T10:02:32.804814+0000 (oldest deadline 2024-05-14T10:02:58.104552+0000) 2024-05-14T10:03:06.702 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:06 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:06.559+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.105029+0000 front 2024-05-14T10:02:38.104999+0000 (oldest deadline 2024-05-14T10:03:03.405132+0000) 2024-05-14T10:03:06.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:06 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:06.689+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:06.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:06 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:06.689+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:06.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:06 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:06.689+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:06.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:06 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:06.689+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.496559+0000 front 2024-05-14T10:02:40.496632+0000 (oldest deadline 2024-05-14T10:03:03.396558+0000) 2024-05-14T10:03:06.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:06 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:06.652+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:06.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:06 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:06.652+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:06.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:06 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:06.652+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:06.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:06 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:06.652+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:07.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:07.408+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:07.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:07.408+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:07.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:07.408+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:07.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:07.408+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:07.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:07 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:07.408+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:07.701 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:07 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:07.601+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:03:07.701 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:07 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:07.601+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:03:07.702 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:07 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:07.601+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.804562+0000 front 2024-05-14T10:02:32.804814+0000 (oldest deadline 2024-05-14T10:02:58.104552+0000) 2024-05-14T10:03:07.702 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:07 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:07.601+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.105029+0000 front 2024-05-14T10:02:38.104999+0000 (oldest deadline 2024-05-14T10:03:03.405132+0000) 2024-05-14T10:03:07.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:07 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:07.379+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:07.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:07 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:07.379+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:07.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:07 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:07.379+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:07.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:07 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:07.379+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:07.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:07 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:07.379+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:07.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:07 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:07.693+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:07.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:07 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:07.693+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:07.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:07 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:07.693+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:07.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:07 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:07.693+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.496559+0000 front 2024-05-14T10:02:40.496632+0000 (oldest deadline 2024-05-14T10:03:03.396558+0000) 2024-05-14T10:03:07.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:07 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:07.645+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:07.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:07 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:07.645+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:07.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:07 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:07.645+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:07.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:07 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:07.645+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:08.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:08 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:08.361+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:08.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:08 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:08.361+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:08.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:08 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:08.361+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:08.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:08 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:08.361+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:08.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:08 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:08.361+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:08.701 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:08 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:08.637+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:03:08.702 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:08 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:08.637+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:03:08.702 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:08 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:08.637+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.804562+0000 front 2024-05-14T10:02:32.804814+0000 (oldest deadline 2024-05-14T10:02:58.104552+0000) 2024-05-14T10:03:08.702 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:08 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:08.637+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.105029+0000 front 2024-05-14T10:02:38.104999+0000 (oldest deadline 2024-05-14T10:03:03.405132+0000) 2024-05-14T10:03:08.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:08 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:08.409+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:08.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:08 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:08.409+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:08.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:08 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:08.409+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:08.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:08 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:08.409+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:08.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:08 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:08.409+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:08.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:08 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:08.696+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:08.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:08 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:08.696+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:08.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:08 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:08.696+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:08.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:08 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:08.696+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.496559+0000 front 2024-05-14T10:02:40.496632+0000 (oldest deadline 2024-05-14T10:03:03.396558+0000) 2024-05-14T10:03:08.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:08 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:08.637+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:08.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:08 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:08.637+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:08.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:08 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:08.637+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:08.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:08 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:08.637+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:09.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:09 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:09.376+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:09.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:09 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:09.376+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:09.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:09 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:09.376+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:09.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:09 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:09.376+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:09.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:09 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:09.376+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:09.701 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:09 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:09.659+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:13.500833+0000 front 2024-05-14T10:02:13.500727+0000 (oldest deadline 2024-05-14T10:02:38.200824+0000) 2024-05-14T10:03:09.701 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:09 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:09.659+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.403076+0000 front 2024-05-14T10:02:23.403096+0000 (oldest deadline 2024-05-14T10:02:46.903143+0000) 2024-05-14T10:03:09.702 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:09 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:09.659+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.804562+0000 front 2024-05-14T10:02:32.804814+0000 (oldest deadline 2024-05-14T10:02:58.104552+0000) 2024-05-14T10:03:09.702 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:09 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7[54518]: 2024-05-14T10:03:09.659+0000 7f12b9946700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.105029+0000 front 2024-05-14T10:02:38.104999+0000 (oldest deadline 2024-05-14T10:03:03.405132+0000) 2024-05-14T10:03:09.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:09 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:09.430+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:09.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:09 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:09.430+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:09.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:09 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:09.430+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:09.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:09 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:09.430+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:09.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:09 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:09.430+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:09.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:09 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:09.709+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:09.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:09 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:09.709+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:09.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:09 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:09.709+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:09.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:09 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:09.709+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.496559+0000 front 2024-05-14T10:02:40.496632+0000 (oldest deadline 2024-05-14T10:03:03.396558+0000) 2024-05-14T10:03:09.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:09 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:09.666+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:09.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:09 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:09.666+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:09.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:09 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:09.666+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:09.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:09 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:09.666+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:10.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:10 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:10.347+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:10.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:10 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:10.347+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:10.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:10 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:10.347+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:10.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:10 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:10.347+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:10.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:10 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:10.347+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:10.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:10 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:10.749+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:10.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:10 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:10.749+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:10.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:10 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:10.749+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:10.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:10 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:10.749+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.496559+0000 front 2024-05-14T10:02:40.496632+0000 (oldest deadline 2024-05-14T10:03:03.396558+0000) 2024-05-14T10:03:10.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:10 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:10.652+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:10.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:10 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:10.652+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:10.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:10 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:10.652+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:10.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:10 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:10.652+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:10.951 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:10 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:10.460+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:10.951 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:10 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:10.460+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:10.951 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:10 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:10.460+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:10.952 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:10 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:10.460+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:10.952 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:10 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:10.460+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:11.278 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:10 smithi026 podman[78635]: 2024-05-14 10:03:10.979321472 +0000 UTC m=+5.626379169 container died 3e89494dd66055dc70f8df1738cdfb51cddff1c23ae78591247af4fcf2ec976e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-14T10:03:11.636 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:11 smithi026 podman[78635]: 2024-05-14 10:03:11.377572931 +0000 UTC m=+6.024630626 container cleanup 3e89494dd66055dc70f8df1738cdfb51cddff1c23ae78591247af4fcf2ec976e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:03:11.637 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:11 smithi026 bash[78635]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7 2024-05-14T10:03:11.637 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:11.474+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:11.637 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:11.474+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:11.637 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:11.474+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:11.637 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:11.474+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:11.637 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:11.474+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:11.637 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:11.474+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:11.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:11 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:11.344+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:11.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:11 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:11.344+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:11.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:11 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:11.344+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:11.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:11 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:11.344+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:11.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:11 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:11.344+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:11.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:11 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:11.736+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:11.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:11 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:11.736+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:11.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:11 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:11.736+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:11.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:11 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:11.736+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.496559+0000 front 2024-05-14T10:02:40.496632+0000 (oldest deadline 2024-05-14T10:03:03.396558+0000) 2024-05-14T10:03:11.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:11 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:11.640+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:11.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:11 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:11.640+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:11.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:11 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:11.640+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:11.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:11 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:11.640+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:11.951 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:11 smithi026 podman[78652]: 2024-05-14 10:03:11.637317203 +0000 UTC m=+0.661438243 container remove 3e89494dd66055dc70f8df1738cdfb51cddff1c23ae78591247af4fcf2ec976e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-14T10:03:12.240 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:11 smithi026 podman[78741]: 2024-05-14 10:03:11.862505813 +0000 UTC m=+0.017534868 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T10:03:12.240 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:11 smithi026 podman[78741]: 2024-05-14 10:03:11.979881666 +0000 UTC m=+0.134910705 container create f6b8ead2263fad0e83e013a135cabfb38365c0b1a58022f97f1eca4b871fd930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True) 2024-05-14T10:03:12.240 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:12 smithi026 podman[78741]: 2024-05-14 10:03:12.240798449 +0000 UTC m=+0.395827496 container init f6b8ead2263fad0e83e013a135cabfb38365c0b1a58022f97f1eca4b871fd930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:03:12.549 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:12 smithi026 podman[78741]: 2024-05-14 10:03:12.244285951 +0000 UTC m=+0.399315000 container start f6b8ead2263fad0e83e013a135cabfb38365c0b1a58022f97f1eca4b871fd930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-14T10:03:12.549 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:12 smithi026 podman[78741]: 2024-05-14 10:03:12.307270998 +0000 UTC m=+0.462300061 container attach f6b8ead2263fad0e83e013a135cabfb38365c0b1a58022f97f1eca4b871fd930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-14T10:03:12.549 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:12 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:12.469+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:12.550 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:12 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:12.469+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:12.550 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:12 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:12.469+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:12.550 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:12 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:12.469+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:12.550 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:12 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:12.469+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:12.550 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:12 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:12.469+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:12.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:12 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:12.386+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:12.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:12 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:12.386+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:12.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:12 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:12.386+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:12.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:12 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:12.386+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:12.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:12 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:12.386+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:12.918 INFO:journalctl@ceph.osd.7.smithi026.stdout:May 14 10:03:12 smithi026 podman[78741]: 2024-05-14 10:03:12.550256121 +0000 UTC m=+0.705285169 container died f6b8ead2263fad0e83e013a135cabfb38365c0b1a58022f97f1eca4b871fd930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240507, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-14T10:03:12.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:12 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:12.682+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:12.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:12 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:12.682+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:12.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:12 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:12.682+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:12.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:12 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:12.682+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:12.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:12 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:12.771+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:12.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:12 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:12.771+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:12.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:12 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:12.771+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:12.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:12 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:12.771+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.496559+0000 front 2024-05-14T10:02:40.496632+0000 (oldest deadline 2024-05-14T10:03:03.396558+0000) 2024-05-14T10:03:12.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:12 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:12.771+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.297521+0000 front 2024-05-14T10:02:49.297540+0000 (oldest deadline 2024-05-14T10:03:12.197700+0000) 2024-05-14T10:03:13.213 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.7.service' 2024-05-14T10:03:13.259 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:03:13.259 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-14T10:03:13.259 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-14T10:03:13.260 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.8 2024-05-14T10:03:13.437 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:13 smithi029 systemd[1]: Stopping Ceph osd.8 for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:03:13.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:13 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:13.383+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:13.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:13 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:13.383+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:13.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:13 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:13.383+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:13.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:13 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:13.383+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:13.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:13 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:13.383+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:13.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:13 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:13.442+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:13.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:13 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:13.442+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:13.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:13 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:13.442+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:13.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:13 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:13.442+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:13.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:13 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:13.442+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:13.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:13 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:13.443+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:13.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:13 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:13.584+0000 7f6823f02700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:03:13.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:13 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:13.584+0000 7f6823f02700 -1 osd.8 97 *** Got signal Terminated *** 2024-05-14T10:03:13.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:13 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:13.584+0000 7f6823f02700 -1 osd.8 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T10:03:13.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:13 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:13.769+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:13.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:13 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:13.769+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:13.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:13 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:13.769+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:13.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:13 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:13.769+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.496559+0000 front 2024-05-14T10:02:40.496632+0000 (oldest deadline 2024-05-14T10:03:03.396558+0000) 2024-05-14T10:03:13.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:13 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:13.769+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.297521+0000 front 2024-05-14T10:02:49.297540+0000 (oldest deadline 2024-05-14T10:03:12.197700+0000) 2024-05-14T10:03:13.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:13 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:13.648+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:13.939 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:13 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:13.648+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:13.939 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:13 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:13.648+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:13.939 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:13 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:13.648+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:13.939 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:13 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:13.648+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:14.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:14 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:14.341+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:14.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:14 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:14.341+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:14.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:14 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:14.341+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:14.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:14 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:14.341+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:14.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:14 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:14.341+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:14.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:14 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:14.341+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:48.290628+0000 front 2024-05-14T10:02:48.290718+0000 (oldest deadline 2024-05-14T10:03:14.190618+0000) 2024-05-14T10:03:14.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:14 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:14.439+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:14.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:14 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:14.439+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:14.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:14 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:14.439+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:14.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:14 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:14.439+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:14.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:14 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:14.439+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:14.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:14 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:14.439+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:14.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:14 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:14.753+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:14.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:14 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:14.753+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:14.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:14 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:14.753+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:14.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:14 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:14.753+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.496559+0000 front 2024-05-14T10:02:40.496632+0000 (oldest deadline 2024-05-14T10:03:03.396558+0000) 2024-05-14T10:03:14.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:14 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:14.753+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.297521+0000 front 2024-05-14T10:02:49.297540+0000 (oldest deadline 2024-05-14T10:03:12.197700+0000) 2024-05-14T10:03:14.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:14 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:14.628+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:14.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:14 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:14.628+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:14.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:14 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:14.628+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:14.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:14 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:14.628+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:14.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:14 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:14.628+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:15.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:15 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:15.312+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:15.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:15 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:15.312+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:15.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:15 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:15.312+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:15.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:15 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:15.312+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:15.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:15 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:15.312+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:15.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:15 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:15.312+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:48.290628+0000 front 2024-05-14T10:02:48.290718+0000 (oldest deadline 2024-05-14T10:03:14.190618+0000) 2024-05-14T10:03:15.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:15 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:15.451+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:15.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:15 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:15.451+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:15.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:15 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:15.451+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:15.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:15 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:15.451+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:15.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:15 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:15.451+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:15.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:15 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:15.451+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:15.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:15 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:15.716+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:15.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:15 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:15.716+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:15.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:15 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:15.716+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:15.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:15 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:15.716+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.496559+0000 front 2024-05-14T10:02:40.496632+0000 (oldest deadline 2024-05-14T10:03:03.396558+0000) 2024-05-14T10:03:15.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:15 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:15.716+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.297521+0000 front 2024-05-14T10:02:49.297540+0000 (oldest deadline 2024-05-14T10:03:12.197700+0000) 2024-05-14T10:03:15.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:15 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:15.627+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:15.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:15 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:15.627+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:15.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:15 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:15.627+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:15.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:15 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:15.627+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:15.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:15 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:15.627+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:16.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:16 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:16.269+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:16.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:16 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:16.269+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:16.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:16 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:16.269+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:16.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:16 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:16.269+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:16.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:16 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:16.269+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:16.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:16 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:16.269+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:48.290628+0000 front 2024-05-14T10:02:48.290718+0000 (oldest deadline 2024-05-14T10:03:14.190618+0000) 2024-05-14T10:03:16.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:16 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:16.432+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:16.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:16 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:16.432+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:16.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:16 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:16.432+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:16.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:16 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:16.432+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:16.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:16 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:16.432+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:16.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:16 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:16.432+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:16.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:16 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:16.668+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:16.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:16 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:16.668+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:16.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:16 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:16.668+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:16.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:16 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:16.668+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.496559+0000 front 2024-05-14T10:02:40.496632+0000 (oldest deadline 2024-05-14T10:03:03.396558+0000) 2024-05-14T10:03:16.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:16 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:16.668+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.297521+0000 front 2024-05-14T10:02:49.297540+0000 (oldest deadline 2024-05-14T10:03:12.197700+0000) 2024-05-14T10:03:16.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:16 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:16.648+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:16.939 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:16 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:16.648+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:16.939 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:16 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:16.648+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:16.939 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:16 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:16.648+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:16.939 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:16 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:16.648+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:17.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:17 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:17.248+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:17.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:17 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:17.248+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:17.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:17 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:17.248+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:17.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:17 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:17.248+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:17.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:17 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:17.248+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:17.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:17 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:17.248+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:48.290628+0000 front 2024-05-14T10:02:48.290718+0000 (oldest deadline 2024-05-14T10:03:14.190618+0000) 2024-05-14T10:03:17.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:17 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:17.449+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:17.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:17 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:17.449+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:17.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:17 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:17.449+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:17.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:17 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:17.449+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:17.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:17 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:17.449+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:17.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:17 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:17.449+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:17.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:17 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:17.621+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:15.491294+0000 front 2024-05-14T10:02:15.491362+0000 (oldest deadline 2024-05-14T10:02:40.191271+0000) 2024-05-14T10:03:17.937 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:17 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:17.621+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:21.892608+0000 front 2024-05-14T10:02:21.892637+0000 (oldest deadline 2024-05-14T10:02:50.093539+0000) 2024-05-14T10:03:17.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:17 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:17.621+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:31.794531+0000 front 2024-05-14T10:02:31.794833+0000 (oldest deadline 2024-05-14T10:02:57.694592+0000) 2024-05-14T10:03:17.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:17 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:17.621+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.496559+0000 front 2024-05-14T10:02:40.496632+0000 (oldest deadline 2024-05-14T10:03:03.396558+0000) 2024-05-14T10:03:17.938 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:17 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8[54486]: 2024-05-14T10:03:17.621+0000 7f681c4b0700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.297521+0000 front 2024-05-14T10:02:49.297540+0000 (oldest deadline 2024-05-14T10:03:12.197700+0000) 2024-05-14T10:03:17.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:17 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:17.684+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:17.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:17 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:17.684+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:17.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:17 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:17.684+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:17.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:17 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:17.684+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:17.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:17 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:17.684+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:18.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:18 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:18.254+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:18.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:18 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:18.254+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:18.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:18 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:18.254+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:18.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:18 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:18.254+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:18.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:18 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:18.254+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:18.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:18 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:18.254+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:48.290628+0000 front 2024-05-14T10:02:48.290718+0000 (oldest deadline 2024-05-14T10:03:14.190618+0000) 2024-05-14T10:03:18.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:18 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:18.423+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:18.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:18 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:18.423+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:18.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:18 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:18.423+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:18.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:18 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:18.423+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:18.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:18 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:18.423+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:18.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:18 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:18.423+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:19.181 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:18 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:18.720+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:19.181 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:18 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:18.721+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:19.181 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:18 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:18.721+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:19.181 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:18 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:18.721+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:19.181 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:18 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:18.721+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:19.437 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:19 smithi029 podman[79295]: 2024-05-14 10:03:19.181336748 +0000 UTC m=+5.733440151 container died 862829554ce49de1b8d73a0f16822909276d84ca9e6383633db70d2578c814be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T10:03:19.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:19 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:19.291+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:19.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:19 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:19.291+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:19.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:19 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:19.291+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:19.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:19 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:19.291+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:19.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:19 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:19.291+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:19.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:19 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:19.291+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:48.290628+0000 front 2024-05-14T10:02:48.290718+0000 (oldest deadline 2024-05-14T10:03:14.190618+0000) 2024-05-14T10:03:19.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:19 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:19.438+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:19.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:19 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:19.438+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:19.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:19 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:19.438+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:19.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:19 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:19.438+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:19.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:19 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:19.438+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:19.703 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:19 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:19.438+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:19.902 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:19 smithi029 podman[79295]: 2024-05-14 10:03:19.589940805 +0000 UTC m=+6.142044202 container cleanup 862829554ce49de1b8d73a0f16822909276d84ca9e6383633db70d2578c814be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T10:03:19.902 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:19 smithi029 bash[79295]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8 2024-05-14T10:03:19.903 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:19 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:19.675+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:19.903 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:19 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:19.675+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:19.903 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:19 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:19.675+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:19.903 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:19 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:19.675+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:19.903 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:19 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:19.675+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:20.187 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:19 smithi029 podman[79310]: 2024-05-14 10:03:19.90291243 +0000 UTC m=+0.712628336 container remove 862829554ce49de1b8d73a0f16822909276d84ca9e6383633db70d2578c814be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=) 2024-05-14T10:03:20.483 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:20 smithi029 podman[79398]: 2024-05-14 10:03:20.129170007 +0000 UTC m=+0.018363133 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T10:03:20.483 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:20 smithi029 podman[79398]: 2024-05-14 10:03:20.238278693 +0000 UTC m=+0.127471790 container create 9b36f52fc358ae019d1494241c6350393e484d8c25ed6f74bcbcece78eaa74c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T10:03:20.483 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:20 smithi029 podman[79398]: 2024-05-14 10:03:20.484102157 +0000 UTC m=+0.373295262 container init 9b36f52fc358ae019d1494241c6350393e484d8c25ed6f74bcbcece78eaa74c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-05-14T10:03:20.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:20 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:20.308+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:20.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:20 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:20.308+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:20.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:20 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:20.308+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:20.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:20 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:20.308+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:20.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:20 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:20.308+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:20.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:20 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:20.308+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:48.290628+0000 front 2024-05-14T10:02:48.290718+0000 (oldest deadline 2024-05-14T10:03:14.190618+0000) 2024-05-14T10:03:20.791 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:20 smithi029 podman[79398]: 2024-05-14 10:03:20.488098985 +0000 UTC m=+0.377292096 container start 9b36f52fc358ae019d1494241c6350393e484d8c25ed6f74bcbcece78eaa74c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-14T10:03:20.791 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:20 smithi029 podman[79398]: 2024-05-14 10:03:20.601810372 +0000 UTC m=+0.491003495 container attach 9b36f52fc358ae019d1494241c6350393e484d8c25ed6f74bcbcece78eaa74c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8-deactivate, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-14T10:03:20.791 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:20 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:20.708+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:20.791 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:20 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:20.708+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:20.791 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:20 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:20.708+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:20.791 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:20 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:20.708+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:20.791 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:20 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:20.708+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:20.951 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:20 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:20.468+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:20.952 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:20 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:20.468+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:20.952 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:20 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:20.468+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:20.952 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:20 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:20.468+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:20.952 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:20 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:20.468+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:20.952 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:20 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:20.468+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:21.091 INFO:journalctl@ceph.osd.8.smithi029.stdout:May 14 10:03:20 smithi029 podman[79398]: 2024-05-14 10:03:20.792010105 +0000 UTC m=+0.681203211 container died 9b36f52fc358ae019d1494241c6350393e484d8c25ed6f74bcbcece78eaa74c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-14T10:03:21.419 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.8.service' 2024-05-14T10:03:21.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:03:21.459 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-14T10:03:21.459 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-14T10:03:21.459 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.9 2024-05-14T10:03:21.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:21 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:21.316+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:21.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:21 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:21.316+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:21.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:21 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:21.316+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:21.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:21 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:21.316+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:21.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:21 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:21.316+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:21.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:21 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:21.316+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:48.290628+0000 front 2024-05-14T10:02:48.290718+0000 (oldest deadline 2024-05-14T10:03:14.190618+0000) 2024-05-14T10:03:21.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:21 smithi002 systemd[1]: Stopping Ceph osd.9 for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:03:21.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:21 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:21.428+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:21.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:21 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:21.428+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:21.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:21 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:21.428+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:21.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:21 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:21.428+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:21.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:21 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:21.428+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:21.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:21 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:21.428+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:22.171 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:21 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:21.783+0000 7f9cfdf3f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:03:22.171 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:21 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:21.783+0000 7f9cfdf3f700 -1 osd.9 97 *** Got signal Terminated *** 2024-05-14T10:03:22.171 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:21 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:21.783+0000 7f9cfdf3f700 -1 osd.9 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T10:03:22.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:21 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:21.731+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:22.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:21 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:21.731+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:22.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:21 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:21.731+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:22.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:21 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:21.731+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:22.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:21 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:21.731+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:22.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:22 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:22.321+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:22.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:22 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:22.321+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:22.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:22 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:22.321+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:22.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:22 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:22.321+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:22.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:22 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:22.321+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:22.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:22 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:22.321+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:48.290628+0000 front 2024-05-14T10:02:48.290718+0000 (oldest deadline 2024-05-14T10:03:14.190618+0000) 2024-05-14T10:03:22.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:22 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:22.394+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:22.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:22 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:22.394+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:22.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:22 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:22.394+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:22.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:22 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:22.394+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:22.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:22 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:22.394+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:22.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:22 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:22.394+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:23.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:22 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:22.726+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:23.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:22 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:22.726+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:23.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:22 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:22.726+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:23.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:22 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:22.726+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:23.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:22 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:22.726+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:23.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:23 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:23.345+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:23.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:23 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:23.345+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:23.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:23 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:23.345+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:23.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:23 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:23.346+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:23.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:23 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:23.346+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:23.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:23 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:23.346+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:48.290628+0000 front 2024-05-14T10:02:48.290718+0000 (oldest deadline 2024-05-14T10:03:14.190618+0000) 2024-05-14T10:03:23.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:23 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:23.374+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:23.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:23 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:23.374+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:23.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:23 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:23.374+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:23.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:23 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:23.374+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:23.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:23 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:23.374+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:23.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:23 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:23.374+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:24.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:23 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:23.761+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:24.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:23 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:23.761+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:24.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:23 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:23.761+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:24.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:23 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:23.761+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:24.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:23 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:23.761+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:24.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:24 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:24.324+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:24.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:24 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:24.324+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:24.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:24 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:24.324+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:24.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:24 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:24.324+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:24.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:24 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:24.324+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:24.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:24 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:24.324+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:48.290628+0000 front 2024-05-14T10:02:48.290718+0000 (oldest deadline 2024-05-14T10:03:14.190618+0000) 2024-05-14T10:03:24.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:24 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:24.327+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:24.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:24 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:24.327+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:24.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:24 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:24.327+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:24.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:24 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:24.327+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:24.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:24 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:24.327+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:24.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:24 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:24.327+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:24.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:24 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:24.327+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:01.958504+0000 front 2024-05-14T10:03:01.958439+0000 (oldest deadline 2024-05-14T10:03:23.658511+0000) 2024-05-14T10:03:25.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:24 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:24.752+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:25.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:24 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:24.752+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:25.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:24 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:24.752+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:25.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:24 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:24.752+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:25.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:24 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:24.752+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:25.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:25 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:25.362+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:25.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:25 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:25.362+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:25.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:25 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:25.362+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:25.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:25 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:25.362+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:25.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:25 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:25.362+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:25.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:25 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:25.362+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:48.290628+0000 front 2024-05-14T10:02:48.290718+0000 (oldest deadline 2024-05-14T10:03:14.190618+0000) 2024-05-14T10:03:25.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:25 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:25.362+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:01.792108+0000 front 2024-05-14T10:03:01.792143+0000 (oldest deadline 2024-05-14T10:03:25.292355+0000) 2024-05-14T10:03:25.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:25 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:25.319+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:25.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:25 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:25.319+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:25.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:25 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:25.319+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:25.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:25 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:25.319+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:25.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:25 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:25.319+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:25.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:25 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:25.319+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:25.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:25 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:25.319+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:01.958504+0000 front 2024-05-14T10:03:01.958439+0000 (oldest deadline 2024-05-14T10:03:23.658511+0000) 2024-05-14T10:03:26.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:25 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:25.767+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:26.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:25 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:25.767+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:26.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:25 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:25.767+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:26.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:25 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:25.767+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:26.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:25 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:25.767+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:26.671 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:26 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:26.391+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:05.479703+0000 front 2024-05-14T10:02:05.479734+0000 (oldest deadline 2024-05-14T10:02:29.480396+0000) 2024-05-14T10:03:26.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:26 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:26.391+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:16.482560+0000 front 2024-05-14T10:02:16.482496+0000 (oldest deadline 2024-05-14T10:02:38.683410+0000) 2024-05-14T10:03:26.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:26 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:26.391+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.285610+0000 front 2024-05-14T10:02:23.285691+0000 (oldest deadline 2024-05-14T10:02:48.585652+0000) 2024-05-14T10:03:26.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:26 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:26.391+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:32.587300+0000 front 2024-05-14T10:02:32.587384+0000 (oldest deadline 2024-05-14T10:02:57.788388+0000) 2024-05-14T10:03:26.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:26 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:26.391+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:37.789147+0000 front 2024-05-14T10:02:37.788950+0000 (oldest deadline 2024-05-14T10:03:03.088916+0000) 2024-05-14T10:03:26.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:26 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:26.391+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:48.290628+0000 front 2024-05-14T10:02:48.290718+0000 (oldest deadline 2024-05-14T10:03:14.190618+0000) 2024-05-14T10:03:26.672 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:26 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9[63765]: 2024-05-14T10:03:26.391+0000 7f9cf64ed700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:01.792108+0000 front 2024-05-14T10:03:01.792143+0000 (oldest deadline 2024-05-14T10:03:25.292355+0000) 2024-05-14T10:03:26.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:26 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:26.345+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:26.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:26 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:26.345+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:26.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:26 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:26.345+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:26.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:26 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:26.345+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:26.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:26 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:26.345+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:26.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:26 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:26.345+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:26.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:26 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:26.345+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:01.958504+0000 front 2024-05-14T10:03:01.958439+0000 (oldest deadline 2024-05-14T10:03:23.658511+0000) 2024-05-14T10:03:27.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:26 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:26.730+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:27.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:26 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:26.730+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:27.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:26 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:26.730+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:27.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:26 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:26.730+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:27.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:26 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:26.730+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:27.657 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:27 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:27.299+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:27.657 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:27 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:27.299+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:27.657 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:27 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:27.299+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:27.657 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:27 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:27.299+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:27.657 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:27 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:27.299+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:27.657 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:27 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:27.299+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:27.657 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:27 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:27.299+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:01.958504+0000 front 2024-05-14T10:03:01.958439+0000 (oldest deadline 2024-05-14T10:03:23.658511+0000) 2024-05-14T10:03:27.921 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:27 smithi002 podman[108974]: 2024-05-14 10:03:27.479424138 +0000 UTC m=+5.809718173 container died 2e4b54fb77549e3ae352f6c7fa248f9288835e773865cbf8137159f06534a929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T10:03:28.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:27 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:27.746+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:28.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:27 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:27.746+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:28.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:27 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:27.746+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:28.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:27 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:27.746+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:28.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:27 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:27.746+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:28.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:27 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:27.746+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:28.233 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:28 smithi002 podman[108974]: 2024-05-14 10:03:28.008141003 +0000 UTC m=+6.338434983 container cleanup 2e4b54fb77549e3ae352f6c7fa248f9288835e773865cbf8137159f06534a929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-14T10:03:28.234 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:28 smithi002 bash[108974]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9 2024-05-14T10:03:28.647 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:28 smithi002 podman[108992]: 2024-05-14 10:03:28.234468701 +0000 UTC m=+0.746366779 container remove 2e4b54fb77549e3ae352f6c7fa248f9288835e773865cbf8137159f06534a929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True) 2024-05-14T10:03:28.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:28 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:28.272+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:28.924 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:28 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:28.272+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:28.924 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:28 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:28.272+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:28.924 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:28 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:28.272+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:28.924 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:28 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:28.272+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:28.924 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:28 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:28.272+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:28.924 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:28 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:28.272+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:01.958504+0000 front 2024-05-14T10:03:01.958439+0000 (oldest deadline 2024-05-14T10:03:23.658511+0000) 2024-05-14T10:03:28.925 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:28 smithi002 podman[109080]: 2024-05-14 10:03:28.5505674 +0000 UTC m=+0.017751291 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T10:03:28.925 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:28 smithi002 podman[109080]: 2024-05-14 10:03:28.652144963 +0000 UTC m=+0.119328852 container create 890ef43c3083b48d18858b9f7d09de007e708c6ceb22cbc59ba19552e62ca8e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:03:28.925 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:28 smithi002 podman[109080]: 2024-05-14 10:03:28.871475186 +0000 UTC m=+0.338659088 container init 890ef43c3083b48d18858b9f7d09de007e708c6ceb22cbc59ba19552e62ca8e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T10:03:28.925 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:28 smithi002 podman[109080]: 2024-05-14 10:03:28.874950381 +0000 UTC m=+0.342134278 container start 890ef43c3083b48d18858b9f7d09de007e708c6ceb22cbc59ba19552e62ca8e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-14T10:03:29.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:28 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:28.750+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:29.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:28 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:28.750+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:29.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:28 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:28.750+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:29.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:28 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:28.750+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:29.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:28 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:28.750+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:29.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:28 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:28.750+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:29.421 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:28 smithi002 podman[109080]: 2024-05-14 10:03:28.979764779 +0000 UTC m=+0.446948667 container attach 890ef43c3083b48d18858b9f7d09de007e708c6ceb22cbc59ba19552e62ca8e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:03:29.421 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 10:03:29 smithi002 podman[109080]: 2024-05-14 10:03:29.175953963 +0000 UTC m=+0.643137857 container died 890ef43c3083b48d18858b9f7d09de007e708c6ceb22cbc59ba19552e62ca8e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-14T10:03:29.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:29 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:29.304+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:29.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:29 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:29.304+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:29.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:29 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:29.304+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:29.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:29 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:29.304+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:29.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:29 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:29.305+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:29.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:29 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:29.305+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:29.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:29 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:29.305+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:01.958504+0000 front 2024-05-14T10:03:01.958439+0000 (oldest deadline 2024-05-14T10:03:23.658511+0000) 2024-05-14T10:03:29.739 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.9.service' 2024-05-14T10:03:29.779 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:03:29.779 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-14T10:03:29.779 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-14T10:03:29.779 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.10 2024-05-14T10:03:30.073 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:29 smithi026 systemd[1]: Stopping Ceph osd.10 for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:03:30.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:29 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:29.729+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:30.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:29 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:29.729+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:30.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:29 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:29.729+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:30.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:29 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:29.729+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:30.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:29 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:29.729+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:30.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:29 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:29.729+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:30.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:30 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:30.073+0000 7fb3ed638700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:03:30.451 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:30 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:30.073+0000 7fb3ed638700 -1 osd.10 97 *** Got signal Terminated *** 2024-05-14T10:03:30.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:30 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:30.073+0000 7fb3ed638700 -1 osd.10 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T10:03:30.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:30 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:30.292+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:30.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:30 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:30.292+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:30.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:30 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:30.292+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:30.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:30 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:30.292+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:30.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:30 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:30.292+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:30.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:30 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:30.292+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:30.452 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:30 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:30.292+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:01.958504+0000 front 2024-05-14T10:03:01.958439+0000 (oldest deadline 2024-05-14T10:03:23.658511+0000) 2024-05-14T10:03:31.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:30 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:30.699+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:31.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:30 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:30.699+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:31.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:30 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:30.699+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:31.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:30 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:30.699+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:31.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:30 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:30.699+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:31.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:30 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:30.699+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:31.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:31 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:31.289+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:31.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:31 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:31.289+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:31.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:31 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:31.289+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:31.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:31 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:31.289+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:31.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:31 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:31.289+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:31.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:31 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:31.289+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:31.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:31 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:31.289+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:01.958504+0000 front 2024-05-14T10:03:01.958439+0000 (oldest deadline 2024-05-14T10:03:23.658511+0000) 2024-05-14T10:03:31.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:31 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:31.721+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:31.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:31 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:31.721+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:31.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:31 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:31.721+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:31.937 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:31 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:31.721+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:31.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:31 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:31.721+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:31.938 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:31 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:31.721+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:32.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:32 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:32.312+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:32.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:32 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:32.312+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:32.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:32 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:32.312+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:32.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:32 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:32.312+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:32.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:32 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:32.312+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:32.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:32 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:32.312+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:32.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:32 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:32.312+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:01.958504+0000 front 2024-05-14T10:03:01.958439+0000 (oldest deadline 2024-05-14T10:03:23.658511+0000) 2024-05-14T10:03:33.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:32 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:32.760+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:33.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:32 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:32.760+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:33.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:32 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:32.760+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:33.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:32 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:32.760+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:33.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:32 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:32.760+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:33.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:32 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:32.760+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:33.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:33 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:33.333+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:33.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:33 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:33.333+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:33.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:33 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:33.333+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:33.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:33 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:33.333+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:33.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:33 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:33.333+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:33.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:33 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:33.333+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:33.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:33 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:33.333+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:01.958504+0000 front 2024-05-14T10:03:01.958439+0000 (oldest deadline 2024-05-14T10:03:23.658511+0000) 2024-05-14T10:03:34.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:33 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:33.733+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:34.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:33 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:33.733+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:34.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:33 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:33.733+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:34.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:33 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:33.733+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:34.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:33 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:33.733+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:34.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:33 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:33.733+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:34.701 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:34 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:34.343+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:03.148089+0000 front 2024-05-14T10:02:03.148307+0000 (oldest deadline 2024-05-14T10:02:29.048059+0000) 2024-05-14T10:03:34.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:34 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:34.343+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-05-14T10:02:14.349553+0000 front 2024-05-14T10:02:14.349607+0000 (oldest deadline 2024-05-14T10:02:38.449731+0000) 2024-05-14T10:03:34.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:34 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:34.343+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:23.651800+0000 front 2024-05-14T10:02:23.651753+0000 (oldest deadline 2024-05-14T10:02:49.551741+0000) 2024-05-14T10:03:34.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:34 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:34.343+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:29.552520+0000 front 2024-05-14T10:02:29.552710+0000 (oldest deadline 2024-05-14T10:02:54.252472+0000) 2024-05-14T10:03:34.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:34 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:34.343+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:38.953618+0000 front 2024-05-14T10:02:38.953554+0000 (oldest deadline 2024-05-14T10:03:03.053841+0000) 2024-05-14T10:03:34.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:34 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:34.343+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:49.455263+0000 front 2024-05-14T10:02:49.455426+0000 (oldest deadline 2024-05-14T10:03:10.555136+0000) 2024-05-14T10:03:34.702 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:34 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10[61541]: 2024-05-14T10:03:34.343+0000 7fb3e5be6700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:01.958504+0000 front 2024-05-14T10:03:01.958439+0000 (oldest deadline 2024-05-14T10:03:23.658511+0000) 2024-05-14T10:03:35.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:34 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:34.726+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:35.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:34 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:34.726+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:35.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:34 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:34.726+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:35.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:34 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:34.726+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:35.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:34 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:34.726+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:35.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:34 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:34.726+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:35.886 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:35 smithi026 podman[78858]: 2024-05-14 10:03:35.634209365 +0000 UTC m=+5.649424368 container died d702f2a68103c05434ba644b84c11f473ffeb7aaf3432420ca0e23e9f5a23f82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-14T10:03:36.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:35 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:35.747+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:36.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:35 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:35.747+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:36.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:35 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:35.747+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:36.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:35 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:35.747+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:36.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:35 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:35.747+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:36.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:35 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:35.747+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:36.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:35 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:35.747+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2024-05-14T10:03:09.686010+0000 front 2024-05-14T10:03:09.686036+0000 (oldest deadline 2024-05-14T10:03:35.585699+0000) 2024-05-14T10:03:36.201 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:35 smithi026 podman[78858]: 2024-05-14 10:03:35.9818613 +0000 UTC m=+5.997076295 container cleanup d702f2a68103c05434ba644b84c11f473ffeb7aaf3432420ca0e23e9f5a23f82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-05-14T10:03:36.202 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:35 smithi026 bash[78858]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10 2024-05-14T10:03:36.599 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:36 smithi026 podman[78873]: 2024-05-14 10:03:36.291276886 +0000 UTC m=+0.659866770 container remove d702f2a68103c05434ba644b84c11f473ffeb7aaf3432420ca0e23e9f5a23f82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-14T10:03:36.876 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:36 smithi026 podman[78963]: 2024-05-14 10:03:36.600434985 +0000 UTC m=+0.104082860 container create 66d14fda408d515fcbc20b6e8baf3284be582bd57a61505e1b7c75652fcc7626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10-deactivate, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:03:36.877 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:36 smithi026 podman[78963]: 2024-05-14 10:03:36.514107609 +0000 UTC m=+0.017755478 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T10:03:36.877 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:36 smithi026 podman[78963]: 2024-05-14 10:03:36.802547379 +0000 UTC m=+0.306195238 container init 66d14fda408d515fcbc20b6e8baf3284be582bd57a61505e1b7c75652fcc7626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-14T10:03:36.877 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:36 smithi026 podman[78963]: 2024-05-14 10:03:36.805734937 +0000 UTC m=+0.309382807 container start 66d14fda408d515fcbc20b6e8baf3284be582bd57a61505e1b7c75652fcc7626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-05-14T10:03:36.877 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:36 smithi026 podman[78963]: 2024-05-14 10:03:36.877402884 +0000 UTC m=+0.381050746 container attach 66d14fda408d515fcbc20b6e8baf3284be582bd57a61505e1b7c75652fcc7626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-14T10:03:37.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:36 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:36.723+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:37.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:36 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:36.723+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:37.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:36 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:36.723+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:37.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:36 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:36.723+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:37.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:36 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:36.723+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:37.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:36 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:36.723+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:37.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:36 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:36.723+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2024-05-14T10:03:09.686010+0000 front 2024-05-14T10:03:09.686036+0000 (oldest deadline 2024-05-14T10:03:35.585699+0000) 2024-05-14T10:03:37.202 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:37 smithi026 podman[78963]: 2024-05-14 10:03:37.109523214 +0000 UTC m=+0.613171079 container died 66d14fda408d515fcbc20b6e8baf3284be582bd57a61505e1b7c75652fcc7626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.vendor=CentOS) 2024-05-14T10:03:37.654 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.10.service' 2024-05-14T10:03:37.682 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:37 smithi026 podman[78989]: 2024-05-14 10:03:37.628763365 +0000 UTC m=+0.509283427 container remove 66d14fda408d515fcbc20b6e8baf3284be582bd57a61505e1b7c75652fcc7626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-05-14T10:03:37.682 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:37 smithi026 systemd[1]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.10.service: Deactivated successfully. 2024-05-14T10:03:37.682 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:37 smithi026 systemd[1]: Stopped Ceph osd.10 for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T10:03:37.682 INFO:journalctl@ceph.osd.10.smithi026.stdout:May 14 10:03:37 smithi026 systemd[1]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.10.service: Consumed 11min 18.563s CPU time. 2024-05-14T10:03:37.699 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:03:37.699 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-14T10:03:37.699 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-14T10:03:37.699 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.11 2024-05-14T10:03:38.009 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:37 smithi029 systemd[1]: Stopping Ceph osd.11 for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:03:38.009 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:37 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:37.767+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:38.009 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:37 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:37.767+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:38.009 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:37 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:37.767+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:38.009 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:37 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:37.767+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:38.009 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:37 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:37.767+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:38.009 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:37 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:37.767+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:38.009 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:37 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:37.767+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2024-05-14T10:03:09.686010+0000 front 2024-05-14T10:03:09.686036+0000 (oldest deadline 2024-05-14T10:03:35.585699+0000) 2024-05-14T10:03:38.009 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:38.008+0000 7fb80b69d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:03:38.009 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:38.008+0000 7fb80b69d700 -1 osd.11 97 *** Got signal Terminated *** 2024-05-14T10:03:38.009 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:38.008+0000 7fb80b69d700 -1 osd.11 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T10:03:39.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:38.799+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:39.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:38.799+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:39.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:38.799+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:39.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:38.799+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:39.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:38.799+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:39.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:38.799+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:39.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:38 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:38.799+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2024-05-14T10:03:09.686010+0000 front 2024-05-14T10:03:09.686036+0000 (oldest deadline 2024-05-14T10:03:35.585699+0000) 2024-05-14T10:03:40.089 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:39 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:39.781+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:40.089 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:39 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:39.781+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:40.089 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:39 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:39.781+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:40.090 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:39 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:39.781+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:40.090 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:39 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:39.781+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:40.090 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:39 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:39.781+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:40.090 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:39 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:39.781+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2024-05-14T10:03:09.686010+0000 front 2024-05-14T10:03:09.686036+0000 (oldest deadline 2024-05-14T10:03:35.585699+0000) 2024-05-14T10:03:41.105 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:40 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:40.765+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:41.105 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:40 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:40.765+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:41.105 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:40 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:40.765+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:41.105 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:40 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:40.765+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:41.105 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:40 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:40.765+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:41.105 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:40 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:40.765+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:41.105 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:40 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:40.765+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2024-05-14T10:03:09.686010+0000 front 2024-05-14T10:03:09.686036+0000 (oldest deadline 2024-05-14T10:03:35.585699+0000) 2024-05-14T10:03:42.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:41 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:41.752+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:42.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:41 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:41.752+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:42.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:41 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:41.752+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:42.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:41 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:41.752+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:42.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:41 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:41.752+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:42.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:41 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:41.753+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:42.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:41 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:41.753+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2024-05-14T10:03:09.686010+0000 front 2024-05-14T10:03:09.686036+0000 (oldest deadline 2024-05-14T10:03:35.585699+0000) 2024-05-14T10:03:42.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:41 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:41.753+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6820 osd.8 since back 2024-05-14T10:03:15.586134+0000 front 2024-05-14T10:03:15.586090+0000 (oldest deadline 2024-05-14T10:03:41.486166+0000) 2024-05-14T10:03:43.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:42 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:42.773+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T10:02:07.171665+0000 front 2024-05-14T10:02:07.171632+0000 (oldest deadline 2024-05-14T10:02:29.471781+0000) 2024-05-14T10:03:43.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:42 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:42.773+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6804 osd.2 since back 2024-05-14T10:02:24.575773+0000 front 2024-05-14T10:02:24.575733+0000 (oldest deadline 2024-05-14T10:02:48.675855+0000) 2024-05-14T10:03:43.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:42 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:42.773+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T10:02:28.676303+0000 front 2024-05-14T10:02:28.676419+0000 (oldest deadline 2024-05-14T10:02:53.976433+0000) 2024-05-14T10:03:43.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:42 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:42.773+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2024-05-14T10:02:40.779530+0000 front 2024-05-14T10:02:40.779562+0000 (oldest deadline 2024-05-14T10:03:03.079537+0000) 2024-05-14T10:03:43.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:42 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:42.773+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6812 osd.5 since back 2024-05-14T10:02:46.981838+0000 front 2024-05-14T10:02:46.981870+0000 (oldest deadline 2024-05-14T10:03:12.882036+0000) 2024-05-14T10:03:43.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:42 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:42.773+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T10:03:02.184277+0000 front 2024-05-14T10:03:02.184332+0000 (oldest deadline 2024-05-14T10:03:26.884286+0000) 2024-05-14T10:03:43.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:42 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:42.773+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2024-05-14T10:03:09.686010+0000 front 2024-05-14T10:03:09.686036+0000 (oldest deadline 2024-05-14T10:03:35.585699+0000) 2024-05-14T10:03:43.188 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:42 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11[61261]: 2024-05-14T10:03:42.773+0000 7fb807c53700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.29:6820 osd.8 since back 2024-05-14T10:03:15.586134+0000 front 2024-05-14T10:03:15.586090+0000 (oldest deadline 2024-05-14T10:03:41.486166+0000) 2024-05-14T10:03:43.881 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:43 smithi029 podman[79503]: 2024-05-14 10:03:43.606330302 +0000 UTC m=+5.703238728 container stop a47901cf6ce0e212e8314506348904013fba63ec45d108ac1677291317b06faa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-14T10:03:43.881 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:43 smithi029 podman[79503]: 2024-05-14 10:03:43.606845528 +0000 UTC m=+5.703753952 container died a47901cf6ce0e212e8314506348904013fba63ec45d108ac1677291317b06faa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-14T10:03:44.192 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:43 smithi029 podman[79503]: 2024-05-14 10:03:43.987648828 +0000 UTC m=+6.084557251 container cleanup a47901cf6ce0e212e8314506348904013fba63ec45d108ac1677291317b06faa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507) 2024-05-14T10:03:44.192 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:43 smithi029 bash[79503]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11 2024-05-14T10:03:44.618 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:44 smithi029 podman[79532]: 2024-05-14 10:03:44.2987546 +0000 UTC m=+0.685353756 container remove a47901cf6ce0e212e8314506348904013fba63ec45d108ac1677291317b06faa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T10:03:44.931 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:44 smithi029 podman[79621]: 2024-05-14 10:03:44.522209606 +0000 UTC m=+0.017820147 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T10:03:44.931 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:44 smithi029 podman[79621]: 2024-05-14 10:03:44.625601754 +0000 UTC m=+0.121212314 container create cb24e7a98d2139877882e806dfd335d53480d9b533f94d67e80c1df77f7e846c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-14T10:03:44.931 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:44 smithi029 podman[79621]: 2024-05-14 10:03:44.847895167 +0000 UTC m=+0.343505700 container init cb24e7a98d2139877882e806dfd335d53480d9b533f94d67e80c1df77f7e846c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-14T10:03:44.931 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:44 smithi029 podman[79621]: 2024-05-14 10:03:44.851218071 +0000 UTC m=+0.346828613 container start cb24e7a98d2139877882e806dfd335d53480d9b533f94d67e80c1df77f7e846c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11-deactivate, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-14T10:03:44.931 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:44 smithi029 podman[79621]: 2024-05-14 10:03:44.931458731 +0000 UTC m=+0.427069273 container attach cb24e7a98d2139877882e806dfd335d53480d9b533f94d67e80c1df77f7e846c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, ceph=True) 2024-05-14T10:03:45.187 INFO:journalctl@ceph.osd.11.smithi029.stdout:May 14 10:03:45 smithi029 podman[79621]: 2024-05-14 10:03:45.153994204 +0000 UTC m=+0.649604745 container died cb24e7a98d2139877882e806dfd335d53480d9b533f94d67e80c1df77f7e846c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-14T10:03:45.840 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@osd.11.service' 2024-05-14T10:03:45.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:03:45.884 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-14T10:03:45.885 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-14T10:03:45.885 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.a 2024-05-14T10:03:46.171 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 10:03:45 smithi002 systemd[1]: Stopping Ceph mds.a for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:03:46.671 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 10:03:46 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-a[70867]: 2024-05-14T10:03:46.192+0000 7efdbfcb5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:03:46.671 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 10:03:46 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-a[70867]: 2024-05-14T10:03:46.192+0000 7efdbfcb5700 -1 mds.a *** got signal Terminated *** 2024-05-14T10:03:48.934 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 10:03:48 smithi002 podman[109197]: 2024-05-14 10:03:48.676276715 +0000 UTC m=+2.580184676 container died 73d41c4e0cf74a51b9ecabc66d601c96caf70f1a4f1b083ae463f58e4eb513a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-14T10:03:49.202 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 10:03:49 smithi002 podman[109197]: 2024-05-14 10:03:49.025708716 +0000 UTC m=+2.929616668 container cleanup 73d41c4e0cf74a51b9ecabc66d601c96caf70f1a4f1b083ae463f58e4eb513a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-a, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-14T10:03:49.202 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 10:03:49 smithi002 bash[109197]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-a 2024-05-14T10:03:49.439 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.a.service' 2024-05-14T10:03:49.463 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 10:03:49 smithi002 podman[109211]: 2024-05-14 10:03:49.201897762 +0000 UTC m=+0.521353787 container remove 73d41c4e0cf74a51b9ecabc66d601c96caf70f1a4f1b083ae463f58e4eb513a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-a, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-14T10:03:49.463 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 10:03:49 smithi002 systemd[1]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.a.service: Deactivated successfully. 2024-05-14T10:03:49.463 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 10:03:49 smithi002 systemd[1]: Stopped Ceph mds.a for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T10:03:49.463 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 10:03:49 smithi002 systemd[1]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.a.service: Consumed 1.304s CPU time. 2024-05-14T10:03:49.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:03:49.477 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-14T10:03:49.477 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-14T10:03:49.477 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.d 2024-05-14T10:03:49.813 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 10:03:49 smithi002 systemd[1]: Stopping Ceph mds.d for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:03:50.171 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 10:03:49 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-d[69602]: 2024-05-14T10:03:49.811+0000 7fb6f64a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:03:50.171 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 10:03:49 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-d[69602]: 2024-05-14T10:03:49.811+0000 7fb6f64a6700 -1 mds.d *** got signal Terminated *** 2024-05-14T10:03:51.555 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 10:03:51 smithi002 podman[109316]: 2024-05-14 10:03:51.281079283 +0000 UTC m=+1.566334164 container died 038ad3835cf8df63f6ba2b16461490194dd7da12261f312f0bae3c1a8ec7072f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-14T10:03:51.921 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 10:03:51 smithi002 podman[109316]: 2024-05-14 10:03:51.646028482 +0000 UTC m=+1.931283356 container cleanup 038ad3835cf8df63f6ba2b16461490194dd7da12261f312f0bae3c1a8ec7072f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T10:03:51.921 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 10:03:51 smithi002 bash[109316]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-d 2024-05-14T10:03:52.079 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.d.service' 2024-05-14T10:03:52.113 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:03:52.114 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-14T10:03:52.114 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-14T10:03:52.114 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.g 2024-05-14T10:03:52.671 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 10:03:52 smithi002 systemd[1]: Stopping Ceph mds.g for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:03:52.672 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 10:03:52 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-g[70443]: 2024-05-14T10:03:52.440+0000 7f9fabb56700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:03:52.672 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 10:03:52 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-g[70443]: 2024-05-14T10:03:52.440+0000 7f9fabb56700 -1 mds.g *** got signal Terminated *** 2024-05-14T10:03:56.750 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 10:03:56 smithi002 podman[109436]: 2024-05-14 10:03:56.45510255 +0000 UTC m=+4.109409875 container died f3fcf57575ab9c44eaa2e1d7a9f92f752b63a96b79a46ac199eaee71b976fecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-g, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-14T10:03:57.067 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 10:03:56 smithi002 podman[109436]: 2024-05-14 10:03:56.866553371 +0000 UTC m=+4.520860705 container cleanup f3fcf57575ab9c44eaa2e1d7a9f92f752b63a96b79a46ac199eaee71b976fecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-14T10:03:57.067 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 10:03:56 smithi002 bash[109436]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-g 2024-05-14T10:03:57.305 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.g.service' 2024-05-14T10:03:57.327 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 10:03:57 smithi002 podman[109450]: 2024-05-14 10:03:57.067586854 +0000 UTC m=+0.612151352 container remove f3fcf57575ab9c44eaa2e1d7a9f92f752b63a96b79a46ac199eaee71b976fecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-14T10:03:57.327 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 10:03:57 smithi002 systemd[1]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.g.service: Deactivated successfully. 2024-05-14T10:03:57.328 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 10:03:57 smithi002 systemd[1]: Stopped Ceph mds.g for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T10:03:57.328 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 10:03:57 smithi002 systemd[1]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.g.service: Consumed 1.305s CPU time. 2024-05-14T10:03:57.339 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:03:57.339 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-14T10:03:57.340 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-14T10:03:57.340 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.j 2024-05-14T10:03:57.653 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 10:03:57 smithi002 systemd[1]: Stopping Ceph mds.j for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:03:57.921 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 10:03:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-j[69159]: 2024-05-14T10:03:57.653+0000 7f924ca8f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:03:57.921 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 10:03:57 smithi002 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-j[69159]: 2024-05-14T10:03:57.653+0000 7f924ca8f700 -1 mds.j *** got signal Terminated *** 2024-05-14T10:03:59.412 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 10:03:59 smithi002 podman[109558]: 2024-05-14 10:03:59.121365441 +0000 UTC m=+1.554137637 container died 810e8ad44c3b4568b9f3d9d613a28ed6f5871e5c6719e3f50b18b54df0d0e3ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-05-14T10:03:59.671 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 10:03:59 smithi002 podman[109558]: 2024-05-14 10:03:59.512110383 +0000 UTC m=+1.944882570 container cleanup 810e8ad44c3b4568b9f3d9d613a28ed6f5871e5c6719e3f50b18b54df0d0e3ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-j, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-14T10:03:59.738 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 10:03:59 smithi002 bash[109558]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-j 2024-05-14T10:03:59.946 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.j.service' 2024-05-14T10:03:59.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:03:59.979 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-14T10:03:59.979 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-14T10:03:59.980 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.b 2024-05-14T10:04:00.304 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 10:04:00 smithi026 systemd[1]: Stopping Ceph mds.b for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:04:00.586 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 10:04:00 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b[66741]: 2024-05-14T10:04:00.304+0000 7ff8ee1ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:04:00.586 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 10:04:00 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b[66741]: 2024-05-14T10:04:00.304+0000 7ff8ee1ed700 -1 mds.b *** got signal Terminated *** 2024-05-14T10:04:02.951 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 10:04:02 smithi026 podman[79069]: 2024-05-14 10:04:02.698172605 +0000 UTC m=+2.505162447 container died 968bdd4c0b6373c1a07bb45e26ca37dc9545f3500f7ff4c71ab3e6d8685fddec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:04:03.288 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 10:04:03 smithi026 podman[79069]: 2024-05-14 10:04:03.062657728 +0000 UTC m=+2.869647569 container cleanup 968bdd4c0b6373c1a07bb45e26ca37dc9545f3500f7ff4c71ab3e6d8685fddec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True) 2024-05-14T10:04:03.288 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 10:04:03 smithi026 bash[79069]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b 2024-05-14T10:04:03.547 INFO:journalctl@ceph.mds.b.smithi026.stdout:May 14 10:04:03 smithi026 podman[79095]: 2024-05-14 10:04:03.288800314 +0000 UTC m=+0.585378402 container remove 968bdd4c0b6373c1a07bb45e26ca37dc9545f3500f7ff4c71ab3e6d8685fddec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-14T10:04:03.551 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.b.service' 2024-05-14T10:04:03.587 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:04:03.588 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-14T10:04:03.588 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-14T10:04:03.588 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.e 2024-05-14T10:04:03.916 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 10:04:03 smithi026 systemd[1]: Stopping Ceph mds.e for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:04:04.201 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 10:04:03 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e[66118]: 2024-05-14T10:04:03.916+0000 7f3dd2f17700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:04:04.201 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 10:04:03 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e[66118]: 2024-05-14T10:04:03.916+0000 7f3dd2f17700 -1 mds.e *** got signal Terminated *** 2024-05-14T10:04:06.950 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 10:04:06 smithi026 podman[79201]: 2024-05-14 10:04:06.692490647 +0000 UTC m=+2.880088747 container stop eb79eacbc98a660363ccf3b839297da2af590244dbc44b39237ab2d8ef71654c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-05-14T10:04:06.950 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 10:04:06 smithi026 podman[79201]: 2024-05-14 10:04:06.718611734 +0000 UTC m=+2.906209841 container died eb79eacbc98a660363ccf3b839297da2af590244dbc44b39237ab2d8ef71654c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-14T10:04:07.201 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 10:04:07 smithi026 podman[79201]: 2024-05-14 10:04:07.066167406 +0000 UTC m=+3.253765505 container cleanup eb79eacbc98a660363ccf3b839297da2af590244dbc44b39237ab2d8ef71654c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-14T10:04:07.202 INFO:journalctl@ceph.mds.e.smithi026.stdout:May 14 10:04:07 smithi026 bash[79201]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-e 2024-05-14T10:04:07.447 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.e.service' 2024-05-14T10:04:07.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:04:07.481 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-14T10:04:07.481 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-14T10:04:07.481 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.h 2024-05-14T10:04:07.811 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 10:04:07 smithi026 systemd[1]: Stopping Ceph mds.h for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:04:08.201 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 10:04:07 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T10:04:07.811+0000 7fe8be0fa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:04:08.201 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 10:04:07 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h[66422]: 2024-05-14T10:04:07.811+0000 7fe8be0fa700 -1 mds.h *** got signal Terminated *** 2024-05-14T10:04:10.951 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 10:04:10 smithi026 podman[79322]: 2024-05-14 10:04:10.695916504 +0000 UTC m=+2.992506839 container died 58bf2a4d34111b0e7f4ebb901fe9bd9e86866099b7c0ed64e0f00eb6b09e44e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-14T10:04:11.337 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 10:04:11 smithi026 podman[79322]: 2024-05-14 10:04:11.111556386 +0000 UTC m=+3.408146720 container cleanup 58bf2a4d34111b0e7f4ebb901fe9bd9e86866099b7c0ed64e0f00eb6b09e44e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T10:04:11.337 INFO:journalctl@ceph.mds.h.smithi026.stdout:May 14 10:04:11 smithi026 bash[79322]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-h 2024-05-14T10:04:11.558 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.h.service' 2024-05-14T10:04:11.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:04:11.593 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-14T10:04:11.593 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-14T10:04:11.593 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.k 2024-05-14T10:04:11.923 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 10:04:11 smithi026 systemd[1]: Stopping Ceph mds.k for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:04:12.201 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 10:04:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-k[65819]: 2024-05-14T10:04:11.923+0000 7f1b89da7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:04:12.201 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 10:04:11 smithi026 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-k[65819]: 2024-05-14T10:04:11.923+0000 7f1b89da7700 -1 mds.k *** got signal Terminated *** 2024-05-14T10:04:15.814 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 10:04:15 smithi026 podman[79444]: 2024-05-14 10:04:15.546412915 +0000 UTC m=+3.727855784 container died d4e5220632df18aaa51bb2d978f139ae91584f21d8d4b1f1b737c60880e555a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-k, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507) 2024-05-14T10:04:16.098 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 10:04:15 smithi026 podman[79444]: 2024-05-14 10:04:15.890060269 +0000 UTC m=+4.071503127 container cleanup d4e5220632df18aaa51bb2d978f139ae91584f21d8d4b1f1b737c60880e555a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-k, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T10:04:16.099 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 10:04:15 smithi026 bash[79444]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-k 2024-05-14T10:04:16.329 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.k.service' 2024-05-14T10:04:16.350 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 10:04:16 smithi026 podman[79458]: 2024-05-14 10:04:16.099263508 +0000 UTC m=+0.535488884 container remove d4e5220632df18aaa51bb2d978f139ae91584f21d8d4b1f1b737c60880e555a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-k, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-14T10:04:16.350 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 10:04:16 smithi026 systemd[1]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.k.service: Deactivated successfully. 2024-05-14T10:04:16.351 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 10:04:16 smithi026 systemd[1]: Stopped Ceph mds.k for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T10:04:16.351 INFO:journalctl@ceph.mds.k.smithi026.stdout:May 14 10:04:16 smithi026 systemd[1]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.k.service: Consumed 1.300s CPU time. 2024-05-14T10:04:16.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:04:16.361 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-14T10:04:16.362 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-14T10:04:16.362 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.c 2024-05-14T10:04:16.668 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 10:04:16 smithi029 systemd[1]: Stopping Ceph mds.c for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:04:16.937 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 10:04:16 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-c[66381]: 2024-05-14T10:04:16.667+0000 7f9997c4e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:04:16.937 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 10:04:16 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-c[66381]: 2024-05-14T10:04:16.667+0000 7f9997c4e700 -1 mds.c *** got signal Terminated *** 2024-05-14T10:04:19.039 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 10:04:18 smithi029 podman[79726]: 2024-05-14 10:04:18.724201496 +0000 UTC m=+2.158144572 container died 1128ca72187843c129e5b30f52047885e6d63590c3c0c28c49fbba62773ae7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T10:04:19.356 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 10:04:19 smithi029 podman[79726]: 2024-05-14 10:04:19.146621533 +0000 UTC m=+2.580564616 container cleanup 1128ca72187843c129e5b30f52047885e6d63590c3c0c28c49fbba62773ae7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-14T10:04:19.356 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 10:04:19 smithi029 bash[79726]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-c 2024-05-14T10:04:19.596 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.c.service' 2024-05-14T10:04:19.617 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 10:04:19 smithi029 podman[79751]: 2024-05-14 10:04:19.357285345 +0000 UTC m=+0.626886736 container remove 1128ca72187843c129e5b30f52047885e6d63590c3c0c28c49fbba62773ae7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-c, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T10:04:19.617 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 10:04:19 smithi029 systemd[1]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.c.service: Deactivated successfully. 2024-05-14T10:04:19.617 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 10:04:19 smithi029 systemd[1]: Stopped Ceph mds.c for 50d5aebe-11d3-11ef-bc98-c7b262605968. 2024-05-14T10:04:19.618 INFO:journalctl@ceph.mds.c.smithi029.stdout:May 14 10:04:19 smithi029 systemd[1]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.c.service: Consumed 53.613s CPU time. 2024-05-14T10:04:19.632 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:04:19.632 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-14T10:04:19.632 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-14T10:04:19.632 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.f 2024-05-14T10:04:19.937 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 10:04:19 smithi029 systemd[1]: Stopping Ceph mds.f for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:04:20.437 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 10:04:19 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-f[66076]: 2024-05-14T10:04:19.942+0000 7fb26d203700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:04:20.437 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 10:04:19 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-f[66076]: 2024-05-14T10:04:19.942+0000 7fb26d203700 -1 mds.f *** got signal Terminated *** 2024-05-14T10:04:23.012 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 10:04:22 smithi029 podman[79860]: 2024-05-14 10:04:22.744347599 +0000 UTC m=+2.890372166 container died 56f5739413b1245e958cd2673b854786dba1237267b538167bdb24fabb507b60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-14T10:04:23.374 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 10:04:23 smithi029 podman[79860]: 2024-05-14 10:04:23.105787064 +0000 UTC m=+3.251811632 container cleanup 56f5739413b1245e958cd2673b854786dba1237267b538167bdb24fabb507b60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD) 2024-05-14T10:04:23.374 INFO:journalctl@ceph.mds.f.smithi029.stdout:May 14 10:04:23 smithi029 bash[79860]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-f 2024-05-14T10:04:23.526 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.f.service' 2024-05-14T10:04:23.560 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:04:23.560 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-14T10:04:23.560 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-14T10:04:23.560 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.i 2024-05-14T10:04:23.895 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 10:04:23 smithi029 systemd[1]: Stopping Ceph mds.i for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:04:24.187 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 10:04:23 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-i[66685]: 2024-05-14T10:04:23.894+0000 7fef4ac33700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:04:24.187 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 10:04:23 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-i[66685]: 2024-05-14T10:04:23.894+0000 7fef4ac33700 -1 mds.i *** got signal Terminated *** 2024-05-14T10:04:27.005 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 10:04:26 smithi029 podman[79981]: 2024-05-14 10:04:26.737414305 +0000 UTC m=+2.953086949 container died 9e4f54c74b3e06c278ce449e189e9bc81565ec0e609c714995171ba796d697aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-14T10:04:27.392 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 10:04:27 smithi029 podman[79981]: 2024-05-14 10:04:27.090131091 +0000 UTC m=+3.305803732 container cleanup 9e4f54c74b3e06c278ce449e189e9bc81565ec0e609c714995171ba796d697aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507) 2024-05-14T10:04:27.392 INFO:journalctl@ceph.mds.i.smithi029.stdout:May 14 10:04:27 smithi029 bash[79981]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-i 2024-05-14T10:04:27.535 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.i.service' 2024-05-14T10:04:27.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:04:27.567 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-14T10:04:27.567 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-14T10:04:27.567 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.l 2024-05-14T10:04:27.937 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 10:04:27 smithi029 systemd[1]: Stopping Ceph mds.l for 50d5aebe-11d3-11ef-bc98-c7b262605968... 2024-05-14T10:04:27.937 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 10:04:27 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-l[65770]: 2024-05-14T10:04:27.894+0000 7f511ca3a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T10:04:27.937 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 10:04:27 smithi029 ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-l[65770]: 2024-05-14T10:04:27.894+0000 7f511ca3a700 -1 mds.l *** got signal Terminated *** 2024-05-14T10:04:32.181 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 10:04:31 smithi029 podman[80100]: 2024-05-14 10:04:31.91833302 +0000 UTC m=+4.127199953 container died e34a19116bb8a0da72d9dc9e95adf2686acb24884e7c67409e4f0ed32a2c7ed7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T10:04:32.437 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 10:04:32 smithi029 podman[80100]: 2024-05-14 10:04:32.264515838 +0000 UTC m=+4.473382767 container cleanup e34a19116bb8a0da72d9dc9e95adf2686acb24884e7c67409e4f0ed32a2c7ed7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-l, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T10:04:32.437 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 10:04:32 smithi029 bash[80100]: ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-l 2024-05-14T10:04:32.711 INFO:journalctl@ceph.mds.l.smithi029.stdout:May 14 10:04:32 smithi029 podman[80114]: 2024-05-14 10:04:32.459704907 +0000 UTC m=+0.539112537 container remove e34a19116bb8a0da72d9dc9e95adf2686acb24884e7c67409e4f0ed32a2c7ed7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-50d5aebe-11d3-11ef-bc98-c7b262605968-mds-l, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD) 2024-05-14T10:04:32.715 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50d5aebe-11d3-11ef-bc98-c7b262605968@mds.l.service' 2024-05-14T10:04:32.746 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T10:04:32.746 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-14T10:04:32.747 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-14T10:04:32.950 INFO:teuthology.orchestra.run.smithi002.stdout:Deleting cluster with fsid: 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T10:04:35.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-14T10:04:35.623 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T10:04:38.271 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-14T10:04:38.470 INFO:teuthology.orchestra.run.smithi029.stdout:Deleting cluster with fsid: 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T10:04:41.049 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T10:04:41.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T10:04:41.115 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T10:04:41.141 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-14T10:04:41.143 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670/remote/smithi002/crash 2024-05-14T10:04:41.144 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/crash -- . 2024-05-14T10:04:41.182 INFO:teuthology.orchestra.run.smithi002.stderr:tar: /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-14T10:04:41.182 INFO:teuthology.orchestra.run.smithi002.stderr:tar: Error is not recoverable: exiting now 2024-05-14T10:04:41.184 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670/remote/smithi026/crash 2024-05-14T10:04:41.185 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/crash -- . 2024-05-14T10:04:41.226 INFO:teuthology.orchestra.run.smithi026.stderr:tar: /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-14T10:04:41.227 INFO:teuthology.orchestra.run.smithi026.stderr:tar: Error is not recoverable: exiting now 2024-05-14T10:04:41.228 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670/remote/smithi029/crash 2024-05-14T10:04:41.229 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/crash -- . 2024-05-14T10:04:41.272 INFO:teuthology.orchestra.run.smithi029.stderr:tar: /var/lib/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-14T10:04:41.272 INFO:teuthology.orchestra.run.smithi029.stderr:tar: Error is not recoverable: exiting now 2024-05-14T10:04:41.273 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-14T10:04:41.274 DEBUG:teuthology.orchestra.run.smithi002:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-14T10:04:41.379 INFO:teuthology.orchestra.run.smithi002.stdout:2024-05-14T09:37:11.678685+0000 mds.h (mds.1) 2 : cluster [WRN] Scrub error on inode 0x1000000022c (/volumes/qa/sv_1/5bd6dab4-7d53-4a9f-a809-40f5c08c1a08/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-05-14T10:04:41.379 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-14T10:04:41.379 INFO:tasks.cephadm:Compressing logs... 2024-05-14T10:04:41.379 DEBUG:teuthology.orchestra.run.smithi002:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T10:04:41.382 DEBUG:teuthology.orchestra.run.smithi026:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T10:04:41.384 DEBUG:teuthology.orchestra.run.smithi029:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T10:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-14T10:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.log 2024-05-14T10:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-14T10:04:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.7.log 2024-05-14T10:04:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.b.log 2024-05-14T10:04:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-14T10:04:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-14T10:04:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mgr.y.log 2024-05-14T10:04:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.h.log 2024-05-14T10:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.10.log 2024-05-14T10:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-14T10:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.e.log 2024-05-14T10:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-14T10:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mon.b.log 2024-05-14T10:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.k.log 2024-05-14T10:04:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-14T10:04:41.436 INFO:teuthology.orchestra.run.smithi002.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-14T10:04:41.437 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.log 2024-05-14T10:04:41.437 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-14T10:04:41.439 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.g.log 2024-05-14T10:04:41.439 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.j.log 2024-05-14T10:04:41.439 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-14T10:04:41.439 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-14T10:04:41.440 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mgr.x.log 2024-05-14T10:04:41.440 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mon.a.log 2024-05-14T10:04:41.445 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.6.log 2024-05-14T10:04:41.445 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-14T10:04:41.445 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.a.log 2024-05-14T10:04:41.445 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-14T10:04:41.445 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.a.log: --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.d.log 2024-05-14T10:04:41.445 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.9.log 2024-05-14T10:04:41.445 INFO:teuthology.orchestra.run.smithi002.stderr: 87.1% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-14T10:04:41.445 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-14T10:04:41.445 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-14T10:04:41.445 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 94.6% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.g.log.gz 2024-05-14T10:04:41.446 INFO:teuthology.orchestra.run.smithi002.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-14T10:04:41.446 INFO:teuthology.orchestra.run.smithi029.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-14T10:04:41.446 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.log 2024-05-14T10:04:41.446 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-14T10:04:41.446 INFO:teuthology.orchestra.run.smithi029.stderr:gzip/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.f.log 2024-05-14T10:04:41.447 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-14T10:04:41.447 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-14T10:04:41.447 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.c.log 2024-05-14T10:04:41.447 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mon.c.log 2024-05-14T10:04:41.447 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mgr.z.log 2024-05-14T10:04:41.447 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.c.log: /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.i.log 2024-05-14T10:04:41.447 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-14T10:04:41.447 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.11.log 2024-05-14T10:04:41.447 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.i.log: /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.8.log 2024-05-14T10:04:41.447 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-14T10:04:41.447 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.l.log 2024-05-14T10:04:41.447 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-14T10:04:41.448 INFO:teuthology.orchestra.run.smithi002.stderr: 94.6% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.d.log.gz 2024-05-14T10:04:41.489 INFO:teuthology.orchestra.run.smithi002.stderr: 93.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-14T10:04:41.511 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 84.0% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-14T10:04:41.531 INFO:teuthology.orchestra.run.smithi002.stderr: 83.9% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-14T10:04:41.538 INFO:teuthology.orchestra.run.smithi026.stderr: 94.4% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.k.log.gz 2024-05-14T10:04:41.598 INFO:teuthology.orchestra.run.smithi002.stderr: 90.4% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-14T10:04:41.622 INFO:teuthology.orchestra.run.smithi026.stderr: 90.5% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-14T10:04:41.626 INFO:teuthology.orchestra.run.smithi026.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-14T10:04:41.639 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-volume.log: /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 93.6% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mgr.z.log.gz 2024-05-14T10:04:41.689 INFO:teuthology.orchestra.run.smithi029.stderr: 84.0% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-14T10:04:41.736 INFO:teuthology.orchestra.run.smithi026.stderr: 93.5% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mgr.y.log.gz 2024-05-14T10:04:41.738 INFO:teuthology.orchestra.run.smithi002.stderr: 91.1% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-14T10:04:41.828 INFO:teuthology.orchestra.run.smithi029.stderr: 90.5% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-14T10:04:41.866 INFO:teuthology.orchestra.run.smithi029.stderr: 94.5% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.l.log.gz 2024-05-14T10:04:41.899 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-14T10:04:41.950 INFO:teuthology.orchestra.run.smithi029.stderr: 87.1% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-14T10:04:41.991 INFO:teuthology.orchestra.run.smithi029.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-14T10:04:41.996 INFO:teuthology.orchestra.run.smithi026.stderr: 87.1% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-14T10:04:42.028 INFO:teuthology.orchestra.run.smithi029.stderr: 91.0% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-14T10:04:42.365 INFO:teuthology.orchestra.run.smithi002.stderr: 94.6% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.j.log.gz 2024-05-14T10:04:42.849 INFO:teuthology.orchestra.run.smithi002.stderr: 94.6% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.a.log.gz 2024-05-14T10:04:43.333 INFO:teuthology.orchestra.run.smithi002.stderr: 90.5% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mgr.x.log.gz 2024-05-14T10:04:54.728 INFO:teuthology.orchestra.run.smithi002.stderr: 90.2% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mon.a.log.gz 2024-05-14T10:04:54.861 INFO:teuthology.orchestra.run.smithi029.stderr: 91.9% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mon.c.log.gz 2024-05-14T10:06:21.977 INFO:teuthology.orchestra.run.smithi002.stderr: 91.0% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.6.log.gz 2024-05-14T10:07:03.762 INFO:teuthology.orchestra.run.smithi002.stderr: 91.0% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.9.log.gz 2024-05-14T10:07:22.805 INFO:teuthology.orchestra.run.smithi026.stderr: 93.4% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mon.b.log.gz 2024-05-14T10:08:01.462 INFO:teuthology.orchestra.run.smithi002.stderr: 91.3% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-14T10:08:21.730 INFO:teuthology.orchestra.run.smithi002.stderr: 91.0% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-14T10:08:21.733 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-05-14T10:08:21.733 INFO:teuthology.orchestra.run.smithi002.stderr:real 3m40.336s 2024-05-14T10:08:21.733 INFO:teuthology.orchestra.run.smithi002.stderr:user 4m51.317s 2024-05-14T10:08:21.733 INFO:teuthology.orchestra.run.smithi002.stderr:sys 0m9.874s 2024-05-14T10:10:37.064 INFO:teuthology.orchestra.run.smithi026.stderr: 94.9% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.e.log.gz 2024-05-14T10:14:08.131 INFO:teuthology.orchestra.run.smithi029.stderr: 94.9% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.c.log.gz 2024-05-14T10:15:58.642 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.10.log.gz 2024-05-14T10:15:59.285 INFO:teuthology.orchestra.run.smithi029.stderr: 95.0% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.f.log.gz 2024-05-14T10:20:05.179 INFO:teuthology.orchestra.run.smithi026.stderr: 91.0% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.7.log.gz 2024-05-14T10:20:15.438 INFO:teuthology.orchestra.run.smithi029.stderr: 90.9% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.8.log.gz 2024-05-14T10:20:48.332 INFO:teuthology.orchestra.run.smithi026.stderr: 91.2% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-14T10:22:04.283 INFO:teuthology.orchestra.run.smithi029.stderr: 91.2% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.11.log.gz 2024-05-14T10:23:00.464 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-14T10:23:48.735 INFO:teuthology.orchestra.run.smithi029.stderr: 91.2% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-14T10:26:53.325 INFO:teuthology.orchestra.run.smithi029.stderr: 90.9% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-14T10:31:48.041 INFO:teuthology.orchestra.run.smithi029.stderr: 93.2% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.i.log.gz 2024-05-14T10:31:48.112 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-05-14T10:31:48.112 INFO:teuthology.orchestra.run.smithi029.stderr:real 27m6.713s 2024-05-14T10:31:48.112 INFO:teuthology.orchestra.run.smithi029.stderr:user 10m16.923s 2024-05-14T10:31:48.112 INFO:teuthology.orchestra.run.smithi029.stderr:sys 0m28.713s 2024-05-14T10:40:45.170 INFO:teuthology.orchestra.run.smithi026.stderr: 93.1% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.h.log.gz 2024-05-14T10:42:32.348 INFO:teuthology.orchestra.run.smithi026.stderr: 93.2% -- replaced with /var/log/ceph/50d5aebe-11d3-11ef-bc98-c7b262605968/ceph-mds.b.log.gz 2024-05-14T10:42:32.389 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-05-14T10:42:32.390 INFO:teuthology.orchestra.run.smithi026.stderr:real 37m50.990s 2024-05-14T10:42:32.390 INFO:teuthology.orchestra.run.smithi026.stderr:user 13m58.299s 2024-05-14T10:42:32.390 INFO:teuthology.orchestra.run.smithi026.stderr:sys 0m38.658s 2024-05-14T10:42:32.390 INFO:tasks.cephadm:Archiving logs... 2024-05-14T10:42:32.391 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670/remote/smithi002/log 2024-05-14T10:42:32.392 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T10:43:22.827 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670/remote/smithi026/log 2024-05-14T10:43:22.828 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T10:45:26.621 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670/remote/smithi029/log 2024-05-14T10:45:26.622 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T10:46:54.304 INFO:tasks.cephadm:Removing cluster... 2024-05-14T10:46:54.305 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 --force 2024-05-14T10:46:54.508 INFO:teuthology.orchestra.run.smithi002.stdout:Deleting cluster with fsid: 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T10:46:55.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 --force 2024-05-14T10:46:56.624 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T10:46:57.602 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 50d5aebe-11d3-11ef-bc98-c7b262605968 --force 2024-05-14T10:46:58.685 INFO:teuthology.orchestra.run.smithi029.stdout:Deleting cluster with fsid: 50d5aebe-11d3-11ef-bc98-c7b262605968 2024-05-14T10:46:59.595 INFO:tasks.cephadm:Removing cephadm ... 2024-05-14T10:46:59.595 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-14T10:46:59.616 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-14T10:46:59.650 DEBUG:teuthology.orchestra.run.smithi029:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-14T10:46:59.666 INFO:tasks.cephadm:Teardown complete 2024-05-14T10:46:59.666 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-14T10:46:59.679 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-14T10:46:59.679 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T10:46:59.681 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T10:46:59.693 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T10:46:59.831 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-14T10:46:59.842 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-14T10:46:59.843 DEBUG:teuthology.orchestra.run.smithi002:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T10:46:59.845 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T10:46:59.848 DEBUG:teuthology.orchestra.run.smithi029:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T10:46:59.861 INFO:teuthology.orchestra.run.smithi002.stderr:bash: line 1: ntpq: command not found 2024-05-14T10:46:59.862 INFO:teuthology.orchestra.run.smithi029.stderr:bash: line 1: ntpq: command not found 2024-05-14T10:46:59.863 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-05-14T10:46:59.866 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T10:46:59.866 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2024-05-14T10:46:59.866 INFO:teuthology.orchestra.run.smithi002.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +963us[ +981us] +/- 57ms 2024-05-14T10:46:59.867 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 28 -692us[ -674us] +/- 60ms 2024-05-14T10:46:59.867 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 -657us[ -639us] +/- 78ms 2024-05-14T10:46:59.867 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T10:46:59.954 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T10:46:59.954 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-05-14T10:46:59.954 INFO:teuthology.orchestra.run.smithi026.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 +1028us[+1036us] +/- 57ms 2024-05-14T10:46:59.954 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 94 -630us[ -629us] +/- 59ms 2024-05-14T10:46:59.954 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -580us[ -580us] +/- 78ms 2024-05-14T10:46:59.954 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T10:46:59.976 INFO:teuthology.orchestra.run.smithi029.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T10:46:59.976 INFO:teuthology.orchestra.run.smithi029.stdout:=============================================================================== 2024-05-14T10:46:59.976 INFO:teuthology.orchestra.run.smithi029.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 +1078us[+1085us] +/- 58ms 2024-05-14T10:46:59.976 INFO:teuthology.orchestra.run.smithi029.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 25 -572us[ -565us] +/- 60ms 2024-05-14T10:46:59.976 INFO:teuthology.orchestra.run.smithi029.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 -538us[ -531us] +/- 78ms 2024-05-14T10:46:59.976 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T10:46:59.992 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-14T10:47:00.005 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-14T10:47:00.006 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-14T10:47:00.158 DEBUG:teuthology.orchestra.run.smithi002:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-14T10:47:00.300 DEBUG:teuthology.orchestra.run.smithi026:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-14T10:47:00.445 DEBUG:teuthology.orchestra.run.smithi029:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-14T10:47:00.562 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T10:47:00.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T10:47:00.595 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T10:47:00.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T10:47:00.627 DEBUG:teuthology.orchestra.run.smithi029:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T10:47:00.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T10:47:00.655 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-14T10:47:00.666 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-14T10:47:00.746 INFO:teuthology.task.internal:Duration was 5595.777436 seconds 2024-05-14T10:47:00.747 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-14T10:47:00.758 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-14T10:47:00.758 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T10:47:00.761 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T10:47:00.763 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T10:47:00.842 INFO:teuthology.orchestra.run.smithi029.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T10:47:00.845 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T10:47:00.847 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T10:47:01.281 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-14T10:47:01.281 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T10:47:01.281 DEBUG:teuthology.orchestra.run.smithi002:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T10:47:01.307 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2024-05-14T10:47:01.307 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T10:47:01.338 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi029.front.sepia.ceph.com 2024-05-14T10:47:01.339 DEBUG:teuthology.orchestra.run.smithi029:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T10:47:01.369 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-14T10:47:01.369 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T10:47:01.372 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T10:47:01.382 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T10:47:01.450 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-14T10:47:01.450 DEBUG:teuthology.orchestra.run.smithi002:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T10:47:01.458 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T10:47:01.463 DEBUG:teuthology.orchestra.run.smithi029:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T10:47:02.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-14T10:47:02.178 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-14T10:47:02.178 DEBUG:teuthology.orchestra.run.smithi002:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T10:47:02.220 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T10:47:02.262 DEBUG:teuthology.orchestra.run.smithi029:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T10:47:02.304 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-14T10:47:02.316 DEBUG:teuthology.orchestra.run.smithi002:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T10:47:02.320 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T10:47:02.322 DEBUG:teuthology.orchestra.run.smithi029:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T10:47:02.349 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = core 2024-05-14T10:47:02.379 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2024-05-14T10:47:02.388 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = core 2024-05-14T10:47:02.442 DEBUG:teuthology.orchestra.run.smithi002:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T10:47:02.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T10:47:02.458 DEBUG:teuthology.orchestra.run.smithi026:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T10:47:02.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T10:47:02.474 DEBUG:teuthology.orchestra.run.smithi029:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T10:47:02.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T10:47:02.498 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-14T10:47:02.510 INFO:teuthology.task.internal:Transferring archived files... 2024-05-14T10:47:02.511 DEBUG:teuthology.misc:Transferring archived files from smithi002:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670/remote/smithi002 2024-05-14T10:47:02.511 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T10:47:02.564 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670/remote/smithi026 2024-05-14T10:47:02.565 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T10:47:02.604 DEBUG:teuthology.misc:Transferring archived files from smithi029:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670/remote/smithi029 2024-05-14T10:47:02.604 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T10:47:02.643 INFO:teuthology.task.internal:Removing archive directory... 2024-05-14T10:47:02.643 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T10:47:02.646 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T10:47:02.648 DEBUG:teuthology.orchestra.run.smithi029:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T10:47:02.692 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-14T10:47:02.704 INFO:teuthology.task.internal:Not uploading archives. 2024-05-14T10:47:02.704 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-14T10:47:02.758 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-14T10:47:02.758 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T10:47:02.761 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T10:47:02.763 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T10:47:02.774 INFO:teuthology.orchestra.run.smithi002.stdout: 265296 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 10:47 /home/ubuntu/cephtest 2024-05-14T10:47:02.778 INFO:teuthology.orchestra.run.smithi026.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 10:47 /home/ubuntu/cephtest 2024-05-14T10:47:02.778 INFO:teuthology.orchestra.run.smithi029.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 10:47 /home/ubuntu/cephtest 2024-05-14T10:47:02.780 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-14T10:47:02.792 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-14T10:47:02.948 INFO:teuthology.nuke:Checking targets against current locks 2024-05-14T10:47:02.974 DEBUG:teuthology.nuke:shortname: smithi002 2024-05-14T10:47:02.975 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T10:47:03.001 DEBUG:teuthology.nuke:shortname: smithi026 2024-05-14T10:47:03.001 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T10:47:03.005 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 09:05:16.603156', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNKU1NbO3urIqJGt41AqPMnvxC3UOVVOMzAaBWER7gOR1mrCXB1fR6aBOlXoumsmo/kJS/Z9omknmy5yQTbbI8w='} 2024-05-14T10:47:03.024 DEBUG:teuthology.nuke:shortname: smithi029 2024-05-14T10:47:03.025 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T10:47:03.029 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 09:05:16.605221', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCn5pQSoMMrBGz6P2ksGr1UGMUGVYy2pyh2vnuWCt3viIRL2EcKGTtqzFV/evaUwD8UJ6Y7bXAuPwsljxLg9PpE='} 2024-05-14T10:47:03.052 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi029.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705670', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 09:05:16.606985', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKoROA0d8fvC+XMKMpSo9QWZxKU/Q/yFOwkGDQgdv7uhoR9B/jXsXxbZ+QcJo7kGpT8dw1aLLR3Go0yYzVT7b5M='} 2024-05-14T10:47:03.062 INFO:teuthology.orchestra.console.smithi002:Power off 2024-05-14T10:47:03.062 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T10:47:03.085 DEBUG:teuthology.orchestra.console.smithi002:power off output: Chassis Power Control: Down/Off 2024-05-14T10:47:03.085 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T10:47:03.101 INFO:teuthology.orchestra.console.smithi026:Power off 2024-05-14T10:47:03.101 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T10:47:03.115 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2024-05-14T10:47:03.127 DEBUG:teuthology.orchestra.console.smithi026:power off output: Chassis Power Control: Down/Off 2024-05-14T10:47:03.127 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T10:47:03.143 INFO:teuthology.orchestra.console.smithi029:Power off 2024-05-14T10:47:03.144 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T10:47:03.153 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2024-05-14T10:47:03.166 DEBUG:teuthology.orchestra.console.smithi029:power off output: Chassis Power Control: Down/Off 2024-05-14T10:47:03.166 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T10:47:03.192 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is on 2024-05-14T10:47:07.116 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T10:47:07.154 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T10:47:07.194 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T10:47:07.227 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2024-05-14T10:47:07.265 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2024-05-14T10:47:07.302 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is on 2024-05-14T10:47:11.230 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T10:47:11.266 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T10:47:11.303 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T10:47:11.341 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2024-05-14T10:47:11.376 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is off 2024-05-14T10:47:11.413 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is off 2024-05-14T10:47:11.442 INFO:teuthology.orchestra.console.smithi002:Power off completed 2024-05-14T10:47:11.477 INFO:teuthology.orchestra.console.smithi026:Power off completed 2024-05-14T10:47:11.514 INFO:teuthology.orchestra.console.smithi029:Power off completed 2024-05-14T10:47:11.620 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} duration: 5595.777436494827 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_vshankar@teuthology status: fail success: false 2024-05-14T10:47:11.620 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-14T10:47:11.701 INFO:teuthology.run:FAIL